First of all, let me say that I believe you are correct. Physical death did enter the world when Adam and Eve sinned. I believe there had been no death before this point. However, I think the spritual death is more significant. Satan used the distinction between the two to trick Eve. He knew God was speaking of spiritual death, yet he ignored that in his conversation with her.
I think the first thing to address is what is the effect of sin. It separates us from God:
Isa 59:2 But your iniquities have separated you from your God; And your sins have hidden His face from you, So that He will not hear.
Sin leads to destruction.
You asked about when sin is upon us, when we sin or when we are born. I know many believe in original sin, but I have never understood why they do. Ezekiel 18 tells us we are only responsible for our own deeds, and only will be judged by those deeds. In Matthew 18, Jesus says,
Mat 18:2 Then Jesus called a little child to Him, set him in the midst of them, Mat 18:3 and said, "Assuredly, I say to you, unless you are converted and become as little children, you will by no means enter the kingdom of heaven.
If we were born with sin upon us, why would he make such a statement? Why would he want us to be converted and be sinful? The logic doesn't seem to support original sin to me. Therefore, it us upon us when we sin, with one caveat. The person has to be aware of their sin (knowing right and wrong, or capable of understanding it). After all, I am sure these children Jesus spoke of were not infants. And we all know at some point, children do things wrong and have to be corrected. But there was no sin to their account at this time.
The spiritual "deadness" of Adam is passed on because sin has entered the world. It is our nature now. By Adam and Eve's sin, they gained the knowledge of right and wrong (morality) and they have the freewill to choose one or the other. Their offspring and all mankind now has this knowledge, because they brought it in. This is how it is passed on: the knowledge, and sin itself.
All men die spiritually if they are not redeemed.
Rom 6:1 What shall we say then? Shall we continue in sin that grace may abound?
Rom 6:2 Certainly not! How shall we who died to sin live any longer in it?
Rom 6:3 Or do you not know that as many of us as were baptized into Christ Jesus were baptized into His death?
Rom 6:4 Therefore we were buried with Him through baptism into death, that just as Christ was raised from the dead by the glory of the Father, even so we also should walk in newness of life.
Rom 6:5 For if we have been united together in the likeness of His death, certainly we also shall be in the likeness of His resurrection,
Rom 6:6 knowing this, that our old man was crucified with Him, that the body of sin might be done away with, that we should no longer be slaves of sin.
Rom 6:7 For he who has died has been freed from sin.
Rom 6:8 Now if we died with Christ, we believe that we shall also live with Him,
Rom 6:9 knowing that Christ, having been raised from the dead, dies no more. Death no longer has dominion over Him.
and
Eph 2:1 And you He made alive, who were dead in trespasses and sins,
Eph 2:2 in which you once walked according to the course of this world, according to the prince of the power of the air, the spirit who now works in the sons of disobedience,
Eph 2:3 among whom also we all once conducted ourselves in the lusts of our flesh, fulfilling the desires of the flesh and of the mind, and were by nature children of wrath, just as the others.
Eph 2:4 But God, who is rich in mercy, because of His great love with which He loved us,
Eph 2:5 even when we were dead in trespasses, made us alive together with Christ (by grace you have been saved),
Christ redeemed us and gave us an opportunity to obey the gospel and live a life for Him, and we will gain eternal life.
We don't die spiritually "in Adam." I think the word "in" is being given a little too much weight. By his actions, sin and death entered the world. Because of this, we have been given the gospel, an opportunity to escape that everlasting spiritual death.
I hope this helps. If not, please ask questions, and I will attempt to clarify further.
May God bless you in your studies.
Heb 11:6 But without faith it is impossible to please Him, for he who comes to God must believe that He is, and that He is a rewarder of those who diligently seek Him.