Wow this is a big topic. God is so much bigger than us, and there are some things we will never have a definitive answer to. But here's my take on it!
This is something that we actually discussed in this weeks mens group by pure chance. My answer to the question was basically, ask God, I don't know for sure, but here's my opinion. We are working our way through
Luke 12:35-48, and most relevant to this discussion up to verse 39. Something we read was
the parable of the 10 virgins, where 5 were foolish and 5 were wise bringing spare oil. The foolish ran out of oil and while they were buying more the bridegroom came and they were shut out of the marriage feast.
My take on this is, if you have drifted so far from God that you don't even feel saved anymore (and don't get me wrong we all stumble a little sometimes), then I would think it debatable if you were truly saved in the first place. I know that for most of the my life I've been a Christian, I'd go to church every week, I looked great and Godly on the outside, but I wasn't truly saved, I was just pretending. But God knew. I was one of the foolish virgins, and though in the past I'd felt like I'd genuinely committed my life to the Lord. I wasn't walking the walk.
If Jesus had returned when I was in that place, I just can't feel it would have been right for me to have been allowed into heaven, because I wasn't in the right place then.
Now things are different - I love the Lord, I commit myself to my walk with him daily, and now I know, I KNOW I'm saved. But perhaps more pertinent to this discussion is that now I'm here, I am NEVER going back. God is my all and I'm not willing to sacrifice that for anything.
So from my perspective at least - if you are truly saved, you have that personal relationship with God, and reinforce that daily. And when you get to that place, there's no desire to go back because you know what you now have, and where you've come from - and that stays with you always.