The whole point of the parable of the prodigal son is to show us that we can leave the father and go back to the world. And when he did come back, what did the father say to the other son? Your brother was dead, and is now alive.
Those to whom Christ said "Depart from Me." once professed their faith, but it did not save them in the end because they didn't obey God's laws.
Those who left God and His doctrine to follow falsehood, after knowing the truth, can never return because that is like crucifying Christ all over again- like you said. They were once saved but are saved no longer. Like God said, those who remain faithful UNTIL the end will be saved.
Its like if you got into Noah's ark, you were saved. But if you got back out, then back in- you removed yourself from a safe state. But what really determines it is if you are in the ark when the flood comes (the end of your life). If the flood comes, and you are not in the ark, and you say, "But I was once in the ark in my lifetime" it matters not. You not only have to get saved, you have to stay saved. And for one to say well then they were never truly saved to begin with is like saying they were never in the ark to begin with- that's rediculous.
The Bible says those who leave once knowing the truth and knowing God cannot come back because that is like crucifying Christ all over AGAIN- indicates that Christ was already crucified in their case- they were already once saved, yet are no longer.