Hymenaeus, Philetus, and Alexander didn't keep it... they departed from it.
Hymenaeus and Alexander in 1 Timothy 1, are "handed over to satan". This does not mean that Paul takes away their salvation.
18 Timothy, my son, I am giving you this command in keeping with the prophecies once made about you, so that by recalling them you may fight the battle well,
19 holding on to faith and a good conscience, which some have rejected and so have suffered shipwreck with regard to the faith.
20 Among them are Hymenaeus and Alexander,
whom I have handed over to Satan to be taught not to blaspheme.
He is being put out of the congregation of believers. We see another example of this in 1 Corinthians 5...
1 It is actually reported that there is sexual immorality among you, and of a kind that even pagans do not tolerate: A man is sleeping with his father’s wife.
2 And you are proud! Shouldn’t you rather have gone into mourning and have put out of your fellowship the man who has been doing this? 3 For my part, even though I am not physically present, I am with you in spirit. As one who is present with you in this way, I have already passed judgment in the name of our Lord Jesus on the one who has been doing this.
4 So when you are assembled and I am with you in spirit, and the power of our Lord Jesus is present,
5 hand this man over to Satan for the destruction of the flesh,
[a][b] so that his spirit may be saved on the day of the Lord.
In 2 Timothy 2,
14 Keep reminding God’s people of these things. Warn them before God against quarreling about words; it is of no value, and only ruins those who listen.
15 Do your best to present yourself to God as one approved, a worker who does not need to be ashamed and who correctly handles the word of truth.
16 Avoid godless chatter, because those who indulge in it will become more and more ungodly.
17 Their teaching will spread like gangrene. Among them are
Hymenaeus and Philetus,
18 who have departed from the truth. They say that the resurrection has already taken place, and they destroy the faith of some. 19 Nevertheless, God’s solid foundation stands firm, sealed with this inscription: “The Lord knows those who are his,” and, “Everyone who confesses the name of the Lord must turn away from wickedness.”
2 Tim 2, we see that "departing from the truth" is that they were preaching something that was false.