The time may be different for different people. In 1972, after I was saved, when I was in full time ministry, the Lord told me I would have to suffer a lot, and I did. He also told me when I am old, I will be free to whatever I like (as long as it's not sinful, of course).
When I completed the 9 years of ministry, He told me again, that when I got old, I would be free to enjoy whatever I wanted.
When I approached retirement, say 4 years ago, He told me again. And again two years ago, and 1 year ago, and last month, etc.
Lately, everything I ask Him if I should do, He just reminds me of that. If I tell Him I need something to enjoy life, He provides it. But He has made it clear. He simply okays what I want to do; He does not assign me to anything anymore, except very short term, and always in line with my choices.
Again, this is very personal, and Jon, you do whatever God says.