While I agree that if God has called the two to be together, there's nothing more to question. But for me personally, in all the churches I've attended, the pastor's wife had a very active role in leading Bible classes and ministering to the women of the congregation.
If a 50-year-old pastor has a congregation full of women who are willing to be led by a 20-year-old woman, then God bless them, but I doubt that would happen.
By some standards, I'm at middle-age myself and the other women still often pat me on the head with a "There, there, little girl."
So if a pastor with a 20-year-old wife is leading a congregation full of teenage girls, that might work. But if his flock includes older women, there is probably going to be a lot of unrest among the women, as they won't feel they have anyone to lead them or go to with their concerns.
I realize this could also happen if a young minister and his young wife are called to lead a congregation, but I think it would be slightly different in a situation where the couple has a 30-year age difference. I think people would be divided over taking the pastor seriously and not his wife.
And if the pastor were older (80, and his wife was 50), the opposite might occur. People might see his wife as being more in touch with the congregation if she's around the median age of most of the members, whereas he might be seen as pretty much just a figurehead in the background.
I'm definitely not saying that God couldn't call a couple with a large age difference to lead a congregation, but I do think they would run into their own unique set of challenges from the people.