Allegorical approach is often seen as dangerous because people can make up their own minds, apply Scripture to their own lives as how it ministers to them... it takes control from the leaders of the church. When you can decide for yourself what Scripture says, instead of buying into doctrinal and how-to-live legalism, then you don't need the priest, pastor, preacher to tell you what to think - it's pure, spiritual liberty.
Doesn't bode well for those who need control for financial or political reasons. People who condemn allegorical thought, I've found, tend to be very legalistic, and think everyone should believe, live and worship a certain way - an army of religious robots.
Note, I am not saying all literal approach Christians are legalistic - just seems that kind of reading tends towards it. There are certainly plenty of liberal, or "freethinking" Christians that are dogmatic, as well.
Also, I believe there's should be order - but people can disagree and still be orderly. I doubt you will find any church where every memember thinks/believes exactly the same things - many are holding back their true thoughts for fear of being stigmatized. (There's a number of reason why one wouldn't just go to another church - family/friends go to that one, it's the closest, they like some people there, it's a family tradition and expected, etc.)
I talked to a man in a Lutheran church one time, that said he really like it, but whispered that he didn't think the pastor should be called Reverend - and it was clear that he kept that to himself, but he took what he didn't like and focused on the good.