Asking a man not to care about appearances is asking him not to be a man. It's wired into him, down to his very DNA. Now, don't get me wrong - as with anything in creation it can be (and has been) taken to excess - but "attractiveness" (which is more than physical appearance - it means that the person has qualities about them that draw you to them) is and should be part of the equation. Even Song of Songs seems to paint the picture that the sexes are different, and that the male one is more visually driven than emotionally driven. Doesn't mean that every guy requires a supermodel to consider her a prospect, but the reality is that most men don't. But she does have to be attractive. Whatever "attractive" works out to be, physical appearance is always going to be part of the formula.
Besides, I know a fair amount of Godly, single men who remain single because they lack a certain attractiveness about them.