I love threads like this. There are many things I never think about and non-clergy baptizing others is one of them.
If I had to take a stab, my guess is that though it is not commanded, the examples the Bible does provide are not laymen baptizing, but men who had a level of spiritual/Church authority.
My guess (and it's a very unsubstantiated, spit balling kind of guess) is that it would be permissible if that non pastor/elder/deacon was all you had at hand.