I was only trying to point out that not allowing your sons father in the house isnot always needed,

I really think you should talk to his dad, try and keep it civil and just talk about what concerns you have, discuss reasonable ground rules (like the gf not coming over etc) and do tell him you need time.