My situation was very similar to yours. My ex cheeted on me when I was pregnant with my second child. We seperated for 1.5 years then tried to reconcile. That worked for about a year and he was back to his old ways again. The second time we seperated I decided I would not let him in my house no matter how much I needed him because I had to be strong and it just caused to much confusion with my children. He takes them sometimes on the weekdays but he goes somewhere with them and brings them home later. Everyother weekend he takes them to the house he lives at. It has made things so much easier for me and the children. When he comes to your home it causes the children to think he is home and if you arent going to reconcile you need to make it clear that he is not going to be in the home because he dosent live there. It is not selfish to hold your ground. If you are going to move on then you can not let him in your home anymore. Need a babysitter take them to him or have a friend babysit.