Gem, we've all done things in the past that we may be ashamed of. Making mistakes is how we grow as a person.
I don't know if you are still with this man you have been talking about lately, but I'd steer clear of him as much as possible, though I realise you share children.
When we live in the past, we miss out on so many fantastic opportunities that are right in front of us. I did this when I was younger, and now I'm older, I realise I wasted my time pining after a man who would never care for me how I wanted to be cared for. I turned down a date with another really lovely guy - he left me one message after that, hoping he'd see me again but if not, to take care. I never called him back and I regret it to this day. You know why? Cos I was stuck in the past, and pining over a bloke I'd known for a long time, & I didn't want to admit to myself that we'd probably never get together. I missed out on probably a wonderful opportunity.
Let me ask you this: if you stay living in the past, and regretting everything, where is that going to get you? This sort of rumination quite often leads to depression, & can cause clinical depression & other problems in the long run.
Please talk to someone non-judgemental about what you are feeling. This will get it all off your chest, and gives it less power over you in the long run.
I also find it helps to write a diary. That helps to get it all out physically on paper, then you can put it away somewhere safe, instead of brooding on the past all the time.
If you feel like you have to confess, contact the person/s you hurt by writing a letter to them, or phoning them if you think this will help. But this is exactly why the saying 'let sleeping dogs lie' came about - sometimes, we just have to accept we did some thing/s we regret, and move on.
It's all we have - that and heaps of prayer.