Look at Saul in the OT.
He choose to disobey God. He choose to sin.
1Sa 28:6 And when Saul enquired of the LORD, the LORD answered him not, neither by dreams, nor by Urim, nor by prophets.
Sin, his chooses, seperated him from God.
When we sin we are seperated from God
God does not leave us or forsake us. We leave God, we forsake Him by sinning.
He will often seek us out and if we repent He will restore peace in us.
Can we have Guilt and peace at the same time? No
We can't have sin and the presence of God at the same time.
Isa 59:2 But your iniquities have separated between you and your God, and your sins have hid his face from you, that he will not hear.
1Jn 1:9 KJV If we confess our sins, he is faithful and just to forgive us our sins, and to cleanse us from all unrighteousness.
Isa 48:22 There is no peace, saith the LORD, unto the wicked.
Isa 57:21 There is no peace, saith my God, to the wicked.
Peace comes from God and iniquity drives God away.
He wants to be near us but our sin seperates us.
Repent and turn from your sins.
I'm sure you have felt the Holy Spirit talk to your Heart.
If you keep ignoring God and harden your heart you can not hear Him anymore.
Using Saul, once a king of Israel and not Saul who became Paul, is a very poor example. Do you use a prophet for direction or do you understand that we are no longer under the long shadow of the law and having to sacrifice animals because we sin.
Ephesians 1:7 - In him we have redemption through his blood, the forgiveness of our trespasses, according to the riches of his grace,
1 John 1:9 - If we confess our sins, he is faithful and just to forgive us our sins and to cleanse us from all unrighteousness.