Lets explain this in an easier concept. Patience. When you pray for patience, God doesn't zap you with patience, He gives you things to test your patience. Why? Because Real growth comes through struggle, effort, facing your obstacles. When you want more patience, God puts you in a position to have to exercise what patience you have. Why? Think of it as a muscle, if you don't exercise your muscle, it doesn't grow, right? What happens if you don't exercise your patience? It doesn't grow. So when you want more patience, you have to have your patience tested, then you practice being more patient, to grow it.
Same will happen with your faith. If you want more faith, God won't zap it into you, He will test your faith, so that when you pass that test, your faith will be stronger. If anything, your struggle with your faith is God answering your prayer.
But we also do have our flesh, which actively fights against our spiritual growth. We have an enemy in satan, who also actively fights against our spiritual growth. So you have two sources fighting against you, so it's only natural that you will have problems in your attempt to move forward in your spiritual life.
In reality, everything you just said is normal and makes sense and are good signs that you're making right choices.
Just remember, there is little growth when times are easy, but in times of trials and suffering is when we grow the most, and that growth sticks in us deeper and changes us more. Keep pushing persevering.