Hi Angelique,
Don't feel like, because a family member wants to come visit, you have to open your door to them. If this person doesn't want help, keep praying for him that he will want his life changed. I feel bad for him, I'm sure you do too, but you can't help a person until they ask for help. Have you been able to preach the gospel to him?
If I were in the same situation I'm not sure what I'd do. I'd definitely be praying and asking for God to guide me and give me wisdom. The bible says that if anyone lacks wisdom he should pray and God will give him wisdom liberally.
If he insists on coming for a visit, maybe family members could chip in and find an inexpensive motel or one of those long stays that can sometimes be cheaper. That's what I would do so that he wouldn't be in my home with a young baby or in the parents home who are older and would find it difficult.
I read where you're scared. I use to be scared of demonic stuff too when I was younger in the Lord. But it was simply the lack of knowledge or understanding the demonic world that caused the fear.
The Lord says that Greater is He that is in us, than he who's in the world. When I finally understood what that means, God gave me peace about the demonic world. I don't have to be scared of a demon.
You see, Jesus lives in you and you are In Christ. There's power In Him, there's protection In Him, there's wisdom In Him, there's understanding and knowledge In Him. Everything we need for life and Godliness is In Christ.
If we come across a demon, the Lord gives the church the ability to cast them out. But it needs to be that your family member is willing and wants to accept Jesus into their life, because if you cast the demon out and they don't accept the Lord as their savior, that demon could go and get 7 more demons and your family member would be worse off than before.
So pray for His peace in your heart, ask for wisdom from God for this situation and then as God leads you, come up with a plan that would benefit everyone concerned and protect the family.