Avoiding people? Hmmm.
Fundamentalist secondary separation has caused great harm.
I used to hear 1 Corinthians 5:10 preached by several as if it were a warning that associating with the lost would cause God to "take us out of this world." That is one gravely misused text that has caused me to appreciate and seek proper context.
We cannot be salt to the world, and light to the world (Matthew 5:13-14) if we isolate and insulate ourselves from the lost.
This text would be difficult to heed if we are to avoid the lost: 1 Peter 3:15.
I happen to work around lots of lost people, and lately have had opportunity to witness to several of them the Gospel of Christ. I know, "Calvies" don't witness, right?
As far as the "sermon" thing, that's a touchy subject. The mere mention of a Biblical point to some is a sermon and they can become antagonistic about mentioning that. I try to avoid a person in that state of mind and pray for a good opportunity.
We do not always have to mention Scripture to the lost, why not just have general conversation and seek good opportunity and an open door of God's leading to go deeper in witness? Years ago I knew a guy who would hunt down the newly hired people at work and hit them between the eyes with the Gospel as soon as he found them. Not very wise imho.
Funny thing here, God tells us to avoid those who cause division &c, those within the body, obviously; Romans 16:17.