I'm sorry you have to put up with this. Just think- it'll stop sooner or later. Are you new there?
I kind of have the same situation as you too. Most people I mention him to think I'm interested in dating him and say "You can't marry a non-christian." Hold on now- he's my co-worker, someone I don't know well. He might be kind and make good company, but marry him? Jeez, people.
I can sympathize for you here lol I wish I could but ointment and a band-aid on it for you, but time should take care of it. I think yo're doing the right thing by not giving your co-workers a reaction.