This is why I do not like mixed signals.
I would ask him exactly what his intentions are.
I would also explain to him how I feel about the entire situation.
I would tell him that he needs to figure out exactly what he wants, and when he is done, to then tell me what that answer is.
Once all of that happens, you can decide what you want to do regarding everything.