Jesus also says He will draw all men to Him (John 12:32).
God desires all to come to repentance that none perish...
At some point, the will of man is involved.
Punishment makes no sense otherwise.
Also, in regards to punishment. Mankind sinned against God. Every last one of us. We are guilty collectively and without excuse. Again, man has a will, but it is bound by our nature, and that nature is to sin. We choose, but we choose wrongly. That's our own fault. I would refer you to Romans 9 since Paul addressed the issue:
14 What shall we say then? Is there injustice on God's part? By no means! 15 For he says to Moses, “I will have mercy on whom I have mercy, and I will have compassion on whom I have compassion.” 16 So then it depends not on human will or exertion, but on God, who has mercy. 17 For the Scripture says to Pharaoh, “For this very purpose I have raised you up, that I might show my power in you, and that my name might be proclaimed in all the earth.” 18 So then he has mercy on whomever he wills, and he hardens whomever he wills.
19 You will say to me then, “Why does he still find fault? For who can resist his will?” 20 But who are you, O man, to answer back to God? Will what is molded say to its molder, “Why have you made me like this?” 21 Has the potter no right over the clay, to make out of the same lump one vessel for honorable use and another for dishonorable use? 22 What if God, desiring to show his wrath and to make known his power, has endured with much patience vessels of wrath prepared for destruction, 23 in order to make known the riches of his glory for vessels of mercy, which he has prepared beforehand for glory—
All of mankind is condemned already. In his grace, he has chosen some to save, not for them or because of them, or any merit on their part, but ultimately for his own glory and worship.