The context is the elect of whom God waits patiently for their repentance and fulfillment in time which is the point of the context. But you don't like contextual interpretation over your tradition. The "usward" refers to the elect, and to them only, as none other will be saved, note also 2 Timothy 2:8-10.
You don't like God electing whom He wills. Right?
"The world" doesn't mean each and every person who has ever lived. It means those other than Jews as well, something striking to a Jew, who penned this, which would, shockingly to them, include Gentiles thus, "the world."
But again, you don't like contextual interpretations going against your tradition, right?