The Holy Spirit is thought by most to be the "Spirit of God" that "moved over the waters" in Genesis 1. The Holy Spirit certainly came down on prophets, and sometimes on the elders of Israel in the desert, so He manifested many times before Jesus was on earth.
The Holy Spirit is never mentioned in a physical form in the Bible, other than as a dove, descending on Jesus at His Baptism. Of course, God can do whatever He wants, but most people experience the HS strictly as a presence. I doubt the word "sit" is appropriate for a spirit, but if you mean will He "abide" or "remain" with God forever, the answer is yes, as He is God also.