As the account at Pentecost reveals those who believed in Jesus still needed to repent and have their sins remitted in water baptism and receive the Holy Ghost in order to be born again. This truth is witnessed over and over in Scripture. And we know from scripture that sin cannot enter the kingdom of God. Note just a few of the scripture references:
Jesus said, "I tell you, Nay: but, except ye repent, ye shall all likewise perish." Luke 13:3
"But ye are not in the flesh, but in the Spirit, if so be that the Spirit of God dwell in you. Now if any man have not the Spirit of Christ, he is none of his." Rom. 8:9
Scripture reveals that receiving the Holy Ghost/Spirit does not happen the instant a person believes in Jesus:
Luke 11:13 Jesus said, "If ye then, being evil, know how to give good gifts unto your children: how much more shall your heavenly Father give the Holy Spirit to them that ASK him?"
Acts 19:1-7 Paul asked, "Have you received the Holy Ghost since you believed?"
The account reveals the men submitted to baptism in the name of Jesus. And afterward received the Holy Ghost as well.
Acts 8:15-17 "Now when the apostles which were at Jerusalem heard that Samaria had received the word of God, they sent unto them Peter and John: Who, when they were come down, prayed for them, that they might receive the Holy Ghost:
For as yet he was fallen upon none of them: only they were baptized in the name of the Lord Jesus.)
Then laid they their hands on them, and they received the Holy Ghost."
Acts 2:36-42 "Therefore let all the house of Israel know assuredly, that God hath made the same Jesus, whom ye have crucified, both Lord and Christ.
Now when they heard this, they were pricked in their heart, ... (At this point they believed in Jesus and asked Peter what they had to do) ...
Then Peter said unto them, Repent, and be baptized every one of you in the name of Jesus Christ for the remission of sins, and ye shall receive the gift of the Holy Ghost.
For the promise is unto you, and to your children, and to all that are afar off, even as many as the Lord our God shall call.
And with many other words did he testify and exhort, saying, Save yourselves from this untoward generation.
Then they that gladly received his word were baptized: and the same day there were added unto them about three thousand souls.
And they continued stedfastly in the apostles' doctrine and fellowship, and in breaking of bread, and in prayers."
Acts 10:43-48 - In this case people received the Holy Ghost while Peter was still speaking. The concensus as why this happened was in order to convince the Jews that God intended for Gentiles to be given the opportunity to acquire the NT rebirth. Notice Cornelius and his household still had to submit to being water baptized in the name of Jesus after receiving the Holy Ghost (Acts 10:43-48).