The early Church was the Catholic Church. It taught infallibly, gave us the New Testament and was made up of three ranks of clergy, bishop, priest and deacon.
The idea of “Scripture Alone” didn’t exist nor could it have as the printing press would not be invented for more than a thousand years.
The Martyrdom of Polycarp
When finally he concluded his prayer, after remembering all who had at any time come his way – small folk and great folk, distinguished and undistinguished, and the whole Catholic Church throughout the world – the time for departure came. So they placed him on an ass, and brought him into the city on a great Sabbath (The Martyrdom of Polycarp 8 [A.D. 110]).
Irenaeus
The Catholic Church possesses one and the same faith throughout the whole world, as we have already said (Against Heresies 1:10 [A.D. 189]).
Since therefore we have such proofs, it is not necessary to seek the truth among others which it is easy to obtain from the Church; since the apostles, like a rich man [depositing his money] in a bank, lodged in her hands most copiously all things pertaining to the truth: so that every man, whosoever will, can draw from her the water of life. For she is the entrance to life; all others are thieves and robbers. On this account we are bound to avoid them, but to make choice of the things pertaining to the Church with the utmost diligence, and to lay hold of the tradition of the truth. For how stands the case? Suppose there should arise a dispute relative to some important question among us. Should we not have recourse to the most ancient churches with which the apostles held constant intercourse, and learn from them what is certain and clear in regard to the present question? For how should it be if the apostles themselves had not left us writings? Would it not be necessary [in that case] to follow the course of the tradition which they handed down to those to whom they did commit the churches? (ibid. 3:4).
Tertullian
Where was Marcion then, that shipmaster of Pontus, the zealous student of Stoicism? Where was Valentinus then, the disciple of Platonism? For it is evident that those men lived not so long ago – in the reign of Antoninus for the most part – and that they at first were believers in the doctrine of the Catholic Church, in the church of Rome under the episcopate of the blessed Eleutherus, until on account of their ever restless curiosity, with which they even infected the brethren, they were more than once expelled (The Prescription Against Heretics 22,30 [A.D.200])
Clement of Alexandria
A multitude of other pieces of advice to particular persons is written in the holy books: some for presbyters, some for bishops and deacons; and others for widows, of whom we shall have opportunity to speak elsewhere (The Instructor of Children 3:12:97:2 [A.D. 191]).
Even here in the Church the gradations of bishops, presbyters, and deacons happen to be imitations, in my opinion, of the angelic glory and of that arrangement which, the Scriptures say, awaits those who have followed in the footsteps of the apostles and who have lived in complete righteousness according to the gospel (Stromateis 6:13:107:2 [post-A.D. 202]).
Hippolytus
When a deacon is to be ordained, he is chosen after the fashion of those things said above, the bishop alone in like manner imposing his hands upon him as we have prescribed. In the ordaining of a deacon, this is the reason why the bishop alone is to impose his hands upon him: He is not ordained to the priesthood, but to serve the bishop and to fulfill the bishop's command. He has no part in the council of the clergy, but is to attend to his own duties and is to acquaint the bishop with such matters as are needful. . . . On a presbyter [priest], however, let the presbyters impose their hands because of the common and like Spirit of the clergy. Even so, the presbyter has only the power to receive [the Spirit], and not the power to give [the Spirit]. That is why a presbyter does not ordain the clergy; for at the ordaining of a presbyter, he but seals while the bishop ordains. (Apostolic Tradition 9 [ca. A.D. 215]).
Origen
Not fornication only, but even marriages make us unfit for ecclesiastical honors; for neither a bishop, nor a presbyter, nor a deacon, nor a widow is able to be twice married (Homilies on Luke, 17 [ca. A.D. 235]).
Cyprian
The spouse of Christ cannot be defiled; she is uncorrupted and chaste. She knows one home . . . Does anyone believe that this unity which comes from divine strength, which is closely connected with the divine sacraments, can be broken asunder in the Church and be separated by the divisions of colliding wills? He who does not hold this unity, does not hold the law of God, does not hold the faith of the Father and the Son, does not hold life and salvation (On the Unity of the Catholic Church 6 [A.D. 251]).
Peter speaks there, on whom the Church was to be built, teaching and showing in the name of the Church, that although a rebellious and arrogant multitude of those who will not hear or obey may depart, yet the Church does not depart from Christ; and they are the Church who are a people united to the priest, and the flock which adheres to its pastor. Whence you ought to know that the bishop is in the Church, and the Church in the bishop; and if any one be not with the bishop, that he is not in the Church, and that those flatter themselves in vain who creep in, not having peace with God’s priests, and think that they communicate secretly with some; while the Church which is Catholic and one, is not cut nor divided, but is indeed connected and bound together by the cement of priests who cohere with one another (Letters 66 [A.D. 253]).
Jesus Christ's prayer to God the Father -
"I pray also for those who will believe in me through their message, that all of them may be ONE, Father, just as you are in me and I am inyou. May they also be in us so that the world may believe that you have sent me. I have given them the glory that you gave me, that they may be one as we are one— I in them and you in me—so that they may be brought to complete unity. Then the world will know that you sent me and have loved them even as you have loved me"
- John 17:20-23
