"The Church received from the apostles the tradition of giving baptism even to infants. The apostles, to whom were committed the secrets of the divine sacraments, knew there are in everyone innate strains of [original] sin, which must be washed away through water and the Spirit" (Commentaries on Romans 5:9 [A.D. 248]).
Here we see historical evidence for infant baptism. If the Apostles thought it was ok I think it's ok aswell!
Irenaeus
"He [Jesus] came to save all through himself; all, I say, who through him are reborn in God: infants, and children, and youths, and old men. Therefore he passed through every age, becoming an infant for infants, sanctifying infants; a child for children, sanctifying those who are of that age . . . [so that] he might be the perfect teacher in all things, perfect not only in respect to the setting forth of truth, perfect also in respect to relative age" (Against Heresies 2:22:4 [A.D. 189]).
"‘And [Naaman] dipped himself . . . seven times in the Jordan’ [2 Kgs. 5:14]. It was not for nothing that Naaman of old, when suffering from leprosy, was purified upon his being baptized, but [this served] as an indication to us. For as we are lepers in sin, we are made clean, by means of the sacred water and the invocation of the Lord, from our old transgressions, being spiritually regenerated as newborn babes, even as the Lord has declared: ‘Except a man be born again through water and the Spirit, he shall not enter into the kingdom of heaven’ [John 3:5]" (Fragment34 [A.D. 190]).
Hippolytus
"Baptize first the children, and if they can speak for themselves let them do so. Otherwise, let their parents or other relatives speak for them" (The Apostolic Tradition 21:16 [A.D. 215]).
Origen
"Every soul that is born into flesh is soiled by the filth of wickedness and sin. . . . In the Church, baptism is given for the remission of sins, and, according to the usage of the Church, baptism is given even to infants. If there were nothing in infants which required the remission of sins and nothing in them pertinent to forgiveness, the grace of baptism would seem superfluous" (Homilies on Leviticus 8:3 [A.D. 248]).
"The Church received from the apostles the tradition of giving baptism even to infants. The apostles, to whom were committed the secrets of the divine sacraments, knew there are in everyone innate strains of [original] sin, which must be washed away through water and the Spirit" (Commentaries on Romans 5:9 [A.D. 248]).
John Chrysostom
"You see how many are the benefits of baptism, and some think its heavenly grace consists only in the remission of sins, but we have enumerated ten honors [it bestows]! For this reason we baptize even infants, though they are not defiled by [personal] sins, so that there may be given to them holiness, righteousness, adoption, inheritance, brotherhood with Christ, and that they may be his [Christ’s] members" (Baptismal Catecheses in Augustine, Against Julian 1:6:21 [A.D. 388]).
Augustine
"What the universal Church holds, not as instituted [invented] by councils but as something always held, is most correctly believed to have been handed down by apostolic authority. Since others respond for children, so that the celebration of the sacrament may be complete for them, it is certainly availing to them for their consecration, because they themselves are not able to respond" (On Baptism, Against the Donatists 4:24:31 [A.D. 400]).
"The custom of Mother Church in baptizing infants is certainly not to be scorned, nor is it to be regarded in any way as superfluous, nor is it to be believed that its tradition is anything except apostolic" (The Literal Interpretation of Genesis 10:23:39 [A.D. 408]).
"Cyprian was not issuing a new decree but was keeping to the most solid belief of the Church in order to correct some who thought that infants ought not be baptized before the eighth day after their birth. . . . He agreed with certain of his fellow bishops that a child is able to be duly baptized as soon as he is born" (Letters 166:8:23 [A.D. 412]).
"By this grace baptized infants too are ingrafted into his [Christ’s] body, infants who certainly are not yet able to imitate anyone. Christ, in whom all are made alive . . . gives also the most hidden grace of his Spirit to believers, grace which he secretly infuses even into infants. . . . It is an excellent thing that the Punic [North African] Christians call baptism salvation and the sacrament of Christ’s Body nothing else than life. Whence does this derive, except from an ancient and, as I suppose, apostolic tradition, by which the churches of Christ hold inherently that without baptism and participation at the table of the Lord it is impossible for any man to attain either to the kingdom of God or to salvation and life eternal? This is the witness of Scripture, too. . . . If anyone wonders why children born of the baptized should themselves be baptized, let him attend briefly to this. . . . The sacrament of baptism is most assuredly the sacrament of regeneration" (Forgiveness and the Just Deserts of Sin, and the Baptism of Infants 1:9:10; 1:24:34; 2:27:43 [A.D. 412]).