Letter 49 to Pammachius
10. (…) See my express declaration that marriage is allowed in the Gospel, yet that those who are married cannot receive the rewards of chastity so long as they render their due one to another. If married men feel indignant at this statement, let them vent their anger not on me but on the Holy Scriptures; nay, more, upon all bishops, presbyters, and deacons, and the whole company of priests and levites, who know that they cannot offer sacrifices if they fulfil the obligations of marriage.
(...)
21. Therefore, as I was going to say, the virgin Christ and the virgin Mary have dedicated in themselves the first fruits of virginity for both sexes. The apostles have either been virgins or, though married, have lived celibate lives. Those persons who are chosen to be bishops, priests, and deacons are either virgins or widowers; or at least when once they have received the priesthood, are vowed to perpetual chastity.
(trans. Fremantle, Lewis, and Martley 1893)