Calvary Church is one of two Episcopal churches in Bolivar County; however, it was not the first. That distinction goes to Grace Church, Rosedale. Grace Church was founded in 1879, the same year its church building was completed. Calvary is yoked with Grace Church, and now Calvary holds services at Grace once each quarter. Calvary Episcopal Church was formed in July 1921, as a mission church. By 1925, there were 52 communicants. The founding members worshiped in various public buildings, including the Bolivar County Courthouse and the First Christian Church until 1928; then, the first church, a yellow stucco building located on Court Street, was erected. This building was dedicated as Calvary Church in 1937 by Bishop Theodore Bratton. The next construction at Calvary was the current rectory, completed in May 1953. Its first occupants were the Reverend Duncan Gray, Jr. and his family. Many renovations and additions have been made to the rectory since that time.
The Reverend Duncan Gray, Jr. was instrumental in obtaining a grant and an interest-free loan from the national church. This was offered to mission churches with great potential of becoming parish churches. With this money from the national church and additional available funds, it was decided that it was time to expand and plans were made for a new sanctuary. With the original church occupying the site of the proposed new building, it was necessary to either destroy the old building or move it to a new location. It was decided the
original church be moved and now serves as the Rayner Parish Hall. The first service in the new church was Christmas Eve 1956. In early 1957, the structure was dedicated by Bishop Duncan Gray.
Under the leadership of the Reverend Ray Pradat (1970-73), there was a substantial growth in membership. On January 29, 1977, at the 150th Annual Council in Oxford, Mississippi, Calvary Episcopal Church became a parish with the Reverend James Pulliam as its first rector. (An interesting side note in the history of Calvary Cleveland: Bishop Duncan Gray served the church during its early organizational days (1925-26), Bishop
Duncan Gray, Jr. served as Vicar 1953-57, and Bishop Duncan M. Gray, III spent some of his formative years in Cleveland while his father was at Calvary). In order to accommodate the increase in numbers, a duplex located next door to the rectory was purchased and converted into a Sunday school and office complex. Some recent improvements to other buildings include the addition of three large stained glass windows (memorials) in the church proper, extensive landscaping, a fountain outside the church, and renovations to the interior of the church itself. As part of the interior renovation, a new sacristy and vesting room were added on the west side and a covered walkway connecting the church and parish hall was added on the east side. Recently the ECW and Men’s Group collaborated to update the Rectors’ and Church office. Major renovations to the Rayner House were carried out in 2010.