In the early development of manufacturing, areas that afforded the potential for water power were prime locations for community growth. In 1814, the Cotton Factory was built in a remote section of Nelson to take advantage of the water power from the outflow of the dammed Factory Lake, now known as Granite Lake. A community began to develop around the factory. A small number of the residents here were Baptists who met for services in the garret of the Cotton Factory. Holding beliefs that differed from the established Congregational Church in the village of Nelson, they decided to build their own church.
In 1819 John Breed, Jerry Felt, and Oliver Heald, all Baptists, constructed a new brick church on the shore of Factory Lake conveniently located for their service of baptism by immersion. It is reported that even if there was ice on the Lake, it would be broken for baptisms if needed.
After several years, financial problems caused the original builders to sell the church to the Universalist Church. It was again sold in 1862 to a stock company and was known as Union Hall for the next 29 years. The building served as a community meeting place and was the site of dances, shows, and occasional religious services and Sunday school sessions. There is an old framed announcement near the clerk’s desk in the Nelson Town Office advertising a “Fancy Dress Ball” to be held at the Union Hall on April 5, 1871 that attests to social activities there. It also was the meeting place of the Granite Lake Grange No. 115 for several years.
In 1891 the building was bought by the United Methodist Church and has remained so for more than 125 years. It was renamed the Chapel-by-the-Lake in 1943 by Rev. Fay Gemmel, the pastor at that time. In 1938 during the depression a local Nelson artist, Albert Duvall Quigley who was employed by the WPA, painted a mural on three walls of the Chapel. It is remembered by some of the older town residents as portraying life in Nelson in the early days. The mural was painted over with no photographs or drawings made to document Quigley’s work. According to Bill Steiniger who edited a history of the Chapel in 2017, this happened in 1958.
A major renovation of the church building was taken in 1972. After the long life of the building with only a bell tower, a steeple was added in 1995. Today the building remains a community center in addition to the weekly religious services with many activities sponsored by the church.