The mill site is the lowest of three on the stream that empties Granite Lake. The broken grindstone near the mill’s tailrace is evidence of its early use for grinding grain into flour.  Joseph Baker likely built the mill in the 1790’s as a saw and gristmill. By 1816 it had been purchased by the Nelson Cotton and Woolen Company as part of their establishment of a cotton mill on the stream just below the millpond.  Asa Stone probably operated it as a gristmill under their ownership and later purchased it.

In the 1840’s it was used solely as a sawmill operated by a succession of owners. George B. Jenkins (1856-1859) installed three circular saws and a shaft and belting system to transfer power.  Frederick Taylor (1864-1868) installed woodworking equipment and manufactured trunk cleats. Frank B. Hardy (1871 to 1892) likely carried on that work and made shingles. The mill probably acted as a subcontractor making chair parts for the Colony chair factory and Frederick Taylor’s chair shop, both just upstream. The mill seems to have gone out of use about 1890.