Oxford was Benton County's first town. Commissioners appointed by the Indiana General Assembly selected the location in May 1843 to be the site of the county seat on land donated by Henry W. Ellsworth and David Atkinson. It was first called Milroy in honor of Samuel Milroy, one of the commissioners, but a town in Rush County already bore that name.

In October 1843 the commissioners changed it to Hartford, after Hartford, Connecticut (the home of Ellsworth and Watkins), but they discovered this name was also already being used in Indiana, so in December Judge David J. McConnell awarded the town its present name. Lots went up for sale at this time, with buyers allowed to pay for the lots in three installments over 18 months.

The first building erected in Oxford was a two-story, wooden frame courthouse, designed and built by Francis Boynton, and the second was a hotel, built by Henry L. Ellsworth and operated by James L. Hatton. The first home was that of Aaron Wood, who also ran a small store that supplied groceries, beer and whiskey.Other early establishments in Oxford included the Earl & Carnahan dry goods store and Barnes & Dailey drug store.

The town's first lawyer was Jacob Benedict and its first physician Dr. Mayo. An 1883 history also makes mention of the community's first literary society, the Oxford Lyceum. Many additions and expansions to the town were made over subsequent decades, adding several streets and more than 300 lots. On July 7, 1869, a vote was held to determine if Oxford should be incorporated, and passed with 41 in favor and 28 against.

The town was dealt a blow, however, when in 1874 it lost its status as county seat to more centrally-located Fowler, 9 miles to the north.Standardbred horse Dan Patch is arguably the town's best-known native, and was foaled in an Oxford barn on April 29, 1896. The horse gained celebrity status for setting world speed records that remained unbroken until 1960.

A 1949 film, The Great Dan Patch, dramatized the horse's origins in Oxford and rise to fame. He continues to be honored at the town's annual Dan Patch Days festival.The David S. Heath House and Presbyterian Church Building are listed on the National Register of Historic Places.Additional significant historic properties include the Messner Building and the Masonic Building.