The first inhabitants of the area currently known as Greenwood were the Delaware Indians (Lenape). In 1818, the Treaty of St. Mary's opened central Indiana to European American settlement, and by 1823 the first cabin in northern Johnson County was erected by settlers John B. and Isaac Smock on land now occupied by Greenwood Park Mall.

Greenwood was first known as "Smocktown" or "Smock's Settlement" in honor of the Smock brothers, and became "Greenfield" in 1825. Since this clashed with another Greenfield located in Hancock County, the name of the settlement was changed to "Greenwood" in 1833.Greenwood was incorporated as a town under Indiana law in 1864.Greenwood was an early and key cog in the Electric Indianapolis Interurban Railway System.

In 1895, Henry L. Smith proposed and organized the Indianapolis, Greenwood & Franklin Company and graded the line to Greenwood. The Indianapolis, Greenwood & Franklin Railway was opened between Indianapolis and Greenwood on January 1, 1900, and, according to Indianapolis historian Jacob Piatt Dunn, was the Hoosier capital's first real interurban electric railway.

The railway followed what is now Madison Avenue. The J.T. Polk Canning Company was essential to Greenwood's early growth. The cannery was a major employer and canned a variety of vegetables grown in Indiana. Later, the company expanded into the dairy market and provided milk delivery to customers. At one point the cannery was the largest canning operation west of Baltimore.

The cannery was eventually purchased by the Stokely-Van Camp company and retained operations in Greenwood until the 1950s. Portions of the cannery are still standing on Main Street and have been repurposed for professional office space.Greenwood became a fifth-class city in 1960.In 1965, an Indiana Civil Rights Commission report found that Greenwood had recently been one of 19 sundown towns in Indiana, where African Americans were not allowed to live or stay after dark.

The city had been exclusively white since the 1920s.Greenwood Commercial Historic District was listed on the National Register of Historic Places in 1991. This district encompasses 25 buildings and 3.2 acres (1.3 hectares).In 2010, the Greenwood City Council approved a measure to change the official status of Greenwood to second-class city in accordance with Indiana Code Title 36, Article 4, Chapter 1.On July 17, 2022, three people were killed and two others wounded after a mass shooting occurred at the food court of Greenwood Park Mall.

The gunman was subsequently shot and killed by an armed civilian who witnessed the shooting.