The original inhabitants of the land where Alhambra now sits are the Tongva. The San Gabriel Mission was founded nearby on September 8, 1771, as part of the Spanish conquest and occupation of Alta California. The land that would later become Alhambra was part of a 300,000 acre land grant given to Manuel Nieto, a soldier from the Los Angeles Presidio.

In 1820 Mexico won its independence from the Spanish crown and lands once ruled by them became part of the Mexican Republic. These lands then transferred into the hands of the United States following the defeat in the Mexican–American War. A wealthy developer, Benjamin Davis Wilson, married Ramona Yorba, daughter of Bernardo Yorba, who owned the land which would become Alhambra.

With the persuasion of his daughter, Ruth, Yorba named the land after a book she was reading, Washington Irving's Tales of the Alhambra, which he was inspired to write by his extended visit to the Alhambra palace in Granada, Spain. Alhambra was founded as a suburb of Los Angeles that remained an unincorporated area during the mid-19th century.

The first school in Alhambra was Ramona Convent Secondary School, built on hillside property donated by the prominent James de Barth Shorb family. Thirteen years before the city was incorporated, several prominent San Gabriel Valley families interested in the Catholic education of their daughters established the school in 1890.

The city's first public high school, Alhambra High School, was established in 1898, five years before the city's incorporation. On July 11, 1903, the City of Alhambra was incorporated. The Alhambra Fire Department was established in 1906. Alhambra is promoted as a "city of homes", and many of its homes have historical significance.

They include styles such as craftsman, bungalow, Spanish Mediterranean, Spanish colonial, Italian beaux-arts, and arts and crafts. Twenty-six single-family residential areas have been designated historic neighborhoods by the city, including the Bean Tract (formerly owned by early resident Jacob Bean), the Midwick Tract (site of the former Midwick Country Club), the Airport Tract (formerly the landing pad for Alhambra Airport), and the Emery Park area.

There are also a large number of condominiums, rental apartments, and mixed-use residential/commercial buildings, especially in the downtown area. Alhambra's main business district, at the intersection of Main and Garfield, has been a center of commerce since 1895. By the 1950s, it had taken on an upscale look and was "the" place to go in the San Gabriel Valley.

While many of the classic historical buildings have been torn down over the years, the rebuilding of Main Street has led to numerous dining, retail, and entertainment establishments. Alhambra has experienced waves of new immigrants, beginning with Italians in the 1950s, Mexicans in the 1960s, and Chinese in the 1980s.

As a result, a very active Chinese business district has developed on Valley Boulevard, including Chinese supermarkets, restaurants, shops, banks, realtors, and medical offices. The Valley Boulevard corridor has become a national hub for many Asian-owned bank headquarters, and there are other nationally recognised retailers in the city. The historic Garfield Theatre, located at Valley Boulevard and Garfield Avenue from 1925 until 2001, was formerly a vaudeville venue and is rumored to have hosted the Gumm Sisters, featuring a very young Judy Garland.

Faded from its original glory, for its last few years it was purchased and ran Chinese-language films, and in 2001 went out of business. Subsequently, developers have remodeled the dilapidated building, turning it into a vibrant commercial center with many Chinese stores and eateries. In 2003, actress Lana Clarkson was shot to death in the Alhambra home of record producer Phil Spector.

Spector lived in Alhambra's largest and most notable residence, the Pyrenees Castle, built in 1926. In 2009, Spector was convicted of second-degree murder in connection with Clarkson's death.