The area has been inhabited by the Nlaka'pamux (formerly the Thompson Salish) people for over 10,000 years. It is one of the earliest locations settled by non-natives in the Southern Interior of British Columbia. It was founded during the Fraser Canyon Gold Rush of 1858–59, when it was known as "The Forks".

Lytton was on the route of the Gold Rush in 1858. The same year, Lytton was named for Edward Bulwer-Lytton, the British Colonial Secretary and a novelist. For many years Lytton was a stop on major transportation routes, namely, the River Trail from 1858, Cariboo Wagon Road in 1862, the Canadian Pacific Railway in the 1880s, the Cariboo Highway in the 1920s, and the Trans-Canada Highway in the 1950s. However, it has become much less important since the construction of the Coquihalla Highway in 1987 which uses a more direct route to the British Columbia Interior.

The town was destroyed by a wildlife on June 30, 2021.

During summer heat waves, Lytton is often the hottest spot in Canada, despite being north of 50° in latitude. Due to the dry summer air and a relatively low elevation of 230 m (750 ft), summer afternoon shade temperatures sometimes reach 35 °C (95 °F) and up and occasionally top 40 °C (104 °F). Hot summer temperatures are made more tolerable by low humidity. However, the heat can be intense with usually clear blue skies and blazing sunlight – heat also radiates from the valley's slopes, and forest fires are not uncommon during the summer.

Lytton's climate is also characterized by relatively short and mild winters (although December and January average monthly temperatures are just below freezing), with Pacific maritime influence during the winter ensuring thick cloud cover much of the time. Cold snaps originating from Arctic outflow occur from time to time, but tend to be short-lived, and mountains to the north block extreme cold from penetrating the Fraser Canyon.

Lytton is much drier than communities to the south, but certainly wetter than some of the driest spots in the British Columbia interior such as Spences Bridge, Kamloops, and Osoyoos. However, Lytton has the driest summers in the interior of British Columbia, and indeed, one of the driest summers of all places in Canada. Maximum precipitation occurs in the cooler months, with the late autumn and early winter constituting the wettest time of the year.

Lytton is on Highway 1 (Trans-Canada Highway) between Vancouver and Kamloops. It is approximately 3½ hours from Vancouver and 1½ hours north of Hope.

Although railways pass through Lytton, VIA Rail Canada's closest train stops are in Ashcroft, 80 km to the north, and North Bend/Boston Bar, 44 km to the south.

Inland Ferries. Operated under contract for British Columbia's Ministry of Transportation and Infrastructure, inland ferries are vehicle ferries that connect roads across rivers and lakes. Routes operate throughout the year, but some river ferries may not operate during parts of the Spring due to river conditions. Free. 

• 50.24736-121.591921 Lytton Reaction Ferry. Daily 12:45 AM - 2:30 AM, 3:00 AM - 4:45 AM, 5:00AM - 8:45 AM, 9:00 AM - 10:30 AM, 11:00 AM - 12:45 PM, 1:00 PM - 4:30 PM, 4:45 PM - 6:30 PM, 7:00 PM - 8:45 PM, 9:00 AM - 12:30 AM. Crosses the Fraser River about 2 km north of Lytton using a turn off from Highway 12. 5 minutes crossing. Operates on demand.  

On the river's west side are Indian reserve communities of the Lytton First Nation and the Stein Valley Nlaka'pamux Heritage Park via trails from the confluence of the Stein River with the Fraser. From the ferry, a route known as the West Side Road leads through the Nesikep and Texas Creek areas to Lillooet and British Columbia Highway 99; the route south from the ferry is much more difficult but leads to North Bend-Boston Bar. When the ferry is out of service due to ice or low water levels on the Fraser River, pedestrian access is available via a walkway on the Canadian National Railway bridge crossing the river.