When revolutionary soldier and physician Andrew Walthour settled Walthourville, he turned its land into the richest cotton plantation in the region. By 1844, residents considered the town one of the wealthiest communities in Southern Georgia.
In the 20th century, visitors came to know the town for its lavish homes, those belonging to elite town residents Captain Edward Fleming, George W. Walthour, Dr. Raymond Harris and others.
Because mayors serve long terms, the town has had relatively few and interestingly, many of them have been women. In fact, Walthourville seated an African-American woman as mayor before any other town in the state did.
A famous hometown personality is Robert "Bobby" Walthour, the two-time world champion cyclist.