Before and during the Civil War, the Hunt-Phelan House welcomed politicians and high-ranking military men of every stripe. Both Jefferson Davis and Andrew Johnson visited here, and in 1861, Gen. Leonidas Polk stayed in the house while he organized the Provisional Army of Tennessee. As the war progressed, the owner, Col. William R. Hunt, served as superintendent of the Confederate ordnance works at Columbus, Mississippi. Designed by famed architect Robert Mills, the original part of the Federal-style house was built ca. 1840 by George H. Wyatt. Retired land commissioner Eli M. Driver purchased the property five years later and left it to his daughter, Sarah Elizabeth, when he died in 1851. She and her husband built a service wing at the rear and added a large Greek Revival portico to the facade.
