In 1846, Mr. Whitfields first wife died. After she died, Mr. Whitfield hired a nanny, Evelyn Carter, to help with the house chores and to take care of the children. Ms. Carter would sing songs to the family and even knew how to play various instruments. Not known when, but Ms. Carter died after a long battle of an unknown illness on a cold winters day. Ms. Carter made sure to let Mr. Whitfield know, that if she happen to die before him, she was to be buried in a family cemetery back in Virginia. However, when Ms. Carter died, it was far too cold to get her back to Virginia, so Mr. Whitfield buried her under the house. It is said that Ms. Carter still haunts the plantation due to the fact that she never made it home to be buried the way she requested. People have stated that the voices they hear, the songs that are being sung and the piano being played, even though most of the keys no longer work and the tune is way out of whack, is all from Ms. Carter. Silk skirts rustling, footsteps in the guest stairway and the piano being played at midnight during the restoration of the mansion in 1970's, have also been heard.
There is also a river that flows by the plantation called Tombigbee River. In December 1858, a steamboat by the name of Eliza Battle, had caught fire and most of the two hundred plus passengers either drowned or they died of hypothermia. Mr. Whitfield went down to the river to see if he could be of any help, not sure if he was able to or not. He did manage to paint a portrait of the fire, and it still is hanging in the plantation to this day. People have claimed that they can see the Eliza Battle still burning.
After Mr. Whitfields death, his decedents took over the home and the 480 acres that surrounded it, until 1923. There were two other families that took over the property after that, and then in 1966 the Alabama Historical Commission bought the home and the property. The house was turned into a museum in 1971, and it remains a museum until this day.