Supposedly, Mary Jane's first husband Charles William Barnum ran off to South America with one of the maids who worked at the family's railroad boarding house. Mary Jane, alone with just her small son, found life to be very hard. She later met and married E. H. Patterson, who also worked for the railroad. After a time they traded the hotel for a farm west of Ponchatoula on Highway 22, known then as Perrins Ferry Road. The farm was known as "The Van Dyke Place." After a few years they moved to Blood River, where two children were born to them.
The 1870 US Census lists her birthplace as Canada.