The Barnum Family, 1350-1907 calls him Edson Barnum.
In the 1850 US Census for Southbury, New Haven County, Connecticut the family of James E. Barnum was enumerated as follows:
Dwelling #8; Family #8
James E. Barnum, 39, M, Farmer, b. Connecticut
Cynthia C. Barnum, 31, F, b. Connecticut
James E. Barnum, 5, M, b. Connecticut, Attended school within the year
Chas. F. Barnum, 2, M, b. Connecticut
Ann E. Hinman, 10, F, b. Connecticut, Attended school within the year