He married twice. He married Margaret Cherry in Clarke County, Alabama, February 14, 1842. He married Elizabeth Ann Gill in Clarke County, Alabama, February 17, 1845. She was the daughter of unknown Gill and Charlotte unknown.

Andrew Jackson Daffin and Elizabeth Ann Gill had the following children:

Nannett Daffin  was born Abt. 1846.

Alice Daffin  was born in Clarke County, Alabama 1848. She married J. R. Lane in Grove Hill, Clarke County, Alabama, November 22, 1869. 

Mary Daffin  was born August, 1850.

Marcus Lafayette Daffin was born April, 1853.
 

Emma L. Daffin was born Abt. 1854. She married Thomas C. Tucker in Falls County, Texas, October 3, 1876. 

Samuel V. Daffin was born August, 1854.
 

Leona Daffin was born Abt. 1857. She married John F. Cook in Falls County, Texas, November 17, 1881. 

Josephine Daffin  was born 1862. She married H. H. Bryon in Falls County, Texas, December 9, 1886. 
 

Jennie Daffin was born 1864.

Mattie Daffin  was born 1866.

He married Frances A. Duncan in Grove Hill, Clarke Co., Alabama, May 26, 1845. 

George was listed as the head of a family on the 1850 Census in Clarke County, Alabama.

George Daffin and Frances A. Duncan had the following children:

Edwin was listed as the head of a family on the 1880 Census in Pickens County, Alabama.
 

Benjamin Daffin was born 1849.

Caledonia Daffin was born 1852.

Jackson Daffin was born 1854.

Timothy Daffin was born 1855.

Jefferson Daffin was born 1856.