It was Galgut's third nomination for the 50,000 pounds ($68,175) English language literary award.
"It's taken a long while to get here and now that I have, I kind of feel that I shouldn't be here," he said in his acceptance speech.
"This has been a great year for African writing and I would like to accept this on behalf of all the stories told and untold, the writers heard and unheard from the remarkable continent that I'm part of. Please keep listening to us, there's a lot more to come."
Starting in the mid-1980s and set just outside Pretoria, "The Promise" transitions through several decades and South African presidents.
Told through four family funerals, it begins with a young girl hearing her father promise her dying mother their maid will get the deeds to the annexe where she lives - something he later reneges on.
"'The Promise' astonished us from the outset as a penetrating and incredibly well-constructed account of a white South African family navigating the end of apartheid and its aftermath. On each reading we felt that the book grew," historian Maya Jasanoff, chair of the judges, said in a statement.
"With an almost deceptive narrative economy, it offers moving insights into generational divides; meditates on what makes a fulfilling life - and how to process death; and explores the capacious metaphorical implications of 'promise' in relation to modern South Africa."
Galgut, who wrote his first novel at the age of 17, follows in the footsteps of past South African Booker Prize winners Nadine Gordimer and J. M. Coetzee.
He has been nominated for the Booker Prize twice before: in 2003 for "The Good Doctor" and in 2010 for "In a Strange Room".