Elgar defied a hostile Indian attack and body blows from balls that struck him on a difficult pitch to deliver a match-winning performance as he led a successful 240-run chase, fittingly hitting a boundary to secure the victory.
South Africa reached 243-3 near the end of a rain-restricted fourth day to level the series after India had won the first Test in Pretoria by 113 runs, with the deciding match to come in Cape Town, starting on Tuesday.
The home team were 118-2 overnight but had to wait as the first two sessions on Thursday were washed out by rain.
Elgar and Rassie van der Dussen eventually resumed their run chase at 3.45pm local time (1345 GMT) under cloudy skies with the outcome still in the balance as the pitch was expected to favour the formidable India attack.
But the two batsmen continued their partnership and added 57 more runs in the first hour’s play as they took on the tourists’ bowling, before Van der Dussen edged Mohammed Shami to first slip and was out for 40.
South Africa still needed 65 to win when the wicket went down and might have quickly spiralled into trouble had Shardul Thakur held onto a catch when new batsman Temba Bavuma hit the second ball he faced straight back at the bowler.
Thakur was unable to hold onto a stinging shot, and Bavuma went on to make full use of his good fortune to finish 23 not out.
But Elgar was the hero, showing a dogged fighting spirit and leading by example to hand his young team an important and morale-boosting win. He faced 188 balls, hitting 10 boundaries, and was deserving of a century.
"I’d like to think that the knocks just gave me added motivation," he said.
"The bigger picture is to win and I wanted to show our young group that sometimes taking the blows can be worth it."
It was the first time South Africa had beaten India at the Wanderers and keeps alive their hopes of denying the world's top-ranked team a first-ever series victory in South Africa.
"I felt we could have scored 60 or so more runs in the first innings and given ourselves a bit of a lead. That's really where we let the game slip," added India's stand-in skipper KL Rahul.