Rahul proved the steadying factor in the batting line-up, after taking over from Virat Kohli as skipper just before the start of play, before being dismissed for 50.
Kohli had to pull out of the test because of an upper back strain, handing over the captaincy to Rahul, who played an organised and circumspect innings before a stray shot saw him caught at fine leg by a diving Kagiso Rabada off the bowling of Marco Jansen.
It proved the big wicket of the session for the home team, who added two more after having India 53-3 at lunch.
It was a second wicket for Jansen, who had claimed the first scalp of the test when opener Mayank Agarwal feathered a slight touch to the tall left-arm quick to depart for a brisk 26 off 37 balls.
Duanne Olivier then took two wickets in two balls before lunch with Cheteshwar Pujara caught off a top edge for three followed by Ajinkya Rahane, out first ball as he was caught by Keegan Petersen at second slip.
The 29-year-old Olivier was returning to test cricket for the first time in almost three years, having exploded onto the scene in 2017 and snagging 48 wickets in 10 tests at an average below 20, but then taking up a Kolpak contract with English county Yorkshire that meant he could not play for South Africa.
But after returning home when the Kolpak system fell away due to the United Kingdom's exit from the European Union, Olivier finished the top wicket-taker in the domestic four-day competition and has won back a test place.
After lunch, Rahul and Hanuma Vihari, who was brought into the team to replace Kohli, put together a 42-run partnership before Vihari was caught at short leg by a flying one-handed catch by Rassie van der Dussen off Rabada.
At tea, Rishabh Pant was 13 not out and Ravichandran Ashwin had scored 24 runs off only 21 balls.
India won the first test in Pretoria last week and are looking for a first series triumph in South Africa.