Sprint great Singh, who was nicknamed the "Flying Sikh", ran at three successive Olympic Games and finished fourth in the 400 metres in Rome in 1960 when the first four home all ran faster than the previous world record.
His death due to COVID-19 complications comes days after his wife, former India volleyball captain Nirmal Kaur, had died due to similar reasons.
"In the passing away of Shri Milkha Singh Ji, we have lost a colossal sportsperson, who captured the nation's imagination and had a special place in the hearts of countless Indians," Modi tweeted.
"His inspiring personality endeared himself to millions. Anguished by his passing away.
"I had spoken to Shri Milkha Singh Ji just a few days ago. Little did I know it would be our last conversation. Several budding athletes will derive strength from his life journey. My condolences to his family and many admirers all over the world."
Singh, the father of former Asian Tour number one golfer Jeev Milkha Singh, was a popular figure throughout his life and inspired a biopic titled "Bhaag Milkha Bhaag" (Run, Milkha, Run) which was released in 2013.
The two-times Asian Games champion in 400m and Commonwealth champion in 1958 famously sold his life story to the film maker for a cut price one rupee in the hope that the biopic might inspire the youth of India.