The Old Trafford Test was cancelled last Friday barely two hours before its scheduled start as the tourists, who were leading the series 2-1, failed to field a side after their physiotherapist tested positive for COVID-19.
"The players refused to play but you can't blame them," Ganguly told Indian newspaper The Telegraph on Monday.
"Physio Yogesh Parmar was such a close contact of the players ... he mixed freely with the players and even performed their COVID-19 tests.
"He also used to give them a massage, he was part of their everyday lives.
"The players were devastated when they came to know that he had tested positive for COVID-19. They feared they must have contracted the disease and were dead scared."
Former England captain Michael Vaughan has suggested that India's players were "petrified" of testing positive ahead of the IPL, which resumes on Sept 19 in the United Arab Emirates.
But England and Wales Cricket Board CEO Tom Harrison has said the cancellation had nothing to do with the IPL and Ganguly echoed those comments.
"The BCCI will never be an irresponsible board. We value other boards too," he said, adding that the match is likely to be held as a one-off match next year.