His admission came on Wednesday in the wake of protest by the left parties including the Communist Party of Bangladesh or CPB against the corruption in the banking sector in the capital.
Muhith spoke to the reporters after he attended a meeting of the cabinet committee on government procurement.
"There are weaknesses in the banking sector, that's all."
The finance minister, however, advised the central bank to stay alert to avert scams similar to the ones that rocked Sonali Bank and BASIC Bank.
He also said the government was worried about the repeated failure, corruption and scam in state-owned banks. Likewise, some private banks were also suffering from similar issues, he added.
The finance minister said the left parties took up the protest programme for the 'sake of their own survival.'
A joint procession of CPB, BaSoD and Ganatantrik Baam Morcha was stopped by the police at the Dainik Bangla intersection on Wednesday when the leaders and activists of the parties tried to march towards the central bank. They also locked in a brief scuffle with the police around noon.
"The government facilitated this plundering," said BaSoD leader Rajekuzzaman Ratan at the rally at Dainik Bangla square.
Criticising the protest procession, the finance minister said, "This is nothing but nonsense. They don't have any existence; they have a few leaders; that's all. They are doing this carrying the red flags for their survival."
Muhith is known for his attachment to left politics during his university days. He was a member of the cultural wing of the CPB-leaning student organisation, the Chhatra Union, when he was a student of Dhaka University in the 1950s.