"Mr Sayedul was elected MP six times from that area. It's unbelievable that he made such illogical, unacceptable comments," Awami League Joint General Secretary Hanif said at a press conference on Saturday.
"He will face organisational action if someone can provide evidence," he added.
On Oct 30, mobs attacked Hindu temples, houses and businesses in Sayedul's constituency Brahmanbarhia's Nasirnagar following a Facebook post allegedly insulting Islam.
The minister, who went to visit his area three days after the attacks, accused the media of 'exaggerating' the attacks and also reportedly made a derogatory comment about the Hindus.
One daily newspaper reported that he called the Hindus 'Malaun', a derogatory term derived from an Arabic derivative meaning 'accursed' or 'deprived of God's Mercy'.
Protesters across the country also demanded Sayedul's resignation as there were similar attacks two days after his visit. A temple and several houses were torched at that time.
On Friday, Bangladesh Puja Udjapon Committee General Secretary Taposh Kumar Paul said, "The Hindus were attacked again after his visit. It proves that he is linked with the incident."
Many are blaming factional feud in Awami League for the attacks.
The party has suspended three of its leaders in Nasirnagar over the attacks. They are known to be loyal to Sayedul.
Referring to the Awami League’s move, its rival BNP has said it is 'clear' that the party was 'involved' in the attacks.
Awami League leader Hanif, however, said the leaders were temporarily expelled for their 'failure' to stop the attacks.