He said the chief justice was the ultimate decision maker on the matter.
"Chief Justice is the guardian of Supreme Court. He will make the call ultimately," the Law Minister said.
"But we must ensure that the debate should not create any chaos. After all, the Supreme Court is a holy place," added the minister.
The minister made the remarks while talking to reporters after inaugurating a training course of the International Labour Organization in capital Dhaka on Tuesday.
Prime Minister Sheikh Hasina concurred with the demand on Apr 11 when a delegation headed by Hefazot chief Amir Shah Ahmad Shafi met her at her official residence, the Ganabhaban.
Islamist groups have noted that the statue resembles the Roman symbol of justice, 'Lady Justice,' and thus 'pollutes the apex court.'
When asked on the observation, Anisul said: "It has sparked some questions. Now you take the hint."
He highlighted that labourers were being deprived for an insufficient number of labour courts and the prolonged trials.
He suggested shifting the three labour courts from Dhaka's Motijheel to Narayanganj and Tongi.