The arrestees are Shahahurul Isla, 30, and Milon Mia, 40, both from the Upazila.
They were detained for suspicious movement when the leaders of the Upazila council and the Awami League, and officials of the local administration were paying their respects to the language martyrs at the monument around 12:30am on Sunday.
Later, the crude bombs and the knife were recovered from them, said Afzal Hossain, inspector at Gobindaganj Police Station.
The police started a case against them under the Explosives Act and the Arms Act. A court sent them to jail in the afternoon.
Mehedi Hasan, OC of Gobindaganj Police Station, said the court is set to hear on Monday a petition seeking a five-day remand of the duo to know why they were roaming in the area with the bombs and knife.
The bombs have been deactivated, the police said.