Several firefighting units have been dispatched to the factory in the Bayezid Textile area
According to locals, some Islami Chhatra Shibir activists took out a procession at the city's Dorgar Gate area around 6.30 am.
Shibir is the student front of the Jamaat-e-Islami.
They tried to block the road towards the airport by setting flammable objects on fire and exploded a few handmade bombs, but fled before the police could reach the spot.
Sylhet police's Additional Deputy Commissioner Rahmat Ullah said no incidents of violence were reported elsewhere on the city.
Additional police personnel have been deployed at key points in the city to avert violence, he said.