The battle inside the Cadereyta prison outside the city of Monterrey began overnight, officials said. Anxious family members gathered outside the prison during the day waiting for news of their loved ones.
A protest by inmates overnight had spiralled out of control, one prisoner was killed and a number of guards were taken hostage, Nuevo Leon state security spokesman Aldo Fasci told reporters.
Police were sent in to calm the disturbance but the fight soon escalated to involve around 250 inmates, he said.
"Around five in the afternoon, the decision was made by security forces to use lethal force to prevent the murder of guards as well as stop the murder of inmates," Fasci said.
"If we had not taken this decision, we would be talking about many more deaths," he said.
Rival gangs that are involved in drug trafficking, human smuggling, fuel theft, kidnapping and extortion have fuelled bloody battles in Mexico's overcrowded prisons.
At least nine were killed in a prison fight in the state of Tamaulipas on the US border with Texas and at least 28 inmates were killed in July in a battle in a prison in the Pacific resort of Acapulco.