Based on interviews with some 550 workers in 53 H&M and Gap supplier factories in Bangladesh, Cambodia, India, Indonesia, and Sri Lanka, rights groups said women were at "daily risk" of violence, and faced retaliation if they reported the attacks.
The coalition has investigated the factories for several years as efforts mount to push Western brands into improving safety along their supply chains and render them slave-free.
Sweden's H&M - the world's No. 2 clothes group after Zara owner Inditex - said it would review the findings of the recent report by the civil society groups and unions.
"We will go through every section of the report and follow up on (a) factory level with our local teams based in each production country," a company spokesman said in a statement.
"All forms of abuse or harassment are against everything that H&M group stands for."
US retailer Gap said it was "deeply concerned about the troubling allegations raised by this report".
"Our global team is currently conducting our due diligence to investigate and address these issues," a spokeswoman said.
The charities said they had found widespread sex harassment, verbal and physical abuse - such as slapping - and threats of retaliation when women refused sexual advances from bosses.
A separate report published last month by the coalition of rights groups found similar abuse of women at supplier factories in Asia for US-based Walmart, the world's largest retailer.
Walmart said last month that it was reviewing the "concerning" accounts cited in the report.
The Ethical Trading Initiative (ETI), a group of trade unions, firms and charities of which both Gap and H&M are members, said it expected the retailers to work with the suppliers to ensure that women have swift access to remedy.
"These allegations are deeply concerning," said Debbie Coulter of the ETI. "Gender-based violence is unacceptable under any circumstances, and brands need to make sure that women working in their supply chain are protected."
Campaigners told the Thomson Reuters Foundation last month that the level of pressure and harassment faced by the workers in the three separate reports was approaching forced labour.
"Any time you have retaliation against workers, and coercion and control ... you are coming close to the line of forced labour," Jennifer Rosenbaum of Global Labor Justice (GLJ), a network of worker and migrant organisations, said last month.
The reports have been published amid meetings hosted by the United Nations' International Labour Organization (ILO) to work on the first global convention against workplace harassment after the #MeToo campaign thrust the issue into the spotlight.