Such an agreement to beef up measures against pandemics is expected to be ready in May 2024, covering issues from data sharing and genome sequencing of emerging viruses to equitable distribution of vaccines and drugs derived from research.
"The adoption of this decision is cause for celebration and a cause for hope that we all need," WHO director-general Tedros Adhanom Ghebreyesus told the meeting of health ministers.
"Of course there is still a long road ahead. There are still differences of opinion about what a new accord could or should contain," he said, calling for continued cooperation.
In the meantime, countries should abide by the WHO's 2005 International Health Regulations, he said.
The decision, entitled "The World Together", was adopted by consensus at a special assembly of the 194 nations that are members of the UN health body, drawing applause at the end of a three-day meeting.
"The text before us is the product of extensive discussions, of frank exchanges and of compromises," said Australia's ambassador Sally Mansfield, who co-chaired the working group.
The European Union (EU) had pushed for agreement on an international legally binding treaty, along with about 70 countries, but Brazil, India and the United States were among those reluctant to commit to a treaty, diplomats said.
"We call for an ambitious process in developing this treaty - let us all demonstrate our multilateral commitment and engagement towards a binding instrument," Ambassador Lotte Knudsen, head of the EU delegation to the United Nations in Geneva, said in a statement on Wednesday.
The United States welcomed the decision saying in a statement: "This momentous step represents our collective responsibility to work together to advance health security and to make the global health system stronger and more responsive."
More than 262.22 million people have been reported infected with SARS-CoV-2, the virus responsible for COVID-19, and 5.46 million killed since it emerged in China in December 2019.
The WHO says China has still not shared some of its early data that might help pinpoint the origin of the virus.
China said on Monday that it was ready to work on developing a new international agreement which it hoped would avoid "politicisation, stigmatisation and instrumentalisation".