Two gunmen were killed on Saturday night. It was initially reported that four gunmen had attacked the hotel.
Interior Ministry spokesman Najib Danish said at least five other people had been killed and six wounded, a lower casualty total than earlier feared, while 153 people, including 41 foreigners had been evacuated.
As day broke on Sunday, thick clouds of black smoke could be seen pouring from the building. Several armoured US military vehicles with heavy machine guns could be seen close to the hotel along with Afghan police units.
The raid came just days after a US embassy warning of possible attacks on hotels in Kabul.
There was no immediate claim of responsibility.
The raid was the latest in a long series of attacks which have underlined the city's precarious situation and the ability of militants to mount high profile operations aimed at undermining confidence in the Western-backed government.
According to one witness, who did not want to be named, the attackers took hotel staff and guests hostage.
The Intercontinental Hotel, an imposing 1960s structure set on a hilltop and heavily protected like most public buildings in Kabul, was previously attacked by Taliban fighters in 2011.
It is one of two main luxury hotels in the city and had been due to host an information technology conference on Sunday. More than 100 IT managers and engineers were on site when the attack took place, Ahmad Waheed, an official at the telecommunications ministry, said.
The attack, just days after a United Nations Security Council visit to Kabul to allow senior representatives of member states to assess the situation in Afghanistan, may lead to a further tightening of security in Kabul.
Large areas of the city centre are already closed off behind high concrete blast walls and police checkpoints but the ability of the attackers to get into a well-protected hotel frequented by both government officials and foreigners demonstrated how difficult it remains to prevent high profile attacks.
Danish said a private company had taken over security of the hotel about three weeks ago.
The State Department said on Saturday it was monitoring the situation and was in contact with Afghan authorities to determine whether any US citizens had been affected.
Captain Tom Gresback, spokesman for the NATO-led Resolute Support mission in Afghanistan, said they were also watching closely but it was not clear what role international forces were taking in suppressing the attack.
"Afghan National Defense and Security Forces are leading the response efforts. According to initial reports, no Resolute Support or (US forces) members were injured in this incident," he said in an emailed statement.
Although Resolute Support says the Taliban has come under pressure after the United States increased assistance to Afghan security forces and stepped up air strikes against insurgents, security remains precarious.
As pressure on the battlefield has increased, security officials have warned that the danger of attacks on high-profile targets in Kabul and other cities would increase.
After repeated attacks in Kabul, notably an incident last May in which a truck bomber killed at least 150 people outside the German embassy, security has been further tightened.
While it shares the same name, the hotel in Kabul is not part of InterContinental Hotels Group (IHG), which issued a statement in 2011 saying that "the hotel Inter-continental in Kabul is not part of IHG and has not been since 1980".