Dozens of people, mostly soldiers, were also wounded in the bombing, which left blood and body parts scattered over a wide area.
Yemen's defence minister and chief of staff were both present at the event, a rehearsal for Tuesday's National Day parade, but neither was hurt.
Officials suspect the attack was carried out by a rogue soldier.
A man claiming to speak for militant group Ansar al-Sharia said in a telephone call to Reuters it was behind the attack.
An Ansar al-Sharia spokesman subsequently confirmed the claim, saying it was in response to the "crimes" of the security forces who are fighting to dislodge militants from their strongholds in the south of the country.
The authenticity of the claim could not immediately be verified.
The attack is also sure to heighten concern in Washington, for whom Yemen is a frontline state in its global war on Islamic militants.
A US military instructor was shot and seriously wounded on Sunday in an attack also claimed by Ansar al-Sharia, which is affiliated to al Qaeda in the Arabian Peninsula (AQAP)
Militants have exploited political instability to gain a foothold in a country paralysed for most of 2011 by protests that eventually unseated President Ali Abdullah Saleh.
The huge explosion left scenes of carnage, with bloodied victims strewn across the 10-lane road where the rehearsal was held on Monday morning not far from the presidential palace.
A police source said 63 soldiers were killed, some instantly and others dying later in hospital. At least 60 people were also wounded, he said.
"We had just finished the parade. We were saluting our commander when a huge explosion went off," said soldier Amr Habib. "It was a gruesome attack. Many soldiers were killed and others had their arms and legs blown off."
One of the investigators said preliminary findings suggested the suicide bomber was a rogue soldier rather than a man in a disguise.
"The suicide bomber was dressed in a military uniform. He had a belt of explosives underneath," said a man who identified himself as Colonel Amin al-Alghabati, his hands and uniform flecked with blood.
The usual security procedure for such an event would be for checks to be made on the soldiers at their bases before they are transported to the site of the parade in army vehicles
The wounded were ferried to hospital in taxis.
"Most of the injuries are to the head, we have dozens paralysed. We expect the death toll to rise. Most of the injured here are boys in their teens. Sanaa's hospitals are overwhelmed," said doctor Mohsen al-Dhahari.
The impoverished state has seen a spate of deadly attacks since President Abd-Rabbu Mansour Hadi replaced Saleh in February saying he would extinguish an Islamist insurgency, which until now has been concentrated in the south.
Yemeni troops closed in on the southern militant-held town of Jaar on Sunday in heavy fighting, part of a new US-backed offensive launched earlier this month to regain control of territory and towns seized by Ansar al-Sharia.
The parade was scheduled for Tuesday to mark the unification of north and south Yemen, previously separate states, which were merged in 1990.