The jewels in the crown were two architecturally-stunning buildings -- the main "Bird's Nest" stadium and the "Water Cube" aquatics centre, described by International Olympic Committee President Jacques Rogge as "beautiful" and "unprecedented" venues.
"The successful hosting of the Olympics was not only splendid for Chinese sports, it ... excited the passion of one billion people about sport," China's sports minister Liu Peng was cited as saying by state media last year, summing up the Games legacy for Beijing.
Yet today both places are better known for the steady stream of curious tourists they attract -- some 4.61 million visitors in 2011 -- rather than as locations for major sporting events.
While the Bird's Nest does host the odd football match or track and field competition, it has also been the site of a "winter wonderland" theme park and concerts.
The stadium's management estimates that at the current rate, it will take some three decades to recoup the 3 billion yuan ($480 million) cost of building it.
The neighbouring Water Cube lost an estimated 11 million yuan last year, even with a state subsidy and revenue from an attached water park built after the Olympics to capitalise on its fame.
"The cost for building Olympic venues was substantial. But the organisers failed to consider overall how to use the venues after the Olympics when building these sites or even bidding for the Olympics," said Yan Qiang, chief sports editor of NetEase Media Group.
"For sports venues, the more frequently they are used, the longer they will last, the better protection they will receive, and society will benefit that much more," Yan added. "I think Beijing has a severe shortage in this regards."
Other venues have fared even worse than the Bird's Nest or Water Cube.
The kayaking venue sits all but abandoned, what water remaining in it being sucked up by a large pipe to quench a surrounding park in the midst of a typically parched Beijing spring, during a recent visit by a Reuters journalist.
The rowing venue, located in a remote and hard to reach northeastern suburb, now hosts mostly small dinghys.
Neither sport is well-known in China, which partly explains the almost total abandonment.
Some sites, such as for table tennis and wrestling, were built inside universities.
"They were given these huge venues ... and they had no event management experience, and they weren't allowed to get any before the Olympic Games," said Susan Brownell, professor of anthropology and expert on Chinese sports at the University of Missouri-St. Louis.
"After the Games were over they were learning from scratch in terms of how to manage an event," she added.
Plus, in Communist Party-controlled China, there is the added concern over large crowds in the current tense run-up to a once-in-a-decade handover of power for the country's top leadership, which will happen in the autumn.
"In order to hold a major sports event you have to bring thousands of people together, and that's a public assembly. In the current political atmosphere there's just a lot of fear of large public assemblies," Brownell said.
Even the trumpeted closing of polluting factories to improve Beijing's notoriously poor air has only had a limited effect and the city is still regularly cloaked in a think pall of choking smog.
Where once Chinese swelled with pride at the hosting of the Olympics, especially after the country topped the gold medal table in 2008, some now criticise the venues for their wastefulness.
"I think the building materials are very expensive and wasteful," said tourist Li Fang.
The Water Cube "changes water everyday, which is a huge waste of water resources. It also consumes lots of electricity when the lights are on. I think it's better to devote these resources to people's daily life. These expenses are totally unnecessary", the 21-year-old added.