The shortlist of candidates to host the world's biggest multi-sports event was announced at the IOC's executive board meetings in Quebec City.
Baku, which invested more than a billion dollars in sports venues in the past 10 years, was bidding for a second successive time after falling at the first hurdle for the 2016 Games.
Doha, which was proposing to hold the 2020 Olympics in October rather than the usual July/August schedule to avoid the Gulf Arab state's searing summer heat, is already hosting the 2022 World Cup soccer tournament.
The IOC will announce the winner between bids from Tokyo (Japan), Madrid (Spain) and İstanbul (Turkey) in September 2013.
Rome (Italy) pulled out of the running in February due to the country's efforts to head off a debt crisis.
London will host the 2012 Summer Games from July 27-Aug. 12 while the Brazilian city of Rio de Janeiro will stage the 2016 Olympics.