"One of the things I've learned as president is you represent the entire country," Obama said on CBS's "Late Show with David Letterman."
"My expectation is if you want to be president, you've got to work for everybody, not just for some," he said to applause from the studio audience.
In the video, the first part of which was published on Monday by the liberal Mother Jones magazine, Romney told donors that 47 percent of Americans would back Obama no matter what and "my job is not to worry about those people."
Romney said they did not pay income taxes and were people "who are dependent upon government, who believe that they are victims, who believe the government has a responsibility to care for them."
Obama said he tried to reach out to people who did not support him after his victory four years ago.
"When I won in 2008, 47 percent of people who voted, voted for John McCain. They didn't vote for me," Obama said.
"And what I said on election night was, even though you didn't vote for me, I hear your voices and I'm going to work as hard as I can to be your president."
Obama, who as a candidate in 2008 made a damaging gaffe of his own that white rural voters clung to guns or religion, indicated he thought Romney should admit his remarks were a mistake.
"When you run for president, you are under a microscope all the time. All of us make mistakes. That incident in 2008, I immediately said, 'I regret this,'" Obama said.
"What I think people want to make sure of is you're not writing off a big chunk of the country."
Despite the focus on the video that was shaking up the presidential campaign, Obama's appearance was not wholly serious in tone. Upon arriving on stage, Obama told the comedian that he looked sharp.
"You haven't seen me naked," Letterman replied.
"We're going to keep it that way," Obama retorted.
Letterman opened his show, which airs later on Tuesday, by poking fun at an incident at the Republican National Convention in which film star Clint Eastwood delivered a diatribe addressed to an imaginary Obama in an empty chair.
"I want to know before we get started if you want to say something to the empty chair," Letterman said. Obama smiled.