In Tunisia, there was confusion among officials over who authorised the extradition.
Libyan interim Prime Minister Abdurrahim El-Keib told reporters on Sunday that the Justice Ministry was holding Al Baghdadi Ali al-Mahmoudi in prison.
"The suspect is in the custody of the judicial police as per an arrest warrant ordered by the prosecutor general's office charging him with committing crimes against the Libyan people," said Keib.
A Reuters team visited Mahmoudi, 62, late on Sunday in his heavily guarded prison but authorities did not allow any interviews or interaction with the former prime minister.
Wearing a white traditional Libyan robe and clutching a cane, the bearded Mahmoudi was helped into an office in the prison by a number of policemen. Sitting silently in a chair flanked by his guards, Mahmoudi looked exhausted and weak, his hair dishevelled, his eyes fixed on a news programme on a television in the room.
"A team led by the military's Chief of Staff, a member of the judicial police and a medical doctor flew to Tunis in a helicopter where authorities handed over Mahmoudi according to an official agreement with Tunisia's government," the general manager of Libya's prisons Khaled al-Shareef told Reuters.
But a spokesman for Tunisian President Moncef al-Marzouki said the president had not authorised any such handover.
"The president did not sign any documents authorising the handover of Baghdadi and he said that if the information about the handover is confirmed, it is up to the prime minister's office to assume responsibility because this happened without the approval of the president," the spokesman said in a statement posted on Marzouki's official Facebook page.
Mahmoudi served as the ousted Libyan leader's prime minister from 2006 until he fled to neighbouring Tunisia around the time that rebel fighters took Tripoli in August last year.
His extradition could establish a precedent for other countries that have given refuge to or arrested members of Gaddafi's old entourage.
Libya's government and the International Criminal Court - which indicted Gaddafi's son Saif al-Islam in June for crimes against humanity stemming from the crackdown on last year's revolt - have argued for months over where he should be tried.
Tripoli considers it a matter of national pride and a mark of Libya's transformation for Saif al-Islam and other Gaddafi loyalists to be tried in Libya. But human rights groups question whether Libya's justice system can meet international legal standards and say Saif al-Islam should be handed to the ICC.
Mabrouk Khorchid, Mahmoudi's lawyer in Tunisia, said neither he nor Mahmoudi's family had been given any warning that he was about to be extradited.
"I believe this is a state crime and is against human rights," he said. "This is a sad moment for human rights in Tunisia. I think he's going to be tortured and treated illegally and believe that those who handed him over bear part of the responsibility."
Khorchid said he had not been allowed to see his client for 20 days and had heard that Mahmoudi had been placed in solitary confinement and had suffered a nervous breakdown since Tunisia's justice minister said last month that an extradition was imminent.
The fate of the former Libyan prime minister has been a dilemma for the Tunisian authorities.
As the first Arab country to stage a successful revolt last year, Tunisia sees itself as standard-bearer for human rights in the region. It has therefore been reluctant to hand over Mahmoudi to a jurisdiction where, rights activists say, he is unlikely to be given a fair trial.
At the same time, Tunisia's economy is stuttering, raising the risk of social unrest that could loosen the government's hold on power. Tunisia badly needs Libyan trade and investment, something Tripoli may have linked to Mahmoudi's extradition.