President Saleh had promised to step down but failed to sign a deal to do so
Questions are being asked whether Mr Saleh - who has several times promised to give up power - will return, a BBC correspondent in Yemen says.
His departure leaves him in a much weakened position.
An uprising demanding that Mr Saleh leave power has led to violence bringing Yemen close to civil war.
Yemen's acting president - previously the vice-president who under the constitution replaces Mr Saleh in his absence - Abd-Rabbu Mansour Hadi, has taken over, including command of the armed forces and security services.
He is to meet members of the military and Mr Saleh's sons, al-Arabiya television is reporting, quoting sources.
He has held talks with the US ambassador in Yemen - in what will be seen as support for the handover of power, says BBC Middle East correspondent Jon Leyne in Cairo.
Everything suggests that Mr Saleh's visit to Saudi Arabia for medical treatment will be a one-way trip, our correspondent says.
If President Saleh is gone for good, a new battle for power will begin, with the vice-president, President Saleh's eldest son, tribal leaders and a popular protest movement attempting to take control, he adds.
The president's departure has prompted celebrations in the Yemeni capital, Sanaa. Thousands of people came out on the streets chanting and waving flags and banners. Many youths celebrated at Sanaa's University Square.
But explosions and gunfire have been heard in the city, with correspondents reporting renewed fighting between Yemeni military and fighters of President Saleh's rival Ahmar tribe.
The BBC has also learnt that at least five people were killed in a grenade attack at a military compound north of Sanaa.
A military source inside the army's fist division, led by defected Gen Ali Mohsen, has told the BBC a bomb went off in the compound by mistake.
Five people are also reported to have been killed in gunfights in the flashpoint city of Taiz in the south of the country, witnesses and reports say.
Family networkThere is a question mark over whether the defiant president, who has promised to give up power several times, will return. His vice-president is now the acting president.
Many here see this as the second stage of the Gulf Countries proposal aimed at putting an end to Yemen's political crisis which President Saleh has so far refused to sign. Stage one was getting President Saleh to hand over power to his vice-president which would then be followed by early elections.
Many here see his departure as end of his power and the country has now entered a transitional phase after Saleh.
Sources close to President Saleh have confirmed to the BBC that some members of the family have left the country with him, including his wife.
Reports also say the prime minister and other ministers are in Riyadh for treatment.
But his son, Ahmad, and his nephews, Ammar and Yehia, who are Mr Saleh's military commanders, are reported to still be in the country.
Ahmad commands the elite Republican Guard, and other relatives control security and intelligence units.
Ammar and Yehia have co-operated with the United States in fighting terrorism. Many in Yemen believe their presence in this transitional period is essential and welcomed by regional and international powers, says the BBC's Lina Sinjab in Sanaa.
Mr Saleh flew to the Saudi capital, Riyadh, on Saturday on a medical plane to have an operation after being hit by shrapnel three inches (7.6cm) below the heart in a rocket attack on the presidential compound on Friday.
Uncertainty surrounded his whereabouts for much of Saturday.
A Gulf nation diplomatic source told BBC Arabic that the decision to transfer Mr Saleh to Riyadh was taken after Saudi doctors consulted with a German medical team.
Ceasefire reportedMr Saleh and several senior officials were praying at the al-Nahdayn mosque inside the presidential compound in the south of Sanaa on Friday afternoon at the time of the attack.
The mosque was originally thought to have been hit by rockets, but there are now suggestions someone may have planted a bomb there.
The attack on the mosque left seven of Mr Saleh's bodyguards dead and several officials wounded Government officials have accused armed tribesmen allied to Mr Saleh's opponents of carrying out the attack which killed seven people, but they have denied it. An analyst close to President Saleh has suggested that Friday's attack was a bomb planted by a member of Saleh's own people but it's hard to confirm.
The president broadcast an audio message on Friday after he was wounded, but did not appear in public.
In the broadcast, he blamed the attack on an "outlaw gang" of his tribal foes - an accusation denied by Sheikh Sadeq al-Ahmar, the head of the Hashid tribal federation, whose fighters have been clashing with security forces.
More than 160 people have been killed in the fighting that began on 23 May and has brought Yemen to the brink of civil war.
The prominent Ahmar family has been financing the opposition and helping sustain protesters, who have been demanding Mr Saleh's resignation since January despite a crackdown that has left at least 350 people dead.
Western and regional powers have been urging Mr Saleh, who has ruled the country for 33 years, to sign a Gulf Co-operation Council-brokered deal that would see him hand over power to his deputy in return for an amnesty from prosecution.
He has agreed to sign on several occasions, but then backed out.
Hundreds of people have died in violent clashes which have escalated in the past few weeks.
