They will be responsible for security in the south-western regions of Gedo, Bay and Bakool, the AU said.
Ethiopia's contribution takes the AU force to the 22,000-strong level mandated by the UN Security Council.
hiopian forces have been operating in neighbouring Somalia for several years, helping the UN-backed government fight the al-Qaeda-aligned al-Shabab group.
Last year, the UN chief Ban Ki-moon asked for a "surge" of extra troops for the AU force in Somalia, known as Amisom, fearing reversals in advances made over the last few years.
BBC international development correspondent
The Ethiopians will be based in Baidoa, about half way between Mogadishu and the Ethiopian border. It has been heavily defended since being taken over a year ago by Ugandan troops in the Amisom force. Al-Shabab has significant positions in the area and attacks Amisom garrisons almost every day.
The 4,395 Ethiopians are a mixture of fresh troops and soldiers who were already in Somalia on a mission which Addis Ababa sees as defending its borders and many Somalis see as an assault on their sovereignty.
Amisom intends to reshuffle its forces now the reinforcements have arrived. There has long been talk of a big Amisom offensive.
But co-ordination between the various Amisom national contingents is sometimes poor. Speaking on condition of anonymity, a senior officer in one contingent told me late last year that he "wasn't told" when another contingent attacked an al-Shabab position near his area of operations.
Under fire on a dangerous Somali road
Crossing the al-Shabab frontline
Together with government forces, Amisom, has driven al-Shabab from some key cities, including the capital, Mogadishu, in August 2011.
The BBC's international development correspondent Mark Doyle says the troops from the Ethiopian army - one of most battle-hardened in Africa - will be based in Baidoa, some 300km (185 miles) north-west of Mogadishu.
There was a flag ceremony on Wednesday morning in the town to welcome them and hand over the security of the region.
"The Ethiopian deployment will permit Burundian and Ugandan forces to move into parts of Lower and Middle Shabelle," the AFP news agency quotes an Amisom statement as saying.
Ethiopia first entered Somalia in 2006 to remove the now-defunct Union of Islamic Courts (UIC), which had ruled most of southern Somalia for six months that year.
Al-Shabab emerged as the radical youth wing of the UIC as it battled Ethiopian troops.
Our correspondent says that in the 1970s, Somalia and Ethiopia fought a bitter war over their border area and as a consequence many Somalis, who are fiercely nationalist when faced by any foreign forces, have a particular hatred of Ethiopians.
Nonetheless, Amisom will welcome the new troops on its side, he says.
Its soldiers are hit almost daily by al-Shabab roadside bombs, ambushes and rocket attacks, he adds.
The first contingent of Amisom troops arrived in Somalia in March 2007, with Burundi, Djibouti, Kenya, Sierra Leone and Uganda now providing the force's soldiers.
Despite Amisom gains, Islamist fighters still hold sway over many small towns and much of rural Somalia where they have imposed a strict version of Islamic law.
They also control a number of small coastal ports which they use for the lucrative export of charcoal, which fetches high prices in Arab Gulf states.