The clampdown has led to clashes in the capital, Riyadh, with at least five people killed.
Saudi authorities say they are trying to reduce the 12% unemployment rate among native Saudis.
An estimated nine million migrant workers are in Saudi Arabia.
They are said to make up more than half the workforce, filling manual, clerical and service jobs.
Ethiopia's ambassador in Riyadh, Muhammed Hassan Kabiera, said the embassy had been informed by Saudi officials that some 23,000 Ethiopians had so far handed themselves in.
Some of them have already been repatriated, with the first group arriving in Ethiopia's capital, Addis Ababa, on Wednesday, reports from Ethiopia say.
In renewed clashes on Wednesday in Riyadh's Manfuhah district, a Sudanese national was killed, Saudi Arabia's state-owned SPA news agency reports.
Illegal migrants "rioted, hurling rocks at passersby and cars", it quoted police as saying.
Police said they intervened and "controlled" the situation, SPA reports.
Manfuhah is home to many migrants, mostly from East Africa.
On Sunday, Ethiopia's Foreign Affairs Minister Tedros Adhanom said he had information that three Ethiopian citizens had been killed in clashes since last week.
However, Saudi authorities say two foreigners have been killed, along with three Saudis.
Riyadh governor Prince Khaled bin Bandar bin Abdulaziz said the clamp down was aimed at illegal migrants, and not any "specific group".
"We will continue these campaigns until we ensure all residents in our country are staying legally," he is quoted by al-Riyadh newspaper as saying.
Earlier this month, the authorities began rounding up the migrants following the expiry of a seven-month amnesty for them to formalise their status.
Nearly a million Bangladeshis, Indians, Filipinos, Nepalis, Pakistanis and Yemenis are estimated to have left the country in the past three months.
More than 30,000 Yemenis have reportedly crossed to their home country in the past two weeks.
Four million other migrants obtained work permits before the deadline expired.