We want the government to abrogate the law and to release all political and prisoners of conscience immediately, Negasso Gidada, the leader of the opposition Unity for Democratic Justice (UDJ), told AFP.
Several opposition members and journalists, including dissident blogger Eskinder Nega, have been jailed under the 2009 legislation.
The government accused the group of glorifying convicted criminals and said the threat of terrorism in Ethiopia needed to be taken seriously.
These people are downplaying the danger that this country has been facing... its not a potential threat, it's already there, said government spokesman Redwan Hussein.
Negasso said he was briefly arrested ahead of the protests along with the single member of parliament from an opposition party, Girma Seifu, and 60 other people.
Redwan said he did not know of anybody who had been detained.
The demonstration is part of a three-month campaign launched by UDJ, which has held protests in several cities across the country.
Negasso estimated the crowd at 80,000 people, while Redwan said only a few hundred had gathered.
The group, led by five people dressed in prison uniforms, chanted slogans demanding freedom and the release of prisoners, including UDJ member Andualem Arage who is serving a life sentence after he was convicted under the law.
Rights groups have accused Ethiopia's anti-terrorism legislation of being vague and used to stifle peaceful dissent.
Last year, two Swedish journalists jailed on terror-related offences were released after serving 13 months of their 11-year sentence.