That is exactly the reason why the law forces the accuser to face the accused.
To do otherwise makes a mockery of all of our laws.
During the Inquisition, prisoners were forced to name a certain number of 'others' under torture (witches, traitors, malcontents, etc.) or they would be killed. They would frequently simply name a few names they knew at random, and then those people would be arrested with no charge, accuser, and no evidence, and the system would perpetuate that way. All the accusers were nameless.
A person could accuse his neighbor or a shop owner anonymously for no reason other than he wanted his property or maybe his even wife. Anonymous accusers do in fact change justice from a matter of actual 'law' into more of a political thing.