No doubt, I was oversimplifying the situation.
Just for fun, consider ergot has similar effects on humans, if ingested, to those of LSD.
Now imagine thousands of tripping, hungry, ticked off peasants...
Wheat made finer flour and was used for cake, and to some degree, for bread, but rye was a staple for bread as well--especially for poor folks. Contaminated rye would mean the peasants had no bread to eat...hence 'let them eat cake' (of uncontaminated wheat flour).