It would be easy for a male to become involuntarily aroused in a female bathroom. Regardless of the 'ick' factor, female urine contains pheromones which can stimulate a man into sexual arousal. "The scent of a woman" is a real thing. And when they take their pants or dress off and pull their panties down, the scent alone of doing that is simulating.
The olfactory stimulations happening in that environment to a male combined with the simple psychology of knowing that two feet away from you is a woman or girl with her skirt hiked up and her panties down could easily be difficult for some males, especially young males, to endure without release of some kind. No doubt many of the men using the female bathroom are using it for purposes other than urination or defecation.
So, the idea of, "I'm only here to use the lady's bathroom" is a very viable, and I would say likely, occurrence. It probably happens a lot more than we know. If you hear some guy in the stall next to you huffing and puffing and grunting, you can guess what he is doing.