Neither kits, cats, sacks, nor wives were going to St. Ives. Only one person - the narrator - was definitely going to St. Ives.
You are correct but you are wrong.
The riddle does not ask if the riddler was going to St. Ives.
It asks: "Kits, cats, sacks, wives, How many (of those!) were going to St. Ives?"
I was cruising from Ft. Lauderdale to Los Angeles in 2013 when I pondered this lovely riddle and posed it to my poker buddies. They all agreed with my analysis.
The reflexive "how many" refers to the immediately precedent subject.
Going through Panama, I won a big pot at six card hi/low Monte with a club wheel, ace through five, both ways.
Declare before showing. One chip in your hand for low, two in your hand for high, three for both ways. You have to win both ways or you lose.
__________________________
The Overcrowded PrisonA prison was overcrowded, and the warden decided to release a prisoner by giving a test.
He had three prisoners brought to his office and explained his proposition.
"The guard behind you has a duffel bag with 5 hats in it. Three are black and two are red.
He will place one hat on each of you from behind you. If you can tell me the color of your hat, you will be released to go home. But you must agree not to guess and if you guess wrong, you understand you will be shot immediately. Agreed?"
[No lose proposition. All agree. Hats are placed on the three.]
First prisoner looks at other two: "I don't know."
Second prisoner looks at other two: "I don't know."
Third prisoner is blind: "I know."
Does he or doesn't he? Explain or you will be shot.