Maybe yes, maybe not. If he's clearly off his rocker, he may have a tough time getting another gun and all that ammo.
But I assume that means you're also fine with leaving him with all his guns and ammo as well. Right?
Of course I am. If you don't have a reason to commit him, and you don't have a reason to arrest him, what the hell makes you think you have a reason to take his property?
But then - He's off his meds...
People taking medicine for psychosis would not have been able to purchase the gun in the first place, and if he is off his meds, you have a reason to commit him.
so your entire drama is rendered moot.