Why base it on the existence or nonexistence of a penis? How do you enforce that? You’re in a bathroom, ideally no one is seeing your genitals.

Just get rid of gendered bathrooms in the first place. A toilet doesn’t care what shape of butthole poops into it