I don’t think it started as a derogatory term, but it became so. People would use it to imply that someone was ugly or otherwise defective, so no husband would want them. A man unattached is still seen as more acceptable than a woman unattached even today.
Oh my goodness yes! Not to mention the whole if you don’t dress “modestly” it’s your fault if you get unwanted attention thing. It’s a grooming ground.