It might have more to do with men being able to safely father so many more children. 50ish seems the record for mothers, but fathers could have hundreds of kids. On top of that, maternal mortality rates were high, so a matriarch has a chance of dying with every kid.