I guess it depends on the type of headphone you are using. I could see your problem with those short wire earplugs with one wire to each ear where the wires are hanging between ears and phone. Not so much with the longer ones that only go to one ear and have generally thicker wires like an over-ear headset.
Only if you lean your phone towards you instead of slightly away from you.
Which is more often that you think
I guess it depends on the type of headphone you are using. I could see your problem with those short wire earplugs with one wire to each ear where the wires are hanging between ears and phone. Not so much with the longer ones that only go to one ear and have generally thicker wires like an over-ear headset.