For me, at least, it correlates to the direction that provides the widest/least obstructed/most visibly clear/least disruptive/least hazardous direction to give onward travel. Clearly the direction that that boils down to varies according to the individual situation.

I don’t recall being alone with a single central obstruction in the middle of an otherwise deserted and symmetrical street with no other influencing factors enough times to have noted any innate bias on my part.