My job requires me I’m the office maybe 3 times a year. Why does that need to be hybrid?

At this point, you are already in an hybrid situation: why do you think that “hybrid” only means “x days a week” and maybe not something like “every time I need” (be it 3 days a week or 3 days a year) ?