They don’t have to be fighters for it to be a headache. During a civil war you have to deal with feeding, securing, housing, etc. all of those people when areas inevitably collapse or are taken over for military operations and people evacuate (i.e. refugees).

Then there are people who do support whichever side and do small acts of sabotage, espionage, etc.