LLMs are getting better at math btw.

There are also techniques considered to have the LLM call functions like a calculator to address arithmetic.