Like with most things in comedy, you will need to hear his timing, context, and the cadence of his voice. You can still not think it is funny, but there clearly was a joke being told.