Some species appear blue. Mud daubers aren’t a specific species iirc. Some look like plain old stretched out bees and some look like this.

Hell some are completely black