Yes, it does make a difference, but like with many other things you should not fall into the trap of decision making paralysis.

Your current instance lemmy.world is the largest and perfectly adequate for the majority. It also costs nothing to make accounts on other instances and is done in less time than it takes to decide between them.

That said some reasons to choose one instance over another:

Federation with other instances

Some instances choose not to federate with others. Common reasons being political ideology or NSFW/piracy/violent content. Others might be more liberal and leave it up to their users to block whatever within their own apps. As someone already mentioned world seems to have defederated dbzer0 the piracy focused instance. Some political instances that often also get defederated are the far left ones like hexbear and Lemmy grad (seems like those are defederated by lemmy.world as well).

Server location and performance

Especially country focused instances like lemmy.ca or feddit.uk will have their servers in their own countries. You can of course access them from anywhere, but a European user might have a better time choosing an instance with servers located in Europe, while someone in America might have lower latency with one located there.

Alternate frontends

If you are accessing Lemmy through a browser rather than an app, you might enjoy alternate frontends that change the design. Your current one lemmy.world for example offers 4 different designs that can also be found in the sidebar alexandrite, photon, voyager mobile and one looking like old.reddit.

General ideology of the instance

Your choice of instance might also tell others something about you. If you choose a country specific instance people seeing your profile name might assume you come from there, if you choose one with a particular political view people will probably assume you hold similar views. Same goes for other instances that are related to things like sexuality or hobbies.