It kind of does. If the public server you're on dosen't have the option to talk to other servers turned on (and this is not uncommon), well, you can only talk to people on the same server. If they do, and your server can find the other server and various conditions for inter server communication are met, things just work.
In short? Safe option is to use the same server as the people you want to talk to. Else, in many cases it dosen't matter, but make sure you check that the people you want to talk to can see you