The pattern is usually that A&B would send a request to a relay server which is visible to both. As the requests are outgoing, the ISP doesn't require A or B to support incoming requests.
The simplest implementation is for the server to then act as an intermediary between the two. This creates some additional latency and server load.
Alternatively, the server may tell B or A try to create a connection to the other client. If either connection is possible - one of A or B's ISP is configured to allow incoming requests - then the conversation continues with the direct connection rather than being relayed through the server.