If you look into the /System/Library/LaunchDaemons/ntalk.plist then you can see the following entry:
<dict> <key>Listeners</key> <dict> <key>SockServiceName</key> <string>ntalk</string> <key>SockType</key> <string>dgram</string> </dict> </dict> The value of SockType is dgram and because of that the program listens on a UDP and not a TCP port.
If you call sudo nmap -sU -p 518 localhost then you should see that the port is opened.
vanemery.com: Using ntalkd as a Private, Secure, One-on-One Text Chat Service:
ntalkd listens on UDP port 518. Once the actual session is setup, there are two UDP streams and one TCP connection that exist between clients.