To identify the magic port, you can use nmap while inside the wifi network, and scan the IP address of your VPS for all UDP and TCP ports:
nmap -sU -sS -p1-65535 <vps ip>
The idea here is that the firewall at the wifi end is blocking packets leaving the local network, but any that get through, must be via open ports. So on the VPS side, you run
tcpdump -i <interface name> host <public IP address of wifi router>
You will need to work out the public address by going to http://whatismyip.com
We are not interested in the results that nmap comes back with, we want to see what tcpdump sees - any packet that makes it to the VPS will have passed through the firewall, so the destination port of the packet will tell us which ports are open:
13:23:32.805549 IP <wifi router>.2154 > <vps>.ssh
The above fragment shows that a packet arrived on the ssh port, which is 22, which must be permitted through the firewall.
Note that while you are able to do DNS queries, it does not follow that port 53 is open to the internet. The usual case is that you are permitted contact to controlled DNS servers, and it is those that can forward DNS requests out to the internet - much like in a domestic setting you often set your router to be the DNS server for the network, and it is the router that resolves queries.
If it is the case that port 53 is open only to specific DNS server, then you can get around it using an IP over DNS tunnel. If you have a VPS running a DNS server and you have a domain name you can can control, you could use iodine which allows you to tunnel IP over DNS queries, and so removes the need for OpenVPN (though running OpenVPN inside the tunnel will ensure your packets are protected. You could also do the same with ssh).