I want to put my patch panel behind the sheet rock to accommodate where my wires are in the wall. There is no space for a switch so can I bypass the switch?
Without a multi-port switch, then the patch panel is basically just a bunch of extension Ethernet cables connecting directly to the router. Depending on your usage/needs maybe the router has enough ports to accommodate your networking needs?
If so then yes, just run each patch panel Ethernet cable directly to a free LAN port on the router; you’ll have an 1-to-1 relation between ports on the panel and ports on the router.
But if your router has—let’s say—only 4 Ethernet LAN ports and you need 8 Ethernet LAN ports, then that won’t work cleanly.
Maybe you could run one of the patch panel ports into an Ethernet switch outside of the wall to an area where you need extra connectivity. But if you were to do that, then why waste time with the patch panel?
It all depends on your networking goals and the overall layout of your physical environment as to whether directly extending the router via a patch panel—instead of going through a switch—is worth it.