As long as you don't depend on the execution order (.vimrc is sourced first, .gvimrc last, after all plugins; most options have no such dependency), it's an arbitrary decision where to place the GUI-only settings.
You can just put those into ~/.vimrc, surrounded by
:if has('gui_running') .. :endif Caveat
Apparently, if you start terminal Vim and only later switch to GUI mode (via :gui; this works f.e. on Linux Vim), the .gvimrc file is belatedly sourced, so that's one special case where your consolidated approach wouldn't work.