Yes of course, it is possible to install Xcode on a maco. But as you said yourself, you need OSX 10.8.4 or newer. So you have to install OSX 10.8.
OSX 10.8 needs an Mac mini Early 2009 or newer.
To check how old your mac mini is, follow these instructions:
- Apple Menu->About this Mac.
- Click the More Info...-button.
- Now you see the year of your mac under the model name.
If your model is able to install it, you can download mountain lion through the app store.