you've installed rvm in system mode, and thus rvm directs all gem installs to the system folder, where administrative privileges are needed.
You need to completely remove rvm, and then reinstall either rvm in user mode (not sure how, I just know it's possible from reading things.) or installing rbenv. Both those will put your ruby environment (and therefore the gems as well) in your home directory where administrative privileges are not needed.
Do mind that RVM and Rbenv DO NOT PLAY WELL TOGETHER. never under any circumstance install both. (I've had to wipe my iMac once after accidentally installing both.)
Homebrew is system centric, and you probably won't be able to install rvm/rbenv in user mode with that. I recommend using the general install guides from the developers instead.