If the output looks too blocky or pixelated, that's due to a too low bitrate. I see you set 500 kBit/s, but that's much too low for achieving reasonable quality with video at 720p or above.
The VP8 encoding guide lists a few options you have when using the libvpx encoder. I agree with you that using a constant rate factor is the correct approach to achieving the same level of visual quality, regardless of the video resolution.
You were using CRF 18 for the x264 encoder, which is considered very high quality (maybe not visually lossless, but the quality loss should not be perceivable). For libvpx, the CRF range is a little different. 10 is a good default (it would be 23 for x264), and so I'd probably recommend using something like CRF 6–7 for libvpx.
Note that the -qmin
parameter must be set equal or lower to the CRF you're choosing. Otherwise it can't encode.
Remember that the resulting bitrate has its upper bound set at whatever you specify with -b:v
. Therefore, if your CRF is too low, and your bitrate is set to -b:v 2M
, your final bitrate will be 2 MBit/s.
But really, there is nothing better than just trying out different values. It all depends on what your input looks like or what your target application for the videos is. If you're just doing some web streaming, you don't need such high quality video. If you want offline storage, then a lower CRF would of course be better.