- Any and all of used now VCS can store and handle binary-files in repositories for almost any size ("don't store giant files in repo" is recommendation, not limitation). Some VCS just do it better, than another; and some VCS handle Big Data in repo better, than another
record the reason for the change and being able to fetch the file as it existed at that check-in
is core of VCS and can't be governing parameters
- For changing binary-data storing new version as not diff is almost common rule for VCSes (except applying different tricks in different VCSes for reducing deltas in storages), thus - which VCS to use it your choice and responsibility, I can only note some recent discussions about large files under version control, in which I participated on StackOverflow (top three answers) and repeat my personal opinion - Mercurial
All the kinds of changes anticipated are common tasks for any data, stored in VCS (perform change of content, store it) and aren't unique for audio-files (changing is changing regardless of what changes)
While it would be super nice to a have a unified diff view like is possible with text
You can at least try to get it: Foobar2000 with Binary Comparator plugin (answer found here on SU, in very useful in common topic) can (?!... not tried, not tested) compare (in GUI?!) two files of supported by Foobar2000 formats. Or (if it'll work on Win7 /old project, not updated from 2008/ and and will be usable for your tasks) see at Audio DiffMaker's DYF-files (additional object for storing in repo for any changeset, which change audio-data)
While you can add/change MP3-tags by any external tool, you can compare tags (fast and dirty search gave me at the first lines screenshot of Beyond Compare): Beyond Compare can be used as default diff|mergetool in Mercurial (TortoiseHG), Foobar2000 can be (probably) assigned as special mergetool for mp3-files
Storage should be of change deltas instead of dumb wholesale copies of each commit.
It isn't possible (in common case, see above p.2), but for Git with LFS or Mercurial with LargeFiles we have special case (can meet your needs better, than usual "all in repo"): all files for all changesets stored in independent external storage (full files), changesets in repos have only "link" to files, on local workplace you have downloaded only one big file (not the whole set for full history in your clone of repo for DVCS-case)... and all additional old versions needed for direct diffing (think again about using DYF from Audio DiffMaker): you'll must to have one giant storage, but will have some "saving" of local space, compared to "files in repo" case