After doing a little more research, I may have to give up on a pre-written solution and just write my own.
PyFilesystem looks like it has potential, particularly the MultiFS fs type. Its FUSE code needs a patch to work with fuse4x.
The MultiFS documentation isn't clear what whether it will create a directory on the writable fs, but I'll give it a try.
UPDATE
The patch introduced a bug when it moved this line before the iconv CDLL loading:
_libfuse = CDLL(_libfuse_path)
Just move it right after and it should work on OS X. (A general fix requires also adding the line after the if _system in
block.)
So I tried the MultiFS and it does not create missing directories on the writable FS. But I'm guessing it's easy to add (even though I'm not a python expert). Also, trying to overwrite or append to a file on one of the read-only FS's doesn't consistently create the corresponding path on the writable one. I get a lot of errors and hangs (the fuse process then starts using a lot of CPU). This solution still looks very promising though.