It seems that it cannot be done this way. The problem is that the VBR header is written to the beginning of the file, but is only calculated at the end of the encoding. This requires seeking in the file, which is not possible if the output is a pipe.
I ran an strace on all of the variants above. In the 2nd version (when writing to a normal file), I get the following at the end:
lseek(4, 0, SEEK_SET) = 0 write(4, "\377\373\220d\0\0\0"..., 417) = 417 close(4) = 0 In the 1st version, where I use - as output argument, lame doesn't even try to write the header. In the 3rd version, however, it tries but fails because the output is a pipe.
lseek(4, 0, SEEK_SET) = -1 ESPIPE (Illegal seek) It also writes an error message at the end, which is easily overlooked because of the other output it normally prints to stderr, unless I run it with the --silent option (which I did, to make the strace output cleaner):
fatal error: can't update LAME-tag frame! The solution to this problem is either to write to a temporary file, then pipe that further, or to use constant bitrate encoding (in which case no additional header is written at the beggining of the file after encoding).