First of all, you have to differentiate between a video/audio codec, and a multimedia container. I've talked in way more detail about the difference between codecs and containers in another Super User answer. Briefly put, a container is there to hold various audio and video (and subtitle, data, image, …) bitstreams.
This process of merging bitstreams into one container is indeed often referred to as "multiplexing". You could call the process of interleaving audio/video to a container multiplexing, but strictly speaking this term applies to MPEG-2 Transport Stream multiplexing. We'll get to that later.
Now, there are various container formats, but they all serve the same purposes: Dealing with synchronization of the streams inside. You want to make sure that parts belonging together are played at the same time. The container carries meta information that specifically tells a playback device where to look and what to play at, let's say 00:02:01.250.
Other things that containers allow you to do are including information about the stream itself. For example, in many streaming contexts, the player needs to know the size of the video, its frame rate, various other things. When you stream an MP4 file (MPEG-4 Part 14 is a container, not a codec), you want the MOOV atom to be placed at the beginning of the file, so the player can parse it right away. Usually, this "atom" of data would sit at the end of the file where it's not really useful because you'd have to download the entire file before streaming. This is why there are tools to move it to the start, like qt-faststart.
What's important though is that in any multimedia application that multiplexes containers from individual bitstreams, those streams are encoded first. There's no "integrated A/V signal" that is encoded—the container will simply wrap what's there and won't touch the actual audio and video data. So, your video is fed to a video encoder, your audio is fed to an audio encoder. The result from both of these tasks is merged/interleaved in the container. The container will take care of assembling the final bitstream that can be transmitted in a streaming context, or saved as a file.
Now, there are many containers that are suited for different applications. MPEG-2 transport streams are commonly used in TV broadcasting. Here, multiple TV channels are multiplexed into one container, which is then transmitted over the air, cable or satellite. The method of multiplexing is time-division multiplexing. Here, "multiplexing" is also the correct technical term. For other containers, it's not used that often. MPEG-4 Part 14 (MP4, we've briefly mentioned this before) on the other hand is very common in the web and mobile context, and it's becoming more and more of a standard for exchanging files, even in the professional field.
There are other containers, such as MKV, Ogg, MPEG Program Streams, FLV, and many more. Which one to choose depends on the application, as you can see, but this would go far beyond the scope here.