By compiling linux kernel are we building a totally standalone OS which doesn't depend on any linux destributions?
No. Linux by itself is not an operating system. It is just a kernel. A userland, comprising a collection of system libraries and basic software, is necessary to facilitate your interaction with the computer. In what we commonly mean when we refer to "Linux", this is provided by a separate project called the GNU Project. Technically, we ought to (and some do) call it "GNU/Linux".
Well when I want to create the initrd file using initramfs commmand why is it adding some info about my running linux destrubution ( I'm running Ubuntu 10.4) ?
Because your initrd contains the collection of programs that initially set up your system on boot, load necessary drivers and daemons, find and mount the requisite partition(s) on your hard drive(s), etc. It is actually customized not only to your distribution, but to your particular installation, configuration, and computer. It is not generally interchangeable with anyone else's initrd. That's why it has to be generated by your system.
How should I compile and add modules to my compiled linux and where should I put them?
If you are compiling your own kernel, presumably you are doing make menuconfig in the kernel source directory prior to compiling, which allows you to select which features will be compiled, and whether they will be integrated into the kernel binary or made available as modules.
This is a summery of what I've been doing:
There are a number of reasons this won't work the way you want it to.
getting the kernel source (3.4.1) and compiling it using "make all" command ( I use my current linux .config )
If you are running Ubuntu 10.04, which predates the 3.x branch of the Linux kernel by quite a bit, let alone 3.4. It will most likely not work correctly, at least not without an enormous amount of extra work. The config files changed quite a bit too, and I'm frankly surprised it's not throwing out errors left and right about things in your (presumably for Linux 2.6) config file not matching up with how things are done in 3.4. Also, you probably want to be downloading Ubuntu's kernel source package and not the plain vanilla kernel source from kernel.org, as Ubuntu patches the kernel to include needed features and make it play nice with everything else in the system. Finally, you are missing a number of steps, which I'll go into later.
Putting the compiled bzImage to the /boot directory of my memory stick
I'm not sure what your intention is with this, but if it's to have a full bootable Linux installation on your memory stick, there's a bit more to it than that -- you don't have any userland stuff at all on that memory stick, for instance. What this will actually do, if set up properly, is (try to) run the Ubuntu installation on your hard drive with the kernel on your memory stick. I believe this should work, but why do that rather than just install the kernel on your hard drive? You can install it side by side with your existing one so you can freely switch between them.
Getting into (initramfs) terminal after an error stating that could not find the "/lib/modules/3.4.1/modules.dep" file !!!
This goes back to the missed steps I mentioned earlier. You need to make modules_install your new kernel to get its modules placed in /lib/modules.
HOWEVER...
None of this really matters, because ideally you should not compile a kernel the "traditional" way at all, unless you have a very specific need to. These days, should you actually need to compile your own kernel, most distributions have a way to do it that uses their custom kernel version (if applicable), automates the build process, installs the kernel as a package so you can cleanly install and remove it as necessary, and configures your bootloader to give you a choice of the new or old kernel(s). In Ubuntu you should follow these instructions: Kernel/Compile - Ubuntu Wiki
Which is not to say you can't compile a kernel the "old-fashioned" way just for kicks/for educational purposes -- I compiled my first kernel that way, but this was before there were better ways to do it. If your system being not-broken is important to you, use the recommended method for your distro, use their source, and use their package management.