Most modern sound card can natively support the most common sampling frequencies (the ones you mention) without quality loss. And as user55325 states, anything above 44.1 kHz is not really needed for the (normal) human ear. Most of the quality loss experienced with digital audio starts after the conversion from digital sound to analog, and depends on the used filters (a good digital -> analog conversion needs a filter to remove the high frequency digitalization effects), the amplifier, and most important the speakers (or headphones or ...)
Another thing that will influence quality is: Do you use compression on this audio files (MP3, WMA, Flac ...), if so, then most of the time (except when you use a lossless compression format) this compression will influence the quality more than the chosen sampling frequency.
If you do not use any form of compression (or use lossless compression) I would keep the files in the format I got them, because any manipulation in sampling frequency will cause a small degradation in quality (most noticeable when downsampling = from a higher frequency to a lower; and upsampling will give you no benefits - you cannot improve the quality of the sound above the quality of the original). Upsampling gives also the disadvantage of larger files, so you will need more disk space to hold the same files.
About the 96kHz files: those can perfectly be used for further processing, this is (one of) the format(s) studios use when they do a recording (most multi-track) and then combine all this to put it on a CD (at 44.1 kHz) or in a movie (48 kHz). When you have them in this format, I would keep them like this, except when you need the disk space, but I would never use it as format for all my files.
