You said
*If I remove the 4-pin CPU power connector and just leave the 24-pin ATX connector (http://www.legitreviews.com/images/reviews/418/gigabyte_ds3_labeled.jpg) and if I plug in the power cable, CPU fan starts spinning. But it doesn't so with both the connectors plugged in.
When you said this I agreed with your statement that it wasn't the power-supply. Instead I believe it is the CPU. You see if your using a modern power-supply it won't give power to the board unless the board meets some requirements. This is a very logical process because if the board is broken and you apply power it might (will) damage the things attached. My suspicion is that either the CPU or the 4-pin connector is unhappy. When you said the fan starts up with it unplugged it was somewhat of a give away.