To answer my own question: This is a bug in the Windows 10 calculator.
The problem did not exist in Windows 7, and it was NOT fixed in the first big update for Windows 10. As of December 2015, the bug is still there.
As usual there is no official acknowledgement. However, for a discussion on the Microsoft servers, see e.g. https://answers.microsoft.com/en-us/windows/forum/windows_7-winapps/windows-calculator-decimal-separator-is-always/d6e80dc7-9738-4a61-8457-c5e5d28beeb8?db=5&auth=1