Загрузки на веб-сайтах иногда имеют контрольную сумму MD5, что позволяет людям подтвердить целостность файла. Я слышал, что это позволяет не только мгновенно идентифицировать поврежденные файлы до того, как они вызовут проблему, но и для того, чтобы любые вредоносные изменения могли быть легко обнаружены.
Я следую логике в том, что касается повреждения файлов, но если кто-то намеренно хочет загрузить вредоносный файл, он может сгенерировать соответствующую контрольную сумму MD5 и опубликовать ее на сайте загрузки вместе с измененным файлом. Это обмануло любого, кто загружал файл, и думал, что оно не изменилось.
Как контрольные суммы MD5 могут обеспечить какую-либо защиту от намеренно измененных файлов, если нет способа узнать, была ли скомпрометирована сама контрольная сумма?
I have heard this is to allow [...] for any malicious changes to be detected also.
Well you heard wrong, then. MD5 (or SHA or whatever) checksums are provided (next to downloads links, specifically) only for verifying a correct download. The only thing they aim to guarantee is that you have the same file as the server. Nothing more, nothing less. If the server is compromised, you’re SOL. It’s really as simple as that.
The solution used by some package management systems such as dpkg is to sign the hash: use the hash as input to one of the public key signing algorithms. See http://www.pgpi.org/doc/pgpintro/#p12
If you have the public key of the signatory, you can verify the signature, which proves the hash is unmodified. This just leaves you with the problem of getting the right public key in advance, although if someone once tampers with the key distribution they also have to tamper with everything you might verify with it otherwise you'll spot that something strange is going on.
Your assumption is correct. There is an exception though. If the server providing the file and the page where the hash is are not managed by the same entity. In that case the software developer may want to say "hey people download this from that place but only believe if hash = xxxx". (This might be usefull for CDN's as an example). I guess this was the reason why someone did it in the first place. Than others just followed thinking how cool it would be to show the hash. Not even thinking how useful it is not even both the file and the hash are on the same location.
Having this said, this is worth what it is. Don't assume too much about security as others already stated. If and only if you can absolutely trust the original hash, than the file is good. Otherwise an attacker with enough motivation and knowledge can tamper both file and the hash, even if these are in different servers and managed by different entities.
Иногда контрольные суммы предоставляются надежно, а загрузка - нет. Поскольку MD5 поврежден, контрольные суммы безопасности MD5 являются более слабыми, чем более безопасные контрольные суммы, но до того, как MD5 был сломан, надежно предоставленный MD5 (например, тот, который был подписан с PGP или GPG или Gatekeeper, или извлечен по HTTPS), который соответствовал MD5 загрузка была убедительным доказательством того, что полученная загрузка была доступной серверу.
Я писал о плачевном отсутствии надежных контрольных сумм в течение многих лет, здесь .
Пользователи не должны загружать ненадежные исполняемые файлы по ненадежным сетям и запускать их из-за риска атак MITM. См., Например, «Незащищенность в системах автоматического обновления» П. Руиссена, Р. Влотуиса.
Приложение 2014 года: Нет, это НЕ неправильно, «контрольные суммы, размещенные на веб-страницах, используются для обнаружения вредоносных изменений», потому что это роль, которую они могут выполнять. Они помогают защитить от случайного повреждения и, если они обслуживаются по протоколу HTTPS или с проверенной подписью (или еще лучше, оба), помогают защитить от злонамеренного повреждения! Я получил контрольные суммы по HTTPS и подтвердил, что они совпадают с загрузками HTTP много раз.
В настоящее время двоичные файлы часто распространяются с подписанными, автоматически проверенными хешами, но даже это не совсем безопасно .
Выдержка из приведенной выше ссылки: «Приложение KeRanger было подписано с действительным сертификатом разработки приложений для Mac; следовательно, оно смогло обойти защиту от Apple Gatekeeper». ... »С тех пор Apple аннулировала испорченный сертификат и обновила антивирусную сигнатуру XProtect, а Transmission Project удалил вредоносные программы установки со своего веб-сайта. Palo Alto Networks также обновила фильтрацию URL-адресов и предотвращение угроз, чтобы не допустить воздействия KeRanger на системы. Технический анализ
Два инсталлятора Transmission, зараженные KeRanger, были подписаны законным сертификатом, выданным Apple. Разработчик перечислил этот сертификат турецкой компании с идентификатором Z7276PX673, который отличался от идентификатора разработчика, использовавшегося для подписи предыдущих версий установщика Transmission. В информации о подписи кода мы обнаружили, что эти установщики были созданы и подписаны утром 4 марта. "
Приложения 2016 года:
@ Cornstalks: Re. Ваш комментарий ниже: Неверно. Как в настоящее время отмечено в статье Википедии об атаках на столкновения, на которую вы ссылаетесь: «В 2007 году была обнаружена атака столкновения с выбранным префиксом на MD5», и «злоумышленник может выбрать два произвольно разных документа, а затем добавить различные вычисленные значения, которые в результате получаются в целом». документы, имеющие одинаковое значение хеш-функции. " Таким образом, даже если MD5 предоставляется безопасно и злоумышленник не может изменить его, злоумышленник все равно МОЖЕТ использовать атаку коллизии с выбранным префиксом с вредоносным ПО с выбранным префиксом, что означает, что MD5 НЕ безопасен для криптографических целей. Во многом это объясняется тем, что US-CERT заявил, что MD5 «следует считать криптографически взломанным и непригодным для дальнейшего использования».
Еще пара вещей: CRC32 - это контрольная сумма. MD5, SHA и т. Д. - это больше, чем контрольные суммы; они предназначены для надежных хэшей. Это означает, что они должны быть очень устойчивы к столкновениям. В отличие от контрольной суммы, защищенный хэш защищенного хэша защищает от атаки "человек посередине" (MITM), когда MITM находится между сервером и пользователем. Он не защищает от атаки, когда сам сервер подвергается риску. Чтобы защититься от этого, люди обычно полагаются на что-то вроде PGP, GPG, Gatekeeper и т. Д.
This is really a problem. Showing checksums on the same site as the file to download is insecure. A person who can change the file can also change the checksum. The checksum should be shown through a complete separated system but this is hardly feasible, because how to tell the user in a safe way where the checksum can be found.
A possible solution is the use of signed files.
(BTW: MD5 is unsafe anywhere and shouldn't be used anymore.)
This is the precise reason posted checksums often carry a disclaimer saying "This cannot protect against malicious modification of the file". So, the short answer is "they can't provide any protection whatsoever against a deliberately altered file" (although, if the page is delivered over HTTPS, HTTPS itself protects against modification; if the file isn't delivered over HTTPS but the checksum is, then that might help some, but isn't a common case). Whoever told you that checksums posted on web pages are used to detect malicious modifications was wrong, because this is not a role they can perform; all they do is help protect against accidental corruption, and lazy malicious corruption (if someone doesn't bother to intercept the page giving you the checksum).
If you want to protect against deliberate modification, you need to either keep people from messing with the checksum, or make it impossible for anyone else to generate a valid checksum. The former can involve giving it out in person or similar (so the checksum itself is trusted); the latter goes to digital signature algorithms (where you need to securely get your public key to the downloader; in TLS, this is done by ultimately trusting certificate authorities directly and having them verify everyone else; it can also be done over a web of trust, but the point is that something has to be securely transferred at some point, and just posting something on your site isn't enough).
How can MD5 checksums provide any protection against deliberately altered files if there is no way of knowing if the checksum itself has been compromised?
You are entirely correct. The goal, then, would be to make your "if" wrong — if we know that a secure cryptographic hash of a file isn't compromised, then we know that the file isn't compromised either.
For example, if you post a hash of a file on your website, and then link to a copy of the file on a third-party mirror server — common practice in old-fashioned free software distribution — your users can be protected against some types of attacks. If the mirror server is malicious or compromised, but your website is okay, the mirror won't be able to subvert your file.
If your website uses HTTPS, or you sign the hash with gpg, your file can also be (mostly) protected from network attackers like malicious Wi-Fi hotspots, rogue Tor exit nodes, or NSA.
How can MD5 checksums provide any protection against deliberately altered files if there is no way of knowing if the checksum has not been compromised either?
This is a really good question. In general, your assessment of MD5 manipulation is spot on. But I believe the value of MD5 checksums on downloads is superficial at best. Perhaps after you download a file you can check the MD5 you have against a website, but I tend to see side-by-side MD5 storage as a “receipt” or something that is nice to have but not reliable. As such, I have generally never cared about MD5 checksums from downloaded files, but I can speak from my experience creating ad-hoc server-based MD5 processes.
Basically what I have done when a client wants to sweep a file system for MD5 checksums is to have them be generated into CSV files that maps filename, path, MD5—and other sundry file info—into a structured data format that I then have ingested into a database for comparison and storage.
So using your example, while an MD5 checksum might sit next to a file in it’s own text file, the authority record MD5 checksum would be stored in a non-connected database system. So if someone somehow hacked into a fileshare to manipulate data, that intruder would not have any access to the MD5 authority records or the connected history.
Recently I discovered a nice piece of library archival software called ACE Audit manager which basically is a Java application designed to sit and watch a filesystem for changes. It logs changes via MD5 changes. And it operates on a similar philosophy as my ad-hoc process—store the checksums in a database—but it takes it a step further but creating an MD5 checksum of MD5 checksums which is known as a hash tree or Merkle tree.
So let’s say you have 5 files in a collection. Those 5 files in ACE Audit manager would then get another—let’s call it “parent”—checksum that is a hash generated from the 5 MD5 checksums of each file. So if someone were to tamper with just one file, the hash for the file would change and so would the hash for the whole “parent” collection.
In general the way you need to look at MD5 checksums and related integrity hashes is unless they are not connected to some non-direct storage for the MD5 hashes themselves, they can be corrupted. And their value as a long term data integrity tool is equivalent to a cheap lock that comes “free” on a new piece of luggage; if you are serious about locking your luggage you will get a lock that cannot be opened in 5 second with a paperclip.
You can't modify the MD5 checksum without also modifying the file. If you download the file, then download the hash, and then your computation of the file's has doesn't match what is given, either the hash or the file is wrong or incomplete.
If you want to "tie" the file to something external, such as author, machine, etc. it needs to be signed, using a PKI-type process with certificates. The file author, etc. can sign the file with his/her private key, and you can verify signatures with the public key, which should be publicly available, itself signed by a CA both you and the author trust, and downloadable, preferably from multiple locations.
Modifying the file would make the signature invalid, so this can be used to verify file integrity too.
Hashes indicate whether your version of the file (the "download") differs from the server's version. They offer no guarantee to the authenticity of the file.
Digital signatures (asymmetric encryption + hash function) can be used to verify that the file has not been modified by anyone who does not have the corresponding private key.
The file's creator hashes the file and encrypts the hash using their (secret) private key. That way, anyone with the corresponding (non-secret) public key can verify that the hash matches the file, but while the file contents can be modified, nobody can replace the corresponding hash with one that matches the file (after hash is decrypted using the public key) - unless they manage to brute-force the private key, or gain access to it somehow.
What's to stop Mr "A.Hacker" from simply modifying the file, then signing it with their own private key?
In order to validate the file, you need to compare its hash to the one that you obtained by decrypting the associated digital signature. If you think the file is from "I.M.Awesome", then you decrypt the hash using his key and the hash does not match the file, since the hash was encrypted using A.Hacker's key.
Digital signatures hence allow one to detect both accidental and malicious changes.
But how do we get I.M.Awesome's public key in the first place? How can we ensure that when we obtained his/her key, it wasn't actually A.Hacker's key being served by a compromised server or a man-in-the-middle attack? This is where certificate chains and trusted root certificates come in, neither of which are perfectly secure [1] solutions to the problem, and both of which should probably be well explained on Wikipedia and on other questions on SO.
[1] Upon inspection, the root certificates that shipped with the Microsoft OS on my work PC include a certificate from the U.S. Government. Anyone with access to the corresponding private key cough NSA cough can therefore serve content to my web browser that passes the "is there a padlock in the address bar" check. How many people will actually bother to click on the padlock and see who's key-pair is being used to "secure" the connection?