Message Digest 5, or MD5, was created by Ronald Rivest in 1991 to replace his previous version, MD4, which was deemed insecure after the results of cryptanalytic work by Bert den Boer and Antoon Bosselaers showed MD4 weaknesses. Interestingly enough, MD4 is the algorithm behind the computation of NT-hash password digests on Microsoft Windows NT, XP and Vista.

MD5 is a cryptographic hash function that takes an arbitrary block of data and returns a bit string of a fixed size commonly called the hash value or the message digest. Think of a hash like a digital fingerprint. MD5 produces a 128-bit message digest that is represented in 32-bit hexadecimal format. The basic operation of a hash function takes a message (input data) and encodes it using a series of rounds where algorithms modify the data to produce a hash value (bit string).

Recent MD5 flaws have been disclosed and as a result of these disclosures, US-CERT has stated that MD5 “should be considered cryptographically broken and unsuitable for further use”. As a result of these flaws, U.S. government applications will be required to move to the SHA-2 family of hash functions by 2010. Another interesting side note: the U.S. National Institute of Standards and Technology (NIST) held an open competition to find a new hash function capable of replacing SHA-2. This new hash function will be called SHA-3 and is expected to become a Federal Information Processing Standard (FIPS) sometime around 2012.

These recent flaws, amongst others, have altered MD5′s use in applications. One use that still commonly exists is the ability to use MD5 checksums to determine if a transferred file was altered in any way during transmission. Errors introduced during transmission could lead to a poor copy of the file existing at the destination system. An entity somewhere between the source and destination could alter the file in some way as well. To check the integrity of the transmitted file the destination system could perform an MD5 checksum test.

Many files available for download will have an MD5 checksum posted as well. Using the provided MD5 checksum we can verify the integrity of our file after transmission by computing an MD5 message digest and comparing the two values. If the two values match then the transmitted file was not altered during transmission.

I’m going to use the md5sum command on my Linux system. If you run Unix or Linux you should be able to use this command without any system modifications. For Windows users, a variety of programs exist. Try to Google “windows md5 checksum” and you should find a few good programs that can check MD5 hashes.

Let’s say we want to download BackTrack 4 Pre Release and verify that our file is not altered. remote-exploit provides the following MD5 checksum:

b0485da6194d75b30cda282ceb629654

We can compare the provided MD5 message digest with our own by typing the following at the command prompt:

md5sum bt4-pre-final.iso

After this command my system produced the following:

b0485da6194d75b30cda282ceb629654  bt4-pre-final.iso

As you can see the two message digests are the same and therefore the file was not altered during transmission by possible transmission errors or by any other means of altercation. Comparing a 32-bit hexadecimal string by hand or eye can be quite cumbersome so let’s make this process a bit easier to manage. Create a text file (.txt) containing the MD5 checksum provided by remote-exploit followed by TWO spaces and then the exact name of the file using your favorite editor. This is what my text file (bt4_md5.txt) looks like:

b0485da6194d75b30cda282ceb629654 bt4-pre-final.iso

Then run the following command:

md5sum -c bt4_md5.txt

This is what md5sum spits back at me:

bt4-pre-final.iso: OK

The power behind creating a text file with a listing of MD5 hashes and their respective file names is that you can check multiple files at once just by running the above command. This is beneficial as it allows for easy integration into your shell scripts if you find yourself routinely checking MD5 hashes.