boost::uuids::detail::sha1 computes incorrect result on vs2010sp1.iso

[frankrq2009@…]

I wrote a program to compute the CRC32 and sha1 checksum, I found it can generate correct sha1 result only if the size of the checked file is not too big(e.g.,the size of virtualbox or linux kernel source), but when I checked vs2010sp1.iso which is around 1.6GB, the crc32 is correct, the sha1 is not the same as the output of sha1sum. The version of boost that I tested are 1.45.0,1.46.0 and 1.46.1, both on windows 7 and linux 2.6.36. I attach my source code here.

#include <stdio.h> #ifdef WIN32 #include <io.h> #else #include <sys/types.h> #endif #include <fcntl.h> #include <string.h> #include <errno.h> #include <exception> #include <boost/crc.hpp> #include <boost/uuid/sha1.hpp>

const int bufSize = 4096; int main(int argc, charargv) {

if(argc < 2) {

printf("Useage:\t%s FileName1 FileName2 ...\n",argv[0]); return 1;

}

char buf[bufSize];

try {

for(int i = 1; i < argc; ++i) {

boost::crc_32_type crc; boost::uuids::detail::sha1 sha;

int fd=::open(argv[i], O_RDONLY); if(fd<0) {

printf("Failed to open file %s, error:%s\n", argv[i], strerror(errno)); continue;

} while(1) {

int n=::read(fd, buf, bufSize); if(n<=0)

break;

crc.process_bytes(buf, n); sha.process_bytes(buf, n);

}; ::close(fd); unsigned int dig[5];

sha.get_digest(dig);

printf("%s :\nCRC32 = %08x\nSHA1 = ",argv[i], crc.checksum()); for(i=0; i<5; ++i)

printf("%04x",dig[i]);

printf("\n");

} return 0;

} catch(std::exception &e) {

printf("Found an exception with '%s'\n",e.what()); return 2;

} catch(...) {

printf("Found an unknown exception.\n"); return 3;

}

}

[Andy Tompkins]

Fixed with release commit 76405 and 76406

The digest was incorrect for messages longer than 536,870,911 bytes (and vs2010sp1.iso is).

The sha1 implementation now handles messages as long as the sha1 specification, 2^{64 bits.}

[Andy Tompkins]

Note also that the given program has a bug. The ::open also needs the flag _O_BINARY.

I added a test to hash 1 million 'a's, but I do need to add tests for even longer messages.