7zipsfx and files >2GB
Posted: 16 Apr 2007, 14:12
Thank you for your 7zipsfx.
It's very good that you added the file attributes and file dates support to 7ZIP's ANSI-C decoder.
However, your implementation still cannot extract files that a bigger than 2GB from 7zip archives. In file 7zExtract.c we can see this:
*outBuffer = (Byte *)allocMain->Alloc((size_t)unPackSize);
*outBufferSize = (size_t)unPackSize;
so decompressor tries to allocate entire uncompressed file size. If a file size is bigger than 2GB, this will always fail on x86. Although, considering a file size is, say, 1.5GB, it is not a good practice to decompress entire file in memory either. Even if such a huge memory allocation succeeds, it will lead to much swapping and slowing down.
Can this be fixed? It would be nice to do decompression in smaller chunks, calling SzDecode in a loop, and writing decompressed data chunk to a file after each iteration (for example via a callback function).
Can you implement this?
It's very good that you added the file attributes and file dates support to 7ZIP's ANSI-C decoder.
However, your implementation still cannot extract files that a bigger than 2GB from 7zip archives. In file 7zExtract.c we can see this:
*outBuffer = (Byte *)allocMain->Alloc((size_t)unPackSize);
*outBufferSize = (size_t)unPackSize;
so decompressor tries to allocate entire uncompressed file size. If a file size is bigger than 2GB, this will always fail on x86. Although, considering a file size is, say, 1.5GB, it is not a good practice to decompress entire file in memory either. Even if such a huge memory allocation succeeds, it will lead to much swapping and slowing down.
Can this be fixed? It would be nice to do decompression in smaller chunks, calling SzDecode in a loop, and writing decompressed data chunk to a file after each iteration (for example via a callback function).
Can you implement this?