The zlib, gzip and bz2 modules provide essential data and file compression tools. Data files are often built for speedy processing, and may contain characters which are meaningless spacing. This extraneous data can be reduced in size, or compressed.

For example, a .tar file is often compressed using GZip to create a .tar.gz file, sometimes called a .tgz file. In the case of the ZIP file archive, the compression algorithms are already part of the zipfile module.

These modules are very flexible and can be used in a variety of ways by an application program. The interface to zlib and bz2 are very similar. The interface to gzip is greatly simplified.

Data Compression and Decompression. The zlib and bz2 modules provide essential data compression and decompression algorithms. Each module provides a compress and decompress function which will compress a string into a sequence of bytes. For relatively short strings, there may be no reduction in size.

Note that the interfaces for the zlib and bz2 modules are designed to be nearly identical. The results, however, are not compatible at all; these are different algorithms for data compression.

In addition to simple compression and decompression, zlib provides some additional features for computing various kinds of checksums of the data.

Simple File Compression and Decompression. The zlib and bz2 modules provide functions for compression and decompression of files or streams. Each module provides a way to create a compressor object, as well as process a file while compressing or decompressing it.

The following methods will create a compressor object or a decompressor object. A compressor objects has methods which will compress a sequence of bytes incrementally. A decompressor object, conversely, has methods to uncomress a sequence of bytes. These objects can work with chunks of data rather than the complete contents of a file, making it possible to compress or uncompress large amounts of data.

Example 33.4. compdecomp.py

#/usr/bin/env python
"""Compress a file using a compressor object."""
import zlib, bz2, os

def compDecomp( compObj, srcName, dstName ):
    source= file( srcName, "r" )
    dest= file( dstName, "w" )
    block= source.read( 2048 )
    while block:
        cBlock= compObj.compress( block )
        dest.write(cBlock)
        block= source.read( 2048 )
    cBlock= compObj.flush()
    dest.write( cBlock )
    source.close()
    dest.close()

compObj1= zlib.compressobj()
compDecomp( compObj1, "../python.xml", "python.xml.gz" )
print "source", os.stat("../python.xml").st_size/1024, "k"
print "dest", os.stat("python.xml.gz").st_size/1024, "k"

compObj2= bz2.BZ2Compressor()
compDecomp( compObj2, "../python.xml", "python.xml.bz" )
print "source", os.stat("../python.xml").st_size/1024, "k"
print "dest", os.stat("python.xml.bz").st_size/1024, "k"

	We define a function, compdecomp, which applies a compressor object to a source file and produces a destination file. This reads the file in small blocks of 2048 bytes, compresses each block, and writes the compressed block to the destination file. The final block will generally be less than 2048 bytes. The final call to compObj.flush notifies the compressor object that there will be no more data; the value returned is the tail end of the compressed data.
	We create a compressor, compObj1, from the zlib.compressobj function. This object is used by our compDecomp function to compress a sample file. In this case, the sample file is 1,208K, the resulting file is 301K, about ¼ the original size.
	We create a compressor, compObj2, from the bz2.BZ2Compressor class. This object is used by our compDecomp function to compress a sample file. The resulting file is 228K, about ⅕ the original size.

Decompression uses a function nearly identical to the compDecomp function. The decompress version calls the decompress method of a decompObj instead of the compress method of a compObj.

Gzip File Handling. The gzip module provides function and class definitions that make it easy to handle simple Gzip files. These allow you to open a compressed file and read it as if it were already decompressed. They also allow you to open a file and write to it, having the data automatically compressed as you write.

Once this file is open, it can perform ordinary read, readline, readlines, write, writeline, and writelines operations. If you open the file with 'rb' mode, the various read functions will decompress the file's contents as the file is read. If you open the file with 'wb' or 'ab' modes, the various write functions will compress the data as they write to the file.