Skip to content
  • Sean Purcell's avatar
    squashfs: Add zstd support · 87bf54bb
    Sean Purcell authored
    Add zstd compression and decompression support to SquashFS. zstd is a
    great fit for SquashFS because it can compress at ratios approaching xz,
    while decompressing twice as fast as zlib. For SquashFS in particular,
    it can decompress as fast as lzo and lz4. It also has the flexibility
    to turn down the compression ratio for faster compression times.
    
    The compression benchmark is run on the file tree from the SquashFS archive
    found in ubuntu-16.10-desktop-amd64.iso [1]. It uses `mksquashfs` with the
    default block size (128 KB) and and various compression algorithms/levels.
    xz and zstd are also benchmarked with 256 KB blocks. The decompression
    benchmark times how long it takes to `tar` the file tree into `/dev/null`.
    See the benchmark file in the upstream zstd source repository located under
    `contrib/linux-kernel/squashfs-benchmark.sh` [2] for details.
    
    I ran the benchmarks on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
    The VM is running on a MacBook Pro with a 3.1 GHz Intel Core i7 processor,
    16 GB of RAM, and a SSD.
    
    | Method         | Ratio | Compression MB/s | Decompression MB/s |
    |----------------|-------|------------------|--------------------|
    | gzip           |  2.92 |               15 |                128 |
    | lzo            |  2.64 |              9.5 |                217 |
    | lz4            |  2.12 |               94 |                218 |
    | xz             |  3.43 |              5.5 |                 35 |
    | xz 256 KB      |  3.53 |              5.4 |                 40 |
    | zstd 1         |  2.71 |               96 |                210 |
    | zstd 5         |  2.93 |               69 |                198 |
    | zstd 10        |  3.01 |               41 |                225 |
    | zstd 15        |  3.13 |             11.4 |                224 |
    | zstd 16 256 KB |  3.24 |              8.1 |                210 |
    
    This patch was written by Sean Purcell <me@seanp.xyz>, but I will be
    taking over the submission process.
    
    [1] http://releases.ubuntu.com/16.10/
    [2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/squashfs-benchmark.sh
    
    zstd source repository: https://github.com/facebook/zstd
    
    
    
    Signed-off-by: default avatarSean Purcell <me@seanp.xyz>
    Signed-off-by: default avatarNick Terrell <terrelln@fb.com>
    Signed-off-by: default avatarChris Mason <clm@fb.com>
    Acked-by: default avatarPhillip Lougher <phillip@squashfs.org.uk>
    87bf54bb