Gzip: Difference between revisions

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
imported>Connoisseurship
File format: uppercase hex
imported>Okterakt
Undid revision 1321715321 by ~2025-32930-53 (talk)
 
Line 23: Line 23:
}}
}}


'''gzip''' is a [[file format]] and a [[software application]] used for [[Data compression|file compression and decompression]]. The program was created by [[Jean-loup Gailly]]<!--Person prefers lowercase l (ell), see his article--> and [[Mark Adler]] as a [[free software]] replacement for the [[compress]] program used in early [[Unix]] systems, and intended for use by [[GNU]] (from which the "g" of gzip is derived). Version 0.1 was first publicly released on 31 October 1992, and version 1.0 followed in February 1993.
'''gzip''' is a [[file format]] and a [[software application]] for file [[Data compression|compression and decompression]]. The program was created by [[Jean-loup Gailly]]<!--Person prefers lowercase l (ell), see his article--> and [[Mark Adler]] as a [[free software]] replacement for the [[compress]] program used in early [[Unix]] systems, and intended for use by [[GNU]] (from which the "g" of gzip is derived). Version 0.1 was first publicly released on 31 October 1992, and version 1.0 followed in February 1993.


The decompression of the ''gzip'' format can be implemented as a [[streaming algorithm]], an important{{why|date=January 2025}} feature for [[HTTP compression|Web protocols]], [[Electronic data interchange|data interchange]] and [[Extract, transform, load|ETL]] (in [[Standard streams|standard pipes]]) applications.
As the file format can be decompressed via a [[streaming algorithm]], it is commonly used in stream-based technology such as [[HTTP compression|Web protocols]], [[Electronic data interchange|data interchange]] and [[Extract, transform, load|ETL]] (in [[Standard streams|standard pipes]]).


== File format ==
== File format ==
{{Infobox file format
{{Infobox file format
| name = gzip (file format)
| name = gzip (file format)
| extension = .gz
| extension = <code>.gz</code>, <code>.tgz</code>, <code>.gzip</code>
| mime = application/gzip<ref>{{cite IETF|title=The 'application/zlib' and 'application/gzip' Media Types|rfc=6713|publisher=Internet Engineering Task Force|access-date=2014-03-01}}</ref>
| mime = <code>application/gzip</code><ref>{{cite IETF|title=The 'application/zlib' and 'application/gzip' Media Types|rfc=6713|publisher=Internet Engineering Task Force|access-date=2014-03-01}}</ref>
| uniform type = org.gnu.gnu-zip-archive
| magic = <code>1F 8B</code>
| magic = <code>1F 8B</code>
| owner = Jean-loup Gailly and Mark Adler
| owner = Jean-loup Gailly and Mark Adler
Line 38: Line 37:
| url = {{URL|http://gzip.org}} (obsolete)
| url = {{URL|http://gzip.org}} (obsolete)
| open = Yes
| open = Yes
| standard = [[IETF]] {{IETF RFC|1952}}
| uniform_type = <code>org.gnu.gnu-zip-archive</code>, <code>org.gnu.gnu-zip-tar-archive</code>
| max_size = Unlimited
}}
}}


gzip is based on the [[DEFLATE]] algorithm, which is a combination of [[LZ77 and LZ78|LZ77]] and [[Huffman coding]]. DEFLATE was intended as a replacement for [[LZW]] and other [[patent]]-encumbered [[data compression]] [[algorithm]]s which, at the time, limited the usability of the [[compress]] utility and other popular archivers.
A gzip file (described in the table below) contains a 10-[[byte]] header, optional extra header fields, a [[DEFLATE]]-compressed [[Payload (computing)|payload]] and an 8-byte trailer.


The gzip file format (described in the table below) contains a 10-byte header, optional extra headers,<ref>{{Cite web |last=Deutsch |first=L. Peter |date=May 1996 |title=GZIP file format specification version 4.3 |url=https://tools.ietf.org/html/rfc1952.html |journal= |language=en |publisher=Internet Engineering Task Force |doi=10.17487/RFC1952 |access-date=2019-07-23}}</ref> a deflate-compressed [[Payload (computing)|payload]] and an 8-byte trailer.<ref>{{cite web|author1=Jean-loup Gailly|title=GNU Gzip|url=https://www.gnu.org/software/gzip/manual/gzip.html#index-options-4|website=Gnu.org|access-date=11 October 2015|archive-date=15 October 2015|archive-url=https://web.archive.org/web/20151015064748/http://www.gnu.org/software/gzip/manual/gzip.html#index-options-4|url-status=live}}</ref>
gzip is based on the DEFLATE algorithm, which is a combination of [[LZ77 and LZ78|LZ77]] and [[Huffman coding]]. DEFLATE was intended as a replacement for [[LZW]] and other [[patent]]-encumbered [[data compression]] [[algorithm]]s which, at the time, limited the usability of the [[compress]] utility and other popular archivers.


[[File:Targzip.svg|thumb|gzip can be combined with the [[tar (file format)|tar]] program to compress multiple files.]]
[[File:Targzip.svg|thumb|gzip can be combined with the [[tar (file format)|tar]] program to compress multiple files.]]


Although its file format also allows for multiple such streams to be [[Concatenation|concatenated]] (gzipped files are simply decompressed concatenated as if they were originally one file),<ref>{{cite web| title=GNU Gzip: Advanced usage| url=https://www.gnu.org/software/gzip/manual/html_node/Advanced-usage.html| website=Gnu.org| access-date=2012-11-28| archive-date=24 December 2012| archive-url=https://web.archive.org/web/20121224211417/https://www.gnu.org/software/gzip/manual/html_node/Advanced-usage.html| url-status=live}}</ref> gzip is normally used to compress just single files.<ref>{{cite web| title=Can gzip compress several files into a single archive?| url=http://www.gzip.org/#faq16|website=Gnu.org| access-date=2010-01-27 |archive-url=https://web.archive.org/web/20100722004012/http://www.gzip.org/#faq16 |archive-date=2010-07-22}}</ref> Compressed archives are typically created by assembling collections of files into a single [[tar (file format)|tar]] archive (also called [[tar (file format)|tarball]]),<ref>{{cite web|title=tarball, The Jargon File, version 4.4.7|url=http://www.catb.org/jargon/html/T/tarball.html|website=Catb.org|access-date=2010-01-27|archive-date=20 March 2017|archive-url=https://web.archive.org/web/20170320183848/http://www.catb.org/jargon/html/T/tarball.html|url-status=live}}</ref> and then compressing that archive with gzip. The final compressed file usually has the extension {{mono|.tar.gz}} or {{mono|.tgz}}.
Although multiple streams may be [[Concatenation|concatenated]] (gzipped files are simply decompressed concatenated as if they were originally one file),<ref>{{cite web| title=GNU Gzip: Advanced usage| url=https://www.gnu.org/software/gzip/manual/html_node/Advanced-usage.html| website=Gnu.org| access-date=2012-11-28| archive-date=24 December 2012| archive-url=https://web.archive.org/web/20121224211417/https://www.gnu.org/software/gzip/manual/html_node/Advanced-usage.html| url-status=live}}</ref> normally only a single file is compressed.<ref>{{cite web| title=Can gzip compress several files into a single archive?| url=http://www.gzip.org/#faq16|website=Gnu.org| access-date=2010-01-27 |archive-url=https://web.archive.org/web/20100722004012/http://www.gzip.org/#faq16 |archive-date=2010-07-22}}</ref> Compressed archives are typically created by assembling collections of files into a single [[tar (file format)|tar]] archive (also called [[tar (file format)|tarball]]),<ref>{{cite web|title=tarball, The Jargon File, version 4.4.7|url=http://www.catb.org/jargon/html/T/tarball.html|website=Catb.org|access-date=2010-01-27|archive-date=20 March 2017|archive-url=https://web.archive.org/web/20170320183848/http://www.catb.org/jargon/html/T/tarball.html|url-status=live}}</ref> and then compressing that archive with gzip. The final compressed file usually has the extension <code>.tar.gz</code> or <code>.tgz</code>.


gzip is not to be confused with the [[ZIP (file format)|ZIP]] archive format, which also uses [[Deflate|DEFLATE]]. The ZIP format can hold collections of files without an external archiver, but is less compact than compressed [[tar (computing)|tarballs]] holding the same data, because it compresses files individually and cannot take advantage of redundancy between files ([[solid compression]]).
gzip is not to be confused with the [[ZIP (file format)|ZIP]] archive format, which also uses [[Deflate|DEFLATE]]. The ZIP format can hold collections of files without an external archiver, but is less compact than compressed [[tar (computing)|tarballs]] holding the same data, because it compresses files individually and cannot take advantage of redundancy between files ([[solid compression]]).
The gzip file format is also not to be confused with that of the compress utility, based on LZW, with extension {{mono|.Z}}; however, the gunzip utility is able to decompress .Z files.<ref>{{cite web | title=GNU Gzip | website=The GNU Operating System and the Free Software Movement | date=2023-02-05 | url=https://www.gnu.org/software/gzip/manual/gzip.html | access-date=2024-04-03 |quote=gunzip can currently decompress files created by gzip, zip, compress or pack. The detection of the input format is automatic. }}</ref>
The gzip file format is also not to be confused with that of the compress utility, based on LZW, with extension <code>.Z</code>; however, the gunzip utility is able to decompress <code>.Z</code> files.<ref>{{cite web | title=GNU Gzip | website=The GNU Operating System and the Free Software Movement | date=2023-02-05 | url=https://www.gnu.org/software/gzip/manual/gzip.html | access-date=2024-04-03 |quote=gunzip can currently decompress files created by gzip, zip, compress or pack. The detection of the input format is automatic. }}</ref>
 
=== File structure ===
All multi-byte values are encoded in [[little-endian]].<ref>{{Cite IETF|rfc=1952|title=Overall conventions|page=4}}</ref>


{| class="wikitable"
{| class="wikitable"
|+GZIP file format specification<ref>{{Cite IETF|rfc=1952|title=Member format|page=5}}</ref>
![[Offset (computer science)|Offset]] (bytes)
![[Offset (computer science)|Offset]]<br/>(in&nbsp;bytes)
!Field<ref>{{Cite IETF|rfc=1952|title=Member format|page=5}}</ref>
!Field
!Size (bytes)
!Size<br/>(in&nbsp;bytes)
!Description
!Description
|-
|-
Line 61: Line 65:
|ID1
|ID1
|1
|1
| rowspan="2" |GZIP file format identifier. '''Must be''' '''<code>1F 8B</code>'''.
| rowspan="2" |'''[[File format#Magic number|Magic number]]'''. Must be <code>1F 8B</code>.
|-
|-
|1
|1
Line 70: Line 74:
|CM
|CM
|1
|1
|'''Compression method'''.
|'''Compression method'''. Must be 8&nbsp;(Deflate).
 
* '''0–7''': Reserved.
* '''8''': Deflate.
|-
|-
|3
|3
|FLG
|FLG
|1
|1
|'''Flags'''. Reserved bits must be zero.
|'''[[Flag (programming)|Flags]]'''. Reserved bits must be zero.
 
* '''Bit 0''' ([[Least significant bit|LSb]]): <code>FTEXT</code>. Set by the compressor to indicate the file encoding is probably [[ASCII]].
* '''Bit 0''' ([[Least significant bit|LSb]]): <code>FTEXT</code>. Set by the compressor to indicate the file encoding is probably [[ASCII]].
* '''Bit 1''': <code>FHCRC</code>.
* '''Bit 1''': <code>FHCRC</code>
* '''Bit 2''': <code>FEXTRA</code>.
* '''Bit 2''': <code>FEXTRA</code>
* '''Bit 3''': <code>FNAME</code>.
* '''Bit 3''': <code>FNAME</code>
* '''Bit 4''': <code>FCOMMENT</code>.
* '''Bit 4''': <code>FCOMMENT</code>
* '''Bit 5''': Reserved.
* '''Bit 5''': Reserved
* '''Bit 6''': Reserved.
* '''Bit 6''': Reserved
* '''Bit 7''' ([[Most significant bit|MSb]]): Reserved.
* '''Bit 7''' ([[Most significant bit|MSb]]): Reserved
|-
|-
|4
|4
|MTIME
|MTIME
|4
|4
|Time the file was '''last modified'''. Encoded (in [[little-endian]]) as seconds since the [[UNIX epoch]]. If the compressed data did not come from a file, MTIME is the timestamp '''when compression started'''. 0 means the timestamp is not available.
|[[Unix time]] when the file was '''last modified'''. If the compressed data did not come from a file, <code>MTIME</code> is the Unix time '''when compression started'''. 0 means no [[timestamp]] is available.
|-
|-
|8
|8
Line 100: Line 100:


* Deflate-specific flags.
* Deflate-specific flags.
** '''0''': None ('''default value''').
** '''0''': None ('''default value''')
** '''2''': Best compression (level 9).
** '''2''': Best compression (level 9)
** '''4''': Fastest compression (level 1).
** '''4''': Fastest compression (level 1)
|-
|-
|9
|9
|OS
|OS
|1
|1
|'''Filesystem''' on which compression occurred.
|'''[[Filesystem]]''' on which compression occurred.
* '''255''': Unknown ('''default value''')
* '''255''': Unknown ('''default value''')
* '''0''': FAT filesystem (MS-DOS, OS/2, NT/Win32)
* '''0''': [[File Allocation Table|FAT]] filesystem ([[MS-DOS]], [[OS/2]], [[Windows NT|NT]]/[[Win32]])
* '''1''': Amiga
* '''1''': [[Amiga]]
* '''2''': VMS (or OpenVMS)
* '''2''': [[OpenVMS]] (or VMS)
* '''3''': Unix
* '''3''': [[Unix]]
* '''4''': VM/CMS
* '''4''': [[VM/CMS]]
* '''5''': Atari TOS
* '''5''': [[Atari TOS]]
* '''6''': HPFS filesystem (OS/2, NT)
* '''6''': [[High Performance File System|HPFS]] filesystem (OS/2, NT)
* '''7''': Macintosh
* '''7''': [[Macintosh]]
* '''8''': Z-System
* '''8''': Z-System
* '''9''': CP/M
* '''9''': [[CP/M]]
* '''10''': TOPS-20
* '''10''': [[TOPS-20]]
* '''11''': NTFS filesystem (NT)
* '''11''': [[NTFS]] filesystem (NT)
* '''12''': QDOS
* '''12''': [[86-DOS|QDOS]]
* '''13''': Acorn RISCOS
* '''13''': Acorn [[RISCOS]]
|-
|-
|10
|10
|XLEN
|XLEN
|0 or 2
|0 or 2
| rowspan="2" |'''Extra field''' is a sequence of subfields. Present if the '''FEXTRA''' flag is set. <code>XLEN</code> (little-endian) is the size in bytes of the extra field. Each subfield starts with <code>SI1 SI2</code> (a two-byte identifier; typically two ASCII letters) followed by a two-byte <code>LEN</code> (little-endian) value indicating the remaining number of bytes in the subfield. Subfield IDs with <code>SI2 = 0</code> are reserved for future use.
| rowspan="2" |'''Extra field''' is a sequence of subfields. <code>XLEN</code> is the size in bytes of the extra field. Both are present if the '''FEXTRA''' flag is set. Each subfield starts with <code>SI1 SI2</code> (a two-byte identifier; typically two ASCII letters with some [[mnemonic]] value) followed by a two-byte <code>LEN</code> value indicating the remaining number of bytes in the subfield. Subfield IDs with <code>SI2 = 0</code> are reserved for future use.
|-
|-
|12
|12
Line 133: Line 133:
|0 or <code>XLEN</code>
|0 or <code>XLEN</code>
|-
|-
|{{varies}}
|rowspan=6" {{varies}}
|File name
|File name
|{{varies|0 or varies}}
|{{varies|0 or varies}}
|Name of the '''file being compressed'''. Present if the '''FNAME''' flag is set. [[Null-terminated string|Null-terminated]]. Encoded as [[ISO/IEC 8859-1|ISO 8859-1]] (<code>latin-1</code>). Converted to lowercase on [[case-insensitive]] filesystems. Empty if the compressed data did not come from a named file.
|[[Null-terminated string|Null-terminated]] '''name of the file being compressed'''. Present if the '''FNAME''' flag is set. Encoded as [[ISO/IEC 8859-1|ISO 8859-1]] (<code>latin-1</code>). Converted to [[lowercase]] on [[case-insensitive]] filesystems. Empty if the compressed data did not come from a named file.
|-
|-
|{{varies}}
|File comment
|File comment
|{{varies|0 or varies}}
|{{varies|0 or varies}}
|'''File comment''' intended for human consumption. Present if the '''FCOMMENT''' flag is set. Null-terminated. Encoded as ISO 8859-1 (<code>latin-1</code>). [[Newline|Newlines]] should use a single line feed (LF) character.
|Null-terminated '''file comment''' intended for human consumption. Present if the '''FCOMMENT''' flag is set. Encoded as ISO 8859-1 (<code>latin-1</code>). [[Newline|Newlines]] should use a single line feed (LF) character.
|-
|-
|{{varies}}
|CRC16
|CRC16
|0 or 2
|0 or 2
|'''CRC16''' of all bytes in the gzip header up to (not including) this field. Present if the '''FHCRC''' flag is set.
|Two least significant bytes of the [[CRC-32]] (ISO 3309) of all bytes in the gzip file up to (not including) this field. Present if the '''FHCRC''' flag is set.
|-
|-
|{{varies}}
|Compressed data
|Compressed data
|{{varies}}
|{{varies}}
|The '''compressed data'''.
|The '''compressed data'''.
|-
|-
|{{varies}}
|CRC32
|CRC32
|4
|4
|'''CRC32''' of the '''uncompressed data'''. Encoded as little-endian.
|'''CRC-32''' (ISO 3309) of the '''uncompressed data'''.  
|-
|-
|{{varies}}
|ISIZE
|ISIZE
|4
|4
|'''Size''' (in bytes) of the '''uncompressed data''' modulo <math>2^{32}</math>. Encoded as little-endian.
|'''Size''' (in bytes) of the '''uncompressed data''' modulo <math>2^{32}</math>.
|}
|}


Line 186: Line 181:
The [[tar (computing)|tar]] utility included in most Linux distributions can extract .tar.gz files by passing the {{mono|z}} option, e.g., {{mono|tar -zxf file.tar.gz}}, where <code>-z</code> instructs decompression, <code>-x</code> means extraction, and <code>-f</code> specifies the name of the compressed archive file to extract from. Optionally, <code>-v</code> (''verbose'') lists files as they are being extracted.<ref>{{cite web |title=How To Extract / Unzip tar.gz Files From Linux Command Line |url=https://phoenixnap.com/kb/extract-tar-gz-files-linux-command-line |website=Knowledge Base by phoenixNAP |access-date=12 January 2022 |date=14 November 2019}}</ref>
The [[tar (computing)|tar]] utility included in most Linux distributions can extract .tar.gz files by passing the {{mono|z}} option, e.g., {{mono|tar -zxf file.tar.gz}}, where <code>-z</code> instructs decompression, <code>-x</code> means extraction, and <code>-f</code> specifies the name of the compressed archive file to extract from. Optionally, <code>-v</code> (''verbose'') lists files as they are being extracted.<ref>{{cite web |title=How To Extract / Unzip tar.gz Files From Linux Command Line |url=https://phoenixnap.com/kb/extract-tar-gz-files-linux-command-line |website=Knowledge Base by phoenixNAP |access-date=12 January 2022 |date=14 November 2019}}</ref>


[[zlib]] is an abstraction of the DEFLATE algorithm in library form which includes support both for the gzip file format and a lightweight [[Data stream#General|data stream]] format in its API. The zlib stream format, DEFLATE, and the gzip file format were standardized respectively as RFC 1950, RFC 1951, and RFC 1952.
The [[zlib]] library implements the DEFLATE algorithm, and supports both the gzip file format and a lightweight [[Data stream#General|data stream]] format. The zlib stream format, DEFLATE, and the gzip file format were standardized respectively as RFC 1950, RFC 1951, and RFC 1952.


The gzip format is used in [[HTTP compression]], a technique used to speed up the sending of [[HTML]] and other content on the [[World Wide Web]]. It is one of the three standard formats for HTTP compression as specified in RFC 2616. This [[Request for Comments|RFC]] also specifies a zlib format (called "DEFLATE"), which is equal to the gzip format except that gzip adds eleven bytes of overhead in the form of headers and trailers. Still, the gzip format is sometimes recommended over zlib because [[Internet Explorer]] does not implement the standard correctly and cannot handle the zlib format as specified in RFC 1950.<ref>{{cite web |first=Eric |last=Lawrence |title=Compressing the Web |url=http://blogs.msdn.com/b/ieinternals/archive/2014/10/21/http-compression-optimize-file-formats-with-deflate.aspx |date=21 November 2014 |website=MSDN Blogs > IEInternals |publisher=[[Microsoft]] |access-date=2 November 2015 |archive-date=28 October 2015 |archive-url=https://web.archive.org/web/20151028172035/http://blogs.msdn.com/b/ieinternals/archive/2014/10/21/http-compression-optimize-file-formats-with-deflate.aspx |url-status=live }}</ref>
The gzip format is used in [[HTTP compression]], a technique used to speed up the sending of [[HTML]] and other content on the [[World Wide Web]]. It is one of the three standard formats for HTTP compression as specified in RFC 2616. This [[Request for Comments|RFC]] also specifies a zlib format (called "DEFLATE"), which is equal to the gzip format except that gzip adds eleven bytes of overhead in the form of headers and trailers. Still, the gzip format is sometimes recommended over zlib because [[Internet Explorer]] does not implement the standard correctly and cannot handle the zlib format as specified in RFC 1950.<ref>{{cite web |first=Eric |last=Lawrence |title=Compressing the Web |url=http://blogs.msdn.com/b/ieinternals/archive/2014/10/21/http-compression-optimize-file-formats-with-deflate.aspx |date=21 November 2014 |website=MSDN Blogs > IEInternals |publisher=[[Microsoft]] |access-date=2 November 2015 |archive-date=28 October 2015 |archive-url=https://web.archive.org/web/20151028172035/http://blogs.msdn.com/b/ieinternals/archive/2014/10/21/http-compression-optimize-file-formats-with-deflate.aspx |url-status=live }}</ref>
Line 200: Line 195:
==See also==
==See also==
{{Portal|Free and open-source software}}
{{Portal|Free and open-source software}}
* [[Comparison of file archivers]]
* {{Annotated link|Brotli}}
* [[Open format|Free file format]]
* {{Annotated link|Libarc}}
* [[List of archive formats]]
* {{Annotated link|Comparison of file archivers}}
* [[List of Unix commands]]
* {{Annotated link|List of archive formats}}
* [[Libarc]]
* {{Annotated link|List of POSIX commands}}
* [[Brotli]]
* {{Annotated link|Open file format}}
* [[zlib]]
* {{Annotated link|zlib}}


==Notes==
==Notes==

Latest revision as of 00:38, 14 November 2025

Template:Short description Template:Use dmy dates Template:Lowercase title Script error: No such module "Infobox".Template:Template otherScript error: No such module "Check for unknown parameters".Template:Main other

gzip is a file format and a software application for file compression and decompression. The program was created by Jean-loup Gailly and Mark Adler as a free software replacement for the compress program used in early Unix systems, and intended for use by GNU (from which the "g" of gzip is derived). Version 0.1 was first publicly released on 31 October 1992, and version 1.0 followed in February 1993.

As the file format can be decompressed via a streaming algorithm, it is commonly used in stream-based technology such as Web protocols, data interchange and ETL (in standard pipes).

File format

Script error: No such module "Infobox".Template:Template otherScript error: No such module "Check for unknown parameters".

A gzip file (described in the table below) contains a 10-byte header, optional extra header fields, a DEFLATE-compressed payload and an 8-byte trailer.

gzip is based on the DEFLATE algorithm, which is a combination of LZ77 and Huffman coding. DEFLATE was intended as a replacement for LZW and other patent-encumbered data compression algorithms which, at the time, limited the usability of the compress utility and other popular archivers.

File:Targzip.svg
gzip can be combined with the tar program to compress multiple files.

Although multiple streams may be concatenated (gzipped files are simply decompressed concatenated as if they were originally one file),[1] normally only a single file is compressed.[2] Compressed archives are typically created by assembling collections of files into a single tar archive (also called tarball),[3] and then compressing that archive with gzip. The final compressed file usually has the extension .tar.gz or .tgz.

gzip is not to be confused with the ZIP archive format, which also uses DEFLATE. The ZIP format can hold collections of files without an external archiver, but is less compact than compressed tarballs holding the same data, because it compresses files individually and cannot take advantage of redundancy between files (solid compression). The gzip file format is also not to be confused with that of the compress utility, based on LZW, with extension .Z; however, the gunzip utility is able to decompress .Z files.[4]

File structure

All multi-byte values are encoded in little-endian.[5]

Offset (bytes) Field[6] Size (bytes) Description
0 ID1 1 Magic number. Must be 1F 8B.
1 ID2 1
2 CM 1 Compression method. Must be 8 (Deflate).
3 FLG 1 Flags. Reserved bits must be zero.
  • Bit 0 (LSb): FTEXT. Set by the compressor to indicate the file encoding is probably ASCII.
  • Bit 1: FHCRC
  • Bit 2: FEXTRA
  • Bit 3: FNAME
  • Bit 4: FCOMMENT
  • Bit 5: Reserved
  • Bit 6: Reserved
  • Bit 7 (MSb): Reserved
4 MTIME 4 Unix time when the file was last modified. If the compressed data did not come from a file, MTIME is the Unix time when compression started. 0 means no timestamp is available.
8 XFL 1 Extra flags.
  • Deflate-specific flags.
    • 0: None (default value)
    • 2: Best compression (level 9)
    • 4: Fastest compression (level 1)
9 OS 1 Filesystem on which compression occurred.
10 XLEN 0 or 2 Extra field is a sequence of subfields. XLEN is the size in bytes of the extra field. Both are present if the FEXTRA flag is set. Each subfield starts with SI1 SI2 (a two-byte identifier; typically two ASCII letters with some mnemonic value) followed by a two-byte LEN value indicating the remaining number of bytes in the subfield. Subfield IDs with SI2 = 0 are reserved for future use.
12 Extra field 0 or XLEN
rowspan=6" Template:Varies File name Template:Varies Null-terminated name of the file being compressed. Present if the FNAME flag is set. Encoded as ISO 8859-1 (latin-1). Converted to lowercase on case-insensitive filesystems. Empty if the compressed data did not come from a named file.
File comment Template:Varies Null-terminated file comment intended for human consumption. Present if the FCOMMENT flag is set. Encoded as ISO 8859-1 (latin-1). Newlines should use a single line feed (LF) character.
CRC16 0 or 2 Two least significant bytes of the CRC-32 (ISO 3309) of all bytes in the gzip file up to (not including) this field. Present if the FHCRC flag is set.
Compressed data Template:Varies The compressed data.
CRC32 4 CRC-32 (ISO 3309) of the uncompressed data.
ISIZE 4 Size (in bytes) of the uncompressed data modulo 232.

Implementations

Script error: No such module "Infobox".Template:Template otherScript error: No such module "Check for unknown parameters".Template:Main other Various implementations of the program have been written. The most commonly known is the GNU Project's implementation using Lempel-Ziv coding (LZ77). OpenBSD's version of gzip is actually the compress program, to which support for the gzip format was added in OpenBSD 3.4. The "g" in this specific version stands for gratis.[7] FreeBSD, DragonFly BSD and NetBSD use a BSD-licensed implementation instead of the GNU version; it is actually a command-line interface for zlib intended to be compatible with the GNU implementations' options.[8] These implementations originally come from NetBSD, and support decompression of bzip2 and the Unix pack format.

An alternative compression program achieving 3-8% better compression is Zopfli. It achieves gzip-compatible compression using more exhaustive algorithms, at the expense of compression time required. It does not affect decompression time.

pigz, written by Mark Adler, is compatible with gzip and speeds up compression by using all available CPU cores and threads.[9]

Damage recovery

Data in blocks prior to the first damaged part of the archive is usually fully readable. Data from blocks not demolished by damage that are located afterward may be recoverable through difficult workarounds.[10]

Derivatives and other uses

The tar utility included in most Linux distributions can extract .tar.gz files by passing the Template:Mono option, e.g., Template:Mono, where -z instructs decompression, -x means extraction, and -f specifies the name of the compressed archive file to extract from. Optionally, -v (verbose) lists files as they are being extracted.[11]

The zlib library implements the DEFLATE algorithm, and supports both the gzip file format and a lightweight data stream format. The zlib stream format, DEFLATE, and the gzip file format were standardized respectively as RFC 1950, RFC 1951, and RFC 1952.

The gzip format is used in HTTP compression, a technique used to speed up the sending of HTML and other content on the World Wide Web. It is one of the three standard formats for HTTP compression as specified in RFC 2616. This RFC also specifies a zlib format (called "DEFLATE"), which is equal to the gzip format except that gzip adds eleven bytes of overhead in the form of headers and trailers. Still, the gzip format is sometimes recommended over zlib because Internet Explorer does not implement the standard correctly and cannot handle the zlib format as specified in RFC 1950.[12]

zlib DEFLATE is used internally by the Portable Network Graphics (PNG) format.

Since the late 1990s, bzip2, a file compression utility based on a block-sorting algorithm, has gained some popularity as a gzip replacement. It produces considerably smaller files (especially for source code and other structured text), but at the cost of memory and processing time (up to a factor of 4).[13]

AdvanceCOMP, Zopfli, libdeflate and 7-Zip can produce gzip-compatible files, using an internal DEFLATE implementation with better compression ratios than gzip itself—at the cost of more processor time compared to the reference implementation.Script error: No such module "Unsubst".

Research published in 2023 showed that simple lossless compression techniques such as gzip could be combined with a k-nearest-neighbor classifier to create an attractive alternative to deep neural networks for text classification in natural language processing. This approach has been shown to equal and in some cases outperform conventional approaches such as BERT due to low resource requirements, e.g. no requirement for GPU hardware.[14]

See also

Script error: No such module "Portal".

Notes

Template:Reflist

References

Template:Refbegin

Template:Refend

External links

Template:Archive formats Template:Compression software implementations Template:GNU

  1. Script error: No such module "citation/CS1".
  2. Script error: No such module "citation/CS1".
  3. Script error: No such module "citation/CS1".
  4. Script error: No such module "citation/CS1".
  5. Template:Cite IETF
  6. Template:Cite IETF
  7. Script error: No such module "citation/CS1".
  8. Script error: No such module "citation/CS1".
  9. Script error: No such module "citation/CS1".
  10. Recovering a damaged .gz file – Jean-loup Gailly, GZip.org
  11. Script error: No such module "citation/CS1".
  12. Script error: No such module "citation/CS1".
  13. Script error: No such module "citation/CS1".
  14. Script error: No such module "Citation/CS1".