extract gzip files

[Mark Brethen]

For *.gz files (no tar), what’s the proper way to extract them to extract_dir?




Mark Brethen
***@gmail.com

[Mark Brethen]

I can override the extract phase and use a foreach block with something like this:

STEM=$(basename "${f}" .gz)
gzip -c "${f}” >${extract_dir}/"${STEM}”


Mark Brethen

Post by Mark Brethen
For *.gz files (no tar), what’s the proper way to extract them to extract_dir?
Mark Brethen

[Mark Brethen]

This doesn’t work:

extract.suffix = .gz
extract.cmd = gunzip
extract.pre_args = -c
extract.post_args = "> ${workpath}"

so, I’m using this:

extract {
foreach f ${distfiles} {
set stem [file rootname [file tail ${f}]]
system -W ${distpath} "gunzip -c ${f} > ${workpath}/${stem}"
}
}

have to provide the extracted filename.

Mark Brethen

Post by Mark Brethen
STEM=$(basename "${f}" .gz)
gzip -c "${f}” >${extract_dir}/"${STEM}”
Mark Brethen

Post by Mark Brethen
For *.gz files (no tar), what’s the proper way to extract them to extract_dir?
Mark Brethen

[Rainer Müller]

Post by Mark Brethen
extract.suffix = .gz
extract.cmd = gunzip
extract.pre_args = -c
extract.post_args = "> ${workpath}"

This cannot work as ${workpath} is a directory and you cannot redirect
output to a directory, you would have to specify a regular file.

Can you rely on the extracted filename that is stored in the .gz? Then
most extract options can be left on the default values:

extract.pre_args -d
extract.post_args
extract.mkdir yes

By default, the extract command will be executed inside extract.dir,
which would be ${workpath}. The extract.mkdir option will create
${worksrcpath} first and extract to the new directory. This should
simplifies the following configure/build/destroot commands that will be
executed in ${worksrcpath} by default.

Rainer

[Joshua Root]

Post by Rainer Müller

Post by Mark Brethen
extract.suffix = .gz
extract.cmd = gunzip
extract.pre_args = -c
extract.post_args = "> ${workpath}"

This cannot work as ${workpath} is a directory and you cannot redirect
output to a directory, you would have to specify a regular file.
Can you rely on the extracted filename that is stored in the .gz? Then
extract.pre_args -d
extract.post_args
extract.mkdir yes
By default, the extract command will be executed inside extract.dir,
which would be ${workpath}. The extract.mkdir option will create
${worksrcpath} first and extract to the new directory. This should
simplifies the following configure/build/destroot commands that will be
executed in ${worksrcpath} by default.

Unfortunately I don't think this will work correctly, since even if you
add the -k option to avoid deleting the .gz file from the distpath, gzip
will still try to create the decompressed file in the distpath as well.

- Josh

[Ryan Schmidt]

Post by Mark Brethen
For *.gz files (no tar), what’s the proper way to extract them to extract_dir?

MacPorts doesn't have specific support for this built-in, I guess because it wasn't thought to be a common enough use case. But there are several ports in the tree that do this manually; you can probably find them by grepping for "extract.suffix .gz".

Just as we recently added a use_tar option to handle uncompressed tarballs, even though that is uncommon, we could add an option to handle gz-/bz2-/xz-/etc. compressed non-tar files. But there might be some confusion about what to call the options. We already have options use_bzip2 and use_xz, for example, which handle bz2- and xz-compressed tarballs, respectively; there is no use_gz or use_gzip, but if there were one, for consistency it would have to mean a gz-compressed tarball; there is no such option because that is the default behavior.

Ultimately, I would like MacPorts to be able to automatically extract almost any file, without needing to be told what compression format was used. See https://lists.macports.org/pipermail/macports-dev/2016-March/032671.html . But every port that provides custom extract phase settings is an obstacle to that (i.e. is something that might break if we release a new version of MacPorts with that functionality, depending on how it's implemented), hence my desire to first provide options to accommodate most ports' extraction needs without the need to override any extract phase settings.

[Mark Brethen]

Abiword used this solution:

distname AbiWord-${version}-10.2
extract.suffix .dmg.gz
extract.post_args > ${workpath}/${distname}.dmg

but I seven files, all .ps.gz, will this work with distfiles instead?

Alternatively, I could use extract {} and then change destroot to

destroot {
xinstall -d ${destroot}${prefix}/share/doc/${name}
foreach f ${distfiles} {
set stem [file rootname [file tail ${f}]]
system -W ${distpath} "gunzip -c ${f} > ${destroot}${prefix}/share/doc/${name}//${stem}"
}
}

Mark Brethen

Post by Ryan Schmidt

For *.gz files (no tar), what’s the proper way to extract them to extract_dir?

MacPorts doesn't have specific support for this built-in, I guess because it wasn't thought to be a common enough use case. But there are several ports in the tree that do this manually; you can probably find them by grepping for "extract.suffix .gz".
Just as we recently added a use_tar option to handle uncompressed tarballs, even though that is uncommon, we could add an option to handle gz-/bz2-/xz-/etc. compressed non-tar files. But there might be some confusion about what to call the options. We already have options use_bzip2 and use_xz, for example, which handle bz2- and xz-compressed tarballs, respectively; there is no use_gz or use_gzip, but if there were one, for consistency it would have to mean a gz-compressed tarball; there is no such option because that is the default behavior.
Ultimately, I would like MacPorts to be able to automatically extract almost any file, without needing to be told what compression format was used. See https://lists.macports.org/pipermail/macports-dev/2016-March/032671.html . But every port that provides custom extract phase settings is an obstacle to that (i.e. is something that might break if we release a new version of MacPorts with that functionality, depending on how it's implemented), hence my desire to first provide options to accommodate most ports' extraction needs without the need to override any extract phase settings.

[Mark Brethen]

This works, avoids overriding extract, but is less direct:

extract.cmd cp
extract.pre_args
extract.post_args ${workpath}/.

destroot {
xinstall -d ${destroot}${prefix}/share/doc/${name}
foreach f ${distfiles} {
set stem [file rootname [file tail ${f}]]
system -W ${workpath} "gunzip -c ${f} > ${destroot}${prefix}/share/doc/${name}/${stem}"
}
}

Preference?


Mark Brethen

Post by Mark Brethen
distname AbiWord-${version}-10.2
extract.suffix .dmg.gz
extract.post_args > ${workpath}/${distname}.dmg
but I seven files, all .ps.gz, will this work with distfiles instead?
Alternatively, I could use extract {} and then change destroot to
destroot {
xinstall -d ${destroot}${prefix}/share/doc/${name}
foreach f ${distfiles} {
set stem [file rootname [file tail ${f}]]
system -W ${distpath} "gunzip -c ${f} > ${destroot}${prefix}/share/doc/${name}//${stem}"
}
}
Mark Brethen

Post by Ryan Schmidt

For *.gz files (no tar), what’s the proper way to extract them to extract_dir?

MacPorts doesn't have specific support for this built-in, I guess because it wasn't thought to be a common enough use case. But there are several ports in the tree that do this manually; you can probably find them by grepping for "extract.suffix .gz".
Just as we recently added a use_tar option to handle uncompressed tarballs, even though that is uncommon, we could add an option to handle gz-/bz2-/xz-/etc. compressed non-tar files. But there might be some confusion about what to call the options. We already have options use_bzip2 and use_xz, for example, which handle bz2- and xz-compressed tarballs, respectively; there is no use_gz or use_gzip, but if there were one, for consistency it would have to mean a gz-compressed tarball; there is no such option because that is the default behavior.
Ultimately, I would like MacPorts to be able to automatically extract almost any file, without needing to be told what compression format was used. See https://lists.macports.org/pipermail/macports-dev/2016-March/032671.html <https://lists.macports.org/pipermail/macports-dev/2016-March/032671.html> . But every port that provides custom extract phase settings is an obstacle to that (i.e. is something that might break if we release a new version of MacPorts with that functionality, depending on how it's implemented), hence my desire to first provide options to accommodate most ports' extraction needs without the need to override any extract phase settings.

[Ryan Schmidt]

Post by Mark Brethen

Post by Mark Brethen
distname AbiWord-${version}-10.2
extract.suffix .dmg.gz
extract.post_args > ${workpath}/${distname}.dmg
but I seven files, all .ps.gz, will this work with distfiles instead?

No, since it specifies the name of the file to write the output to.

Post by Mark Brethen
extract.cmd cp
extract.pre_args
extract.post_args ${workpath}/.
destroot {
xinstall -d ${destroot}${prefix}/share/doc/${name}
foreach f ${distfiles} {
set stem [file rootname [file tail ${f}]]
system -W ${workpath} "gunzip -c ${f} > ${destroot}${prefix}/share/doc/${name}/${stem}"
}
}

At that point, you may as well skip the extract part entirely by setting "extract.only", and ungzipping each file in destroot directly from its location in distpath. That would save the time of copying the gzipped files.