New Source Format

Status

Details

With the rise of tools such as dpatch and dbs, it's becoming increasingly apparent that the existing format of source packages isn't sufficient for everybody's needs. A source format that allows multiple pristine source files and multiple patches, as well as including the debian directory itself, is certainly in demand.

One of the most common suggestions is actually to invent a .src.deb format which would be a single ar or tar file containing the rest of the files.

The problem with this is it massively increases the amount of mirror traffic as for every minor patch update you'd have to ship the entire updated source package, including upstream source; currently you only upload a new .diff.gz file.

Another suggestion is a format that groups the upstream sources together into one file, and the patches together into another file, .patches.tar.gz for example. While seemingly sensible, this isn't really any different to what we have today.

The third suggestion is simply to upload them all separately, this would be the most difficult to manage but give the most benefit. A revision that updated a particular patch would only upload that patch and a new changelog.

Wig And Pen Format

This format was designed in a [http://www.wigandpen.com.au/ Canberra pub] (thus the name) by BrendanODea, JamesTroup and ScottJamesRemnant. Rather than being a radical diversion from the current format, it's an evolution to cope with the more common uses today.

Native packages remain unchanged, a single .tar.gz accompanied by a .dsc file.

For non-native packages, the unpacking of the .orig.tar.gz will remain the same; however you're permitted to add additional original upstream tarballs by naming them package_version.orig-name.tar.gz. These tarballs will be unpacked as package-version/name, ie. underneath the traditional source directory. Any numeric suffix to name is removed from the resulting subdirectory, which allows updated versions of such additional tarballs.

Changes to either set of tarballs can be stored in the .diff.gz, alterations to any additional tarball would simply include the subdirectory name in the patch hunk filenames.

As an alternative to a .diff.gz, the debian directory may be tarred up into a .debian.tar.gz <!> this extension is provisional, and sub-optimal. This is unpacked by dpkg-source into the right place. Providing both a .diff.gz or a .debian.tar.gz is illegal; as is providing a .debian.tar.gz for a native source package.

If you use a .debian.tar.gz, you will be able to place patch files in debian/patches; these will be automatically applied in ASCIIbetical order by dpkg-source when the tarball is unpacked. This means that there is no need to concern yourself with them in debian/rules, and no need for an unpack target or attempting to unapply them in a clean target. dpkg-source uses the same naming convention as run-parts when determining which patches to apply (upper and lower case letters, digits, underscores, and hyphens).

In all instances, .bz2 may be used in place of .gz for source packages that benefit from it.

Eventually native packages and the "Version 1.0" format (.diff.gz) would be deprecated, and all source packages would consist of two tarballs.

Example (picking on glibc):

glibc_2.3.2.ds1.orig.tar.gz
glibc_2.3.2.ds1.orig-nptl.tar.gz
glibc_2.3.2.ds1.orig-posixthreads.tar.gz
glibc_2.3.2.ds1-20ubuntu13.debian.tar.gz
glibc_2.3.2.ds1-20ubuntu13.dsc

This would be unpacked as:

glibc-2.3.2.ds1/
glibc-2.3.2.ds1/nptl/
glibc-2.3.2.ds1/posixthreads/
glibc-2.3.2.ds1/debian/

With any patch files found in glibc-2.3.2.ds1/debian/patches/ automatically applied with -p1 in the top-level.

Notes

It may also be worth allowing updating of the main upstream tarball without bumping the source version number, by allowing the upload of foo_1.0.orig-1.tar.gz to supersede foo_1.0.orig.tar.gz.

To work well, the patches should be pulled from a version control system (arch, svn, baz-ng, darcs, etc), rather than using a "debian/rules commit" approach. baz-ng and darcs are probably lightweight enough to work really well. Thoughts at NewSourceFormatAndVersionControl.

Package Building

Presently dpkg-source only has support for unpacking of v2.0 format source packages.

Proposed: a smallish change to the operation of dpkg-source -b such that prior to the normal build code the following steps would occur:

If the exit status was success and ../package_ver.dsc was created, containing a file list with correct checksums then the normal build process would be skipped.

This would allow packages to "plug in" a build process appropriate for the package: a tool as described above to gather patches from a version control system, a variant on dpatch, a purpose-built script, whatever.

The default behaviour would still remain to create basic v1.0 (orig.tar.gz + diff.gz) source packages, which for the majority of packages is sufficient.

dpkg: NewSourceFormat (last edited 2005-08-18 09:06:37 by BrendanODea)