Package: quilt
Version: 0.42-2
Severity: wishlist
Tags: patch
The *.txt version of the manual is suitale for off-line reading for
development tools (console, less(1), emacs(1), grep(1) ...).
File attached. It was converted/edited directly from quilt.pdf
Jari
-- System Information:
Debian Release: testing/unstable
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'stable')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/dash
Kernel: Linux 2.6.14-2-686
Locale: LANG=C, LC_CTYPE=C (charmap=ISO-8859-1) (ignored: LC_ALL set to en_US)
Versions of packages quilt depends on:
ii bzip2 1.0.3-2 high-quality block-sorting file co
ii diffstat 1.41-1 produces graph of changes introduc
ii gawk 1:3.1.5-1 GNU awk, a pattern scanning and pr
ii gettext 0.14.5-2 GNU Internationalization utilities
ii libc6 2.3.5-13 GNU C Library: Shared libraries an
ii patch 2.5.9-4 Apply a diff file to an original
quilt recommends no packages.
-- no debconf information
Title
How To Survive With Many Patches or Introduction to Quilt(*)
by Andreas Gruunbacher, SuSE Labs, [EMAIL PROTECTED], June 12, 2005
(*) Quilt is a GPL licensed project hosted on GNU Savannah.
Some ideas for this document were taken from docco.txt in
Andrew Morton's patch management scripts package[1]. The text
in the examples was taken from A Midsummer Night's Dream by
William Shakespeare.
Abstract
After looking at different strategies for dealing with software
packages that consist of a base software package on top of which a
number of patches are applied, this document introduces the script
collection quilt, which was specifically written to help deal with
multiple patches and common patch management tasks.
1 Introduction
In the old days, vendor specific software packages in the open source
worldconsisted of a file with the official version of the software,
plus a patch file with the additional changes needed to adapt the
package to specific needs. Theofficial software package was usually
contained in a package.tar.gz file, whilethe patch was found in
package.diff. Instead of modifying the official packagesources, local
changes were kept separate. When building the software package,
the tar archive was extracted, and the patch was applied.Over time,
the patch file ended up containing several independent changes. Of
those changes, some were integrated into later versions of the
software, whileother add-ons or adaptations remain external. Whenever
a new official version was integrated, the patch needed to be
revised: changes that were alreadyintegrated in the official version
needed to be split from changes that were not.
A big improvement was to allow multiple patches in a vendor package,
andthis is also how patches are handled today: a number of patches is
applied on top of each other. Each patch usually consists of a
logically related setof changes. When some patches get integrated
upstream, those patches can simply be removed from the vendor
specific package. The remaining patches frequently continue to apply
cleanly. Some of the remaining patches may haveto be maintained
across a range of upstream versions because they are too specific for
the upstream software package, etc. These patches often get out
ofsync, and need to be updated.
For the majority of packages, the number of patches remains
relatively low,so maintaining those patches without tools is
feasible. A number of packages have dozens of patches, however. At
the extreme end is the kernel source package (kernel-source-2.4.x )
with more than 1 000 patches. The difficulty of managing such a vast
number of patches without tools can easily be imagined.This document
discusses different strategies of dealing with large sets of patches.
Patches are usually generated by the diff utility, and applied with
thepatch utility. Different patch file formats are defined as part of
the specification of the diff utility in POSIX.1[3]. The most
commonly used format today,unified diff, is not covered by POSIX.1,
however. A good description of patch file formats is found in the GNU
diff info pages [4].The question we try to answer in this document is
how patches are best kept up to date in face of changes both to the
upstream software package, and tothe patches that precede them. After
looking at some existing approaches, a collection of patch management
scripts known as quilt is described[2], startingwith basic concepts,
and progressing towards more advanced tasks.
2 Existing Approaches
The minimal solution for updating a patch is to apply all preceding
patches.Then, a copy of the resulting source tree is created(*1) The
next patch in the sequence of patches (which is the one to be
updated) is applied to only one ofthese source trees. This source
tree is then modified until it reflects the desiredresult. The new
version of the patch is distilled by comparing the two sourcetrees
with diff, and writing the result into a file.
This simple approach is rather error prone, and leaves much to be
desired.Several people have independently written scripts that
automate and improve upon this process.A version control system like
CVS or RCS may be a reasonable alternative in some cases. The version
control system is brought in the state of the workingtree with a
number of patches applied. Then the next patch is applied. After the
working tree is updated as required, a diff between the repository
copy andthe working tree is created (with cvs diff, etc). In this
scenario the version control system is used to store and compare
against the old repository versiononly. The full version control
overhead is paid, while only a small fraction of its functionality is
needed. Switching between different patches is not simplified.
One of the most advanced approaches is Andrew Morton's patch
manage-ment scripts [1]. The author of this document found that none
of the available solutions would scale up to the specific
requirements of the SUSE kernel-sourcepackage, and started to improve
Andrew Morton's scripts until they became what they are now [2].
(*1) The two copies can also be hard-linked with each other, which
significantly speeds up both the copying and the final "diffing". If
hard links are used, care must be taken that the tools used to update
one copy of the source tree will create new files, and will not
overwrite shared files. Editors like emacs(1) and vi(1), and
utilities like patch(1), support this.
3 Quilt: Basic Concepts and Operation
The remainder of this document discusses the script collection quilt.
With quilt, all work occurs within a single directory tree. Since
version 0.30, commands can be invoked from anywhere within the source
tree. Commands are of the form "quilt cmd" similar to cvs(1) commands.
They can be abbreviatedas long as the specified part of the command
is unique. All commands print some help text with "quilt cmd -h."
Quilt manages a stack of patches. Patches are applied incrementally
on top of the base tree plus all preceding patches. They can be
pushed on top of the stack (quilt push), and popped off the stack
(quilt pop). Commands are availablefor querying the contents of the
series file (quilt series, see below), the contents of the stack
(quilt applied, quilt previous, quilt top), and the patches that are
not applied at a particular moment (quilt next, quilt unapplied). By
default, most commands apply to the topmost patch on the stack.
When files in the working directory are changed, those changes become
partof the working state of the topmost patch, provided that those
files are part of the patch. Files that are not part of a patch must
be added before modifying them so that quilt is aware of the original
versions of the files. The "quilt refresh" command regenerates a
patch. After the refresh, the patch and the working state are the
same.
Patch files are located in the patches sub-directory of the source
tree (see Figure 1). The QUILT_PATCHES environment variable can be
used to override this location. The patches directory may contain
sub-directories. Directory "patches" may also be a symbolic link
instead of a directory.
A file called "series" contains a list of patch file names that
defines the order inwhich patches are applied. Unless there are means
by which series files can be generated automatically (see Section
5.8), they are usually provided along witha set of patches. In
series, each patch file name is on a separate line. Patch files are
identified by pathnames that are relative to the patches directory;
patches may be in sub-directories below the patches directory. Lines
in the series file that start with a hash character (#) are ignored.
When quilt adds, removes or renames patches, it automatically updates
the "series" file. Users of quilt can modify series files while some
patches are applied, as long as the applied patches remain in their
original order.
Different "series" files can be used to assemble patches in different
ways, corresponding for example to different development branches.
work/ -+- ...
+- patches/
| +- series
| +- patch2.diff
| +- patch1.diff
| +- ...
|
+- .pc/
|
+- applied-patches/
|
+- patch1.diff/ -+- ...
+- patch2.diff/ -+- ...
+- ...
Figure 1: Quilt files in a source tree.
Before a patch is applied (or "pushed on the stack"), copies of all
files the patch modifies are saved to the .pc/patch directory(*2).
The patch is added to the list of currently applied patches
(.pc/applied-patches). Later when a patch is regenerated (quilt
refresh), the backup copies in .pc/patch are compared with the
current versions of the files in the source tree using GNU diff.
Documentation related to a patch can be put at the beginning of a
patchfile. Quilt is careful to preserve all text that precedes the
actual patch when doing a refresh.
The series file is looked up in the root of the source tree, in the
patches directory, and in the .pc directory. The first series file
that is found is used. This may also be a symbolic link, or a file
with multiple hard links. Usually, only one series file is used for a
set of patches, so the patches sub-directory is aconvenient location.
While patches are applied to the source tree, the .pc directory is
essentialfor many operations, including taking patches off the stack
(quilt pop), andrefreshing patches (quilt refresh). Files in the .pc
directory are automaticallyremoved when they are no longer needed, so
usually there is no need to clean up manually. The QUILT_PC
environment variable can be used to override thelocation of the .pc
directory.
(*2) The patch name with extensions stripped is used as the name of
the sub-directory below the .pc directory. GNU patch, which quilt
uses internally to apply patches, creates backup files and applies
the patch in one step.
4 An Example
This section demonstrates how new patches are created and updated,
and how conflicts are resolved. Let's start with a short text file:
Yet mark'd I where the bolt of Cupid fell:
It fell upon a little western flower,
Before milk-white, now purple with love's wound,
And girls call it love-in-idleness.
New patches are created with quilt new. A new patch automatically
becomesthe topmost patch on the stack. Files must be added with quilt
add before they are modified. Note that this is slightly different
from the CVS style of interaction: with CVS, files are in the
repository, and adding them before committing (but after modifying
them) is enough. Files are usually added and immediately modified. The
command quilt edit adds a file and loads it into the default editor.
(The environment variable EDITOR specifies which is the default
editor. If EDITOR is not set, vi(1) is used.)
$ quilt new flower.diff
Patch flower.diff is now on top
$ quilt edit Oberon.txt
File Oberon.txt added to patch flower.diff
Let's assume that the following lines were added to Oberon.txt
during editing:
The juice of it on sleeping eye-lids laid
Will make a man or woman madly dote
Upon the next live creature that it sees.
The actual patch file is created (and later updated) with
quilt refresh. The result is as follows(*3):
$ quilt refresh
$ cat patches/flower.diff
Index: example1/Oberon.txt
===================================================================
--- example1.orig/Oberon.txt
+++ example1/Oberon.txt
@@ -2,3 +2,6 @@
It fell upon a little western flower,
Before milk-white, now purple with love's wound,
And girls call it love-in-idleness.
+The juice of it on sleeping eye-lids laid
+Will make a man or woman madly dote
+Upon the next live creature that it sees.
(*3) Timestamps in patches are omitted from the output in the examples.
Now let's assume that a line in the text has been overlooked, and
must be inserted. The file Oberon.txt is already part of the patch
flower.diff, so it canimmediately be modified in an editor.
Alternatively, quilt edit can be used; it simply opens up the default
editor if the file is already part of the patch. After the line is
added, we use "quilt diff -z" to see a diff of the changes we made:
$ quilt diff -z
Index: example1/Oberon.txt
===================================================================
--- example1.orig/Oberon.txt
+++ example1/Oberon.txt @@ -2,6 +2,7 @@
It fell upon a little western flower,
Before milk-white, now purple with love's wound,
And girls call it love-in-idleness.
+Fetch me that flower; the herb I shew'd thee once:
The juice of it on sleeping eye-lids laid
Will make a man or woman madly dote
Upon the next live creature that it sees.
A diff of the topmost patch can be generated with quilt diff without
arguments. This does not modify the actual patch file. The changes
are only added to the patch file by updating it with "quilt refresh".
Then we remove the patch from the stack with "quilt pop":
$ quilt refresh
Refreshed patch flower.diff
$ quilt pop
Removing flower.diff
Restoring Oberon.txt
No patches applied
Next, let's assume that Oberon.txt was modified "upstream": The word
girl did not fit very well, and so it was replaced with maiden.
Oberon.txt now contains:
Yet mark'd I where the bolt of Cupid fell:
It fell upon a little western flower,
Before milk-white, now purple with love's wound,
And maidens call it love-in-idleness.
This causes flower.diff to no longer apply cleanly. When we try to
push flower.diff on the stack with quilt push, we get the following
result:
$ quilt push
Applying flower.diff
patching file Oberon.txt
Hunk #1 FAILED at 2.
1 out of 1 hunk FAILED -- rejects in file Oberon.txt
Patch flower.diff does not apply (enforce with -f)
Quilt does not automatically apply patches that have rejects. Patches
that do not apply cleanly can be "force-applied" with "quilt push
-f", which leaves reject files in the source tree for each file that
has conflicts. Those conflicts must be resolved manually, after which
the patch can be updated (quilt refresh). Quilt remembers when a
patch has been force-applied. It refuses to push further patches on
top of such patches, and it does not remove them from the stack. A
force-applied patch may be "force-removed" from the stack with "quilt
pop -f",however. Here is what happens when force-applying flower.diff:
$ quilt push -f
Applying flower.diff
patching file Oberon.txt
Hunk #1 FAILED at 2.
1 out of 1 hunk FAILED -- saving rejects to file Oberon.txt.rej
Applied flower.diff (forced; needs refresh)
After re-adding the lines of verse from flower.diff to Oberon.txt, we
update the patch with "quilt refresh".
$ quilt edit Oberon.txt
$ quilt refresh
Refreshed patch flower.diff
Our final version of Oberon.txt contains:
Yet mark'd I where the bolt of Cupid fell:
It fell upon a little western flower,
Before milk-white, now purple with love's wound,
And maidens call it love-in-idleness.
Fetch me that flower; the herb I shew'd thee once:
The juice of it on sleeping eye-lids laid
Will make a man or woman madly dote
Upon the next live creature that it sees.
5 Further Commands and Concepts
This section introduces a few more basic commands, and then describes
additional concepts that may not be immediately obvious. We do not
describe all of the features of quilt here since many quilt commands
are quite intuitive; furthermore, help text that describes the
available options for each command is available via "quilt <cmd> -h".
The "quilt top" command shows the name of the topmost patch. The
"quilt files" command shows which files a patch contains. The "quilt
patches" command showswhich patches modify a specified file. With our
previous example, we get the following results:
$ quilt top flower.diff
$ quilt files
Oberon.txt
$ quilt patches
Oberon.txt
flower.diff
The "quilt push" and "quilt pop" commands optionally take a number or
a patchname as argument. If a number is given, the specified number
of patches is added (quilt push) or removed (quilt pop). If a patch
name is given, patchesare added (quilt push) or removed (quilt pop)
until the specified patch is on top of the stack. With the -a option,
all patches in the series file are added (quilt push), or all applied
patches are removed from the stack (quilt pop).
5.1 Patch Strip Levels
Quilt assumes that patches are applied with a strip level of one (the
-p1 option of GNU patch) by default: the topmost directory level of
file names in patches is stripped off. Quilt remembers the strip
level of each patch in the series file. When generating a diff (quilt
diff) or updating a patch (quilt refresh), a different strip level
can be specified, and the series file will be updated accordingly.
Quilt can apply patches with an arbitrary strip level, and produces
patches with a strip level of zero or one. With a strip level of one,
the name of the directory that contains the working tree is used as
the additional path component. (So in our example, Oberon.txt is
contained in directory example1.)
5.2 Importing Patches
The quilt import command automates the importing of patches into the
quilt system. The command copies a patch into the patches directory
and adds it to the series file. For patch strip levels other than one,
the strip level is added after the patch file name. (An entry for
a.diff with strip level zero might read" a.diff -p0".)
Another common operation is to incorporate a fix or similar that
comes as a patch into the topmost patch. This can be done by hand by
first adding all the files contained in the additional patch to the
topmost patch with "quilt add"(*4) and then applying the patch to the
working tree. The "quilt fold" command combines these steps.
(*4) The lsdiff(1) utility, which is part of the patchutils package,
generates a list of files affected by a patch.
5.3 Sharing patches with others
For sharing a set of patches with someone else, the series file which
contains the list of patches and how they are applied, and the patches
themselves are all that's needed. The .pc directory only contains
quilt's working state, and should not be distributed. Make sure that
all the patches are up-to-date, and refresh patches when necessary.
The -combine option of quilt diff can be used for generating a single
patch out of all the patches in the "series" file.
5.4 Merging with upstream
The concept of merging your patches with upstream is identical to
applying your patches on a more recent version of the software.
Before merging, make sure to pop all your patches using "quilt pop -a".
Then, update your codebase. Finally, remove obsoleted patches from the
series file and "quilt push" the remaining ones, resolve conflicts and
"quilt refresh" patches as needed.
5.5 Forking
There are situations in which updating a patch in-place is not ideal:
the same patch may be used in more than one series file. It may also
serve as convenient documentation to retain old versions of patches,
and create new ones underdifferent names. This can be done by hand by
creating a copy of a patch (which must not be applied), and updating
the patch name in the series file.
The "quilt fork" command simplifies this: it creates a copy of the
topmost patch in the series, and updates the "series" file. Unless a
patch name is explicitly specified, "quilt fork" will generate the
following sequence of patch names: patch.diff, patch-2.diff,
patch-3.diff etc.
5.6 Dependencies
When the number of patches in a project grows large, it becomes
increasingly difficult to find the right place for adding a new patch
in the patch series. At a certain point, patches will get inserted at
the end of the patch series, because finding the right place has
become too complicated. In the long run, a mess accumulates.
To help avoid this by keeping the big picture, the "quilt graph"
command generates dot graphs showing the dependencies between
patches(*5). The ouput of this command can be visualized using the
tools from AT&T Research's Graph Visualization Project (GraphViz,
<http://www.graphviz.org/>). The "quilt graph" command supports
different kinds of graphs.
(*5) The quilt graph command computes dependencies based on which
patches modify which files, and optionally will also check for
overlapping changes in the files. While the former approach will
often result in false positives, the latter approach may result in
false negatives (that is, quilt graph may overlook dependencies).
5.7 Advanced Diffing
Quilt allows us to diff and refresh patches that are applied, but are
not on top of the stack ("quilt diff -P <patch>" and "quilt refresh
<patch>"). This is useful in several cases, for example, when patches
applied higher on the stack modify some of the files that this patch
modifies. We can picture this as a shadow which the patches higher on
the stack throw on the files they modify. When refreshing a patch,
changes to files that are not shadowed (and thus were last modified
bythe patch that is being refreshed) are taken into account. The
modifications that the patch contains for shadowed files will not be
updated.
The "quilt diff" command allows us to merge multiple patches into one
by optionally specifying the range of patches to include (see quilt
diff -h). The combined patch will only modify each file contained in
these patches once. The result of applying the combined patch is the
same as applying all the patches in the specified range in sequence.
Sometimes it is convenient to use a tool other than GNU diff for
comparing files; for example, a graphical diff replacement like
tkdiff(1). Quilt will not use tools other than GNU diff when updating
patches (quilt refresh), but "quilt diff" can be passed the
--diff=utility argument. With this argument, the specified utility is
invoked for each file that is being modified with the original file
and new file as arguments. For new files, the first argument will be
/dev/null. Forremoved files, the second argument will be /dev/null.
When "quilt diff" is passed a list of file names, the diff will be
limited to thosefiles. With the -R parameter, the original and new
files are swapped, which results in a reverse diff.
Sometimes it is useful to create a diff between an arbitrary state of
the working tree and the current version. This can be used to create
a diff between different versions of a patch (see Section 5.5), etc.
To this end, quilt allows us to take a snapshot of the working
directory (quilt snapshot). Later, a diff against this state of the
working tree can be created with "quilt diff --snapshot".
Currently, only a single snapshot is supported. It is stored in the
.pc/.snapdirectory. To recover the disk space the snapshot occupies,
it can be removed with "quilt snapshot -d", or by removing the
.pc/.snap directory manually.
5.8 Working with RPM Packages
Several Linux distributions are based on the RPM Package Manager[5].
RPMpackages consist of a spec that defines how packages are built,
and a number of additional files like tar archives, patches, etc.
Most RPM packages contain anofficial software package plus a number
of patches. Before these patches can be manipulated with quilt, a
series file must be created that lists the patches along with their
strip levels.
The quilt setup command automates this for most RPM packages. When
given a spec file as its argument, it performs the %prep section of
the spec file, which is supposed to extract the official software
package, and apply the patches. In this run, quilt remembers the tar
archives and the patches that are applied and creates a series file.
Based on that series file, quilt setup extracts the archives, and
copies the patches into the patches sub-directory. Some
meta-information like the archive names are stored as comments in the
series file. quilt setup also accepts a series file as argument
(which must contain some meta-information), and sets up the working
tree from the series file in this case.
6 Customizing Quilt
Upon startup, quilt evaluates the file $HOME/.quiltrc in the user's
home directory, or the file specified with the --quiltrc option. This
file is a regular bash script. Default options can be passed to any
command by defining a QUILT_<COMMAND>_ARGS variable (for example,
QUILT_ DIFF_ARGS="-color=auto" causes the output of "quilt diff" to
be syntax colored when writing to a terminal). In addition to that,
quilt recognizes the following variables:
o QUILT_DIFF_OPTS; Additional options that quilt shall pass to GNU
diff when generating patches. A useful setting for C source code
is " -p", which causes GNU diff to show in the resulting patch
which function a change is in.
o QUILT_PATCH_OPTS; Additional options that quilt shall pass to GNU
patch when applying patches. (For example, some versions of GNU
patch support the --unified-reject-files option for generating
reject files in unified diff style.
o QUILT_PATCHES; The location of patch files (see Section 3). This
settingdefaults to "patches".
7 Pitfalls and Known Problems
As mentioned earlier, files must be added to patches before they can
be modified. If this step is overlooked, one of the following problems
will occur: If the file is included in a patch further below on the
stack, the changes will appear in that patch when it is refreshed, and
for that patch the quilt pop command will fail before it is refreshed.
If the file is not included in any applied patch, the original file
in the working tree is modified. Patch files may modify the same file
more than once. GNU patch has a bug that corrupts backup files in
this case. A fix is available, and will be integrate din a later
version of GNU patch. The fix has already been integrated into the
SUSE version of GNU patch.
There are some packages that assume that it's a good idea to remove
all empty files throughout a working tree, including the .pc
directory. The "make clean" target in the linux kernel sources is an
example. Quilt uses zero-length files in .pc to mark files added by
patches, so such packages may corrupt the .pc directory. A workaround
is to create a symbolic link .pc in the working tree that points to a
directory outside.
It may happen that the files in the patches directory gets out of
sync with the working tree (e.g., they may accidentally get updated
by CVS or similar). Files in the .pc directory may also become
inconsistent, particularly if files are not added before modifying
them (quilt add / quilt edit). If this happens, it may be possible to
repair the source tree, but often the best solution is to start over
with a scratch working directory and the patches sub-directory. There
is no need to keep any files from the .pc directory in this case.
8 Summary
We have shown how the script collection quilt solves various problems
that occur when dealing with patches to software packages. Quilt is
an obvious improvement over directly using the underlying tools (GNU
patch, GNU diff,etc.), and offers many features not available with
competing solutions. Join the club! The quilt project homepage is
<http://savannah.nongnu.org/projects/quilt/>. There is a development
mailing list at <http://mail.nongnu.org/mailman/listinfo/quilt-dev>.
Additional features that fit into quilt's modeof operation are always
welcome, of course.
References
[1] Andrew Morton: Patch Management Scripts,
<http://lwn.net/Articles/13518/> and
<http://www.zip.com.au/~akpm/linux/patches/patch-scripts-0.9>.
[2] Andreas Gruunbacher et al.: Patchwork Quilt,
<http://savannah.nongnu.org/projects/quilt>.
[3] IEEE Std. 1003.1-2001: Standard for Information Technology,
Portable Operating System Interface (POSIX), Shell and Utilities, diff
command, pp. 317. Online version available from the The Austin Common
Standards Revision Group, <http://www.opengroup.org/austin/>.
[4] GNU diff info pages (info Diff), section Output Formats.
[5] Edward C. Bailey: Maximum RPM: Taking the Red Hat Package
Manager to the Limit, <http://www.rpm.org/max-rpm/>.
End of file
