apt-diff: a tool to diff filesystem content against APT
(I'm not subscribed to this list, please CC me on replies.) I have recently written a tool I call "apt-diff" to compare filesystem content against APT. It is intended for investigating problems where packaged files get modified/deleted after installing them from APT (e.g., by user customization, accidental deletion, etc.). I originally wrote it after clobbering the packaged ALSA installation on my computer with "make install" and needing a good way to detect which packaged files had been modified so that I could restore them. I also find it useful for figuring out what customizations I have made to my system after I have forgotten about them. Conceptually apt-diff is meant to act like "svn diff", "git diff", etc., except that instead of comparing to a source code repository it compares to the APT repositories. I've designed it for processing many files/packages in batch, so diff'ing the entire filesystem is doable. It's sort of like an APT-aware version of debsums. The code is at https://github.com/TristanSchmelcher/apt-diff My hope is for this to become part of the APT stack in Debian and I'm posting here to see if there is any interest in that from the Debian developer community. I think it might be helpful for it to be invoked from reportbug to diagnose/triage issues where files or configurations have been changed. Plus I think it's just a nice tool to have available for power users. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlkti=xrf33uwabvtcgxg1jdazubkuz77exf4g3y...@mail.gmail.com
Re: Re: apt-diff: a tool to diff filesystem content against APT
> how does it deal with configurations generated in postinstall? Only files shipped by a package (i.e., appear in its .list file) can be diff'ed, so if a configuration file was generated from scratch in a postinst then apt-diff can't show a diff for it. But it does keep track of how many files it has seen that aren't owned by any package, and there is an option to print the list, so you can use that to see what generated files are present in a directory tree. But at that point you sort of have to know what you're looking for. Sort of analogous to an untracked file in a VCS system. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1291782138.22848.16.ca...@tinygod3
Re: apt-diff: a tool to diff filesystem content against APT
On Tue, Dec 7, 2010 at 11:30 PM, Goswin von Brederlow wrote: > Tristan Schmelcher writes: > >>> how does it deal with configurations generated in postinstall? >> >> Only files shipped by a package (i.e., appear in its .list file) can be >> diff'ed, so if a configuration file was generated from scratch in a >> postinst then apt-diff can't show a diff for it. But it does keep track >> of how many files it has seen that aren't owned by any package, and >> there is an option to print the list, so you can use that to see what >> generated files are present in a directory tree. But at that point you >> sort of have to know what you're looking for. >> >> Sort of analogous to an untracked file in a VCS system. > > Isn't debsums already filling that nieche? > > MfG > Goswin > I find debsums to be too basic for my needs. apt-diff is my attempt to improve upon it. I often want to answer the question "how does package X on my machine compare to a pristine installation?" debsums only gives part of the answer. It can't check files that are missing md5sums (which includes a lot of conffiles) and when it finds a discrepancy I still need to manually download the package and diff the files myself. apt-diff solves both problems. It also allows for checking by path instead of by package, which is useful when I want to check a whole directory tree that contains files from many packages. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktinrj=ysveqkzsgkpx1sfpxoeh1+jdrkc286g...@mail.gmail.com
Re: apt-diff: a tool to diff filesystem content against APT
On Tue, Dec 7, 2010 at 11:05 PM, Lucas Nussbaum wrote: > On 07/12/10 at 17:45 -0800, Tristan Schmelcher wrote: >> (I'm not subscribed to this list, please CC me on replies.) >> >> I have recently written a tool I call "apt-diff" to compare filesystem >> content against APT. It is intended for investigating problems where >> packaged files get modified/deleted after installing them from APT >> (e.g., by user customization, accidental deletion, etc.). I originally >> wrote it after clobbering the packaged ALSA installation on my >> computer with "make install" and needing a good way to detect which >> packaged files had been modified so that I could restore them. I also >> find it useful for figuring out what customizations I have made to my >> system after I have forgotten about them. >> >> Conceptually apt-diff is meant to act like "svn diff", "git diff", >> etc., except that instead of comparing to a source code repository it >> compares to the APT repositories. I've designed it for processing many >> files/packages in batch, so diff'ing the entire filesystem is doable. >> It's sort of like an APT-aware version of debsums. The code is at >> https://github.com/TristanSchmelcher/apt-diff >> >> My hope is for this to become part of the APT stack in Debian and I'm >> posting here to see if there is any interest in that from the Debian >> developer community. I think it might be helpful for it to be invoked >> from reportbug to diagnose/triage issues where files or configurations >> have been changed. Plus I think it's just a nice tool to have >> available for power users. > > Hi, > > How does it compare to the cruft package? > > - Lucas > Different problem domains. cruft compares the set of file paths on disk with the set of packaged file paths (and other expected files), whereas apt-diff compares the _content_ of packaged files. (It can list unexpected files if run with --no-ignore-extras, but that's not its intended purpose.) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlkti=9z=9fagrqspufuptanrevrto=j4=ttjyxo...@mail.gmail.com
Re: apt-diff: a tool to diff filesystem content against APT
On Thu, Dec 9, 2010 at 11:12 PM, Goswin von Brederlow wrote: > Osamu Aoki writes: > >> On Fri, Dec 10, 2010 at 12:06:45AM +0900, Osamu Aoki wrote: >>> Hi, >>> >>> On Wed, Dec 08, 2010 at 12:52:28PM -0800, Tristan Schmelcher wrote: >>> > On Tue, Dec 7, 2010 at 11:30 PM, Goswin von Brederlow >>> > wrote: >>> > > Tristan Schmelcher writes: >>> > >>> how does it deal with configurations generated in postinstall? >>> > I find debsums to be too basic for my needs. apt-diff is my attempt to >>> > improve upon it. I often want to answer the question "how does package >>> > X on my machine compare to a pristine installation?" debsums only >>> > gives part of the answer. It can't check files that are missing >>> > md5sums (which includes a lot of conffiles) >> ~ >>> Wrong. >> >> OOps.. I now see your point. Sorry. >> >>> If you configure debsum correctly following manpage, md5sums are >>> available for all packages. Use -g option to initialize it and make >>> sure to set the debconf boolean value debsums/apt-autogen to be "true". >>> I understand that this is an exra hussle. >> >> This is talking about files missing md5sums due to packaging practice of >> the maintainer. conffiles are not debsums interest. > > Does debsums support ucf? Can it support ucf? > > What I mean is that often generated conffiles are installed with ucf and > ucf keeps track of the original generated checksum and the possibly > localy modified file and such. Does/could debsums tap into that info to > find generated conffiles that were changed? It would be pretty easy to tap into--the md5sums are all in /var/lib/ucf/hashfile and it's already in the format used by md5sum -c. But that still leaves a lot to be desired. On my machine there are only 26 entries in that file, whereas there are 1663 conffiles without md5sums.* As far as I know there's no metadata available to check the integrity of those files--they have to be compared to the original files in the .deb as apt-diff does. *As determined by this script: for file in /var/lib/dpkg/info/*.conffiles; do for line in $(cat $file); do line=$(sed -r 's/^\///' <<<$line) md5sums=$(sed -r 's/conffiles/md5sums/' <<<$file) if ! grep -qF $line $md5sums; then echo $line fi done done | wc > >> By the way, etckeeper is the way to track /etc history to me. Some >> conffile are generated by postinst. So not all files in /etc are in >> package as file. >> >>> Making this easy is step in right direction as long as it is written in >>> effective way. (In order to improve debsum, we should know it well. >>> Debsum tries to avoid downloading package as much as possible. >> >> Osamu) > > MfG > Goswin > -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktikewzhqu8efobe1cfpp-+iga-okukqh_zwp3...@mail.gmail.com
Re: Re: apt-diff: a tool to diff filesystem content against APT
> The *.conffiles files in /var/lib/dpkg/info have the checksums of all
Don't seem to ...
$ cat /var/lib/dpkg/info/*.conffiles | head
/etc/a2ps.cfg
/etc/a2ps-site.cfg
/etc/emacs/site-start.d/50a2ps.el
/etc/bash_completion.d/ack-grep
/etc/acpi/events/powerbtn
/etc/acpi/powerbtn.sh
/etc/init/acpid.conf
/etc/default/acpid
/etc/acpi/events/ac
/etc/acpi/events/asus-brightness-down
No md5sums there.
> See also dpkg-query -f '${Conffiles}' -W .
Aha! It gets them from /var/lib/dpkg/status. Should be easy to tap into.
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/1292092333.14733.6.ca...@tinygod3
Re: Re: apt-diff: a tool to diff filesystem content against APT
>> See also dpkg-query -f '${Conffiles}' -W .
>
> Aha! It gets them from /var/lib/dpkg/status. Should be easy to tap into.
>
>
I've changed apt-diff to make use of this, and it has improved speed
by 25% when diff'ing my entire filesystem! Takes only 6 minutes now.
:)
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/aanlktim7z3sbcrz7eggf+nfcyczu=0z_614ppzbjd...@mail.gmail.com
Re: Re: apt-diff: a tool to diff filesystem content against APT
On Thu, Dec 16, 2010 at 3:00 PM, Tristan Schmelcher
wrote:
>>> See also dpkg-query -f '${Conffiles}' -W .
>>
>> Aha! It gets them from /var/lib/dpkg/status. Should be easy to tap into.
>>
>>
>
> I've changed apt-diff to make use of this, and it has improved speed
> by 25% when diff'ing my entire filesystem! Takes only 6 minutes now.
> :)
>
Oops, I mean 4 minutes.
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/aanlktimqhqjqyqdqsqzohrzhgx2i4yaqiekwi20yi...@mail.gmail.com
Bug#773402: ITP: libgaminggear -- Provides functionality for gaming input devices
Package: wnpp Severity: wishlist Owner: Tristan Schmelcher * Package name: libgaminggear Version : 0.5.0 Upstream Author : Stefan Achatz * URL : https://sourceforge.net/projects/libgaminggear/ * License : GPL2 Programming Lang: C Description : Provides functionality for gaming input devices libgaminggear is a support library for userspace device input drivers to facilitate implementation of advanced features commonly found in gaming peripherals, such as macros. This library needs to be packaged in order to package roccat-tools. I am planning to package and maintain both of them (I will need a sponsor). -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141217234656.20481.6611.reportbug@kat-Dell-System-XPS-L322X
