Package: dpkg Version: 1.15.7.2 Severity: normal Hello,
first of all, I'm fully aware of [1] and [2]. However, solution implemented in #578635 is not good enough and slows down dpkg considerably on ext4 and even twice as much on btrfs [3]. So it is pretty clear that dpkg is not ready for the age of modern file systems. I personally find performance after #578635 almost acceptable on my main / where I definitely don't want it to produce zero-length files but dpkg has other use cases where such a slow down is not acceptable for the benefit it brings. For example, sometimes zero-length files after unclean shutdown are not critical but barely minor problem. One of them is chroots, esp. buildd ones. sbuild and pbuilder install the same packages over and over again in the (copies of) the chroot and slow dpkg performance takes much developer time even on new fast machines. However, in case of power failure, it is not important what happens to that (copy of) the chroot and how many zero-length files dpkg is going to leave it with. The (copy of) the chroot will either be `rm -rf` or fixed manually anyway. So huge dpkg slowdown is for no benefit at all in this case, it only wastes developer time and considerably prolongs package build time. What is more, frequent sync()s (#578635), obviously, affect ext3 and other file systems as well esp. when there is much other parallel disk I/O going on at the same time. Therefore, I would like to be able to effectively disable fsync()/sync() at runtime via dpkg command line switch and alternatively, via entry in /etc/dpkg/dpkg.conf (yes, this includes fsync() calls on database updates, as they are the major source of slow down as well). Of course, it would be even better if dpkg itself were fixed to be crash-tolerant and fast on the file systems with delayed allocation, but it seems that it is not going to be easy to implement. [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578635 [2] https://bugzilla.kernel.org/show_bug.cgi?id=15910 [3] nosync.c: void sync() { } int fsync(int fildes) { return 0; } $ gcc -shared -fPIC -O2 -g nosync.c -o libnosync.so -------------------------------------------------------------------- On the BTRFS file system: # unset LD_PRELOAD # time apt-get -y build-dep kde4libs The following NEW packages will be installed: adduser automoc bison bsdmainutils ca-certificates cdbs cmake cmake-data comerr-dev debhelper emacsen-common file flex fontconfig fontconfig-config gettext gettext-base groff-base hspell html2text intltool-debian krb5-multidev libacl1-dev libasound2 libasound2-dev libaspell-dev libaspell15 libasyncns0 libattica-dev libattica0 libattr1-dev libaudio2 libavahi-client3 libavahi-common-data libavahi-common-dev libavahi-common3 libbz2-dev libcap2 libclucene0ldbl libcroco3 libcups2 libcups2-dev libcurl3-gnutls libdbus-1-3 libdbus-1-dev libdbus-glib-1-2 libdrm2 libeggdbus-1-0 libenchant-dev libenchant1c2a libexiv2-6 libexpat1 libfam-dev libfam0 libflac8 libfontconfig1 libfreetype6 libgcrypt11 libgcrypt11-dev libgif-dev libgif4 libgl1-mesa-dev libgl1-mesa-glx libglib2.0-0 libglib2.0-dev libglu1-mesa libglu1-mesa-dev libgnutls-dev libgnutls26 libgpg-error-dev libgpg-error0 libgssapi-krb5-2 libgssrpc4 libhunspell-1.2-0 libice-dev libice6 libidn11 libilmbase-dev libilmbase6 libiodbc2 libjasper-dev libjasper1 libjpeg62 libjpeg62-dev libk5crypto3 libkadm5clnt-mit7 libkadm5srv-mit7 libkdb5-4 libkeyutils-dev libkeyutils1 libkrb5-3 libkrb5-dev libkrb5support0 liblcms1 libldap-2.4-2 liblzma-dev libmagic1 libmng1 libncursesw5 libogg0 libopenexr-dev libopenexr6 libpcre3 libpcre3-dev libpcrecpp0 libphonon-dev libphonon4 libpng12-0 libpng12-dev libpolkit-agent-1-0 libpolkit-gobject-1-0 libpolkit-qt-1-0 libpolkit-qt-1-dev libpopt0 libpthread-stubs0 libpthread-stubs0-dev libpulse-mainloop-glib0 libpulse0 libqca2 libqca2-dev libqt4-assistant libqt4-dbus libqt4-designer libqt4-dev libqt4-help libqt4-multimedia libqt4-network libqt4-opengl libqt4-opengl-dev libqt4-qt3support libqt4-script libqt4-scripttools libqt4-sql libqt4-svg libqt4-test libqt4-webkit libqt4-xml libqt4-xmlpatterns libqtcore4 libqtgui4 libraptor1 librasqal2 librdf0 libsasl2-2 libsm-dev libsm6 libsndfile1 libsoprano-dev libsoprano4 libsqlite3-0 libssl-dev libssl0.9.8 libstreamanalyzer-dev libstreamanalyzer0 libstreams-dev libstreams0 libtasn1-3 libtasn1-3-dev libtiff4 libunistring0 libutempter-dev libutempter0 libvorbis0a libvorbisenc2 libwrap0 libx11-6 libx11-data libx11-dev libxau-dev libxau6 libxcb1 libxcb1-dev libxcursor-dev libxcursor1 libxdamage1 libxdmcp-dev libxdmcp6 libxext-dev libxext6 libxfixes-dev libxfixes3 libxi-dev libxi6 libxml2 libxml2-dev libxml2-utils libxrender-dev libxrender1 libxslt1-dev libxslt1.1 libxss-dev libxss1 libxt6 libxtst-dev libxtst6 libxxf86vm1 m4 man-db mesa-common-dev openssl pkg-config pkg-kde-tools po-debconf procps qt4-qmake shared-desktop-ontologies shared-mime-info soprano-daemon ttf-dejavu-core ucf x11-common x11proto-core-dev x11proto-fixes-dev x11proto-input-dev x11proto-kb-dev x11proto-record-dev x11proto-render-dev x11proto-scrnsaver-dev x11proto-xext-dev xsltproc xtrans-dev zlib1g-dev 0 upgraded, 222 newly installed, 0 to remove and 0 not upgraded. Need to get 0B/92.5MB of archives. After this operation, 273MB of additional disk space will be used. ... real 12m35.082s user 0m6.760s sys 0m14.969s # time dpkg -P <222 kde4libs build-depends> ... real 11m21.294s user 0m2.488s sys 0m9.305s # export LD_PRELOAD=libnosync.so # time apt-get -y build-dep kde4libs ... real 0m31.748s user 0m6.496s sys 0m6.576s # time dpkg -P <222 kde4libs build-depends> ... real 0m11.788s user 0m2.160s sys 0m2.732s nosync version is ~24 / 60 times faster. -------------------------------------------------------------------- On the ext4 file system: # unset LD_PRELOAD # time apt-get -y build-dep kde4libs ... real 6m38.181s user 0m7.328s sys 0m6.816s # time dpkg -P <222 kde4libs build-depends> ... real 4m49.146s user 0m2.504s sys 0m2.020s # export LD_PRELOAD=libnosync.so # time apt-get -y build-dep kde4libs ... real 0m34.390s user 0m6.888s sys 0m4.816s # time dpkg -P <222 kde4libs build-depends> ... real 0m10.010s user 0m2.072s sys 0m1.564s nosync version is ~11 / 34 times faster. -------------------------------------------------------------------- -- System Information: Debian Release: squeeze/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (101, 'experimental') Architecture: amd64 (x86_64) Kernel: Linux 2.6.35-rc4 (SMP w/4 CPU cores) Locale: LANG=lt_LT.UTF-8, LC_CTYPE=lt_LT.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages dpkg depends on: ii coreutils 8.5-1 GNU core utilities ii libbz2-1.0 1.0.5-4 high-quality block-sorting file co ii libc6 2.11.2-2 Embedded GNU C Library: Shared lib ii libselinux1 2.0.94-1 SELinux runtime shared libraries ii xz-utils 4.999.9beta+20100527-1 XZ-format compression utilities ii zlib1g 1:1.2.3.4.dfsg-3 compression library - runtime dpkg recommends no packages. Versions of packages dpkg suggests: ii apt 0.7.25.3 Advanced front-end for dpkg -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
