tags 412889 + patch severity 412889 wishlist thanks Hi,
On Wed, Feb 28, 2007 at 10:34:36AM -0800, Carl Van Arsdall wrote: > When trying to create an apt repository with fai-mirror on a 64bit > systems some statements report errors. > > When trying to make a 32bit mirror on a 64 machine I attempted to use > the following methods: Currently, the documentation does not indicate that this should be possible. I'm thus downgrading the bug to wishlist. > First I tried only including the I386 class > #fai-mirror -c I386 /srv/fai/mirror/ > > Then I attempted to exclude AMD64 > #fai-mirror -x AMD64 /srv/fai/mirror Including or excluding classes only affects the list of packages that is going to be included in the mirror. It does not have any effect on the architecture being used. > The third uses linux32 that fakes out queries to commands like uname so > that they report a 32bit architecture > > #linux32 fai-mirror /srv/fai/mirror fai-mirror is using apt-get / aptitude to download the packages which use a build-in default value for the architecture to be used. Anyway, I've stepped up to add support to specify the architecture(s) that should be included in the mirror. The attached patch adds support for the -a command line option to fai-mirror which accepts a comma-separated list of architectures. That patch is against the current trunk but should apply to older versions of fai-mirror as well. Example usage: fai-mirror -B -a amd64,i386,powerpc /srv/fai/mirror So far, I've only done fairly basic tests, so more testing and feedback would be much appreciated. Anyway, I think that the patch could be included in the post-Lenny version of FAI as this should ensure further testing and it basically includes an optional option only which should not affect the normal behavior. There is one issue though which is currently known to possibly cause problems: If the -B option is _not_ used, fai-mirror reads a list of additional packages to be included from information included in the nfsroot. That might include packages though which are not available for the specified architecture (e.g. I could not build amd64 or i386 mirrors on powerpc without using -B). This could be solved by generating the appropriate list of packages on the fly rather than reading it from $NFSROOT/var/tmp/base-pkgs.lis. This would require rather large code changes, so I did not implement it so far as I only consider this to be a very minor issue. Any feedback and comments are mostly welcome! Cheers, Sebastian -- Sebastian "tokkee" Harl +++ GnuPG-ID: 0x8501C7FC +++ http://tokkee.org/ Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety. -- Benjamin Franklin
From 4af9fa8484d4d85a701d934888ef8ada535d61f2 Mon Sep 17 00:00:00 2001
From: Sebastian Harl <[EMAIL PROTECTED]>
Date: Sat, 6 Sep 2008 19:07:28 +0200
Subject: [PATCH] fai-mirror: Added support for the -a command line option.
This option may be used to specify the architectures which should be included
in the mirror.
Signed-off-by: Sebastian Harl <[EMAIL PROTECTED]>
---
bin/fai-mirror | 111 +++++++++++++++++++++++++++++++++++++++--------------
man/fai-mirror.1 | 5 ++
2 files changed, 86 insertions(+), 30 deletions(-)
diff --git a/bin/fai-mirror b/bin/fai-mirror
index 1d14609..187d504 100755
--- a/bin/fai-mirror
+++ b/bin/fai-mirror
@@ -151,6 +151,8 @@ initialize() {
}
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
delete_base_packages() {
+ local arch=$1
+ shift
# now delete all packages that are already included in base.tgz
local p
@@ -162,14 +164,18 @@ delete_base_packages() {
fi
echo "Removing packages that are already included in base.tgz"
for p in $(cat $NFSROOT/var/tmp/base-pkgs.lis); do
- if [ -f $archivedir/${p}_*.deb ]; then
- [ $verbose -eq 1 ] && echo "deleting package $p"
- rm $archivedir/${p}_*.deb
+ # There might be more than one file - even for this architecture.
+ local files=$(ls $archivedir/${p}_{$arch,all}.deb 2> /dev/null)
+ for file in $files; do
+ if [ -f $file ]; then
+ [ $verbose -eq 1 ] && echo "deleting package $p"
+ rm $file
# else commands only for debugging
# else
# echo "package $p not found"
# ls $archivedir/${p}_*.deb
- fi
+ fi
+ done
done
}
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@@ -225,7 +231,8 @@ preserve=0
verbose=0
add=1
qflag=-qq
-while getopts "Bvhx:pc:C:" opt ; do
+archs="`dpkg-architecture -qDEB_HOST_ARCH`"
+while getopts "Bvhx:pc:C:a:" opt ; do
case "$opt" in
B) add=0 ;;
C) cfdir=$OPTARG ;;
@@ -234,6 +241,7 @@ while getopts "Bvhx:pc:C:" opt ; do
c) cclasses="$OPTARG";;
p) preserve=1;;
v) verbose=1; vflag=-v; qflag='';;
+ a) archs="$OPTARG";;
?) die 1 "Unknown option";;
esac
done
@@ -290,17 +298,26 @@ fi
# TODO: use -p to preserve sources.list
perl -p -e 's/file:/copy:/' $cfdir/apt/sources.list > $aptcache/etc/apt/sources.list
-echo "Getting package information"
-apt-get $quiet $aptoptions update >/dev/null
-
-set-classes
-echo "Downloading packages for classes:" $classes
-FAI=$FAI_CONFIGDIR install_packages -d $vflag
-[ $add -eq 1 ] && add_base_packages
-get_addpackages
-umount_dirs
-trap "" EXIT ERR
-[ $add -eq 0 ] && delete_base_packages
+archs=${archs//,/ }
+for arch in $archs; do
+ arch=$(echo $arch | tr '[[:upper:]]' '[[:lower:]]')
+ aptoptions_noarch="$aptoptions"
+ aptoptions=" -o APT::Architecture=$arch $aptoptions_noarch"
+
+ echo "Getting package information for architecture $arch"
+ apt-get $quiet $aptoptions update >/dev/null
+
+ set-classes
+ echo "Downloading packages for classes:" $classes
+ FAI=$FAI_CONFIGDIR install_packages -d $vflag
+ [ $add -eq 1 ] && add_base_packages
+ get_addpackages
+ umount_dirs
+ trap "" EXIT ERR
+ [ $add -eq 0 ] && delete_base_packages $arch
+
+ aptoptions="$aptoptions_noarch"
+done
# create mirror directory structure
echo "Calling apt-move"
@@ -315,21 +332,55 @@ CONTENTS=no
PKGCOMP='none gzip'
EOF
apt-move $quiet -c $aptmovefile update
-# since Packages.gz from apt-move does not include packages from my
-# repository, let's use apt-ftparchive for generiating correct index
-# files
-pfilegz=$(find $mirrordir/dists -name Packages.gz)
-pfile=$(find $mirrordir/dists -name Packages)
-pdist=$(cd $mirrordir/dists ; ls)
+
+# apt-move does not handle Packages files for multiple architectures,
+# so we're creating those ourselves. Also, we're creating our own
+# Release files.
+#
+# We trust in dists and components being created by apt-move.
+echo "Creating Packages and Release files"
cd $mirrordir
-# md5sums of apt-move are not valid, when we recreate Packages.gz using
-# apt-ftparchive, but we can use the header of the Release file
-grep -B99 MD5Sum: $mirrordir/dists/$pdist/Release | grep -v MD5Sum: > $mirrordir/tmpfile
-rm $mirrordir/dists/$pdist/Release
-apt-ftparchive packages pool > $pfile
-gzip -c $pfile > $pfilegz
-apt-ftparchive release dists/$pdist >> tmpfile
-mv tmpfile dists/$pdist/Release
+dists=$(cd $mirrordir/dists ; find * -maxdepth 0 -type d | xargs echo -n)
+for dist in $dists; do
+ distdir="$mirrordir/dists/$dist"
+
+ components=$(cd $distdir ; find * -maxdepth 0 -type d | xargs echo -n)
+ for component in $components; do
+ componentdir="$distdir/$component"
+
+ for arch in $archs; do
+ archdir="$componentdir/binary-$arch"
+ mkdir -p "$archdir"
+ rm -rf $archdir/Packages{,gz}
+
+ dpkg-scanpackages --arch $arch . > $archdir/Packages
+ gzip -c $archdir/Packages > $archdir/Packages.gz
+
+ cat > $archdir/Release <<EOF
+Archive: $dist
+Component: $component
+Origin: fai-mirror
+Label: fai-mirror
+Architecture: $arch
+EOF
+ done
+ done
+
+ mirror_archs=$(cd $distdir ; \
+ ls */ | grep '^binary-' | cut -d'-' -f2- | xargs echo -n)
+ cat > $distdir/Release <<EOF
+Origin: fai-mirror
+Label: fai-mirror
+Suite: $dist
+Codename: unknown
+Date: `date -Ru`
+Architectures: $mirror_archs
+Components: $components
+Description: unknown
+EOF
+
+ apt-ftparchive release dists/$dist >> $distdir/Release
+done
echo "$0 finished."
echo -n "Mirror size and location: ";du -sh $mirrordir
diff --git a/man/fai-mirror.1 b/man/fai-mirror.1
index daac1aa..4ffab54 100644
--- a/man/fai-mirror.1
+++ b/man/fai-mirror.1
@@ -71,6 +71,11 @@ Show simle help and version.
Exclude these classes. Do not add the packages of these classes to
the mirror.
.TP
+.B "\-a " ARCH[,ARCH]
+Create the mirror for the given architectures. Default is the host
+architecture. This has no influence on the list of classes. This might have to
+be combined with -B to prevent architecture specific packages to be pulled in.
+.TP
.B \-p
Preserve existing sources.list. NOT YET IMPLEMENTED.
.TP
--
1.6.0.1.216.g1b23a
signature.asc
Description: Digital signature
