I've done further testing and revised my patch. I've tested debpool by
uploading debian specific packages (those that don't have orig.tar.gz
tarballs), uploading package builds with the orig tarball, uploading
package builds without the orig tarball, binary-only builds,
binary-only builds without binary-all packages, and source-only
builds. I've even tested uploading udebs. I've also made sure that
messages were logged when appropriate. All packages were uploaded and
placed in their proper directories and the Packages/Sources files were
generated correctly. A major change from the last patch is that
package file for binary-all packages will be generated and used to
generate the Packages list for all archs, not just the arch used to
build the binary-all package.
This patch will also contain some fixes for udeb support and for
keeping the pool tree minimized.
diff -urN debpool_released/bin/debpool debpool/bin/debpool
--- debpool_released/bin/debpool 2007-12-03 14:10:49.000000000 -0500
+++ debpool/bin/debpool 2007-12-05 23:51:32.000000000 -0500
@@ -206,11 +206,14 @@
}
}
- my($dsc_data) = Parse_DSC("$Options{'incoming_dir'}/$dscfile");
- if ($with_source && !defined($dsc_data)) {
- Log_Message("Failure parsing dsc file '$dscfile': " .
+ my($dsc_data);
+ if ($with_source) {
+ $dsc_data = Parse_DSC("$Options{'incoming_dir'}/$dscfile");
+ if ($with_source && !defined($dsc_data)) {
+ Log_Message("Failure parsing dsc file '$dscfile': " .
$DebPool::Packages::Error, LOG_GENERAL, LOG_ERROR);
- next;
+ next;
+ }
}
my($package) = $changes_data->{'Source'};
diff -urN debpool_released/debian/changelog debpool/debian/changelog
--- debpool_released/debian/changelog 2007-12-03 14:10:48.000000000 -0500
+++ debpool/debian/changelog 2007-12-06 00:26:48.000000000 -0500
@@ -1,3 +1,23 @@
+debpool (0.2.4) experimental; urgency=low
+
+ [ Andres Mejia ]
+ * Added udeb support. Thanks Free Ekanayaka. Closes: #245526
+ * Fixed problem with packages with multiple '+' characters. Fixes part of bug
+ 322593.
+ * Added instruction on optional use of GnuPG. Closes: #454100
+ * Added Homepage and Vcs entries in control file. Closes: #453905
+ * Changed debpool recommends leaving out libdigest-md5-perl. Closes: #454102
+ * Fixed problem with missing newlines in .package files.
+ Thanks Timo Weingärtner. Closes: #400402
+ * Fixed problem with use of Vcs entries. Closes: 454173
+ * Fixed problem with binary only uploads. Closes: 322593
+ + Also fixed problem with source only uploads and binary only uploads
+ without binary-all packages.
+ * Added fix to keep pool tree minimized. Thanks Magnus Holmgren.
+ + Closes: 350503
+
+ -- Andres Mejia <[EMAIL PROTECTED]> Thu, 06 Dec 2007 00:26:38 -0500
+
debpool (0.2.3) experimental; urgency=low
* Handle sections with no subsection more usefully in
diff -urN debpool_released/debian/control debpool/debian/control
--- debpool_released/debian/control 2007-12-03 14:10:48.000000000 -0500
+++ debpool/debian/control 2007-12-03 02:46:21.000000000 -0500
@@ -4,11 +4,14 @@
Maintainer: Joel Aelwyn <[EMAIL PROTECTED]>
Standards-Version: 3.7.2.0
Build-Depends: debhelper (>= 4)
+Vcs-SVN: svn://svn.debian.org/svn/debpool
+Vcs-Browser: http://svn.debian.org/wsvn/debpool/?op=log
+Homepage: http://alioth.debian.org/projects/debpool
Package: debpool
Architecture: all
Depends: ${perl:Depends}
-Recommends: gnupg, libdigest-md5-perl, libdigest-sha1-perl, libproc-daemon-perl
+Recommends: gnupg, libdigest-sha1-perl, libproc-daemon-perl
Description: pool-based Debian package archiver
DebPool is a package archive maintenance utility designed with a goal of
removing any dependency on code not shipped as part of the core Debian
@@ -17,7 +20,7 @@
It is capable of all of the following:
* Tracking multiple distributions (however, it does *not* include
unstable -> testing promotion scripts).
- * Generating Release files (requires libdigest-{md5,sha1}-perl)
+ * Generating Release files (requires libdigest-sha1-perl)
* Verifying package signatures (requires gnupg).
* Signing release files (requires Release files and gnupg).
* Running in single-pass or daemon modes.
diff -urN debpool_released/debian/README.GnuPG debpool/debian/README.GnuPG
--- debpool_released/debian/README.GnuPG 2007-12-03 14:10:48.000000000 -0500
+++ debpool/debian/README.GnuPG 2007-12-03 02:43:34.000000000 -0500
@@ -68,3 +68,33 @@
$Options{'gpg_sign_key'} to the key ID of your archive signing key.
Note that this won't have any effect unless you also enable Release
file generation (but it won't hurt anything, either).
+
+Optional setup of GnuPG under debpool directory.
+
+1. Make sure that the gnupg package is installed.
+
+2. Make a directory under debpool. (Let's assume you're naming the directory
+ "s3kr1t" like from the mini-dinstall package.)
+
+ sudo mkdir /var/cache/debpool/.s3kr1t
+
+3. Ensure that the newly created directory is under mode 0700.
+
+ sudo chmod 0700 /var/cache/debpool/.s3kr1t
+
+4. Export the GNUPGHOME variable used by gpg to point to the newly created
+ directory. This will make gpg use the directory specified for GNUPGHOME
+ instead of the default ~/.gnupg directory.
+
+ export GNUPGHOME=/var/cache/debpool/.s3kr1t
+
+5. Generate an archive signing key by using 'sudo gpg --gen-key', record the
+ passphrase in some text file (for example, under $GNUPGHOME/passphrase), and
+ make sure it's in mode 0600.
+
+6. Edit the appropriate Config.pm file (/etc/debpool/Config.pm or
+ ~/.debpool/Config.pm), set $Options{'sign_release'} to 1 and
+ $Options{'gpg_sign_key'} to the key ID of your archive signing key.
+ Note that this won't have any effect unless you also enable Release
+ file generation (but it won't hurt anything, either). Also, make sure to set
+ $Options{'gpg_home'} to point to the newly created directory.
diff -urN debpool_released/share/DebPool/Config.pm debpool/share/DebPool/Config.pm
--- debpool_released/share/DebPool/Config.pm 2007-12-03 14:10:49.000000000 -0500
+++ debpool/share/DebPool/Config.pm 2007-12-05 23:48:27.000000000 -0500
@@ -590,7 +590,7 @@
=cut
-$Options{'sections'} = [ 'main', 'contrib', 'non-free' ];
+$Options{'sections'} = [ 'main', 'contrib', 'non-free', 'debian-installer' ];
$OptionDefs{'sections'} = 'sections=s@';
=item B<archs> => I<array of architecture names>
diff -urN debpool_released/share/DebPool/DB.pm debpool/share/DebPool/DB.pm
--- debpool_released/share/DebPool/DB.pm 2007-12-03 14:10:49.000000000 -0500
+++ debpool/share/DebPool/DB.pm 2007-12-05 23:48:38.000000000 -0500
@@ -236,7 +236,7 @@
foreach $hashref (@files) {
my($filename) = $hashref->{'Filename'};
- if ($filename =~ m/^([^_]+)_([^_]+)_.+\.deb/) {
+ if ($filename =~ m/^([^_]+)_([^_]+)_.+\.u?deb/) {
my($package) = $1;
my($version) = $2;
diff -urN debpool_released/share/DebPool/Dirs.pm debpool/share/DebPool/Dirs.pm
--- debpool_released/share/DebPool/Dirs.pm 2007-12-03 14:10:49.000000000 -0500
+++ debpool/share/DebPool/Dirs.pm 2007-12-05 23:49:02.000000000 -0500
@@ -60,6 +60,7 @@
@EXPORT_OK = qw(
&Archfile
&Create_Tree
+ &Tree_Mkdir
&Monitor_Incoming
&PoolBasePath
&PoolDir
@@ -69,7 +70,7 @@
);
%EXPORT_TAGS = (
- 'functions' => [qw(&Archfile &Create_Tree &Monitor_Incoming
+ 'functions' => [qw(&Archfile &Create_Tree &Tree_Mkdir &Monitor_Incoming
&PoolBasePath &PoolDir &Scan_Changes &Scan_All
&Strip_Subsection)],
'vars' => [qw()],
@@ -198,19 +199,10 @@
my($section);
foreach $section (@{$Options{'sections'}}) {
+ next if $section =~ m/\s*\/debian-installer/;
if (!Tree_Mkdir("$pool_dir/$section", $pool_dir_mode)) {
return 0;
}
-
- my($letter);
- foreach $letter ('a' .. 'z') {
- if (!Tree_Mkdir("$pool_dir/$section/$letter", $pool_dir_mode)) {
- return 0;
- }
- if (!Tree_Mkdir("$pool_dir/$section/lib$letter", $pool_dir_mode)) {
- return 0;
- }
- }
}
return 1;
diff -urN debpool_released/share/DebPool/Packages.pm debpool/share/DebPool/Packages.pm
--- debpool_released/share/DebPool/Packages.pm 2007-12-03 14:10:49.000000000 -0500
+++ debpool/share/DebPool/Packages.pm 2007-12-06 00:20:39.000000000 -0500
@@ -340,7 +340,7 @@
for $count (0..$#changes) {
if ($found) {
- if ($changes[$count] =~ m/^\s*$/) { # Blank line
+ if ($changes[$count] =~ m/^(\s*$|\S)/) { # End of Files entry
$found = 0; # No longer in Files
} elsif ($changes[$count] =~ m/\s*([[:xdigit:]]+)\s+(\d+)\s+(\S+)\s+(\S+)\s+(\S+)/) {
my($md5, $size, $sec, $pri, $file) = ($1, $2, $3, $4, $5);
@@ -488,7 +488,7 @@
for $count (0..$#dsc) {
if ($found) {
- if ($dsc[$count] =~ m/^\s*$/) { # Blank line
+ if ($dsc[$count] =~ m/^(\s*$|\S)/) { # End of Files entry
$found = 0; # No longer in Files
} elsif ($dsc[$count] =~ m/\s*([[:xdigit:]]+)\s+(\d+)\s+(\S+)/) {
my($md5, $size, $file) = ($1, $2, $3);
@@ -567,14 +567,31 @@
($Options{'pool_dir'}, PoolDir($source, $section), $source));
my($version) = Get_Version($distribution, $source, 'meta');
my($target) = "$pool/${source}_" . Strip_Epoch($version);
- $target .= '.package';
+ $target .= "_$arch\.package";
+ my($target_all) = "$pool/${source}_" . Strip_Epoch($version);
+ $target_all .= "_all\.package";
+
+ # Check for any binary-arch packages
+ if (-e $target) {
+ if (!open(PKG_ARCH, '<', "$target")) {
+ my($msg) = "Skipping package entry for all packages from ";
+ $msg .= "${source}: couldn't open '$target' for reading: $!";
- if (!open(PKG, '<', "$target")) {
- my($msg) = "Skipping package entry for all packages from ";
- $msg .= "${source}: couldn't open '$target' for reading: $!";
+ Log_Message($msg, LOG_GENERAL, LOG_ERROR);
+ next;
+ }
+ }
+
+ # Check for any binary-all packages
+ if (-e $target_all) {
+ if (!open(PKG_ALL, '<', "$target_all")) {
+ my($msg) = "Skipping package entry for all packages ";
+ $msg .= "from ${source}: couldn't open '$target_all' for";
+ $msg .= " reading: $!";
- Log_Message($msg, LOG_GENERAL, LOG_ERROR);
- next;
+ Log_Message($msg, LOG_GENERAL, LOG_ERROR);
+ next;
+ }
}
# Playing around with the record separator ($/) to make this
@@ -583,15 +600,26 @@
my($backup_RS) = $/;
$/ = "";
- my(@entries) = <PKG>;
- close(PKG);
+ my(@arch_entries);
+ if (-e $target) { # Write entries from arch packages
+ @arch_entries = <PKG_ARCH>;
+ close(PKG_ARCH);
+ }
+
+ my(@all_entries);
+ if (-e $target_all) { # Write entries from all packages
+ @all_entries = <PKG_ALL>;
+ close(PKG_ALL);
+ }
$/ = $backup_RS;
# Pare it down to the relevant entries, and print those out.
- @entries = grep(/\nArchitecture: ($arch|all)\n/, @entries);
- print $tmpfile_handle @entries;
+ @arch_entries = grep(/\nArchitecture: ($arch)\n/, @arch_entries);
+ @all_entries = grep(/\nArchitecture: all\n/, @all_entries);
+ print $tmpfile_handle @arch_entries;
+ print $tmpfile_handle @all_entries;
}
}
@@ -625,22 +653,16 @@
my($pkg_ver) = $chg_hashref->{'Version'};
my($guess_section) = Guess_Section($chg_hashref);
- my($pkg_dir) = join('/',
- ($pool_dir, PoolDir($pkg_name, $guess_section), $pkg_name));
+ my($pkg_pool_subdir) = join('/',
+ ($pool_dir, PoolDir($pkg_name, $guess_section)));
+ my($pkg_dir) = join('/', ($pkg_pool_subdir, $pkg_name));
- # Make sure the package directory exists (and is a directory!)
+ # Create the directory or error out
- if (! -e $pkg_dir) {
- if (!mkdir($pkg_dir)) {
- $Error = "Failed to mkdir '$pkg_dir': $!";
- return 0;
- }
- if (!chmod($Options{'pool_dir_mode'}, $pkg_dir)) {
- $Error = "Failed to chmod '$pkg_dir': $!";
- return 0;
- }
- } elsif (! -d $pkg_dir) {
- $Error = "Target '$pkg_dir' is not a directory.";
+ if (!Tree_Mkdir($pkg_pool_subdir, $Options{'pool_dir_mode'})) {
+ return 0;
+ }
+ if (!Tree_Mkdir($pkg_dir, $Options{'pool_dir_mode'})) {
return 0;
}
@@ -661,19 +683,25 @@
# Generate and install .package and .source metadata files.
- my($pkg_file) = Generate_Package($chg_hashref);
+ my(@pkg_archs) = @{$chg_hashref->{'Architecture'}};
+ @pkg_archs = grep(!/source/, @pkg_archs); # Source is on it's own.
- if (!defined($pkg_file)) {
- $Error = "Failed to generate .package file: $Error";
- return undef;
- }
+ my($target);
+ foreach my $pkg_arch (@pkg_archs) {
+ my($pkg_file) = Generate_Package($chg_hashref, $pkg_arch);
- my($target) = "$pkg_dir/${pkg_name}_" . Strip_Epoch($pkg_ver) . '.package';
+ if (!defined($pkg_file)) {
+ $Error = "Failed to generate .package file: $Error";
+ return undef;
+ }
- if (!Move_File($pkg_file, $target, $Options{'pool_file_mode'})) {
- $Error = "Failed to move '$pkg_file' to '$target': ";
- $Error .= $DebPool::Util::Error;
- return 0;
+ $target = "$pkg_dir/${pkg_name}_" . Strip_Epoch($pkg_ver) . "_$pkg_arch" . '.package';
+
+ if (!Move_File($pkg_file, $target, $Options{'pool_file_mode'})) {
+ $Error = "Failed to move '$pkg_file' to '$target': ";
+ $Error .= $DebPool::Util::Error;
+ return 0;
+ }
}
if (defined($dsc) && defined($dsc_hashref)) {
@@ -719,6 +747,9 @@
$chg_hashref->{'Files'});
$ComponentDB{$distribution}->{$pkg_name} = $component;
}
+ if ( $section eq 'debian-installer' ) {
+ $component .= '/debian-installer';
+ }
return 1;
}
@@ -820,6 +851,21 @@
my($package, $changes_hashref) = @_;
+ # Checking for version of package being installed
+ my($changes_version) = $changes_hashref->{'Version'};
+ # Checking for binary only upload
+ my($with_source) = undef;
+ # Checking for binary-all packages in binary only upload
+ my($with_indep) = undef;
+ for my $temp (@{$changes_hashref->{'Architecture'}}) {
+ if ('source' eq $temp) {
+ $with_source = 1;
+ }
+ if ('all' eq $temp) {
+ $with_indep = 1;
+ }
+ }
+
my($installed_dir) = $Options{'installed_dir'};
my($pool_dir) = $Options{'pool_dir'};
@@ -872,7 +918,7 @@
$bin_package = $1;
$version = $2;
$deb = 1;
- } elsif ($file =~ m/^([^_]+)_([^_]+)\.package$/) { # package metadata
+ } elsif ($file =~ m/^([^_]+)_([^_]+)_.+\.package$/) { # package metadata
$bin_package = $1;
$version = $2;
} elsif ($file =~ m/^([^_]+)_([^_]+)\.source$/) { # source metadata
@@ -884,8 +930,20 @@
next;
}
- # Skip it if we recognize it as a valid version.
+ # Skip files if we recognize it as a valid version.
+ # Skipping dsc, diff.gz, and orig tarball files if doing a binary only
+ # upload
+ if (!$with_source) {
+ $src = 0;
+ # Skip binary-all packages in a binary only upload without
+ # binary-all packages as long as they're of the same changes
+ # version
+ if ((!$with_indep) &&
+ ($file =~ m/\Q_${changes_version}_all.\Eu?deb/)) {
+ $deb = 0;
+ }
+ }
my($matched) = 0;
my($dist);
foreach $dist (@{$Options{'realdists'}}) {
@@ -956,7 +1014,7 @@
use DebPool::Dirs qw(:functions);
use DebPool::Logging qw(:functions :facility :level);
- my($changes_data) = @_;
+ my($changes_data, $arch) = @_;
my($source) = $changes_data->{'Source'};
my(@files) = @{$changes_data->{'Files'}};
my($pool_base) = PoolBasePath();
@@ -966,96 +1024,89 @@
my($tmpfile_handle, $tmpfile_name) = tempfile();
my(@packages) = @{$changes_data->{'Binary'}};
- my(@architectures) = @{$changes_data->{'Architecture'}};
- @architectures = grep(!/source/, @architectures); # Source is on it's own.
- my($package, $arch);
+ my($package);
foreach $package (@packages) {
- foreach $arch (@architectures) {
- # Construct a pattern to match the filename and nothing else.
- # This used to be an exact match using the source version, but
- # Debian's standards are sort of insane, and the version number
- # on binary files is not always the same as that on the source
- # file (nor is it even something simple like "source version
- # without the epoch" -- it is more or less arbitrary, as long
- # as it is a well-formed version number).
-
- my($filepat) = "${package}_.*_${arch}\\.deb";
- $filepat =~ s/\+/\\\+/;
+ # Construct a pattern to match the filename and nothing else.
+ # This used to be an exact match using the source version, but
+ # Debian's standards are sort of insane, and the version number
+ # on binary files is not always the same as that on the source
+ # file (nor is it even something simple like "source version
+ # without the epoch" -- it is more or less arbitrary, as long
+ # as it is a well-formed version number).
+ my($filepat) = qr/^\Q${package}_\E.*\Q_${arch}.\Eu?deb/;
+ my($section) = Guess_Section($changes_data);
+ my($pool) = join('/', (PoolDir($source, $section), $source));
- my($section) = Guess_Section($changes_data);
- my($pool) = join('/', (PoolDir($source, $section), $source));
-
- my($marker) = -1;
- my($count) = 0;
+ my($marker) = -1;
+ my($count) = 0;
- # Step through each file, match against filename. Save matches
- # for later use.
+ # Step through each file, match against filename. Save matches
+ # for later use.
- for $count (0..$#files) {
- if ($files[$count]->{'Filename'} =~ m/$filepat/) {
- $marker = $count;
- }
+ for $count (0..$#files) {
+ if ($files[$count]->{'Filename'} =~ m/$filepat/) {
+ $marker = $count;
}
+ }
- # The changes file has a stupid quirk; it puts all binaries from
- # a package in the Binary: line, even if they weren't built (for
- # example, an Arch: all doc package when doing an arch-only build
- # for a port). So if we didn't find a .deb file for it, assume
- # that it's one of those, and skip, rather than choking on it.
+ # The changes file has a stupid quirk; it puts all binaries from
+ # a package in the Binary: line, even if they weren't built (for
+ # example, an Arch: all doc package when doing an arch-only build
+ # for a port). So if we didn't find a .deb file for it, assume
+ # that it's one of those, and skip, rather than choking on it.
- next if (-1 == $marker);
+ next if (-1 == $marker);
- # Run Dpkg_Info to grab the dpkg --info data on the package.
+ # Run Dpkg_Info to grab the dpkg --info data on the package.
- my($file) = $files[$marker]->{'Filename'};
- my($info) = Dpkg_Info("$Options{'pool_dir'}/$pool/$file");
-
- # Dump all of our data into the metadata tempfile.
-
- print $tmpfile_handle "Package: $package\n";
+ my($file) = $files[$marker]->{'Filename'};
+ my($info) = Dpkg_Info("$Options{'pool_dir'}/$pool/$file");
- if (defined($info->{'Priority'})) {
- print $tmpfile_handle "Priority: $info->{'Priority'}\n";
- }
+ # Dump all of our data into the metadata tempfile.
- if (defined($info->{'Section'})) {
- print $tmpfile_handle "Section: $info->{'Section'}\n";
- }
+ print $tmpfile_handle "Package: $package\n";
- if (defined($info->{'Essential'})) {
- print $tmpfile_handle "Essential: $info->{'Essential'}\n";
- }
+ if (defined($info->{'Priority'})) {
+ print $tmpfile_handle "Priority: $info->{'Priority'}\n";
+ }
+
+ if (defined($info->{'Section'})) {
+ print $tmpfile_handle "Section: $info->{'Section'}\n";
+ }
- print $tmpfile_handle "Installed-Size: $info->{'Installed-Size'}\n";
+ if (defined($info->{'Essential'})) {
+ print $tmpfile_handle "Essential: $info->{'Essential'}\n";
+ }
- print $tmpfile_handle "Maintainer: $changes_data->{'Maintainer'}\n";
- print $tmpfile_handle "Architecture: $arch\n";
- print $tmpfile_handle "Source: $source\n";
- print $tmpfile_handle "Version: $changes_data->{'Version'}\n";
-
- # All of the inter-package relationships go together, and any
- # one of them can potentially be empty (and omitted).
+ print $tmpfile_handle "Installed-Size: $info->{'Installed-Size'}\n";
- my($field);
- foreach $field (@Relationship_Fields) {
- if (defined($info->{$field})) {
- print $tmpfile_handle "${field}: $info->{$field}\n";
- }
+ print $tmpfile_handle "Maintainer: $changes_data->{'Maintainer'}\n";
+ print $tmpfile_handle "Architecture: $arch\n";
+ print $tmpfile_handle "Source: $source\n";
+ print $tmpfile_handle "Version: $changes_data->{'Version'}\n";
+
+ # All of the inter-package relationships go together, and any
+ # one of them can potentially be empty (and omitted).
+
+ my($field);
+ foreach $field (@Relationship_Fields) {
+ if (defined($info->{$field})) {
+ print $tmpfile_handle "${field}: $info->{$field}\n";
}
-
- # And now, some stuff we can grab out of the parsed changes
- # data far more easily than anywhere else.
-
- print $tmpfile_handle "Filename: $pool_base/$pool/$file\n";
-
- print $tmpfile_handle "Size: $files[$marker]->{'Size'}\n";
- print $tmpfile_handle "MD5sum: $files[$marker]->{'MD5Sum'}\n";
-
- print $tmpfile_handle "Description: $info->{'Description'}";
}
+ # And now, some stuff we can grab out of the parsed changes
+ # data far more easily than anywhere else.
+
+ print $tmpfile_handle "Filename: $pool_base/$pool/$file\n";
+
+ print $tmpfile_handle "Size: $files[$marker]->{'Size'}\n";
+ print $tmpfile_handle "MD5sum: $files[$marker]->{'MD5Sum'}\n";
+
+ print $tmpfile_handle "Description: $info->{'Description'}";
+
print $tmpfile_handle "\n";
}
@@ -1117,7 +1168,8 @@
print $tmpfile_handle 'Architecture: ';
print $tmpfile_handle join(' ', @{$dsc_data->{'Architecture'}}) . "\n";
- print $tmpfile_handle "Standards-Version: $dsc_data->{'Standards-Version'}\n";
+ print $tmpfile_handle "Standards-Version: $dsc_data->{'Standards-Version'}\n"
+ if exists $dsc_data->{'Standards-Version'};
print $tmpfile_handle "Format: $dsc_data->{'Format'}\n";
print $tmpfile_handle "Directory: " . join('/',
(PoolBasePath(), PoolDir($source, $section), $source)) . "\n";
