Hi
This is probably overkill but I've attached a diff that show my patches
to image.sh. It's just a hack so far to make it do what I want and not
meant as general purpose. Use the changes you need for your application.
Changes (only meant to work for building usb images on amd64 and i386)
- mount devfs so pkg will work properly
- add "install packages from official repo" option so you won't have to
build your own packages for each jail
- downloaded packages from official repo is cached locally to avoid
excessive downloads
- hack for i386 so packages are installed properly
- increase swap space to allow for kernel core dumps (for usb image)
- execute post-install script "overlay.sh" in the jail if provided in
the overlay folder
Bapt: Maybe you'll find some of these changes useful? :)
/Johannes
On 6/11/19 8:52 AM, [email protected] wrote:
> I'm having the same issue with poudriere image; could you please let
> me know what you did to fix it? I'm assuming the image.sh you're
> referring to is /usr/local/share/poudriere/image.sh, but I'm not sure
> where the change would need to be made.
>
> Thanks in advance,
> Joseph
>
>
> On 2019-06-04 13:02, Johannes Lundberg wrote:
>> On 6/3/19 10:44 PM, Baptiste Daroussin wrote:
>>> On Tue, Jun 04, 2019 at 07:32:16AM +0200, O. Hartmann wrote:
>>>> Hello List,
>>>>
>>>> lately I ran into a serious problem installing packages in a nanoBSD
>>>> environment, in which the package repository server is "remotely"
>>>> on site. The
>>>> issue as documented below occurs on both 12-STABLE r348529 and
>>>> CURRENT r348600
>>>> and must have been introduced shortly, since the last known good
>>>> installation
>>>> with the environment of ours was on 21st May 2019.
>>>>
>>>> As far as I know,, the package installation is performed via
>>>> "chroot'ed"
>>>> environment and somehow /dev/null is out of a sudden not accessible
>>>> anymore
>>>> while pkg tries to delegate some output to /dev/null.
>>>>
>>>> What happened here?
>>>>
>>>> Kind regards and thanks in advance,
>>>>
>>>> oh
>>>>
>>>> [...]
>>>> All repositories are up to date.
>>>> The following 10 package(s) will be affected (of 0 checked):
>>>>
>>>> New packages to be INSTALLED:
>>>> python3: 3_3 [zeit4]
>>>> sudo: 1.8.27_1 [zeit4]
>>>> devcpu-data: 1.22 [zeit4]
>>>> python36: 3.6.8_2 [zeit4]
>>>> readline: 8.0.0 [zeit4]
>>>> indexinfo: 0.3.1 [zeit4]
>>>> libffi: 3.2.1_3 [zeit4]
>>>> gettext-runtime: 0.19.8.1_2 [zeit4]
>>>> openldap-sasl-client: 2.4.47 [zeit4]
>>>> cyrus-sasl: 2.1.27 [zeit4]
>>>>
>>>> Number of packages to be installed: 10
>>>>
>>> What is new is that pkg is using /dev/null as input when running
>>> script? this is
>>> new since pkg 1.11 . Somehow this does not seems to be avaalaible in
>>> your
>>> environement.
>>
>> Hi
>>
>> Same things applies to poudriere-image. I had to add a mount devfs
>> command to the image.sh script.
>>
>>>
>>> Best regards,
>>> Bapt
>>
>> _______________________________________________
>> [email protected] mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to
>> "[email protected]"
--- image.sh.orig 2019-06-06 08:24:36.445264000 -0700
+++ image.sh 2019-06-08 17:44:57.852022000 -0700
@@ -40,6 +40,7 @@
-n imagename -- The name of the generated image
-o outputdir -- Image destination directory
-p portstree -- Ports tree
+ -P -- Install packages from official repo
-s size -- Set the image size
-t type -- Type of image can be one of (default iso+zmfs):
-- iso, iso+mfs, iso+zmfs, usb, usb+mfs, usb+zmfs,
@@ -112,7 +113,7 @@
. ${SCRIPTPREFIX}/common.sh
HOSTNAME=poudriere-image
-while getopts "c:f:h:j:m:n:o:p:s:t:X:z:" FLAG; do
+while getopts "c:f:h:j:m:n:o:Pp:s:t:X:z:" FLAG; do
case "${FLAG}" in
c)
[ -d "${OPTARG}" ] || err 1 "No such extract directory: ${OPTARG}"
@@ -146,6 +147,9 @@
OPTARG="${SAVED_PWD}/${OPTARG}"
OUTPUTDIR=${OPTARG}
;;
+ P)
+ PKGREPO=1
+ ;;
p)
PTNAME=${OPTARG}
;;
@@ -239,7 +243,6 @@
mkdir -p ${WRKDIR}/out
[ -z "${EXCLUDELIST}" ] || cat ${EXCLUDELIST} > ${excludelist}
cat >> ${excludelist} << EOF
-usr/src
var/db/freebsd-update
var/db/etcupdate
boot/kernel.old
@@ -351,8 +354,11 @@
make -C ${mnt}/usr/src DESTDIR=${WRKDIR}/world BATCH_DELETE_OLD_FILES=yes SRCCONF=${WRKDIR}/src.conf delete-old delete-old-libs
[ ! -d "${EXTRADIR}" ] || cp -fRPp ${EXTRADIR}/ ${WRKDIR}/world/
-mv ${WRKDIR}/world/etc/login.conf.orig ${WRKDIR}/world/etc/login.conf
-cap_mkdb ${WRKDIR}/world/etc/login.conf
+if [ -e "${WRKDIR}/world/etc/login.conf.orig" ]; then
+ # No login.conf.orig on 13.0-CURRENT-i386
+ mv ${WRKDIR}/world/etc/login.conf.orig ${WRKDIR}/world/etc/login.conf
+ cap_mkdb ${WRKDIR}/world/etc/login.conf
+fi
# Set hostname
if [ -n "${HOSTNAME}" ]; then
@@ -367,12 +373,15 @@
local REPOS_DIR=$(mktemp -dt poudriere_repo)
local ABI_FILE
- # This pkg rquery is always ran in host so we need a host-centric
- # repo.conf always.
- cat > "${REPOS_DIR}/repo.conf" <<-EOF
- FreeBSD: { enabled: false }
- local: { url: file:///${WRKDIR}/world/tmp/packages }
- EOF
+ if [ -n "${PKGREPO}" ]; then
+ else
+ # This pkg rquery is always ran in host so we need a host-centric
+ # repo.conf always.
+ cat > "${REPOS_DIR}/repo.conf" <<-EOF
+ FreeBSD: { enabled: false }
+ local: { url: file:///${WRKDIR}/world/tmp/packages }
+ EOF
+ fi
export REPOS_DIR PKG_DBDIR
# Always need this from host.
@@ -387,36 +396,83 @@
# install packages if any is needed
if [ -n "${PACKAGELIST}" ]; then
mkdir -p ${WRKDIR}/world/tmp/packages
- ${NULLMOUNT} ${POUDRIERE_DATA}/packages/${MASTERNAME} ${WRKDIR}/world/tmp/packages
+ mount -t devfs devfs ${WRKDIR}/world/dev
+ chroot "${WRKDIR}/world" env /usr/sbin/pwd_mkdb -p /etc/master.passwd
+ [ -n "${RESOLV_CONF}" ] && cp -v "${RESOLV_CONF}" "${WRKDIR}/world/etc/"
+ mkdir -p ${POUDRIERE_DATA}/pkgcache/${MASTERNAME}
+ mkdir -p ${WRKDIR}/world/var/cache/pkg
if [ "${arch}" == "${host_arch}" ]; then
- cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF
- FreeBSD: { enabled: false }
- local: { url: file:///tmp/packages }
- EOF
- convert_package_list "${PACKAGELIST}" | \
- xargs chroot "${WRKDIR}/world" env \
- REPOS_DIR=/tmp ASSUME_ALWAYS_YES=yes \
- pkg install
+ if [ -n "${PKGREPO}" ]; then
+ ${NULLMOUNT} ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} \
+ ${WRKDIR}/world/var/cache/pkg
+ convert_package_list "${PACKAGELIST}" | \
+ xargs chroot "${WRKDIR}/world" env \
+ ASSUME_ALWAYS_YES=yes \
+ pkg install
+ umount ${WRKDIR}/world/var/cache/pkg
+ else
+ ${NULLMOUNT} ${POUDRIERE_DATA}/packages/${MASTERNAME} ${WRKDIR}/world/tmp/packages
+ cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF
+ FreeBSD: { enabled: false }
+ local: { url: file:///tmp/packages }
+ EOF
+ convert_package_list "${PACKAGELIST}" | \
+ xargs chroot "${WRKDIR}/world" env \
+ REPOS_DIR=/tmp ASSUME_ALWAYS_YES=yes \
+ pkg install
+ umount ${WRKDIR}/world/tmp/packages
+ fi
else
- cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF
- FreeBSD: { enabled: false }
- local: { url: file:///${WRKDIR}/world/tmp/packages }
- EOF
- (
- export ASSUME_ALWAYS_YES=yes SYSLOG=no \
- REPOS_DIR="${WRKDIR}/world/tmp/" \
- ABI_FILE="${WRKDIR}/world/usr/lib/crt1.o"
- pkg -r "${WRKDIR}/world/" install pkg
+ if [ -n "${PKGREPO}" ]; then
+ # Try do same as for amd64 (we only care about i386)
+ ${NULLMOUNT} ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} \
+ ${WRKDIR}/world/var/cache/pkg
convert_package_list "${PACKAGELIST}" | \
- xargs pkg -r "${WRKDIR}/world/" install
- )
+ xargs chroot "${WRKDIR}/world" env \
+ ASSUME_ALWAYS_YES=yes \
+ pkg install
+ umount ${WRKDIR}/world/var/cache/pkg
+ # ${NULLMOUNT} ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} \
+ # ${WRKDIR}/world/var/cache/pkg
+ # (
+ # export ASSUME_ALWAYS_YES=yes SYSLOG=no \
+ # ABI_FILE="${WRKDIR}/world/usr/lib/crt1.o"
+ # pkg -r "${WRKDIR}/world/" install pkg
+ # convert_package_list "${PACKAGELIST}" | \
+ # xargs pkg -r "${WRKDIR}/world/" install
+ # )
+ # umount ${WRKDIR}/world/var/cache/pkg
+ else
+ ${NULLMOUNT} ${POUDRIERE_DATA}/packages/${MASTERNAME} ${WRKDIR}/world/tmp/packages
+ cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF
+ FreeBSD: { enabled: false }
+ local: { url: file:///${WRKDIR}/world/tmp/packages }
+ EOF
+ (
+ export ASSUME_ALWAYS_YES=yes SYSLOG=no \
+ REPOS_DIR="${WRKDIR}/world/tmp/" \
+ ABI_FILE="${WRKDIR}/world/usr/lib/crt1.o"
+ pkg -r "${WRKDIR}/world/" install pkg
+ convert_package_list "${PACKAGELIST}" | \
+ xargs pkg -r "${WRKDIR}/world/" install
+ )
+ umount ${WRKDIR}/world/tmp/packages
+ fi
fi
- rm -rf ${WRKDIR}/world/var/cache/pkg
- umount ${WRKDIR}/world/tmp/packages
+ umount ${WRKDIR}/world/dev
rmdir ${WRKDIR}/world/tmp/packages
+ rm -rf ${WRKDIR}/world/var/cache/pkg
rm ${WRKDIR}/world/var/db/pkg/repo-* 2>/dev/null || :
fi
+
+if [ -e "${WRKDIR}/world/overlay.sh" ]; then
+ export MASTERNAME
+ mount -t devfs devfs ${WRKDIR}/world/dev
+ chroot "${WRKDIR}/world" env LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib /overlay.sh
+ umount ${WRKDIR}/world/dev
+fi
+
case ${MEDIATYPE} in
*mfs)
cat >> ${WRKDIR}/world/etc/fstab <<-EOF
@@ -574,8 +630,8 @@
mkimg -s gpt -b ${mnt}/boot/pmbr \
-p efi:=${mnt}/boot/boot1.efifat \
-p freebsd-boot:=${mnt}/boot/gptboot \
- -p freebsd-ufs:=${WRKDIR}/raw.img \
- -p freebsd-swap::1M \
+ -p freebsd-ufs/gfx-root:=${WRKDIR}/raw.img \
+ -p freebsd-swap/gfx-swap::1G \
-o ${OUTPUTDIR}/${FINALIMAGE}
;;
tar)
_______________________________________________
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[email protected]"
