Hi Olaf and Faustin, @Olaf
Thanks for your quick response and suggestions. On 20/11/18 18:50, Olaf van der Spek wrote: > IMO apt shouldn't be run in such a way that packages get removed > automatically though.. If you have any specific suggestions on how to ensure that apt won't remove packages, I'd be interested to hear. Also, IIRC there have been cases where removal of old packages were required (I think that was the case with Samba security updates within Jessie?! - Although perhaps I am confused). @Faustin Thanks to you too for your prompt reply, apologies that my response has been a little slow... I'll aim to provide as much relevant info as possible, if there is anything else you need please ask. Hopefully it's not too waffley and/or includes too much irrelevant info... (I'm often told that I need to turn verbosity down...) On 21/11/18 04:57, Faustin Lammler wrote: > Are you able to provide a step-by-step procedure? > Possibly the easiest way to reproduce the issue would be to download our v15.0 (Stretch based) LAMP appliance ISO[1] (signed hash file here[2]) and install it to a VM and NOT run the initial firstboot "security updates" script (i.e. select "skip" when asked). Once logged in as root, you can then poke around inside and see exactly what is going on. [1] http://mirror.turnkeylinux.org/turnkeylinux/images/iso/turnkey-lamp-15.0-stretch-amd64.iso [2] http://mirror.turnkeylinux.org/turnkeylinux/images/iso/turnkey-lamp-15.0-stretch-amd64.iso.hash Then the issue can be reproduced by running 'turnkey-install-security-updates' That will only install updates from Debian (and TurnKey) security repos. ---- In an effort to assist you to avoid that though, here's some more info which may help. The process that turnkey-install-security-updates uses is a little convoluted, but essentially it runs this: apt-get update apt-get autoclean -y apt-get dist-upgrade -y -o APT::Get::Show-Upgraded=true \ -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/security.sources.list \ -o Dir::Etc::sourceparts=nonexistent \ -o DPkg::Options::=--force-confdef \ -o DPkg::Options::=--force-confold FWIW the security.sources.list: deb http://archive.turnkeylinux.org/debian stretch-security main deb http://security.debian.org/ stretch/updates main deb http://security.debian.org/ stretch/updates contrib #deb http://security.debian.org/ stretch/updates non-free If I run the above dist-upgrade command (after apt-get update and without the -y switch), I get this: Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages were automatically installed and are no longer required: galera-3 libaio1 libjemalloc1 lsof mariadb-client-core-10.1 mariadb-common mariadb-server-core-10.1 socat Use 'apt autoremove' to remove them. The following packages will be REMOVED: default-mysql-server mariadb-client-10.1 mariadb-server-10.1 mysql-server The following NEW packages will be installed: linux-image-4.9.0-8-amd64 The following packages will be upgraded: curl git git-core git-man libcurl3 libcurl3-gnutls libfuse2 libmariadbclient18 libpython2.7 libpython2.7-minimal libpython2.7-stdlib libpython3.5-minimal libpython3.5-stdlib linux-image-4.9.0-7-amd64 linux-image-amd64 mariadb-client-core-10.1 mariadb-common mariadb-server-core-10.1 openssh-client openssh-server openssh-sftp-server python2.7 python2.7-minimal python3.5 python3.5-minimal ssh 26 upgraded, 1 newly installed, 4 to remove and 0 not upgraded. Need to get 107 MB of archives. After this operation, 68.6 MB of additional disk space will be used. Do you want to continue? [Y/n] Obviously some of those packages are irrelevant to this issue, but figured it best to not omit anything. If I then allow it to install those updates (and uninstall default-mysql-server mariadb-client-10.1 mariadb-server-10.1 & mysql-server), then reinstall default-mysql-server, here's what I get: root@lamp ~# apt-get install default-mysql-server Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libconfig-inifiles-perl mariadb-client-10.1 mariadb-server-10.1 Suggested packages: mariadb-test netcat-openbsd tinyca Recommended packages: libterm-readkey-perl libhtml-template-perl The following NEW packages will be installed: default-mysql-server libconfig-inifiles-perl mariadb-client-10.1 mariadb-server-10.1 0 upgraded, 4 newly installed, 0 to remove and 25 not upgraded. Need to get 11.3 MB of archives. After this operation, 125 MB of additional disk space will be used. So it appears likely that the offending dependency (as suggested by Olaf) is libconfig-inifiles-perl ?! > If not, dpgk -l could help to understand what apt dependencies may be > problematic. And what happened when you reinstalled mariadb. Which > command did you used, what was the output? > Alternatively (or as well as?), you can see all the packages installed by apt/dpkg in the appliance manifest[3]. [3] http://mirror.turnkeylinux.org/turnkeylinux/metadata/turnkey-lamp/15.0-stretch-amd64/turnkey-lamp-15.0-stretch-amd64.manifest If you would prefer a full dpkg -l output, I'm happy to also provide that. As noted above, it appears that offending dependency is libconfig-inifiles-perl. It is not installed (at all) by default on our servers. Prior to (re)installing default-mysql-server: root@lamp ~# apt policy libconfig-inifiles-perl libconfig-inifiles-perl: Installed: (none) Candidate: 2.94-1 Version table: 2.94-1 500 500 http://deb.debian.org/debian stretch/main amd64 Packages Below, I've noted more details of upgradable packages from the above LAMP appliance prior to the upgrade. As above, obviously some which are irrelevant. apache2/stable 2.4.25-3+deb9u6 amd64 [upgradable from: 2.4.25-3+deb9u5] apache2-bin/stable 2.4.25-3+deb9u6 amd64 [upgradable from: 2.4.25-3+deb9u5] apache2-data/stable 2.4.25-3+deb9u6 all [upgradable from: 2.4.25-3+deb9u5] apache2-utils/stable 2.4.25-3+deb9u6 amd64 [upgradable from: 2.4.25-3+deb9u5] base-files/stable 9.9+deb9u6 amd64 [upgradable from: 9.9+deb9u5] confconsole/stretch 1.1.0+2+g6c2aad9 all [upgradable from: 1.1.0] curl/stable,stable 7.52.1-5+deb9u8 amd64 [upgradable from: 7.52.1-5+deb9u6] git/stable,stable 1:2.11.0-3+deb9u4 amd64 [upgradable from: 1:2.11.0-3+deb9u3] git-core/stable,stable 1:2.11.0-3+deb9u4 all [upgradable from: 1:2.11.0-3+deb9u3] git-man/stable,stable 1:2.11.0-3+deb9u4 all [upgradable from: 1:2.11.0-3+deb9u3] gnupg/stable 2.1.18-8~deb9u3 amd64 [upgradable from: 2.1.18-8~deb9u2] gnupg-agent/stable 2.1.18-8~deb9u3 amd64 [upgradable from: 2.1.18-8~deb9u2] gpgv/stable 2.1.18-8~deb9u3 amd64 [upgradable from: 2.1.18-8~deb9u2] grub-common/stable 2.02~beta3-5+deb9u1 amd64 [upgradable from: 2.02~beta3-5] grub-pc/stable 2.02~beta3-5+deb9u1 amd64 [upgradable from: 2.02~beta3-5] grub-pc-bin/stable 2.02~beta3-5+deb9u1 amd64 [upgradable from: 2.02~beta3-5] grub2-common/stable 2.02~beta3-5+deb9u1 amd64 [upgradable from: 2.02~beta3-5] hdparm/stable 9.51+ds-1+deb9u1 amd64 [upgradable from: 9.51+ds-1] libcurl3/stable,stable 7.52.1-5+deb9u8 amd64 [upgradable from: 7.52.1-5+deb9u6] libcurl3-gnutls/stable,stable 7.52.1-5+deb9u8 amd64 [upgradable from: 7.52.1-5+deb9u6] libfuse2/stable 2.9.7-1+deb9u2 amd64 [upgradable from: 2.9.7-1] libgnutls30/stable 3.5.8-5+deb9u4 amd64 [upgradable from: 3.5.8-5+deb9u3] libmariadbclient18/stable 10.1.37-0+deb9u1 amd64 [upgradable from: 10.1.26-0+deb9u1] libpython2.7/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable from: 2.7.13-2+deb9u2] libpython2.7-minimal/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable from: 2.7.13-2+deb9u2] libpython2.7-stdlib/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable from: 2.7.13-2+deb9u2] libpython3.5-minimal/stable,stable 3.5.3-1+deb9u1 amd64 [upgradable from: 3.5.3-1] libpython3.5-stdlib/stable,stable 3.5.3-1+deb9u1 amd64 [upgradable from: 3.5.3-1] libseccomp2/stable 2.3.1-2.1+deb9u1 amd64 [upgradable from: 2.3.1-2.1] libsystemd0/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4] libudev1/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4] linux-image-4.9.0-7-amd64/stable 4.9.110-3+deb9u2 amd64 [upgradable from: 4.9.110-1] linux-image-amd64/stable,stable 4.9+80+deb9u6 amd64 [upgradable from: 4.9+80+deb9u5] mariadb-client-10.1/stable 10.1.37-0+deb9u1 amd64 [upgradable from: 10.1.26-0+deb9u1] mariadb-client-core-10.1/stable 10.1.37-0+deb9u1 amd64 [upgradable from: 10.1.26-0+deb9u1] mariadb-common/stable 10.1.37-0+deb9u1 all [upgradable from: 10.1.26-0+deb9u1] mariadb-server-10.1/stable 10.1.37-0+deb9u1 amd64 [upgradable from: 10.1.26-0+deb9u1] mariadb-server-core-10.1/stable 10.1.37-0+deb9u1 amd64 [upgradable from: 10.1.26-0+deb9u1] openssh-client/stable,stable 1:7.4p1-10+deb9u4 amd64 [upgradable from: 1:7.4p1-10+deb9u3] openssh-server/stable,stable 1:7.4p1-10+deb9u4 amd64 [upgradable from: 1:7.4p1-10+deb9u3] openssh-sftp-server/stable,stable 1:7.4p1-10+deb9u4 amd64 [upgradable from: 1:7.4p1-10+deb9u3] python2.7/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable from: 2.7.13-2+deb9u2] python2.7-minimal/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable from: 2.7.13-2+deb9u2] python3.5/stable,stable 3.5.3-1+deb9u1 amd64 [upgradable from: 3.5.3-1] python3.5-minimal/stable,stable 3.5.3-1+deb9u1 amd64 [upgradable from: 3.5.3-1] ssh/stable,stable 1:7.4p1-10+deb9u4 all [upgradable from: 1:7.4p1-10+deb9u3] systemd/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4] systemd-sysv/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4] tklbam/stretch 1.4.1+37+g8117cd6 all [upgradable from: 1.4.1+32+g07acc1c] tzdata/stable 2018g-0+deb9u1 all [upgradable from: 2018e-0+deb9u1] udev/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4] > Olaf's guess maybe true but there can be lot's of reason why APT decided > to remove mariadb-* packages. > As noted above, it appears that Olaf was right. The offending dependency appears to be libconfig-inifiles-perl. FWIW libconfig-inifiles-perl appears to be a (new?) dependency of mariadb-client-10.1, which is in turn a dependency of mariadb-server-10.1: root@lamp ~# apt-cache depends mariadb-client-10.1 | grep libconfig-inifiles-perl Depends: libconfig-inifiles-perl root@lamp ~# apt-cache depends mariadb-server-10.1 | grep mariadb-client-10.1 Depends: mariadb-client-10.1 My guess is that this dependency was previously a "recommends" and is now a hard "depends" (see below). Does that seem likely? > Finally, do you at turnkeylinux set some non default apt preferences? Assuming you mean config in general (rather than strictly "preferences") yes we do. I suspect the one that may have been a causal fact in this case is not installing "recommends" by default: root@lamp ~# cat /etc/apt/apt.conf.d/05recommends // Don't consider recommends as dependencies and install them by default APT::Install-Recommends "false"; There are some other conf snippets in apt.conf.d, but without further checking (i.e. comparison with Debian default) I'm not 100% sure which ones are default and which are added by us. Even then, I'm not sure which ones are significant? On face value, none seem to be of real significance in this scenario. E.g.: root@lamp ~# cat /etc/apt/apt.conf.d/70debconf // Pre-configure all packages with debconf before they are installed. // If you don't like it, comment it out. DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt || true";}; Having said that I note that there is a 01autoremove script which notes some packages to never autoremove. Might be worth further investiagation to mitigate against the cahnce of this in the future? Hope that is all useful and relevant. Anything further you need, please ask. Regards, Jeremy
signature.asc
Description: OpenPGP digital signature
