Package: ifupdown
Version: 0.8.44
Severity: critical
Justification: breaks unrelated software
Hey.
I run a large storage cluster (50 nodes or so, Dell, HPE servers) at the local
university, which I’ve recently upgraded from bookworm to trixie.
Today I migrated the last node, which is the core node of the storage system
an noticed that the system completely failed, which after a short search
tured out to be that dCache (the software) can’t connect to the locally running
PostgreSQL DB, which turned out to be because when the PostgreSQL started (after
a reboot) it couldn’t bind to the IPv6 address (and thereafter never tried
again).
At first I thought the reason is because of a buggy PostgreSQL systemd script
(#1099664, where I’d agree with the original reporter, that
network-online.target
should be used as per its ad network.target’s documentation in
systemd.special(7)).
So I overwrote that with a drop-in conf, but no help.
Only then I’ve notice, that many more services on all the 50 nodes were
affected.
In principle everything that tries to bind to IPv6, doesn’t compltely faily if
it
can’t and never retries again "later".
So things like ssh, apache httpd, ... they all fail to bind to IPv6 on boot
(but it works fine if I restart these services later).
ntp seems to be an exception, it fails during boot (I see a log message) but
apparently
re-binds to the address later.
So my next guess was that ifupdown is the "offender" ;-) (and it seems it is).
First, I changed /usr/lib/systemd/system/networking.service’s ExecStart= to:
ExecStart=-/bin/sh -c 'if [ -f /run/network/restart-hotplug ]; then
/usr/sbin/ifup -a --read-environment --allow=hotplug; fi; echo CALESTYO;
ifconfig'
which gave rather counter-indicative results:
Here it starts:
Dec 10 22:33:02 lcg-lrz-dcache0 systemd[1]: Starting networking.service - Raise
network interfaces...
Dec 10 22:33:02 lcg-lrz-dcache0 systemd[1]: Mounted
proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
Dec 10 22:33:02 lcg-lrz-dcache0 systemd[1]: Finished systemd-binfmt.service -
Set Up Additional Binary Formats.
Dec 10 22:33:02 lcg-lrz-dcache0 systemd[1]: Reached target sysinit.target -
System Initialization.
Dec 10 22:33:02 lcg-lrz-dcache0 systemd[1]: Started anacron.timer - Trigger
anacron every hour.
...
<loads of other boring stuff>
...
Dec 10 22:33:04 lcg-lrz-dcache0 dbus-daemon[1787]: [system] Successfully
activated service 'org.freedesktop.PolicyKit1'
Dec 10 22:33:04 lcg-lrz-dcache0 polkitd[2105]: Acquired the name
org.freedesktop.PolicyKit1 on the system bus
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Started udisks2.service - Disk
Manager.
Dec 10 22:33:04 lcg-lrz-dcache0 udisksd[1818]: Acquired the name
org.freedesktop.UDisks2 on the system message bus
Okay now the command is about to end:
(and suprise the IPv6 *is* set... BUT... look at the timestamp 22:33:04)
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2338]: CALESTYO
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: eth0:
flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: inet 129.187.131.210 netmask
255.255.255.0 broadcast 129.187.131.255
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: inet6
fe80::b67a:f1ff:fe7d:dca2 prefixlen 64 scopeid 0x20<link>
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: inet6 2001:4ca0:0:201:a::
prefixlen 64 scopeid 0x0<global>
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: ether b4:7a:f1:7d:dc:a2
txqueuelen 1000 (Ethernet)
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: RX packets 0 bytes 0 (0.0 B)
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: RX errors 0 dropped 0
overruns 0 frame 0
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: TX packets 3 bytes 298
(298.0 B)
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: TX errors 0 dropped 0
overruns 0 carrier 0 collisions 0
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: device interrupt 145 memory
0xdf800000-df81ffff
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: lo: flags=73<UP,LOOPBACK,RUNNING>
mtu 65536
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: inet 127.0.0.1 netmask
255.0.0.0
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: inet6 ::1 prefixlen 128
scopeid 0x10<host>
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: loop txqueuelen 1000 (Local
Loopback)
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: RX packets 0 bytes 0 (0.0 B)
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: RX errors 0 dropped 0
overruns 0 frame 0
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: TX packets 0 bytes 0 (0.0 B)
Dec 10 22:33:04 lcg-lrz-dcache0 sh[2400]: TX errors 0 dropped 0
overruns 0 carrier 0 collisions 0
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Finished networking.service - Raise
network interfaces.
Here networking.service is done
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Reached target network.target -
Network.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Reached target
network-online.target - Network is Online.
here the targets ...
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Started apt-listbugs.timer - Hourly
check for daily apt-listbugs preferences cleanup.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Reached target timers.target -
Timer Units.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Starting apache2.service - The
Apache HTTP Server...
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Started fail2ban.service - Fail2Ban
Service.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Starting ntpsec.service - Network
Time Service...
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Starting postfix.service - Postfix
Mail Transport Agent (main/default instance)...
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2588]: INIT: ntpd ntpsec-1.2.3: Starting
Dec 10 22:33:04 lcg-lrz-dcache0 ntp-systemd-wrapper[2588]: 2025-12-10T22:33:04
ntpd[2588]: INIT: ntpd ntpsec-1.2.3: Starting
Dec 10 22:33:04 lcg-lrz-dcache0 ntp-systemd-wrapper[2588]: 2025-12-10T22:33:04
ntpd[2588]: INIT: Command line: /usr/sbin/ntpd -p /run/ntpd.pid -c
/etc/ntpsec/ntp.conf -g -u ntpsec:ntpsec
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2588]: INIT: Command line: /usr/sbin/ntpd
-p /run/ntpd.pid -c /etc/ntpsec/ntp.conf -g -u ntpsec:ntpsec
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: INIT: precision = 0.244 usec (-22)
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: INIT: successfully locked into RAM
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: CONFIG: readconfig: parsing file:
/etc/ntpsec/ntp.conf
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: CLOCK: leapsecond file
('/usr/share/zoneinfo/leap-seconds.list'): good hash signature
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: CLOCK: leapsecond file
('/usr/share/zoneinfo/leap-seconds.list'): loaded, expire=2026-06-28T00:00Z
last=2017-01-01T00:00Z ofs=37
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: INIT: Using SO_TIMESTAMPNS(ns)
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: Listen and drop on 0 v6wildcard
[::]:123
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: Listen and drop on 1 v4wildcard
0.0.0.0:123
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: Listen normally on 2 lo
127.0.0.1:123
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: Listen normally on 3 eth0
129.187.131.210:123
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: Listen normally on 4 lo
[::1]:123
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: bind(21) AF_INET6
2001:4ca0:0:201:a::#123 flags 0x1 failed: Cannot assign requested address
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: unable to create socket on eth0
(5) for 2001:4ca0:0:201:a::#123
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: failed to init interface for
address 2001:4ca0:0:201:a::
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: bind(21) AF_INET6
fe80::b67a:f1ff:fe7d:dca2%6#123 flags 0x1 failed: Cannot assign requested
address
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: unable to create socket on eth0
(6) for fe80::b67a:f1ff:fe7d:dca2%6#123
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: failed to init interface for
address fe80::b67a:f1ff:fe7d:dca2%6
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: IO: Listening on routing socket on
fd #21 for interface updates
here, ntp failed to bind (timestamp still 22:33:04)
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Starting
[email protected] - PostgreSQL Cluster 17-dcache...
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: INIT: MRU 10922 entries, 13 hash
bits, 65536 bytes
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: CLOCK: kernel reports TIME_ERROR:
0x41: Clock Unsynchronized
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: CLOCK: kernel reports TIME_ERROR:
0x41: Clock Unsynchronized
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: INIT: Built with OpenSSL 3.5.0 8
Apr 2025, 30500000
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: INIT: Running with OpenSSL 3.5.4 30
Sep 2025, 30500040
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: NTSc: Using system default root
certificates.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Starting ssh.service - OpenBSD
Secure Shell server...
Dec 10 22:33:04 lcg-lrz-dcache0 apachectl[2580]: AH00548: NameVirtualHost has
no effect and will be removed in the next release
/etc/apache2/sites-enabled/00-default.vhost.conf:203
Dec 10 22:33:04 lcg-lrz-dcache0 apachectl[2580]: (99)Cannot assign requested
address: AH00072: make_sock: could not bind to address [2001:4ca0:0:201:a::]:80
Dec 10 22:33:04 lcg-lrz-dcache0 apachectl[2580]: no listening sockets
available, shutting down
Dec 10 22:33:04 lcg-lrz-dcache0 apachectl[2580]: AH00015: Unable to open logs
here, apache failed to bind (timestamp still 22:33:04)
Dec 10 22:33:04 lcg-lrz-dcache0 postfix[2583]: postfix: Postfix is using
backwards-compatible default settings
Dec 10 22:33:04 lcg-lrz-dcache0 postfix[2583]: postfix: See
https://www.postfix.org/COMPATIBILITY_README.html for details
Dec 10 22:33:04 lcg-lrz-dcache0 postfix[2583]: postfix: To disable backwards
compatibility use "postconf compatibility_level=3.6" and "postfix reload"
Dec 10 22:33:04 lcg-lrz-dcache0 postfix[2583]: Postfix is using
backwards-compatible default settings
Dec 10 22:33:04 lcg-lrz-dcache0 postfix[2583]: See
https://www.postfix.org/COMPATIBILITY_README.html for details
Dec 10 22:33:04 lcg-lrz-dcache0 postfix[2583]: To disable backwards
compatibility use "postconf compatibility_level=3.6" and "postfix reload"
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Starting
systemd-user-sessions.service - Permit User Sessions...
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Starting zookeeper.service - LSB:
centralized coordination service...
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-needrestart.service: Deactivated successfully.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Finished
prometheus-node-exporter-needrestart.service - needrestart metric collection
for Prometheus node exporter’s textfile collector.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-needrestart.service: Consumed 543ms CPU time, 68.3M
memory peak.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-smartmon.service: Deactivated successfully.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Finished
prometheus-node-exporter-smartmon.service - Collect SMART metrics for
prometheus-node-exporter.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-smartmon.service: Consumed 1.214s CPU time, 11M memory
peak.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-smartraid.service: Deactivated successfully.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Finished
prometheus-node-exporter-smartraid.service - SmartRAID metric collection for
Prometheus node exporter’s textfile collector.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: apache2.service: Control process
exited, code=exited, status=1/FAILURE
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: apache2.service: Failed with result
'exit-code'.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Failed to start apache2.service -
The Apache HTTP Server.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: apache2.service: Triggering
OnFailure= dependencies.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Started ntpsec.service - Network
Time Service.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Finished
systemd-user-sessions.service - Permit User Sessions.
Dec 10 22:33:04 lcg-lrz-dcache0 ntpd[2595]: statistics directory
/var/log/ntpsec/ does not exist or is unwriteable, error No such file or
directory
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Created slice
system-mail\x2dsystemd\x2dunit\x2dstatus.slice - Slice
/system/mail-systemd-unit-status.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Started [email protected] - Getty
on tty1.
Dec 10 22:33:04 lcg-lrz-dcache0 sshd[2670]: error: Bind to port 22 on
2001:4ca0:0:201:a:: failed: Cannot assign requested address.
here, sshd failed to bind (timestamp still 22:33:04)
Dec 10 22:33:04 lcg-lrz-dcache0 sshd[2670]: Server listening on 129.187.131.210
port 22.
Dec 10 22:33:04 lcg-lrz-dcache0 sshd[2670]: Server listening on 127.0.1.1 port
22.
Dec 10 22:33:04 lcg-lrz-dcache0 sshd[2670]: Server listening on ::1 port 22.
Dec 10 22:33:04 lcg-lrz-dcache0 sshd[2670]: error: Bind to port 22 on ::1
failed: Address already in use.
Dec 10 22:33:04 lcg-lrz-dcache0 sshd[2670]: Server listening on 127.0.0.1 port
22.
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Starting
[email protected] - send systemd unit status via
email to `root`...
Dec 10 22:33:04 lcg-lrz-dcache0 systemd[1]: Started [email protected]
- Serial Getty on ttyS0.
Dec 10 22:33:05 lcg-lrz-dcache0 systemd[1]: Reached target getty.target - Login
Prompts.
Dec 10 22:33:05 lcg-lrz-dcache0 systemd[1]: Started ssh.service - OpenBSD
Secure Shell server.
Dec 10 22:33:05 lcg-lrz-dcache0 systemd[1]: Started zookeeper.service - LSB:
centralized coordination service.
Dec 10 22:33:05 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-apt.service: Deactivated successfully.
Dec 10 22:33:05 lcg-lrz-dcache0 systemd[1]: Finished
prometheus-node-exporter-apt.service - Collect apt metrics for
prometheus-node-exporter.
Dec 10 22:33:05 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-apt.service: Consumed 1.070s CPU time, 111.1M memory
peak.
Dec 10 22:33:05 lcg-lrz-dcache0 fail2ban-server[2534]: Server ready
Dec 10 22:33:05 lcg-lrz-dcache0 postfix[2827]: postfix: Postfix is using
backwards-compatible default settings
Dec 10 22:33:05 lcg-lrz-dcache0 postfix[2827]: postfix: See
https://www.postfix.org/COMPATIBILITY_README.html for details
Dec 10 22:33:05 lcg-lrz-dcache0 postfix[2827]: postfix: To disable backwards
compatibility use "postconf compatibility_level=3.6" and "postfix reload"
Dec 10 22:33:05 lcg-lrz-dcache0 postfix[2827]: Postfix is using
backwards-compatible default settings
Dec 10 22:33:05 lcg-lrz-dcache0 postfix[2827]: See
https://www.postfix.org/COMPATIBILITY_README.html for details
Dec 10 22:33:05 lcg-lrz-dcache0 postfix[2827]: To disable backwards
compatibility use "postconf compatibility_level=3.6" and "postfix reload"
Dec 10 22:33:05 lcg-lrz-dcache0 ntpd[2595]: DNS: dns_probe: ptbtime1.ptb.de,
cast_flags:1, flags:21801
Dec 10 22:33:05 lcg-lrz-dcache0 postfix/master[2836]: daemon started -- version
3.10.5, configuration /etc/postfix
Dec 10 22:33:05 lcg-lrz-dcache0 systemd[1]: Started postfix.service - Postfix
Mail Transport Agent (main/default instance).
Dec 10 22:33:06 lcg-lrz-dcache0 systemd[1]:
[email protected]: Deactivated successfully.
Dec 10 22:33:06 lcg-lrz-dcache0 systemd[1]: Finished
[email protected] - send systemd unit status via
email to `root`.
Dec 10 22:33:06 lcg-lrz-dcache0 systemd[1]:
[email protected]: Consumed 442ms CPU time,
692.1M memory peak.
Dec 10 22:33:06 lcg-lrz-dcache0 postfix/pickup[2837]: 7436E29D5FFF: uid=0
from=<root>
Dec 10 22:33:06 lcg-lrz-dcache0 postfix/cleanup[2850]: 7436E29D5FFF:
message-id=<[email protected]>
Dec 10 22:33:06 lcg-lrz-dcache0 postfix/qmgr[2838]: 7436E29D5FFF:
from=<[email protected]>, size=2038, nrcpt=1 (queue active)
Dec 10 22:33:06 lcg-lrz-dcache0 postfix/cleanup[2850]: 79F8529D6000:
message-id=<[email protected]>
Dec 10 22:33:06 lcg-lrz-dcache0 postfix/local[2853]: 7436E29D5FFF:
to=<[email protected]>, orig_to=<root>, relay=local, delay=0.06,
delays=0.04/0.01/0/0.01, dsn=2.0.0, status=sent (forwarded as 79F8529D6000)
Dec 10 22:33:06 lcg-lrz-dcache0 postfix/qmgr[2838]: 79F8529D6000:
from=<[email protected]>, size=2195, nrcpt=1 (queue active)
Dec 10 22:33:06 lcg-lrz-dcache0 postfix/qmgr[2838]: 7436E29D5FFF: removed
Dec 10 22:33:07 lcg-lrz-dcache0 systemd[1]: Started
[email protected] - PostgreSQL Cluster 17-dcache.
Dec 10 22:33:07 lcg-lrz-dcache0 systemd[1]: Starting postgresql.service -
PostgreSQL RDBMS...
Dec 10 22:33:07 lcg-lrz-dcache0 systemd[1]: Finished postgresql.service -
PostgreSQL RDBMS.
postgres simply seems to ignore the ones it cannot bind to... so no error
message and no failure, I guess
Dec 10 22:33:07 lcg-lrz-dcache0 systemd[1]: Reached target multi-user.target -
Multi-User System.
Dec 10 22:33:07 lcg-lrz-dcache0 systemd[1]: Reached target graphical.target -
Graphical Interface.
Dec 10 22:33:07 lcg-lrz-dcache0 systemd[1]: Startup finished in 14.680s
(kernel) + 12.411s (userspace) = 27.092s.
here it seeems ntp re-binds (and then succeeds).
Dec 10 22:33:07 lcg-lrz-dcache0 ntpd[2595]: IO: Listen normally on 7 eth0
[2001:4ca0:0:201:a::]:123
Dec 10 22:33:07 lcg-lrz-dcache0 ntpd[2595]: IO: Listen normally on 8 eth0
[fe80::b67a:f1ff:fe7d:dca2%6]:123
Dec 10 22:33:07 lcg-lrz-dcache0 ntpd[2595]: IO: new interface(s) found: waking
up resolver
Because of the identical timestamps I figured the IPv6 might not be confiured
yet and the log output just unfortunately added due to the concurrency (yes,
seems
to be so), so I changed the unit to use:
ExecStart=-/bin/sh -c 'if [ -f /run/network/restart-hotplug ]; then
/usr/sbin/ifup -a --read-environment --allow=hotplug; fi; echo CALESTYO;
ifconfig; sleep 30'
With that, things work again:
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2272]: CALESTYO
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: eth0:
flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: inet 129.187.131.210 netmask
255.255.255.0 broadcast 129.187.131.255
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: inet6 2001:4ca0:0:201:a::
prefixlen 64 scopeid 0x0<global>
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: inet6
fe80::b67a:f1ff:fe7d:dca2 prefixlen 64 scopeid 0x20<link>
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: ether b4:7a:f1:7d:dc:a2
txqueuelen 1000 (Ethernet)
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: RX packets 0 bytes 0 (0.0 B)
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: RX errors 0 dropped 0
overruns 0 frame 0
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: TX packets 0 bytes 0 (0.0 B)
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: TX errors 0 dropped 0
overruns 0 carrier 0 collisions 0
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: device interrupt 145 memory
0xdf800000-df81ffff
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: lo: flags=73<UP,LOOPBACK,RUNNING>
mtu 65536
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: inet 127.0.0.1 netmask
255.0.0.0
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: inet6 ::1 prefixlen 128
scopeid 0x10<host>
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: loop txqueuelen 1000 (Local
Loopback)
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: RX packets 0 bytes 0 (0.0 B)
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: RX errors 0 dropped 0
overruns 0 frame 0
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: TX packets 0 bytes 0 (0.0 B)
Dec 10 22:37:02 lcg-lrz-dcache0 sh[2338]: TX errors 0 dropped 0
overruns 0 carrier 0 collisions 0
here it’s configured
Dec 10 22:37:02 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-smartraid.service: Deactivated successfully.
Dec 10 22:37:02 lcg-lrz-dcache0 systemd[1]: Finished
prometheus-node-exporter-smartraid.service - SmartRAID metric collection for
Prometheus node exporter’s textfile collector.
Dec 10 22:37:02 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-needrestart.service: Deactivated successfully.
Dec 10 22:37:02 lcg-lrz-dcache0 systemd[1]: Finished
prometheus-node-exporter-needrestart.service - needrestart metric collection
for Prometheus node exporter’s textfile collector.
Dec 10 22:37:02 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-needrestart.service: Consumed 558ms CPU time, 68.3M
memory peak.
Dec 10 22:37:02 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-smartmon.service: Deactivated successfully.
Dec 10 22:37:02 lcg-lrz-dcache0 systemd[1]: Finished
prometheus-node-exporter-smartmon.service - Collect SMART metrics for
prometheus-node-exporter.
Dec 10 22:37:02 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-smartmon.service: Consumed 1.243s CPU time, 12.1M
memory peak.
Dec 10 22:37:03 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-apt.service: Deactivated successfully.
Dec 10 22:37:03 lcg-lrz-dcache0 systemd[1]: Finished
prometheus-node-exporter-apt.service - Collect apt metrics for
prometheus-node-exporter.
Dec 10 22:37:03 lcg-lrz-dcache0 systemd[1]:
prometheus-node-exporter-apt.service: Consumed 998ms CPU time, 111M memory peak.
notice the 30 s jump:
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Finished networking.service - Raise
network interfaces.
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Reached target network.target -
Network.
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Reached target
network-online.target - Network is Online.
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Started apt-listbugs.timer - Hourly
check for daily apt-listbugs preferences cleanup.
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Reached target timers.target -
Timer Units.
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Starting apache2.service - The
Apache HTTP Server...
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Started fail2ban.service - Fail2Ban
Service.
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Starting ntpsec.service - Network
Time Service...
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Starting postfix.service - Postfix
Mail Transport Agent (main/default instance)...
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2643]: INIT: ntpd ntpsec-1.2.3: Starting
Dec 10 22:37:32 lcg-lrz-dcache0 ntp-systemd-wrapper[2643]: 2025-12-10T22:37:32
ntpd[2643]: INIT: ntpd ntpsec-1.2.3: Starting
Dec 10 22:37:32 lcg-lrz-dcache0 ntp-systemd-wrapper[2643]: 2025-12-10T22:37:32
ntpd[2643]: INIT: Command line: /usr/sbin/ntpd -p /run/ntpd.pid -c
/etc/ntpsec/ntp.conf -g -u ntpsec:ntpsec
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2643]: INIT: Command line: /usr/sbin/ntpd
-p /run/ntpd.pid -c /etc/ntpsec/ntp.conf -g -u ntpsec:ntpsec
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: INIT: precision = 0.245 usec (-22)
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Starting
[email protected] - PostgreSQL Cluster 17-dcache...
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: INIT: successfully locked into RAM
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: CONFIG: readconfig: parsing file:
/etc/ntpsec/ntp.conf
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: CLOCK: leapsecond file
('/usr/share/zoneinfo/leap-seconds.list'): good hash signature
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: CLOCK: leapsecond file
('/usr/share/zoneinfo/leap-seconds.list'): loaded, expire=2026-06-28T00:00Z
last=2017-01-01T00:00Z ofs=37
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: INIT: Using SO_TIMESTAMPNS(ns)
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: IO: Listen and drop on 0 v6wildcard
[::]:123
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: IO: Listen and drop on 1 v4wildcard
0.0.0.0:123
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: IO: Listen normally on 2 lo
127.0.0.1:123
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: IO: Listen normally on 3 eth0
129.187.131.210:123
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: IO: Listen normally on 4 lo
[::1]:123
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: IO: Listen normally on 5 eth0
[2001:4ca0:0:201:a::]:123
ntp succeeds on the frist try
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: IO: Listen normally on 6 eth0
[fe80::b67a:f1ff:fe7d:dca2%6]:123
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: IO: Listening on routing socket on
fd #23 for interface updates
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Starting ssh.service - OpenBSD
Secure Shell server...
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: INIT: MRU 10922 entries, 13 hash
bits, 65536 bytes
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: CLOCK: kernel reports TIME_ERROR:
0x41: Clock Unsynchronized
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: CLOCK: kernel reports TIME_ERROR:
0x41: Clock Unsynchronized
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: INIT: Built with OpenSSL 3.5.0 8
Apr 2025, 30500000
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: INIT: Running with OpenSSL 3.5.4 30
Sep 2025, 30500040
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: NTSc: Using system default root
certificates.
Dec 10 22:37:32 lcg-lrz-dcache0 postfix[2641]: postfix: Postfix is using
backwards-compatible default settings
Dec 10 22:37:32 lcg-lrz-dcache0 postfix[2641]: postfix: See
https://www.postfix.org/COMPATIBILITY_README.html for details
Dec 10 22:37:32 lcg-lrz-dcache0 postfix[2641]: postfix: To disable backwards
compatibility use "postconf compatibility_level=3.6" and "postfix reload"
Dec 10 22:37:32 lcg-lrz-dcache0 postfix[2641]: Postfix is using
backwards-compatible default settings
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Starting
systemd-user-sessions.service - Permit User Sessions...
Dec 10 22:37:32 lcg-lrz-dcache0 postfix[2641]: See
https://www.postfix.org/COMPATIBILITY_README.html for details
Dec 10 22:37:32 lcg-lrz-dcache0 postfix[2641]: To disable backwards
compatibility use "postconf compatibility_level=3.6" and "postfix reload"
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Starting zookeeper.service - LSB:
centralized coordination service...
Dec 10 22:37:32 lcg-lrz-dcache0 apachectl[2655]: AH00548: NameVirtualHost has
no effect and will be removed in the next release
/etc/apache2/sites-enabled/00-default.vhost.conf:203
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Started ntpsec.service - Network
Time Service.
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Started apache2.service - The
Apache HTTP Server.
Dec 10 22:37:32 lcg-lrz-dcache0 systemd[1]: Finished
systemd-user-sessions.service - Permit User Sessions.
Dec 10 22:37:32 lcg-lrz-dcache0 ntpd[2649]: statistics directory
/var/log/ntpsec/ does not exist or is unwriteable, error No such file or
directory
Dec 10 22:37:32 lcg-lrz-dcache0 sshd[2684]: Server listening on
2001:4ca0:0:201:a:: port 22.
Dec 10 22:37:32 lcg-lrz-dcache0 sshd[2684]: Server listening on 129.187.131.210
port 22.
Dec 10 22:37:32 lcg-lrz-dcache0 sshd[2684]: Server listening on 127.0.1.1 port
22.
Dec 10 22:37:32 lcg-lrz-dcache0 sshd[2684]: Server listening on ::1 port 22.
Dec 10 22:37:32 lcg-lrz-dcache0 sshd[2684]: error: Bind to port 22 on ::1
failed: Address already in use.
Dec 10 22:37:32 lcg-lrz-dcache0 sshd[2684]: Server listening on 127.0.0.1 port
22.
ssh succeeds on the frist try
Probably 30s is too much... but ideally it should just work
perfectly out of the box.
Still no idea why it doesn't work as is, because even within the ExecStart=
command, ifconfig already shows the IPv6 to be set... and the other services,
despite the identical timestamp, should still come afterwards.
Any ideas?
Thanks,
Chris.
-- Package-specific info:
--- /etc/network/interfaces:
allow-auto lo
allow-auto eth0
iface lo inet loopback
dns-search grid.lrz.de
iface eth0 inet static
address 129.187.131.210
netmask 255.255.255.0
gateway 129.187.131.254
dns-nameserver 10.156.33.53
dns-nameserver 129.187.5.1
iface eth0 inet6 static
address 2001:4CA0:0:201:A:0:0:0
netmask 64
gateway 2001:4CA0:0:201::1
dns-nameserver 2001:4CA0::53:1
dns-nameserver 2001:4CA0::53:2
--- up and down scripts installed:
/etc/network/if-down.d:
total 12
-rwxr-xr-x 1 root root 207 Oct 28 11:24 postfix
-rwxr-xr-x 1 root root 331 Apr 23 2025 resolvconf
-rwxr-xr-x 1 root root 759 Dec 9 2022 resolved
/etc/network/if-post-down.d:
total 0
/etc/network/if-pre-up.d:
total 4
-rwxr-xr-x 1 root root 348 Aug 23 21:10 ethtool
/etc/network/if-up.d:
total 20
-rwxr-xr-x 1 root root 1042 Apr 23 2025 000resolvconf
-rwxr-xr-x 1 root root 1689 Aug 23 21:10 ethtool
-rwxr-xr-x 1 root root 207 Oct 28 11:24 postfix
-rwxr-xr-x 1 root root 4665 Aug 20 2024 resolved
-- System Information:
Debian Release: 13.2
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500,
'stable-debug'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 6.12.57+deb13-amd64 (SMP w/64 CPU threads; PREEMPT)
Locale: LANG=en_DE.UTF-8, LC_CTYPE=en_DE.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages ifupdown depends on:
ii adduser 3.152
ii iproute2 6.15.0-1
ii libc6 2.41-12
Versions of packages ifupdown recommends:
pn dhcpcd-base | dhcp-client <none>
Versions of packages ifupdown suggests:
pn ppp <none>
pn rdnssd <none>
-- no debconf information
-- debsums errors found:
debsums: changed file /usr/lib/systemd/system/networking.service (from ifupdown
package)
[that's only because of my debug changes, see above]
