To confirm that this isn't shiftfs related and that we were just causing
the issue to be hidden, I've run the same test on OpenSuse tumbleweed.
I chose that distro because it's apparmor-enabled, has snapd and a 5.4
kernel.
```
localhost:~ # snap install docker
docker 18.09.9 from Canonical* installed
localhost:~ # auth_folder=/var/snap/docker/common/auth
localhost:~ # mkdir -p $auth_folder
localhost:~ # docker run --entrypoint htpasswd registry:2 -Bbn user passwd >
$auth_folder/htpasswd
Unable to find image 'registry:2' locally
2: Pulling from library/registry
486039affc0a: Pulling fs layer
ba51a3b098e6: Pulling fs layer
8bb4c43d6c8e: Pulling fs layer
6f5f453e5f2d: Pulling fs layer
42bc10b72f42: Pulling fs layer
6f5f453e5f2d: Waiting
42bc10b72f42: Waiting
ba51a3b098e6: Download complete
486039affc0a: Verifying Checksum
486039affc0a: Download complete
8bb4c43d6c8e: Verifying Checksum
8bb4c43d6c8e: Download complete
6f5f453e5f2d: Verifying Checksum
6f5f453e5f2d: Download complete
42bc10b72f42: Verifying Checksum
42bc10b72f42: Download complete
486039affc0a: Pull complete
ba51a3b098e6: Pull complete
8bb4c43d6c8e: Pull complete
6f5f453e5f2d: Pull complete
42bc10b72f42: Pull complete
Digest: sha256:7d081088e4bfd632a88e3f3bcd9e007ef44a796fddfe3261407a3f9f04abe1e7
Status: Downloaded newer image for registry:2
localhost:~ # docker run -d -p 5000:5000 --restart=always --name registry \
> -v $auth_folder:/auth \
> -e "REGISTRY_AUTH=htpasswd" \
> -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
> -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
> registry:2
cba1ec94734a8a198fa0c474d9873233958fad6cdafe93d2ccf4d701ecab55ff
localhost:~ # docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
cba1ec94734a registry:2 "/entrypoint.sh /etc…" 5 seconds ago
Restarting (2) Less than a second ago registry
localhost:~ # uname -a
Linux localhost 5.4.10-1-default #1 SMP Thu Jan 9 15:45:45 UTC 2020 (556a6fe)
x86_64 x86_64 x86_64 GNU/Linux
localhost:~ #
```
As you can see, the exact same thing happen there. So this is an
apparmor kernel bug or some issue with the snapd or docker snap, this
isn't a shiftfs bug and reverting the change would just expose a
different bug rather than actually fix things.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1879690
Title:
Docker registry doesn't stay up and keeps restarting
Status in linux package in Ubuntu:
Incomplete
Status in linux source package in Eoan:
Fix Committed
Status in linux source package in Focal:
Fix Committed
Bug description:
[Impact]
The change applied for bug 1857257 and its followup fix bug 1876645, which
were released on focal and eoan -updates, introduced a regression on overlayfs,
breaking docker snap.
[Test case]
See original bug report.
[Fix]
While we don't have a final fix the solution for now is to revert the
following commits:
UBUNTU: SAUCE: overlayfs: fix shitfs special-casing
UBUNTU: SAUCE: overlayfs: use shiftfs hacks only with shiftfs as underlay
[Regression potential]
Low. Reverting these two commits will introduce back the issue reported on
bug 1857257, but will fix the other use cases which was broken by the latest
release.
Original bug report.
-----------------------------------
Tested kernels:
Focal 5.4.0-31.35
Eoan 5.3.0-53.47
To reproduce:
1) Spin up a cloud image
2) snap install docker
3) auth_folder=/var/snap/docker/common/auth
4) mkdir -p $auth_folder
5) docker run --entrypoint htpasswd registry:2 -Bbn user passwd >
$auth_folder/htpasswd
6) docker run -d -p 5000:5000 --restart=always --name registry \
-v $auth_folder:/auth \
-e "REGISTRY_AUTH=htpasswd" \
-e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
-e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
registry:2
On a good kernel 'docker ps' shows something like:
# docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
a346b65b4509 registry:2 "/entrypoint.sh /etc…" 14 seconds
ago Up 12 seconds 0.0.0.0:5000->5000/tcp registry
On a bad kernel:
docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
0322374f1b1d registry:2 "/entrypoint.sh /etc…" 5 seconds
ago Restarting (2) 1 second ago registry
Note status 'Restarting' on the bad kernel.
This seems to be introduce by any of the following commits:
b3bdda24f1bc UBUNTU: SAUCE: overlayfs: fix shitfs special-casing
6f18a8434050 UBUNTU: SAUCE: overlayfs: use shiftfs hacks only with shiftfs as
underlay
629edd70891c UBUNTU: SAUCE: shiftfs: record correct creator credentials
cfaa482afb97 UBUNTU: SAUCE: shiftfs: fix dentry revalidation
Kernels that don't have these commits seem fine.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1879690/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
