Performing verification for focal.

This will cover both the kernel and the cifs-utils package. both in
-proposed.

This is going to be long, as we need to test:
* patched kernel, patched cifs-utils
* patched kernel, existing cifs-utils
* existing kernel, patched cifs-utils

I started a fresh focal VM, with:

kernel 5.4.0-216-generic from -updates
cifs-utils 2:6.9-1ubuntu0.2 from -updates

I then followed the instructions to about step 34.

root@focal-dc:/home/ubuntu# kinit administra...@samba-dc.example.com
Password for administra...@samba-dc.example.com: 
Warning: Your password will expire in 16 days on Wed Jun 11 06:13:06 2025
root@focal-dc:/home/ubuntu# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: administra...@samba-dc.example.com

Valid starting     Expires            Service principal
05/25/25 06:26:20  05/25/25 16:26:20  
krbtgt/samba-dc.example....@samba-dc.example.com
        renew until 05/26/25 06:26:17
root@focal-dc:/home/ubuntu# mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0 
//samba-dc.example.com/demo /mnt/testshare1
root@focal-dc:/home/ubuntu# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: administra...@samba-dc.example.com

Valid starting     Expires            Service principal
05/25/25 06:26:20  05/25/25 16:26:20  
krbtgt/samba-dc.example....@samba-dc.example.com
        renew until 05/26/25 06:26:17
05/25/25 06:26:31  05/25/25 16:26:20  cifs/samba-dc.example.com@
        renew until 05/26/25 06:26:17
05/25/25 06:26:31  05/25/25 16:26:20  
cifs/samba-dc.example....@samba-dc.example.com
        renew until 05/26/25 06:26:17
# journalctl -b0
kernel: Key type cifs.spnego registered
kernel: Key type cifs.idmap registered
kernel: CIFS: Attempting to mount //samba-dc.example.com/demo
kernel: No dialect specified on mount. Default has changed to a more secure 
dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the >
cifs.upcall[2085]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.27;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x80f
cifs.upcall[2086]: ver=2
cifs.upcall[2086]: host=samba-dc.example.com
cifs.upcall[2086]: ip=192.168.122.27
cifs.upcall[2086]: sec=1
cifs.upcall[2086]: uid=0
cifs.upcall[2086]: creduid=0
cifs.upcall[2086]: user=root
cifs.upcall[2086]: pid=2063
cifs.upcall[2085]: get_cachename_from_process_env: pid == 0
cifs.upcall[2085]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
cifs.upcall[2085]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
cifs.upcall[2085]: handle_krb5_mech: obtained service ticket
cifs.upcall[2085]: Exit status 0

root@focal-dc:/home/ubuntu# stat /mnt/testshare1
  File: /mnt/testshare1
  Size: 0               Blocks: 0          IO Block: 1048576 directory
Device: 35h/53d Inode: 260995      Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-04-30 06:16:07.320000000 +0000
Modify: 2025-04-30 06:16:07.320000000 +0000
Change: 2025-04-30 06:16:07.320000000 +0000
 Birth: -
 
# docker run -it -v /mnt/testshare1:/mnt/shared --name cifstest ubuntu:20.04 
/bin/bash
root@focal-dc:/home/ubuntu# docker start 2f1a7761412a
2f1a7761412a
root@focal-dc:/home/ubuntu# docker exec -it 2f1a7761412a bash
root@2f1a7761412a:/# stat /mnt/shared
  File: /mnt/shared
  Size: 0               Blocks: 0          IO Block: 1048576 directory
Device: 35h/53d Inode: 260995      Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-04-30 18:16:07.320000000 +1200
Modify: 2025-04-30 18:16:07.320000000 +1200
Change: 2025-04-30 18:16:07.320000000 +1200
 Birth: -

root@fcec5b069772:/# vim /etc/krb5.conf
        default_ccache_name = /tmp/krb5cc_00%{uid}

Now back on the host:

root@focal-dc:/home/ubuntu# kdestroy -c /tmp/krb5cc_0
root@focal-dc:/home/ubuntu# ss -K dport 445
Netid          State          Recv-Q           Send-Q                      
Local Address:Port                        Peer Address:Port                  
Process          
tcp            ESTAB          0                0                          
192.168.122.27:36352                     192.168.122.27:microsoft-ds            
  

On the docker container:

root@fcec5b069772:/# stat /mnt/shared
stat: cannot statx '/mnt/shared': Required key not available

On the host:
# journalctl -f
cifs.upcall[2305]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.27;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x8ff
cifs.upcall[2306]: ver=2
cifs.upcall[2306]: host=samba-dc.example.com
cifs.upcall[2306]: ip=192.168.122.27
cifs.upcall[2306]: sec=1
cifs.upcall[2306]: uid=0
cifs.upcall[2306]: creduid=0
cifs.upcall[2306]: user=root
cifs.upcall[2306]: pid=2303
cifs.upcall[2305]: get_cachename_from_process_env: pid == 0
cifs.upcall[2305]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_000
cifs.upcall[2305]: get_tgt_time: unable to get principal
cifs.upcall[2305]: krb5_get_init_creds_keytab: -1765328228
cifs.upcall[2305]: Exit status 1
kernel: CIFS VFS: Verify user has a krb5 ticket and keyutils is installed
kernel: CIFS VFS: \\samba-dc.example.com Send error in SessSetup = -126

Note the line:
get_existing_cc: default ccache is FILE:/tmp/krb5cc_000

In this case, cifs.upcall tried to use the kerberos credential cache from the
container namespace, instead of the host namespace where the mount was actually
first mounted in.

So we can reproduce the issue.

Next we will test:
* patched kernel, patched cifs-utils
====================================

kernel: 5.4.0-218-generic from -proposed
cifs-utils 2:6.9-1ubuntu0.3 from -security-proposed

First, we make sure existing behaviour is maintained with the default
option:

root@focal-dc:/home/ubuntu# mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0 
//samba-dc.example.com/demo /mnt/testshare1
root@focal-dc:/home/ubuntu# stat /mnt/testshare1
  File: /mnt/testshare1
  Size: 0               Blocks: 0          IO Block: 1048576 directory
Device: 36h/54d Inode: 260995      Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-04-30 06:16:07.320000000 +0000
Modify: 2025-04-30 06:16:07.320000000 +0000
Change: 2025-04-30 06:16:07.320000000 +0000
 Birth: -
# journalctl -b0
kernel: CIFS: Attempting to mount //samba-dc.example.com/demo
kernel: No dialect specified on mount. Default has changed to a more secure 
dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the >
cifs.upcall[1908]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.27;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x770;upcall_target=app
cifs.upcall[1909]: ver=2
cifs.upcall[1909]: host=samba-dc.example.com
cifs.upcall[1909]: ip=192.168.122.27
cifs.upcall[1909]: sec=1
cifs.upcall[1909]: uid=0
cifs.upcall[1909]: creduid=0
cifs.upcall[1909]: user=root
cifs.upcall[1909]: pid=1904
cifs.upcall[1909]: upcall_target=app
cifs.upcall[1908]: upcall_target=app, switching namespaces to application thread
cifs.upcall[1908]: get_cachename_from_process_env: pid == 0
cifs.upcall[1908]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
cifs.upcall[1908]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
cifs.upcall[1908]: handle_krb5_mech: obtained service ticket
cifs.upcall[1908]: Exit status 0

Note, upcall_target=app is enabled by default.

In the docker container:
root@focal-dc:/home/ubuntu# docker start 2f1a7761412a
2f1a7761412a
root@focal-dc:/home/ubuntu# docker exec -it 2f1a7761412a bash
root@2f1a7761412a:/# stat /mnt/shared
  File: /mnt/shared
  Size: 0               Blocks: 0          IO Block: 1048576 directory
Device: 36h/54d Inode: 260995      Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-04-30 18:16:07.320000000 +1200
Modify: 2025-04-30 18:16:07.320000000 +1200
Change: 2025-04-30 18:16:07.320000000 +1200
 Birth: -
 
On the host:
root@focal-dc:/home/ubuntu# kdestroy -c /tmp/krb5cc_0
root@focal-dc:/home/ubuntu# ss -K dport 445
Netid          State          Recv-Q           Send-Q                      
Local Address:Port                        Peer Address:Port                  
Process          
tcp            ESTAB          0                0                          
192.168.122.27:54162                     192.168.122.27:microsoft-ds            
       

On the container:
root@2f1a7761412a:/# stat /mnt/shared
stat: cannot statx '/mnt/shared': Required key not available

On the host:

# journalctl -f
cifs.upcall[2043]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.27;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x7fa;upcall_target=app
cifs.upcall[2044]: ver=2
cifs.upcall[2044]: host=samba-dc.example.com
cifs.upcall[2044]: ip=192.168.122.27
cifs.upcall[2044]: sec=1
cifs.upcall[2044]: uid=0
cifs.upcall[2044]: creduid=0
cifs.upcall[2044]: user=root
cifs.upcall[2044]: pid=2042
cifs.upcall[2044]: upcall_target=app
cifs.upcall[2043]: upcall_target=app, switching namespaces to application thread
cifs.upcall[2043]: get_cachename_from_process_env: pid == 0
cifs.upcall[2043]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_000
cifs.upcall[2043]: check_service_ticket_exists: unable to get client principal 
from cache: No credentials cache found (filename: /tmp/krb5cc_000)
cifs.upcall[2043]: get_tgt_time: unable to get principal
cifs.upcall[2043]: main: valid TGT is not present in credential cache
cifs.upcall[2043]: krb5_get_init_creds_keytab: -1765328228
cifs.upcall[2043]: Exit status 1
kernel: CIFS VFS: Verify user has a krb5 ticket and keyutils is installed
kernel: CIFS VFS: \\samba-dc.example.com Send error in SessSetup = -126

We still have existing behaviour by default:
cifs.upcall[2043]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_000
and
cifs.upcall[2043]: check_service_ticket_exists: unable to get client principal 
from cache: No credentials cache found (filename: /tmp/krb5cc_000)
which means we will not cause any regressions.

Next, we will use the new mount option, "upcall_target=mount":

# mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0,upcall_target=mount
 //samba-dc.example.com/demo /mnt/testshare1
# journalctl -b0
kernel: CIFS: Attempting to mount //samba-dc.example.com/demo
kernel: No dialect specified on mount. Default has changed to a more secure 
dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the >
cifs.upcall[2159]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.27;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x86b;upcall_target=mount
cifs.upcall[2160]: ver=2
cifs.upcall[2160]: host=samba-dc.example.com
cifs.upcall[2160]: ip=192.168.122.27
cifs.upcall[2160]: sec=1
cifs.upcall[2160]: uid=0
cifs.upcall[2160]: creduid=0
cifs.upcall[2160]: user=root
cifs.upcall[2160]: pid=2155
cifs.upcall[2160]: upcall_target=mount
cifs.upcall[2159]: upcall_target=mount, not switching namespaces to application 
thread
cifs.upcall[2159]: get_cachename_from_process_env: pid == 0
cifs.upcall[2159]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
cifs.upcall[2159]: main: valid service ticket exists in credential cache
cifs.upcall[2159]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
cifs.upcall[2159]: handle_krb5_mech: obtained service ticket
cifs.upcall[2159]: Exit status 0

Now we have upcall_target=mount set.

In the container:
root@focal-dc:/home/ubuntu# docker start 2f1a7761412a
2f1a7761412a
root@focal-dc:/home/ubuntu# docker exec -it 2f1a7761412a bash
root@2f1a7761412a:/# stat /mnt/shared
  File: /mnt/shared
  Size: 0               Blocks: 0          IO Block: 1048576 directory
Device: 36h/54d Inode: 260995      Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-04-30 18:16:07.320000000 +1200
Modify: 2025-04-30 18:16:07.320000000 +1200
Change: 2025-04-30 18:16:07.320000000 +1200
 Birth: -
 
On the host:
root@focal-dc:/home/ubuntu# kdestroy -c /tmp/krb5cc_0
root@focal-dc:/home/ubuntu# ss -K dport 445
Netid          State          Recv-Q           Send-Q                      
Local Address:Port                        Peer Address:Port                  
Process          
tcp            ESTAB          0                0                          
192.168.122.27:45856                     192.168.122.27:microsoft-ds            
         

On the container:
root@2f1a7761412a:/# stat /mnt/shared
stat: cannot stat '/mnt/shared': Required key not available

On the host:
# journalctl -f
cifs.upcall[2301]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.27;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x8fc;upcall_target=mount
cifs.upcall[2302]: ver=2
cifs.upcall[2302]: host=samba-dc.example.com
cifs.upcall[2302]: ip=192.168.122.27
cifs.upcall[2302]: sec=1
cifs.upcall[2302]: uid=0
cifs.upcall[2302]: creduid=0
cifs.upcall[2302]: user=root
cifs.upcall[2302]: pid=2300
cifs.upcall[2302]: upcall_target=mount
cifs.upcall[2301]: upcall_target=mount, not switching namespaces to application 
thread
cifs.upcall[2301]: get_cachename_from_process_env: pid == 0
cifs.upcall[2301]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
cifs.upcall[2301]: check_service_ticket_exists: unable to get client principal 
from cache: No credentials cache found (filename: /tmp/krb5cc_0)
cifs.upcall[2301]: get_tgt_time: unable to get principal
cifs.upcall[2301]: main: valid TGT is not present in credential cache
cifs.upcall[2301]: krb5_get_init_creds_keytab: -1765328378
cifs.upcall[2301]: Exit status 1
kernel: CIFS VFS: Verify user has a krb5 ticket and keyutils is installed
kernel: CIFS VFS: \\samba-dc.example.com Send error in SessSetup = -126

Now that "mount" is used, we correctly only use the credential cache from the
host namespace, which is FILE:/tmp/krb5cc_0, and we don't leak any data between
the container or the host, fixing the security issue.

Next we will test:
* patched kernel, existing cifs-utils
=====================================

kernel: 5.4.0-218-generic from -proposed
cifs-utils: 2:6.9-1ubuntu0.2 from -updates

If we were to manually specify upcall_target=app on the mount command
line:

root@samba-dc:/home/ubuntu# mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0,upcall_target=app 
//samba-dc.example.com/demo /mnt/testshare1
# journalctl -f
kernel: Key type cifs.spnego registered
kernel: Key type cifs.idmap registered
kernel: CIFS: Attempting to mount //samba-dc.example.com/demo
kernel: No dialect specified on mount. Default has changed to a more secure 
dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the >
cifs.upcall[1565]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.27;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x60a;upcall_target=app
cifs.upcall[1566]: ver=2
cifs.upcall[1566]: host=samba-dc.example.com
cifs.upcall[1566]: ip=192.168.122.27
cifs.upcall[1566]: sec=1
cifs.upcall[1566]: uid=0
cifs.upcall[1566]: creduid=0
cifs.upcall[1566]: user=root
cifs.upcall[1566]: pid=1546
cifs.upcall[1565]: get_cachename_from_process_env: pid == 0
cifs.upcall[1565]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
cifs.upcall[1565]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
cifs.upcall[1565]: handle_krb5_mech: obtained service ticket
cifs.upcall[1565]: Exit status 0

The mount continues successfully. The existing cifs-utils does not complain
about it.

Test with no "upcall_target". e.g.:

# mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0 
//samba-dc.example.com/demo /mnt/testshare1
# journalctl -b0
kernel: CIFS: Attempting to mount //samba-dc.example.com/demo
kernel: No dialect specified on mount. Default has changed to a more secure 
dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the >
cifs.upcall[1579]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.27;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x627;upcall_target=app
cifs.upcall[1580]: ver=2
cifs.upcall[1580]: host=samba-dc.example.com
cifs.upcall[1580]: ip=192.168.122.27
cifs.upcall[1580]: sec=1
cifs.upcall[1580]: uid=0
cifs.upcall[1580]: creduid=0
cifs.upcall[1580]: user=root
cifs.upcall[1580]: pid=1575
cifs.upcall[1579]: get_cachename_from_process_env: pid == 0
cifs.upcall[1579]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
cifs.upcall[1579]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
cifs.upcall[1579]: handle_krb5_mech: obtained service ticket
cifs.upcall[1579]: Exit status 0

Note, that we still see "upcall_target=app" appended to key description even
though this is with existing cifs-utils. The mount still succeeds as normal.

Next we will test:
* existing kernel, patched cifs-utils
=====================================

kernel 5.4.0-216-generic from -updates
cifs-utils 2:6.9-1ubuntu0.3 from -security-proposed

If we were to manually specify upcall_target=app on the mount command
line:

# mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0,upcall_target=app 
//samba-dc.example.com/demo /mnt/testshare1
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log 
messages (dmesg)

# journalctl -b0
kernel: CIFS: Unknown mount option "upcall_target=app"

cifs-utils accepts it, but the kernel does not, and fails with an unknown
parameter. This is okay, because the existing kernel does not have support for
the new parameter, and users would be manually be adding it to their mount
parameters anyway. If they went to this effort, they would notice their kernel
is out of date and would upgrade their kernel to get support.

Testing with no "upcall_target":

# mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0 
//samba-dc.example.com/demo /mnt/testshare1
# mount -l
//samba-dc.example.com/demo on /mnt/testshare1 type cifs 
(rw,relatime,vers=3.1.1,sec=krb5i,cruid=0,cache=strict,username=root,uid=0,forceuid,gid=0,forcegid,addr=192.168.122.27,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,user=root)
# journalctl -f
kernel: CIFS: Attempting to mount //samba-dc.example.com/demo
kernel: No dialect specified on mount. Default has changed to a more secure 
dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the >
cifs.upcall[2699]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.27;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0xa85
cifs.upcall[2700]: ver=2
cifs.upcall[2700]: host=samba-dc.example.com
cifs.upcall[2700]: ip=192.168.122.27
cifs.upcall[2700]: sec=1
cifs.upcall[2700]: uid=0
cifs.upcall[2700]: creduid=0
cifs.upcall[2700]: user=root
cifs.upcall[2700]: pid=2693
cifs.upcall[2699]: upcall_target=app, switching namespaces to application thread
cifs.upcall[2699]: get_cachename_from_process_env: pid == 0
cifs.upcall[2699]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
cifs.upcall[2699]: main: valid service ticket exists in credential cache
cifs.upcall[2699]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
cifs.upcall[2699]: handle_krb5_mech: obtained service ticket
cifs.upcall[2699]: Exit status 0

Mounting shares without any additional mount parameters still works correctly.
cifs-utils does not append upcall_target=app to the mount command line, so
existing kernel continues to work correctly. Note, it does now mention:

cifs.upcall[2699]: upcall_target=app, switching namespaces to
application thread

to say that it uses the current process namespace, which is the same as
existing behaviour.

We have covered all situations of the testing matrix. Both the kernel in
-proposed and cifs-utils in -security-proposed fix the issue, and don't cause
any issues being installed independently with or without each other.

Happy to mark verified for focal.

** Tags removed: verification-needed-focal-linux
** Tags added: verification-done-focal verification-done-focal-linux

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/2099914

Title:
  CVE-2025-2312 cifs.upcall could access incorrect kerberos credentials
  cache

Status in cifs-utils package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in cifs-utils source package in Focal:
  In Progress
Status in linux source package in Focal:
  Fix Committed
Status in cifs-utils source package in Jammy:
  In Progress
Status in linux source package in Jammy:
  Fix Committed
Status in cifs-utils source package in Noble:
  In Progress
Status in linux source package in Noble:
  Fix Committed
Status in cifs-utils source package in Oracular:
  In Progress
Status in linux source package in Oracular:
  Fix Committed
Status in cifs-utils source package in Plucky:
  Fix Released
Status in linux source package in Plucky:
  Fix Released

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/2099914

  [Impact]

  This is CVE-2025-2312, where namespace confusion by cifs.upcall may lead to
  disclosing sensitive data from the host or container Kerberos credentials 
cache
  by accessing the wrong credential cache that doesn't belong to the current 
user.

  Consider the following scenario:

  A CIFS/SMB file share is mounted on a host node using Kerberos
  authentication.

  During the session setup phase, the Linux kernel's cifs.ko module makes an
  upcall to user space to retrieve the Kerberos service ticket from the 
credential
  cache.

  In typical (non-container) environments, this process works correctly, but in
  containerized environments, the upcall may be directed to a different 
namespace
  than intended, leading to issues. For example:

  a) The file share is mounted on the host node at /mnt/testshare1, meaning the
  Kerberos credential cache is stored in the host's namespace.
  b) A Docker container is created, and the file share path /mnt/testshare1 is
  exported to the container at /sharedpath.
  c) When the service ticket expires and the SMB connection is lost, before the
  ticket is refreshed in the credential cache, an application inside the 
container
  performs a file operation. This triggers the kernel to attempt a session
  reconnect.
  d) During the session setup, a Kerberos ticket is needed, so the kernel 
invokes
  the cifs.upcall binary using the request_key function. However, cifs.upcall
  switches to the namespace of the caller (i.e., the container), causing it to
  attempt to read the credential cache from the container's namespace. But since
  the original mount happened in the host namespace, the credential cache is
  located on the host, not in the container. This results in the upcall failing
  to access the correct credential cache or accessinng credential cache which
  doesn't belong to correct user.

  [Fix]

  The fix adds a "upcall_target" mount parameter that needs to be present in 
both
  the kernel and cifs-utils. "upcall_target" specifies what namespace to find 
the
  kerberos credential cache, and takes options "mount" being the host namespace,
  or "app", being the container namespace. The language is intended to suit
  Kubernetes based usecases.

  The kernel requires the following commit:

  commit db363b0a1d9e6b9dc556296f1b1007aeb496a8cf
  Author: Ritvik Budhiraja <rbudhir...@microsoft.com>
  Date:  Mon Nov 11 11:43:51 2024 +0000
  Subject: CIFS: New mount option for cifs.upcall namespace resolution
  Link: 
https://github.com/torvalds/linux/commit/db363b0a1d9e6b9dc556296f1b1007aeb496a8cf

  This landed in 6.13 mainline, and is already in plucky. Oracular is a clean
  cherry pick, noble and jammy requires a context adjustment backport and focal
  needed a heavy backport.

  Test packages are available in the following ppa:

  https://launchpad.net/~vpeixoto/+archive/ubuntu/cifs-backport

  In addition, a userspace fix is also needed in cifs-utils, with the following
  commits:

  commit 89b679228cc1be9739d54203d28289b03352c174
  From: Ritvik Budhiraja <rbudhir...@microsoft.com>
  Date: Tue, 19 Nov 2024 06:07:58 +0000
  Subject: CIFS.upcall to accomodate new namespace mount opt
  Link: 
https://git.samba.org/?p=cifs-utils.git;a=commit;h=89b679228cc1be9739d54203d28289b03352c174

  commit cf63240489431e98033e599a7c9437b59494a2e4
  From: Ritvik Budhiraja <rbudhir...@microsoft.com>
  Date: Thu, 30 Jan 2025 14:13:10 +0000
  Subject: cifs-utils: add documentation for upcall_target
  Link: 
https://git.samba.org/?p=cifs-utils.git;a=commit;h=cf63240489431e98033e599a7c9437b59494a2e4

  These were a part of 7.2 upstream. Plucky already has this release, so we just
  need to fix oracular, noble, jammy and focal.

  Test packages are available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf407276-test

  If you install the test packages, you can now use the upcall_target argument
  with either "mount" or "app" options.

  [Testcase]

  Some knowledge of kerberos will go a long way to help you make this
  all work.

  We should be able to do all testing on the same VM.

  1) Create a fresh VM
  2) sudo apt update
  3) sudo apt upgrade
  4) sudo hostnamectl set-hostname samba-dc
  5) sudo vim /etc/hosts
  Add an entry with its IP address, e.g.:
  192.168.122.124 samba-dc samba-dc.example.com
  6) sudo apt install -y samba smbclient winbind libpam-winbind libnss-winbind 
krb5-kdc libpam-krb5 cifs-utils
  Focal:
  sudo apt install keyutils
  Oracular:
  sudo apt install samba-ad-dc
  Note: skip config of kerberos KDC.
  7) sudo rm /etc/krb5.conf
  8) sudo rm /etc/samba/smb.conf
  9) sudo samba-tool domain provision --server-role=dc --use-rfc2307 
--dns-backend=SAMBA_INTERNAL --realm=samba-dc.EXAMPLE.COM --domain=SAMBA 
--adminpass=Password1
  10) sudo cp /var/lib/samba/private/krb5.conf /etc/krb5.conf
  11) sudo systemctl mask smbd nmbd winbind
  12) sudo systemctl disable smbd nmbd winbind
  13) sudo systemctl stop smbd nmbd winbind
  14) sudo systemctl unmask samba-ad-dc
  15) sudo systemctl start samba-ad-dc
  16) sudo systemctl enable samba-ad-dc
  17) sudo reboot
  18) sudo systemctl stop systemd-resolved
  19) sudo systemctl disable systemd-resolved
  20) cat << EOF >> /etc/resolv.conf
  nameserver 192.168.122.124
  search SAMBA
  EOF
  sudo vim /etc/samba/smb.conf
  Change forwarder to 8.8.8.8
  21) sudo reboot
  22) host -t SRV _ldap._tcp.samba-dc.example.com
  _ldap._tcp.samba-dc.example.com has SRV record 0 100 389 
samba-dc.samba-dc.example.com.
  23) $ smbclient -L localhost -N
  Anonymous login successful

   Sharename Type Comment
   --------- ---- -------
   sysvol Disk
   netlogon Disk
   IPC$ IPC IPC Service (Samba 4.13.17-Ubuntu)
  SMB1 disabled -- no workgroup available
  24) $ smbclient //localhost/netlogon -UAdministrator -c 'ls'
  Enter SAMBA\Administrator's password:
    . D 0 Mon Feb 28 04:23:22 2022
    .. D 0 Mon Feb 28 04:23:27 2022

    9983232 blocks of size 1024. 7995324 blocks available
  25) kinit administrator
  Password for administra...@samba-dc.example.com:
  Warning: Your password will expire in 41 days on Wed May 21 02:51:02 2025
  26) klist
  Ticket cache: FILE:/tmp/krb5cc_1000
  Default principal: administra...@samba-dc.example.com

  Valid starting     Expires            Service principal
  04/09/25 02:53:27  04/09/25 12:53:27  
krbtgt/samba-dc.example....@samba-dc.example.com
   renew until 04/10/25 02:53:22
  27) Create a share:
  28) sudo mkdir -p /srv/samba/Demo/
  29) sudo vim /etc/samba/smb.conf
  [Demo]
   path = /srv/samba/Demo/
   read only = no
  30) sudo chmod 0770 /srv/samba/Demo/
  31) smbclient -U Administrator //samba-dc.example.com/demo
  Password for [SAMBA\Administrator]:
  Try "help" to get a list of possible commands.
  smb: \>
  32) smbclient -U Administrator --use-krb5-ccache=/tmp/krb5cc_1000 
//samba-dc.example.com/demo
  Try "help" to get a list of possible commands.
  smb: \>
  33) klist
  Ticket cache: FILE:/tmp/krb5cc_1000
  Default principal: administra...@samba-dc.example.com

  Valid starting     Expires            Service principal
  04/09/25 02:53:27  04/09/25 12:53:27  
krbtgt/samba-dc.example....@samba-dc.example.com
   renew until 04/10/25 02:53:22
  04/09/25 02:58:16  04/09/25 12:53:27  cifs/samba-dc.example....@example.com
   renew until 04/10/25 02:53:22
   Ticket server: cifs/samba-dc.example....@samba-dc.example.com
  04/09/25 02:58:16  04/09/25 12:53:27  
cifs/samba-dc.example....@samba-dc.example.com
   renew until 04/10/25 02:53:22
  34) sudo -s
  35) # kinit administra...@samba-dc.example.com
  Password for administra...@samba-dc.example.com:
  Warning: Your password will expire in 41 days on Wed May 21 02:51:02 2025
  # klist
  Ticket cache: FILE:/tmp/krb5cc_0
  Default principal: administra...@samba-dc.example.com

  Valid starting     Expires            Service principal
  04/09/25 03:26:10  04/09/25 13:26:10  
krbtgt/samba-dc.example....@samba-dc.example.com
   renew until 04/10/25 03:26:06
  36) # mkdir /mnt/testshare1
  # mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0 
//samba-dc.example.com/demo /mnt/testshare1
  37) # klist
  Ticket cache: FILE:/tmp/krb5cc_0
  Default principal: administra...@samba-dc.example.com

  Valid starting     Expires            Service principal
  04/09/25 03:26:10  04/09/25 13:26:10  
krbtgt/samba-dc.example....@samba-dc.example.com
   renew until 04/10/25 03:26:06
  04/09/25 03:30:26  04/09/25 13:26:10  cifs/samba-dc.example.com@
   renew until 04/10/25 03:26:06
   Ticket server: cifs/samba-dc.example....@samba-dc.example.com
  38) journalctl
  kernel: netfs: FS-Cache loaded
  kernel: Key type cifs.spnego registered
  kernel: Key type cifs.idmap registered
  kernel: CIFS: No dialect specified on mount. Default has changed to a more 
secure dialect, SMB2.1 or later (e.g. SMB3.1.1), from CIFS (SMB1). T>
  kernel: CIFS: enabling forceuid mount option implicitly because uid= option 
is specified
  kernel: CIFS: enabling forcegid mount option implicitly because gid= option 
is specified
  kernel: CIFS: Attempting to mount //samba-dc.example.com/demo
  cifs.upcall[1805]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.124;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x6ee
  cifs.upcall[1806]: ver=2
  cifs.upcall[1806]: host=samba-dc.example.com
  cifs.upcall[1806]: ip=192.168.122.124
  cifs.upcall[1806]: sec=1
  cifs.upcall[1806]: uid=0
  cifs.upcall[1806]: creduid=0
  cifs.upcall[1806]: user=root
  cifs.upcall[1806]: pid=1774
  cifs.upcall[1805]: get_cachename_from_process_env: pid == 0
  cifs.upcall[1805]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
  cifs.upcall[1805]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
  cifs.upcall[1805]: handle_krb5_mech: using native krb5
  cifs.upcall[1805]: handle_krb5_mech: obtained service ticket
  cifs.upcall[1805]: Exit status 0

  Take note of the line:
  get_existing_cc: default ccache is FILE:/tmp/krb5cc_0

  39) # stat /mnt/testshare1
    File: /mnt/testshare1
    Size: 0             Blocks: 0          IO Block: 1048576 directory
  Device: 0,41  Inode: 297860      Links: 2
  Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
  Access: 2025-04-09 02:54:45.264000000 +0000
  Modify: 2025-04-09 02:54:45.264000000 +0000
  Change: 2025-04-09 02:54:45.264000000 +0000
   Birth: 2025-04-09 02:54:45.264000000 +0000
  40) sudo apt install docker.io
  41) docker pull ubuntu:24.04
  42) docker run -it -v /mnt/testshare1:/mnt/shared --name cifstest 
ubuntu:24.04 /bin/bash
  43) root@685c7e420afc:/# stat /mnt/shared
    File: /mnt/shared
    Size: 0             Blocks: 0          IO Block: 1048576 directory
  Device: 0,41  Inode: 297860      Links: 2
  Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
  Access: 2025-04-09 02:54:45.264000000 +0000
  Modify: 2025-04-09 02:54:45.264000000 +0000
  Change: 2025-04-09 02:54:45.264000000 +0000
   Birth: 2025-04-09 02:54:45.264000000 +0000
  root@685c7e420afc:/# ls /mnt/shared
  44) root@685c7e420afc:/# apt install krb5-user vim
  45) root@685c7e420afc:/# vim /etc/krb5.conf
  Under libdefaults, add default_ccache_name = /tmp/krb5cc_00%{uid} save and 
exit.
  46) Back on the host in root, clear initial kerberos crediental cache and 
disconnect cifs connections.
  # kdestroy -c /tmp/krb5cc_0
  # ss -K dport 445
  47) Back in the container:
  root@685c7e420afc:/# stat /mnt/shared
  48) Back on the host in root:
  # journalctl
  kernel: CIFS: VFS: Verify user has a krb5 ticket and keyutils is installed
  kernel: CIFS: VFS: \\samba-dc.example.com Send error in SessSetup = -126
  cifs.upcall[2804]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.124;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0xaf3
  cifs.upcall[2805]: ver=2
  cifs.upcall[2805]: host=samba-dc.example.com
  cifs.upcall[2805]: ip=192.168.122.124
  cifs.upcall[2805]: sec=1
  cifs.upcall[2805]: uid=0
  cifs.upcall[2805]: creduid=0
  cifs.upcall[2805]: user=root
  cifs.upcall[2805]: pid=2803
  cifs.upcall[2804]: get_cachename_from_process_env: pid == 0
  cifs.upcall[2804]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_000
  cifs.upcall[2804]: get_tgt_time: unable to get principal
  cifs.upcall[2804]: krb5_get_init_creds_keytab: -1765328378
  cifs.upcall[2804]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
  cifs.upcall[2804]: handle_krb5_mech: using GSS-API
  cifs.upcall[2804]: GSS-API error init_sec_context: No credentials were 
supplied, or the credentials were unavailable or inaccessible
  cifs.upcall[2804]: GSS-API error init_sec_context: No Kerberos credentials 
available (default cache: /tmp/krb5cc_000)
  cifs.upcall[2804]: handle_krb5_mech: failed to obtain service ticket via GSS 
(458752)
  cifs.upcall[2804]: Unable to obtain service ticket
  cifs.upcall[2804]: Exit status 458752

  Note that it now tries to read /tmp/krb5cc_000 from container namespace 
instead
  of /tmp/krb5cc_0 from host namespace.

  If you install the test packages from the following ppas:

  https://launchpad.net/~vpeixoto/+archive/ubuntu/cifs-backport
  https://launchpad.net/~mruffell/+archive/ubuntu/sf407276-test

  When you initially mount the cifs filesystem, use the new mount option
  upcall_target=mount.

  # mount -t cifs -o
  
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0,upcall_target=mount
  //samba-dc.example.com/demo /mnt/testshare1

  Repeat the testcase. When we disconnect the cifs connection and try stat 
inside
  the container, the kerberos crediental cache should be /tmp/krb5cc_0 in the
  host namespace.

  get_existing_cc: default ccache is FILE:/tmp/krb5cc_0

  A successful run with upcall_target=mount and fixed cifs-utils should
  look like:

  cifs.upcall[2122]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.124;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x849;upcall_target=mount
  cifs.upcall[2123]: ver=2
  cifs.upcall[2123]: host=samba-dc.example.com
  cifs.upcall[2123]: ip=192.168.122.124
  cifs.upcall[2123]: sec=1
  cifs.upcall[2123]: uid=0
  cifs.upcall[2123]: creduid=0
  cifs.upcall[2123]: user=root
  cifs.upcall[2123]: pid=2121
  cifs.upcall[2123]: upcall_target=mount
  cifs.upcall[2122]: upcall_target=mount, not switching namespaces to 
application thread
  cifs.upcall[2122]: get_cachename_from_process_env: pid == 0
  cifs.upcall[2122]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
  cifs.upcall[2122]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
  cifs.upcall[2122]: handle_krb5_mech: using native krb5
  cifs.upcall[2122]: handle_krb5_mech: obtained service ticket
  cifs.upcall[2122]: Exit status 0

  Specific Testcases Of Existing / Patched Packages:

  patched kernel, existing cifs-utils
  -----------------------------------

  When specifying "upcall_target" on mount command line, e.g.:

  # mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0,upcall_target=app 
//samba-dc.example.com/demo /mnt/testshare1
  # journalctl -f
  kernel: CIFS: Attempting to mount //samba-dc.example.com/demo
  cifs.upcall[1540]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.124;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x600;upcall_target=app
  cifs.upcall[1541]: ver=2
  cifs.upcall[1541]: host=samba-dc.example.com
  cifs.upcall[1541]: ip=192.168.122.124
  cifs.upcall[1541]: sec=1
  cifs.upcall[1541]: uid=0
  cifs.upcall[1541]: creduid=0
  cifs.upcall[1541]: user=root
  cifs.upcall[1541]: pid=1536
  cifs.upcall[1540]: get_cachename_from_process_env: pid == 0
  cifs.upcall[1540]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
  cifs.upcall[1540]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
  cifs.upcall[1540]: handle_krb5_mech: using native krb5
  cifs.upcall[1540]: handle_krb5_mech: obtained service ticket
  cifs.upcall[1540]: Exit status 0

  Test with no "upcall_target". e.g.:

  # mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0 
//samba-dc.example.com/demo /mnt/testshare1
  # journalctl -f
  Apr 30 04:23:35 samba-dc kernel: CIFS: Attempting to mount 
//samba-dc.example.com/demo
  Apr 30 04:23:35 samba-dc cifs.upcall[1560]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.124;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x614;upcall_target=app
  Apr 30 04:23:35 samba-dc cifs.upcall[1561]: ver=2
  Apr 30 04:23:35 samba-dc cifs.upcall[1561]: host=samba-dc.example.com
  Apr 30 04:23:35 samba-dc cifs.upcall[1561]: ip=192.168.122.124
  Apr 30 04:23:35 samba-dc cifs.upcall[1561]: sec=1
  Apr 30 04:23:35 samba-dc cifs.upcall[1561]: uid=0
  Apr 30 04:23:35 samba-dc cifs.upcall[1561]: creduid=0
  Apr 30 04:23:35 samba-dc cifs.upcall[1561]: user=root
  Apr 30 04:23:35 samba-dc cifs.upcall[1561]: pid=1556
  Apr 30 04:23:35 samba-dc cifs.upcall[1560]: get_cachename_from_process_env: 
pid == 0
  Apr 30 04:23:35 samba-dc cifs.upcall[1560]: get_existing_cc: default ccache 
is FILE:/tmp/krb5cc_0
  Apr 30 04:23:35 samba-dc cifs.upcall[1560]: handle_krb5_mech: getting service 
ticket for samba-dc.example.com
  Apr 30 04:23:35 samba-dc cifs.upcall[1560]: handle_krb5_mech: using native 
krb5
  Apr 30 04:23:35 samba-dc cifs.upcall[1560]: handle_krb5_mech: obtained 
service ticket
  Apr 30 04:23:35 samba-dc cifs.upcall[1560]: Exit status 0

  existing kernel, patched cifs-utils
  -----------------------------------

  When specifying "upcall_target" on mount command line, e.g.:

  # mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0,upcall_target=app 
//samba-dc.example.com/demo /mnt/testshare1
  mount error(22): Invalid argument
  Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log 
messages (dmesg)
  # journalctl -f
  kernel: cifs: Unknown parameter 'upcall_target'

  Test with no "upcall_target". e.g.:

  # mount -t cifs -o 
cruid=root,user=root,sec=krb5i,uid=0,gid=0,cred=/tmp/krb5cc_0 
//samba-dc.example.com/demo /mnt/testshare1
  # journalctl -f
  kernel: CIFS: Attempting to mount //samba-dc.example.com/demo
  cifs.upcall[10899]: key description: 
cifs.spnego;0;0;39010000;ver=0x2;host=samba-dc.example.com;ip4=192.168.122.124;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x2a8d
  cifs.upcall[10900]: ver=2
  cifs.upcall[10900]: host=samba-dc.example.com
  cifs.upcall[10900]: ip=192.168.122.124
  cifs.upcall[10900]: sec=1
  cifs.upcall[10900]: uid=0
  cifs.upcall[10900]: creduid=0
  cifs.upcall[10900]: user=root
  cifs.upcall[10900]: pid=10893
  cifs.upcall[10899]: upcall_target=app, switching namespaces to application 
thread
  cifs.upcall[10899]: get_cachename_from_process_env: pid == 0
  cifs.upcall[10899]: get_existing_cc: default ccache is FILE:/tmp/krb5cc_0
  cifs.upcall[10899]: main: valid service ticket exists in credential cache
  cifs.upcall[10899]: handle_krb5_mech: getting service ticket for 
samba-dc.example.com
  cifs.upcall[10899]: handle_krb5_mech: using native krb5
  cifs.upcall[10899]: handle_krb5_mech: obtained service ticket
  cifs.upcall[10899]: Exit status 0

  Note the line:
  cifs.upcall[10899]: upcall_target=app, switching namespaces to application 
thread

  [Where problems can occur]

  We are adding a new mount option to cifs in both the kernel and in
  cifs-utils.

  Existing cifs-utils packages need to not break when making upcalls to kernels
  that have this new upcall_target option, and existing kernels need to not 
break
  when using new cifs-utils packages that set upcall_target without the 
necessary
  in kernel support.

  We need to be careful to test three scenarios:
  * patched kernel, patched cifs-utils
  * patched kernel, existing cifs-utils
  * existing kernel, patched cifs-utils

  The default option is "app" and "app" has the same behaviour of pre-
  patch, that is, to use the credential cache of the calling process
  namespace. This should not introduce any behaviour change to existing
  setups. Not specifying any option at mount time defaults to "app"
  automatically. Users must opt into using "mount" themselves.

  If a regression were to occur, it could affect mounting of cifs / smb shares 
and
  users would not be able to access their data.

  Additionally, if a regression were to occur, we could also further confuse 
what
  namespace is to be used for accessing the user's kerberos credentials cache,
  which could disclose data from the host or container namespace to the 
incorrect
  namespace.

  [Other info]

  CVE-2025-2312
  https://ubuntu.com/security/CVE-2025-2312
  https://nvd.nist.gov/vuln/detail/CVE-2025-2312

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cifs-utils/+bug/2099914/+subscriptions


-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to     : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to