Neither came back with anything # ldapsearch -x -h 172.16.100.161 -D "cn=directory manager" -W -b "dc=itmodev,dc=gov" '(uid=ldap/comipa01.itmodev.gov)' Enter LDAP Password: # extended LDIF # # LDAPv3 # base <dc=itmodev,dc=gov> with scope subtree # filter: (uid=ldap/comipa01.itmodev.gov) # requesting: ALL #
# search result search: 2 result: 0 Success # numResponses: 1 [root@comipa02 ~]# ldapsearch -x -h 172.16.100.161 -D "cn=directory manager" -W -b "dc=itmodev,dc=gov" '(uid=ldap/*.gov)' uid Enter LDAP Password: # extended LDIF # # LDAPv3 # base <dc=itmodev,dc=gov> with scope subtree # filter: (uid=ldap/*.gov) # requesting: uid # # search result search: 2 result: 0 Success # numResponses: 1 -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Rich Megginson Sent: Tuesday, November 10, 2015 11:04 AM To: [email protected] Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos authentication error) On 11/10/2015 08:18 AM, Gronde, Christopher (Contractor) wrote: > Thank you! I should have caught that... > > I changed the log level and then restarted dirsrv and attempted to start > krb5kdc and got the following... <snip> [10/Nov/2015:10:12:02 -0500] conn=5 fd=64 slot=64 connection from 172.16.100.208 to 172.16.100.161 [10/Nov/2015:10:12:02 -0500] conn=5 op=0 BIND dn="" method=sasl version=3 mech=GSSAPI [10/Nov/2015:10:12:03 -0500] conn=5 op=0 RESULT err=14 tag=97 nentries=0 etime=1, SASL bind in progress [10/Nov/2015:10:12:03 -0500] conn=5 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI [10/Nov/2015:10:12:03 -0500] conn=5 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress [10/Nov/2015:10:12:03 -0500] conn=5 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI [10/Nov/2015:10:12:03 -0500] conn=Internal op=-1 SRCH base="dc=itmodev,dc=gov" scope=2 filter="(uid=ldap/comipa01.itmodev.gov)" attrs=ALL [10/Nov/2015:10:12:03 -0500] conn=Internal op=-1 RESULT err=0 tag=48 nentries=0 etime=0 [10/Nov/2015:10:12:03 -0500] conn=5 op=2 RESULT err=49 tag=97 nentries=0 etime=0 [10/Nov/2015:10:12:03 -0500] conn=5 op=3 UNBIND [10/Nov/2015:10:12:03 -0500] conn=5 op=3 fd=64 closed - U1 <snip> This is the SASL bind. It thinks the principal in the Kerberos credential is "ldap/comipa01.itmodev.gov", and the SASL map tells the code to look for something with uid=ldap/comipa01.itmodev.gov under dc=itmodev,dc=gov. However, this entry is not found: RESULT err=0 tag=48 nentries=0. nentries=0 means no entries matched the search criteria. You can do the search yourself with ldapsearch: ldapsearch -x -h 172.16.100.161 -D "cn=directory manager" -W -b "dc=itmodev,dc=gov" '(uid=ldap/comipa01.itmodev.gov)' If you want to find out if there is some other ldap principal, do a search like this: ldapsearch -x -h 172.16.100.161 -D "cn=directory manager" -W -b "dc=itmodev,dc=gov" '(uid=ldap/*.gov)' uid >> Ran into an error trying to set that >> >> # ldapmodify -a -D "cn=directory manager" -W Enter LDAP Password: >> dn: cn=config >> changetype: modify >> replace: nsslapd-acesslog-level >> : 260 >> >> modifying entry "cn=config" >> ldap_modify: Server is unwilling to perform (53) >> additional info: Unknown attribute nsslapd-acesslog-level >> will be ignored >> >> [root@comipa02 ~]# ldapmodify -a -D "cn=config" -W Enter LDAP >> Password: >> ldap_bind: Inappropriate authentication (48) >> >> -----Original Message----- >> From: Ludwig Krispenz [mailto:[email protected]] >> Sent: Tuesday, November 10, 2015 9:48 AM >> To: Gronde, Christopher (Contractor) <[email protected]> >> Cc: [email protected] >> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos >> authentication error) >> >> >> On 11/10/2015 03:32 PM, Gronde, Christopher (Contractor) wrote: >>> How do I change that log setting? Is that done in LDAP? Using ldapmodify? >> yes, >> ldapmodify ... >> dn: cn=config >> changetype: modify >> replace: nsslapd-acesslog-level >> nsslapd-acesslog-level: 260 >>> -----Original Message----- >>> From: [email protected] >>> [mailto:[email protected]] On Behalf Of Ludwig >>> Krispenz >>> Sent: Tuesday, November 10, 2015 9:03 AM >>> To: [email protected] >>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos >>> authentication error) >>> >>> >>> On 11/10/2015 02:40 PM, Alexander Bokovoy wrote: >>>> On Tue, 10 Nov 2015, Gronde, Christopher (Contractor) wrote: >>>>> Where can I verify or change the credentials it is trying to use? >>>>> Is it my LDAP password? >>>> No, according to your logs, it is your LDAP master trying to >>>> replicate (push changes) to your LDAP replica: >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 fd=64 slot=64 connection from >>>>>> <MASTER_IP> to <REPLICA_IP> >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 BIND dn="" method=sasl >>>>>> version=3 mech=GSSAPI >>> err=49 could also be a result if the entry which is mapped from the >>> principal is not found in the directory. A bit more info could be gained by >>> enabling logging of internal searches. >>> Set nsslapd-acesslog-level: 260 >>> >>> and then look what internal searches are done during the gssapi >>> authentication >>>> If that is true, it would be ldap/<master> Kerberos principal >>>> talking to ldap/<replica> Kerberos principal. If that fails, it >>>> means master and replica KDCs have different understanding of both >>>> ldap/<master> and ldap/<replica> keys which most likely means keys >>>> were rotated on master and weren't propagated to replica. >>>> >>>> How to solve it? One possibility is to set master's hostname as KDC >>>> address in krb5.conf on replica, forcing LDAP server on replica to >>>> use master's KDC. I'm absolutely not sure this will actually work >>>> but at least it allows to see if we are indeed dealing with >>>> inconsistent state of service principals' keys. >>>> >>>>> -----Original Message----- >>>>> From: Alexander Bokovoy [mailto:[email protected]] >>>>> Sent: Tuesday, November 10, 2015 8:18 AM >>>>> To: Gronde, Christopher (Contractor) >>>>> <[email protected]> >>>>> Cc: Rob Crittenden <[email protected]>; [email protected] >>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos >>>>> authentication error) >>>>> >>>>> On Tue, 10 Nov 2015, Gronde, Christopher (Contractor) wrote: >>>>>> When I tried to start the service again I got no response from >>>>>> tail of the log, but this is a repeating entry I see in the >>>>>> access log >>>>>> >>>>>> [09/Nov/2015:15:01:04 -0500] conn=1 fd=64 slot=64 connection from >>>>>> 127.0.0.1 to 127.0.0.1 >>>>>> [09/Nov/2015:15:01:04 -0500] conn=1 op=-1 fd=64 closed - B1 >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 fd=64 slot=64 connection from >>>>>> <MASTER_IP> to <REPLICA_IP> >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 BIND dn="" method=sasl >>>>>> version=3 mech=GSSAPI >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 RESULT err=14 tag=97 >>>>>> nentries=0 etime=0, SASL bind in progress >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=1 BIND dn="" method=sasl >>>>>> version=3 mech=GSSAPI >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=1 RESULT err=14 tag=97 >>>>>> nentries=0 etime=0, SASL bind in progress >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=2 BIND dn="" method=sasl >>>>>> version=3 mech=GSSAPI >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=2 RESULT err=49 tag=97 >>>>>> nentries=0 etime=0 >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=3 UNBIND >>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=3 fd=64 closed - U1 >>>>>> >>>>>> Does anyone know what err=14 or err=49 are? >>>>> err=14 means SASL bind in progress -- i.e. multi-round processing >>>>> is ongoing. This is normal for SASL GSSAPI. >>>>> >>>>> err=49 is wrong password or username, i.e. credentials were incorrect. >>>>> It may also mean that LDAP server side was unable to process >>>>> Kerberos negotiation due to not having a current Kerberos ticket >>>>> for own service >>>>> (LDAP) and trying to request it from the Kerberos KDC but Kerberos >>>>> KDC is down. >>>>> >>>>>> -----Original Message----- >>>>>> From: Rob Crittenden [mailto:[email protected]] >>>>>> Sent: Monday, November 09, 2015 3:26 PM >>>>>> To: Gronde, Christopher (Contractor) >>>>>> <[email protected]>; Alexander Bokovoy >>>>>> <[email protected]> >>>>>> Cc: [email protected] >>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos >>>>>> authentication error) >>>>>> >>>>>> Gronde, Christopher (Contractor) wrote: >>>>>>> Nothing bad came back and there is definitely data in the tree. >>>>>> Ok, I guess I'd try to start the kdc again and then watch the >>>>>> 389-ds access log (buffered) to: >>>>>> >>>>>> 1. See if it is binding at all >>>>>> 2. See what the search is and what, if any, results were returned >>>>>> >>>>>> This would be in /var/log/dirsrv/slapd-YOUR_REALM/access >>>>>> >>>>>> rob >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Rob Crittenden [mailto:[email protected]] >>>>>>> Sent: Monday, November 09, 2015 11:46 AM >>>>>>> To: Gronde, Christopher (Contractor) >>>>>>> <[email protected]>; Alexander Bokovoy >>>>>>> <[email protected]> >>>>>>> Cc: [email protected] >>>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos >>>>>>> authentication error) >>>>>>> >>>>>>> Gronde, Christopher (Contractor) wrote: >>>>>>>> I restarted dirsrv and attempted to start krb5kdc and this is >>>>>>>> what the error log shows >>>>>>>> >>>>>>>> # tail /var/log/dirsrv/slapd-ITMODEV-GOV/errors >>>>>>>> [09/Nov/2015:11:01:02 -0500] - WARNING: userRoot: entry cache >>>>>>>> size 10485760B is less than db size 28016640B; We recommend to >>>>>>>> increase the entry cache size nsslapd-cachememsize. >>>>>>>> [09/Nov/2015:11:01:02 -0500] - slapd started. Listening on All >>>>>>>> Interfaces port 389 for LDAP requests >>>>>>>> [09/Nov/2015:11:06:04 -0500] - slapd shutting down - signaling >>>>>>>> operation threads >>>>>>>> [09/Nov/2015:11:06:04 -0500] - slapd shutting down - closing >>>>>>>> down internal subsystems and plugins >>>>>>>> [09/Nov/2015:11:06:04 -0500] - Waiting for 4 database threads >>>>>>>> to stop >>>>>>>> [09/Nov/2015:11:06:04 -0500] - All database threads now stopped >>>>>>>> [09/Nov/2015:11:06:04 -0500] - slapd stopped. >>>>>>>> [09/Nov/2015:11:14:20 -0500] - 389-Directory/1.2.11.15 >>>>>>>> B2015.247.1737 starting up >>>>>>>> [09/Nov/2015:11:14:20 -0500] - WARNING: userRoot: entry cache >>>>>>>> size 10485760B is less than db size 28016640B; We recommend to >>>>>>>> increase the entry cache size nsslapd-cachememsize. >>>>>>>> [09/Nov/2015:11:14:20 -0500] - slapd started. Listening on All >>>>>>>> Interfaces port 389 for LDAP requests >>>>>>> Ok, that's good. >>>>>>> >>>>>>> I'd do something like this to see what is in the db (substitute >>>>>>> example.com with your domain): >>>>>>> >>>>>>> $ ldapsearch -x -D 'cn=Directory Manager' -W -s one -b >>>>>>> cn=kerberos,dc=example,dc=com >>>>>>> >>>>>>> (don't post the output as it would include the kerberos master key). >>>>>>> >>>>>>> If that returns nothing that's bad. >>>>>>> >>>>>>> If it succeeds I'd broaden the search base a bit to see what >>>>>>> data you do >>>>>>> have: >>>>>>> >>>>>>> $ ldapsearch -x -D 'cn=Directory Manager' -W -b >>>>>>> cn=groups,cn=accounts,dc=example,dc=com >>>>>>> >>>>>>> I picked groups because usually groups << users in numbers. This >>>>>>> is just to see if you have data in the tree. >>>>>>> >>>>>>> Let us know if either or both turns up nothing. >>>>>>> >>>>>>> rob >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Alexander Bokovoy [mailto:[email protected]] >>>>>>>> Sent: Monday, November 09, 2015 10:51 AM >>>>>>>> To: Gronde, Christopher (Contractor) >>>>>>>> <[email protected]> >>>>>>>> Cc: [email protected] >>>>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos >>>>>>>> authentication error) >>>>>>>> >>>>>>>> On Mon, 09 Nov 2015, Gronde, Christopher (Contractor) wrote: >>>>>>>>> Hello all! >>>>>>>>> >>>>>>>>> On my replica IPA server after fixing a cert issue that had >>>>>>>>> been going on for sometime, I have all my certs figured out >>>>>>>>> but the krb5kdc service will not start. >>>>>>>>> >>>>>>>>> # service krb5kdc start >>>>>>>>> Starting Kerberos 5 KDC: krb5kdc: cannot initialize realm >>>>>>>>> ITMODEV.GOV - see log file for details [FAILED] >>>>>>>>> >>>>>>>>> # cat /var/log/krb5kdc.log >>>>>>>>> krb5kdc: Server error - while fetching master key K/M for >>>>>>>>> realm ITMODEV.GOV >>>>>>>>> krb5kdc: Server error - while fetching master key K/M for >>>>>>>>> realm ITMODEV.GOV >>>>>>>>> krb5kdc: Server error - while fetching master key K/M for >>>>>>>>> realm ITMODEV.GOV >>>>>>>>> >>>>>>>>> I found this article online: >>>>>>>>> http://research.imb.uq.edu.au/~l.rathbone/ldap/kerberos.shtml >>>>>>>>> >>>>>>>>> Which stated it might be because The slave KDC does not have a >>>>>>>>> stash file (.k5.EXAMPLE.COM). You need to create one. Tried >>>>>>>>> the command >>>>>>>>> listed: >>>>>>>>> >>>>>>>>> # kdb5_util stash >>>>>>>>> kdb5_util: Server error while retrieving master entry >>>>>>>>> >>>>>>>>> No further information found on the proceeding error above for >>>>>>>>> the kdb5_util command. >>>>>>>>> >>>>>>>>> Any thoughts? >>>>>>>> First: don't use instructions which are not related to IPA, please. >>>>>>>> >>>>>>>> FreeIPA has its own LDAP driver for KDC and instructions for >>>>>>>> anything else do not apply here at all. >>>>>>>> >>>>>>>> If you see 'Server error - while fetching master key ..' it >>>>>>>> means KDC LDAP driver was unable to contact LDAP server. Does >>>>>>>> LDAP server work on the replica? What is in its error log >>>>>>>> (/var/log/dirsrv/slapd-ITMODEV-GOV/errors)? >>>>>>>> >>>>>>>> -- >>>>>>>> / Alexander Bokovoy >>>>>>>> >>>>>>>> >>>>> -- >>>>> / Alexander Bokovoy >>>>> >>> -- >>> Manage your subscription for the Freeipa-users mailing list: >>> https://www.redhat.com/mailman/listinfo/freeipa-users >>> Go to http://freeipa.org for more info on the project >>> > > -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
