Hi Mark, thanks for chipping in. Does anyone else have any suggestions before I wipe and reinstall the replica entirely?
Peter ________________________________________ From: Mark Reynolds <[email protected]> Sent: Friday, 6 June 2025 17:32 To: FreeIPA users list Cc: Kroon PC, Peter Subject: Re: [Freeipa-users] Re: Replication issues [You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] On 6/6/25 11:01 AM, Kroon PC, Peter via FreeIPA-users wrote: > Hi Mark, > > thanks for the quick reply. > Server-B has the following in the access log (twice, actually): > ``` > [06/Jun/2025:14:55:33.573567003 +0000] conn=8984 fd=372 slot=372 connection > from 192.168.12.58 to 192.168.13.128 > [06/Jun/2025:14:55:33.613280772 +0000] conn=8984 op=0 BIND dn="" method=sasl > version=3 mech=GSSAPI > [06/Jun/2025:14:55:33.614444924 +0000] conn=8984 op=0 RESULT err=49 tag=97 > nentries=0 wtime=0.000406209 optime=0.001176425 etime=0.001580230 - > SASL(-13): authentication failure: GSSAPI Failure: gss_accept_sec_context > [06/Jun/2025:14:55:33.696712718 +0000] conn=8984 op=1 UNBIND > [06/Jun/2025:14:55:33.696753716 +0000] conn=8984 op=1 fd=372 Disconnect - > Cleanly Closed Connection - U1 > ``` > I don't fully understand why it's trying GSSAPI after I entered the directory > manager password. You authenticated as Directory Manager to the CLI, but it invokes the replication agreement credentials to do the replication initialization behind the scenes. > I'd assume it'd use simple bind. > In the security log there's the following, nothing about the failed gssapi > bind. > ``` > { "date": "[06\/Jun\/2025:14:55:00.776221689 +0000] ", "utc_time": > "1749221700.776221689", "event": "BIND_SUCCESS", "dn": "cn=directory > manager", "bind_method": "SIMPLE", "root_dn": true, "client_ip": > "192.168.12.58", "server_ip": "192.168.13.128", "ldap_version": 3, "conn_id": > 8983, "op_id": 0, "msg": "" } > ``` This is a different connection, but the other failed bind should have been logged. Might have a bug somewhere :-( > > I'm about to leave the office for a long weekend, so I probably won't respond > again before Tuesday. So you "might" need to do an offline initialization to fix this, but I suspect there is an official way the IPA team recommends to fix this (sorry not as familiar with IPA as I am DS). So maybe someone from the core IPA team and provide their recommendation. Mark > > Peter > > ________________________________________ > From: Mark Reynolds <[email protected]> > Sent: Friday, 6 June 2025 15:26 > To: FreeIPA users list > Cc: Kroon PC, Peter > Subject: Re: [Freeipa-users] Replication issues > > [You don't often get email from [email protected]. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > Hi Peter, > > So the credentials the replication agreement is using are not valid (for > whatever reason). Please check the directory server access log for > "err=49" and you should see the bind DN and more details on why the > authentication failed. > > HTH, > > Mark > > On 6/6/25 6:56 AM, Kroon PC, Peter via FreeIPA-users wrote: >> Hello world, >> >> I have 3 IPA servers that are supposed to all replicate with each other. For >> one server this stopped working. On all servers I have ipa-server >> 4.12.2-14.el9_6 on Rocky Linux 9.6. >> I'll call my servers A, B, and C. Server A cannot replicate with neither >> server B nor C. B and C can replicate eachother without issues. My first >> suspicion was a firewall, but I've already confirmed that that's not the >> issue. >> Here's the sanitized CLI outputs, all from server A: >> ``` >> $ ipa-healthcheck >> { >> "source": "ipahealthcheck.ds.replication", >> "check": "ReplicationCheck", >> "result": "CRITICAL", >> "uuid": "9c96a61a-d917-44de-907a-d5c7a4df667e", >> "when": "20250606095130Z", >> "duration": "1.072988", >> "kw": { >> "key": "DSREPLLE0001", >> "items": [ >> "Replication", >> "Agreement" >> ], >> "msg": "The replication agreement >> (server-A.example.com-to-server-C.example.com) under \"dc=example,dc=com\" >> is not in synchronization." >> } >> }, >> { >> "source": "ipahealthcheck.ds.replication", >> "check": "ReplicationCheck", >> "result": "CRITICAL", >> "uuid": "6c29d38e-a216-414e-b47a-d9302f38da67", >> "when": "20250606095131Z", >> "duration": "1.073047", >> "kw": { >> "key": "DSREPLLE0001", >> "items": [ >> "Replication", >> "Agreement" >> ], >> "msg": "The replication agreement (metoserver-B.example.com) under >> \"dc=example,dc=com\" is not in synchronization." >> } >> }, >> { >> "source": "ipahealthcheck.ds.replication", >> "check": "ReplicationCheck", >> "result": "CRITICAL", >> "uuid": "445f686b-27d9-47e0-aaa8-bb8d5f3416d4", >> "when": "20250606095131Z", >> "duration": "1.073054", >> "kw": { >> "key": "DSREPLLE0001", >> "items": [ >> "Replication", >> "Agreement" >> ], >> "msg": "The replication agreement (catoserver-B.example.com) under >> \"o=ipaca\" is not in synchronization." >> } >> }, >> { >> "source": "ipahealthcheck.ds.replication", >> "check": "ReplicationCheck", >> "result": "CRITICAL", >> "uuid": "bb6fb4b1-54fc-45dc-bb33-c8c3562f5765", >> "when": "20250606095131Z", >> "duration": "1.073058", >> "kw": { >> "key": "DSREPLLE0001", >> "items": [ >> "Replication", >> "Agreement" >> ], >> "msg": "The replication agreement >> (server-A.example.com-to-server-C.example.com) under \"o=ipaca\" is not in >> synchronization." >> } >> } >> ``` >> ``` >> $ ipa-replica-manage list >> ipa-replica-manage list >> Directory Manager password: >> >> server-A.example.com: master >> server-B.example.com: master >> server-C.example.com: master >> ``` >> Ok, things are broken, let's try force-sync. >> ``` >> $ ipa-replica-manage force-sync --from server-B.example.com --verbose --debug >> <snip imports> >> ipa: DEBUG: found 1 A records for server-A.example.com.: 192.168.12.58 >> ipa: DEBUG: The DNS response does not contain an answer to the question: >> server-A.example.com. IN AAAA >> Directory Manager password: >> >> ipa: DEBUG: Created connection context.ldap2_139632322538896 >> ipa: DEBUG: found 1 A records for server-A.example.com.: 192.168.12.58 >> ipa: DEBUG: The DNS response does not contain an answer to the question: >> server-A.example.com. IN AAAA >> ipa: DEBUG: found 1 A records for server-B.example.com.: 192.168.13.128 >> ipa: DEBUG: The DNS response does not contain an answer to the question: >> server-B.example.com. IN AAAA >> ipa: DEBUG: retrieving schema for SchemaCache >> url=ldaps://server-A.example.com:636 conn=<ldap.ldapobject.SimpleLDAPObject >> object at 0x7efeaefd2460> >> ipa: DEBUG: Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state' >> ipa: DEBUG: Loading Index file from >> '/var/lib/ipa/sysrestore/sysrestore.index' >> ipa: DEBUG: retrieving schema for SchemaCache >> url=ldapi://%2Frun%2Fslapd-BIN-BIOINF-NL.socket >> conn=<ldap.ldapobject.SimpleLDAPObject object at 0x7efeaefd2610> >> ipa: DEBUG: retrieving schema for SchemaCache >> url=ldaps://server-B.example.com:636 conn=<ldap.ldapobject.SimpleLDAPObject >> object at 0x7efeae1ee1f0> >> ipa: INFO: Setting agreement >> cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping >> tree,cn=config schedule to 2358-2359 0 to force synch >> ipa: INFO: Deleting schedule 2358-2359 0 from agreement >> cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping >> tree,cn=config >> ipa: INFO: Replication Update in progress: False: status: Error (49) Problem >> connecting to replica - LDAP error: Invalid credentials (connection error): >> start: 19700101000000: end: 19700101000000 >> ipa: DEBUG: Destroyed connection context.ldap2_139632322538896 >> ``` >> Maybe I can re-initialize? >> ``` >> $ ipa-replica-manage re-initialize --from server-B.example.com --verbose >> --debug >> <snip import> >> <snip DNS responses, same as above> >> Directory Manager password: >> ipa: DEBUG: Created connection context.ldap2_140512488991280 >> ipa: DEBUG: retrieving schema for SchemaCache >> url=ldaps://server-A.example.com:636 conn=<ldap.ldapobject.SimpleLDAPObject >> object at 0x7fcb9c310dc0> >> ipa: DEBUG: retrieving schema for SchemaCache >> url=ldaps://server-B.example.com:636 conn=<ldap.ldapobject.SimpleLDAPObject >> object at 0x7fcb9c1e10a0> >> ipa: INFO: Setting agreement >> cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping >> tree,cn=config schedule to 2358-2359 0 to force synch >> ipa: INFO: Deleting schedule 2358-2359 0 from agreement >> cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping >> tree,cn=config >> Update in progress, 15 seconds elapsed >> [ldaps://server-B.example.com:636] reports: Update failed! Status: [Error >> (49) - LDAP error: Invalid credentials - no response received] >> ipa: DEBUG: Destroyed connection context.ldap2_140512488991280 >> ``` >> Is it a firewall or password issue? No: >> ``` >> $ ldapwhoami -H ldaps://server-B.example.com -D 'cn=directory manager' -W >> Enter LDAP Password: >> dn: cn=directory manager >> ``` >> From server-B: >> ``` >> server-B $ ldapwhoami -H ldaps://server-A.example.com -D 'cn=directory >> manager' -W >> Enter LDAP Password: >> dn: cn=directory manager >> ``` >> I also checked that `ipa-replica-manage list-ruv` produces the same output >> for all 3 servers. >> Does anyone have any more ideas I could explore? >> >> Many thanks in advance! >> Peter > -- > Identity Management Development Team > -- Identity Management Development Team -- _______________________________________________ FreeIPA-users mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/[email protected] Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
