fserver is healthy, all services up and running and its certificates valid. It also has a full set of users. On the contrary, mserver has certificates that have expired and is the one whose tomcat fails to start. mserver misses few users, but I think I can handle this by recreating them, if needed.

So, I guess the proper approach is to stop mserver from being renewal master, renew its certs and sync it with fserver's dbase. If I am correct, could you point me to relative instructions? Or am I totally lost?

Petros

On 3/21/25 11:08 PM, Rob Crittenden wrote:
Petros Triantafyllidis via FreeIPA-users wrote:
Sorry for misunderstanding, I thought I made it clear: replication is
broken. Wherever I create a user, this user does not exist in the other
server.
There should only be one renewal master.

The way it should work is that one server, the renewal master, handles
renewing the CA subsystem certificates. It sticks the updated
certificates into LDAP and the other servers pull the updated certs from
there. The other servers update their own server-specific certificates
(HTTP, LDAP, etc) themselves.

Earlier in the thread you said that mserver failed to start tomcat but
listed output from fserver. Is mserver in fact ok and has valid
certificates for years to come?

You'll want to start trying to find any users that only exist on one of
the servers. Bringing fserver back into the fold is likely to be
destructive so you want to preserve as much as you can.

You effectively have a split-brain. Two independent systems with mostly
the same data each claiming to be authoritative but not actually having
all the same data.

rob


On 3/21/25 6:01 PM, Florence Blanc-Renaud wrote:
Hi,

On Fri, Mar 21, 2025 at 3:33 PM Petros Triantafyllidis via
FreeIPA-users <[email protected]> wrote:

     Thanks for your answer flo,
     ipa config-show shows both servers as renewal masters, depending
     where you run it. I guess this might be a cause of the problems:

     [root@mserver ~]# ipa config-show | grep renewal
       IPA CA renewal master: mserver.example.com
     <http://mserver.example.com>

     [root@fserver ~]# ipa config-show | grep renewal
       IPA CA renewal master: fserver.example.com
     <http://fserver.example.com>

     Even when I force the service to start on mserver, pki-tomcat
     still fails and consequently ipa-healthcheck reports the same errors.

Is the replication broken even after the service is force-started?

flo

     Assuming that I'd like fserver (the healthy one) to be the CA
     renewal master, how do I proceed? Should I run ipa-cert-fix on
     mserver ?

     Thanks again,
     Petros


     On 3/21/25 15:34, Florence Blanc-Renaud wrote:
     Hi,

     On Thu, Mar 20, 2025 at 10:06 PM Petros Triantafyllidis via
     FreeIPA-users <[email protected]> wrote:

         Hi,
          I have two IPA servers 4.10.2-8.el9_3 (fserver & mserver)
         running Rocky 9.3. I realized that some of the most recently
         created users, had problems logging in. One strange thing was
         that when listing their home directory, in place of owner
         there was their uidnumber instead their username.
         One of the servers (mserver) fails to start pki-tomcat and
         suspected a certificate issue (some show expiration a month
         ago). Below I show some info (sanitized) and I could use some
         help:

         *[root@fserver]# ipa-healthcheck*
         Internal server error 503 Server Error: Service Unavailable
         for url:
         https://mserver.example.com:443/ca/rest/certs/search?size=3
         [
           {
             "source":
         "pki.server.healthcheck.clones.connectivity_and_data",
             "check": "ClonesConnectivyAndDataCheck",
             "result": "ERROR",
             "uuid": "ae2033bb-9595-4907-8b6d-0db6d13813c3",
             "when": "20250320202815Z",
             "duration": "0.605725",
             "kw": {
               "status": "ERROR:  pki-tomcat : Internal error testing
         CA clone. Host: mserver.example.com
         <http://mserver.example.com> Port: 443"
             }
           },
           {
             "source": "ipahealthcheck.ds.replication",
             "check": "ReplicationCheck",
             "result": "ERROR",
             "uuid": "cd8ecc04-1e11-4229-b0e9-605fc08cc2af",
             "when": "20250320202818Z",
             "duration": "0.381935",
             "kw": {
               "key": "DSREPLLE0003",
               "items": [
                 "Replication",
                 "Agreement"
               ],
               "msg": "The replication agreement
         (metomserver.example.com <http://metomserver.example.com>)
         under \"dc=IPA,dc=ss,dc=lan\" is not in
         synchronization.\nStatus message: error (18) can't acquire
         replica (incremental update transient warning.  backing off,
         will retry update later.)"
             }
           },
           {
             "source": "ipahealthcheck.ds.replication",
             "check": "ReplicationCheck",
             "result": "ERROR",
             "uuid": "2178f7ef-f4fc-426f-a4c4-f357c3540baa",
             "when": "20250320202818Z",
             "duration": "0.381965",
             "kw": {
               "key": "DSREPLLE0003",
               "items": [
                 "Replication",
                 "Agreement"
               ],
               "msg": "The replication agreement
         (catomserver.example.com <http://catomserver.example.com>)
         under \"o=ipaca\" is not in synchronization.\nStatus message:
         error (18) can't acquire replica (incremental update
         transient warning.  backing off, will retry update later.)"
             }
           }

         
===================================================================================

         *[root@mserver ~]# ipa-healthcheck *
         Expired Cert: ocsp_signing
         Expired Cert: subsystem
         Expired Cert: audit_signing
         Internal server error
         HTTPConnectionPool(host='mserver.example.com
         <http://mserver.example.com>', port=8080): Max retries
         exceeded with url: /ca/rest/securityDomain/domainInfo (Caused
         by NewConnectionError('<urllib3.connection.HTTPConnection
         object at 0x7f1a6ea9c6d0>: Failed to establish a new
         connection: [Errno 111] Connection refused'))
         Internal server error
         HTTPSConnectionPool(host='mserver.example.com
         <http://mserver.example.com>', port=8443): Max retries
         exceeded with url: /ca/admin/ca/getStatus (Caused by
         NewConnectionError('<urllib3.connection.HTTPSConnection
         object at 0x7f1a6e9194f0>: Failed to establish a new
         connection: [Errno 111] Connection refused'))
         [
           {
             "source": "ipahealthcheck.meta.services",
             "check": "pki_tomcatd",
             "result": "ERROR",
             "uuid": "1f169946-8a47-4d93-ae38-f8072abf82e1",
             "when": "20250320203343Z",
             "duration": "0.000577",
             "kw": {
               "status": false,
               "msg": "pki_tomcatd: not running"
             }
           },
           {
             "source": "pki.server.healthcheck.certs.expiration",
             "check": "CASystemCertExpiryCheck",
             "result": "ERROR",
             "uuid": "d659a57c-f625-462d-b6d5-1a60d8216953",
             "when": "20250320203344Z",
             "duration": "0.143464",
             "kw": {
               "cert_id": "ocsp_signing",
               "expiry_date": "Feb 17 2025",
               "msg": "Certificate has ALREADY EXPIRED"
             }
           },
           {
             "source": "pki.server.healthcheck.certs.expiration",
             "check": "CASystemCertExpiryCheck",
             "result": "ERROR",
             "uuid": "7232e7cb-3cc2-4ff2-9953-954ef2e5d3b9",
             "when": "20250320203344Z",
             "duration": "0.280452",
             "kw": {
               "cert_id": "subsystem",
               "expiry_date": "Feb 17 2025",
               "msg": "Certificate has ALREADY EXPIRED"
             }
           },
           {
             "source": "pki.server.healthcheck.certs.expiration",
             "check": "CASystemCertExpiryCheck",
             "result": "ERROR",
             "uuid": "117eece4-37dd-45cb-bf6c-acdfa29fb525",
             "when": "20250320203344Z",
             "duration": "0.349712",
             "kw": {
               "cert_id": "audit_signing",
               "expiry_date": "Feb 17 2025",
               "msg": "Certificate has ALREADY EXPIRED"
             }
           },
           {
             "source": "pki.server.healthcheck.meta.connectivity",
             "check": "DogtagCACertsConnectivityCheck",
             "result": "CRITICAL",
             "uuid": "bb5c2f08-e28e-47d7-9752-404f83fb67a8",
             "when": "20250320203345Z",
             "duration": "0.035959",
             "kw": {
               "msg": "Internal server error. Is your CA subsystem and
         LDAP database up?",
               "instance_name": "pki-tomcat",
               "exception":
         "HTTPSConnectionPool(host='mserver.example.com
         <http://mserver.example.com>', port=8443): Max retries
         exceeded with url: /ca/admin/ca/getStatus (Caused by
         NewConnectionError('<urllib3.connection.HTTPSConnection
         object at 0x7f1a6e9194f0>: Failed to establish a new
         connection: [Errno 111] Connection refused'))"
             }
           },
           {
             "source": "ipahealthcheck.dogtag.ca
         <http://ipahealthcheck.dogtag.ca>",
             "check": "DogtagCertsConnectivityCheck",
             "result": "ERROR",
             "uuid": "2c134180-e055-41fe-bd8e-8aa9ca4f56a6",
             "when": "20250320203346Z",
             "duration": "0.423802",
             "kw": {
               "key": "cert_show_ra",
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/41': [SSL:
         SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert certificate
         expired (_ssl.c:2633)",
               "serial": "41",
               "msg": "Request for certificate failed: {error}"
             }
           },
           {
             "source": "ipahealthcheck.ds.replication",
             "check": "ReplicationCheck",
             "result": "ERROR",
             "uuid": "55b716e8-43e9-44b0-8764-4263d283dc2d",
             "when": "20250320203347Z",
             "duration": "0.346086",
             "kw": {
               "key": "DSREPLLE0003",
               "items": [
                 "Replication",
                 "Agreement"
               ],
               "msg": "The replication agreement
         (metofserver.example.com <http://metofserver.example.com>)
         under \"dc=IPA,dc=ss,dc=lan\" is not in
         synchronization.\nStatus message: error (18) can't acquire
         replica (incremental update transient warning.  backing off,
         will retry update later.)"
             }
           },
           {
             "source": "ipahealthcheck.ds.replication",
             "check": "ReplicationCheck",
             "result": "ERROR",
             "uuid": "37bc48b0-7eca-4576-8e61-b30b1bde621b",
             "when": "20250320203347Z",
             "duration": "0.346109",
             "kw": {
               "key": "DSREPLLE0003",
               "items": [
                 "Replication",
                 "Agreement"
               ],
               "msg": "The replication agreement
         (catofserver.example.com <http://catofserver.example.com>)
         under \"o=ipaca\" is not in synchronization.\nStatus message:
         error (18) can't acquire replica (incremental update
         transient warning.  backing off, will retry update later.)"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertmongerExpirationCheck",
             "result": "ERROR",
             "uuid": "3170174f-6bb6-4afc-82f0-a795791036ed",
             "when": "20250320203347Z",
             "duration": "0.010887",
             "kw": {
               "key": "20240325182332",
               "expiration_date": "20250217085937Z",
               "msg": "Request id {key} expired on {expiration_date}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertmongerExpirationCheck",
             "result": "ERROR",
             "uuid": "727c67e4-f647-4942-b9f4-2861ffd244a8",
             "when": "20250320203347Z",
             "duration": "0.013823",
             "kw": {
               "key": "20240325182333",
               "expiration_date": "20250217085837Z",
               "msg": "Request id {key} expired on {expiration_date}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertmongerExpirationCheck",
             "result": "ERROR",
             "uuid": "83c634aa-24d0-41df-88c3-401a0ce804f4",
             "when": "20250320203347Z",
             "duration": "0.016737",
             "kw": {
               "key": "20240325182337",
               "expiration_date": "20250217085847Z",
               "msg": "Request id {key} expired on {expiration_date}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertmongerExpirationCheck",
             "result": "ERROR",
             "uuid": "d8ff80a5-f947-48a2-b97c-078becf2f8f9",
             "when": "20250320203347Z",
             "duration": "0.019678",
             "kw": {
               "key": "20240325182339",
               "expiration_date": "20250217085927Z",
               "msg": "Request id {key} expired on {expiration_date}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertfileExpirationCheck",
             "result": "ERROR",
             "uuid": "31b38213-3510-4a6d-b01f-4aef8f01fdfe",
             "when": "20250320203347Z",
             "duration": "0.059710",
             "kw": {
               "key": "20240325182332",
               "expiration_date": "20250217085937Z",
               "msg": "Request id {key} expired on {expiration_date}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertfileExpirationCheck",
             "result": "ERROR",
             "uuid": "140abdf2-c6a6-4f5a-9c4e-1381ad9ffef2",
             "when": "20250320203347Z",
             "duration": "0.103873",
             "kw": {
               "key": "20240325182333",
               "expiration_date": "20250217085837Z",
               "msg": "Request id {key} expired on {expiration_date}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertfileExpirationCheck",
             "result": "ERROR",
             "uuid": "bb70dc48-0382-43a5-80c9-1303302d0332",
             "when": "20250320203347Z",
             "duration": "0.148327",
             "kw": {
               "key": "20240325182337",
               "expiration_date": "20250217085847Z",
               "msg": "Request id {key} expired on {expiration_date}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertfileExpirationCheck",
             "result": "ERROR",
             "uuid": "d7c88743-8fb3-4793-b193-5e7a6a963e4b",
             "when": "20250320203348Z",
             "duration": "0.192067",
             "kw": {
               "key": "20240325182339",
               "expiration_date": "20250217085927Z",
               "msg": "Request id {key} expired on {expiration_date}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPAOpenSSLChainValidation",
             "result": "ERROR",
             "uuid": "faab3d0d-c726-4d93-a4fd-b3f47cbee7a0",
             "when": "20250320203351Z",
             "duration": "0.016884",
             "kw": {
               "key": "/var/lib/ipa/ra-agent.pem",
               "reason": "O = IPA.SS.LAN, CN = IPA RA\nerror 10 at 0
         depth lookup: certificate has expired\nerror
         /var/lib/ipa/ra-agent.pem: verification failed\n",
               "msg": "Certificate validation for {key} failed: {reason}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertRevocation",
             "result": "ERROR",
             "uuid": "c962600a-5f2e-4000-995e-0d6e2c51bf6c",
             "when": "20250320203351Z",
             "duration": "0.438354",
             "kw": {
               "key": "20240325182332",
               "serial": 41,
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/41': [SSL:
         SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert certificate
         expired (_ssl.c:2633)",
               "msg": "Request for certificate serial number {serial}
         in request {key} failed: {error}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertRevocation",
             "result": "ERROR",
             "uuid": "524f71a3-92aa-424c-a660-f48efef17684",
             "when": "20250320203351Z",
             "duration": "0.513622",
             "kw": {
               "key": "20240325182333",
               "serial": 37,
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/37': [SSL:
         SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert certificate
         expired (_ssl.c:2633)",
               "msg": "Request for certificate serial number {serial}
         in request {key} failed: {error}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertRevocation",
             "result": "ERROR",
             "uuid": "c2f9fc6f-0561-45e9-aa2b-c741b893a173",
             "when": "20250320203351Z",
             "duration": "0.591474",
             "kw": {
               "key": "20240325182337",
               "serial": 38,
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/38': [SSL:
         SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert certificate
         expired (_ssl.c:2633)",
               "msg": "Request for certificate serial number {serial}
         in request {key} failed: {error}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertRevocation",
             "result": "ERROR",
             "uuid": "a373bde5-64bd-4cdb-9db1-4b9a565f6d60",
             "when": "20250320203351Z",
             "duration": "0.667891",
             "kw": {
               "key": "20240325182339",
               "serial": 40,
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/40': [SSL:
         SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert certificate
         expired (_ssl.c:2633)",
               "msg": "Request for certificate serial number {serial}
         in request {key} failed: {error}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertRevocation",
             "result": "ERROR",
             "uuid": "4adb473e-4604-4f6b-85de-aeda264b2bfd",
             "when": "20250320203352Z",
             "duration": "0.749218",
             "kw": {
               "key": "20240325182340",
               "serial": 1,
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/1': [SSL:
         SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert certificate
         expired (_ssl.c:2633)",
               "msg": "Request for certificate serial number {serial}
         in request {key} failed: {error}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertRevocation",
             "result": "ERROR",
             "uuid": "3cd2e311-a4cc-47a1-86fa-c80ae7c35535",
             "when": "20250320203352Z",
             "duration": "0.826249",
             "kw": {
               "key": "20240325182341",
               "serial": 805175299,
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/805175299':
         [SSL: SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert
         certificate expired (_ssl.c:2633)",
               "msg": "Request for certificate serial number {serial}
         in request {key} failed: {error}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertRevocation",
             "result": "ERROR",
             "uuid": "3355e9e4-e18d-48b5-9a1f-018ea8a02018",
             "when": "20250320203352Z",
             "duration": "0.865347",
             "kw": {
               "key": "20240325182024",
               "serial": 805175298,
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/805175298':
         [SSL: SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert
         certificate expired (_ssl.c:2633)",
               "msg": "Request for certificate serial number {serial}
         in request {key} failed: {error}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertRevocation",
             "result": "ERROR",
             "uuid": "43779b90-74cd-4db1-a1b2-19b30f1400ac",
             "when": "20250320203352Z",
             "duration": "0.947838",
             "kw": {
               "key": "20240325182004",
               "serial": 805175297,
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/805175297':
         [SSL: SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert
         certificate expired (_ssl.c:2633)",
               "msg": "Request for certificate serial number {serial}
         in request {key} failed: {error}"
             }
           },
           {
             "source": "ipahealthcheck.ipa.certs",
             "check": "IPACertRevocation",
             "result": "ERROR",
             "uuid": "76a26aa3-c56f-4984-8abc-c5113d25f2e2",
             "when": "20250320203352Z",
             "duration": "0.992108",
             "kw": {
               "key": "20240325182408",
               "serial": 268304393,
               "error": "cannot connect to
         'https://mserver.example.com:443/ca/rest/certs/268304393':
         [SSL: SSLV3_ALERT_CERTIFICATE_EXPIRED] sslv3 alert
         certificate expired (_ssl.c:2633)",
               "msg": "Request for certificate serial number {serial}
         in request {key} failed: {error}"
             }
           }
         ]

         
===================================================================================

         *[root@mserver]# ipactl -d status*

         [...]

         ipa-custodia Service: RUNNING
         ipa: DEBUG: request POST
         http://mserver.example.com:8080/ca/admin/ca/getStatus
         ipa: DEBUG: request body ''
         ipa: DEBUG: httplib request failed:
         Traceback (most recent call last):
           File
         "/usr/lib/python3.9/site-packages/ipapython/dogtag.py", line
         271, in _httplib_request
             conn.request(method, path, body=request_body,
         headers=headers)
           File "/usr/lib64/python3.9/http/client.py", line 1285, in
         request
             self._send_request(method, url, body, headers,
         encode_chunked)
           File "/usr/lib64/python3.9/http/client.py", line 1331, in
         _send_request
             self.endheaders(body, encode_chunked=encode_chunked)
           File "/usr/lib64/python3.9/http/client.py", line 1280, in
         endheaders
             self._send_output(message_body,
         encode_chunked=encode_chunked)
           File "/usr/lib64/python3.9/http/client.py", line 1040, in
         _send_output
             self.send(msg)
           File "/usr/lib64/python3.9/http/client.py", line 980, in send
             self.connect()
           File "/usr/lib64/python3.9/http/client.py", line 946, in
         connect
             self.sock = self._create_connection(
           File "/usr/lib64/python3.9/socket.py", line 844, in
         create_connection
             raise err
           File "/usr/lib64/python3.9/socket.py", line 832, in
         create_connection
             sock.connect(sa)
         ConnectionRefusedError: [Errno 111] Connection refused
         ipa: DEBUG: Failed to check CA status: cannot connect to
         'http://mserver.example.com:8080/ca/admin/ca/getStatus':
         [Errno 111] Connection refused
         pki-tomcatd Service: STOPPED
         [...]

         
===================================================================================

         *[root@mserver ~]# getcert list*
         Number of certificates and requests being tracked: 9.
         Request ID '20240325182004':
                 status: MONITORING
                 stuck: no
                 key pair storage:
         
type=NSSDB,location='/etc/dirsrv/slapd-IPA-SS-LAN',nickname='Server-Cert',token='NSS
         Certificate
         DB',pinfile='/etc/dirsrv/slapd-IPA-SS-LAN/pwdfile.txt'
                 certificate:
         
type=NSSDB,location='/etc/dirsrv/slapd-IPA-SS-LAN',nickname='Server-Cert',token='NSS
         Certificate DB'
                 CA: IPA
                 issuer: CN=Certificate Authority,O=IPA.SS.LAN
                 subject: CN=mserver.example.com
         <http://mserver.example.com>,O=IPA.SS.LAN
                 issued: 2024-03-25 20:20:06 EET
                 expires: 2026-03-26 20:20:06 EET
                 dns: mserver.example.com <http://mserver.example.com>
                 principal name: ldap/[email protected]
                 key usage:
         digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
                 eku: id-kp-serverAuth,id-kp-clientAuth
                 profile: caIPAserviceCert
                 pre-save command:
                 post-save command:
         /usr/libexec/ipa/certmonger/restart_dirsrv IPA-SS-LAN
                 track: yes
                 auto-renew: yes
         Request ID '20240325182024':
                 status: MONITORING
                 stuck: no
                 key pair storage:
         
type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/mserver.example.com-443-RSA'
                 certificate:
         type=FILE,location='/var/lib/ipa/certs/httpd.crt'
                 CA: IPA
                 issuer: CN=Certificate Authority,O=IPA.SS.LAN
                 subject: CN=mserver.example.com
         <http://mserver.example.com>,O=IPA.SS.LAN
                 issued: 2024-03-25 20:20:25 EET
                 expires: 2026-03-26 20:20:25 EET
                 dns: mserver.example.com
         <http://mserver.example.com>,ipa-ca.IPA.SS.LAN
                 principal name: HTTP/[email protected]
                 key usage:
         digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
                 eku: id-kp-serverAuth,id-kp-clientAuth
                 profile: caIPAserviceCert
                 pre-save command:
                 post-save command:
         /usr/libexec/ipa/certmonger/restart_httpd
                 track: yes
                 auto-renew: yes
         Request ID '20240325182332':
                 status: CA_WORKING
                 stuck: no
                 key pair storage:
         type=FILE,location='/var/lib/ipa/ra-agent.key'
                 certificate:
         type=FILE,location='/var/lib/ipa/ra-agent.pem'
                 CA: dogtag-ipa-ca-renew-agent
                 issuer: CN=Certificate Authority,O=IPA.SS.LAN
                 subject: CN=IPA RA,O=IPA.SS.LAN
                 issued: 2023-02-28 10:59:37 EET
                 expires: 2025-02-17 10:59:37 EET
                 key usage:
         digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
                 eku: id-kp-serverAuth,id-kp-clientAuth
                 profile: caSubsystemCert
                 pre-save command:
         /usr/libexec/ipa/certmonger/renew_ra_cert_pre
                 post-save command:
         /usr/libexec/ipa/certmonger/renew_ra_cert
                 track: yes
                 auto-renew: yes
         Request ID '20240325182333':
                 status: CA_WORKING
                 stuck: no
                 key pair storage:
         
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
         cert-pki-ca',token='NSS Certificate DB',pin set
                 certificate:
         
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
         cert-pki-ca',token='NSS Certificate DB'
                 CA: dogtag-ipa-ca-renew-agent
                 issuer: CN=Certificate Authority,O=IPA.SS.LAN
                 subject: CN=CA Audit,O=IPA.SS.LAN
                 issued: 2023-02-28 10:58:37 EET
                 expires: 2025-02-17 10:58:37 EET
                 key usage: digitalSignature,nonRepudiation
                 profile: caSignedLogCert
                 pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
                 post-save command:
         /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert
         cert-pki-ca"
                 track: yes
                 auto-renew: yes
         Request ID '20240325182337':
                 status: CA_WORKING
                 stuck: no
                 key pair storage:
         
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
         cert-pki-ca',token='NSS Certificate DB',pin set
                 certificate:
         
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
         cert-pki-ca',token='NSS Certificate DB'
                 CA: dogtag-ipa-ca-renew-agent
                 issuer: CN=Certificate Authority,O=IPA.SS.LAN
                 subject: CN=OCSP Subsystem,O=IPA.SS.LAN
                 issued: 2023-02-28 10:58:47 EET
                 expires: 2025-02-17 10:58:47 EET
                 eku: id-kp-OCSPSigning
                 profile: caOCSPCert
                 pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
                 post-save command:
         /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
         cert-pki-ca"
                 track: yes
                 auto-renew: yes
         Request ID '20240325182339':
                 status: CA_WORKING
                 stuck: no
                 key pair storage:
         type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
         cert-pki-ca',token='NSS Certificate DB',pin set
                 certificate:
         type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
         cert-pki-ca',token='NSS Certificate DB'
                 CA: dogtag-ipa-ca-renew-agent
                 issuer: CN=Certificate Authority,O=IPA.SS.LAN
                 subject: CN=CA Subsystem,O=IPA.SS.LAN
                 issued: 2023-02-28 10:59:27 EET
                 expires: 2025-02-17 10:59:27 EET
                 key usage:
         digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
                 eku: id-kp-serverAuth,id-kp-clientAuth
                 profile: caSubsystemCert
                 pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
                 post-save command:
         /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
         cert-pki-ca"
                 track: yes
                 auto-renew: yes
         Request ID '20240325182340':
                 status: MONITORING
                 stuck: no
                 key pair storage:
         type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
         cert-pki-ca',token='NSS Certificate DB',pin set
                 certificate:
         type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
         cert-pki-ca',token='NSS Certificate DB'
                 CA: dogtag-ipa-ca-renew-agent
                 issuer: CN=Certificate Authority,O=IPA.SS.LAN
                 subject: CN=Certificate Authority,O=IPA.SS.LAN
                 issued: 2017-06-20 18:03:50 EEST
                 expires: 2037-06-20 18:03:50 EEST
                 key usage:
         digitalSignature,nonRepudiation,keyCertSign,cRLSign
                 profile: caCACert
                 pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
                 post-save command:
         /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
         cert-pki-ca"
                 track: yes
                 auto-renew: yes
         Request ID '20240325182341':
                 status: MONITORING
                 stuck: no
                 key pair storage:
         type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
         cert-pki-ca',token='NSS Certificate DB',pin set
                 certificate:
         type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
         cert-pki-ca',token='NSS Certificate DB'
                 CA: dogtag-ipa-ca-renew-agent
                 issuer: CN=Certificate Authority,O=IPA.SS.LAN
                 subject: CN=mserver.example.com
         <http://mserver.example.com>,O=IPA.SS.LAN
                 issued: 2024-03-25 20:22:48 EET
                 expires: 2026-03-15 20:22:48 EET
                 key usage:
         digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
                 eku:
         id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
                 profile: caServerCert
                 pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
                 post-save command:
         /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
         cert-pki-ca"
                 track: yes
                 auto-renew: yes
         Request ID '20240325182408':
                 status: MONITORING
                 stuck: no
                 key pair storage:
         type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
                 certificate:
         type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
                 CA: IPA
                 issuer: CN=Certificate Authority,O=IPA.SS.LAN
                 subject: CN=mserver.example.com
         <http://mserver.example.com>,O=IPA.SS.LAN
                 issued: 2024-03-25 20:24:13 EET
                 expires: 2026-03-26 20:24:13 EET
                 dns: mserver.example.com <http://mserver.example.com>
                 principal name: krbtgt/[email protected]
                 key usage:
         digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
                 eku: id-kp-serverAuth,id-pkinit-KPKdc
                 profile: KDCs_PKINIT_Certs
                 pre-save command:
                 post-save command:
         /usr/libexec/ipa/certmonger/renew_kdc_cert
                 track: yes
                 auto-renew: yes

         
===================================================================================



     From your logs,

       * replication between your servers is broken (could be related
         to the expired certs or the cause for expired certs...)
       * on mserver:
           o PKI server not running
           o the shared PKI certificates are expired (Feb 17 2025):
             ra-agent.pem + auditSigningCert cert-pki-ca +
             ocspSigningCert cert-pki-ca + subsystemCert cert-pki-ca

     Can you check which server is the CA renewal master?
     kinit admin
     ipa config-show | grep renewal
       IPA CA renewal master: *server.ipa.test*

     Then you can force the startup of ipa services on mserver:
     ipactl start --ignore-service-failures

     At this point, check if the replication is working (for instance
     with ipa-healthcheck or by creating a user on mserver and
     ensuring it is present on fserver and vice-versa). Then let us
     know the situation, depending on your answers you will have to
     follow one of the sections of
     
https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/managing_certificates_in_idm/renewing-expired-system-certificates-when-idm-is-offline_managing-certificates-in-idm#renewing-expired-system-certificates-when-idm-is-offline_managing-certificates-in-idm

     flo

     --
     _______________________________________________
     FreeIPA-users mailing list -- [email protected]
     To unsubscribe send an email to
     [email protected]
     Fedora Code of Conduct:
     https://docs.fedoraproject.org/en-US/project/code-of-conduct/
     List Guidelines:
     https://fedoraproject.org/wiki/Mailing_list_guidelines
     List Archives:
     
https://lists.fedorahosted.org/archives/list/[email protected]
     Do not reply to spam, report it:
     https://pagure.io/fedora-infrastructure/new_issue

--
_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedorahosted.org/archives/list/[email protected]
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Reply via email to