On Wed, Sep 16, 2020 at 11:29:05AM +1000, Fraser Tweedale via FreeIPA-users
wrote:
> On Tue, Sep 15, 2020 at 02:20:49PM -0000, Boris Sukhinin via FreeIPA-users
> wrote:
> > I try to add a new replica to a cluster of 3 freeipa servers.
> > ipa-replica-install --setup-ca fails with an error:
> > [5/28]: configuring certificate server instance
> > ipaserver.install.dogtaginstance: CRITICAL Failed to configure CA instance:
> > Command '/usr/sbin/pkispawn -s CA -f /tmp/tmp_3KfOZ' returned non-zero exit
> > status 1
> > ipaserver.install.dogtaginstance: CRITICAL See the installation logs and
> > the following files/directories for more information:
> > ipaserver.install.dogtaginstance: CRITICAL /var/log/pki/pki-tomcat
> > [error] RuntimeError: CA configuration failed.
> >
> > /var/log/pki/pki-tomcat/ca/debug gives more info:
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: === Processing ocsp_signing
> > cert ===
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: === Processing sslserver cert
> > ===
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]:
> > SystemConfigService.processKeyPair(sslserver)
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService:
> > san_server_cert not found
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService: loading
> > existing key pair from NSS database
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils:
> > loadKeyPair(Server-Cert cert-pki-ca, )
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService: storing
> > key pair into CS.cfg
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils:
> > storeKeyPair(sslserver)
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]:
> > SystemConfigService.processCert(sslserver)
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: SystemConfigService: checking
> > sslserver cert in NSS database
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configCert: caType is local
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configCert: caType is remote
> > (revised)
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils:
> > updateConfig() for certTag sslserver
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: updateConfig() done
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configCert: remote CA
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: confgCert: tag: sslserver
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertRequestPanel: got public
> > key
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertRequestPanel: got private
> > key
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils: For this
> > Cloned CA, always use its Master CA to generate the 'sslserver' certificate
> > to avoid any changes which may have been made to the X500Name directory
> > string encoding order.
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils:
> > injectSAN: false
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configRemoteCert: tag:
> > sslserver : setting profileId to: caInternalAuthServerCert
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: configRemoteCert: tag:
> > sslserver calculated profileId: caInternalAuthServerCert
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertUtil: content:
> > {xmlOutput=[true], cert_request_type=[pkcs10],
> > profileId=[caInternalAuthServerCert], cert_request=[...cut...],
> > requestor_name=[CA-srvXXX.local.domain-8443],
> > sessionID=[6184760106499759096]}
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: ConfigurationUtils: POST
> > https://srvYYY.local.domain:443/ca/ee/ca/profileSubmit
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertUtil: status: 1
> > [15/Sep/2020:15:42:06][http-bio-8443-exec-3]: CertUtil: error: Request
> > 19990005 - Server Internal Error
> > java.io.IOException: Request 19990005 - Server Internal Error
> > at
> > com.netscape.cms.servlet.csadmin.CertUtil.createRemoteCert(CertUtil.java:103)
> > at
> > com.netscape.cms.servlet.csadmin.ConfigurationUtils.configRemoteCert(ConfigurationUtils.java:2737)
> > at
> > com.netscape.cms.servlet.csadmin.ConfigurationUtils.configCert(ConfigurationUtils.java:2593)
> > at
> > org.dogtagpki.server.rest.SystemConfigService.processCert(SystemConfigService.java:484)
> > at
> > org.dogtagpki.server.rest.SystemConfigService.processCerts(SystemConfigService.java:303)
> > at
> > org.dogtagpki.server.rest.SystemConfigService.configure(SystemConfigService.java:166)
> > at
> > org.dogtagpki.server.rest.SystemConfigService.configure(SystemConfigService.java:101)
> > <...long stacktrace...>
> >
> > LDAP access logs on existing server show that certificate request ID was
> > reused. ADD queries for request as well as for certificate have failed, but
> > nevertheless the request was overwritten with a third query:
> > [15/Sep/2020:15:42:06.259655845 +0300] conn=2608 op=1804 ADD
> > dn="cn=19990005,ou=ca,ou=requests,o=ipaca"
> > [15/Sep/2020:15:42:06.260525381 +0300] conn=2608 op=1804 RESULT err=68
> > tag=105 nentries=0 etime=0.0001251547
> > [15/Sep/2020:15:42:06.295774388 +0300] conn=2608 op=1805 ADD
> > dn="cn=536805381,ou=certificateRepository,ou=ca,o=ipaca"
> > [15/Sep/2020:15:42:06.296210847 +0300] conn=2608 op=1805 RESULT err=68
> > tag=105 nentries=0 etime=0.0000755866
> > [15/Sep/2020:15:42:06.301651990 +0300] conn=2608 op=1806 MOD
> > dn="cn=19990005,ou=ca,ou=requests,o=ipaca"
> > [15/Sep/2020:15:42:06.315983780 +0300] conn=2608 op=1806 RESULT err=0
> > tag=103 nentries=0 etime=0.0014827998 csn=5f60b69f000000110000
> >
> > I can confirm that each server has its own request ID range configured in
> > /etc/pki/pki-tomcat/ca/CS.cfg:
> > 1. dbs.beginRequestNumber=19970001
> > dbs.endRequestNumber=19980000
> > 2. dbs.beginRequestNumber=19980001
> > dbs.endRequestNumber=19990000
> > 3. dbs.beginRequestNumber=19990001
> > dbs.endRequestNumber=20000000
> >
> > So my questions are:
> >
> > 1. How does PKI track request IDs and prevent their reuse?
> >
>
> There are two aspects to this. One is replica range management.
> You have already checked the replica ranges so that seems OK. More
> details about replica range management in this blog post[1], if you
> are interested.
>
> [1]
> https://frasertweedale.github.io/blog-redhat/posts/2019-07-26-dogtag-replica-ranges.html
>
> The other aspect is VLV indices. During startup, Dogtag uses a VLV
> search to find the next unused identifier in its active range. If
> VLV indicies are corrupt or incomplete, it can result in this
> behaviour.
>
> > 2. Does overwriting certificate request in LDAP affect certificate
> > renewal or have any other negative consequences?
> >
> It depends on the verison of FreeIPA. Older versions used
> serial-based renewal for system certificates. This can cause big
> problems if there are conflicts or old requests have been
> overwritten. Since freeipa-4.8.1 we perform a "fresh enrolment"
> when renewing system certificates, to avoid such problems.
>
> > 3. What's the best way to fix this problem? LDAP backups are
> > available and restoring o=ipaca is an option. Domain database
> > rollback is not an option though.
> >
> Next step is to stop Dogtag, rebuild VLV indices, restart Dogtag and
> see if the problem went away. I'm not quite sure how to do this.
> I'll reply again when I work it out - or maybe someone familiar with
> DS can advise.
>
You can use the following script to reindex the VLV indices:
$ /bin/cp /usr/share/pki/ca/conf/vlvtasks.ldif .
$ sed -i "s/{instanceId}/pki-tomcat/g" vlvtasks.ldif
$ sed -i "s/{database}/ipaca/g" vlvtasks.ldif
$ ldapadd -x -D "cn=Directory Manager" -w $DM_PASS -f vlvtasks.ldif
Then wait for the task to complete successfully; check the progress
with this search:
$ ldapsearch -x -D "cn=Directory Manager" -w $DM_PASS \
-b "cn=index1160589769,cn=index,cn=tasks,cn=config"
Note that the task object is only retained for 10 seconds after the
task finishes. If you want to keep it a longer time, change the
`ttl' attribute in vlvtasks.ldif before creating the object.
After reindexing is complete, restart Dogtag.
Cheers,
Fraser
_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/[email protected]
