Sorry for the red herring, the enrollment issue was someone updating a secret improperly that was breaking the auth. Enrollment is not affected.
On Mon, Feb 10, 2025 at 9:49 AM Rob Crittenden <[email protected]> wrote: > Russell Long wrote: > > Just to add, and I apologize for not noticing this before, but we do not > > use it for anything other than internal FreeIPA operations, but it > > appears the CA itself is not working. I cannot enroll new hosts because > > of this. Login to existing hosts seems to still work without issue > > however. > > I'm surprised it is affecting enrollment. Are you obtaining a > certificate for the host at the same time ala --request-cert? > > If not how is it failing? > > rob > > > > > On Thu, Feb 6, 2025 at 12:45 PM Rob Crittenden via FreeIPA-users > > <[email protected] > > <mailto:[email protected]>> wrote: > > > > Russell and I did a bit of offline troubleshooting but unfortunately > > didn't find anything. > > > > I'm cc'ing a couple of the PKI developers. They would know know how > to > > verify that the webapp(s) are registered properly and may be able to > > tell us why we're seeing 404's. > > > > rob > > > > Russell Long wrote: > > > With pki-tomcatd@pki-tomcat running, the ipa-cert-show gives a > > > connection error: > > > > > > [root@ipa-primary ~]# openssl x509 -serial -noout -in > /etc/ipa/ca.crt > > > serial=01 > > > [root@ipa-primary ~]# ipa cert-show 01 > > > ipa: ERROR: Request failed with status 404: Non-2xx response from > CA > > > REST API: 404. (404) > > > > > > --Russ > > > > > > On Wed, Feb 5, 2025 at 2:06 PM Rob Crittenden <[email protected] > > <mailto:[email protected]> > > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > > > Russell Long wrote: > > > > Here is the obfuscated sosreport. > > > > > > It looks like the CA restart that happened immediately before > > the first > > > 404 was successful. At least it doesn't report any errors > > beyond the > > > usual LDAP connection failures at startup. > > > > > > The startup looks to be basically done around 2025-02-03 > 14:37:56 > > > > > > The CA returned 404's at 2025-02-03T19:39:34Z in the upgrade > log. > > > > > > We don't have the tomcat access logs in the sosreport so we > > can't see > > > the requests but they would likely also report 404 and no > > other details. > > > > > > If you manually start things can you communicate with the CA? > > > > > > # ipactl --skip-version-check > > > > > > Does the CA start? If not add --ignore-service-failures > > > > > > Once everything else is up and settled, if the CA start failed > > run: > > > systemctl restart pki-tomcatd@pki-tomcat > > > > > > And see if that is successful. I think it should succeed since > it > > > appears to have done so in the recent past. > > > > > > If so try a basic cert command: > > > # openssl x509 -serial -noout -in /etc/ipa/ca.crt > > > # ipa cert-show <that serial number> > > > > > > Does it give you data or a connection error? > > > > > > rob > > > > > > > > > > > > > > > On Wed, Feb 5, 2025 at 2:33 AM Alexander Bokovoy > > > <[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>> > > > > <mailto:[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>>> wrote: > > > > > > > > On Пан, 03 лют 2025, Russell Long via FreeIPA-users > wrote: > > > > >Here is the log, sorry for the delay. Logs are > > redacted, but the > > > > only thing > > > > >changed was the domain names and DNs. > > > > > > > > The upgrade log chokes on the CA application not being > > > registered in > > > > tomcat container (the corresponding /ca/rest/... path is > > > giving 404 > > > > error). > > > > > > > > So we get back to the same point as before. An upgrade > > has been in > > > > progress but somehow was interrupted. Directory server > was > > > having some > > > > of listeners disabled to avoid external communication > during > > > the upgrade > > > > and those listeners weren't recovered due to an > > interruption. You > > > > recovered some of them but it looks like there is still > > > something that > > > > messes up. > > > > > > > > If you are saying all services are working fine, just the > > > upgrade kicks > > > > in every time 'ipactl restart' is run (which is part of > > > ipa.service > > > > machinery), it means the logged IPA data version is > > older than > > > what IPA > > > > sees in the RPM database. Temporarily, this can be fixed > by > > > looking at > > > > /var/lib/ipa/sysupgrade/sysupgrade.state and changing > > > ipa.data_version > > > > value to be exact same as the RPM package > > version-release values. > > > > > > > > However, it would help to understand why an upgrade > > causes CA > > > apps to > > > > fail to register with the tomcat container. It looks > like we > > > have at > > > > least three such cases on this list over past week or > > so, all > > > on CentOS > > > > 9 Stream, so there might be something? > > > > > > > > May be you can install sos report tool and collect a > larger > > > amount of > > > > data altogether so that we can see a greater picture? > > > > > > > > # dnf install sos > > > > # sos report > > > --profile={identity,security,system,services,network} > > > > --clean -a > > > > > > > > This should produce logs with consistently obfuscated > > > hostnames and > > > > domains across all files. You can add more domains to > > > obfuscate with > > > > `--domains={domain1,domain2,..}` to `sos report` tool. > > > > > > > > > > > > > > > > > > > > > > > >On Wed, Jan 29, 2025 at 4:51 PM Rob Crittenden > > > <[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>> > > > > <mailto:[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>>> wrote: > > > > > > > > > >> Russ Long via FreeIPA-users wrote: > > > > >> > Things are functional, however IPA still thinks it > > needs an > > > > upgrade, so > > > > >> any time the service restarts, it breaks again. > > > > >> > > > > > >> > > > > >> If you have time to run the upgrade again and send us > a > > > compressed > > > > >> /var/log/ipaupgrade.log we can see if we can identify > the > > > root cause. > > > > >> > > > > >> rob > > > > >> > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > / Alexander Bokovoy > > > > Sr. Principal Software Engineer > > > > Security / Identity Management Engineering > > > > Red Hat Limited, Finland > > > > > > > > > > > -- > > _______________________________________________ > > FreeIPA-users mailing list -- [email protected] > > <mailto:[email protected]> > > To unsubscribe send an email to > > [email protected] > > <mailto:[email protected]> > > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > > > https://lists.fedorahosted.org/archives/list/[email protected] > > Do not reply to spam, report it: > > https://pagure.io/fedora-infrastructure/new_issue > > > >
-- _______________________________________________ FreeIPA-users mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/[email protected] Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
