Hi Florence,
Thank you for lending help! Please see below ipareplica-conncheck.log.
Thanks.
Kathy.
[root@rh-master2 /]# cat /var/log/ipareplica-conncheck.log
2022-07-12T21:34:41Z DEBUG /usr/sbin/ipa-replica-conncheck was invoked with
options: {'master': 'rh-master1.example.com', 'auto_master_check': True,
'realm': 'EXAMPLE.COM', 'kdc': None, 'principal': None, 'ca_cert_file':
None, 'replica': None, 'check_ca': False, 'hostname': None, 'debug': True,
'quiet': False, 'log_to_file': True}
2022-07-12T21:34:41Z DEBUG missing options might be asked for interactively
later
2022-07-12T21:34:41Z DEBUG IPA version 4.9.8-7.module+el8.6.0+14337+19b76db2
2022-07-12T21:34:41Z INFO Check connection from replica to remote master '
rh-master1.example.com':
2022-07-12T21:34:41Z INFO Directory Service: Unsecure port (389): OK
2022-07-12T21:34:41Z INFO Directory Service: Secure port (636): OK
2022-07-12T21:34:41Z INFO Kerberos KDC: TCP (88): OK
2022-07-12T21:34:41Z INFO Kerberos Kpasswd: TCP (464): OK
2022-07-12T21:34:41Z INFO HTTP Server: Unsecure port (80): OK
2022-07-12T21:34:41Z INFO HTTP Server: Secure port (443): OK
2022-07-12T21:34:41Z INFO
The following list of ports use UDP protocol and would need to be
checked manually:
2022-07-12T21:34:41Z INFO Kerberos KDC: UDP (88): SKIPPED
2022-07-12T21:34:41Z INFO Kerberos Kpasswd: UDP (464): SKIPPED
2022-07-12T21:34:41Z INFO
Connection from replica to master is OK.
2022-07-12T21:34:41Z INFO Start listening on required ports for remote
master check
2022-07-12T21:34:41Z DEBUG Starting listening thread.
2022-07-12T21:34:41Z DEBUG Original thread stopped
2022-07-12T21:34:41Z DEBUG 389 tcp: Started listening
2022-07-12T21:34:41Z DEBUG 636 tcp: Started listening
2022-07-12T21:34:41Z DEBUG 88 tcp: Started listening
2022-07-12T21:34:41Z DEBUG 88 udp: Started listening
2022-07-12T21:34:41Z DEBUG 464 tcp: Started listening
2022-07-12T21:34:41Z DEBUG 464 udp: Started listening
2022-07-12T21:34:41Z DEBUG 80 tcp: Started listening
2022-07-12T21:34:41Z DEBUG 443 tcp: Started listening
2022-07-12T21:34:41Z DEBUG Ports opened, notify original thread
2022-07-12T21:34:41Z DEBUG Original thread resumed
2022-07-12T21:34:41Z INFO Get credentials to log in to remote master
2022-07-12T21:34:41Z DEBUG KRB5CCNAME set to None
2022-07-12T21:34:41Z INFO Check RPC connection to remote master
2022-07-12T21:34:41Z DEBUG importing all plugin modules in
ipaclient.remote_plugins.schema$70235405...
2022-07-12T21:34:41Z DEBUG importing plugin module
ipaclient.remote_plugins.schema$70235405.plugins
2022-07-12T21:34:41Z DEBUG importing all plugin modules in
ipaclient.plugins...
2022-07-12T21:34:41Z DEBUG importing plugin module ipaclient.plugins.vault
......
2022-07-12T21:34:42Z DEBUG found session_cookie in persistent storage for
principal '[email protected]', cookie:
'ipa_session=MagBearerToken=HHqtX4P3Da2m4Dxfh3Qf3Sl0KwkH2NguR6fScYccnvkKFj7jUMWIEY6noxPE0n6sqxXYoYXpmcWvzvNYba2e5Cws4XCI9MdsCTv5a3U9IACWokYhFYwMemYOlF6y2B6am20AlJqi1%2bpRhlrmtVEYwgJAyy%2flvPO%2bkY%2fyuYuNGwRdrO9Db1FQiZl2C%2fp69gL%2bB1EZlkdLZGpORvUbfbgQ9g%3d%3d'
2022-07-12T21:34:42Z DEBUG setting session_cookie into context
'ipa_session=MagBearerToken=HHqtX4P3Da2m4Dxfh3Qf3Sl0KwkH2NguR6fScYccnvkKFj7jUMWIEY6noxPE0n6sqxXYoYXpmcWvzvNYba2e5Cws4XCI9MdsCTv5a3U9IACWokYhFYwMemYOlF6y2B6am20AlJqi1%2bpRhlrmtVEYwgJAyy%2flvPO%2bkY%2fyuYuNGwRdrO9Db1FQiZl2C%2fp69gL%2bB1EZlkdLZGpORvUbfbgQ9g%3d%3d;'
2022-07-12T21:34:42Z DEBUG trying
https://rh-master1.example.com/ipa/session/json
2022-07-12T21:34:42Z DEBUG New HTTP connection (rh-master1.example.com)
2022-07-12T21:34:42Z DEBUG HTTP connection destroyed (rh-master1.example.com
)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ipalib/rpc.py", line 731, in
single_request
response.msg)
xmlrpc.client.ProtocolError: <ProtocolError for
rh-master1.example.com/ipa/session/json: 401 Unauthorized>
2022-07-12T21:34:42Z DEBUG trying
https://rh-master1.example.com/ipa/session/json
2022-07-12T21:34:42Z DEBUG New HTTP connection (rh-master1.example.com)
2022-07-12T21:34:42Z DEBUG HTTP connection destroyed (rh-master1.example.com
)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ipalib/rpc.py", line 731, in
single_request
response.msg)
xmlrpc.client.ProtocolError: <ProtocolError for
rh-master1.example.com/ipa/session/json: 401 Unauthorized>
2022-07-12T21:34:42Z INFO Connection to
https://rh-master1.example.com/ipa/session/json failed with <ProtocolError
for rh-master1.example.com/ipa/session/json: 401 Unauthorized>
2022-07-12T21:34:42Z DEBUG trying
https://centos-master1.example.com/ipa/session/json
2022-07-12T21:34:42Z DEBUG New HTTP connection (centos-master1.example.com)
2022-07-12T21:34:42Z DEBUG received Set-Cookie (<class
'list'>)'['ipa_session=MagBearerToken=drYz94JyUtAu7Ee3Q1V0eTjOz%2fmm5%2fYzUFaAielYZ6NKCbN6UoE1IAQEbjaf4Jkj95xXlVSPGRNflzkax6vWYaFV%2fdQONIYjDja%2f2CO0Txioly7YlTXM0GwPzfsF9xz%2b%2ffZbt%2bNx1nGME2q8lmmLSEwVjNciQUw4gWxE%2bLrCOoqcBNy1kkCSc6K6qaZQkNL2CT24TZ%2b4pCBn0LdFUwHCwA%3d%3d;path=/ipa;httponly;secure;']'
2022-07-12T21:34:42Z DEBUG storing cookie
'ipa_session=MagBearerToken=drYz94JyUtAu7Ee3Q1V0eTjOz%2fmm5%2fYzUFaAielYZ6NKCbN6UoE1IAQEbjaf4Jkj95xXlVSPGRNflzkax6vWYaFV%2fdQONIYjDja%2f2CO0Txioly7YlTXM0GwPzfsF9xz%2b%2ffZbt%2bNx1nGME2q8lmmLSEwVjNciQUw4gWxE%2bLrCOoqcBNy1kkCSc6K6qaZQkNL2CT24TZ%2b4pCBn0LdFUwHCwA%3d%3d;'
for principal [email protected]
2022-07-12T21:34:42Z DEBUG Created connection
context.rpcclient_139909940596464
2022-07-12T21:34:42Z DEBUG raw: ping(version='2.245')
2022-07-12T21:34:42Z DEBUG ping(version='2.245')
2022-07-12T21:34:42Z DEBUG [try 1]: Forwarding 'ping/1' to json server '
https://centos-master1.example.com/ipa/session/json'
2022-07-12T21:34:42Z DEBUG HTTP connection keep-alive (
centos-master1.example.com)
2022-07-12T21:34:42Z DEBUG received Set-Cookie (<class
'list'>)'['ipa_session=MagBearerToken=koarwtvmwt9048z3u2fOYK%2b8hp6xaOi1zYAOg1CvaJDmb1qMrCRiHcIy1IQUx1B7KYZRTQiJqdPSF4RgyJsqhJJTNR%2fgXddTDqgDuUDByRrcJ7sZk8%2bdBPVFg8upcRqE70fOIyZnc1%2f3WQ15pMsMziAhQYv2%2bX1JVXS6u4XxOzRvOfmkwdXv4dectfMCxevVfwBelgOa%2bGkdksahtmr8SA%3d%3d;path=/ipa;httponly;secure;']'
2022-07-12T21:34:42Z DEBUG storing cookie
'ipa_session=MagBearerToken=koarwtvmwt9048z3u2fOYK%2b8hp6xaOi1zYAOg1CvaJDmb1qMrCRiHcIy1IQUx1B7KYZRTQiJqdPSF4RgyJsqhJJTNR%2fgXddTDqgDuUDByRrcJ7sZk8%2bdBPVFg8upcRqE70fOIyZnc1%2f3WQ15pMsMziAhQYv2%2bX1JVXS6u4XxOzRvOfmkwdXv4dectfMCxevVfwBelgOa%2bGkdksahtmr8SA%3d%3d;'
for principal [email protected]
2022-07-12T21:34:42Z INFO Execute check on remote master
2022-07-12T21:34:42Z DEBUG [try 1]: Forwarding 'server_conncheck' to json
server 'https://centos-master1.example.com/ipa/session/json'
2022-07-12T21:34:42Z DEBUG HTTP connection keep-alive (
centos-master1.example.com)
2022-07-12T21:34:42Z DEBUG received Set-Cookie (<class
'list'>)'['ipa_session=MagBearerToken=VxSP42dIPlrB1LQhDGoh3Zngcsj8usqNW1Jyersj%2fISvtKI7gYdBnDN5%2f7UqCOZZKvR1sOMAiTyvYvEfis0oTZHm8a5gR8Q4jxUVDzxkVncfI2%2f4Qh6dCpyDDpBXeqZFJdz4lY%2fDYX2%2be0%2b3h9eKTMeKijX8lWzT3U3BZei8l0loY7pl3Gw60JAj4lIZ%2biBORRAFhxEdgFjstOM3Bazh%2bA%3d%3d;path=/ipa;httponly;secure;']'
2022-07-12T21:34:42Z DEBUG storing cookie
'ipa_session=MagBearerToken=VxSP42dIPlrB1LQhDGoh3Zngcsj8usqNW1Jyersj%2fISvtKI7gYdBnDN5%2f7UqCOZZKvR1sOMAiTyvYvEfis0oTZHm8a5gR8Q4jxUVDzxkVncfI2%2f4Qh6dCpyDDpBXeqZFJdz4lY%2fDYX2%2be0%2b3h9eKTMeKijX8lWzT3U3BZei8l0loY7pl3Gw60JAj4lIZ%2biBORRAFhxEdgFjstOM3Bazh%2bA%3d%3d;'
for principal [email protected]
2022-07-12T21:34:42Z DEBUG Destroyed connection
context.rpcclient_139909940596464
2022-07-12T21:34:42Z ERROR ERROR: Remote master check failed with following
error message(s):
invalid 'cn': must be "centos-master1.example.com"
2022-07-12T21:34:42Z DEBUG Stopping listening thread.
2022-07-12T21:34:43Z DEBUG 389 tcp: Stopped listening
2022-07-12T21:34:43Z DEBUG 636 tcp: Stopped listening
2022-07-12T21:34:43Z DEBUG 88 tcp: Stopped listening
2022-07-12T21:34:43Z DEBUG 88 udp: Stopped listening
2022-07-12T21:34:43Z DEBUG 464 tcp: Stopped listening
2022-07-12T21:34:43Z DEBUG 464 udp: Stopped listening
2022-07-12T21:34:43Z DEBUG 80 tcp: Stopped listening
2022-07-12T21:34:43Z DEBUG 443 tcp: Stopped listening
[root@rh-master2 /]#
On Wed, Jul 13, 2022 at 12:08 AM Florence Blanc-Renaud <[email protected]>
wrote:
>
>
> On Wed, Jul 13, 2022 at 12:46 AM Kathy Zhu via FreeIPA-users <
> [email protected]> wrote:
>
>> Hi Rob,
>>
>> On a different topic, we started migration from Centos 7 to Red Hat 8.6 over
>> the weekend, after adding the first Red Hat master and moved CA renewal and
>> CRL generation roles to it, then we tried to add the second Red Hat master
>> via the first Red Hat master, after many tries without success. The keytabs
>> on the first master seem messed up. We wonder if it is possible or safe to
>> back out.
>>
>> The current domain is all Centos 7 masters and one Red Hat 8 master with CA
>> renewal and CRL generation role. By backing out, I mean to move the CA
>> renewal and CRL generation role back to a Centos 7 master, then remove the
>> Red Hat 8 master. So we will be back to the way before the migration. Then
>> we have a freshly installed red hat server and try the migration process
>> again.
>>
>> Is this safe to do?
>>
>> Hi,
> I don't see any issue with moving back CA renewal and CRL generation roles
> back to the Centos 7 server. But maybe you can share the failed
> installation logs for us to help you troubleshoot the replica installation
> issue?
>
> flo
>
>> Thanks.
>>
>> Kathy.
>>
>>
>>
>> On Tue, Jul 12, 2022 at 2:18 PM Kathy Zhu wrote:
>>
>>> Hi Rob,
>>>
>>> Thank you!
>>>
>>> We are migrating to Red Hat 8.6, that master will be replaced. So far,
>>> we do not see any issue yet.
>>>
>>> The outputs from "dsconf slapd-EXAMPLE-COM repl-conflict list o=ipaca"
>>> are binaries. Have no clue what that means :-).
>>>
>>> Many thanks for your help! It made our domain cleaner. Appreciate it.
>>>
>>> Kathy.
>>>
>>> On Tue, Jul 12, 2022 at 2:03 PM Rob Crittenden <[email protected]>
>>> wrote:
>>>
>>>> Kathy Zhu via FreeIPA-users wrote:
>>>
>>> > Hi Rob,
>>>> >
>>>> > Thank you!
>>>> >
>>>> > It worked! There were 4 bad entries! However, I made a mistake by
>>>> > deleting a valid one :-(. Could you please share how to add it back?
>>>> Or
>>>> > should I reinstall it?
>>>>
>>>> I don't know how to re-add one or what repercussions there are (the CA
>>>> is still treated very much like a black box). Re-installing is the
>>>> safest bet.
>>>>
>>>> >
>>>> > ipa-healthcheck is no longer complain about the same. However, I still
>>>> > see the warning:
>>>> >
>>>> > # ipa-healthcheck --failures-only --output-type=human
>>>> >
>>>> > Unhandler rdtype 256
>>>> >
>>>> > Unhandler rdtype 256
>>>> >
>>>> > Unhandler rdtype 256
>>>> > ...
>>>> >
>>>> > Unhandler rdtype 256
>>>> >
>>>> > WARNING: ipahealthcheck.ds.replication.ReplicationCheck.DSREPLLE0002:
>>>> > There were 118 conflict entries found under the replication suffix
>>>> > "dc=corp,dc=nuro,dc=team".
>>>> >
>>>> > WARNING: ipahealthcheck.ds.replication.ReplicationCheck.DSREPLLE0002:
>>>> > There were 15 conflict entries found under the replication suffix
>>>> "o=ipaca".
>>>> > #
>>>> >
>>>> > Note the last line :
>>>> >
>>>> > There were 15 conflict entries found under the replication suffix
>>>> "o=ipaca".
>>>> >
>>>> > We have 11 valid ones plus 4 old removed ones, that is total 15.
>>>> > Somewhere in IPA still shows 15.
>>>>
>>>> They must be there somewhere. It is a 389-ds check that returns these
>>>> results. I'd try: dsconf slapd-YOUR_INSTANCE repl-conflict list o=ipaca
>>>>
>>>> rob
>>>>
>>>> >
>>>> > Many thanks.
>>>> >
>>>> > Kathy.
>>>> >
>>>> > On Mon, Jul 11, 2022 at 7:24 PM Rob Crittenden <[email protected]
>>>> > <mailto:[email protected]>> wrote:
>>>> >
>>>> > Kathy Zhu via FreeIPA-users wrote:
>>>> > > Hi Team,
>>>> > >
>>>> > >
>>>> > > We are migrating from Centos 7 IPA to Red Hat 8.6. After adding
>>>> the
>>>> > > first Red Hat master, it reported error:
>>>> > >
>>>> > >
>>>> > > # ipa-healthcheck
>>>> > > --source=pki.server.healthcheck.clones.connectivity_and_data
>>>> > >
>>>> > > Internal server error HTTPSConnectionPool(host='
>>>> ipa4.example.com
>>>> > <http://ipa4.example.com>
>>>> > > <http://ipa4.example.com>', port=443): Max retries exceeded
>>>> with url:
>>>> > > /ca/rest/certs/search?size=3 (Caused by
>>>> > > NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection
>>>> object
>>>> > > at 0x7f0611b6d5c0>: Failed to establish a new connection:
>>>> [Errno -2]
>>>> > > Name or service not known',))
>>>> > >
>>>> > > [
>>>> > >
>>>> > > {
>>>> > >
>>>> > > "source":
>>>> "pki.server.healthcheck.clones.connectivity_and_data",
>>>> > >
>>>> > > "check": "ClonesConnectivyAndDataCheck",
>>>> > >
>>>> > > "result": "ERROR",
>>>> > >
>>>> > > "uuid": "bfb9aeac-2e86-4d1d-ac2a-3cb62300527e",
>>>> > >
>>>> > > "when": "20220711221016Z",
>>>> > >
>>>> > > "duration": "0.768881",
>>>> > >
>>>> > > "kw": {
>>>> > >
>>>> > > "status": "ERROR: pki-tomcat : Internal error testing CA
>>>> clone.
>>>> > > Host: ipa4.example.com <http://ipa4.example.com>
>>>> > <http://ipa4.example.com> Port: 443"
>>>> > >
>>>> > > }
>>>> > >
>>>> > > }
>>>> > >
>>>> > > ]
>>>> > >
>>>> > > #
>>>> > >
>>>> > >
>>>> > > ipa4 was a master we had years ago. it did not show up as a
>>>> dangling
>>>> > > master in the domain. However, it remains in pki DB. How to
>>>> safely
>>>> > clean
>>>> > > it out from pki DB?
>>>> >
>>>> > IPA wasn't cleaning up the security domain on server removal until
>>>> > relatively recently.
>>>> >
>>>> > You can find the list of servers with:
>>>> >
>>>> > # pki securitydomain-host-find
>>>> >
>>>> > You can remove one with with:
>>>> >
>>>> > # pki -d /etc/pki/pki-tomcat/alias/ -n 'subsystemCert
>>>> cert-pki-ca' -C
>>>> > /etc/pki/pki-tomcat/alias/pwdfile.txt securitydomain-host-del 'CA
>>>> > ipa.example.test 443'
>>>> >
>>>> > Be very careful as you can remove valid ones just as easily.
>>>> >
>>>> > > Another interesting fact I like to point out - Centos 7
>>>> > ipa-healthcheck
>>>> > > does not report this.
>>>> >
>>>> > The epel-7 build of ipa-healthcheck I did was a one-off. The
>>>> differences
>>>> > were just too great to keep it in sync. It's an incentive to
>>>> upgrade to
>>>> > RHEL 8 (or 9).
>>>> >
>>>> > rob
>>>> >
>>>>
>>>> _______________________________________________
>> FreeIPA-users mailing list -- [email protected]
>> To unsubscribe send an email to
>> [email protected]
>> Fedora Code of Conduct:
>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>> https://lists.fedorahosted.org/archives/list/[email protected]
>> Do not reply to spam on the list, report it:
>> https://pagure.io/fedora-infrastructure
>>
>
_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/[email protected]
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
