Any idea if there are other steps I can consider? If not, should I simply rebuild the system? If I do go this route, is there a way to backup all the Puppet configurations set for servers and services that can be reimported in a fresh install? Would it also be suggested to go a non-Docker route due to stability?
On Thursday, November 15, 2018 at 1:33:43 PM UTC-8, Morgan Rhodes wrote: > > Hi Rohit, > > No, unfortunately, it's not just a change in your docker-compose.yml. When > you're generating the certs for your puppetserver, you'll want to make sure > you're passing the `--dns_alt_names=<altnames>`, so it would be something > like: > puppet cert generate puppet4.psd401.net --dns_alt_names=puppet, > puppet.psd401.net > > Afterwards, you can confirm that your certificate has all of the altnames > with `puppet cert list --all`, you should see something like: > $ puppet cert list --all > + "puppet4.psd401.net" (SHA256) <fingerprint> (alt names: "DNS:puppet", > "DNS:puppet4.psd401.net") > > On Tue, Nov 13, 2018 at 11:23 AM Rohit <[email protected] > <javascript:>> wrote: > >> Hello Morgan, >> >> Apologies for the late response here, some of our Puppet services had >> started working but it looks like the same issue has arised and I am not >> entirely sure why. I did check the docker-entrypoint.sh file and indeed see >> the very exact response as you posted. However my question is for the >> "altname" that you suggested, would I change this in the docker-compose.yml >> file? I also realize the full docker-compose.yml did not show up in my >> previous post but have attached it again in a separate file. >> >> >> On Friday, October 19, 2018 at 4:38:12 PM UTC-7, Morgan Rhodes wrote: >> >>> When you look at the output of `puppet cert list all` does the >>> certificate for your puppetmaster also include the alt name 'puppet'? >>> (Something like 'alt names: "DNS:puppet", "DNS:testpuppet"'). If not, I'm >>> guessing that's your problem. >>> >>> You mentioned in your earlier email that you were using puppetdb 4.2.0. >>> I'm assuming you're running the puppet/puppetdb:4.2.0 container. To get the >>> container entrypoint, I start the container manually with a custom >>> entrypoint so I can look around, there should be a file >>> 'docker-entrypoint.sh' in the root directory of the container. >>> >>> $ docker run --rm -it --entrypoint /bin/bash puppet/puppetdb:4.2.0 >>> root@e09f677618d7:/# ls >>> Dockerfile bin boot dev docker-entrypoint.sh etc home lib lib64 >>> media mnt opt proc root run sbin srv sys tmp usr var >>> root@e09f677618d7:/# cat docker-entrypoint.sh >>> #!/bin/bash >>> >>> if [ ! -d "/etc/puppetlabs/puppetdb/ssl" ]; then >>> while ! nc -z puppet 8140; do >>> sleep 1 >>> done >>> set -e >>> /opt/puppetlabs/bin/puppet agent --verbose --onetime --no-daemonize >>> --waitforcert 120 >>> /opt/puppetlabs/server/bin/puppetdb ssl-setup -f >>> fi >>> >>> exec /opt/puppetlabs/server/bin/puppetdb "$@" >>> root@e09f677618d7:/# >>> >>> The docker-entrypoint.sh script in that version of the container doesn't >>> have any logic for a puppetserver with a non-default name, which means when >>> it runs `puppet agent --verbose --onetime --no-daemonize --waitforcert 120` >>> it will connect to the host named 'puppet'. From the link you have set up >>> in your docker-compose.yml, I'm assuming your puppetserver container name >>> is 'puppet' with the hostname 'puppet4.psd401.net'. Since the container >>> name is 'puppet', the puppetdb container is able to resolve 'puppet' as >>> 'puppet4....', so when it runs puppet agent -t it can connect to the host, >>> but certificate validation will fail if puppet isn't listed as one of the >>> valid altnames for the puppet container. >>> >>> On Fri, Oct 19, 2018 at 11:35 AM Rohit <[email protected]> >>> wrote: >>> >> >>>> 1. puppet_db is trying to connect our.puppet.domain, there is no >>>> docker-entrypoint.sh script that I was able to find. >>>> 1. For reference, this is the docker-compose.yml: >>>> 2. >>>> >>>> puppetdb: >>>> >>>> container_name: puppet_db >>>> >>>> hostname: puppetdb.peninsula.wednet.edu >>>> >>>> dns: >>>> >>>> - 10.0.0.7 >>>> >>>> image: puppet/puppetdb:latest >>>> >>>> ports: >>>> >>>> - 8087:8080 >>>> >>>> - 8088:8081 >>>> >>>> depends_on: >>>> >>>> - puppet >>>> >>>> links: >>>> >>>> - puppet:puppet4.psd401.net >>>> >>>> - puppetdbpostgres:postgres >>>> >>>> volumes: >>>> >>>> - ./puppet-client.conf:/etc/puppetlabs/puppet/puppet.conf >>>> >>>> - ./puppetdb_conf:/etc/puppetlabs/puppetdb/conf.d >>>> >>>> - ./puppetdb_ssl:/etc/puppetlabs/puppet/ssl/ >>>> >>>> networks: >>>> >>>> puppet: >>>> >>>> ipv4_address: 172.19.0.4 >>>> >>>> restart: always >>>> >>>> >>>> 2. The hostname that the puppetdb container is trying to connect >>>> to is indeed the one listed on the certificate name on the puppet >>>> servers >>>> cert. >>>> >>>> >>>> On Friday, October 19, 2018 at 10:09:56 AM UTC-7, Morgan Rhodes wrote: >>>> >>>>> A few things to verify: >>>>> >>>>> 1) what hostname is your puppetdb container trying to connect to >>>>> puppetserver at? >>>>> a) This should be in your docker-entrypoint.sh script in the >>>>> puppetdb container. Likely either 'puppet' or '$PUPPETSERVER_HOSTNAME' >>>>> depending on what variables you have set in your compose file and what >>>>> version of the puppetdb container you have. >>>>> >>>>> 2) Is the hostname your puppetdb container is trying to connect to >>>>> listed as one of the certificate names for your puppet server's cert? >>>>> a) For example, in my puppetserver container when I run `puppet >>>>> cert list --all` I see: >>>>> >>>>> + "testserver" (SHA256) >>>>> F0:31:6D:1D:03:82:C0:84:0D:FA:2B:28:5B:52:CB:18:88:87:61:5F:5A:F5:7E:AB:A2:73:29:44:BC:57:D0:99 >>>>> >>>>> (alt names: "DNS:testserver", "DNS:foo") >>>>> >>>>> if my puppetdb container tries to connect to that host over any >>>>> names other than 'testserver' or 'foo' I get a certificate verify failed >>>>> error. >>>>> >>>>> >>>>> On Fri, Oct 19, 2018 at 9:02 AM Rohit <[email protected]> >>>>> wrote: >>>>> >>>> Hello Morgan, >>>>>> >>>>>> If you are refferring to the cert being in the conf/ssl/certs folder, >>>>>> then yes, our.puppet.domain.pem is in the folder. When running the >>>>>> 'puppet >>>>>> cert list --all' I see three certificates (in the SHA256 format): >>>>>> >>>>>> - computername.our.puppet.domain >>>>>> - our.puppet.domain >>>>>> - servername.our.puppet.domain >>>>>> >>>>>> If it is a DNS issue, do I have to likely change something from the >>>>>> docker-compose side? >>>>>> >>>>>> >>>>>> On Thursday, October 18, 2018 at 2:14:54 PM UTC-7, Morgan Rhodes >>>>>> wrote: >>>>>>> >>>>>>> Hi Rohit, >>>>>>> >>>>>>> Is the hostname from `/CN=our.puppet.domain` showing up in your >>>>>>> puppetserver's certificate? You can verify that with `puppet cert list >>>>>>> --all` on the puppetserver container. This looks like a DNS issue. >>>>>>> >>>>>>> On Thursday, October 18, 2018 at 11:41:16 AM UTC-7, Rohit wrote: >>>>>>>> >>>>>>>> Hello, we currently have a puppet docker container setup and are >>>>>>>> experiencing certificate issues. Basically, in our docker setup (on >>>>>>>> our >>>>>>>> main server) I had generated and signed new certificates, but the >>>>>>>> puppet_db >>>>>>>> container keeps restarting. Here are logs from the puppet_db container: >>>>>>>> >>>>>>>> ‘Error: Could not retrieve catalog from remote server: >>>>>>>> SSL_connect returned=1 errno=0 state=error: certificate verify failed: >>>>>>>> [unable to get local issuer certificate for /CN=our.puppet.domain] >>>>>>>> Error: Could not retrieve catalog; skipping run >>>>>>>> Error: Could not send report: SSL_connect returned=1 errno=0 >>>>>>>> state=error: certificate verify failed: [unable to get local issuer >>>>>>>> certificate for /CN=our.puppet.domain]’ >>>>>>>> >>>>>>>> I have tried series of steps to solve this problem as it looks like >>>>>>>> Puppet is not functioning correctly as our servers are not properly >>>>>>>> listening to the host server. Any idea what I can do to solve this >>>>>>>> problem? >>>>>>>> For reference, we are running Puppet_DB version 4.2 and Puppet Server >>>>>>>> version 2.7.2, all of which is set up on a docker container >>>>>>>> environment on >>>>>>>> one server. >>>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Puppet Users" group. >>>>>> >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to [email protected]. >>>>> >>>>> >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- >>>>> Morgan Rhodes >>>>> [email protected] >>>>> Release Engineer >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Puppet Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>> Morgan Rhodes >>> [email protected] >>> Release Engineer >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com >> >> <https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > Morgan Rhodes > [email protected] > Release Engineer > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/9d319859-8f13-4e6e-97c9-c2366152a4e3%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
