On Fri, Nov 30, 2018 at 7:39 AM Darragh Bailey <[email protected]>
wrote:
>
>
> On Thursday, November 29, 2018 at 11:27:13 AM UTC, Darragh Bailey wrote:
>>
>>
>>
>> On Wednesday, November 28, 2018 at 6:32:44 PM UTC, Eric Sorenson wrote:
>>>
>>> Hi Darragh, the fact that the error message contains a '400' error
>>> suggests the problem happens on the server when it receives the report.
>>>
>>> My first guess given that error message is also that there's a mix of
>>> versions installed, but it's weird that it only happens on some reports.
>>> Maybe there is something malformed in those reports that triggers a
>>> different code path on the server.
>>>
>>
>> Indeed there is, I missed that there were errors further back in the
>> output log previously and assumed the failure to post the report
>> successfully was the error. Seems they are not compilation errors but
>> rather runtime errors where something unexpected occurred in applying the
>> catalog.
>>
>> For successful runs there is no problem in posting back the report, this
>> at least now gives me something to reproduce consistently, just add an
>> error somewhere that will trigger it.
>>
>
> So confirmed I can reproduce this by simply adding the following block to
> the site.pp for our vagrant environment
>
> exec { 'break the run':
> command => '/bin/false',
> }
>
> However it only triggers the error on the first run, once there is a
> successful run, can't trigger the problem subsequently.
>
This makes me think it's related to the transactionstore functionality (it
keeps track of what the "current" value is the last time puppet applied a
catalog, so that it can tell if a change is due to an intentional manifest
change or the system drifted and needed to be remediated). I say that
because the transaction store records information differently depending if
the event was successful or not. Might want to look in `puppet config print
transactionstorefile --section agent` on the agent to see if there are any
clues.
Also earlier you mentioned:
> Warning: Event['previous_value'] contains a Process::Status value. It
will be converted to the String 'pid 30408 exit 1'
The "previous_value" for an exec should always be "notrun" or the desired
value (due to the onlyif/unless/creates checks causing the exec to skip). I
wonder if somehow the previous system from the transactionstore is ending
up in the next report?
>>
>>> You can save a copy of the reports by adding `store` to the type of
>>> report submission on the master: `reports = https,store` and see what they
>>> look like. They should go into a subdirectory
>>> of /opt/puppetlabs/puppet/cache/reports
>>>
>>
>>> HTH
>>> --eric0
>>>
>>
>> Ah, fantastic, didn't think of that! That will definitely help, many
>> thanks.
>>
>>
> So apparently 'store' is the default based on
> https://puppet.com/docs/puppet/5.4/reporting_about.html#configuring-reporting
> and it should be 'http' just in case anyone else encounters this.
>
> But it doesn't appear to have any effect in capturing the report being
> sent. Might need to make use of nginx and report_port as a way to forward
> on the request while logging it's content it so that I can grab what was
> sent.
>
> Looking on the agent node I see the
> /var/lib/puppet/state/last_run_summary.yaml containing
>
> events:
> failure: 1
> success: 362
> total: 363
>
> But unfortunately there's no /var/lib/puppet/state/last_run_report.yaml
> that corresponds, as it doesn't appear to get updated due to the failure,
> so there's no actual record of the report being sent to the master.
>
You may be running into https://tickets.puppetlabs.com/browse/PUP-6708?
> --
> Darragh Bailey
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/45e5e998-f3a1-4aa1-8066-5bed2a4dab29%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/45e5e998-f3a1-4aa1-8066-5bed2a4dab29%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
Josh Cooper | Software Engineer
[email protected] | @coopjn
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CA%2Bu97ukEMbCtZZzZEP5L1Y1Gjjwy1BwMuUCGrwOcU3Cpu49TcQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
