On Fri, Jan 29, 2016 at 2:04 AM, Erik Dalén <[email protected]> wrote:
> Well, there was an old thread here about this. The conclusions were > summarized here: http://projects.puppetlabs.com/issues/2198#note-41 > Yes, and that thinking was somewhat captured in this epic in Jira: https://tickets.puppetlabs.com/browse/PUP-146 > > AFAIK none of that has been implemented though. > That's correct. Not lack of will, just lack of time :( However, in PUP-146 someone pointed out there's an old patch that could perhaps be revived and used as a starting point for getting this functionality into mainline puppet. If someone wants to take a pass at reviving/rebasing that patch, I'd love to collaborate on getting this into upstream. > And perhaps bigger refactorings of the RAL would be better to allow > concurrent processing as well as batching. > Absolutely. Even more work, but very desirable IMHO, would be to allow both parallel catalog application and batching (depending on how you look at it, batching could be seen as a special case of parallel application). True parallel catalog application is something I've always seen as a potential win from a native (C++) rewrite of the agent. Kylo > On Fri, 29 Jan 2016 at 03:18 Shawn Ferry <[email protected]> wrote: > >> >> I had looked at post_resource_eval but discarded it given the docs >> description for cleanup. If I define ‘cleanup’ as actually making the >> desired changes it would seem to work. >> >> I over simplified slow_command, it’s not particularly slow on query it’s >> just slow on change so if I’ve already determined that nothing needs to >> change I’m not really paying a penalty for doing it. >> >> Shawn >> >> On Jan 28, 2016, at 7:05 PM, Gary Larizza <[email protected]> wrote: >> >> I don't know if it would help, but did anything ever get decided from the >> post_xxx_eval propositions in this thread --> >> https://groups.google.com/forum/#!msg/puppet-dev/8j2IjZ1Ilog/KTsMaATo4DAJ >> >> That would give you a method that could be called at specific times, BUT, >> again, it's just another hook AFTER every resource was synchronized (and so >> wouldn't stop the slow command from being synchronized every time). >> >> On Thu, Jan 28, 2016 at 11:36 AM, Shawn Ferry <[email protected]> >> wrote: >> >>> Yeah, I don’t like how the composite resource removes the individual >>> resource-y nature of each thing. It would work but ‘all_the_true_things’, >>> ‘all_the_false_things’, and ‘all_the_removed_things’ is unpalatable. >>> >>> That is what I’m finding as well and prefetch is working, I was hoping >>> that there were higher level hooks I was missing that would let me carry >>> some execution across resources something like a ‘flush_provider’. >>> >>> I’m wondering if I can create resources dynamically for the required >>> changes execute, update, and fall back to individual resource modification; >>> ‘batched_slow_command’ unless that really sounds viable I’ll just shelve >>> this as something we need to live with. I’m afraid that while it could >>> probably be made to work it will be overly messy, confusing, and the >>> applicability is too specific to this one case. >>> >>> Thanks >>> Shawn >>> >>> >>> On Jan 28, 2016, at 1:31 PM, Luke Kanies <[email protected]> wrote: >>> >>> We’ve tried multiple times to build something to support this within the >>> RAL, but we’ve never been able to come up with something sufficiently >>> robust. >>> >>> We do have ‘prefetch’ for combining discovery commands, but nothing for >>> combining the execution of the commands, AFAIK. >>> >>> >>> On Jan 27, 2016, at 1:03 PM, Michael Smith <[email protected]> >>> wrote: >>> >>> >>> I'm not aware of anything that supports this explicitly. If making your >>> own type and provider, you could provide a parameter that takes an array of >>> instances to apply and use that (kind of like the file resource has an >>> overload parameter 'path'). You could then write >>> >>> slow_command { 'things_1_2_3_4': >>> things => ['thing1', 'thing2', 'thing3', 'thing4'], >>> value => true, >>> } >>> >>> but any dependencies would have to be expressed against the namevar >>> 'things_1_2_3_4'. >>> >>> ---- >>> >>> I've been part of discussions about how this would be nice for package >>> providers (`yum install a b c` is faster than calling `yum install a; yum >>> install b; yum install c`). However some extensions to the provider >>> interface would be needed to make it happen. >>> >>> Conceptually providers would need a way to state they support merging >>> individual declarations, some way of representing a pluralization of the >>> type, and processing in the graph to merge instances of the same >>> type/provider pair that don't have intermediate dependencies. >>> >>> Catalog ordering limits the use of this to your use case, where you >>> declare several instances of the same type consecutively. It would be more >>> interesting without catalog ordering, where we could merge instances of a >>> type declared in different modules as long as they don't have intermediate >>> dependencies. >>> >>> On Wed, Jan 27, 2016 at 10:50 AM, Shawn Ferry <[email protected]> >>> wrote: >>> >>>> I have a command that takes one or more effectively free form arguments >>>> and executes somewhat slowly and sometimes if things are changing much more >>>> slowly. Lets say 30s nominal execution in the simple case. >>>> >>>> I’m not finding a list of hooks to see if anything is appropriate. >>>> Flush would be great if I was processing a bunch of individual arguments >>>> for a resource but I need something that allows me to defer these updates >>>> until the end of processing for a provider and flush them together. >>>> >>>> Am I missing an alternate method to do something like this or the >>>> correct place in the docs? >>>> >>>> >>>> Thanks >>>> Shawn >>>> >>>> >>>> If I have something like the following it takes at least 2 minutes >>>> >>>> slow_command { [‘thing1’, 'thing2', ‘thing3’, ’thing4']: >>>> value => true >>>> } >>>> >>>> /tmp/slow_command thing1=true >>>> /tmp/slow_command thing2=true >>>> /tmp/slow_command thing3=true >>>> /tmp/slow_command thing4=true >>>> >>>> If I manually invoke or exec it it takes 30s >>>> >>>> /tmp/slow_command thing1=true thing2=true thing3=true thing4=true >>>> >>>> >>>> :::slow_command::: >>>> #!/bin/sh >>>> sleep 30 >>>> echo $* >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Puppet Developers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/puppet-dev/9D1DB354-8D30-448E-A461-54C67D4B8B26%40oracle.com >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Michael Smith >>> Developer, Puppet Labs >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Puppet Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/puppet-dev/CABy1mM%2Bkvusazxe6Wz%3DMR4NKbT%2B6Xndpw5f9U%3D0j%3D4g6nFML9Q%40mail.gmail.com >>> <https://groups.google.com/d/msgid/puppet-dev/CABy1mM%2Bkvusazxe6Wz%3DMR4NKbT%2B6Xndpw5f9U%3D0j%3D4g6nFML9Q%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> — >>> @puppetmasterd | http://about.me/lak | http://puppetlabs.com/ >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Puppet Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/puppet-dev/A76EB163-0090-4E6B-8BDF-BADB6DA6B6CC%40puppetlabs.com >>> <https://groups.google.com/d/msgid/puppet-dev/A76EB163-0090-4E6B-8BDF-BADB6DA6B6CC%40puppetlabs.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Puppet Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/puppet-dev/76A99F23-2F09-4AE1-A8A1-355193A2AAAA%40oracle.com >>> <https://groups.google.com/d/msgid/puppet-dev/76A99F23-2F09-4AE1-A8A1-355193A2AAAA%40oracle.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Gary Larizza >> Professional Services Engineer >> Puppet Labs >> >> http://garylarizza.com >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/CAMQzncLJZMJG-hLSLYRq9auhWffTY-eharhYwai6ALaxc6o_Qw%40mail.gmail.com >> <https://groups.google.com/d/msgid/puppet-dev/CAMQzncLJZMJG-hLSLYRq9auhWffTY-eharhYwai6ALaxc6o_Qw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/2B10EFB4-3B81-4AF8-A278-C9E661C1AF9F%40oracle.com >> <https://groups.google.com/d/msgid/puppet-dev/2B10EFB4-3B81-4AF8-A278-C9E661C1AF9F%40oracle.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/CAAAzDLfUF%2B6hJ%3DwuArMfoKCsixOox-0A0Ezf-mpUjcdHs%2BhVqQ%40mail.gmail.com > <https://groups.google.com/d/msgid/puppet-dev/CAAAzDLfUF%2B6hJ%3DwuArMfoKCsixOox-0A0Ezf-mpUjcdHs%2BhVqQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Kylo Ginsberg | [email protected] | irc: kylo | twitter: @kylog -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CALsUZFELQ03O_8r03RuQmLnhoojw6D3UN885hPUErJbp88qG3w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
