Andy, did this on a vagrant box with eyaml and file backends. The hierarchy has 7 levels.
Debug: PROFILE [apply] 1 Setup server facts for compiling: took 0.0000 seconds Debug: PROFILE [apply] 2.1 Compile: Set node parameters: took 0.0000 seconds Debug: PROFILE [apply] 2.2 Compile: Created settings scope: took 0.0100 seconds Debug: hiera(): [eyaml_backend]: Hiera eYAML backend starting Debug: hiera(): [eyaml_backend]: Set option: datadir = /vagrant/hiera/ Debug: hiera(): [eyaml_backend]: Set option: pkcs7_private_key = /vagrant/keys/private_key.pkcs7.pem Debug: hiera(): [eyaml_backend]: Set option: pkcs7_public_key = /vagrant/keys/public_key.pkcs7.pem Debug: hiera(): [eyaml_backend]: Looking up somekey in eYAML backend Debug: hiera(): [eyaml_backend]: Looking for data source it-mil1/dev/nodes/dnsns001 Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/nodes/dnsns001.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source it-mil1/dev/roles/dnsns Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/roles/dnsns.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source it-mil1/dev/global Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/global.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source nodes/dnsns001 Debug: hiera(): [eyaml_backend]: Looking for data source roles/dnsns Debug: hiera(): Cannot find datafile /vagrant/hiera/roles/dnsns.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source dev Debug: hiera(): Cannot find datafile /vagrant/hiera/dev.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source global Debug: hiera(): Hiera File backend starting Debug: hiera(): Looking up somekey in File backend Debug: hiera(): Hiera File_backend: looking for data source 'it-mil1/dev/nodes/dnsns001' Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/nodes/dnsns001.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'it-mil1/dev/roles/dnsns' Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/roles/dnsns.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'it-mil1/dev/global' Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/global.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'nodes/dnsns001' Debug: hiera(): Cannot find datafile /vagrant/hiera/nodes/dnsns001.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'roles/dnsns' Debug: hiera(): Cannot find datafile /vagrant/hiera/roles/dnsns.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'dev' Debug: hiera(): Cannot find datafile /vagrant/hiera/dev.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'global' Debug: PROFILE [apply] 2.3.1 Called hiera: took 0.0336 seconds Debug: hiera(): [eyaml_backend]: Set option: datadir = /vagrant/hiera/ Debug: hiera(): [eyaml_backend]: Set option: pkcs7_private_key = /vagrant/keys/private_key.pkcs7.pem Debug: hiera(): [eyaml_backend]: Set option: pkcs7_public_key = /vagrant/keys/public_key.pkcs7.pem Debug: hiera(): [eyaml_backend]: Looking up otherkey in eYAML backend Debug: hiera(): [eyaml_backend]: Looking for data source it-mil1/dev/nodes/dnsns001 Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/nodes/dnsns001.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source it-mil1/dev/roles/dnsns Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/roles/dnsns.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source it-mil1/dev/global Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/global.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source nodes/dnsns001 Debug: hiera(): [eyaml_backend]: Looking for data source roles/dnsns Debug: hiera(): Cannot find datafile /vagrant/hiera/roles/dnsns.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source dev Debug: hiera(): Cannot find datafile /vagrant/hiera/dev.eyaml, skipping Debug: hiera(): [eyaml_backend]: Looking for data source global Debug: hiera(): Looking up otherkey in File backend Debug: hiera(): Hiera File_backend: looking for data source 'it-mil1/dev/nodes/dnsns001' Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/nodes/dnsns001.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'it-mil1/dev/roles/dnsns' Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/roles/dnsns.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'it-mil1/dev/global' Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/global.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'nodes/dnsns001' Debug: hiera(): Cannot find datafile /vagrant/hiera/nodes/dnsns001.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'roles/dnsns' Debug: hiera(): Cannot find datafile /vagrant/hiera/roles/dnsns.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'dev' Debug: hiera(): Cannot find datafile /vagrant/hiera/dev.d, skipping Debug: hiera(): Hiera File_backend: looking for data source 'global' Debug: PROFILE [apply] 2.3.2 Called hiera: took 0.0156 seconds On Friday, May 16, 2014 6:29:28 PM UTC+2, Andy Parker wrote: > > On Thu, May 15, 2014 at 10:24 PM, David Schmitt <[email protected]<javascript:> > > wrote: > >> Hi, >> >> >> On 16.05.2014 02:17, Alessandro Franceschi wrote: >> >>> >>> >>> On Thursday, May 15, 2014 9:55:05 PM UTC+2, John Bollinger wrote: >>> >>> I am concerned about the impact. It is already somewhat costly for >>> Puppet to evaluate data bindings for class parameters, and adding >>> bindings for resource parameters (even just for resources of defined >>> types) will magnify that. Note that the cost scales with the >>> aggregate number of defined parameters for all declared resources, >>> independent whether any data are actually bound. In fact, the cases >>> were no data are bound are the most costly, because hiera must then >>> search the entire hierarchy. >>> >>> >>> Yes, these are valid and convincing points. >>> Anyway if we find data binding useful for classes and can bear the >>> performance overhead, I suppose we can do the same for defined types. >>> >> >> I very much like the idea for data binding on all parameters, but this >> statement is really not universally true. I've got catalogs with only tens >> of classes but thousands of resources. That works out to 100x more hiera >> calls. :-/ >> >> > John makes a very good point about the performance cost. The impact of > putting data bindings on everything is made even worse by the fact that it > is an all or nothing price you have to pay, there isn't any way to use it > for just a subset of your catalog. > > As a little number collection, would some of you mind doing a little > experiment and posting the numbers? > > Run "puppet apply -e 'notice(hiera("somekey", 1), hiera("otherkey", 1))' > --profile --debug" > > Make sure that it is using your real hiera config with your real data > files or else it won't be an accurate reflection of the cost. You'll get > output that should contain something like this: > > Debug: PROFILE [apply] 2.2 Compile: Created settings scope: took 0.0103 > seconds > Debug: hiera(): Hiera YAML backend starting > Debug: hiera(): Looking up somekey in YAML backend > Debug: hiera(): Looking for data source common > Debug: hiera(): Cannot find datafile /var/lib/hiera/common.yaml, skipping > Debug: PROFILE [apply] 2.3.1 Called hiera: took 0.0019 seconds > Debug: hiera(): Looking up otherkey in YAML backend > Debug: hiera(): Looking for data source common > Debug: hiera(): Cannot find datafile /var/lib/hiera/common.yaml, skipping > Debug: PROFILE [apply] 2.3.2 Called hiera: took 0.0002 seconds > > I don't have hiera setup on this machine so it didn't do anything, but you > can see how much overhead *per-parameter* data bindings introduces. So in a > system with no hierarchy, and no data files (and in fact no hiera config) > it adds about 2 usec per parameter. > > If some of you could post results as well as estimates of the number of > defined types and parameters that you have in a catalog that would be great. > > >> >> Regards, David >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit https://groups.google.com/d/ >> msgid/puppet-dev/5375A109.7020209%40dasz.at. >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Andrew Parker > [email protected] <javascript:> > Freenode: zaphod42 > Twitter: @aparker42 > Software Developer > > *Join us at PuppetConf 2014 <http://www.puppetconf.com/>, September > 22-24 in San Francisco* > *Register by May 30th to take advantage of the Early Adopter discount > <http://links.puppetlabs.com/puppetconf-early-adopter> **—**save $349!* > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/63ce435d-0f8a-447d-ac3c-9999c3b2874a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
