Cesar, For general catalog compilation time improvements, you might be best served by using Puppet Server ( https://docs.puppet.com/puppetserver/1.2/services_ master_puppetserver.html ) instead of passenger.
In 2014, my team & I performed our own sort of benchmark tests of the puppet master server. From https://indico.cern.ch/event/ 320819/contributions/742968/ download the PDF https://indico.cern.ch/event/320819/contributions/742968/ attachments/619020/851687/Configuration_Management_ Change_Management-James-Pryor.pdf and please see pages 32 - 37 that show the load on the single puppet master that was running Apache + passenger. Approximately six months later 2015, another test was done on the same hardware with the same software, but with the replacement of Apache & Passenger with Puppet Server 1.x. From https://indico.cern.ch/event/ 346931/contributions/817788/ download the PDF https://indico.cern.ch/event/ 346931/contributions/817788/attachments/684633/940417/ Configuration_Management_at_the_RACF.pdf and check pages 3 - 8 and see that compilation times were effectively cut in half. So in summary, in an apples-to-apples comparison on the same hardware, Puppet Server 1.x is close to twice as fast as Apache & Passenger. Now there are some differences between Puppet Server & Passenger which is documented here: https://docs.puppet.com/puppetserver/1.2/puppetserver_vs_passenger.html Now all that said, Puppet 3.x has been End of Life-ed as of 2017-01-01 ( https://groups.google.com/forum/#!topic/puppet-announce/tSe_4KTS7t0 ). You should consider working toward a migration path from Puppet 3.3 to the latest Puppet 3.8.x , and then look toward testing with Puppet 4.x, and either land on Puppet 4.x or maybe even wait for Puppet 5.x (which is hinted at coming in 2017 https://groups.google.com/d/ msg/puppet-dev/-H1pHJM6NLE/II3X8Cw6AwAJ). https://docs.puppet.com/puppet/4.10/index.html#upgrading-from-puppet-3 https://docs.puppet.com/puppet/4.10/upgrade_major_pre.html Regards, James On Thu, Apr 20, 2017 at 6:32 AM, Cesar <[email protected]> wrote: > Hi all, > > we have some performance issues with Puppet 3.3 catalog compilation (we > are using Passenger to host Puppet master processes). > It looks like a single Puppet master process is being used for the catalog > compilation, and importing the catalogs is taking ~3 minutes. > > Apr 19 10:21:14 ieatlms3841 puppet-master[6700]: Handling request: GET > /production/node/ieatrcxb3742 > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: Using cached facts for > ieatrcxb3742 > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: Caching node for > ieatrcxb3742 > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: node supports formats: > pson b64_zlib_yaml yaml raw > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: PROFILE [69862410013880] > 1.1 Rendered result in Puppet::Network::Format[pson]: took 0.0014 seconds > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: PROFILE [69862410013880] > 1.2 Sent response: took 0.0000 seconds > Apr 19 10:21:15 ieatlms3841 puppet-master[6700]: PROFILE [69862410013880] > 1 Processed request GET /production/node/ieatrcxb3742: took 0.0159 seconds > ... > > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.1 Setup server facts for compiling: took 0.0172 seconds > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: 'replace facts' command > for ieatrcxb3742 submitted to PuppetDB with UUID > 9a23c445-5203-4987-9faf-a187ac7dd0b3 > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.2 Found facts: took 0.1696 seconds > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: Using cached facts for > ieatrcxb3742 > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: Caching node for > ieatrcxb3742 > Apr 19 10:21:20 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.3 Found node information: took 0.0133 seconds > *Apr 19 10:21:20* ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatlms3841.pp' in environment production > Apr 19 10:21:27 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3739.pp' in environment production > Apr 19 10:21:42 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3740.pp' in environment production > Apr 19 10:21:51 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3742.pp' in environment production > Apr 19 10:21:56 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3743.pp' in environment production > Apr 19 10:22:05 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3744.pp' in environment production > Apr 19 10:22:11 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb3755.pp' in environment production > ... > > Apr 19 10:24:03 ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb5894.pp' in environment production > *Apr 19 10:24:06 *ieatlms3841 puppet-master[6700]: importing > '/etc/puppet/manifests/plugins/ieatrcxb6430.pp' in environment production > Apr 19 10:24:14 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4.1 Compile: Set node parameters: took 0.0025 seconds > Apr 19 10:24:14 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4.2 Compile: Created settings scope: took 0.0295 seconds > Apr 19 10:24:14 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4.3 Compile: Evaluated main: took 0.0083 seconds > > ... > > Apr 19 10:24:29 ieatlms3841 puppet-master[6700]: (Firewall[1018 vmmonitord > ipv6]) Adding default for require > Apr 19 10:24:29 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4.7 Compile: Finished catalog: took 0.1885 seconds > Apr 19 10:24:31 ieatlms3841 puppet-master[6700]: PROFILE [69862429450080] > 1.4 Compiled catalog for ieatrcxb3742 in environment production: took > *191.6076* seconds > Apr 19 10:24:31 ieatlms3841 puppet-master[6700]: Compiled catalog for > ieatrcxb3742 in environment production in 191.61 seconds > Apr 19 10:24:31 ieatlms3841 puppet-master[6700]: Caching catalog for > ieatrcxb3742 > > > I was wondering whether there is any way to reduce catalog compilation? > Could the catalog compilation be split across different Puppet master > processes or something like that? > > > Thanks, > Cesar. > > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. > If you have received this email in error please notify the system manager. > This message contains confidential information and is intended only for the > individual named. If you are not the named addressee you should not > disseminate, distribute or copy this e-mail. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/puppet-users/c04da5d1-bb08-46a1-ab95-ffe3e52069a2%40googlegroups.com > <https://groups.google.com/d/msgid/puppet-users/c04da5d1-bb08-46a1-ab95-ffe3e52069a2%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAOsPUcit6_%3DvQPunUb%2BZpdqAG-yjEGaoVHya1HZVQgHvWff51w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
