Ok, so I finally opted for inserting a full node from one DB into another, since getting the resources should be quite easy with a GET.
But then I run into a problem when running the replace catalog command (with an empty catalog): curl -vv -G -H "Accept: application/json" 'http://localhost:8080/v2/commands' --data-urlencode 'payload={"command":"replace catalog","version": 2,"payload":"{ \"metadata\": {\"api_version\": 1}, \"data\": {\"name\": \"puppet09.cscs.ch\",\"version\": \"1111\", \"edges\": [], \"resources\": [] } }"}' Which returns ok. The puppetdb.log shows: 2013-10-15 10:42:02,060 ERROR [command-proc-45] [puppetdb.command] [43c41752-e7d4-4725-b062-7b3a9b1ac8ec] [replace catalog] Retrying after attempt 8, due to: org.postgresql.util.PSQLException: ERROR: syntax error at or near ")" Position: 66 I have looked again and again into the data, and syntax seems to be correct. Anybody tried this? Hints? Thanks a lot, Pablo Fernandez On 10/10/2013 05:23 PM, Throwe, Jesse wrote: > That would really depend on the approach ultimately taken for the > multi-master. If the puppet master (or agent in masterless mode) > forwards the same fact/report/node/etc data to multiple puppetdb > instances then yes the scale breaks horribly and does not meet part of > this request. If there is some kind of store-and-forward mechanism in > puppetdb itself, it would be possible to extend the forward to say > 'only if it matches these rules'. This could definitely be useful in > a variety of ways beyond simple HA purposes. > > > > On Thu, Oct 10, 2013 at 11:07 AM, Pablo Fernandez > <[email protected] <mailto:[email protected]>> wrote: > > Yes, I saw that... this may be a solution for replicating full > PuppetDBs (if you tackle the backlog during a long shortage), but > that does not scale very well if you have 5-10 PuppetDBs all being > replicated to a single, bigger one. > > Besides, it would not cover cases when you want to have only some > nodes replicated, or even only some resources. > > > > On 10/10/2013 04:35 PM, Throwe, Jesse wrote: >> It sounds like a shade of #19321 - >> https://projects.puppetlabs.com/issues/19321 >> >> >> On Thu, Oct 10, 2013 at 10:31 AM, Wolf Noble <[email protected] >> <mailto:[email protected]>> wrote: >> >> I'd be interested in accomplishing a similar thing. I want to >> have a central reporting puppetdb which I can (outside of >> puppet) query for multisite information without tying >> isolated environments together via a single postgres db, or >> using multi-master replication. >> >> my needs do not involve making each puppet site aware of each >> other's data, but it would be nice to have one db which has >> all the data for my digging pleasure. >> >> from my understanding this is not currently supported. >> >> I was thinking something could be done to ask puppetdb at >> site A (what updates have you had in the past N minutes/ >> since the last time I checked in with you/ since >> timestamp(N)), and then iterate through the collected data >> and feed it into a reporting puppetdb instance, however I've >> not gone beyond thinking about it. >> >> >> >> >> >> On Thu, Oct 10, 2013 at 7:12 AM, Pablo Fernandez >> <[email protected] <mailto:[email protected]>> wrote: >> >> Thanks Ken, >> >> I took a look at the Command API and seems to be quite >> interesting, but >> it triggers a few more questions. >> >> As a reminder (also for others), what I wanted to do was >> to add exported >> resources from Node_A (managed by PuppetMaster_A that has >> its own >> PuppetDB_A) into a different PuppetDB_B, where Node_A is >> unknown, in >> order for other Nodes_B (managed by PuppetMaster_B) be >> aware and >> possibly realize those exported resources. >> >> Taking a look the Commands API seem to capabilities to >> change a whole >> catalog, or facts. Do they need to be existing catalogs, >> or could I add >> new ones? >> I am thinking: If I want to add Node_A's exported >> resource(s) to >> PuppetDB_B, I could "artificially" add its whole catalog to >> PuppetDB_B... but the Command API is "replace catalog", >> no "add >> catalog". Would it be rejected? How does the first >> catalog of a node get >> introduced into the DB? And moreover, how do I query a >> whole catalog? >> >> Another possibility would be to include the resource >> inside an existing >> Node_B catalog on the PuppetDB_B (after all, exported >> resources don't >> really mater what node do they belong, as long as they >> have the right >> attributes, right?), but the Catalog wire format seems to >> require the >> whole set of resources, so, if I define the resources of >> Node_B with >> some exported resources from Node_A, I believe I would >> lose the original >> Node_B's resources. The only possibility would be to >> query Node_B full >> catalog, and *add also* those resources from Node_A >> before sending them >> to the DB... and this seems to me like a bit dirty :) >> >> Am I missing anything? Or is there another way to >> implemen what I need >> with Puppet 3.2? >> >> Thanks again, >> BR/Pablo >> >> >> On 10/10/2013 09:16 AM, Ken Barber wrote: >> >>> Is there a mechanism to pull and push some exported >> resources from one >> >>> PuppetDB to another? I guess this could be done with >> the API, but has >> >>> anybody tried it, or will it work at all? >> > Not provided by us, yet. We've had a lot of discussions >> internally >> > about this kind of thing. How about you outline your >> requirements in >> > the form of a feature request here: >> > >> > http://projects.puppetlabs.com/projects/puppetdb/issues >> > >> >> Did anybody try to push data to a PuppetDB by hand? >> The API does not >> >> seem to have the PUT or POST methods. >> > The client obviously pushes uses POST :-). The >> documentation for the >> > mechanism is here: >> > >> > http://docs.puppetlabs.com/puppetdb/1.5/api/commands.html >> > >> > ken. >> > >> >> -- >> You received this message because you are subscribed to >> the Google Groups "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails >> from it, send an email to >> [email protected] >> <mailto:puppet-users%[email protected]>. >> To post to this group, send email to >> [email protected] >> <mailto:[email protected]>. >> Visit this group at >> http://groups.google.com/group/puppet-users. >> For more options, visit >> https://groups.google.com/groups/opt_out. >> >> >> -- >> You received this message because you are subscribed to the >> Google Groups "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from >> it, send an email to >> [email protected] >> <mailto:puppet-users%[email protected]>. >> To post to this group, send email to >> [email protected] >> <mailto:[email protected]>. >> Visit this group at http://groups.google.com/group/puppet-users. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> -- >> You received this message because you are subscribed to the >> Google Groups "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected] >> <mailto:[email protected]>. >> To post to this group, send email to >> [email protected] <mailto:[email protected]>. >> Visit this group at http://groups.google.com/group/puppet-users. >> For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google > Groups "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, > send an email to [email protected] > <mailto:puppet-users%[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at http://groups.google.com/group/puppet-users. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- > You received this message because you are subscribed to the Google > Groups "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/puppet-users. > For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
