Re: Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Anilkumar Gingade
>> One other idea that hasn't been mentioned is making parallel the only way My vote is to support both option; we could make parallel default but having an option to take snapshot at one node may be useful for use-cases where: - Easier to manage snapshot at one file location; in a large cluster e

Re: Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Nick Reich
With minimal code change, it is possible to enable the use of —dir for both standard and parallel export/import, allowing —file to function only for standard exports (and optionally, make it depricated in favor of the —dir option). While not inherently opposed to forcing all Partitioned Region sna

Re: Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Michael Stolz
One other idea that hasn't been mentioned is making parallel the only way for Partitioned Regions, and having --file serve the purpose of defining both a path and a filename pattern where the bucket ID or whatever we're using gets automatically inserted before the .gfd extension. No need for a new

Re: Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Nick Reich
Parallel export will write the data to files on the bucket primary for each bucket, distributing the work (and therefore files) to all the members. That would be a big enough deviation from the current behavior (single file on single machine), that I think it makes it worth having the additional op

Re: Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Dan Smith
I don't really like the idea of adding a separate command. It really is the same command - you just want to have the parallel flag interact with other options. A separate command would be more confusing for users, and more of a maintenance issue as we add more options to export. Having a --path th

Re: Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Jacob Barrett
On Tue, Aug 22, 2017 at 1:49 PM Nick Reich wrote: > The idea of deprecating —file in favor of path is interesting. I wonder if > instead of making them mutually exclusive to start, having —path be able to > support both modes from the start would be better? That way —file could > still be used fo

Re: Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Nick Reich
I thought about a mutually exclusive —file and —dir, but in that case, -—file is required for standard and —path required for parallel export, which I think could be better than overloading —file, but still has potential for confusion. The idea of deprecating —file in favor of path is interesting.

Re: Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Jacob Barrett
How about deprecate —file and replace with —path? In the transition make them mutually exclusive and —path required for —parallel. Any reason to not just make all export parallel rather than supporting two different modes? -Jake Sent from my iPhone > On Aug 22, 2017, at 12:27 PM, Kenneth Ho

Re: Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Kenneth Howe
I agrees that overloading the “file” option seems like a bad idea. As an alternative to separate commands, what about mutually exclusive options, ‘—file’ and ‘—dir’? If you go for implementing the new functionality as a separate command, I would suggest calling the gfsh commands: “export data-p

Adding parallel import/export of snapshots to gfsh

2017-08-22 Thread Nick Reich
Team, I am working on exposing the parallel export/import of snapshots through gfsh and would appreciate input on the best approach to adding to / updating the existing interface. Currently, ExportDataCommand and ImportDataCommand take a region name, a member to run the command on, and a file loc