The ‘export logs’ command deals with this by having —start-time and —end-time parameters that can be used to specify a particular time window. I’m not sure whether statistics files roll over and have timestamps or not. This could be one approach to limiting the size of the data returned for ‘export artifacts'. We could also check the size of the artifacts before sending them back, and prompt for confirmation from the user if it was going to exceed some given threshold.
> On Jan 26, 2017, at 2:02 PM, Udo Kohlmeyer <ukohlme...@pivotal.io> wrote: > > I agree with dan... We need to be able to limit how much data is being > collected and sent. > > In many cases clients don't clean up the files so they might drag a lot of > stuff over. > > > On 1/26/17 13:52, Dan Smith wrote: >> +1 for export artifacts and +1 for zip-file >> >> I'm also generally in favor of gathering the artifacts on the gfsh client >> but I am a little concerned that in some cases that could be a lot of data >> going to the client (10s or 100s of GBs), especially if logs and stats >> aren't being cleaned up on a regular basis. Maybe we need a cap on the >> amount of data sent back? >> >> -Dan >> >> On Thu, Jan 26, 2017 at 1:35 PM, John Blum <jb...@pivotal.io> wrote: >> >>> +1 >>> >>> On Thu, Jan 26, 2017 at 1:05 PM, Jared Stewart <jstew...@pivotal.io> >>> wrote: >>> >>>> I would like to propose a new ‘export artifacts’ GFSH command, as well as >>>> some changes to existing GFSH export commands. (See >>>> https://geode.apache.org/docs/guide/tools_modules/gfsh/ >>>> command-pages/export.html <https://geode.apache.org/ >>>> docs/guide/tools_modules/gfsh/command-pages/export.html> for a list of >>>> the current commands.) >>>> >>>> There are some inconsistencies in the destinations of the current GFSH >>>> export commands. Some commands (e.g. 'export logs’) export to a given >>>> directory on the executing member (server or locator). Other commands >>>> (e.g. ‘export cluster-configuration’) export to a given directory on the >>>> GFSH client machine. (This makes far more sense, and is what I would have >>>> expected as a user.) I propose to reconcile all export commands to >>> target >>>> an export directory on the GFSH client machine, rather than on the >>>> executing cluster member. >>>> Export cluster-configuration (which exports the existing cluster >>>> configuration of a cluster to a zip file) has parameters for both >>>> —zip-file-name and —dir. I think it makes more sense to have one >>>> parameter, say —zip-file, that can take either a relative path to a zip >>>> file (“—zip-file=myExport.zip”, “—zip-file=logs/myExport.zip") or an >>>> absolute path including directories (“—zip-file=/logs/myExport.zip”) >>>> rather than having two separate parameters. >>>> I propose to add an ‘export artifacts’ GFSH command that would export all >>>> log files and stat files from all the members of a cluster to a single >>> zip >>>> file on the GFSH client machine. This would provide a convenient >>> mechanism >>>> for an administrator to collect together all of the files necessary to >>>> troubleshoot problems in their Geode cluster. >>>> >>>> - Jared >>> >>> >>> >>> -- >>> -John >>> john.blum10101 (skype) >>> >
