I strongly suggest you consult with a local statistician. Your description is far too vague (to me anyway) to make any sense of and probably requires a good deal of back and forth between you and a competent data analyst to pin down what the issues and constraints are. For example, what constitutes "interesting" patterns? -- do the data need to be analyzed in real time? -- Are visual displays sufficient or is some kind of numeric indication needed? ... etc. etc.
One tentative word of advice, though: with this much data, do not get embroiled with P values or other measures of statistical "significance": anything you can see will be "significant." -- Bert On Mon, Apr 2, 2012 at 8:42 PM, Hasan Diwan <hasan.di...@gmail.com> wrote: > I have approximately 2.5 million rows from a number of sensor > readings. Having plotted these, I can see a given pattern (say a spike > in the amplitude away from the mean). I would now like to automate > this procedure as we're expecting a great deal more data in the near > future. Is there any package or function that will make this possible? > Many thanks! I suppose, I could do something like: > amplitude <- abs(sensorReadings[sensorReadings[,2] > 1.2 * > median(sensorReadings[,2]),]) > Is this the most efficient way to do what I want or not? > -- H > > -- > Sent from my mobile device > Envoyait de mon portable > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. -- Bert Gunter Genentech Nonclinical Biostatistics Internal Contact Info: Phone: 467-7374 Website: http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
