Re: using solr for time series data

2012-01-19 Thread Ted Dunning
Take a look at openTSDB. You might want to use that as is, or steal some of the concepts. The major idea to snitch is the idea of using a single row of hte data base (document in Lucene or Solr) to hold many data points. Thus, you could consider having documents with the following fields: key:

using solr for time series data

2012-01-19 Thread Robert Stewart
I have a project where the client wants to store time series data (maybe in SOLR if it can work). We want to store daily "prices" over last 20 years (about 6000 values with associate dates), for up to 500,000 entities. This data currently exists in a SQL database. Access to SQL is too slow for c

Re: SolR for time-series data

2011-12-05 Thread Michael Kuhlmann
Hi Alan, Solr can do this fast and easy, but I wonder if a simple key-value-store won't fit better for your suits. Do you really only need to query be chart_id, or do you also need to query by time range? In either case, as long as your data fits into an in-memory database, I would suggest

Re: SolR for time-series data

2011-12-04 Thread Ted Dunning
Sax is attractive, but I have found it lacking in practice. My primary issue is that in order to get sufficient recall for practical matching problems, I had to do enough query expansion that the speed advantage of inverted indexes went away. The OP was asking for blob storage, however, and I thi

Re: SolR for time-series data

2011-12-04 Thread Grant Ingersoll
Definitely should be possible. As an aside, I've also thought one could do more time series stuff. Have a look at the iSax stuff by Shieh and Koegh: http://www.cs.ucr.edu/~eamonn/iSAX/iSAX.html On Dec 3, 2011, at 12:10 PM, Alan Miller wrote: > Hi, > > I have a webapp that plots a bunch of t

Re: SolR for time-series data

2011-12-03 Thread Óscar Marín Miró
Hi Alan, at my job we had a really succesful implementation similar to what you are proposing. With a classic RDBM, we hit serious performance issues so, we moved to solr to display time series of data. The 'trick' was to facet on a date field, to get 'counts' of data for a time series on a specifi

SolR for time-series data

2011-12-03 Thread Alan Miller
Hi, I have a webapp that plots a bunch of time series data which is just a series of doubles coupled with a timestamp. Every chart in my webapp has a chart_id in my db and i am wondering if it would be effective to usr solr to serve the data to my app instead of keeping the data in my rdbms. Cur