sure. the processes we run to do linkage take hours. we're processing ~600k records, bouncing our users data up against a few data sources that act as 'sources of truth' for us for the sake of this linkage. we get the top 3 results and run some quick checks on it algorithmically to determine if we have a match. we use parallel requests of 100 at a time.
solr isn't built for this sort of purpose specifically, i'm pretty sure, but even so i'm imagining/hoping there is a way to give it a bit more processing power. thanks for any continued discussion! -- *John Blythe* Product Manager & Lead Developer 251.605.3071 | j...@curvolabs.com www.curvolabs.com 58 Adams Ave Evansville, IN 47713 On Thu, Jun 2, 2016 at 12:49 PM, Erick Erickson <erickerick...@gmail.com> wrote: > Without having a lot more data it's hard to say anything helpful. > > _What_ is slow? What does "data linkage" mean exactly? Etc. > > Best, > Erick > > On Thu, Jun 2, 2016 at 9:33 AM, John Blythe <j...@curvolabs.com> wrote: > > hi all, > > > > having lots of processing happening using multiple solr cores to do some > > data linkage with our customers' transactional data. it runs pretty > slowly > > at the moment. we were wondering if there were some solr or jetty tunings > > that we could implement to help make it more powerful and efficient. it > > currently is using less than 2GB on our box, can we open it up to use > more > > memory and get speedier as a result? > > > > thanks for any tips! >
