First thank you for your response. With that said the on the fly calculations
that Solr would be tasked to preform would be as follows. Perform a store
location Overlap Analysis of McDonald's (14,000 locations), Subway(20,000
locations), and Duncan Donuts(4,000 locations) utilizing their Geocodes
returning all locations that are within 5 miles radius.
I will absolutely be using the Solr index down the road with my pre-calculated
value data to get the best of both worlds. However the current frontend will
need to be re-programed to take advantage of this. This will be a phase 2
initiative.
The main reason for the question was we are optimizing the nightly
per-calculation job to only do the calculations on the deltas which will reduce
the nightly jobs processing time by 85%. Which will make it less likely to
fail and when it does fail quicker to re-process.
If the process pre-calculates without issue the speed at which the SQL query
returns data is super-fast and not the issue at all. Thx again and I look
forward to any addition replies and/or feedback.
Regards,
Joe
Joseph Costello
Chief Information Officer
F&D Reports | Creditntell | ARMS
===========================
Information Clearinghouse Inc. & Market Service Inc.
310 East Shore Road, Great Neck, NY 11023
email: jose...@fdreports.com | Tel: 800.789.0123 ext 112 | Cell: 516.263.6555 |
www.informationclearinghouseinc.com
Need Help? Click here to request IT Support.
-----Original Message-----
From: Shawn Heisey [mailto:apa...@elyograg.org]
Sent: Wednesday, October 17, 2018 11:30 AM
To: solr-user@lucene.apache.org
Subject: Re: Solr performing Calculations vs. Pulling data Values Directly From
DB Question
On 10/17/2018 9:19 AM, Joseph Costello - F&D Reports wrote:
>
> Any feedback from the group on the question below.
>
> The question was will solr performing distance calculations (10,000++)
> on the fly, perform faster than SQL query simply pulling
> pre-calculated distance values directly from the database.
>
If your database contains pre-calculated values, then pulling those is likely
to be faster than doing calculations on the fly. Whether that's true in
practice depends on precisely what kinds of calculation you are doing, and what
must be done to obtain the values that go into the calculation.
If your database has pre-calculated values, and you need those in search
results, why not just put the pre-calculated values into your Solr index when
you build it? One of the key things done with a search engine for performance
is handling as much as possible at index time, so there's less work to do at
query time.
Thanks,
Shawn
