Re: Edismax query using different strings for different fields

2020-06-07 Thread David Zimmermann
then recombining the results. But that way I don’t know if the resulting scores are comparable? Can I assume a score of 15 from the English edismax is better than a score of 13 from the German edismax? Best regards David On 5 Jun 2020, at 19:39, Erick Erickson mailto:erickerick...@gmail.com

Re: Script to check if solr is running

2020-06-08 Thread David Hastings
> > Why have a cold backup and then switch? > my current set up is: 1. master indexer 2. master slave on a release/commit basis 3. 3 live slave searching nodes in two data different centers the three live nodes are in front of nginx load balancing and they are mostly hot but not all of them, i f

Re: Getting rid of zookeeper

2020-06-09 Thread David Hastings
Zookeeper is annoying to both set up and manage, but then again the same thing can be said about solr cloud. not certain why you would want to deal with either On Tue, Jun 9, 2020 at 3:29 PM S G wrote: > Hello, > > I recently stumbled across KIP-500: Replace ZooKeeper with a Self-Managed > Meta

using solr to extarct keywords from a long text?

2020-06-10 Thread David Zimmermann
up as standalone and not in cloud mode. Best David

Re: using solr to extarct keywords from a long text?

2020-06-11 Thread David Zimmermann
Hi Mikhail Your suggested solution does seem to work for me. Thank you so much for the help! Best regards David For future reference in case someone else wants do the same, here are some more details about the steps needed: - The more like this handler is not in the default solrconfig.xml

Question about Atomic Update

2020-06-15 Thread david . davila
tested with Solr 7.4 and Solr 4.10 Thanks, David

Re: Question about Atomic Update

2020-06-15 Thread david . davila
Hi Erick, Thank you for your answer. Unfortunatelly our most important field is that text field, so, we need to index it. We will have to assume that big documents takes a long time to index. Best, David David Dávila Atienza AEAT - Departamento de Informática Tributaria Subdirección de

Re: How to determine why solr stops running?

2020-06-16 Thread David Hastings
To add to this, i generally have solr start with this: -Xms31000m-Xmx31000m and the only other thing that runs on them are maria db gallera cluster nodes that are not in use (aside from replication) the 31gb is not an accident either, you dont want 32gb. On Tue, Jun 16, 2020 at 11:26 AM Shawn H

Re: How to determine why solr stops running?

2020-06-16 Thread David Hastings
the sum of the heap allocations across all your JVMs should be below > that percentage. See Uwe Schindler's mmapdirectiry blog... > > Shot in the dark... > > On Tue, Jun 16, 2020, 11:51 David Hastings > wrote: > > > To add to this, i generally have solr start with

Re: Solr 7.6 optimize index size increase

2020-06-16 Thread David Hastings
I cant give you a 100% true answer but ive experienced this, and what "seemed" to happen to me was that the optimize would start, and that will drive the size up by 3 fold, and if you out of disk space in the process the optimize will quit since, it cant optimize, and leave the live index pieces in

Re: Facet Performance

2020-06-17 Thread David Smiley
I strongly recommend setting indexed=true on a field you facet on for the purposes of efficient refinement (fq=field:value). But it strictly isn't required, as you have discovered. ~ David On Wed, Jun 17, 2020 at 9:02 AM Michael Gibney wrote: > facet.method=enum works by executing

Re: Master Slave Terminology

2020-06-17 Thread David Smiley
priv...@lucene.apache.org but it should have been public and expect it to spill out to the dev list today. ~ David On Wed, Jun 17, 2020 at 11:14 AM Mike Drob wrote: > Hi Jan, > > Can you link to the discussion? I searched the dev list and didn’t see > anything, is it on slack

Re: Getting rid of Master/Slave nomenclature in Solr

2020-06-19 Thread David Cumings
ntinue this conversation here while making sure that we converge > without much bike-shedding. > > -Anshum > -- David Cumings AU: +61 498 137 841 US: +1 (929) 291-0801 UK: +44 7725 057 500 <-- Currently in the UK IN: +91 82771 96058 d...@cumings.com

Re: How to determine why solr stops running?

2020-06-29 Thread David Hastings
> > > On Tue, Jun 16, 2020 at 1:00 PM David Hastings < > hastings.recurs...@gmail.com> > wrote: > > > me personally, around 290gb. as much as we could shove into them > > > > On Tue, Jun 16, 2020 at 12:44 PM Erick Erickson > > > wrote: > &g

Re: How to determine why solr stops running?

2020-06-29 Thread David Hastings
script, you _should_ have had very clear > > evidence that that was the cause. > > > > If you were not running the killer script, the apologies for not asking > > about that > > in the first place. Java’s performance is unpredictable when OOMs happen, > > which is th

Re: unified highlighter performance in solr 8.5.1

2020-07-03 Thread David Smiley
ing with '0') and additional performance benefit from that. What do you think Nandor, Michal? I'm hoping a change in settings (+ some better notes/docs on this) could slip into an 8.6, all done by myself ASAP. ~ David On Fri, Jun 19, 2020 at 2:32 PM Nándor Mátravölgyi wrote: > Hi

Re: Out of memory errors with Spatial indexing

2020-07-03 Thread David Smiley
class="solr.RptWithGeometrySpatialField" which internally is based off a combination of a course grid and storing the original vector geometry for accurate verification: The internally coarser grid will lessen the impact of that pole bug. ~ David Smiley Apache Lucene/Solr Search

Re: unified highlighter performance in solr 8.5.1

2020-07-03 Thread David Smiley
;true"? We agree on better documenting the perf trade-off. Thanks again for working on these settings, BTW. ~ David On Fri, Jul 3, 2020 at 1:25 PM Nándor Mátravölgyi wrote: > Since the issue seems to be affecting the highlighter differently > based on which mode it is using,

Re: unified highlighter performance in solr 8.5.1

2020-07-04 Thread David Smiley
Here's my PR, which includes some edits to the ref guide docs where I tried to clarify these settings a little too. https://github.com/apache/lucene-solr/pull/1651 ~ David On Sat, Jul 4, 2020 at 8:44 AM Nándor Mátravölgyi wrote: > I guess that's fair. Let's have hl.fragsiz

Re: Out of memory errors with Spatial indexing

2020-07-06 Thread David Smiley
I believe you are experiencing this bug: LUCENE-5056 <https://issues.apache.org/jira/browse/LUCENE-5056> The fix would probably be adjusting code in here org.apache.lucene.spatial.query.SpatialArgs#calcDistanceFromErrPct ~ David Smiley Apache Lucene/Solr Search Developer http://www.linked

sorting help

2020-07-15 Thread David Hastings
howdy, i have a field that sorts fine all other content, and i cant seem to debug why it wont sort for me on this one chunk of it. "sort":"alphatitle asc", "debugQuery":"on", "_":"1594733127740"}}, "response ":{"numFound":3,"start":0,"docs":[ { "title":"Money orders", { "title":"Finance, consolidat

Re: sorting help

2020-07-15 Thread David Hastings
ercaseFilter in front of your patternreplace, > you’re removing uppercase characters. > > Best, > Erick > > > On Jul 15, 2020, at 3:06 PM, David Hastings < > hastings.recurs...@gmail.com> wrote: > > > > howdy, > > i have a field that sorts fine all ot

bin/solr auth enable

2020-07-24 Thread David Glick
When I issue “bin/solr auth enable -prompt true -blockUnknown true”, I get a Null Pointer Exception. I’m using the 8.5.1 release. Am I doing something wrong? Thanks. Sent from my iPhone

Re: Meow attacks

2020-07-28 Thread David Hastings
so, your zookeeper/solr servers have public facing addresses/ports? On Tue, Jul 28, 2020 at 4:41 PM Odysci wrote: > Folks, > > I suspect one of our Zookeeper installations on AWS was subject to a Meow > attack ( > > https://arstechnica.com/information-technology/2020/07/more-than-1000-database

Re: solr query returns items with spaces removed

2020-07-29 Thread David Hastings
"Oh, and returning 100K docs is an anti-pattern, if you really need that many docs consider cursorMark and/or Streaming." er, i routinely ask for 2+ million records into a single file based on a query. I mean not into a web application or anything, its meant to be processed after the fact, but so

Re: bin/solr auth enable

2020-07-31 Thread David Glick
/solr auth enable -prompt true Both fail with the NPE. Thanks, David. Sent from my iPhone > On Jul 31, 2020, at 7:03 PM, Jason Gerlowski wrote: > > Hi David, > > I tried this out locally but couldn't reproduce. The command you > provided above works just fine for me.

Re: org.apache.lucene.util.fst.FST taking up lot of Java Heap Memory

2020-08-05 Thread David Smiley
What is the Solr field type definition for this field? And what sort of spatial data do you add here -- just points or what? ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Mon, Aug 3, 2020 at 10:09 PM sanjay dutt wrote: > Hello Solr commun

Re: org.apache.lucene.util.fst.FST taking up lot of Java Heap Memory

2020-08-07 Thread David Smiley
you are probably not using Solr 8.4.0 or beyond, which moved to having the FSTs off-heap -- at least the ones associated with the field indexes. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Thu, Aug 6, 2020 at 8:19 PM sanjay dutt wrote: >

Re: Multiple "df" fields

2020-08-11 Thread David Hastings
why not use a copyfield for indexing? On Tue, Aug 11, 2020 at 9:59 AM Edward Turner wrote: > Hi all, > > Is it possible to have multiple "df" fields? (We think the answer is no > because our experiments did not work when adding multiple "df" values to > solrconfig.xml -- but we just wanted to do

Number of times in document

2020-08-12 Thread David Hastings
Is there any way to do a query for the minimum number of times a phrase or string exists in a document? This has been a request from some users as other search services (names not to be mentioned) have such a functionality. Ive been using solr since 1.4 and i think ive tried finding this ability

[CVE-2020-13941] Apache Solr information disclosure vulnerability

2020-08-14 Thread David Smiley
to trusted paths * Prevent remote connection when using Windows UNC Paths ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley

Re: SOLR indexing takes longer time

2020-08-18 Thread David Hastings
Another thing to mention is to make sure the indexer you build doesnt send commits until its actually done. Made that mistake with some early in house indexers. On Tue, Aug 18, 2020 at 9:38 AM Charlie Hull wrote: > 1. You could write some code to pull the items out of Mongo and dump > them to d

Re: Error on searches containing specific character pattern

2020-09-03 Thread David Smiley
cf1ff/lucene/core/src/java/org/apache/lucene/util/QueryBuilder.java#L653 If you can reproduce this with the "techproducts" schema, please share the complete query. If there's a problem here, I suspect the synonyms you have may be pertinent. ~ David Smiley Apache Lucene/Solr S

Re: What is the Best way to block certain types of queries/ query patterns in Solr?

2020-09-03 Thread David Smiley
support arbitrary parameters you pass to Solr as-is that you don't know about in advance (i.e. use an allow-list). ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Mon, Aug 31, 2020 at 10:57 AM Mark Robinson wrote: > Hi, > I had come across

HEY, are you using the Analytics contrib?

2020-09-03 Thread David Smiley
Solr maintainers continue to maintain it. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley

Re: Master/Slave

2020-09-30 Thread David Hastings
>whether we should expect Master/Slave replication also to be deprecated it better not ever be depreciated. it has been the most reliable mechanism for its purpose, solr cloud isnt going to replace standalone, if it does, thats when I guess I stop upgrading or move to elastic On Wed, Sep 30, 202

Re: Solr endpoint on the public internet

2020-10-08 Thread David Hastings
Well that’s why I suggested deleting the update handler :) > On Oct 8, 2020, at 2:52 PM, Walter Underwood wrote: > > Let me know where it is and I’ll delete all the documents in your collection. > It is easy, just one HTTP request. > > https://gist.github.com/nz/673027/313f70681daa985ea13ba33a

Re: Solr endpoint on the public internet

2020-10-08 Thread David Hastings
dler. And block Config API to avoid attackers creating new > handlers. > > Regards, > Alex. > >> On Thu, 8 Oct 2020 at 14:54, David Hastings wrote: >> >> Well that’s why I suggested deleting the update handler :) >> >>>> On Oct 8, 2020, at 2:52

Re: converting string to solr.TextField

2020-10-16 Thread David Hastings
"If you want to keep the same field name, you need to delete all of the documents in the index, change the schema, and reindex." actually doesnt re-indexing a document just delete/replace anyways assuming the same id? On Fri, Oct 16, 2020 at 3:07 PM Alexandre Rafalovitch wrote: > Just as a side

Re: converting string to solr.TextField

2020-10-16 Thread David Hastings
and we > need to be free to make important improvements with time." > > And all that aside, you have to re-index all the docs anyway or > your search results will be inconsistent. So leaving aside the > impossible task of covering all the possibilities on the fly, it’s > b

Re: converting string to solr.TextField

2020-10-16 Thread David Hastings
ng all the docs into an > existing index, things like changing from stored=true to > stored=false, adding new fields, deleting fields (although the > meta-data for the field is still kept around) etc. > > > On Oct 16, 2020, at 3:57 PM, David Hastings < > hastings.recurs...@

Re: Solr 8.6.3

2020-10-22 Thread David Smiley
cted the warning about this in 8.7, so you won't see that again. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Thu, Oct 15, 2020 at 4:13 PM Kris Gurusamy wrote: > I've just downloaded solr 8.6.3 and trying to create DIH for loading >

Re: [ANNOUNCE] Apache Solr 8.7.0 released

2020-11-09 Thread David Smiley
FYI an updated Docker image was just published a few hours ago: https://hub.docker.com/_/solr ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Wed, Nov 4, 2020 at 9:06 AM Atri Sharma wrote: > 3/11/2020, Apache Solr™ 8.7 available > > The L

Re: Frequent Index Replication Failure in solr.

2020-11-13 Thread David Hastings
looks like youre repeater is grabbing a file that the master merged into a different file, why not lower how often you go from master->repeater, and/or dont commit so often so you can make the index faster On Fri, Nov 13, 2020 at 12:13 PM Parshant Kumar wrote: > All,please help on this > > On Tu

Re: Faceting: !terms vs mincount precedence

2020-11-17 Thread David Smiley
ul. I know my response isn't a direct answer to your question RE mincount... perhaps it can be made to work? ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Tue, Nov 17, 2020 at 8:21 AM Jason Gerlowski wrote: > Hey all, > > I was usi

Re: data import handler deprecated?

2020-11-30 Thread David Smiley
the audience of news / release notes), the functionality has *moved*. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Mon, Nov 30, 2020 at 8:04 AM Eric Pugh wrote: > You don’t need to abandon DIH right now…. You can just use the Github > h

solr 7.x sql query returns null

2019-04-18 Thread David Barnett
I have a large solr 7.3 collection 400m + documents. I’m trying to use the Solr JDBC driver to query the data but I get a java.io.IOException: Failed to execute sqlQuery 'select id from document limit 10' against JDBC connection 'jdbc:calcitesolr:'. Error while executing SQL "select id from doc

Re: solr 7.x sql query returns null

2019-04-18 Thread David Barnett
Hi Joel Yes it was always setup as cloud. On Thu, 18 Apr 2019, 15:17 Joel Bernstein, wrote: > Was the original index a Solr Cloud index? > > > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Thu, Apr 18, 2019 at 7:48 AM David Barnett > wrote: > >

Re: solr 7.x sql query returns null

2019-04-18 Thread David Barnett
joelsolr.blogspot.com/ > > > > > > On Thu, Apr 18, 2019 at 7:48 AM David Barnett > > wrote: > > > > > I have a large solr 7.3 collection 400m + documents. > > > > > > I’m trying to use the Solr JDBC driver to query the data but I get a > &g

Re: solr 7.x sql query returns null

2019-04-18 Thread David Barnett
> different. > > But as Shawn mentioned the stack trace is not coming from Solr. Is there > more in the logs beyond the Calcite exception? > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Thu, Apr 18, 2019 at 11:04 AM Shawn Heisey wrote: > > >

Re: solr 7.x sql query returns null

2019-04-18 Thread David Barnett
hema is > different. > > But as Shawn mentioned the stack trace is not coming from Solr. Is there > more in the logs beyond the Calcite exception? > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Thu, Apr 18, 2019 at 11:04 AM Shawn Heisey wrote: > > &g

Re: solr 7.x sql query returns null

2019-04-19 Thread David Barnett
; > > If you want to create a jira ticket and attach your schema we can track > > this down. I'll probably attach a special binary to the ticket which has > > additional logging so we can can find out what field is causing the > > problem. > > > > J

Re: solr 7.x sql query returns null

2019-04-19 Thread David Barnett
the ticket which has > additional logging so we can can find out what field is causing the problem. > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Thu, Apr 18, 2019 at 1:38 PM David Barnett wrote: > > > Hi Joel, besides the solr log is there anywhere e

Re: Compound Primary Keys

2019-04-24 Thread David Hastings
another thing to consider doing is just merge the two fields into the id value: "id": "USER_RECORD_12334", since its a string. On Wed, Apr 24, 2019 at 2:35 PM Gus Heck wrote: > Hi Vivek > > Solr is not a database, nor should one try to use it as such. You'll need > to adjust your thinking some

Re: Unable to tag queries (q) in SOLR >= 7.2

2019-04-30 Thread David Smiley
ery parser there and *not* defType (don't set defType or set it to "lucene" which is the default). ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Tue, Apr 30, 2019 at 8:17 AM Fredrik Rodland wrote: > Hi. > > I seems SOLR-115

Re: Reverse-engineering existing installation

2019-05-02 Thread David Smiley
Consider trying to diff configs from a default at the version it was copied from, if possible. Even better, the configs should be in source control and then you can browse history with commentary and sometimes links to issue trackers and code reviews. Also a big part that you can’t see by staring

Re: Search using filter query on multivalued fields

2019-05-03 Thread David Hastings
another option is to index dynamically, so you would index in this case, or this is what i would do: INGREDIENT_SALT_i:40 INGREDIENT_EGG_i:20 etc and query INGREDIENT_SALT_i:[20 TO *] or an arbitrary max value, since these are percentages INGREDIENT_SALT_i:[20 TO 100] On Fri, May 3, 2019 at 12:

Re: Date format issue in solr select query.

2019-05-09 Thread David Smiley
to a string stored field. This is necessary because primitive field types (date, float, int, etc.) normalize the input when the value is internally stored. Perhaps it shouldn't do that -- as you show here the surface form (original) may indicate the precision. ~ David Smiley Apache Lucene/S

Re: Solr query takes a too much time in Solr 6.1.0

2019-05-10 Thread David Hastings
first inclination is your index is cold. On Fri, May 10, 2019 at 9:32 AM vishal patel wrote: > We have 2 shards and 2 replicas in Live environment. we have multiple > collections. > Some times some query takes much time(QTime=52552). There are so many > documents indexing and searching within m

Re: Streaming Expression: get the value of the array at the specified position

2019-05-10 Thread David Hastings
no. On Fri, May 10, 2019 at 11:09 AM Nazerke S wrote: > Hi, > > I am interested in getting the value of the array at the given index. For > example, > > let(echo="b", a=array(1,2,3,4,5), b=getAt(a, 2)) should return 3. > > Is there a way to get access an array's element by indexing? > > Thanks!

Re: Range query syntax on a polygon field is returning all documents

2019-05-12 Thread David Smiley
t supposedly is much more efficient for Geo3D specifically. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Wed, Mar 20, 2019 at 2:00 PM David Smiley wrote: > Hi Mitchell, > > Seems like there's a bug based on wh

Slow ReadProcessor read fields Warnings - Ideas to investigate?

2019-05-22 Thread David Winter
regards David Winter

Re: Empty rows from /export?

2019-05-31 Thread David Hastings
> Ah. So docValues are managed by Solr outside of Lucene. Interesting. i was under the impression docValues are in lucene, and he is just saying that an optimize is not a re-index, its just taking the actual files that already exist in your index and arranging them and removing deletions, an optim

Re: strange behavior

2019-06-06 Thread David Hastings
audit_author.name:Burley,%20S.K. translates to audit_author.name:Burley, DEFAULT_OPERATOR DEFAULT_FIELD:S.K. On Thu, Jun 6, 2019 at 2:46 PM Wendy2 wrote: > > Hi, > > Why "AND" didn't work anymore? > > I use Solr 7.3.1 and edismax parser. > Could someone explain to me why the following query

Re: Urgent help on solr optimisation issue !!

2019-06-07 Thread David Santamauro
Erick, on 6.0.1, optimize with maxSegments only merges down to the specified number. E.g., given an index with 75 segments, optimize with maxSegments=74 will only merge 2 segments leaving 74 segments. It will choose a segment to merge that has deleted documents, but does not merge every segment

Re: Urgent help on solr optimisation issue !!

2019-06-07 Thread David Santamauro
/clarification/ ... expungeDeletes will merge every segment *touched by the current commit* that has a deleted document. On 6/7/19, 10:07 AM, "David Santamauro" wrote: Erick, on 6.0.1, optimize with maxSegments only merges down to the specified number. E.g., given an ind

Re: Urgent help on solr optimisation issue !!

2019-06-07 Thread David Santamauro
5. Best, Erick > On Jun 7, 2019, at 7:07 AM, David Santamauro wrote: > > Erick, on 6.0.1, optimize with maxSegments only merges down to the specified number. E.g., given an index with 75 segments, optimize with maxSegments=74 will only merge 2 segments leaving 74

Re: Re: Query takes a long time Solr 6.1.0

2019-06-07 Thread David Hastings
There isnt anything wrong aside from your query is poorly thought out. On Fri, Jun 7, 2019 at 11:04 AM vishal patel wrote: > Any one is looking my issue?? > > Get Outlook for Android > > > From: vishal patel > Sent: Thursday, June 6, 2019 5

Re: Urgent help on solr optimisation issue !!

2019-06-07 Thread David Santamauro
I use the same algorithm and for me, initialMaxSegments is always the number of segments currently in the index (seen, e.g, in the SOLR admin UI). finalMaxSegments depends on what kind of updates have happened. If I know that "older" documents are untouched, then I'll usually use -60% or even -7

Re: highlighting not working as expected

2019-06-10 Thread David Smiley
Please try hl.method=unified and tell us if that helps. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Mon, Jun 3, 2019 at 4:06 AM Martin Frank Hansen (MHQ) wrote: > Hi, > > I am having some difficulties making highlighting work. For so

Issue with Solr 7.7.2 - ClassCastException: org.apache.solr.common.util.ByteArrayUtf8CharSequence

2019-06-11 Thread David Winter
org.apache.solr.update.processor.AtomicUpdateDocumentMerger.doSet(AtomicUpdateDocumentMerger.java:319) Mit freundlichen Grüßen / Kind regards David Winter

Re: Large Filter Query

2019-06-26 Thread David Hastings
you can use the !terms operator and send them separated by a comma: {!terms f=id}id1,id2,..id1499,id1500 and run facets normally On Wed, Jun 26, 2019 at 2:31 PM Lucky Sharma wrote: > Hi all, > > What we are doing is, we will be having a set of unique Ids of solr > document at max 1500, we

Re: Large Filter Query

2019-06-26 Thread David Hastings
yeah there is a performance hit but that is expected. in my scenario i pass sometimes a few thousand using this method, but i pre-process my results since its a set. you will not have any issues if you are using POST with the uri length. On Wed, Jun 26, 2019 at 3:02 PM Lucky Sharma wrote: > Th

Re: Getting list of unique values in a field

2019-07-12 Thread David Hastings
just use a facet on the field should work yes? On Fri, Jul 12, 2019 at 9:39 AM Steven White wrote: > Hi everyone, > > One of my indexed field is as follows: > > multiValued="false" indexed="true" required="true" stored="false"/> > > It holds the file extension of the files I'm indexing. Th

Re: Getting list of unique values in a field

2019-07-12 Thread David Hastings
nks David. But is there a SolrJ sample code on how to do this? I need > to see one, or at least the API, so I know how to make the call. > > Steven > > On Fri, Jul 12, 2019 at 9:42 AM David Hastings < > hastings.recurs...@gmail.com> > wrote: > > > just use a facet o

Re: How to query against dynamic fields without listing them all?

2019-07-14 Thread David Santamauro
Hi Steven, You can dump all the dynamic fields into a copyField Then you can just set "qf":"CC_COMP_NAME_ALL" On 7/14/19, 10:42 AM, "Steven White" wrote: Hi everyone, In my schema, I have the following field: When I index, I create dynamic field

Re: Solr Geospatial Polygon Indexing/Querying Issue

2019-07-25 Thread David Smiley
to that. Also, FYI geodist() is a function that can take arguments directly which makes more sense when multiple spatial fields are in play. Sadly this aspect is not documented. Suffice it to say, if you do geodist(latLng) (maybe quoted?) then it'll use that field, and parse "pt"

Re: Ranking

2019-07-27 Thread David Hastings
I can’t imagine this is actually true unless you have a default copy field and I is in one of them. Also the letter “I” is a bizarre test case > On Jul 27, 2019, at 3:40 PM, Steven White wrote: > > Hi everyone, > > I have 2 files like so: > > FA has the letter "i" only 2 times, and the file s

Re: Solr Geospatial Polygon Indexing/Querying Issue

2019-07-30 Thread David Smiley
On Tue, Jul 30, 2019 at 4:41 PM Sanders, Marshall (CAI - Atlanta) < marshall.sande...@coxautoinc.com> wrote: > I’ll explain the context around the use case we’re trying to solve and > then attempt to respond as best I can to each of your points. What we have > is a list of documents that in our c

[CVE-2019-0193] Apache Solr, Remote Code Execution via DataImportHandler

2019-07-31 Thread David Smiley
The DataImportHandler, an optional but popular module to pull in data from databases and other sources, has a feature in which the whole DIH configuration can come from a request's "dataConfig" parameter. The debug mode of the DIH admin screen uses this to allow convenient debugging / development o

Re: more like this query parser with faceting

2019-08-12 Thread David Hastings
The easiest way will be to pass in a filter query (fq) On Mon, Aug 12, 2019 at 2:40 PM Szűcs Roland wrote: > Hi All, > > Is there any tutorial or example how to use more like this functionality > when we have some other constraints set by the user through faceting > parameters like price range,

Re: more like this query parser with faceting

2019-08-12 Thread David Hastings
should be fine, https://cwiki.apache.org/confluence/display/solr/MoreLikeThisHandler for more info On Mon, Aug 12, 2019 at 2:49 PM Szűcs Roland wrote: > Hi David, > Thanks the fast reply. Am I right that I can combine fq with mlt only if I > use more like this as a query parser? >

Re: Moving to solrcloud from single instance

2019-08-12 Thread David Hastings
I actually never had a problem with the index being larger than the memory for a standalone instance, but the entire index is on an SSD at least one my end On Mon, Aug 12, 2019 at 3:43 PM Erie Data Systems wrote: > I am starting the planning stages of moving from a single instance of solr > 8 to

Re: Solutio for long time highlighting

2019-08-30 Thread David Smiley
html Still... there is perhaps some value in multi-threading the highlighting for huge docs, but I think we ultimately found no need after re-engineering the highlighter. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Wed, Aug 28, 2019 at 10:36 AM SOLR

Re: ExecutorService support in SolrIndexSearcher

2019-08-30 Thread David Smiley
nd in particular Solr's means of flipping bits in a big bitset to accumulate the DocSet had to be careful so that multiple threads don't try to overwrite the same underlying "long" in the long[]. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davi

Re: upgrading from solr4 to solr8 searches taking 4 to 10 times as long to return

2019-09-05 Thread David Smiley
th debug=timing to see information on each of the components. That may give you a strong clue. If it's in the QueryComponent which actually executes the underlying search then you have some further digging to do. Use a profiler like JVisualVM. ~ David Smiley Apache Lucene/Solr S

Re: upgrading from solr4 to solr8 searches taking 4 to 10 times as long to return

2019-09-07 Thread David Smiley
from docValues=true to false to see if it's a docValues perf issue compared to uninverting. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Sat, Sep 7, 2019 at 3:06 PM Russell Bahr wrote: > Hi David, > I ran the *:* query 10 times against all

Re: upgrading from solr4 to solr8 searches taking 4 to 10 times as long to return

2019-09-07 Thread David Smiley
remove grouping; it's a complexity weight on our codebase. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Sat, Sep 7, 2019 at 5:15 PM David Smiley wrote: > 10s of seconds to respond to a simple match-all query, especially to just > a singl

Re: Query field alias - issue with circular reference

2019-09-08 Thread David Smiley
No but this seems like a decent enhancement request. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Fri, Aug 9, 2019 at 3:07 AM Jaroslaw Rozanski wrote: > Hi Folks, > > > > Question about query field aliases. > > > &

Re: Migrating Bounding box from Lucene to Solr

2019-09-09 Thread David Smiley
other hand, is in the UK. It's also unclear what field type you are using. If you have a polygon then use RptWithGeometrySpatialField and provide it as-such using either WKT or GeoJSON. Supplying a list of points runs the risk that the query won't actually intersect those points. ~

Re: Solr Backup restore

2019-09-13 Thread David Smiley
It would help if you could devise a simple set of command line steps to reproduce/demonstrate the problem using the "bin/solr -e solrcloud" setup. The problem you see ought to be reproducible here if there is a problem. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linke

Re: SolR: How to sort (or boost) by Availability dates

2019-09-24 Thread David Hastings
It sounds like you want to do a normal search but only show available items. You could simply just add a fq parameter with dynamic values based on the current date fq=avaiable_from:[$todays_date TO *] AND available_to[* TO $todays_date] On Tue, Sep 24, 2019 at 9:41 AM Audrey Lorberfeld - audrey.l

Re: Re: Need urgent help with Solr spatial search using SpatialRecursivePrefixTreeFieldType

2019-09-30 Thread David Smiley
ction (e.g. min/max/sum). ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Mon, Sep 30, 2019 at 10:22 AM Anushka Gupta < anushka_gu...@external.mckinsey.com> wrote: > Hi, > > > > I want to be able to filter on different cities

Re: Re: Need urgent help with Solr spatial search using SpatialRecursivePrefixTreeFieldType

2019-10-01 Thread David Smiley
ading space after 'fq'. This is a syntax parsing gotcha that has to do with how embedded queries are parsed, which is what you need to do as you need to compose two with an operator. It'd be kinda awkard to fix that gotcha in Solr. There are other techniques too, but this is th

Re: Protecting Tokens from Any Analysis

2019-10-08 Thread David Hastings
Another thing to add to the above, > > IT:ibm. In this case, we would want to maintain the colon and the > capitalization (otherwise “it” would be taken out as a stopword). > stopwords are a thing of the past at this point. there is no benefit to using them now with hardware being so cheap. On Tu

Re: Re: Protecting Tokens from Any Analysis

2019-10-09 Thread David Hastings
ordability of > hardware...can you expand? I'm not sure I understand. > > > > -- > > Audrey Lorberfeld > > Data Scientist, w3 Search > > IBM > > audrey.lorberf...@ibm.com > > > > > > On 10/8/19, 1:01 PM, "David Hastings" >

Re: Re: Protecting Tokens from Any Analysis

2019-10-09 Thread David Hastings
bm.com wrote: > > > > Hey Alex, > > > > Thank you! > > > > Re: stopwords being a thing of the past due to the affordability of > hardware...can you expand? I'm not sure I understand. > > > > -- > > Audrey Lorberfeld > > Data Scient

Re: Windows Production

2019-10-09 Thread David Barnett
7;m working on a project and I need solr in production mode on windows > server. > > Regards > Suleiman Hassan > App4legal > -- *David Barnett* O&G Technology Consulting Ltd oand...@gmail.com +44 (0) 7753 235608

Re: Protecting Tokens from Any Analysis

2019-10-09 Thread David Hastings
Google in relevance tests, probably because of phrase IDF. > > More Like This could do the same thing, but it seems to be really slow and > not especially useful as a search component. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blo

Re: Re: Re: Re: Protecting Tokens from Any Analysis

2019-10-09 Thread David Hastings
oh and by 'non stop' i mean close enough for me :) On Wed, Oct 9, 2019 at 2:59 PM David Hastings wrote: > if you have anything close to a decent server you wont notice it all. im > at about 21 million documents, index varies between 450gb to 800gb > depending on merges, and

<    5   6   7   8   9   10   11   12   13   14   >