Re: query parsing output in analysis page and query page are different

2017-08-07 Thread Erick Erickson
Your problem is probably that the query _parser_ gets in there before the input gets to the analysis chain. When you use the admin/analysis page, it's as though the query parser has already broken the query up and assigned it. Add to that that wildcard queries have their own quirks when parsing an

Re: Storing data in Solr

2017-08-07 Thread Erick Erickson
Well, a very common pattern is to use Solr to search, storing just enough in each field (stored="true") to return to the user search results that give enough information to determine whether they want to look at the original document. When the click on a choice (or a link like "download PDF") then

Re: Storing data in Solr

2017-08-07 Thread sg1973
I have written the code to publish to Solr but i am wondering what is the right way to do it. Is directly putting data in Solr OK or putting it in a separate cache and then building solr on top of it? what are the pros and cons of each? -- View this message in context: http://lucene.472066.n3.n

Re: Storing data in Solr

2017-08-07 Thread Deepak Vohra
Solr indexes data for search and if search is the main criterion Solr should be used. On Mon, 8/7/17, sg1973 wrote: Subject: Storing data in Solr To: solr-user@lucene.apache.org Received: Monday, August 7, 2017, 6:55 PM Hello All, I am new to

Re: Storing data in Solr

2017-08-07 Thread Deepak Vohra
Which database is to be integrated? Solr provides Data Import Handlers for several databases including Oracle and MySQL. On Mon, 8/7/17, sg1973 wrote: Subject: Storing data in Solr To: solr-user@lucene.apache.org Received: Monday, August 7, 2017,

Storing data in Solr

2017-08-07 Thread sg1973
Hello All, I am new to Solr and have a question. I have to load about 1 million records from a DB table (with say 30 columns/row) and then run various search queries on it. I see 2 ways to do it. Store the data directly in Solr versus store in in a cache and then search on it using Solr. I am tryin

query parsing output in analysis page and query page are different

2017-08-07 Thread radha krishnan
Hi, I created the following fieldType in schema.xml mapping.txt contains the following (replacing dot with white space) "." => " " and using the above in the field 1. in the analysis page on the solr UI (http://localhost:8984/solr/#/tenant1-core-1/analysis)

Re: Custom FunctionQuery in Solr

2017-08-07 Thread Erick Erickson
bq: The goal is to take a field called 'foo' from all the documents and modify that field with a custom function. I don't really understand that. You use the contents of the field to modify the score of a doc with a custom function query. If you want to modify the field that's returned with a doc

Re: SynonymFilterFactory needs bounce for every change

2017-08-07 Thread Erick Erickson
Did you try reloading the relevant collections? The synonyms are read at core load time and reloading the collection should do that. bq: Also can we have synonyms defined for certain documents and not common to all the documents indexed Not that I know of Best, Erick On Mon, Aug 7, 2017 at 3:57

SynonymFilterFactory needs bounce for every change

2017-08-07 Thread Abhijit Pawar
Hi, I am using SynonymFilterFactory in my custom fieldType and I observed SOLR picks up new synonyms from the synonyms.txt file only when I bounce the server. Is there something we can do to get the synonyms picked up without bouncing server? Also can we have synonyms defined for certain document

Re: Limiting the number of queries/updates to Solr

2017-08-07 Thread S G
I tried using the Jetty's QoS filter for rate limiting the queries. It has a good option to apply different rates per URL pattern. However, the same is not being picked up by Solr and the details of the same are shared on https://stackoverflow.com/questions/45536986/why-is-this-qos-jetty-filter-no

Re: Indexing a CSV that contains double quotes

2017-08-07 Thread Ahmet Arslan
Hi Devon, I mean this: curl 'http://10.0.1.24:8983/solr/products/update?commit=true&encapsulator=";' --data-binary @solrItmList.csv -H 'Content-type:application/csv' Ahmet On Monday, August 7, 2017, 9:00:13 PM GMT+3, O'Shaughnessy, Devon wrote: Hi Ahmet, I'm afraid I don't understan

Custom FunctionQuery in Solr

2017-08-07 Thread Ali Husain
I'm trying to implement something like this in Solr 6.6 http://www.solrtutorial.com/custom-solr-functionquery.html Custom Solr FunctionQueries - Solr Tutorial.com www.solrtutorial.com Custom Solr FunctionQueries. Solr FunctionQueries a

Re: Indexing a CSV that contains double quotes

2017-08-07 Thread O'Shaughnessy, Devon
Hi Ahmet, I'm afraid I don't understand, do you think you could clarify a little bit? Thanks, Devon O'Shaughnessy Developer/Analyst Upper Lakes Foods p: 800.879.1265 | ext: 4135 w: upperlakesfoods.com [1498580146444_PastedImage] __

Re: Indexing a CSV that contains double quotes

2017-08-07 Thread Ahmet Arslan
Hi Devon, I think you need to supply encapsulator=" parameter-value pair. Ahmet On Monday, August 7, 2017, 7:57:45 PM GMT+3, O'Shaughnessy, Devon wrote:    Hello all, I'm pretty new at Solr, having only worked with in a couple weeks, and I'm guessing I'm having a newbie problem of

Indexing a CSV that contains double quotes

2017-08-07 Thread O'Shaughnessy, Devon
Hello all, I'm pretty new at Solr, having only worked with in a couple weeks, and I'm guessing I'm having a newbie problem of some sort. I'm a little confused about how Solr works with double quotes within strings. I'm uploading a CSV to Solr once a day containing some item data, some of which

Re: Architecture: How to commercialise a Solr based application?

2017-08-07 Thread Paul Smith Parker
Hello Charlie, Thank you for your reply. I am not trying to build anything like Searchblox nor Lucidworks Fusion. My expertise is in my customer’s domain, and I would like to sell them a product that fulfils their need to search their content in a smart way: easy-to-use admin UIs, clever inges

Re: solr hangs

2017-08-07 Thread Erick Erickson
I've seen this error happen when there are, well, too many threads simultaneously active and the stack space exhausts available memory, here's some discussion: https://stackoverflow.com/questions/36898701/how-does-java-jvm-allocate-stack-for-each-thread With your settings, you can _create_ 65K th

Re: Reg. recommended Solr 6.x version to use in prod

2017-08-07 Thread Erick Erickson
I'd go with 6.6. Technically you do not have to re-index, although if you can it's preferable. Solr (well, mostly Lucene) will allow reading "one major version back", so any 6x Solr should work fine with a 5x index. As the segments get merged, they'll be brought up to the 6x format. There's also t

Re: IndexReaders cannot exceed 2 Billion

2017-08-07 Thread Yago Riveiro
You have the maximum number of docs in a single shard. If I'm not wrong, the only solution is split the index in more shards (if you are running solrcloud mode). -- /Yago Riveiro On 7 Aug 2017, 16:48 +0100, Wael Kader , wrote: > Hello, > > I faced an issue that is making me go crazy. > I am ru

IndexReaders cannot exceed 2 Billion

2017-08-07 Thread Wael Kader
Hello, I faced an issue that is making me go crazy. I am running SOLR saving data on HDFS and I have a single node setup with an index that has been running fine until today. I know that 2 billion documents is too much on a single node but it has been running fine for my requirements and it was pr

Reg. recommended Solr 6.x version to use in prod

2017-08-07 Thread Hari Baskar
Hi,    We are currently running Solr 5.1 in production. We do not use SolrCloud now . We plan to move to 6.x and use SolrCloud. Any specific recommendations on the recommended Solr 6.x version to use in prod. We are open to reindex all the collections during this migration. Outr major use is ful

Re: solr hangs

2017-08-07 Thread hawk....@139.com
Hi Shawn, The software config of the server is: OS: CentOS 7 Solr Version: 6.5 The system sends delete requests to the Solr server, the requests are around 5k to 6k.The exception occurred from time to time, and we have to restart the node to continue the service. The max threads setting os t

Re: solr hangs

2017-08-07 Thread Shawn Heisey
On 8/7/2017 12:35 AM, hawk@139.com wrote: > Below is the OOM exception. > > 2017-08-07 12:45:48.446 WARN (qtp33524623-4275) [c:taoke s:shard2 > r:core_node4 x:taoke_shard2_replica1] o.e.j.u.t.QueuedThreadPool > java.lang.OutOfMemoryError: unable to create new native thread The last line that

Re: MongoDb vs Solr

2017-08-07 Thread Charlie Hull
On 05/08/2017 12:28, GW wrote: For The Guardian, Solr is the new database | Lucidworks

Re: Architecture: How to commercialise a Solr based application?

2017-08-07 Thread Charlie Hull
Hi Paul, You should be aware you're doing something that has been tried many times before - there are lots of Lucene-based 'packaged' search products out there, from Searchblox to Lucidworks Fusion to Attivio. It's not a small task. You should focus not on the technology (you could build this

Re: Get results in multiple orders (multiple boosts)

2017-08-07 Thread Luca Dall'Osto
Hello Rick, thanks for your answer. Yes, I compose solr query from frontend request, but I'm not able to sort by a custom order, only by natural order (for example: sort=category desc, source desc, /*...*/ ). How do you set a custom sort order in solr? Thanks Luca On Friday, August 4, 2