Parse solr json object

Hi, for enabling highlighting i want to parse json object. for readilibility i included xml format of that json object. please tell me how should i parse this object using $.each("", function(i,item){ so that i could get highlighted result. − − − These elegant and fluid earrings have six r

commit time and lock

I am using solrj to index the data. I have around 5 docs indexed. As at the time of commit due to lock server stop giving response so I was calculating commit time: double starttemp = System.currentTimeMillis(); server.add(docs); server.commit(); System.out.println("total time in commit = " +

Re: Problem with CSV update handler

Hello! Once again thanks for the response ;) So the solution is to generate the data files once again and either adding the space after doubled encapsulator or changing the encapsulator to the character that does not occur in the filed values (of course the one taht will be split). -- Regards,

Re: How to highlight a field in searchresult using solr

Jan Høydahl / Cominvent, without parsing json object for i am able to show the result just using the url with hl=true and hl.fl="somefield". and i got the field highlighted. but what i want is to highlight only data i pass in the query ,in that "somefield" not full field data. How can i do this

Re: Searching in Traditional / Simplified Chinese Record

By "changing the record", i mean translate them word by word using software. Sorry i m new for this kind of modification. For synonyms filter, would there be a big table and result in degrade of indexing performance? I have tried using filter like ICUTransformFilterFactory but it seems not wor

Best way to index source files?

I have a directory of source files (text) and want to index them into SOLR. What is the easiest way to do that? Thanks. I could not see how to use DIH handler to do itŠ Thanks.

Re: Extending Solr Highlighter to pull information from external source

I found https://issues.apache.org/jira/browse/SOLR-1397 but there is not much going on there LUCENE-1522 has a lot of fascinating discussion on this topic though There is a couple of long lived issues in jira for this (I'd like to try to se

Re: Update JSON Invalid

On Mon, Jun 20, 2011 at 11:25 PM, Shawn Heisey wrote: > On 6/20/2011 8:08 PM, entdeveloper wrote: >> >> Technically, yes, it's valid json, but most libraries treat the json >> objects >> as maps, and with multiple "add" elements as the keys, you cannot properly >> deserialize. >> >> As an example,

Re: Update JSON Invalid

On 6/20/2011 8:08 PM, entdeveloper wrote: Technically, yes, it's valid json, but most libraries treat the json objects as maps, and with multiple "add" elements as the keys, you cannot properly deserialize. As an example, try putting this into jsonlint.com, and notice it trims off one of the doc

Update JSON Invalid

I'm looking at the wiki article about updating the index with json and the format doesn't seem well formed to me. http://wiki.apache.org/solr/UpdateJSON Technically, yes, it's valid json, but most libraries treat the json objects as maps, and with multiple "add" elements as the keys, you cannot pr

Re: Extending Solr Highlighter to pull information from external source

(11/06/20 21:41), Jamie Johnson wrote: I am trying to index data where I'm concerned that storing the contents of a specific field will be a bit of a hog so we are planning to retrieve this information as needed for highlighting from an external source. I am looking to extend the default solr hi

Re: Problem with CSV update handler

Multi-valued CSV fields are double encoded. We start with: "aaa ""bbb""ccc"' Then decoding one leve, we get: aaa "bbb"ccc Decoding again to get individual values results in a decode error because the encapsulator appears unescaped in the middle of the second value (i.e. invalid CSV). One easier

Re: Optimize taking two steps and extra disk space

On 6/20/2011 3:18 PM, Michael McCandless wrote: With segmentsPerTier at 35 you will easily cross 70 segs in the index... If you want optimize to run in a single merge, I would lower sementsPerTier and mergeAtOnce (maybe back to the 10 default), and set your maxMergeAtOnceExplicit to 70 or higher.

Re: Problem with CSV update handler

Hi! Yonik, thanks for the reply. I just realized that the example I gave was not full - the error is returned by Solr only when the field is multivalued and the values in the fields are splited. For example, the following curl command give me the mentioned error: curl 'http://localhost:8983/sol

Re: Problem with CSV update handler

This works fine for me: curl http://localhost:8983/solr/update/csv -H 'Content-type:text/plain' -d 'id,name "1","aaa ""bbb"" ccc"' -Yonik http://www.lucidimagination.com On Mon, Jun 20, 2011 at 3:17 PM, Rafał Kuć wrote: > Hello! > >  I have a question about the CSV update handler. Lets say I h

Re: Optimize taking two steps and extra disk space

On Mon, Jun 20, 2011 at 4:00 PM, Shawn Heisey wrote: > On 6/20/2011 12:31 PM, Michael McCandless wrote: >> >> Actually, TieredMP has two different params (different from the >> previous default LogMP): >> >>   * segmentsPerTier controls how many segments you can tolerate in the >> index (bigger nu

Re: Optimize taking two steps and extra disk space

On 6/20/2011 12:31 PM, Michael McCandless wrote: Actually, TieredMP has two different params (different from the previous default LogMP): * segmentsPerTier controls how many segments you can tolerate in the index (bigger number means more segments) * maxMergeAtOnce says how many segments

Problem with CSV update handler

Hello! I have a question about the CSV update handler. Lets say I have the following file sent to CSV update handler using curl: id,name "1","aaa ""bbb""ccc" It throws an error, saying that: Error 400 java.io.IOException: (line 0) invalid char between encapsulated token end delimiter If I cha

CSV update handler question

Hello! I have a question about the CSV update handler. Lets say I have the following file sent to CSV update handler using curl: id,name "1","aaa ""bbb""ccc" It throws an error, saying that: Error 400 java.io.IOException: (line 0) invalid char between encapsulated token end delimiter If I cha

spellcheck.q results in null error when used with suggester

I have been playing around with the new suggestion feature in Solr 3.1 but have run into issues when using spellcheck.q. I initiate the suggestion handler using: http://www.domain.tld:8983/solr/suggest?spellcheck.build=true http://www.domain.tld:8983/solr/suggest?spellcheck.reload=true And the

Re: Optimize taking two steps and extra disk space

On Sun, Jun 19, 2011 at 12:35 PM, Shawn Heisey wrote: > On 6/19/2011 7:32 AM, Michael McCandless wrote: >> >> With LogXMergePolicy (the default before 3.2), optimize respects >> mergeFactor, so it's doing 2 steps because you have 37 segments but 35 >> mergeFactor. >> >> With TieredMergePolicy (def

Re: Searching in Traditional / Simplified Chinese Record

Wayne I am not sure what you mean by 'changing the record'. One option would be to implement something like the synonyms filter to generate the TC for SC when you index the document, which would index both the TC and the SC in the same location. That way your users would be able to search with

Re: Caching queries.

Thanks, this is exactly what I'm looking for! -- View this message in context: http://lucene.472066.n3.nabble.com/Caching-queries-tp3078271p3087497.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: upgrading to Tika 0.9 on Solr 1.4.1

I've unsuccessfully attempted to go down this road - there are API changes, some of which I was able to solve by taking code snippets from Solr 3.1. Some extraction-related tests for wouldn't pass (look for 'Solr 1.4.1 and Tika 0.9 - some tests not passing' in the archive). Ultimately, I de

Multiple Solrs on the same box

First, a couple of assumptions. We have boxes with a large amount (~70Gb) of memory which we're running Solr under. We've currently set -Xmx to 25Gb with the GC settings -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing We're reluctant to up the -Xmx

Re: query about the server configuration

thanks.. However I am still not clear about that How can I achieve the requirement: 1. I have around 1 file to index daily. 2. I need to index in every 5 minutes 3. size of the files are 50mb per day 4. 10 to 20 concurrent users please help.. regards On Monday 20 June 2011 06:07 PM, Erick

Highlighting map use unique key field?

Hi - A simple yes or no question, I think. I want to retrieve highlighting result from a QueryResponse. I know to use the following: Map>> highlighting = resp.getHighlighting(); Most of the examples I've seen use the document uid to extract the results like so: String key = resultDec.getFi

Replication Problem

Hello List, What's wrong with my replication? One of my slaves logs this from time to time. The slave runs on 3.1.0. 20.06.2011 12:31:32 org.apache.solr.common.SolrException log SCHWERWIEGEND: ClientAbortException: java.io.IOException at org.apache.catalina.connector.OutputBuffer.realWr

Re: Problem with SolrTestCaseJ4

Thnx a lot! :) /Jakob Den 20. juni 2011 kl. 13:51 skrev Robert Muir : > you must use junit 4.7.x, not junit 4.8.x > > On Mon, Jun 20, 2011 at 6:21 AM, Jakob Vad Nielsen > wrote: >> Hi, >> >> I'm trying to create some integrations tests within my project using JUnit >> and the SolrTestCaseJ4 (

Re: How to highlight a field in searchresult using solr

Hi, You have asked the question before in another thread, and I have tried to answer http://search-lucene.com/m/586GL1pMaEc1 You DO need to get the highlighted result from the highlighting tag separately. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com Solr Training -

Re: Complex situation

It'll start to get ugly, but simply add clauses to your facet.query. Note that you can submit as many facet.query entries as you want. You can also configure them as defaults in your request handler... Something like facet.query=startdate:[* TO NOW] AND enddate:[NOW TO *] AND openingHours:800 Not

Re: Master Slave Replication in Solr Cloud - What happens if the master is not available?

You're right, thanks! 2011/6/20 Erick Erickson : > Hmmm, be a little careful here with terminology. > Shards may be unnecessary if you  can put your whole index > on a single searcher. It's preferable to   simply have each > slave hold a complete copy of the index, no sharding necessary. > > Best

Re: Complex situation

with this facet.query=startdate:[* TO NOW] AND enddate:[NOW TO *] i will get all the results?? Now i get the startdate and enddate from my db with the DIH. My schema.xml looks like this: When i use the facet.query i only get a count with companies. What i want is a count for openinghours.

Re: PorterStemFilter kills JVM

if you can create a issue, with a reproducible test, we can try to come up with a workaround... no promises but I'd be willing to give it a shot. On Mon, Jun 20, 2011 at 10:11 AM, Bernd Fehling wrote: > > Now this is a good one, PorterStemFilter kills JVM (reproducible). > > Should I post this on

Re: Extending Solr Highlighter to pull information from external source

Yes that sounds about right. I also have in mind an optimization for highlighting so it doesn't need to pull the whole field value. The fast vector highlighter is working with offsets into the field, and should work better w/random access into the field value(s). But that should come as a la

Re: upgrading to Tika 0.9 on Solr 1.4.1

Hi Surendra, On Jun 20, 2011, at 4:59 AM, Surendra wrote: > Hey Chris > > I have added tika-core 0.9 and tika-parsers 0.9 to Solr1.4.1 (extraction/lib) > after building them using the source provided by TIKA. Now I have an issue > with > this. I am working with extracting PDF content using Solr

PorterStemFilter kills JVM

Now this is a good one, PorterStemFilter kills JVM (reproducible). Should I post this on lucene list, because of org.apache.lucene.analysis.PorterStemFilter ? Regards Bernd # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x7f80132057ac, pid=

Re: Master Slave Replication in Solr Cloud - What happens if the master is not available?

Hmmm, be a little careful here with terminology. Shards may be unnecessary if you can put your whole index on a single searcher. It's preferable to simply have each slave hold a complete copy of the index, no sharding necessary. Best Erick On Mon, Jun 20, 2011 at 9:45 AM, Mark Schoy wrote: >

Re: SolrJ and Range Faceting

Martin, I believe this does work. http://wiki.apache.org/solr/SolrQuerySyntax shows an example ( createdate:[1976-03-06T23:59:59.999Z TO 1976-03-06T23:59:59.999Z+1YEAR) and it worked on my dataset as well. So it looks like it should be as easy as appending the gap to the value. On Mon, Jun 13,

Re: Extending Solr Highlighter to pull information from external source

perhaps it should be an array that gets returned to be consistent with getValues(fieldName); On Mon, Jun 20, 2011 at 9:59 AM, Jamie Johnson wrote: > Yes, in that case the code becomes > > if(!schemaField.stored()){ > > > SchemaField keyField = schema.getUniqueKeyField(); >

Re: Extending Solr Highlighter to pull information from external source

Yes, in that case the code becomes if(!schemaField.stored()){ SchemaField keyField = schema.getUniqueKeyField(); String key = doc.getValues(keyField.getName())[0]; docTexts = doc.getValues(fieldName); if(key != null && key.length() > 0){

Re: Extending Solr Highlighter to pull information from external source

Another option for determining whether to go to external storage would be to examine the SchemaField, see if it is stored, and if not, try to fetch from a file or whatever. That way you won't have to configure anything. -Mike On 06/20/2011 09:46 AM, Jamie Johnson wrote: In my case chucking t

How to highlight a field in searchresult using solr

Hi, I want to include highlighting feature in my search application using solr. I did necessary changes in config file for highlighting feature. when i run the url with hl=true&hl.fl=somefield i got the . now i want to display these highlighted fields in my search.jsp page. To display the search

Re: Extending Solr Highlighter to pull information from external source

In my case chucking the external storage is simply not an option. I'll definitely share anything I find, the following is a very simple example of adding text to the default solr highlighter (had to copy a large portion of the class since the method that actually does the highlighting is private

Re: Master Slave Replication in Solr Cloud - What happens if the master is not available?

Thanks for your answer Erick. So the easiest way will be to set up 2 shard cluster with shard replicas ;) 2011/6/20 Erick Erickson : > No, there's nothing built into Solr to automatically promote a slave > to a master. > > You have several choices here. One is to build a new master and > reindex

Re: Extending Solr Highlighter to pull information from external source

Mike I would be very interested in the answer to that question too. My hunch is that the answer is no too. I have a few text databases that range from 200MB to about 60GB with which I could run some tests. I will have some downtime in early July and will post results. From what I can tell the

Re: Master Slave Replication in Solr Cloud - What happens if the master is not available?

No, there's nothing built into Solr to automatically promote a slave to a master. You have several choices here. One is to build a new master and reindex from scratch. Another is to configure your slave as a new master and then bring up a new machine and have it replicate. Now make that new machi

Re: Complex situation

DateMath (http://wiki.apache.org/solr/SolrQuerySyntax) contains the keyword NOW, and since you can form your facet queries as arbitrary Solr queries, something like facet.query=startdate:[* TO NOW] AND enddate:[NOW TO *] should work? Best Erick On Mon, Jun 20, 2011 at 3:01 AM, roySolr wrote: > H

Re: Request handle solrconfig.xml Spellchecker

Look in your solr/data/spellcheck directory. Is there anything in there? It should look like a regular Solr index, you should even be able to invoke Luke on it. If there's nothing there, then something's wrong with your build process, probably your configuration in solrconfig.xml Have you made

Re: Extending Solr Highlighter to pull information from external source

I'd be very interested in this, as well, if you do it before me and are willing to share... A related question I have tried to ask on this list, and have never really gotten a good answer to, is whether it makes sense to just chuck the external storage and treat the lucene index as the primary

Re: Showing facet of first N docs

2011/6/18 Dmitry Kan > Do you mean you would like to boost the facets that contain the most of the > lemmas? > That would be good, but I'd prefer getting facets, for example, from first 50 of 500 docs only . > What is the user query in this case and if possible, what is the use case > (may be

Questions about Solr MLTHanlder, performance, Indexes

Hi folks, I am new to Solr, and using it for web application. I have been experimenting with it and have a couple of doubts which I was unable to resolve by Google. Our portal allows users to upload content and the fields we use are - title, description, transcript, tags. Now each of the content h

How to add unextracted field when using Sorl Cell

Hello, I would like to transform my existing Lucene application to Solr but I'm struggling with one thing (most important though). I would like to index XHTML files using ExtractingRequestHandler - no problem with that. But, I have a custom Tokenizer which expects well formed xml (whole xhtml docu

Re: Request handle solrconfig.xml Spellchecker

Try http://localhost:8983/solr/spellcheck?q=komputer&spellcheck=true If you want the spellchecker to be enabled in your default search handler, you need to include this in your default search-hanlder config: spellcheck -- Jan Høydahl, search solution architect Cominvent AS - www.co

Extending Solr Highlighter to pull information from external source

I am trying to index data where I'm concerned that storing the contents of a specific field will be a bit of a hog so we are planning to retrieve this information as needed for highlighting from an external source. I am looking to extend the default solr highlighting capability to work with inform

Re: query about the server configuration

>From Ahmet Arslan's email to Ranveer: It uses HttpClient under the hood. You can pass httpClient to its constructor too. It seems that MultiThreadedHttpConnectionManager has setMaxConnectionsPerHost method. String serverPath = "http://localhost:8983/solr";; HttpClient client = new HttpClient(ne

Re: paging and maintaingin a cursor just like ScrollableResultSet

See the queryResultCache setting in your solrconfig.xml file. For a given query it keeps a (configurable) list of results, you could simply make this very large, at the usual memory cost. Best Erick On Sun, Jun 19, 2011 at 6:02 PM, Hiller, Dean x66079 wrote: > As you probably know, using Query

Re: about the SolrServer server = new CommonsHttpSolrServer(URL);

I suggest you just try some things and see. The real problem isn't the SolrJ side, it's whether your Solr instance can handle the load. But assuming that your Solr instance can keep up, there's no reason setting this to 30-40 would be a problem. You might also think about using something like jMe

Master Slave Replication in Solr Cloud - What happens if the master is not available?

Hi, if I use a master slave replication in Solr Cloud and the master crashes, can the slave automatically switch to master mode? Or is there another way to index documents after the master is down? Thanks.

Re: Weird optimize performance degradation

Hmmm, that is odd, anyone else want to chime in here? But optimizing isn't going to help with the strange commit times, it'll only make it worse. It's not doing you much if any good, so I'd think about not optimizing About the commit times in general. Depending upon when the merge happens, lo

Re: Solr Multithreading

OK, *how* does it fail when indexing attachments? What have you tried? Best, Erick On Sun, Jun 19, 2011 at 10:35 AM, Rahul Warawdekar wrote: > Hi, > > I am currently working on a search based project which involves > indexing data from a SQL Server database including attachments using > DIH. > F

Re: why too many open files?

(11/06/20 16:16), Jason, Kim wrote: Hi, Mark I think FileNotFoundException will be worked around by raise the ulimit. I just want to know why segments are created more than mergeFactor. During the googling, I found contents concerning mergeFactor: http://web.archiveorange.com/archive/v/bH0vUQzfY

Re: upgrading to Tika 0.9 on Solr 1.4.1

Mattmann, Chris A (388J jpl.nasa.gov> writes: > > Hi Jo, > > You may consider checking out Tika trunk, where we recently have a Tika JAX-RS web service [1] committed as > part of the tika-server module. You could probably wire DIH into it and accomplish the same thing. > > Cheers, > Chris >

Re: Problem with SolrTestCaseJ4

you must use junit 4.7.x, not junit 4.8.x On Mon, Jun 20, 2011 at 6:21 AM, Jakob Vad Nielsen wrote: > Hi, > > I'm trying to create some integrations tests within my project using JUnit > and the SolrTestCaseJ4 (from Solr-test-framework 3.2.0) helper class. The > problem is that I'm getting an Ass

Re: Multiple Cores

I would say it all depends on what you are trying to do. Unlike a relational database, in Solr the data does not need to be normalised, you need to put everything into an index so that you can achieve whatever feature it is that you want. For example, you may search on customer and want a facette

Problem with SolrTestCaseJ4

Hi, I'm trying to create some integrations tests within my project using JUnit and the SolrTestCaseJ4 (from Solr-test-framework 3.2.0) helper class. The problem is that I'm getting an AssertionError for LuceneTestCase.java: java.lang.AssertionError: ensure your setUp() calls super.setUp()!!! at o

Searching in Traditional / Simplified Chinese Record

Hi, I 've recently make change to my schema.xml to support import of Chinese Record. What i want to do is to search both Traditional Chinese(TC) (e.g. ?? )and Simplified Chinese (SC) (e.g. ??) Record when in the same query. I know I can do that by encoding all SC Record to TC. I want to chang

Re: Multiple Cores

On Mon, Jun 20, 2011 at 10:08 AM, jboy79 wrote: > Hi, I am new to SOLR and would like to know if multiple cores is the best way > to deal with having a product and customer index. If the indexes are going to have very different fields, it is probably the case that you want to go with multiple Sol

Solr Clustering For Multiple Pages

Hi How can i create cluster for all records. Currently i am sending clustering=true param to solr and it give the cluster in response , but it give for 10 rows because rows=10 . So please suggest me how can i get the cluster for all records . How can i search with in cluster . e.g clus

Re: relevant result for query with boost factor on parameters

>but if suppose field1 does not contain both the term *"rock" and "roll", >* >*special attention *then field 2 results should take the priority (show >the results which has both the terms first and then show the results with >respect to boost factor or relevance) >if both the fields do not con

Re: relevant result for query with boost factor on parameters

You can try following: 1. Try to increase boost for fields(say, field-1^100, field-2^20), and pass field-3 as a filtered query(using fq parameter). This way field-3 won't effect the scoring. 2. Some implicit factors like length normalization would defer the results, so, you can also switch it off

Full-import of DIH use too much resource of Database

Dear all, My DB system is Microsoft SQL Server 2008, it contents approximately 200.000 Records. Each time full-import to Solr Index(we use DIH for this task ), It use too much database resource. Any query requests to DB are too slow. Are you know solution for this scenario or may be you know

Re: why too many open files?

12 shards on the same machine? > Hi, All > > I have 12 shards and ramBufferSizeMB=512, mergeFactor=5. > But solr raise java.io.FileNotFoundException (Too many open files). > mergeFactor is just 5. How can this happen? > Below is segments of some shard. That is too many segments over mergFactor. >

Re: Why are not query keywords treated as a set?

That only removed tokens on the same position, as the wiki explains. Gabrielle, why would you expect that? You input two tokens so you query for two tokens, why would it be a `set` ? > this might help in your analysis chain > > http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.Re

RE: Building Solr 3.2 from sources - can't get war

> https://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_2/ Thanks, that worked and as I now see was quite obvious (ashamed). > Building dist succeeds, but I just tried dist-war on my checked out 3.2 > and it failed, ending with the following error: > > BUILD FAILED > /opt/ncindex/src/

Multiple Cores

Hi, I am new to SOLR and would like to know if multiple cores is the best way to deal with having a product and customer index. If this is the case how do you go about indexing on multiple cores. Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Multiple-Cores-tp3084817p3

Re: why too many open files?

Hi, Mark I think FileNotFoundException will be worked around by raise the ulimit. I just want to know why segments are created more than mergeFactor. During the googling, I found contents concerning mergeFactor: http://web.archiveorange.com/archive/v/bH0vUQzfYcdtZoocG2C9 Yonik wrote: "mergeFactor

Re: Showing facet of first N docs

On Thu, 2011-06-16 at 12:39 +0200, Tommaso Teofili wrote: > Do you know if it is possible to show the facets for a particular field > related only to the first N docs of the total number of results? It collides with the inner working in Solr, as faceting does not process the doc-IDs from the match

Re: Complex situation

Hi, I want all the results, not only the results for current season. Let say i search for "supermarket", i get results 1, 2 and 3 in my response(previous post) Then i want on the left part some facets with openinghours. Let's say it is today 02/08/2011. Then my facets looks like this: 18:00(2) 20