Re: DataImportHandlerException for custom DIH Transformer

Resurrecting an old thread. I faced exact problem as Tommy and the jar was in {solr.home}/lib as Noble had suggested. My custom transformer overrides following method as per the specification of Transformer class. public Object transformRow(Map row, Context context); But, in the cod

Re: stream.url

Hi Hoss, Thanks for reply and it got working The reason was as you said i was not double escaping i used %2520 for whitespace and it is working now Thanks, satya

Re: Query result ranking - Score independent

My request was very simple: q= astronomy^0 And Solr returned the exception. Maybe the zero boost factor is not causing the exception? 1) We indexed n documents with a Schema.xml. 2)Then we changed some field type in the Schema.xml 3)Then we indexed other m documents Maybe this could cause the exc

Phrase search + multi-word index time expanded synonym

Hello, well, first, here's the field type that is searched : ignoreCase="true" expand="true"/> mapping="mapping-ISOLatin1Accent.txt"/> mapping="mapping-ISOLatin1Accent.txt"/> here's the synonym from the synonyms-fr.txt file : ... PS,Parti socialiste ... and here's the query :

Multi core schema file

I have a query regarding multi core schema files which I can't find the answer to. In a multi core set up does each core need it's own schema file? I have two cores with their own schema files, which are a straight copy of each other, and I just wanted to maintain one schema file. I see there's

Creating a sub-index from another

Hej, I have a Solr Index with several million documents. I need to implement some text mining processes and I would like to create a million documents index from the original for some tests. How can I do it? Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com

Solr searching "harri" finds "harry"

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hey, I've got a Solr server up and running over about 10 million rows, I have a text column that I use for my main search and a couple of int/str fields used for faceting. Whenever I search for the term "harri" as q=title:harri it will match the term

Re: Phrase search + multi-word index time expanded synonym

see the analysis.jsp with debug verbose and see what happens at index time and search time during analysis with your data Also u can use debugQuery=on for seeing what actually parsed query is. - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/Phrase-search-multi

Re: Solr searching "harri" finds "harry"

U have not provided much detail about analysis of that field,but I am sure that problem because of stemming u can see by analysis page or by debugQuery=on parameter. To prevent stemming u have to put words in protword.txt on which u do not need any stemming - Grijesh -- View this message in

Re: Phrase search + multi-word index time expanded synonym

On 08/09/2010 12:21, Grijesh.singh wrote: see the analysis.jsp with debug verbose and see what happens at index time and search time during analysis with your data Also u can use debugQuery=on for seeing what actually parsed query is. - Grijesh I've found a first solution by myself, usi

Re: Creating a sub-index from another

you need a separate solr core for that and have to write a processor which process with your original index ,then generate the xml data and push to the new core.That is the simple way that i have used many times. - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.c

Re: Solr searching "harri" finds "harry"

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I have "harry" as a protected word in protword.txt Here is the xml definition for my text column On 08/09/10 1

RE: Advice requested. How to map 1:M or M:M relationships with support for facets

Thank you for your advice. Tim -Original Message- From: Lance Norskog [mailto:goks...@gmail.com] Sent: Tuesday, September 07, 2010 11:01 PM To: solr-user@lucene.apache.org Subject: Re: Advice requested. How to map 1:M or M:M relationships with support for facets These days the best pract

Re: Multi core schema file

solr.xml allows you to mention the other properties as well like instanceDir, config,schema in the cores/core tag So , sharing the entire conf dir may not be possible , but it is possible to share solrconfig.xml and schema.xml U can see the detail parameters at wiki page http://wiki.apache.org/

Re: Solr searching "harri" finds "harry"

have u restart the solr after adding words in protwords and reindex the data? - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-searching-harri-finds-harry-tp1438486p1438735.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr searching "harri" finds "harry"

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes to restart, no to re-index. Was hoping that wouldn't be necessary. I'll do that now. On 08/09/10 11:48, Grijesh.singh wrote: > > have u restart the solr after adding words in protwords and reindex the data? > > - > Grijesh -BEGIN PGP SIG

Re: Solr searching "harri" finds "harry"

yes reindexing is necessary for protwords,synanym update - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-searching-harri-finds-harry-tp1438486p1438802.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Batch update, order of evaluation

This would be surprising behavior, if you can reliably reproduce this it's worth a JIRA. But (and I'm stretching a bit here) are you sure you're committing at the end of the batch AND are you sure you're looking after the commit? Here's the scenario: Your updated document is a position 1 and 100 i

Re: list of filters/factories/Input handlers/blah blah

See the javadocs at: http://lucene.apache.org/solr/api/org/apache/solr/analysis/package-summary.html also see: http://wiki.apache.org/solr/LanguageAnalysis Both of these

Re: Alphanumeric wildcard search problem

Ah, thanks. That reconciles our differing results. Best Erick On Wed, Sep 8, 2010 at 2:58 AM, Hasnain wrote: > > The real problem was this tag > > > text > > and I was quering like this q=r-1* instead of q=mat_nr:r-1* > so whatever fieldType I use for mat_nr, it was using "text" fieldType whic

Re: Batch update, order of evaluation

Thanks, I'll create a deliberate test tomorrow feed some random data through it several times to see what happens. I'm also working on simply improving the buffer to handle the situation internally, but a few hours of testing isn't a big deal. Ta, Greg On 8 September 2010 21:41, Erick Erickson

Re: Query result ranking - Score independent

The change in the schema shouldn't matter (emphasis on the should). What version of SOLR are you using? I tried this query and it works just fine for me, I'm using 1.4.1 Best Erick On Wed, Sep 8, 2010 at 4:38 AM, Alessandro Benedetti < benedetti.ale...@gmail.com> wrote: > My request was very si

Re: Query result ranking - Score independent

Ooops, hit send too quickly. Could you show us the entire URL you send that produces the error? Erick On Wed, Sep 8, 2010 at 7:58 AM, Erick Erickson wrote: > The change in the schema shouldn't matter (emphasis on the should). > > What version of SOLR are you using? I tried this query and it work

Re: Solr, c/s type ?

I'll guess he means client/server. On Tue, Sep 7, 2010 at 5:52 PM, Chris Hostetter wrote: > > : Subject: Solr, c/s type ? > : > : i'm wondering c/s type is possible (not http web type). > : if possible, could i get the material about it? > > You're going t oneed to provide more info exaplining wh

RE: Solr, c/s type ?

> I'll guess he means client/server. HTTP is a client/server protocol, isn't it?

Re: Null Pointer Exception with shards&facets where some shards have no values for some facets.

On Tue, Sep 7, 2010 at 8:31 PM, Ron Mayer wrote: > Short summary: >  * Mixing Facets and Shards give me a NullPointerException >    when not all docs have all facets. https://issues.apache.org/jira/browse/SOLR-2110 I believe the underlying real issue stemmed from your use of a complex key "invol

How to import data with a different date format

Hi, I am attempting to import some of our data into SOLR. I did it the quickest way I know because I literally only have 2 days to import the data and do some queries for a proof-of-concept. So I have this data in XML format and I wrote a short XSLT script to convert it to the format in solr/e

Invariants on a specific fq value

Hi, I have an index with several collections. Every document has a collection field that specifies the collection it belongs to. To make querying easier (and restrict exposed parameters) i have a request handler for each collection. The request handlers are largely the same and preset all para

RE: How to import data with a different date format

No. The Datefield [1] will not accept it any other way. You could, however, fool your boss and dump your dates in an ordinary string field. But then you cannot use some of the nice date features.   [1]: http://lucene.apache.org/solr/api/org/apache/solr/schema/DateField.html   -Original mess

Re: How to import data with a different date format

That was my first thought :-) But it would be nice to be able to do date queries. I guess when I export the data I can just add 00:00:00Z. Thanks. - Original Message From: Markus Jelsma To: solr-user@lucene.apache.org Sent: Wed, September 8, 2010 11:34:32 AM Subject: RE: How to import

RE: Re: How to import data with a different date format

Your format (MM/DD/) is not compatible.   -Original message- From: Rico Lelina Sent: Wed 08-09-2010 19:03 To: solr-user@lucene.apache.org; Subject: Re: How to import data with a different date format That was my first thought :-) But it would be nice to be able to do date queries.

Re: How to import data with a different date format

I think Markus is spot-on given the fact that you have 2 days. Using a string field is quickest. However, if you absolutely MUST have functioning dates, there are three options I can think of: 1> can you make your XSLT transform the dates? Confession; I'm XSLT-ignorant 2> use DIH and DateTransform

Re: Invariants on a specific fq value

I just found out about 'invariants', and I found out about another thing too: "appends". (I don't think either of these are actually documented anywhere?). I think maybe "appends" rather than "invariants", with your fq you want always to be there might be exactly what you want? I actually

Re: Re: How to import data with a different date format

It will work. The original data is in XML format. I have an XSLT that transforms the data into the same format as that in exampledocs: ... - Original Message From: Markus Jelsma To: solr-user@lucene.apache.org Sent: Wed, September 8, 2010 12:06:39 PM Subject: RE: Re: How to impor

Re: How to import data with a different date format

I'm going with option 1, converting MM/DD/ to -MM-DD (which is fairly easy in XSLT) and then adding T00:00:00Z to it. Thanks. - Original Message From: Erick Erickson To: solr-user@lucene.apache.org Sent: Wed, September 8, 2010 12:09:55 PM Subject: Re: How to import data with

RE: Re: How to import data with a different date format

Ah, that answers Erick's question. And mine ;)   -Original message- From: Rico Lelina Sent: Wed 08-09-2010 19:25 To: solr-user@lucene.apache.org; Subject: Re: How to import data with a different date format I'm going with option 1, converting MM/DD/ to -MM-DD (which is fairly e

Solr Highlighting Question

Thanks for taking time to read through this. I'm using a checkout from the solr 3.x branch My problem is with the highlighter and wildcards I can get the highlighter to work with wild cards just fine, the problem is that solr is returning the term matched, when what I want it to do is highlig

Re: How to import data with a different date format

Just throwing it out there, I'd consider a different approach for an actual real app, although it might not be easier to get up quickly. (For quickly, yeah, I'd just store it as a string, more on that at bottom). If none of your dates have times, they're all just full days, I'm not sure you re

Re: How to import data with a different date format

I'm really thinking, once you convert to -MM-DD anyway, you might be better off just sticking this in a string field, rather than using a date field at all. The extra precision in the date field is going to make things confusing later, I predict. Especially for a quick and dirty prototype,

RE: Re: Invariants on a specific fq value

Interesting! I haven't met the appends method before and i'll be sure to give it a try tomorrow. Try, the wiki [1] is not very clear on what it really does.   More suggestions before tomorrow?   [1]: http://wiki.apache.org/solr/SolrSecurity#Path_Based_Authentication   -Original message

Re: Invariants on a specific fq value

Ah, I NEVER would have thought to look for these defaults/invariants/appends stuff under 'security', that's why I never found it! I can see now why it's sort of a security issue, but I, like you, use them just for convenience instead, and think of defaults, invariants, and appends as all in th

Re: How to import data with a different date format

how SOLR-savvy you are, so pardon if this is something you already know. But lots of people trip up over the "string" field type, which is NOT tokenized. You usually want "text" unless it's some sort of ID So it might be worth it to do some searching earlier rather than later Why wo

Re: Re: Invariants on a specific fq value

2010 at 1:32 PM, Markus Jelsma wrote: > Interesting! I haven't met the appends method before and i'll be sure to give > it a try tomorrow. Try, the wiki [1] is not very clear on what it really does. Here's a comment from the example solrconfig.xml: -Yonik http://lucenerevolution.org Luce

RE: Re: Re: Invariants on a specific fq value

Sounds great! I'll be very sure to put it to the test tomorrow and perhaps add documentation on these types to the solrconfigxml wiki page for reference.     -Original message- From: Yonik Seeley Sent: Wed 08-09-2010 19:38 To: solr-user@lucene.apache.org; Subject: Re: Re: Invariants o

Re: How to import data with a different date format

I'm doing something similar for dates/times/timestamps. I'm actually trying to do, "'now' is within the range of what appointments(date/time from and to combos, i.e. timestamps). Fairly simple search of: What items have a start time BEFORE now, and an end time AFTER now? My thoughts were to

Re: Invariants on a specific fq value

If there is no default or request-provided value, will the "appends" still be used? I suspect so, but let us know, perhaps by adding it to the wiki page! Markus Jelsma wrote: Sounds great! I'll be very sure to put it to the test tomorrow and perhaps add documentation on these types to the so

Re: How to import data with a different date format

That was a general comment on SOLR string types. Mostly I wanted to prompt Rico to try some searching before getting too hung up on indexing refinements. I'd far rather demo a prototype being able to say "Dates don't work yet, but you can search" than "searching is broken to pieces, but dates work

Re: How to import data with a different date format

So the standard 'int' field in Solr 1.4 is a "trie based" field, although the example "int" type in the default solrconfig.xml has a "precision" set to 0, which means it's not really doing "trie" things. If you set the precision to something greater than 0, as in the default example "tint" type

Re: How to import data with a different date format

So now, vs when 'trie' came out, Solr has an INT field that IS 'trie', right? And nothing date/timestamp related has come out since, making 'trie'/INT the field of choice for timestamps, right? Seems like the fastest choice. I will have to read up on it. Seems like my original choice to use u

RE: Re: Re: Invariants on a specific fq value

: Sounds great! I'll be very sure to put it to the test tomorrow and : perhaps add documentation on these types to the solrconfigxml wiki page : for reference. SolrConfigXml wouldn't really be an appropriate place to document this -- it's not a general config item, it's a feature of the Search

RE: Re: Re: Invariants on a specific fq value

Excellent! You already made my day for tomorrow! I'll check it's behavior with fq parameters specifying the a filter for the same field! -Original message- From: Chris Hostetter Sent: Wed 08-09-2010 21:04 To: solr-user@lucene.apache.org; Subject: RE: Re: Re: Invariants on a specific fq v

Re: How to import data with a different date format

: If none of your dates have times, they're all just full days, I'm not sure you : really need the date type at all. : : Convert the date to number-of-days since epoch integer. (Most languages will : have a way to do this, but I don't know about pure XSLT). Store _that_ in a : 1.4 'int' field.

Randomly slow response times for range queries

Hello all, I am running two range queries on a double value as filter queries using Solr 1.4, and for the most part am getting great performance (qTime < 100ms). However, at certain QPS, I start getting very slow queries (2000+ms). I've tried this using the new trie fields, and using standar sdou

Re: Is semicolon a character that needs escaping?

: I am using 1.3 without a sort param which explains it, I think. It would : be nice to update to 1.4 but we try to avoid such actions on a : production server as long as everything runs fine (the semicolon thing : was only reported recently). if you don't currenlty use "sort" at all, then adding

Re: How to use TermsComponent when I need a filter

: Subject: How to use TermsComponent when I need a filter : In-Reply-To: <8ffbbf6788bd5842b5a7274ef0f6837e01c3d...@msex85.morningstar.com> : References: <8ffbbf6788bd5842b5a7274ef0f6837e01c3d...@msex85.morningstar.com> http://people.apache.org/~hossman/#threadhijack Thread Hijacking on Mailing Li

How to use TermsComponent when I need a filter

Hi, I have a solr index, which for simplicity is just a list of names, and a list of associations. (either a multivalue field e.g. {A1, A2, A3, A6} or a string concatenation list e.g. "A1 A2 A3 A6") I want to be able to provide autocomplete but with a specific association. E.g. Names beginn

Re: Randomly slow response times for range queries

Well, throw enough queries at any server and it'll slow right down, so how many are we talking here? But no, there're no SOLR issues like this that I know of. That said, you could be getting cache thrashing. You could be getting garbage collection by the JVM. You could be executing commits somehow

RE: DataImportHandlerException for custom DIH Transformer

I am experiencing a similar situation? Any comments? -Original Message- From: Shashikant Kore [mailto:shashik...@gmail.com] Sent: Wednesday, September 08, 2010 2:54 AM To: solr-user@lucene.apache.org Subject: Re: DataImportHandlerException for custom DIH Transformer Resurrecting an old

Re: Implementing synonym NewBie

I believe the synonym filter does not find phrases, only individual words. It is possible that you could use the Shingle tools to create terms that are word pairs. This would be very inefficient. On Tue, Sep 7, 2010 at 6:23 AM, Jak Akdemir wrote: > If you think to improve your synonyms file by t

Re: How to retrieve the full corpus

If you want to do a mass scan of an index, the most scalable way is to make a variation of the Lucene CheckIndex program. Unfortunately, CheckIndex does not know any of the Solr types. But first, you should try the above techniques because they are much much easier. On Mon, Sep 6, 2010 at 7:59 AM

Re: How to import data with a different date format

Solr 1.4 was the first tagged release with trie fields. And Solr 1.4+ also includes a 'date' field based on 'trie' just for dates. If your dates are actually going to include hour/minute/second, not just calendar day-of-month, then I'd definitely use the built in solr trie date field, that's

Delta Import with something other than Date

Hi, I have a table that I want to index, and the table has no datetime stamp. However, the table is append only so the primary key can only go up. Is it possible to store the last primary key, and use some delta query="select id where id>${last_id_value}" Cheers, David

Re: Null Pointer Exception with shards&facets where some shards have no values for some facets.

Yonik Seeley wrote: > On Tue, Sep 7, 2010 at 8:31 PM, Ron Mayer wrote: >> Short summary: >> * Mixing Facets and Shards give me a NullPointerException >>when not all docs have all facets. > > https://issues.apache.org/jira/browse/SOLR-2110 > > I believe the underlying real issue stemmed from

Re: Delta Import with something other than Date

Of course you can store whatever you want in a solr index. And if you store an integer as a Solr 1.4 "int" type, you can certainly query for all documents that have greater than some specified integer in a field. You can't use SQL to query Solr though. I'm not sure what you're really asking?

RE: Delta Import with something other than Date

Currently DIH delta import uses the SQL query of type "select id from item where last_modified > ${dataimporter.last_index_time}" What I need is some field like ${dataimporter.last_primary_key} wiki.apache.org/solr/DataImportHandler I am thinking of storing the last primary key externally and calli

Need Advice for Finding Freelance Solr Expert

Hi, We need someone who knows Solr to help us prepare and index some data. Any advice on where to find people who know Solr? Thanks, John

Re: Randomly slow response times for range queries

Well I am only sending about 50 QPS at it at the time that it temporarily slows down, and then it's able to get all the way up to 100 QPS+ with no problems (until the next random queries). I suppose it could be the garbage collection. Is there a good way to limit this? -- View this message in con

Re: Randomly slow response times for range queries

Also, does anyone know the best precisionStep to use on a trie field (float) definition to achieve optimal performance? -- View this message in context: http://lucene.472066.n3.nabble.com/Randomly-slow-response-times-for-range-queries-tp1441724p1443096.html Sent from the Solr - User mailing list

Re: Need Advice for Finding Freelance Solr Expert

There's a page on the Solr/Lucene site for this. I myself will be in the market for one late this year. Dennis Gearon Signature Warning EARTH has a Right To Life, otherwise we all die. Read 'Hot, Flat, and Crowded' Laugh at http://www.yert.com/film.php --- On Wed, 9/8/10, Jo

Re: How to import data with a different date format

I already have the issue of how to store between different databases, languages, platforms, and frameworks. Settling on LONGINT/unix timestamp solves the problem on all fronts. I may even send them to the browser and have the JScript convert them to date/times (maybe ;-) So, it's *nix timestam

Re: Null Pointer Exception with shards&facets where some shards have no values for some facets.

I just checked in the last part of those changes that should eliminate any restriction on key. But, that last part dealt with escaping keys that contained whitespace or } Your example really should have worked after my previous 2 commits. Perhaps not all of the servers got successfully upgraded? Ca

Re: Null Pointer Exception with shards&facets where some shards have no values for some facets.

Yonik Seeley wrote: > I just checked in the last part of those changes that should eliminate > any restriction on key. > But, that last part dealt with escaping keys that contained whitespace or } > Your example really should have worked after my previous 2 commits. > Perhaps not all of the servers

Re: Creating a sub-index from another

: I have a Solr Index with several million documents. I need to implement some : text mining processes and I would like to create a million documents index : from the original for some tests. Which million documents do you want? If you're just looking for a one time kind of experimental test ind

Re: Solr Highlighting Question

Anybody? On 09/08/2010 11:26 AM, Jed Glazner wrote: Thanks for taking time to read through this. I'm using a checkout from the solr 3.x branch My problem is with the highlighter and wildcards I can get the highlighter to work with wild cards just fine, the problem is that solr is retur

[ANN] Webinar, Sep 15: Mastering the Power of Faceted Search

Folks, here's an upcoming Solr webinar sponsored by my employer. It's Hoss on faceting, so it should be good! -Yonik http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8 --- Webinar Details Join us for a free webc

Re: Multi core schema file

A demonstration of this feature would be a good addition to the example/multicore directory. On Wed, Sep 8, 2010 at 3:45 AM, Grijesh.singh wrote: > > solr.xml allows you to mention the other properties as well like > instanceDir, config,schema in the cores/core tag > > So , sharing the entire con

Re: Query result ranking - Score independent

Generally speaking it is a bad idea to change the schema without reindexing. I found several little things that could go wrong back when I had a huge index and could not reindex. On Wed, Sep 8, 2010 at 4:58 AM, Erick Erickson wrote: > Ooops, hit send too quickly. Could you show us the entire URL

Re: Solr Highlighting Question

(10/09/09 2:26), Jed Glazner wrote: Thanks for taking time to read through this. I'm using a checkout from the solr 3.x branch My problem is with the highlighter and wildcards I can get the highlighter to work with wild cards just fine, the problem is that solr is returning the term matched

Re: Randomly slow response times for range queries

Here's a good place to start with garbage collection: http://www.lucidimagination.com/blog/2009/09/19/java-garbage-collection-boot-camp-draft/ About precision step. Optimal means different things for different indexes. You have to provide a lot more detail about your problem space before anyone ca

Re: Solr, c/s type ?

I'd just like to use solr for in-house which is not web application. But I don't know how should i do? Thanks, -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-c-s-type-tp1392952p1444175.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Delta Import with something other than Date

https://issues.apache.org/jira/browse/SOLR-1499 This is a patch (not committed) that queries a Solr instance and returns the values as a DIH document. This allows you to do a sort query to Solr, ask for the first result, and continue indexing after that. Scary, but it works. Lance David Yan

Re: Solr, c/s type ?

You would set up a Java server (container) and run Solr/Lucene. Not sure how to do the following, but then you block the standard port for Solr/Lucene on that machine from being accessible except locally. In whatever code/applicaiton that you are working with, on that machine, you then use it'

RE: Solr, c/s type ?

You _could_ use SolrJ with EmbeddedSolrServer. But personally I wouldn't unless there's a reason to. There's no automatic reason not to use the ordinary Solr HTTP api, even for an in-house application which is not a web application. Unless you have a real reason to use embedded solr, I'd use

Re: Distance sorting with spatial filtering

It says that the field "sum(1)" is not indexed. You don't have a field called 'sum(1)'. I know there has been a lot of changes in query parsing, and sorting by functions may be on the list. But the _val_ trick is the older one and, and you noted, still works. The _val_ trick sets the ranking va

Re: Delta Import with something other than Date

On 09.09.2010, at 00:44, David Yang wrote: > Currently DIH delta import uses the SQL query of type "select id from > item where last_modified > ${dataimporter.last_index_time}" > What I need is some field like ${dataimporter.last_primary_key} > wiki.apache.org/solr/DataImportHandler > I am thinki

Re: Distance sorting with spatial filtering

According to the documentation, sorting by function has been a feature since Solr 1.5. It seems like a major regression if this no longer works. http://wiki.apache.org/solr/FunctionQuery#Sort_By_Function The _val_ trick does not seem to work if used with a query term, although I can try some more