Group.query

2012-09-25 Thread Peter Kirk
Hi I have "products" which belong to one or more "groups". Products are documents in Solr, while the groups are fields (eg. group_1_bool:true). For example: Prod1 => group1, group2 Prod2 => group1, group2 Prod3 => group1 Prod4 => group2 I would like to execute a query which results in the grou

Access Error after changing Solr Code

2012-09-25 Thread deniz
Hello, I had changed Solr Beta 4.0 source for learning and testing. I have added a funcitonality for this ticket which was opened after my post on this forum: http://issues.apache.org/jira/browse/SOLR-3862 Basically, now I am able to delete an entry from a multivalued queue, by using curl http r

Re: Indexing in Solr: invalid UTF-8

2012-09-25 Thread Robert Muir
On Tue, Sep 25, 2012 at 2:02 PM, Patrick Oliver Glauner wrote: > Hi > Thanks. But I see that 0xd835 is missing in this list (see my exceptions). > > What's the best way to get rid of all of them in Python? I am new to unicode > in Python but I am sure that this use case is quite frequent. > I do

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread 韦震宇
Hi, Tanguy I would do as your suggestion. Best Regards! Monton - Original Message - From: "Tanguy Moal" To: ; Sent: Tuesday, September 25, 2012 11:05 PM Subject: Re: How can I create about 10 independent indexes in Solr? That is an interesting issue... I was wondering if relyi

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread 韦震宇
Hi, Eskildsen The customer is company, so the active user is tenfold more than the customers. I'm choicing the "a single index" and splitting index into multiple shards. Best Regards! Monton - Original Message - From: "Toke Eskildsen" To: Sent: Tuesday, September 25, 2012 1

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread 韦震宇
Hi, Bordis Thank you very much for your precious experience! I would choice the "single index" solution. I'm sorry my name is not pronounced good in english ,you would call me monton. Best Regards! Monton - Original Message - From: "Christian Bordis" To: Sent: Tuesday, Sept

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Otis Gospodnetic
Hi, Have a look at ElasticSearch. While it's probably no better at handling 100K cores than Solr it, ES has document and query routing functionality that can help you route documents and queries to specific shards, thus avoiding needing to query all 100K users' data. I think Solr doesn't have thi

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread 韦震宇
Hi,Tom I'm prefer performance, so if the performance of "single index" solution is better than "mutilcore" solution, I would choice the former. Best Regards! - Original Message - From: "Tom Mortimer" To: Sent: Tuesday, September 25, 2012 6:06 PM Subject: Re: How can I create about

Re: Untokenized lowercase string

2012-09-25 Thread am
Just wanted to confirm that this: ...works beautifully for untokenized lowercase values. Starting spaces and spaces in the middle work fine. -- View this message in context: http://lucene.472066.n3.nabble.com/Untokenized-lowercase-string-tp4010296p4010351.html Sent from the

Re: How to boost date field while boosting a text field?

2012-09-25 Thread Erick Erickson
Did you just try ?q={!boost b=recip(ms(NOW,date),3.16e-11,1,1)}ipad^2&fl=date Works just fine. Appending &debugQuery=on will show you why the doc ranked as it did, although the output can be cryptic... Best Erick On Tue, Sep 25, 2012 at 11:56 AM, srinalluri wrote: > Thanks Hoss for your reply

Re: Re: Re: Solr Cell Questions

2012-09-25 Thread Erick Erickson
OK, I was thinking more along the lines of this blog: http://searchhub.org/dev/2012/02/14/indexing-with-solrj/ which uses Tika directly to process the docs on the client (wherever you run it) and only sends the results to Solr The SolrJ program you're referencing uses a different approach...

Re: Making a case to migrate from GSA to Solr

2012-09-25 Thread Otis Gospodnetic
Hi Matthew, Solr can do everything you mentioned. Re 3) This won't show anywhere in the UI, but you will get errors in the indexer during indexing. Re 5) You may want to search search-lucene.com for info about boolean queries, which are not true boolean, in case true boolean is really what you n

Re: Untokenized lowercase string

2012-09-25 Thread am
That sounds right, thanks! I missed KeywordTokenizerFactory, with a name like that it did not sound like what I wanted. I expected NullTokenizerFactory or something standing out like that :) Jack Krupansky-2 wrote > Use the KeywordTokenizerFactory for your text field tokenizer to keep the > text

Re: Untokenized lowercase string

2012-09-25 Thread am
Alexandre Rafalovitch wrote > Each field has a type. Each type defines what happens with the text. > You can certainly select to do one thing but not another. Understood. But it seemed to me that only TextField allows adding filters to it and filters go in conjunction with tokenizers. I could not

Wouldn't be nice: free public example solr server

2012-09-25 Thread Alexandre Rafalovitch
I was just thinking that there is now a couple of SOLR hosting providers (e.g. https://www.dotcloud.com/, but I think there were several). Wouldn't be nice if one of them had a fully setup SOLR4 instance with the 'example' configuration loaded. Then, anybody could go experiment with it even before

Re: Untokenized lowercase string

2012-09-25 Thread Jack Krupansky
Use the KeywordTokenizerFactory for your text field tokenizer to keep the text from being tokenized, and then use the LowerCaseFilterFactory token filter to do the lowercasing. Unfortunately, "string" (StrField) does not support analysis. -- Jack Krupansky -Original Message- From: am

Re: Untokenized lowercase string

2012-09-25 Thread Alexandre Rafalovitch
Each field has a type. Each type defines what happens with the text. You can certainly select to do one thing but not another. Just look towards the bottom of the schema.xml and compare field types definition for string and text, it should be fairly obvious. You'll most probably make up a new type

Untokenized lowercase string

2012-09-25 Thread am
I am new to Solr. Just wetting my feet, trying to set it up and to migrate our in-house search to it. Is it possible to define a field type that is not tokenized, but has a lowercase filtering? I'm sure I can do it in java code, but I am looking for an XML file solution. Basically "Foo Bar" and "f

Re: DIH - 'url' element in 'entity' does not work with url parameters

2012-09-25 Thread Jack Krupansky
In XML, "&" needs to be written as "&". In this case "amp" is the "entity name". -- Jack Krupansky -Original Message- From: Billy Newman Sent: Tuesday, September 25, 2012 5:28 PM To: solr-user@lucene.apache.org Subject: DIH - 'url' element in 'entity' does not work with url parameters

DIH - 'url' element in 'entity' does not work with url parameters

2012-09-25 Thread Billy Newman
Hello All. I am trying to configure the DIH to import XML data from a URL. The url attribute contains parameters in he link and I cannot get it to work. http://test.org/data?version=1.0&type=test"; processor="XPathEntityProcessor"

Re: Making a case to migrate from GSA to Solr

2012-09-25 Thread vikash . iitb
Solr can handle all of your pain points. You can sort with any indexed field. It returns correct count. Faceting is trivial. OR conditions are totally fine, it can handle really complex conditional statements. Sent from my iPad On 26-Sep-2012, at 12:48 AM, Matthew Shapiro wrote: > Hi all, I

Making a case to migrate from GSA to Solr

2012-09-25 Thread Matthew Shapiro
Hi all, I don't know if this is the correct mailing list, so I apologize if it isn't. I wasn't sure what other list it would go to. Anyways, my company a while back (before I started) got Google envy and decided to purchase a GSA system to store our searchable data. While the GSA seems ok for a

Re: AutoIndexing

2012-09-25 Thread Amit Nithian
There's a couple ways to accomplish this from easy to hard depending on your database schema: 1) Use DB trigger -> I don't like triggers too much b/c to me they couple your database layer with your application layer which leads to untestable and sometimes unmaintainable code -> Also it gets dif

RE: Indexing in Solr: invalid UTF-8

2012-09-25 Thread Patrick Oliver Glauner
Hi Thanks. But I see that 0xd835 is missing in this list (see my exceptions). What's the best way to get rid of all of them in Python? I am new to unicode in Python but I am sure that this use case is quite frequent. Patrick From: Markus Jelsma [markus.j

RE: Indexing in Solr: invalid UTF-8

2012-09-25 Thread Markus Jelsma
Hi - you need to get rid of all non-character code points. http://unicode.org/cldr/utility/list-unicodeset.jsp?a=[:Noncharacter_Code_Point=True:] -Original message- > From:Patrick Oliver Glauner > Sent: Tue 25-Sep-2012 18:47 > To: solr-user@lucene.apache.org > Subject: Indexing in Solr

Indexing in Solr: invalid UTF-8

2012-09-25 Thread Patrick Oliver Glauner
Hello We use Solr 3.1 and Jetty to index previously extracted fulltexts from PDFs, DOC etc. Our indexing script is written in Python 2.4 using solrpy: [...] text = remove_control_characters(text) # except \r, \t, \n utext = unicode(text, 'utf-8') SOLR_CONNECTION.add(id=recid, fulltext=utext) [..

Re: How to more gracefully handle field format exceptions?

2012-09-25 Thread Chris Hostetter
: In my thinking (and for this situation), it would be much better to : just ignore the malformed field and keep the doc - is there any way to : configure this or enable this behavior instead? 4.0-ALPHA added some new UpdateProcessorFactories that can be configured in your solrconfig.xml when yo

Re: How to boost date field while boosting a text field?

2012-09-25 Thread srinalluri
Thanks Hoss for your reply. I already seen that Solr wiki, but I am unable to merge that solution with my query. The example in the wiki gave to search for one string. I tried this, but the date is not sorted order, why? ?q={!boost b=recip(ms(NOW,date),3.16e-11,1,1)}ipad&fl=date How to merge '

Re: complex Solr Sorting Expression in 4.0 alpha

2012-09-25 Thread lavesh
How to specify a range using if query ? please help me in re-framing this using if query sort=map(sum(SCORE,11),77,77,sum(SCORE,33),sum(SCORE,44)) desc -- View this message in context: http://lucene.472066.n3.nabble.com/complex-Solr-Sorting-Expression-in-4-0-alpha-tp4010160p4010188.html Sent

Re: complex Solr Sorting Expression in 4.0 alpha

2012-09-25 Thread Jack Krupansky
Sorry, but as the wiki for the "map" function states, "min,max,target[, and optional default value] are constants", not full function value expressions. Good improvement to request though. See: http://wiki.apache.org/solr/FunctionQuery#map You probably need to resort to "if" functions. -- Jac

Re: /solr/dataimport not found

2012-09-25 Thread Michael Della Bitta
On Tue, Sep 25, 2012 at 4:18 AM, johnohod wrote: > *Michael Della Bitta:* > /solr/admin/dataimport.jsp shows a gray page with the text: "Select > handler". Are there any links to click on on that page? If not, it means the handler isn't configured correctly or the DIH JARs are not available. Mic

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Tanguy Moal
That is an interesting issue... I was wondering if relying on dynamic fields could be an option... Something like : * : * customer : string * *_field_a1 : type_a * *_field_a2 : type_a * *_field_b1 : type_b * ... And the prefix each field by the customer name, so for customer1, indexed documents

Re: mod function not working - Solr 4 beta

2012-09-25 Thread Jack Krupansky
Final release date? Uhhh... they're working on it! There is already one RC, but another is expected. Next week maybe? But that's just my personal guess. -- Jack Krupansky -Original Message- From: reeuv Sent: Tuesday, September 25, 2012 10:56 AM To: solr-user@lucene.apache.org Subject:

Re: mod function not working - Solr 4 beta

2012-09-25 Thread Jack Krupansky
It was added to 4.0 after the beta release. See: http://svn.apache.org/viewvc?view=revision&revision=1379233 Author:hossman Date:Thu Aug 30 23:49:05 2012 UTC (3 weeks, 4 days ago) Log Message:SOLR-2768: new "mod(x,y)" function for computing the modulus of two value sources (merge r1

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Toke Eskildsen
On Tue, 2012-09-25 at 04:21 +0200, 韦震宇 wrote: > The company I'm working in have a website to server more than 10 > customers, and every customer should have it's own search cataegory. > So I should create independent index for every customer. How many of the customers are active at any given t

Re: Solr Cell Questions

2012-09-25 Thread Jack Krupansky
Maybe we should even contemplate direct support for Tika/SolrCell in SolrJ - call it SolrJCell. This might also make it a lot easier for apps to apply post-processing after document parser but before data is sent to Solr. And maybe even have an option for multi-process support (invoke Tika as a

Re: Solr Cell Questions

2012-09-25 Thread Alexandre Rafalovitch
Are you by any chance committing after every file being indexed? That could cause the speed issues. Also, have you tried to optimize your indexer's java memory params. I use this for mine which used to run out of memory as well: java -server -Xms512m -Xmx2048m Regards, Alex. P.s. I may have so

Antwort: Re: Re: Solr Cell Questions

2012-09-25 Thread Johannes . Schwendinger
The difference with solr cell is, that i'am sending every single document to solr cell and don't collect them until i have a couple of them in my memory. Using mainly the code form here: http://wiki.apache.org/solr/ExtractingRequestHandler#SolrJ Erick Erickson schrieb am 25.09.2012 15:47:34:

complex Solr Sorting Expression in 4.0 alpha

2012-09-25 Thread lavesh
I want to perform complex sorting expression for which i have upgraded to solr-aplha 4.0 apache-solr-4.0.0-BETA/select?q=:&sort=map(sum(SCORE,11),77,77,sum(SCORE,33),sum(SCORE,44)) desc but when i try to run this , its not working. can anyone help me for such complex queries of solr -- View t

Re: Re: Solr Cell Questions

2012-09-25 Thread Erick Erickson
bq: how many documents per minute, second, what ever can i put into solr Too many variables to say. I've seen several thousand truly simple docs/sec. But since you're doing the Tika processing that's probably going to be your limiting factor. And it'll be many fewer... I don't understand your OOM

Re: /solr/dataimport not found

2012-09-25 Thread Mukund Chouhan
Hi John, I have created one POC with MySql database which is working fine. I have attached conf folder zip file. please create dummy database and try this. Thanks and Regards, Mukund Chouhan On Tue, Sep 25, 2012 at 1:48 PM, johnohod [via Lucene] < ml-node+s472066n4010087...@n3.nabble.com> wr

Re: Copyfield query

2012-09-25 Thread Rafał Kuć
Hello! As you can is in the http://wiki.apache.org/solr/SchemaXml#Copy_Fields the actual copying is done before analysis and indexing, so it doesn't matter if you store fields you use as source for your copy fields. -- Regards, Rafał Kuć Sematext :: http://sematext.com/ :: Solr - Lucene

AW: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Christian Bordis
Hi weizhenyu! (can you write down your name in roman latters) We also tried to provide dedicated cores for each customer in one of our early system design to get maximum config possibilities. 40k Webshops => 40k Solr Cores, which didn't work out. Problems we encountered: - Threads: Solr creat

Re: Join in Solr 3.6

2012-09-25 Thread Rafał Kuć
Hello! If you talk about this: https://issues.apache.org/jira/browse/SOLR-2272 than it is only available since 4.0-alpha. -- Regards, Rafał Kuć Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > Hi. I have read there is "join" functionality in Solr 4 beta. > Is there

Join in Solr 3.6

2012-09-25 Thread Peter Kirk
Hi. I have read there is "join" functionality in Solr 4 beta. Is there a "join" in Solr 3.6? Thanks, Peter

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Toke Eskildsen
On Tue, 2012-09-25 at 12:06 +0200, Tom Mortimer wrote: > Why do you think that the indexes should be independent? What would be > the problem with using a single index and filter queries? Ranking will be done with regard to the full corpus instead of just the single customer. Whether that is an is

Re: AW: AW: auto completion search with solr using NGrams in SOLR

2012-09-25 Thread aniljayanti
Hi, Finally got Solved using TermComponents. schema.xml -- URL : http://localhost:8080/solr/terms?*terms.fl=autocomplete_text&omitHeader=true&terms.sort=index&terms.prefix=mad* Response : *1 1

Re: DIH problem

2012-09-25 Thread Gian Marco Tagliani
Ok, I'll try to verify if there is the same issue in the 4.0 and I'll open the issue in Jira. thanks -- Gian Marco On Sat, Sep 22, 2012 at 9:34 PM, Dyer, James wrote: > Gian, > > Even if you can't write a failing unit test (if you did it would be > awesome), please open a JIRA issue on this a

Re: AutoIndexing

2012-09-25 Thread Tom Mortimer
I'm afraid I don't have any DIH experience myself, but some googling suggests that using a postgresql trigger to start a delta import might be one approach: http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command and http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFull

RE: AutoIndexing

2012-09-25 Thread darshan
My Document is Database(yes RDBMS) and software for it is postgresql, where any change in it's table should be reflected, without re-indexing. I am indexing it via DIH process Thanks, Darshan -Original Message- From: Tom Mortimer [mailto:tom.m.f...@gmail.com] Sent: Tuesday, September 25,

Re: How can I create about 100000 independent indexes in Solr?

2012-09-25 Thread Tom Mortimer
Hi, Why do you think that the indexes should be independent? What would be the problem with using a single index and filter queries? Tom On 25 Sep 2012, at 03:21, 韦震宇 wrote: > Dear all, >The company I'm working in have a website to server more than 10 > customers, and every customer

Re: AutoIndexing

2012-09-25 Thread Tom Mortimer
Hi Darshan, Can you give us some more details, e.g. what do you mean by "database"? A RDBMS? Which software? How are you indexing it (or intending to index it) to Solr? etc... cheers, Tom On 25 Sep 2012, at 09:55, darshan wrote: > Hi All, > >Is there any way where I can aut

Re: UIMA for lemmatization

2012-09-25 Thread Tommaso Teofili
Hi, I think you'd better ask this on u...@uima.apache.org list as this is more related to Apache UIMA itself rather than to Apache Solr. Regards, Tommaso 2012/9/25 abhayd > hi > I m new to UIMA. Solr doea not have lemmatization component, i was > thinking > of using UIMA for this. > > Is t

Listening to index updates in SolrJ?

2012-09-25 Thread Mathias Hodler
Hi, I'm using Embedded Solr 4.0 with SolrJ. In solrconfig.xml you can specify a RunExecutableListener. Is there something similar in SolrJ, so I can get an event, if the index gets updated? This can be very useful if using SolrCloud, to get an event if other shards updating the index. Thanks.

Antwort: Re: Solr Cell Questions

2012-09-25 Thread Johannes . Schwendinger
Thank you Erick for your respone, I've already tried what you've suggested and got some out of memory exceptions. Because of this i like the solution with solr Cell where i can send the file directly to solr via stream and don't collect them in my memory. And another question that came to my

AutoIndexing

2012-09-25 Thread darshan
Hi All, Is there any way where I can auto-index whenever there is changes in my database. Thanks, Darshan

Re: /solr/dataimport not found

2012-09-25 Thread johnohod
*Michael Della Bitta:* /solr/admin/dataimport.jsp shows a gray page with the text: "Select handler". *Chris Hostetter:* 1) 3.6.1 2) Yes. I've tried every alternative way of writing the url that I can think of. 3) I think so. I can see the /solr/admin-page and click around without problems. I haven

Re: Admin-UI: multiple facet

2012-09-25 Thread Stefan Matheis
Hey Alex Right now it's not possible .. but we already have an Issue for that: https://issues.apache.org/jira/browse/SOLR-3838 There is no real reason for that limitation .. it's just not done, yet :) Stefan On Tuesday, September 25, 2012 at 1:54 AM, Alexandre Rafalovitch wrote: > Hello, >

Re: SolrJ - IOException

2012-09-25 Thread Toke Eskildsen
On Tue, 2012-09-25 at 01:50 +0200, balaji.gandhi wrote: > I am encountering this error randomly (under load) when posting to Solr > using SolrJ. > > Has anyone encountered a similar error? > > org.apache.solr.client.solrj.SolrServerException: IOException occured when > talking to server at: http: