highlighting not working with Solr 3.0 trunk?

2011-01-07 Thread Teruhiko Kurosaka
I've downloaded http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x and ran ant there. I've followed the tutorial but highlighting on analyzer debug screen isn't working. This link found in the tutorial doesn't show any highlight. http://localhost:8983/solr/admin/analysis.jsp?name=nam

Internal Server Error when indexing a pdf file

2011-01-07 Thread Alessandro Marino
Hi, I was trying to use Solr Cell (through the Java API) to index a pdf file. The class has been extracted from http://wiki.apache.org/solr/ContentStreamUpdateRequestExample public class Solr { public static void main(String[] args) { try { String solrId = "beautiful_stm.pdf"; in

Solr indexing socket timeout errors

2011-01-07 Thread Burton-West, Tom
Hello all, We are getting intermittent socket timeout errors (see below). Out of about 600,000 indexing requests, 30 returned these socket timeout errors. We haven't been able to correlate these with large merges, which tends to slow down the indexing response rate. Does anyone know where we

Indexing Issue between Mac OS X 10.5 and 10.6

2011-01-07 Thread Kevin Murdoff
Greetings Everyone - I am hoping someone can help me with this unusual issue I have here. Issue Indexing information in a database (i.e. /dataimport [full-import]) succeeds when I perform this function on a Mac OS X 10.6 with Java 1.6, but fails when I attempt the same indexing task on a 10.5

Re: DIH - Closing ResultSet in JdbcDataSource

2011-01-07 Thread Adam Estrada
This is my configuration which seems to work just fine. >From there it's just a matter of running the select statement and mapping it against the correct fields in your index. Adam On Fri, Jan 7, 2011 at 2:40 PM, Shane Perry wrote: > Hi, > > I am in the process of migrating our system

Re: solrconfig luceneMatchVersion 2.9.3

2011-01-07 Thread Chris Hostetter
: there is no more trunk support for 2.9 indexes. : : So I tried the suggested solution to execute an optimize to convert a 2.9.3 : index to a 3.x index. : : However, when I tried to the optimize a 2.9.3 index using the Solr 4.0 trunk : version with luceneMatchVersion set to LUCENE_30 in the sol

Re: How do I troubleshoot Schema / Document mismatches?

2011-01-07 Thread Ahmet Arslan
> When I use the post.jar tool, I don't get any meaningful > errors if there's > some kind of mismatch between the schema and the XML > 's I'm loading. > > All I get is: > FATAL: Solr returned an error: Internal Server Error > > There's no information about what fields were missing, > additional

Re: schema.xml in other than conf folder

2011-01-07 Thread Chris Hostetter
: Thanks for your response. Our production environment is a read only file : system. It is not allowing to modify or create new files under conf folder : at runtime. So copy config through replication is not working for us. if your entire production server is read only, where do you keep your

How do I troubleshoot Schema / Document mismatches?

2011-01-07 Thread danieltalsky
When I use the post.jar tool, I don't get any meaningful errors if there's some kind of mismatch between the schema and the XML 's I'm loading. All I get is: FATAL: Solr returned an error: Internal Server Error There's no information about what fields were missing, additional fields, wrong data,

DIH - Closing ResultSet in JdbcDataSource

2011-01-07 Thread Shane Perry
Hi, I am in the process of migrating our system from Postgres 8.4 to Solr 1.4.1. Our system is fairly complex and as a result, I have had to define 19 base entities in the data-config.xml definition file. Each of these entities executes 5 queries. When doing a full-import, as each entity comple

RE: Custom match scoring

2011-01-07 Thread Nelson Branco
Ok, I have look at it and it almost solves my problem... My rules list demand to counting only once each token, not each field... Any idea on how it can be done? Currently I'm using pure logic to accomplish this, more or less like "FieldA:token1 OR (FieldB:token1 AND -FieldA:token1) OR (FieldC:t

RE: Custom match scoring

2011-01-07 Thread Nelson Branco
Hum, if so, it may resolve the problem. I didn't know that. I´ll take a look. Thanks. -- Nelson Branco SAPO Mapas/GIS -Original Message- From: Chris Hostetter [mailto:hossman_luc...@fucit.org] Sent: quarta-feira, 5 de Janeiro de 2011 02:12 To: solr-user@lucene.apache.org Subject: RE: C

Re: solrconfig luceneMatchVersion 2.9.3

2011-01-07 Thread Johannes Goll
according to http://www.mail-archive.com/solr-user@lucene.apache.org/msg40491.html there is no more trunk support for 2.9 indexes. So I tried the suggested solution to execute an optimize to convert a 2.9.3 index to a 3.x index. However, when I tried to the optimize a 2.9.3 index using the Solr

Re: Improving Solr performance

2011-01-07 Thread mike anderson
Making sure the index can fit in memory (you don't have to allocate that much to Solr, just make sure it's available to the OS so it can cache it -- otherwise you are paging the hard drive, which is why you are probably IO bound) has been the key to our performance. We recently opted to use less RA

Re: Improving Solr performance

2011-01-07 Thread Toke Eskildsen
On Fri, 2011-01-07 at 10:57 +0100, supersoft wrote: [5 shards, 100GB, ~20M documents] ... [Low performance for concurrent searches] > Using JConsole for monitoring the server java proccess I checked that Heap > Memory and the CPU Usages don't reach the upper limits so the server > shouldn't per

Lucene Scorer Extension?

2011-01-07 Thread dante stroe
Hello, What I am trying to do is build a personalized search engine. The aim is to have the resulting documents' scores depend on users' preferences. I've already built some Solr plugins (request handlers mainly), however I am not sure that what I am trying to do can be achieved by a plugin.

Re: Optimize a Index

2011-01-07 Thread Erick Erickson
Hmmm, certainly try optimize, but often the problem is in how you query. Providing some examples of slow queries and the time it takes to get them would help. Also, running with &debugQuery=on will show you a QTime field in the response header that is the number of milliseconds the actual query to

Re: Improving Solr performance

2011-01-07 Thread François Schiettecatte
It sounds like your system is I/O bound and I suspect (bet even) that all your index files are on the same disk drive. Also you have only 8GB of RAM for 100GB of index, so while your SOLR instance will cache some stuff and the balance will be used for caching file blocks, there really isn't enou

DIH Transformer

2011-01-07 Thread Bernd Fehling
Hi list, currently the Transformers return row but can I skip or drop a row from the Transformer? If so, what should I return in that case, an empty row? Regards, Bernd

Re: Improving Solr performance

2011-01-07 Thread supersoft
The reason of this distribution is the kind of the documents. In spite of having the same schema structure (and solr conf), a document belongs to 1 of 5 different kinds. Each kind corresponds to a concrete shard and due to this, the implemented client tool avoids searching in all the shards when

RE: Improving Solr performance

2011-01-07 Thread Grijesh.singh
open a new mail conversation for that - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/Improving-Solr-performance-tp2210843p2211300.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Improving Solr performance

2011-01-07 Thread Hong-Thai Nguyen
Hi, Always interesting question! Anyone could purpose a generic (and approximate) equation: Search_time = F(Nb_of_servers, RAM_size_per_server, CPU_of_servers, Nb_of_shards, Nb_of_documents, Total_size_of_documents or Average_size_of_a_document, Nb_requests_in_minute, Nb_indexed_fields_in_inde

Re: Improving Solr performance

2011-01-07 Thread Grijesh.singh
shards are used when index size become huge and performance going down . shards mean distributed indexes. But if you will put all shards on same machine as multicore then it will not help too much on performance. and also shards distributes indexes near equals in size. There is also not enough Ra

Re: Improving Solr performance

2011-01-07 Thread Grijesh.singh
shards are used when index size become huge and performance going down . shards mean distributed indexes. But if you will put all shards on same machine as multicore then it will not help too much on performance. and also shards distributes indexes near equals in size. There is also not enough Ra

Re: Improving Solr performance

2011-01-07 Thread supersoft
1 - Yes, all the shards are in the same machine 2 - The machine RAM is 7.8GB and I assign 3.4GB to Solr server 3 - The shards sizes (GB) are 17, 5, 3, 11, 64 -- View this message in context: http://lucene.472066.n3.nabble.com/Improving-Solr-performance-tp2210843p2211135.html Sent from the Solr -

Re: Improving Solr performance

2011-01-07 Thread Grijesh.singh
Some questions- 1-Are all shards on same machine 2-What is your Ram Size 3-What are the size of index on each shards in GB - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/Improving-Solr-performance-tp2210843p2210878.html Sent from the Solr - User mailing list

Saravanan Chinnadurai/Actionimages is out of the office.

2011-01-07 Thread Saravanan . Chinnadurai
I will be out of the office starting 07/01/2011 and will not return until 17/01/2011. Please email to itsta...@actionimages.com for any urgent issues. Action Images is a division of Reuters Limited and your data will therefore be protected in accordance with the Reuters Group Privacy / Data P

Improving Solr performance

2011-01-07 Thread supersoft
have deployed a 5-sharded infrastructure where: shard1 has 3124422 docs shard2 has 920414 docs shard3 has 602772 docs shard4 has 2083492 docs shard5 has 11915639 docs Indexes total size: 100GB The OS is Linux x86_64 (Fedora release 8) with vMem equal to 7872420 and I run the server using Jetty (f

Improving Solr performance

2011-01-07 Thread supersoft
have deployed a 5-sharded infrastructure where: shard1 has 3124422 docs shard2 has 920414 docs shard3 has 602772 docs shard4 has 2083492 docs shard5 has 11915639 docs Indexes total size: 100GB The OS is Linux x86_64 (Fedora release 8) with vMem equal to 7872420 and I run the server using Jetty (f

Re: forward slash not working in my solr search

2011-01-07 Thread dhanesh
On 1/6/2011 2:45 PM, Grijesh.singh wrote: use as pharase it will work like "Computer / IT" for you Here IT is a stopword when you are trying query as category:Computer / IT parsed as category:Computer:IT but IT is a stopword for default search field what you have selected so second query remo

Re: Input raw log file

2011-01-07 Thread Grijesh.singh
There is a csv update handler in solr you can use it by modifying your logfile - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/Input-raw-log-file-tp2210043p2210673.html Sent from the Solr - User mailing list archive at Nabble.com.

Optimize a Index

2011-01-07 Thread Jörg Agatz
Hallo, i have a Index withe 800.000 Dokuments, and now i hope it will be Faster, if i optimize the Index, it sounds good ;-) But i cant find an Example to Optimize one of milticors or all cors.. Maby one of you have a little example for that .. King