Re: Can I rebuild an index and remove some fields?

for method 2, delete is wrong. we can't delete terms. you also should hack with the tii and tis file. On Tue, Feb 14, 2012 at 2:46 PM, Li Li wrote: > method1, dumping data > for stored fields, you can traverse the whole index and save it to > somewhere else. > for indexed but not stored field

Re: Can I rebuild an index and remove some fields?

method1, dumping data for stored fields, you can traverse the whole index and save it to somewhere else. for indexed but not stored fields, it may be more difficult. if the indexed and not stored field is not analyzed(fields such as id), it's easy to get from FieldCache.StringIndex. But for

Re: Re:how to monitor solr in newrelic

How to run the java agent.Can you please guide the steps.Because i have tried a lot but no luck -- View this message in context: http://lucene.472066.n3.nabble.com/how-to-monitor-solr-in-newrelic-tp3739567p3742858.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: What versions support compressed text fields?

Yes, that (TextField, StrField) looks like ancient Lucene stuff from the previous decade :) Otis  Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html - Original Message - > From: Walter Underwood > To: solr-user@lucene.apache.org

Re: What versions support compressed text fields?

So this statement is false? "Field types that store text (TextField, StrField) support compression of stored contents" http://wiki.apache.org/solr/SchemaXml And the attributes compressed and compressThreshold are no longer supported? wunder On Feb 13, 2012, at 8:15 PM, Otis Gospodnetic wrote:

Re: What versions support compressed text fields?

Hi Wunder, Unless I missed something, you simply need to do compressing/uncompressing on your own - old Solr/Lucene versions would do that for you, but that's ancient history. Otis  Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html --

Poll: how to report # of docs in index over time

Hello, Quick poll for those who have an opinion about what index size monitoring should report in terms of the number of documents in the index. Poll: http://blog.sematext.com/2012/02/13/poll-solr-index-size-monitoring/ For example, imagine that in some 5-minute time period (say 10:00 AM to 10:

Re: Need help with graphing function (MATH)

Hi, assuming you have x and want to generate y, then maybe - if x < 50, y = 150 - if x > 175, y = 60 - otherwise : either y = (100/(e^((x -50)/75)^2)) + 50 http://www.wolframalpha.com/input/?i=plot++%28100%2F%28e ^%28%28x+-50%29%2F75%29^2%29%29+%2B+50%2C+x%3D50..175 - or maybe y =sin((x+5)/38

Re: Solr 3.5 not starting on CentOS 6 or RHEL 5

Is /tmp a separate file system? There are problems with people mounting /tmp with 'noexec' as a security precaution, which then causes Solr to fail. On Mon, Feb 13, 2012 at 4:06 PM, Bernhardt, Russell (CIV) wrote: > A software package we use recently upgraded to Solr 3.5 (from 1.4.1) and now >

Solr 3.5 not starting on CentOS 6 or RHEL 5

A software package we use recently upgraded to Solr 3.5 (from 1.4.1) and now we're having problems getting the Solr server to start up under RHEL 5 or CentOS 6. I upgraded our local install of Java to the latest from Oracle and it didn't help, even removed the local OpenJDK just to be sure. Wh

Re: Re:how to monitor solr in newrelic

Just run the java agent as indicated, solr will be detected and 3 new menu items will be available automatically on the app: solr caches, solr updates, solr requests. On Mon, Feb 13, 2012 at 10:56 AM, solr wrote: > My question is how to check solr perfromance in newrelic.Basically we have > ja

Re: Permissions and user to acess administrative interface

Thanks for the responses. I will create rules via htaccess. Regards Vasconcelos 2012/2/13 Ge, Yao (Y.) > I can only speak from my experience with Tomcat. > First make sure the available authentication modes are available by > checking server.xml. > I added a few roles in tomcat-users.xml and

What versions support compressed text fields?

I've looked at the wiki and the changelog, and I'm still confused about what versions support compressed fields. We have an index which is rapidly growing through 100Gb, and I'd like to turn on text field compression without reindexing. Is that possible? We are on 3.3.0. wunder -- Walter Under

Can I rebuild an index and remove some fields?

Lets say I have a large index (100M docs, 1TB, split up between 10 indexes). And a bunch of the "stored" and "indexed" fields are not used in search at all. In order to save memory and disk, I'd like to rebuild that index *without* those fields, but I don't have original documents to rebuild e

RE: spellcheck configuration not providing suggestions or corrections

thank you sooo much - that was it. also - thank you for the tip on which field to hit, eg itemDesc in stead of itemDescSpell. thank you, mark -- View this message in context: http://lucene.472066.n3.nabble.com/spellcheck-configuration-not-providing-suggestions-or-corrections-tp3740877p3741783

Re: How to use nested query in fq?

: I am using Solr 3.5, and would like to use a fq like : 'getField(getDoc(uuid:workspace_${workspaceId})), "isPublic"):true? ... : The use case is that I have workspace objects and workspace contains many : sub-objects, such as work files, comments, datasets and so on. And : workspace has

Re: spellcheck configuration not providing suggestions or corrections

you have put this true Maybe you need to put true Alex. -Original Message- From: Dyer, James To: solr-user Sent: Mon, Feb 13, 2012 12:43 pm Subject: RE: spellcheck configuration not providing suggestions or corrections That would be it, I tbinkl. Your request is to "/sele

Re: solr taking too long to update a document

Sharing the solution to our problems: *What was wrong: *We used to update the slave and the master solr at the same time, and all of the webapps screens were searching on the slave, where we cache almost everything. Updating the cache takes time. *What we Changed:* We now (on the screen where th

Re: Symbols in synonyms

: is it good practice, common, or even possible to put symbols in my list of : synonyms? it entirely depends on your usecases, and wether you want words with those symbols to have synonyms. : I'm having trouble indexing and searching for "A&E", with it being split on : the &. that sounds like

Little hint for: java.net.SocketException: Too many open files

Hi together, We're running several instances of SOLR (3.5) on Apache Tomcat (6.0) on Ubuntu 10.xx. After adding another instance (maybe the 14th or 15th for the developers sandboxes), tomcat rise the exception "java.net.SocketException: Too many open files" . After reading some several sites I've

RE: spellcheck configuration not providing suggestions or corrections

That would be it, I tbinkl. Your request is to "/select", but you've put spellchecking into "/search". Try "/search" instead. Also, I doubt its the problem, but try removing the trailing CRLFs from your query. Also, typically you'd still query against the main field ("itemDesc" in your case)

RE: spellcheck configuration not providing suggestions or corrections

hello thank you for the suggestion - however this did not work. i went in to solrconfig and change the count to 20 - then restarted the server and then did a reimport. is it possible that i am not firing the request handler that i think i am firing ? default false

Re: OR-FilterQuery

Hi, have a look at: http://search-lucene.com/m/Z8lWGEiKoI I think not much had changed since then. Regards, Em Am 13.02.2012 20:17, schrieb spr...@gmx.eu: > Hi, > > how efficent is such an query: > > q=some text > fq=id:(1 OR 2 OR 3...) > > Should I better use q:some text AND id:(1 OR 2 OR 3

OR-FilterQuery

Hi, how efficent is such an query: q=some text fq=id:(1 OR 2 OR 3...) Should I better use q:some text AND id:(1 OR 2 OR 3...)? Is the Filter Cache used for the OR'ed fq? Thank you

RE: spellcheck configuration not providing suggestions or corrections

The one thing that jumps out is you have "spellcheck.count" set to 1. Try 10 and see if you get results. The spellcherker uses a 2-pass algorithm and if the "count" is too small, all the good suggestions can get eliminated in the first pass. So you often need a "count" of maybe 10 even if you

Solr binary response for C#?

Admittedly I'm new to this, but the project we're working on feeds results from Solr to an ASP.net application. Currently we are using XML, but our payloads can be rather large, some up to 17MB. We are looking for a way to minimize that payload and increase performance and I'm curious if there's

Re: New segment file created too often

can you post your config file? I found there are 2 places to config ramBufferSizeMB in latest svn of 3.6's example solrconfig.xml. trying to modify them both? false 10 32 1 1000 . false 32 10

Re: New segment file created too often

as far as I know, there are three situation it will be flushed to a new segment: RAM buffer for posting data structure is used up; added doc numbers are exceeding threshold and there are many deletions in a segment but your configuration seems it is not likely to flush many small segments. 1024 21

Re: New segment file created too often

Hi, I am using solr 3.5. As I understood it, NRT is a solr 4 feature, but solr 4 is not released yet. I understand commit after adding each document is expensive, but the application requires that documents be available after adding to the index. What I don't understand is why new segment files

Re: New segment file created too often

Commit is called after adding each document you should add enough documents and then calling a commit. commit is a cost operation. if you want to get latest feeded documents, you could use NRT On Tue, Feb 14, 2012 at 12:47 AM, Huy Le wrote: > Hi, > > I am using solr 3.5. I seeing solr keep

New segment file created too often

Hi, I am using solr 3.5. I seeing solr keeps creating new segment files (<1MB files) so often that it triggers segment merge about every one minute. I search the news archive, but could not find any info on this issue. I am indexing about 10 docs of less 2KB each every second. Commit is called

RE: Permissions and user to acess administrative interface

I can only speak from my experience with Tomcat. First make sure the available authentication modes are available by checking server.xml. I added a few roles in tomcat-users.xml and add individual user id/password to these roles. For example you can separate by Search, Update, Admin roles. Modif

Re: how to monitor solr in newrelic

Why are you asking us? This is a standard feature of Newrelic, ask them. They should have the answer. http://blog.newrelic.com/2010/05/11/got-apache-solr-search-server-use-rpm-to-monitor-troubleshoot-and-tune-solr-operations/ You can use Solr with any servlet container. We use Tomcat in producti

Re: Permissions and user to acess administrative interface

Hi Anderson, you will need to rearrange the JSPs a little bit to do what you want. If you do so, you can create rules via .htaccess. Otherwise I would suggest you to look for a commercial distribution of Solr which might fit your needs. Regards, Em Am 13.02.2012 16:48, schrieb Anderson vasconce

Re: Re:how to monitor solr in newrelic

My question is how to check solr perfromance in newrelic.Basically we have javaagent .Bu these are instaling in tomcat,jetty,websphere etc.. I have installed standard solr distribution.So jetty is default.But while installing newrelic in solr thats not finding jetty scirpt.Because jetty **.jar file

[commercial] soleami - The Solr Query Log Visualization Tool Launched

We are excited to announce the launch of soleami - a free visualization service for Apache Solr query log. soleami - Visualize the needs of your visitors. http://soleami.com/ Soleami summarizes Solr query logs of past 12 months by search keyword. It is a very useful tool for you to see various in

Filtering suggester suggestions

Hi, I'm using Solr 3.5 and making auto-complete suggestions using Suggester. >From the SOLR-2010 issue I understood it's possible to filter suggestions through the fq parameter using collations; in fact I read "Only return collations that are guara

custom scoring

Hello all: We'd like to score the matching documents using a combination of SOLR's IR score with another application-specific score that we store within the documents themselves (i.e. a float field containing the app-specific score). In particular, we'd like to calculate the final score doing some

Re: Format version is not supported in file 'segments_gw': 0 (needs to be between -9 and -10). This version of Lucene only supports indexes created with release 3.0 and later.

I have no idea how to check that. Also I have no idea how I would attach a trunk Solr to a 1.4 index, so it's not very likely I actually did that :) My Solr folder looks like this: C:\My Dropbox\Dropbox\inetpub\apache-solr-4.0-2010-10-12_08-05-48\example\example-DIH\solr How would I check what ve

Re: Queried value and Indexed value are the same still no match in the query result

query and index analysis is different. word delimiters are set up differently and no ngram filter. look at the index and query an field analysis.jsp output you should see what filter in the query chain fails to match. (choose verbose output) On 13 February 2012 11:12, Dirceu Vieira wrote: > Hi

Re: Format version is not supported in file 'segments_gw': 0 (needs to be between -9 and -10). This version of Lucene only supports indexes created with release 3.0 and later.

It *looks* like you've attached a trunk Solr to a 1.4 index, is that possible? Best Erick On Sun, Feb 12, 2012 at 12:42 PM, PeterKerk wrote: > Im getting the error below all of a sudden. AFAIK i haven't touched anything > with Lucene. > I did add a fields to my schema and data-config.xml: > data

Re: Distributed search: RequestHandler

Hmmm, *how* does this not work? What are you seeing when you try this and what to do you expect to see? Also, what does attaching &debugQuery=on show you? Best Erick On Sat, Feb 11, 2012 at 1:34 PM, ku3ia wrote: > Hi! > > I'm using Solr 3.5. I have two shards. Now I'm using default and my own >

Re: Recovering from database connection resets in DataimportHandler

I'd seriously consider using SolrJ and your favorite JDBC driver instead. It's actually quite easy to create one, although as always it may be a bit intimidating to get started. This allows you much finer control over error conditions than DIH does, so may be more suited to your needs. Best Erick

Re:how to monitor solr in newrelic

Hi I think you can add &debugQuery =true to query url. At 2012-02-13 17:06:42,solr wrote: >How to monitor solr perfromance in newrelic .If there are any agents please >suggest > >-- >View this message in context: >http://lucene.472066.n3.nabble.com/how-to-monitor-solr-in-newrelic-tp3

Re: Joining multicore to return top results

>From your example, it rather looks like you've moved some DB tables into separate cores and are trying to do some SQL-like operations. Stop that! ... Solr really isn't built for this kind of operation. I know this goes against all your DB training, but can you simply de-normalize all the data an

Re: Highlighting stopwords

Hmm, now the synonyms aren't highlighted anymore. OK back to basic (im using trunk and FVH). What is the way to go about if I want to search on a field without stopwords, but still want to highlight the stopwords? (and still highlight synonyms and stemmed words)? -- View this message in contex

Re: SolrCloud Replication Question

Yes, I have the following layout on the FS ./bootstrap.sh ./example (standard example directory from distro containing jetty jars, solr confs, solr war, etc) ./slice1 - start.sh -solr.xml - slice1_shard1 - data - slice2_shard2 -data ./slice2 - start.sh - solr.xml -slice2_shard1

Re: Highlighting stopwords

I got it fixed now I think. I thought that if you used it like "hl.q=spell Checker" it would use the query analysis of the field that was being highlighted as default. But in my case it needs to be "hl.q=content_hl:(spell Checker)" for it to work. The behavour I got default made no sense whatsoev

Re: SolrCloud Replication Question

Do you have unique dataDir for each instance? 13.2.2012 14.30 "Jamie Johnson" kirjoitti:

Re: Highlighting stopwords

Koji Sekiguchi wrote > > (12/02/11 21:19), O. Klein wrote: >> >> Koji Sekiguchi wrote >>> >>> (12/01/24 9:31), O. Klein wrote: Let's say I search for "spellcheck solr" on a website that only contains info about Solr, so "solr" was added to the stopwords.txt. The query that >>>

Re: Query for exact part of sentence

It is still not working after reindexing. Below you can find the output of the filed analysis. Any idea what can be wrong? Index Analyzer org.apache.solr.analysis.HTMLStripCharFilterFactory {luceneMatchVersion=LUCENE_35} text"123 456" org.apache.solr.analysis.KeywordTokenizerFactory {luce

Re: Do we need reindexing from solr 1.4.1 to 3.5.0?

Hi, If you keep the old schema and solrconfig, you should be able to simply upgrade your solr.war. Solr 3.5 will read the old index format and write the new one. Once you've upgraded the index will not be readable by 1.4.1. Test this on a test environment to verify. -- Jan Høydahl, search solu

Re: SolrCloud Replication Question

I don't see any errors in the log. here are the following scripts I'm running, and to create the cores I run the following commands curl 'http://localhost:8501/solr/admin/cores?action=CREATE&name=slice1_shard1&collection=collection1&shard=slice1&collection.configName=config1' curl 'http://local

Re: correct usage of StreamingUpdateSolrServer?

Whats the output of jstack $PID ? If the program does not exit, there must be some non-daemon threads still running. smime.p7s Description: S/MIME cryptographic signature

Re: Queried value and Indexed value are the same still no match in the query result

Hi Lee, Thanks for you reply! Yes, we actually need those filters. This dynamic field is parsing the meta data concerning each video and they may have different content. If I understand where you're going with your comment you mean that I probably should plan it better and create field types that

Re: Solr monitoring: Newrelic

So how to install newrelic javaagent in standard solr pkg -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-monitoring-Newrelic-tp3042889p3739798.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Queried value and Indexed value are the same still no match in the query result

Hi You have a lot of language processing for a field which contains, at least in your example non words. Do you need the synonyms, two lots of stemming, etc what is the field for? >>" I don't believe that this last point is what actually causes >> my unsatisfactory results" it probably is

Re: Queried value and Indexed value are the same still no match in the query result

Hi, Anybody has any thoughts about this? I'm really struggling whit these problems, any hints would be very welcome! Regards, Dirceu On Fri, Feb 10, 2012 at 4:45 PM, Dirceu Vieira wrote: > Hi Guys, > > Would someone have time to help me understand what's happening here: > > I have a dynamic f

Re: Do we need reindexing from solr 1.4.1 to 3.5.0?

One thought... If your old (1.4.1) Solr index has all fields are 'stored' (stored="true" in schema.xml for field), then you can easily create a migration tool which would pick the data from 1.4.1 Solr instances and put it into 3.5.0. Bhavnik On 2/13/2012 12:42 PM, Kashif Khan wrote: Do we need