Re: Solr 1.3 highlighting problem

[Grijesh]

Have you recently change its type from unstored to stored?
if so you have to reindex .

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-1-3-highlighting-problem-tp2918089p2922117.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: edismax available in solr 3.1?

[Grijesh]

See this
http://www.gettingahead.co.in/blogwp/2011/03/technical/solr/shalini/some-info-about-extended-dismax-query-parser/

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/edismax-available-in-solr-3-1-tp2910613p2922128.html
Sent from the Solr - User mailing list archive at Nabble.com.

Indexing Mails

[Jörg Agatz]

Hallo..

I search a Way to indexing e-Mails fast and easy to safe in Solr

A lot of users do this too, but i cant find a howTo or something like that.

when you have or know where i can find a HowTo..
it will nice if you can help me..


King

RE: Solr 3.1 / Java 1.5: Exception regarding analyzer implementation

[Uwe Schindler]

Hi,

> On 09.05.11 11:04, Martin Jansen wrote:
> > I just attempted to set up an instance of Solr 3.1 in Tomcat 5.5
> > running in Java 1.5.  It fails with the following exception on start-up:
> >
> >> java.lang.AssertionError: Analyzer implementation classes or at least
> >> their tokenStream() and reusableTokenStream() implementations must
> be
> >> final at
> >> org.apache.lucene.analysis.Analyzer.assertFinal(Analyzer.java:57)
> 
> In the meantime I solved the issue by installing Java 1.6.  Works without
a
> problem now, but I'm wondering if Solr 3.1 is intentionally incompatible
to
> Java 1.5 or if if happened by mistake.

Solr 3.1 is compatible with Java 1.5 and runs fine with that. The exception
you are seeing should not happen for Analyzers that are shipped with
Solr/Lucene, they can only happen if you wrote your own
Analyzer/TokenStreams that are not declared final as requested. In that case
the error will also happen with Java 6.

BUT: This is only an assertion to make development and debugging easier. The
assertions should not run in production mode, as they may affect performance
(seriously)! You should check you java command line for -ea parameters and
remove them on production.

The reason why this assert hits you in one of your tomcat installations
could also be related to some instrumentation tools you have enabled in this
tomcat. Lot's of instrumentation tools may dynamically change class bytecode
and e.g. make them unfinal. In that case the assertion of course fails (with
assertions enabled). Before saying Solr 3.1 is not compatible with Java 1.5:

- Disable assertions in production (by removing -ea command line parameters,
see http://download.oracle.com/javase/1.4.2/docs/guide/lang/assert.html)
- Check your configuration if you have some instrumentation enabled.

Both of the above points may not affect you on the other server that runs
fine with Java 6.

Uwe

Re: Replication Clarification Please

[Alexander Kanarsky]

Ravi,

as far as I remember, this is how the replication logic works (see
SnapPuller class, fetchLatestIndex method):

> 1. Does the Slave get the whole index every time during replication or
> just the delta since the last replication happened ?


It look at the index version AND the index generation. If both slave's
version and generation are the same as on master, nothing gets
replicated. if the master's generation is greater than on slave, the
slave fetches the delta files only (even if the partial merge was done
on the master) and put the new files from master to the same index
folder on slave (either index or index., see further
explanation). However, if the master's index generation is equals or
less than one on slave, the slave does the full replication by
fetching all files of the master's index and place them into a
separate folder on slave (index.). Then, if the fetch is
successfull, the slave updates (or creates) the index.properties file
and puts there the name of the "current" index folder. The "old"
index. folder(s) will be kept in 1.4.x - which was treated
as a bug - see SOLR-2156 (and this was fixed in 3.1). After this, the
slave does commit or reload core depending whether the config files
were replicated. There is another bug in 1.4.x that fails replication
if the slave need to do the full replication AND the config files were
changed - also fixed in 3.1 (see SOLR-1983).

> 2. If there are huge number of queries being done on slave will it
> affect the replication ? How can I improve the performance ? (see the
> replications details at he bottom of the page)


>From my experience the half of the replication time is a time when the
transferred data flushes to the disk. So the IO impact is important.

> 3. Will the segment names be same be same on master and slave after
> replication ? I see that they are different. Is this correct ? If it
> is correct how does the slave know what to fetch the next time i.e.
> the delta.


They should be the same. The slave fetches the changed files only (see
above), also look at SnapPuller code.

> 4. When and why does the index. folder get created ? I see
> this type of folder getting created only on slave and the slave
> instance is pointing to it.


See above.

> 5. Does replication process copy both the index and index.
folder ?


index. folder gets created only of the full replication
happened at least once. Otherwise, the slave will use the index
folder.

> 6. what happens if the replication kicks off even before the previous
> invocation has not completed ? will the 2nd invocation block or will
> it go through causing more confusion ?


There is a lock (snapPullLock in ReplicationHandler) that prevents two
replications run simultaneously. If there is no bug, it should just
return silently from the replication call. (I personally never had
problem with this so it looks there is no bug :)

> 7. If I have to prep a new master-slave combination is it OK to copy
> the respective contents into the new master-slave and start solr ? or
> do I have have to wipe the new slave and let it replicate from its new
> master ?


If the new master has a different index, the slave will create a new
 folder. There is no need to wipe it.

> 8. Doing an 'ls | wc -l' on index folder of master and slave gave 194
> and 17968 respectively...I slave has lot of segments_xxx files. Is
> this normal ?


No, it looks like in your case the slave continues to replicate to the
same folder for a long time period but the old files are not getting
deleted bu some reason. Try to restart the slave or do core reload on
it to see if the old segments gone.

-Alexander

Re: Indexing Mails

[Gora Mohanty]

On Tue, May 10, 2011 at 1:36 PM, Jörg Agatz  wrote:
> Hallo..
>
> I search a Way to indexing e-Mails fast and easy to safe in Solr

> A lot of users do this too, but i cant find a howTo or something like that.

http://wiki.apache.org/solr/DataImportHandler#Indexing_Emails
which directs one to: http://wiki.apache.org/solr/MailEntityProcessor

Regards,
Gora

Re: Indexing Mails

[Jörg Agatz]

will the E-Mail ID, and the recent E-Mail Ids, indext too?

and witch fiels i have to create in schema.xml?

Facet Count Based on Dates

[Jasneet Sabharwal]

I have a schema which has field Polarity which is of type "text" and it 
can have three values 0,1 or -1 and CreatedAt which is of type "date".


*How can I get count of polarity based on dates. For example, it gives 
the output that on 5/1/2011 there were 10 counts of 0, 10 counts of 1 
and 10 counts of -1

*

If I use the facet query like this :-

http://localhost:8983/solor/select/?q=*:*&facet=true&facet.field=Polarity

Then I get the count of the complete database


531477
530682


The query : 
http://localhost:8983/solr/select/?q=*:*%20AND%20CreatedAt:[2011-03-10T00:00:00Z%20TO%202011-03-18T23:59:59Z]&facet=true&facet.date=CreatedAt&facet.date.start=2011-03-10T00:00:00Z&facet.date.end=2011-03-18T23:59:59Z&facet.date.gap=%2B1DAY 



Would give me the count of data per day, like this:


0
276262
183929
196853
2967
22762
11299
37433
14359
+1DAY
2011-03-10T00:00:00Z
2011-03-19T00:00:00Z


How will I be able to get the Polarity count for each date like:-

2011-03-10T00:00:00Z
Polarity
0 = 100
1 = 500
-1 = 200

--
Thanks&  Regards

Jasneet Sabharwal

Re: How to set a common field to several values types ?

[Grijesh]

Yes you can but one doc will contain one type of data other wise it will be
overwritten by latest one

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-set-a-common-field-to-several-values-types-tp2922192p2922206.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 1.3 highlighting problem

[nicksnels1]

I indeed changed the type from unstored to stored. But I did do a reindex
(several times already) and a commit, restarted Tomcat, restarted PC,
nothing works. When I do a search the text field shows up in the results.
But that is the only thing that has changed.  tag
only shows id's. I have no idea what is wrong?

Kind regards,

Nick

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-1-3-highlighting-problem-tp2918089p2922482.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 1.3 highlighting problem

[Markus Jelsma]

You're using the string fieldType to get highlighting to work but it must be 
tokenized. Use a fieldType with a tokenizer.

See also: http://wiki.apache.org/solr/FieldOptionsByUseCase

> Hi,
> 
> I'm using the "old" 1.3 Solr version on one of my sites and I decided to
> add a highlighting feature. Unfortunately I can not get it to work. I'm
> doing some testing in the Sorl admin interface without much luck. Below is
> some information that describes the problem.
> 
> I would like to highlight text in the field text, schema.xml config of
> text:
> 
> 
> 
> Query in the solr admin interface:
> 
> http://127.0.0.1:8080/solr/select?indent=on&version=2.2&q=solr&start=0&rows
> =10&fl=*%2Cscore&qt=standard&wt=standard&explainOther=&hl=on&hl.fl=text
> 
> I get back two results, both of the text fields contain the query "solr".
> In the highlight tag I get only the IDs:
> 
> 
> 
> Any ideas what may be causing this and how I can debug it? Thanks.
> 
> Kind regards,
> 
> Nick
> 
> 
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-1-3-highlighting-problem-tp2918089
> p2918089.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 1.3 highlighting problem

[Grijesh]

Oh!
I did't  see the "string" 

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-1-3-highlighting-problem-tp2918089p2922532.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Facet Count Based on Dates

[Grijesh]

Have you looked at Pivot Faceting
http://wiki.apache.org/solr/HierarchicalFaceting
http://wiki.apache.org/solr/SimpleFacetParameters#Pivot_.28ie_Decision_Tree.29_Faceting-1

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Facet-Count-Based-on-Dates-tp2922371p2922541.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Indexing Mails

[Grijesh]

Look at  http://wiki.apache.org/solr/MailEntityProcessor
http://wiki.apache.org/solr/MailEntityProcessor  it will give you the all
informations

You may have to create the fields as
single valued fields :

messageId
subject
from
sentDate
xMailer
multi valued fields

allTo
flags : possible flags are 'answered', 'deleted', 'draft', 'flagged' ,
'recent', 'seen'
content
attachment
attachmentNames; 

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Indexing-Mails-tp2922327p2922547.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: tomcat and multicore processors

[Grijesh]

I think OS will manage it when required OS will use other core of your CPU
also. Also you can use some memory management(GC Options) settings with
java.

You can see about memory management a blog posted by "Mark Miller" at
Lucidimagination.com 

http://www.gettingahead.co.in/blogwp/2011/03/technical/solr/shalini/memory-management-and-garbage-collection-algorithm/
http://www.gettingahead.co.in/blogwp/2011/03/technical/solr/shalini/memory-management-and-garbage-collection-algorithm/
 

about detail of memory management

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/tomcat-and-multicore-processors-tp2917973p2922565.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to set a common field to several values types ?

[cocowww]

Thanks for your answer! 

It is pretty weird because all my tries are wrong.

If I try to index data with this field in the schema :



It works perfectly (in the case of indexing extract from file (via solr
cell/tika)), but it fails if I try to index plain text data! Is there
something wrong ?
And the field will never be able to index plain text data! (In any case, I
haven't seen it).

Another way to resolve the situation is to have 2 fields :
Then, if I use these fields :

 
 

The first is to index data from extracted files, and the second is to index
plain text data.
With this kind of schema, I am able to index both values types, without any
mistakes/problems. (but both fields have the same features!).

Did you have any suggestions in order to help me ? (to summarize, my goal is
to have only one field to share several data types)

I have a (dummy?) question to ask : can we index data using the
requestHandler "/update/extract/" without extracting any documents ? (I
assume my evils are there   )

Thank you (again!)

--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-set-a-common-field-to-several-values-types-tp2922192p2922573.html
Sent from the Solr - User mailing list archive at Nabble.com.

Boosting score of a document without deleting and adding another document

[karan veer singh]

I want to add score for documents dynamically. For 
this, the method that I know right now is to use "add". Using this 
method, everytime I need to get the contents of that document, and then 
add another record with the boost field. Is there a way in which I can 
just specify the boost attribute value, and not have to delete and add 
the whole document?


Thanks

Re: How to set a common field to several values types ?

[Grijesh]

Which way you are using to index the Plain text data?
As I see your both the fields are same then why not use same field for both
the data.
What is the need of extracting request handler if you are not going to
extract any document?

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-set-a-common-field-to-several-values-types-tp2922192p2922601.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Boosting score of a document without deleting and adding another document

[Grijesh]

NO, You have to update the whole document .There is no way to update only
some of fields or any field attribute in solr/Lucene currently.

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Boosting-score-of-a-document-without-deleting-and-adding-another-document-tp2922597p2922610.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Boosting score of a document without deleting and adding another document

[karan veer singh]

Any other workaround? Maybe by fiddling around with the source?
> From: kvs_cheezy_...@hotmail.com
> To: solr-user@lucene.apache.org
> Subject: Boosting score of a document without deleting and adding another 
> document
> Date: Tue, 10 May 2011 16:17:25 +0530
> 
> 
> 
> I want to add score for documents dynamically. For 
> this, the method that I know right now is to use "add". Using this 
> method, everytime I need to get the contents of that document, and then 
> add another record with the boost field. Is there a way in which I can 
> just specify the boost attribute value, and not have to delete and add 
> the whole document?
> 
> 
> Thanks  
  

Re: Solr 3.1 Upgrade - Reindex necessary ?

[Grijesh]

>1. Do we have to reindex all content again to use Solr 3.1 ? 

>2. If we don't reindex all content are there any potential issues ? (I 
>read somewhere that first commit would change the 1.4.1 format to 3.1. 
>have the analyzer's behavior changed which warrants reindexing ?) 
>3. Apart from deploying the new solr 3.1 war; Is it just enough to set 
>"LUCENE_31"  to get all the 
>goodies and bug fixes of the LUCENE/SOLR 3.1 ? 

HI Solr-3.1 version usage the the latest version of Lucene jars so if you
are planning to Upgrade then it is necessary to Re index all the content
with Solr3.1 version.

Not re-indexing will possibly cause of index corruption because newer
version of lucene will create indexes in Newer version which is backward
compatible for read only.

setting  LUCENE_31 is not enough
because it will not get the lucene 3.1 jar automatically.

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-3-1-Upgrade-Reindex-necessary-tp2919679p2922645.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Boosting score of a document without deleting and adding another document

[Grijesh]

You can add the boost at the time of indexing the doc at first time.
You can see the function queries at query time to boost the documents
dynamically.
You can see around DefaultSimilarityClass to tweak around scores

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Boosting-score-of-a-document-without-deleting-and-adding-another-document-tp2922597p2922655.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Boosting score of a document without deleting and adding another document

[karan veer singh]

What I want to do is : everytime an item is searched, I get the ID of that 
document, and now I want to increase this document's score.Function queries at 
query time won't really help with this. Any way to do this other than adding 
the entire document again?
> From: kvs_cheezy_...@hotmail.com
> To: solr-user@lucene.apache.org
> Subject: Boosting score of a document without deleting and adding another 
> document
> Date: Tue, 10 May 2011 16:17:25 +0530
> 
> 
> 
> I want to add score for documents dynamically. For 
> this, the method that I know right now is to use "add". Using this 
> method, everytime I need to get the contents of that document, and then 
> add another record with the boost field. Is there a way in which I can 
> just specify the boost attribute value, and not have to delete and add 
> the whole document?
> 
> 
> Thanks  
  

RE: Boosting score of a document without deleting and adding another document

[lboutros]

Perhaps the query elevation component is what you are looking for :

http://wiki.apache.org/solr/QueryElevationComponent

Ludovic.

-
Jouve
France.
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Boosting-score-of-a-document-without-deleting-and-adding-another-document-tp2922597p2922707.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Boosting score of a document without deleting and adding another document

[Grijesh]

Nope can't do,
  you can keep the counter at application end and update the time-stamp of
documents which have been searched then update/index the document with
counter or assign the boost to document being updated.

if keeping counter then by using function query you can influence the score
of hire/lower count .I have used this approach for a property portal to get
up the least contacted properties.

The QueryElevationComponent can not do that job because it will put your
elevated doc on top only not affected by score any way. also need reload
solr after updating the elevate.xml

-
Thanx: 
Grijesh 
www.gettinhahead.co.in 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Boosting-score-of-a-document-without-deleting-and-adding-another-document-tp2922597p2922754.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 1.3 highlighting problem

[nicksnels1]

Hi Markus,

thanks for the tip. I replaced the string field with a tokenizer field and
now it works. Thank you!

Kind regards,

Nick

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-1-3-highlighting-problem-tp2918089p2922785.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to set a common field to several values types ?

[cocowww]

Grijesh wrote:
> 
> Which way you are using to index the Plain text data?
> 
I'm using Solr in a Php application, and have to index documents (with tika)
and plain text data.
The php interface uses curl to send xml data to the solr server.


Grijesh wrote:
> 
> As I see your both the fields are same then why not use same field for
> both the data.
> 
I just was trying to show what I found  weird in my tries!


Grijesh wrote:
> 
> What is the need of extracting request handler if you are not going to
> extract any document?
> 
My question is rather this : "Can I use the extracting request handler even
if I don't have to extract any documents ?" (in order to prevent me from any
adaptation to the both cases (documents/plain text))


--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-set-a-common-field-to-several-values-types-tp2922192p2922849.html
Sent from the Solr - User mailing list archive at Nabble.com.

Building hierarcies of query object instead of flat string queries

[Geir Gullestad Pettersen]

Is it possible to create hierarchies of query operator objects and pass that
to solr instead of constructing a long string query?

This would enable client applications to construct complex queries and have
them verified by the compiler, example (psuedo code):

q = BooleanQuery(
  minNumberShouldMatch => 1,
  booleanClauses => ClauseList(
MustMatch(DisjunctionMaxQuery(
  PhraseQuery("features","apache solr", 2),
  PhraseQuery("name","apache solr", 3)
),
ShouldMatch(DisjunctionMaxQuery(
  TermQuery("features","search", 2),
  TermQuery("name","search", 3)
),
ShouldMatch(DisjunctionMaxQuery(
  TermQuery("features","server", 2),
  TermQuery("name","server", 3)
));

solr.search( q );


However, i cannot see that the Solr api supports this.

Any thoughts?


Thanks,
Geir

DIH entity threads (multithreading)

[Jamroz Marcin]

Hallo,

I have a problem using threads option in entity in DIH it just does not work, 
it either hangs it self or fails to import anything.

Does this feature even work ?

Without threads the import works fine, just too slow

**
Diese E-Mail wurde auf Viren ueberprueft.
mailswee...@it2media.de
**

Re: Is it possible to build Solr as a maven project?

[Gabriele Kahlout]

> sorry, this was not the target I used (this one should work too, but...),
>

Can we expand on the but...?

$ wget http://apache.panu.it//lucene/solr/3.1.0/apache-solr-3.1.0-src.tgz
$ tar xf apache-solr-3.1.0-src.tgz
$ cd apache-solr-3.1.0
$ ant generate-maven-artifacts
*generate-maven-artifacts:

get-maven-poms:

BUILD FAILED
/Users/simpatico/Downloads/apache-solr-3.1.0/build.xml:59: The following
error occurred while executing this line:
/Users/simpatico/Downloads/apache-solr-3.1.0/lucene/build.xml:445: The
following error occurred while executing this line:
/Users/simpatico/Downloads/apache-solr-3.1.0/build.xml:45:
/Users/simpatico/Downloads/apache-solr-3.1.0/dev-tools/maven does not exist.
*


Now for those that build this, it must have worked sometime. How? Or is this
a bug in the release?
Looking the revisions history of the build script I might be referring to
LUCENE-2490  but I'm not
sure I understand the solution out. I've checked out dev-tools but even with
it things don't work (tried the one with 3.1.0 relesase).





> the one I used is get-maven-poms. That will just create pom files and copy
> them to their right target locations.
>
> I'm using netbeans and I'm using the plugin "Automatic Projects" to do
> everything inside the IDE.
>
> Which version of Solr are you using ?
>
> Ludovic.
>
> 2011/5/4 Gabriele Kahlout [via Lucene] <
> ml-node+2898211-2124746009-383...@n3.nabble.com>
>
> > generate-maven-artifacts:
> >[mkdir] Created dir: /Users/simpatico/SOLR_HOME/build/maven
> >[mkdir] Created dir: /Users/simpatico/SOLR_HOME/dist/maven
> > [copy] Copying 1 file to
> > /Users/simpatico/SOLR_HOME/build/maven/src/maven
> > [artifact:install-provider] Installing provider:
> > org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
> >
> > *BUILD FAILED*
> > /Users/simpatico/SOLR_HOME/*build.xml:800*: The following error occurred
> > while executing this line:
> > /Users/simpatico/SOLR_HOME/common-build.xml:274: artifact:deploy doesn't
> > support the "uniqueVersion" attribute
> >
> >
> > *build.xml:800: * > pom.xml="src/maven/solr-parent-pom.xml.template"/>
> >
> > removed "uniquVersion" attirubte:
> >
> > generate-maven-artifacts:
> > [artifact:install-provider] Installing provider:
> > org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
> > [artifact:deploy] Deploying to
> file:///Users/simpatico/SOLR_HOME/dist/maven
> >
> > [artifact:deploy] [INFO] Retrieving previous build number from remote
> > [artifact:deploy] [INFO] Retrieving previous metadata from remote
> > [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact
> > org.apache.solr:solr-parent'
> > [artifact:deploy] [INFO] Retrieving previous metadata from remote
> > [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot
> > org.apache.solr:solr-parent:1.4.2-SNAPSHOT'
> >  [copy] Copying 1 file to /Users/simpatico/SOLR_HOME/build/maven/lib
> > [artifact:install-provider] Installing provider:
> > org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
> > [artifact:deploy] Deploying to
> file:///Users/simpatico/SOLR_HOME/dist/maven
> >
> > [artifact:deploy] [INFO] Retrieving previous build number from remote
> > [artifact:deploy] [INFO] Retrieving previous metadata from remote
> > [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact
> > org.apache.solr:solr-commons-csv'
> > [artifact:deploy] [INFO] Retrieving previous metadata from remote
> > [artifact:deploy] [INFO] Uploading project information for
> solr-commons-csv
> >
> > 1.4.2-SNAPSHOT
> > [artifact:deploy] [INFO] Retrieving previous metadata from remote
> > [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot
> > org.apache.solr:solr-commons-csv:1.4.2-SNAPSHOT'
> >  [copy] Copying 1 file to
> > /Users/simpatico/SOLR_HOME/build/maven/contrib/dataimporthandler
> > [artifact:install-provider] Installing provider:
> > org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
> >
> > BUILD FAILED
> > /Users/simpatico/SOLR_HOME/build.xml:809: The following error occurred
> > while
> > executing this line:
> > */Users/simpatico/SOLR_HOME/common-build.xml:274: artifact:deploy doesn't
> > support the nested "attach" element*
> >
> >
>
>
> -
> Jouve
> France.
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Is-it-possible-to-build-Solr-as-a-maven-project-tp2898068p2898315.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Regards,
K. Gabriele

--- unchanged since 20/9/10 ---
P.S. If the subject contains "[LON]" or the addressee acknowledges the
receipt within 48 hours then I don't resend the email.
subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧ time(x)
< Now + 48h) ⇒ ¬resend(I, this).

If an email is sent by a sender that is not a trusted contact or the email
does not contain a valid code then the email is not received. A valid code
starts with a hyphen and ends with "X".
∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSender

Re: Solr security

[Anthony Wlodarski]

The WIKI has a loose interpretation of how to set-up Jetty securely.  
Please take a look at the article I wrote here:  
http://anthonyw.net/2011/04/securing-jetty-and-solr-with-php-authentication/.  
Even if PHP is not your language that sits on top of Solr you can still 
use the first part of the tutorial.  If you are using Tomcat I would 
recommend looking here: 
http://blog.comtaste.com/2009/02/securing_your_solr_server_on_t.html


Regards,

-Anthony

On 05/09/2011 05:28 PM, Jan Høydahl wrote:

Hi,

You can simply configure a firewall on your Solr server to only allow access 
from your frontend server. Whether you use the built-in software firewall of 
Linux/Windows/Whatever or use some other FW utility is a choice you need to 
make. This is by design - you should never ever expose your backend services, 
whether it's a search server or a database server, to the public.

Read more about Solr security on the WIKI: 
http://wiki.apache.org/solr/SolrSecurity

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

On 9. mai 2011, at 20.57, Brian Lamb wrote:


Hi all,

Is it possible to set up solr so that it will only execute dataimport
commands if they come from localhost?

Right now, my application and my solr installation are on different servers
so any requests are formatted http://domain:8983 instead of
http://localhost:8983. I am concerned that when I launch my application,
there will be the potential for abuse. Is the best solution to have
everything reside on the same server?

What are some other solutions?

Thanks,

Brian Lamb


--
Anthony Wlodarski
Lead Software Engineer
Get2Know.me (http://www.get2know.me)
Office: 646-285-0500 x217
Fax: 646-285-0400

Re: DIH entity threads (multithreading)

[Gora Mohanty]

On Tue, May 10, 2011 at 5:58 PM, Jamroz Marcin
 wrote:
> Hallo,
>
> I have a problem using threads option in entity in DIH it just does not work, 
> it either hangs it self or fails to import anything.
>
> Does this feature even work ?
>
> Without threads the import works fine, just too slow
[...]

Which version of Solr are you using?

Regards,
Gora

Re: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.handler.dataimport.DataImportHandler'

[vrpar...@gmail.com]

now its working gr8,

thanks stefan,scott 

--
View this message in context: 
http://lucene.472066.n3.nabble.com/org-apache-solr-common-SolrException-Error-loading-class-org-apache-solr-handler-dataimport-DataImpo-tp2865625p2922985.html
Sent from the Solr - User mailing list archive at Nabble.com.

how to do offline adding/updating index

[vrpar...@gmail.com]

Hello all,

indexing with dataimporthandler runs every hour (new records will be added,
some records will be updated) note :large data

requirement is when indexing is in progress, searching (on already indexed
data) should not affect

so should i use multicore-with merge and swap or delta query or any other
way?

Thanks

--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-do-offline-adding-updating-index-tp2923035p2923035.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Boosting score of a document without deleting and adding another document

[Ahmet Arslan]

> What I want to do is : everytime an item is searched, I get
> the ID of that document, and now I want to increase this
> document's score.Function queries at query time won't really
> help with this. Any way to do this other than adding the
> entire document again?

Looks like you need ExternalFileField.

http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html

RE: how to do offline adding/updating index

[Jonathan Rochkind]

One approach is to use Solr's replication features.  Index to a 'master', 
periodically replicate to 'slave' on which all the searching is done. 

That's what I do; my master and slave are in fact on the same server (one with 
a bunch of CPUs and RAM however), although not alternate cores in a multi-core 
setup. I in fact put them in different containers (different tomcat or jetty 
instances) to isolate them as much as possible (don't want an accidental OOM on 
one effecting the other).This seems to work out pretty well -- although I 
think that while the replication operation is actually going on, performance on 
the slave is indeed effected somewhat, it's not completely without side effect. 
 

It's possible using some kind of 'swapping' technique would eliminate that, as 
you suggest, but I haven't tried it. Certainly a delta query for indexing 
imports is always a good idea if it will work for you, but with or without 
you'll probably need some other setup in addition to isolate your indexing from 
your searching, either replication or a method of 'swapping', indexing to a new 
Solr index and then swapping the indexes out. 

From: vrpar...@gmail.com [vrpar...@gmail.com]
Sent: Tuesday, May 10, 2011 9:22 AM
To: solr-user@lucene.apache.org
Subject: how to do offline adding/updating index

Hello all,

indexing with dataimporthandler runs every hour (new records will be added,
some records will be updated) note :large data

requirement is when indexing is in progress, searching (on already indexed
data) should not affect

so should i use multicore-with merge and swap or delta query or any other
way?

Thanks

--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-do-offline-adding-updating-index-tp2923035p2923035.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Is it possible to build Solr as a maven project?

[Steven A Rowe]

Hi Gabriele,

There are some Maven instructions here (not in Lucene/Solr 3.1 because I just 
wrote the file a couple of days ago):


My recommendation, since the Solr 3.1 source tarball does not include 
dev-tools/, is to check out the 3.1-tagged sources from Subversion:

svn co http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_1

and then follow the instructions in the above-linked README.maven.  I did that 
just now and it worked for me.  The results are in solr/package/maven/.

Please write back if you run into any problems.

Steve


From: Gabriele Kahlout [mailto:gabri...@mysimpatico.com]
Sent: Tuesday, May 10, 2011 8:37 AM
To: boutr...@gmail.com
Cc: solr-user@lucene.apache.org; Steven A Rowe; ryan...@gmail.com
Subject: Re: Is it possible to build Solr as a maven project?


sorry, this was not the target I used (this one should work too, but...),

Can we expand on the but...?

$ wget 
http://apache.panu.it//lucene/solr/3.1.0/apache-solr-3.1.0-src.tgz
$ tar xf apache-solr-3.1.0-src.tgz
$ cd apache-solr-3.1.0
$ ant generate-maven-artifacts
generate-maven-artifacts:

get-maven-poms:

BUILD FAILED
/Users/simpatico/Downloads/apache-solr-3.1.0/build.xml:59: The following error 
occurred while executing this line:
/Users/simpatico/Downloads/apache-solr-3.1.0/lucene/build.xml:445: The 
following error occurred while executing this line:
/Users/simpatico/Downloads/apache-solr-3.1.0/build.xml:45: 
/Users/simpatico/Downloads/apache-solr-3.1.0/dev-tools/maven does not exist.



Now for those that build this, it must have worked sometime. How? Or is this a 
bug in the release?
Looking the revisions history of the build script I might be referring to 
LUCENE-2490 but I'm not sure 
I understand the solution out. I've checked out dev-tools but even with it 
things don't work (tried the one with 3.1.0 relesase).




the one I used is get-maven-poms. That will just create pom files and copy
them to their right target locations.

I'm using netbeans and I'm using the plugin "Automatic Projects" to do
everything inside the IDE.

Which version of Solr are you using ?

Ludovic.

2011/5/4 Gabriele Kahlout [via Lucene] <
ml-node+2898211-2124746009-383...@n3.nabble.com>

> generate-maven-artifacts:
>[mkdir] Created dir: /Users/simpatico/SOLR_HOME/build/maven
>[mkdir] Created dir: /Users/simpatico/SOLR_HOME/dist/maven
> [copy] Copying 1 file to
> /Users/simpatico/SOLR_HOME/build/maven/src/maven
> [artifact:install-provider] Installing provider:
> org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
>
> *BUILD FAILED*
> /Users/simpatico/SOLR_HOME/*build.xml:800*: The following error occurred
> while executing this line:
> /Users/simpatico/SOLR_HOME/common-build.xml:274: artifact:deploy doesn't
> support the "uniqueVersion" attribute
>
>
> *build.xml:800: * pom.xml="src/maven/solr-parent-pom.xml.template"/>
>
> removed "uniquVersion" attirubte:
>
> generate-maven-artifacts:
> [artifact:install-provider] Installing provider:
> org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
> [artifact:deploy] Deploying to 
> file:///Users/simpatico/SOLR_HOME/dist/maven
>
> [artifact:deploy] [INFO] Retrieving previous build number from remote
> [artifact:deploy] [INFO] Retrieving previous metadata from remote
> [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact
> org.apache.solr:solr-parent'
> [artifact:deploy] [INFO] Retrieving previous metadata from remote
> [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot
> org.apache.solr:solr-parent:1.4.2-SNAPSHOT'
>  [copy] Copying 1 file to /Users/simpatico/SOLR_HOME/build/maven/lib
> [artifact:install-provider] Installing provider:
> org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
> [artifact:deploy] Deploying to 
> file:///Users/simpatico/SOLR_HOME/dist/maven
>
> [artifact:deploy] [INFO] Retrieving previous build number from remote
> [artifact:deploy] [INFO] Retrieving previous metadata from remote
> [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact
> org.apache.solr:solr-commons-csv'
> [artifact:deploy] [INFO] Retrieving previous metadata from remote
> [artifact:deploy] [INFO] Uploading project information for solr-commons-csv
>
> 1.4.2-SNAPSHOT
> [artifact:deploy] [INFO] Retrieving previous metadata from remote
> [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot
> org.apache.solr:solr-commons-csv:1.4.2-SNAPSHOT'
>  [copy] Copying 1 file to
> /Users/simpatico/SOLR_HOME/build/maven/contrib/dataimporthandler
> [artifact:install-provider] Installing provider:
> org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
>
> BUILD FAILED
> /Users/simpatico/SOLR_HOME/build.xml:809: The following error occurred
> while
> executing this line:
> */Use

Re: how to do offline adding/updating index

[Michael McCandless]

Under the hood, Lucene can support this by keeping multiple commit
points in the index.

So you'd make a new commit whenever you finish indexing the updates
from each hour, and record that this is the last "searchable" commit.

Then you are free to commit while indexing the next hour's worth of
changes, but these commits are not marked as searchable.

But... this is a low level Lucene capability and I don't know of any
plans for Solr to support multiple commit points in the index.

Mike

http://blog.mikemccandless.com

On Tue, May 10, 2011 at 9:22 AM, vrpar...@gmail.com  wrote:
> Hello all,
>
> indexing with dataimporthandler runs every hour (new records will be added,
> some records will be updated) note :large data
>
> requirement is when indexing is in progress, searching (on already indexed
> data) should not affect
>
> so should i use multicore-with merge and swap or delta query or any other
> way?
>
> Thanks
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/how-to-do-offline-adding-updating-index-tp2923035p2923035.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Is it possible to build Solr as a maven project?

[Gabriele Kahlout]

On Tue, May 10, 2011 at 3:50 PM, Steven A Rowe  wrote:

> Hi Gabriele,
>
> There are some Maven instructions here (not in Lucene/Solr 3.1 because I
> just wrote the file a couple of days ago):
> <
> http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_3_1/dev-tools/maven/README.maven
> >
>
> My recommendation, since the Solr 3.1 source tarball does not include
> dev-tools/, is to check out the 3.1-tagged sources from Subversion:
>
> svn co http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_1
>
> and then follow the instructions in the above-linked README.maven.  I did
> that just now and it worked for me.  The results are in solr/package/maven/.
>

I did that and i think they worked for me but i didn't get nutch to work
with it, so I preferred to revert to what is officially supported (not even,
but...).

I'll be trying and report back. Thank you in advance.


>
> Please write back if you run into any problems.
>
> Steve
>
>
> From: Gabriele Kahlout [mailto:gabri...@mysimpatico.com]
> Sent: Tuesday, May 10, 2011 8:37 AM
> To: boutr...@gmail.com
> Cc: solr-user@lucene.apache.org; Steven A Rowe; ryan...@gmail.com
> Subject: Re: Is it possible to build Solr as a maven project?
>
>
> sorry, this was not the target I used (this one should work too, but...),
>
> Can we expand on the but...?
>
> $ wget http://apache.panu.it//lucene/solr/3.1.0/apache-solr-3.1.0-src.tgz<
> http://apache.panu.it/lucene/solr/3.1.0/apache-solr-3.1.0-src.tgz>
> $ tar xf apache-solr-3.1.0-src.tgz
> $ cd apache-solr-3.1.0
> $ ant generate-maven-artifacts
> generate-maven-artifacts:
>
> get-maven-poms:
>
> BUILD FAILED
> /Users/simpatico/Downloads/apache-solr-3.1.0/build.xml:59: The following
> error occurred while executing this line:
> /Users/simpatico/Downloads/apache-solr-3.1.0/lucene/build.xml:445: The
> following error occurred while executing this line:
> /Users/simpatico/Downloads/apache-solr-3.1.0/build.xml:45:
> /Users/simpatico/Downloads/apache-solr-3.1.0/dev-tools/maven does not exist.
>
>
>
> Now for those that build this, it must have worked sometime. How? Or is
> this a bug in the release?
> Looking the revisions history of the build script I might be referring to
> LUCENE-2490 but I'm not
> sure I understand the solution out. I've checked out dev-tools but even with
> it things don't work (tried the one with 3.1.0 relesase).
>
>
>
>
> the one I used is get-maven-poms. That will just create pom files and copy
> them to their right target locations.
>
> I'm using netbeans and I'm using the plugin "Automatic Projects" to do
> everything inside the IDE.
>
> Which version of Solr are you using ?
>
> Ludovic.
>
> 2011/5/4 Gabriele Kahlout [via Lucene] <
> ml-node+2898211-2124746009-383...@n3.nabble.com ml-node%2b2898211-2124746009-383...@n3.nabble.com>>
>
> > generate-maven-artifacts:
> >[mkdir] Created dir: /Users/simpatico/SOLR_HOME/build/maven
> >[mkdir] Created dir: /Users/simpatico/SOLR_HOME/dist/maven
> > [copy] Copying 1 file to
> > /Users/simpatico/SOLR_HOME/build/maven/src/maven
> > [artifact:install-provider] Installing provider:
> > org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
> >
> > *BUILD FAILED*
> > /Users/simpatico/SOLR_HOME/*build.xml:800*: The following error occurred
> > while executing this line:
> > /Users/simpatico/SOLR_HOME/common-build.xml:274: artifact:deploy doesn't
> > support the "uniqueVersion" attribute
> >
> >
> > *build.xml:800: * > pom.xml="src/maven/solr-parent-pom.xml.template"/>
> >
> > removed "uniquVersion" attirubte:
> >
> > generate-maven-artifacts:
> > [artifact:install-provider] Installing provider:
> > org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
> > [artifact:deploy] Deploying to
> file:///Users/simpatico/SOLR_HOME/dist/maven
> >
> > [artifact:deploy] [INFO] Retrieving previous build number from remote
> > [artifact:deploy] [INFO] Retrieving previous metadata from remote
> > [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact
> > org.apache.solr:solr-parent'
> > [artifact:deploy] [INFO] Retrieving previous metadata from remote
> > [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot
> > org.apache.solr:solr-parent:1.4.2-SNAPSHOT'
> >  [copy] Copying 1 file to /Users/simpatico/SOLR_HOME/build/maven/lib
> > [artifact:install-provider] Installing provider:
> > org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2
> > [artifact:deploy] Deploying to
> file:///Users/simpatico/SOLR_HOME/dist/maven
> >
> > [artifact:deploy] [INFO] Retrieving previous build number from remote
> > [artifact:deploy] [INFO] Retrieving previous metadata from remote
> > [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact
> > org.apache.solr:solr-commons-csv'
> > [artifact:deploy] [INFO] Retrieving previous metadata from remote
> > [artifact:deploy] [INFO] Uploading project information for
> solr-commons-csv
> >
> > 1.4.2-SNAPSHOT
> > [artifact:deploy] [INFO] Retrieving previou

Re: Is it possible to build Solr as a maven project?

[lboutros]

Very nice Steve ! Thanks again. (I'm building from svn so that's perfect for
me)
Is this file referenced somewhere in the wiki ?

Ludovic.

-
Jouve
France.
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Is-it-possible-to-build-Solr-as-a-maven-project-tp2898068p2923151.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Is it possible to build Solr as a maven project?

[Steven A Rowe]

Hi Ludovic,

On 5/10/2011 at 10:02 AM, lboutros wrote:
> Very nice Steve ! Thanks again. (I'm building from svn so that's perfect
> for me)
> Is this file referenced somewhere in the wiki ?

Not yet, no.  Probably should be linked from the HowToContribute pages for 
Lucene and Solr.  Feel free to add the link yourself.

Steve

RE: Is it possible to build Solr as a maven project?

[Steven A Rowe]

On 5/10/2011 at 9:57 AM, Gabriele Kahlout wrote:
> On Tue, May 10, 2011 at 3:50 PM, Steven A Rowe  wrote:
> > 
[...]
> > svn co http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_1
> 
> I did that and i think they worked for me but i didn't get nutch to work
> with it, so I preferred to revert to what is officially supported (not
> even, but...).

FYI, the "official" Solr source distribution is generated by running "svn 
export" using the same revision as the release tag, but excluding dev-tools/.  
(I wrote the build pieces to generate the source release.)  

Checking a release tag out of Subversion is pretty damn official.

Steve

RE: SolrQuery API for adding group filter

[Dyer, James]

I recently set up a solrj application that uses Solr Trunk and grouping.  I 
didn't see where there was any explicit support in solrj for grouping (in 
Trunk...Maybe there is in the old SOLR-236 version).  But you can set any 
parameters on the request like this:

SolrQuery query = new SolrQuery("...");
query.set("group", true);
query.set("group.field", "FIELD_TO_GROUP_ON");

Getting the response back is the tricky part.  You have to parse the raw 
response because once again, I didn't see any explicit support.  Here's a 
simple example how to do it.  I did some quick editing on this and stripped out 
all of the null checks, etc for clarity.  You'll also need additional code if 
your "group.limit" is something other than 1...

QueryResponse resp = server.query( ... );
...
NamedList respNL = resp.getResponse();
NamedList groupInfo = (NamedList) respNL.get("grouped");
NamedList thisGroupInfo = (NamedList) groupInfo.get("FIELD_TO_GROUP_ON");
Number totalUngrouped = (Number) thisGroupInfo.get("matches");
long totalNumberOfUngroupedDocuments = totalUngrouped.longValue();
List groupData = (List) thisGroupInfo.get("groups");
int numberOfGroupsReturnedOnThisPage = groupData.size();
for(Object o : groupData) {
NamedList thisGroup = (NamedList) o;
SolrDocumentList sdl = (SolrDocumentList) thisGroup.get("doclist");
long totalDocsInThisGroup = sdl.getNumFound();
int totalDocsReturnedForThisGroup = sdl.size();
SolrDocument groupedDoc = sdl.get(0);
//do something with the document here...
}

As an alternative to getting the response back like this, I believe if you 
specify "group.format=simple" and "group.main=true", then the response will 
come back with a normal SolrDocumentList and you won't need code like above.  
One caveat to this is my testing showed a significant performance and/or memory 
usage hit from using these two parameters.  I didn't dig any deeper to try and 
figure out why.

James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311

-Original Message-
From: arian487 [mailto:akarb...@tagged.com] 
Sent: Monday, May 09, 2011 9:44 PM
To: solr-user@lucene.apache.org
Subject: SolrQuery API for adding group filter

There doesn't seem to be API to add a group (like group.field or group=true). 
I'm very new to this so I'm wondering how I'd go about adding a group query
much like how I use 'addFilterQuery' to add an fq.  Thanks.  

--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrQuery-API-for-adding-group-filter-tp2921539p2921539.html
Sent from the Solr - User mailing list archive at Nabble.com.

SolrHome ends with /./ - is this normal?

[Gabriele Kahlout]

 Hello,

I'm having trouble getting Solr 3.1 to work with nutch-1.3.  I'm not sure
where the problem is, but I'm wondering why does the solrHome path end with
/./.

cwd=/Applications/NetBeans/apache-tomcat-7.0.6/bin
SolrHome=/Users/simpatico/apache-solr-3.1.0/solr/./

In the web.xml of solr:

   
   solr/home

${user.home}/apache-solr-3.1.0/solr
   java.lang.String



-- 
Regards,
K. Gabriele

--- unchanged since 20/9/10 ---
P.S. If the subject contains "[LON]" or the addressee acknowledges the
receipt within 48 hours then I don't resend the email.
subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧ time(x)
< Now + 48h) ⇒ ¬resend(I, this).

If an email is sent by a sender that is not a trusted contact or the email
does not contain a valid code then the email is not received. A valid code
starts with a hyphen and ends with "X".
∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
L(-[a-z]+[0-9]X)).

RE: Is it possible to build Solr as a maven project?

[lboutros]

Steve,

I'm not used to update wikis, but I've added a small part after the IntelliJ
part here :

http://wiki.apache.org/solr/HowToContribute

Ludovic.

-
Jouve
France.
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Is-it-possible-to-build-Solr-as-a-maven-project-tp2898068p2923247.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrHome ends with /./ - is this normal?

[Gabriele Kahlout]

It apparently is normal, and my issue is indeed with nutch.

I've modified post.sh from the example docs to use the solr in
http://localhost:8080/apache-solr-3.1-SNAPSHOT and now finally data made it
to the index.
$ post.sh solr.xml monitor.xml

With nutch I'm at:

$ svn info
Path: .
URL: http://svn.apache.org/repos/asf/nutch/branches/branch-1.3
Repository Root: http://svn.apache.org/repos/asf
Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
Revision: *1101459*
Node Kind: directory
Schedule: normal
Last Changed Author: markus
Last Changed Rev: 1101280
Last Changed Date: 2011-05-10 02:46:04 +0200 (Tue, 10 May 2011)

Does this work for you? All I've done is svn co nutch 1.3 and execute my
script which up to now worked.


On Tue, May 10, 2011 at 4:11 PM, Gabriele Kahlout
wrote:

> Hello,
>
> I'm having trouble getting Solr 3.1 to work with nutch-1.3.  I'm not sure
> where the problem is, but I'm wondering why does the solrHome path end with
> /./.
>
> cwd=/Applications/NetBeans/apache-tomcat-7.0.6/bin
> SolrHome=/Users/simpatico/apache-solr-3.1.0/solr/./
>
> In the web.xml of solr:
>
>
>solr/home
>
> ${user.home}/apache-solr-3.1.0/solr
>java.lang.String
> 
>
>
> --
> Regards,
> K. Gabriele
>
> --- unchanged since 20/9/10 ---
> P.S. If the subject contains "[LON]" or the addressee acknowledges the
> receipt within 48 hours then I don't resend the email.
> subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧
> time(x) < Now + 48h) ⇒ ¬resend(I, this).
>
> If an email is sent by a sender that is not a trusted contact or the email
> does not contain a valid code then the email is not received. A valid code
> starts with a hyphen and ends with "X".
> ∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
> L(-[a-z]+[0-9]X)).
>
>


-- 
Regards,
K. Gabriele

--- unchanged since 20/9/10 ---
P.S. If the subject contains "[LON]" or the addressee acknowledges the
receipt within 48 hours then I don't resend the email.
subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧ time(x)
< Now + 48h) ⇒ ¬resend(I, this).

If an email is sent by a sender that is not a trusted contact or the email
does not contain a valid code then the email is not received. A valid code
starts with a hyphen and ends with "X".
∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
L(-[a-z]+[0-9]X)).

Re: SolrHome ends with /./ - is this normal?

[Gabriele Kahlout]

>From solr logs:

May 10, 2011 4:33:20 PM org.apache.solr.common.SolrException log
*SEVERE: org.apache.solr.common.SolrException: ERROR:unknown field 'content'
*
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:321)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:147)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:55)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:244)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFilter.java:393)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:244)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:550)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:380)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:243)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:188)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:166)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:288)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)


in conf/schema.xml:
   



**

in conf/solrindex-mapping.xml:



In recent solr I think this has been renamed into text?

Solr's conf/schema.xml:
via copyField further on in this schema  -->
*   *

On Tue, May 10, 2011 at 4:30 PM, Gabriele Kahlout
wrote:

> It apparently is normal, and my issue is indeed with nutch.
>
> I've modified post.sh from the example docs to use the solr in
> http://localhost:8080/apache-solr-3.1-SNAPSHOT and now finally data made
> it to the index.
> $ post.sh solr.xml monitor.xml
>
> With nutch I'm at:
>
> $ svn info
> Path: .
> URL: http://svn.apache.org/repos/asf/nutch/branches/branch-1.3
> Repository Root: http://svn.apache.org/repos/asf
> Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
> Revision: *1101459*
> Node Kind: directory
> Schedule: normal
> Last Changed Author: markus
> Last Changed Rev: 1101280
> Last Changed Date: 2011-05-10 02:46:04 +0200 (Tue, 10 May 2011)
>
> Does this work for you? All I've done is svn co nutch 1.3 and execute my
> script which up to now worked.
>
>
>
> On Tue, May 10, 2011 at 4:11 PM, Gabriele Kahlout <
> gabri...@mysimpatico.com> wrote:
>
>> Hello,
>>
>> I'm having trouble getting Solr 3.1 to work with nutch-1.3.  I'm not sure
>> where the problem is, but I'm wondering why does the solrHome path end with
>> /./.
>>
>> cwd=/Applications/NetBeans/apache-tomcat-7.0.6/bin
>> SolrHome=/Users/simpatico/apache-solr-3.1.0/solr/./
>>
>> In the web.xml of solr:
>>
>>
>>solr/home
>>
>> ${user.home}/apache-solr-3.1.0/solr
>>java.lang.String
>> 
>>
>>
>> --
>> Regards,
>> K. Gabriele
>>
>> --- unchanged since 20/9/10 ---
>> P.S. If the subject contains "[LON]" or the addressee acknowledges the
>> receipt within 48 hours then I don't resend the email.
>> subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧
>> time(x) < Now + 48h) ⇒ ¬resend(I, this).
>>
>> If an email is sent by a sender that is not a trusted contact or the email
>> does not contain a valid code then the email is not received. A valid code
>> starts with a hyphen and ends with "X".
>> ∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
>> L(-[a-z]+[0-9]X)).
>>
>>
>
>
> --
> Regards,
> K. Gabriele
>
> --- unchanged since 20/9/10 ---
> P.S. If the subject contains "[LON]" or the addressee 

Re: SolrHome ends with /./ - is this normal?

[Markus Jelsma]

There is a working example schema in Nutch' conf directory.

On Tuesday 10 May 2011 16:40:02 Gabriele Kahlout wrote:
> From solr logs:
> 
> May 10, 2011 4:33:20 PM org.apache.solr.common.SolrException log
> *SEVERE: org.apache.solr.common.SolrException: ERROR:unknown field
> 'content' *
> at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:321)
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdatePro
> cessorFactory.java:60) at
> org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:147) at
> org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentS
> treamHandlerBase.java:55) at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase
> .java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:
> 356) at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java
> :252) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicatio
> nFilterChain.java:244) at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterC
> hain.java:210) at
> org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFilte
> r.java:393) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicatio
> nFilterChain.java:244) at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterC
> hain.java:210) at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.j
> ava:240) at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.j
> ava:161) at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:16
> 4) at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:10
> 0) at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:550)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.jav
> a:118) at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:380)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:243)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Htt
> p11Protocol.java:188) at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Htt
> p11Protocol.java:166) at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java
> :288) at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.j
> ava:886) at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:
> 908) at java.lang.Thread.run(Thread.java:680)
> 
> 
> in conf/schema.xml:
>
> 
> 
>  required="true"/>
> **
> 
> in conf/solrindex-mapping.xml:
> 
> 
> 
> In recent solr I think this has been renamed into text?
> 
> Solr's conf/schema.xml:
> via copyField further on in this schema  -->
> *multiValued="true"/>*
> 
> On Tue, May 10, 2011 at 4:30 PM, Gabriele Kahlout
> 
> wrote:
> > It apparently is normal, and my issue is indeed with nutch.
> > 
> > I've modified post.sh from the example docs to use the solr in
> > http://localhost:8080/apache-solr-3.1-SNAPSHOT and now finally data made
> > it to the index.
> > $ post.sh solr.xml monitor.xml
> > 
> > With nutch I'm at:
> > 
> > $ svn info
> > Path: .
> > URL: http://svn.apache.org/repos/asf/nutch/branches/branch-1.3
> > Repository Root: http://svn.apache.org/repos/asf
> > Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
> > Revision: *1101459*
> > Node Kind: directory
> > Schedule: normal
> > Last Changed Author: markus
> > Last Changed Rev: 1101280
> > Last Changed Date: 2011-05-10 02:46:04 +0200 (Tue, 10 May 2011)
> > 
> > Does this work for you? All I've done is svn co nutch 1.3 and execute my
> > script which up to now worked.
> > 
> > 
> > 
> > On Tue, May 10, 2011 at 4:11 PM, Gabriele Kahlout <
> > 
> > gabri...@mysimpatico.com> wrote:
> >> Hello,
> >> 
> >> I'm having trouble getting Solr 3.1 to work with nutch-1.3.  I'm not
> >> sure where the problem is, but I'm wondering why does the solrHome path
> >> end with /./.
> >> 
> >> cwd=/Applications/NetBeans/apache-tomcat-7.0.6/bin
> >> SolrHome=/Users/simpatico/apache-solr-3.1.0/solr/./
> >> 
> >> In the web.xml of solr:
> >>
> >>
> >>solr/home
> >> 
> >> ${user.home}/apache-solr-3.1.0/solr
> >> 
> >>java.lang.String
> >> 
> >> 
> >> 
> >> --
> >> Regards,
> >> K. Gabriele
> >> 
> >> --- unchanged since 20/9/10 ---
> >> P.S. If the subject contains "[LON]" or the addressee acknowledges the
> >> receipt within 48 hours then I don't resend the email.
> >> subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧
> >> time(x) < Now + 48h) ⇒ ¬resend(I, this).
> >> 
> >> If an email is 

Re: SolrHome ends with /./ - is this normal?

[Gabriele Kahlout]

I don't get you, are you talking about conf/schema.xml? That's what I'm
referring to. Am i supposed to do something with the nutch's
conf/schema.xml?

On Tue, May 10, 2011 at 4:46 PM, Markus Jelsma
wrote:

> There is a working example schema in Nutch' conf directory.
>
> On Tuesday 10 May 2011 16:40:02 Gabriele Kahlout wrote:
> > From solr logs:
> >
> > May 10, 2011 4:33:20 PM org.apache.solr.common.SolrException log
> > *SEVERE: org.apache.solr.common.SolrException: ERROR:unknown field
> > 'content' *
> > at
> >
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:321)
> > at
> >
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdatePro
> > cessorFactory.java:60) at
> > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:147) at
> > org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
> > at
> >
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentS
> > treamHandlerBase.java:55) at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase
> > .java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:
> > 356) at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java
> > :252) at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicatio
> > nFilterChain.java:244) at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterC
> > hain.java:210) at
> >
> org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFilte
> > r.java:393) at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicatio
> > nFilterChain.java:244) at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterC
> > hain.java:210) at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.j
> > ava:240) at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.j
> > ava:161) at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:16
> > 4) at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:10
> > 0) at
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:550)
> > at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.jav
> > a:118) at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:380)
> > at
> >
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:243)
> > at
> >
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Htt
> > p11Protocol.java:188) at
> >
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Htt
> > p11Protocol.java:166) at
> >
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java
> > :288) at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.j
> > ava:886) at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:
> > 908) at java.lang.Thread.run(Thread.java:680)
> >
> >
> > in conf/schema.xml:
> >
> > 
> > 
> >  > required="true"/>
> > * indexed="true"/>*
> >
> > in conf/solrindex-mapping.xml:
> > 
> > 
> >
> > In recent solr I think this has been renamed into text?
> >
> > Solr's conf/schema.xml:
> > via copyField further on in this schema  -->
> > *> multiValued="true"/>*
> >
> > On Tue, May 10, 2011 at 4:30 PM, Gabriele Kahlout
> >
> > wrote:
> > > It apparently is normal, and my issue is indeed with nutch.
> > >
> > > I've modified post.sh from the example docs to use the solr in
> > > http://localhost:8080/apache-solr-3.1-SNAPSHOT and now finally data
> made
> > > it to the index.
> > > $ post.sh solr.xml monitor.xml
> > >
> > > With nutch I'm at:
> > >
> > > $ svn info
> > > Path: .
> > > URL: http://svn.apache.org/repos/asf/nutch/branches/branch-1.3
> > > Repository Root: http://svn.apache.org/repos/asf
> > > Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
> > > Revision: *1101459*
> > > Node Kind: directory
> > > Schedule: normal
> > > Last Changed Author: markus
> > > Last Changed Rev: 1101280
> > > Last Changed Date: 2011-05-10 02:46:04 +0200 (Tue, 10 May 2011)
> > >
> > > Does this work for you? All I've done is svn co nutch 1.3 and execute
> my
> > > script which up to now worked.
> > >
> > >
> > >
> > > On Tue, May 10, 2011 at 4:11 PM, Gabriele Kahlout <
> > >
> > > gabri...@mysimpatico.com> wrote:
> > >> Hello,
> > >>
> > >> I'm having trouble getting Solr 3.1 to work with nutch-1.3.  I'm not
> > >> sure where the problem is, but I'm wondering why does the solrHome
> path
> > >> end with /./.
> > >>
> > >> cwd=/Applications/NetBeans/apache-tomcat-7.0.6/bin
> > >> SolrHome=/Users/simpatico/apache-solr-3.1.0/solr/./
> > >>
> > >> In the web.xml of solr:
> > >>   

Re: Solr 3.1 Upgrade - Reindex necessary ?

[Ravi Solr]

Thanks Grijesh for responding. I meant that I will use the Lucene 3.1
jars for indexing also from now on. My current index already has a
million docs indexed with solr 1.4.1 version, I read somewhere that
once server is upgraded to 3.1, it is said that the first commit will
change the indexes to 3.1 format automatically. Is this true or do I
have to literally reindex the million docs again ?

Thanks,
Ravi Kiran Bhaskar

On Tuesday, May 10, 2011, Grijesh  wrote:
>>1. Do we have to reindex all content again to use Solr 3.1 ?
>
>>2. If we don't reindex all content are there any potential issues ? (I
>>read somewhere that first commit would change the 1.4.1 format to 3.1.
>>have the analyzer's behavior changed which warrants reindexing ?)
>>3. Apart from deploying the new solr 3.1 war; Is it just enough to set
>>"LUCENE_31"  to get all the
>>goodies and bug fixes of the LUCENE/SOLR 3.1 ?
>
> HI Solr-3.1 version usage the the latest version of Lucene jars so if you
> are planning to Upgrade then it is necessary to Re index all the content
> with Solr3.1 version.
>
> Not re-indexing will possibly cause of index corruption because newer
> version of lucene will create indexes in Newer version which is backward
> compatible for read only.
>
> setting  LUCENE_31 is not enough
> because it will not get the lucene 3.1 jar automatically.
>
> -
> Thanx:
> Grijesh
> www.gettinhahead.co.in
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Solr-3-1-Upgrade-Reindex-necessary-tp2919679p2922645.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: SolrHome ends with /./ - is this normal?

[Markus Jelsma]

You need to use the schema.xml shipped with Nutch in Solr. It provides most 
fields that you need.

On Tuesday 10 May 2011 17:31:33 Gabriele Kahlout wrote:
> I don't get you, are you talking about conf/schema.xml? That's what I'm
> referring to. Am i supposed to do something with the nutch's
> conf/schema.xml?
> 
> On Tue, May 10, 2011 at 4:46 PM, Markus Jelsma
> 
> wrote:
> > There is a working example schema in Nutch' conf directory.
> > 
> > On Tuesday 10 May 2011 16:40:02 Gabriele Kahlout wrote:
> > > From solr logs:
> > > 
> > > May 10, 2011 4:33:20 PM org.apache.solr.common.SolrException log
> > > *SEVERE: org.apache.solr.common.SolrException: ERROR:unknown field
> > > 'content' *
> > > 
> > > at
> > 
> > org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:32
> > 1)
> > 
> > > at
> > 
> > org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateP
> > ro
> > 
> > > cessorFactory.java:60) at
> > > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:147) at
> > > org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
> > > 
> > > at
> > 
> > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Conten
> > tS
> > 
> > > treamHandlerBase.java:55) at
> > 
> > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBa
> > se
> > 
> > > .java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
> > > 
> > > at
> > 
> > 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:
> > > 356) at
> > 
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.ja
> > va
> > 
> > > :252) at
> > 
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> > io
> > 
> > > nFilterChain.java:244) at
> > 
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> > rC
> > 
> > > hain.java:210) at
> > 
> > org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFil
> > te
> > 
> > > r.java:393) at
> > 
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> > io
> > 
> > > nFilterChain.java:244) at
> > 
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> > rC
> > 
> > > hain.java:210) at
> > 
> > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
> > .j
> > 
> > > ava:240) at
> > 
> > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
> > .j
> > 
> > > ava:161) at
> > 
> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
> > 16
> > 
> > > 4) at
> > 
> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
> > 10
> > 
> > > 0) at
> > > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:55
> > > 0)
> > > 
> > > at
> > 
> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
> > av
> > 
> > > a:118) at
> > 
> > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:38
> > 0)
> > 
> > > at
> > 
> > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:243
> > )
> > 
> > > at
> > 
> > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(H
> > tt
> > 
> > > p11Protocol.java:188) at
> > 
> > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(H
> > tt
> > 
> > > p11Protocol.java:166) at
> > 
> > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.ja
> > va
> > 
> > > :288) at
> > 
> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor
> > .j
> > 
> > > ava:886) at
> > 
> > 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:
> > > 908) at java.lang.Thread.run(Thread.java:680)
> > > 
> > > in conf/schema.xml:
> > >
> > >
> > > 
> > >  > > indexed="true"/>  > > indexed="true"
> > > 
> > > required="true"/>
> > > 
> > > * > 
> > indexed="true"/>*
> > 
> > > in conf/solrindex-mapping.xml:
> > > 
> > > 
> > > 
> > > 
> > > In recent solr I think this has been renamed into text?
> > > 
> > > Solr's conf/schema.xml:
> > > via copyField further on in this schema  -->
> > > 
> > > *> > multiValued="true"/>*
> > > 
> > > On Tue, May 10, 2011 at 4:30 PM, Gabriele Kahlout
> > > 
> > > wrote:
> > > > It apparently is normal, and my issue is indeed with nutch.
> > > > 
> > > > I've modified post.sh from the example docs to use the solr in
> > > > http://localhost:8080/apache-solr-3.1-SNAPSHOT and now finally data
> > 
> > made
> > 
> > > > it to the index.
> > > > $ post.sh solr.xml monitor.xml
> > > > 
> > > > With nutch I'm at:
> > > > 
> > > > $ svn info
> > > > Path: .
> > > > URL: http://svn.apache.org/repos/asf/nutch/branches/branch-1.3
> > > > Repository Root: http://svn.apache.org/repos/asf
> > > > Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
> > > > Revision: *1101459*
> > > > Node Kind: directory
> > > > Schedule

Re: SolrHome ends with /./ - is this normal?

[Gabriele Kahlout]

You mean that I should copy it from nutch into solr?

$ cp $NUTCH_HOME/conf/schema.xml $SOLR_HOME/conf/schema.xml

After restarting tomcat, and re-executing the script nothing changed.

On Tue, May 10, 2011 at 5:35 PM, Markus Jelsma
wrote:

> You need to use the schema.xml shipped with Nutch in Solr. It provides most
> fields that you need.
>
> On Tuesday 10 May 2011 17:31:33 Gabriele Kahlout wrote:
> > I don't get you, are you talking about conf/schema.xml? That's what I'm
> > referring to. Am i supposed to do something with the nutch's
> > conf/schema.xml?
> >
> > On Tue, May 10, 2011 at 4:46 PM, Markus Jelsma
> >
> > wrote:
> > > There is a working example schema in Nutch' conf directory.
> > >
> > > On Tuesday 10 May 2011 16:40:02 Gabriele Kahlout wrote:
> > > > From solr logs:
> > > >
> > > > May 10, 2011 4:33:20 PM org.apache.solr.common.SolrException log
> > > > *SEVERE: org.apache.solr.common.SolrException: ERROR:unknown field
> > > > 'content' *
> > > >
> > > > at
> > >
> > >
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:32
> > > 1)
> > >
> > > > at
> > >
> > >
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateP
> > > ro
> > >
> > > > cessorFactory.java:60) at
> > > > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:147)
> at
> > > > org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
> > > >
> > > > at
> > >
> > >
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Conten
> > > tS
> > >
> > > > treamHandlerBase.java:55) at
> > >
> > >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBa
> > > se
> > >
> > > > .java:129) at
> org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
> > > >
> > > > at
> > >
> > >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:
> > > > 356) at
> > >
> > >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.ja
> > > va
> > >
> > > > :252) at
> > >
> > >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> > > io
> > >
> > > > nFilterChain.java:244) at
> > >
> > >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> > > rC
> > >
> > > > hain.java:210) at
> > >
> > >
> org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFil
> > > te
> > >
> > > > r.java:393) at
> > >
> > >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> > > io
> > >
> > > > nFilterChain.java:244) at
> > >
> > >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> > > rC
> > >
> > > > hain.java:210) at
> > >
> > >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
> > > .j
> > >
> > > > ava:240) at
> > >
> > >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
> > > .j
> > >
> > > > ava:161) at
> > >
> > >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
> > > 16
> > >
> > > > 4) at
> > >
> > >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
> > > 10
> > >
> > > > 0) at
> > > >
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:55
> > > > 0)
> > > >
> > > > at
> > >
> > >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
> > > av
> > >
> > > > a:118) at
> > >
> > >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:38
> > > 0)
> > >
> > > > at
> > >
> > >
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:243
> > > )
> > >
> > > > at
> > >
> > >
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(H
> > > tt
> > >
> > > > p11Protocol.java:188) at
> > >
> > >
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(H
> > > tt
> > >
> > > > p11Protocol.java:166) at
> > >
> > >
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.ja
> > > va
> > >
> > > > :288) at
> > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor
> > > .j
> > >
> > > > ava:886) at
> > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:
> > > > 908) at java.lang.Thread.run(Thread.java:680)
> > > >
> > > > in conf/schema.xml:
> > > >
> > > >
> > > > 
> > > >  > > > indexed="true"/>  > > > indexed="true"
> > > >
> > > > required="true"/>
> > > >
> > > > * > >
> > > indexed="true"/>*
> > >
> > > > in conf/solrindex-mapping.xml:
> > > > 
> > > >
> > > > 
> > > >
> > > > In recent solr I think this has been renamed into text?
> > > >
> > > > Solr's conf/schema.xml:
> > > > via copyField further on in this schema  -->
> > > >
> > > > *> > > multiValued="true"/>*
> > > >
> > > > On Tue, May 10, 2011 at 4:30 PM, Gabriele Kahlout
> > > >
> > > > wrote:
> > > > > It apparently is normal, and my issue is indeed with nutch.
> > > > >
> > > > > I've

Re: SolrHome ends with /./ - is this normal?

[Gabriele Kahlout]

actually something changed, I managed to crawl and index some pages (the
other must have to do with regex-urls). Thank you!

Was this always necessary? Any pointer discussing why it's needed?

On Tue, May 10, 2011 at 5:40 PM, Gabriele Kahlout
wrote:

> You mean that I should copy it from nutch into solr?
>
> $ cp $NUTCH_HOME/conf/schema.xml $SOLR_HOME/conf/schema.xml
>
> After restarting tomcat, and re-executing the script nothing changed.
>
>
> On Tue, May 10, 2011 at 5:35 PM, Markus Jelsma  > wrote:
>
>> You need to use the schema.xml shipped with Nutch in Solr. It provides
>> most
>> fields that you need.
>>
>> On Tuesday 10 May 2011 17:31:33 Gabriele Kahlout wrote:
>> > I don't get you, are you talking about conf/schema.xml? That's what I'm
>> > referring to. Am i supposed to do something with the nutch's
>> > conf/schema.xml?
>> >
>> > On Tue, May 10, 2011 at 4:46 PM, Markus Jelsma
>> >
>> > wrote:
>> > > There is a working example schema in Nutch' conf directory.
>> > >
>> > > On Tuesday 10 May 2011 16:40:02 Gabriele Kahlout wrote:
>> > > > From solr logs:
>> > > >
>> > > > May 10, 2011 4:33:20 PM org.apache.solr.common.SolrException log
>> > > > *SEVERE: org.apache.solr.common.SolrException: ERROR:unknown field
>> > > > 'content' *
>> > > >
>> > > > at
>> > >
>> > >
>> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:32
>> > > 1)
>> > >
>> > > > at
>> > >
>> > >
>> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateP
>> > > ro
>> > >
>> > > > cessorFactory.java:60) at
>> > > > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:147)
>> at
>> > > > org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
>> > > >
>> > > > at
>> > >
>> > >
>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Conten
>> > > tS
>> > >
>> > > > treamHandlerBase.java:55) at
>> > >
>> > >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBa
>> > > se
>> > >
>> > > > .java:129) at
>> org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
>> > > >
>> > > > at
>> > >
>> > >
>>
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:
>> > > > 356) at
>> > >
>> > >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.ja
>> > > va
>> > >
>> > > > :252) at
>> > >
>> > >
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
>> > > io
>> > >
>> > > > nFilterChain.java:244) at
>> > >
>> > >
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
>> > > rC
>> > >
>> > > > hain.java:210) at
>> > >
>> > >
>> org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFil
>> > > te
>> > >
>> > > > r.java:393) at
>> > >
>> > >
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
>> > > io
>> > >
>> > > > nFilterChain.java:244) at
>> > >
>> > >
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
>> > > rC
>> > >
>> > > > hain.java:210) at
>> > >
>> > >
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
>> > > .j
>> > >
>> > > > ava:240) at
>> > >
>> > >
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
>> > > .j
>> > >
>> > > > ava:161) at
>> > >
>> > >
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
>> > > 16
>> > >
>> > > > 4) at
>> > >
>> > >
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
>> > > 10
>> > >
>> > > > 0) at
>> > > >
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:55
>> > > > 0)
>> > > >
>> > > > at
>> > >
>> > >
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
>> > > av
>> > >
>> > > > a:118) at
>> > >
>> > >
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:38
>> > > 0)
>> > >
>> > > > at
>> > >
>> > >
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:243
>> > > )
>> > >
>> > > > at
>> > >
>> > >
>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(H
>> > > tt
>> > >
>> > > > p11Protocol.java:188) at
>> > >
>> > >
>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(H
>> > > tt
>> > >
>> > > > p11Protocol.java:166) at
>> > >
>> > >
>> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.ja
>> > > va
>> > >
>> > > > :288) at
>> > >
>> > >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor
>> > > .j
>> > >
>> > > > ava:886) at
>> > >
>> > >
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:
>> > > > 908) at java.lang.Thread.run(Thread.java:680)
>> > > >
>> > > > in conf/schema.xml:
>> > > >
>> > > >
>> > > > > indexed="true"/>
>> > > > > > > > indexed="true"/> > > > > indexed="true"
>> > > >
>> > > > required="true"/>
>> > > >
>> > > > *> > >
>> > > indexed="true"/>*
>> > >
>> >

Re: SolrHome ends with /./ - is this normal?

[Markus Jelsma]

Yes, that schema is configured for the fields Nutch can generate. It's 
recommended to remove the Solr index after changing the schema. Then you must 
reindex. The 

On Tuesday 10 May 2011 17:40:39 Gabriele Kahlout wrote:
> You mean that I should copy it from nutch into solr?
> 
> $ cp $NUTCH_HOME/conf/schema.xml $SOLR_HOME/conf/schema.xml
> 
> After restarting tomcat, and re-executing the script nothing changed.
> 
> On Tue, May 10, 2011 at 5:35 PM, Markus Jelsma
> 
> wrote:
> > You need to use the schema.xml shipped with Nutch in Solr. It provides
> > most fields that you need.
> > 
> > On Tuesday 10 May 2011 17:31:33 Gabriele Kahlout wrote:
> > > I don't get you, are you talking about conf/schema.xml? That's what I'm
> > > referring to. Am i supposed to do something with the nutch's
> > > conf/schema.xml?
> > > 
> > > On Tue, May 10, 2011 at 4:46 PM, Markus Jelsma
> > > 
> > > wrote:
> > > > There is a working example schema in Nutch' conf directory.
> > > > 
> > > > On Tuesday 10 May 2011 16:40:02 Gabriele Kahlout wrote:
> > > > > From solr logs:
> > > > > 
> > > > > May 10, 2011 4:33:20 PM org.apache.solr.common.SolrException log
> > > > > *SEVERE: org.apache.solr.common.SolrException: ERROR:unknown field
> > > > > 'content' *
> > > > > 
> > > > > at
> > 
> > org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:32
> > 
> > > > 1)
> > > > 
> > > > > at
> > 
> > org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateP
> > 
> > > > ro
> > > > 
> > > > > cessorFactory.java:60) at
> > > > > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:147)
> > 
> > at
> > 
> > > > > org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
> > > > > 
> > > > > at
> > 
> > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Conten
> > 
> > > > tS
> > > > 
> > > > > treamHandlerBase.java:55) at
> > 
> > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBa
> > 
> > > > se
> > > > 
> > > > > .java:129) at
> > 
> > org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
> > 
> > > > > at
> > 
> > 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:
> > > > > 356) at
> > 
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.ja
> > 
> > > > va
> > > > 
> > > > > :252) at
> > 
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> > 
> > > > io
> > > > 
> > > > > nFilterChain.java:244) at
> > 
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> > 
> > > > rC
> > > > 
> > > > > hain.java:210) at
> > 
> > org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFil
> > 
> > > > te
> > > > 
> > > > > r.java:393) at
> > 
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> > 
> > > > io
> > > > 
> > > > > nFilterChain.java:244) at
> > 
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> > 
> > > > rC
> > > > 
> > > > > hain.java:210) at
> > 
> > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
> > 
> > > > .j
> > > > 
> > > > > ava:240) at
> > 
> > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
> > 
> > > > .j
> > > > 
> > > > > ava:161) at
> > 
> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
> > > > 16
> > > > 
> > > > > 4) at
> > 
> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
> > > > 10
> > > > 
> > > > > 0) at
> > 
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:55
> > 
> > > > > 0)
> > > > > 
> > > > > at
> > 
> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
> > 
> > > > av
> > > > 
> > > > > a:118) at
> > 
> > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:38
> > 
> > > > 0)
> > > > 
> > > > > at
> > 
> > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:243
> > 
> > > > )
> > > > 
> > > > > at
> > 
> > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(H
> > 
> > > > tt
> > > > 
> > > > > p11Protocol.java:188) at
> > 
> > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(H
> > 
> > > > tt
> > > > 
> > > > > p11Protocol.java:166) at
> > 
> > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.ja
> > 
> > > > va
> > > > 
> > > > > :288) at
> > 
> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor
> > 
> > > > .j
> > > > 
> > > > > ava:886) at
> > 
> > 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:
> > > > > 908) at java.lang.Thread.run(Thread.java:680)
> > > > > 
> > > > > in conf/schema.xml:
> > > > >
> > > > >
> > > > >  > > > > indexed="true"/>  > > > > stored="false"
> > > > > indexed="true"/>  > > > > indexed="true"
> > > > > 
> > > > > required="true"/>
>

Re: Solr security

[Brian Lamb]

Great posts all. I will give these a look and come up with something based
on these recommendations. I'm sure as I begin implementing something, I will
have more questions arise.

On Tue, May 10, 2011 at 9:00 AM, Anthony Wlodarski <
anth...@tinkertownlabs.com> wrote:

> The WIKI has a loose interpretation of how to set-up Jetty securely.
>  Please take a look at the article I wrote here:
> http://anthonyw.net/2011/04/securing-jetty-and-solr-with-php-authentication/.
>  Even if PHP is not your language that sits on top of Solr you can still use
> the first part of the tutorial.  If you are using Tomcat I would recommend
> looking here:
> http://blog.comtaste.com/2009/02/securing_your_solr_server_on_t.html
>
> Regards,
>
> -Anthony
>
>
> On 05/09/2011 05:28 PM, Jan Høydahl wrote:
>
>> Hi,
>>
>> You can simply configure a firewall on your Solr server to only allow
>> access from your frontend server. Whether you use the built-in software
>> firewall of Linux/Windows/Whatever or use some other FW utility is a choice
>> you need to make. This is by design - you should never ever expose your
>> backend services, whether it's a search server or a database server, to the
>> public.
>>
>> Read more about Solr security on the WIKI:
>> http://wiki.apache.org/solr/SolrSecurity
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>> On 9. mai 2011, at 20.57, Brian Lamb wrote:
>>
>>  Hi all,
>>>
>>> Is it possible to set up solr so that it will only execute dataimport
>>> commands if they come from localhost?
>>>
>>> Right now, my application and my solr installation are on different
>>> servers
>>> so any requests are formatted http://domain:8983 instead of
>>> http://localhost:8983. I am concerned that when I launch my application,
>>> there will be the potential for abuse. Is the best solution to have
>>> everything reside on the same server?
>>>
>>> What are some other solutions?
>>>
>>> Thanks,
>>>
>>> Brian Lamb
>>>
>>
> --
> Anthony Wlodarski
> Lead Software Engineer
> Get2Know.me (http://www.get2know.me)
> Office: 646-285-0500 x217
> Fax: 646-285-0400
>
>

RE: SolrQuery API for adding group filter

[arian487]

I'm actually using php but I get what you're saying.  I think I understand
what I need to do.  Thanks a lot man!

--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrQuery-API-for-adding-group-filter-tp2921539p2923701.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Synonym Filter disable at query time

[mtraynham]

Just a heads up on a solution.

copyField wasn't need, but a new fieldType and a non-indexed, non-stored
field was added.

Within a new Synonym processor that executes right before the
AnalyzerQueryNodeProcessor, I was able to modify the field name for each
node to point at the new field.  Therefore I could build out the necessary
synonym values from the tokenizer and then reassign them all back to the
original field with whatever boosts they needed.  This allowed me to retain
the original value match, to keep it's boost at 1 and then boost the
synonyms according to a user specified boost value.  Works perfectly.

Thanks again for the help.

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Synonym-Filter-disable-at-query-time-tp2919876p2923775.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Replication Clarification Please

[Ravi Solr]

Hello Mr. Kanarsky,
Thank you very much for the detailed explanation,
probably the best explanation I found regarding replication. Just to
be sure, I wanted to test solr 3.1 to see if it alleviates the
problems...I dont think it helped. The master index version and
generation are greater than the slave, still the slave replicates the
entire index form master (see replication admin screen output below).
Any idea why it would get the whole index everytime even in 3.1 or am
I misinterpreting the output ? However I must admit that 3.1 finished
the replication unlike 1.4.1 which would hang and be backed up for
ever.

Master  http://masterurl:post/solr-admin/searchcore/replication
Latest Index Version:null, Generation: null
Replicatable Index Version:1296217097572, Generation: 12726

Poll Interval   00:03:00

Local Index Index Version: 1296217097569, Generation: 12725

Location: /data/solr/core/search-data/index
Size: 944.32 MB
Times Replicated Since Startup: 148
Previous Replication Done At: Tue May 10 12:32:42 EDT 2011
Config Files Replicated At: null
Config Files Replicated: null
Times Config Files Replicated Since Startup: null
Next Replication Cycle At: Tue May 10 12:35:41 EDT 2011

Current Replication Status  Start Time: Tue May 10 12:32:41 EDT 2011
Files Downloaded: 18 / 108
Downloaded: 317.48 KB / 436.24 MB [0.0%]
Downloading File: _ayu.nrm, Downloaded: 4 bytes / 4 bytes [100.0%]
Time Elapsed: 17s, Estimated Time Remaining: 23902s, Speed: 18.67 KB/s


Thanks,
Ravi Kiran Bhaskar

On Tue, May 10, 2011 at 4:10 AM, Alexander Kanarsky
 wrote:
> Ravi,
>
> as far as I remember, this is how the replication logic works (see
> SnapPuller class, fetchLatestIndex method):
>
>> 1. Does the Slave get the whole index every time during replication or
>> just the delta since the last replication happened ?
>
>
> It look at the index version AND the index generation. If both slave's
> version and generation are the same as on master, nothing gets
> replicated. if the master's generation is greater than on slave, the
> slave fetches the delta files only (even if the partial merge was done
> on the master) and put the new files from master to the same index
> folder on slave (either index or index., see further
> explanation). However, if the master's index generation is equals or
> less than one on slave, the slave does the full replication by
> fetching all files of the master's index and place them into a
> separate folder on slave (index.). Then, if the fetch is
> successfull, the slave updates (or creates) the index.properties file
> and puts there the name of the "current" index folder. The "old"
> index. folder(s) will be kept in 1.4.x - which was treated
> as a bug - see SOLR-2156 (and this was fixed in 3.1). After this, the
> slave does commit or reload core depending whether the config files
> were replicated. There is another bug in 1.4.x that fails replication
> if the slave need to do the full replication AND the config files were
> changed - also fixed in 3.1 (see SOLR-1983).
>
>> 2. If there are huge number of queries being done on slave will it
>> affect the replication ? How can I improve the performance ? (see the
>> replications details at he bottom of the page)
>
>
> >From my experience the half of the replication time is a time when the
> transferred data flushes to the disk. So the IO impact is important.
>
>> 3. Will the segment names be same be same on master and slave after
>> replication ? I see that they are different. Is this correct ? If it
>> is correct how does the slave know what to fetch the next time i.e.
>> the delta.
>
>
> They should be the same. The slave fetches the changed files only (see
> above), also look at SnapPuller code.
>
>> 4. When and why does the index. folder get created ? I see
>> this type of folder getting created only on slave and the slave
>> instance is pointing to it.
>
>
> See above.
>
>> 5. Does replication process copy both the index and index.
> folder ?
>
>
> index. folder gets created only of the full replication
> happened at least once. Otherwise, the slave will use the index
> folder.
>
>> 6. what happens if the replication kicks off even before the previous
>> invocation has not completed ? will the 2nd invocation block or will
>> it go through causing more confusion ?
>
>
> There is a lock (snapPullLock in ReplicationHandler) that prevents two
> replications run simultaneously. If there is no bug, it should just
> return silently from the replication call. (I personally never had
> problem with this so it looks there is no bug :)
>
>> 7. If I have to prep a new master-slave combination is it OK to copy
>> the respective contents into the new master-slave and start solr ? or
>> do I have have to wipe the new slave and let it replicate from its new
>> master ?
>
>
> If the new master has a different index, the slave will cr

RE: Synonym Filter disable at query time

[Robert Petersen]

Very nice! Good job! :)

-Original Message-
From: mtraynham [mailto:mtrayn...@digitalsmiths.com] 
Sent: Tuesday, May 10, 2011 9:44 AM
To: solr-user@lucene.apache.org
Subject: RE: Synonym Filter disable at query time

Just a heads up on a solution.

copyField wasn't need, but a new fieldType and a non-indexed, non-stored
field was added.

Within a new Synonym processor that executes right before the
AnalyzerQueryNodeProcessor, I was able to modify the field name for each
node to point at the new field.  Therefore I could build out the
necessary
synonym values from the tokenizer and then reassign them all back to the
original field with whatever boosts they needed.  This allowed me to
retain
the original value match, to keep it's boost at 1 and then boost the
synonyms according to a user specified boost value.  Works perfectly.

Thanks again for the help.

--
View this message in context:
http://lucene.472066.n3.nabble.com/Synonym-Filter-disable-at-query-time-
tp2919876p2923775.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Boosting score of a document without deleting and adding another document

[Michael McCandless]

Lucene also lets you change the norms directly, from an IndexReader.

In theory this could do what you're seeking, I believe.  But it's
rather expert... and I don't think it's exposed through to Solr (?).

Mike

http://blog.mikemccandless.com

On Tue, May 10, 2011 at 9:39 AM, Ahmet Arslan  wrote:
>
>> What I want to do is : everytime an item is searched, I get
>> the ID of that document, and now I want to increase this
>> document's score.Function queries at query time won't really
>> help with this. Any way to do this other than adding the
>> entire document again?
>
> Looks like you need ExternalFileField.
>
> http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html
>

Solrj Questions

[Rohit]

Hi,

 

I am trying to use solrj for my application, my code is given below,

 

query.add("q", "simplify360");

query.add("facet", "true");

query.add("facet.range", "createdOnGMTDate");

query.add("facet.range.start",
"2010-08-01T00:00:00Z+330MINUTES");

query.add("facet.range.end", "2011-05-31T00:00:00Z+330MINUTES");

query.add("facet.range.gap", "+1DAY");

//query.add("wt","json");

//query.add("wt.mime-type","application/json");



System.err.println(query.toString());

 

The code executes fine and when i execute the url on solr server, i get the
following result for faceting,

 













0

0

0

0

0

0

+1DAY

2010-01-01T00:00:00Z

2011-05-31T00:00:00Z









 

 

1) How can i retrieve these values in java, 

2) Also if there is anyway i can convert the json response to the json java
object

 

 

Regards,

Rohit

Re: edismax available in solr 3.1?

[Jan Høydahl]

It's already much better than the old dismax, and probably more stable.
If you do not require the ability to lock down what fields people should be 
allowed to search for, I suggest you start using it right away. If you require 
to lock that down, I suggest you help out finishing that part of eDismax and 
then use it :)

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

On 10. mai 2011, at 01.05, cyang2010 wrote:

> Is it a formal feature that solr 3.1 support?  Or still as experimental
> feature?  If it is experimental feature, i would still be hesitating to use
> it.
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/edismax-available-in-solr-3-1-tp2910613p2920975.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: edismax available in solr 3.1?

[Markus Jelsma]

Jan, is there already an issue for that?

> It's already much better than the old dismax, and probably more stable.
> If you do not require the ability to lock down what fields people should be
> allowed to search for, I suggest you start using it right away. If you
> require to lock that down, I suggest you help out finishing that part of
> eDismax and then use it :)
> 
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
> 
> On 10. mai 2011, at 01.05, cyang2010 wrote:
> > Is it a formal feature that solr 3.1 support?  Or still as experimental
> > feature?  If it is experimental feature, i would still be hesitating to
> > use it.
> > 
> > --
> > View this message in context:
> > http://lucene.472066.n3.nabble.com/edismax-available-in-solr-3-1-tp29106
> > 13p2920975.html Sent from the Solr - User mailing list archive at
> > Nabble.com.

Re: Total Documents Failed : How to find out why

[Erick Erickson]

OK, then your log is probably just coming out to the console. you can
start it as "java -jar start.jar > file.log 2>&1" and keep a permanent
record of the log if you're on windows

Best
Erick

On Mon, May 9, 2011 at 7:32 AM, Rohit  wrote:
> Hi Erick,
>
> Thats exactly how I am starting solr.
>
> Regards,
> Rohit
>
> -Original Message-
> From: Erick Erickson [mailto:erickerick...@gmail.com]
> Sent: 09 May 2011 16:57
> To: solr-user@lucene.apache.org
> Subject: Re: Total Documents Failed : How to find out why
>
> First you need to find your logs. That folder should not
> be empty regardless of whether DIH is working correctly
> or not.
>
> I'm assuming here that you're just doing the "java -jar star.jar"
> in the example directory, if this isn't the case how are you
> starting Solr/Jetty?
>
> Best
> Erick
>
> On Mon, May 9, 2011 at 3:26 AM, Rohit  wrote:
>> Hi,
>>
>> I am running the solr index and post indexing I get these results, how can
> I
>> know which documents failed and why?
>>
>> 1
>> 5170850
>> 0
>> 2011-05-08 23:40:09
>> Indexing completed. Added/Updated: 2972300 documents. Deleted
> 0
>> documents.
>> 2011-05-09 00:13:48
>> 2011-05-09 00:13:48
>> 2972300
>> 2198550
>> 0:33:40.945
>>
>> Running solr on jetty right now and the console shows no error, also "
>> \Solr\example\logs " folder is empty.
>>
>> Thanks,
>> Rohit
>>
>>
>>
>>
>
>

Re: edismax available in solr 3.1?

[Chris Hostetter]

: It's already much better than the old dismax, and probably more stable. 
: If you do not require the ability to lock down what fields people should 
: be allowed to search for, I suggest you start using it right away. If 
: you require to lock that down, I suggest you help out finishing that 
: part of eDismax and then use it :)

Right ... the reason it's marked "experimental" in 3.1 is because there is  
are still open questions as to what the "default" behavior should be, 
and what new params should be added to change that behavior, which may 
change between now and whenever edismax is no longer marked "experimental"

But the code most certainly works, and works well -- otherwise we wouldn't 
have included it in 3.1...

https://issues.apache.org/jira/browse/SOLR-2058
https://issues.apache.org/jira/browse/SOLR-2368


-Hoss

RE: SolrQuery API for adding group filter

[arian487]

I actually have another question unrelated to this (but related to grouping). 
I'm wondering if I can do a more complex grouping, such as grouping by a
field and also making sure it matches some other criteria (such as date). 
For example, currently it might group 5 items from some field, but the 5th
item for example is from a really far date which I don't want grouped with
these more recent items.  

Basically I want it to look like this:

Group1 all has common field 'x' and ALSO is items from today
Group2 all has common field 'x' again but now its items are from yesterday,
etc...

I'm having trouble figuring out how that'd work, any help would be
appreciated!

--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrQuery-API-for-adding-group-filter-tp2921539p2924232.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 3.1 Upgrade - Reindex necessary ?

[Chris Hostetter]

: Thanks Grijesh for responding. I meant that I will use the Lucene 3.1
: jars for indexing also from now on. My current index already has a
: million docs indexed with solr 1.4.1 version, I read somewhere that
: once server is upgraded to 3.1, it is said that the first commit will
: change the indexes to 3.1 format automatically. Is this true or do I
: have to literally reindex the million docs again ?

index versioning happens on a segment basis, so once you start using Solr 
3.1, as new docs are added and segments are merged those segments will be 
updated to the new file format -- the way to ensure that "all" segments 
are updated is to optimize your index.

: >>1. Do we have to reindex all content again to use Solr 3.1 ?

you should not need to, know.

: >>3. Apart from deploying the new solr 3.1 war; Is it just enough to set
: >>"LUCENE_31"  to get all the
: >>goodies and bug fixes of the LUCENE/SOLR 3.1 ?

It's not mandatory to change the  to upgrade -- if 
you do want to change the  then you should reindex, 
as that change causes analyzers/query parsers to behave differently (in 
ways thta might be incompatible with how they behave previously.

this change is unrelated to the index fileformat -- optimizing your index 
to force the 3.1 fileformat has no impact on how what esoteric/broken 
behavior a tokenizer might have had in the past that changed once the 
 setting is updated.

The purpose of  is to say "i want the behavior of 
X.Y, even when it's been decided that that behavior was bad, because it's 
what matches the terms i've already indexed"


-Hoss

RE: Solr 3.1 / Java 1.5: Exception regarding analyzer implementation

[Chris Hostetter]

: The reason why this assert hits you in one of your tomcat installations
: could also be related to some instrumentation tools you have enabled in this
: tomcat. Lot's of instrumentation tools may dynamically change class bytecode
: and e.g. make them unfinal. In that case the assertion of course fails (with
: assertions enabled). Before saying Solr 3.1 is not compatible with Java 1.5:

It's also possible that whatever packaging of Java you are using; and the 
way your are running tomcat; is subtlely differnet between your Java5 and 
Java6 installs such that Java5 is run with assertions by default but Java6 
is not.

-Hoss

Re: Total Documents Failed : How to find out why

[Chris Hostetter]

: Subject: Total Documents Failed : How to find out why
: References: <96637.83834...@web121717.mail.ne1.yahoo.com>
:  <1304627391.29391.156.ca...@lysdexic.healthline.com>
: In-Reply-To: <1304627391.29391.156.ca...@lysdexic.healthline.com>


http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists

When starting a new discussion on a mailing list, please do not reply to 
an existing message, instead start a fresh email.  Even if you change the 
subject line of your email, other mail headers still track which thread 
you replied to and your question is "hidden" in that thread and gets less 
attention.   It makes following discussions in the mailing list archives 
particularly difficult.


-Hoss

Re: Controlling webapp startup

[Chris Hostetter]

: 2) Use the usual Solr servlet apparatus, but defer its startup until
: other code in the webapp makes up its mind about configuration and
: calls System.setProperty to locate the solr home and data directories.

I would suggest that route ... basically just create your own war, and 
subclass SolrDispatchFilter, do whatever configuration processing you want 
in your init(FilterConfig) method before delegating to super.init(...)


-Hoss

Multiple character wildcard searches looks for 0 or more characters

[Satyam Kumar Asis]

Hello,

I am using solr search application, try to search text with wildcard
character getting some weird result.

Case :- 1 q=tes*

Result: test,  test1, test2

Case:- 1 q=test*

Result: NONE

Kindly help me to solve above problem.

Thanks & Regards,
Satyam

Re: Solrj Questions

[Chris Hostetter]

: Subject: Solrj Questions
: References: 
: <19898929-d0b9-47fb-b62a-5fd0437a8...@cominvent.com>
: <4dc936de.9050...@tinkertownlabs.com>
:  
: In-Reply-To: 

http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists

When starting a new discussion on a mailing list, please do not reply to 
an existing message, instead start a fresh email.  Even if you change the 
subject line of your email, other mail headers still track which thread 
you replied to and your question is "hidden" in that thread and gets less 
attention.   It makes following discussions in the mailing list archives 
particularly difficult.


-Hoss

Re: Solr 3.1 Upgrade - Reindex necessary ?

[Ravi Solr]

Hoss,
 Thank you very much for clearly delineating the difference.
Just to be clear - My intent to move to 3.1 was driven by my desire to
improve my replication performance - Deducing from your explanation, I
believe the replication/indexing related changes/bug fixes like the
following will be available to me even without specifying
"LUCENE_31" am I right ??

faster exact PhraseQuery; merging favors segments with deletions;
primary key lookup is faster; IndexWriter.addIndexes(Directory[]) uses
file copy instead of merging; various Directory performance
improvements; compound file is dynamically turned off for large
segments; fully deleted segments are dropped on commit; faster
snowball analyzers (in contrib); ConcurrentMergeScheduler is more
careful about setting priority of merge threads.

Ravi Kiran Bhaskar

On Tue, May 10, 2011 at 2:49 PM, Chris Hostetter
 wrote:
>
> : Thanks Grijesh for responding. I meant that I will use the Lucene 3.1
> : jars for indexing also from now on. My current index already has a
> : million docs indexed with solr 1.4.1 version, I read somewhere that
> : once server is upgraded to 3.1, it is said that the first commit will
> : change the indexes to 3.1 format automatically. Is this true or do I
> : have to literally reindex the million docs again ?
>
> index versioning happens on a segment basis, so once you start using Solr
> 3.1, as new docs are added and segments are merged those segments will be
> updated to the new file format -- the way to ensure that "all" segments
> are updated is to optimize your index.
>
> : >>1. Do we have to reindex all content again to use Solr 3.1 ?
>
> you should not need to, know.
>
> : >>3. Apart from deploying the new solr 3.1 war; Is it just enough to set
> : >>"LUCENE_31"  to get all the
> : >>goodies and bug fixes of the LUCENE/SOLR 3.1 ?
>
> It's not mandatory to change the  to upgrade -- if
> you do want to change the  then you should reindex,
> as that change causes analyzers/query parsers to behave differently (in
> ways thta might be incompatible with how they behave previously.
>
> this change is unrelated to the index fileformat -- optimizing your index
> to force the 3.1 fileformat has no impact on how what esoteric/broken
> behavior a tokenizer might have had in the past that changed once the
>  setting is updated.
>
> The purpose of  is to say "i want the behavior of
> X.Y, even when it's been decided that that behavior was bad, because it's
> what matches the terms i've already indexed"
>
>
> -Hoss

Re: how to do offline adding/updating index

[Mike Sokolov]

I think the key question here is what's the best way to perform indexing 
without affecting search performance, or without affecting it much.  If 
you have a batch of documents to index (say a daily batch that takes an 
hour to index and merge), you'd like to do that on an offline system, 
and then when ready, bring that index up for searching.  but using 
Lucene's multiple commit points assumes you use the same box for search 
and indexing doesn't it?


Something like this is what I have in mind (simple 2-server config here):

Box 1 is live and searching
Box 2 is offline and ready to index

loading begins on Box 2...
loading complete on Box 2 ...
commit, optimize

Swap Box 1 and Box 2 ( with a load balancer or application config?)
Box 2 is live and searching
Box 1 is offline and ready to index

To make the best use of your resources, you'd then like to start using 
Box 1 for searching (until indexing starts up again).  Perhaps if your 
load balancing is clever enough, it could be sensitive to the decreased 
performance of the indexing box and just send more requests to the other 
one(s).  That's probably ideal.


-Mike S


Under the hood, Lucene can support this by keeping multiple commit
points in the index.

So you'd make a new commit whenever you finish indexing the updates
from each hour, and record that this is the last "searchable" commit.

Then you are free to commit while indexing the next hour's worth of
changes, but these commits are not marked as searchable.

But... this is a low level Lucene capability and I don't know of any
plans for Solr to support multiple commit points in the index.

Mike

http://blog.mikemccandless.com

On Tue, May 10, 2011 at 9:22 AM, vrpar...@gmail.com  wrote:
   

Hello all,

indexing with dataimporthandler runs every hour (new records will be added,
some records will be updated) note :large data

requirement is when indexing is in progress, searching (on already indexed
data) should not affect

so should i use multicore-with merge and swap or delta query or any other
way?

Thanks

--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-do-offline-adding-updating-index-tp2923035p2923035.html
Sent from the Solr - User mailing list archive at Nabble.com.

 

Re: Multiple character wildcard searches looks for 0 or more characters

[Ahmet Arslan]

> I am using solr search application, try to search text with
> wildcard
> character getting some weird result.
> 
> Case :- 1 q=tes*
> 
> Result: test,  test1, test2
> 
> Case:- 1 q=test*
> 
> Result: NONE
> 
> Kindly help me to solve above problem.

Do you have stemming filter defined in your analyzer definition? It may be 
reducing "test" to "tes".

Re: how to do offline adding/updating index

[kenf_nc]

Master/slave replication does this out of the box, easily. Just set the slave
to update on Optimize only. Then you can update the master as much as you
want. When you are ready to update the slave (the search instance), just
optimize the master. On the slave's next cycle check it will refresh itself,
quickly, efficiently, minimal impact to search performance. No need to build
extra moving parts for swapping search servers or anything like that.

--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-do-offline-adding-updating-index-tp2923035p2924426.html
Sent from the Solr - User mailing list archive at Nabble.com.

Test Post

[Bob Sandiford]

Hi, all.

Sorry for the 'spam' - I'm just testing that my posts are actually being seen.  
I've sent a few queries over the past couple of weeks and haven't had a single 
response :(

Anyways - if one or two would respond to this, I'd appreciate it - just to let 
me know that I'm being ignored, vs unseen :)

Thanks!

Bob Sandiford | Lead Software Engineer | SirsiDynix
P: 800.288.8020 X6943 | bob.sandif...@sirsidynix.com
www.sirsidynix.com
Join the conversation - you may even get an iPad or Nook out of it!

[cid:image002.jpg@01CC0F1E.A9E7FB90]Like us 
on Facebook!

[cid:image004.jpg@01CC0F1E.A9E7FB90]Follow us 
on Twitter!

Re: Test Post

[Jasneet Sabharwal]

Hi Bob

I just joined Solr Mailing List and I can see your post :)

Hope your query get solved.

Regards
Jasneet Sabharwal
On 11-05-2011 01:59, Bob Sandiford wrote:


Hi, all.

Sorry for the 'spam' -- I'm just testing that my posts are actually 
being seen.  I've sent a few queries over the past couple of weeks and 
haven't had a single response L


Anyways -- if one or two would respond to this, I'd appreciate it -- 
just to let me know that I'm being ignored, vs unseen J



Thanks!

*Bob Sandiford *| Lead Software Engineer | SirsiDynix

P: 800.288.8020 X6943 | _Bob.Sandiford@sirsidynix.com_

www.sirsidynix.com 

*/Join the conversation - you may even get an iPad or Nook out of it!/*

**

Description: Description: Description: Description: Screen shot 
2011-04-28 at 3.32.20 PM.png *Like 
us on Facebook!*


**

Description: Description: Description: Description: Screen shot 
2011-04-28 at 3.32.49 PM.png 
*Follow us on Twitter!*





--
Regards

Jasneet Sabharwal
Software Developer
NextGen Invent Corporation
+91-9871228582

Re: how to do offline adding/updating index

[Mike Sokolov]

Thanks - that sounds like what I was hoping for.  So the I/O during 
replication will have *some* impact on search performance, but 
presumably much less than reindexing and merging/optimizing?


-Mike


Master/slave replication does this out of the box, easily. Just set the slave
to update on Optimize only. Then you can update the master as much as you
want. When you are ready to update the slave (the search instance), just
optimize the master. On the slave's next cycle check it will refresh itself,
quickly, efficiently, minimal impact to search performance. No need to build
extra moving parts for swapping search servers or anything like that.

--
View this message in context: 
http://lucene.472066.n3.nabble.com/how-to-do-offline-adding-updating-index-tp2923035p2924426.html
Sent from the Solr - User mailing list archive at Nabble.com.
   

Re: how to do offline adding/updating index

[Markus Jelsma]

Replication large files can be bad for OS page cache as files being written are 
also written to the page cache. Search latency can grow due to I/O for getting 
the current index version back into memory. Also, Solr cache warming can casue 
a doubling of your heap usage.

Frequent replication in an environment with large files and high query load is 
something one should measure before going in production.

> Thanks - that sounds like what I was hoping for.  So the I/O during
> replication will have *some* impact on search performance, but
> presumably much less than reindexing and merging/optimizing?
> 
> -Mike
> 
> > Master/slave replication does this out of the box, easily. Just set the
> > slave to update on Optimize only. Then you can update the master as much
> > as you want. When you are ready to update the slave (the search
> > instance), just optimize the master. On the slave's next cycle check it
> > will refresh itself, quickly, efficiently, minimal impact to search
> > performance. No need to build extra moving parts for swapping search
> > servers or anything like that.
> > 
> > --
> > View this message in context:
> > http://lucene.472066.n3.nabble.com/how-to-do-offline-adding-updating-ind
> > ex-tp2923035p2924426.html Sent from the Solr - User mailing list archive
> > at Nabble.com.

search by url in Solr?

[Anurag]

Hello everyone,
My Query
q=google.com
should give the ouput as
http://www.googe.com
In my search engine when i put q=google.com, it does not give the above
output, rather it shows no result.
but putting
q=url:google.com gives the correct output as http://www.google.com
Now my question how can i make this feature deafult. That's is when user
queries as google.com he gets the ouput as www.google.com , Also i want to
maintain Normal search features for the content search. That is , i dont
want to loose the existing features normal terms or say content search.

Please guide me


-
Kumar Anurag

--
View this message in context: 
http://lucene.472066.n3.nabble.com/search-by-url-in-Solr-tp2924632p2924632.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: search by url in Solr?

[Luis Cappa Banda]

Hello.

One option is to specify a default search field in your schema.xml. If your
query doesn´t include an specific field the query parser will use the
default one to launch the query. You should include in your schema.xml
something like this:




 url

Re: search by url in Solr?

[Anurag]

thanks! , it worked.
Can i mention
 url,content
to inlcude two default fields.?

On Wed, May 11, 2011 at 3:02 AM, Rakudten [via Lucene] <
ml-node+2924686-576776982-146...@n3.nabble.com> wrote:

> Hello.
>
> One option is to specify a default search field in your schema.xml. If your
>
> query doesn´t include an specific field the query parser will use the
> default one to launch the query. You should include in your schema.xml
> something like this:
>
>
> 
>
>  url
>
>
> --
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/search-by-url-in-Solr-tp2924632p2924686.html
>  To unsubscribe from search by url in Solr?, click 
> here.
>
>



-- 
Kumar Anurag


-
Kumar Anurag

--
View this message in context: 
http://lucene.472066.n3.nabble.com/search-by-url-in-Solr-tp2924632p2926107.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Replication Clarification Please

[Bill Bell]

OK let me rephrase.

In solrconfig.xml there is a setting called mergeFactor. The default is
usually 10.
Practically it means there are 10 segments. If you are doing fast delta
indexing (adding a couple documents, then committing),
You will cycle through all 10 segments pretty fast.

It appears that if you do go past the 10 segments without replicating, the
only recourse is for the replicator to do a full index replication instead
of a delta index replication...

Does that help?


On 5/9/11 9:24 AM, "Ravi Solr"  wrote:

>Hello Mr. Bell,
>   Thank you very much for patiently responding to my
>questions. We optimize once in every 2 days. Can you kindly rephrase
>your answer, I could not understand - "if the amount of time if > 10
>segments, I believe that might also trigger a whole index, since you
>cycled all the segments.In that case I think you might want to
>increase the mergeFactor."
>
>The current index folder details and sizes are given below
>
>MASTER
>--
>   5K   search-data/spellchecker2
> 480M  search-data/index
>   5K   search-data/spellchecker1
>   5K   search-data/spellcheckerFile
> 480M   search-data
>
>SLAVE
>--
>   2K   search-data/index.20110509103950
> 419M   search-data/index
> 2.3G   search-data/index.20110429042508  > SLAVE is pointing to
>this directory
>   5K   search-data/spellchecker1
>   5K  search-data/spellchecker2
>   5K   search-data/spellcheckerFile
> 2.7G   search-data
>
>Thanks,
>
>Ravi Kiran Bhaskar
>
>On Sat, May 7, 2011 at 11:49 PM, Bill Bell  wrote:
>> I did not see answers... I am not an authority, but will tell you what I
>> think
>>
>> Did you get some answers?
>>
>>
>> On 5/6/11 2:52 PM, "Ravi Solr"  wrote:
>>
>>>Hello,
>>>Pardon me if this has been already answered somewhere and I
>>>apologize for a lengthy post. I was wondering if anybody could help me
>>>understand Replication internals a bit more. We have a single
>>>master-slave setup (solr 1.4.1) with the configurations as shown
>>>below. Our environment is quite commit heavy (almost 100s of docs
>>>every 5 minutes), and all indexing is done on Master and all searches
>>>go to the Slave. We are seeing that the slave replication performance
>>>gradually decreases and the speed decreases < 1kbps and ultimately
>>>gets backed up. Once we reload the core on slave it will be work fine
>>>for sometime and then it again gets backed up. We have mergeFactor set
>>>to 10 and ramBufferSizeMB is set to 32MB and solr itself is running
>>>with 2GB memory and locktype is simple on both master and slave.
>>
>> How big is your index? How many rows and GB ?
>>
>> Every time you replicate, there are several resets on caching. So if you
>> are constantly
>> Indexing, you need to be careful on how that performance impact will
>>apply.
>>
>>>
>>>I am hoping that the following questions might help me understand the
>>>replication performance issue better (Replication Configuration is
>>>given at the end of the email)
>>>
>>>1. Does the Slave get the whole index every time during replication or
>>>just the delta since the last replication happened ?
>>
>>
>> It depends. If you do an OPTIMIZE every time your index, then you will
>>be
>> sending the whole index down.
>> If the amount of time if > 10 segments, I believe that might also
>>trigger
>> a whole index, since you cycled all the segments.
>> In that case I think you might want to increase the mergeFactor.
>>
>>
>>>
>>>2. If there are huge number of queries being done on slave will it
>>>affect the replication ? How can I improve the performance ? (see the
>>>replications details at he bottom of the page)
>>
>> It seems that might be one way the you get the index.* directories. At
>> least I see it more frequently when there is huge load and you are
>>trying
>> to replicate.
>> You could replicate less frequently.
>>
>>>
>>>3. Will the segment names be same be same on master and slave after
>>>replication ? I see that they are different. Is this correct ? If it
>>>is correct how does the slave know what to fetch the next time i.e.
>>>the delta.
>>
>> Yes they better be. In the old days you could just rsync the data
>> directory from master and slave and reload the core, that worked fine.
>>
>>>
>>>4. When and why does the index. folder get created ? I see
>>>this type of folder getting created only on slave and the slave
>>>instance is pointing to it.
>>
>> I would love to know all the conditions... I believe it is supposed to
>> replicate to index.*, then reload to point to it. But sometimes it gets
>> stuck in index.* land and never goes back to straight index.
>>
>> There are several bug fixes for this in 3.1.
>>
>>>
>>>5. Does replication process copy both the index and index.
>>>folder ?
>>
>> I believe it is supposed to copy the segment or whole index/ from master
>> to index.* on slave.
>>
>>>
>>>6. what happens if the replication kicks off even before the previous
>>>invocation has not completed ? will the 2nd invocation

Re: tomcat and multicore processors

[Bill Bell]

This is fine. Solr needs lots of memory though... :) It may not be as fast
as you need, but you can test that.

On 5/9/11 2:33 AM, "solr_begin...@onet.pl"  wrote:

>Hi,
> 
>Is that possible that solr on tomcat on windows 2008 is using only one
>core of processor? Do I need configure something to use more cores?
> 
>Best Regards,
>Solr_Beginner

AW: search by url in Solr?

[Martin Rödig]

Hello,

if you want to have more than one searchfield, you can use the Dismax or 
eDismax Queryparser. There you can set more than one field to search.
Example:


dismax
explicit
*,score
text^1.0 title^0.05 author^0.2 shi_quelle^0.4 
shi_year adrp_keywords^0.5 shi_path^2.0
...

Qf are the Query fields in wich the parser will search.
The numbers with the ^ are boost Factors.

The Dismax have many more nice features look at it on 
http://wiki.apache.org/solr/DisMaxQParserPlugin

Mit freundlichen Grüßen
M.Sc. Dipl.-Inf. (FH) Martin Rödig
 
SHI Elektronische Medien GmbH

Postadresse: Watzmannstr. 23, 86316 Friedberg
Besuchsadresse: Curt-Frenzel-Str. 12, 86167 Augsburg
Tel.: 0821 7482633 18
Tel.: 0821 7482633 0 (Zentrale)
Fax: 0821 7482633 29

Internet: http://www.shi-gmbh.com
Registergericht Augsburg HRB 17382
Geschäftsführer: Peter Spiske
Steuernummer: 103/137/30412

-Ursprüngliche Nachricht-
Von: Anurag [mailto:anurag.it.jo...@gmail.com] 
Gesendet: Mittwoch, 11. Mai 2011 07:05
An: solr-user@lucene.apache.org
Betreff: Re: search by url in Solr?

thanks! , it worked.
Can i mention
 url,content
to inlcude two default fields.?

On Wed, May 11, 2011 at 3:02 AM, Rakudten [via Lucene] <
ml-node+2924686-576776982-146...@n3.nabble.com> wrote:

> Hello.
>
> One option is to specify a default search field in your schema.xml. If 
> your
>
> query doesn´t include an specific field the query parser will use the 
> default one to launch the query. You should include in your schema.xml 
> something like this:
>
>
> 
>
>  url
>
>
> --
>  If you reply to this email, your message will be added to the 
> discussion
> below:
>
> http://lucene.472066.n3.nabble.com/search-by-url-in-Solr-tp2924632p292
> 4686.html  To unsubscribe from search by url in Solr?, click 
> here.
>
>



--
Kumar Anurag


-
Kumar Anurag

--
View this message in context: 
http://lucene.472066.n3.nabble.com/search-by-url-in-Solr-tp2924632p2926107.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Boosting score of a document without deleting and adding another document

[karan veer singh]

Is the keyField over here the same thing as the "score" of the field?
> Date: Tue, 10 May 2011 06:39:32 -0700
> From: iori...@yahoo.com
> Subject: RE: Boosting score of a document without deleting and adding another 
> document
> To: solr-user@lucene.apache.org
> 
> 
> > What I want to do is : everytime an item is searched, I get
> > the ID of that document, and now I want to increase this
> > document's score.Function queries at query time won't really
> > help with this. Any way to do this other than adding the
> > entire document again?
> 
> Looks like you need ExternalFileField.
> 
> http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html