Re: Meaning of the colors in the solrcloud view

[Stefan Matheis]

Hi Markus

There is an open issue for integrating a color legend:
https://issues.apache.org/jira/browse/SOLR-3915

Stefan 


On Sunday, November 18, 2012 at 6:30 AM, Markus.Mirsberger wrote:

> Hi,
> 
> can someone give me a hint about the meaning of the different colors 
> inside the cloud admin view graphics? I see oange, yellow, gray colors 
> but dont find anything in the documentation about the meaning 
> (especially the gray color:) ).
> 
> I have an external zookeeper server with 4 nodes (2 shards with 2 
> replications) and suddenly the following effect (everything worked fine 
> for a few days):
> 
> - I have 3 collections already running with data and for every 
> collection the graphic shows me now one serveraddress in gray color.
> - When I try to create a new collection through the collection api the 
> cores will be created on all servers except one ( the same shown in 
> other collections in gray color)
> 
> I already restarted the servers. I also tried to clean the zookeeper 
> data, upload my configurations again and restart everything. Always the 
> same effect.
> 
> 
> Thanks,
> Markus

Re: admin query showing unstored fields

[Reik Schatz]

Hi,

yes all good thanks. It would be nice on this page:
http://wiki.apache.org/solr/SchemaXml#Fields under "Common field options"
and also "Data Types - common options" to specify what Solr will use as
default is if an attribute is not specified (i.e. field will be stored if
the stored attribute is not specified for a Field)

Reik


On Fri, Nov 16, 2012 at 8:18 PM, Jack Krupansky wrote:

> So, you're all set?
>
> Maybe you could suggest where in the wiki the text could be clarified to
> help others avoid the same confusion.
>
>
> -- Jack Krupansky
>
> -Original Message- From: Reik Schatz
> Sent: Friday, November 16, 2012 8:35 AM
>
> To: solr-user@lucene.apache.org
> Subject: Re: admin query showing unstored fields
>
> Hi Jack,
>
> I just did some testing again and can confirm it works! I am new to Solr so
> by reading through 
> http://wiki.apache.org/solr/**SchemaXml#FieldsI
>  was under
> the impression that not specifying the stored attribute at all, would
> default to stored="false" - which apparently is not the case. So running
> without the stored attribute in both the fieldType and the field element
> will store the contents. When I added stored="false" to either fieldType or
> the field element it worked. As for the 1.1 schema, I'am not upgrading from
> a previous version. I was looking for the most minimal configuration to get
> started with Solr, so I just copied some snippets from the web, which
> probably were older :)
>
>
> On Fri, Nov 16, 2012 at 5:09 PM, Jack Krupansky *
> *wrote:
>
>  I just noticed that you are using an old schema version, 1.1. Any reason
>> for that? This suggests that you had an existing, old, Solr that you
>> migrated to 4.0. What Solr release was it? Was the stored attribute
>> working
>> as you expected before you upgraded to Solr 4.0? I don't know the specific
>> semantics of stored fields with such an old schema version, but I don't
>> see
>> any doc that suggests that the stored attribute would behave differently.
>>
>> I did just try your scenario in a fresh copy of the Solr 4.0 example and
>> it works as expected - "players" CANNOT be returned on a query. So, please
>> try your scenario on such a fresh copy of Solr 4.0 and see if you can
>> reproduce your symptom. Are there any other steps you took that led up to
>> the unexpected behavior?
>>
>> The bottom line is that unless you did the two scenarios that Erik and I
>> suggested, or maybe Schema version 1.1 has some odd, undocumented quirks,
>> there must be some other steps you are performing that cause the
>> unexpected
>> behavior.
>>
>>
>> -- Jack Krupansky
>>
>> -Original Message- From: Reik Schatz
>> Sent: Friday, November 16, 2012 3:02 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: admin query showing unstored fields
>>
>> I did this test. Here is my schema.xml (setting stored="false" explicitly
>> though it should be default):
>>
>> 
>>
>>
>>> precisionStep="32" omitNorms="true" omitTermFreqAndPositions="true"
>> />
>>
>> 
>>
>>> class="org.apache.lucene.analysis.standard.StandardAnalyzer" />
>>
>>
>>
>>
>>
>>> stored="false" />
>>
>>> stored="false" />
>>
>>id
>>players
>>
>>
>>
>> 
>>
>> I indexed a single document via the sol4j api:
>>
>> // todo: make addings Fields extensible, i.e. by creating a
>> SolrInputDocumentBuilder class using multiple ReportToField subclasses
>>final SolrInputDocument doc = new SolrInputDocument();
>>doc.addField("id", String.valueOf(UUID.
>> randomUUID().toString()));
>>
>>
>>
>>doc.addField("players", "One morning, when Gregor Samsa woke from
>> troubled dreams, he found himself transformed in his bed into a horrible
>> vermin. He lay on his armour-like back, and if he lifted his head a little
>> he could see his brown belly, slightly domed and divided by arches into
>> stiff sections. The bedding was hardly able to cover it and seemed ready
>> to
>> slide off any moment. His many legs, pitifully thin compared with the size
>> of the rest of him, waved about helplessly as he looked. \"What's happened
>> to me? \" he thought. It wasn't a dream. His room, a proper human room
>> although a little too small, lay peacefully between its four familiar
>> walls. A collection of textile samples lay spread out on the table - Samsa
>> was a travelling salesman - and above it there hung a picture that he had
>> recently cut out of an illustrated magazine and housed in a nice, gilded
>> frame. It showed a lady fitted out with a fur hat and fur boa who sat
>> upright, raising a heavy fur muff that covered the whole of her lower arm
>> towards the viewer. Gregor then turned to look out the window at the dull
>> weather.");
>>
>>// dynamic score field no shown here
>>
>>return doc;
>>
>>
>> Then I went to 
>> http://localhost:8080/solr/#/history/query

Re: Solr Delta Import Handler not working

[Spadez]

Update! Thank you to Lance for the help. Based on your suggestion I have
fixed up a few things.

*My Dataconfig now has the filename pattern fixed and root entity=true*
/
  
  

  
  

  
/

*My data.xml has a corrected date format with "T":*
/

123
Delta Import 2
This is my long description
This is

Google
England
2007-12-31T22:29:59
Google
www.google.com
45.17614,45.17614

/



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-Delta-Import-Handler-not-working-tp4020897p4020925.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Execute an independent query from the main query

[Erick Erickson]

Depending on your use-case, you might be able to use the join
functionality. Be aware that this doesn't return the values of the
"from" field, so it may not be suitable.

_why_ do you want to do this? I'm wondering if this is an _XY_
problem.

Best,
Erick


On Sun, Nov 18, 2012 at 9:14 AM, Indika Tantrigoda wrote:

> Hi All,
>
> I would like to get results of an query that is different from the main
> query as a new field. This query needs to be independent from any filter
> queries applied to the main query. I was trying to achieve this by
> fl=_external_query_result:query($myQuery), however that result seems to be
> governed by any filter queries applied to the main query ? Is it possible
> to have a completely separate query in the fl list and return its result
> along with the results (per results), or would I need to create a separate
> query on the client side to get the results of the independent query (based
> on the results from the first query) ?
>
> Thanks in advance,
> Indika
>

Re: Execute an independent query from the main query

[Luis Cappa Banda]

Hello!

When queries become more and more complex and you need to apply one second
query with the resultant docs from the first one, or re-sort results, or
maybe add some promotional or special docs to the response, I recommend to
develop a Web App module that implements that complex business logic and
dispatches queries from your Client App to your Solr back-end. That module,
let's call Search Engine, lets you play with all those special use cases.
If you are familiar with Java I suggest you to have a look at the
combination between SolrJ and Spring framework or Jersey.

Regards,

- Luis Cappa.
El 18/11/2012 15:15, "Indika Tantrigoda"  escribió:

> Hi All,
>
> I would like to get results of an query that is different from the main
> query as a new field. This query needs to be independent from any filter
> queries applied to the main query. I was trying to achieve this by
> fl=_external_query_result:query($myQuery), however that result seems to be
> governed by any filter queries applied to the main query ? Is it possible
> to have a completely separate query in the fl list and return its result
> along with the results (per results), or would I need to create a separate
> query on the client side to get the results of the independent query (based
> on the results from the first query) ?
>
> Thanks in advance,
> Indika
>

SolrCloud - Zookeeper Questions

[deniz]

Hi All,

I have been digging web for answers to some of my questions in my mind but
till now I am still not clear on them..

first (and main)question is about zookeeper itself.. are there any tutorials
in detail how we can use it with solr other than wikies? SolrCloud wiki page
is useful for sure, but i want to know how to control some operations like
removing a shard/replica temporarily and then adding back again and so on... 
I did some trials for that but basically it was playing with the command
line witb zKClient.sh, which I am not sure if it is the correct was to
administer SolrCloud. 

anyone has experience with zookeeper? 




-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-Zookeeper-Questions-tp4020971.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Date math query syntax

[Jack Krupansky]

You can do numeric math in function queries, but not string operations, and 
there is no provision for doing string operations in a query using the 
result of an embedded function query.


You haven't shown us a full query.

Sounds like we have another candidate for "XY Problem". First you need to 
tell us what you are really trying to accomplish, THEN a solution can be 
pursued.


Take a look at the "ms" function query for date calculations - which is 
separate from so-called "date-math" which is a constant offset from a base 
date literal value.


-- Jack Krupansky

-Original Message- 
From: Indika Tantrigoda

Sent: Sunday, November 18, 2012 3:11 PM
To: solr-user@lucene.apache.org
Subject: Date math query syntax

Hi All,

I am trying to use date math along with a filed
value. 1970-01-01T04:32:010Z+(_val_:prep_time)MINUTES is the date math
query I am using. prep_time is of type int. I am having trouble properly
formatting the string resulting in exceptions. I am guessing that I am not
properly formatting the query with quotes and/or parenthesis.

Thanks,
Indika 

Re: SolrCloud - Zookeeper Questions

[Mark Miller]

On Nov 18, 2012, at 10:02 PM, deniz  wrote:

> removing a shard/replica temporarily and then adding back again 

I'd unload them with http cmds (without using the options that delete things on 
disk) and then create them again when I wanted them back.

- Mark

Per user document exclusions

[Christian Jensen]

Hi,

We have a need to allow each user to 'exclude' individual documents in the
results. We can easily do this now within the RDBMS using a FTS index and a
query with 'OUTER LEFT JOIN WHERE NULL' type of thing.

Can Solr do this somehow? Heavy customization is not a problem - I would
bet this has already been done. I would like to avoid multiple trips back
and forth from either the DB or SOLR if possible.

Thanks!
Christian

-- 

*Christian Jensen*
724 Ioco Rd
Port Moody, BC V3H 2W8
+1 (778) 996-4283
christ...@jensenbox.com

Re: SolrCloud - Zookeeper Questions

[deniz]

Mark Miller-3 wrote
> On Nov 18, 2012, at 10:02 PM, deniz <

> denizdurmus87@

> > wrote:
> 
>> removing a shard/replica temporarily and then adding back again 
> 
> I'd unload them with http cmds (without using the options that delete
> things on disk) and then create them again when I wanted them back.
> 
> - Mark


so then, the reason zookeeper is a part of SolrCloud is only for storing
config information? 

and instead of using the steps here
http://lucidworks.lucidimagination.com/display/solr/Using+ZooKeeper+to+Manage+Configuration+Files
can we directly change the configs and then reloading by http would do the
new configs valid? 



-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-Zookeeper-Questions-tp4020971p4020981.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Date math query syntax

[Indika Tantrigoda]

Hi Jack,

Thanks for the response. Let me explain my question better.

I have a filter query to limit the query results,
start_time:[* TO 1970-01-01T10:55:002Z] AND end_time:[1970-01-01T10:55:002Z
TO *] AND session_time_range_available:true

and I also use the same query as a function query in fl using the
if(exists(query), x, y) where query is,
{!edismax}(start_time:[* TO 1970-01-01T10:55:002Z] AND
end_time:[1970-01-01T10:55:002Z TO *] AND session_time_range_available:true)

I need to extend the end_time range query based on the logic, similar to
end_time:[1970-01-01T10:55:002Z+30MINUTUES TO *]

How much I need to offset the end_time range is defined as an int type in
the schema. I was trying to use the field value query (_val_:field) to
retrive the end_time offset, resulting in the query being
1970-01-01T04:32:010Z+(_val_:**prep_time)MINUTES

However the query seems to have syntax errors. Or is it allowed to use the
_val_:field in such instances ?

As an extension I'll also have two offsets resulting in something similar
to
1970-01-01T04:32:010Z+(_val_:**prep_time)MINUTES+(_val_:extend_time)MINUTES

Thanks,
Indika

On 19 November 2012 09:17, Jack Krupansky  wrote:

> You can do numeric math in function queries, but not string operations,
> and there is no provision for doing string operations in a query using the
> result of an embedded function query.
>
> You haven't shown us a full query.
>
> Sounds like we have another candidate for "XY Problem". First you need to
> tell us what you are really trying to accomplish, THEN a solution can be
> pursued.
>
> Take a look at the "ms" function query for date calculations - which is
> separate from so-called "date-math" which is a constant offset from a base
> date literal value.
>
> -- Jack Krupansky
>
> -Original Message- From: Indika Tantrigoda
> Sent: Sunday, November 18, 2012 3:11 PM
> To: solr-user@lucene.apache.org
> Subject: Date math query syntax
>
>
> Hi All,
>
> I am trying to use date math along with a filed
> value. 1970-01-01T04:32:010Z+(_val_:**prep_time)MINUTES is the date math
> query I am using. prep_time is of type int. I am having trouble properly
> formatting the string resulting in exceptions. I am guessing that I am not
> properly formatting the query with quotes and/or parenthesis.
>
> Thanks,
> Indika
>

SolrCloud Error after leader restarts

[deniz]

Hello,

for test purposes, I am running two zookeepers on ports 2181 and 2182. and i
have two solr instances running on different machines...

For the one which is running on my local and acts as leader:
 java -Dbootstrap_conf=true -DzkHost=localhost:2181 -jar start.jar

and for the one which acts as follower, on a remote machine:
java -Djetty.port=7574 -DzkHost=:2182 -jar start.jar

until this point everything is smooth and i can see the configs on both
zookeeper hosts when i connect with zkCli.sh. 

just to see what happens and check recovery stuff, i have killed the solr
which is running on my local and tried to index some files by using the
follewer, which was failed... this is normal as writes are routed into the
leader...

the point that i dont understand is here:

when i restart the leader with the same command on terminal, after normal
logs, it start showing this 


Nov 19, 2012 2:15:18 PM org.apache.solr.common.SolrException log
SEVERE: SnapPull failed :org.apache.solr.common.SolrException: Index fetch
failed : 
at 
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:400)
at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:297)
at
org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:151)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:405)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:220)
Caused by: org.apache.lucene.index.IndexNotFoundException: no segments* file
found in org.apache.lucene.store.RAMDirectory@1e75e89
lockFactory=org.apache.lucene.store.NativeFSLockFactory@128e909: files: []
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:741)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:630)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:343)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:639)
at 
org.apache.solr.update.SolrIndexWriter.(SolrIndexWriter.java:75)
at 
org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:62)
at
org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:191)
at
org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:77)
at 
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:354)
... 4 more

Nov 19, 2012 2:15:18 PM org.apache.solr.common.SolrException log
SEVERE: Error while trying to recover:org.apache.solr.common.SolrException:
Replication for recovery failed.
at
org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:154)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:405)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:220)


it fails to recover after shutdown... why does this happen? 


 



-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-Error-after-leader-restarts-tp4020985.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: document clustering or tagging

[Stanislaw Osinski]

Stanislaw Osinski, stanislaw.osin...@carrotsearch.com
http://carrotsearch.com

I have very huge solr index. I want to tag all documents with terms that
> better represent that document like  this
> <
> http://search.carrotsearch.com/carrot2-webapp/search?source=web&view=folders&skin=fancy-compact&query=rugby+in+london&results=100&algorithm=lingo3g&EToolsDocumentSource.country=ALL&EToolsDocumentSource.language=ALL&EToolsDocumentSource.safeSearch=false
> >
> . Does this type of clustering results is also come under document tagging?
>

No, this type of clustering will not solve your problem because it's suited
for small/medium collections of documents (search results) rather than the
whole index. For your specific problem I'd recommend some keyword /
keyphrase extractor, which would generate tags for each document separately.

Staszek

Re: error opening index solr 4.0 with lukeall-4.0.0-ALPHA.jar

[Bernd Fehling]

I think there is already a BETA available:
http://luke.googlecode.com/svn/trunk/

Changes in unreleased version:
* Update to 4.0.0_BETA.
* Issue 22: term vectors could not be accessed if a field was not stored. Fixed
  also several other wrong assumptions about field flags.

You might try that one.

Regards
Bernd


Am 16.11.2012 17:16, schrieb Miguel Ángel Martín:
> hi all:
> 
> i can open an index create with  solr 4.0. with luke version=>
>  lukeall-4.0.0-ALPHA.jar
> 
> i have the error:
> 
> Format version is not supported (resource:
> NIOFSIndexInput(path="/Users/desa/data/index/_2.tvx")): 1 (needs to be
> between 0 and 0)
>  at
> org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:148)
> at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:130)
>  at
> org.apache.lucene.codecs.lucene40.Lucene40TermVectorsReader.(Lucene40TermVectorsReader.java:108)
> at
> org.apache.lucene.codecs.lucene40.Lucene40TermVectorsFormat.vectorsReader(Lucene40TermVectorsFormat.java:107)
>  at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:118)
> at org.apache.lucene.index.SegmentReader.(SegmentReader.java:55)
>  at
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
> at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:752)
>  at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
>  at org.getopt.luke.Luke.openIndex(Luke.java:967)
> at org.getopt.luke.Luke.openOk(Luke.java:696)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>  at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
>  at thinlet.Thinlet.invokeImpl(Thinlet.java:4579)
> at thinlet.Thinlet.invoke(Thinlet.java:4546)
>  at thinlet.Thinlet.handleMouseEvent(Thinlet.java:3937)
> at thinlet.Thinlet.processEvent(Thinlet.java:2917)
>  at java.awt.Component.dispatchEventImpl(Component.java:4744)
> at java.awt.Container.dispatchEventImpl(Container.java:2141)
>  at java.awt.Component.dispatchEvent(Component.java:4572)
> at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4619)
>  at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4280)
> at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4210)
>  at java.awt.Container.dispatchEventImpl(Container.java:2127)
> at java.awt.Window.dispatchEventImpl(Window.java:2489)
>  at java.awt.Component.dispatchEvent(Component.java:4572)
> at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:704)
>  at java.awt.EventQueue.access$400(EventQueue.java:82)
> at java.awt.EventQueue$2.run(EventQueue.java:663)
>  at java.awt.EventQueue$2.run(EventQueue.java:661)
> at java.security.AccessController.doPrivileged(Native Method)
>  at
> java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
> at
> java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:98)
>  at java.awt.EventQueue$3.run(EventQueue.java:677)
> at java.awt.EventQueue$3.run(EventQueue.java:675)
>  at java.security.AccessController.doPrivileged(Native Method)
> at
> java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
>  at java.awt.EventQueue.dispatchEvent(EventQueue.java:674)
> at
> java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)
>  at
> java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)
> at
> java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)
>  at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
> at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
>  at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
> o
> 
> 
> any ideas?
> 
> 
> I,ve created another index with lucene 4.0 and this luke open the index
> well.
> 
> thanks in advance
> 

-- 
*
Bernd FehlingBielefeld University Library
Dipl.-Inform. (FH)LibTec - Library Technology
Universitätsstr. 25  and Knowledge Management
33615 Bielefeld
Tel. +49 521 106-4060   bernd.fehling(at)uni-bielefeld.de

BASE - Bielefeld Academic Search Engine - www.base-search.net
*

Re: error opening index solr 4.0 with lukeall-4.0.0-ALPHA.jar

[Toke Eskildsen]

On Mon, 2012-11-19 at 08:10 +0100, Bernd Fehling wrote:
> I think there is already a BETA available:
> http://luke.googlecode.com/svn/trunk/

> You might try that one.

That doesn't work either for Lucene 4.0.0 indexes, same for source
trunk. I did have some luck with downloading the source and changing the
dependencies to Lucene 4.0.0 final (4 or 5 JARs, AFAIR). It threw a
non-fatal exception upon index open, something about subReaders not
being accessible throught the metod it used (sorry for being vague, it
was on my home machine and some days ago), so I'm guessing that not all
functionality works. It was possible to inspect some documents and that
was what I needed at the time.