from:"Victor"

Multiple search analyzers on the same field type possible?

2011-10-12 Thread Victor

I would like to do the following in solr/lucene:

For a demo I would like to index a certain field once, but be able to query
it in 2 different ways. The first way is to query the field using a synonym
list and the second way is to query the same field without using a synonym
list. The reason I want to do this is that I want the synonym list to be
flexible and do not want to re-index everything when the list changes. Also,
I want to be able to let the user decide if he/she wants to use the synonym
list while querying.

I had hoped that a solution like this would be possible:


  
...
  
  
...
  
  
...
  


And then use some kind of parameter in the url to select either query1 or
query2, but this does not seem possible in solr/lucene. 

Maybe I can use a solution using the  command, but so far I have
not been successful in getting this to work.

I still hope this is possible, thanks in advance for your help on this.


--
View this message in context: 
http://lucene.472066.n3.nabble.com/Multiple-search-analyzers-on-the-same-field-type-possible-tp3417898p3417898.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Multiple search analyzers on the same field type possible?

2011-10-13 Thread Victor

I looked at the copyfield solution and found it not suitable for what I am
trying for. I defined a new  using a  that uses a synonym
filter for the query analyzer. Then I used a  command to fill it
with the data that I want. Since I do not want to create another index I set
the index parameter to false, "indexed=false". I found that it is impossible
to query on this field (which is logical, since solr does it's querying
based on indexes).

I guess what I would need is an expansion of the solr functionality of the q
parameter, like:

q=::

Should I wait for it? :-)

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Multiple-search-analyzers-on-the-same-field-type-possible-tp3417898p3418672.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Multiple search analyzers on the same field type possible?

2011-10-13 Thread Victor

Or, alternatively, it would be nice to link a field to another field so that
it can use the index of that field.

The whole point of different "query analyzers" on the same index would make
the whole solr/lucene more flexible I think. But let's wait and see, maybe
it is possible to do this and I am just missing it.

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Multiple-search-analyzers-on-the-same-field-type-possible-tp3417898p3418771.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Multiple search analyzers on the same field type possible?

2011-10-13 Thread Victor

Sorry Erick, my last post and your's crossed each other.

I am reluctant to use another index (or a multi-value index) since I think
it will increase the storage I need for those indexes without adding
functionality (and storage could be an issue for me).

But first let's see if I understand you correctly:


  
...
  
  
... no synonyms
  
   
 


  
... (the same as "fieldA")
  
  
... (the same as "fieldA" + )
  
   
 






User wants to query without synonyms:
1) q=desc_no_synonyms:"hot" fl=desc_no_synonyms

User wants to query with synonyms:
2) q=desc_yes_synonyms:"hot" fl=desc_no_synonyms

In case 1) the user gets the description with only "hot" in it,
in case 2) the user gets the description with "hot" or "warm" in it.

I understand that "fieldB" does not store the contents, but it will create
an extra index or expand an already existing one, right?

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Multiple-search-analyzers-on-the-same-field-type-possible-tp3417898p3418874.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Multiple search analyzers on the same field type possible?

2011-10-14 Thread Victor

Hi Erick,

I work for a very big library and we store huge amounts of data. Indexing
some of our collections can take days and the index files can get very big.
We are a non-profit organisation, so we want to provide maximum service to
our customers but at the same time we are bound to a fixed budget and want
to keep costs as low as possible (including disk space). Our customers vary
from academic people that want to do very precise searches to common users
who want to seach in a more general way. The library now wants to implement
some form of stemming, but we have had one demo in the past with a stemmer
that returned results that did not please my internal customer (another
department).

So my wish list looks like this:

1) Implement stemming
2) Give the end user the possibility to turn stemming on or off for their
searches
3) Have maximum control over the stemmer without the need to reindex if we
change something there
4) Prevent the need for more storage (to keep the operations people happy)

So far I have been able to satisfy 1,2 and 3. I am using a synonyms list at
query time to apply my stemming. The synonym list I build as follows:

a) load a library (a text file with 1 word per line)
b) remove stop words from the list
c) link words that have the same stem

Bullet c) is a little bit more sophisticated, because I do not link words
that are already part of a pre-defined synonym list that contains
exceptions.

All this I do to keep maximum control over the behaviour of the stemmer.
Since this is a demo and it will be used to convince other people in my
organisation that stemming could be worth implementing, I need to be able to
adjust its behaviour quickly.

So far processing speed has not been an issue, but disk storage has.
Generally, at index time we remove as few tokens as possible and our objects
are complete books, news papers (from 1618 until 1995), etc . So you can
imagine that our indexes get very, very big.

--
View this message in context:
http://lucene.472066.n3.nabble.com/Multiple-search-analyzers-on-the-same-field-type-possible-tp3417898p3420946.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Multiple search analyzers on the same field type possible?

2011-10-14 Thread Victor

Hi Erick,

First of all, thanks for your posts, I really appreciate this!

1) Yes, we have tested alternative stemmers, but I admit that a definite
decission has not been made yet. Anyway, we definately do not want to create
a stemmed index because of storage issues and we definately want to be able
to allow the end-user to turn it on and off. So choosing a different stemmer
does not solve my problem of wanting to switch between stemming/non-stemming
without additional indexes.

2) Rant granted :) And I definately agree with you. It is always a challenge
to find a balance between what a customer wants and how far you really want
to go to in achieving a solution (that does not conflict too much with the
maintainability of the system).

But, I do think that the requirements are not that outragious. It seems to
me reasonable that once you have created an index it would be nice to be
able to use that index in different ways. After all, the only thing I want
is apply different query analyzers (mind you, I am not formatting the
tokens, what could possibly result in index/query token conflicts, I am
merely expanding query possibilities here by adding synonyms, the rest stays
the same).

Another good example could be that you want to index a field that contains
text in different languages. Would it not be nice then to be able to define
optimized query analyzers on that field, one for each language? You could
then access them using the q parameter: q=::,
where is the name of the query analyzer to use. It seems to me to
be a nice feature. Could be a big change though, because I assume that at
the moment the analyzers have hard-coded names ("index" and "query").

3) Yep, I was also looking into this (because other options seemed to be
vaporizing). Don't know if I'm going to use suffixes or maybe add a trigger
word like @stem@. Depends on what the scope of the called method is. I
prefer the trigger word @stem@ variant because I can then just insert that
without needing to parse the query string to find out what the actual seach
words are that I need to suffix.

Cheers and again, thanks for helping me on this,
Victor

--
View this message in context:
http://lucene.472066.n3.nabble.com/Multiple-search-analyzers-on-the-same-field-type-possible-tp3417898p3421522.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Multiple search analyzers on the same field type possible?

2011-10-14 Thread Victor

I've spent today writing my own SynonymFilter and SynonymFilterFactory. And
it works!

I've followed Erick's advice and pre- and postfixed all the words that I
want to stem with a @. So, if I want to stem the word car, I injest it in
the query as @car@.

My adapted synonymfilter recognizes the pre/postfixing, removes the @
characters and continues as usual (which means the synonym filter will do
what it is supposed to be doing). If no "stemming tags" are found, it aborts
the synonym lookup part of the process for that token an returns
immediately.

So: 
car --> car
cars --> cars
@car@ --> car and cars

Mission accomplished, no extra storage needed, current index can stay as it
is, end user can switch between stemming and no stemming when he/she wants
too.

I think I saved a lot of money today.

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Multiple-search-analyzers-on-the-same-field-type-possible-tp3417898p3422060.html
Sent from the Solr - User mailing list archive at Nabble.com.

Which open-source crawler to use with SolrJ and Postgresql ?

2016-02-16 Thread Victor D'agostino


Hi

I am building a Solr 5 architecture with 3 Solr nodes and 1 zookeeper.
The database backend is postgresql 9 on RHEL 6.

I am looking for a free open-source crawler which use SolrJ.

What do you guys recommend ?

Best regards
Victor d'Agostino




Ce message et les éventuels documents joints peuvent contenir des informations confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou publication, totale ou partielle et quel qu'en soit le moyen est formellement interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité de ce message n'est pas assurée et la société émettrice ne peut être tenue pour responsable de son contenu.

Re: Which open-source crawler to use with SolrJ and Postgresql ?

2016-02-16 Thread Victor D'agostino


Hi,

Thanks for your help.
Nutch is exactly what i'm looking for and i'm feeling lucky the solr 
cloud support has just been comited !


I'll try the trunk version and wait until the 1.12 version is released.

Regards
Victor


Nutch has Solr 5 cloud support in trunk, i committed it earlier this month.
https://issues.apache.org/jira/browse/NUTCH-2197

Markus
  
-Original message-

From:Emir Arnautovic 
Sent: Tuesday 16th February 2016 16:26
To: solr-user@lucene.apache.org
Subject: Re: Which open-source crawler to use with SolrJ and Postgresql ?

Hi,
It is most common to use Nutch as crawler, but it seems that it still
does not have support for SolrCloud (if I am reading this ticket
correctly https://issues.apache.org/jira/browse/NUTCH-1662). Anyway, I
would recommend Nutch with standard http client.

Regards,
Emir

On 16.02.2016 16:02, Victor D'agostino wrote:

Hi

I am building a Solr 5 architecture with 3 Solr nodes and 1 zookeeper.
The database backend is postgresql 9 on RHEL 6.

I am looking for a free open-source crawler which use SolrJ.

What do you guys recommend ?

Best regards
Victor d'Agostino













Ce message et les éventuels documents joints peuvent contenir des informations confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou publication, totale ou partielle et quel qu'en soit le moyen est formellement interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité de ce message n'est pas assurée et la société émettrice ne peut être tenue pour responsable de son contenu.

Re: Solr 5.5.0 ClassNotFoundException solr.MockTokenizerFactory after DIH setup

2016-03-19 Thread Victor D'agostino

trib/extraction/lib/xmlbeans-2.6.0.jar
file:/data/solr-5.5.0/contrib/extraction/lib/boilerpipe-1.1.0.jar
file:/data/solr-5.5.0/contrib/extraction/lib/bcmail-jdk15-1.45.jar
file:/data/solr-5.5.0/contrib/extraction/lib/juniversalchardet-1.0.3.jar
file:/data/solr-5.5.0/contrib/extraction/lib/xmpcore-5.1.2.jar
file:/data/solr-5.5.0/contrib/extraction/lib/poi-ooxml-3.11.jar
file:/data/solr-5.5.0/contrib/extraction/lib/poi-ooxml-schemas-3.11.jar
file:/data/solr-5.5.0/contrib/extraction/lib/tika-java7-1.7.jar
file:/data/solr-5.5.0/contrib/extraction/lib/vorbis-java-tika-0.6.jar
file:/data/solr-5.5.0/contrib/extraction/lib/fontbox-1.8.8.jar
file:/data/solr-5.5.0/contrib/extraction/lib/metadata-extractor-2.6.2.jar
file:/data/solr-5.5.0/dist/solr-cell-5.5.0.jar
file:/data/solr-5.5.0/contrib/langid/lib/jsonic-1.2.7.jar
file:/data/solr-5.5.0/contrib/langid/lib/langdetect-1.1-20120112.jar
file:/data/solr-5.5.0/dist/solr-langid-5.5.0.jar
file:/data/solr-5.5.0/contrib/velocity/lib/commons-collections-3.2.2.jar
file:/data/solr-5.5.0/contrib/velocity/lib/commons-beanutils-1.8.3.jar
file:/data/solr-5.5.0/contrib/velocity/lib/velocity-tools-2.0.jar
file:/data/solr-5.5.0/contrib/velocity/lib/velocity-1.7.jar
file:/data/solr-5.5.0/dist/solr-velocity-5.5.0.jar

Thanks for your help



 Message original 
*Sujet: *Re: Solr 5.5.0 ClassNotFoundException solr.MockTokenizerFactory 
after DIH setup

*De : *Erick Erickson 
*Pour : *solr-user 
*Date : *17/03/2016 17:43

Likely you have some old jars in the classpath somehow. The first parts of
the
log should show you exactly what jars are loaded. It's tedious to go through
since there are a lot of them, but it's something to check.

If you have the hardware, try putting it on a machine that's never had Solr
on it?

Best,
Erick

On Thu, Mar 17, 2016 at 9:39 AM, Victor D'agostino <
victor.d.agost...@fiducial.net> wrote:


H guys

I have a java.lang.ClassNotFoundException: solr.MockTokenizerFactory after
a fresh 5.5.0 setup with DIH and a collection named "db".

The tgz file is from
https://urldefense.proofpoint.com/v2/url?u=http-3A__apache.crihan.fr_dist_lucene_solr_5.5.0_solr-2D5.5.0.tgz&d=CwIFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=GIbD6pb1nH9ZrxFDfhl_c8kJe7NkpbmXG1YHXBYFth8&m=gmvHOPcbr4doaPRxi2-DRqQCz_NluHiEuBIk5vc7N6I&s=-sNXRbZQambKg7w1PpYj24DaEkM6I9BkIjbMk-p8c5E&e=

Any idea why this class is missing at startup ?
Should i download tierce parties jar files or export the CLASSPATH ?


Here is the full log :

2016-03-17 16:32:53.767 INFO  (coreLoadExecutor-6-thread-1) [   ]
o.a.s.u.SolrIndexConfig IndexWriter infoStream solr logging is enabled
2016-03-17 16:32:53.811 INFO  (coreLoadExecutor-6-thread-1) [   ]
o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
2016-03-17 16:32:54.731 INFO  (coreLoadExecutor-6-thread-1) [   ]
o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
2016-03-17 16:32:54.755 INFO  (coreLoadExecutor-6-thread-1) [   ]
o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but
managed schema resource managed-schema not found - loading non-managed
schema schema.xml instead
2016-03-17 16:32:55.020 INFO  (coreLoadExecutor-6-thread-1) [   ]
o.a.s.s.IndexSchema [db] Schema name=test
2016-03-17 16:32:55.153 ERROR (coreLoadExecutor-6-thread-1) [   ]
o.a.s.c.CoreContainer Error creating core [db]: Could not load conf for
core db: Can't load schema /data/solr-5.5.0/server/solr/db/conf/schema.xml:
Plugin init failure for [schema.xml] fieldType "failtype1": Plugin init
failure for [schema.xml] analyzer/tokenizer: Error loading class
'solr.MockTokenizerFactory'
org.apache.solr.common.SolrException: Could not load conf for core db:
Can't load schema /data/solr-5.5.0/server/solr/db/conf/schema.xml: Plugin
init failure for [schema.xml] fieldType "failtype1": Plugin init failure
for [schema.xml] analyzer/tokenizer: Error loading class
'solr.MockTokenizerFactory'
 at
org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:84)
 at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:812)
 at
org.apache.solr.core.CoreContainer.access$000(CoreContainer.java:87)
 at
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:467)
 at
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:458)
 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
 at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: Can't load schema
/data/solr-5.5.0/server/solr/db/conf/schema.xml: Plugin init failure for
[schema.xml] fieldType

Solr 5.5.0 ClassNotFoundException solr.MockTokenizerFactory after DIH setup

2016-03-19 Thread Victor D'agostino

oader.create(FieldTypePluginLoader.java:104)
at 
org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:53)
at 
org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:152)

... 17 more
Caused by: org.apache.solr.common.SolrException: Error loading class 
'solr.MockTokenizerFactory'
at 
org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:558)
at 
org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:627)
at 
org.apache.solr.schema.FieldTypePluginLoader$2.create(FieldTypePluginLoader.java:341)
at 
org.apache.solr.schema.FieldTypePluginLoader$2.create(FieldTypePluginLoader.java:334)
at 
org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:152)

... 21 more
Caused by: java.lang.ClassNotFoundException: solr.MockTokenizerFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at 
java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:278)
at 
org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:542)

... 25 more



Regards
Victor





Ce message et les éventuels documents joints peuvent contenir des informations confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou publication, totale ou partielle et quel qu'en soit le moyen est formellement interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité de ce message n'est pas assurée et la société émettrice ne peut être tenue pour responsable de son contenu.

Cant access new docs without restarting Solr (java.nio.file.NoSuchFileException)

2016-03-22 Thread Victor D'agostino

ommit=false}
161763 INFO  (commitScheduler-13-thread-1) [c:db s:shard1 r:core_node1 
x:db_shard1_replica1] o.a.s.c.SolrDeletionPolicy 
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(MMapDirectory@/data/solr-5.5.0/server/solr/db_shard1_replica1/data/index 
lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c585e91; 
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
commit{dir=NRTCachingDirectory(MMapDirectory@/data/solr-5.5.0/server/solr/db_shard1_replica1/data/index 
lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c585e91; 
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
161763 INFO  (commitScheduler-13-thread-1) [c:db s:shard1 r:core_node1 
x:db_shard1_replica1] o.a.s.c.SolrDeletionPolicy newest commit 
generation = 4
161765 INFO  (commitScheduler-13-thread-1) [c:db s:shard1 r:core_node1 
x:db_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening 
Searcher@6aca2d82[db_shard1_replica1] realtime
161766 INFO  (commitScheduler-13-thread-1) [c:db s:shard1 r:core_node1 
x:db_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
161826 INFO  (commitScheduler-14-thread-1) [c:db s:shard2 r:core_node2 
x:db_shard2_replica1] o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
162656 INFO  (commitScheduler-14-thread-1) [c:db s:shard2 r:core_node2 
x:db_shard2_replica1] o.a.s.c.SolrDeletionPolicy 
SolrDeletionPolicy.onCommit: commits: num=2
commit{dir=NRTCachingDirectory(MMapDirectory@/data/solr-5.5.0/server/solr/db_shard2_replica1/data/index 
lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c585e91; 
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
commit{dir=NRTCachingDirectory(MMapDirectory@/data/solr-5.5.0/server/solr/db_shard2_replica1/data/index 
lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c585e91; 
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
162656 INFO  (commitScheduler-14-thread-1) [c:db s:shard2 r:core_node2 
x:db_shard2_replica1] o.a.s.c.SolrDeletionPolicy newest commit 
generation = 2
162662 INFO  (commitScheduler-14-thread-1) [c:db s:shard2 r:core_node2 
x:db_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening 
Searcher@55cd91c3[db_shard2_replica1] realtime
162663 INFO  (commitScheduler-14-thread-1) [c:db s:shard2 r:core_node2 
x:db_shard2_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush



Here are my index files :

[root@LXLYOSOL30 ~]# ls -l 
/data/solr-5.5.0/server/solr/db_shard1_replica1/data/index/

total 124
-rw-rw-r-- 1 solr solr  253 22 mars  18:20 _0.fdt
-rw-rw-r-- 1 solr solr   84 22 mars  18:20 _0.fdx
-rw-rw-r-- 1 solr solr 2179 22 mars  18:20 _0.fnm
-rw-rw-r-- 1 solr solr  197 22 mars  18:20 _0_Lucene50_0.doc
-rw-rw-r-- 1 solr solr  113 22 mars  18:20 _0_Lucene50_0.pos
-rw-rw-r-- 1 solr solr 1907 22 mars  18:20 _0_Lucene50_0.tim
-rw-rw-r-- 1 solr solr  603 22 mars  18:20 _0_Lucene50_0.tip
-rw-rw-r-- 1 solr solr   59 22 mars  18:20 _0.nvd
-rw-rw-r-- 1 solr solr   98 22 mars  18:20 _0.nvm
-rw-rw-r-- 1 solr solr  491 22 mars  18:20 _0.si
-rw-rw-r-- 1 solr solr  252 22 mars  18:29 _1.fdt
-rw-rw-r-- 1 solr solr   84 22 mars  18:29 _1.fdx
-rw-rw-r-- 1 solr solr 2179 22 mars  18:29 _1.fnm
-rw-rw-r-- 1 solr solr  197 22 mars  18:29 _1_Lucene50_0.doc
-rw-rw-r-- 1 solr solr  113 22 mars  18:29 _1_Lucene50_0.pos
-rw-rw-r-- 1 solr solr 1907 22 mars  18:29 _1_Lucene50_0.tim
-rw-rw-r-- 1 solr solr  603 22 mars  18:29 _1_Lucene50_0.tip
-rw-rw-r-- 1 solr solr   59 22 mars  18:29 _1.nvd
-rw-rw-r-- 1 solr solr   98 22 mars  18:29 _1.nvm
-rw-rw-r-- 1 solr solr  491 22 mars  18:29 _1.si
-rw-rw-r-- 1 solr solr  244 22 mars  18:30 _2.fdt
-rw-rw-r-- 1 solr solr   84 22 mars  18:30 _2.fdx
-rw-rw-r-- 1 solr solr 2179 22 mars  18:30 _2.fnm
-rw-rw-r-- 1 solr solr  168 22 mars  18:30 _2_Lucene50_0.doc
-rw-rw-r-- 1 solr solr  101 22 mars  18:30 _2_Lucene50_0.pos
-rw-rw-r-- 1 solr solr 1849 22 mars  18:30 _2_Lucene50_0.tim
-rw-rw-r-- 1 solr solr  603 22 mars  18:30 _2_Lucene50_0.tip
-rw-rw-r-- 1 solr solr   59 22 mars  18:30 _2.nvd
-rw-rw-r-- 1 solr solr   98 22 mars  18:30 _2.nvm
-rw-rw-r-- 1 solr solr  491 22 mars  18:30 _2.si
-rw-rw-r-- 1 solr solr  227 22 mars  18:30 segments_4
-rw-rw-r-- 1 solr solr0 22 mars  18:18 write.lock


Any help ?

Regards
Victor



Ce message et les éventuels documents joints peuvent contenir des informations 
confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions 
de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute 
utilisation de ce message non conforme à sa destination, toute diffusion ou 
publication, totale ou partielle et quel qu'en soit le moyen est formellement 
interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité 
de ce message n'est pas assurée et la société émettrice ne peut être tenue pour 
responsable de son contenu.

Re: Cant access new docs without restarting Solr (java.nio.file.NoSuchFileException)

2016-03-23 Thread Victor D'agostino


Hi

It looks like this bug :
http://mail-archives.apache.org/mod_mbox/lucene-dev/201603.mbox/%3cjira.12933432.1453490528000.198540.1456947498...@atlassian.jira%3E

Regards


 Message original 
*Sujet: *Cant access new docs without restarting Solr 
(java.nio.file.NoSuchFileException)

*De : *Victor D'agostino 
*Pour : *solr-user@lucene.apache.org
*Date : *22/03/2016 18:39

Hi

I've setup a Solr Cloud 5.5.0 ensemble with ZooKeeper.

If I post a few docs with curl it seems ok :
[root@LXLYOSOL30 ~]# curl --noproxy '*' 
http://lxlyosol30:8983/solr/db/update --data-binary 
@/data/conf-cpm3/test.txt -H 'Content-type:application/xml'



0name="QTime">18




But when I go to the admin page on my first shard I got :
Luke is not configured
although I have in solrconfig.xml and ZooKeeper the line 
class="org.apache.solr.handler.admin.LukeRequestHandler" />



If I restart Solr i can see in the stats the new docs have been added !

Statistics :

Last Modified: 3 minutes ago
Num Docs:5
Max Doc:5
Heap Memory Usage:-1

Deleted Docs:0
Version:22
Segment Count:2

Instance :

CWD:/data/solr-5.5.0/server
Instance:/data/solr-5.5.0/server/solr/db_shard1_replica1
Data:/data/solr-5.5.0/server/solr/db_shard1_replica1/data
Index:/data/solr-5.5.0/server/solr/db_shard1_replica1/data/index
Impl:org.apache.solr.core.NRTCachingDirectoryFactory



In the logs I can see a java.nio.file.NoSuchFileException :

149746 INFO  (qtp609396627-17) [c:db s:shard1 r:core_node1 
x:db_shard1_replica1] o.a.s.c.S.Request [db_shard1_replica1] 
webapp=/solr path=/admin/file 
params={file=admin-extra.menu-bottom.html&_=1458667795848&contentType=text/html;charset%3Dutf-8} 
status=404 QTime=1
149780 ERROR (qtp609396627-20) [c:db s:shard1 r:core_node1 
x:db_shard1_replica1] o.a.s.h.RequestHandlerBase 
java.nio.file.NoSuchFileException: 
/data/solr-5.5.0/server/solr/db_shard1_replica1/data/index/segments_2
at 
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at 
sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at 
sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at 
sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:97)

at java.nio.file.Files.readAttributes(Files.java:1686)
at java.nio.file.Files.size(Files.java:2275)
at 
org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:210)
at 
org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:127)
at 
org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:592)
at 
org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:137)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:2082)
at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:670)
at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:458)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)

at org.eclipse.jetty.server.Server.handle(Server.java:499)
at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at 
org.eclip

Setting up a two nodes Solr Cloud 5.4.1 environment

2016-03-25 Thread Victor D'agostino


Hi guys

I am trying to set up a Solr Cloud environment of two Solr 5.4.1 nodes 
but the data are always indexed on the first node although the unique id 
is a GUID.


It looks like I can't add an additional node. Could you tell me where 
i'm wrong ?


I try to set up a collection named "db" with two shards on each node. 
Without replica. The config is named "copiemail3".



On node n°1 I put schema.xml, solrconfig.xml, etc. in 
$SOLRHOME/configsets/copiemail3/conf/

Then i do a upconfig to zkp1 with zkcli.sh
I start Solr and create my collection with the API.
 wget --no-proxy 
"http://$HOSTNAME:8983/solr/admin/collections?numShards=2&collection.configName=copiemail3&route.name=compositeId&maxShardsPerNode=2&router.field=mail_id&name=db&replicationFactor=1&action=CREATE";
My two first shards are created, Cloud is enabled and i also enable ping 
with the API :
 wget --no-proxy 
"http://$HOSTNAME:8983/solr/db_shard1_replica1/admin/ping?action=enable";
 wget --no-proxy 
"http://$HOSTNAME:8983/solr/db_shard2_replica1/admin/ping?action=enable";

Finally I restart Solr

On node n°2
I start Solr and create the two shards with the cores API (collections 
API won't work because i use compositeId routing mode) :
 wget --no-proxy 
"http://$HOSTNAME:8983/solr/admin/cores?action=CREATE&schema=schema.xml&shard=shard3&instanceDir=db_shard3_replica1&indexInfo=false&name=db_shard3_replica1&config=solrconfig.xml&collection=db&dataDir=data";
 wget --no-proxy 
"http://$HOSTNAME:8983/solr/admin/cores?action=CREATE&schema=schema.xml&shard=shard4&instanceDir=db_shard4_replica1&indexInfo=false&name=db_shard4_replica1&config=solrconfig.xml&collection=db&dataDir=data";

Like node 1 i activate the ping and restart Solr.

On each Solr admin interface I can see ZooKeeper config is good (4 
alived nodes). Cloud schema seems ok because i have my db collection, 4 
shards on two nodes (all leaders).



Search request are well distributed but as I said before the data are 
always indexed on the two shards on the first node :




[root@node1 ~]# /data/solr-5.4.1/bin/solr healthcheck -c db -z 
localhost:2181

{
  "collection":"db",
  "status":"healthy",
  "numDocs":1,
  "numShards":4,
  "shards":[
{
  "shard":"shard1",
  "status":"healthy",
  "replicas":[{
  "name":"core_node2",
  "url":"http://10.69.220.46:8983/solr/db_shard1_replica1/";,
  "numDocs":5175,
  "status":"active",
  "uptime":"0 days, 0 hours, 14 minutes, 35 seconds",
  "memory":"339.3 MB (%14.1) of 2.4 GB",
  "leader":true}]},
{
  "shard":"shard2",
  "status":"healthy",
  "replicas":[{
  "name":"core_node1",
  "url":"http://10.69.220.46:8983/solr/db_shard2_replica1/";,
  "numDocs":4825,
  "status":"active",
  "uptime":"0 days, 0 hours, 14 minutes, 35 seconds",
  "memory":"339.3 MB (%14.1) of 2.4 GB",
  "leader":true}]},
{
  "shard":"shard3",
  "status":"healthy",
  "replicas":[{
  "name":"core_node3",
  "url":"http://10.69.220.47:8983/solr/db_shard3_replica1/";,
  "numDocs":0,
  "status":"active",
  "uptime":"0 days, 0 hours, 13 minutes, 44 seconds",
  "memory":"177 MB (%7.4) of 2.4 GB",
  "leader":true}]},
{
  "shard":"shard4",
  "status":"healthy",
  "replicas":[{
  "name":"core_node4",
  "url":"http://10.69.220.47:8983/solr/db_shard4_replica1/";,
  "numDocs":0,
  "status":"active",
  "uptime":"0 days, 0 hours, 13 minutes, 44 seconds",
  "memory":"177 MB (%7.4) of 2.4 GB",
  "leader":true}]}]}


Thanks for your help

Victor




Ce message et les éventuels documents joints peuvent contenir des informations confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou publication, totale ou partielle et quel qu'en soit le moyen est formellement interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité de ce message n'est pas assurée et la société émettrice ne peut être tenue pour responsable de son contenu.

Re: Setting up a two nodes Solr Cloud 5.4.1 environment

2016-03-29 Thread Victor D'agostino


Hi Erick

Thanks for your help, here is what I've done.

1. I deleted zookeepers and Solr installations.
2. I setup zookeepers on my two servers.
3. I successfully setup Solr Cloud node 1 with the same API call (1 
collection named db and two cores) :
 wget --no-proxy 
"http://$HOSTNAME:8983/solr/admin/collections?numShards=2&collection.configName=copiemail3&route.name=compositeId&maxShardsPerNode=2&router.field=mail_id&name=db&replicationFactor=1&action=CREATE";


4. I didn't use the core API anymore.
I tried to setup node 2 with the collection API 
<https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api8> 
and here is the error message (shards can be added only to 'implicit' 
collections) :


*Request :*
wget --no-proxy 
"http://$HOSTNAME:8983/solr/admin/collections?action=CREATESHARD&collection=db&shard=db_shard3_replica1";

*
**Error log**:*
2016-03-29 08:49:09.422 INFO  (qtp2085805465-13) [   ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :createshard with 
params shard=db_shard3_replica1&action=CREATESHARD&collection=db
2016-03-29 08:49:09.425 ERROR (qtp2085805465-13) [   ] 
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: shards 
can be added only to 'implicit' collections
at 
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation$10.call(CollectionsHandler.java:468)
at 
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:176)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)

at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:438)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:223)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)

at org.eclipse.jetty.server.Server.handle(Server.java:499)
at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)

at java.lang.Thread.run(Thread.java:745)


If I do a status check on Solr node 2 (lxlyosol31) I can see ZooKeeper 
is ok but node 2 is not in the cluster :


/etc/init.d/solr status

Found 1 Solr nodes:

Solr process 3883 running on port 8983
{
  "solr_home":"/data/solr-5.4.1/server/solr",
  "version":"5.4.1 1725212 - jpountz - 2016-01-18 11:51:45",
  "startTime":"2016-03-29T07:49:15.192Z",
  "uptime":"0 days, 0 hours, 7 minutes, 48 seconds",
  "memory":"259.7 MB (%10.8) of 2.4 GB",
  "cloud":{
"ZooKeeper":"lxlyosol30:2181,lxlyosol31:2181",
"liveNodes":"2",
"collections":"1"}}


Regards
Victor


 Message original 
*Sujet: *Re: Setting up a two nodes Solr Cloud 5.4.1 environment
*De : *Erick Erickson 
*Pour : *solr-user 
*Date : *25/03/2016 19:44

bq:  (collections API won't work because i use compositeId routing mode)

This had better NOT be true or SolrCloud is horribly broken. compositeId is
the default and it's tested a all the time by unit tests. So is implicit for
that matter.

One question I

Re: Setting up a two nodes Solr Cloud 5.4.1 environment

2016-03-29 Thread Victor D'agostino


Hi guys

It seems I tried to add two additional shards on a existing Solr 
ensemble and this is not supported (or I didn't find how).


So after setting ZooKeeper I first setup my node n°2 and then setup my 
node n°1 with
wget --no-proxy 
"http://node1:8983/solr/admin/collections?&collection.configName=x&name=db&replicationFactor=1&action=CREATE&numShards=4&maxShardsPerNode=2";


Because node n°2 was already up then two shards are created on each node.

Regards
Victor

 Message original 
*Sujet: *Re: Setting up a two nodes Solr Cloud 5.4.1 environment
*De : *Victor D'agostino 
*Pour : *solr-user@lucene.apache.org
*Copie à : *Erick Erickson 
*Date : *29/03/2016 09:58

Hi Erick

Thanks for your help, here is what I've done.

1. I deleted zookeepers and Solr installations.
2. I setup zookeepers on my two servers.
3. I successfully setup Solr Cloud node 1 with the same API call (1 
collection named db and two cores) :
 wget --no-proxy 
"http://$HOSTNAME:8983/solr/admin/collections?numShards=2&collection.configName=copiemail3&route.name=compositeId&maxShardsPerNode=2&router.field=mail_id&name=db&replicationFactor=1&action=CREATE";


4. I didn't use the core API anymore.
I tried to setup node 2 with the collection API 
<https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api8> 
and here is the error message (shards can be added only to 'implicit' 
collections) :


*Request :*
wget --no-proxy 
"http://$HOSTNAME:8983/solr/admin/collections?action=CREATESHARD&collection=db&shard=db_shard3_replica1";

*
**Error log**:*
2016-03-29 08:49:09.422 INFO  (qtp2085805465-13) [   ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :createshard 
with params shard=db_shard3_replica1&action=CREATESHARD&collection=db
2016-03-29 08:49:09.425 ERROR (qtp2085805465-13) [   ] 
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: 
shards can be added only to 'implicit' collections
at 
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation$10.call(CollectionsHandler.java:468)
at 
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:176)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:438)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:223)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)

at org.eclipse.jetty.server.Server.handle(Server.java:499)
at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)

at java.lang.Thread.run(Thread.java:745)


If I do a status check on Solr node 2 (lxlyosol31) I can see ZooKeeper 
is ok but node 2 is not in the cluster :


/etc/init.d/solr status

Found 1 Solr nodes:

Solr process 3883 running on port 8983
{
  "solr_home":"/data/solr-5.4.1/server/solr",
  "version":"5.4.1 1725212 - jpountz - 2016-01-18 11:51:45",
  "startTime":"2016-03-29T07:49:15.192Z",
  "uptime

Error opening new searcher

2016-05-20 Thread Victor D'agostino


Hi guys

What doest this "try again later" log means in solr--console.log :

193899678 WARN  (qtp1393423910-18329) [c:db s:shard3 r:core_node3 
x:db_shard3_replica1] o.a.s.u.p.DistributedUpdateProcessor Error sending 
update to http://10.69.212.22:8983/solr
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: 
Error from server at http://10.69.212.22:8983/solr/db_shard3_replica1: 
Error opening new searcher. exceeded limit of maxWarmingSearchers=2, try 
again later.


Am I supposed to resend the document or will it be inserted just fine 
later ?


And is it possible to set the log timestamp "193899678" to a human 
readable format ?


Regards

Victor



Ce message et les éventuels documents joints peuvent contenir des informations confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou publication, totale ou partielle et quel qu'en soit le moyen est formellement interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité de ce message n'est pas assurée et la société émettrice ne peut être tenue pour responsable de son contenu.

Re: Error opening new searcher

2016-05-20 Thread Victor D'agostino


Hi Shawn

Ok I am going to comit less often then.

I have planned to set the console log from INFO to WARN but this kind of 
log was not in the solr.log regular log file !


Regards
Victor

 Message original 
*Sujet: *Re: Error opening new searcher
*De : *Shawn Heisey 
*Pour : *solr-user@lucene.apache.org
*Date : *20/05/2016 11:40

On 5/20/2016 1:46 AM, Victor D'agostino wrote:

What doest this "try again later" log means in solr--console.log :

You should really disable console logging entirely.  I assume you're
running at least version 5.0?


193899678 WARN  (qtp1393423910-18329) [c:db s:shard3 r:core_node3
x:db_shard3_replica1] o.a.s.u.p.DistributedUpdateProcessor Error
sending update to http://10.69.212.22:8983/solr
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://10.69.212.22:8983/solr/db_shard3_replica1:
Error opening new searcher. exceeded limit of maxWarmingSearchers=2,
try again later.

Am I supposed to resend the document or will it be inserted just fine
later ?

Most likely the update itself was fine -- the error was when opening a
new searcher, which is something that happens at commit time.  You would
need to check the solr.log file on the server with address 10.69.212.22
to be sure.  This particular error message means that you are committing
too frequently -- two previous commits with openSearcher=true were not
yet finished before a third commit with openSearcher=true was started.


And is it possible to set the log timestamp "193899678" to a human
readable format ?

Check the timestamp in the solr.log file.  Like I said above -- the
console log should be disabled entirely.  You should be able to remove
it as a logging destination by editing resources/log4j.properties
(assuming 5.x or 6.x).

Thanks,
Shawn






Ce message et les éventuels documents joints peuvent contenir des informations 
confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions 
de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute 
utilisation de ce message non conforme à sa destination, toute diffusion ou 
publication, totale ou partielle et quel qu'en soit le moyen est formellement 
interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité 
de ce message n'est pas assurée et la société émettrice ne peut être tenue pour 
responsable de son contenu.

Re: Error opening new searcher

2016-05-23 Thread Victor D'agostino

Hi Erick
Thanks for your help, it is alright now.

Have a good day
Victor

Message original
*Sujet: *Re: Error opening new searcher
*De : *Erick Erickson
*Pour : *solr-user
*Date : *20/05/2016 17:57

Actually, it almost certainly _is_ in the regular Solr log file, just
which one. The file logging
rolls over, which is why you have solr.log, solr.log.1 etc. Likely the
message is in one of
those unless it happened a long time ago. Those Solr logs are really a
window that spans
some time period, how long depends on how much log traffic you're
generating. If you
need a longer window, adjust the log4j.properties file.

The console log file will accumulate all messages forever, which is why Shawn
recommends you disable it. If you monitor it and set just the console
logging to WARN
it won't grow very quickly (at least it better not) but you then have to monitor
it yourself.

Best,
Erick

On Fri, May 20, 2016 at 4:19 AM, Victor D'agostino
wrote:

Hi Shawn

Ok I am going to comit less often then.

I have planned to set the console log from INFO to WARN but this kind of log
was not in the solr.log regular log file !

Regards
Victor

Message original
*Sujet: *Re: Error opening new searcher
*De : *Shawn Heisey
*Pour : *solr-user@lucene.apache.org
*Date : *20/05/2016 11:40

On 5/20/2016 1:46 AM, Victor D'agostino wrote:

What doest this "try again later" log means in solr--console.log :

You should really disable console logging entirely. I assume you're
running at least version 5.0?

193899678 WARN (qtp1393423910-18329) [c:db s:shard3 r:core_node3
x:db_shard3_replica1] o.a.s.u.p.DistributedUpdateProcessor Error
sending update to
https://urldefense.proofpoint.com/v2/url?u=http-3A__10.69.212.22-3A8983_solr&d=CwIFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=GIbD6pb1nH9ZrxFDfhl_c8kJe7NkpbmXG1YHXBYFth8&m=XvSFBhKqCFis7ZDQP0Npk9DWx5YWGwcP1wx5ZWQMbm4&s=0LMx0511pQ6Au2CeRhtuzD-zDGiCaxoBE0MnbvneS5I&e=
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at
https://urldefense.proofpoint.com/v2/url?u=http-3A__10.69.212.22-3A8983_solr_db-5Fshard3-5Freplica1-3A&d=CwIFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=GIbD6pb1nH9ZrxFDfhl_c8kJe7NkpbmXG1YHXBYFth8&m=XvSFBhKqCFis7ZDQP0Npk9DWx5YWGwcP1wx5ZWQMbm4&s=GqsWQqbS0ryjcM6Cb_zPBkg0oljISX0dNnIjuJBoR1A&e=
Error opening new searcher. exceeded limit of maxWarmingSearchers=2,
try again later.

Am I supposed to resend the document or will it be inserted just fine
later ?

Most likely the update itself was fine -- the error was when opening a
new searcher, which is something that happens at commit time. You would
need to check the solr.log file on the server with address 10.69.212.22
to be sure. This particular error message means that you are committing
too frequently -- two previous commits with openSearcher=true were not
yet finished before a third commit with openSearcher=true was started.

And is it possible to set the log timestamp "193899678" to a human
readable format ?

Check the timestamp in the solr.log file. Like I said above -- the
console log should be disabled entirely. You should be able to remove
it as a logging destination by editing resources/log4j.properties
(assuming 5.x or 6.x).

Thanks,
Shawn

Ce message et les éventuels documents joints peuvent contenir des
informations confidentielles. Au cas où il ne vous serait pas destiné, nous
vous remercions de bien vouloir le supprimer et en aviser immédiatement
l'expéditeur. Toute utilisation de ce message non conforme à sa destination,
toute diffusion ou publication, totale ou partielle et quel qu'en soit le
moyen est formellement interdite. Les communications sur internet n'étant
pas sécurisées, l'intégrité de ce message n'est pas assurée et la société
émettrice ne peut être tenue pour responsable de son contenu.

Ce message et les éventuels documents joints peuvent contenir des informations
confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions
de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute
utilisation de ce message non conforme à sa destination, toute diffusion ou
publication, totale ou partielle et quel qu'en soit le moyen est formellement
interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité
de ce message n'est pas assurée et la société émettrice ne peut être tenue pour
responsable de son contenu.

json.nl=arrarr json formating option with the php lib

2016-06-23 Thread Victor D'agostino


Hi guyes

I try to add the json.nl=arrarr option in a solr request in addition of 
wt=json but i can not find how to do this using the php lib.


Is there any option or way to create a custom option field ?

Regards

Victor






Ce message et les éventuels documents joints peuvent contenir des informations confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou publication, totale ou partielle et quel qu'en soit le moyen est formellement interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité de ce message n'est pas assurée et la société émettrice ne peut être tenue pour responsable de son contenu.

Re: json.nl=arrarr json formating option with the php lib

2016-06-23 Thread Victor D'agostino


Hi again

Same question for the "indent" parameters i'd like to turn off.

Regards

Victor

 Message original 
*Sujet: *json.nl=arrarr json formating option with the php lib
*De : *Victor D'agostino 
*Pour : *solr-user@lucene.apache.org
*Date : *23/06/2016 16:31

Hi guys

I try to add the json.nl=arrarr option in a solr request in addition 
of wt=json but i can not find how to do this using the php lib.


Is there any option or way to create a custom option field ?

Regards

Victor










Ce message et les éventuels documents joints peuvent contenir des informations 
confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions 
de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute 
utilisation de ce message non conforme à sa destination, toute diffusion ou 
publication, totale ou partielle et quel qu'en soit le moyen est formellement 
interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité 
de ce message n'est pas assurée et la société émettrice ne peut être tenue pour 
responsable de son contenu.

Re: json.nl=arrarr json formating option with the php lib

2016-06-23 Thread Victor D'agostino


Hi guys

SolrQuery implements SolrModifiableParams so we can use the setParam 
method :


$solr_query = new SolrQuery();
$solr_query->setParam('json.nl', 'arrarr');
$solr_query->setParam('indent', 'off');

json.nl is ok but the php lib force indent to "on" so the request will 
looks like json.nl=arrarr&indent=on&indent=off resulting with 
indentation enabled in contradiction with
http://php.net/manual/fr/solrparams.setparam.php which say setParam will 
erase previous value and add the new one.


Regards
Victor


 Message original 
*Sujet: *Re: json.nl=arrarr json formating option with the php lib
*De : *Victor D'agostino 
*Pour : *solr-user@lucene.apache.org
*Date : *23/06/2016 16:50

Hi again

Same question for the "indent" parameters i'd like to turn off.

Regards

Victor

 Message original ----
*Sujet: *json.nl=arrarr json formating option with the php lib
*De : *Victor D'agostino 
*Pour : *solr-user@lucene.apache.org
*Date : *23/06/2016 16:31

Hi guys

I try to add the json.nl=arrarr option in a solr request in addition 
of wt=json but i can not find how to do this using the php lib.


Is there any option or way to create a custom option field ?

Regards

Victor










Ce message et les éventuels documents joints peuvent contenir des 
informations confidentielles. Au cas où il ne vous serait pas destiné, 
nous vous remercions de bien vouloir le supprimer et en aviser 
immédiatement l'expéditeur. Toute utilisation de ce message non 
conforme à sa destination, toute diffusion ou publication, totale ou 
partielle et quel qu'en soit le moyen est formellement interdite. Les 
communications sur internet n'étant pas sécurisées, l'intégrité de ce 
message n'est pas assurée et la société émettrice ne peut être tenue 
pour responsable de son contenu. 





Ce message et les éventuels documents joints peuvent contenir des informations 
confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions 
de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute 
utilisation de ce message non conforme à sa destination, toute diffusion ou 
publication, totale ou partielle et quel qu'en soit le moyen est formellement 
interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité 
de ce message n'est pas assurée et la société émettrice ne peut être tenue pour 
responsable de son contenu.

How to update from Solr Cloud 5.4.1 to 5.5.1

2016-08-26 Thread D'agostino Victor


Hi guys

I've got a tree nodes Solr Cloud 5.4.1 with zookeeper 3.4.8 in 
production serving 72.000.000 documents.
Documents types are easy ones : string, date, text, boolean, and multi 
valued string but reindexing would take two weeks.


I would like to upgrade solr to (at least) version 5.5.1 for a bug fix 
(https://issues.apache.org/jira/browse/SOLR-8779).


How can I do that ? Is there a safe procedure anywhere ?

Best regards
Victor d'Agostino





Ce message et les éventuels documents joints peuvent contenir des informations confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou publication, totale ou partielle et quel qu'en soit le moyen est formellement interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité de ce message n'est pas assurée et la société émettrice ne peut être tenue pour responsable de son contenu.

Re: How to update from Solr Cloud 5.4.1 to 5.5.1

2016-08-26 Thread D'agostino Victor

Hi Erick

Thanks for your reply.
That's what I though but i wasn't sure :)

Do you know in which version index format changes and if I should update
to a higher version ?

And about ZooKeeper ; the 3.4.8 is fine or should I update it too ?

Have a good day
Victor

Message original
*Sujet: *Re: How to update from Solr Cloud 5.4.1 to 5.5.1
*De : *Erick Erickson
*Pour : *solr-user
*Date : *26/08/2016 17:40

First of course I would always back up my indexes, but then I'm paranoid.

But 5.5.1 should be drop-in for 5.4.1. There are no index format changes
you need to worry about. You can install 5.5.1 in a new directory on your
box and start it up with the same SOLR_HOME as your 5.4.1 setup (after
shutting down the Solr 5.4.1 of course) and you should be fine.

Best,
Erick

2016-08-26 4:26 GMT-07:00 D'agostino Victor :

Hi guys

I've got a tree nodes Solr Cloud 5.4.1 with zookeeper 3.4.8 in production
serving 72.000.000 documents.
Documents types are easy ones : string, date, text, boolean, and multi
valued string but reindexing would take two weeks.

I would like to upgrade solr to (at least) version 5.5.1 for a bug fix
(https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SOLR-2D8779&d=DQIFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=GIbD6pb1nH9ZrxFDfhl_c8kJe7NkpbmXG1YHXBYFth8&m=64wqAkfWd4MYvnvXDQtmqtauM4cEjSJovZ_5MMxbzGY&s=N9YtmP-PRMS6F_VQtrm2ClVsFocMf4dcnN0buu-AmXQ&e=
).

How can I do that ? Is there a safe procedure anywhere ?

Best regards
Victor d'Agostino

Ce message et les éventuels documents joints peuvent contenir des informations
confidentielles. Au cas où il ne vous serait pas destiné, nous vous remercions
de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute
utilisation de ce message non conforme à sa destination, toute diffusion ou
publication, totale ou partielle et quel qu'en soit le moyen est formellement
interdite. Les communications sur internet n'étant pas sécurisées, l'intégrité
de ce message n'est pas assurée et la société émettrice ne peut être tenue pour
responsable de son contenu.

SolrIndexSearcher#getDocList() method returns zero results, if query includes tdate range query

2017-04-25 Thread Victor Solakhian

We have code that uses *SolrIndexSearcher#getDocList()* method to get
document IDs for the query.

First a Solr query string is generated from UI, then the following code
creates a Lucene Query

org.apache.lucene.search.Query query = parser.parse(solrQueryString);

where parser is org.apache.lucene.queryparser.classic.QueryParser and then
the following is used to get the document IDs:

DocList docList = indexSearcher.getDocList(query, filterList, sort,
start, length, 0);

The code worked perfectly in Solr 4.5. Now, in Solr 5.5.4, it works only if
the query does not contain a date range query. For example, solrQueryString:

"(+c_class:(Industry.government)) AND +valid_date:[2015-10-21 TO
2017-04-21] AND -class:(TitleCodeMiddle.Board) AND +company:[* TO *] AND
+(has_email:(true) OR has_phone:(true) OR c_has_phone:(true)) AND
+c_class:(Industry.government)"

was parsed to the Lucene query:

"+(+c_class:industry.government) +valid_date:[2015-10-21 TO 2017-04-21]
-class:titlecodemiddle.board +company:[* TO *] +(has_email:T has_phone:T
c_has_phone:T) +c_class:industry.government"

which contains "+valid_date:[2015-10-21 TO 2017-04-21]", returns ZERO
results, although the same query (actually the Solr equivalent) returns
3326 records when used in Solr Admin.

Here is the definition of the "valid_date" field:

   

   

For a similar query without the range query:

+(+c_class:industry.government) -class:titlecodemiddle.board +company:[* TO
*] +(has_email:T has_phone:T c_has_phone:T) +c_class:industry.government

our code returns 5629 results (same as Solr Admin).

I tried to use different formats for date in the Solr query (according to
what I was able to find on the web for Lucene date format):

   - "+valid_date:[2015-10-21 TO 2017-04-21]"
   - "+valid_date:[20151021 TO 20170421]"
   - "+valid_date:[2015-10-21T04:00:00.000 TO 2017-04-21T04:00:00.000]"
   - "+valid_date:[2015-10-21T04:00:00.000Z TO 2017-04-21T04\:00\:00]"
   - "+valid_date:[2015-10-21T00:00:00Z TO 2017-04-21T00:00:00Z]"

Just out of curiosity, I even generated "+valid_date:[XXX TO XXX]" just to
see that SolrIndexSearcher#getDocList() method does not check for correct
syntax and returns ZERO results.

Does anybody know what is happening and what is the proper date format for
Lucene range query in v. 5.5.4?

Thanks,

Victor

Re: SolrIndexSearcher#getDocList() method returns zero results, if query includes tdate range query

2017-04-25 Thread Victor Solakhian

Rick,

Solr Admin does not generate a query. I use it just to confirm that the
query generated by our code returns results.

getDocList() method also does not generate a query, It returns a list of
document IDs for the query created by the QueryParser.parse(query,...).
method.

Thanks,

Victor

On Tue, Apr 25, 2017 at 12:44 PM, Rick Leir  wrote:

> Victor,
> When you do a query in SolrAdmin, the generated query is shown in at the
> top of the page. Can you compare that with the query that getDocList
> generates? Or did I misunderstand your question.
> Cheers -- Rick
>
> On April 25, 2017 11:34:17 AM EDT, Victor Solakhian 
> wrote:
> >We have code that uses *SolrIndexSearcher#getDocList()* method to get
> >document IDs for the query.
> >
> >First a Solr query string is generated from UI, then the following code
> >creates a Lucene Query
> >
> >  org.apache.lucene.search.Query query = parser.parse(solrQueryString);
> >
> >where parser is org.apache.lucene.queryparser.classic.QueryParser and
> >then
> >the following is used to get the document IDs:
> >
> >DocList docList = indexSearcher.getDocList(query, filterList, sort,
> >start, length, 0);
> >
> >The code worked perfectly in Solr 4.5. Now, in Solr 5.5.4, it works
> >only if
> >the query does not contain a date range query. For example,
> >solrQueryString:
> >
> >"(+c_class:(Industry.government)) AND +valid_date:[2015-10-21 TO
> >2017-04-21] AND -class:(TitleCodeMiddle.Board) AND +company:[* TO *]
> >AND
> >+(has_email:(true) OR has_phone:(true) OR c_has_phone:(true)) AND
> >+c_class:(Industry.government)"
> >
> >was parsed to the Lucene query:
> >
> >"+(+c_class:industry.government) +valid_date:[2015-10-21 TO 2017-04-21]
> >-class:titlecodemiddle.board +company:[* TO *] +(has_email:T
> >has_phone:T
> >c_has_phone:T) +c_class:industry.government"
> >
> >which contains "+valid_date:[2015-10-21 TO 2017-04-21]", returns ZERO
> >results, although the same query (actually the Solr equivalent) returns
> >3326 records when used in Solr Admin.
> >
> >Here is the definition of the "valid_date" field:
> >
> > 
> >
> >>sortMissingLast="true" precisionStep="6" positionIncrementGap="0"
> >omitNorms="true"/>
> >
> >For a similar query without the range query:
> >
> >+(+c_class:industry.government) -class:titlecodemiddle.board
> >+company:[* TO
> >*] +(has_email:T has_phone:T c_has_phone:T)
> >+c_class:industry.government
> >
> >our code returns 5629 results (same as Solr Admin).
> >
> >I tried to use different formats for date in the Solr query (according
> >to
> >what I was able to find on the web for Lucene date format):
> >
> >   - "+valid_date:[2015-10-21 TO 2017-04-21]"
> >   - "+valid_date:[20151021 TO 20170421]"
> >   - "+valid_date:[2015-10-21T04:00:00.000 TO 2017-04-21T04:00:00.000]"
> >   - "+valid_date:[2015-10-21T04:00:00.000Z TO 2017-04-21T04\:00\:00]"
> >   - "+valid_date:[2015-10-21T00:00:00Z TO 2017-04-21T00:00:00Z]"
> >
> >Just out of curiosity, I even generated "+valid_date:[XXX TO XXX]" just
> >to
> >see that SolrIndexSearcher#getDocList() method does not check for
> >correct
> >syntax and returns ZERO results.
> >
> >Does anybody know what is happening and what is the proper date format
> >for
> >Lucene range query in v. 5.5.4?
> >
> >Thanks,
> >
> >Victor
>
> --
> Sorry for being brief. Alternate email is rickleir at yahoo dot com

Re: SolrIndexSearcher#getDocList() method returns zero results, if query includes tdate range query

2017-04-25 Thread Victor Solakhian

Hi Chris,

I think you are leading me to  the right direction.

I'm not sure how that would have worked in Solr 4.5, ... unless
> perhaps your definition of a "date" field was different in the schema's
> you used in 4.5, and did not involve a Trie based date field  (the very
> old legacy date format ields used a simple String based encoding that
> might have worked)


You are right. In Solr 4.5 we had:




I will need some time to digest all information you provided. I will let
you know.

Thank you very much.

Victor



On Tue, Apr 25, 2017 at 2:45 PM, Chris Hostetter 
wrote:

> : The correct way for a plugin to do the sort of thing you are trying to do
> : would be to use an instance of SolrQueryParser -- see for example the
> code
> : in LuceneQParser and how it uses SolrQueryParser ... you'll most likeley
> : just want to use LuceneQParser directly in your plugin to simplify
> things.
>
> ...or depending on how low level you want to deal with things, consider
> using IndexSchema.getField(...).getFieldType().getRangeQuery(null, ...) to
> build your range Query object directly from the low/high end points
> provided as input instead of needing to build a string just to parse it
> again.
>
>
> -Hoss
> http://www.lucidworks.com/
>

Re: SolrIndexSearcher#getDocList() method returns zero results, if query includes tdate range query

2017-04-28 Thread Victor Solakhian

I used org.apache.solr.search.LuceneQParser instead of
org.apache.lucene.queryparser.classic.QueryParser and now our code works.
Here are some excerpts:
...
QParser qParser = getParser(core, solrQueryString);
Query query = qParser.parse();
...

private QParser getParser(final SolrCore corer, final String queryStr) {
SolrQueryRequest req = new LocalSolrQueryRequest(core, queryStr,
null, 0, -1, new HashMap());

QParser qParser = null;
try {
qParser = LuceneQParser.getParser(queryStr, null, req);
} catch (SyntaxError e) {
log.error("Failed to get LuceneQParser", e);
}

return qParser;
}

Thanks again,

Victor

On Tue, Apr 25, 2017 at 6:19 PM, Victor Solakhian 
wrote:

> Hi Chris,
>
> I think you are leading me to  the right direction.
>
> I'm not sure how that would have worked in Solr 4.5, ... unless
>> perhaps your definition of a "date" field was different in the schema's
>> you used in 4.5, and did not involve a Trie based date field  (the very
>> old legacy date format ields used a simple String based encoding that
>> might have worked)
>
>
> You are right. In Solr 4.5 we had:
>
> 
>  omitNorms="true"/>
>
> I will need some time to digest all information you provided. I will let
> you know.
>
> Thank you very much.
>
> Victor
>
>
>
> On Tue, Apr 25, 2017 at 2:45 PM, Chris Hostetter  > wrote:
>
>> : The correct way for a plugin to do the sort of thing you are trying to
>> do
>> : would be to use an instance of SolrQueryParser -- see for example the
>> code
>> : in LuceneQParser and how it uses SolrQueryParser ... you'll most likeley
>> : just want to use LuceneQParser directly in your plugin to simplify
>> things.
>>
>> ...or depending on how low level you want to deal with things, consider
>> using IndexSchema.getField(...).getFieldType().getRangeQuery(null, ...)
>> to
>> build your range Query object directly from the low/high end points
>> provided as input instead of needing to build a string just to parse it
>> again.
>>
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>
>

Solr does not recognize language

2014-04-29 Thread Victor Pascual

Dear all,

I'm a new user of Solr. I've managed to index a bunch of documents (in
fact, they are tweets) and everything works quite smoothly.

Nevertheless it looks like Solr doesn't detect the language of my documents
nor remove stopwords accordingly so I can extract the most frequent terms.

I've added this piece of XML to my solrconfig.xml as well as the Tika lib
jars.


   
  
text
lang
  


   
 

There is no error in the tomcat log file, so I have no clue of why this
isn't working.
Any hint on how to solve this problem will be much appreciated!

Re: Solr does not recognize language

2014-04-29 Thread Victor Pascual

Hi Ahmet,

thanks for your reply. Adding &update.chain=langid to my query doesn't
work: IP:8080/solr/select/?q=*%3A*&update.chain=langid
Regarding defining the chain in an UpdateRequestHandler... sorry for the
lame question but shall I paste those three lines to solrconfig.xml, or
shall I add them somewhere else?

There is not UpdateRequestHandler in my solrconfig.

Thanks!


On Tue, Apr 29, 2014 at 3:13 PM, Ahmet Arslan  wrote:

> Hi,
>
> Did you attach your chain to a UpdateRequestHandler?
>
> You can do it by adding &update.chain=langid to the URL or defining it in
> a defaults section as follows
>
> 
>  langid
>    
>
>
>
> On Tuesday, April 29, 2014 3:18 PM, Victor Pascual <
> vic...@mobilemediacontent.com> wrote:
> Dear all,
>
> I'm a new user of Solr. I've managed to index a bunch of documents (in
> fact, they are tweets) and everything works quite smoothly.
>
> Nevertheless it looks like Solr doesn't detect the language of my documents
> nor remove stopwords accordingly so I can extract the most frequent terms.
>
> I've added this piece of XML to my solrconfig.xml as well as the Tika lib
> jars.
>
> 
>
> class="org.apache.solr.update.processor.LangDetectLanguageIdentifierUpdateProcessorFactory">
>   
> text
> lang
>   
> 
> 
>
>  
>
> There is no error in the tomcat log file, so I have no clue of why this
> isn't working.
> Any hint on how to solve this problem will be much appreciated!
>

Re: Solr does not recognize language

2014-05-05 Thread Victor Pascual

Thank you very much for you help Ahmet.

However the language detection is still not workin. :(
My solrconfig.xml didn't contain that lst section inside the update
requestHandler.
That's the content I added:

 class="solr.XmlUpdateRequestHandler">
>
>  langid
>
> 
>


>
> class="org.apache.solr.update.processor.LangDetectLanguageIdentifierUpdateProcessorFactory">
>   
> text
> lang
>   
> 
> 
>
>  


Now, your suggested query
http://localhost:8080/solr/update?commit=true&update.chain=langid returns


> 
> 0
> 14
> 
> 

And there is still no lang field in my documents.
Any idea what am I doing wrong?



On Tue, Apr 29, 2014 at 5:33 PM, Ahmet Arslan  wrote:

> Hi,
>
> solr/update should be used, not /solr/select
>
> curl 'http://localhost:8983/solr/update?commit=true&update.chain=langid'
>
> By the way don't you have following definition in your solrconfig.xml?
>
>  
>
>  langid
>
>   
>
>
>
> On Tuesday, April 29, 2014 4:50 PM, Victor Pascual <
> vic...@mobilemediacontent.com> wrote:
> Hi Ahmet,
>
> thanks for your reply. Adding &update.chain=langid to my query doesn't
> work: IP:8080/solr/select/?q=*%3A*&update.chain=langid
> Regarding defining the chain in an UpdateRequestHandler... sorry for the
> lame question but shall I paste those three lines to solrconfig.xml, or
> shall I add them somewhere else?
>
> There is not UpdateRequestHandler in my solrconfig.
>
> Thanks!
>
>
>
> On Tue, Apr 29, 2014 at 3:13 PM, Ahmet Arslan  wrote:
>
> > Hi,
> >
> > Did you attach your chain to a UpdateRequestHandler?
> >
> > You can do it by adding &update.chain=langid to the URL or defining it in
> > a defaults section as follows
> >
> > 
> >  langid
> >
> >
> >
> >
> > On Tuesday, April 29, 2014 3:18 PM, Victor Pascual <
> > vic...@mobilemediacontent.com> wrote:
> > Dear all,
> >
> > I'm a new user of Solr. I've managed to index a bunch of documents (in
> > fact, they are tweets) and everything works quite smoothly.
> >
> > Nevertheless it looks like Solr doesn't detect the language of my
> documents
> > nor remove stopwords accordingly so I can extract the most frequent
> terms.
> >
> > I've added this piece of XML to my solrconfig.xml as well as the Tika lib
> > jars.
> >
> > 
> > >
> >
> class="org.apache.solr.update.processor.LangDetectLanguageIdentifierUpdateProcessorFactory">
> >   
> > text
> > lang
> >   
> > 
> > 
> >
> >  
> >
> > There is no error in the tomcat log file, so I have no clue of why this
> > isn't working.
> > Any hint on how to solve this problem will be much appreciated!
> >
>
>

Re: Solr does not recognize language

2014-05-05 Thread Victor Pascual

Why this should be a problem?
Both files start with 


On Mon, May 5, 2014 at 11:44 AM, Frankcis  wrote:

> i think you should check your scheme.xml and solrconfig.xml encoding
> format =
> utf-8。
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-does-not-recognize-language-tp4133711p4134643.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Solr does not recognize language

2014-05-05 Thread Victor Pascual

Hi there,

I'm indexing my documents using mysolr. I mainly generate a lost of json
objects and the run: solr.update(documents_array,'json')


On Mon, May 5, 2014 at 1:08 PM, Ahmet Arslan  wrote:

> Hi Victor,
>
> How do you index your documents? Your last config looks correct. However
> for example if you use data import handler you need to add update.chain
> there too. Same as extraction request hadler if you are using sole-cell.
>
>  class="org.apache.solr.handler.dataimport.DataImportHandler">
> 
>   /home/username/data-config.xml
>   langid
> 
>   
>
> By the way The URL
> http://localhost:8080/solr/update?commit=true&update.chain=langid was
> just an example and meant to feed xml update messages by POST method. Not
> to use in a browser.
>
> Ahmet
>
> On Monday, May 5, 2014 11:04 AM, Victor Pascual <
> vic...@mobilemediacontent.com> wrote:
>
> Thank you very much for you help Ahmet.
>
> However the language detection is still not workin. :(
> My solrconfig.xml didn't contain that lst section inside the update
> requestHandler.
> That's the content I added:
>
>>  class="solr.XmlUpdateRequestHandler">
> >   
> > langid
> >   
> >
> >
>
>
> >class="org.apache.solr.update.processor.LangDetectLanguageIdentifierUpdateProcessorFactory">
> >  
> >text
> >lang
> >  
> >
> >
> >   
> > 
>
> Now, your suggested query
> http://localhost:8080/solr/update?commit=true&update.chain=langid returns
>
> 
> >
> >0
> >14
> >
> >
> And there is still no lang field in my documents.
> Any idea what am I doing wrong?
>
>
>
>
> On Tue, Apr 29, 2014 at 5:33 PM, Ahmet Arslan  wrote:
>
> Hi,
> >
> >solr/update should be used, not /solr/select
> >
> >curl 'http://localhost:8983/solr/update?commit=true&update.chain=langid'
> >
> >By the way don't you have following definition in your solrconfig.xml?
> >
> > 
> >
> >   
> > langid
> >   
> >  
> >
> >
> >
> >
> >On Tuesday, April 29, 2014 4:50 PM, Victor Pascual <
> vic...@mobilemediacontent.com> wrote:
> >Hi Ahmet,
> >
> >thanks for your reply. Adding &update.chain=langid to my query doesn't
> >work: IP:8080/solr/select/?q=*%3A*&update.chain=langid
> >Regarding defining the chain in an UpdateRequestHandler... sorry for the
> >lame question but shall I paste those three lines to solrconfig.xml, or
> >shall I add them somewhere else?
> >
> >There is not UpdateRequestHandler in my solrconfig.
> >
> >Thanks!
> >
> >
> >
> >On Tue, Apr 29, 2014 at 3:13 PM, Ahmet Arslan  wrote:
> >
> >> Hi,
> >>
> >> Did you attach your chain to a UpdateRequestHandler?
> >>
> >> You can do it by adding &update.chain=langid to the URL or defining it
> in
> >> a defaults section as follows
> >>
> >> 
> >>  langid
> >>
> >>
> >>
> >>
> >> On Tuesday, April 29, 2014 3:18 PM, Victor Pascual <
> >> vic...@mobilemediacontent.com> wrote:
> >> Dear all,
> >>
> >> I'm a new user of Solr. I've managed to index a bunch of documents (in
> >> fact, they are tweets) and everything works quite smoothly.
> >>
> >> Nevertheless it looks like Solr doesn't detect the language of my
> documents
> >> nor remove stopwords accordingly so I can extract the most frequent
> terms.
> >>
> >> I've added this piece of XML to my solrconfig.xml as well as the Tika
> lib
> >> jars.
> >>
> >> 
> >> >>
> >>
> class="org.apache.solr.update.processor.LangDetectLanguageIdentifierUpdateProcessorFactory">
> >>   
> >> text
> >> lang
> >>   
> >> 
> >> 
> >>
> >>  
> >>
> >> There is no error in the tomcat log file, so I have no clue of why this
> >> isn't working.
> >> Any hint on how to solve this problem will be much appreciated!
> >>
> >
> >
>

Re: Solr does not recognize language

2014-05-06 Thread Victor Pascual

Thank you very much Ahmet for your help.
It finally worked!

For anyone interested, all your hints where more than useful. I basically
had two problems:
- Didn't have my language detection chain in the update/json requestHandler
- Didn't create the field where the detected language should be stored

Again, thanks for your help!


On Mon, May 5, 2014 at 5:19 PM, Ahmet Arslan  wrote:

> Hi Victor,
>
> I don't know mysolr, I assume you are using /update/json, lets add your
> chain to defaults section.
>
>   
>
> 
>  application/json
>  langid
>
>   
>
>
>
>
> On Monday, May 5, 2014 4:06 PM, Victor Pascual <
> vic...@mobilemediacontent.com> wrote:
> Hi there,
>
> I'm indexing my documents using mysolr. I mainly generate a lost of json
> objects and the run: solr.update(documents_array,'json')
>
>
>
> On Mon, May 5, 2014 at 1:08 PM, Ahmet Arslan  wrote:
>
> > Hi Victor,
> >
> > How do you index your documents? Your last config looks correct. However
> > for example if you use data import handler you need to add update.chain
> > there too. Same as extraction request hadler if you are using sole-cell.
> >
> >  > class="org.apache.solr.handler.dataimport.DataImportHandler">
> > 
> >   /home/username/data-config.xml
> >   langid
> > 
> >   
> >
> > By the way The URL
> > http://localhost:8080/solr/update?commit=true&update.chain=langid was
> > just an example and meant to feed xml update messages by POST method. Not
> > to use in a browser.
> >
> > Ahmet
> >
> > On Monday, May 5, 2014 11:04 AM, Victor Pascual <
> > vic...@mobilemediacontent.com> wrote:
> >
> > Thank you very much for you help Ahmet.
> >
> > However the language detection is still not workin. :(
> > My solrconfig.xml didn't contain that lst section inside the update
> > requestHandler.
> > That's the content I added:
> >
> >> >  class="solr.XmlUpdateRequestHandler">
> > >   
> > > langid
> > >   
> > >
> > >
> >
> >
> > >>
> class="org.apache.solr.update.processor.LangDetectLanguageIdentifierUpdateProcessorFactory">
> > >  
> > >text
> > >lang
> > >  
> > >
> > >
> > >   
> > > 
> >
> > Now, your suggested query
> > http://localhost:8080/solr/update?commit=true&update.chain=langidreturns
> >
> > 
> > >
> > >0
> > >14
> > >
> > >
> > And there is still no lang field in my documents.
> > Any idea what am I doing wrong?
> >
> >
> >
> >
> > On Tue, Apr 29, 2014 at 5:33 PM, Ahmet Arslan  wrote:
> >
> > Hi,
> > >
> > >solr/update should be used, not /solr/select
> > >
> > >curl 'http://localhost:8983/solr/update?commit=true&update.chain=langid
> '
> > >
> > >By the way don't you have following definition in your solrconfig.xml?
> > >
> > > 
> > >
> > >   
> > > langid
> > >   
> > >  
> > >
> > >
> > >
> > >
> > >On Tuesday, April 29, 2014 4:50 PM, Victor Pascual <
> > vic...@mobilemediacontent.com> wrote:
> > >Hi Ahmet,
> > >
> > >thanks for your reply. Adding &update.chain=langid to my query doesn't
> > >work: IP:8080/solr/select/?q=*%3A*&update.chain=langid
> > >Regarding defining the chain in an UpdateRequestHandler... sorry for the
> > >lame question but shall I paste those three lines to solrconfig.xml, or
> > >shall I add them somewhere else?
> > >
> > >There is not UpdateRequestHandler in my solrconfig.
> > >
> > >Thanks!
> > >
> > >
> > >
> > >On Tue, Apr 29, 2014 at 3:13 PM, Ahmet Arslan 
> wrote:
> > >
> > >> Hi,
> > >>
> > >> Did you attach your chain to a UpdateRequestHandler?
> > >>
> > >> You can do it by adding &update.chain=langid to the URL or defining it
> > in
> > >> a defaults section as follows
> > >>
> > >> 
> > >>  langid
> > >>
> > >>
> > >>
> > >>
> > >> On Tuesday, April 29, 2014 3:18 PM, Victor Pascual <
> > >> vic...@mobilemediacontent.com> wrote:
> > >> Dear all,
> > >>
> > >> I'm a new user of Solr. I've managed to index a bunch of documents (in
> > >> fact, they are tweets) and everything works quite smoothly.
> > >>
> > >> Nevertheless it looks like Solr doesn't detect the language of my
> > documents
> > >> nor remove stopwords accordingly so I can extract the most frequent
> > terms.
> > >>
> > >> I've added this piece of XML to my solrconfig.xml as well as the Tika
> > lib
> > >> jars.
> > >>
> > >> 
> > >> > >>
> > >>
> >
> class="org.apache.solr.update.processor.LangDetectLanguageIdentifierUpdateProcessorFactory">
> > >>   
> > >> text
> > >> lang
> > >>   
> > >> 
> > >> 
> > >>
> > >>  
> > >>
> > >> There is no error in the tomcat log file, so I have no clue of why
> this
> > >> isn't working.
> > >> Any hint on how to solve this problem will be much appreciated!
> > >>
> > >
> > >
> >
>
>

Use cases - Production examples: datas, queries, cluster hardware and conf, and statistics

2015-02-12 Thread Victor Rondel

Hi everyone,

I am considering moving one or several Solr clusters to production.
Although Solr's documentation and community is *great*, I am strongly
startled not to find any *complete use-case story* stretching from
application(s) needs and data considerations to hardware ones.
Indeed, I understand why "what/how much hardware / configuration /
sharding" questions are systematically replied with both "it depends"
followed by "test".
But then, what about a few complete descriptions, out of so many
elasticsearch users, from data use case to cluster's internals, along with
a few performance and nodes stats?

So here are questions, before moving to production :

Are there any *complete* use cases around? Could you share some? By
complete I mean including *at least some* of the following :

   1. *Application needs and scope*
   2. *Indexing Data indications* : data volume, documents mapping,
   documents / indexes volume
   3. *Searching Data indications* : different applications, queries, use
   of facets - filters - pivot facets, concurrent indexing
   4. *Cluster Hardware* : machines' hardware (RAM, Disks/SSD -
   DAS-JBOD/SAN/NAS), JVM heap / OS Cache, nb of machines, back office network
   5. *Cluster Configuration* : one or several indexes, sharding,
   replication, master nodes, data nodes, use of over-sharding at start-up,
   use of re-indexing
   6. *Benchmaks *: queries response times, QPS, with or without concurrent
   indexing, memory heap sweet spot, nodes stats

For those interested, here is the (not *complete*) best-among-very-few
exemples I've stumbled upon so far :

   - Perfs with hardware and query description :

   
http://fr.slideshare.net/charliejuggler/lucene-solrlondonug-meetup28nov2014-solr-es-performance

Multy-tenancy and quarantee of service per application (tenant)

2015-02-12 Thread Victor Rondel

Hi everyone,

I am wondering about multy-tenancy and garantee of service in SolrCloud :

*Multy-tenant cluster* : Is there a way to *guarantee a level of service* /
capacity planning for *each tenant* using the cluster (its *own collections*)
?


Thanks,

Streaming search results

2013-04-03 Thread Victor Miroshnikov

Is it possible to stream search results from Solr? Seems that this feature is 
missing.

I see two options to solve this: 

1. Using search results pagination feature
The idea is to implement a smart proxy that will stream chunks from search 
results using pagination.

2. Implement Solr plugin with search streaming feature (is that possible at 
all?)

First option is easy to implement and reliable, though I dont know what are the 
drawbacks.

Regards,
Viktor

Re: Search data who does not have "x" field

2013-04-09 Thread Victor Ruiz

Sorry, I didnt explain my self good, I mean , you have to create an
additional field 'hasCategory' in your schema, and then, before indexing,
set the field 'hasCategory' in the indexed document as true, if your
document has categories, or set it to false, if it has any. With this you
will save computation time, since the query for a boolean field is much
easier for Solr than checking for an empty string field. 

The query should be => q=*:*&fq=hasCategory:true 


anurag.jain wrote
> "another solution would be to add a boolean field, hasCategory, and use it
> for filtering 
> q=
> 
> &fq=hasCategory:true "
> 
> 
> I am not getting result.
> 
> 
> i am trying
> 
> localhost:8983/search?q=*:*&fq=category:true
> 
> it is giving zero result.
> 
> by the way first technique is working fine.





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Search-data-who-does-not-have-x-field-tp4046959p4054763.html
Sent from the Solr - User mailing list archive at Nabble.com.

corrupted index in slave?

2013-04-09 Thread Victor Ruiz

xSearcher.doc(SolrIndexSearcher.java:568)\n\tat
org.apache.solr.handler.component.RealTimeGetComponent.process(RealTimeGetComponent.java:176)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:1817)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)\n\tat
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)\n\tat
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)\n\tat
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)\n\tat
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)\n\tat
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)\n\tat
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)\n\tat
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)\n\tat
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)\n\tat
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)\n\tat
org.mortbay.jetty.Server.handle(Server.java:326)\n\tat
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)\n\tat
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:926)\n\tat
org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)\n\tat
org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)\n\tat
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)\n\tat
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)\n\tat
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)\n","code":500}}


Thanks in advance, regards
Victor




--
View this message in context: 
http://lucene.472066.n3.nabble.com/corrupted-index-in-slave-tp4054769.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: corrupted index in slave?

2013-04-09 Thread Victor Ruiz

sorry I forgot to say, the exceptions are not for every document, but only
for a few...

regards,
Victor

Victor Ruiz wrote
> Hi guys,
> 
> I'm getting exceptions in a Solr slave, when accessing TermVector
> component and RealTimeGetHandler. The weird thing is, that in the master
> and in one of the 2 slaves, the documents are ok, and the same query
> doesnt return any exception. For now, the only way I have to solve the
> problem is deleting these documents and indexing them again.
> 
> I upgraded Solr from 4.0 directly to 4.2, then to 4.2.1 last week These
> exceptions seems to appear since the upgrade to 4.2.
> I didn't run the script for migrating the index files (as I did in the
> migration from 3.6 to 4.0), should I? Has the format of the index changed?
> If not, is that a known bug? If it's, sorry I couldn't find it in JIRA.
> 
> These are the exceptions I get:
> 
> {"responseHeader":{"status":500,"QTime":1},"response":{"numFound":1,"start":0,"docs":[{"itemid":"105266867","text":"exklusiver
> kann man kaum würzen  safran ist das teuerste gewürz der welt handverlesen
> und in mühevoller kleinstarbeit hergestellt ist safran sehr selten und
> wird in winzigen mengen gehandelt und
> verwendet","title":"safran","domainid":4287,"date_i":"2012-11-21T17:01:23Z","date":"2012-11-21T17:01:09Z","category":["kultur","literatur","gesellschaft","umwelt","trinken","essen"]}]},"termVectors":["uniqueKeyFieldName","itemid","105266867",["uniqueKey","105266867"]],"error":{"trace":"java.lang.ArrayIndexOutOfBoundsException\n\tat
> org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)\n\tat
> org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)\n\tat
> org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.get(CompressingTermVectorsReader.java:493)\n\tat
> org.apache.lucene.index.SegmentReader.getTermVectors(SegmentReader.java:175)\n\tat
> org.apache.lucene.index.BaseCompositeReader.getTermVectors(BaseCompositeReader.java:97)\n\tat
> org.apache.lucene.index.IndexReader.getTermVector(IndexReader.java:385)\n\tat
> org.apache.solr.handler.component.TermVectorComponent.process(TermVectorComponent.java:313)\n\tat
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)\n\tat
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\tat
> org.apache.solr.core.SolrCore.execute(SolrCore.java:1817)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)\n\tat
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)\n\tat
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)\n\tat
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)\n\tat
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)\n\tat
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)\n\tat
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)\n\tat
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)\n\tat
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)\n\tat
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)\n\tat
> org.mortbay.jetty.Server.handle(Server.java:326)\n\tat
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)\n\tat
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:926)\n\tat
> org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)\n\tat
> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)\n\tat
> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)\n\tat
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)\n\tat
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)\n","code":500}}
> 
> 
> {"error":{"trace":"java.lang.ArrayIndexOutOfBoundsException\n\tat
> org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)\n\tat
> org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)\n\tat
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.vi

SolrCloud vs Solr master-slave replication

2013-04-12 Thread Victor Ruiz

Hi,

I've just posted this week an issue today with our Solr index:
http://lucene.472066.n3.nabble.com/corrupted-index-in-slave-td4054769.html,

Today, that error started to happen constantly for almost every request, and
I created a JIRA issue becaue I thought it was a bug
https://issues.apache.org/jira/browse/SOLR-4707

As you can read, at the end it was due to a fail in the Solr master-slave
replication, and now I don't know if we should think about migrating to
SolrCloud, since Solr master-slave replications seems not to fit to our
requirements:

* index size:  ~20 million documents, ~9GB
* ~1200 updates/min
* ~1 queries/min (distributed over 2 slaves)  MoreLikeThis, RealTimeGet,
TermVectorComponent, SearchHandler

I would thank you if anyone could help me to answer these questions:

* Would it be advisable to migrate to SolrCloud? Would it have impact on the
replication performance? 
* In that case, what would have better performance? to maintain a copy of
the index in every server, or to use shard servers?
* How many shards and replicas would you advice for ensuring high
availability? 

Kind Regards,

Victor



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-vs-Solr-master-slave-replication-tp4055541.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud vs Solr master-slave replication

2013-04-15 Thread Victor Ruiz

Hi Shawn,

thank you for your reply. 

I'll check if network card drivers are ok. About the RAM, the JVM max heap
size is currently 6GB, but it never reaches the maximum, tipically the used
RAM is not more than 5GB. should I assign more RAM? I've read that excess of
RAM assigned could have also a bad effect on the performance. Apart of the
RAM used by JVM, the server has more than 10GB of unused RAM, which should
be enough to cache the index.

About SolrCloud, I know it doesn't use master-slave replication, but
incremental updates, item by item. That's why I thought it could work for
us, since our bottleneck appear to be the replication cycles. But another
point is, if the indexing occurs in all servers, 1200 updates/min could also
overload the servers? and therefore have a worst performance than with
master-slave replication?

Regards,
Victor





--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-vs-Solr-master-slave-replication-tp4055541p4055995.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud vs Solr master-slave replication

2013-04-18 Thread Victor Ruiz

Thank you again for your answer Shawn. 

Network card seems to work fine, but we've found segmentation faults, so now
our hosting provider is going to run a full hw check. Hopefully they'll
replace the server and problem wil be solved

Regards,
Victor





--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-vs-Solr-master-slave-replication-tp4055541p4056925.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud vs Solr master-slave replication

2013-04-18 Thread Victor Ruiz

Also, I forgot to say... the same error started to happen again.. the index
is again corrupted :(



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-vs-Solr-master-slave-replication-tp4055541p4056926.html
Sent from the Solr - User mailing list archive at Nabble.com.

MoreLikeThis boosted by Date

2013-03-04 Thread Victor Ruiz

Hi,

In my Solr config I have a request handler that boosts newer items, using
date field:

   


true
10
itemid,score
{!boost b=$bf v=$qq}
recip(ms(NOW,date),0.6,1,1) 



And I use also the MoreLikeThis handler
   

true
false
50
itemid,score
title,text
1
1
3
15
true
text title^5



My question is: would it be possible to add a date boost to the items
returned by MoreLikeThis? Or to chain the requests in some way, that is, to
call my dateBoost handler with the itemids returned by MoreLikeThis. 
I can get the result I want by sending a second query when I get the results
of MoreLikeThis:

http://localhost:8983/solr/mlt?q=itemid:item0 => item1 item2 item3 item4

http://localhost:8983/solr/select?qq=itemid:(item1 item2 item3
item4)&qt=dateBoost

I've been working with Solr for almost 2 years, and I've not found yet the
way to do it, if it exists... but I still wonder if there's any way to get
the same output saving the 2nd query. 

Thanks in advance,
Victor




--
View this message in context: 
http://lucene.472066.n3.nabble.com/MoreLikeThis-boosted-by-Date-tp4044513.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr replication takes long time

2013-03-11 Thread Victor Ruiz

Hi guys,

I have a problem with Solr replication. I have 2 solr servers (Solr 4.0.0) 1
master and 1 slave (8 processors,16GB RAM ,Ubuntu 11, ext3, each). In
every server, there are 2 independent instances of solr running (I tried
also multicore config, but having independent instances has for me better
performance), every instance having a differente collection. So, we have 2
masters in server 1, and 2 slaves in server 2.

Index size is currently (for the biggest collection) around 17 million
documents, with a total size near 12 GB. The files transferred every
replication cycle are typically not more than 100, with a total size not
bigger than 50MB. The other collection is not that big, just around 1
million docs and not bigger than 2 GB and not a high update ratio. The big
collection has a load around 200 queries per second (MoreLikeThis,
RealTimeGetHandler , TermVectorComponent mainly), and for the small one it
is below 50 queries per second

Replication has been working for long time with any problem, but in the last
weeks the replication cycles started to take long and long time for the big
collection, even more than 2 minutes, some times even more. During that
time, slaves are so overloaded, that many queries are timing out, despite
the timeout in my clients is 30 seconds.

The servers are in same LAN, gigabit ethernet, so the broadband should not
be the bottleneck.

Since the index is receiving frequents updates and deletes (update handler
receives more than 200 request per second for the big collection, but not
more than 5 per second for the small one), I tried to use the
maxCommitsToKeep attribute, to ensure that no file was deleted during
replication, but it has no effect.

My solrconfig.xml in the big collection is like that:

LUCENE_40

10
1

6HOUR

2000
3

500

${solr.data.dir:}

2048

true

*:*
date:[NOW/DAY-7DAY TO
NOW/DAY+1DAY]
1000

true

${enable.master:false}
commit
startup
startup
schema.xml,solrconfig.xml,stopwords_de.txt,stopwords_en.txt,mapping-FoldToASCII.txt,mapping-FoldToASCII_de.txt

${enable.slave:false}
http://${MASTER_HOST}:${MASTER_PORT}/solr/${MASTER_CORE}
05:00
${MASTER_HTTP_USER}
${MASTER_HTTP_PWD}

*:*

Poll interval is now set to 5 min, I tried to reduce it to 2 min and to
increase it up to 10, with no effect, the replication is always taking so
long., even with a poll time of 2 minutes, when there are only a few megas
to replicate.

Any idea suggestion about what could be the problem?

Thanks in advance,
Victor

--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-replication-takes-long-time-tp4046388.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr replication takes long time

2013-03-11 Thread Victor Ruiz

no, Solr 4.0.0, I wanted to update to Solr 4.1 but I read that there was an
issue with the replication, so I decided not to try it for now


Mark Miller-3 wrote
> Are you using Solr 4.1?
> 
> - Mark
> 
> On Mar 11, 2013, at 1:53 PM, Victor Ruiz <

> bik1979@

> > wrote:
> 
>> Hi guys,
>> 
>> I have a problem with Solr replication. I have 2 solr servers (Solr
>> 4.0.0) 1
>> master and 1 slave (8 processors,16GB RAM ,Ubuntu 11,  ext3,  each). In
>> every server, there are 2 independent instances of solr running (I tried
>> also multicore config, but having independent instances has for me better
>> performance), every instance having a differente collection. So, we have
>> 2
>> masters in server 1, and 2 slaves in server 2.
>> 
>> Index size is currently (for the biggest collection) around 17 million
>> documents, with a total size near 12 GB. The files transferred every
>> replication cycle are typically not more than 100, with a total size not
>> bigger than 50MB. The other collection is not that big, just around 1
>> million docs and not bigger than 2 GB and not a high update ratio. The
>> big
>> collection has a load around 200 queries per second (MoreLikeThis,
>> RealTimeGetHandler , TermVectorComponent mainly), and for the small one
>> it
>> is below 50 queries per second
>> 
>> Replication has been working for long time with any problem, but in the
>> last
>> weeks the replication cycles started to take long and long time for the
>> big
>> collection, even more than 2 minutes, some times even more. During that
>> time, slaves are so overloaded, that many queries are timing out, despite
>> the timeout in my clients is 30 seconds. 
>> 
>> The servers are in same LAN, gigabit ethernet, so the broadband should
>> not
>> be the bottleneck.
>> 
>> Since the index is receiving frequents updates and deletes (update
>> handler
>> receives more than 200 request per second for the big collection, but not
>> more than 5 per second for the small one), I tried to use the
>> maxCommitsToKeep attribute, to ensure that no file was deleted during
>> replication, but it has no effect. 
>> 
>> My solrconfig.xml in the big collection is like that:
>> 
>> 
>> 
>> 
> 
>> 
>>  
> 
> LUCENE_40
> 
>> 
>>  
> >
> 
> class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>> 
>> 
>>  
> 
>>  
> 
> 3
> 
>> 
>>  
> 
>>  
>>  
> 
> 10
> 
>>  
> 
> 1
> 
>>  
>>  
> 
> 6HOUR
> 
>> 
>>  
> 
>> 
>>  
> 
>> 
>>  
> 
>> 
>>  
> 
>> 
>>  
> 
>>  
> 
> 2000
> 
>>  
> 
> 3
> 
>>  
> 
>> 
>>  
> 
>>  
> 
> 500
> 
>>  
> 
>> 
>>  
> 
>>  
> 
> ${solr.data.dir:}
> 
>>  
> 
>> 
>>  
> 
>> 
>>  
> 
>>  
> 
> 2048
> 
>> 
>>  
> >
>   class="solr.FastLRUCache"
>>  size="2048"
>>  initialSize="1024"
>>  autowarmCount="1024"/>
>> 
>>  
> >
>   class="solr.LRUCache"
>>  size="2048"
>>  initialSize="1024"
>>  autowarmCount="1024"/>
>> 
>>  
>>  
> >
>   class="solr.LRUCache"
>>  size="2048"
>>  initialSize="1024"
>>  autowarmCount="1024"/>
>> 
>>  
> 
> true
> 
>> 
>>  
> 
> 50
> 
>> 
>>  
> 
> 50
> 
>> 
>>  
> 
>>  
> 
>>  
> 
>>

Re: Solr replication takes long time

2013-03-11 Thread Victor Ruiz

Thanks for your answer Mark. I think I'll try to update to 4.2. I'll keep you
updated.

Anyway, I'd not say that the full index is replicated, I've been monitoring
the replication process in the Solr admin console and there I see that
usually not more than 50-100 files are transferrend, the total size is
rarely greater than 50MB. Is this info trustable?

Victor

Mark Miller-3 wrote
> Okay - yes, 4.0 is a better choice for replication than 4.1.
> 
> It almost sounds like you may be replicating the full index rather than
> just changes or something. 4.0 had a couple issues as well - a couple
> things that were discovered while writing stronger tests for 4.2.
> 
> 4.2 is spreading onto mirrors now.
> 
> - Mark
> 
> On Mar 11, 2013, at 2:00 PM, Victor Ruiz <

> bik1979@

> > wrote:
> 
>> no, Solr 4.0.0, I wanted to update to Solr 4.1 but I read that there was
>> an
>> issue with the replication, so I decided not to try it for now
>> 
>> 
>> Mark Miller-3 wrote
>>> Are you using Solr 4.1?
>>> 
>>> - Mark
>>> 
>>> On Mar 11, 2013, at 1:53 PM, Victor Ruiz <
>> 
>>> bik1979@
>> 
>>> > wrote:
>>> 
>>>> Hi guys,
>>>> 
>>>> I have a problem with Solr replication. I have 2 solr servers (Solr
>>>> 4.0.0) 1
>>>> master and 1 slave (8 processors,16GB RAM ,Ubuntu 11,  ext3,  each). In
>>>> every server, there are 2 independent instances of solr running (I
>>>> tried
>>>> also multicore config, but having independent instances has for me
>>>> better
>>>> performance), every instance having a differente collection. So, we
>>>> have
>>>> 2
>>>> masters in server 1, and 2 slaves in server 2.
>>>> 
>>>> Index size is currently (for the biggest collection) around 17 million
>>>> documents, with a total size near 12 GB. The files transferred every
>>>> replication cycle are typically not more than 100, with a total size
>>>> not
>>>> bigger than 50MB. The other collection is not that big, just around 1
>>>> million docs and not bigger than 2 GB and not a high update ratio. The
>>>> big
>>>> collection has a load around 200 queries per second (MoreLikeThis,
>>>> RealTimeGetHandler , TermVectorComponent mainly), and for the small one
>>>> it
>>>> is below 50 queries per second
>>>> 
>>>> Replication has been working for long time with any problem, but in the
>>>> last
>>>> weeks the replication cycles started to take long and long time for the
>>>> big
>>>> collection, even more than 2 minutes, some times even more. During that
>>>> time, slaves are so overloaded, that many queries are timing out,
>>>> despite
>>>> the timeout in my clients is 30 seconds. 
>>>> 
>>>> The servers are in same LAN, gigabit ethernet, so the broadband should
>>>> not
>>>> be the bottleneck.
>>>> 
>>>> Since the index is receiving frequents updates and deletes (update
>>>> handler
>>>> receives more than 200 request per second for the big collection, but
>>>> not
>>>> more than 5 per second for the small one), I tried to use the
>>>> maxCommitsToKeep attribute, to ensure that no file was deleted during
>>>> replication, but it has no effect. 
>>>> 
>>>> My solrconfig.xml in the big collection is like that:
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
> 
>>>> 
>>>>
>>> 
> 
>>> LUCENE_40
>>> 
> 
>>>> 
>>>>
>>> 
> >
>>> 
>>>  
>>> class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>>>> 
>>>> 
>>>>
>>> 
> 
>>>>
>>> 
> 
>>> 3
>>> 
> 
>>>> 
>>>>
>>> 
> 
>>>>
>>>>
>>> 
> 
>>> 10
>>> 
> 
>>>>
>>> 
> 
>>> 1
>>> 
> 
>>>>
>>>>
>>> 
> 
>>> 6HOUR
>>> 
> 
>>>> 
>>>>
>>> 
> 
>>>> 
>>>&

Re: Solr replication takes long time

2013-03-13 Thread Victor Ruiz

After upgrading to 4.2, the problem is not yet solved, in this image you can
see, how slow is the transfer speed. At least, after the update the master
is not blocked during replication
 

Any idea?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-replication-takes-long-time-tp4046388p4046951.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Search data who does not have "x" field

2013-03-13 Thread Victor Ruiz

add this to your query, or filter query: 

q=&fq=-category:[* TO *]

another solution would be to add a boolean field, hasCategory, and use it
for filtering
q=&fq=hasCategory:true


Victor


anurag.jain wrote
> Hi all,
> 
> I am facing a problem. 
> 
> Problem is:
> 
> I have updated 250 data to solr. 
> 
> 
> and some of data have "category" field and some of don't have.
> 
> for example.
> 
> 
> {
> "id":"321",
> "name":"anurag",
> "category":"x"
> },
> {
> "id":"3",
> "name":"john"
> }
> 
> 
> now i want to search that data who does not have that field. 
> 
> what query should like. 
> 
> 
> please reply 
> 
> It is very urgent - i have to complete this task by today itself 

> 
> thanks in advance.





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Search-data-who-does-not-have-x-field-tp4046959p4046961.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr replication takes long time

2013-03-13 Thread Victor Ruiz

While looking at Solr logs, I found a java.lang.OutOfMemoryError: Java heap
space that was happening 2 times per hour 
So I tried to increase the max memory heap assigned to JVM (-Xmx) and since
then  the servers are not crashing, even though the replication takes still
long time to complete. But for now, the 2 slaves can handle with no problems
all the queries.


Regards,
Victor



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-replication-takes-long-time-tp4046388p4046993.html
Sent from the Solr - User mailing list archive at Nabble.com.

High response times after snapshot install

2008-12-29 Thread Victor Hogemann


Greetings,

Before anything I'd like to thank the Solr team for such great piece  
of software! Thank you.


Now... back to my problem. I'm experiencing slowdowns on a Solr 1.3  
multicore installation after a snapshot install, response times go up  
from 5ms to above 5sec, and don't go down unless we restart the servers.


Our index has more than 20 fields, and the slave servers got some  
50hits/sec. We're running it on Weblogic, with 6 Weblogic Nodes/Solr  
instances per machine, sharing the same index on the filesystem.


Any insight on what is happening?

Thanks.

Victor Hogemann - Ideais Tecnologia

Tel (RJ) +55 +21 3553-1301 (R217)
Tel (SP) +55 +11 4063-0443 (R217)
Fax +55 +21 3553-1302

Rua da Assembléia, nº 98, 9º andar
Centro | Rio de Janeiro | Brasil
CEP 20011-000

Re: High response times after snapshot install

2008-12-30 Thread Victor Hogemann


Hello Otis,

Thank you for the reply.

But, if the high response time is caused by IO and CPU usage... it  
shouldn't come back to regular response time after a while? Also,  
we're not seeing that much CPU usage.


What if some requests got stuck, and the old searchers are not being  
destroyed? Would it account for the high response times?


PS.: Sorry my poor english, I'm not a native speaker :-)

Victor Hogemann - Ideais Tecnologia

Tel (RJ) +55 +21 3553-1301 (R217)
Tel (SP) +55 +11 4063-0443 (R217)
Fax +55 +21 3553-1302

Rua da Assembléia, nº 98, 9º andar
Centro | Rio de Janeiro | Brasil
CEP 20011-000

On Dec 29, 2008, at 4:26 PM, Otis Gospodnetic wrote:


Hello Victor,

The high response times could be caused by long and intense warming  
of the new searcher(s), which could be causing high IO (it sounds  
like all instances share the same index on the same file system) and  
high CPU usage (lots of cache items being copied from the old  
searcher).



Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message ----

From: Victor Hogemann 
To: Solr-User 
Sent: Monday, December 29, 2008 7:53:44 AM
Subject: High response times after snapshot install

Greetings,

Before anything I'd like to thank the Solr team for such great  
piece of

software! Thank you.

Now... back to my problem. I'm experiencing slowdowns on a Solr 1.3  
multicore
installation after a snapshot install, response times go up from  
5ms to above

5sec, and don't go down unless we restart the servers.

Our index has more than 20 fields, and the slave servers got some  
50hits/sec.
We're running it on Weblogic, with 6 Weblogic Nodes/Solr instances  
per machine,

sharing the same index on the filesystem.

Any insight on what is happening?

Thanks.

Victor Hogemann - Ideais Tecnologia

Tel (RJ) +55 +21 3553-1301 (R217)
Tel (SP) +55 +11 4063-0443 (R217)
Fax +55 +21 3553-1302

Rua da Assembléia, nº 98, 9º andar
Centro | Rio de Janeiro | Brasil
CEP 20011-000

Using Solr's zkcli.sh

2020-09-01 Thread Victor Kretzer

Thank you in advance. This is my first time using a mailing list like this so 
hopefully I am doing so correctly.

I am attempting to setup SolrCloud (Solr 6.6.6) and an external zookeeper 
ensemble on Azure. I have three dedicated to the zookeeper ensemble and two for 
solr all running Ubuntu 18.04 LTS. I've been relying on the following documents:


  *   Taking Solr to 
Production<https://lucene.apache.org/solr/guide/6_6/taking-solr-to-production.html#taking-solr-to-production>
  *   Enbabling 
SSL<https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html#enabling-ssl>

I was able to complete the stand-alone portion of Enabling SSL on each of the 
solr machines and have successfully navigated to the Admin page using 
https://private.address/solr.


I am now trying to complete the section, SSL with SolrCloud, but I cannot get 
past the Configure Zookeeper section. Whenever I try to run
server/scripts/cloud-scripts/zkcli.sh it says:
it says
-bash: server/scripts/cloud-scripts/zkcli.sh: Permission denied

I've tried using sudo server/...  but then it says:
sudo: server/scripts/cloud-scripts/zkcli.sh: command not found

What am I doing wrong? Any help getting this set up would be greatly 
appreciated.

Thanks,

Victor

RE: Using Solr's zkcli.sh

2020-09-02 Thread Victor Kretzer

Vincent --

Your suggestion worked perfectly. After using chmod I'm now able to use the 
zkcli script. Thank you so much for the quick save.

Victor



Victor Kretzer
Sitecore Developer
Application Services
GDC IT Solutions
Office: 717-262-2080 ext. 151

www.gdcitsolutions.com

-Original Message-
From: Vincent Brehin  
Sent: Wednesday, September 2, 2020 6:10 AM
To: solr-user@lucene.apache.org
Subject: Re: Using Solr's zkcli.sh

Hi Victor,
For me also it's a first post to the list. Even if I am quite old to solr, I am 
a recent subscriber.
I guess you used install_solr_service.sh script for installing (either directly 
or through ansible role, or another wrapper).
IIRC this script removes exec permission for some other commands, including 
zkcli.
So you should first launch "sudo chmod a+x 
server/scripts/cloud-scripts/zkcli.sh" , then you should be able to use the 
command.
Let us know !
Vincent


Le mar. 1 sept. 2020 à 23:35, Victor Kretzer  a écrit :

> Thank you in advance. This is my first time using a mailing list like 
> this so hopefully I am doing so correctly.
>
> I am attempting to setup SolrCloud (Solr 6.6.6) and an external 
> zookeeper ensemble on Azure. I have three dedicated to the zookeeper 
> ensemble and two for solr all running Ubuntu 18.04 LTS. I've been 
> relying on the following
> documents:
>
>
>   *   Taking Solr to Production<
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fluce
> ne.apache.org%2Fsolr%2Fguide%2F6_6%2Ftaking-solr-to-production.html%23
> taking-solr-to-production&data=02%7C01%7CVictorKretzer%40gdcit.com
> %7Cfb9f75bdcd7b485a8ace08d84f285bac%7C87b66f08478c40adbb095e93796da295
> %7C1%7C0%7C637346382497269184&sdata=A1jZsEDZ%2FtEGKyvjixLyjYI7F%2F
> 0AQaZ38zM4hZbwuqw%3D&reserved=0
> >
>   *   Enbabling SSL<
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fluce
> ne.apache.org%2Fsolr%2Fguide%2F6_6%2Fenabling-ssl.html%23enabling-ssl&
> amp;data=02%7C01%7CVictorKretzer%40gdcit.com%7Cfb9f75bdcd7b485a8ace08d
> 84f285bac%7C87b66f08478c40adbb095e93796da295%7C1%7C0%7C637346382497269
> 184&sdata=kDnYjqRCKqZ%2BjZpW3dEFdxi6lL1Glp%2BVPXozsJpAn9Y%3D&r
> eserved=0>
>
> I was able to complete the stand-alone portion of Enabling SSL on each 
> of the solr machines and have successfully navigated to the Admin page 
> using 
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fprivate.address%2Fsolr&data=02%7C01%7CVictorKretzer%40gdcit.com%7Cfb9f75bdcd7b485a8ace08d84f285bac%7C87b66f08478c40adbb095e93796da295%7C1%7C0%7C637346382497269184&sdata=aIPyGtMTFuZ3p7lmOi%2B11CRFZ8f3BQPm4NDU42zrhDI%3D&reserved=0.
>
>
> I am now trying to complete the section, SSL with SolrCloud, but I 
> cannot get past the Configure Zookeeper section. Whenever I try to run 
> server/scripts/cloud-scripts/zkcli.sh it says:
> it says
> -bash: server/scripts/cloud-scripts/zkcli.sh: Permission denied
>
> I've tried using sudo server/...  but then it says:
> sudo: server/scripts/cloud-scripts/zkcli.sh: command not found
>
> What am I doing wrong? Any help getting this set up would be greatly 
> appreciated.
>
> Thanks,
>
> Victor
>

SolrCloud (6.6.6) SSL Setup - Unable to create collection

2020-09-03 Thread Victor Kretzer

BACKGROUND: I'm attempting to setup SolrCloud (Solr 6.6.6) with an external 
zookeeper ensemble on Azure. I have three dedicated vms for the zookeeper 
ensemble and two for solr all running Ubuntu 18.04 LTS. I'm new to Solr (and 
Linux) and have been heavily relying on the Solr Ref Guide 6.6, most recently 
the following section on enabling ssl:



https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html



So far I have:

Installed and setup zookeeper

Installed Solr (using install_solr_service.sh script) on both vms.

Followed the steps under Basic SSL Setup, generating certificates on each of 
the nodes.

Set the cluster-wide property to https per the Configure Zookeeper section of 
SolrCloud in the document

Started both nodes and have been able to navigate to them in my browser with 
https



If I do bin/solr status I get:



Solr process 13106 running on port 8983

{

  "solr_home":"/opt/solr-6.6.6/cloud/test2",

  "version":"6.6.6 68fa249034ba8b273955f20097700dc2fbb7a800 - ishan - 
2019-03-29 09:13:13",

  "startTime":"2020-09-03T18:15:34.092Z",

  "uptime":"0 days, 0 hours, 43 minutes, 29 seconds",

  "memory":"52.7 MB (%10.7) of 490.7 MB",

  "cloud":{

"ZooKeeper":"zk1:2181,zk2:2181,zk3:2181/solr",

"liveNodes":"2",

"collections":"0"}}







THE ISSUE

When I try to create a collection using the steps outlined in the above 
document, I get the following error:



azureuser@solr-node-01-test:/opt/solr$ sudo bin/solr create -c mycollection 
-shards 2 -force



Connecting to ZooKeeper at zk1:2181,zk2:2181,zk3:2181/solr ...

INFO  - 2020-09-03 18:21:26.784; 
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider; Cluster at 
zk1:2181,zk2:2181,zk3:2181/solr ready

Re-using existing configuration directory mycollection



Creating new collection 'mycollection' using command:

https://Solr1:8983/solr/admin/collections?action=CREATE&name=mycollection&numShards=2&replicationFactor=1&maxShardsPerNode=1&collection.configName=mycollection



ERROR: Failed to create collection 'mycollection' due to: 
{Solr2:8983_solr=org.apache.solr.client.solrj.SolrServerException:IOException 
occured when talking to server at: https://Solr2:8983/solr}

*I've attached logs at the bottom of this email.



QUESTIONS:

What am I doing wrong and how can I fix it?

Was I right to create separate certificates on each of the nodes (one cert on 
vm1, another cert on vm 2)?

Do I need to copy the certs for each node into the other (if so how)?



CONCLUSION

Thank you so much in advance and if there's any other information you need 
please let me know.

Victor

2020-09-03 18:15:35.240 INFO  
(zkCallback-5-thread-1-processing-n:Solr1:8983_solr) [   ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
2020-09-03 18:15:40.124 INFO  (qtp401424608-45) [   ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
2020-09-03 18:15:40.124 INFO  (qtp401424608-45) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/cores 
params={indexInfo=false&wt=json&_=1599156956818} status=0 QTime=23
2020-09-03 18:15:40.134 INFO  (qtp401424608-20) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/info/system params={wt=json&_=1599156956818} 
status=0 QTime=29
2020-09-03 18:15:40.171 INFO  (qtp401424608-13) [   ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params 
action=LIST&wt=json&_=1599156956818 and sendToOCPQueue=true
2020-09-03 18:15:40.172 INFO  (qtp401424608-13) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/collections 
params={action=LIST&wt=json&_=1599156956818} status=0 QTime=1
2020-09-03 18:15:40.174 INFO  (qtp401424608-16) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/info/system params={wt=json&_=1599156956818} 
status=0 QTime=8
2020-09-03 18:15:58.225 INFO  (qtp401424608-14) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/cores 
params={indexInfo=false&wt=json&_=1599156974989} status=0 QTime=0
2020-09-03 18:15:58.231 INFO  (qtp401424608-13) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/info/system params={wt=json&_=1599156974989} 
status=0 QTime=7
2020-09-03 18:15:58.258 INFO  (qtp401424608-20) [   ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params 
action=LIST&wt=json&_=1599156974989 and sendToOCPQueue=true
2020-09-03 18:15:58.258 INFO  (qtp401424608-20) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/collections 
params={action=LIST&wt=json&_=1599156974989} status=0 QTime=0
2020-09-03 18:15:58.263 INFO  (qtp401424608-21) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/info/system params={wt=json&_=1599156974989} 
sta

RE: SolrCloud (6.6.6) SSL Setup - Unable to create collection

2020-09-04 Thread Victor Kretzer

I solved my problem by using just the certificate from my first node and 
copying that to the second node. I'm not sure whether all three are necessary, 
but I copied: 
*   solr-ssl.keystore.jks
*   solr-ssl-keystore.p12
*   solr-ssl.pem.
If you originally made separate certificates for each node, make sure that on 
the additional nodes you remove those cert files before adding the files from 
the first node. I moved mine to a backup folder I created because I wasn't sure 
what I was trying would work but I think that was unnecessary.

Victor 

-Original Message-----
From: Victor Kretzer  
Sent: Thursday, September 3, 2020 3:03 PM
To: solr-user@lucene.apache.org
Subject: SolrCloud (6.6.6) SSL Setup - Unable to create collection

BACKGROUND: I'm attempting to setup SolrCloud (Solr 6.6.6) with an external 
zookeeper ensemble on Azure. I have three dedicated vms for the zookeeper 
ensemble and two for solr all running Ubuntu 18.04 LTS. I'm new to Solr (and 
Linux) and have been heavily relying on the Solr Ref Guide 6.6, most recently 
the following section on enabling ssl:



https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flucene.apache.org%2Fsolr%2Fguide%2F6_6%2Fenabling-ssl.html&data=02%7C01%7CVictorKretzer%40gdcit.com%7Ca124b9385d4a4eab744408d8503c0457%7C87b66f08478c40adbb095e93796da295%7C1%7C0%7C637347565991726245&sdata=BE%2BPvrXsVzAR67Aoe3D%2FxMruuRlY2Img4aBHeuKpJY8%3D&reserved=0



So far I have:

Installed and setup zookeeper

Installed Solr (using install_solr_service.sh script) on both vms.

Followed the steps under Basic SSL Setup, generating certificates on each of 
the nodes.

Set the cluster-wide property to https per the Configure Zookeeper section of 
SolrCloud in the document

Started both nodes and have been able to navigate to them in my browser with 
https



If I do bin/solr status I get:



Solr process 13106 running on port 8983

{

  "solr_home":"/opt/solr-6.6.6/cloud/test2",

  "version":"6.6.6 68fa249034ba8b273955f20097700dc2fbb7a800 - ishan - 
2019-03-29 09:13:13",

  "startTime":"2020-09-03T18:15:34.092Z",

  "uptime":"0 days, 0 hours, 43 minutes, 29 seconds",

  "memory":"52.7 MB (%10.7) of 490.7 MB",

  "cloud":{

"ZooKeeper":"zk1:2181,zk2:2181,zk3:2181/solr",

"liveNodes":"2",

"collections":"0"}}







THE ISSUE

When I try to create a collection using the steps outlined in the above 
document, I get the following error:



azureuser@solr-node-01-test:/opt/solr$ sudo bin/solr create -c mycollection 
-shards 2 -force



Connecting to ZooKeeper at zk1:2181,zk2:2181,zk3:2181/solr ...

INFO  - 2020-09-03 18:21:26.784; 
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider; Cluster at 
zk1:2181,zk2:2181,zk3:2181/solr ready

Re-using existing configuration directory mycollection



Creating new collection 'mycollection' using command:

https://Solr1:8983/solr/admin/collections?action=CREATE&name=mycollection&numShards=2&replicationFactor=1&maxShardsPerNode=1&collection.configName=mycollection



ERROR: Failed to create collection 'mycollection' due to: 
{Solr2:8983_solr=org.apache.solr.client.solrj.SolrServerException:IOException 
occured when talking to server at: https://Solr2:8983/solr}

*I've attached logs at the bottom of this email.



QUESTIONS:

What am I doing wrong and how can I fix it?

Was I right to create separate certificates on each of the nodes (one cert on 
vm1, another cert on vm 2)?

Do I need to copy the certs for each node into the other (if so how)?



CONCLUSION

Thank you so much in advance and if there's any other information you need 
please let me know.

Victor

2020-09-03 18:15:35.240 INFO  
(zkCallback-5-thread-1-processing-n:Solr1:8983_solr) [   ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
2020-09-03 18:15:40.124 INFO  (qtp401424608-45) [   ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
2020-09-03 18:15:40.124 INFO  (qtp401424608-45) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/cores 
params={indexInfo=false&wt=json&_=1599156956818} status=0 QTime=23
2020-09-03 18:15:40.134 INFO  (qtp401424608-20) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/info/system params={wt=json&_=1599156956818} 
status=0 QTime=29
2020-09-03 18:15:40.171 INFO  (qtp401424608-13) [   ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params 
action=LIST&wt=json&_=1599156956818 and sendToOCPQueue=true
2020-09-03 18:15:40.172 INFO  (qtp401424608-13) [   ] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/collections 
params={action=LIST&wt=json&_=1599156956818} status=0 QTime=1
2020-09-03 18:15:40.174 INFO  (qtp

Need to update SOLR_HOME in the solr service script and getting errors

2020-09-16 Thread Victor Kretzer

My setup is two solr nodes running on separate Azure Ubuntu 18.04 LTS vms using 
an external zookeeper assembly.
I installed Solr 6.6.6 using the install file and then followed the steps for 
enabling ssl. I am able to start solr, add collections and the like using 
bin/solr script.

Example:
/opt/solr$ sudo bin/solr start -cloud -s cloud/test2 -force

However, if I restart the machine or attempt to start solr using the installed 
service, it naturally goes back to the default SOLR_HOME in the 
/etc/default/solr.in.sh script: "/var/solr/data"

I've tried updating SOLR_HOME to "/opt/solr/cloud/test2" but then when I start 
the service I see the following error on the Admin Dashboard:
SolrCore Initialization Failures
mycollection_shard1_replica1: 
org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: 
/opt/solr-6.6.6/cloud/test2/mycollection_shard1_replica1/data/index/write.lock
Please check your logs for more information

[cid:image001.png@01D68C18.8565BFB0]

I'm including what I believe to be the pertinent information from the logs 
below:
I suspect this is a permission issue because the solr user created by the 
install script isn't allowed access to  /opt/solr but I'm new to Linux and 
haven't completely wrapped my head around the way permissions work with it. Am 
I correct in guessing the cause of the error and, if so, how do I correct this 
so that the service can be used to run my instances?

java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: 
Unable to create core [mycollection_shard1_replica1]
  at java.util.concurrent.FutureTask.report(FutureTask.java:122)
  at java.util.concurrent.FutureTask.get(FutureTask.java:192)
  at 
org.apache.solr.core.CoreContainer.lambda$load$6(CoreContainer.java:594)
  at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
  at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Unable to create core 
[mycollection_shard1_replica1]
  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:966)
  at 
org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565)
  at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  ... 5 more
Caused by: org.apache.solr.common.SolrException: 
/opt/solr-6.6.6/cloud/test2/mycollection_shard1_replica1/data/index/write.lock
  at org.apache.solr.core.SolrCore.(SolrCore.java:977)
  at org.apache.solr.core.SolrCore.(SolrCore.java:830)
  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950)
  ... 7 more
Caused by: java.nio.file.AccessDeniedException: 
/opt/solr-6.6.6/cloud/test2/mycollection_shard1_replica1/data/index/write.lock
  at 
sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
  at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
  at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
  at 
sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
  at java.nio.channels.FileChannel.open(FileChannel.java:287)
  at java.nio.channels.FileChannel.open(FileChannel.java:335)
  at 
org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:113)
  at 
org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)
  at 
org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
  at 
org.apache.lucene.store.FilterDirectory.obtainLock(FilterDirectory.java:104)
  at 
org.apache.lucene.index.IndexWriter.isLocked(IndexWriter.java:4776)
  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:709)
  at org.apache.solr.core.SolrCore.(SolrCore.java:923)


Thanks for the help,
Victor

RE: Need to update SOLR_HOME in the solr service script and getting errors

2020-09-17 Thread Victor Kretzer

Hi Mark. 

Thanks for taking the time to explain it so clearly. It makes perfect sense to 
me now and using chown solved the problem. Thanks again and have a great day.

Victor

-Original Message-
From: Mark H. Wood  
Sent: Thursday, September 17, 2020 9:59 AM
To: solr-user@lucene.apache.org
Subject: Re: Need to update SOLR_HOME in the solr service script and getting 
errors

On Wed, Sep 16, 2020 at 02:59:32PM +, Victor Kretzer wrote:
> My setup is two solr nodes running on separate Azure Ubuntu 18.04 LTS vms 
> using an external zookeeper assembly.
> I installed Solr 6.6.6 using the install file and then followed the steps for 
> enabling ssl. I am able to start solr, add collections and the like using 
> bin/solr script.
> 
> Example:
> /opt/solr$ sudo bin/solr start -cloud -s cloud/test2 -force
> 
> However, if I restart the machine or attempt to start solr using the 
> installed service, it naturally goes back to the default SOLR_HOME in the 
> /etc/default/solr.in.sh script: "/var/solr/data"
> 
> I've tried updating SOLR_HOME to "/opt/solr/cloud/test2"

That is what I would do.

> but then when I start the service I see the following error on the Admin 
> Dashboard:
> SolrCore Initialization Failures
> mycollection_shard1_replica1: 
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrExcept
> ion: 
> /opt/solr-6.6.6/cloud/test2/mycollection_shard1_replica1/data/index/wr
> ite.lock Please check your logs for more information
> 
> I'm including what I believe to be the pertinent information from the logs 
> below:

You did well.

> I suspect this is a permission issue because the solr user created by the 
> install script isn't allowed access to  /opt/solr but I'm new to Linux and 
> haven't completely wrapped my head around the way permissions work with it. 
> Am I correct in guessing the cause of the error and, if so, how do I correct 
> this so that the service can be used to run my instances?

Yes, the stack trace actually tells you explicitly that the problem is 
permissions on that file.  Follow the chain of "Caused by:" and you'll see:

  Caused by: java.nio.file.AccessDeniedException: 
/opt/solr-6.6.6/cloud/test2/mycollection_shard1_replica1/data/index/write.lock

Since, in the past, you have started Solr using 'sudo', this probably means 
that write.lock is owned by 'root'.  Solr creates this file with permissions 
that allow only the owner to write it.  If the service script runs Solr as any 
other user (and it should!) then Solr won't be able to open this file for 
writing, and because of this it won't complete the loading of that core.

You should find out what user account is used by the service script, and 
'chown' Solr's entire working directories tree to be owned by that user.  Then, 
refrain from ever running Solr as 'root' or the problem may recur.  Use the 
normal service start/stop mechanism for controlling your Solr instances.

--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu

Setting up SolrCloud Behind Azure Application Gateway

2020-11-12 Thread Victor Kretzer

I'm attempting to set up SolrCloud for use with Sitecore 9.0.2. I want to set 
up my Azure Application Gateway with a TSL cert. I want a private IP for 
Sitecore and a public IP for accessing the Solr Admin Dashboard. My goal is to 
use Application Gateway for the TSL and then route to the backend using http 
protocol.

I currently have the following configuration:
* 2 SolrCloud 6.6.6 nodes on 2 Azure Ubuntu 18.04 
LTS VMs
* 3 Zookeeper nodes on 3 Azure Ubuntu VMs
* A VPN with the IPs of all the above
* An application Gateway with:
o public listener on port 443
o public listener on port 80 (to 
eliminate the cert as a cause of my issues)
o backend pool for the two 
sorlCloud VMs
o an HTTP setting for Backend port 
8983

I can access the dashboard for the nodes using:
* 
http://:8983/solr/#/<http://%3cnode-pub-ip%3e:8983/solr/#/>

But not when using either of the following:
* 
https:///solr/#<https://%3capp-gtwy-ip%3e/solr/#> with a public 
listener on port 443
* 
http:///solr/#<http://%3capp-gtwy-ip%3e/solr/#> with a public 
listener on port 80

The private IPs of both SolrCloud VMs are reporting healthy on port 8983 with a 
302-status code according to the default Backend Health monitor on Application 
Gateway.

I greatly appreciate any help provided.

Thanks,

Victor

SOLR running on Azure web app/services

2019-07-09 Thread Victor Casas

Hello, is Solr supported on Azure web app or Azure Service? If so, is there any 
white paper on how to get this done or recommendations/suggestions??


Victor Casas
Web Admin

SolrCloud keeps crashing

2020-12-21 Thread Victor Kretzer

My setup:
3 SolrCloud 6.6.6 nodes and 3 zookeeper 3.4.14 nodes running on 3 Azure Ubuntu 
18.04 LTS VMs (1 solr/1 zk per machine).

My issue:
Every few days (1-3 days usually) I come on to find 2 of me 3 nodes down. I'm 
looking at the logs and not seeing an out of memory error. I do see in the 
solr_gc.logs that the gc is running more and more frequently. I also see some 
illegal type errors in the solr logs. But I'm not sure what the actual cause of 
the crash is and my understanding of garbage collection is rudimentary, at best.
Is there an obvious cause in the logs that I'm not understanding or do I need 
to turn to some other resource to trouble these issues?

Below is part of my logs. I can include more if helpful but they are very long. 
(>10k lines for solr.log, >53k lines for the gc.log). Please let me know if 
there is any additional information I can provide and thank you in advance for 
your help.


***
solr.log
***
2020-12-20 08:49:02.802 ERROR (qtp401424608-8936) 
[c:sitecore_submittals_index_sec s:shard1 r:core_node1 
x:sitecore_submittals_index_sec_shard1_replica2] o.a.s.s.HttpSolrCall 
null:org.apache.velocity.exception.MethodInvocationException: Invocation of 
method 'toChars' in  class java.lang.Class threw exception 
java.lang.IllegalArgumentException at custom.vm[line 1, column 376]
at 
org.apache.velocity.runtime.parser.node.ASTMethod.handleInvocationException(ASTMethod.java:243)
at 
org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:187)
at 
org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:280)
at 
org.apache.velocity.runtime.parser.node.ASTReference.value(ASTReference.java:567)
at 
org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:151)
at 
org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:280)
at 
org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:369)
at 
org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at 
org.apache.velocity.runtime.directive.Foreach.render(Foreach.java:420)
at 
org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207)
at 
org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
at org.apache.velocity.Template.merge(Template.java:356)
at org.apache.velocity.Template.merge(Template.java:260)
at 
org.apache.solr.response.VelocityResponseWriter.write(VelocityResponseWriter.java:169)
at 
org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
at 
org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:810)
at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:539)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at 
org.eclipse.jetty.serv

Indexing the directory

2009-03-16 Thread Huang, Zijian(Victor)


> Hi, all:
> I am new to SOLR, can anyone please tell me what do I do to index
> a some text files in a local directory?
> 
> Thanks
> 
> Victor
> 
>

Question about incremental index update

2009-03-18 Thread Huang, Zijian(Victor)

Hi:
   Is it easy to do daily incremental index update in Solr assuming the
index is around 1G? In terms of giving a document an ID to facilitate
index update, is it using the URL a good way to do so? 

Thanks


Victor

RE: Question about incremental index update

2009-03-18 Thread Huang, Zijian(Victor)

Hi, Otis:
   so does Solr already has some kind of libraries build-in, which it
can automatically detect the different within two set of crawled
documents and update the index to the newer one? 
I mean the document ID in Slor xml doc format. Inside the Solr wiki,
it tells me that I can update a particular doc by its ID if I assigned
one previously. I am thinking if using the url as the doc ID will be a
good thing to do.

Thanks

Vic

-Original Message-
From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] 
Sent: Wednesday, March 18, 2009 12:43 PM
To: solr-user@lucene.apache.org
Subject: Re: Question about incremental index update


Victor,

Daily updates (or hourly or more frequent) are not going to be a
problem.  I don't follow your question about document ID and using URL.


Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message 
> From: "Huang, Zijian(Victor)" 
> To: solr-user@lucene.apache.org
> Sent: Wednesday, March 18, 2009 2:51:59 PM
> Subject: Question about incremental index update
> 
> Hi:
>Is it easy to do daily incremental index update in Solr assuming 
> the index is around 1G? In terms of giving a document an ID to 
> facilitate index update, is it using the URL a good way to do so?
> 
> Thanks
> 
> 
> Victor

Page-Rank algorithm

2009-03-19 Thread Huang, Zijian(Victor)

Hi, 
   Do you guys know if there is some versions of the page-rank algorithm
already implemented in Solr(Lucene)? If not, how hard is it to
implement. I am trying to improve the ranking relevance for Solr.

Thanks


Vic

I need to index files larger than 300 Mb, helpme please

2017-03-13 Thread Victor Hugo Olvera Morales

How can I index files with more than 300 MB in weight in solr-6.2.1

Re: Multiple search analyzers on the same field type possible?

2011-10-16 Thread Victor van der Wolf

I don't think this will be a problem. I'll contact you tomorrow directly by
email for some details.


--
View this message in context: 
http://lucene.472066.n3.nabble.com/Multiple-search-analyzers-on-the-same-field-type-possible-tp3417898p3426678.html
Sent from the Solr - User mailing list archive at Nabble.com.

68 matches

Mail list logo