AW: CoreContainer#createAndLoad, existing cores not loaded
https://issues.apache.org/jira/browse/SOLR-6718
looks like I am not alone with my "weird" questions/ideas ;)
And I should really switch over to 5 ;)
-Ursprüngliche Nachricht-
Von: Clemens Wyss DEV [mailto:clemens...@mysign.ch]
Gesendet: Donnerstag, 29. Januar 2015 08:08
An: solr-user@lucene.apache.org
Betreff: AW: CoreContainer#createAndLoad, existing cores not loaded
Thx Shawn. I am running latest-greatest Solr (4.10.3) Solr home is e.g.
/opt/webs//WebContent/WEB-INF/solr
the core(s) reside in
/opt/webs//WebContent/WEB-INF/solr/cores
Should these be found by core discovery?
If not, how can I configure coreRootDirectory in sorl.xml to be "cores folder
below "
${coreRootDirectory:/cores}
Note:
the solr.xml is to be used for any of our 150sites we host. Therefore like it
to be "generic" -> /cores
-Ursprüngliche Nachricht-
Von: Shawn Heisey [mailto:apa...@elyograg.org]
Gesendet: Mittwoch, 28. Januar 2015 17:08
An: solr-user@lucene.apache.org
Betreff: Re: CoreContainer#createAndLoad, existing cores not loaded
On 1/28/2015 8:52 AM, Clemens Wyss DEV wrote:
> My problem:
> I create cores dynamically using container#create( CoreDescriptor ) and then
> add documents to the very core(s). So far so good.
> When I restart my app I do
> container = CoreContainer#createAndLoad(...) but when I then call
> container.getAllCoreNames() an empty list is returned.
>
> What cores should be loaded by the container if I call
> CoreContainer#createAndLoad(...)
> ? Where does the container lookup the "existing cores"?
If the solr.xml is the old format, then cores are defined in solr.xml, in the
section of that config file.
There is a new format for solr.xml that is supported in version 4.4 and later
and will be mandatory in 5.0. If that format is present, then Solr will use
core discovery -- starting from either the solr home or a defined
coreRootDirectory, solr will search for core.properties files and treat each
directory where one is found as a core's instanceDir.
http://wiki.apache.org/solr/Solr.xml%204.4%20and%20beyond
Thanks,
Shawn
Solr source problem
Hi, I want to add solr source to eclipse by link instruction: http://wise.99acres.com/tutorials/tutorial-solr-code-run-from-eclipse-wtp-tomcat/, but cannot success. (fail at step 7) This is error message C:\Users\anhletung\Downloads\solr-4.10.0-src\solr-4.10.0\build.xml:111: The foll owing error occurred while executing this line: C:\Users\anhletung\Downloads\solr-4.10.0-src\solr-4.10.0\lucene\build.xml:194: T he following error occurred while executing this line: C:\Users\anhletung\Downloads\solr-4.10.0-src\solr-4.10.0\lucene\common-build.xml :434: impossible to resolve dependencies: resolve failed - see output for details Can you help me?
Building Solr 5 from svn sources
Looks like trunk is Solr 6? Should I build Solr 5 from http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x
Re: Solr source problem
Hi, The error message "impossible to resolve dependencies" suggests dependency problems. seems to be problems at "ant ivy-bootstrap" (step 6) Try step 6 again and check logs... Regards, Tomoko 2015-01-29 18:14 GMT+09:00 L� T�ng Anh : > Hi, > > I want to add solr source to eclipse by link instruction: > > http://wise.99acres.com/tutorials/tutorial-solr-code-run-from-eclipse-wtp-tomcat/ > , > but cannot success. (fail at step 7) > > This is error message > C:\Users\anhletung\Downloads\solr-4.10.0-src\solr-4.10.0\build.xml:111: The > foll > owing error occurred while executing this line: > > C:\Users\anhletung\Downloads\solr-4.10.0-src\solr-4.10.0\lucene\build.xml:194: > T > he following error occurred while executing this line: > > C:\Users\anhletung\Downloads\solr-4.10.0-src\solr-4.10.0\lucene\common-build.xml > :434: impossible to resolve dependencies: > resolve failed - see output for details > > > Can you help me? >
Re: Building Solr 5 from svn sources
Yes, trunk is now for 6.0. branch_5x is for 5.1. 5.0 release branch is lucene_solr_5_0 http://mail-archives.apache.org/mod_mbox/lucene-dev/201501.mbox/%3CCAKiERN4-qbj7BF%3DJgui4xUFKujwuP%2BodkZPesVT51xnXG1om_w%40mail.gmail.com%3E 2015-01-29 19:29 GMT+09:00 Clemens Wyss DEV : > Looks like trunk is Solr 6? > Should I build Solr 5 from > http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x >
AW: Building Solr 5 from svn sources
Thx! -Ursprüngliche Nachricht- Von: Tomoko Uchida [mailto:tomoko.uchida.1...@gmail.com] Gesendet: Donnerstag, 29. Januar 2015 13:39 An: solr-user@lucene.apache.org Betreff: Re: Building Solr 5 from svn sources Yes, trunk is now for 6.0. branch_5x is for 5.1. 5.0 release branch is lucene_solr_5_0 http://mail-archives.apache.org/mod_mbox/lucene-dev/201501.mbox/%3CCAKiERN4-qbj7BF%3DJgui4xUFKujwuP%2BodkZPesVT51xnXG1om_w%40mail.gmail.com%3E 2015-01-29 19:29 GMT+09:00 Clemens Wyss DEV : > Looks like trunk is Solr 6? > Should I build Solr 5 from > http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x >
Re: SpellingQueryConverter and query parsing
Thank you, James, I'll do that. ResponseBuilder carries around with it the QParser, Query, and query string, so getting suggestions from parsed query terms shouldn't be a big deal. What looks to be hard is rewriting the original query with the suggestions. That's probably why the regex is used instead of the parser. -Scott On Tue, Jan 27, 2015 at 1:37 PM, Dyer, James wrote: > Having worked with the spellchecking code for the last few years, I've > often wondered the same thing, but I never looked seriously into it. I'm > sure there's probably some serious hurdles, hence the Query Converter. The > easy thing to do here is to use "spellcheck.q", and then pass in > space-delimited keywords. This bypasses the query converter entirely for > custom situations like yours. > > But please, if you find a way to plug the actual query parser into > spellcheck, consider opening a jira & contributing the code, even if what > you end up with isn't in a final polished state for general use. > > James Dyer > Ingram Content Group > > > -Original Message- > From: Scott Stults [mailto:sstu...@opensourceconnections.com] > Sent: Tuesday, January 27, 2015 11:26 AM > To: solr-user@lucene.apache.org > Subject: SpellingQueryConverter and query parsing > > Hello! > > SpellingQueryConverter "parses" the incoming query in sort of a quick and > dirty way with a regular expression. Is there a reason the query string > isn't parsed with the _actual_ parser, if one was configured for that type > of request? Even better, could the parsed query object be added to the > response in some way so that the query wouldn't need to be parsed twice? > The individual terms could then be visited and substituted in-place without > needing to worry about preserving the meaning of operators in the query. > > The motive in my question is, I may need to implement a QueryConverter > because I'm using a custom parser, and using that parser in the > QueryConverter itself seems like the right thing to do. That wasn't done > though in SpellingQueryConverter, so I wan't to find out why before I go > blundering into a known minefield. > > > Thanks! > -Scott > -- Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC | 434.409.2780 http://www.opensourceconnections.com
Search most common phrases by part of phrase
I have not trivial task and want to understand if Solr is right instrument to solve it . Simplified example: i have search field on my site for searching by product description. and next descriptions in mysql db: Id Desc 1 this is my test document number one. also checking search within phrases. 2 this is my test document number two 3 this is another group 4 this is first group 5 this is first test 6 this is your test When user type some text in search field ex. "is" he should get next results(top three phrases which contains "is"): "this is", "is my", "is first". Is someone has any ideas how to do that ? -- View this message in context: http://lucene.472066.n3.nabble.com/Search-most-common-phrases-by-part-of-phrase-tp4182890.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Search most common phrases by part of phrase
Hi Sylkaalex, Sounds like auto-complete stuff. http://blog.trifork.com/2012/02/15/different-ways-to-make-auto-suggestions-with-solr/ Ahmet On Thursday, January 29, 2015 4:37 PM, sylkaalex wrote: I have not trivial task and want to understand if Solr is right instrument to solve it . Simplified example: i have search field on my site for searching by product description. and next descriptions in mysql db: Id Desc 1 this is my test document number one. also checking search within phrases. 2 this is my test document number two 3 this is another group 4 this is first group 5 this is first test 6 this is your test When user type some text in search field ex. "is" he should get next results(top three phrases which contains "is"): "this is", "is my", "is first". Is someone has any ideas how to do that ? -- View this message in context: http://lucene.472066.n3.nabble.com/Search-most-common-phrases-by-part-of-phrase-tp4182890.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Define Id when using db dih
Hi, You can create one sing uuid : https://wiki.apache.org/solr/UniqueKey Alternatively, you can combine multiple fields from your table to create a key. TemplateTransformer would be handy. It could be done in select statement too. Ahmet On Thursday, January 29, 2015 9:27 AM, SolrUser1543 wrote: Hi, I am using data import handler and import data from oracle db. I have a problem that the table I am importing from has no one column which is defined as a key. How should I define the key in the data config file ? Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Define-Id-when-using-db-dih-tp4182797.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: replicas goes in recovery mode right after update
Hi Erick, @ichattopadhyaya beat me to it already yesterday. So we are good -cheers Vijay On Wed, Jan 28, 2015 at 1:30 PM, Erick Erickson wrote: > Vijay: > > Thanks for reporting this back! Could I ask you to post a new patch with > your correction? Please use the same patch name > (SOLR-5850.patch), and include a note about what you found (I've already > added a comment). > > Thanks! > Erick > > On Wed, Jan 28, 2015 at 9:18 AM, Vijay Sekhri > wrote: > > > Hi Shawn, > > Thank you so much for the assistance. Building is not a problem . Back in > > the days I have worked with linking, compiling and building C , C++ > > software . Java is a piece of cake. > > We have built the new war from the source version 4.10.3 and our > > preliminary tests have shown that our issue (replicas in recovery on high > > load)* is resolved *. We will continue to do more testing and confirm . > > Please note that the *patch is BUGGY*. > > > > It removed the break statement within while loop because of which, > whenever > > we send a list of docs it would hang (API CloudSolrServer.add) , but it > > would work if send one doc at a time. > > > > It took a while to figure out why that is happening. Once we put the > break > > statement back it worked like a charm. > > Furthermore the patch has > > > > > solr/solrj/src/java/org/apache/solr/client/solrj/impl/ConcurrentUpdateSolrClient.java > > which should be > > > > > solr/solrj/src/java/org/apache/solr/client/solrj/impl/ConcurrentUpdateSolrServer.java > > > > Finally checking if(!offer) is sufficient than using if(offer == false) > > Last but not the least having a configurable queue size and timeouts > > (managed via solrconfig) would be quite helpful > > Thank you once again for your help. > > > > Vijay > > > > On Tue, Jan 27, 2015 at 6:20 PM, Shawn Heisey > wrote: > > > > > On 1/27/2015 2:52 PM, Vijay Sekhri wrote: > > > > Hi Shawn, > > > > Here is some update. We found the main issue > > > > We have configured our cluster to run under jetty and when we tried > > full > > > > indexing, we did not see the original Invalid Chunk error. However > the > > > > replicas still went into recovery > > > > All this time we been trying to look into replicas logs to diagnose > the > > > > issue. The problem seem to be at the leader side. When we looked into > > > > leader logs, we found the following on all the leaders > > > > > > > > 3439873 [qtp1314570047-92] WARN > > > > org.apache.solr.update.processor.DistributedUpdateProcessor – Error > > > > sending update > > > > *java.lang.IllegalStateException: Queue full* > > > > > > > > > > > > > There is a similar bug reported around this > > > > https://issues.apache.org/jira/browse/SOLR-5850 > > > > > > > > and it seem to be in OPEN status. Is there a way we can configure the > > > queue > > > > size and increase it ? or is there a version of solr that has this > > issue > > > > resolved already? > > > > Can you suggest where we go from here to resolve this ? We can > repatch > > > the > > > > war file if that is what you would recommend . > > > > In the end our initial speculation about solr unable to handle so > many > > > > update is correct. We do not see this issue when the update load is > > less. > > > > > > Are you in a position where you can try the patch attached to > > > SOLR-5850? You would need to get the source code for the version > you're > > > on (or perhaps a newer 4.x version), patch it, and build Solr yourself. > > > If you have no experience building java packages from source, this > might > > > prove to be difficult. > > > > > > Thanks, > > > Shawn > > > > > > > > > > > > -- > > * > > Vijay Sekhri > > * > > > -- * Vijay Sekhri *
Re: AW: CoreContainer#createAndLoad, existing cores not loaded
On 1/29/2015 12:08 AM, Clemens Wyss DEV wrote:
> Thx Shawn. I am running latest-greatest Solr (4.10.3)
> Solr home is e.g.
> /opt/webs//WebContent/WEB-INF/solr
> the core(s) reside in
> /opt/webs//WebContent/WEB-INF/solr/cores
> Should these be found by core discovery?
> If not, how can I configure coreRootDirectory in sorl.xml to be "cores folder
> below "
>
> ${coreRootDirectory:/cores}
>
> Note:
> the solr.xml is to be used for any of our 150sites we host. Therefore like it
> to be "generic" -> /cores
The first thing to say is that it is a bad idea to put your solr home
inside the extracted WAR. The "WEB-INF" above is what makes me think
you have done this. Extracted war directories can get wiped out at any
time, and we often recommend wiping them out manually in order to clear
up certain problems.
If there will only be core.properties files in that cores directory and
not any other location under the solr home, just remove
coreRootDirectory from the config entirely. It will default to the solr
home and everything in "cores" will be found with no problem. That
would only be a problem if you have core.properties files in some other
directory that is also underneath the solr home. I think that's a bad
idea ... if you want a staging area where you create new cores manually,
keep it outside the solr home, to avoid confusing either yourself or Solr.
Relative paths should be possible in coreRootDirectory, but they are
currently resolved from the current working directory instead of the
solr home. There's a bug report for that issue, but it is only fixed in
5.0, which isn't out yet. I placed a note with the info above on the
Jira issue. It's the last comment right now:
https://issues.apache.org/jira/browse/SOLR-6718
Since I converted to core discovery, my solr.xml file is extremely
minimal -- all the defaults work perfectly for me:
Thanks,
Shawn
AW: Building Solr 5 from svn sources
Why are solr*jars not being built? All others (including lucene) are built. -Ursprüngliche Nachricht- Von: Clemens Wyss DEV [mailto:clemens...@mysign.ch] Gesendet: Donnerstag, 29. Januar 2015 13:47 An: solr-user@lucene.apache.org Betreff: AW: Building Solr 5 from svn sources Thx! -Ursprüngliche Nachricht- Von: Tomoko Uchida [mailto:tomoko.uchida.1...@gmail.com] Gesendet: Donnerstag, 29. Januar 2015 13:39 An: solr-user@lucene.apache.org Betreff: Re: Building Solr 5 from svn sources Yes, trunk is now for 6.0. branch_5x is for 5.1. 5.0 release branch is lucene_solr_5_0 http://mail-archives.apache.org/mod_mbox/lucene-dev/201501.mbox/%3CCAKiERN4-qbj7BF%3DJgui4xUFKujwuP%2BodkZPesVT51xnXG1om_w%40mail.gmail.com%3E 2015-01-29 19:29 GMT+09:00 Clemens Wyss DEV : > Looks like trunk is Solr 6? > Should I build Solr 5 from > http://svn.apache.org/repos/asf/lucene/dev/branches/branch_5x >
Re: AW: CoreContainer#createAndLoad, existing cores not loaded
On 1/29/2015 12:13 AM, Clemens Wyss DEV wrote: > BTW: > None of my core folders contains a core.properties file ... ? Could it be due > to the fact that I am (so far) running only EmbeddedSolrServer, hence no real > Solr-Server? I'm not sure how it's working without core.properties files, unless your solr.xml file has a element and the cores are defined there. The old-style solr.xml file works in 4.10.3. The minimum config you need in a core.properties file is nothing at all -- it just needs to exist. The name of the core defaults to the name of the directory where it is found, and the dataDir defaults to "data" relative to the directory where the core.properties was found. I myself only have "name" and "dataDir" defined in my core.properties files, because I need both to be different from the default. [root@bigindy5 cores]# cat s0_0/core.properties #Written by CorePropertiesLocator #Mon Jan 26 23:38:42 MST 2015 name=s0build dataDir=../../data/s0_0 [root@bigindy5 cores]# cat s0_1/core.properties #Written by CorePropertiesLocator #Mon Jan 26 23:38:42 MST 2015 name=s0live dataDir=../../data/s0_1 To answer your other question about whether you should go with 5.0 ... that depends on how close you are to your production deployment and how willing you are to deal with bleeding edge software. The first release of a new major version is always a risky proposition ... I plan to wait for the 5.2 or 5.3 release before rolling it into production, and waiting long enough after the release to make sure that no big problems are reported by users. Minor Lucene/Solr releases happen on a very quick timeline, so I won't have to wait very long. The following info is available on the dev list, so I'm not giving away any secrets when I tell you that the first release candidate for 5.0 has failed to pass the vote. There will be a new one soon that includes some additional bugfixes. If all goes well with that candidate, there should be a release announcement within a week of the new RC being cut. Thanks, Shawn
AW: AW: CoreContainer#createAndLoad, existing cores not loaded
> to put your solr home inside the extracted WAR
We are NOT using war's
>coreRootDirectory
I don't have this property in my sorl.xml
> If there will only be core.properties files in that cores directory
Again, I see no core.properties file. I am creating my cores through
CoreContainer.createCore( CordeDescriptor). The folder(s) are created but no
core.properties file
-Ursprüngliche Nachricht-
Von: Shawn Heisey [mailto:apa...@elyograg.org]
Gesendet: Donnerstag, 29. Januar 2015 17:28
An: solr-user@lucene.apache.org
Betreff: Re: AW: CoreContainer#createAndLoad, existing cores not loaded
On 1/29/2015 12:08 AM, Clemens Wyss DEV wrote:
> Thx Shawn. I am running latest-greatest Solr (4.10.3) Solr home is
> e.g.
> /opt/webs//WebContent/WEB-INF/solr
> the core(s) reside in
> /opt/webs//WebContent/WEB-INF/solr/cores
> Should these be found by core discovery?
> If not, how can I configure coreRootDirectory in sorl.xml to be "cores folder
> below "
>
> name="coreRootDirectory">${coreRootDirectory:/cores}
>
> Note:
> the solr.xml is to be used for any of our 150sites we host. Therefore
> like it to be "generic" -> /cores
The first thing to say is that it is a bad idea to put your solr home inside
the extracted WAR. The "WEB-INF" above is what makes me think you have done
this. Extracted war directories can get wiped out at any time, and we often
recommend wiping them out manually in order to clear up certain problems.
If there will only be core.properties files in that cores directory and not any
other location under the solr home, just remove coreRootDirectory from the
config entirely. It will default to the solr home and everything in "cores"
will be found with no problem. That would only be a problem if you have
core.properties files in some other directory that is also underneath the solr
home. I think that's a bad idea ... if you want a staging area where you
create new cores manually, keep it outside the solr home, to avoid confusing
either yourself or Solr.
Relative paths should be possible in coreRootDirectory, but they are currently
resolved from the current working directory instead of the solr home. There's
a bug report for that issue, but it is only fixed in 5.0, which isn't out yet.
I placed a note with the info above on the Jira issue. It's the last comment
right now:
https://issues.apache.org/jira/browse/SOLR-6718
Since I converted to core discovery, my solr.xml file is extremely minimal --
all the defaults work perfectly for me:
Thanks,
Shawn
Re: AW: Building Solr 5 from svn sources
"ant package" in the solr directory will generate the distribution tars. That's the ultimate build. Regards, Alex. Sign up for my Solr resources newsletter at http://www.solr-start.com/ On 29 January 2015 at 12:12, Shawn Heisey wrote: > On 1/29/2015 10:03 AM, Clemens Wyss DEV wrote: >> Why are solr*jars not being built? All others (including lucene) are built. > > What steps are you taking, and what is not there that you expect to be > there? > > Thanks, > Shawn >
AW: AW: Building Solr 5 from svn sources
In the README it only says "ant compile", so I was expecting to get the solr*jars from this task. ">ant jar" does the trick, what else ;) Sorry for the noise ... -Ursprüngliche Nachricht- Von: Shawn Heisey [mailto:apa...@elyograg.org] Gesendet: Donnerstag, 29. Januar 2015 18:13 An: solr-user@lucene.apache.org Betreff: Re: AW: Building Solr 5 from svn sources On 1/29/2015 10:03 AM, Clemens Wyss DEV wrote: > Why are solr*jars not being built? All others (including lucene) are built. What steps are you taking, and what is not there that you expect to be there? Thanks, Shawn
AW: AW: CoreContainer#createAndLoad, existing cores not loaded
No -element in my solr.xml My folder structure looks alike: [solr] -solr.xml -[configsets] ... -[cores] --[de] ---[data] ... --[fr] ... --[it] ... So you say, if I copy an empty core.proerties file into the [cores] folder it should work? Even with 4.10.3? Thanks also for all your warnings going the "bleeding edge"-way! -Ursprüngliche Nachricht- Von: Shawn Heisey [mailto:apa...@elyograg.org] Gesendet: Donnerstag, 29. Januar 2015 18:10 An: solr-user@lucene.apache.org Betreff: Re: AW: CoreContainer#createAndLoad, existing cores not loaded On 1/29/2015 12:13 AM, Clemens Wyss DEV wrote: > BTW: > None of my core folders contains a core.properties file ... ? Could it be due > to the fact that I am (so far) running only EmbeddedSolrServer, hence no real > Solr-Server? I'm not sure how it's working without core.properties files, unless your solr.xml file has a element and the cores are defined there. The old-style solr.xml file works in 4.10.3. The minimum config you need in a core.properties file is nothing at all -- it just needs to exist. The name of the core defaults to the name of the directory where it is found, and the dataDir defaults to "data" relative to the directory where the core.properties was found. I myself only have "name" and "dataDir" defined in my core.properties files, because I need both to be different from the default. [root@bigindy5 cores]# cat s0_0/core.properties #Written by CorePropertiesLocator #Mon Jan 26 23:38:42 MST 2015 name=s0build dataDir=../../data/s0_0 [root@bigindy5 cores]# cat s0_1/core.properties #Written by CorePropertiesLocator #Mon Jan 26 23:38:42 MST 2015 name=s0live dataDir=../../data/s0_1 To answer your other question about whether you should go with 5.0 ... that depends on how close you are to your production deployment and how willing you are to deal with bleeding edge software. The first release of a new major version is always a risky proposition ... I plan to wait for the 5.2 or 5.3 release before rolling it into production, and waiting long enough after the release to make sure that no big problems are reported by users. Minor Lucene/Solr releases happen on a very quick timeline, so I won't have to wait very long. The following info is available on the dev list, so I'm not giving away any secrets when I tell you that the first release candidate for 5.0 has failed to pass the vote. There will be a new one soon that includes some additional bugfixes. If all goes well with that candidate, there should be a release announcement within a week of the new RC being cut. Thanks, Shawn
Re: AW: Building Solr 5 from svn sources
On 1/29/2015 10:03 AM, Clemens Wyss DEV wrote: > Why are solr*jars not being built? All others (including lucene) are built. What steps are you taking, and what is not there that you expect to be there? Thanks, Shawn
Re: AW: Building Solr 5 from svn sources
For working locally (i.e. not distributing a package) I always use "ant example" (although this target may be changed). "ant dist" gives me jars to link against for SolrJ programs. FWIW, Erick On Thu, Jan 29, 2015 at 9:25 AM, Alexandre Rafalovitch wrote: > "ant package" in the solr directory will generate the distribution > tars. That's the ultimate build. > > Regards, >Alex. > > Sign up for my Solr resources newsletter at http://www.solr-start.com/ > > > On 29 January 2015 at 12:12, Shawn Heisey wrote: > > On 1/29/2015 10:03 AM, Clemens Wyss DEV wrote: > >> Why are solr*jars not being built? All others (including lucene) are > built. > > > > What steps are you taking, and what is not there that you expect to be > > there? > > > > Thanks, > > Shawn > > >
Re: AW: Building Solr 5 from svn sources
I would recommend 'ant server' instead. 'and example' is not recommended anymore as that target is on track for removal. On Thu, Jan 29, 2015 at 10:06 AM, Erick Erickson wrote: > For working locally (i.e. not distributing a package) I always use "ant > example" (although > this target may be changed). > > "ant dist" gives me jars to link against for SolrJ programs. > > FWIW, > Erick > > On Thu, Jan 29, 2015 at 9:25 AM, Alexandre Rafalovitch > > wrote: > > > "ant package" in the solr directory will generate the distribution > > tars. That's the ultimate build. > > > > Regards, > >Alex. > > > > Sign up for my Solr resources newsletter at http://www.solr-start.com/ > > > > > > On 29 January 2015 at 12:12, Shawn Heisey wrote: > > > On 1/29/2015 10:03 AM, Clemens Wyss DEV wrote: > > >> Why are solr*jars not being built? All others (including lucene) are > > built. > > > > > > What steps are you taking, and what is not there that you expect to be > > > there? > > > > > > Thanks, > > > Shawn > > > > > > -- Anshum Gupta http://about.me/anshumgupta
Re: AW: Building Solr 5 from svn sources
I do ant package because I want to have identical file system layout to the final distribution. Is there a target that does that? In my attempts to find it, all other targets had jars, etc all over the place. But that's my use-case, not OP's. So, possibly not relevant. Regards, Alex. Sign up for my Solr resources newsletter at http://www.solr-start.com/ On 29 January 2015 at 13:16, Anshum Gupta wrote: > I would recommend 'ant server' instead. 'and example' is not recommended > anymore as that target is on track for removal. > > On Thu, Jan 29, 2015 at 10:06 AM, Erick Erickson > wrote: > >> For working locally (i.e. not distributing a package) I always use "ant >> example" (although >> this target may be changed). >> >> "ant dist" gives me jars to link against for SolrJ programs. >> >> FWIW, >> Erick >> >> On Thu, Jan 29, 2015 at 9:25 AM, Alexandre Rafalovitch > > >> wrote: >> >> > "ant package" in the solr directory will generate the distribution >> > tars. That's the ultimate build. >> > >> > Regards, >> >Alex. >> > >> > Sign up for my Solr resources newsletter at http://www.solr-start.com/ >> > >> > >> > On 29 January 2015 at 12:12, Shawn Heisey wrote: >> > > On 1/29/2015 10:03 AM, Clemens Wyss DEV wrote: >> > >> Why are solr*jars not being built? All others (including lucene) are >> > built. >> > > >> > > What steps are you taking, and what is not there that you expect to be >> > > there? >> > > >> > > Thanks, >> > > Shawn >> > > >> > >> > > > > -- > Anshum Gupta > http://about.me/anshumgupta
Re: AW: AW: CoreContainer#createAndLoad, existing cores not loaded
On 1/29/2015 10:15 AM, Clemens Wyss DEV wrote: >> to put your solr home inside the extracted WAR > We are NOT using war's > >> coreRootDirectory > I don't have this property in my sorl.xml > >> If there will only be core.properties files in that cores directory > Again, I see no core.properties file. I am creating my cores through > CoreContainer.createCore( CordeDescriptor). The folder(s) are created but no > core.properties file I am pretty clueless when it comes to the embedded server, but if you are creating the cores in the java code every time you create the container, I bet what I'm telling you doesn't apply at all. The solr.xml file may not even be used. The recommendation these days is to NOT use the embedded server. There are too many limitations and it doesn't receive as much user testing as the webapp. Start Solr as a separate process and access it over http. The overhead of http on a LAN is minimal, and over localhost it's almost nothing. To do that, you would just need to change your code to use one of the client objects. That would probably be HttpSolrServer, which is renamed to HttpSolrClient in 5.0. They share the same parent object as EmbeddedSolrServer. Most of the relevant methods used come from the parent class, so you would need very few code changes. Thanks, Shawn
Re: AW: Building Solr 5 from svn sources
On 1/29/2015 12:05 PM, Alexandre Rafalovitch wrote: > I do ant package because I want to have identical file system layout > to the final distribution. Is there a target that does that? In my > attempts to find it, all other targets had jars, etc all over the > place. > > But that's my use-case, not OP's. So, possibly not relevant. The 'dist' target is what I've typically used in the past. On 4.x versions, it creates the dist directory, with the war and all the jars included. With 5.0 this doesn't create a war, because that was specifically excluded, with the idea that users should be using the bin/solr script, not loading the webapp into their own servlet container. If you DO want to create a war, it looks like you can either do 'ant server' or 'ant server-war'. It will be created as the following, relative to root of the checkout: solr/server/webapps/solr.war I personally will be creating and using the war. My setup was originally homegrown from the jetty included in the example with version 1.4.0, and upgraded with each Solr version since then. I will be using this setup until such time as Solr completely embeds the HTTP server into an actual Solr application. Thanks, Shawn
delete from specific shard
We have duplicate records in two shards and want to delete one set of duplicate records from one shard. curl --proxy "" 'http://host.abc.com:8983/solr/collection1/update?shards=localhost:8983/solr/collection1&commit=true' -H "Content-Type: text/xml" --data-binary '.' This delete command do not honor the shards parameter and deletes records across the shards. How do we delete records from specific shard? Thanks.
Key/Value Query
Hi, I need your help on the following query: *Indexed data (field name is 'text'):* ** *I'm also a cat* *I'm pig* *I'm a cat* *I'm a hero* *I'm a superman* *I'm dog* I want to query for *text:(i'm cat) OR text:(i'm dog) OR text:(i'm god)* And I would like to get the result set as follows in a *single query*: *i'm cat => I'm a cat* *i'm dog => I'm dog* *i'm god => * Currently, I'm firing 3 individual queries, each one for "i'm dog", "i'm cat", "i'm god" with rows=1 to get the best result on the top and programatically, building the result set. Question is, how do we fire this in one query (to make it faster, than making multiple calls) and get the result in a similar format? If the question is vague, please let me know and I would be happy to clarify. Thanks Trilok
Re: Key/Value Query
That's a bit hard. I think some of this information is in the debug, if you enable it. But that's expensive. You don't by any chance trying to run incoming documents against a bunch of preset queries, do you? Because if you do, Luwak might be something to check out. https://github.com/flaxsearch/luwak And one day, it might be in Solr Regards, Alex. Sign up for my Solr resources newsletter at http://www.solr-start.com/ On 29 January 2015 at 19:46, Trilok Prithvi wrote: > Hi, > > I need your help on the following query: > > *Indexed data (field name is 'text'):* > ** > *I'm also a cat* > *I'm pig* > *I'm a cat* > *I'm a hero* > *I'm a superman* > *I'm dog* > > I want to query for *text:(i'm cat) OR text:(i'm dog) OR text:(i'm god)* > > And I would like to get the result set as follows in a *single query*: > *i'm cat => I'm a cat* > *i'm dog => I'm dog* > *i'm god => * > > Currently, I'm firing 3 individual queries, each one for "i'm dog", "i'm > cat", "i'm god" with rows=1 to get the best result on the top and > programatically, building the result set. > > Question is, how do we fire this in one query (to make it faster, than > making multiple calls) and get the result in a similar format? If the > question is vague, please let me know and I would be happy to clarify. > > Thanks > Trilok
Re: SolrCloud result correctness compared with single core
Pretty helpful, thanks Erick! 2015-01-24 9:48 GMT+08:00 Erick Erickson : > you might, but probably not enough to notice. At 50G, the tf/idf > stats will _probably_ be close enough you won't be able to tell. > > That said, recently distributed tf/idf has been implemented but > you need to ask for it, see SOLR-1632. This is Solr 5.0 though. > > I've rarely seen it matter except in fairly specialized situations. > Consider a single core. Deleted documents still count towards > some of the tf/idf stats. So your scoring could theoretically > change after, say, an optimize. > > So called "bottom line" is that yes, the scoring may change, but > IMO not any more radically than was possible with single cores, > and I wouldn't worry about unless I had evidence that it was > biting me. > > Best > Erick > > On Fri, Jan 23, 2015 at 2:52 PM, Yandong Yao wrote: > > > Hi Guys, > > > > As the main scoring mechanism is based tf/idf, so will same query running > > against SolrCloud return different result against running it against > single > > core with same data sets as idf will only count df inside one core? > > > > eg: Assume I have 100GB data: > > A) Index those data using single core > > B) Index those data using SolrCloud with two cores (each has 50GB data > > index) > > > > Then If I query those with same query like 'apple', then will I get > > different result for A and B? > > > > > > Regards, > > Yandong > > >
Change date field until minute to zero for group by
I got date fields like this.
{
date:"2013-11-01T16:14:55.372Z",
latency:1
},
{
date:"2013-11-01T16:14:00.372Z",
latency:6
},
{
date:"013-11-01T16:15:17.496Z",
latency:3
}
I want to see the count to group by the date field(until minutes) like
that.
{
date:"2013-11-01T16:14:55.372Z",
latency:6
},
{
date:"013-11-01T16:15:17.496Z",
latency:3
}
So I have to change the date field format to Raw Query Parameters.
How do I do? Help me please. It didn't work.
My Raw Query Parameters :
group=true&group.field=log_ts_tdt&group.facet=true&group.func=rint(div(ms(log_ts_tdt),mul(24,mul(60,mul(60,1000)
--
View this message in context:
http://lucene.472066.n3.nabble.com/Change-date-field-until-minute-to-zero-for-group-by-tp4183042.html
Sent from the Solr - User mailing list archive at Nabble.com.
solrj returning no results but curl can get them
I'm stumped. I've got some solrj 3.6.1 code that works fine against three of
my request handlers but not the fourth. The very odd thing is that I have no
trouble retrieving results with curl against all of the result handlers.
My solrj code sets some parameters:
ModifiableSolrParams params = new ModifiableSolrParams();
params.set("fl","*,score");
params.set("rows","500");
params.set("qt","/"+product);
params.set("hl", "on");
params.set("hl.fl", "title snippet");
params.set("hl.fragsize",50);
params.set("hl.simple.pre","");
params.set("hl.simple.post","");
queryString = "(" + queryString + s[s.length-1] + ")";
I have various request handlers that key off of the product value. I'll call
the one that doesn't work "myproduct".
I send the parameter string to catalina.out for debugging:
System.out.println(params.toString());
I get this:
fl=*%2Cscore&rows=500&qt=%2Fmyproduct&hl=on&hl.fl=title+snippet&hl.fragsize=50
&hl.simple.pre=%3Cspan+class%3D%22hlt%22%3E
&hl.simple.post=%3C%2Fspan%3E&q=title%3A%28brain%29+OR+snippet%3A%28brain%29
I get no results when I let the solrj code do the search although the code
works fine with the other three products.
To convince myself that there is nothing wrong with the data I unencode the
parameter string and run this command:
curl "http://myserver/myapp/myproduct\
fl=*,score&rows=500&qt=/myproduct&hl=on&hl.fl=title+snippet&hl.fragsize=50\
&hl.simple.pre=&hl.simple.post=\
&q=title:brain%20OR%20snippet:brain"
It runs just fine.
How can I debug this?
Thanks very much.
--
View this message in context:
http://lucene.472066.n3.nabble.com/solrj-returning-no-results-but-curl-can-get-them-tp4183053.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Cannot reindex to add a new field
For this I prefer TemplateTransformer to RegexTransformer - its not a regex, just a pattern, and so should be more efficient to use TemplateTransformer. A script will also work, of course. On Tue, Jan 27, 2015 at 5:54 PM, Alexandre Rafalovitch wrote: > On 27 January 2015 at 17:47, Carl Roberts > wrote: > > > commonField="false" regex=":" replaceWith=" "/> > > Yes, that works because the transformer copies it, not the > EntityProcessor. So, no conflict on xpath. > > Regards, >Alex. > > > Sign up for my Solr resources newsletter at http://www.solr-start.com/ >
AW: AW: AW: CoreContainer#createAndLoad, existing cores not loaded
> The recommendation these days is to NOT use the embedded server We would love to, as it is clear that this is not the "Solr-way" to go. The reason for us building upon EmbeddedSolrServer is, we have more than 150sites, each with ist own index (core). If we'd go client server then we could no easily update the solr server(s) without also updating all clients (i.e. the 150 sites) at same time. And having a dedicated Solr server for every client/site is not really an option, is it? Or can for example a 4.10.3 client "talk" to a Solr 5/6 Server? Also when updating the Solr server, doesn't that also require a re-index of all data as the Luncene-storage format might have changed? -Ursprüngliche Nachricht- Von: Shawn Heisey [mailto:apa...@elyograg.org] Gesendet: Donnerstag, 29. Januar 2015 20:30 An: solr-user@lucene.apache.org Betreff: Re: AW: AW: CoreContainer#createAndLoad, existing cores not loaded On 1/29/2015 10:15 AM, Clemens Wyss DEV wrote: >> to put your solr home inside the extracted WAR > We are NOT using war's > >> coreRootDirectory > I don't have this property in my sorl.xml > >> If there will only be core.properties files in that cores directory > Again, I see no core.properties file. I am creating my cores through > CoreContainer.createCore( CordeDescriptor). The folder(s) are created > but no core.properties file I am pretty clueless when it comes to the embedded server, but if you are creating the cores in the java code every time you create the container, I bet what I'm telling you doesn't apply at all. The solr.xml file may not even be used. The recommendation these days is to NOT use the embedded server. There are too many limitations and it doesn't receive as much user testing as the webapp. Start Solr as a separate process and access it over http. The overhead of http on a LAN is minimal, and over localhost it's almost nothing. To do that, you would just need to change your code to use one of the client objects. That would probably be HttpSolrServer, which is renamed to HttpSolrClient in 5.0. They share the same parent object as EmbeddedSolrServer. Most of the relevant methods used come from the parent class, so you would need very few code changes. Thanks, Shawn
