SOLR-1632 will certainly help. But trying to predict whether your core
A or core B will appear first doesn't really seem like a good use of
time. If you actually have a setup like you describe, add &debug=all
to your query on both cores and you'll see all the gory detail of how
the scores are calcu
IMO each mega of memory saved has more impact that 0.001 less in latency … an
OOM is killer, a lag of 2 second … is not catastrophic.
—
/Yago Riveiro
On Tue, Mar 10, 2015 at 4:03 PM, Erick Erickson
wrote:
> Just to pile on:
> I admire your bravery! I'll add to the other comments only by saying
You simply cannot compare scores from two separate queries, comparing
them is meaningless.
This appears to be an XY problem, you're asking _how_ to do something
without telling us _what_ the end goal here is.
>From your description, I really have no idea what you're trying to do.
You might revie
Ahhh, ok. When you reloaded the cores, did you do it core-by-core?
Yes, but maybe we reloaded the wrong core or something like that. We
also noticed that the startTime doesn't update in the admin-ui while
switching between cores (you have to reload the page). We still use
4.8.1, so maybe it i
Would like to do it during querying.
Thanks,
Ashish
On Tue, Mar 10, 2015 at 11:07 PM, Alexandre Rafalovitch
wrote:
> Is that during indexing or during query phase?
>
> Indexing has UpdateRequestProcessors (e.g.
> http://www.solr-start.com/info/update-request-processors/ )
> Query has Components
Actually the reason I did not use the solr script was that I didn't really
get how to make a window service out of it from nssm.exe. I tried doing a
.bat that called solr with start -p 8983 but seems it just loops my command
rather then run it.
Thanks for the help / Karl
On 11 March 2015 at 23:08
Hi,
I understand that in Solr 5.0, they provide a REST API to do real-time
update of the schema using Curl. However, I could not do that for my
eariler version of Solr 4.10.1.
Would like to check, is this function available for the earlier version of
Solr, and is the curl syntax the same as Solr
On 3/10/2015 1:39 PM, Ryan, Michael F. (LNG-DAY) wrote:
> You'll need to wrap the date in quotes, since it contains a colon:
>
> String a = "speechDate:\"1992-07-10T17:33:18Z\"";
You could also escape the colons with a backslash. Here's another way
to do it that doesn't require quotes or manual e
I tried to run Solr over HDFS following
https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS
when I was testing map reduce way of index generation.
However, when I run Solr on MapRFS, solr gave error that could not
recognize maprfs:// scheme in URI.
Have anyone met similar issues
Hi Zheng,
*** I understand that in Solr 5.0, they provide a REST API to do real-time
update of the schema using Curl ** *. Would please help me how to do this?
I need to update both schema.xml and solrconfig.xml in Solr 5.0 in
SolrCloud.
Your help is appreciated..
*Thanks Again..*
On Thu, Mar
Hello,
when I display one of my core's schema, lots of fields appear:
"fields":[{
"name":"_root_",
"type":"string",
"indexed":true,
"stored":false},
{
"name":"_version_",
"type":"long",
"indexed":true,
"stored":true},
On 3/11/2015 8:56 AM, SolrUser1543 wrote:
> Client application which queries solr needs to increase a number of
> simultaneously connections in order to improve performance ( in additional
> to get solr results, it needs to get an internal resources like images. )
> But this increment has improved
Hello,
my solr logs say:
INFO - 2015-03-12 08:49:34.900; org.apache.solr.core.RequestHandlers; created
/analysis/document: solr.DocumentAnalysisRequestHandler
WARN - 2015-03-12 08:49:34.919; org.apache.solr.core.SolrResourceLoader; Solr
loaded a deprecated plugin/analysis class [solr.admin.Ad
Anyone please suggest
With Regards
Aman Tandon
On Sat, Mar 7, 2015 at 9:55 PM, Aman Tandon wrote:
> Hi,
>
> Please suggest me what should be the tests which i should run to check the
> availability, query time, etc in my solr cloud setup.
>
> With Regards
> Aman Tandon
>
Hi,
i want to sort my documents by a given order. The order is defined by a
list of ids.
My current solution is:
list of ids: 15, 5, 1, 10, 3
query: q=*:*&fq=(id:((15) OR (5) OR (1) OR (10) OR
(3)))&sort=query($idqsort) desc,id asc&idqsort=id:((15^5) OR (5^4) OR
(1^3) OR (10^2) OR (3^1))&s
On 3/11/2015 10:53 AM, SolrUser1543 wrote:
does it apply to solr 4.10 ? or only to solr 5 ?
The information I provided is not version-specific. It would apply to
either version you listed and at least some of the previous 4.x versions.
Thanks,
Shawn
Hi,
I've a field which is being used for result grouping. Here's the field
definition.
This started once I did a rolling update from 4.7 to 5.0. I started getting
the error on any group by query --> "SolrDispatchFilter null:java.lang.
IllegalStateException: unexpected docvalues type NONE for
I investigated my tomcat7 configuration.
I have founded that we work in BIO mode.
I consider to switch to NIO mode.
what are recommendation in this case?
--
View this message in context:
http://lucene.472066.n3.nabble.com/increase-connections-on-tomcat-tp4192405p4192602.html
Sent from th
Hi. Erick..
Would please help me distinguish between
Uploading a Configuration Directory and Linking a Collection to a
Configuration Set ?
On Thu, Mar 12, 2015 at 2:01 AM, Nitin Solanki wrote:
> Thanks a lot Erick.. It will be helpful.
>
> On Wed, Mar 11, 2015 at 9:27 PM, Erick Er
Well, the new way will be the only way eventually, so either you learn
the old way then switch or learn it now ;)...
But if you insist you could start with a heap size of 4G like this:
java -Xmx4G -Xms4G -jar start.jar
Best,
Erick
On Wed, Mar 11, 2015 at 1:09 PM, Karl Kildén wrote:
> Thanks!
Hi,
I am indexing documents on Solr 4.10.2. While indexing, I am
getting this error in log -
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
at
Hi,
I am not sure but when i am looking into the server/lib directory then i am
able to see the version 8.1 with all those lib files present in that
folder. So i am guessing its version 8.1.
I confirmed it by downloading the new jetty server which was jetty-9.2 and
i found the same version on jet
You could also check the default memory by starting solr with the -V
parameter for verbose output. It will show your output like this.
If your are startinf solr with script present in bin directory using this
command
*./solr -c -V*
Using Solr root directory: /data/solr/aman/solr_cloud/solr-5.0.0
Just a small correction
> If your are startinf solr with script present in bin directory using this
> command
> *./solr -c -V*
*./solr start -c -V*
With Regards
Aman Tandon
On Thu, Mar 12, 2015 at 4:05 PM, Aman Tandon
wrote:
> You could also check the default memory by starting solr with the
Yes, Solr 5.0 uses Jetty 8.
FYI, the upcoming release 5.1 will move to Jetty 9.
Also, just in case it matters -- as noted in the 5.0 release notes, the use
of Jetty is now an implementation detail and we might move away from it in
the future -- so you shouldn't be depending on Solr using Jetty or
Hi,
I am using Oracle 11g2 and we are having a schema where few tables are having
more than 100 million rows (some of them are Varchar2 100 bytes). And we have
to frequently do the LIKE based search on those tables. Sometimes we need to
join the tables also. Insert / Updates are also happening
Hi,
Securing various Solr APIs has once again surfaced as a discussion in the
developer list. See e.g. SOLR-7236
Would be useful to get some feedback from Solr users about needs "in the field".
Please reply to this email and let us know what security aspect(s) would be
most important for your c
that would explain it!
Luke tool (http://github.com/DmitryKey/luke) is also useful for such cases
or generally, when in need to check the field contents:
On Wed, Mar 11, 2015 at 12:50 PM, wrote:
> Hello,
>
> I found the reason: the query to store ymds in SOLR was invalid ("json"
> and "litera
>> What is the modern equivalent of Luke?
It is same Luke, but polished:
http://github.com/DmitryKey/luke
On Thu, Mar 12, 2015 at 11:03 AM, wrote:
> Hello,
>
> my solr logs say:
>
> INFO - 2015-03-12 08:49:34.900; org.apache.solr.core.RequestHandlers;
> created /analysis/document: solr.Docume
Hi,
Things you have mentioned would be useful for our use-case.
On top we've seen these two requests for securing Solr:
1. Encrypting the index (with a customer private key for instance). There
are certainly other ways to go about this, like using virtual private
clouds, but having the feature i
On 3/11/2015 10:45 PM, Aman Tandon wrote:
>> You may need to manually remove the 127.0.1.1 entries from zookeeper
>> after you fix the IP address problem.
>
>
> How to do that?
The zkcli script included with Solr should have everything you need --
getfile, putfile, and clear ... but that would b
On 3/11/2015 7:38 AM, phi...@free.fr wrote:
> does anyone if it is possible to create a directory resource in the
> solr-jetty configuration files?
>
> In Tomcat 8, you can do the following:
>
>
>
> className="org.apache.catalina.webresources.DirResourc
On 3/12/2015 2:00 AM, Zheng Lin Edwin Yeo wrote:
> I understand that in Solr 5.0, they provide a REST API to do real-time
> update of the schema using Curl. However, I could not do that for my
> eariler version of Solr 4.10.1.
>
> Would like to check, is this function available for the earlier ver
On 3/12/2015 5:03 AM, Pratik Thaker wrote:
> I am using Oracle 11g2 and we are having a schema where few tables are having
> more than 100 million rows (some of them are Varchar2 100 bytes). And we have
> to frequently do the LIKE based search on those tables. Sometimes we need to
> join the tab
Hi Shawn,
here is the Jetty Mailing List's reply concerning my question.
Unfortunately, this solution won't work with SOLR Jetty, because its version is
< 9.
Philippe
--
Just ensure you don't have a /WEB-INF/ directory, and you can u
Not unless you can somehow codify that sort order at index time, but
I'm assuming the sort order changes dynamically.
You can also sort by function, but that's not really useful.
Or, if these are relatively short lists, you can sort at the app layer.
Best,
Erick
On Thu, Mar 12, 2015 at 2:16 AM,
By and large, I really never use linking. But it's about associating a
config set
you've _already_ uploaded with a collection.
So uploading is pushing the configset from your local machine up to Zookeeper,
and linking is using that uploaded, named configuration with an
arbitrary collection.
But u
On 3/12/2015 8:17 AM, phi...@free.fr wrote:
> here is the Jetty Mailing List's reply concerning my question.
>
> Unfortunately, this solution won't work with SOLR Jetty, because its version
> is < 9.
The trunk branch of the Solr source code (version 6.0 development) is
already running Jetty 9.2.9
About <1>. Gotta be careful here about what would be promised. You
really _can't_ encrypt the _indexed_ terms in a meaningful way and
still search. And, as you well know, you can reconstruct documents
from the indexed terms. It's lossy, but still coherent enough to give
security folks fits.
For in
Actually I ran across a neat IntelliJ plugin that you could install
and directly edit ZK files. And I'm pretty sure there are stand-alone
programs that do this, but they are all outside Solr.
I'm not sure what "real time update of the schema" is for, would you
(Zheng) explain further? Collections
On 3/12/2015 9:18 AM, Erick Erickson wrote:
> By and large, I really never use linking. But it's about associating a
> config set
> you've _already_ uploaded with a collection.
>
> So uploading is pushing the configset from your local machine up to Zookeeper,
> and linking is using that uploaded, n
Yes, the admin handlers are deprecated because they are now implicit - no
need to specify them in solrconfig. Yeah, the doc is very unclear on that
point, but in CHANGES.TXT: "*AdminHandlers is deprecated , /admin/* are
implicitly defined, /get ,/replication and handlers are also implicitly
registe
Hi Hoss,
I created a wrapper class, compiled a jar and included an
org.apache.lucene.codecs.Codec file in META-INF/services in the jar file
with an entry for the wrapper class :HTPostingsFormatWrapper. I created a
collection1/lib directory and put the jar there. (see below)
I'm getting the drea
Hi guys,
I saw an issue in Jira (https://issues.apache.org/jira/browse/SOLR-7234)
with status Resolved, but the resolution is not identified in the issue.
I am facing the exact same problem.. and not able to identified the
solution.
In the last comment of the issue, is said that this kind
Well, I think I've narrowed down the issue. The error is happening when I'm
trying to do a rolling update from Solr 4.7 (which is our current version)
to 5.0 . I'm able to re-produce this couple of times. If I do a fresh index
on a 5.0, it works. Not sure if there's any other way to mitigate it.
On 3/11/2015 4:45 PM, shamik wrote:
> multiValued="false" required="false" omitNorms="true" docValues="true"
> /> 3/11/2015, 2:14:30 PM ERROR SolrDispatchFilter
> null:java.lang.IllegalStateException: unexpected docvalues type NONE
> for field 'DocumentType' (expected=SORTED). Use UninvertingReade
The answer meant it was most likely something user has done not quite
understanding Solr's behavior. Not a bug. I'd ignore that case and
just explain what your issue actually is.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On
Do you have any really old segments in that index? Could be worth
trying to optimize them down to one in latest format first.
Like Shawn, this is just a "one more idea" proposal.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
O
If you cannot trust your root users you probably have bigger problems than with
search... I think it has been suggested to encrypt on codec or directory level
as well. Yep, here is the JIRA
https://issues.apache.org/jira/browse/LUCENE-2228 :)
--
Jan Høydahl, search solution architect
Cominvent
Hello Alex,
I'm trying to add a new document, using solrj and the error "This IndexSchema
is not mutable" is raised when inserting the document in the solr index.
My index in solr, is configured with classicSchemaIndexFactory.
If I change it to AutoManaged the insert is done without any problems
For those who have not yet taken the leap to SSD goodness because they are
afraid of flash wear, the burnout test from The Tech Report seems worth a read.
The short story is that they wrote data to the drives until they wore out. All
tested drives survived considerably longer than guaranteed, bu
what does your schema.xml look like?
what does your solrconfig.xml look like?
what does the document you are indexing look like?
what is the full error with stack trace from your server logs?
details matter.
https://wiki.apache.org/solr/UsingMailingLists
: Date: Thu, 12 Mar 2015 20:27:05 +0
Wow, "optimize" worked like a charm. This really addressed the docvalues
issue. A follow-up question, is it recommended to run optimize in a
Production Solr index ? Also, in a Sorl cloud mode, do we need to run
optimize on each instance / each shard / any instance ?
Appreciate your help Alex.
-
Manual optimize is no longer needed for modern Solr. It does great
optimization automatically. The only reason I recommended it here is
to make sure that all segments are brought up to the latest version
and the deleted documents are purged. That's something that also would
happen automatically eve
Hi All,
I am having a solr cloud cluster of 20 nodes with each node having close to
20 Million records and total index size is around 400GB ( 20GB per node X 20
nodes ). I am trying to know the best way to dump out the entire solr data
in say CSV format.
I use successive queries by incrementing
Well, it's cursor or nothing. Well, or some sort of custom code to
manually read Lucene indexes (good luck with deleted items, etc).
I think your understanding is correct.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12 Ma
Thanks for sharing Toke!
Reliability should not be a problem for a Solr cloud environment. A corrupted
index cannot be loaded due to exceptions so the core should not enter an active
state. However, what would happen if parts of the data become corrupted but can
still be processed by the codec
Hi,
I’m currently working with indexes that need document level security. Based on
the user logged in, query results would omit documents that this user doesn’t
have access to, with LDAP integration and such.
I think that would be nice to have on a future Solr release.
Henrique.
> On Mar 12,
On 3/12/2015 12:43 PM, Pedro Figueiredo wrote:
> I saw an issue in Jira (https://issues.apache.org/jira/browse/SOLR-7234)
> with status Resolved, but the resolution is not identified in the issue.
>
> I am facing the exact same problem.. and not able to identified the
> solution.
I believe the pro
Lucene 5 has added a lot of various CRCs to catch index corruption
situations. I don't know if it is 'perfect', but there was certainly a
lot of work.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12 March 2015 at 18:39, Ma
Jan - we don't really need any security for our products, nor for most clients.
However, one client does deal with very sensitive data so we proposed to
encrypt the transfer of data and the data on disk through a Lucene Directory.
It won't fill all gaps but it would adhere to such a client's gui
Hello Alexandre - if you, and others, allow me to be a bit lazy right now; are
there unit tests that input corrupted segments, where not the structure but the
data is affected, to the codec?
Thanks,
Markus
-Original message-
> From:Alexandre Rafalovitch
> Sent: Thursday 12th March
Are there any results of off-heap cache vs JRE 8 with G1GC?
On 10 March 2015 at 11:13, Alexandre Rafalovitch wrote:
> Ask and you shall receive:
> SOLR-7210 Off-Heap filter cache
> SOLR-7211 Off-Heap field cache
> SOLR-7212 Parameter substitution
> SOLR-7214 JSON Facet API
> SOLR-7216 JSON Reque
Thanks Alex for quick response. I wanted to avoid reading the lucene index to
prevent complications of merging deleted info. Also I would like to do this
on very frequent basis as well like once in two or three days.
I am wondering if the issues that I faced while scraping the index towards
higher
Without cursor, you are rerunning a full search every time. So, slow
down is entirely expected.
With cursor, you do not. It does an internal skip based on cursor
value. I think the sort is there to ensure the value is stable.
Basically, you need to use the cursor.
Regards,
Alex.
Solr Ana
Well, I don't know this issue to such level of granularity. Perhaps others do.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12 March 2015 at 18:57, Markus Jelsma wrote:
> Hello Alexandre - if you, and others, allow me to b
Hello - i would assume off-heap would out perform any heap based data
structure. G1 is only useful if you deal with very large heaps, and it eats CPU
at the same time. As much as G1 is better than CMS in same cases, you would
still have less wasted CPU time and resp. less STW events.
Anyway. if
Thanks Alex for explanation. Actually since I am scraping all the contents
from Solr, I am doing a generic query of *:* So I think it should not take
so much time right?
But as you say probably the internal skips using the cursor might be more
efficient than the skip done with increasing the start
Hi folks,
I googled and tried without success so I ask you: how can I modify the
setting of a field to store it ?
It is interesting to note that I did not add _text field so I guess it
is a default one. Maybe it is normal that it is not showed on the result
but actually this is my real probl
Wait, step back. This is confusing. What's your real problem you are
trying to solve?
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12 March 2015 at 19:50, Mirko Torrisi wrote:
> Hi folks,
>
> I googled and tried without su
I would love to see record level (or even field level) restricted access in
Solr / Lucene.
This should be group level, LDAP like or some rule base (which can be dynamic).
If the solution means having a second core, so be it.
The following is the closest I found:
https://wiki.apache.org/solr/S
Hello,
I have written a python script to do 2 documents indexing
each time on Solr. I have 28 GB RAM with 8 CPU.
When I started indexing, at that time 15 GB RAM was freed. While indexing,
all RAM is consumed but **not** a single document is indexed. Why so?
And it through *HTTPError:
What's your commit strategy? Explicit commits? Soft commits/hard
commits (in solrconfig.xml)?
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 12 March 2015 at 23:19, Nitin Solanki wrote:
> Hello,
> I have written a
Hi Alexandre,
*Hard Commit* is :
${solr.autoCommit.maxTime:3000}
false
*Soft Commit* is :
${solr.autoSoftCommit.maxTime:300}
And I am committing 2 documents each time.
Is it good config for committing?
Or I am good something wrong ?
On Fri, Mar 13, 2015 a
Thanks Shawn and Erick for explanation...
On Thu, Mar 12, 2015 at 9:02 PM, Shawn Heisey wrote:
> On 3/12/2015 9:18 AM, Erick Erickson wrote:
> > By and large, I really never use linking. But it's about associating a
> > config set
> > you've _already_ uploaded with a collection.
> >
> > So uploa
Hi,
I want to retrieve the parent document which contain "Test Street" in street
field or if any of it's child contain "Test Street" in childStreet field.
So, I've used the following syntax.
q=street:"Test Street" OR {!parent which="type:parent"}childStreet:"Test
Street"
If the query after the OR
76 matches
Mail list logo
