Re: SolrCloud not distributing documents across shards

2013-04-03 Thread Mark Miller
On Apr 3, 2013, at 5:53 PM, Michael Della Bitta wrote: > From what I can tell, the Collections API has been hardened > significantly since 4.2 I did a lot of work here for 4.2.1 - there was a lot to improve. Hopefully there is much less now, but if anyone finds anything, I'll fix any JIRA's.

Re: Solr 4.2 Cloud Replication Replica has higher version than Master?

2013-04-03 Thread Mark Miller
nts >>> without any issues as of right now. I'll keep going and add more to see if >>> I find anything. >>> >>> >>> On Wed, Apr 3, 2013 at 4:01 PM, Jamie Johnson wrote: >>> >>>> ok, so that's not a deal breaker for

Re: Solr 4.2 Cloud Replication Replica has higher version than Master?

2013-04-03 Thread Mark Miller
the other hand, it's nicer to detect that faster. Your tradeoff to make. - Mark On Apr 3, 2013, at 7:46 PM, Mark Miller wrote: > Yeah. Are you using the concurrent low pause garbage collector? > > This means the overseer wasn't able to communicate with zk for 15 seconds - &

Re: Solr 4.2 Cloud Replication Replica has higher version than Master?

2013-04-03 Thread Mark Miller
dler.doHandle(SessionHandler.java:231) > > Nothing other than this in the log jumps out as interesting though. > > > On Wed, Apr 3, 2013 at 7:47 PM, Mark Miller wrote: > >> This shouldn't be a problem though, if things are working as they are >> supposed to.

[ANNOUNCE] Apache Solr 4.2.1 released

2013-04-03 Thread Mark Miller
April 2013, Apache Solr™ 4.2.1 available The Lucene PMC is pleased to announce the release of Apache Solr 4.2.1 Solr is the popular, blazing fast, open source NoSQL search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted searc

Re: hl.usePhraseHighlighter defaults to true but Query form and wiki suggest otherwise

2013-04-03 Thread Mark Miller
It was def intentional to make it default to true, but I believe that was changed at one point from initially defaulting to false - the doc was probably not updated and that slipped into he UI. Thanks for looking into this. - Mark On Apr 3, 2013, at 8:50 PM, Timothy Potter wrote: > Minor issu

Re: Solr 4.2 Cloud Replication Replica has higher version than Master?

2013-04-03 Thread Mark Miller
rFactory.newReader(StandardIndexReaderFactory.java:34) >at > org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:169) >... 18 more > > > > On Wed, Apr 3, 2013 at 8:54 PM, Jamie Johnson wrote: > >> Thanks I will try that. >> &

Re: Does solr cloud support rename or swap function for collection?

2013-04-06 Thread Mark Miller
4.2 and 4.2.1 have collection aliasing (similar to what we had with SolrCore aliasing at one point). You can use that to have one url and swap the collection search by it by the scenes. - Mark On Apr 6, 2013, at 6:28 AM, bradhill99 wrote: > Hi, > We are using solr 4.1 and we create a collecti

Re: Does solr cloud support rename or swap function for collection?

2013-04-07 Thread Mark Miller
On Apr 7, 2013, at 9:44 AM, bradhill99 wrote: > Thanks Mark for this great feature but I suggest you can update the wiki > too. Yeah, I've stopped updating the wiki for a while now looking back - paralysis on how to handle versions (I didn't want to do the std 'this applies to 4.1', 'this ap

Re: Does solr cloud support rename or swap function for collection?

2013-04-07 Thread Mark Miller
what I > can. > > Cheers, > > Tim > > On 07/04/13 12:28 PM, Mark Miller wrote: >> On Apr 7, 2013, at 9:44 AM, bradhill99 wrote: >> >>> Thanks Mark for this great feature but I suggest you can update the wiki >>> too. >> >> Yeah, I

Re: Index Replication Failing in Solr 4.2.1

2013-04-11 Thread Mark Miller
I was looking for this msg the other day and couldn't find it offhand… +1, please add this to JIRA so someone can look into it and it does not get lost! - Mark On Apr 11, 2013, at 11:17 AM, Otis Gospodnetic wrote: > Hi Umesh, > > The attachment didn't make it through. Could you please add

Re: CloudSolrServer vs ConcurrentUpdateSolrServer for indexing

2013-04-13 Thread Mark Miller
On Apr 13, 2013, at 11:07 AM, J Mohamed Zahoor wrote: > Hi > > This question has come up many times in the list with lots of variations > (which confuses me a lot). > > Iam using Solr 4.1. one collection , 6 shards, 6 machines. > I am using CloudSolrServer inside each mapper to index my docu

Re: zkState changes too often

2013-04-16 Thread Mark Miller
Are you using a the concurrent low pause garbage collector or perhaps G1? Are you able to use something like visualvm to pinpoint what the bottleneck might be? Otherwise, keep raising the timeout. This means Solr and Zk are not able to talk for that much time - either something needs to be tun

Re: updateLog in Solr 4.2

2013-04-16 Thread Mark Miller
Can you file a JIRA issue? - minimum you should get a better error. - Mark On Apr 12, 2013, at 9:17 AM, vicky desai wrote: > If i disable update log in solr 4.2 then i get the following exception > SEVERE: :java.lang.NullPointerException >at > org.apache.solr.cloud.ShardLeaderElectionCo

Re: SolrCloud Leader Response Mechanism

2013-04-16 Thread Mark Miller
Leaders don't have much to do with querying - the node that you query will determine what other nodes it has to query to search the whole index and do a scatter/gather for you. (Though in some cases that request can be proxied to another node) - Mark On Apr 16, 2013, at 7:48 AM, Furkan KAMACI

Re: Push/pull model between leader and replica in one shard

2013-04-16 Thread Mark Miller
On Apr 16, 2013, at 1:36 AM, SuoNayi wrote: > Hi, can someone explain more details about what model is used to sync docs > between the lead and > replica in the shard? > The model can be push or pull.Supposing I have only one shard that has 1 > leader and 2 replicas, > when the leader receive

Re: updateLog in Solr 4.2

2013-04-17 Thread Mark Miller
On Apr 17, 2013, at 9:17 AM, vicky desai wrote: > If updateLog tag is manadatory than why is it given as a parameter in > solrconfig.xml Because its not mandatory. - Mark

Re: Push/pull model between leader and replica in one shard

2013-04-17 Thread Mark Miller
I > >> Really nice presentation. >> >> >> 2013/4/17 Mark Miller >> >>> >>> On Apr 16, 2013, at 1:36 AM, SuoNayi wrote: >>> >>>> Hi, can someone explain more details about what model is used to sync >>> docs bet

Re: Solr 4.2 Startup Detects Corrupt Log And is Really Slow to Start

2013-04-17 Thread Mark Miller
On Apr 17, 2013, at 1:42 PM, Shawn Heisey wrote: > On 4/17/2013 10:29 AM, Umesh Prasad wrote: >> We use DIH and have turned off the Auto commit because we have to sometimes >> build index from Scratch (clean=true) and we not want to >> Our master server sees a lot of restarts, sometimes 2-3 time

Re: zkState changes too often

2013-04-18 Thread Mark Miller
On Apr 18, 2013, at 8:40 AM, jmozah wrote: > > > On 16-Apr-2013, at 11:16 PM, Mark Miller wrote: > >> Are you using a the concurrent low pause garbage collector or perhaps G1? > > > I use the default one which comes in jdk 1.7. It varies by platform, but 99% t

Re: Rogue query killed several replicas with OOM, after recovering - match all docs query problem

2013-04-20 Thread Mark Miller
Yeah, thats no good. You might hit each node with distrib=false to get the doc counts. Which ones have what you think are the right counts and which the wrong - eg is it all replicas that are off, or leaders as well? You say several replicas - do you mean no leaders went down? You might look c

Re: CloudSolrServer and update requests

2013-04-21 Thread Mark Miller
https://issues.apache.org/jira/browse/SOLR-3154 - Mark On Apr 21, 2013, at 1:28 PM, Timothy Potter wrote: > Today is my day for conceptual questions ;-) > > From what I understand, CloudSolrServer is "smart" because it uses > cluster state information pulled from Zookeeper to send update > req

Re: Rogue query killed several replicas with OOM, after recovering - match all docs query problem

2013-04-22 Thread Mark Miller
n a JIRA once I have a little more understanding of what >>> happened. Hopefully I can piece together some story after going over >>> the logs. >>> >>> As for replica / leader, I suspect some leaders went down but >>> fail-over to new leaders seemed to work

Re: Rogue query killed several replicas with OOM, after recovering - match all docs query problem

2013-04-22 Thread Mark Miller
What do you know about the # of docs you *should*? Do you have that mean when taking the bad replica out of the equation? - Mark On Apr 22, 2013, at 4:33 PM, Mark Miller wrote: > Bummer on the log loss :( > > Good info though. Somehow that replica became active without actually

Re: Soft Commit and Document Cache

2013-04-22 Thread Mark Miller
Yup - all of the top level caches are. It's a trade off - don't NRT more than you need to. - Mark On Apr 22, 2013, at 6:16 PM, Niran Fajemisin wrote: > Hi all, > > A quick (and hopefully simply) question: Does the document cache (or any of > the other caches for that matter), get invalidated

Re: Rogue query killed several replicas with OOM, after recovering - match all docs query problem

2013-04-22 Thread Mark Miller
io to > try to re-create as it was a perfect storm of high indexing throughput > and a rogue query. > > Tim > > On Mon, Apr 22, 2013 at 2:41 PM, Mark Miller wrote: >> What do you know about the # of docs you *should*? Do you have that mean >> when taking the bad

Re: Update on shards

2013-04-23 Thread Mark Miller
If you use jetty - which you should :) It's what we test with. Tomcat only gets user testing. If you use tomcat, this won't work in 4.2 or 4.2.1, but probably will in 4.3 (we are voting on 4.3 now). No clue on other containers. - Mark On Apr 23, 2013, at 10:59 AM, Michael Della Bitta wrote:

Re: Update on shards

2013-04-23 Thread Mark Miller
you use tomcat, this won't work in 4.2 or 4.2.1" > > Can you explain more what won't be at Tomcat and what will change at 4.3? > > 2013/4/23 Mark Miller > >> If you use jetty - which you should :) It's what we test with. Tomcat only >> gets user te

Re: Update on shards

2013-04-23 Thread Mark Miller
lr proxying? Is it same thing with that: when > I use SolrCloud and if I send document any of the nodes at my cluster the > document will be routed into the leader of appropriate shard. So you mean I > can not do that if I use Tomcat? > > 2013/4/23 Mark Miller > >> The

Re: What is cluster overseer at SolrCloud?

2013-04-23 Thread Mark Miller
On Apr 23, 2013, at 2:53 PM, Furkan KAMACI wrote: > When I read about SolrCloud wiki there writes something about "cluster > overseer". What is the role of that at read and write processes? How can I > see which node is overseer at my cluster? The Overseer's main responsibility is to write the

Re: Update on shards

2013-04-23 Thread Mark Miller
On Apr 23, 2013, at 2:49 PM, Shawn Heisey wrote: > What exactly is the 'request proxying' thing that doesn't work on tomcat? Is > this something different from basic SolrCloud operation where you send any > kind of request to any server and they get directed where they need to go? I > haven'

Re: Update on shards

2013-04-23 Thread Mark Miller
ed at repository (I mean when it is > ready) this feature will work for Tomcat too at a stable version? > > > 2013/4/23 Mark Miller > >> >> On Apr 23, 2013, at 2:49 PM, Shawn Heisey wrote: >> >>> What exactly is the 'request proxying' thing

Re: Using Solr For a Real Search Engine

2013-04-23 Thread Mark Miller
Tomcat should work just fine in most cases. The downside to Tomcat is that all of the devs generally run Jetty since it's the default. Also, all of our units tests run against Jetty - in fact, a specific version of Jetty. Usually, Solr will run fine in other webapps. Many, many users run Solr in

Re: Does SolrCloud supports QueryElevationComponent?

2013-04-23 Thread Mark Miller
No, I'm fairly sure we added support a year or less back. - Mark On Apr 23, 2013, at 5:56 PM, Furkan KAMACI wrote: > When I read Lucidworks' Solr Guide I saw that: > > "Distributed searching does not support the QueryElevationComponent, which > configures the > top results for a given query re

Re: JVM Parameters to Startup Solr?

2013-04-24 Thread Mark Miller
On Apr 24, 2013, at 4:02 AM, Furkan KAMACI wrote: > Lucidworks Solr Guide says that: > > "If you are using Sun's JVM, add the -server command-line option when you > start Solr. This tells the JVM that it should optimize for a long running, > server process. If the Java runtime on your system is

Re: JVM Parameters to Startup Solr?

2013-04-24 Thread Mark Miller
VM options to > kill on OOM? I vaguely recall a message from Mark about this sometime > ago: > > -XX:OnOutOfMemoryError="kill -9 %p" -XX:+HeapDumpOnOutOfMemoryError > > On Wed, Apr 24, 2013 at 9:13 AM, Mark Miller wrote: >> >> On Apr 24, 2013, at 4:02 AM

Re: JVM Parameters to Startup Solr?

2013-04-24 Thread Mark Miller
On Apr 24, 2013, at 12:00 PM, Mark Miller wrote: >> -XX:OnOutOfMemoryError="kill -9 %p" -XX:+HeapDumpOnOutOfMemoryError The way I like to handle this is to have the OOM trigger a little script or set of cmds that logs the issue and kills the process. Then if you have the pr

Re: JVM Parameters to Startup Solr?

2013-04-24 Thread Mark Miller
heap dump, that has the same info, but a log is just a nice compact little history of events. - Mark > > Thanks. > Tim > > On Wed, Apr 24, 2013 at 10:03 AM, Mark Miller wrote: >> >> On Apr 24, 2013, at 12:00 PM, Mark Miller wrote: >> >>>> -XX:OnOut

Re: Update on shards

2013-04-24 Thread Mark Miller
update (or accept the update). - Mark On Apr 23, 2013, at 3:38 PM, Mark Miller wrote: > We have a 3rd release candidate for 4.3 being voted on now. > > I have never tested this feature with Tomcat - only Jetty. Users have > reported it does not work with Tomcat. That leads one to

Re: How to Clean Zookeeper Data for Solr

2013-04-25 Thread Mark Miller
What are you doing to clean zk? You should be able to simply use the ZkCli clear cmd: http://wiki.apache.org/solr/SolrCloud#Command_Line_Util Just make sure you stop your Solr instances before clearing it. Clearing out zk from under a running Solr instance is not a good thing to do. This shoul

Re: How to Clean Zookeeper Data for Solr

2013-04-25 Thread Mark Miller
Of course deleting the collection and then recreating it should also work - if it doesn't, there is a bug to address. - Mark On Apr 25, 2013, at 12:00 PM, Mark Miller wrote: > What are you doing to clean zk? > > You should be able to simply use the ZkCli clear

Re: How to Clean Zookeeper Data for Solr

2013-04-25 Thread Mark Miller
lr.data.dir=/home/solr-4.2.1/solr/data -jar start.jar > > However when I look at the graph at Admin GUI there is only one shard but > two replicas? What is the problem why it is not two shards? > > > 2013/4/25 Mark Miller > >> Of course deleting the collection and t

Re: Log Monitor System for SolrCloud and Logging to log4j at SolrCloud?

2013-04-26 Thread Mark Miller
Slf4j is meant to work with existing frameworks - you can set it up to work with log4j, and Solr will use log4j by default in the about to be released 4.3. http://wiki.apache.org/solr/SolrLogging - Mark On Apr 26, 2013, at 7:19 AM, Furkan KAMACI wrote: > I want to use GrayLog2 to monitor my l

Re: createNodeSet

2013-04-29 Thread Mark Miller
What version of Solr? That should work in Jetty in 4.2 and not before and in Tomcat in 4.3 and not before. - Mark On Apr 29, 2013, at 10:19 AM, Arkadi Colson wrote: > When I first do a linkconfig the route:implicit seems to be gone! So > recreating the collection will solve this. The problem

Re: Solr 4.2 rollback not working

2013-05-04 Thread Mark Miller
Rollback is unsupported in SolrCloud - this is a bug really - you should get an error back. - Mark On May 1, 2013, at 11:12 PM, Dipti Srivastava wrote: > Hi All, > WE have setup a 4.2 Solr cloud with 4 nodes and while the add/update/delete > operations are working we are not able to perform

Re: Pulling Config Folder from Zookeeper at SolrCloud

2013-05-04 Thread Mark Miller
The ZkCli cmd line tool allows you to pull config down too - so just pull the latest down, update it, and push it back up. Alternatively, there may be ZK UI tools that let you do this if you look. - Mark On May 2, 2013, at 1:45 AM, Furkan KAMACI wrote: > I use the same folder naming conventio

Re: disaster recovery scenarios for solr cloud and zookeeper

2013-05-05 Thread Mark Miller
When Solr loses it's connection to ZooKeeper, updates will start being rejected. Read requests will continue as normal. This is regardless of how long ZooKeeper is down. So it's pretty simple - when you lost the ability to talk to ZK, everything keeps working based on the most recent clustersta

Re: zookeeper errors

2013-05-05 Thread Mark Miller
It sounds like you probably need to raise the default 15 sec zk client timeout. We have it default to a fairly aggressive setting. For high load envs, you may have to bring it to 30 or 45 seconds. Also, be sure you are not using a stop the world gc collector - CMS (the concurrent low pause coll

Re: disaster recovery scenarios for solr cloud and zookeeper

2013-05-06 Thread Mark Miller
; >> -- Jack Krupansky >> >> -Original Message- From: Mark Miller >> Sent: Sunday, May 05, 2013 2:19 PM >> To: solr-user@lucene.apache.org >> Subject: Re: disaster recovery scenarios for solr cloud and zookeeper >> >> >> When Solr l

Re: Solr 4.3 and SLF4j

2013-05-06 Thread Mark Miller
You need all the same jars that are in the lib/ext folder of the default jetty distribution. Those are the logging jars, those are what you need. All you can do is swap out impls (see the SLF4j documentation). You must have all those jars as a start, and if you don't want to use log4j, you can s

Re: Solr Cloud with large synonyms.txt

2013-05-07 Thread Mark Miller
On May 6, 2013, at 12:32 PM, Son Nguyen wrote: > I did some researches on internet and found out that because Zookeeper znode > size limit is 1MB. I tried to increase the system property "jute.maxbuffer" > but it won't work. > Does anyone have experience of dealing with it? Perhaps hit up the

Re: Solr Cloud with large synonyms.txt

2013-05-07 Thread Mark Miller
On May 7, 2013, at 10:24 AM, Mark Miller wrote: > > On May 6, 2013, at 12:32 PM, Son Nguyen wrote: > >> I did some researches on internet and found out that because Zookeeper znode >> size limit is 1MB. I tried to increase the system property "jute.maxbuffer"

Re: Solr Cloud with large synonyms.txt

2013-05-07 Thread Mark Miller
ZK (I have only one ZK instance) and > Solr, but it still didn't work. > But I read somewhere that ZK is not really designed for keeping large data > files, so this solution - increasing jute.maxbuffer (if I can implement it) > should be just temporary. > > Son > > ---

Re: Solr 4.2 rollback not working

2013-05-09 Thread Mark Miller
At the least it should throw an exception if you try rollback with SolrCloud - though now there is discussion about removing it entirely. But yes, it's not supported and there are no real plans to support it. - Mark On May 9, 2013, at 7:21 AM, mark12345 wrote: > > So for all current versions

Re: Solr Admin Page Says Leader is Down, Replica is Up, Zookeeper Says Thar They are Both Active

2013-05-14 Thread Mark Miller
The actual state is a mix of the clusterstate.json and the ephemeral live nodes - a node may be listed as active or whatever, and if it's live node is not up, it doesn't matter - it's considered down. - Mark On May 14, 2013, at 8:08 AM, Furkan KAMACI wrote: > Node is shown as down at admin pa

Re: Using the Collections API

2013-05-15 Thread Mark Miller
Yeah, I use both on an empty Solr - what is the error? - Mark On May 15, 2013, at 6:53 AM, A.Eibner wrote: > Hi, > > I just wanted to ask, if anyone is using the collections API to create > collections, > or if not how they use the coreAPI to create a collection with replication ? > > Becaus

Re: SEVERE:IOException occured when talking to server

2013-05-15 Thread Mark Miller
I'd use Jetty for SolrCloud - much, much, much better tested. Here is a note on something similar around tomcat: http://stackoverflow.com/questions/10570672/get-nohttpresponseexception-for-load-testing Perhaps that helps, perhaps not. The root cause is: org.apache.http.NoHttpResponseException:

Re: SOLR Cloud Collection Management quesiotn.

2013-05-15 Thread Mark Miller
You cannot currently adjust the number of replicas with the collections api - you have to use the core admin api. Which means you determine the replica placement based on what server you hit with the core admin api. http://wiki.apache.org/solr/SolrCloud#Creating_cores_via_CoreAdmin Create 2 mor

Re: Using the Collections API

2013-05-15 Thread Mark Miller
On May 15, 2013, at 12:26 PM, Jared Rodriguez wrote: > the cores in the collection stay offline even if there are no > material changes. I've used reload - if you are having trouble with it, please post more details or file a JIRA issue. - Mark

Re: Compatible collections SOLR4 / SOLRCloud?

2013-05-15 Thread Mark Miller
They need to be similar enough to satisfy the particular queries. - Mark On May 15, 2013, at 12:23 PM, Marcin wrote: > Hi there, > > I am trying to figure out what SOLR means by compatible collection in order > to be able to run the following query: > > |Query all shards of multiple compatib

Re: question about the file data/index.properties

2013-05-15 Thread Mark Miller
It's fairly meaningless from a user perspective, but it happens when an index is replicated that cannot be simply merged with the existing index files and needs a new directory. - Mark On May 15, 2013, at 5:38 PM, Bill Au wrote: > I am running 2 separate 4.3 SolrCloud clusters. On one of the

Re: Migrating from 4.2.1 to 4.3.0

2013-05-16 Thread Mark Miller
Your solr webapp context appears to be "" rather than "solr". There was a JIRA issue in 4.3 that may have affected this, but I only saw it from a distance, so just a guess. What does it say in solr.xml for the context (an attribute on ) - Mark On May 16, 2013, at 2:02 PM, "M. Flatterie" wrote

Re: Controlling which node(s) hold(s) a collection

2013-05-16 Thread Mark Miller
You can control simply with the CoreAdmin api - the core is created at the location of whatever url you use…simply fire the creates at whatever nodes you want the collection to live on. The collections api also takes a list of nodes names to use optionally. - Mark On May 16, 2013, at 7:34 PM,

Re: Zookeeper Ensemble Startup Parameters For SolrCloud?

2013-05-17 Thread Mark Miller
The way Solr uses ZK, unless you are also using ZK with something else, I wouldn't worry about it at all. In a steady state, the cluster won't even really talk to ZK in any intensive manner at all. - Mark On May 16, 2013, at 5:07 PM, Furkan KAMACI wrote: > Hi Shawn; > > I will have totally 1

Re: Using the Collections API

2013-05-17 Thread Mark Miller
What version of Solr? I think there was a bug a couple versions back (perhaps introduced in 4.1 if I remember right) that made it so creates were not spread correctly. - Mark

Re: Core admin action "CREATE" fails for existing core

2013-05-23 Thread Mark Miller
Yes, this did change - it's actually a protection for a previous change though. There was a time when you did a core reload by just making a new core with the same name and closing the old core - that is no longer really supported though - the proper way to do this is to use SolrCore#reload, and

Re: Core admin action "CREATE" fails for existing core

2013-05-23 Thread Mark Miller
n May 23, 2013, at 12:29 PM, Alan Woodward wrote: >> >>> I think the wiki needs to be updated to reflect this? >>> http://wiki.apache.org/solr/CoreAdmin >>> >>> If somebody adds me as an editor (AlanWoodward), I'll do it. >>> >>> Alan

Re: Collections API Reload killing my cloud

2013-05-30 Thread Mark Miller
https://issues.apache.org/jira/browse/SOLR-4805 - Mark On May 30, 2013, at 3:09 PM, davers wrote: > Everytime I try to do a reload using the collections API my entire cloud goes > down and I cannot search it. The solrconfig.xml and schema.xml are good > because when I just restart tomcat everyt

Re: SolrCloud Load Balancer "weight"

2013-06-03 Thread Mark Miller
On Jun 3, 2013, at 3:33 PM, Tim Vaillancourt wrote: > Should I JIRA this? Thoughts? Yeah - it's always been in the back of my mind - it's come up a few times - eventually we would like nodes to report some stats to zk to influence load balancing. - mark

Re: How to Get Cluster State By Solrj?

2013-06-03 Thread Mark Miller
It actually accepts a comma separated list of zk host addresses (your quorum). Same format as zk describes in it's docs. To get the cluster state, get the ZkStateReader from the CloudSolrServer and then it's getClusterState or something. - Mark On Jun 3, 2013, at 5:30 PM, Furkan KAMACI wrote:

Re: Leader election deadlock after restarting leader in 4.2.1

2013-06-03 Thread Mark Miller
Thanks - I can try and look into this perhaps next week. You might copy the details into a JIRA issue to prevent it from getting lost though... - Mark On Jun 3, 2013, at 4:46 PM, John Guerrero wrote: > SOLR 4.2.1, tomcat 6.0.35, CentOS 6.2 (2.6.32-220.4.1.el6.x86_64 #1 SMP), > java 6u27 64 bit

Re: problem with zkcli.sh linkconfig

2013-06-05 Thread Mark Miller
Sounds like a bug - we probably don't have a test that updates a link - if you can make a JIRA issue, I'll be happy to look into it soon. - Mark On Jun 4, 2013, at 8:16 AM, Shawn Heisey wrote: > I've got Solr 4.2.1 running SolrCloud. I need to change the config set > associated with a collect

Re: HdfsDirectoryFactory

2013-06-07 Thread Mark Miller
Eagle eye man. Yeah, we plan on contributing hdfs support for Solr. I'm flying home today and will create a JIRA issue for it shortly after I get there. - Mark On Jun 6, 2013, at 6:16 PM, Jamie Johnson wrote: > I've seen reference to an HdfsDirectoryFactory in the new Cloudera Search > along

Re: index merge question

2013-06-08 Thread Mark Miller
On Jun 8, 2013, at 12:52 PM, Jamie Johnson wrote: > When merging through the core admin ( > http://wiki.apache.org/solr/MergingSolrIndexes) what is the policy for > conflicts during the merge? So for instance if I am merging core 1 and > core 2 into core 0 (first example), what happens if core

Re: Lucene/Solr Filesystem tunings

2013-06-08 Thread Mark Miller
Turning swappiness down to 0 can have some decent performance impact. - http://en.wikipedia.org/wiki/Swappiness In the past, I've seen better performance with ext3 over ext4 around commits/fsync. Test were actually enough slower (lots of these operations), that I made a special ext3 partition w

Re: Why clusterstate.json says active for a killed Solr Node?

2013-06-09 Thread Mark Miller
The true current state is the live nodes info combined with the clusterstate.json. If a node is not live, whatever is in clusterstate.json is simply it's last state, not the current one. - Mark On Sun, Jun 9, 2013 at 4:40 PM, Furkan KAMACI wrote: > I want to get cluster state of my SolrCloud an

Re: Why clusterstate.json says active for a killed Solr Node?

2013-06-09 Thread Mark Miller
s(if not: could you tell me is there > any example code part at Solr source code)? By the way what does active means > for clusterstate.json? > > 2013/6/10 Mark Miller > The true current state is the live nodes info combined with the > clusterstate.json. If a node i

Re: Get Statistics With CloudSolrServer?

2013-06-09 Thread Mark Miller
On Jun 9, 2013, at 7:52 PM, Furkan KAMACI wrote: > There is a stat,st,cs section at admin page and gives information as like: > > Last Modified, Num Docs, Max Doc and etc. How can I get such kind of > information using CloudSolrServer with Solrj? There is an admin request handler that exposes

Re: external zookeeper with SolrCloud

2013-06-10 Thread Mark Miller
This might be https://issues.apache.org/jira/browse/SOLR-4899 - Mark On Jun 10, 2013, at 5:59 PM, "Joshi, Shital" wrote: > Hi, > > > > We're setting up 5 shard SolrCloud with external zoo keeper. When we bring up > Solr nodes while the zookeeper instance is not up and running, we see this

Re: index merge question

2013-06-11 Thread Mark Miller
meone updates a doc > that update could be lost from a solr perspective. I guess what would need > to happen to ensure the correct information was indexed would be to record > the start time and reindex the information that changed since then? > On Jun 8, 2013 2:37 PM, "Mark Mill

Re: Why there is no getter method for defaultCollection at CloudSolrServer?

2013-06-11 Thread Mark Miller
On Jun 11, 2013, at 4:51 AM, Furkan KAMACI wrote: > Why there is no getter method for defaultCollection at CloudSolrServer? Want to create a JIRA issue to add it? - Mark

Re: external zookeeper with SolrCloud

2013-06-11 Thread Mark Miller
o build Solr 4.4 from SVN > repository? It's java, so it's pretty easy - you might find some help here: http://wiki.apache.org/solr/HowToContribute - Mark > > -----Original Message- > From: Mark Miller [mailto:markrmil...@gmail.com] > Sent: Monday, June 10, 2013 8:05

Re: index merge question

2013-06-11 Thread Mark Miller
the concepts of a unique > Key(solr land concept) > > http://wiki.apache.org/solr/MergingSolrIndexes has a cautionary note at the > end > > we do frequent index merges for which we externally run map/reduce ( java > code using lucene api's) jobs to merge & validate merged indi

Re: What is Difference Between Down and Gone At Admin Cloud Page?

2013-06-12 Thread Mark Miller
On Jun 12, 2013, at 3:19 AM, Furkan KAMACI wrote: > What is Difference Between Down and Gone At Admin Cloud Page? If I remember right, Down can mean the node is still actively working towards something - eg, without action by you, it might go into recovering or active state. Gone means it has

Re: Solr Cloud Hangs consistently .

2013-06-17 Thread Mark Miller
Could you give a simple stack trace dump as well? It's likely the distributed update deadlock that has been reported a few times now - I think usually with a replication factor greater than 2, but I can't be sure. The deadlock involves sending docs concurrently to replicas and I wouldn't have e

Re: Solr Cloud Hangs consistently .

2013-06-17 Thread Mark Miller
If it actually happens with replicationFactor=1, it doesn't likely have anything to do with the update handler issue I'm referring to. In some cases like these, people have better luck with Jetty than Tomcat - we test it much more. For instance, it's setup to help avoid search side distributed d

Re: Avoiding OOM fatal crash

2013-06-17 Thread Mark Miller
There is a java cmd line arg that lets you run a command on OOM - I'd configure it to log and kill -9 Solr. Then use runit or something to supervice Solr - so that if it's killed, it just restarts. I think that is the best way to deal with OOM's. Other than that, you have to write a middle laye

Re: Shard splitting and document routing

2013-06-18 Thread Mark Miller
No, the hash ranges are split and new docs go to both new shards. - Mark On Jun 18, 2013, at 12:25 PM, Otis Gospodnetic wrote: > Hi, > > Imagine a (common) situation where you use document routing and you > end up with 1 large shards (e.g. 1 large user with lots of docs). > Shard splitting wi

Re: shard failure, leader transition took 11s (seems high?)

2013-06-24 Thread Mark Miller
It will take a short bit of a time before a new leader takes over when a leader goes - that's expected - how long it takes will vary. Some things will do short little retries to kind of deal with this, but you are alerted those updates failed, so you have to deal with that as you would other upd

Re: Waiting until we see more replicas up message??

2013-06-24 Thread Mark Miller
This is a safety mechanism - you can turn it off by configuring leaderVoteWait to 0 in solr.xml. This is meant to protect the case where you stop a shard or it fails and then the first node to get started back up has stale data - you don't want it to just become the leader. So we wait to see ev

New NearRealtime feature in Solr trunk.

2011-09-04 Thread Mark Miller
For those that would like to start diving in, I've started documenting the recent NearRealtime work (available on trunk) on the wiki: http://wiki.apache.org/solr/NearRealtimeSearch - Mark Miller lucidimagination.com 2011.lucene-eurocon.org | Oct 17-20 | Barcelona

Re: StreamingUpdateSolrServer#handleError

2011-09-05 Thread Mark Miller
n't see how to keep > track of failures, i'd like to discover > which documents failed during the request. > > thanks in advance! > > -- > Leonardo S Souza - Mark Miller lucidimagination.com 2011.lucene-eurocon.org | Oct 17-20 | Barcelona

Re: SolrCloud and TermsComponent

2011-09-07 Thread Mark Miller
SolrCloud? Are you using trunk? I seem to remember a bug from a long time ago on the SolrCloud branch with this...if that's it, should be fixed though. - Mark Miller lucidimagination.com 2011.lucene-eurocon.org | Oct 17-20 | Barcelona

Re: Solr Cloud - is replication really a feature on the trunk?

2011-09-10 Thread Mark Miller
nd how they are organized in shards. >> >> >>> because you have a slightly different config between Node 1 & 2: >>> http://lucene.472066.n3.nabble.com/Replication-setup-with-SolrCloud-Zk-td2952602.html >> >> >> I have two shards, each shard having a master and a slave core. >> Cores are located so that master and slave are on different nodes. >> This protects search (but not indexing) from node failure. >> - Mark Miller lucidimagination.com 2011.lucene-eurocon.org | Oct 17-20 | Barcelona

Re: SolrCloud Feedback

2011-09-11 Thread Mark Miller
a reference only >>> >>> d) How are admins supposed to update configs in ZK? Install their favourite >>> ZK editor? >>> >>> e) We should perhaps not be so afraid to make ZK a requirement for Solr in >>> v4. Ideally you should interact with a 1-

Re: Solr Cloud Number of Shard Limitation?

2011-09-27 Thread Mark Miller
their own code, but we have not yet done this in trunk. What that places the upper limit at, I don't know - I imagine it takes quite a few shards before it ends up being too much of a problem - they shard by user I believe, so lot's of shards. - Mark Miller lucidimagination.com 201

Re: Solr Cloud Number of Shard Limitation?

2011-09-28 Thread Mark Miller
, read the data for those not in the list, and get your new state efficiently. - Mark Miller lucidimagination.com 2011.lucene-eurocon.org | Oct 17-20 | Barcelona On Sep 28, 2011, at 10:35 PM, Jamie Johnson wrote: > Thanks Mark found the TODO in ZkStateReader.java > > // TODO: - possib

Re: SolrCloud: is there a programmatic way to create an ensemble

2011-09-29 Thread Mark Miller
gt; (Solr launched with -DzkRun). >>> >>> I would like to create an ensemble out of them. I know about -DzkHost >>> parameter, but can I achieve the same programmatically? Either with >>> SolrJ or REST API? >>> >>> Thanks, >>> Yury >>> >> - Mark Miller lucidimagination.com 2011.lucene-eurocon.org | Oct 17-20 | Barcelona

Re: basic solr cloud questions

2011-09-30 Thread Mark Miller
>> don't know if this does this right now as I am not familiar with the >> source code. If someone could answer this it would be great otherwise >> I'll post back eventually when I do become familiar. >> >> Cheers, >> - Pulkit >> - Mark Miller lucidimagination.com 2011.lucene-eurocon.org | Oct 17-20 | Barcelona

Re: Automate startup/shutdown of SolrCloud Shards

2011-09-30 Thread Mark Miller
, "8983", "solr") zkController.uploadConfigDir(directory, configName); zkController.close(); - Mark Miller lucidimagination.com 2011.lucene-eurocon.org | Oct 17-20 | Barcelona

<    1   2   3   4   5   6   7   8   9   10   >