High Cpu sys usage

[YouPeng Yang]

Hi
  We are using Solr Cloud 4.6 in our production for searching service
since 2 years ago.And now it has 700GB in one cluster which is  comprised
of 3 machines with ssd. At beginning ,everything go well,while more and
more business services interfered with our searching service .And a problem
 which we haunted with is just like a  nightmare . That is the cpu sys
usage is often growing up to  over 10% even higher, and as a result the
machine will hang down because system resources have be drained out.We have
to restart the machine manually.
  We have done some google,and we also know the high cpu sys usage is due
to system kernel.But we do not have further more information about the
problem other than the solution.
  Is it perhaps because of the increase of the search concurrence?
  Do you guys have experiences like us.How do you solve it?
  Please any suggestion will be appreciated.

Re: High Cpu sys usage

[YouPeng Yang]

Hi all
  Thanks for your reply.I do some investigation for much time.and I will
post some logs of the 'top' and IO in a few days when the crash come again.

2016-03-08 10:45 GMT+08:00 Shawn Heisey :

> On 3/7/2016 2:23 AM, Toke Eskildsen wrote:
> > How does this relate to YouPeng reporting that the CPU usage increases?
> >
> > This is not a snark. YouPeng mentions kernel issues. It might very well
> > be that IO is the real problem, but that it manifests in a non-intuitive
> > way. Before memory-mapping it was easy: Just look at IO-Wait. Now I am
> > not so sure. Can high kernel load (Sy% in *nix top) indicate that the IO
> > system is struggling, even if IO-Wait is low?
>
> It might turn out to be not directly related to memory, you're right
> about that.  A very high query rate or particularly CPU-heavy queries or
> analysis could cause high CPU usage even when memory is plentiful, but
> in that situation I would expect high user percentage, not kernel.  I'm
> not completely sure what might cause high kernel usage if iowait is low,
> but no specific information was given about iowait.  I've seen iowait
> percentages of 10% or less with problems clearly caused by iowait.
>
> With the available information (especially seeing 700GB of index data),
> I believe that the "not enough memory" scenario is more likely than
> anything else.  If the OP replies and says they have plenty of memory,
> then we can move on to the less common (IMHO) reasons for high CPU with
> a large index.
>
> If the OS is one that reports load average, I am curious what the 5
> minute average is, and how many real (non-HT) CPU cores there are.
>
> Thanks,
> Shawn
>
>

Re: High Cpu sys usage

[YouPeng Yang]

Hello
   The problem appears several times ,however I could not capture the top
output .My script is as follows code.
I check the sys cpu usage whether it exceed 30%.the other metric
information can be dumpped successfully except the top .
Would you like to check my script that I am not able to figure out what is
wrong.

-
#!/bin/bash

while :
  do
sysusage=$(mpstat 2 1 | grep -A 1 "%sys" | tail -n 1 | awk '{if($6 <
30) print 1; else print 0;}' )

if [ $sysusage -eq 0 ];then
#echo $sysusage
#perf record -o perf$(date +%Y%m%d%H%M%S).data  -a -g -F 1000 sleep
30
file=$(date +%Y%m%d%H%M%S)
top -n 2 >> top$file.data
iotop -b -n 2  >> iotop$file.data
iostat >> iostat$file.data
netstat -an | awk '/^tcp/ {++state[$NF]} END {for(i in state) print
i,"\t",state[i]}' >> netstat$file.data
fi
sleep 5
  done
You have new mail in /var/spool/mail/root


---------

2016-03-08 21:39 GMT+08:00 YouPeng Yang :

> Hi all
>   Thanks for your reply.I do some investigation for much time.and I will
> post some logs of the 'top' and IO in a few days when the crash come again.
>
> 2016-03-08 10:45 GMT+08:00 Shawn Heisey :
>
>> On 3/7/2016 2:23 AM, Toke Eskildsen wrote:
>> > How does this relate to YouPeng reporting that the CPU usage increases?
>> >
>> > This is not a snark. YouPeng mentions kernel issues. It might very well
>> > be that IO is the real problem, but that it manifests in a non-intuitive
>> > way. Before memory-mapping it was easy: Just look at IO-Wait. Now I am
>> > not so sure. Can high kernel load (Sy% in *nix top) indicate that the IO
>> > system is struggling, even if IO-Wait is low?
>>
>> It might turn out to be not directly related to memory, you're right
>> about that.  A very high query rate or particularly CPU-heavy queries or
>> analysis could cause high CPU usage even when memory is plentiful, but
>> in that situation I would expect high user percentage, not kernel.  I'm
>> not completely sure what might cause high kernel usage if iowait is low,
>> but no specific information was given about iowait.  I've seen iowait
>> percentages of 10% or less with problems clearly caused by iowait.
>>
>> With the available information (especially seeing 700GB of index data),
>> I believe that the "not enough memory" scenario is more likely than
>> anything else.  If the OP replies and says they have plenty of memory,
>> then we can move on to the less common (IMHO) reasons for high CPU with
>> a large index.
>>
>> If the OS is one that reports load average, I am curious what the 5
>> minute average is, and how many real (non-HT) CPU cores there are.
>>
>> Thanks,
>> Shawn
>>
>>
>

Re: High Cpu sys usage

[YouPeng Yang]

Hi Shawn

   Actually,there are three Solr instances(The top three PIDs is the three
instances),and the datafile size of the stuff is 851G,592G,49G respectively
,and more and more data will be added as time going.I think it may be rare
as the large scope as my solrcloud service .and
it is now one of the most important core service in my company.
Just as you suggest,the increasing size of data make us to devide our
SolrCloud service into smaller application clusters.and we do have
 separated our collection into smaller shards .and I know   there must be
some abnormal things on the service when time is going.however the unknown
reason high sys cpu is right now as a nightmare.So I look for help from our
community.
   Would you have some experience as me and how you solve this problem?




Best Regards




2016-03-17 14:16 GMT+08:00 Shawn Heisey :

> On 3/16/2016 8:27 PM, YouPeng Yang wrote:
> > Hi Shawn
> >Here is my top screenshot:
> >
> >https://www.dropbox.com/s/jaw10mkmipz943y/topscreen.jpg?dl=0
> >
> >It is captured when my system is normal.And I have reduced the memory
> > size down to 48GB originating  from 64GB.
>
> It looks like you have at least two Solr instances on this machine, one
> of which has over 600GB of index data, and the other has over 500GB of
> data.  There may be as many as ten Solr instances, but I cannot tell for
> sure what those Java processes are.
>
> If my guess is correct, this means that there's over a terabyte of index
> data, but you only have about 100GB of RAM available to cache it.  I
> don't think this is enough RAM for good performance, even if the disks
> are SSD.  You'll either need a lot more memory in each machine, or more
> machines.  The data may need to be divided into more shards.
>
> I am not seeing any evidence here of high CPU.  The system only shows
> about 12 percent total CPU usage, and very little of it is system (kernel).
>
> Thanks,
> Shawn
>
>

Re: High Cpu sys usage

[YouPeng Yang]

Hi
  To Patrick: Never mind .Thank you for your suggestion all the same.
  To Otis. We do not use SPM. We monintor the JVM just use jstat becasue my
system went well before ,so we do not need  other tools.
But SPM is really awesome .

  Still looking for help.

Best Regards

2016-03-18 6:01 GMT+08:00 Patrick Plaatje :

> Yeah, I did’t pay attention to the cached memory at all, my bad!
>
> I remember running into a similar situation a couple of years ago, one of
> the things to investigate our memory profile was to produce a full heap
> dump and manually analyse that using a tool like MAT.
>
> Cheers,
> -patrick
>
>
>
>
> On 17/03/2016, 21:58, "Otis Gospodnetić" 
> wrote:
>
> >Hi,
> >
> >On Wed, Mar 16, 2016 at 10:59 AM, Patrick Plaatje 
> >wrote:
> >
> >> Hi,
> >>
> >> From the sar output you supplied, it looks like you might have a memory
> >> issue on your hosts. The memory usage just before your crash seems to be
> >> *very* close to 100%. Even the slightest increase (Solr itself, or
> possibly
> >> by a system service) could caused the system crash. What are the
> >> specifications of your hosts and how much memory are you allocating?
> >
> >
> >That's normal actually - http://www.linuxatemyram.com/
> >
> >You *want* Linux to be using all your memory - you paid for it :)
> >
> >Otis
> >--
> >Monitoring - Log Management - Alerting - Anomaly Detection
> >Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> >
> >
> >
> >
> >>
> >
> >
> >>
> >>
> >> On 16/03/2016, 14:52, "YouPeng Yang"  wrote:
> >>
> >> >Hi
> >> > It happened again,and worse thing is that my system went to crash.we
> can
> >> >even not connect to it with ssh.
> >> > I use the sar command to capture the statistics information about
> it.Here
> >> >are my details:
> >> >
> >> >
> >> >[1]cpu(by using sar -u),we have to restart our system just as the red
> font
> >> >LINUX RESTART in the logs.
> >>
> >>
> >--
> >> >03:00:01 PM all  7.61  0.00  0.92  0.07  0.00
> >> >91.40
> >> >03:10:01 PM all  7.71  0.00  1.29  0.06  0.00
> >> >90.94
> >> >03:20:01 PM all  7.62  0.00  1.98  0.06  0.00
> >> >90.34
> >> >03:30:35 PM all  5.65  0.00 31.08  0.04  0.00
> >> >63.23
> >> >03:42:40 PM all 47.58  0.00 52.25  0.00  0.00
> >> > 0.16
> >> >Average:all  8.21  0.00  1.57  0.05  0.00
> >> >90.17
> >> >
> >> >04:42:04 PM   LINUX RESTART
> >> >
> >> >04:50:01 PM CPU %user %nice   %system   %iowait%steal
> >> >%idle
> >> >05:00:01 PM all  3.49  0.00  0.62  0.15  0.00
> >> >95.75
> >> >05:10:01 PM all  9.03  0.00  0.92  0.28  0.00
> >> >89.77
> >> >05:20:01 PM all  7.06  0.00  0.78  0.05  0.00
> >> >92.11
> >> >05:30:01 PM all  6.67  0.00  0.79  0.06  0.00
> >> >92.48
> >> >05:40:01 PM all  6.26  0.00  0.76  0.05  0.00
> >> >92.93
> >> >05:50:01 PM all  5.49  0.00  0.71  0.05  0.00
> >> >93.75
> >>
> >>
> >--
> >> >
> >> >[2]mem(by using sar -r)
> >>
> >>
> >--
> >> >03:00:01 PM   1519272 196633272 99.23361112  76364340 143574212
> >> >47.77
> >> >03:10:01 PM   1451764 196700780 99.27361196  76336340 143581608
> >> >47.77
> >> >03:20:01 PM   1453400 196699144 99.27361448  76248584 143551128
> >> >47.76
> >> >03:30:35 PM   1513844 196638700 99.24361648  76022016 143828244
> >> >47.85
> >> >03:42:40 PM   1481108 196671436 99.25361676  75718320 144478784
> >> >48.07
> >> >Average:  5051607 193100937 97.45362421  

Re: High Cpu sys usage

[YouPeng Yang]

Hi Shawn
   Here is my top screenshot:

   https://www.dropbox.com/s/jaw10mkmipz943y/topscreen.jpg?dl=0

   It is captured when my system is normal.And I have reduced the memory
size down to 48GB originating  from 64GB.


  We have two hardware clusters ,each is comprised of 3 machines,and On one
cluster we deploy 3 different SolrCloud application clusters,the above top
screenshot is the machine crached 4:30PM  yesterday.

  To be convenient,I post a top sceenshot of  another machine  of the other
cluster:

   https://www.dropbox.com/s/p3j3bpcl8l2i1nt/another64GBnodeTop.jpg?dl=0

  On this machine ,the biggest  Solrcloud node  which jvm memory size is
64GB holds 730GB index size.The machine hung up for a long time just at
yesterday middle night.

We also have capture the iotop when it hung up.

   https://www.dropbox.com/s/keqqjabmon9f1ea/anthoer64GBnodeIotop.jpg?dl=0

as the iotop shows the process jdb2  is writing large .I think it will be
helpfull.

Best Regards





2016-03-17 7:35 GMT+08:00 Shawn Heisey :

> On 3/16/2016 8:59 AM, Patrick Plaatje wrote:
> > From the sar output you supplied, it looks like you might have a memory
> issue on your hosts. The memory usage just before your crash seems to be
> *very* close to 100%. Even the slightest increase (Solr itself, or possibly
> by a system service) could caused the system crash. What are the
> specifications of your hosts and how much memory are you allocating?
>
> It's completely normal for a machine, especially a machine running Solr
> with a very large index, to run at nearly 100% memory usage.  The
> "Average" line from the sar output indicates 97.45 percent usage, but it
> also shows 81GB of memory in the "kbcached" column -- this is memory
> that can be instantly claimed by any program that asks for it.  If we
> discount this 81GB, since it is instantly available, the "true" memory
> usage is closer to 70 percent than 100.
>
> https://en.wikipedia.org/wiki/Page_cache
>
> If YouPeng can run top and sort it by memory usage (press shift-M), then
> grab a screenshot, that will be helpful for more insight.  Here's an
> example of this from one of my servers, shared on dropbox:
>
> https://www.dropbox.com/s/qfuxhw20q0y1ckx/linux-8gb-heap.png?dl=0
>
> This is a server with 64GB of RAM and 110GB of index data.  About 48GB
> of my memory is used by the disk cache.  I've got slightly less than
> half my index data in the cache.
>
> Thanks,
> Shawn
>
>

Re: High Cpu sys usage

[YouPeng Yang]

Hi
 It happened again,and worse thing is that my system went to crash.we can
even not connect to it with ssh.
 I use the sar command to capture the statistics information about it.Here
are my details:


[1]cpu(by using sar -u),we have to restart our system just as the red font
LINUX RESTART in the logs.
--
03:00:01 PM all  7.61  0.00  0.92  0.07  0.00
91.40
03:10:01 PM all  7.71  0.00  1.29  0.06  0.00
90.94
03:20:01 PM all  7.62  0.00  1.98  0.06  0.00
90.34
03:30:35 PM all  5.65  0.00 31.08  0.04  0.00
63.23
03:42:40 PM all 47.58  0.00 52.25  0.00  0.00
 0.16
Average:all  8.21  0.00  1.57  0.05  0.00
90.17

04:42:04 PM   LINUX RESTART

04:50:01 PM CPU %user %nice   %system   %iowait%steal
%idle
05:00:01 PM all  3.49  0.00  0.62  0.15  0.00
95.75
05:10:01 PM all  9.03  0.00  0.92  0.28  0.00
89.77
05:20:01 PM all  7.06  0.00  0.78  0.05  0.00
92.11
05:30:01 PM all  6.67  0.00  0.79  0.06  0.00
92.48
05:40:01 PM all  6.26  0.00  0.76  0.05  0.00
92.93
05:50:01 PM all  5.49  0.00  0.71  0.05  0.00
93.75
--

[2]mem(by using sar -r)
--
03:00:01 PM   1519272 196633272 99.23361112  76364340 143574212
47.77
03:10:01 PM   1451764 196700780 99.27361196  76336340 143581608
47.77
03:20:01 PM   1453400 196699144 99.27361448  76248584 143551128
47.76
03:30:35 PM   1513844 196638700 99.24361648  76022016 143828244
47.85
03:42:40 PM   1481108 196671436 99.25361676  75718320 144478784
48.07
Average:  5051607 193100937 97.45362421  81775777 142758861
47.50

04:42:04 PM   LINUX RESTART

04:50:01 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit
%commit
05:00:01 PM 154357132  43795412 22.10 92012  18648644 134950460
44.90
05:10:01 PM 136468244  61684300 31.13219572  31709216 134966548
44.91
05:20:01 PM 135092452  63060092 31.82221488  32162324 134949788
44.90
05:30:01 PM 133410464  64742080 32.67233848  32793848 134976828
44.91
05:40:01 PM 132022052  66130492 33.37235812  33278908 135007268
44.92
05:50:01 PM 130630408  67522136 34.08237140  33900912 135099764
44.95
Average:136996792  61155752 30.86206645  30415642 134991776
44.91
--


As the blue font parts show that my hardware crash from 03:30:35.It is hung
up until I restart it manually at 04:42:04
ALl the above information just snapshot the performance when it crashed
while there is nothing cover the reason.I have also
check the /var/log/messages and find nothing useful.

Note that I run the command- sar -v .It shows something abnormal:

02:50:01 PM  11542262  9216 76446   258
03:00:01 PM  11645526  9536 76421   258
03:10:01 PM  11748690  9216 76451   258
03:20:01 PM  11850191  9152 76331   258
03:30:35 PM  11972313 10112132625   258
03:42:40 PM  12177319 13760340227   258
Average:  8293601  8950 68187   161

04:42:04 PM   LINUX RESTART

04:50:01 PM dentunusd   file-nr  inode-nrpty-nr
05:00:01 PM 35410  7616 35223 4
05:10:01 PM137320  7296 42632 6
05:20:01 PM247010  7296 42839 9
05:30:01 PM358434  7360 42697 9
05:40:01 PM471543  7040 4292910
05:50:01 PM583787  7296 4283713


and I check the man info about the -v option :

*-v*  Report status of inode, file and other kernel tables.  The following
values are displayed:
   *dentunusd*
Number of unused cache entries in the directory cache.
*file-nr*
Number of file handles used by the system.
*inode-nr*
Number of inode handlers used by the system.
*pty-nr*
Number of pseudo-terminals used by the system.


Is the any clue about the crash? Would you please give me some suggestions?


Best Regards.


2016-03-16 14:01 GMT+08:00 YouPeng Yang :

> Hello
>The problem appears several times ,however I could not capture t

Re: High Cpu sys usage

[YouPeng Yang]

Hi
  Our system still goes down as times going.We found lots of threads are
WAITING.Here is the threaddump that I copy from the web page.And 4 pictures
for it.
  Is there any relationship with my problem?


https://www.dropbox.com/s/h3wyez091oouwck/threaddump?dl=0
https://www.dropbox.com/s/p3ctuxb3t1jgo2e/threaddump1.jpg?dl=0
https://www.dropbox.com/s/w0uy15h6z984ntw/threaddump2.jpg?dl=0
https://www.dropbox.com/s/0frskxdllxlz9ha/threaddump3.jpg?dl=0
https://www.dropbox.com/s/46ptnly1ngi9nb6/threaddump4.jpg?dl=0


Best Regards

2016-03-18 14:35 GMT+08:00 YouPeng Yang :

> Hi
>   To Patrick: Never mind .Thank you for your suggestion all the same.
>   To Otis. We do not use SPM. We monintor the JVM just use jstat becasue
> my system went well before ,so we do not need  other tools.
> But SPM is really awesome .
>
>   Still looking for help.
>
> Best Regards
>
> 2016-03-18 6:01 GMT+08:00 Patrick Plaatje :
>
>> Yeah, I did’t pay attention to the cached memory at all, my bad!
>>
>> I remember running into a similar situation a couple of years ago, one of
>> the things to investigate our memory profile was to produce a full heap
>> dump and manually analyse that using a tool like MAT.
>>
>> Cheers,
>> -patrick
>>
>>
>>
>>
>> On 17/03/2016, 21:58, "Otis Gospodnetić" 
>> wrote:
>>
>> >Hi,
>> >
>> >On Wed, Mar 16, 2016 at 10:59 AM, Patrick Plaatje 
>> >wrote:
>> >
>> >> Hi,
>> >>
>> >> From the sar output you supplied, it looks like you might have a memory
>> >> issue on your hosts. The memory usage just before your crash seems to
>> be
>> >> *very* close to 100%. Even the slightest increase (Solr itself, or
>> possibly
>> >> by a system service) could caused the system crash. What are the
>> >> specifications of your hosts and how much memory are you allocating?
>> >
>> >
>> >That's normal actually - http://www.linuxatemyram.com/
>> >
>> >You *want* Linux to be using all your memory - you paid for it :)
>> >
>> >Otis
>> >--
>> >Monitoring - Log Management - Alerting - Anomaly Detection
>> >Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>> >
>> >
>> >
>> >
>> >>
>> >
>> >
>> >>
>> >>
>> >> On 16/03/2016, 14:52, "YouPeng Yang" 
>> wrote:
>> >>
>> >> >Hi
>> >> > It happened again,and worse thing is that my system went to crash.we
>> can
>> >> >even not connect to it with ssh.
>> >> > I use the sar command to capture the statistics information about
>> it.Here
>> >> >are my details:
>> >> >
>> >> >
>> >> >[1]cpu(by using sar -u),we have to restart our system just as the red
>> font
>> >> >LINUX RESTART in the logs.
>> >>
>> >>
>> >--
>> >> >03:00:01 PM all  7.61  0.00  0.92  0.07  0.00
>> >> >91.40
>> >> >03:10:01 PM all  7.71  0.00  1.29  0.06  0.00
>> >> >90.94
>> >> >03:20:01 PM all  7.62  0.00  1.98  0.06  0.00
>> >> >90.34
>> >> >03:30:35 PM all  5.65  0.00 31.08  0.04  0.00
>> >> >63.23
>> >> >03:42:40 PM all 47.58  0.00 52.25  0.00  0.00
>> >> > 0.16
>> >> >Average:all  8.21  0.00  1.57  0.05  0.00
>> >> >90.17
>> >> >
>> >> >04:42:04 PM   LINUX RESTART
>> >> >
>> >> >04:50:01 PM CPU %user %nice   %system   %iowait%steal
>> >> >%idle
>> >> >05:00:01 PM all  3.49  0.00  0.62  0.15  0.00
>> >> >95.75
>> >> >05:10:01 PM all  9.03  0.00  0.92  0.28  0.00
>> >> >89.77
>> >> >05:20:01 PM all  7.06  0.00  0.78  0.05  0.00
>> >> >92.11
>> >> >05:30:01 PM all  6.67  0.00  0.79  0.06  0.00
>> >> >92.48
>> >> >05:40:01 PM all  6.26  0.00  0.76  0.05  0.00
>> >> >92.93
>> >> >05:50:01 PM all  5.49  0.00   

Re: High Cpu sys usage

[YouPeng Yang]

Hi Toke
  The number of collection is just 10.One of collection has 43 shards,each
shard has two replicas.We continue  importing data from oracle all the time
while our systems provide searching service.
   There are "Overlapping onDeckSearchers"  in my solr.logs. What is the
meaning about the "Overlapping onDeckSearchers" ,We set the the <
maxWarmingSearchers>20 and true.Is it right ?



Best Regard.


2016-03-29 22:31 GMT+08:00 Toke Eskildsen :

> On Tue, 2016-03-29 at 20:12 +0800, YouPeng Yang wrote:
> >   Our system still goes down as times going.We found lots of threads are
> > WAITING.Here is the threaddump that I copy from the web page.And 4
> pictures
> > for it.
> >   Is there any relationship with my problem?
>
> That is a lot of commitScheduler-threads. Do you have hundreds of
> collections in your cloud?
>
>
> Try grepping for "Overlapping onDeckSearchers" in your solr.logs to see
> if you got caught in a downwards spiral of concurrent commits.
>
> - Toke Eskildsen, State and University Library, Denmark
>
>
>

Re: High Cpu sys usage

[YouPeng Yang]

Hi
  Thanks you Erik.
   The main collection that stores our trade data is set to the softcomit
when we import data using DIH. As you guess that the softcommit intervals
is " 1000 " and we have autowarm counts to 0.However
there is some collections that store our meta info in which we commit after
each add.and these metadata collections just hold a few docs.


Best Regards


2016-03-30 12:25 GMT+08:00 Erick Erickson :

> Do not, repeat NOT try to "cure" the "Overlapping onDeckSearchers"
> by bumping this limit! What that means is that your commits
> (either hard commit with openSearcher=true or softCommit) are
> happening far too frequently and your Solr instance is trying to do
> all sorts of work that is immediately thrown away and chewing up
> lots of CPU. Perhaps this will help:
>
>
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> I'd guess that you're
>
> > commiting every second, or perhaps your indexing client is committing
> after each add. If the latter, do not do this and rely on the
> autocommit settings
> and if the formaer make those intervals as long as you can stand.
>
> > you may have your autowarm counts in your solrconfig.xml file set at
> very high numbers (let's see the filterCache settings, the queryResultCache
> settings etc.).
>
> I'd _strongly_ recommend that you put the on deck searchers back to
> 2 and figure out why you have so many overlapping searchers.
>
> Best,
> Erick
>
> On Tue, Mar 29, 2016 at 8:57 PM, YouPeng Yang 
> wrote:
> > Hi Toke
> >   The number of collection is just 10.One of collection has 43
> shards,each
> > shard has two replicas.We continue  importing data from oracle all the
> time
> > while our systems provide searching service.
> >There are "Overlapping onDeckSearchers"  in my solr.logs. What is the
> > meaning about the "Overlapping onDeckSearchers" ,We set the the <
> > maxWarmingSearchers>20 and true > useColdSearcher>.Is it right ?
> >
> >
> >
> > Best Regard.
> >
> >
> > 2016-03-29 22:31 GMT+08:00 Toke Eskildsen :
> >
> >> On Tue, 2016-03-29 at 20:12 +0800, YouPeng Yang wrote:
> >> >   Our system still goes down as times going.We found lots of threads
> are
> >> > WAITING.Here is the threaddump that I copy from the web page.And 4
> >> pictures
> >> > for it.
> >> >   Is there any relationship with my problem?
> >>
> >> That is a lot of commitScheduler-threads. Do you have hundreds of
> >> collections in your cloud?
> >>
> >>
> >> Try grepping for "Overlapping onDeckSearchers" in your solr.logs to see
> >> if you got caught in a downwards spiral of concurrent commits.
> >>
> >> - Toke Eskildsen, State and University Library, Denmark
> >>
> >>
> >>
>

Re: Questions on SolrCloud core state, when will Solr recover a "DOWN" core to "ACTIVE" core.

[YouPeng Yang]

Hi
   We have used Solr4.6 for 2 years,If you post more logs ,maybe we can
fixed it.

2016-04-21 6:50 GMT+08:00 Li Ding :

> Hi All,
>
> We are using SolrCloud 4.6.1.  We have observed following behaviors
> recently.  A Solr node in a Solrcloud cluster is up but some of the cores
> on the nodes are marked as down in Zookeeper.  If the cores are parts of a
> multi-sharded collection with one replica,  the queries to that collection
> will fail.  However, when this happened, if we issue queries to the core
> directly, it returns 200 and correct info.  But once Solr got into the
> state, the core will be marked down forever unless we do a restart on Solr.
>
> Has anyone seen this behavior before?  Is there any to get out of the state
> on its own?
>
> Thanks,
>
> Li
>

Re: maximum number of shards per SolrCloud

[YouPeng Yang]

Hi Shawn
Thanks a lot. It is  greatly  helpful.


2014-04-23 0:43 GMT+08:00 Shawn Heisey :

> On 4/22/2014 10:02 AM, yypvsxf19870706 wrote:
>
>>  I am curious of the influences when have more than 2G docs in a
>> core.And we plan to have  5g docs/core.
>>
>> Please give me some suggestion about how to plan num of docs in a
>> core ?
>>
>
> One Solr core contains one Lucene index.  It can't be divided further than
> that without a significant redesign.  Quick note: Although SolrCloud can
> handle five billion documents with no problem, you can't have five billion
> documents in a single shard/core.
>
> The only hard limitation in the entire system is that you can't have more
> than approximately 2 billion documents in a single Lucene index.  This is
> because a Java integer (which is a signed 32-bit number) is what gets used
> for internal Lucene document identifiers. Deleted documents count against
> that limit.  It is theoretically possible to overcome this limitation, but
> it would be a MAJOR change to Lucene, requiring major changes in Solr as
> well.
>
> The other limitations you can run into with a large SolrCloud are mostly a
> matter of configuration, system resources, and scaling to multiple servers.
>  They are not hard limitations in the software.
>
> I would never put more than about 1 billion documents in a single core.
>  For performance reasons, it would be a good idea to never exceed a few
> hundred million.  When a high query rate is required, loading only one Solr
> core per server may be a requirement.
>
> Thanks,
> Shawn
>
>

error when update.distrib=FROMLEADER

[YouPeng Yang]

Within Solr Cloud 4.6.0,I have a master core and indexing to it using
DIH.In the meantime,I also update indexes with solrJ.
Things go well until I create a replication of the master,Exception comes
out as [1].

When do I get this exception?


[1]---
org.apache.solr.common.SolrException: Internal Server Error request:
http://10.1.22.1:8082/solr/tv_201404_r1/update?update.distrib=FROMLEADER&distrib.from=http%3A%2F%2F10.1.22.1%3A8080%2Fsolr%2Ftv_201404%2F&wt=javabin&version=2
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:240)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
4495502413 [http-bio-8080-exec-114] INFO
org.apache.solr.update.processor.LogUpdateProcessor  – [tv_201404]
webapp=/solr path=/update params={wt=javabin&version=2}
{add=[LKLPOSLK008354580025420424103438 (1466234083906420736)]} 0 2
4495502413 [updateExecutor-1-thread-234] INFO
org.apache.solr.update.processor.DistributedUpdateProcessor  – try and ask
http://10.1.22.1:8082/solr to recover

UnsupportedOperationException when DIH in SolrCloud

[YouPeng Yang]

SolrCloud 4.6.0

  I am using SolrCloud 4.6.0 with a master and replica.I adopt the
DistributedUpdateProcessorFactory to distribute the doc between the master
and replica.
  Firstly,If the master and replica both empty,the DIH will be succeed.
  And then, DIH again,the replica always thow an
UnsupportedOperationException as [1].
  the

  I have check the src,Finally find the FunctionValues.longVal function:
  public long longVal(int doc) { throw new UnsupportedOperationException();
}
 and the implement class  StrDocValues  does not override the function.

  And ,I come a idea that,the uniqueKey of the doc that is imported is a
string FiledType. As result the StrDocValues.longVal just throw the
UnsupportedOperationException .

  If I really have the uniqueKey be string,is there a way to overwhelm this
error?

Any suggestion wiil be appreciated.











-
80711457 [http-bio-8082-exec-19] ERROR org.apache.solr.core.SolrCore  –
java.lang.UnsupportedOperationException
at
org.apache.lucene.queries.function.FunctionValues.longVal(FunctionValues.java:46)
at
org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:200)
at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:736)
at
org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:183)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:685)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:89)
at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:151)

Re: UnsupportedOperationException when DIH in SolrCloud

[YouPeng Yang]

Hi Mikhail Khludnev

   Thanks very much.
   I go through the src,and make sure that the reason about the
UnsupportedOperationException
is that the  FunctionValues be something unsupportable,like the string type
do not override the longVal of the FunctionValue class so that it throws
the exeception.
  And your hint sheds light on me : the _version_ is the key.
  Recalling that ,originally, the _version_ was set to be string,although I
had change it to type long and  reload the core,the new time _version_ do
not really take effect . The _version_ stilll kept being StrField which
make sure with some debuggings. After I have restart the web
containner(tomcat),It works(Thank godness),no UnsupportedOperationException
comes out.the DIH goes well with the DistributedUpdateProcessorFactory.

  Thanks again.


2014-04-24 19:09 GMT+08:00 yypvsxf19870706 :

> yes , i am sure？
>
> 发自我的 iPhone
>
> 在 2014-4-24，18:30，Mikhail Khludnev  写道：
>
> > Are you sure that field _version_ is declared correctly in schema.xml?
> >
> >
> > On Thu, Apr 24, 2014 at 12:30 PM, YouPeng Yang <
> yypvsxf19870...@gmail.com>wrote:
> >
> >> SolrCloud 4.6.0
> >>
> >>  I am using SolrCloud 4.6.0 with a master and replica.I adopt the
> >> DistributedUpdateProcessorFactory to distribute the doc between the
> master
> >> and replica.
> >>  Firstly,If the master and replica both empty,the DIH will be succeed.
> >>  And then, DIH again,the replica always thow an
> >> UnsupportedOperationException as [1].
> >>  the
> >>
> >>  I have check the src,Finally find the FunctionValues.longVal function:
> >>  public long longVal(int doc) { throw new
> UnsupportedOperationException();
> >> }
> >> and the implement class  StrDocValues  does not override the function.
> >>
> >>  And ,I come a idea that,the uniqueKey of the doc that is imported is a
> >> string FiledType. As result the StrDocValues.longVal just throw the
> >> UnsupportedOperationException .
> >>
> >>  If I really have the uniqueKey be string,is there a way to overwhelm
> this
> >> error?
> >>
> >> Any suggestion wiil be appreciated.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> -
> >> 80711457 [http-bio-8082-exec-19] ERROR org.apache.solr.core.SolrCore  –
> >> java.lang.UnsupportedOperationException
> >>at
> >>
> >>
> org.apache.lucene.queries.function.FunctionValues.longVal(FunctionValues.java:46)
> >>at
> >>
> >>
> org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:200)
> >>at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:736)
> >>at
> >> org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:183)
> >>at
> >>
> >>
> org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:685)
> >>at
> >>
> >>
> org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
> >>at
> >>
> >>
> org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
> >>at
> >>
> >>
> org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:89)
> >>at
> >>
> >>
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:151)
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> > Principal Engineer,
> > Grid Dynamics
> >
> > <http://www.griddynamics.com>
> > 
>

Re: DIH issues with 4.7.1

[YouPeng Yang]

Hi
   I have just compare the difference between the version 4.6.0 and 4.7.1.
Notice that the time in the getConnection function   is declared with the
System.nanoTime in 4.7.1 ,while System.currentTimeMillis().
  Curious about the resson for the change.the benefit of it .Is it
neccessory?
   I have read the SOLR-5734 ,
https://issues.apache.org/jira/browse/SOLR-5734
   Do some google about the difference of currentTimeMillis and nano,but
still can not figure out it.




2014-04-26 2:24 GMT+08:00 Shawn Heisey :

> On 4/25/2014 11:56 AM, Hutchins, Jonathan wrote:
>
>> I recently upgraded from 4.6.1 to 4.7.1 and have found that the DIH
>> process that we are using takes 4x as long to complete.  The only odd
>> thing I notice is when I enable debug logging for the dataimporthandler
>> process, it appears that in the new version each sql query is resulting in
>> a new connection opened through jdbcdatasource (log:
>> http://pastebin.com/JKh4gpmu).  Were there any changes that would affect
>> the speed of running a full import?
>>
>
> This is most likely the problem you are experiencing:
>
> https://issues.apache.org/jira/browse/SOLR-5954
>
> The fix will be in the new 4.8 version.  The release process for 4.8 is
> underway right now.  A second release candidate was required yesterday.  If
> no further problems are encountered, the release should be made around the
> middle of next week.  If problems are encountered, the release will be
> delayed.
>
> Here's something very important that has been mentioned before:  Solr 4.8
> will require Java 7.  Previously, Java 6 was required.  Java 7u55 (the
> current release from Oracle as I write this) is recommended as a minimum.
>
> If a 4.7.3 version is built, this is a fix that we should backport.
>
> Thanks,
> Shawn
>
>

Re: DIH issues with 4.7.1

[YouPeng Yang]

Hi Mark Miller
  Sorry to get you in these discussion .
  I notice that Mark Miller report this issure in
https://issues.apache.org/jira/browse/SOLR-5734 according to
https://issues.apache.org/jira/browse/SOLR-5721,but it just happened with
the zookeeper.
  If I just do DIH with JDBCDataSource ,I do not think it will get the
problem.
  Please give some hints

 >> Bonus,just post the last mail I send about the problem:
   I have just compare the difference between the version 4.6.0 and 4.7.1.
Notice that the time in the getConnection function   is declared with the
System.nanoTime in 4.7.1 ,while System.currentTimeMillis().
  Curious about the resson for the change.the benefit of it .Is it
neccessory?
   I have read the SOLR-5734 ,
https://issues.apache.org/jira/browse/SOLR-5734
   Do some google about the difference of currentTimeMillis and nano,but
still can not figure out it.

Thank you very much.


2014-04-26 20:31 GMT+08:00 YouPeng Yang :

> Hi
>I have just compare the difference between the version 4.6.0 and 4.7.1.
> Notice that the time in the getConnection function   is declared with the
> System.nanoTime in 4.7.1 ,while System.currentTimeMillis().
>   Curious about the resson for the change.the benefit of it .Is it
> neccessory?
>I have read the SOLR-5734 ,
> https://issues.apache.org/jira/browse/SOLR-5734
>Do some google about the difference of currentTimeMillis and nano,but
> still can not figure out it.
>
>
>
>
> 2014-04-26 2:24 GMT+08:00 Shawn Heisey :
>
> On 4/25/2014 11:56 AM, Hutchins, Jonathan wrote:
>>
>>> I recently upgraded from 4.6.1 to 4.7.1 and have found that the DIH
>>> process that we are using takes 4x as long to complete.  The only odd
>>> thing I notice is when I enable debug logging for the dataimporthandler
>>> process, it appears that in the new version each sql query is resulting
>>> in
>>> a new connection opened through jdbcdatasource (log:
>>> http://pastebin.com/JKh4gpmu).  Were there any changes that would affect
>>> the speed of running a full import?
>>>
>>
>> This is most likely the problem you are experiencing:
>>
>> https://issues.apache.org/jira/browse/SOLR-5954
>>
>> The fix will be in the new 4.8 version.  The release process for 4.8 is
>> underway right now.  A second release candidate was required yesterday.  If
>> no further problems are encountered, the release should be made around the
>> middle of next week.  If problems are encountered, the release will be
>> delayed.
>>
>> Here's something very important that has been mentioned before:  Solr 4.8
>> will require Java 7.  Previously, Java 6 was required.  Java 7u55 (the
>> current release from Oracle as I write this) is recommended as a minimum.
>>
>> If a 4.7.3 version is built, this is a fix that we should backport.
>>
>> Thanks,
>> Shawn
>>
>>
>

Re: DIH issues with 4.7.1

[YouPeng Yang]

Hi Rafał Kuć
  I got it,the point is  many operating systems measure time in units of
tens of milliseconds,and the  System.currentTimeMillis() is  just base on
operating system.
  In my case,I just do DIH with a crontable, Is there any possiblity to get
in that trouble?I am really can not picture what the situation may lead to
the problem.


Thanks very much.


2014-04-26 20:49 GMT+08:00 YouPeng Yang :

> Hi Mark Miller
>   Sorry to get you in these discussion .
>   I notice that Mark Miller report this issure in
> https://issues.apache.org/jira/browse/SOLR-5734 according to
> https://issues.apache.org/jira/browse/SOLR-5721,but it just happened with
> the zookeeper.
>   If I just do DIH with JDBCDataSource ,I do not think it will get the
> problem.
>   Please give some hints
>
>  >> Bonus,just post the last mail I send about the problem:
>
>I have just compare the difference between the version 4.6.0 and 4.7.1.
> Notice that the time in the getConnection function   is declared with the
> System.nanoTime in 4.7.1 ,while System.currentTimeMillis().
>   Curious about the resson for the change.the benefit of it .Is it
> neccessory?
>I have read the SOLR-5734 ,
> https://issues.apache.org/jira/browse/SOLR-5734
>Do some google about the difference of currentTimeMillis and nano,but
> still can not figure out it.
>
> Thank you very much.
>
>
> 2014-04-26 20:31 GMT+08:00 YouPeng Yang :
>
> Hi
>>I have just compare the difference between the version 4.6.0 and
>> 4.7.1. Notice that the time in the getConnection function   is declared
>> with the System.nanoTime in 4.7.1 ,while System.currentTimeMillis().
>>   Curious about the resson for the change.the benefit of it .Is it
>> neccessory?
>>I have read the SOLR-5734 ,
>> https://issues.apache.org/jira/browse/SOLR-5734
>>Do some google about the difference of currentTimeMillis and nano,but
>> still can not figure out it.
>>
>>
>>
>>
>> 2014-04-26 2:24 GMT+08:00 Shawn Heisey :
>>
>> On 4/25/2014 11:56 AM, Hutchins, Jonathan wrote:
>>>
>>>> I recently upgraded from 4.6.1 to 4.7.1 and have found that the DIH
>>>> process that we are using takes 4x as long to complete.  The only odd
>>>> thing I notice is when I enable debug logging for the dataimporthandler
>>>> process, it appears that in the new version each sql query is resulting
>>>> in
>>>> a new connection opened through jdbcdatasource (log:
>>>> http://pastebin.com/JKh4gpmu).  Were there any changes that would
>>>> affect
>>>> the speed of running a full import?
>>>>
>>>
>>> This is most likely the problem you are experiencing:
>>>
>>> https://issues.apache.org/jira/browse/SOLR-5954
>>>
>>> The fix will be in the new 4.8 version.  The release process for 4.8 is
>>> underway right now.  A second release candidate was required yesterday.  If
>>> no further problems are encountered, the release should be made around the
>>> middle of next week.  If problems are encountered, the release will be
>>> delayed.
>>>
>>> Here's something very important that has been mentioned before:  Solr
>>> 4.8 will require Java 7.  Previously, Java 6 was required.  Java 7u55 (the
>>> current release from Oracle as I write this) is recommended as a minimum.
>>>
>>> If a 4.7.3 version is built, this is a fix that we should backport.
>>>
>>> Thanks,
>>> Shawn
>>>
>>>
>>
>

How to delete docs only in a spec shard within a collection

[YouPeng Yang]

Hi
  I have a colloection with 3 shards.
  I want to delete some docs in one shard with the command:

  
http://10.1.22.1:8082/solr/tv_201402/update?&stream.body=BEGINTIME:["2014-03-01
00:00:00" TO *]&shards=tv_201402&commit=true
 As the red font exepression, It is supposed that docs only in the shard
tv_201402  whose BEGINTIME field were larger than "2014-03-01 00:00:00"
would be deleted.However ,as a result, the docs on the other shards such
astv_201403 ,tv_201404 whose BEGINTIME were also larger than 2014-03-01
00:00:00  were deleted too.
  This is not the effientlty right thing I want.

  How to delete docs only in a spec shard  within a collection which
contains several shards?

Regards.

Re: How to delete docs only in a spec shard within a collection

[YouPeng Yang]

Hi
  Anyone gives some suggestions.


Regards


2014-05-19 11:31 GMT+08:00 YouPeng Yang :

> Hi
>   I have a colloection with 3 shards.
>   I want to delete some docs in one shard with the command:
>
>   
> http://10.1.22.1:8082/solr/tv_201402/update?&stream.body=BEGINTIME:["2014-03-01
> 00:00:00" TO *]&shards=tv_201402&commit=true
>  As the red font exepression, It is supposed that docs only in the shard
> tv_201402  whose BEGINTIME field were larger than "2014-03-01 00:00:00"
> would be deleted.However ,as a result, the docs on the other shards such
> astv_201403 ,tv_201404 whose BEGINTIME were also larger than 2014-03-01
> 00:00:00  were deleted too.
>   This is not the effientlty right thing I want.
>
>   How to delete docs only in a spec shard  within a collection which
> contains several shards?
>
> Regards.
>

Re: How to delete docs only in a spec shard within a collection

[YouPeng Yang]

Hi
  I have checked the src in
DistributedUpdateProcessor.doDeleteByQuery(DeleteUpdateCommand cmd) as[1]

  According to the source code,it seems that  ShardParams._ROUTE_ (_route_)
and the ShardParams.SHARD_KEYS(shard.keys)
would be used to find the right shard.
  Compared to my last two posts,the shards is not the right properity to
identify the right shards that really need to delete the docs.


  Am I right?
  Does it need to add the processeing  about the shards to unify the
process?


[1]---
..omitted..
  String route = params.get(ShardParams._ROUTE_);
  if(route == null) route = params.get(ShardParams.SHARD_KEYS);//
deprecated . kept for backcompat
  Collection slices = coll.getRouter().getSearchSlices(route,
params, coll);
  List leaders =  new ArrayList<>(slices.size());
  for (Slice slice : slices) {
 ..omitted..
 cmdDistrib.distribDelete(cmd, leaders, outParams);
 ..omitted..
   }

..omitted..

-


Regards.


2014-05-19 15:46 GMT+08:00 YouPeng Yang :

> Hi
>   Anyone gives some suggestions.
>
>
> Regards
>
>
> 2014-05-19 11:31 GMT+08:00 YouPeng Yang :
>
> Hi
>>   I have a colloection with 3 shards.
>>   I want to delete some docs in one shard with the command:
>>
>>   
>> http://10.1.22.1:8082/solr/tv_201402/update?&stream.body=BEGINTIME:["2014-03-01
>> 00:00:00" TO *]&shards=tv_201402&commit=true
>>  As the red font exepression, It is supposed that docs only in the shard
>> tv_201402  whose BEGINTIME field were larger than "2014-03-01 00:00:00"
>> would be deleted.However ,as a result, the docs on the other shards such
>> astv_201403 ,tv_201404 whose BEGINTIME were also larger than 2014-03-01
>> 00:00:00  were deleted too.
>>   This is not the effientlty right thing I want.
>>
>>   How to delete docs only in a spec shard  within a collection which
>> contains several shards?
>>
>> Regards.
>>
>
>

Re: How to delete docs only in a spec shard within a collection

[YouPeng Yang]

Hi

  It seems that ther _route_ parameter make work as [1]. In  my last
posts,It said that  the shards should be same with the  _route_ and
shard.keys during update process to  make sure consistent.

 I will  should file a JIRA to improve that.

[1]:http://10.1.22.1:8082/solr/tv_201402/update?stream.body=BEGINTIME:["2014-03-01
00:00:00" TO "2014-03-01 00:00:10"]&_route_
=tv_201402&commit=true





2014-05-19 19:23 GMT+08:00 Jack Krupansky :

> Normally a delete-by-query needs to be distributed to all shards since it
> is not known which document keys it will affect.
>
> It sounds like SolrCloud does not support the feature you would really
> like: support for distrb=false.
>
> You can file a Jira request for a feature "improvement."
>
> -- Jack Krupansky
>
> -Original Message- From: YouPeng Yang
> Sent: Monday, May 19, 2014 3:46 AM
> To: solr-user@lucene.apache.org
> Subject: Re: How to delete docs only in a spec shard within a collection
>
>
> Hi
>  Anyone gives some suggestions.
>
>
> Regards
>
>
> 2014-05-19 11:31 GMT+08:00 YouPeng Yang :
>
>  Hi
>>   I have a colloection with 3 shards.
>>   I want to delete some docs in one shard with the command:
>>
>>
>> http://10.1.22.1:8082/solr/tv_201402/update?&stream.body=<
>> delete>BEGINTIME:["2014-03-01
>> 00:00:00" TO *]&shards=tv_201402&commit=true
>>  As the red font exepression, It is supposed that docs only in the shard
>> tv_201402  whose BEGINTIME field were larger than "2014-03-01 00:00:00"
>> would be deleted.However ,as a result, the docs on the other shards such
>> astv_201403 ,tv_201404 whose BEGINTIME were also larger than 2014-03-01
>> 00:00:00  were deleted too.
>>   This is not the effientlty right thing I want.
>>
>>   How to delete docs only in a spec shard  within a collection which
>> contains several shards?
>>
>> Regards.
>>
>>
>

the whole web instance hangs when optimize one core.

[YouPeng Yang]

Hi.

   I am using solr4.6, in one my core it contains 50 million docs,and I am
just click the optimized button on the overview page of the core,and the
whole web instance hangs,one phenomenon is the DIH on another core hanged.

  Is it a known problem or something wrong with my env?


Regards

Re: How to optimize single shard only?

[YouPeng Yang]

Hi Marcin

  Thanks to your mail,now I know why my cloud hangs when I just click the
optimize button on the overview page of the shard.


2014-05-20 15:25 GMT+08:00 Ahmet Arslan :

> Hi Marcin,
>
> just a guess, pass distrib=false ?
>
>
>
> Ahmet
>
>
> On Tuesday, May 20, 2014 10:23 AM, Marcin Rzewucki 
> wrote:
> Hi,
>
> Do you know how to optimize index on a single shard only ? I was trying to
> use "optimize=true&waitFlush=true&shard.keys=myshard" but it does not work
> - it optimizes all shards instead of just one.
>
> Kind regards.
>
>

Re: How to optimize single shard only?

[YouPeng Yang]

Hi
 Maybe you can try _router_=myshard? I will check the source code ,note you
later.


2014-05-20 17:19 GMT+08:00 YouPeng Yang :

> Hi Marcin
>
>   Thanks to your mail,now I know why my cloud hangs when I just click the
> optimize button on the overview page of the shard.
>
>
> 2014-05-20 15:25 GMT+08:00 Ahmet Arslan :
>
> Hi Marcin,
>>
>> just a guess, pass distrib=false ?
>>
>>
>>
>> Ahmet
>>
>>
>> On Tuesday, May 20, 2014 10:23 AM, Marcin Rzewucki 
>> wrote:
>> Hi,
>>
>> Do you know how to optimize index on a single shard only ? I was trying to
>> use "optimize=true&waitFlush=true&shard.keys=myshard" but it does not work
>> - it optimizes all shards instead of just one.
>>
>> Kind regards.
>>
>>
>

Re: How to optimize single shard only?

[YouPeng Yang]

Hi
  My DIH work indeed hangs, I have only four shards,each has a master and a
replica.Maybe jvm memory size is very low.it was 3G while the size of every
my core is almost 16GB.

 I also have found that the size of the master increased during the
optimization(you can check on the overview page of the core.).the
phenomenon is very werid. Is it because that the collection overall
optimization will comput and copy  all the docs of the whole collection.


Version Gen Size   Master (Searching)
1400501330248
 98396
   29.83 GB
 Master (Replicable)
1400501330888
 98397
-


  After I have check source code,unfortunatly,it seems the optimize action
distrib overall the collection.you can reference the
SolrCmdDistributor.distribCommit.


2014-05-20 17:27 GMT+08:00 Marcin Rzewucki :

> Well, it should not hang if all is configured fine :) How many shards and
> memory you have ? Note that optimize rewrites index so you might need
> additional disk space for this process. Optimizing works fine however I'd
> like to be able to do it on a single shard as well.
>
>
> On 20 May 2014 11:19, YouPeng Yang  wrote:
>
> > Hi Marcin
> >
> >   Thanks to your mail,now I know why my cloud hangs when I just click the
> > optimize button on the overview page of the shard.
> >
> >
> > 2014-05-20 15:25 GMT+08:00 Ahmet Arslan :
> >
> > > Hi Marcin,
> > >
> > > just a guess, pass distrib=false ?
> > >
> > >
> > >
> > > Ahmet
> > >
> > >
> > > On Tuesday, May 20, 2014 10:23 AM, Marcin Rzewucki <
> mrzewu...@gmail.com>
> > > wrote:
> > > Hi,
> > >
> > > Do you know how to optimize index on a single shard only ? I was trying
> > to
> > > use "optimize=true&waitFlush=true&shard.keys=myshard" but it does not
> > work
> > > - it optimizes all shards instead of just one.
> > >
> > > Kind regards.
> > >
> > >
> >
>

commit on a spec shard with SolrCloud

[YouPeng Yang]

Hi
  Doing DIH to one of shards in my SolrCloud Colleciton.I notice that every
time do ing commit in the shard,all the other shards do commit too.
  I have check the source code DistributedUpdateProcessor.processCommit ,it
said
that processCommit would extend to all the shard in the collection.
  What I want to achieve is to do dih ,commit,update(including delete) in
respective shard.
  Refer to DistributedUpdateProcessor.java: To DIH ,I need add the
shards=myshard in the request .To the update,the _route_ or shard.keys will
be use to achieve this, By the way,I have file JIRA about to make the
action consist with the shards parameter instead of  _route_ or shard.keys when
doing update(including delete) .However ,to commit,there is no parameter
used to  impose commit  only on a spec shard.
  So, I am thinking to make some changes to
DistributedUpdateProcessor.processCommit to achieve only commit to a spec
shard.
  Is there any suggestions about  what I am thinking?  Or is there any side
effect along with the change?

ping an unloaded core with a replica returns ok

[YouPeng Yang]

Hi
   As the title.I am using solr 4.6 with solrCloud. One of my leader core
within  a shard  have bean unloaded,the ping to the unloaded core and
return OK.
 Is it normal?
 How to send the right ping request to the core,and get the no ok?

weird drastic query latency during performance testing and DIH import delay after performance testing

[YouPeng Yang]

Hi
  I build my SolrCloud using Solr 4.6.0 (java version:1.7.0_45). In my
cloud,I have a collection with 30 shard,and each shard has one replica.
each core of the shard contains nearly  50 million docs  that is 15GB in
size,so does the replica.
  Before applying my cloud in the real world,I do a performance test with
JMeter 2.11.
  The scenario of the my test is simple:100 threads sending requests for 20
seconds ,and these requests are only sent to  a specific core of a specific
shard.the request is similar to the following :
 http://IP:port/solr/tv_201407/select?q=*:*&fq=BEGINTIME:["2014-06-01
00:00:00"+TO+*]+AND+(CONTACT:${user})+AND (TV_STATE:"00")&shards=tv_201407
&rows=2000&sort=BEGINTIME+desc.

  I encountered the drastic  query latency during performance testing and
DIH import delay after performance testing.Please help me. I have tested
 several times and get the same problem and can not handle it by myself.Any
suggestion will be apprecaited.

 The following steps describes what I have done .

Step 1: Before the test,the DIH import job is very fast.As the statistics
[1], the DIH importing takes only 1s for 10 docs.
[1]---
Indexing completed. Added/Updated: 10 documents. Deleted 0 documents.
(Duration: 01s)
Requests: 1 (1/s), Fetched: 10 (10/s), Skipped: 0, Processed: 10 (10/s)
Started: less than a minute ago
---

Step 2:  Then ,Doing the test under the caches are cleaned. The summery
statistics data is as [2]. Although I have clean the caches,I never think
the query latency becomes so drastic that it cannot be acceptable in my
real application.
  The red font describes the latency of the query performance test on the
core tv_201407 of the shard tv_201407 .

  So would you experts can give some hints about the drastic  query latency
?

[2]---
[solr@solr2 test]$ ../bin/jmeter.sh  -n -t solrCoudKala20140401.jmx  -l
logfile_solrCloud_20.jtl
Creating summariser 
Created the tree successfully using solrCoudKala20140401.jmx
Starting the test @ Wed Jul 16 15:59:28 CST 2014 (1405497568104)
Waiting for possible shutdown message on port 4445
aggregate +  1 in   8.1s =0.1/s Avg:  8070 Min:  8070 Max:  8070 Err:
0 (0.00%) Active: 100 Started: 100 Finished: 0
aggregate +103 in  13.4s =7.7/s Avg:  8027 Min:  4191 Max:  8434 Err:
0 (0.00%) Active: 97 Started: 100 Finished: 3
aggregate =104 in  13.4s =7.7/s Avg:  8027 Min:  4191 Max:  8434 Err:
0 (0.00%)
aggregate + 96 in 7s =   14.5/s Avg:  6160 Min:  5295 Max:  6625 Err:
0 (0.00%) Active: 0 Started: 100 Finished: 100
aggregate =200 in15s =   13.6/s Avg:  7131 Min:  4191 Max:  8434 Err:
0 (0.00%)
Tidying up ...@ Wed Jul 16 15:59:43 CST 2014 (1405497583461)
... end of run
[solr@solr2 test]$
---
Step 3:To be continued,after the test,I do the DIH importing job again
using  the same import expresion.However the performance of the DIH becomes
so unacceptable.
to import  the 10 docs takes 2 m 15 s [3]!
  Having noticing that ,solr can fetched the 10 docs fast,the processing is
slow.

[3]---
*Indexing completed. Added/Updated: 10 documents. Deleted 0 documents.
(Duration: 2m 15s)*
Requests: 1 (0/s), Fetched: 10 (0/s), Skipped: 0, Processed: 10 (0/s)
Started: about an hour ago
---

 By the way. jvm gc goes normal,and there is no long full gc during the
test. the load of my system(rhel 6.5) are also normal.

Regards

To warm the whole cache of Solr other than the only autowarmcount

[YouPeng Yang]

Hi

   I think it is wonderful to have caches autowarmed when commit or soft
commit happens. However ,If I want to warm the whole cache other than the
only autowarmcount,the default the auto warming operation will take long
long ~~long time.So it comes up with that maybe it good idea  to just
change the reference of the caches of the newsearcher with the caches of
the oldsearcher. It will increase the time of the autowarming,also increase
the query time of NRT.
  It is just not a mature idea.I am pust this idea,and hope to get more
hints or help to make more cleat about the idea.



regards

Re: To warm the whole cache of Solr other than the only autowarmcount

[YouPeng Yang]

To Matt

  Thank you,your opinion is very valuable ,So I have checked the source
codes about how the cache warming  up. It seems to just put items of the
old caches into the new caches.
  I will pull Mark Miller into this discussion.He is the one of the
developer of the Solr whom  I had  contacted with.

 To Mark Miller

   Would you please check out what we are discussing in the last two
posts.I need your help.


Regards.





2014-07-25 2:50 GMT+08:00 Matt Kuiper (Springblox) <
matt.kui...@springblox.com>:

> I don't believe this would work.  My understanding (please correct if I
> have this wrong) is that the underlying Lucene document ids have a
> potential to change and so when a newSearcher is created the caches must be
> regenerated and not copied.
>
> Matt
>
> -Original Message-
> From: YouPeng Yang [mailto:yypvsxf19870...@gmail.com]
> Sent: Thursday, July 24, 2014 10:26 AM
> To: solr-user@lucene.apache.org
> Subject: To warm the whole cache of Solr other than the only autowarmcount
>
> Hi
>
>I think it is wonderful to have caches autowarmed when commit or soft
> commit happens. However ,If I want to warm the whole cache other than the
> only autowarmcount,the default the auto warming operation will take long
> long ~~long time.So it comes up with that maybe it good idea  to just
> change the reference of the caches of the newsearcher with the caches of
> the oldsearcher. It will increase the time of the autowarming,also increase
> the query time of NRT.
>   It is just not a mature idea.I am pust this idea,and hope to get more
> hints or help to make more cleat about the idea.
>
>
>
> regards
>

Re: To warm the whole cache of Solr other than the only autowarmcount

[YouPeng Yang]

Hi Erick

We do the DIH job from the DB and committed frequently.It takes a long time
to autowarm the filterCaches after commit or soft commit  happened when
setting the autowarmcount=1024,which I do think is small enough.
So It comes up an idea that whether it  could  directly pass the reference
of the caches   over to the new caches so that the autowarm processing will
take much fewer time .



2014-07-28 2:30 GMT+08:00 Erick Erickson :

> Why do you think you _need_ to autowarm the entire cache? It
> is, after all, an LRU cache, the theory being that the most recent
> queries are most likely to be reused.
>
> Personally I'd run some tests on using small autowarm counts
> before getting at all mixed up in some complex scheme that
> may not be useful at all. Say an autowarm count of 16. Then
> measure using that, then say 32 then... Insure you have a real
> problem before worrying about a solution! ;)
>
> Best,
> Erick
>
>
> On Fri, Jul 25, 2014 at 6:45 AM, Shawn Heisey  wrote:
>
> > On 7/24/2014 8:45 PM, YouPeng Yang wrote:
> > > To Matt
> > >
> > >   Thank you,your opinion is very valuable ,So I have checked the source
> > > codes about how the cache warming  up. It seems to just put items of
> the
> > > old caches into the new caches.
> > >   I will pull Mark Miller into this discussion.He is the one of the
> > > developer of the Solr whom  I had  contacted with.
> > >
> > >  To Mark Miller
> > >
> > >Would you please check out what we are discussing in the last two
> > > posts.I need your help.
> >
> > Matt is completely right.  Any commit can drastically change the Lucene
> > document id numbers.  It would be too expensive to determine which
> > numbers haven't changed.  That means Solr must throw away all cache
> > information on commit.
> >
> > Two of Solr's caches support autowarming.  Those caches use queries as
> > keys and results as values.  Autowarming works by re-executing the top N
> > queries (keys) in the old cache to obtain fresh Lucene document id
> > numbers (values).  The cache code does take *keys* from the old cache
> > for the new cache, but not *values*.  I'm very sure about this, as I
> > wrote the current (and not terribly good) LFUCache.
> >
> > Thanks,
> > Shawn
> >
> >
>

Re: To warm the whole cache of Solr other than the only autowarmcount

[YouPeng Yang]

Hi Shawn
  No affense to your work,I am still confusing about the cache warm
processing about your explanation.So I check the warm method of
FastLRUCache as [1].
  As far as I see,there is no values refresh during the the warm
processing. the  *regenerator.regenerateItem* just put the old value to the
new cache.

 Did I miss anything?

[1]--
  public void warm(SolrIndexSearcher searcher, SolrCache old) {
if (regenerator == null) return;
long warmingStartTime = System.nanoTime();
FastLRUCache other = (FastLRUCache) old;
// warm entries
if (isAutowarmingOn()) {
  int sz = autowarm.getWarmCount(other.size());
  Map items = other.cache.getLatestAccessedItems(sz);
  Map.Entry[] itemsArr = new Map.Entry[items.size()];
  int counter = 0;
  for (Object mapEntry : items.entrySet()) {
itemsArr[counter++] = (Map.Entry) mapEntry;
  }
  for (int i = itemsArr.length - 1; i >= 0; i--) {
try {
  boolean continueRegen = regenerator.regenerateItem(searcher,
  this, old, itemsArr[i].getKey(), itemsArr[i].getValue());
  if (!continueRegen) break;
}
catch (Exception e) {
  SolrException.log(log, "Error during auto-warming of key:" +
itemsArr[i].getKey(), e);
}
  }
}
warmupTime = TimeUnit.MILLISECONDS.convert(System.nanoTime() -
warmingStartTime, TimeUnit.NANOSECONDS);
  }


2014-07-25 21:45 GMT+08:00 Shawn Heisey :

> On 7/24/2014 8:45 PM, YouPeng Yang wrote:
> > To Matt
> >
> >   Thank you,your opinion is very valuable ,So I have checked the source
> > codes about how the cache warming  up. It seems to just put items of the
> > old caches into the new caches.
> >   I will pull Mark Miller into this discussion.He is the one of the
> > developer of the Solr whom  I had  contacted with.
> >
> >  To Mark Miller
> >
> >Would you please check out what we are discussing in the last two
> > posts.I need your help.
>
> Matt is completely right.  Any commit can drastically change the Lucene
> document id numbers.  It would be too expensive to determine which
> numbers haven't changed.  That means Solr must throw away all cache
> information on commit.
>
> Two of Solr's caches support autowarming.  Those caches use queries as
> keys and results as values.  Autowarming works by re-executing the top N
> queries (keys) in the old cache to obtain fresh Lucene document id
> numbers (values).  The cache code does take *keys* from the old cache
> for the new cache, but not *values*.  I'm very sure about this, as I
> wrote the current (and not terribly good) LFUCache.
>
> Thanks,
> Shawn
>
>

Class name of parsing the fq clause

[YouPeng Yang]

Hi
   I search the solr with fq clause,which is like:
   fq=BEGINTIME:[2013-08-25T16:00:00Z TO *] AND BUSID:(M3 OR M9)


   I am curious about the parsing process . I want to study it.
   What is the Java file name describes  the parsing  process of the fq
clause.


  Thanks

Regards.

Re: Class name of parsing the fq clause

[YouPeng Yang]

HI Jack

  Thanks a lot for your explanation.


2013/10/21 Jack Krupansky 

> Start with org.apache.solr.handler.**component.QueryComponent#**prepare
> which fetches the fq parameters and indirectly invokes the query parser(s):
>
> String[] fqs = req.getParams().getParams(**CommonParams.FQ);
> if (fqs!=null && fqs.length!=0) {
>   List filters = rb.getFilters();
>   // if filters already exists, make a copy instead of modifying the
> original
>   filters = filters == null ? new ArrayList(fqs.length) : new
> ArrayList(filters);
>   for (String fq : fqs) {
> if (fq != null && fq.trim().length()!=0) {
>   QParser fqp = QParser.getParser(fq, null, req);
>   filters.add(fqp.getQuery());
> }
>   }
>   // only set the filters if they are not empty otherwise
>   // fq=&someotherParam= will trigger all docs filter for every request
>   // if filter cache is disabled
>   if (!filters.isEmpty()) {
> rb.setFilters( filters );
>
> Note that this line actually invokes the parser:
>
>   filters.add(fqp.getQuery());
>
> Then in org.apache.lucene.search.**Query.QParser#getParser:
>
> QParserPlugin qplug = req.getCore().getQueryPlugin(**parserName);
> QParser parser =  qplug.createParser(qstr, localParams, req.getParams(),
> req);
>
> And for the common case of the Lucene query parser, org.apache.solr.search.
> **LuceneQParserPlugin#**createParser:
>
> public QParser createParser(String qstr, SolrParams localParams,
> SolrParams params, SolrQueryRequest req) {
>  return new LuceneQParser(qstr, localParams, params, req);
> }
>
> And then in org.apache.lucene.search.**Query.QParser#getQuery:
>
> public Query getQuery() throws SyntaxError {
>  if (query==null) {
>query=parse();
>
> And then in org.apache.lucene.search.**Query.LuceneQParser#parse:
>
> lparser = new SolrQueryParser(this, defaultField);
>
> lparser.setDefaultOperator
>  (QueryParsing.**getQueryParserDefaultOperator(**getReq().getSchema(),
>  getParam(QueryParsing.OP)));
>
> return lparser.parse(qstr);
>
> And then in org.apache.solr.parser.**SolrQueryParserBase#parse:
>
> Query res = TopLevelQuery(null);  // pass null so we can tell later if an
> explicit field was provided or not
>
> And then in org.apache.solr.parser.**QueryParser#TopLevelQuery, the
> parsing begins.
>
> And org.apache.solr.parser.**QueryParser.jj is the grammar for a basic
> Solr/Lucene query, and org.apache.solr.parser.**QueryParser.java is
> generated by JFlex, and a lot of the logic is in the base class of the
> generated class, org.apache.solr.parser.**SolrQueryParserBase.java.
>
> Good luck! Happy hunting!
>
> -- Jack Krupansky
>
> -Original Message- From: YouPeng Yang
> Sent: Monday, October 21, 2013 2:57 AM
> To: solr-user@lucene.apache.org
> Subject: Class name of parsing the fq clause
>
>
> Hi
>   I search the solr with fq clause,which is like:
>   fq=BEGINTIME:[2013-08-25T16:**00:00Z TO *] AND BUSID:(M3 OR M9)
>
>
>   I am curious about the parsing process . I want to study it.
>   What is the Java file name describes  the parsing  process of the fq
> clause.
>
>
>  Thanks
>
> Regards.
>

deleteByQuery does not work with SolrCloud

[YouPeng Yang]

Hi
  I am using SolrCloud withing solr 4.4 ,and I try the SolrJ API
deleteByQuery to delete the Index as :
 CloudSolrServer cloudServer = new CloudSolrServer(myZKhost)
 cloudServer.connect()
 cloudServer.setDefaultCollection
 cloudServer.deleteByQuery("indexname:shardTv_20131010");
 cloudServer.commit();

It seems not to work.
I also have do some google,unfortunately there is no help.

Do I miss anything?


Thanks

Regard

Re: deleteByQuery does not work with SolrCloud

[YouPeng Yang]

Hi Erick

  It can get hits on  this documents.
  And I try this :

myhost/solr/mycore/update?stream.body=name:shardTv_20131010&commit=true
  the document could be deleted.


Regards


2013/10/23 Erick Erickson 

> The first thing I'd do is go in to the browser UI and make sure you can get
> hits on documents, something like
> blah/collection/q=indexname:shardTv_20131010
>
> Best,
> Erick
>
>
> On Wed, Oct 23, 2013 at 8:20 AM, YouPeng Yang  >wrote:
>
> > Hi
> >   I am using SolrCloud withing solr 4.4 ,and I try the SolrJ API
> > deleteByQuery to delete the Index as :
> >  CloudSolrServer cloudServer = new CloudSolrServer(myZKhost)
> >  cloudServer.connect()
> >  cloudServer.setDefaultCollection
> >  cloudServer.deleteByQuery("indexname:shardTv_20131010");
> >  cloudServer.commit();
> >
> > It seems not to work.
> > I also have do some google,unfortunately there is no help.
> >
> > Do I miss anything?
> >
> >
> > Thanks
> >
> > Regard
> >
>

how to increase each index file size

[YouPeng Yang]

Hi
  I'm using the SolrCloud integreted with HDFS,I found there are lots of
small size files.
  So,I'd like to increase  the index  file size  while doing DIH
full-import. Any suggestion to achieve this goal.


Regards.

Re: how to increase each index file size

[YouPeng Yang]

Hi Erick

   Thanks for your reply.


Regards


2013/12/4 Erick Erickson 

> Why do you want to do this? Are you seeing performance problems?
> If not, I'd just ignore this problem, premature optimization and all that.
>
> If you _really_ want to do this, your segments files are closed every
> time you to a commit, opensearcher=true|false doesn't matter.
>
> BUT, the longer these are the bigger your transaction log will be,
> which may lead to other issues, particularly on restart. See:
>
> http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> The key is the section on truncating the tlog.
>
> And note the sizes of these segments will change as they're
> merged anyway.
>
> Best,
> Erick
>
>
> On Wed, Dec 4, 2013 at 4:42 AM, YouPeng Yang  >wrote:
>
> > Hi
> >   I'm using the SolrCloud integreted with HDFS,I found there are lots of
> > small size files.
> >   So,I'd like to increase  the index  file size  while doing DIH
> > full-import. Any suggestion to achieve this goal.
> >
> >
> > Regards.
> >
>

Cannot create core

[YouPeng Yang]

Hi

 I get an weird problem.
 I try to create a core within Solr4.6.

Firstly, on my solr web server tomcat, a log come out[1]:
Then lots of Overseer  Info  logs come out as [2]. And then the creation
failed.

I have also notice that there is a lot of qn in the Overseer  on the
zookeeper:

[zk: localhost:2181(CONNECTED) 45] ls /overseer/queue
[qn-38, qn-37, qn-39, qn-34, qn-33,
qn-36, qn-44, qn-35, qn-42, qn-43,
qn-30, qn-31, qn-40, qn-32, qn-41]


Why are there so many qn.? I do not know the reason to the core creation
failed.
Please help .



[1]
..
293699 [http-bio-8081-exec-1] INFO  org.apache.solr.cloud.ZkController  –
publishing core=reportCore_201310 state=down
293700 [http-bio-8081-exec-1] INFO  org.apache.solr.cloud.ZkController  –
numShards not found on descriptor - reading it from system property
293704 [http-bio-8081-exec-1] INFO  org.apache.solr.cloud.ZkController  –
waiting to find shard id in clusterstate for reportCore_201310


[2]---
136661 [Thread-2] INFO  org.apache.solr.common.cloud.ZkStateReader  –
Updating cloud state from ZooKeeper...
136663 [Thread-2] INFO  org.apache.solr.cloud.Overseer  – Update state
numShards=null message={
  "operation":"state",
  "state":"down",
  "base_url":"http://10.1.22.1:8080/solr";,
  "core":"reportCore_201310",
  "roles":null,
  "node_name":"10.1.22.1:8080_solr",
  "shard":"reportCore_201310",
  "shard_range":null,
  "shard_state":"active",
  "shard_parent":null,
  "collection":"repCore",
  "numShards":null,
  "core_node_name":"reportCore_201310"}
136663 [Thread-2] ERROR org.apache.solr.cloud.Overseer  – Exception in
Overseer main queue loop
java.lang.NullPointerException
at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.updateState(Overseer.java:360)
at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:204)
at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:166)
at java.lang.Thread.run(Thread.java:662)

Re: Cannot create core

[YouPeng Yang]

Hi
  I have to add some necessary information.
  I failed to  create a core by using :
http://10.1.22.1:8081/solr/admin/cores?action=CREATE&name=reportCore_201309&shard=reportCore_201309&collection.configName=myconf&schema=rep_core_schema.xml&config=solrconfigHDFS_report_his.xml&collection=repCore&;
coreNodeName=reportCore_201309

 However succeed to create a core by using:


http://10.1.22.1:8081/solr/admin/cores?action=CREATE&name=reportCore_201309&shard=reportCore_201309&collection.configName=myconf&schema=rep_core_schema.xml&config=solrconfigHDFS_report_his.xml&collection=repCore


I just to upgrade to solr 4.6, the coreNodeName  goes well in solr 4.4 .

please help.

Regards


2013/12/17 YouPeng Yang 

> Hi
>
>  I get an weird problem.
>  I try to create a core within Solr4.6.
>
> Firstly, on my solr web server tomcat, a log come out[1]:
> Then lots of Overseer  Info  logs come out as [2]. And then the creation
> failed.
>
> I have also notice that there is a lot of qn in the Overseer  on the
> zookeeper:
>
> [zk: localhost:2181(CONNECTED) 45] ls /overseer/queue
> [qn-38, qn-37, qn-39, qn-34,
> qn-33, qn-36, qn-44, qn-35, qn-42,
> qn-43, qn-30, qn-31, qn-40, qn-32,
> qn-41]
>
>
> Why are there so many qn.? I do not know the reason to the core creation
> failed.
> Please help .
>
>
>
> [1]
> ..
> 293699 [http-bio-8081-exec-1] INFO  org.apache.solr.cloud.ZkController  –
> publishing core=reportCore_201310 state=down
> 293700 [http-bio-8081-exec-1] INFO  org.apache.solr.cloud.ZkController  –
> numShards not found on descriptor - reading it from system property
> 293704 [http-bio-8081-exec-1] INFO  org.apache.solr.cloud.ZkController  –
> waiting to find shard id in clusterstate for reportCore_201310
>
>
>
> [2]---
> 136661 [Thread-2] INFO  org.apache.solr.common.cloud.ZkStateReader  –
> Updating cloud state from ZooKeeper...
> 136663 [Thread-2] INFO  org.apache.solr.cloud.Overseer  – Update state
> numShards=null message={
>   "operation":"state",
>   "state":"down",
>   "base_url":"http://10.1.22.1:8080/solr";,
>   "core":"reportCore_201310",
>   "roles":null,
>   "node_name":"10.1.22.1:8080_solr",
>   "shard":"reportCore_201310",
>   "shard_range":null,
>   "shard_state":"active",
>   "shard_parent":null,
>   "collection":"repCore",
>   "numShards":null,
>   "core_node_name":"reportCore_201310"}
> 136663 [Thread-2] ERROR org.apache.solr.cloud.Overseer  – Exception in
> Overseer main queue loop
> java.lang.NullPointerException
> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.updateState(Overseer.java:360)
> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:204)
> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:166)
> at java.lang.Thread.run(Thread.java:662)
>
>
>

Re: Cannot create core

[YouPeng Yang]

Hi
   Any one help me?

  I can only set coreNodeName  mannully to one core in the same
collection.It fails when I add a new core to the existed collection.

  please,please




2013/12/18 YouPeng Yang 

> Hi
>   I have to add some necessary information.
>   I failed to  create a core by using :
>
> http://10.1.22.1:8081/solr/admin/cores?action=CREATE&name=reportCore_201309&shard=reportCore_201309&collection.configName=myconf&schema=rep_core_schema.xml&config=solrconfigHDFS_report_his.xml&collection=repCore&;
> coreNodeName=reportCore_201309
>
>  However succeed to create a core by using:
>
>
> http://10.1.22.1:8081/solr/admin/cores?action=CREATE&name=reportCore_201309&shard=reportCore_201309&collection.configName=myconf&schema=rep_core_schema.xml&config=solrconfigHDFS_report_his.xml&collection=repCore
>
>
> I just to upgrade to solr 4.6, the coreNodeName  goes well in solr 4.4 .
>
> please help.
>
> Regards
>
>
> 2013/12/17 YouPeng Yang 
>
>> Hi
>>
>>  I get an weird problem.
>>  I try to create a core within Solr4.6.
>>
>> Firstly, on my solr web server tomcat, a log come out[1]:
>> Then lots of Overseer  Info  logs come out as [2]. And then the creation
>> failed.
>>
>> I have also notice that there is a lot of qn in the Overseer  on the
>> zookeeper:
>>
>> [zk: localhost:2181(CONNECTED) 45] ls /overseer/queue
>> [qn-38, qn-37, qn-39, qn-34,
>> qn-33, qn-36, qn-44, qn-35, qn-42,
>> qn-43, qn-30, qn-31, qn-40, qn-32,
>> qn-41]
>>
>>
>> Why are there so many qn.? I do not know the reason to the core creation
>> failed.
>> Please help .
>>
>>
>>
>> [1]
>> ..
>> 293699 [http-bio-8081-exec-1] INFO  org.apache.solr.cloud.ZkController  –
>> publishing core=reportCore_201310 state=down
>> 293700 [http-bio-8081-exec-1] INFO  org.apache.solr.cloud.ZkController  –
>> numShards not found on descriptor - reading it from system property
>> 293704 [http-bio-8081-exec-1] INFO  org.apache.solr.cloud.ZkController  –
>> waiting to find shard id in clusterstate for reportCore_201310
>>
>>
>>
>> [2]---
>> 136661 [Thread-2] INFO  org.apache.solr.common.cloud.ZkStateReader  –
>> Updating cloud state from ZooKeeper...
>> 136663 [Thread-2] INFO  org.apache.solr.cloud.Overseer  – Update state
>> numShards=null message={
>>   "operation":"state",
>>   "state":"down",
>>   "base_url":"http://10.1.22.1:8080/solr";,
>>   "core":"reportCore_201310",
>>   "roles":null,
>>   "node_name":"10.1.22.1:8080_solr",
>>   "shard":"reportCore_201310",
>>   "shard_range":null,
>>   "shard_state":"active",
>>   "shard_parent":null,
>>   "collection":"repCore",
>>   "numShards":null,
>>   "core_node_name":"reportCore_201310"}
>> 136663 [Thread-2] ERROR org.apache.solr.cloud.Overseer  – Exception in
>> Overseer main queue loop
>> java.lang.NullPointerException
>> at
>> org.apache.solr.cloud.Overseer$ClusterStateUpdater.updateState(Overseer.java:360)
>> at
>> org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:204)
>> at
>> org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:166)
>> at java.lang.Thread.run(Thread.java:662)
>>
>>
>>
>

update doc with a xml-format string

[YouPeng Yang]

Hi solr users

  I have a  string field to store a xml string. Now I want to update the
field.I use the command.

http://10.7.23.122:8080/solr/meta_core/update?stream.body=shardTv_20131031"REP_DATE>20130930
and REP_DATE<20131003
"&commit=true

The red color string is what I want to update .However it give out an error:


 
  400
  3


 unexpected XML tag doc/root
400



Is there any way to acheive my requirement?


Any suggestion will be appreciated.

Regards

Re: update doc with a xml-format string

[YouPeng Yang]

Hi
   Thanks for your reply.

   The   is actually what I want to update the doc. That
is I intend to  update the xml string to one of the fields of my doc.
The url I have not found  I want.
   Any way,thanks a lot.

Regards.


2013/12/20 Gora Mohanty 

> On 20 December 2013 13:57, YouPeng Yang  wrote:
> > Hi solr users
> >
> >   I have a  string field to store a xml string. Now I want to update the
> > field.I use the command.
> >
> > http://10.7.23.122:8080/solr/meta_core/update?stream.body=
>  > name="name">shardTv_20131031 > name="jobFirstRunExpress" >" > type=\"string\" isChangeable=\"true\"
> >>REP_DATE>20130930
> > and 
> >>REP_DATE<20131003
> > "&commit=true
> >
> > The red color string is what I want to update .However it give out an
> error:
>
> Um, the error message is pretty clear. Your XML is not in the format
> that Solr expects: You have a  tag that Solr does not understand.
> Please see http://wiki.apache.org/solr/UpdateXmlMessages for the
> required XML format.
>
> Regards,
> Gora
>

Re: update doc with a xml-format string

[YouPeng Yang]

Hi
  thanks.
  I add the CDATA like :


update?stream.body=shardTv_20131031&commit=true

 and the field  jobFirstRunExpress in the doc just like :

jobFirstRunExpress":"REP_DATE>20130930 and REP_DATE<20131003"

 it seems  the xml tags were omitted.






2013/12/20 Gora Mohanty 

> On 20 December 2013 14:18, YouPeng Yang  wrote:
> > Hi
> >Thanks for your reply.
> >
> >The   is actually what I want to update the doc. That
> > is I intend to  update the xml string to one of the fields of my doc.
> [...]
>
> Ah, sorry, I missed that. As others have followed up, you need
> to escape the XML that is within that field.
>
> Regards,
> Gora
>

Re: update doc with a xml-format string

[YouPeng Yang]

Hi Daniel

The CDATA block works. the raw data is actually what I want.
Thanks a lot

Regards



2013/12/20 yypvsxf19870706 

> Your suggestions light me. The type is just string.
> I will try the left advices .
> Thanks a lot.
>
> 发自我的 iPhone
>
> 在 2013-12-20，19:09，Daniel Collins  写道：
>
> > What's the schema definition for that field?  Are you stripping HTML in
> > your analyzer chain? Can you run it through the analyzer screen in the
> > admin UI to confirm that the raw data goes through as you expect?
> > Can you add a document via the admin UI and see that the data in the
> index
> > is correct? Have you ever had valid data in that field?
> >
> > That would already confirm whether the problem is in your index
> definition
> > or your indexing code.
> >
> > You still haven't given us much to go on...
> >
> >
> > On 20 December 2013 10:57, YouPeng Yang 
> wrote:
> >
> >> Hi
> >>  thanks.
> >>  I add the CDATA like :
> >>
> >>
> >> update?stream.body= >> name="name">shardTv_20131031 >> name="jobFirstRunExpress"
> >&commit=true
> >>
> >> and the field  jobFirstRunExpress in the doc just like :
> >>
> >> jobFirstRunExpress":"REP_DATE>20130930 and REP_DATE<20131003"
> >>
> >> it seems  the xml tags were omitted.
> >>
> >>
> >>
> >>
> >>
> >>
> >> 2013/12/20 Gora Mohanty 
> >>
> >>> On 20 December 2013 14:18, YouPeng Yang 
> >> wrote:
> >>>> Hi
> >>>>   Thanks for your reply.
> >>>>
> >>>>   The   is actually what I want to update the doc.
> >> That
> >>>> is I intend to  update the xml string to one of the fields of my doc.
> >>> [...]
> >>>
> >>> Ah, sorry, I missed that. As others have followed up, you need
> >>> to escape the XML that is within that field.
> >>>
> >>> Regards,
> >>> Gora
> >>
>

Re: update doc with a xml-format string

[YouPeng Yang]

Hi Daniel

The CDATA block works. the raw data is actually what I want.
Thanks a lot

Regards



2013/12/20 yypvsxf19870706 

> Your suggestions light me. The type is just string.
> I will try the left advices .
> Thanks a lot.
>
> 发自我的 iPhone
>
> 在 2013-12-20，19:09，Daniel Collins  写道：
>
> > What's the schema definition for that field?  Are you stripping HTML in
> > your analyzer chain? Can you run it through the analyzer screen in the
> > admin UI to confirm that the raw data goes through as you expect?
> > Can you add a document via the admin UI and see that the data in the
> index
> > is correct? Have you ever had valid data in that field?
> >
> > That would already confirm whether the problem is in your index
> definition
> > or your indexing code.
> >
> > You still haven't given us much to go on...
> >
> >
> > On 20 December 2013 10:57, YouPeng Yang 
> wrote:
> >
> >> Hi
> >>  thanks.
> >>  I add the CDATA like :
> >>
> >>
> >> update?stream.body= >> name="name">shardTv_20131031 >> name="jobFirstRunExpress"
> >&commit=true
> >>
> >> and the field  jobFirstRunExpress in the doc just like :
> >>
> >> jobFirstRunExpress":"REP_DATE>20130930 and REP_DATE<20131003"
> >>
> >> it seems  the xml tags were omitted.
> >>
> >>
> >>
> >>
> >>
> >>
> >> 2013/12/20 Gora Mohanty 
> >>
> >>> On 20 December 2013 14:18, YouPeng Yang 
> >> wrote:
> >>>> Hi
> >>>>   Thanks for your reply.
> >>>>
> >>>>   The   is actually what I want to update the doc.
> >> That
> >>>> is I intend to  update the xml string to one of the fields of my doc.
> >>> [...]
> >>>
> >>> Ah, sorry, I missed that. As others have followed up, you need
> >>> to escape the XML that is within that field.
> >>>
> >>> Regards,
> >>> Gora
> >>
>

Werid exception: SolrCore 'collection1' is not available due to init failure

[YouPeng Yang]

Hi users

 I get a very werid problem within solr 4.6
 I  just  want to reload a core :
http://10.7.23.125:8080/solr/admin/cores?action=RELOAD&core=reportCore_201210_r1

 However it give out an exception[1].As the exception the SolrCore
'collection1' does not exist. I create a default core not with the name
'collection1'.

 I am not clear about the exception.And  I thought that the exception
should not come out. Does it need to make an addtional setup avoid this
exception?


[1]-
1016924 [http-bio-8080-exec-10] ERROR
org.apache.solr.servlet.SolrDispatchFilter
?.null:org.apache.solr.common.SolrException: SolrCore 'collection1' is not
available due to init failure: No such core: collection1
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:818)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.solr.common.SolrException: No such core: collection1
at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:675)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:717)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:178)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:662)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
... 16 more

Questions about integrateing SolrCloud with HDFS

[YouPeng Yang]

Hi users

 Solr supports for writing and reading its index and transaction log files
to the HDFS distributed filesystem.
  **I am curious about that there are any other futher improvement about
the integration with HDFS.*
  **For the solr  native replication  will make multiple copies  of the
master node's index. Because of the native replication of HDFS,there is no
need to do that.It just to need that multiple cores in solrcloud share the
same index directory in HDFS?*


   The above supposition is what I want to achive when we are integrating
SolrCloud with HDFS (Solr 4.6).
   To make sure of our application high available,we still have  to take
the solr   replication with   some tricks.

   Firstly ,noting that  solr's index directory is made up of
*collectionName/coreNodeName/data/index *

*collectionName/coreNodeName/data/tlog*
   So to achive this,we want to create multi cores that use the same  hdfs
index directory .

  I have tested this  within solr 4.4 by expilcitly indicating  the same
coreNodeName.

  For example:
  Step1, a core was created with the name=core1 and shard=core_shard1 and
collection=clollection1 and coreNodeName=*core1*
  Step2. create another core  with the name=core2 and shard=core_shard1 and
collection=clollection1 and coreNodeName=
*core1*
*  T*he two core share the same shard ,collection and coreNodeName.As a
result,the two core will get the same index data which is stored in the
hdfs directory :
  hdfs://myhdfs/*clollection1*/*core1*/data/index
  hdfs://myhdfs/*clollection1*/*core1*/data/tlog

  Unfortunately*, *as the solr 4.6 was released,we upgraded . the above
goal failed. We could not create a core with both expilcit shard and
coreNodeName.
Exceptions are as [1].
*  Can some give some help?*


Regards
[1]--
64893635 [http-bio-8080-exec-1] INFO  org.apache.solr.cloud.ZkController
?.publishing core=hdfstest3 state=down
64893635 [http-bio-8080-exec-1] INFO  org.apache.solr.cloud.ZkController
?.numShards not found on descriptor - reading it from system property
64893698 [http-bio-8080-exec-1] INFO  org.apache.solr.cloud.ZkController
?.look for our core node name



64951227 [http-bio-8080-exec-17] INFO  org.apache.solr.core.SolrCore
?.[reportCore_201208] webapp=/solr path=/replication
params={slave=false&command=details&wt=javabin&qt=/replication&version=2}
status=0 QTime=107


65213770 [http-bio-8080-exec-1] INFO  org.apache.solr.cloud.ZkController
?.waiting to find shard id in clusterstate for hdfstest3
65533894 [http-bio-8080-exec-1] ERROR org.apache.solr.core.SolrCore
?.org.apache.solr.common.SolrException: Error CREATEing SolrCore
'hdfstest3': Could not get shard id for core: hdfstest3
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:535)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:152)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:662)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.solr.common.SolrException: Could not get shard id for
core: hdfstest3
at
org.apache.solr.cloud.ZkController.waitForShardId(ZkController.java:1302)
at
org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1248)
at
org.apache.solr.cloud.ZkController.preRegister(ZkControll

Cannot creat a core with both shard and coreNodeName

[YouPeng Yang]

Hi
  I am using  solr.4.6. when I create a core with the request:
  http://10.7.23.125:8081/solr/admin/cores?action=CREATE&schema=schema.xml&;
*shard*=reportCore_201202_b0&*coreNodeName*
=core_node1&collection.configName=myconf&name=reportCore_201202_b0&action=CREATE&config=solrconfig2.xml.bak&collection=repCore&node_name=10.7.23.125:8081
_solr

  As above ,I have explicitely indicate the *shard and *

*coreNodeName.*
  However after waiting for a 320s,it give out the exception [1].
By the way,if  assign either  *shard *or *coreNodeName , *it goes success.

  I have posted this problem for several time.but did not get  anyone'
attention.
 I really need some help on the problem.Please help me.

  Why could I not indicate the *shard and **coreNodeName.*

Regards.

[1]---
13971123 [http-bio-8081-exec-14] ERROR org.apache.solr.core.SolrCore
?.org.apache.solr.common.SolrException: Error CREATEing SolrCore
'reportCore_201202_b0': Could not get shard id for core:
reportCore_201202_b0
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:535)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:152)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:662)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.solr.common.SolrException: Could not get shard id for
core: reportCore_201202_b0
at
org.apache.solr.cloud.ZkController.waitForShardId(ZkController.java:1302)
at
org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1244)
at
org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1396)
at
org.apache.solr.core.CoreContainer.preRegisterInZk(CoreContainer.java:942)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:492)
... 20 more

Maybe a bug for solr 4.6 when create a new core

[YouPeng Yang]

Hi
   Merry Christmas.

   Before this mail,I am in trouble with a weird problem  for a few days
when to create a new core with both explicite shard and coreNodeName. And I
have posted a few mails  in the mailist,no one ever gives any
suggestions,maybe  they did not  encounter the same problem.
  I have to go through the srcs to check out the reason. Thanks god, I find
it. The reason to the problem,maybe be a bug, so I would like to report it
hoping to get your endorsement and confirmation.


In class org.apache.solr.cloud.Overseer the Line 360:
-
  if (sliceName !=null && collectionExists &&
!"true".equals(state.getCollection(collection).getStr("autoCreated"))) {
Slice slice = state.getSlice(collection, sliceName);
if (slice.getReplica(coreNodeName) == null) {
  log.info("core_deleted . Just return");
  return state;
}
  }
-
the slice needs to be checked null .because I create a new core with both
explicite shard and coreNodeName, the state.getSlice(collection,
sliceName)  may return a null.So it needs to be checked ,or there will be
an NullpointException
-
  if (sliceName !=null && collectionExists &&
!"true".equals(state.getCollection(collection).getStr("autoCreated"))) {
Slice slice = state.getSlice(collection, sliceName);
if (*slice != null &&* slice.getReplica(coreNodeName) == null) {
  log.info("core_deleted . Just return");
  return state;
}
  }
-

 *Querstion 1*: Is this OK with the whole solr project,I have no aware
about the influences about the change,as right now ,it goes right. Please
make confirm about this.

  After I fixed this prolem,I can create a core with the request:
http://10.7.23.122:8080/solr/admin/cores?action=CREATE&name=Test&;
*shard=Test*
&collection.configName=myconf&schema=schema.xml&config=solrconfigLocal_default.xml&collection=defaultcol&
*coreNodeName=Test*

  However when I create a replica within the same shard Test:
http://10.7.23.122:8080/solr/admin/cores?action=CREATE&*name=Test1*&;
*shard=Test*
&collection.configName=myconf&schema=schema.xml&config=solrconfigLocal_default.xml&collection=defaultcol&
*coreNodeName=Test1*

  It response an error:


400
 29


  Error CREATEing SolrCore 'Test1': Test1 is
removed
  400



I aslo find the reason the in the class  org.apache.solr.cloud.ZkController
line 1369~ 1384[1]
 As the src here,it needs to check the autoCreated within an existing
collection
when the coreNodeName and shard were assigned manully. the autoCreated
property of a collection is not equal with true, it throws an exeption.

  *Question2*: Why does it need  to check the 'autoCreated', and how could
I go through this check, or Is this another bug?




[1]-
try {
  if(cd.getCloudDescriptor().getCollectionName() !=null &&
cd.getCloudDescriptor().getCoreNodeName() != null ) {
//we were already registered

if(zkStateReader.getClusterState().hasCollection(cd.getCloudDescriptor().getCollectionName())){
DocCollection coll =
zkStateReader.getClusterState().getCollection(cd.getCloudDescriptor().getCollectionName());
 if(!"true".equals(coll.getStr("autoCreated"))){
   Slice slice =
coll.getSlice(cd.getCloudDescriptor().getShardId());
   if(slice != null){
 if(slice.getReplica(cd.getCloudDescriptor().getCoreNodeName())
== null) {
   log.info("core_removed This core is removed from ZK");
   throw new SolrException(ErrorCode.NOT_FOUND,coreNodeName +"
is removed");
 }
   }
 }
}
  }
--

Re: Maybe a bug for solr 4.6 when create a new core

[YouPeng Yang]

Hi Mark.

   Thanks for your reply.

I will file a JIRA issue about the NPE.

   By the way,would you look through the Question 2. After I create a new
core with explicite shard and coreNodeName successfully,I can not create a
replica for above new core also with explicite coreNodeName and the same
shard and collection
  Request url as following:

http://10.7.23.122:8080/solr/admin/cores?action=CREATE&name=Test1&shard=Test&collection.configName=myconf&schema=schema.xml&config=solrconfigLocal_default.xml&collection=defaultcol&coreNodeName=Test1

 It responses an error:


   400
 29
  
  
 Error CREATEing SolrCore 'Test1': Test1 is
removed
 400
  
  

   I find out that in the src class in  org.apache.solr.cloud.
ZkController line 1369~ 1384:
   As the code says,when I indicate a  coreNodeName and collection
explicitly,it goes to check a 'autoCreated' property of the Collection
which I have already created.

  My question :Why does it need to check the 'autoCreated' property,any
jira about this 'autoCreated' property? How can I make  through the check?


[1]-
try {
  if(cd.getCloudDescriptor().getCollectionName() !=null &&
 cd.getCloudDescriptor().getCoreNodeName() != null ) {
//we were already registered

 
if(zkStateReader.getClusterState().hasCollection(cd.getCloudDescriptor().getCollectionName())){
DocCollection coll =
 
zkStateReader.getClusterState().getCollection(cd.getCloudDescriptor().getCollectionName());
 if(!"true".equals(coll.getStr("autoCreated"))){
   Slice slice =
 coll.getSlice(cd.getCloudDescriptor().getShardId());
   if(slice != null){
 if(slice.getReplica(cd.getCloudDescriptor().getCoreNodeName())
 == null) {
   log.info("core_removed This core is removed from ZK");
   throw new SolrException(ErrorCode.NOT_FOUND,coreNodeName +"
 is removed");
 }
   }
 }
}
  }
 
--


Regards


2013/12/27 Mark Miller 

> If you are seeing an NPE there, sounds like you are on to something.
> Please file a JIRA issue.
>
> - Mark
>
> > On Dec 26, 2013, at 1:29 AM, YouPeng Yang 
> wrote:
> >
> > Hi
> >   Merry Christmas.
> >
> >   Before this mail,I am in trouble with a weird problem  for a few days
> > when to create a new core with both explicite shard and coreNodeName.
> And I
> > have posted a few mails  in the mailist,no one ever gives any
> > suggestions,maybe  they did not  encounter the same problem.
> >  I have to go through the srcs to check out the reason. Thanks god, I
> find
> > it. The reason to the problem,maybe be a bug, so I would like to report
> it
> > hoping to get your endorsement and confirmation.
> >
> >
> > In class org.apache.solr.cloud.Overseer the Line 360:
> > -
> >  if (sliceName !=null && collectionExists &&
> > !"true".equals(state.getCollection(collection).getStr("autoCreated"))) {
> >Slice slice = state.getSlice(collection, sliceName);
> >if (slice.getReplica(coreNodeName) == null) {
> >  log.info("core_deleted . Just return");
> >  return state;
> >}
> >  }
> > -
> > the slice needs to be checked null .because I create a new core with both
> > explicite shard and coreNodeName, the state.getSlice(collection,
> > sliceName)  may return a null.So it needs to be checked ,or there will be
> > an NullpointException
> > -
> >  if (sliceName !=null && collectionExists &&
> > !"true".equals(state.getCollection(collection).getStr("autoCreated"))) {
> >Slice slice = state.getSlice(collection, sliceName);
> >if (*slice != null &&* slice.getReplica(coreNodeName) ==
> null) {
> >  log.info("core_deleted . Just return");
> >  return state;
> >}
> >  }
> > -
> >
> > *Querstion 1*: Is this OK with the whole solr project,I have no aware
> > about the influences about the change,as right now ,it goes 

REYPLAY_ERR: IOException reading log

[YouPeng Yang]

Hi users
  I have build a SolrCloud  on tomcat.The cloud contains 22 shards with no
replica.Also  the the solrcloud is integrated with HDFS.

   After imported data for oracle to the solrcloud, I restart the tomcat
,it does not comes alive againt.
   It always give an exceptions.

   I'm really have not aware about this excetion. Because My schema do not
contains a BigDecimal type field.

   Could you give any tips?

746635 [recoveryExecutor-44-thread-1] WARN
org.apache.solr.update.UpdateLog  – REYPLAY_ERR: IOException reading log
org.apache.solr.common.SolrException: Invalid Number:
java.math.BigDecimal:238088174
at
org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:396)
at
org.apache.solr.update.AddUpdateCommand.getIndexedId(AddUpdateCommand.java:98)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:582)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at
org.apache.solr.update.UpdateLog$LogReplayer.doReplay(UpdateLog.java:1313)
at org.apache.solr.update.UpdateLog$LogReplayer.run(UpdateLog.java:1202)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
746681 [recoveryExecutor-44-thread-1] WARN
org.apache.solr.update.UpdateLog  – REYPLAY_ERR: IOException reading log
org.apache.solr.common.SolrException: Invalid Number:
java.math.BigDecimal:238088175

Re: Maybe a bug for solr 4.6 when create a new core

[YouPeng Yang]

 Hi Mark

  I have filed a jira about the NPE:
  https://issues.apache.org/jira/browse/SOLR-5580


2013/12/27 YouPeng Yang 

> Hi Mark.
>
>Thanks for your reply.
>
> I will file a JIRA issue about the NPE.
>
>By the way,would you look through the Question 2. After I create a new
> core with explicite shard and coreNodeName successfully,I can not create a
> replica for above new core also with explicite coreNodeName and the same
> shard and collection
>   Request url as following:
>
> http://10.7.23.122:8080/solr/admin/cores?action=CREATE&name=Test1&shard=Test&collection.configName=myconf&schema=schema.xml&config=solrconfigLocal_default.xml&collection=defaultcol&coreNodeName=Test1
>
>  It responses an error:
>
> 
> 
>400
>  29
>   
>   
>  Error CREATEing SolrCore 'Test1': Test1 is
> removed
>  400
>   
>   
>
>I find out that in the src class in  org.apache.solr.cloud.
> ZkController line 1369~ 1384:
>As the code says,when I indicate a  coreNodeName and collection
> explicitly,it goes to check a 'autoCreated' property of the Collection
> which I have already created.
>
>   My question :Why does it need to check the 'autoCreated' property,any
> jira about this 'autoCreated' property? How can I make  through the check?
>
>
> [1]-
> try {
>   if(cd.getCloudDescriptor().getCollectionName() !=null &&
>  cd.getCloudDescriptor().getCoreNodeName() != null ) {
> //we were already registered
>
>
>  
> if(zkStateReader.getClusterState().hasCollection(cd.getCloudDescriptor().getCollectionName())){
> DocCollection coll =
>
>  
> zkStateReader.getClusterState().getCollection(cd.getCloudDescriptor().getCollectionName());
>  if(!"true".equals(coll.getStr("autoCreated"))){
>Slice slice =
>  coll.getSlice(cd.getCloudDescriptor().getShardId());
>if(slice != null){
>  if(slice.getReplica(cd.getCloudDescriptor().getCoreNodeName())
>  == null) {
>log.info("core_removed This core is removed from ZK");
>throw new SolrException(ErrorCode.NOT_FOUND,coreNodeName +"
>  is removed");
>  }
>}
>  }
> }
>   }
>
>  
> --
>
>
> Regards
>
>
> 2013/12/27 Mark Miller 
>
>> If you are seeing an NPE there, sounds like you are on to something.
>> Please file a JIRA issue.
>>
>> - Mark
>>
>> > On Dec 26, 2013, at 1:29 AM, YouPeng Yang 
>> wrote:
>> >
>> > Hi
>> >   Merry Christmas.
>> >
>> >   Before this mail,I am in trouble with a weird problem  for a few days
>> > when to create a new core with both explicite shard and coreNodeName.
>> And I
>> > have posted a few mails  in the mailist,no one ever gives any
>> > suggestions,maybe  they did not  encounter the same problem.
>> >  I have to go through the srcs to check out the reason. Thanks god, I
>> find
>> > it. The reason to the problem,maybe be a bug, so I would like to report
>> it
>> > hoping to get your endorsement and confirmation.
>> >
>> >
>> > In class org.apache.solr.cloud.Overseer the Line 360:
>> > -
>> >  if (sliceName !=null && collectionExists &&
>> > !"true".equals(state.getCollection(collection).getStr("autoCreated"))) {
>> >Slice slice = state.getSlice(collection, sliceName);
>> >if (slice.getReplica(coreNodeName) == null) {
>> >  log.info("core_deleted . Just return");
>> >  return state;
>> >}
>> >  }
>> > -
>> > the slice needs to be checked null .because I create a new core with
>> both
>> > explicite shard and coreNodeName, the state.getSlice(collection,
>> > sliceName)  may return a null.So it needs to be checked ,or there will
>> be
>> > an NullpointException
>> > -
>> >  if (sliceName !=null && collectionExists &&
>> > !&qu

lots of tlog.

[YouPeng Yang]

Hi
  There is  a failed core in my solrcloud cluster(solr 4.6 with hdfs 2.2)
when I start my solrcloud . I noticed that there are lots of tlog files [1]
.
  The start proccess was stuck ,it need to do log replay.However it
encountered error[2].
  I do think that it is abnormal that there are still a lot of tlogs.(I
compare it with other normal cores in my solrCloud, even there are only 1
tlog file in normal cores).  Which conditions wil lead to this abnormal
lots of  tlog files? I try many times hoping to replay the this,but i
failed.
  Please give some suggestions.


Regards
[1]---
..
-rw-r--r--   1 solr solr   23462198 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.676
-rw-r--r--   1 solr solr   26083634 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.677
-rw-r--r--   1 solr solr   25428275 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.678
-rw-r--r--   1 solr solr   15794489 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.679
-rw-r--r--   1 solr solr   23593272 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.680
-rw-r--r--   1 solr solr   23068981 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.681
-rw-r--r--   1 solr solr   21889334 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.682
-rw-r--r--   1 solr solr   23331127 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.683
-rw-r--r--   1 solr solr   22675763 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.684
-rw-r--r--   1 solr solr   21954870 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.685
-rw-r--r--   1 solr solr   21496118 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.686
-rw-r--r--   1 solr solr   20775222 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.687
-rw-r--r--   1 solr solr   24183093 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.688
-rw-r--r--   1 solr solr   24183090 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.689
-rw-r--r--   1 solr solr   24379701 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.690
-rw-r--r--   1 solr solr   25887033 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.691
-rw-r--r--   1 solr solr   25231672 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.692
-rw-r--r--   1 solr solr   15335737 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.693
-rw-r--r--   1 solr solr8529530 2013-12-27 14:33
/solr/repCore/repCore/core_node20/data/tlog/tlog.742

[2]-
133462 [recoveryExecutor-48-thread-1] WARN
org.apache.solr.update.UpdateLog  – Starting log replay hdfs
tlog{file=hdfs://lklcluster/solr/repCore/repCore/core_node2
0/data/tlog/tlog.693 refcount=2} active=false starting pos=0
133576 [recoveryExecutor-48-thread-1] WARN
org.apache.solr.update.UpdateLog  – REYPLAY_ERR: IOException reading log
org.apache.solr.common.SolrException: Invalid Number:
java.math.BigDecimal:238124404
at
org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:396)
at
org.apache.solr.update.AddUpdateCommand.getIndexedId(AddUpdateCommand.java:98)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:582)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at
org.apache.solr.update.UpdateLog$LogReplayer.doReplay(UpdateLog.java:1313)
at org.apache.solr.update.UpdateLog$LogReplayer.run(UpdateLog.java:1202)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
133623 [recoveryExecutor-48-thread-1] WARN
org.apache.solr.update.UpdateLog  – REYPLAY_ERR: IOException reading log
org.apache.solr.common.SolrException: Invalid Number:
java.math.BigDecimal:238124405
at
org.apache.s

Re: Maybe a bug for solr 4.6 when create a new core

[YouPeng Yang]

Hi Mark Miller

   It's great that you have fixed the bug. By the way,there is another
point i want to remind you which I posted it before.I am just posting  it
again.
  Please check it again.whether is the autoCreated check here  is fit?

  IAfter I create a new core with explicite shard and coreNodeName
successfully when I fixed the bug,I can not create a replica for above new
core also with explicite coreNodeName and the same shard and collection
,the Request url as following:

http://10.7.23.122:8080/solr/admin/cores?action=CREATE&name=Test1&shard=Test&collection.configName=myconf&schema=schema.xml&config=solrconfigLocal_default.xml&collection=defaultcol&coreNodeName=Test1

 It responses an error:



   400
 29
  
  
 Error CREATEing SolrCore 'Test1': Test1 is
removed
 400
  
  

   I find out that in the src class in  org.apache.solr.cloud.ZkController
line 1369~ 1384:
   As the code says,when I indicate a  coreNodeName and collection
explicitly,it goes to check a 'autoCreated' property of the Collection
which I have already created.

  My question :Why does it need to check the 'autoCreated' property,any
jira about this 'autoCreated' property? How can I make  through the check?

   [1]---
--
try {
  if(cd.getCloudDescriptor().getCollectionName() !=null &&
 cd.getCloudDescriptor().getCoreNodeName() != null ) {
//we were already registered

 
if(zkStateReader.getClusterState().hasCollection(cd.getCloudDescriptor().getCollectionName())){
DocCollection coll =
 
zkStateReader.getClusterState().getCollection(cd.getCloudDescriptor().getCollectionName());
 if(!"true".equals(coll.getStr("autoCreated"))){
   Slice slice =
 coll.getSlice(cd.getCloudDescriptor().getShardId());
   if(slice != null){
 if(slice.getReplica(cd.getCloudDescriptor().getCoreNodeName())
 == null) {
   log.info("core_removed This core is removed from ZK");
   throw new SolrException(ErrorCode.NOT_FOUND,coreNodeName +"
 is removed");
 }
   }
 }
}
  }
 
----------


2013/12/27 YouPeng Yang 

>  Hi Mark
>
>   I have filed a jira about the NPE:
>   https://issues.apache.org/jira/browse/SOLR-5580
>
>
> 2013/12/27 YouPeng Yang 
>
>> Hi Mark.
>>
>>Thanks for your reply.
>>
>> I will file a JIRA issue about the NPE.
>>
>>By the way,would you look through the Question 2. After I create a new
>> core with explicite shard and coreNodeName successfully,I can not create a
>> replica for above new core also with explicite coreNodeName and the same
>> shard and collection
>>   Request url as following:
>>
>> http://10.7.23.122:8080/solr/admin/cores?action=CREATE&name=Test1&shard=Test&collection.configName=myconf&schema=schema.xml&config=solrconfigLocal_default.xml&collection=defaultcol&coreNodeName=Test1
>>
>>  It responses an error:
>>
>> 
>> 
>>400
>>  29
>>   
>>   
>>  Error CREATEing SolrCore 'Test1': Test1 is
>> removed
>>  400
>>   
>>   
>>
>>I find out that in the src class in  org.apache.solr.cloud.
>> ZkController line 1369~ 1384:
>>As the code says,when I indicate a  coreNodeName and collection
>> explicitly,it goes to check a 'autoCreated' property of the Collection
>> which I have already created.
>>
>>   My question :Why does it need to check the 'autoCreated' property,any
>> jira about this 'autoCreated' property? How can I make  through the check?
>>
>>
>> [1]-
>> try {
>>   if(cd.getCloudDescriptor().getCollectionName() !=null &&
>>  cd.getCloudDescriptor().getCoreNodeName() != null ) {
>> //we were already registered
>>
>>
>>  
>> if(zkStateReader.getClusterState().hasCollection(cd.getCloudDescriptor().getCollectionName())){
>> DocCollection coll =
>>
>>  
>> zkStateReader.getClusterState().getCollection(cd.getCloudDescriptor().getCollectionName());
>>  if(!"true".equals(coll.getStr("autoCreated"))){
>>Slice slice =
>>  coll.getSlice(cd.getCloudDescriptor().getShardId());
>>if(slice != null){

Re: lots of tlog.

[YouPeng Yang]

Hi Mark Miller

   How can a  log replay  fail .
 And I can not figure out the reason of the exception. It seems to no
BigDecimal type field  in my schema.

  Please give some suggestions


The exception :
133462 [recoveryExecutor-48-thread-1] WARN  org.apache.solr.update.
UpdateLog  – Starting log replay hdfs
tlog{file=hdfs://lklcluster/solr/repCore/repCore/core_node2
0/data/tlog/tlog.693 refcount=2} active=false starting pos=0
133576 [recoveryExecutor-48-thread-1] WARN
org.apache.solr.update.UpdateLog  – REYPLAY_ERR: IOException reading log
org.apache.solr.common.SolrException: Invalid Number:
java.math.BigDecimal:238124404
at
org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:396)
at
org.apache.solr.update.AddUpdateCommand.getIndexedId(AddUpdateCommand.java:98)
.

Re: Questions about integrateing SolrCloud with HDFS

[YouPeng Yang]

Hi Mark

   Are there have roadmap or plan about your futher work on Solr and with.
I do anticipate your great work.


Regards


2013/12/27 Mark Miller 

> 1. The exception and change in experience on the move to 4.6 seems like it
> could be a bug we want to investigate.
>
> 2. Solr storing data on hdfs in other ways seems like a different issue /
> improvement.
>
> 3. You shouldn't try and force more than one core to use the same index on
> hdfs. This would be bad.
>
> 4. You really want to use the solr.hdfs.home setting described in the
> documentation IMO.
>
> - Mark
>
> > On Dec 26, 2013, at 1:56 PM, Greg Walters 
> wrote:
> >
> > Mark,
> >
> > I'd be happy to but some clarification first; should this issue be about
> creating cores with overlapping names and the stack trace that YouPeng
> initially described, Solr's behavior when storing data on HDFS or YouPeng's
> other thread (Maybe a bug for solr 4.6 when create a new core) that looks
> like it might be a near duplicate of this one?
> >
> > Thanks,
> > Greg
> >
> >> On Dec 26, 2013, at 12:40 PM, Mark Miller 
> wrote:
> >>
> >> Can you file a JIRA issue?
> >>
> >> - Mark
> >>
> >>> On Dec 24, 2013, at 2:57 AM, YouPeng Yang 
> wrote:
> >>>
> >>> Hi users
> >>>
> >>> Solr supports for writing and reading its index and transaction log
> files
> >>> to the HDFS distributed filesystem.
> >>> **I am curious about that there are any other futher improvement about
> >>> the integration with HDFS.*
> >>> **For the solr  native replication  will make multiple copies  of the
> >>> master node's index. Because of the native replication of HDFS,there
> is no
> >>> need to do that.It just to need that multiple cores in solrcloud share
> the
> >>> same index directory in HDFS?*
> >>>
> >>>
> >>> The above supposition is what I want to achive when we are integrating
> >>> SolrCloud with HDFS (Solr 4.6).
> >>> To make sure of our application high available,we still have  to take
> >>> the solr   replication with   some tricks.
> >>>
> >>> Firstly ,noting that  solr's index directory is made up of
> >>> *collectionName/coreNodeName/data/index *
> >>>
> >>> *collectionName/coreNodeName/data/tlog*
> >>> So to achive this,we want to create multi cores that use the same  hdfs
> >>> index directory .
> >>>
> >>> I have tested this  within solr 4.4 by expilcitly indicating  the same
> >>> coreNodeName.
> >>>
> >>> For example:
> >>> Step1, a core was created with the name=core1 and shard=core_shard1 and
> >>> collection=clollection1 and coreNodeName=*core1*
> >>> Step2. create another core  with the name=core2 and shard=core_shard1
> and
> >>> collection=clollection1 and coreNodeName=
> >>> *core1*
> >>> *  T*he two core share the same shard ,collection and coreNodeName.As a
> >>> result,the two core will get the same index data which is stored in the
> >>> hdfs directory :
> >>> hdfs://myhdfs/*clollection1*/*core1*/data/index
> >>> hdfs://myhdfs/*clollection1*/*core1*/data/tlog
> >>>
> >>> Unfortunately*, *as the solr 4.6 was released,we upgraded . the above
> >>> goal failed. We could not create a core with both expilcit shard and
> >>> coreNodeName.
> >>> Exceptions are as [1].
> >>> *  Can some give some help?*
> >>>
> >>>
> >>> Regards
> >>>
> [1]--
> >>> 64893635 [http-bio-8080-exec-1] INFO
>  org.apache.solr.cloud.ZkController
> >>> ?.publishing core=hdfstest3 state=down
> >>> 64893635 [http-bio-8080-exec-1] INFO
>  org.apache.solr.cloud.ZkController
> >>> ?.numShards not found on descriptor - reading it from system property
> >>> 64893698 [http-bio-8080-exec-1] INFO
>  org.apache.solr.cloud.ZkController
> >>> ?.look for our core node name
> >>>
> >>>
> >>>
> >>> 64951227 [http-bio-8080-exec-17] INFO  org.apache.solr.core.SolrCore
> >>> ?.[reportCore_201208] webapp=/solr path=/replication
> >>>
> params={slave=false&command=details&wt=javabin&qt=/replication&version=2}
&g

High cpu ratio when solr sleep

[YouPeng Yang]

Hi
  I find that the cpu ratio is very high when the tomcat contained solr
4.6 sleep.
The pid 13359  shows that my sleeping solr web container  take high cpu
ratio

  Any insights?


[solr@fkapp1 ~]$ top -d -1  -u solr
top - 17:30:15 up 302 days,  7:10,  5 users,  load average: 4.54, 4.52, 4.47
Tasks: 418 total,   1 running, 412 sleeping,   0 stopped,   5 zombie
Cpu(s): 19.1%us,  0.1%sy,  0.0%ni, 80.8%id,  0.0%wa,  0.0%hi,  0.0%si,
0.0%st
Mem:  32955380k total, 28288212k used,  4667168k free,   503148k buffers
Swap: 37257200k total,87064k used, 37170136k free, 10861500k cached

  PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+
COMMAND

13359 solr  21   0 11.4g 6.7g  12m S 400.5 21.4 491:30.85 java

 3678 solr  15   0 13020 1380  828 R  0.0  0.0   0:19.16
top

 3694 solr  15   0 66092 1556 1228 S  0.0  0.0   0:00.01
bash

Re: High cpu ratio when solr sleep

[YouPeng Yang]

util.concurrent.locks.ReentrantLock$NonfairSync)

..
---




2014/1/16 Otis Gospodnetic 

> I wonder if this would be a good addition to Solr Admin functionality?
>
> Otis
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
> On Wed, Jan 15, 2014 at 6:29 AM, Mikhail Khludnev <
> mkhlud...@griddynamics.com> wrote:
>
> > Hello,
> >
> > Invoke top for particular process displaying threads enabled.
> > Find the hottest thread PID.
> > invoke jstack for this process, find the suspicious thread by "..
> > nid=0x[PID in hex]"
> > ...
> > PROFIT!
> >
> >
> > On Wed, Jan 15, 2014 at 1:38 PM, YouPeng Yang  > >wrote:
> >
> > > Hi
> > >   I find that the cpu ratio is very high when the tomcat contained solr
> > > 4.6 sleep.
> > > The pid 13359  shows that my sleeping solr web container  take high cpu
> > > ratio
> > >
> > >   Any insights?
> > >
> > >
> > > [solr@fkapp1 ~]$ top -d -1  -u solr
> > > top - 17:30:15 up 302 days,  7:10,  5 users,  load average: 4.54, 4.52,
> > > 4.47
> > > Tasks: 418 total,   1 running, 412 sleeping,   0 stopped,   5 zombie
> > > Cpu(s): 19.1%us,  0.1%sy,  0.0%ni, 80.8%id,  0.0%wa,  0.0%hi,  0.0%si,
> > > 0.0%st
> > > Mem:  32955380k total, 28288212k used,  4667168k free,   503148k
> buffers
> > > Swap: 37257200k total,87064k used, 37170136k free, 10861500k cached
> > >
> > >   PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+
> > > COMMAND
> > >
> > > 13359 solr  21   0 11.4g 6.7g  12m S 400.5 21.4 491:30.85 java
> > >
> > >  3678 solr  15   0 13020 1380  828 R  0.0  0.0   0:19.16
> > > top
> > >
> > >  3694 solr  15   0 66092 1556 1228 S  0.0  0.0   0:00.01
> > > bash
> > >
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> > Principal Engineer,
> > Grid Dynamics
> >
> > <http://www.griddynamics.com>
> >  
> >
>

Re: High cpu ratio when solr sleep

[YouPeng Yang]

Hi
  By the ways,after I restart the web container ,the ratio returns normal.
So when does the sutuation come out?



Regards



2014/1/16 YouPeng Yang 

> Hi
>   Thanks for the reply.
>   I get the information as following:
> 
> [solr@fkapp1 ~]$ ps mp  13359  -o THREAD,tid
> USER %CPU PRI SCNT WCHAN  USER SYSTEM   TID
> solr  217   -- - -  - -
> solr  0.0  21- 184466-  - 13359
> solr  0.0  19- - -  - 13360
> solr  0.0  23- 184466-  - 13361
> .
> solr 99.9  14- - -  -  1210
> solr 99.9  14- - -  -  1223
> solr 99.9  14- - -  -  1227
> solr 99.9  14- - -  -  1228
> --
> Definitely, the suspicious threads are : 1210 1223 1227 1228, hexadecimal
> values are 0x4ba,0x4c7,0x4cb,0x4cc,
>
> And then get the thread info about the threads by the jstack tools.
> --
> jstack  -l 13359 > dump.stack
>
> ---
>
> Finally, I find the stack info about the above threads:
>
> I am not clear about the information, what does it mean?
> Is there anything abnormal with the SolrDispatchFilter?
>
>
>
> ---
> "http-bio-8081-exec-820" daemon prio=10 tid=0x2aaac0d02800 nid=0x4cc
> runnable [0x43c87000]
>java.lang.Thread.State: RUNNABLE
> at java.util.WeakHashMap.put(WeakHashMap.java:405)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> - locked <0x000640604558> (a
> org.apache.tomcat.util.net.SocketWrapper)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
>
>Locked ownable synchronizers:
> - <0x00064061a360> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
> "http-bio-8081-exec-802" daemon prio=10 tid=0x2aaac0a03000 nid=0x4ba
> runnable [0x47dc8000]
>java.lang.Thread.State: RUNNABLE
> at java.util.WeakHashMap.get(WeakHashMap.java:355)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:4

Re: High cpu ratio when solr sleep

[YouPeng Yang]

Hi Mikhail Khludnev

   I do confirm that there are no requests at all.And I have checked that
nothing abnormal with jstat .About the autowarming,I have set it up,but
there are no commits or optimize on these cores.

   On the instance ,there are about 22 cores whose datadirs are on the HDFS.

It is werid ,I will contact with you if the problem come again.


Regards


2014/1/17 Mikhail Khludnev 

> Hello,
>
> I can't say anything from this thread dump, but it's really suspicious
> stacks:
>
>java.lang.Thread.State: RUNNABLE
> at java.util.WeakHashMap.get(
> WeakHashMap.java:355)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
>
>java.lang.Thread.State: RUNNABLE
> at java.util.WeakHashMap.put(
> WeakHashMap.java:405)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
>
> Nevertheless, it disproves your initial statement about sleeping solr - it
> handled requests, might not be regular ones. Now you can find in log which
> requests those threads handled at that time.
> How many cores you have in this instance? Are you sure that everything was
> fine with the heap?
>
>
> On Thu, Jan 16, 2014 at 11:36 AM, YouPeng Yang  >wrote:
>
> > Hi
> >   Thanks for the reply.
> >   I get the information as following:
> > 
> > [solr@fkapp1 ~]$ ps mp  13359  -o THREAD,tid
> > USER %CPU PRI SCNT WCHAN  USER SYSTEM   TID
> > solr  217   -- - -  - -
> > solr  0.0  21- 184466-  - 13359
> > solr  0.0  19- - -  - 13360
> > solr  0.0  23- 184466-  - 13361
> > .
> > solr 99.9  14- - -  -  1210
> > solr 99.9  14- - -  -  1223
> > solr 99.9  14- - -  -  1227
> > solr 99.9  14- - -  -  1228
> > --
> > Definitely, the suspicious threads are : 1210 1223 1227 1228, hexadecimal
> > values are 0x4ba,0x4c7,0x4cb,0x4cc,
> >
> > And then get the thread info about the threads by the jstack tools.
> > --
> > jstack  -l 13359 > dump.stack
> >
> > ---
> >
> > Finally, I find the stack info about the above threads:
> >
> > I am not clear about the information, what does it mean?
> > Is there anything abnormal with the SolrDispatchFilter?
> >
> >
> >
> > ---
> > "http-bio-8081-exec-820" daemon prio=10 tid=0x2aaac0d02800 nid=0x4cc
> > runnable [0x43c87000]
> >java.lang.Thread.State: RUNNABLE
> > at java.util.WeakHashMap.put(WeakHashMap.java:405)
> > at
> >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
> > at
> >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
> > at
> >
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> > at
> >
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> > at
> >
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> > at
> >
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> > at
> >
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> > at
> >
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> > at
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
> > at
> >
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> > at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> > at
> >
> >
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
> > at
> >
> >
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> > at
> >
> >
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> > - locked <0x000640604558> (a
> > org.apache.tomcat.util.ne

Fatal full GC

[YouPeng Yang]

Hi
 We build the SolrCloud using solr4.6.0 and jdk1.7.60 ,our cluster contains
360G*3 data(one core with 2 replica).
  Our cluster becomes unstable which means occasionlly it comes out long
time full gc.This is awful,the full gc take long take that the solrcloud
consider it as down.
  Normally full gc happens when the Old Generaion  get 70%,and it is
OK.However In the awfull condition,the percentage is highly above 70% ,and
become  99% so that the long full gc happens,and the node is considered as
down.
   We set he JVM parameters referring to the URL
:*https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning
*, the only difference
is that we change the *-Xms48009m -Xmx48009m* to *-Xms49152M -Xmx81920M* .
  The appendix[1]  is the output of the jstat when the awful full gc
happens.I have marked  the important port with red font hoping to be
helpful.
   By the way,I have notice that Eden part of Young Generation takes 100%
always during the awful condition happens,which I think it is a import
indication.
  The SolrCloud will be used to support our applications as a very
important part.
  Would you please give me any suggestion? Do I need to change the JDK
version?


Any suggestions will be appreciated.

Best Regard


[1]--
   S0 S1 E  O  P YGC YGCTFGCFGCT GCT
 ..ommit..
 33.27  84.37 100.00  70.14  59.94  28070 4396.141143.724 4399.864
100.00   0.00  30.96  70.38  59.94  28077 4397.877143.724 4401.601
  0.00  72.06   0.00  70.66  59.94  28083 4399.554143.724 4403.277
 59.50  49.30 100.00  70.88  59.94  28091 4401.101143.724 4404.825
 76.98 100.00 100.00  71.07  59.94  28098 4402.707143.724 4406.431
100.00  84.59 100.00  71.41  59.94  28105 4404.526143.724 4408.250
100.00  89.60 100.00  71.77  59.94  28111 4406.216143.724 4409.939
100.00 100.00  99.92  72.16  59.94  28116 4407.609143.724 4411.333
100.00 100.00 100.00  72.68  59.94  28120 4409.041143.724 4412.764
100.00 100.00 100.00  73.02  59.94  28126 4410.666143.724 4414.390
 92.06 100.00 100.00  73.37  59.94  28132 4412.389143.724 4416.113
 68.89 100.00 100.00  73.74  59.94  28138 4414.004143.724 4417.728
100.00 100.00 100.00  73.99  59.94  28144 4415.555143.724 4419.278
100.00  56.44 100.00  74.31  59.94  28151 4417.311143.724 4421.034
 65.78  25.37 100.00  74.57  59.94  28159 4419.051143.724 4422.774
 62.41  43.09 100.00  74.76  59.94  28167 4420.740143.724 4424.464
 36.14  15.59 100.00  74.97  59.94  28175 4422.353143.724 4426.077
 91.86  37.75 100.00  75.09  59.94  28183 4423.976143.724 4427.700
 87.88 100.00 100.00  75.30  59.94  28190 4425.713143.724 4429.437
 88.91 100.00 100.00  75.63  59.94  28196 4427.293143.724 4431.017
100.00 100.00 100.00  76.01  59.94  28202 4428.816143.724 4432.539
  0.00 100.00  97.08  76.28  59.94  28208 4430.504143.724 4434.228
 63.42  45.06 100.00  76.57  59.94  28215 4432.018143.724 4435.742
 52.26  35.19 100.00  76.73  59.94  28223 4433.644143.724 4437.367
100.00   0.00  75.24  76.88  59.94  28230 4435.231143.724 4438.955
100.00 100.00 100.00  77.27  59.94  28235 4436.334143.724 4440.057
 87.09 100.00 100.00  77.63  59.94  28242 4438.118143.724 4441.842
 92.06 100.00 100.00  95.77  59.94  28248 4439.763143.724 4443.487
  0.00 100.00  37.93  78.65  59.94  28253 4441.483143.724 4445.207
 68.38  81.73 100.00  79.04  59.94  28260 4442.971143.724 4446.695
100.00 100.00 100.00  79.24  59.94  28267 .706143.724 4448.429
 95.40   0.00   0.00  79.56  59.94  28274 4446.608143.724 4450.332
 53.60   0.00 100.00  79.82  59.94  28283 4448.213143.724 4451.937
100.00  89.81 100.00  80.01  59.94  28291 4449.759143.724 4453.483
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 ..ommit..

Fatal full GC

[YouPeng Yang]

Hi
 We build the SolrCloud using solr4.6.0 and jdk1.7.60 ,our cluster contains
360G*3 data(one core with 2 replica).
  Our cluster becomes unstable which means occasionlly it comes out long
time full gc.This is awful,the full gc take long take that the solrcloud
consider it as down.
  Normally full gc happens when the Old Generaion  get 70%,and it is
OK.However In the awfull condition,the percentage is highly above 70% ,and
become  99% so that the long full gc happens,and the node is considered as
down.
   We set he JVM parameters referring to the URL
:*https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning
*, the only difference
is that we change the *-Xms48009m -Xmx48009m* to *-Xms49152M -Xmx81920M* .
  The appendix[1]  is the output of the jstat when the awful full gc
happens.I have marked  the important port with red font hoping to be
helpful.
   By the way,I have notice that Eden part of Young Generation takes 100%
always during the awful condition happens,which I think it is a import
indication.
  The SolrCloud will be used to support our applications as a very
important part.
  Would you please give me any suggestion? Do I need to change the JDK
version?


Any suggestions will be appreciated.

Best Regard


[1]--
   S0 S1 E  O  P YGC YGCTFGCFGCT GCT
 ..ommit..
 33.27  84.37 100.00  70.14  59.94  28070 4396.141143.724 4399.864
100.00   0.00  30.96  70.38  59.94  28077 4397.877143.724 4401.601
  0.00  72.06   0.00  70.66  59.94  28083 4399.554143.724 4403.277
 59.50  49.30 100.00  70.88  59.94  28091 4401.101143.724 4404.825
 76.98 100.00 100.00  71.07  59.94  28098 4402.707143.724 4406.431
100.00  84.59 100.00  71.41  59.94  28105 4404.526143.724 4408.250
100.00  89.60 100.00  71.77  59.94  28111 4406.216143.724 4409.939
100.00 100.00  99.92  72.16  59.94  28116 4407.609143.724 4411.333
100.00 100.00 100.00  72.68  59.94  28120 4409.041143.724 4412.764
100.00 100.00 100.00  73.02  59.94  28126 4410.666143.724 4414.390
 92.06 100.00 100.00  73.37  59.94  28132 4412.389143.724 4416.113
 68.89 100.00 100.00  73.74  59.94  28138 4414.004143.724 4417.728
100.00 100.00 100.00  73.99  59.94  28144 4415.555143.724 4419.278
100.00  56.44 100.00  74.31  59.94  28151 4417.311143.724 4421.034
 65.78  25.37 100.00  74.57  59.94  28159 4419.051143.724 4422.774
 62.41  43.09 100.00  74.76  59.94  28167 4420.740143.724 4424.464
 36.14  15.59 100.00  74.97  59.94  28175 4422.353143.724 4426.077
 91.86  37.75 100.00  75.09  59.94  28183 4423.976143.724 4427.700
 87.88 100.00 100.00  75.30  59.94  28190 4425.713143.724 4429.437
 88.91 100.00 100.00  75.63  59.94  28196 4427.293143.724 4431.017
100.00 100.00 100.00  76.01  59.94  28202 4428.816143.724 4432.539
  0.00 100.00  97.08  76.28  59.94  28208 4430.504143.724 4434.228
 63.42  45.06 100.00  76.57  59.94  28215 4432.018143.724 4435.742
 52.26  35.19 100.00  76.73  59.94  28223 4433.644143.724 4437.367
100.00   0.00  75.24  76.88  59.94  28230 4435.231143.724 4438.955
100.00 100.00 100.00  77.27  59.94  28235 4436.334143.724 4440.057
 87.09 100.00 100.00  77.63  59.94  28242 4438.118143.724 4441.842
 92.06 100.00 100.00  95.77  59.94  28248 4439.763143.724 4443.487
  0.00 100.00  37.93  78.65  59.94  28253 4441.483143.724 4445.207
 68.38  81.73 100.00  79.04  59.94  28260 4442.971143.724 4446.695
100.00 100.00 100.00  79.24  59.94  28267 .706143.724 4448.429
 95.40   0.00   0.00  79.56  59.94  28274 4446.608143.724 4450.332
 53.60   0.00 100.00  79.82  59.94  28283 4448.213143.724 4451.937
100.00  89.81 100.00  80.01  59.94  28291 4449.759143.724 4453.483
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 88.21 100.00 100.00  80.38  59.94  28298 4451.466143.724 4455.190
 ..ommit..

Re: Fatal full GC

[YouPeng Yang]

Hi
  Thank you very much. we make the change to low down the Heap size ,we are
watching the effect of this change.we will inform you about the result.
  It is really helpful.

Best Regard

2014-09-12 23:00 GMT+08:00 Walter Underwood :

> I agree about the 80Gb heap as a possible problem.
>
> A GC is essentially a linear scan of memory. More memory means a longer
> scan.
>
> We run with an 8Gb heap. I’d try that. Test it by replaying logs from
> production against a test instance. You can use JMeter and the Apache
> access log sampler.
>
>
> https://jmeter.apache.org/usermanual/jmeter_accesslog_sampler_step_by_step.pdf
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/
>
>
> On Sep 12, 2014, at 7:10 AM, Shawn Heisey  wrote:
>
> > On 9/12/2014 7:36 AM, YouPeng Yang wrote:
> >> We build the SolrCloud using solr4.6.0 and jdk1.7.60 ,our cluster
> contains
> >> 360G*3 data(one core with 2 replica).
> >>  Our cluster becomes unstable which means occasionlly it comes out long
> >> time full gc.This is awful,the full gc take long take that the solrcloud
> >> consider it as down.
> >>  Normally full gc happens when the Old Generaion  get 70%,and it is
> >> OK.However In the awfull condition,the percentage is highly above 70%
> ,and
> >> become  99% so that the long full gc happens,and the node is considered
> as
> >> down.
> >>   We set he JVM parameters referring to the URL
> >> :*https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning
> >> <https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning>*, the only
> difference
> >> is that we change the *-Xms48009m -Xmx48009m* to *-Xms49152M
> -Xmx81920M* .
> >>  The appendix[1]  is the output of the jstat when the awful full gc
> >> happens.I have marked  the important port with red font hoping to be
> >> helpful.
> >>   By the way,I have notice that Eden part of Young Generation takes 100%
> >> always during the awful condition happens,which I think it is a import
> >> indication.
> >>  The SolrCloud will be used to support our applications as a very
> >> important part.
> >>  Would you please give me any suggestion? Do I need to change the JDK
> >> version?
> >
> > My GC parameter page is getting around. :)
> >
> > Do you really need an 80GB heap?  I realize that your index is 360GB ...
> > but if you really do need a heap that large, you may need to adjust your
> > configuration so you use a lot less heap memory.
> >
> > The red font you mentioned did not make it through, so I cannot tell
> > what lines you highlighted.
> >
> > I pulled your jstat output into a spreadsheet and calculated the length
> > of each GC.  The longest GC in there took 1.903 seconds.  It's the one
> > that had a GCT of 4450.332.  For an 80GB heap, you couldn't hope for
> > anything better.  Based on what I see here, I don't think GC is your
> > problem.  If I read the other numbers on that 1.903 second GC line
> > correctly (not sure that I am), it dropped your Eden size from 100% to
> > 0% ... suggesting that you really don't need an 80GB heap.
> >
> > How much RAM does this machine have?  For ideal performance, you'll need
> > your index size plus your heap size, which for you right now is 440 GB.
> > Normally you don't need the ideal memory size ... but you do need a
> > *significant* portion of it.  I don't think I'd try running this index
> > with less than 256GB of RAM, and that's assuming a much lower heap size
> > than 80GB.
> >
> > Here's some general info about performance problems and possible
> solutions:
> >
> > http://wiki.apache.org/solr/SolrPerformanceProblems
> >
> > Thanks,
> > Shawn
> >
>
>

How to summarize a String Field ?

[YouPeng Yang]

Hi

   One of my filed called AMOUNT  is  String,and I want to  calculate the
sum of the this filed.
I have try it with the stats component,it only give out the stats
information without sum item just as following:


 
 5000
 24230
 26362
  


   Is there any ways to achieve this object?

Regards

Create core with user-defined the instanceDir

[YouPeng Yang]

Hi All

With the Solr-4.4 ,I try the Hdfs and SolrCloud.
   I create  a SolrCloud core with hdfs using :

http://10.7.23.125:8080/solr/admin/cores?action=CREATE&name=testcore8_1&shard=shard13&collection.configName=myconf&schema=schema.xml&config=solrconfig3.xml&collection=collection1&dataDir=/soData/

  Checking the Hdfs ,I find the created path：
drwxr-xr-x   - hadoop supergroup  0 2013-08-20 18:05
/solrdata/collection1/core_node16
drwxr-xr-x   - hadoop supergroup  0 2013-08-20 18:05
/solrdata/collection1/core_node16/soData
drwxr-xr-x   - hadoop supergroup  0 2013-08-20 18:07
/solrdata/collection1/core_node16/soData/index




 Can i define  the instanceDir by myself ,rather than using the default
instanceDir?

  Also I have try the command:
http://10.7.23.125:8080/solr/admin/cores?action=CREATE&name=testcore8_1&shard=shard13&collection.configName=myconf&schema=schema.xml&config=solrconfig3.xml&collection=collection1&dataDir=/soData/&;
instanceDir=heihei
   However, It does not work...


Regards

Thankyou

Re: Create core with user-defined the instanceDir

[YouPeng Yang]

Hi Erick
 Thanks for your reponse.
 In fact ,I create an core by send the Http request  with parameters
above instead of set the  configuration files.

http://10.7.23.125:8080/solr/admin/cores?action=CREATE&name=testcore8_1&shard=shard13&collection.configName=myconf&schema=schema.xml&config=solrconfig3.xml&collection=collection1&dataDir=/soData/&;
instanceDir=heihei

The core  was actully created  with the path which I have post above:

 drwxr-xr-x   - hadoop supergroup  0 2013-08-20 18:05
/solrdata/collection1/core_node16
drwxr-xr-x   - hadoop supergroup  0 2013-08-20 18:05
/solrdata/collection1/core_node16/soData
drwxr-xr-x   - hadoop supergroup  0 2013-08-20 18:07
/solrdata/collection1/core_node16/soData/index

   Sorry for that I do not make clear.

   Back to the point,the created core has an instanceDir -core_node16- that
is not same with  -heihei- which I defined  through the URL.

   Also ,I try to define the instanceDir in the core.properties
configuration file, It does not work either.

   Anyone has encountered this werid problem?


Regards


2013/8/20 Erick Erickson 

> _How_ does it "not work"? You've given us no information to go on.
>
> Please review:
> http://wiki.apache.org/solr/UsingMailingLists
>
> At a guess, does your heihei directory exist and does it have
> the proper configuration files?
>
> Best
> Erick
>
>
> On Tue, Aug 20, 2013 at 6:38 AM, YouPeng Yang  >wrote:
>
> > Hi All
> >
> > With the Solr-4.4 ,I try the Hdfs and SolrCloud.
> >I create  a SolrCloud core with hdfs using :
> >
> >
> >
> http://10.7.23.125:8080/solr/admin/cores?action=CREATE&name=testcore8_1&shard=shard13&collection.configName=myconf&schema=schema.xml&config=solrconfig3.xml&collection=collection1&dataDir=/soData/
> >
> >   Checking the Hdfs ,I find the created path：
> > drwxr-xr-x   - hadoop supergroup  0 2013-08-20 18:05
> > /solrdata/collection1/core_node16
> > drwxr-xr-x   - hadoop supergroup  0 2013-08-20 18:05
> > /solrdata/collection1/core_node16/soData
> > drwxr-xr-x   - hadoop supergroup  0 2013-08-20 18:07
> > /solrdata/collection1/core_node16/soData/index
> >
> >
> >
> >
> >  Can i define  the instanceDir by myself ,rather than using the default
> > instanceDir?
> >
> >   Also I have try the command:
> >
> >
> http://10.7.23.125:8080/solr/admin/cores?action=CREATE&name=testcore8_1&shard=shard13&collection.configName=myconf&schema=schema.xml&config=solrconfig3.xml&collection=collection1&dataDir=/soData/&;
> > instanceDir=heihei
> >However, It does not work...
> >
> >
> > Regards
> >
> > Thankyou
> >
>

Re: "Path must not end with / character" error during performance tests

[YouPeng Yang]

Hi Erick,Tanya

Happened I saw the mail,and I would like report anothe relative issue.
I create a core through the solr/admin URL.
when I set the  collection name to / , the catalina.out of the tomcat
send lots of exceptions  -Path must
not end with / character-. Worse, It seems never to stop, always continue.

   Is there any pre-check to make sure the input is correct?

  When the exceptions come out ,I have to stop all the tomcats, It shows
that the exceptions continue due to it would switch to the other tomcat
server.




2013/8/21 Erick Erickson 

> It looks like you've specified your zkHost (?) as something like
> machine:port/solr/
>
> rather than
> machine:port/solr
>
> Is that possible?
>
> Best,
> Erick
>
>
> On Tue, Aug 20, 2013 at 12:10 PM, Tanya  wrote:
>
> > Hi,
> >
> > I have integrated SolrCloud search in some system with single shard and
> > works fine on single tests.
> >
> > Recently we started to run performance test and we are getting following
> > exception after a while.
> >
> > Help is really appreciated,
> >
> > Thanks
> > Tanya
> >
> > 2013-08-20 10:45:56,128 [cTaskExecutor-1] ERROR
> > LoggingAspect  - Exception
> >
> > java.lang.RuntimeException: java.lang.IllegalArgumentException: Path must
> > not end with / character
> >
> > at
> > org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:123)
> >
> > at
> > org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:88)
> >
> > at
> > org.apache.solr.common.cloud.ZkStateReader.(ZkStateReader.java:148)
> >
> > at
> >
> >
> org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:147)
> >
> > at
> >
> >
> org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:173)
> >
> > at
> >
> >
> org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
> >
> > at
> >
> >
> com.mcm.search.solr.SolrDefaultIndexingAdapter.updateEntities(SolrDefaultIndexingAdapter.java:109)
> >
> > at
> >
> >
> com.mcm.search.AbstractSearchEngineMediator.updateDocuments(AbstractSearchEngineMediator.java:89)
> >
> > at
> >
> >
> com.alu.dmsp.search.mediator.wrapper.SearchMediatorWrapper.updateDocuments(SearchMediatorWrapper.java:255)
> >
> > at
> >
> >
> com.alu.dmsp.business.module.beans.DiscoverySolrIndexingRequestHandler.executeLogic(DiscoverySolrIndexingRequestHandler.java:122)
> >
> > at
> >
> >
> com.alu.dmsp.business.module.beans.DiscoverySolrIndexingRequestHandler.executeLogic(DiscoverySolrIndexingRequestHandler.java:36)
> >
> > at
> >
> >
> com.alu.dmsp.common.business.BasicBusinessServiceExecutionRequestHandler.executeRequest(BasicBusinessServiceExecutionRequestHandler.java:107)
> >
> > at
> >
> >
> com.alu.dmsp.common.business.BasicBusinessServiceExecutionRequestHandler.execute(BasicBusinessServiceExecutionRequestHandler.java:84)
> >
> > at
> >
> >
> com.alu.dmsp.common.business.beans.BasicBusinessService.executeRequest(BasicBusinessService.java:92)
> >
> > at
> >
> >
> com.alu.dmsp.common.business.beans.BasicBusinessService.execute(BasicBusinessService.java:79)
> >
> > at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> >
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:606)
> >
> > at
> >
> >
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319)
> >
> > at
> >
> >
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> >
> > at
> >
> >
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> >
> > at
> >
> >
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
> >
> > at
> >
> >
> com.alu.dmsp.common.log.LoggingAspect.logWebServiceMethodCall(LoggingAspect.java:143)
> >
> > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> >
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:606)
> >
> > at
> >
> >
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
> >
> > at
> >
> >
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
> >
> > at
> >
> >
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
> >
> > at
> >
> >
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
> >
> > at
> >
> >
> org.springframework.aop.aspectj.Asp

relation between optimize and merge

[YouPeng Yang]

Hi All

   I do have some diffculty with understand the relation between the
optimize and merge
  Can anyone give some tips about the difference.

Regards

when does RAMBufferSize work when commit.

[YouPeng Yang]

Hi all
About the RAMBufferSize  and commit ,I have read the doc :
http://comments.gmane.org/gmane.comp.jakarta.lucene.solr.user/60544

   I can not figure out how do they make work.

  Given the settings:

 10
 
   ${solr.autoCommit.maxDocs:1000}
   false
 

 If the indexs docs up to 1000  and the size of these docs is below 10MB
,it will trigger an commit.

 If the size of the indexed docs reaches to 10MB while the the number is below
1000, it will not trigger an commit , however the index docs will just
be flushed
to disk,it will only commit when the number reaches to 1000?

 Are  the two scenarioes right?


Regards

how to integrate solr with HDFS HA

[YouPeng Yang]

Hi all
I try to integrate solr with HDFS HA.When I start the solr server, it
comes out an exeception[1].
And I do know this is because the hadoop.conf.Configuration  in
HdfsDirectoryFactory.java does not include the HA configuration.
So I want to know ,in solr,is there any way to include my hadoop  HA
configuration ?


[1]---
Caused by: java.lang.IllegalArgumentException:
java.net.UnknownHostException: lklcluster
at
org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:418)
at
org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:164)
at
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:415)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:382)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:123)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2277)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:86)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2311)
at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:2299)
at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:364)
at
org.apache.solr.store.hdfs.HdfsDirectory.(HdfsDirectory.java:59)
at
org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:154)
at
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:350)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:256)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:469)
at org.apache.solr.core.SolrCore.(SolrCore.java:759)

Re: when does RAMBufferSize work when commit.

[YouPeng Yang]

Hi Shawn

Thanks a lot. I got it.

Regards


2013/8/22 Shawn Heisey 

> On 8/22/2013 2:25 AM, YouPeng Yang wrote:
> > Hi all
> > About the RAMBufferSize  and commit ,I have read the doc :
> > http://comments.gmane.org/gmane.comp.jakarta.lucene.solr.user/60544
> >
> >I can not figure out how do they make work.
> >
> >   Given the settings:
> >
> >  10
> >  
> >${solr.autoCommit.maxDocs:1000}
> >false
> >  
> >
> >  If the indexs docs up to 1000  and the size of these docs is below 10MB
> > ,it will trigger an commit.
> >
> >  If the size of the indexed docs reaches to 10MB while the the number is
> below
> > 1000, it will not trigger an commit , however the index docs will just
> > be flushed
> > to disk,it will only commit when the number reaches to 1000?
>
> Your actual config seems to have its wires crossed a little bit.  You
> have the autoCommit.maxDocs value being used in a maxTime tag, not a
> maxDocs tag.  You may want to adjust the variable name or the tag.
>
> If that were a maxDocs tag instead of maxTime, your description would be
> pretty much right on the money.  The space taken in the RAM buffer is
> typically larger than the actual document size, but the general idea is
> sound.
>
> The default for RAMBufferSizeMB in recent Solr versions is 100.  Unless
> you've got super small documents, or you are in a limited memory
> situation and have a lot of cores, I would not go smaller than that.
>
> Thanks,
> Shawn
>
>

Re: how to integrate solr with HDFS HA

[YouPeng Yang]

Hi Greg
   Thanks for your reponse.
   It works



2013/8/23 Greg Walters 

> Finally something I can help with! I went through the same problems you're
> having a short while ago. Check out
> https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS for
> most of the information you need and be sure to check the comments on the
> page as well.
>
> Here's an example from my working setup:
>
> **
>class="solr.HdfsDirectoryFactory">
> true
> 1
>  name="solr.hdfs.blockcache.direct.memory.allocation">true
> 16384
> true
> true
> true
> 16
> 192
> hdfs://nameservice1:8020/solr
> /etc/hadoop/conf.cloudera.hdfs1
>   
> **
>
> Thanks,
> Greg
>
> -Original Message-
> From: YouPeng Yang [mailto:yypvsxf19870...@gmail.com]
> Sent: Friday, August 23, 2013 1:16 AM
> To: solr-user@lucene.apache.org
> Subject: how to integrate solr with HDFS HA
>
> Hi all
> I try to integrate solr with HDFS HA.When I start the solr server, it
> comes out an exeception[1].
> And I do know this is because the hadoop.conf.Configuration  in
> HdfsDirectoryFactory.java does not include the HA configuration.
> So I want to know ,in solr,is there any way to include my hadoop  HA
> configuration ?
>
>
>
> [1]---
> Caused by: java.lang.IllegalArgumentException:
> java.net.UnknownHostException: lklcluster
> at
>
> org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:418)
> at
>
> org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:164)
> at
>
> org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129)
> at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:415)
> at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:382)
> at
>
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:123)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2277)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:86)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2311)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:2299)
> at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:364)
> at
> org.apache.solr.store.hdfs.HdfsDirectory.(HdfsDirectory.java:59)
> at
>
> org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:154)
> at
>
> org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:350)
> at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:256)
> at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:469)
> at org.apache.solr.core.SolrCore.(SolrCore.java:759)
>

Re: custom names for replicas in solrcloud

[YouPeng Yang]

Hi  smanad

   If I do not make a mistake, You can append the coreNodeName parameter to
your creation command:


http://10.7.23.125:8080/solr/admin/cores?action=CREATE&name=dfscore8_3&shard=shard3_3&collection.configName=myconf&schema=schema.xml&config=solrconfig3.xml&collection=collection1&dataDir=/soData/&;
coreNodeName=heihei

   May it be helpful


Regards



2013/8/23 smanad 

> Hi,
>
> I am using Solr 4.3 with 3 solr hosta and with an external zookeeper
> ensemble of 3 servers. And just 1 shard currently.
>
> When I create collections using collections api it creates collections with
> names,
> collection1_shard1_replica1, collection1_shard1_replica2,
> collection1_shard1_replica3.
> Is there any way to pass a custom name? or can I have all the replicas with
> same name?
>
> Any pointers will be much appreciated.
> Thanks,
> -Manasi
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/custom-names-for-replicas-in-solrcloud-tp4086205.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Failure to open existing log file On HDFS

[YouPeng Yang]

Hi solr user

   I'm testing the Solr with HDFS.I happend to stop my hdfs before stopping
the solr . After than I started the solr again. An exception come out[1]
that I could not ignore.
   Can anybody  explain the reason and how to avoid it?

Regards

[1]=
12326 [coreLoadExecutor-4-thread-1] ERROR org.apache.solr.update.UpdateLog
?.Failure to open existing log file (non fatal)
hdfs://lklcluster/solrdata/tlog/tlog.018:org.apache.solr.common.SolrException:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.RecoveryInProgressException):
Failed to close file /solrdata/tlog/tlog.018. Lease
recovery is in progress. Try again later.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2017)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1828)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInt(FSNamesystem.java:2069)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:2046)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.append(NameNodeRpcServer.java:445)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.append(ClientNamenodeProtocolServerSideTranslatorPB.java:285)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:40734)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1741)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1737)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1735)

at
org.apache.solr.update.HdfsTransactionLog.(HdfsTransactionLog.java:118)
at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:166)
at org.apache.solr.update.UpdateHandler.(UpdateHandler.java:134)
at org.apache.solr.update.UpdateHandler.(UpdateHandler.java:94)
at
org.apache.solr.update.DirectUpdateHandler2.(DirectUpdateHandler2.java:96)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:537)
at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:607)
at org.apache.solr.core.SolrCore.(SolrCore.java:819)
at org.apache.solr.core.SolrCore.(SolrCore.java:629)
at org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:270)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:655)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:364)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:356)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.RecoveryInProgressException):
Failed to close file /solrdata/tlog/tlog.018. Lease
recovery is in progress. Try again later.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2017)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1828)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInt(FSNamesystem.java:2069)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:2046)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.append(NameNodeRpcServer.java:445)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.append(ClientNamenodeProtocolServerSideTranslatorPB.java:285)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.ja

Re: solr cloud and DIH, indexation runs only on one shard.

[YouPeng Yang]

Hi jerome.dupont

  please check what is the updateHandler in your solrconfig.xml






 --> by
default，it is  solr.NoOpDistributingUpdateProcessorFactor











db-data-config.xml

sample



  





2013/9/3 

>
> Hello again,
>
> I still trying to index a with solr cloud and dih. I can index but it seems
> that indexation is done on only 1 shard. (my goal was to parallelze that to
> go fast)
> This my conf:
> I have 2 tomcat instances,
> One with zookeeper embedded in solr 4.4.0 started and 1 shard (port 8080)
> The other with the second shard. (port 9180)
> In my admin interface, I see 2 shards, each one is leader
>
>
> When I launch the dih, documents are indexed. But only the shard1 is
> working.
>
> http://localhost:8080/solr-0.4.0-pfd/noticesBIBcollection/dataimportMNb?command=full-import&entity=noticebib&optimize=true&indent=true&clean=true&commit=true&verbose=false&debug=false&wt=json&rows=1000
>
>
> In my first shard, I see messages coming from my indexation process:
> DEBUG 2013-09-03 11:48:57,801 Thread-12
> org.apache.solr.handler.dataimport.URLDataSource  (92) - Accessing URL:
> file:/X:/3/7/002/37002118.xml
> DEBUG 2013-09-03 11:48:57,832 Thread-12
> org.apache.solr.handler.dataimport.URLDataSource  (92) - Accessing URL:
> file:/X:/3/7/002/37002120.xml
> DEBUG 2013-09-03 11:48:57,966 Thread-12
> org.apache.solr.handler.dataimport.LogTransformer  (58) - Notice fichier:
> 3/7/002/37002120.xml
> DEBUG 2013-09-03 11:48:57,966 Thread-12 fr.bnf.solr.BnfDateTransformer
> (696) - NN=37002120
>
> In the second instance, I just have this kind of logs, at it was receiving
> notifications from zookeeper of new updates
> INFO 2013-09-03 11:48:57,323 http-9180-7
> org.apache.solr.update.processor.LogUpdateProcessor  (198) - [noticesBIB]
> webapp=/solr-0.4.0-pfd path=/update params=
> {distrib.from=
> http://172.20.48.237:8080/solr-0.4.0-pfd/noticesBIB/&update.distrib=TOLEADER&wt=javabin&version=2
> }
>  {add=[37001748 (1445149264874307584), 37001757 (1445149264879550464),
> 37001764 (1445149264883744768), 37001786 (1445149264887939072), 37001817
> (1445149264891084800), 37001819 (1445149264896327680), 37001837
> (1445149264900521984), 37001861 (1445149264903667712), 37001869
> (1445149264907862016), 37001963 (1445149264912056320)]} 0 41
>
> I supposed there was a confusion between cores names and collection name,
> and I tried to change the name of the collection, but it solved nothing.
> When I come to dih interfaces, in shard1, I see indexation processing, and
> on shard 2 "no information available"
>
> Is there something specia to do to distributre indexation process?
> Should I run zookeeper on both instances (even if it's not mandatory?
> ...
> Regards
> Jerome
>
>
>
> Fermeture annuelle des sites François-Mitterrand et Richelieu du 2 au 15
> septembre 2013 Avant d'imprimer, pensez à l'environnement.

Invalid Version when slave node pull replication from master node

[YouPeng Yang]

HI solrusers

   I'm testing the replication within SolrCloud .
   I just uncomment the replication section separately on the master and
slave node.
   The replication section setting on the  master node:

 commit
 startup
 schema.xml,stopwords.txt
   
 and on the slave node:
  
 http://10.7.23.124:8080/solr/#/
 00:00:50
   

   After startup, an Error comes out on the slave node :
80110110 [snapPuller-70-thread-1] ERROR org.apache.solr.handler.SnapPuller
?.Master at: http://10.7.23.124:8080/solr/#/ is not available. Index fetch
failed. Exception: Invalid version (expected 2, but 60) or the data in not
in 'javabin' format


 Could anyone help me to solve the problem ?


regards

Re: Invalid Version when slave node pull replication from master node

[YouPeng Yang]

Hi again

  I'm  using Solr4.4.


2013/9/5 YouPeng Yang 

> HI solrusers
>
>I'm testing the replication within SolrCloud .
>I just uncomment the replication section separately on the master and
> slave node.
>The replication section setting on the  master node:
> 
>  commit
>  startup
>  schema.xml,stopwords.txt
>
>  and on the slave node:
>   
>  http://10.7.23.124:8080/solr/#/
>  00:00:50
>
>
>After startup, an Error comes out on the slave node :
> 80110110 [snapPuller-70-thread-1] ERROR
> org.apache.solr.handler.SnapPuller  ?.Master at:
> http://10.7.23.124:8080/solr/#/ is not available. Index fetch failed.
> Exception: Invalid version (expected 2, but 60) or the data in not in
> 'javabin' format
>
>
>  Could anyone help me to solve the problem ?
>
>
> regards
>
>
>
>

Re: Invalid Version when slave node pull replication from master node

[YouPeng Yang]

Hi all
   I solve the problem by add the coreName explicitly according to
http://wiki.apache.org/solr/SolrReplication#Replicating_solrconfig.xml.

   But I want to make sure about that is it necessary to set the coreName
explicitly. Is there any SolrJ API to pull the replication on the slave
node from the master node?


regards



2013/9/5 YouPeng Yang 

> Hi again
>
>   I'm  using Solr4.4.
>
>
> 2013/9/5 YouPeng Yang 
>
>> HI solrusers
>>
>>I'm testing the replication within SolrCloud .
>>I just uncomment the replication section separately on the master and
>> slave node.
>>The replication section setting on the  master node:
>> 
>>  commit
>>  startup
>>  schema.xml,stopwords.txt
>>
>>  and on the slave node:
>>   
>>  http://10.7.23.124:8080/solr/#/
>>  00:00:50
>>
>>
>>After startup, an Error comes out on the slave node :
>> 80110110 [snapPuller-70-thread-1] ERROR
>> org.apache.solr.handler.SnapPuller  ?.Master at:
>> http://10.7.23.124:8080/solr/#/ is not available. Index fetch failed.
>> Exception: Invalid version (expected 2, but 60) or the data in not in
>> 'javabin' format
>>
>>
>>  Could anyone help me to solve the problem ?
>>
>>
>> regards
>>
>>
>>
>>
>

when does one core become 'down'

[YouPeng Yang]

Hi all
   In which situations,one core will become down?And how can I simulate
these situations


Any suggestions will be appreatiated.

Regards

create a core with explicit node_name

[YouPeng Yang]

Hi solr users

   I want to create a core with node_name through the api
CloudSolrServer.query(SolrParams params  ).
  For example:
  ModifiableSolrParams  params  = new ModifiableSolrParams();
params.set("qt", "/admin/cores");
params.set("action", "CREATE");
params.set("name", newcore.getName());
params.set("shard",  newcore.getShardname());
params.set("collection.configName",
newcore.getCollectionconfigname());
params.set("schema", newcore.getSchemaXMLFilename());
params.set("config", newcore.getSolrConfigFilename());
params.set("coreNodeName", newcore.getCorenodename());
params.set("node_name", "10.7.23.124:8080_solr");
params.set("collection", newcore.getCollectionname());

 The newcore encapsulats the create properties about the created core.


  It seems  not to work. As a result,the core was created on the other node.

  Do I need to send the  params directly  to the explicit web server -here
is the 10.7.23.124 - instead of using the CloudSolrServer.query(SolrParams
params  )?




regards

how to make sure all the index docs flushed to the index files

[YouPeng Yang]

Hi

   I'm using  the DIH to import data from  oracle database with Solr4.4
   Finally I get 2.7GB index data and 4.1GB tlog data.And the number of
docs was 1090.

  At first,  I move the 2.7GB index data to another new Solr Server in
tomcat7. After I start the tomcat ,I find the total number of docs was just
half of the orginal number.
  So I thought that maybe the left docs were not commited to index
files,and the  tlog needed to be replayed .

  Sequently , I moved the 2.7GB index data and 4.1GB tlog data to the new
Solr Server in tomcat7.
   After I start the tomcat,an exception comes up as [1].
   Then it halts.I can not access the tomcat server URL.
I noticed  that  the CPU utilization  was high by using the comand: top
-d 1 | grep tomcatPid.
I thought solr was replaying the updatelog.And I wait a long time and it
still was replaying. As results ,I give up.

   So I want to make sure after I finished the DIH import process ,whether
the whole index was flushed into the index data files. Is there any steps I
missed?
   How to make sure all the index were commited into the index files?.







[1]--
19380 [recoveryExecutor-6-thread-1] WARN  org.apache.solr.update.UpdateLog
?.REPLAY_ERR: Exception replaying log
java.lang.UnsupportedOperationException
at
org.apache.lucene.queries.function.FunctionValues.longVal(FunctionValues.java:46)
at
org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:200)
at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:736)
at
org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:183)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:672)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at
org.apache.solr.update.UpdateLog$LogReplayer.doReplay(UpdateLog.java:1313)
at org.apache.solr.update.UpdateLog$LogReplayer.run(UpdateLog.java:1202)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)

Re: how to make sure all the index docs flushed to the index files

[YouPeng Yang]

Hi  Shawn

   Thank your very much for your reponse.

   I lauch the full-import task on the web page of solr/admin . And I do
check the commit option.
The new docs would be committed after the operation.
  The commit option is defferent with the autocommit,right? If the import
datasets are too large that leads to poor performance or
other problems ,such as [1].

   The exception that indicate that -Too many open files-,we thought is
because of the ulimit.




[1]
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149d.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149e.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149f.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149g.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149h.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149i.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149j.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149k.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149l.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149m.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149n.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149o.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149p.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149q.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149r.fdx (Too
many open files)
java.io.FileNotFoundException:
/data/apache-tomcat/webapps/solr/collection1/data/index/_149s.fdx (Too
many open files)



2013/9/17 Shawn Heisey 

> On 9/16/2013 8:26 PM, YouPeng Yang wrote:
> >I'm using  the DIH to import data from  oracle database with Solr4.4
> >Finally I get 2.7GB index data and 4.1GB tlog data.And the number of
> > docs was 1090.
> >
> >   At first,  I move the 2.7GB index data to another new Solr Server in
> > tomcat7. After I start the tomcat ,I find the total number of docs was
> just
> > half of the orginal number.
> >   So I thought that maybe the left docs were not commited to index
> > files,and the  tlog needed to be replayed .
>
> You need to turn on autoCommit in your solrconfig.xml so that there are
> hard commits happening on a regular basis that flush all indexed data to
> disk and start new transaction log files.  I will give you a link with
> some information about that below.
>
> >   Sequently , I moved the 2.7GB index data and 4.1GB tlog data to the new
> > Solr Server in tomcat7.
> >After I start the tomcat,an exception comes up as [1].
> >Then it halts.I can not access the tomcat server URL.
> > I noticed  that  the CPU utilization  was high by using the comand:
> top
> > -d 1 | grep tomcatPid.
> > I thought solr was replaying the updatelog.And I wait a long time and it
> > still was replaying. As results ,I give up.
>
> I don't know what the exception was about, but it is likely that it WAS
> replaying the log.  With 4.1GB of transaction log, that's going to take
> a LONG time, during which Solr will be unavailable.  It always replays
> the entire transaction log.  The key, as mentioned above, is in keeping
> that log small.
>
> Here's a wiki page about the slow startup problem and an example of how
> to configure autoCommit to deal with it:
>
> http://wiki.apache.org/solr/SolrPerformanceProblems#Slow_startup
>
> There's a lot of other good information on that page.
>
> Thanks,
> Shawn
>
>

Re: how to make sure all the index docs flushed to the index files

[YouPeng Yang]

Hi
   Another werid problem.
   When we setup the autocommit properties, we  suppose that the index
fille will created every commited.So that the size of the index files will
be large enough. We do not want to keep too many small files as [1].

   How to control the size of the index files.

[1]
...omited 
548KBindex/_28w_Lucene41_0.doc
289KBindex/_28w_Lucene41_0.pos
1.1Mindex/_28w_Lucene41_0.tim
24Kindex/_28w_Lucene41_0.tip
2.1Mindex/_28w.fdt
766Bindex/_28w.fdx
5KBindex/_28w.fnm
40Kindex/_28w.nvd
79Kindex/_28w.nvm
364Bindex/_28w.si
518KBindex/_28x_Lucene41_0.doc
290KBindex/_28x_Lucene41_0.pos
1.2Mindex/_28x_Lucene41_0.tim
28Kindex/_28x_Lucene41_0.tip
2.1Mindex/_28x.fdt
843Bindex/_28x.fdx
5KBindex/_28x.fnm
40Kindex/_28x.nvd
79Kindex/_28x.nvm
386Bindex/_28x.si
...omited 
-





2013/9/17 YouPeng Yang 

> Hi  Shawn
>
>Thank your very much for your reponse.
>
>I lauch the full-import task on the web page of solr/admin . And I do
> check the commit option.
> The new docs would be committed after the operation.
>   The commit option is defferent with the autocommit,right? If the import
> datasets are too large that leads to poor performance or
> other problems ,such as [1].
>
>The exception that indicate that -Too many open files-,we thought is
> because of the ulimit.
>
>
>
>
>
> [1]
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149d.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149e.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149f.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149g.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149h.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149i.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149j.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149k.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149l.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149m.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149n.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149o.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149p.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149q.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149r.fdx (Too many 
> open files)
>
> java.io.FileNotFoundException: 
> /data/apache-tomcat/webapps/solr/collection1/data/index/_149s.fdx (Too many 
> open files)
>
>
>
> 2013/9/17 Shawn Heisey 
>
>> On 9/16/2013 8:26 PM, YouPeng Yang wrote:
>> >I'm using  the DIH to import data from  oracle database with Solr4.4
>> >Finally I get 2.7GB index data and 4.1GB tlog data.And the number of
>> > docs was 1090.
>> >
>> >   At first,  I move the 2.7GB index data to another new Solr Server in
>> > tomcat7. After I start the tomcat ,I find the total number of docs was
>> just
>> > half of the orginal number.
>> >   So I thought that maybe the left docs were not commited to index
>> > files,and the  tlog needed to be replayed .
>>
>> You need to turn on autoCommit in your solrconfig.xml so that there are
>> hard commits happening on a regular basis that flush all indexed data to
>> disk and start new transaction log files.  I will give you a link with
>> some information about that below.
>>
>> >   Sequently , I moved the 2.7GB index data and 4.1GB tlo

few and huge tlogs

[YouPeng Yang]

Hi
  According to
http://wiki.apache.org/solr/SolrPerformanceProblems#Slow_startup。
  It explains that the tlog file will swith to a new when hard commit
happened.


  However,my tlog shows different.
tlog.003   5.16GB
tlog.004   1.56GB
tlog.002   610.MB

  there are only a fewer tlogs which suppose to be ten files, and each one
is vary huge.Even there are lots of hard commit happened.

 So why the number of the tlog files does not increase ?


  here are settings of the  DirectUpdateHandler2:
 

  
  ${solr.ulog.dir:}
   

  
   120
   100
   false
 


  
 60
   50
   

Re: how to make sure all the index docs flushed to the index files

[YouPeng Yang]

Hi Erick and Shawn

   Thanks a lot


2013/9/17 Erick Erickson 

> Here's a blog about tlogs and commits:
>
> http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> And here's Mike's excellent segment merging blog
>
> http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html
>
> Best,
> Erick
>
>
> On Tue, Sep 17, 2013 at 6:36 AM, Shawn Heisey  wrote:
>
> > On 9/17/2013 12:32 AM, YouPeng Yang wrote:
> > > Hi
> > >Another werid problem.
> > >When we setup the autocommit properties, we  suppose that the index
> > > fille will created every commited.So that the size of the index files
> > will
> > > be large enough. We do not want to keep too many small files as [1].
> > >
> > >How to control the size of the index files.
> >
> > An index segment gets created after every hard commit.   In the listing
> > that you sent, all the files starting with _28w are a single segment.
> > All the files starting with _28x are another segment.
> >
> > Solr should be merging the segments when you get enough of them, unless
> > you have incorrectly set up your merge policy.  The default number of
> > segments that get merged is ten.  When you get ten segments, they will
> > be merged down to one.  This repeats until you have ten merged segments.
> >  At that point, those ten merged segments will be merged to make an even
> > larger segment.
> >
> > You can bump up the number of open files allowed by your operating
> > system.  On Linux, this is controlled by the /etc/security/limits.conf
> > file.  Here are some example config lines for that file:
> >
> > elyograghardnofile  6144
> > elyogragsoftnofile  4096
> > roothardnofile  6144
> > rootsoftnofile  4096
> >
> > Alternatively, you can reduce the required number of files if you turn
> > on the UseCompoundFile setting, which is in the IndexConfig section.
> > This causes Solr to create a single file per index segment instead of
> > several files per segment.  The compound file may be slightly less
> > efficient, but the difference is likely to be very small.
> >
> >
> https://cwiki.apache.org/confluence/display/solr/IndexConfig+in+SolrConfig
> >
> >
> >
>

import partition table from oracle

[YouPeng Yang]

Hi

  I want to import dataset in a partition of a partition table with DIH.
And I would like to explicitly define the partition when I do import job.

 To be specific.
  1. I define the DIH configuration like these


  2.I send the url:
  http://localhost:8983/solr/dataimport?command=full-import&part=p2

  and then the DIHhandler will full import the p2 partition of the table.

Any suggestion will be appreciated.


Regards.

import partition table from oracle

[YouPeng Yang]

Hi

  I want to import dataset in a partition of a partition table with DIH.
And I would like to explicitly define the partition when I do import job.

 To be specific.
  1. I define the DIH configuration like these


  2.I send the url:
  http://localhost:8983/solr/dataimport?command=full-import&part=p2

  and then the DIHhandler will full import the p2 partition of the table.

Any suggestion will be appreciated.


Regards.

Re: import partition table from oracle

[YouPeng Yang]

Hi  Shalin

 Thanks a lot. It is the point that I need.


Regards


2013/9/23 Shalin Shekhar Mangar 

> You can use request parameters in your query e.g.
>
> 
>
> http://wiki.apache.org/solr/DataImportHandler#Accessing_request_parameters
>
> On Mon, Sep 23, 2013 at 8:26 AM, YouPeng Yang 
> wrote:
> > Hi
> >
> >   I want to import dataset in a partition of a partition table with DIH.
> > And I would like to explicitly define the partition when I do import job.
> >
> >  To be specific.
> >   1. I define the DIH configuration like these
> > 
> >
> >   2.I send the url:
> >   http://localhost:8983/solr/dataimport?command=full-import&part=p2
> >
> >   and then the DIHhandler will full import the p2 partition of the table.
> >
> > Any suggestion will be appreciated.
> >
> >
> > Regards.
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

join datas from muliti collections

[YouPeng Yang]

Hi

   I have two collections with different schema.
   And I want to do inner join  like SQL:

  select A.xx,B.xx
  from  A,B
  where A.yy=B.yy

   How can I achieve this in Solr.  I'm using SolrCloud with solr 4.4


regards