If long periods between next invocations, up the scanner lease. See:
<property>
<name>hbase.regionserver.lease.period</name>
<value>60000</value>
<description>HRegion server lease period in milliseconds. Default is
60 seconds. Clients must report in within this period else they are
considered dead.</description>
</property>
St.Ack
On Tue, May 4, 2010 at 7:04 AM, Michelan Arendse <[email protected]> wrote:
> Yes I am waiting long periods between invocation of next. I didn't know that
> I am fetching too much data at once.
>
> I am using HBase 0.20.3. This is my code:
>
> scan.setTimeRange(fromDate.getTime(), toDate.getTime());
> ResultScanner scanner = table.getScanner(scan);
>
> while( (result = scanner.next()) != null) {
> channelRow = getChannelDeliveryRow(Bytes.toString(result.getRow()));
> channelRowList.add(channelRow);
> }
>
> This is some of the output from the log file:
> 2010-05-04 15:27:44,546 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Block cache LRU eviction started. Attempting to free 62791520 bytes
> 2010-05-04 15:27:44,552 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Block cache LRU eviction completed. Freed 62797944 bytes. Priority Sizes:
> Single=279.4997MB (293076672), Multi=224.35243MB (235250576),Memory=0.0MB (0)
>
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of Stack
> Sent: 04 May 2010 03:55 PM
> To: [email protected]
> Subject: Re: Improving HBase scanner
>
> Are you waiting too long between invocations of next? (i.e. > the
> scanner lease period?) Or, perhaps you are fetching too much in the
> one go. If you fetch 1000 at a time -- scanner caching -- and you
> don't get the next batch within the scanner lease period, again you
> will timeout.
>
> St.Ack
>
> On Tue, May 4, 2010 at 1:46 AM, Michelan Arendse <[email protected]>
> wrote:
>> Hi
>>
>> I would like to know how configure HBase to improve the scanner fetching
>> data from the table or another method of using scanner, as my database is
>> very large and scanner times out.
>>
>> Kind Regards,
>>
>> Michelan Arendse
>> Junior Developer | AD:DYNAMO // happy business ;-)
>> Office 0861 Dynamo (0861 396266) | Fax +27 (0) 21 465 2587
>>
>> Advertise Online Instantly - www.addynamo.com<http://www.addynamo.com>
>> <http://www.addynamo.com>
>>
>>
>
