Modified: kylin/site/feed.xml
URL: 
http://svn.apache.org/viewvc/kylin/site/feed.xml?rev=1864154&r1=1864153&r2=1864154&view=diff
==============================================================================
--- kylin/site/feed.xml (original)
+++ kylin/site/feed.xml Thu Aug  1 14:04:52 2019
@@ -19,11 +19,281 @@
     <description>Apache Kylin Home</description>
     <link>http://kylin.apache.org/</link>
     <atom:link href="http://kylin.apache.org/feed.xml"; rel="self" 
type="application/rss+xml"/>
-    <pubDate>Thu, 01 Aug 2019 04:10:17 -0700</pubDate>
-    <lastBuildDate>Thu, 01 Aug 2019 04:10:17 -0700</lastBuildDate>
+    <pubDate>Thu, 01 Aug 2019 06:52:56 -0700</pubDate>
+    <lastBuildDate>Thu, 01 Aug 2019 06:52:56 -0700</lastBuildDate>
     <generator>Jekyll v2.5.3</generator>
     
       <item>
+        <title>Detailed Analysis of refine query cache</title>
+        <description>&lt;hr /&gt;
+
+&lt;h2 id=&quot;part-i-basic-introduction&quot;&gt;Part-I Basic 
Introduction&lt;/h2&gt;
+
+&lt;h3 id=&quot;backgroud&quot;&gt;Backgroud&lt;/h3&gt;
+&lt;p&gt;In the past, query cache are not efficiently used in Kylin due to two 
aspects: &lt;strong&gt;coarse-grained cache expiration strategy&lt;/strong&gt; 
and &lt;strong&gt;lack of external cache&lt;/strong&gt;. Because of the 
aggressive cache expiration strategy, useful caches are often cleaned up 
unnecessarily. Because query caches are stored in local servers, they cannot be 
shared between servers. And because of the size limitation of local cache, not 
all useful query results can be cached.&lt;/p&gt;
+
+&lt;p&gt;To deal with these shortcomings, we change the query cache expiration 
strategy by signature checking and introduce the memcached as Kylin’s 
distributed cache so that Kylin servers are able to share cache between 
servers. And it’s easy to add memcached servers to scale out distributed 
cache.&lt;/p&gt;
+
+&lt;p&gt;These features is proposed and developed by eBay Kylin team. Thanks 
so much for their contribution.&lt;/p&gt;
+
+&lt;h3 id=&quot;related-jira&quot;&gt;Related JIRA&lt;/h3&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;&lt;a 
href=&quot;https://issues.apache.org/jira/browse/KYLIN-2895&quot;&gt;KYLIN-2895 
Refine Query Cache&lt;/a&gt;
+    &lt;ul&gt;
+      &lt;li&gt;&lt;a 
href=&quot;https://issues.apache.org/jira/browse/KYLIN-2899&quot;&gt;KYLIN-2899 
Introduce segment level query cache&lt;/a&gt;&lt;/li&gt;
+      &lt;li&gt;&lt;a 
href=&quot;https://issues.apache.org/jira/browse/KYLIN-2898&quot;&gt;KYLIN-2898 
Introduce memcached as a distributed cache for queries&lt;/a&gt;&lt;/li&gt;
+      &lt;li&gt;&lt;a 
href=&quot;https://issues.apache.org/jira/browse/KYLIN-2894&quot;&gt;KYLIN-2894 
Change the query cache expiration strategy by signature 
checking&lt;/a&gt;&lt;/li&gt;
+      &lt;li&gt;&lt;a 
href=&quot;https://issues.apache.org/jira/browse/KYLIN-2897&quot;&gt;KYLIN-2897 
Improve the query execution for a set of duplicate queries in a short 
period&lt;/a&gt;&lt;/li&gt;
+      &lt;li&gt;&lt;a 
href=&quot;https://issues.apache.org/jira/browse/KYLIN-2896&quot;&gt;KYLIN-2896 
Refine query exception cache&lt;/a&gt;&lt;/li&gt;
+    &lt;/ul&gt;
+  &lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;hr /&gt;
+
+&lt;h2 id=&quot;part-ii-deep-dive&quot;&gt;Part-II Deep Dive&lt;/h2&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;Introduce memcached as a Distributed Query Cache&lt;/li&gt;
+  &lt;li&gt;Segment Level Cache&lt;/li&gt;
+  &lt;li&gt;Query Cache Expiration Strategy by Signature Checking&lt;/li&gt;
+  &lt;li&gt;Other Enhancement&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;h3 
id=&quot;introduce-memcached-as-a-distributed-query-cache&quot;&gt;Introduce 
memcached as a Distributed Query Cache&lt;/h3&gt;
+
+&lt;p&gt;&lt;strong&gt;Memcached&lt;/strong&gt; is a Free and open source, 
high-performance, distributed memory object caching system. It is an in-memory 
key-value store for small chunks of arbitrary data (strings, objects) from 
results of database calls, API calls, or page rendering. It is simple yet 
powerful. Its simple design promotes quick deployment, ease of development, and 
solves many problems facing large data caches. Its API is available for most 
popular languages.&lt;/p&gt;
+
+&lt;p&gt;By KYLIN-2898, Kylin use &lt;strong&gt;Memcached&lt;/strong&gt; as 
distributed cache service, and use &lt;strong&gt;EhCache&lt;/strong&gt; as 
local cache service. When &lt;code 
class=&quot;highlighter-rouge&quot;&gt;RemoteLocalFailOverCacheManager&lt;/code&gt;
 is configured in &lt;code 
class=&quot;highlighter-rouge&quot;&gt;applicationContext.xml&lt;/code&gt;, for 
each cache put/get action, Kylin will first check if remote cache service is 
available, only if remote cache service is unavailable, local cache service 
will be used.&lt;/p&gt;
+
+&lt;p&gt;Firstly, multi query server can share query cache. For each kylin 
server, less jvm memory will be occupied which help to reduce GC pressure. 
Secondly, since memcached is centralized so duplicated cache entry will avoid 
in serval Kylin process. Thirdly, memcached has larger size and easy to scale 
out, this will help to reduce the chance which useful cache entry have to be 
dropped due to limited memory capacity.&lt;/p&gt;
+
+&lt;p&gt;To handle node failure and to scale out memcached cluster, author has 
introduced a consistent hash strategy to smoothly solve such problem. Ketama is 
an implementation of a consistent hashing algorithm, meaning you can add or 
remove servers from the memcached pool without causing a complete remap of all 
keys. Detail could be checked at &lt;a 
href=&quot;https://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-_a_consistent_hashing_algo_for_memcache_clients&quot;&gt;Ketama
 consistent hash strategy&lt;/a&gt;.&lt;/p&gt;
+
+&lt;p&gt;&lt;img 
src=&quot;/images/blog/refine-query-cache/consistent-hashing.png&quot; 
alt=&quot;consistent hashing&quot; /&gt;&lt;/p&gt;
+
+&lt;h3 id=&quot;segment-level-cache&quot;&gt;Segment level Cache&lt;/h3&gt;
+
+&lt;p&gt;Currently Kylin use sql as the cache key, when sql comes, if result 
exists in the cache, it will directly returned the cached result and don’t 
need to query hbase. When there is new segment build or existing segment 
refresh, all related cache result need to be evicted. For some frequently build 
cube such as streaming cube(NRT Streaming or Real-time OLAP), the cache miss 
will increase dramatically, that may decrease the query performance.&lt;/p&gt;
+
+&lt;p&gt;Since for Kylin cube, most historical segments are immutable, the 
same query against historical segments should be always same, don’t need to 
be evicted for new segment building. So we decide to implement the segment 
level cache, it is a complement of the existing front-end cache, the idea is 
similar as the level1/level2 cache in operating system.&lt;/p&gt;
+
+&lt;p&gt;&lt;img 
src=&quot;/images/blog/refine-query-cache/l1-l2-cache.png&quot; 
alt=&quot;l1-l2-cache&quot; /&gt;&lt;/p&gt;
+
+&lt;h3 
id=&quot;query-cache-expiration-strategy-by-signature-checking&quot;&gt;Query 
Cache Expiration Strategy by Signature Checking&lt;/h3&gt;
+
+&lt;p&gt;Currently, to invalid query cache, &lt;code 
class=&quot;highlighter-rouge&quot;&gt;CacheService&lt;/code&gt; will either 
invoke &lt;code 
class=&quot;highlighter-rouge&quot;&gt;cleanDataCache&lt;/code&gt; or &lt;code 
class=&quot;highlighter-rouge&quot;&gt;cleanAllDataCache&lt;/code&gt;. Both 
methods will clear all of the query cache , which is very inefficient and 
unnecessary. In production environment, there’s around hundreds of cubing 
jobs per day, which means the query cache will be cleared very several minutes. 
Then we introduced a signature to upgrade cache invalidation strategy.&lt;/p&gt;
+
+&lt;p&gt;The basic idea is as follows:&lt;br /&gt;
+When put SQLResponse into cache, we add signature for each SQLResponse. To 
calculate signature for SQLResponse, we choose the cube last build time and its 
segments to as input of &lt;code 
class=&quot;highlighter-rouge&quot;&gt;SignatureCalculator&lt;/code&gt;.&lt;br 
/&gt;
+When fetch &lt;code 
class=&quot;highlighter-rouge&quot;&gt;SQLResponse&lt;/code&gt; for cache, 
first check whether the signature is consistent. If not, this cached value is 
overdue and will be invalidate.&lt;/p&gt;
+
+&lt;p&gt;As for the calculation of signature is show as follows:&lt;br /&gt;
+1. &lt;code class=&quot;highlighter-rouge&quot;&gt;toString&lt;/code&gt; of 
&lt;code class=&quot;highlighter-rouge&quot;&gt;ComponentSignature&lt;/code&gt; 
will concatenate member varible into a large String; if a &lt;code 
class=&quot;highlighter-rouge&quot;&gt;ComponentSignature&lt;/code&gt; has 
other &lt;code 
class=&quot;highlighter-rouge&quot;&gt;ComponentSignature&lt;/code&gt; as 
member, toString will be calculated recursively&lt;br /&gt;
+2. return value of &lt;code 
class=&quot;highlighter-rouge&quot;&gt;toString&lt;/code&gt; will be input of 
&lt;code 
class=&quot;highlighter-rouge&quot;&gt;SignatureCalculator&lt;/code&gt;,&lt;br 
/&gt;
+&lt;code 
class=&quot;highlighter-rouge&quot;&gt;SignatureCalculator&lt;/code&gt; encode 
string using MD5 as identifer of signature of query cache&lt;/p&gt;
+
+&lt;p&gt;&lt;img 
src=&quot;/images/blog/refine-query-cache/cache-signature.png&quot; 
alt=&quot;cache-signature&quot; /&gt;&lt;/p&gt;
+
+&lt;h3 id=&quot;other-enhancement&quot;&gt;Other Enhancement&lt;/h3&gt;
+
+&lt;h4 
id=&quot;improve-the-query-execution-for-a-set-of-duplicate-queries-in-a-short-period&quot;&gt;Improve
 the query execution for a set of duplicate queries in a short period&lt;/h4&gt;
+
+&lt;p&gt;If same query enter Kylin at the same time by different client, for 
each query they can not find query cache so they must be calculated 
respectively. And even wrose, if these query are complex, they usually cost a 
long duration so Kylin have less chance to utilize cache query; and them cost 
large computation resources that will make query server has poor performance 
has harm to HBase cluster.&lt;/p&gt;
+
+&lt;p&gt;To reduce the impact of duplicated and complex query, it may be a 
good idea to block query which came later, wait to first one return result as 
far as possible. This lazy strategy is especially useful if you have duplicated 
complex query came in same time. To enbale it, you should set &lt;code 
class=&quot;highlighter-rouge&quot;&gt;kylin.query.lazy-query-enabled&lt;/code&gt;
 to &lt;code class=&quot;highlighter-rouge&quot;&gt;true&lt;/code&gt;. 
Optionlly, you may set &lt;code 
class=&quot;highlighter-rouge&quot;&gt;kylin.query.lazy-query-waiting-timeout-milliseconds&lt;/code&gt;
 to what you think later duplicated query wait duration to meet your 
situation.&lt;/p&gt;
+
+&lt;h4 id=&quot;remove-exception-cache&quot;&gt;Remove exception 
cache&lt;/h4&gt;
+&lt;p&gt;Formerly, query cache has been divided into two part, one part for 
storing success query result, another for failed query result, and they are 
invalidated respectively. It looks like not a good classification criteria 
because it is not fine-grained enough. After query cache signature was 
introduced, we have no reason to take them apart, so exception cache was 
removed.&lt;/p&gt;
+
+&lt;hr /&gt;
+
+&lt;h2 id=&quot;part-iii-how-to-use&quot;&gt;Part-III How to Use&lt;/h2&gt;
+
+&lt;p&gt;To get prepared, you need to install memcached, you may refer to 
https://github.com/memcached/memcached/wiki/Install. Then you should modify 
&lt;code class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; 
and &lt;code 
class=&quot;highlighter-rouge&quot;&gt;applicationContext.xml&lt;/code&gt;.&lt;/p&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;kylin.properties&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;div class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code 
class=&quot;language-groff&quot; 
data-lang=&quot;groff&quot;&gt;kylin.cache.memcached.hosts=10.1.2.42:11211
+kylin.query.cache-signature-enabled=true
+kylin.query.lazy-query-enabled=true
+kylin.metrics.memcached.enabled=true
+kylin.query.segment-cache-enabled=true&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;applicationContext.xml&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;div class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code 
class=&quot;language-groff&quot; 
data-lang=&quot;groff&quot;&gt;&amp;lt;cache:annotation-driven/&amp;gt;
+
+&amp;lt;bean id=&quot;ehcache&quot; 
class=&quot;org.springframework.cache.ehcache.EhCacheManagerFactoryBean&quot;
+      p:configLocation=&quot;classpath:ehcache-test.xml&quot; 
p:shared=&quot;true&quot;/&amp;gt;
+
+&amp;lt;bean id=&quot;remoteCacheManager&quot; 
class=&quot;org.apache.kylin.cache.cachemanager.MemcachedCacheManager&quot;/&amp;gt;
+&amp;lt;bean id=&quot;localCacheManager&quot; 
class=&quot;org.apache.kylin.cache.cachemanager.InstrumentedEhCacheCacheManager&quot;
+      p:cacheManager-ref=&quot;ehcache&quot;/&amp;gt;
+&amp;lt;bean id=&quot;cacheManager&quot; 
class=&quot;org.apache.kylin.cache.cachemanager.RemoteLocalFailOverCacheManager&quot;/&amp;gt;
+
+&amp;lt;bean id=&quot;memcachedCacheConfig&quot; 
class=&quot;org.apache.kylin.cache.memcached.MemcachedCacheConfig&quot;&amp;gt;
+    &amp;lt;property name=&quot;timeout&quot; value=&quot;500&quot;/&amp;gt;
+    &amp;lt;property name=&quot;hosts&quot; 
value=&quot;${kylin.cache.memcached.hosts}&quot;/&amp;gt;
+&amp;lt;/bean&amp;gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
+
+&lt;h3 id=&quot;configuration-for-query-cache&quot;&gt;Configuration for query 
cache&lt;/h3&gt;
+
+&lt;h4 id=&quot;general-part&quot;&gt;General part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.cache-enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;boolean, default 
true&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;whether to enable query 
cache&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.cache-threshold-duration&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;long, in milliseconds, 
default is 2000&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;query duration 
threshold&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.cache-threshold-scan-count&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;long, default is 
10240&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;query scan row count 
threshold&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.cache-threshold-scan-bytes&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;long, default is 1024 * 
1024 (1MB)&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;query scan byte 
threshold&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+
+&lt;h4 id=&quot;memcached-part&quot;&gt;Memcached part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.cache.memcached.hosts&lt;/td&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;host1:port1,host2:port2&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;host list of memcached 
host&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.segment-cache-enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default false&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;wether to enable&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.segment-cache-timeout&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default 2000&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;timeout of 
memcached&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.segment-cache-max-size&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;200 (MB)&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;max size put into 
memcached&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+
+&lt;h4 id=&quot;cache-signature-part&quot;&gt;Cache signature part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.cache-signature-enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default false&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;whether to use signature 
for query cache&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.signature-class&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default is 
org.apache.kylin.rest.signature.FactTableRealizationSetCalculator&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;use which class to 
calculate signature of query cache&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+
+&lt;h4 id=&quot;other-optimize-part&quot;&gt;Other optimize part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.lazy-query-enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;default false&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;whether to block duplicated 
sql query&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.query.lazy-query-waiting-timeout-milliseconds&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;long , in milliseconds, 
default is 60000&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;max druation for blocking 
duplicated sql query&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+
+&lt;h4 id=&quot;metrics-part&quot;&gt;Metrics part&lt;/h4&gt;
+
+&lt;table&gt;
+  &lt;thead&gt;
+    &lt;tr&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf Key&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Conf value&lt;/th&gt;
+      &lt;th style=&quot;text-align: left&quot;&gt;Explanation&lt;/th&gt;
+    &lt;/tr&gt;
+  &lt;/thead&gt;
+  &lt;tbody&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.metrics.memcached.enabled&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;true&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;Enable memcached metrics in 
memcached.&lt;/td&gt;
+    &lt;/tr&gt;
+    &lt;tr&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;kylin.metrics.memcached.metricstype&lt;/td&gt;
+      &lt;td style=&quot;text-align: 
left&quot;&gt;off/performance/debug&lt;/td&gt;
+      &lt;td style=&quot;text-align: left&quot;&gt;refer to 
net.spy.memcached.metrics.MetricType&lt;/td&gt;
+    &lt;/tr&gt;
+  &lt;/tbody&gt;
+&lt;/table&gt;
+</description>
+        <pubDate>Tue, 30 Jul 2019 03:30:00 -0700</pubDate>
+        
<link>http://kylin.apache.org/blog/2019/07/30/detailed-analysis-of-refine-query-cache/</link>
+        <guid 
isPermaLink="true">http://kylin.apache.org/blog/2019/07/30/detailed-analysis-of-refine-query-cache/</guid>
+        
+        
+        <category>blog</category>
+        
+      </item>
+    
+      <item>
         <title>Deep dive into Kylin&#39;s Real-time OLAP</title>
         <description>&lt;h2 id=&quot;preface&quot;&gt;Preface&lt;/h2&gt;
 
@@ -921,114 +1191,114 @@ Security: (depend on your security setti
       </item>
     
       <item>
-        <title>Apache Kylin v3.0.0-alpha Release Announcement</title>
-        <description>&lt;p&gt;The Apache Kylin community is pleased to 
announce the release of Apache Kylin v3.0.0-alpha.&lt;/p&gt;
+        <title>Apache Kylin v3.0.0-alpha 发布</title>
+        <description>&lt;p&gt;近日 Apache Kylin 社区很高å…
´åœ°å®£å¸ƒï¼ŒApache Kylin v3.0.0-alpha 正式发布。&lt;/p&gt;
 
-&lt;p&gt;Apache Kylin is an open source Distributed Analytics Engine designed 
to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data 
supporting extremely large datasets.&lt;/p&gt;
+&lt;p&gt;Apache Kylin 
是一个开源的分布式分析引擎,旨在为极大数据集提供 SQL 
接口和多维分析(OLAP)的能力。&lt;/p&gt;
 
-&lt;p&gt;This is the first release of the new generation v3.x, the main 
feature introduced is the Real-time OLAP. All of the changes can be found in 
the &lt;a href=&quot;/docs/release_notes.html&quot;&gt;release 
notes&lt;/a&gt;. Here we just highlight the main features.&lt;/p&gt;
+&lt;p&gt;这是 Kylin 下一代 v3.x 
的第一个发布版本,用于早期预览,主要的功能是实时 
(Real-time) OLAP。完整的改动列表请参见&lt;a 
href=&quot;/docs/release_notes.html&quot;&gt;release 
notes&lt;/a&gt;;这里挑一些主要改进做说明。&lt;/p&gt;
 
-&lt;h1 id=&quot;important-features&quot;&gt;Important features&lt;/h1&gt;
+&lt;h1 id=&quot;section&quot;&gt;重要新功能&lt;/h1&gt;
 
-&lt;h3 id=&quot;kylin-3654---real-time-olap&quot;&gt;KYLIN-3654 - Real-time 
OLAP&lt;/h3&gt;
-&lt;p&gt;With the newly introduced Kylin real-time receiver and coordinator 
components, Kylin can implement a millisecond-level data preparation delay for 
streaming data from sources like Apache Kafka. This means since v3.0 on,  Kylin 
can support sub-second level OLAP over historical batch data, near real-time 
streaming as well as real-time streaming. The user can use one OLAP platform to 
serve different scenarios. This solution has been deployed and verified in 
early adopters like eBay since 2018. For how to enable it, please refer to 
&lt;a href=&quot;/docs30/tutorial/realtime_olap.html&quot;&gt;this 
tutorial&lt;/a&gt;.&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3654----olap&quot;&gt;KYLIN-3654 - 实时 OLAP&lt;/h3&gt;
+&lt;p&gt;随着引入新的 real-time receiver 和 coordinator 组件,Kylin 
能够实现毫秒级别的数据准备延迟,数据源来自流式数据如 
Apache Kafka。这意味着,从 v3.0 开始,Kylin 
既能够支持历史批量数据的 
OLAP,也支持对流式数据的准实时(Near real-time)以及完å…
¨å®žæ—¶(real-time)分析。用户可以使用一个 OLAP 
平台来服务不同的使用场景。此方案已经在早期用户如 eBay 
得到部署和验证。关于如何使用此功能,请参考&lt;a 
href=&quot;/docs30/tutorial/realtime_olap.html&quot;&gt;æ­¤æ•
 ™ç¨‹&lt;/a&gt;。&lt;/p&gt;
 
-&lt;h3 
id=&quot;kylin-3795---submit-spark-jobs-via-apache-livy&quot;&gt;KYLIN-3795 - 
Submit Spark jobs via Apache Livy&lt;/h3&gt;
-&lt;p&gt;This feature allows the administrator to configure Kylin to integrate 
with Apache Livy (incubating) for Spark job submissions. The Spark job is 
submitted to the Livy Server through Livy’s REST API, instead of starting the 
Spark Driver process in local, which facilitates the management and monitoring 
of the Spark resources, and also releases the pressure of the nodes where the 
Kylin job server is running.&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3795----apache-livy--spark-&quot;&gt;KYLIN-3795 - 通过 
Apache Livy 递交 Spark 任务&lt;/h3&gt;
+&lt;p&gt;这个功能允许管理员为 Kylin 配置使用 Apache Livy 
(incubating) 来完成任务的递交。Spark 作业的提交通过 Livy 的 
REST API 来提交,而无需在本地启动 Spark Driver 
进程,从而方便对 Spark 资源的管理监控,同时也降低对 Kylin 
任务进程所在节点的压力。&lt;/p&gt;
 
-&lt;h3 id=&quot;kylin-3820---a-curator-based-job-scheduler&quot;&gt;KYLIN-3820 
- A curator-based job scheduler&lt;/h3&gt;
-&lt;p&gt;A new job scheduler is added to automatically discover the Kylin 
nodes and do an automatic leader selection among them (only the leader will 
submit jobs). With this feature, you can easily deploy and scale out Kylin 
nodes without manually update the node address in &lt;code 
class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; and 
restart Kylin to take effective.&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3820----curator-&quot;&gt;KYLIN-3820 - 基于 Curator 
的任务节点分配和服务发现&lt;/h3&gt;
+&lt;p&gt;新增一种基于Apache Zookeeper 和 
Curator作业调度器,可以自动发现 Kylin 节点,并自动分é…
ä¸€ä¸ªèŠ‚ç‚¹æ¥è¿›è¡Œä»»åŠ¡çš„ç®¡ç†ä»¥åŠæ•…
障恢复。有了这个功能后,管理员可以更加
容易地部署和扩展 Kylin 节点,而不再需要在 &lt;code 
class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; 中é…
ç½®æ¯ä¸ª Kylin 节点的地址并重启 Kylin 以使之生效。&lt;/p&gt;
 
-&lt;h1 id=&quot;other-enhancements&quot;&gt;Other enhancements&lt;/h1&gt;
+&lt;h1 id=&quot;section-1&quot;&gt;其它改进&lt;/h1&gt;
 
-&lt;h3 
id=&quot;kylin-3716---fastthreadlocal-replaces-threadlocal&quot;&gt;KYLIN-3716 
- FastThreadLocal replaces ThreadLocal&lt;/h3&gt;
-&lt;p&gt;Using FastThreadLocal instead of ThreadLocal can improve Kylin’s 
overall performance to some extent.&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3716---fastthreadlocal--threadlocal&quot;&gt;KYLIN-3716 
- FastThreadLocal 替换 ThreadLocal&lt;/h3&gt;
+&lt;p&gt;使用 Netty 中的 FastThreadLocal 替代 JDK 原生的 
ThreadLocal,可以一定程度上提升 Kylin 
在高并发下的性能。&lt;/p&gt;
 
 &lt;h3 
id=&quot;kylin-3867---enable-jdbc-to-use-key-store--trust-store-for-https-connection&quot;&gt;KYLIN-3867
 - Enable JDBC to use key store &amp;amp; trust store for https 
connection&lt;/h3&gt;
-&lt;p&gt;By using HTTPS, the authentication information used by JDBC is 
protected, making Kylin more secure.&lt;/p&gt;
+&lt;p&gt;通过使用HTTPS,保护了JDBC使用的身份验证信息,使得Kylin更åŠ
 å®‰å…¨&lt;/p&gt;
 
 &lt;h3 
id=&quot;kylin-3905---enable-shrunken-dictionary-default&quot;&gt;KYLIN-3905 - 
Enable shrunken dictionary default&lt;/h3&gt;
-&lt;p&gt;By default, the shrunken dictionary is enabled, and the precise 
counting scene for high cardinal dimensions can significantly reduce the build 
time.&lt;/p&gt;
+&lt;p&gt;默认开启 shrunken 
dictionary,针对高基维进行精确去重的场景,可以显著减少构建用时。&lt;/p&gt;
 
 &lt;h3 
id=&quot;kylin-3839---storage-clean-up-after-the-refreshing-and-deleting-a-segment&quot;&gt;KYLIN-3839
 - Storage clean up after the refreshing and deleting a segment&lt;/h3&gt;
-&lt;p&gt;Clear unnecessary data files in a timely manner&lt;/p&gt;
+&lt;p&gt;更加及时地清除不必要的数据文件&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;Download&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;下载&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;To download Apache Kylin v3.0.0-alpha source code or binary package, 
visit the &lt;a 
href=&quot;http://kylin.apache.org/download&quot;&gt;download&lt;/a&gt; 
page.&lt;/p&gt;
+&lt;p&gt;要下载Apache Kylin 源代码或二进制包,请访问&lt;a 
href=&quot;/download&quot;&gt;下载页面&lt;/a&gt; page.&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;Upgrade&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;升级&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;Follow the &lt;a 
href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;upgrade 
guide&lt;/a&gt;.&lt;/p&gt;
+&lt;p&gt;参考&lt;a 
href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;升级指南&lt;/a&gt;.&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;Feedback&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;反馈&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;If you face issue or question, please send mail to Apache Kylin dev 
or user mailing list: d...@kylin.apache.org , u...@kylin.apache.org; Before 
sending, please make sure you have subscribed the mailing list by dropping an 
email to dev-subscr...@kylin.apache.org or 
user-subscr...@kylin.apache.org.&lt;/p&gt;
+&lt;p&gt;如果您遇到问题或疑问,请发送邮件至 Apache Kylin dev 
或 user 邮件列表:d...@kylin.apache.org,u...@kylin.apache.org; 
在发送之前,请确保您已通过发送电子邮件至 
dev-subscr...@kylin.apache.org 或 user-subscr...@kylin.apache.org 订阅
了邮件列表。&lt;/p&gt;
 
-&lt;p&gt;&lt;em&gt;Great thanks to everyone who 
contributed!&lt;/em&gt;&lt;/p&gt;
+&lt;p&gt;&lt;em&gt;非常感谢所有贡献Apache 
Kylin的朋友!&lt;/em&gt;&lt;/p&gt;
 </description>
         <pubDate>Fri, 19 Apr 2019 13:00:00 -0700</pubDate>
-        
<link>http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</link>
-        <guid 
isPermaLink="true">http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</guid>
+        
<link>http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</link>
+        <guid 
isPermaLink="true">http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</guid>
         
         
-        <category>blog</category>
+        <category>cn_blog</category>
         
       </item>
     
       <item>
-        <title>Apache Kylin v3.0.0-alpha 发布</title>
-        <description>&lt;p&gt;近日 Apache Kylin 社区很高å…
´åœ°å®£å¸ƒï¼ŒApache Kylin v3.0.0-alpha 正式发布。&lt;/p&gt;
+        <title>Apache Kylin v3.0.0-alpha Release Announcement</title>
+        <description>&lt;p&gt;The Apache Kylin community is pleased to 
announce the release of Apache Kylin v3.0.0-alpha.&lt;/p&gt;
 
-&lt;p&gt;Apache Kylin 
是一个开源的分布式分析引擎,旨在为极大数据集提供 SQL 
接口和多维分析(OLAP)的能力。&lt;/p&gt;
+&lt;p&gt;Apache Kylin is an open source Distributed Analytics Engine designed 
to provide SQL interface and multi-dimensional analysis (OLAP) on Big Data 
supporting extremely large datasets.&lt;/p&gt;
 
-&lt;p&gt;这是 Kylin 下一代 v3.x 
的第一个发布版本,用于早期预览,主要的功能是实时 
(Real-time) OLAP。完整的改动列表请参见&lt;a 
href=&quot;/docs/release_notes.html&quot;&gt;release 
notes&lt;/a&gt;;这里挑一些主要改进做说明。&lt;/p&gt;
+&lt;p&gt;This is the first release of the new generation v3.x, the main 
feature introduced is the Real-time OLAP. All of the changes can be found in 
the &lt;a href=&quot;/docs/release_notes.html&quot;&gt;release 
notes&lt;/a&gt;. Here we just highlight the main features.&lt;/p&gt;
 
-&lt;h1 id=&quot;section&quot;&gt;重要新功能&lt;/h1&gt;
+&lt;h1 id=&quot;important-features&quot;&gt;Important features&lt;/h1&gt;
 
-&lt;h3 id=&quot;kylin-3654----olap&quot;&gt;KYLIN-3654 - 实时 OLAP&lt;/h3&gt;
-&lt;p&gt;随着引入新的 real-time receiver 和 coordinator 组件,Kylin 
能够实现毫秒级别的数据准备延迟,数据源来自流式数据如 
Apache Kafka。这意味着,从 v3.0 开始,Kylin 
既能够支持历史批量数据的 
OLAP,也支持对流式数据的准实时(Near real-time)以及完å…
¨å®žæ—¶(real-time)分析。用户可以使用一个 OLAP 
平台来服务不同的使用场景。此方案已经在早期用户如 eBay 
得到部署和验证。关于如何使用此功能,请参考&lt;a 
href=&quot;/docs30/tutorial/realtime_olap.html&quot;&gt;æ­¤æ•
 ™ç¨‹&lt;/a&gt;。&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3654---real-time-olap&quot;&gt;KYLIN-3654 - Real-time 
OLAP&lt;/h3&gt;
+&lt;p&gt;With the newly introduced Kylin real-time receiver and coordinator 
components, Kylin can implement a millisecond-level data preparation delay for 
streaming data from sources like Apache Kafka. This means since v3.0 on,  Kylin 
can support sub-second level OLAP over historical batch data, near real-time 
streaming as well as real-time streaming. The user can use one OLAP platform to 
serve different scenarios. This solution has been deployed and verified in 
early adopters like eBay since 2018. For how to enable it, please refer to 
&lt;a href=&quot;/docs30/tutorial/realtime_olap.html&quot;&gt;this 
tutorial&lt;/a&gt;.&lt;/p&gt;
 
-&lt;h3 id=&quot;kylin-3795----apache-livy--spark-&quot;&gt;KYLIN-3795 - 通过 
Apache Livy 递交 Spark 任务&lt;/h3&gt;
-&lt;p&gt;这个功能允许管理员为 Kylin 配置使用 Apache Livy 
(incubating) 来完成任务的递交。Spark 作业的提交通过 Livy 的 
REST API 来提交,而无需在本地启动 Spark Driver 
进程,从而方便对 Spark 资源的管理监控,同时也降低对 Kylin 
任务进程所在节点的压力。&lt;/p&gt;
+&lt;h3 
id=&quot;kylin-3795---submit-spark-jobs-via-apache-livy&quot;&gt;KYLIN-3795 - 
Submit Spark jobs via Apache Livy&lt;/h3&gt;
+&lt;p&gt;This feature allows the administrator to configure Kylin to integrate 
with Apache Livy (incubating) for Spark job submissions. The Spark job is 
submitted to the Livy Server through Livy’s REST API, instead of starting the 
Spark Driver process in local, which facilitates the management and monitoring 
of the Spark resources, and also releases the pressure of the nodes where the 
Kylin job server is running.&lt;/p&gt;
 
-&lt;h3 id=&quot;kylin-3820----curator-&quot;&gt;KYLIN-3820 - 基于 Curator 
的任务节点分配和服务发现&lt;/h3&gt;
-&lt;p&gt;新增一种基于Apache Zookeeper 和 
Curator作业调度器,可以自动发现 Kylin 节点,并自动分é…
ä¸€ä¸ªèŠ‚ç‚¹æ¥è¿›è¡Œä»»åŠ¡çš„ç®¡ç†ä»¥åŠæ•…
障恢复。有了这个功能后,管理员可以更加
容易地部署和扩展 Kylin 节点,而不再需要在 &lt;code 
class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; 中é…
ç½®æ¯ä¸ª Kylin 节点的地址并重启 Kylin 以使之生效。&lt;/p&gt;
+&lt;h3 id=&quot;kylin-3820---a-curator-based-job-scheduler&quot;&gt;KYLIN-3820 
- A curator-based job scheduler&lt;/h3&gt;
+&lt;p&gt;A new job scheduler is added to automatically discover the Kylin 
nodes and do an automatic leader selection among them (only the leader will 
submit jobs). With this feature, you can easily deploy and scale out Kylin 
nodes without manually update the node address in &lt;code 
class=&quot;highlighter-rouge&quot;&gt;kylin.properties&lt;/code&gt; and 
restart Kylin to take effective.&lt;/p&gt;
 
-&lt;h1 id=&quot;section-1&quot;&gt;其它改进&lt;/h1&gt;
+&lt;h1 id=&quot;other-enhancements&quot;&gt;Other enhancements&lt;/h1&gt;
 
-&lt;h3 id=&quot;kylin-3716---fastthreadlocal--threadlocal&quot;&gt;KYLIN-3716 
- FastThreadLocal 替换 ThreadLocal&lt;/h3&gt;
-&lt;p&gt;使用 Netty 中的 FastThreadLocal 替代 JDK 原生的 
ThreadLocal,可以一定程度上提升 Kylin 
在高并发下的性能。&lt;/p&gt;
+&lt;h3 
id=&quot;kylin-3716---fastthreadlocal-replaces-threadlocal&quot;&gt;KYLIN-3716 
- FastThreadLocal replaces ThreadLocal&lt;/h3&gt;
+&lt;p&gt;Using FastThreadLocal instead of ThreadLocal can improve Kylin’s 
overall performance to some extent.&lt;/p&gt;
 
 &lt;h3 
id=&quot;kylin-3867---enable-jdbc-to-use-key-store--trust-store-for-https-connection&quot;&gt;KYLIN-3867
 - Enable JDBC to use key store &amp;amp; trust store for https 
connection&lt;/h3&gt;
-&lt;p&gt;通过使用HTTPS,保护了JDBC使用的身份验证信息,使得Kylin更åŠ
 å®‰å…¨&lt;/p&gt;
+&lt;p&gt;By using HTTPS, the authentication information used by JDBC is 
protected, making Kylin more secure.&lt;/p&gt;
 
 &lt;h3 
id=&quot;kylin-3905---enable-shrunken-dictionary-default&quot;&gt;KYLIN-3905 - 
Enable shrunken dictionary default&lt;/h3&gt;
-&lt;p&gt;默认开启 shrunken 
dictionary,针对高基维进行精确去重的场景,可以显著减少构建用时。&lt;/p&gt;
+&lt;p&gt;By default, the shrunken dictionary is enabled, and the precise 
counting scene for high cardinal dimensions can significantly reduce the build 
time.&lt;/p&gt;
 
 &lt;h3 
id=&quot;kylin-3839---storage-clean-up-after-the-refreshing-and-deleting-a-segment&quot;&gt;KYLIN-3839
 - Storage clean up after the refreshing and deleting a segment&lt;/h3&gt;
-&lt;p&gt;更加及时地清除不必要的数据文件&lt;/p&gt;
+&lt;p&gt;Clear unnecessary data files in a timely manner&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;下载&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;Download&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;要下载Apache Kylin 源代码或二进制包,请访问&lt;a 
href=&quot;/download&quot;&gt;下载页面&lt;/a&gt; page.&lt;/p&gt;
+&lt;p&gt;To download Apache Kylin v3.0.0-alpha source code or binary package, 
visit the &lt;a 
href=&quot;http://kylin.apache.org/download&quot;&gt;download&lt;/a&gt; 
page.&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;升级&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;Upgrade&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;参考&lt;a 
href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;升级指南&lt;/a&gt;.&lt;/p&gt;
+&lt;p&gt;Follow the &lt;a 
href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;upgrade 
guide&lt;/a&gt;.&lt;/p&gt;
 
-&lt;p&gt;&lt;strong&gt;反馈&lt;/strong&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;Feedback&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;如果您遇到问题或疑问,请发送邮件至 Apache Kylin dev 
或 user 邮件列表:d...@kylin.apache.org,u...@kylin.apache.org; 
在发送之前,请确保您已通过发送电子邮件至 
dev-subscr...@kylin.apache.org 或 user-subscr...@kylin.apache.org 订阅
了邮件列表。&lt;/p&gt;
+&lt;p&gt;If you face issue or question, please send mail to Apache Kylin dev 
or user mailing list: d...@kylin.apache.org , u...@kylin.apache.org; Before 
sending, please make sure you have subscribed the mailing list by dropping an 
email to dev-subscr...@kylin.apache.org or 
user-subscr...@kylin.apache.org.&lt;/p&gt;
 
-&lt;p&gt;&lt;em&gt;非常感谢所有贡献Apache 
Kylin的朋友!&lt;/em&gt;&lt;/p&gt;
+&lt;p&gt;&lt;em&gt;Great thanks to everyone who 
contributed!&lt;/em&gt;&lt;/p&gt;
 </description>
         <pubDate>Fri, 19 Apr 2019 13:00:00 -0700</pubDate>
-        
<link>http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</link>
-        <guid 
isPermaLink="true">http://kylin.apache.org/cn_blog/2019/04/19/release-v3.0.0-alpha/</guid>
+        
<link>http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</link>
+        <guid 
isPermaLink="true">http://kylin.apache.org/blog/2019/04/19/release-v3.0.0-alpha/</guid>
         
         
-        <category>cn_blog</category>
+        <category>blog</category>
         
       </item>
     
@@ -1185,85 +1455,6 @@ The checkpoint info is the smallest part
       </item>
     
       <item>
-        <title>Apache Kylin v2.6.0 正式发布</title>
-        <description>&lt;p&gt;近日Apache Kylin 社区很高å…
´åœ°å®£å¸ƒï¼ŒApache Kylin 2.6.0 正式发布。&lt;/p&gt;
-
-&lt;p&gt;Apache Kylin 
是一个开源的分布式分析引擎,旨在为极大数据集提供 SQL 
接口和多维分析(OLAP)的能力。&lt;/p&gt;
-
-&lt;p&gt;这是继2.5.0 后的一个新功能版本。该版本引å…
¥äº†å¾ˆå¤šæœ‰ä»·å€¼çš„æ”¹è¿›ï¼Œå®Œæ•´çš„æ”¹åŠ¨åˆ—è¡¨è¯·å‚è§&lt;a 
href=&quot;https://kylin.apache.org/docs/release_notes.html&quot;&gt;release 
notes&lt;/a&gt;;这里挑一些主要改进做说明:&lt;/p&gt;
-
-&lt;h3 id=&quot;jdbcsdk&quot;&gt;针对以JDBC为数据源的SDK&lt;/h3&gt;
-&lt;p&gt;Kylin目前已经支持通过JDBC连接包括Amazon Redshift, SQL 
Server在内的多种数据源。&lt;br /&gt;
-为了便于开发者更便利地处理各种 SQL 方言(dialect) 
的不同以更加简单地开发新的 JDBC 数据源,Kylin 
提供了相应的 SDK 和统一的 API 入口:&lt;br /&gt;
-* 同步元数据和数据&lt;br /&gt;
-* 构建 cube&lt;br /&gt;
-* 
当找不到相应的cube来解答查询时,下推查询到数据源&lt;/p&gt;
-
-&lt;p&gt;更多内容参见 KYLIN-3552。&lt;/p&gt;
-
-&lt;h3 id=&quot;memcached--kylin-&quot;&gt;使用 Memcached 作 Kylin 
的分布式缓存&lt;/h3&gt;
-&lt;p&gt;在过去,Kylin 
对查询结果的缓存不是十分高效,主要有以下两个方面的原å›
 ï¼š&lt;br /&gt;
-一个是当 Kylin 的 metadata 发生变化时,会主动盲目地去清
除大量缓存,使得缓存会被频繁刷新而导致利用率降低。&lt;br
 /&gt;
-另一点是由于只使用本地缓存而导致 Kylin server 之间不能å…
±äº«å½¼æ­¤çš„缓存,这样查询的缓存命中率就会降低。&lt;br /&gt;
-本地缓存的一个缺点是大小受到限制,不能像分布式缓存那æ
 ·æ°´å¹³æ‰©å±•。这æ 
·å¯¼è‡´èƒ½ç¼“存的查询结果量受到了限制。&lt;/p&gt;
-
-&lt;p&gt;针对这些缺陷,我们改变了缓存失效的机制,不再主动去æ¸
…理缓存,而是采取如下的方案:&lt;br /&gt;
-1. 在将查询结果放入缓存之前,根据当前的å…
ƒæ•°æ®ä¿¡æ¯è®¡ç®—一个数字签名,并与查询结果一同放å…
¥ç¼“存中&lt;br /&gt;
-2. 从缓存中获取查询结果之后,根据当前的å…
ƒæ•°æ®ä¿¡æ¯è®¡ç®—一个数字签名,对比两者
的数字签名是否一致。如果一致,那么缓存有效;反之,该缓存失效并åˆ
 é™¤&lt;/p&gt;
-
-&lt;p&gt;我们还引入了 Memcached 作为 Kylin 的分布式缓存。这样 
Kylin server 之间可以共享查询结果的缓存,而且由于 Memcached 
server 之间的独立性,非常易于水平拓展,更加
有利于缓存更多的数据。&lt;br /&gt;
-相关开发任务是 KYLIN-2895, KYLIN-2894, KYLIN-2896, KYLIN-2897, 
KYLIN-2898, KYLIN-2899。&lt;/p&gt;
-
-&lt;h3 id=&quot;forkjoinpool--fast-cubing-&quot;&gt;ForkJoinPool 简化 fast 
cubing 的线程模型&lt;/h3&gt;
-&lt;p&gt;在过去进行 fast cubing 时,Kylin 
使用自己定义的一系列线程,如 split 线程,task 线程,main 
线程等等进行并发的 cube 构建。&lt;br /&gt;
-在这个线程模型中,线程之间的å…
³ç³»ååˆ†çš„复杂,而且对异常处理也十分容易出错。&lt;/p&gt;
-
-&lt;p&gt;现在我们引入了 ForkJoinPool,在主线程中处理 split 
逻辑,构建 cuboid 的任务以及子任务都在 fork join 
pool中执行,cuboid 
构建的结果可以被异步的收集并且可以更早地输出给下游的 
merge 操作。更多内容参见 KYLIN-2932。&lt;/p&gt;
-
-&lt;h3 id=&quot;hllcounter-&quot;&gt;改进 HLLCounter 的性能&lt;/h3&gt;
-&lt;p&gt;对于 HLLCounter, 我们从两方面进行了改进:构建 
HLLCounter 和计算调和平均的方式。&lt;br /&gt;
-1. 关于 HLLCounter 
的构建,我们不再使用merge的方式,而是直接copy别的HLLCounter里面的registers&lt;br
 /&gt;
-2. 关于计算 HLLCSnapshot 
里面的调和平均,做了以下三个方面的改进:&lt;br /&gt;
-* 缓存所有的1/2^r&lt;br /&gt;
-* 使用整型相加代替浮点型相加&lt;br /&gt;
-* 删除条件分支,例如无需检查 registers[i] 是不是为0&lt;/p&gt;
-
-&lt;p&gt;更多内容参见 KYLIN-3656。&lt;/p&gt;
-
-&lt;h3 id=&quot;cube-planner-&quot;&gt;改进 Cube Planner 算法&lt;/h3&gt;
-&lt;p&gt;在过去,cube planner 的 phase two 增加未被预计算的 
cuboid 的方式只能通过 mandatory cuboid 的方式。而一个 cuboid 
是否为 mandatory,又有两种方式:&lt;br /&gt;
-手动设置,查询时 rollup 
的行数足够大。这里通过判断查询时 rollup 
的行数是否足够大来判断是否为 mandatory cuboid 
的方式有两大缺陷:&lt;br /&gt;
-* 一是估算 rollup 的行数的算法不是很好&lt;br /&gt;
-* 二是很难设立一个静态的阈值来做判定&lt;/p&gt;
-
-&lt;p&gt;现在我们不再从 rollup 
行数的角度看问题了。一切都是从 cuboid 
行数的角度看问题,这样就和 cost based 的 cube planner 
算法做了统一。&lt;br /&gt;
-为此我们通过使用 rollup 比率来改进了未被预先构建的 cuboid 
的行数的估算,然后让 cost based 的 cube planner 
算法来判定哪些未被构建的 cuboid 
该被构建,哪些该被遗弃。&lt;br /&gt;
-通过这样的改进,无需通过设定静态的阈值来推荐 mandatory 
cuboid 了,而 mandatory cuboid 
只能被手动设置,不能被推荐了。更多内容参见 
KYLIN-3540。&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;下载&lt;/strong&gt;&lt;/p&gt;
-
-&lt;p&gt;要下载Apache Kylin v2.6.0源代码或二进制包,请访问&lt;a 
href=&quot;http://kylin.apache.org/download&quot;&gt;下载页面&lt;/a&gt; 
.&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;升级&lt;/strong&gt;&lt;/p&gt;
-
-&lt;p&gt;参考&lt;a 
href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;升级指南&lt;/a&gt;.&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;反馈&lt;/strong&gt;&lt;/p&gt;
-
-&lt;p&gt;如果您遇到问题或疑问,请发送邮件至 Apache Kylin dev 
或 user 邮件列表:d...@kylin.apache.org,u...@kylin.apache.org; 
在发送之前,请确保您已通过发送电子邮件至 
dev-subscr...@kylin.apache.org 或 user-subscr...@kylin.apache.org订阅
了邮件列表。&lt;/p&gt;
-
-&lt;p&gt;&lt;em&gt;非常感谢所有贡献Apache 
Kylin的朋友!&lt;/em&gt;&lt;/p&gt;
-</description>
-        <pubDate>Fri, 18 Jan 2019 12:00:00 -0800</pubDate>
-        <link>http://kylin.apache.org/cn_blog/2019/01/18/release-v2.6.0/</link>
-        <guid 
isPermaLink="true">http://kylin.apache.org/cn_blog/2019/01/18/release-v2.6.0/</guid>
-        
-        
-        <category>cn_blog</category>
-        
-      </item>
-    
-      <item>
         <title>Apache Kylin v2.6.0 Release Announcement</title>
         <description>&lt;p&gt;The Apache Kylin community is pleased to 
announce the release of Apache Kylin v2.6.0.&lt;/p&gt;
 
@@ -1342,274 +1533,81 @@ By this improvement, we don’t need
       </item>
     
       <item>
-        <title>How Cisco&#39;s Big Data Team Improved the High Concurrent 
Throughput of Apache Kylin by 5x</title>
-        <description>&lt;h2 id=&quot;background&quot;&gt;Background&lt;/h2&gt;
-
-&lt;p&gt;As part of the development group of Cisco’s Big Data team, one of 
our responsibilities is to provide BI reports to our stakeholders. Stakeholders 
rely on the reporting system to check the usage of Cisco’s business 
offerings. These reports are also used as a reference for billing, so they are 
critical to our stakeholders and the business overall.&lt;/p&gt;
-
-&lt;p&gt;The raw data for these reports is sourced across multiple tables in 
our Oracle database. The monthly data volume for one table is in the billions, 
and if a customer wants to run a report for one year, at least one billion to 
two billion rows of data need to be aggregated or processed through other 
operations. Additionally, all results need to be provided in a short amount of 
time. In the course of our research, we discovered Apache Kylin, a distributed 
preprocessing engine for massive datasets based on pre-calculation, which 
enables you to query those massive datasets at sub-second latency.&lt;/p&gt;
-
-&lt;p&gt;With the simulation test using our production data, we found that 
Kylin was ideal for our needs and was indeed capable of providing aggregated 
results on one billion rows of data in one second. However, we still needed to 
undergo additional tests for another use case. For one stakeholder, we provide 
15 charts displayed on a single page. The BI system will send REST API requests 
to Kylin to query the data for each chart asynchronously. Based on the 
production data volume, if there are 20 stakeholders viewing the report on one 
node, they will trigger 15*20 = 300 requests. The high concurrent query 
performance of Kylin is what we needed to test here.&lt;/p&gt;
-
-&lt;h2 id=&quot;the-testing-stage&quot;&gt;The Testing Stage&lt;/h2&gt;
-
-&lt;p&gt;&lt;strong&gt;Precondition&lt;/strong&gt;: To reduce the impact from 
network cost, we deployed the testing tools in the same network environment 
with Kylin. Meanwhile, we turned off the query cache for Kylin to make sure 
that each request was executed on the bottom layer.&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;Testing Tools&lt;/strong&gt;: Aside from our 
traditional testing tool, Apache Jmeter, we also used another open source tool: 
Gatlin (&lt;a 
href=&quot;https://gatling.io/&quot;&gt;https://gatling.io/&lt;/a&gt;) to test 
the same case. We excluded the impact from the tools.&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;Testing Strategy&lt;/strong&gt;: We simulated user 
requests of different sizes by increasing the number of concurrent threads, 
tracking the average response in 60 seconds, finding the bottleneck for Kylin 
query responses, and observing the maximum response time and success 
rate.&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;Testing Results&lt;/strong&gt;:&lt;/p&gt;
-
-&lt;table&gt;
-  &lt;thead&gt;
-    &lt;tr&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Thread&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Handled Queries (in 60 
seconds)&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Handled Queries (per 
second)&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Mean Response Time 
(ms)&lt;/th&gt;
-    &lt;/tr&gt;
-  &lt;/thead&gt;
-  &lt;tbody&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;1&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;773&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;13&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;77&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;15&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;3245&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;54&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;279&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;25&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;3844&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;64&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;390&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;50&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;4912&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;82&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;612&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;75&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;5405&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;90&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;841&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;100&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;5436&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;91&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;1108&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;150&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;5434&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;91&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;1688&lt;/td&gt;
-    &lt;/tr&gt;
-  &lt;/tbody&gt;
-&lt;/table&gt;
-
-&lt;p&gt;Resulting in the line chart as follows:&lt;/p&gt;
-
-&lt;p&gt;&lt;img 
src=&quot;/images/blog/cisco_throughput_5x/handled_queries_1.png&quot; 
alt=&quot;&quot; width=&quot;500px&quot; height=&quot;300px&quot; 
/&gt;&lt;/p&gt;
-
-&lt;p&gt;​&lt;/p&gt;
-
-&lt;p&gt;&lt;strong&gt;Finding&lt;/strong&gt;: When the number of concurrent 
threads reach 75, executed queries per second reach a peak of 90. The number 
does not become better even as we continue to increase the threads. 90 
concurrent query responses in one second only allows 90/15 = 6 users to view a 
report at the same time. Even when we extend the Kylin query nodes to 3, query 
capability with 18 users per second is far behind our business 
demands.&lt;/p&gt;
-
-&lt;h2 id=&quot;root-cause-analysis&quot;&gt;Root Cause Analysis&lt;/h2&gt;
-
-&lt;p&gt;After reading and analyzing the query engine code of Kylin, we 
learned that Kylin’s query performs parallel filtering and calculation in 
HBase’s region server by launching HBase Coprocessor. Based on this 
information, we checked the resource usage of the HBase cluster. The count of 
RPC Tasks processed on the region server did not increase linearly with the 
number of Kylin query requests when high concurrent queries occured. We 
concluded that there was a thread block on the Kylin side.&lt;/p&gt;
-
-&lt;p&gt;We used Flame graph and JProfile to collect and analyze data from the 
Kylin query node and could not find the root cause. Then we tried to catch a 
thread snapshot of Kylin with Jstack. Analyzing the Jstack log, we discovered 
the root cause of the bottleneck causing this concurrent query issue. The 
example is as follows (Kylin version 2.5.0):&lt;/p&gt;
-
-&lt;p&gt;One thread is locked at sun.misc.URLClassPath.getNextLoader. TID is 
0x000000048007a180:&lt;/p&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&quot;Query 
e9c44a2d-6226-ff3b-f984-ce8489107d79-3425&quot; #3425 daemon prio=5 os_prio=0 
tid=0x000000000472b000 nid=0x1433 waiting ]
-   java.lang.Thread.State: BLOCKED (on object monitor)
-    at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:469)
-    - locked &amp;lt;0x000000048007a180&amp;gt; (a sun.misc.URLClassPath)
-    at sun.misc.URLClassPath.findResource(URLClassPath.java:214)
-    at java.net.URLClassLoader$2.run(URLClassLoader.java:569)
-    at java.net.URLClassLoader$2.run(URLClassLoader.java:567)
-    at java.security.AccessController.doPrivileged(Native Method)
-    at java.net.URLClassLoader.findResource(URLClassLoader.java:566)
-    at java.lang.ClassLoader.getResource(ClassLoader.java:1096)
-    at java.lang.ClassLoader.getResource(ClassLoader.java:1091)
-    at 
org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1666)
-    at 
org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:338)
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;43 threads were waiting to lock &amp;lt;0x000000048007a180&amp;gt;  
at the same time:&lt;/p&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;&quot;Query 
f1f0bbec-a3f7-04b2-1ac6-fd3e03a0232d-4002&quot; #4002 daemon prio=5 os_prio=0 
tid=0x00007f27e71e7800 nid=0x1676 waiting ]
-   java.lang.Thread.State: BLOCKED (on object monitor)
-    at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:469)
-    - waiting to lock &amp;lt;0x000000048007a180&amp;gt; (a 
sun.misc.URLClassPath)
-    at sun.misc.URLClassPath.findResource(URLClassPath.java:214)
-    at java.net.URLClassLoader$2.run(URLClassLoader.java:569)
-    at java.net.URLClassLoader$2.run(URLClassLoader.java:567)
-    at java.security.AccessController.doPrivileged(Native Method)
-    at java.net.URLClassLoader.findResource(URLClassLoader.java:566)
-    at java.lang.ClassLoader.getResource(ClassLoader.java:1096)
-    at java.lang.ClassLoader.getResource(ClassLoader.java:1091)
-    at 
org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1666)
-    at 
org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:338)
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;We found that the closest code logic to Kylin was:&lt;/p&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:338)
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;p&gt;Further analyzing the Kylin source code showed we were getting close 
to the resolution.&lt;/p&gt;
-
-&lt;h2 id=&quot;code-analysis&quot;&gt;Code Analysis&lt;/h2&gt;
-
-&lt;p&gt;When Kylin query engine builds a request to HBase Coprocessor, it 
will export Kylin properties (various properties used in Kylin) as Strings. 
This issue is caused by the relative code logic.&lt;/p&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;function private static 
OrderedProperties buildSiteOrderedProps()
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;ul&gt;
-  &lt;li&gt;Each thread will getResouce to load 
“kylin-defaults.properties” (the default properties file that users cannot 
modify).&lt;/li&gt;
-&lt;/ul&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;// 1. load default configurations 
from classpath.
-// we have a kylin-defaults.properties in kylin/core-common/src/main/resources
-URL resource = 
Thread.currentThread().getContextClassLoader().getResource(&quot;kylin-defaults.properties&quot;);
-Preconditions.checkNotNull(resource);
-logger.info(&quot;Loading kylin-defaults.properties from {}&quot;, 
resource.getPath());
-OrderedProperties orderedProperties = new OrderedProperties();
-loadPropertiesFromInputStream(resource.openStream(), orderedProperties);
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
-
-&lt;ul&gt;
-  &lt;li&gt;Loop 10 times to getResouce for  “kylin-defaults” + (i) + 
“.properties”. Thread LOCKED occurs here.&lt;/li&gt;
-&lt;/ul&gt;
-
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;for (int i = 0; i &amp;lt; 10; i++) {
-String fileName = &quot;kylin-defaults&quot; +  + &quot;.properties&quot;;
- URL additionalResource = 
Thread.currentThread().getContextClassLoader().getResource(fileName);
- if (additionalResource != null) {
-        logger.info(&quot;Loading {} from {} &quot;, fileName, 
additionalResource.getPath());
- loadPropertiesFromInputStream(additionalResource.openStream(), 
orderedProperties);
- }
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
+        <title>Apache Kylin v2.6.0 正式发布</title>
+        <description>&lt;p&gt;近日Apache Kylin 社区很高å…
´åœ°å®£å¸ƒï¼ŒApache Kylin 2.6.0 正式发布。&lt;/p&gt;
 
-&lt;p&gt;Those logics were introduced in 2017/6/7, with JIRA ID KYLIN-2659 
&lt;em&gt;Refactor KylinConfig so that all the default configurations are 
hidden in kylin-defaults.properties&lt;/em&gt; reported by Hongbin Ma.&lt;/p&gt;
+&lt;p&gt;Apache Kylin 
是一个开源的分布式分析引擎,旨在为极大数据集提供 SQL 
接口和多维分析(OLAP)的能力。&lt;/p&gt;
 
-&lt;h2 id=&quot;issue-fixing&quot;&gt;Issue Fixing&lt;/h2&gt;
+&lt;p&gt;这是继2.5.0 后的一个新功能版本。该版本引å…
¥äº†å¾ˆå¤šæœ‰ä»·å€¼çš„æ”¹è¿›ï¼Œå®Œæ•´çš„æ”¹åŠ¨åˆ—è¡¨è¯·å‚è§&lt;a 
href=&quot;https://kylin.apache.org/docs/release_notes.html&quot;&gt;release 
notes&lt;/a&gt;;这里挑一些主要改进做说明:&lt;/p&gt;
 
-&lt;p&gt;For the first part of the logic, because kylin-defaults.properties is 
built in kylin-core-common-xxxx.jar, there’s no need to getResource for it 
every time. We moved this logic to getInstanceFromEnv(). This logic gets called 
only once when service starts.&lt;/p&gt;
+&lt;h3 id=&quot;jdbcsdk&quot;&gt;针对以JDBC为数据源的SDK&lt;/h3&gt;
+&lt;p&gt;Kylin目前已经支持通过JDBC连接包括Amazon Redshift, SQL 
Server在内的多种数据源。&lt;br /&gt;
+为了便于开发者更便利地处理各种 SQL 方言(dialect) 
的不同以更加简单地开发新的 JDBC 数据源,Kylin 
提供了相应的 SDK 和统一的 API 入口:&lt;br /&gt;
+* 同步元数据和数据&lt;br /&gt;
+* 构建 cube&lt;br /&gt;
+* 
当找不到相应的cube来解答查询时,下推查询到数据源&lt;/p&gt;
 
-&lt;p&gt;We found one regression issue when fixing this bug. One class, 
CubeVisitService, is a Coprocessor. It will use KylinConfig as util class to 
generate KylinConfig object. It’s dangerous to induce any logic to load 
properties. Due to this, there is no Kylin.properties file in 
Coprocessor.&lt;/p&gt;
+&lt;p&gt;更多内容参见 KYLIN-3552。&lt;/p&gt;
 
-&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre 
class=&quot;highlight&quot;&gt;&lt;code&gt;buildDefaultOrderedProperties();
-&lt;/code&gt;&lt;/pre&gt;
-&lt;/div&gt;
+&lt;h3 id=&quot;memcached--kylin-&quot;&gt;使用 Memcached 作 Kylin 
的分布式缓存&lt;/h3&gt;
+&lt;p&gt;在过去,Kylin 
对查询结果的缓存不是十分高效,主要有以下两个方面的原å›
 ï¼š&lt;br /&gt;
+一个是当 Kylin 的 metadata 发生变化时,会主动盲目地去清
除大量缓存,使得缓存会被频繁刷新而导致利用率降低。&lt;br
 /&gt;
+另一点是由于只使用本地缓存而导致 Kylin server 之间不能å…
±äº«å½¼æ­¤çš„缓存,这样查询的缓存命中率就会降低。&lt;br /&gt;
+本地缓存的一个缺点是大小受到限制,不能像分布式缓存那æ
 ·æ°´å¹³æ‰©å±•。这æ 
·å¯¼è‡´èƒ½ç¼“存的查询结果量受到了限制。&lt;/p&gt;
 
-&lt;p&gt;For the second part, this design should be future-proof and allow 
users to define 10 default properties (and override with each other), but after 
a year and a half, this logic seemed to never be used. However, to reduce risk, 
we kept this logic because it only gets called once during service start up 
which resulted in an insignificant waste of additional time.&lt;/p&gt;
+&lt;p&gt;针对这些缺陷,我们改变了缓存失效的机制,不再主动去æ¸
…理缓存,而是采取如下的方案:&lt;br /&gt;
+1. 在将查询结果放入缓存之前,根据当前的å…
ƒæ•°æ®ä¿¡æ¯è®¡ç®—一个数字签名,并与查询结果一同放å…
¥ç¼“存中&lt;br /&gt;
+2. 从缓存中获取查询结果之后,根据当前的å…
ƒæ•°æ®ä¿¡æ¯è®¡ç®—一个数字签名,对比两者
的数字签名是否一致。如果一致,那么缓存有效;反之,该缓存失效并åˆ
 é™¤&lt;/p&gt;
 
-&lt;h2 id=&quot;performance-testing-after-bug-fixes&quot;&gt;Performance 
Testing After Bug Fixes&lt;/h2&gt;
+&lt;p&gt;我们还引入了 Memcached 作为 Kylin 的分布式缓存。这样 
Kylin server 之间可以共享查询结果的缓存,而且由于 Memcached 
server 之间的独立性,非常易于水平拓展,更加
有利于缓存更多的数据。&lt;br /&gt;
+相关开发任务是 KYLIN-2895, KYLIN-2894, KYLIN-2896, KYLIN-2897, 
KYLIN-2898, KYLIN-2899。&lt;/p&gt;
 
-&lt;p&gt;Based on the same data volume and testing environment, results were 
as follows:&lt;/p&gt;
+&lt;h3 id=&quot;forkjoinpool--fast-cubing-&quot;&gt;ForkJoinPool 简化 fast 
cubing 的线程模型&lt;/h3&gt;
+&lt;p&gt;在过去进行 fast cubing 时,Kylin 
使用自己定义的一系列线程,如 split 线程,task 线程,main 
线程等等进行并发的 cube 构建。&lt;br /&gt;
+在这个线程模型中,线程之间的å…
³ç³»ååˆ†çš„复杂,而且对异常处理也十分容易出错。&lt;/p&gt;
 
-&lt;table&gt;
-  &lt;thead&gt;
-    &lt;tr&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Thread&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Handled Queries (in 60 
seconds)&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Handled Queries (per 
second)&lt;/th&gt;
-      &lt;th style=&quot;text-align: center&quot;&gt;Mean Response Time 
(ms)&lt;/th&gt;
-    &lt;/tr&gt;
-  &lt;/thead&gt;
-  &lt;tbody&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;1&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;2451&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;41&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;12&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;15&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;12422&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;207&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;37&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;25&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;15600&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;260&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;56&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;50&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;18481&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;308&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;129&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;75&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;21055&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;351&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;136&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;100&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;24036&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;400&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;251&lt;/td&gt;
-    &lt;/tr&gt;
-    &lt;tr&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;150&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;28014&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;467&lt;/td&gt;
-      &lt;td style=&quot;text-align: center&quot;&gt;277&lt;/td&gt;
-    &lt;/tr&gt;
-  &lt;/tbody&gt;
-&lt;/table&gt;
+&lt;p&gt;现在我们引入了 ForkJoinPool,在主线程中处理 split 
逻辑,构建 cuboid 的任务以及子任务都在 fork join 
pool中执行,cuboid 
构建的结果可以被异步的收集并且可以更早地输出给下游的 
merge 操作。更多内容参见 KYLIN-2932。&lt;/p&gt;
 
-&lt;p&gt;And the resulting line chart:&lt;/p&gt;
+&lt;h3 id=&quot;hllcounter-&quot;&gt;改进 HLLCounter 的性能&lt;/h3&gt;
+&lt;p&gt;对于 HLLCounter, 我们从两方面进行了改进:构建 
HLLCounter 和计算调和平均的方式。&lt;br /&gt;
+1. 关于 HLLCounter 
的构建,我们不再使用merge的方式,而是直接copy别的HLLCounter里面的registers&lt;br
 /&gt;
+2. 关于计算 HLLCSnapshot 
里面的调和平均,做了以下三个方面的改进:&lt;br /&gt;
+* 缓存所有的1/2^r&lt;br /&gt;
+* 使用整型相加代替浮点型相加&lt;br /&gt;
+* 删除条件分支,例如无需检查 registers[i] 是不是为0&lt;/p&gt;
 
-&lt;p&gt;&lt;img 
src=&quot;/images/blog/cisco_throughput_5x/handled_queries_2.png&quot; 
alt=&quot;&quot; width=&quot;500px&quot; height=&quot;300px&quot; 
/&gt;&lt;/p&gt;
+&lt;p&gt;更多内容参见 KYLIN-3656。&lt;/p&gt;
 
-&lt;p&gt;When the concurrent threads reached 150, Kylin processed 467 requests 
per second. The concurrent query capability increased by five times with linear 
growth . It could be concluded then that the bottleneck was eliminated . We 
didn’t increase the concurrent threads due to the Kylin query engine’s 
settings for cluster load balancing which meant that increasing the concurrent 
connections on a single node increased the workload on the Tomcat server (Max 
connection is 150 in Kylin by default). The thread blocking issue disappeared 
after re-collecting and analyzing the Jstack log.&lt;/p&gt;
+&lt;h3 id=&quot;cube-planner-&quot;&gt;改进 Cube Planner 算法&lt;/h3&gt;
+&lt;p&gt;在过去,cube planner 的 phase two 增加未被预计算的 
cuboid 的方式只能通过 mandatory cuboid 的方式。而一个 cuboid 
是否为 mandatory,又有两种方式:&lt;br /&gt;
+手动设置,查询时 rollup 
的行数足够大。这里通过判断查询时 rollup 
的行数是否足够大来判断是否为 mandatory cuboid 
的方式有两大缺陷:&lt;br /&gt;
+* 一是估算 rollup 的行数的算法不是很好&lt;br /&gt;
+* 二是很难设立一个静态的阈值来做判定&lt;/p&gt;
 
-&lt;p&gt;After the fix, each Kylin node could now handle requests for 467/15 = 
31 users, which meets our business requirement. Additionally, Kylin’s 
concurrent query capability can be further improved by several times once we 
enable query cache, so it is more than sufficient to fulfill our 
needs.&lt;/p&gt;
+&lt;p&gt;现在我们不再从 rollup 
行数的角度看问题了。一切都是从 cuboid 
行数的角度看问题,这样就和 cost based 的 cube planner 
算法做了统一。&lt;br /&gt;
+为此我们通过使用 rollup 比率来改进了未被预先构建的 cuboid 
的行数的估算,然后让 cost based 的 cube planner 
算法来判定哪些未被构建的 cuboid 
该被构建,哪些该被遗弃。&lt;br /&gt;
+通过这样的改进,无需通过设定静态的阈值来推荐 mandatory 
cuboid 了,而 mandatory cuboid 
只能被手动设置,不能被推荐了。更多内容参见 
KYLIN-3540。&lt;/p&gt;
 
-&lt;h2 id=&quot;summary&quot;&gt;Summary&lt;/h2&gt;
+&lt;p&gt;&lt;strong&gt;下载&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;Apache Kylin lets you query massive datasets at sub-second latency, 
thanks to the pre-calculation design of cubes, the optimization of Apache 
Calcite operator in queries, and also the introduction of “Prepared Statement 
Cache” to reduce the cost of Calcite SQL parses. Query performance 
optimization is not easy. We need to pay more attention to impacts on the Kylin 
query engine when new features are introduced or bugs are fixed, since even a 
minor code change could spell disaster. Issues like these in high concurrency 
scenarios can often be hard to reproduce and analyze.&lt;/p&gt;
+&lt;p&gt;要下载Apache Kylin v2.6.0源代码或二进制包,请访问&lt;a 
href=&quot;http://kylin.apache.org/download&quot;&gt;下载页面&lt;/a&gt; 
.&lt;/p&gt;
 
-&lt;p&gt;Lastly, query performance testing should not be limited to a single 
or small set of queries. High concurrecy performance testing should take place 
considering actual business requirements. For enterprise reporting systems, 3 
seconds is the user tolerance limit for new page loading, which includes page 
rendering and network consumption. Ultimately, the backend data service should 
provide a response within 1 second. This is indeed a big challenge in a 
business scenario with big data sets. Fortunately, Kylin easily meets this 
requirement.&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;升级&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;This issue has already been submitted on JIRA as &lt;a 
href=&quot;https://issues.apache.org/jira/browse/KYLIN-3672&quot;&gt;KYLIN-3672&lt;/a&gt;,
 and released in Kylin v2.5.2. Thanks to Shaofeng Shi of Kyligence Inc. for 
help.&lt;/p&gt;
+&lt;p&gt;参考&lt;a 
href=&quot;/docs/howto/howto_upgrade.html&quot;&gt;升级指南&lt;/a&gt;.&lt;/p&gt;
 
-&lt;p&gt;【1】&lt;a 
href=&quot;https://issues.apache.org/jira/browse/KYLIN-3672&quot;&gt;https://issues.apache.org/jira/browse/KYLIN-3672&lt;/a&gt;&lt;/p&gt;
+&lt;p&gt;&lt;strong&gt;反馈&lt;/strong&gt;&lt;/p&gt;
 
-&lt;p&gt;&lt;em&gt;Author Zongwie Li as a Cisco engineer and a team member in 
the company’s Big Data architecture team, currently responsible for OLAP 
platform construction and customer business reporting 
systems.&lt;/em&gt;&lt;/p&gt;
+&lt;p&gt;如果您遇到问题或疑问,请发送邮件至 Apache Kylin dev 
或 user 邮件列表:d...@kylin.apache.org,u...@kylin.apache.org; 
在发送之前,请确保您已通过发送电子邮件至 
dev-subscr...@kylin.apache.org 或 user-subscr...@kylin.apache.org订阅
了邮件列表。&lt;/p&gt;
 
+&lt;p&gt;&lt;em&gt;非常感谢所有贡献Apache 
Kylin的朋友!&lt;/em&gt;&lt;/p&gt;
 </description>
-        <pubDate>Thu, 17 Jan 2019 09:30:00 -0800</pubDate>
-        
<link>http://kylin.apache.org/blog/2019/01/17/cisco-throughput-5x/</link>
-        <guid 
isPermaLink="true">http://kylin.apache.org/blog/2019/01/17/cisco-throughput-5x/</guid>
+        <pubDate>Fri, 18 Jan 2019 12:00:00 -0800</pubDate>
+        <link>http://kylin.apache.org/cn_blog/2019/01/18/release-v2.6.0/</link>
+        <guid 
isPermaLink="true">http://kylin.apache.org/cn_blog/2019/01/18/release-v2.6.0/</guid>
         
         
-        <category>blog</category>
+        <category>cn_blog</category>
         
       </item>
     

Added: kylin/site/images/blog/refine-query-cache/cache-signature.png
URL: 
http://svn.apache.org/viewvc/kylin/site/images/blog/refine-query-cache/cache-signature.png?rev=1864154&view=auto
==============================================================================
Binary file - no diff available.

Propchange: kylin/site/images/blog/refine-query-cache/cache-signature.png
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: kylin/site/images/blog/refine-query-cache/consistent-hashing.png
URL: 
http://svn.apache.org/viewvc/kylin/site/images/blog/refine-query-cache/consistent-hashing.png?rev=1864154&view=auto
==============================================================================
Binary file - no diff available.

Propchange: kylin/site/images/blog/refine-query-cache/consistent-hashing.png
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: kylin/site/images/blog/refine-query-cache/l1-l2-cache.png
URL: 
http://svn.apache.org/viewvc/kylin/site/images/blog/refine-query-cache/l1-l2-cache.png?rev=1864154&view=auto
==============================================================================
Binary file - no diff available.

Propchange: kylin/site/images/blog/refine-query-cache/l1-l2-cache.png
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream


Reply via email to