Author: lidong
Date: Fri Oct 22 05:11:34 2021
New Revision: 1894464
URL: http://svn.apache.org/viewvc?rev=1894464&view=rev
Log:
Update kylin diagram and add new blog
Added:
kylin/site/assets/images/Kylin_diagram.pptx (with props)
kylin/site/blog/2021/10/
kylin/site/blog/2021/10/21/
kylin/site/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/
kylin/site/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html
kylin/site/cn_blog/2021/10/
kylin/site/cn_blog/2021/10/21/
kylin/site/cn_blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/
kylin/site/cn_blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html
kylin/site/images/blog/local-cache/
kylin/site/images/blog/local-cache/Local_cache_stage.png (with props)
kylin/site/images/blog/local-cache/kylin4_local_cache.png (with props)
kylin/site/images/blog/local-cache/local_cache_benchmark_result_ssb.png
(with props)
kylin/site/images/blog/local-cache/local_cache_benchmark_result_tpch1.png
(with props)
kylin/site/images/blog/local-cache/local_cache_benchmark_result_tpch4.png
(with props)
Modified:
kylin/site/assets/images/kylin_diagram.png
kylin/site/blog/index.html
kylin/site/cn/blog/index.html
kylin/site/feed.xml
Added: kylin/site/assets/images/Kylin_diagram.pptx
URL:
http://svn.apache.org/viewvc/kylin/site/assets/images/Kylin_diagram.pptx?rev=1894464&view=auto
==============================================================================
Binary file - no diff available.
Propchange: kylin/site/assets/images/Kylin_diagram.pptx
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Modified: kylin/site/assets/images/kylin_diagram.png
URL:
http://svn.apache.org/viewvc/kylin/site/assets/images/kylin_diagram.png?rev=1894464&r1=1894463&r2=1894464&view=diff
==============================================================================
Binary files - no diff available.
Added:
kylin/site/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html
URL:
http://svn.apache.org/viewvc/kylin/site/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html?rev=1894464&view=auto
==============================================================================
---
kylin/site/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html
(added)
+++
kylin/site/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html
Fri Oct 22 05:11:34 2021
@@ -0,0 +1,356 @@
+<!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+<!doctype html>
+<html>
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>Apache Kylin | Performance optimization of Kylin 4.0 in cloud --
local cache and soft affinity scheduling</title>
+ <meta name="description" content="01 Background IntroductionRecently, the
Apache Kylin community released Kylin 4.0.0 with a new architecture. The
architecture of Kylin 4.0 supports the separ...">
+ <meta name="author" content="Apache Kylin">
+ <link rel="shortcut icon" href="fav.png" type="image/png">
+
+
+
+<link rel="stylesheet" href="/assets/css/animate.css">
+<!-- Bootstrap -->
+<link rel="stylesheet" href="/assets/css/bootstrap.min.css">
+
+<!-- Fonts -->
+<!-- <link rel="stylesheet"
href="http://fonts.googleapis.com/css?family=Alice|Open+Sans:400,300,700"> -->
+
+<!-- Icons -->
+<link rel="stylesheet" href="/assets/css/font-awesome.min.css">
+
+ <!-- Custom styles -->
+ <link rel="stylesheet" href="/assets/css/styles.css">
+ <link rel="stylesheet" href="/assets/css/docs.css">
+ <link rel="stylesheet" href="/assets/css/pygments.css">
+
+ <link rel="canonical"
href="http://kylin.apache.org/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/";>
+ <link rel="alternate" type="application/rss+xml" title="Apache Kylin"
href="http://kylin.apache.org/feed.xml"; />
+
+<!--[if lt IE 9]> <script src="assets/js/html5shiv.js"></script> <![endif]-->
+<!-- Global site tag (gtag.js) - Google Analytics -->
+<script async
src="https://www.googletagmanager.com/gtag/js?id=UA-120788561-1";></script>
+<script>
+ window.dataLayer = window.dataLayer || [];
+ function gtag(){dataLayer.push(arguments);}
+ gtag('js', new Date());
+
+ gtag('config', 'UA-120788561-1');
+</script>
+<script type="text/javascript" src="/assets/js/jquery-1.9.1.min.js"></script>
+<script type="text/javascript" src="/assets/js/nside.js"></script> </script>
+<script type="text/javascript" src="/assets/js/nnav.js"></script> </script>
+<script>
+var _hmt = _hmt || [];
+(function() {
+ var hm = document.createElement("script");
+ hm.src = "https://hm.baidu.com/hm.js?bdc5e03add430c0b72cc0eb91eabfa99";;
+ var s = document.getElementsByTagName("script")[0];
+ s.parentNode.insertBefore(hm, s);
+})();
+</script>
+
+</head>
+
+ <body>
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<header id="header" >
+
+ <!-- Main Menu -->
+ <nav class="navbar navbar-default" role="navigation" id="nav-wrapper">
+ <div class="container-fluid" id="nav">
+ <!--
+ <img class="img-circle" width="40px" height="40px" id="circlelogo"
src="/assets/images/kylin_logo.jpg">
+ -->
+ <!-- Brand and toggle get grouped for better mobile display -->
+ <div class="navbar-header">
+ <img class="navbar-logo" width="46"
src="/assets/images/kylin_logo.png" ></img>
+ <button type="button" class="navbar-toggle collapsed"
data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <ul class="nav icon-navbar">
+ <li><a href="https://twitter.com/apachekylin"; target="_blank"
class="fa fa-twitter fa-lg" title="Twitter: @ApacheKylin" ></a></li>
+ <li><a href="https://github.com/apache/kylin"; target="_blank"
class="fa fa-github-alt fa-lg" title="Github: apache/kylin" ></a></li>
+ <li><a href="https://www.facebook.com/kylinio"; target="_blank"
class="fa fa-facebook fa-lg" title="Facebook: kylin.io" ></a></li>
+ </ul>
+ </div>
+
+ <!-- Collect the nav links, forms, and other content for toggling -->
+ <div class="navbar-collapse collapse" id="bs-example-navbar-collapse-1">
+
+ <ul class="nav navbar-nav">
+
+ <li><a href="/">Home</a></li>
+ <li>
+ <a href="/docs" class="dropdown-toggle" data-toggle="dropdown"
role="button" aria-haspopup="true" aria-expanded="false">Docs<span
class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/docs/">Latest Release(Kylin 4.0.0)</a></li>
+ <li><a href="/docs31/">Kylin 3.1.2</a></li>
+ <li><a href="/docs24/">Kylin 2.4.0</a></li>
+ <li><a href="/archive/">Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="/download">Download</a></li>
+ <li><a href="/community" >Community</a></li>
+ <li>
+ <a href="/development" class="dropdown-toggle"
data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Development<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/development40/">Kylin 4.x</a></li>
+ <li><a href="/development/">Kylin 3.x And Older Versions</a></li>
+ </ul>
+ </li>
+ <li><a href="/blog">Blog</a></li>
+ <li><a href="/cn" >ä¸æç</a></li>
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </div><!-- /.container-fluid -->
+ </nav>
+
+ <div id="head" class="parallax normal-header" >
+ <div class="text-center header-apache">
+ <a href="http://apache.org/foundation/contributing.html"; title="Support
Apache" style="margin-left: 150px;">
+ <div>
+ <img src="https://www.apache.org/images/SupportApache-small.png"; >
+ </div>
+ </a>
+ </div>
+ </div>
+
+ </header>
+
+ <div class="page-content main">
+ <header style=" padding:2em 0 0 ">
+ <div class="container" >
+ <div style=" padding:0 4em">
+ <div class="blog-icon">
+ <img width="30" src="/assets/images/icon_blog_w.png">
+ </div>
+ <h4 class="index-title" style="
float:left;"><span>Apache Kylin⢠Technical Blog</span></h4>
+ </div>
+ </div>
+ </div>
+
+ <div class="container blog">
+ <div>
+ <article class="post-content" >
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<div class="post" style=" padding:2em 4em 4em 4em">
+
+ <header class="post-header">
+ <h1 class="post-title">Performance optimization of Kylin 4.0 in cloud --
local cache and soft affinity scheduling</h1>
+ <p class="post-meta" >Oct 21, 2021 ⢠Yaqian Zhang</p>
+ </header>
+
+ <article class="post-content" >
+ <h2 id="background-introduction">01 Background Introduction</h2>
+<p>Recently, the Apache Kylin community released Kylin 4.0.0 with a new
architecture. The architecture of Kylin 4.0 supports the separation of storage
and computing, which enables kylin users to run Kylin 4.0 in a more flexible
cloud deployment mode with flexible computing resources. With the cloud
infrastructure, users can choose to use cheap and reliable object storage to
store cube data, such as S3. However, in the architecture of separation of
storage and computing, we need to consider that reading data from remote
storage by computing nodes through the network is still a costly operation,
which often leads to performance loss.<br />
+In order to improve the query performance of Kylin 4.0 when using cloud object
storage as the storage, we try to introduce the local cache mechanism into the
Kylin 4.0 query engine. When executing the query, the frequently used data is
cached on the local disk to reduce the delay caused by pulling data from the
remote object storage and achieve faster query response. In addition, in order
to avoid wasting disk space when the same data is cached on a large number of
spark executors at the same time, and the computing node can read more required
data from the local cache, we introduce the scheduling strategy of soft
affinity. The soft affinity strategy is to establish a corresponding
relationship between the spark executor and the data file through some method,
In most cases, the same data can always be read on the same executor, so as to
improve the hit rate of the cache.</p>
+
+<h2 id="implementation-principle">02 Implementation Principle</h2>
+
+<h4 id="local-cache">1. Local Cache</h4>
+
+<p>When Kylin 4.0 executes a query, it mainly goes through the following
stages, in which the stages where local cache can be used to improve
performance are marked with dotted lines:</p>
+
+<p><img src="/images/blog/local-cache/Local_cache_stage.png" alt="" /></p>
+
+<ul>
+ <li>File list cacheï¼Cache the file status on the spark driver side. When
executing the query, the spark driver needs to read the file list and obtain
some file information for subsequent scheduling execution. Here, the file
status information will be cached locally to avoid frequent reading of remote
file directories.</li>
+ <li>Data cacheï¼Cache the data on the spark executor side. You can set the
data cache to memory or disk. If it is set to cache to memory, you need to
appropriately increase the executor memory to ensure that the executor has
enough memory for data cache; If it is cached to disk, you need to set the data
cache directory, preferably SSD disk directory.</li>
+</ul>
+
+<p>Based on the above design, different types of caches are made on the driver
side and the executor side of the query engine of kylin 4.0. The basic
architecture is as follows:</p>
+
+<p><img src="/images/blog/local-cache/kylin4_local_cache.png" alt="" /></p>
+
+<h4 id="soft-affinity-scheduling">2. Soft Affinity Scheduling</h4>
+
+<p>When doing data cache on the executor side, if all data is cached on all
executors, the size of cached data will be very considerable and a great waste
of disk space, and it is easy to cause frequent evict cache data. In order to
maximize the cache hit rate of the spark executor, the spark driver needs to
schedule the tasks of the same file to the same executor as far as possible
when the resource conditions are me, so as to ensure that the data of the same
file can be cached on a specific one or several executors, and the data can be
read through the cache when it is read again.<br />
+To this end, we calculate the target executor list by calculating the hash
according to the file name and then modulo with the executor num. The number of
executors to cache is determined by the number of data cache replications
configured by the user. Generally, the larger the number of cache replications,
the higher the probability of hitting the cache. When the target executors are
unreachable or have no resources for scheduling, the scheduler will fall back
to the random scheduling mechanism of spark. This scheduling method is called
soft affinity scheduling strategy. Although it can not guarantee 100% hit to
the cache, it can effectively improve the cache hit rate and avoid a large
amount of disk space wasted by full cache on the premise of minimizing
performance loss.</p>
+
+<h2 id="related-configuration">03 Related Configuration</h2>
+
+<p>According to the above principles, we implemented the basic function of
local cache + soft affinity scheduling in Kylin 4.0, and tested the query
performance based on SSB data set and TPCH data set respectively.<br />
+Several important configuration items are listed here for users to understand.
The actual configuration will be given in the attachment at the end:</p>
+
+<ul>
+ <li>Enable soft affinity
schedulingï¼kylin.query.spark-conf.spark.kylin.soft-affinity.enabled</li>
+ <li>Enable local
cacheï¼kylin.query.spark-conf.spark.hadoop.spark.kylin.local-cache.enabled</li>
+ <li>The number of data cache replications, that is, how many executors cache
the same data
fileï¼kylin.query.spark-conf.spark.kylin.soft-affinity.replications.num</li>
+ <li>Cache to memory or local directory. Set cache to memory as buff and
cache to local as local:
kylin.query.spark-conf.spark.hadoop.alluxio.user.client.cache.store.type</li>
+ <li>Maximum cache
capacityï¼kylin.query.spark-conf.spark.hadoop.alluxio.user.client.cache.size</li>
+</ul>
+
+<h2 id="performance-benchmark">04 Performance Benchmark</h2>
+
+<p>We conducted performance tests in three scenarios under AWS EMR
environment. When scale factor = 10, we conducted single concurrent query test
on SSB dataset, single concurrent query test and 4 concurrent query test on
TPCH dataset. S3 was configured as storage in the experimental group and the
control group. Local cache and soft affinity scheduling were enabled in the
experimental group, but not in the control group. In addition, we also compare
the results of the experimental group with the results when HDFS is used as
storage in the same environment, so that users can intuitively feel the
optimization effect of local cache + soft affinity scheduling on deploying
Kylin 4.0 on the cloud and using object storage as storage.</p>
+
+<p><img src="/images/blog/local-cache/local_cache_benchmark_result_ssb.png"
alt="" /></p>
+
+<p><img src="/images/blog/local-cache/local_cache_benchmark_result_tpch1.png"
alt="" /></p>
+
+<p><img src="/images/blog/local-cache/local_cache_benchmark_result_tpch4.png"
alt="" /></p>
+
+<p>As can be seen from the above results:</p>
+
+<ol>
+ <li>In the single concurrency scenario of SSB data set, when S3 is used as
storage, turning on the local cache and soft affinity scheduling can achieve
about three times the performance improvement, which can be the same as that of
HDFS, or even improved.</li>
+ <li>Under TPCH data set, when S3 is used as storage, whether single
concurrent query or multiple concurrent query, after local cache and soft
affinity scheduling are enabled, the performance of all queries can be greatly
improved.</li>
+</ol>
+
+<p>However, in the comparison results of Q21 under the 4 concurrent tests of
TPCH dataset, we observed that the results of enabling local cache and soft
affinity scheduling are lower than those when using S3 alone as storage. Here,
it may be that the data is not read through the cache for some reason. The
underlying reason is not further analyzed in this test, in the subsequent
optimization process, we will gradually improve. Moreover, because the query of
TPCH is complex and the SQL types are different, compared with the results of
HDFS, the performance of some SQL is improved, while the performance of some
SQL is slightly insufficient, but generally speaking, it is very close to the
results of HDFS as storage.<br />
+The result of this performance test is a preliminary verification of the
performance improvement effect of local cache + soft affinity scheduling. On
the whole, local cache + soft affinity scheduling can achieve significant
performance improvement for both simple queries and complex queries, but there
is a certain performance loss in the scenario of high concurrent queries.<br />
+If users use cloud object storage as Kylin 4.0 storage, they can get a good
performance experience when local cache + soft affinity scheduling is enabled,
which provides performance guarantee for Kylin 4.0 to use the separation
architecture of computing and storage in the cloud.</p>
+
+<h2 id="code-implementation">05 Code Implementation</h2>
+
+<p>Since the current code implementation is still in the basic stage, there
are still many details to be improved, such as implementing consistent hash,
how to deal with the existing cache when the number of executors changes, so
the author has not submitted PR to the community code base. Developers who want
to preview in advance can view the source code through the following link:</p>
+
+<p><a
href="https://github.com/zzcclp/kylin/commit/4e75b7fa4059dd2eaed24061fda7797fecaf2e35";>The
code implementation of local cache and soft affinity scheduling</a></p>
+
+<h2 id="related-link">06 Related Link</h2>
+
+<p>You can view the performance test result data and specific configuration
through the link:<br />
+<a href="https://github.com/Kyligence/kylin-tpch/issues/9";>The benchmark of
Kylin4.0 with local cache and soft affinity scheduling</a></p>
+
+ </article>
+
+</div>
+
+
+
+
+
+ </article>
+ </div>
+ </div>
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<footer id="underfooter">
+ <div>
+ <div class="row">
+ <div class="col-md-12 widget">
+ <div class="widget-body">
+ <div class="footer-img">
+ <a href="http://www.apache.org";>
+ <img id="asf-logo" height="78px" alt="Apache
Software Foundation" src="/assets/images/apache_footer.png">
+ </a>
+ </div>
+ <p style="padding-top: 11px;">
+ The contents of this website are © 2015 Apache
Software Foundation under the terms of the
+ <a href="http://www.apache.org/licenses/LICENSE-2.0";>
Apache License v2 </a>.
+ </p>
+ <p style="margin-bottom: 11px;">
+ Apache Kylin and its logo are trademarks of the Apache
Software Foundation.
+ </div>
+
+ </div>
+ </div>
+ </div>
+ <!-- /row of widgets -->
+
+ </div>
+ <div></div>
+
+</footer>
+
+ <script src="/assets/js/jquery-1.9.1.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/main.js"></script>
+ </body>
+</html>
+
+
+
+
Modified: kylin/site/blog/index.html
URL:
http://svn.apache.org/viewvc/kylin/site/blog/index.html?rev=1894464&r1=1894463&r2=1894464&view=diff
==============================================================================
--- kylin/site/blog/index.html (original)
+++ kylin/site/blog/index.html Fri Oct 22 05:11:34 2021
@@ -197,6 +197,16 @@ var _hmt = _hmt || [];
<div class="col-md-6 col-lg-6 col-xs-12">
+ <a class="blog-card"
href="/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/">
+ <div class="blog-pic">
+ <img width="20" src="../assets/images/icon_blog_w.png" />
+ </div>
+ <p class="blog-title">Performance optimization of Kylin 4.0 in
cloud -- local cache and soft affinity scheduling</p>
+ <p align="left" class="post-meta">posted: Oct 21, 2021</p>
+ </a>
+ </div>
+
+ <div class="col-md-6 col-lg-6 col-xs-12">
<a class="blog-card"
href="/blog/2021/08/03/How-Meituan-Dominates-Online-Shopping-with-Apache-Kylin/">
<div class="blog-pic">
<img width="20" src="../assets/images/icon_blog_w.png" />
Modified: kylin/site/cn/blog/index.html
URL:
http://svn.apache.org/viewvc/kylin/site/cn/blog/index.html?rev=1894464&r1=1894463&r2=1894464&view=diff
==============================================================================
--- kylin/site/cn/blog/index.html (original)
+++ kylin/site/cn/blog/index.html Fri Oct 22 05:11:34 2021
@@ -199,6 +199,16 @@ var _hmt = _hmt || [];
<div class="col-md-6 col-lg-6 col-xs-12">
+ <a class="blog-card"
href="/cn_blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/">
+ <div class="blog-pic">
+ <img width="20" src="/assets/images/icon_blog_w.png" />
+ </div>
+ <p class="blog-title">Kylin4
äºä¸æ§è½ä¼åï¼æ¬å°ç¼åå软亲åæ§è°åº¦</p>
+ <p align="left" class="post-meta">posted: Oct 21, 2021</p>
+ </a>
+ </div>
+
+ <div class="col-md-6 col-lg-6 col-xs-12">
<a class="blog-card"
href="/cn_blog/2021/08/03/How-Meituan-Dominates-Online-Shopping-with-Apache-Kylin/">
<div class="blog-pic">
<img width="20" src="/assets/images/icon_blog_w.png" />
Added:
kylin/site/cn_blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html
URL:
http://svn.apache.org/viewvc/kylin/site/cn_blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html?rev=1894464&view=auto
==============================================================================
---
kylin/site/cn_blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html
(added)
+++
kylin/site/cn_blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/index.html
Fri Oct 22 05:11:34 2021
@@ -0,0 +1,343 @@
+<!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+<!doctype html>
+<html>
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>Apache Kylin | Kylin4
äºä¸æ§è½ä¼åï¼æ¬å°ç¼åå软亲åæ§è°åº¦</title>
+ <meta name="description" content="01 èæ¯ä»ç»æ¥åï¼Apache Kylin
社åºåå¸äºå
¨æ°æ¶æç Kylin 4.0ãKylin 4.0
çæ¶ææ¯æåå¨å计ç®å离ï¼è¿ä½¿å¾ kylin ç¨æ·å¯ä»¥éåæ´å
çµæ´»ã计ç®èµæºå¯ä»¥å¼¹æ§ä¼¸ç¼©çäºä¸é¨ç½²æ¹å¼æ¥è¿è¡ Kylin
4.0ãåå©äºä¸çåºç¡è®¾æ½ï¼ç¨æ·å¯ä»¥éæ©ä½¿ç¨ä¾¿å®ä¸å¯é
ç对象åå¨æ¥å¨å cube æ°...">
+ <meta name="author" content="Apache Kylin">
+ <link rel="shortcut icon" href="fav.png" type="image/png">
+
+
+
+<link rel="stylesheet" href="/assets/css/animate.css">
+<!-- Bootstrap -->
+<link rel="stylesheet" href="/assets/css/bootstrap.min.css">
+
+<!-- Fonts -->
+<!-- <link rel="stylesheet"
href="http://fonts.googleapis.com/css?family=Alice|Open+Sans:400,300,700"> -->
+
+<!-- Icons -->
+<link rel="stylesheet" href="/assets/css/font-awesome.min.css">
+
+ <!-- Custom styles -->
+ <link rel="stylesheet" href="/assets/css/styles.css">
+ <link rel="stylesheet" href="/assets/css/docs.css">
+ <link rel="stylesheet" href="/assets/css/pygments.css">
+
+ <link rel="canonical"
href="http://kylin.apache.org/cn_blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/";>
+ <link rel="alternate" type="application/rss+xml" title="Apache Kylin"
href="http://kylin.apache.org/feed.xml"; />
+
+<!--[if lt IE 9]> <script src="assets/js/html5shiv.js"></script> <![endif]-->
+<!-- Global site tag (gtag.js) - Google Analytics -->
+<script async
src="https://www.googletagmanager.com/gtag/js?id=UA-120788561-1";></script>
+<script>
+ window.dataLayer = window.dataLayer || [];
+ function gtag(){dataLayer.push(arguments);}
+ gtag('js', new Date());
+
+ gtag('config', 'UA-120788561-1');
+</script>
+<script type="text/javascript" src="/assets/js/jquery-1.9.1.min.js"></script>
+<script type="text/javascript" src="/assets/js/nside.js"></script> </script>
+<script type="text/javascript" src="/assets/js/nnav.js"></script> </script>
+<script>
+var _hmt = _hmt || [];
+(function() {
+ var hm = document.createElement("script");
+ hm.src = "https://hm.baidu.com/hm.js?bdc5e03add430c0b72cc0eb91eabfa99";;
+ var s = document.getElementsByTagName("script")[0];
+ s.parentNode.insertBefore(hm, s);
+})();
+</script>
+
+</head>
+
+ <body>
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<header id="header" >
+
+ <!-- Main Menu -->
+ <nav class="navbar navbar-default" role="navigation" id="nav-wrapper">
+ <div class="container-fluid" id="nav">
+ <!--
+ <img class="img-circle" width="40px" height="40px" id="circlelogo"
src="/assets/images/kylin_logo.jpg">
+ -->
+ <!-- Brand and toggle get grouped for better mobile display -->
+ <div class="navbar-header">
+ <img class="navbar-logo" width="46"
src="/assets/images/kylin_logo.png" ></img>
+ <button type="button" class="navbar-toggle collapsed"
data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <ul class="nav icon-navbar">
+ <li><a href="https://twitter.com/apachekylin"; target="_blank"
class="fa fa-twitter fa-lg" title="Twitter: @ApacheKylin" ></a></li>
+ <li><a href="https://github.com/apache/kylin"; target="_blank"
class="fa fa-github-alt fa-lg" title="Github: apache/kylin" ></a></li>
+ <li><a href="https://www.facebook.com/kylinio"; target="_blank"
class="fa fa-facebook fa-lg" title="Facebook: kylin.io" ></a></li>
+ </ul>
+ </div>
+
+ <!-- Collect the nav links, forms, and other content for toggling -->
+ <div class="navbar-collapse collapse" id="bs-example-navbar-collapse-1">
+
+ <ul class="nav navbar-nav">
+
+ <li><a href="/">Home</a></li>
+ <li>
+ <a href="/docs" class="dropdown-toggle" data-toggle="dropdown"
role="button" aria-haspopup="true" aria-expanded="false">Docs<span
class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/docs/">Latest Release(Kylin 4.0.0)</a></li>
+ <li><a href="/docs31/">Kylin 3.1.2</a></li>
+ <li><a href="/docs24/">Kylin 2.4.0</a></li>
+ <li><a href="/archive/">Archive</a></li>
+ </ul>
+ </li>
+ <li><a href="/download">Download</a></li>
+ <li><a href="/community" >Community</a></li>
+ <li>
+ <a href="/development" class="dropdown-toggle"
data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Development<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/development40/">Kylin 4.x</a></li>
+ <li><a href="/development/">Kylin 3.x And Older Versions</a></li>
+ </ul>
+ </li>
+ <li><a href="/blog">Blog</a></li>
+ <li><a href="/cn" >ä¸æç</a></li>
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </div><!-- /.container-fluid -->
+ </nav>
+
+ <div id="head" class="parallax normal-header" >
+ <div class="text-center header-apache">
+ <a href="http://apache.org/foundation/contributing.html"; title="Support
Apache" style="margin-left: 150px;">
+ <div>
+ <img src="https://www.apache.org/images/SupportApache-small.png"; >
+ </div>
+ </a>
+ </div>
+ </div>
+
+ </header>
+
+ <div class="page-content main">
+ <header style=" padding:2em 0 0 ">
+ <div class="container" >
+ <div style=" padding:0 4em">
+ <div class="blog-icon">
+ <img width="30" src="/assets/images/icon_blog_w.png">
+ </div>
+ <h4 class="index-title" style="
float:left;"><span>Apache Kylin⢠Technical Blog</span></h4>
+ </div>
+ </div>
+ </div>
+
+ <div class="container blog">
+ <div>
+ <article class="post-content" >
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<div class="post" style=" padding:2em 4em 4em 4em">
+
+ <header class="post-header">
+ <h1 class="post-title">Kylin4
äºä¸æ§è½ä¼åï¼æ¬å°ç¼åå软亲åæ§è°åº¦</h1>
+ <p class="post-meta" >Oct 21, 2021 â¢ å¼ äºå©</p>
+ </header>
+
+ <article class="post-content" >
+ <h2 id="section">01 èæ¯ä»ç»</h2>
+<p>æ¥åï¼Apache Kylin 社åºåå¸äºå
¨æ°æ¶æç Kylin 4.0ãKylin 4.0
çæ¶ææ¯æåå¨å计ç®å离ï¼è¿ä½¿å¾ kylin ç¨æ·å¯ä»¥éåæ´å
çµæ´»ã计ç®èµæºå¯ä»¥å¼¹æ§ä¼¸ç¼©çäºä¸é¨ç½²æ¹å¼æ¥è¿è¡ Kylin
4.0ãåå©äºä¸çåºç¡è®¾æ½ï¼ç¨æ·å¯ä»¥éæ©ä½¿ç¨ä¾¿å®ä¸å¯é
ç对象åå¨æ¥å¨å cube æ°æ®ï¼æ¯å¦ S3
çãä¸è¿å¨åå¨ä¸è®¡ç®å离çæ¶æä¸ï¼æ们éè¦èèå°ï¼è®¡ç®èç¹éè¿ç½ç»ä»è¿ç«¯åå¨è¯»åæ°æ®ä»ç¶æ¯ä¸ä¸ªä»£ä»·è¾å¤§çæä½ï¼å¾å¾ä¼
带æ¥æ§è½çæèã<br />
+为äºæé« Kylin 4.0
å¨ä½¿ç¨äºä¸å¯¹è±¡åå¨ä½ä¸ºåå¨æ¶çæ¥è¯¢æ§è½ï¼æ们å°è¯å¨
Kylin 4.0 çæ¥è¯¢å¼æä¸å¼å
¥æ¬å°ç¼åï¼Local
Cacheï¼æºå¶ï¼å¨æ§è¡æ¥è¯¢æ¶ï¼å°ç»å¸¸ä½¿ç¨çæ°æ®ç¼åå¨æ¬å°ç£çï¼åå°ä»è¿ç¨å¯¹è±¡åå¨ä¸æåæ°æ®å¸¦æ¥ç延è¿ï¼å®ç°æ´å¿«çæ¥è¯¢ååºï¼é¤æ¤ä¹å¤ï¼ä¸ºäºé¿å
åæ ·çæ°æ®å¨å¤§é spark executor
ä¸åæ¶ç¼å浪费ç£ç空é´ï¼å¹¶ä¸è®¡ç®èç¹å¯ä»¥æ´å¤çä»æ¬å°ç¼å读åæéæ°æ®ï¼æ们å¼å
¥äº 软äº�
�åæ§ï¼Soft Affinity
ï¼çè°åº¦çç¥ï¼æè°è½¯äº²åæ§çç¥ï¼å°±æ¯éè¿æç§æ¹æ³å¨
spark executor åæ°æ®æ件ä¹é´å»ºç«å¯¹åºå
³ç³»ï¼ä½¿å¾åæ
·çæ°æ®å¨å¤§é¨åæ
åµä¸è½å¤æ»æ¯å¨åä¸ä¸ª executor
ä¸é¢è¯»åï¼ä»èæé«ç¼åçå½ä¸çã</p>
+
+<h2 id="section-1">02 å®ç°åç</h2>
+
+<h4 id="section-2">1.æ¬å°ç¼å</h4>
+<p>å¨ Kylin 4.0 æ§è¡æ¥è¯¢æ¶ï¼ä¸»è¦ç»è¿ä»¥ä¸å 个é¶æ®µï¼å
¶ä¸ç¨è线æ 注åºäºå¯ä»¥ä½¿ç¨æ¬å°ç¼åæ¥æåæ§è½çé¶æ®µï¼</p>
+
+<p><img src="/images/blog/local-cache/Local_cache_stage.png" alt="" /></p>
+
+<ul>
+ <li>File list cacheï¼å¨ spark driver 端对 file status
è¿è¡ç¼åãå¨æ§è¡æ¥è¯¢æ¶ï¼spark driver
éè¦è¯»åæ件å表ï¼è·åä¸äºæ件信æ¯è¿è¡åç»çè°åº¦æ§è¡ï¼è¿éä¼å°
file status ä¿¡æ¯ç¼åå°æ¬å°é¿å
é¢ç¹è¯»åè¿ç¨æ件ç®å½ã</li>
+ <li>Data cacheï¼å¨ spark executor
端对æ°æ®è¿è¡ç¼åãç¨æ·å¯ä»¥è®¾ç½®å°æ°æ®ç¼åå°å
åææ¯ç£çï¼è¥è®¾ç½®ä¸ºç¼åå°å
åï¼åéè¦éå½è°å¤§ executor
memoryï¼ä¿è¯ executor æ足å¤çå
åå¯ä»¥è¿è¡æ°æ®ç¼åï¼è¥æ¯ç¼åå°ç£çï¼éè¦ç¨æ·è®¾ç½®æ°æ®ç¼åç®å½ï¼æ好设置为
SSD
ç£çç®å½ãé¤æ¤ä¹å¤ï¼ç¼åæ°æ®çæ大容éãå¤ä»½æ°éçåå¯ç±ç¨æ·é
ç½®è°æ´ã</li>
+</ul>
+
+<p>åºäºä»¥ä¸è®¾è®¡ï¼å¨ Kylin 4.0 çæ¥è¯¢å¼æ sparder ç driver
端å executor 端åå«åä¸åç±»åçç¼åï¼åºæ¬æ¶æå¦ä¸ï¼</p>
+
+<p><img src="/images/blog/local-cache/kylin4_local_cache.png" alt="" /></p>
+
+<h4 id="section-3">2.软亲åæ§è°åº¦</h4>
+<p>å¨ executor 端å data cache æ¶ï¼å¦æå¨ææç executor
ä¸é½ç¼åå
¨é¨çæ°æ®ï¼é£ä¹ç¼åæ°æ®ç大å°å°ä¼é常å¯è§ï¼æ大ç浪费ç£ç空é´ï¼åæ¶ä¹å®¹æ导è´ç¼åæ°æ®è¢«é¢ç¹æ¸
çã为äºæ大å spark executor çç¼åå½ä¸çï¼spark driver
éè¦å°åä¸æ件ç task å¨èµæºæ¡ä»¶æ»¡è¶³çæ
åµä¸å°½å¯è½è°åº¦å°åæ ·ç executorï¼è¿æ
·å¯ä»¥ä¿è¯ç¸åæ件çæ°æ®è½å¤ç¼åå¨ç¹å®çæ个æè
æå 个
executor ä¸ï¼å次读åæ¶ä¾¿å¯ä»¥éè¿ç¼å读åæ°æ®ã
<br />
+为æ¤ï¼æ们éåæ ¹æ®æ件åè®¡ç® hash ä¹ååä¸ executors num
å模çç»ææ¥è®¡ç®ç®æ executor å表ï¼å¨å¤å°ä¸ª executor
ä¸é¢åç¼åç±ç¨æ·é
ç½®çç¼åå¤ä»½æ°éå³å®ï¼ä¸è¬æ
åµä¸ï¼ç¼åå¤ä»½æ°éè¶å¤§ï¼å»ä¸ç¼åçæ¦çè¶é«ãå½ç®æ
executor åä¸å¯è¾¾æè
没æèµæºä¾è°åº¦æ¶ï¼è°åº¦ç¨åºå°åéå°
spark
çéæºè°åº¦æºå¶ä¸ãè¿ç§è°åº¦æ¹å¼ä¾¿ç§°ä¸ºè½¯äº²åæ§è°åº¦çç¥ï¼å®è½ç¶ä¸è½ä¿è¯
100% å»ä¸ç¼åï¼ä½è½å¤æææé«ç¼åå½ä�
�çï¼å¨å°½éä¸æ失æ§è½çåæä¸é¿å
full cache
浪费大éç£ç空é´ã</p>
+
+<h2 id="section-4">03 ç¸å
³é
ç½®</h2>
+<p>æ ¹æ®ä»¥ä¸åçï¼æä»¬å¨ Kylin 4.0
ä¸å®ç°äºæ¬å°ç¼å+软亲åæ§è°åº¦çåºç¡åè½ï¼å¹¶åå«åºäº
ssb æ°æ®éå tpch æ°æ®éåäºæ¥è¯¢æ§è½æµè¯ã<br />
+è¿éååºå 个æ¯è¾éè¦çé
置项ä¾ç¨æ·äºè§£ï¼å®é
使ç¨çé
ç½®å°å¨ç»å°¾é¾æ¥ä¸ç»åºï¼<br />
+-
æ¯å¦å¼å¯è½¯äº²åæ§è°åº¦çç¥ï¼kylin.query.spark-conf.spark.kylin.soft-affinity.enabled<br
/>
+-
æ¯å¦å¼å¯æ¬å°ç¼åï¼kylin.query.spark-conf.spark.hadoop.spark.kylin.local-cache.enabled<br
/>
+- Data cache çå¤ä»½æ°éï¼å³å¨å¤å°ä¸ª executor
ä¸å¯¹åä¸æ°æ®æ件è¿è¡ç¼åï¼kylin.query.spark-conf.spark.kylin.soft-affinity.replications.num<br
/>
+- ç¼åå°å
åä¸è¿æ¯æ¬å°ç®å½ï¼ç¼åå°å
å设置为
BUFFï¼ç¼åå°æ¬å°è®¾ç½®ä¸º
LOCALï¼kylin.query.spark-conf.spark.hadoop.alluxio.user.client.cache.store.type<br
/>
+-
æ大ç¼å容éï¼kylin.query.spark-conf.spark.hadoop.alluxio.user.client.cache.size</p>
+
+<h2 id="section-5">04 æ§è½å¯¹æ¯</h2>
+<p>æä»¬å¨ AWS EMR ç¯å¢ä¸è¿è¡äº 3 ç§åºæ¯çæ§è½æµè¯ï¼å¨ scale
factor = 10çæ
åµä¸ï¼å¯¹ ssb æ°æ®éè¿è¡å并åæ¥è¯¢æµè¯ãtpch
æ°æ®éè¿è¡å并åæ¥è¯¢ä»¥å 4 并åæ¥è¯¢æµè¯ï¼å®éªç»å对ç
§ç»åé
ç½® s3
ä½ä¸ºåå¨ï¼å¨å®éªç»ä¸å¼å¯æ¬å°ç¼åå软亲åæ§è°åº¦ï¼å¯¹ç
§ç»åä¸å¼å¯ãé¤æ¤ä¹å¤ï¼æ们è¿å°å®éªç»ç»æä¸ç¸åç¯å¢ä¸
hdfs
ä½ä¸ºåå¨æ¶çç»æè¿è¡å¯¹æ¯ï¼ä»¥ä¾¿ç¨æ·å¯ä»¥ç´è§çæåå°
æ¬å°ç¼å+软亲åæ§è°åº¦ 对äºä
¸é¨ç½² Kylin 4.0
使ç¨å¯¹è±¡åå¨ä½ä¸ºåå¨åºæ¯ä¸çä¼åææã</p>
+
+<p><img src="/images/blog/local-cache/local_cache_benchmark_result_ssb.png"
alt="" /></p>
+
+<p><img src="/images/blog/local-cache/local_cache_benchmark_result_tpch1.png"
alt="" /></p>
+
+<p><img src="/images/blog/local-cache/local_cache_benchmark_result_tpch4.png"
alt="" /></p>
+
+<p>ä»ä»¥ä¸ç»æå¯ä»¥çåºï¼<br />
+1. å¨ ssb 10 æ°æ®éå并ååºæ¯ä¸ï¼ä½¿ç¨ s3
ä½ä¸ºåå¨æ¶ï¼å¼å¯æ¬å°ç¼åå软亲åæ§è°åº¦è½å¤è·å¾3åå·¦å³çæ§è½æåï¼å¯ä»¥è¾¾å°ä¸
hdfs ä½ä¸ºåå¨æ¶çç¸åæ§è½çè³è¿æ 5% å·¦å³çæåã<br />
+2. å¨ tpch 10 æ°æ®éä¸ï¼ä½¿ç¨ s3 ä½ä¸ºåå¨æ¶ï¼æ
论æ¯å并åæ¥è¯¢è¿æ¯å¤å¹¶åæ¥è¯¢ï¼å¼å¯æ¬å°ç¼åå软亲åæ§è°åº¦åï¼åºæ¬å¨æææ¥è¯¢ä¸é½è½å¤è·å¾å¤§å¹
度çæ§è½æåã</p>
+
+<p>ä¸è¿å¨ tpch 10 æ°æ®éç 4 并åæµè¯ä¸ç Q21
ç对æ¯ç»æä¸ï¼æ们è§å¯å°ï¼å¼å¯æ¬å°ç¼åå软亲åæ§è°åº¦çç»æåèæ¯åç¬ä½¿ç¨
s3 ä½ä¸ºåå¨æ¶ææä¸éï¼è¿éå¯è½æ¯ç±äºæç§åå
导è´æ²¡æéè¿ç¼å读åæ°æ®ï¼æ·±å±åå
å¨æ¤æ¬¡æµè¯ä¸æ²¡æè¿è¡è¿ä¸æ¥çåæï¼å¨åç»çä¼åè¿ç¨ä¸æ们ä¼éæ¥æ¹è¿ãç±äº
tpch çæ¥è¯¢æ¯è¾å¤æä¸ SQL ç±»ååå¼ï¼ä¸ hdfs
ä½ä¸ºåå¨æ¶çç»æç¸æ¯ï¼ä»ç¶æé¨å sql çæ§è½ç¥æä�
�足ï¼ä¸è¿æ»ä½æ¥è¯´å·²ç»ä¸ hdfs çç»ææ¯è¾æ¥è¿ã<br />
+æ¬æ¬¡æ§è½æµè¯çç»ææ¯ä¸æ¬¡å¯¹ æ¬å°ç¼å+软亲åæ§è°åº¦
æ§è½æåææçåæ¥éªè¯ï¼ä»æ»ä½ä¸æ¥çï¼æ¬å°ç¼å+软亲åæ§è°åº¦
æ
论对äºç®åæ¥è¯¢è¿æ¯å¤ææ¥è¯¢é½è½å¤è·å¾ææ¾çæ§è½æåï¼ä½æ¯å¨é«å¹¶åæ¥è¯¢åºæ¯ä¸åå¨ä¸å®çæ§è½æ失ã<br
/>
+å¦æç¨æ·ä½¿ç¨äºä¸å¯¹è±¡åå¨ä½ä¸º Kylin 4.0 çåå¨ï¼å¨å¼å¯
æ¬å°ç¼å+ 软亲åæ§è°åº¦çæ
åµä¸ï¼æ¯å¯ä»¥è·å¾å¾å¥½çæ§è½ä½éªçï¼è¿ä¸º Kylin 4.0
å¨äºä¸ä½¿ç¨è®¡ç®ååå¨å离æ¶ææä¾äºæ§è½ä¿éã</p>
+
+<h2 id="section-6">05 代ç å®ç°</h2>
+<p>ç±äºç®åç代ç
å®ç°è¿å¤äºæ¯è¾åºç¡çé¶æ®µï¼è¿æ许å¤ç»èéè¦å®åï¼æ¯å¦å®ç°ä¸è´æ§åå¸ãå½
executor æ°éåçååæ¶å¦ä½å¤çå·²æ cache çï¼æ以ä½è
è¿æªå社åºä»£ç åºæ交 PRï¼æ³è¦æåé¢è§çå¼åè
å¯ä»¥éè¿ä¸é¢çé¾æ¥æ¥çæºç ï¼<br />
+<a
href="https://github.com/zzcclp/kylin/commit/4e75b7fa4059dd2eaed24061fda7797fecaf2e35";>Kylin4.0
æ¬å°ç¼å+软亲åæ§è°åº¦ä»£ç å®ç°</a></p>
+
+<h2 id="section-7">06 ç¸å
³é¾æ¥</h2>
+<p>éè¿é¾æ¥å¯æ¥é
æ§è½æµè¯ç»ææ°æ®åå
·ä½é
ç½®ï¼<br />
+<a href="https://github.com/Kyligence/kylin-tpch/issues/9";>Kylin4.0
æ¬å°ç¼å+软亲åæ§è°åº¦æµè¯</a></p>
+
+ </article>
+
+</div>
+
+
+
+
+
+ </article>
+ </div>
+ </div>
+ <!--
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+-->
+
+<footer id="underfooter">
+ <div>
+ <div class="row">
+ <div class="col-md-12 widget">
+ <div class="widget-body">
+ <div class="footer-img">
+ <a href="http://www.apache.org";>
+ <img id="asf-logo" height="78px" alt="Apache
Software Foundation" src="/assets/images/apache_footer.png">
+ </a>
+ </div>
+ <p style="padding-top: 11px;">
+ The contents of this website are © 2015 Apache
Software Foundation under the terms of the
+ <a href="http://www.apache.org/licenses/LICENSE-2.0";>
Apache License v2 </a>.
+ </p>
+ <p style="margin-bottom: 11px;">
+ Apache Kylin and its logo are trademarks of the Apache
Software Foundation.
+ </div>
+
+ </div>
+ </div>
+ </div>
+ <!-- /row of widgets -->
+
+ </div>
+ <div></div>
+
+</footer>
+
+ <script src="/assets/js/jquery-1.9.1.min.js"></script>
+ <script src="/assets/js/bootstrap.min.js"></script>
+ <script src="/assets/js/main.js"></script>
+ </body>
+</html>
+
+
+
+
![https://www.apache.org/images/SupportApache-small.png"](https://www.apache.org/images/SupportApache-small.png"){kind=link}