Repository: kylin Updated Branches: refs/heads/1.x-staging eb5deb31e -> 7d3c4f456
KYLIN-1285 Initial release note for 2.0-alpha Project: http://git-wip-us.apache.org/repos/asf/kylin/repo Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/7d3c4f45 Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/7d3c4f45 Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/7d3c4f45 Branch: refs/heads/1.x-staging Commit: 7d3c4f456180408279ed1a8eb2a22b09932c2827 Parents: eb5deb3 Author: Yang Li <liy...@apache.org> Authored: Tue Feb 9 21:41:43 2016 +0800 Committer: Yang Li <liy...@apache.org> Committed: Tue Feb 9 21:41:43 2016 +0800 ---------------------------------------------------------------------- website/_docs/release_notes.md | 304 +++++++++++++++++++++++++++++++++++- 1 file changed, 303 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kylin/blob/7d3c4f45/website/_docs/release_notes.md ---------------------------------------------------------------------- diff --git a/website/_docs/release_notes.md b/website/_docs/release_notes.md index 04cf209..3adea81 100644 --- a/website/_docs/release_notes.md +++ b/website/_docs/release_notes.md @@ -3,7 +3,7 @@ layout: docs title: Apache Kylin⢠Release Notes categories: gettingstarted permalink: /docs/release_notes.html -version: v1.2 +version: v2.0 since: v0.7.1 --- @@ -16,6 +16,308 @@ or send to Apache Kylin mailing list: * User relative: [u...@kylin.apache.org](mailto:u...@kylin.apache.org) * Development relative: [d...@kylin.apache.org](mailto:d...@kylin.apache.org) + +## v2.0-alpha - 2016-02-09 +_Tag:_ [kylin-2.0-alpha](https://github.com/apache/kylin/tree/kylin-2.0-alpha) + +__Highlights__ + + * [KYLIN-875] - A plugin-able architecture, to allow alternative cube engine / storage engine / data source. + * [KYLIN-1245] - A better MR cubing algorithm, about 1.5 times faster than 1.x by comparing hundreds of jobs. + * [KYLIN-942] - A better storage engine, makes query roughly 2 times faster (especially for slow queries) than 1.x by comparing tens of thousands sqls. + * [KYLIN-738] - Streaming cubing EXPERIMENTAL support, source from kafka, build cube in-mem at minutes interval + * [KYLIN-943] - TopN pre-calculation (more UDFs coming) + * [KYLIN-1065] - ODBC compatible with Tableau 9.1, MS Excel, MS PowerBI + * [KYLIN-1219] - Kylin support SSO with Spring SAML + +__Below generated from JIRA system, pending manual revision.__ + +__New Feature__ + + * [KYLIN-196] - Support Job Priority + * [KYLIN-528] - Build job flow for Inverted Index building + * [KYLIN-596] - Support Excel and Power BI + * [KYLIN-599] - Near real-time support + * [KYLIN-603] - Add mem store for seconds data latency + * [KYLIN-606] - Block level index for Inverted-Index + * [KYLIN-607] - More efficient cube building + * [KYLIN-609] - Add Hybrid as a federation of Cube and Inverted-index realization + * [KYLIN-625] - Create GridTable, a data structure that abstracts vertical and horizontal partition of a table + * [KYLIN-728] - IGTStore implementation which use disk when memory runs short + * [KYLIN-738] - StreamingOLAP + * [KYLIN-749] - support timestamp type in II and cube + * [KYLIN-774] - Automatically merge cube segments + * [KYLIN-868] - add a metadata backup/restore script in bin folder + * [KYLIN-886] - Data Retention for streaming data + * [KYLIN-906] - cube retention + * [KYLIN-943] - Approximate TopN supported by Cube + * [KYLIN-986] - Generalize Streaming scripts and put them into code repository + * [KYLIN-1219] - Kylin support SSO with Spring SAML + * [KYLIN-1277] - Upgrade tool to put old-version cube and new-version cube into a hybrid model + +__Improvement__ + + * [KYLIN-225] - Support edit "cost" of cube + * [KYLIN-589] - Cleanup Intermediate hive table after cube build + * [KYLIN-623] - update Kylin UI Style to latest AdminLTE + * [KYLIN-633] - Support Timestamp for cube partition + * [KYLIN-649] - move the cache layer from service tier back to storage tier + * [KYLIN-655] - Migrate cube storage (query side) to use GridTable API + * [KYLIN-663] - Push time condition down to ii endpoint + * [KYLIN-668] - Out of memory in mapper when building cube in mem + * [KYLIN-671] - Implement fine grained cache for cube and ii + * [KYLIN-673] - Performance tuning for In-Mem cubing + * [KYLIN-674] - IIEndpoint return metrics as well + * [KYLIN-675] - cube&model designer refactor + * [KYLIN-678] - optimize RowKeyColumnIO + * [KYLIN-697] - Reorganize all test cases to unit test and integration tests + * [KYLIN-702] - When Kylin create the flat hive table, it generates large number of small files in HDFS + * [KYLIN-708] - replace BitSet for AggrKey + * [KYLIN-712] - some enhancement after code review + * [KYLIN-717] - optimize OLAPEnumerator.convertCurrentRow() + * [KYLIN-718] - replace aliasMap in storage context with a clear specified return column list + * [KYLIN-719] - bundle statistics info in endpoint response + * [KYLIN-720] - Optimize endpoint's response structure to suit with no-dictionary data + * [KYLIN-721] - streaming cli support third-party streammessage parser + * [KYLIN-726] - add remote cli port configuration for KylinConfig + * [KYLIN-729] - IIEndpoint eliminate the non-aggregate routine + * [KYLIN-734] - Push cache layer to each storage engine + * [KYLIN-752] - Improved IN clause performance + * [KYLIN-753] - Make the dependency on hbase-common to "provided" + * [KYLIN-755] - extract copying libs from prepare.sh so that it can be reused + * [KYLIN-760] - Improve the hasing performance in Sampling cuboid size + * [KYLIN-772] - Continue cube job when hive query return empty resultset + * [KYLIN-773] - performance is slow list jobs + * [KYLIN-783] - update hdp version in test cases to 2.2.4 + * [KYLIN-796] - Add REST API to trigger storage cleanup/GC + * [KYLIN-809] - Streaming cubing allow multiple kafka clusters/topics + * [KYLIN-816] - Allow gap in cube segments, for streaming case + * [KYLIN-822] - list cube overview in one page + * [KYLIN-823] - replace fk on fact table on rowkey & aggregation group generate + * [KYLIN-838] - improve performance of job query + * [KYLIN-844] - add backdoor toggles to control query behavior + * [KYLIN-845] - Enable coprocessor even when there is memory hungry distinct count + * [KYLIN-858] - add snappy compression support + * [KYLIN-866] - Confirm with user when he selects empty segments to merge + * [KYLIN-869] - Enhance mail notification + * [KYLIN-870] - Speed up hbase segments info by caching + * [KYLIN-871] - growing dictionary for streaming case + * [KYLIN-874] - script for fill streaming gap automatically + * [KYLIN-875] - Decouple with Hadoop to allow alternative Input / Build Engine / Storage + * [KYLIN-879] - add a tool to collect orphan hbases + * [KYLIN-880] - Kylin should change the default folder from /tmp to user configurable destination + * [KYLIN-881] - Upgrade Calcite to 1.3.0 + * [KYLIN-882] - check access to kylin.hdfs.working.dir + * [KYLIN-883] - Using configurable option for Hive intermediate tables created by Kylin job + * [KYLIN-893] - Remove the dependency on quartz and metrics + * [KYLIN-895] - Add "retention_range" attribute for cube instance, and automatically drop the oldest segment when exceeds retention + * [KYLIN-896] - Clean ODBC code, add them into main repository and write docs to help compiling + * [KYLIN-901] - Add tool for cleanup Kylin metadata storage + * [KYLIN-902] - move streaming related parameters into StreamingConfig + * [KYLIN-903] - automate metadata cleanup job + * [KYLIN-909] - Adapt GTStore to hbase endpoint + * [KYLIN-919] - more friendly UI for 0.8 + * [KYLIN-922] - Enforce same code style for both intellij and eclipse user + * [KYLIN-926] - Make sure Kylin leaves no garbage files in local OS and HDFS/HBASE + * [KYLIN-927] - Real time cubes merging skipping gaps + * [KYLIN-933] - friendly UI to use data model + * [KYLIN-938] - add friendly tip to page when rest request failed + * [KYLIN-942] - Cube parallel scan on Hbase + * [KYLIN-956] - Allow users to configure hbase compression algorithm in kylin.properties + * [KYLIN-957] - Support HBase in a separate cluster + * [KYLIN-960] - Split storage module to core-storage and storage-hbase + * [KYLIN-973] - add a tool to analyse streaming output logs + * [KYLIN-984] - Behavior change in streaming data consuming + * [KYLIN-987] - Rename 0.7-staging and 0.8 branch + * [KYLIN-1014] - Support kerberos authentication while getting status from RM + * [KYLIN-1018] - make TimedJsonStreamParser default parser + * [KYLIN-1019] - Remove v1 cube model classes from code repository + * [KYLIN-1021] - upload dependent jars of kylin to HDFS and set tmpjars + * [KYLIN-1025] - Save cube change is very slow + * [KYLIN-1036] - Code Clean, remove code which never used at front end + * [KYLIN-1041] - ADD Streaming UI + * [KYLIN-1048] - CPU and memory killer in Cuboid.findById() + * [KYLIN-1058] - Remove "right join" during model creation + * [KYLIN-1061] - "kylin.sh start" should check whether kylin has already been running + * [KYLIN-1064] - restore disabled queries in KylinQueryTest.testVerifyQuery + * [KYLIN-1065] - ODBC driver support tableau 9.1 + * [KYLIN-1068] - Optimize the memory footprint for TopN counter + * [KYLIN-1069] - update tip for 'Partition Column' on UI + * [KYLIN-1095] - Update AdminLTE to latest version + * [KYLIN-1096] - Deprecate minicluster in 2.x staging + * [KYLIN-1099] - Support dictionary of cardinality over 10 millions + * [KYLIN-1101] - Allow "YYYYMMDD" as a date partition column + * [KYLIN-1105] - Cache in AbstractRowKeyEncoder.createInstance() is useless + * [KYLIN-1116] - Use local dictionary for InvertedIndex batch building + * [KYLIN-1119] - refine find-hive-dependency.sh to correctly get hcatalog path + * [KYLIN-1126] - v2 storage(for parallel scan) backward compatibility with v1 storage + * [KYLIN-1135] - Pscan use share thread pool + * [KYLIN-1136] - Distinguish fast build mode and complete build mode + * [KYLIN-1139] - Hive job not starting due to error "conflicting lock present for default mode EXCLUSIVE " + * [KYLIN-1149] - When yarn return an incomplete job tracking URL, Kylin will fail to get job status + * [KYLIN-1154] - Load job page is very slow when there are a lot of history job + * [KYLIN-1157] - CubeMigrationCLI doesn't copy ACL + * [KYLIN-1160] - Set default logger appender of log4j for JDBC + * [KYLIN-1161] - Rest API /api/cubes?cubeName= is doing fuzzy match instead of exact match + * [KYLIN-1162] - Enhance HadoopStatusGetter to be compatible with YARN-2605 + * [KYLIN-1190] - Make memory budget per query configurable + * [KYLIN-1234] - Cube ACL does not work + * [KYLIN-1235] - allow user to select dimension column as options when edit COUNT_DISTINCT measure + * [KYLIN-1237] - Revisit on cube size estimation + * [KYLIN-1239] - attribute each htable with team contact and owner name + * [KYLIN-1244] - In query window, enable fast copy&paste by double clicking tables/columns' names. + * [KYLIN-1245] - Switch between layer cubing and in-mem cubing according to stats + * [KYLIN-1246] - get cubes API update - offset,limit not required + * [KYLIN-1251] - add toggle event for tree label + * [KYLIN-1259] - Change font/background color of job progress + * [KYLIN-1265] - Make sure 2.0 query is no slower than 1.0 + * [KYLIN-1266] - Tune 2.0 release package size + * [KYLIN-1267] - Check Kryo performance when spilling aggregation cache + * [KYLIN-1268] - Fix 2 kylin logs + * [KYLIN-1270] - improve TimedJsonStreamParser to support month_start,quarter_start,year_start + * [KYLIN-1281] - Add "partition_date_end", and move "partition_date_start" into cube descriptor + * [KYLIN-1283] - Replace GTScanRequest's SerDer form Kryo to manual + * [KYLIN-1287] - UI update for streaming build action + * [KYLIN-1297] - Diagnose query performance issues in 2.x versions + * [KYLIN-1301] - fix segment pruning failure in 2.x versions + * [KYLIN-1308] - query storage v2 enable parallel cube visiting + * [KYLIN-1312] - Enhance DeployCoprocessorCLI to support Cube level filter + * [KYLIN-1318] - enable gc log for kylin server instance + * [KYLIN-1323] - Improve performance of converting data to hfile + * [KYLIN-1327] - Tool for batch updating host information of htables + * [KYLIN-1334] - allow truncating string for fixed length dimensions + * [KYLIN-1341] - Display JSON of Data Model in the dialog + * [KYLIN-1350] - hbase Result.binarySearch is found to be problematic in concurrent environments + * [KYLIN-1368] - JDBC Driver is not generic to restAPI json result + +__Bug__ + + * [KYLIN-404] - Can't get cube source record size. + * [KYLIN-457] - log4j error and dup lines in kylin.log + * [KYLIN-521] - No verification even if join condition is invalid + * [KYLIN-632] - "kylin.sh stop" doesn't check whether KYLIN_HOME was set + * [KYLIN-635] - IN clause within CASE when is not working + * [KYLIN-656] - REST API get cube desc NullPointerException when cube is not exists + * [KYLIN-660] - Make configurable of dictionary cardinality cap + * [KYLIN-665] - buffer error while in mem cubing + * [KYLIN-688] - possible memory leak for segmentIterator + * [KYLIN-731] - Parallel stream build will throw OOM + * [KYLIN-740] - Slowness with many IN() values + * [KYLIN-747] - bad query performance when IN clause contains a value doesn't exist in the dictionary + * [KYLIN-748] - II returned result not correct when decimal omits precision and scal + * [KYLIN-751] - Max on negative double values is not working + * [KYLIN-766] - round BigDecimal according to the DataType scale + * [KYLIN-769] - empty segment build fail due to no dictionary + * [KYLIN-771] - query cache is not evicted when metadata changes + * [KYLIN-778] - can't build cube after package to binary + * [KYLIN-780] - Upgrade Calcite to 1.0 + * [KYLIN-797] - Cuboid cache will cache massive invalid cuboid if existed many cubes which already be deleted + * [KYLIN-801] - fix remaining issues on query cache and storage cache + * [KYLIN-805] - Drop useless Hive intermediate table and HBase tables in the last step of cube build/merge + * [KYLIN-807] - Avoid write conflict between job engine and stream cube builder + * [KYLIN-817] - Support Extract() on timestamp column + * [KYLIN-824] - Cube Build fails if lookup table doesn't have any files under HDFS location + * [KYLIN-828] - kylin still use ldap profile when comment the line "kylin.sandbox=false" in kylin.properties + * [KYLIN-834] - optimize StreamingUtil binary search perf + * [KYLIN-837] - fix submit build type when refresh cube + * [KYLIN-873] - cancel button does not work when [resume][discard] job + * [KYLIN-889] - Support more than one HDFS files of lookup table + * [KYLIN-897] - Update CubeMigrationCLI to copy data model info + * [KYLIN-898] - "CUBOID_CACHE" in Cuboid.java never flushes + * [KYLIN-905] - Boolean type not supported + * [KYLIN-911] - NEW segments not DELETED when cancel BuildAndMerge Job + * [KYLIN-912] - $KYLIN_HOME/tomcat/temp folder takes much disk space after long run + * [KYLIN-913] - Cannot find rowkey column XXX in cube CubeDesc + * [KYLIN-914] - Scripts shebang should use /bin/bash + * [KYLIN-918] - Calcite throws "java.lang.Float cannot be cast to java.lang.Double" error while executing SQL + * [KYLIN-929] - can not sort cubes by [Source Records] at cubes list page + * [KYLIN-930] - can't see realizations under each project at project list page + * [KYLIN-934] - Negative number in SUM result and Kylin results not matching exactly Hive results + * [KYLIN-935] - always loading when try to view the log of the sub-step of cube build job + * [KYLIN-936] - can not see job step log + * [KYLIN-944] - update doc about how to consume kylin API in javascript + * [KYLIN-946] - [UI] refresh page show no results when Project selected as [--Select All--] + * [KYLIN-950] - Web UI "Jobs" tab view the job reduplicated + * [KYLIN-951] - Drop RowBlock concept from GridTable general API + * [KYLIN-952] - User can trigger a Refresh job on an non-existing cube segment via REST API + * [KYLIN-967] - Dump running queries on memory shortage + * [KYLIN-975] - change kylin.job.hive.database.for.intermediatetable cause job to fail + * [KYLIN-978] - GarbageCollectionStep dropped Hive Intermediate Table but didn't drop external hdfs path + * [KYLIN-982] - package.sh should grep out "Download*" messages when determining version + * [KYLIN-983] - Query sql offset keyword bug + * [KYLIN-985] - Don't suppoprt aggregation AVG while executing SQL + * [KYLIN-991] - StorageCleanupJob may clean a newly created HTable in streaming cube building + * [KYLIN-992] - ConcurrentModificationException when initializing ResourceStore + * [KYLIN-1001] - Kylin generates wrong HDFS path in creating intermediate table + * [KYLIN-1004] - Dictionary with '' value cause cube merge to fail + * [KYLIN-1020] - Although "kylin.query.scan.threshold" is set, it still be restricted to less than 4 million + * [KYLIN-1026] - Error message for git check is not correct in package.sh + * [KYLIN-1027] - HBase Token not added after KYLIN-1007 + * [KYLIN-1033] - Error when joining two sub-queries + * [KYLIN-1039] - Filter like (A or false) yields wrong result + * [KYLIN-1047] - Upgrade to Calcite 1.4 + * [KYLIN-1066] - Only 1 reducer is started in the "Build cube" step of MR_Engine_V2 + * [KYLIN-1067] - Support get MapReduce Job status for ResourceManager HA Env + * [KYLIN-1075] - select [MeasureCol] from [FactTbl] is not supported + * [KYLIN-1078] - UI - Cannot have comments in the end of New Query textbox + * [KYLIN-1093] - Consolidate getCurrentHBaseConfiguration() and newHBaseConfiguration() in HadoopUtil + * [KYLIN-1106] - Can not send email caused by Build Base Cuboid Data step failed + * [KYLIN-1108] - Return Type Empty When Measure-> Count In Cube Design + * [KYLIN-1113] - Support TopN query in v2/CubeStorageQuery.java + * [KYLIN-1115] - Clean up ODBC driver code + * [KYLIN-1121] - ResourceTool download/upload does not work in binary package + * [KYLIN-1127] - Refactor CacheService + * [KYLIN-1137] - TopN measure need support dictionary merge + * [KYLIN-1138] - Bad CubeDesc signature cause segment be delete when enable a cube + * [KYLIN-1140] - Kylin's sample cube "kylin_sales_cube" couldn't be saved. + * [KYLIN-1151] - Menu items should be aligned when create new model + * [KYLIN-1152] - ResourceStore should read content and timestamp in one go + * [KYLIN-1153] - Upgrade is needed for cubedesc metadata from 1.x to 2.0 + * [KYLIN-1171] - KylinConfig truncate bug + * [KYLIN-1179] - Cannot use String as partition column + * [KYLIN-1180] - Some NPE in Dictionary + * [KYLIN-1181] - Split metadata size exceeded when data got huge in one segment + * [KYLIN-1192] - Cannot edit data model desc without name change + * [KYLIN-1205] - hbase RpcClient java.io.IOException: Unexpected closed connection + * [KYLIN-1211] - Add 'Enable Cache' button in System page + * [KYLIN-1216] - Can't parse DateFormat like 'YYYYMMDD' correctly in query + * [KYLIN-1218] - java.lang.NullPointerException in MeasureTypeFactory when sync hive table + * [KYLIN-1220] - JsonMappingException: Can not deserialize instance of java.lang.String out of START_ARRAY + * [KYLIN-1225] - Only 15 cubes listed in the /models page + * [KYLIN-1226] - InMemCubeBuilder throw OOM for multiple HLLC measures + * [KYLIN-1230] - When CubeMigrationCLI copied ACL from one env to another, it may not work + * [KYLIN-1236] - redirect to home page when input invalid url + * [KYLIN-1250] - Got NPE when discarding a job + * [KYLIN-1260] - Job status labels are not in same style + * [KYLIN-1269] - Can not get last error message in email + * [KYLIN-1271] - Create streaming table layer will disappear if click on outside + * [KYLIN-1274] - Query from JDBC is partial results by default + * [KYLIN-1282] - Comparison filter on Date/Time column not work for query + * [KYLIN-1289] - Click on subsequent wizard steps doesn't work when editing existing cube or model + * [KYLIN-1303] - Error when in-mem cubing on empty data source which has boolean columns + * [KYLIN-1306] - Null strings are not applied during fast cubing + * [KYLIN-1314] - Display issue for aggression groups + * [KYLIN-1315] - UI: Cannot add normal dimension when creating new cube + * [KYLIN-1316] - Wrong label in Dialog CUBE REFRESH CONFIRM + * [KYLIN-1317] - Kill underlying running hadoop job while discard a job + * [KYLIN-1328] - "UnsupportedOperationException" is thrown when remove a data model + * [KYLIN-1330] - UI create model: Press enter will go back to pre step + * [KYLIN-1336] - 404 errors of model page and api 'access/DataModelDesc' in console + * [KYLIN-1337] - Sort cube name doesn't work well + * [KYLIN-1346] - IllegalStateException happens in SparkCubing + * [KYLIN-1347] - UI: cannot place cursor in front of the last dimension + * [KYLIN-1349] - 'undefined' is logged in console when adding lookup table + * [KYLIN-1352] - 'Cache already exists' exception in high-concurrency query situation + * [KYLIN-1356] - use exec-maven-plugin for IT environment provision + * [KYLIN-1357] - Cloned cube has build time information + * [KYLIN-1372] - Query using PrepareStatement failed with multi OR clause + * [KYLIN-1382] - CubeMigrationCLI reports error when migrate cube + * [KYLIN-1396] - minor bug in BigDecimalSerializer - avoidVerbose should be incremented each time when input scale is larger than given scale + * [KYLIN-1400] - kylin.metadata.url with hbase namespace problem + * [KYLIN-1402] - StringIndexOutOfBoundsException in Kylin Hive Column Cardinality Job + * [KYLIN-1414] - Couldn't drag and drop rowkey, js error is thrown in browser console + + ## v1.2 - 2015-12-15 _Tag:_ [kylin-1.2](https://github.com/apache/kylin/tree/kylin-1.2)
