-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------
(Updated July 13, 2015, 9:48 a.m.)
Review request for lens.
Bugs: LENS-619
https://issues.apache.org/jira/browse/LENS-619
Repository: lens
Description (updated)
-------
Lens will always accept a new query from a user and put it in a scheduling
queue for processing.
Next candidate query picked up from scheduling queue for processing will be
launched only if all query constraints evaluated on candidate query and
launched queries allows the candidate query to be launched, otherwise the
candidate query will be added to waiting queries.
When any launched query is finished, a waiting query selector will select
waiting queries for rescheduling using a list of waiting queries selection
policy. Every waiting query selection policy will return a list of eligible
waiting queries to be rescheduled. Query Selector will calculate intersection
of multiple list of eligible waiting queries and add the result of intersection
to scheduling queue for reprocessing.
At initialization, Query Constraints and Waiting Query Selection Policies will
be configured using configuration values.
New Query Constraints and Waiting Query Selection Policies can be added at
runtime, without rebuilding and deploying lens module. Drivers should be
allowed to add more Query Constraints and Waiting Queries Selection policies.
Waiting Queries will be persisted across server restarts.
Query Constraint 1: Allow a candidate query to be launched, only if, cumulative
query cost of all current queries launched by the user, who submitted candidate
query, is less than a cumulative query cost ceiling for launching a new query.
Waiting Query Selection Policy 1: Select all waiting queries of the user whose
query has finished.
Query Constraint 2: Allow a candidate query to be launched, only if, current
concurrent queries launched on the
selected driver are less than max concurrent queries allowed on the driver.
Waiting Query Selection Policy 2: Select all waiting queries of the driver
whose query has finished.
Diffs
-----
lens-api/src/main/java/org/apache/lens/api/Addable.java
5a556918afcdb8967deee84472cd7863aeafc4fc
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java
f594133e8a75603e5bb0c913d62ea189248cfa05
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java
539f0b0b0804257fae6b7ec0af8f2f715175d597
lens-client/src/main/java/org/apache/lens/client/LensStatement.java
f7305fb288167cda088e263a62c77f264d779127
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java
e87ed23429d3172db86c17e4ec5e37e2e903374b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java
0f2b7401b4057b3a93aa5a13066972bc1975142e
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml
99f6f205dfe6f1c16277ac6624106436df2b2b8f
lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
ad27d78e1839fa60df8f2429901b77221be71653
lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java
b56c67ce6909a11b1cdec9f52427683f0b11dd41
lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java
c3fe8eb142569c0b254f551c3cbd18502b6eac99
lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java
71128b7d35e91fadf38a9c3926ce46d42369ead4
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
0cccaef5abf391179b04bca0c0bbe799429f9065
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
6e7ab5dc1c78407a24e389c001ca3100563e3795
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
d72727e7832d02ed2ea90379cd29c7a8e509a8f8
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java
5d7163f112d7c7115ebf8f012ba397bfee63adbc
lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java
7fa5cdb55959a44b52a29884fd629c2479fdfb96
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java
be7232134b59f7126edba569358389a9c26a5c74
lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java
PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java
f9262994debe0b943123d13c58a22c10b2015e80
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java
b52b08ad78a4c74c61376fb3c6bc6284f38b0de9
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java
PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java
0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322
lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java
PRE-CREATION
lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java
39f9f5216acf6e1be7cca6a0851152a78ba66176
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java
190435036c5cf8f009d7fbd536ef1743b70c22d9
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
8a322027905edef9403c45fb13cb985e7e4a2f59
lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java
PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java
715347fab82b5d9d4ad673c45e72900b9a95fc81
lens-server/src/main/resources/lensserver-default.xml
da00c0d2d94ef982646ad10f142165abe9125d3a
lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java
PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java
PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java
PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java
PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java
PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java
PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java
PRE-CREATION
lens-server/src/test/resources/lens-site.xml
3bcbf0e3e44d986a24c00cad84121be72596c64d
lens-server/src/test/resources/log4j.properties
e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b
Diff: https://reviews.apache.org/r/36393/diff/
Testing
-------
Thanks,
Himanshu Gahlaut
