----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36393/#review91731 -----------------------------------------------------------
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (line 96) <https://reviews.apache.org/r/36393/#comment145321> Can the config name include launching? Can you document the new configs added in hivedriver-default.xml and generate apt files? For me, giving comma separated factories - does not look intuitive than to give implementations. But we can hold on that, until i see more code down. lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml (line 224) <https://reviews.apache.org/r/36393/#comment145322> This should just come from max pool size incase of JDBCDriver. These two values cannot be different. lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java (line 123) <https://reviews.apache.org/r/36393/#comment145323> Is this still emitted after removing WAITING state? lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java (lines 91 - 97) <https://reviews.apache.org/r/36393/#comment145324> Any reason for doing this change - materializing in context than reading from driverContext ? lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java (line 350) <https://reviews.apache.org/r/36393/#comment145330> Why are removing synchronized here? We want to make sure setStatus is not done by multiple threads through synchronizing on context lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java (lines 432 - 437) <https://reviews.apache.org/r/36393/#comment145325> Why are these required after driver selection? lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java (line 30) <https://reviews.apache.org/r/36393/#comment145326> What do we mean by estimate is not run yet? Does it enforce querycost is available with query? If so, can we put those details? lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java (line 119) <https://reviews.apache.org/r/36393/#comment145327> This change here has to be published as incompatible and provide an alter command for existing tables in production - as part of release. lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 638) <https://reviews.apache.org/r/36393/#comment145329> I see that doing validation early is failing early wrt invalid trasitions. But how are we making sure the state is fine by this call ? The calls addSessionResourcesToDriver(query); query.getSelectedDriver().executeAsync(query); can be expensive. lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (lines 691 - 692) <https://reviews.apache.org/r/36393/#comment145328> StatusPoller works on a copy for iterating it can move finished queries out of launchedQueries - and it should not result in ConcurrentModificationException for iteration. Not sure how it will work after this change put. lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 770) <https://reviews.apache.org/r/36393/#comment145332> Should process next waiting query happen after following data structure updates? Also, can you check if updateFinishedQuery(ctx) is done with ctx locked. If so, processing next waiting query should not require this lock. Might have to after the lock is given out. lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 854) <https://reviews.apache.org/r/36393/#comment145333> The event time sent here is the when was query queued - which is submission time. not current time of event raising. - Amareshwari Sriramadasu On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/36393/ > ----------------------------------------------------------- > > (Updated July 15, 2015, 8:55 a.m.) > > > Review request for lens. > > > Bugs: LENS-619 > https://issues.apache.org/jira/browse/LENS-619 > > > Repository: lens > > > Description > ------- > > LENS-619: Applying Query Launching Constraints before allowing a query to be > launched > > > Diffs > ----- > > lens-api/src/main/java/org/apache/lens/api/Addable.java > 5a556918afcdb8967deee84472cd7863aeafc4fc > lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java > f594133e8a75603e5bb0c913d62ea189248cfa05 > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java > 539f0b0b0804257fae6b7ec0af8f2f715175d597 > lens-api/src/main/resources/lens-errors.conf > 6130ad78c2871f78ff9bc11cca5702a538639bbf > lens-client/src/main/java/org/apache/lens/client/LensStatement.java > f7305fb288167cda088e263a62c77f264d779127 > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java > e87ed23429d3172db86c17e4ec5e37e2e903374b > lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java > 0f2b7401b4057b3a93aa5a13066972bc1975142e > > lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java > 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae > lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml > 99f6f205dfe6f1c16277ac6624106436df2b2b8f > lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 > > lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java > ad27d78e1839fa60df8f2429901b77221be71653 > > lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java > dea0dfb50657b60fc71a9cb263266dec1dfbdf68 > > lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java > b56c67ce6909a11b1cdec9f52427683f0b11dd41 > > lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java > c3fe8eb142569c0b254f551c3cbd18502b6eac99 > > lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java > 71128b7d35e91fadf38a9c3926ce46d42369ead4 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java > 0cccaef5abf391179b04bca0c0bbe799429f9065 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java > 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb > > lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java > 6e7ab5dc1c78407a24e389c001ca3100563e3795 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java > d72727e7832d02ed2ea90379cd29c7a8e509a8f8 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java > 5d7163f112d7c7115ebf8f012ba397bfee63adbc > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java > 7fa5cdb55959a44b52a29884fd629c2479fdfb96 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java > be7232134b59f7126edba569358389a9c26a5c74 > lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java > f9262994debe0b943123d13c58a22c10b2015e80 > > lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java > b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 > > lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java > PRE-CREATION > > lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java > PRE-CREATION > > lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java > 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 > lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a > > lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java > 39f9f5216acf6e1be7cca6a0851152a78ba66176 > lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java > 190435036c5cf8f009d7fbd536ef1743b70c22d9 > > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java > a778a1ff0b166a6e465bdc711133061c45968b33 > > lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java > PRE-CREATION > lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java > 715347fab82b5d9d4ad673c45e72900b9a95fc81 > lens-server/src/main/resources/lensserver-default.xml > da00c0d2d94ef982646ad10f142165abe9125d3a > > lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java > PRE-CREATION > lens-server/src/test/resources/lens-site.xml > 3bcbf0e3e44d986a24c00cad84121be72596c64d > lens-server/src/test/resources/log4j.properties > e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b > src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d > > Diff: https://reviews.apache.org/r/36393/diff/ > > > Testing > ------- > > All Test cases passed: > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s] > [INFO] Lens .............................................. SUCCESS [4.091s] > [INFO] Lens API .......................................... SUCCESS [29.290s] > [INFO] Lens API for server and extensions ................ SUCCESS [30.638s] > [INFO] Lens Cube ......................................... SUCCESS [5:35.451s] > [INFO] Lens DB storage ................................... SUCCESS [26.042s] > [INFO] Lens Query Library ................................ SUCCESS [21.532s] > [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s] > [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s] > [INFO] Lens Server ....................................... SUCCESS > [23:41.881s] > [INFO] Lens client ....................................... SUCCESS [1:04.296s] > [INFO] Lens CLI .......................................... SUCCESS [4:34.347s] > [INFO] Lens Examples ..................................... SUCCESS [15.041s] > [INFO] Lens Distribution ................................. SUCCESS [21.181s] > [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s] > [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s] > [INFO] Lens Regression ................................... SUCCESS [17.566s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 45:57.904s > [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015 > [INFO] Final Memory: 120M/324M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Himanshu Gahlaut > >
