> On July 15, 2015, 12:18 p.m., Amareshwari Sriramadasu wrote: > > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java, > > line 786 > > <https://reviews.apache.org/r/36393/diff/16/?file=1012522#file1012522line786> > > > > Should process next waiting query happen after following data structure > > updates? > > > > Also, can you check if updateFinishedQuery(ctx) is done with ctx > > locked. If so, processing next waiting query should not require this lock. > > Might have to after the lock is given out. > > Himanshu Gahlaut wrote: > While processing waiting queries, ctx is given as an input to waiting > queries selector and selection policies. While Selection policies are reading > information from ctx, other threads should not do any change of state of > information. Hence ctx is locked during waiting queries selection.
ctx here is finsihed query. Why should finished query be locked for next waiting query selection? On the finished mostly no changes are done. With locking put on by the caller - the calls like cancel can take more time because same thread will be choosing the next query to launch. The user api threads should not be used for launching next query. > On July 15, 2015, 12:18 p.m., Amareshwari Sriramadasu wrote: > > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java, > > line 350 > > <https://reviews.apache.org/r/36393/diff/16/?file=1012502#file1012502line350> > > > > Why are removing synchronized here? > > > > We want to make sure setStatus is not done by multiple threads through > > synchronizing on context > > Himanshu Gahlaut wrote: > Is there any thread which is doing setStatus outside of synchronized(ctx) > block ? If Yes, then we can keep it. Keeping the method synchronized removes the double check of callers locking the object. If ctx is already locked, synchrozied here will be no-op. > On July 15, 2015, 12:18 p.m., Amareshwari Sriramadasu wrote: > > lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java, > > line 123 > > <https://reviews.apache.org/r/36393/diff/16/?file=1012498#file1012498line123> > > > > Is this still emitted after removing WAITING state? > > Himanshu Gahlaut wrote: > Yes. It wil help in looking at the graph of usage of waiting queries > store. I see it is implemented as guage on waiting queries. Seems fine. - Amareshwari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36393/#review91731 ----------------------------------------------------------- On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/36393/ > ----------------------------------------------------------- > > (Updated July 15, 2015, 8:55 a.m.) > > > Review request for lens. > > > Bugs: LENS-619 > https://issues.apache.org/jira/browse/LENS-619 > > > Repository: lens > > > Description > ------- > > LENS-619: Applying Query Launching Constraints before allowing a query to be > launched > > > Diffs > ----- > > lens-api/src/main/java/org/apache/lens/api/Addable.java > 5a556918afcdb8967deee84472cd7863aeafc4fc > lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java > f594133e8a75603e5bb0c913d62ea189248cfa05 > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java > 539f0b0b0804257fae6b7ec0af8f2f715175d597 > lens-api/src/main/resources/lens-errors.conf > 6130ad78c2871f78ff9bc11cca5702a538639bbf > lens-client/src/main/java/org/apache/lens/client/LensStatement.java > f7305fb288167cda088e263a62c77f264d779127 > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java > e87ed23429d3172db86c17e4ec5e37e2e903374b > lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java > 0f2b7401b4057b3a93aa5a13066972bc1975142e > > lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java > 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae > lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml > 99f6f205dfe6f1c16277ac6624106436df2b2b8f > lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 > > lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java > ad27d78e1839fa60df8f2429901b77221be71653 > > lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java > dea0dfb50657b60fc71a9cb263266dec1dfbdf68 > > lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java > b56c67ce6909a11b1cdec9f52427683f0b11dd41 > > lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java > c3fe8eb142569c0b254f551c3cbd18502b6eac99 > > lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java > 71128b7d35e91fadf38a9c3926ce46d42369ead4 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java > 0cccaef5abf391179b04bca0c0bbe799429f9065 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java > 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb > > lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java > 6e7ab5dc1c78407a24e389c001ca3100563e3795 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java > d72727e7832d02ed2ea90379cd29c7a8e509a8f8 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java > 5d7163f112d7c7115ebf8f012ba397bfee63adbc > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java > 7fa5cdb55959a44b52a29884fd629c2479fdfb96 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java > be7232134b59f7126edba569358389a9c26a5c74 > lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java > f9262994debe0b943123d13c58a22c10b2015e80 > > lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java > b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 > > lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java > PRE-CREATION > > lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java > PRE-CREATION > > lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java > 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 > lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a > > lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java > 39f9f5216acf6e1be7cca6a0851152a78ba66176 > lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java > 190435036c5cf8f009d7fbd536ef1743b70c22d9 > > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java > a778a1ff0b166a6e465bdc711133061c45968b33 > > lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java > PRE-CREATION > lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java > 715347fab82b5d9d4ad673c45e72900b9a95fc81 > lens-server/src/main/resources/lensserver-default.xml > da00c0d2d94ef982646ad10f142165abe9125d3a > > lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java > PRE-CREATION > lens-server/src/test/resources/lens-site.xml > 3bcbf0e3e44d986a24c00cad84121be72596c64d > lens-server/src/test/resources/log4j.properties > e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b > src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d > > Diff: https://reviews.apache.org/r/36393/diff/ > > > Testing > ------- > > All Test cases passed: > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s] > [INFO] Lens .............................................. SUCCESS [4.091s] > [INFO] Lens API .......................................... SUCCESS [29.290s] > [INFO] Lens API for server and extensions ................ SUCCESS [30.638s] > [INFO] Lens Cube ......................................... SUCCESS [5:35.451s] > [INFO] Lens DB storage ................................... SUCCESS [26.042s] > [INFO] Lens Query Library ................................ SUCCESS [21.532s] > [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s] > [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s] > [INFO] Lens Server ....................................... SUCCESS > [23:41.881s] > [INFO] Lens client ....................................... SUCCESS [1:04.296s] > [INFO] Lens CLI .......................................... SUCCESS [4:34.347s] > [INFO] Lens Examples ..................................... SUCCESS [15.041s] > [INFO] Lens Distribution ................................. SUCCESS [21.181s] > [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s] > [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s] > [INFO] Lens Regression ................................... SUCCESS [17.566s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 45:57.904s > [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015 > [INFO] Final Memory: 120M/324M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Himanshu Gahlaut > >
