I have not yet tested the fix. It is on my agenda for early next week.
- Mike
On Fri, Aug 23, 2013 at 8:09 AM, Andrew Miskelly <
[email protected]> wrote:
> Slightly different outcome this time around for me. By the end of the day
> all of my AJP threads ended up in a RUNNABLE state, though not apparently
> doing anything. I'm not sure whether this is a progression of this issue or
> something entirely different but the result is the same - an unresponsive
> application.
>
> "ajp-bio-8109-exec-937" daemon prio=10 tid=0x00002aaab533d000 nid=0x14d0
> runnable [0x000000004d9b1000]
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
> at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309)
> at
> org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364)
> at
> org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> - locked <0x0000000785dfec70> (a
> org.apache.tomcat.util.net.SocketWrapper)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
>
> Has anyone else done any testing of the new code?
>
> Regards,
> Andrew
>
>
> On 23/08/2013 09:51, Andrew Miskelly wrote:
>
> Hi Christian and Eric,
>
> Thanks for the work you've put in thus far. I have deployed the updated
> classes and will let you know how things are going once GeoServer has been
> running for a while.
>
> Regards,
> Andrew
>
> On 23/08/2013 01:33, Christian Mueller wrote:
>
> Hi all
>
> I attached LRUAuthenticationCacheImpl.class and
> LRUAuthenticationCacheImpl$1.class for testing. These classes includes the
> code snippet proposed by Eric. Please replace these classes in
>
> WEB-INF/lib/main-<version>.jar
>
> Let me know about the results.
>
> If the problem persists, I will publish a version without the timer
> task. This should resolve the issue. The disadvantage would be that
> authentication tokens are hold in memory longer than needed, the
> functionality would not change.
>
> Cheers
> Christian
>
>
>
>
> On Thu, Aug 22, 2013 at 5:18 PM, Eric Smets <[email protected]> wrote:
>
>> Op 22/08/2013 16:30, Christian Mueller schreef:
>>
>> Hi Eric
>>
>> Only to be sure, the above code snippet solves the issue ?
>>
>> We have only tested this modification, this came up after an internal
>> discussion.
>> We can not confirm that this solves the original issue, but this solution
>> should do a better job.
>>
>> We could use the testscript from Andrew to confirm the solution.
>>
>>
>>
>>
>> If it does, I will publish a modified class file for the other users to
>> test.
>>
>> My current idea was to remove the time task completely, but your
>> solution would be better and safer.
>>
>> Please let me know
>>
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Aug 22, 2013 at 4:09 PM, Eric Smets <[email protected]> wrote:
>>
>>> Hello,
>>>
>>> We had/have on a production machine the same issue.
>>> Following the discuss we looked into the implementation of
>>> LRUAuthenticationCacheImpl
>>>
>>> The origin of the problem is not clear but the result is only one loop
>>> over the keys.
>>>
>>> LRUAuthenticationCacheImpl. Some addional logging is also added but
>>> could be removed
>>>
>>> The changed snipped follows
>>> ----
>>>
>>> TimerTask removeExpiredTask = new TimerTask() {
>>> @Override
>>> public void run() {
>>> LOGGER.fine("Start searching for expired authentication
>>> tokens");
>>> writeLock.lock();
>>> try {
>>> int removed = 0;
>>> long currentTime=System.currentTimeMillis();
>>>
>>> if (cache.size() > 50) LOGGER.info("AuthKey Cache
>>> entries: " + cache.size());
>>>
>>> Iterator<AuthenticationCacheEntry> iter =
>>> cache.values().iterator();
>>>
>>> while (iter.hasNext())
>>> {
>>> if (iter.next().hasExpired(currentTime))
>>> {
>>> iter.remove();
>>> removed++;
>>> }
>>> }
>>> LOGGER.fine("Number of expired authentication tokens
>>> found: " + removed);
>>> } finally {
>>> writeLock.unlock();
>>> }
>>> LOGGER.fine("End searching for expired authentication
>>> tokens");
>>> }
>>> };
>>>
>>> Op 22/08/2013 11:58, Andrea Aime schreef:
>>>
>>> On Thu, Aug 22, 2013 at 11:52 AM, Christian Mueller <
>>> [email protected]> wrote:
>>>
>>>> Hi Andrea
>>>>
>>>> The problem is that most cache implementations have global expire
>>>> time settings. We need global expire times as default but it must be
>>>> possible to assign expire times to individual entries. As an example,
>>>> digest authentication has its own expire times and it does not make sense
>>>> if the cache has expire times larger than the configured expire times of
>>>> the digest authentication filter. A cache entry created by digest
>>>> authentication has expire times limited by the digest authentication
>>>> configuration.
>>>>
>>>> The cleaner thread is here to guarantee that an unused authentication
>>>> token is not cached for a long time period, this is a security risk.
>>>>
>>>>
>>>> I am thinking about using Collections.synchronizedMap(..) and to kick
>>>> out all the locking stuff.
>>>>
>>>
>>> Ugh, it would limit scalability quite a bit. The Guava CacheBuilder
>>> should be a much more performant solution.
>>> Once built you can treat it as a regular map, it handles LRU and
>>> concurrency internally.
>>>
>>> Cheers
>>> Andrea
>>>
>>> --
>>> ==
>>> Our support, Your Success! Visit http://opensdi.geo-solutions.it for
>>> more information.
>>> ==
>>>
>>> Ing. Andrea Aime
>>> @geowolf
>>> Technical Lead
>>>
>>> GeoSolutions S.A.S.
>>> Via Poggio alle Viti 1187
>>> 55054 Massarosa (LU)
>>> Italy
>>> phone: +39 0584 962313 <%2B39%200584%20962313>
>>> fax: +39 0584 1660272 <%2B39%200584%201660272>
>>> mob: +39 339 8844549 <%2B39%20%C2%A0339%208844549>
>>>
>>> http://www.geo-solutions.it
>>> http://twitter.com/geosolutions_it
>>>
>>> -------------------------------------------------------
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Introducing Performance Central, a new site from SourceForge and
>>> AppDynamics. Performance Central is your source for news, insights,
>>> analysis and resources for efficient Application Performance Management.
>>> Visit us
>>> today!http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>>>
>>>
>>>
>>> _______________________________________________
>>> Geoserver-users mailing
>>> [email protected]https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>>
>>>
>>>
>>> --
>>> Eric Smets [email protected]
>>> FKS bvba - Formal and Knowledge Systems http://www.fks.be/
>>> Schampbergstraat 32 Tel: ++32-(0)11-21 49 11
>>> B-3511 Hasselt Fax: ++32-(0)11-22 04 19
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Introducing Performance Central, a new site from SourceForge and
>>> AppDynamics. Performance Central is your source for news, insights,
>>> analysis and resources for efficient Application Performance Management.
>>> Visit us today!
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Geoserver-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>>
>>>
>>
>>
>> --
>> DI Christian Mueller MSc (GIS), MSc (IT-Security)
>> OSS Open Source Solutions GmbH
>>
>>
>>
>> --
>> Eric Smets [email protected]
>> FKS bvba - Formal and Knowledge Systems http://www.fks.be/
>> Schampbergstraat 32 Tel: ++32-(0)11-21 49 11
>> B-3511 Hasselt Fax: ++32-(0)11-22 04 19
>>
>>
>
>
> --
> DI Christian Mueller MSc (GIS), MSc (IT-Security)
> OSS Open Source Solutions GmbH
>
>
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance Management.
> Visit us
> today!http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> Geoserver-users mailing
> [email protected]https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
>
>
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance Management.
> Visit us
> today!http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> Geoserver-users mailing
> [email protected]https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
>
>
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance Management.
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> _______________________________________________
> Geoserver-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
>
------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and
AppDynamics. Performance Central is your source for news, insights,
analysis and resources for efficient Application Performance Management.
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
