Immediately persist sessions to database
I'm experimenting with writing some code to persist a session to the database, similar to org.apache.catalina.session.PersistentManager and org.apache.catalina.session.JDBCStore, but to store it immediately after the end of a user request, when the session goes out of "scope". Any pointers on where to start? I've played a bit with extending or replacing the above classes, but it looks like the backgroundProcess(), which is the starting point of the logic that persists sessions is only called every 10 seconds from the Container. Is there a way for a Manger to get control when a session goes out of scope? Mitch - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Immediately persist sessions to database
That worked great. Thanks. Mitch Claborn Rainer Jung wrote: > On 07.07.2009 18:21, Mitch Claborn wrote: > >> I'm experimenting with writing some code to persist a session to the >> database, similar to org.apache.catalina.session.PersistentManager and >> org.apache.catalina.session.JDBCStore, but to store it immediately after >> the end of a user request, when the session goes out of "scope". Any >> pointers on where to start? >> >> I've played a bit with extending or replacing the above classes, but it >> looks like the backgroundProcess(), which is the starting point of the >> logic that persists sessions is only called every 10 seconds from the >> Container. >> >> Is there a way for a Manger to get control when a session goes out of scope? >> > > You could look at the ReplicationValve which is part of the cluster and > triggers replication at the end of request handling. A valve can > interfere with request processing very early and very late in the > request lifecycle and can use internal Tomcat classes (the main > difference between a valve and a filter). > > Regards, > > Rainer > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > > >
JDBCStore save single threaded
The save(Session) method in org.apache.catalina.session.JDBCStore is single threaded, I assume because it uses a single database Connection for the instance. Is there any reason that I couldn't (or shouldn't) subclass JDBStore and use a connection pool, allowing it to be multi-threaded? Mitch - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Potential change to DeltaManager
Please forgive me if this is the incorrect place or format for discussing this. I'm new to trying to develop for Tomcat. I'm developing a patch for DeltaManager and I'd like to discuss with you developers if it could be considered for inclusion in the base code. Please see details below and comment. Problem: When the "all sessions" message is sent from one node to another, when the receiving node is first starting up, I often run into various errors with one of the sessions and it fails to deserialize. This causes all the remaining sessions in that chunk (sendAllSessionsSize) to be lost by the receiver. The problem with the sessions is totally an application problem, but until I can figure those problems out and solve them I need a way to limit the impact of these problems to just the one session that is in error. I could set sendAllSessionsSize="1" but that would take a LONG time to transmit, and we have many thousands of sessions at any given time. Change details: 1. Update org.apache.catalina.ha.session.DeltaManager.deserializeSessions(byte[]) and org.apache.catalina.ha.session.DeltaSession.doReadObject(ObjectInput) to produce a more detailed error message when a session is in error. New error message includes: the session index in the list of sessions, the session ID, the last field or attribute that was attempted to be read. 2. Introduce new XML attribute verifySerializedSessions for DeltaManager. 3. If verifySerializedSessions="true", org.apache.catalina.ha.session.DeltaManager.serializeSessions(Session[]) will first serialize each session then immediately deserialize it. If all is good, send the session as usual. If any errors are encountered, create and send a dummy session with a known session ID instead. (This keeps the session count, which has already been put in the output stream, correct for the receiving node.) 4. Update org.apache.catalina.ha.session.DeltaManager.deserializeSessions(byte[]) to discard any received session that has the known dummy session ID. -- Mitch
Re: Potential change to DeltaManager
See below for answers to your questions. Status update: I've been running my patch in production for about 16 hours with no problems. I've restarted each Tomcat (3) once and had no problems, but also detected no errors, either on send or receive. I have some code that I used in dev to force an error on a specific combination of session attribute name and value. I'm going to put that in prod so that I can test how it behaves with a large volume of sessions and at least one error. Mitch On 09/21/2018 05:00 PM, Mark Thomas wrote: On 21/09/18 18:02, Mitch Claborn wrote: Please forgive me if this is the incorrect place or format for discussing this. I'm new to trying to develop for Tomcat. This is the right place. Welcome to the Tomcat community. I'm developing a patch for DeltaManager and I'd like to discuss with you developers if it could be considered for inclusion in the base code. Please see details below and comment. Will do. Please note that session replication is not an area I am particularly familiar with so if some of my comments are a little off-base I apologise. Problem: When the "all sessions" message is sent from one node to another, when the receiving node is first starting up, I often run into various errors with one of the sessions and it fails to deserialize. This causes all the remaining sessions in that chunk (sendAllSessionsSize) to be lost by the receiver. Oops. The problem with the sessions is totally an application problem, but until I can figure those problems out and solve them I need a way to limit the impact of these problems to just the one session that is in error. I could set sendAllSessionsSize="1" but that would take a LONG time to transmit, and we have many thousands of sessions at any given time. That seems like a reasonable problem to try and solve. Change details: 1. Update org.apache.catalina.ha.session.DeltaManager.deserializeSessions(byte[]) and org.apache.catalina.ha.session.DeltaSession.doReadObject(ObjectInput) to produce a more detailed error message when a session is in error. New error message includes: the session index in the list of sessions, the session ID, the last field or attribute that was attempted to be read. I'm not sure how useful the index will be but the other information makes sense to me. The index gives me an indication of how many sessions were discarded because of the error. 2. Introduce new XML attribute verifySerializedSessions for DeltaManager. Why would a user not want to enable this feature? The performance hit of the additional deserialization on send? That is the only reason I can think of. 3. If verifySerializedSessions="true", org.apache.catalina.ha.session.DeltaManager.serializeSessions(Session[]) will first serialize each session then immediately deserialize it. If all is good, send the session as usual. If any errors are encountered, create and send a dummy session with a known session ID instead. (This keeps the session count, which has already been put in the output stream, correct for the receiving node.) Ah. Is the issue that serialization works but deserialization does not? That seems a little odd. Can you give an example of how this might go wrong? I am trying to understand the root cause(s) of the problem to determine if the proposed solution is appropriate. I thought DeltaSession simply skipped over attributes that it could not deserialize. DeltaSession does skip attributes that are not serializable. I've had three identifiable errors, none of which I could reproduce at will. 1. A session with a Vector that might have contained nulls. This should not be an issue, but I fixed my code to eliminate nulls in that Vector, since they should not be there anyway. 2. In some of my own objects where I do my own serialization with JSON, there were some fields that I don't serialize that were not marked transient that should have been. Some of those embedded objects were thus serialized by the native serialization and caused some problems. I fixed those. 3. In another of my objects that I serialize with JSON, the JSON string in the serialized session was obviously corrupted and was not a valid JSON hash. I went over the serialization code with a fine tooth come and it appears to be correct. That same code works hundreds of thousands of times a day without error. Especially in the case of #3, I suspect that there might be a concurrency issue - a session being modified in one request while it is being serialized in another. FYI, bordering on TMI: I just recently switched to DeltaManager from a custom session sharing solution where I was doing my own persistence to a database, with no in-memory storage. Concurrency was not an issue in that setup because each request received an independent copy of the session content. I cou
Re: Potential change to DeltaManager
Any further thoughts or comments on this? I think my patch is ready for prime time now. Mitch On 09/22/2018 11:23 AM, Mitch Claborn wrote: See below for answers to your questions. Status update: I've been running my patch in production for about 16 hours with no problems. I've restarted each Tomcat (3) once and had no problems, but also detected no errors, either on send or receive. I have some code that I used in dev to force an error on a specific combination of session attribute name and value. I'm going to put that in prod so that I can test how it behaves with a large volume of sessions and at least one error. Mitch On 09/21/2018 05:00 PM, Mark Thomas wrote: On 21/09/18 18:02, Mitch Claborn wrote: Please forgive me if this is the incorrect place or format for discussing this. I'm new to trying to develop for Tomcat. This is the right place. Welcome to the Tomcat community. I'm developing a patch for DeltaManager and I'd like to discuss with you developers if it could be considered for inclusion in the base code. Please see details below and comment. Will do. Please note that session replication is not an area I am particularly familiar with so if some of my comments are a little off-base I apologise. Problem: When the "all sessions" message is sent from one node to another, when the receiving node is first starting up, I often run into various errors with one of the sessions and it fails to deserialize. This causes all the remaining sessions in that chunk (sendAllSessionsSize) to be lost by the receiver. Oops. The problem with the sessions is totally an application problem, but until I can figure those problems out and solve them I need a way to limit the impact of these problems to just the one session that is in error. I could set sendAllSessionsSize="1" but that would take a LONG time to transmit, and we have many thousands of sessions at any given time. That seems like a reasonable problem to try and solve. Change details: 1. Update org.apache.catalina.ha.session.DeltaManager.deserializeSessions(byte[]) and org.apache.catalina.ha.session.DeltaSession.doReadObject(ObjectInput) to produce a more detailed error message when a session is in error. New error message includes: the session index in the list of sessions, the session ID, the last field or attribute that was attempted to be read. I'm not sure how useful the index will be but the other information makes sense to me. The index gives me an indication of how many sessions were discarded because of the error. 2. Introduce new XML attribute verifySerializedSessions for DeltaManager. Why would a user not want to enable this feature? The performance hit of the additional deserialization on send? That is the only reason I can think of. 3. If verifySerializedSessions="true", org.apache.catalina.ha.session.DeltaManager.serializeSessions(Session[]) will first serialize each session then immediately deserialize it. If all is good, send the session as usual. If any errors are encountered, create and send a dummy session with a known session ID instead. (This keeps the session count, which has already been put in the output stream, correct for the receiving node.) Ah. Is the issue that serialization works but deserialization does not? That seems a little odd. Can you give an example of how this might go wrong? I am trying to understand the root cause(s) of the problem to determine if the proposed solution is appropriate. I thought DeltaSession simply skipped over attributes that it could not deserialize. DeltaSession does skip attributes that are not serializable. I've had three identifiable errors, none of which I could reproduce at will. 1. A session with a Vector that might have contained nulls. This should not be an issue, but I fixed my code to eliminate nulls in that Vector, since they should not be there anyway. 2. In some of my own objects where I do my own serialization with JSON, there were some fields that I don't serialize that were not marked transient that should have been. Some of those embedded objects were thus serialized by the native serialization and caused some problems. I fixed those. 3. In another of my objects that I serialize with JSON, the JSON string in the serialized session was obviously corrupted and was not a valid JSON hash. I went over the serialization code with a fine tooth come and it appears to be correct. That same code works hundreds of thousands of times a day without error. Especially in the case of #3, I suspect that there might be a concurrency issue - a session being modified in one request while it is being serialized in another. FYI, bordering on TMI: I just recently switched to DeltaManager from a custom session sharing solution where I was doing my own persistence to a database, with no
Email from AsyncFileHandler?
I recently wrote an extension of org.apache.juli.AsyncFileHandler that will send an email to a specified address if the message is SEVERE or higher (in a background thread). Is there any interest in putting that ability into AsyncFileHanlder directly? -- Mitch - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Bug 62140 implementation question
Are the available verbs the same for both .sh and .bat? If so, perhaps a text file that contains the actual help text that could be output from both .bat and .sh. Mitch On 10/01/2018 02:48 PM, Marek Czernek wrote: Hi there, I'd like to resolve Bug 62140 [0]. I just wanted to gather some opinions about the implementation details. In my mind, the following solution is quite reasonable: 1. Create new help scripts, such as help.sh and help.bat. These files contain a method for each functional verb that prints some info about the verb and exits with 0. 2. Source the files in both catalina.sh and catalina.bat 3. When user enters catalina.[sh|bat] $verb help (or -h, --help?), execute one of the methods. In my mind, the above solution is quite straightforward. Any gotchas, or obvious problems? [0] https://bz.apache.org/bugzilla/show_bug.cgi?id=62140 Cheers, - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Bug 62140 implementation question
My only thought was the single-source value. If it would be a lot of extra work to do the text file, then I think the original design is fine. Mitch On 10/02/2018 09:56 AM, Marek Czernek wrote: Well, I personally would prefer if it was a soure-able (or call-able for windows) script; otherwise, you'd have to parse the text file to check which part of the text you want to output, since you don't want to output all the text at once, but only for one particular verb. Any particular benefits for a text file over two scripts? It's true it's single-sourced, but given that all scripts are essentially duplicated into .sh and .bat, I don't feel like that outweighs the ease of use and extensibility of the help script files. On 10/2/18 4:45 PM, Mitch Claborn wrote: Are the available verbs the same for both .sh and .bat? If so, perhaps a text file that contains the actual help text that could be output from both .bat and .sh. Mitch On 10/01/2018 02:48 PM, Marek Czernek wrote: Hi there, I'd like to resolve Bug 62140 [0]. I just wanted to gather some opinions about the implementation details. In my mind, the following solution is quite reasonable: 1. Create new help scripts, such as help.sh and help.bat. These files contain a method for each functional verb that prints some info about the verb and exits with 0. 2. Source the files in both catalina.sh and catalina.bat 3. When user enters catalina.[sh|bat] $verb help (or -h, --help?), execute one of the methods. In my mind, the above solution is quite straightforward. Any gotchas, or obvious problems? [0] https://bz.apache.org/bugzilla/show_bug.cgi?id=62140 Cheers, - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Potential change to DeltaManager
FYI: I've created the Bugzilla request and submitted the patch there. https://bz.apache.org/bugzilla/show_bug.cgi?id=62773 Mitch On 9/27/18 5:49 PM, Mark Thomas wrote: Mitch, First some general comments. Projects at the ASF generally operate using lazy consensus meaning if no-one objects after a reasonable amount of time (72 hours is a good starting point for reasonable) then assume you have agreement to proceed. Note that it is ApacheCon NA this week so a number of the committers may be distracted and/or travelling. It sounds like a good next step would be to create a Bugzilla enhancement request and attach your patch. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
