Hey Mark, Thank you for the input. I believe that such a feature will be extremely helpful as well, on top of the queue time.
1. I was thinking - if we want to accurately capture the time taken by the Poller (only the poller, which excludes the time taken after accepting connections.) in reading data, processing it and throwing it into the task queue, would the timer begin when the the first read event for that socket wrapper takes place, in the selector? Since that would be the first instance that the poller actually starts processing that request. - On this note, do you know if there's any way we can differentiate requests at this stage, and "tag" them before they are inserted into the task queue? We will require this tag to accurately measure the time the request is first processed by the Poller, till it's inserted into the queue. - Would capturing the time taken by the Poller only be sufficient as well? Since an end-to-end metric might be better, it might be ideal to start timing from the time the Acceptor accepts the connection. Since there is no way to track that, I'm guessing the first bytes being written into the socket buffer, and the poller thread first processing the socket buffer to read from it might be sufficient? 2. For the queue time code I mentioned above, do you foresee any performance issues / implementation gaps? Thank you! Andy On Mon, Dec 2, 2024 at 7:37 PM Mark Thomas <ma...@apache.org> wrote: > Andy, > > I think you are missing a chunk of time with this approach. I'd look at > recording time on the SocketWrapper when the connection is accepted. Or > at least something along those lines. I haven;t looked too carefully at > the code to see what might be possible. That way you'll also capture the > time taken by the Poller. > > That does raise the interesting question of do you start the timer when > the connection is received or when the first byte is received? Of > course, that is only relevant for the first request on a connection with > keep-alive enabled. > > Also worth keeping in mind that we don't have any way of recording the > time spend waiting for the Acceptor to accept the connection. > > Mark > > On 02/12/2024 10:24, Owner wrote: > > Hi Mark, > > > > Apologies for sending you an email again, could I ask for a quick review > on > > how I made this feature work to see if I have any misconceptions, which > > could affect how I would code it in Tomcat's source code? On top of the > > queue time, I added the request processing time as well as I wanted to > > include percentiles as well (P99, P90 etc). > > > > 1. I utilised my own CustomThreadPoolExecutor, that overrides > > ThreadPoolExecutor's offer() method. It inserts a queue time into a > > ConcurrentHashMap, with the key being the Runnable. It also overwrites > the > > take() and poll() method, retrieving the queue time from the hashmap and > > computing the queue time. > > 2. Since the worker thread handling the request is the one that calls > > take() and poll(), I create a ThreadLocal HashMap and inserted the queue > > time. Utilising the beforeExecute() and afterExecute() hook, I compute > the > > request processing time as well. > > 3. In the afterExecute() hook, I register the queue time and request > > processing time. I also check to ensure that the task is a valid request > (I > > understand some tasks queued to the executor are SocketEvents of ERROR, > > which closes the SocketWrapper), by utilising a Filter to intercept if > the > > request is passed down the filter chain. I will only register metrics for > > those that have been passed down this filter chain. > > > > Thank you! > > > > Regards > > Andy > > > > On Fri, Nov 29, 2024 at 9:58 AM Owner <o8076...@gmail.com> wrote: > > > >> Hey Mark, > >> > >> Thank you for your response. I've actually developed it on my own Spring > >> Boot Application to scrape these metrics, but it involves subclassing > >> TaskQueue (in order to overwrite the offer() method and track the > start/end > >> time), and utilising this subclassed queue when constructing my > >> ExecutorService - am not sure if this might be too specialized to be > pushed > >> out to the community. > >> > >> Was wondering if you have any thoughts on how this feature could be > >> implemented? We could go through the pseudocode based on what I've > >> developed too if it helps. > >> > >> Cheers, > >> Andy > >> > >> > >> On Thu, Nov 28, 2024 at 7:57 PM Mark Thomas <ma...@apache.org> wrote: > >> > >>> On 28/11/2024 09:26, Owner wrote: > >>>> Hi Tomcat Devs, > >>>> > >>>> Hope you guys are having a great day. I'm a new developer to this area > >>> so > >>>> do let me know if I have any misconceptions or errors in my thinking. > >>>> > >>>> I am currently using Tomcat as an embedded web server within a Spring > >>> Boot > >>>> Application, and I utilise Micrometer to expose Tomcat metrics, such > as > >>>> tomcat_servlet_request_seconds. > >>>> > >>>> I was thinking if it'd be useful to include a metric that tracks the > >>>> latency experienced in terms of time spent in the Tomcat queue. I > >>>> understand that each request is added to a TaskQueue (default > >>>> implementation) by the poller thread, and the internal executor > service > >>>> created will assign a thread to each task. However, the current > metrics > >>>> only track the time after polling from the queue, and passed down > >>> through > >>>> filters etc to the dispatcher servlet. > (tomcat_servlet_request_seconds). > >>>> > >>>> In reality, I understand the total delay experienced by the client > >>> should > >>>> include the time spent in the queue as well, since there could be a > case > >>>> where there are no available threads to service incoming requests. > >>>> > >>>> Would this metric serve a good use case? Or are there already similar > >>>> metrics out there? I'm currently on Tomcat 9.0.63 if that helps. > >>> > >>> Sounds useful to me. I'd certainly look at any proposed patch. Whether > >>> it gets merged will depend on various factors (complexity, performance, > >>> etc). > >>> > >>> Mark > >>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > >>> For additional commands, e-mail: dev-h...@tomcat.apache.org > >>> > >>> > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
