Measuring bytes sent and received from and to Tomcat
We have an application that collects data from, and sends data to, remote embedded devices. Traditionally we have used TCP and UDP to send and receive data over satellite. The latest release of our product will be using other communication medium with our devices making HTTP request to our application that is running under JBoss/Tomcat. The way we bill our clients is by charging them a usage fee based on the number of bytes being sent over the air/wire. Because of this, we need to have a accurate count of the number of bytes sent and received from each site, which is uniquely identified by it's IP address. Using either UDP and TCP this is simple as we are in control of the end socket. Is there a way we can do the same thing with Tomcat? It's simple for us to measure the number of byte in the payload of the HTTP request/response, however that isn't enough. We need to know the total number of bytes being sent and received for each HTTP request. Can someone suggest a way I could get an accurate count of these bytes? Thanks, Dave. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Measuring bytes sent and received from and to Tomcat
We looked at using a valve but we weren't sure if it would work. Correct me if I'm wrong, but it appears as though valves are chained together in a calling sequence and that some valves could change the content of the request or response. This means we may not get an accurate measure of the number of total number bytes that make up the request. Also, the AccessLogValve has a pattern code to get the number of bytes sent, excluding the HTTP headers, but does not have a pattern code to get the number of bytes sent, including the HTTP headers, which is what we really need. Have I missed something? Dave. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Yoav Shapira Sent: October 22, 2007 02:36 PM To: Tomcat Developers List Subject: Re: Measuring bytes sent and received from and to Tomcat Hey, On 10/22/07, Dave Rathnow <[EMAIL PROTECTED]> wrote: > Is there a way we can do the same thing with Tomcat? It's simple for > us to measure the number of byte in the payload of the HTTP > request/response, however that isn't enough. We need to know the > total number of bytes being sent and received for each HTTP request. > > Can someone suggest a way I could get an accurate count of these bytes? You can probably start with the AccessLogValve that ships with Tomcat: http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html Out of the box it will get you the complete bytes in the response. See the above docs on how to configure that. If you want to log the complete bytes on the request, I think you'll have to extend the Valve, but it should be pretty easy to do. Yoav - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Measuring bytes sent and received from and to Tomcat
I looked at connectors but wasn't sure if this was what I wanted. To avoid anther wild goose chase I decided to ask. Can you point me in the direction of some documentation where I might be able to get started? Dave. -Original Message- From: Costin Manolache [mailto:[EMAIL PROTECTED] Sent: October 22, 2007 04:28 PM To: Tomcat Developers List Subject: Re: Measuring bytes sent and received from and to Tomcat 'bytes' should be counted at a lower level, in connector. I'm not sure this is something generic enough - but you can make some changes to your tomcat, where read() is done from socket. I guess it would be nice to have a JMX graph with bytes/sec in/out. Costin 'bytes' On 10/22/07, Dave Rathnow <[EMAIL PROTECTED]> wrote: > > > We looked at using a valve but we weren't sure if it would work. > Correct me if I'm wrong, but it appears as though valves are chained > together in a calling sequence and that some valves could change the > content of the request or response. This means we may not get an > accurate measure of the number of total number bytes that make up the > request. > > Also, the AccessLogValve has a pattern code to get the number of bytes > sent, excluding the HTTP headers, but does not have a pattern code to > get the number of bytes sent, including the HTTP headers, which is > what we really need. > > Have I missed something? > > Dave. > > > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf > Of Yoav Shapira > Sent: October 22, 2007 02:36 PM > To: Tomcat Developers List > Subject: Re: Measuring bytes sent and received from and to Tomcat > > Hey, > > On 10/22/07, Dave Rathnow <[EMAIL PROTECTED]> wrote: > > Is there a way we can do the same thing with Tomcat? It's simple > > for us to measure the number of byte in the payload of the HTTP > > request/response, however that isn't enough. We need to know the > > total number of bytes being sent and received for each HTTP request. > > > > Can someone suggest a way I could get an accurate count of these > bytes? > > You can probably start with the AccessLogValve that ships with Tomcat: > http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html > > Out of the box it will get you the complete bytes in the response. > See the above docs on how to configure that. If you want to log the > complete bytes on the request, I think you'll have to extend the > Valve, but it should be pretty easy to do. > > Yoav > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] For > additional commands, e-mail: [EMAIL PROTECTED] > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] For > additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Measuring bytes sent and received from and to Tomcat
For our application, it's not that complicated. All we need to know is the number of bytes sent and received over the wire. We aren't using SSL, redirects, etc. -Original Message- From: Johnny Kewl [mailto:[EMAIL PROTECTED] Sent: October 22, 2007 11:26 PM To: Tomcat Developers List Subject: Re: Measuring bytes sent and received from and to Tomcat --- HARBOR: http://coolharbor.100free.com/index.htm Now Tomcat is also a cool application server --- - Original Message ----- From: "Dave Rathnow" <[EMAIL PROTECTED]> To: "Tomcat Developers List" Sent: Monday, October 22, 2007 10:00 PM Subject: Measuring bytes sent and received from and to Tomcat = Hi there, interesting question, more I think about it, more complicated it gets ;) Dont think its easy from TC, its too sophisticated, compression, SSL, redirects, dispatches, clustering... think its hard to get a true network measurement. I would plunder something like TCPMon https://tcpmon.dev.java.net/source/browse/tcpmon/ Its a NB plugin so can play with it first Its really just a (bind - client) ie port 8080 to 8081 type idea - so its easy to install, and easy to setup across multiple sites, clusters etc etc. Steal this (relay or tunnel) code and just mod it... I think you will be able to modify it for client IP's cookies, special headers... anything and then call it from a browser and get client billing breakdowns maybe... == We have an application that collects data from, and sends data to, remote embedded devices. Traditionally we have used TCP and UDP to send and receive data over satellite. The latest release of our product will be using other communication medium with our devices making HTTP request to our application that is running under JBoss/Tomcat. The way we bill our clients is by charging them a usage fee based on the number of bytes being sent over the air/wire. Because of this, we need to have a accurate count of the number of bytes sent and received from each site, which is uniquely identified by it's IP address. Using either UDP and TCP this is simple as we are in control of the end socket. Is there a way we can do the same thing with Tomcat? It's simple for us to measure the number of byte in the payload of the HTTP request/response, however that isn't enough. We need to know the total number of bytes being sent and received for each HTTP request. Can someone suggest a way I could get an accurate count of these bytes? Thanks, Dave. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Measuring bytes sent and received from and to Tomcat
Hello Again, I was wondering if someone could give me some help with this. I think using a connector is probably the way to go to solve this problem; however, I'm not sure where to start. How do I create my own connector and the plumb it into Tomcat so it will be used. I will be using a separate port other than 8080 for the devices that will be sending and receiveing data so, if possible, I would like to leave the default connector on port 8080. Again, I don't want to reimplement the code that parses the HTTP. All I need to do is count the number of bytes arriving and being sent so if I can reuse code from an existing connector, that would be great. Thanks, Dave. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Costin Manolache Sent: October 22, 2007 04:50 PM To: Dave Rathnow Cc: Tomcat Developers List Subject: Re: Measuring bytes sent and received from and to Tomcat Well, if you want absolute byte - connector seems the only place, there are space and tabs beeing skipped when parsing headers, etc. If you are ok with an estimate - the AccessLogValve is ok, add all the header lengths + method + http/1.1. You'll miss bytes for encodings, spaces. Re. where to add - each connector is different on how it reads/parse the message, you probably want to do it close to the 'read()' call, save it somewhere associated with the request ( a note or attribute ) and read it in a valve or filter. Costin On 10/22/07, Dave Rathnow <[EMAIL PROTECTED]> wrote: > > > I looked at connectors but wasn't sure if this was what I wanted. To > avoid anther wild goose chase I decided to ask. Can you point me in > the direction of some documentation where I might be able to get started? > > Dave. > > -Original Message- > From: Costin Manolache [mailto:[EMAIL PROTECTED] > Sent: October 22, 2007 04:28 PM > To: Tomcat Developers List > Subject: Re: Measuring bytes sent and received from and to Tomcat > > 'bytes' should be counted at a lower level, in connector. I'm not sure > this is something generic enough - but you can make some changes to > your tomcat, where read() is done from socket. > > I guess it would be nice to have a JMX graph with bytes/sec in/out. > > Costin > 'bytes' > > On 10/22/07, Dave Rathnow <[EMAIL PROTECTED]> wrote: > > > > > > We looked at using a valve but we weren't sure if it would work. > > Correct me if I'm wrong, but it appears as though valves are chained > > together in a calling sequence and that some valves could change the > > content of the request or response. This means we may not get an > > accurate measure of the number of total number bytes that make up > > the request. > > > > Also, the AccessLogValve has a pattern code to get the number of > > bytes > > > sent, excluding the HTTP headers, but does not have a pattern code > > to get the number of bytes sent, including the HTTP headers, which > > is what we really need. > > > > Have I missed something? > > > > Dave. > > > > > > -Original Message- > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf > > Of Yoav Shapira > > Sent: October 22, 2007 02:36 PM > > To: Tomcat Developers List > > Subject: Re: Measuring bytes sent and received from and to Tomcat > > > > Hey, > > > > On 10/22/07, Dave Rathnow <[EMAIL PROTECTED]> wrote: > > > Is there a way we can do the same thing with Tomcat? It's simple > > > for us to measure the number of byte in the payload of the HTTP > > > request/response, however that isn't enough. We need to know the > > > total number of bytes being sent and received for each HTTP request. > > > > > > Can someone suggest a way I could get an accurate count of these > > bytes? > > > > You can probably start with the AccessLogValve that ships with Tomcat: > > http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html > > > > Out of the box it will get you the complete bytes in the response. > > See the above docs on how to configure that. If you want to log the > > complete bytes on the request, I think you'll have to extend the > > Valve, but it should be pretty easy to do. > > > > Yoav > > > > > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For > > additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For > > additional commands, e-mail: [EMAIL PROTECTED] > > > > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Measuring bytes sent and received from and to Tomcat
I took a look at lamdaprobe but it only counts the payload and not the HTTP request. Back to my original question.can anyone help? -Original Message- From: Henri Gomez [mailto:[EMAIL PROTECTED] Sent: October 26, 2007 05:31 AM To: Tomcat Developers List Subject: Re: Measuring bytes sent and received from and to Tomcat Well it should works since the Lamba Probe, got these numbers for HTTP and AJP. ie : http://www.lambdaprobe.org/d/screenshots/full/charts.png Regards 2007/10/25, Dave Rathnow <[EMAIL PROTECTED]>: > > Hello Again, > > I was wondering if someone could give me some help with this. I think > using a connector is probably the way to go to solve this problem; > however, I'm not sure where to start. How do I create my own > connector and the plumb it into Tomcat so it will be used. I will be > using a separate port other than 8080 for the devices that will be > sending and receiveing data so, if possible, I would like to leave the > default connector on port 8080. > > Again, I don't want to reimplement the code that parses the HTTP. All > I need to do is count the number of bytes arriving and being sent so > if I can reuse code from an existing connector, that would be great. > > Thanks, > Dave. > > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Costin > Manolache > Sent: October 22, 2007 04:50 PM > To: Dave Rathnow > Cc: Tomcat Developers List > Subject: Re: Measuring bytes sent and received from and to Tomcat > > Well, if you want absolute byte - connector seems the only place, > there are space and tabs beeing skipped when parsing headers, etc. > > If you are ok with an estimate - the AccessLogValve is ok, add all the > header lengths + method + http/1.1. You'll miss bytes for encodings, > spaces. > > Re. where to add - each connector is different on how it reads/parse > the message, you probably want to do it close to the 'read()' call, > save it somewhere associated with the request ( a note or attribute ) > and read it in a valve or filter. > > Costin > > > On 10/22/07, Dave Rathnow <[EMAIL PROTECTED]> wrote: > > > > > > I looked at connectors but wasn't sure if this was what I wanted. > > To avoid anther wild goose chase I decided to ask. Can you point me > > in the direction of some documentation where I might be able to get > started? > > > > Dave. > > > > -Original Message- > > From: Costin Manolache [mailto:[EMAIL PROTECTED] > > Sent: October 22, 2007 04:28 PM > > To: Tomcat Developers List > > Subject: Re: Measuring bytes sent and received from and to Tomcat > > > > 'bytes' should be counted at a lower level, in connector. I'm not > > sure > > > this is something generic enough - but you can make some changes to > > your tomcat, where read() is done from socket. > > > > I guess it would be nice to have a JMX graph with bytes/sec in/out. > > > > Costin > > 'bytes' > > > > On 10/22/07, Dave Rathnow <[EMAIL PROTECTED]> wrote: > > > > > > > > > We looked at using a valve but we weren't sure if it would work. > > > Correct me if I'm wrong, but it appears as though valves are > > > chained > > > > together in a calling sequence and that some valves could change > > > the > > > > content of the request or response. This means we may not get an > > > accurate measure of the number of total number bytes that make up > > > the request. > > > > > > Also, the AccessLogValve has a pattern code to get the number of > > > bytes > > > > > sent, excluding the HTTP headers, but does not have a pattern code > > > to get the number of bytes sent, including the HTTP headers, which > > > is what we really need. > > > > > > Have I missed something? > > > > > > Dave. > > > > > > > > > -Original Message- > > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On > > > Behalf > > > > Of Yoav Shapira > > > Sent: October 22, 2007 02:36 PM > > > To: Tomcat Developers List > > > Subject: Re: Measuring bytes sent and received from and to Tomcat > > > > > > Hey, > > > > > > On 10/22/07, Dave Rathnow <[EMAIL PROTECTED]> wrote: > > > > Is there a way we can do the same thing with Tomcat? It's > > > > simple for us to measure the number of byte in the payload of >
