AccessLogValve using STDOUT

[Rainer Jung]

Hi all,

I don't like it but in managed container environments application 
instances tend to get configured to write any log output to STDOUT (than 
everything is caught and redirected to a log concentrator).


I could be wrong, but I think there is no appropriate way of doing it 
with our standard AccessLogValve. I first thought to add a flag but then 
noticed that the biggest part of the code of AccessLogValve is about 
file management. Furthermore adding the STDOUT feature to it means we 
would either produce lots of warnings for attributes that get ignored 
once the feature is used, or risking that people might not understand 
what they actually configure when enabling STDOUT but still setting file 
and directory attributes.


I did a little experiment by stripping the existing AccessLogValve down 
to just use STDOUT but still allow buffer and encoding configuration. I 
ended up with 220 lines, less than 100 lines with actual code.


I would like to add it, but don't know whether there is enough demand 
for that use case and whether people agree on using a separate class as 
the right solution. Of course it also extends our base 
AbstractAccessLogValve.


Opinion?

Thanks and regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: AccessLogValve using STDOUT

[Romain Manni-Bucau]

Hi Rainer,

There is an abstract access valve do providing a log impl (like [1]) can
enable that - plus other standard stuff like pushing on kafka accesses -
without hardcoding an stdout stream which can not work in docker in some
setup (where tomcat is launched by another process and redirects only
configured logs).

[1]
https://github.com/apache/meecrowave/blob/trunk/meecrowave-core/src/main/java/org/apache/meecrowave/tomcat/LoggingAccessLogPattern.java

Le dim. 11 nov. 2018 11:06, Rainer Jung  a écrit :

> Hi all,
>
> I don't like it but in managed container environments application
> instances tend to get configured to write any log output to STDOUT (than
> everything is caught and redirected to a log concentrator).
>
> I could be wrong, but I think there is no appropriate way of doing it
> with our standard AccessLogValve. I first thought to add a flag but then
> noticed that the biggest part of the code of AccessLogValve is about
> file management. Furthermore adding the STDOUT feature to it means we
> would either produce lots of warnings for attributes that get ignored
> once the feature is used, or risking that people might not understand
> what they actually configure when enabling STDOUT but still setting file
> and directory attributes.
>
> I did a little experiment by stripping the existing AccessLogValve down
> to just use STDOUT but still allow buffer and encoding configuration. I
> ended up with 220 lines, less than 100 lines with actual code.
>
> I would like to add it, but don't know whether there is enough demand
> for that use case and whether people agree on using a separate class as
> the right solution. Of course it also extends our base
> AbstractAccessLogValve.
>
> Opinion?
>
> Thanks and regards,
>
> Rainer
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>

Re: AccessLogValve using STDOUT

[Rainer Jung]

Hi Romain,

Am 11.11.2018 um 11:16 schrieb Romain Manni-Bucau:

Hi Rainer,

There is an abstract access valve do providing a log impl (like [1]) can
enable that - plus other standard stuff like pushing on kafka accesses -
without hardcoding an stdout stream which can not work in docker in some
setup (where tomcat is launched by another process and redirects only
configured logs).


I know, that there's the AbstractAccessLogValve, that's why I wrote "Of 
course it also extends our base AbstractAccessLogValve", so I am using 
it in the STDOUT variant just like the LoggingAccessLogPattern you 
mentioned does.


Yes, another way would be to have a variant that writes via JULI and let 
people configure that (combining with what log framework they like). 
That would be more flexible, but also less performant. The reason, that 
the AccessLogValve doesn't simply use a log framework for the request 
protocol is (I think) only performance. If that is correct and still 
relevant, then using a less performing approach for a seemingly simpler 
STDOUT output would be surprising.


Regards,

Rainer


[1]
https://github.com/apache/meecrowave/blob/trunk/meecrowave-core/src/main/java/org/apache/meecrowave/tomcat/LoggingAccessLogPattern.java

Le dim. 11 nov. 2018 11:06, Rainer Jung  a écrit :


Hi all,

I don't like it but in managed container environments application
instances tend to get configured to write any log output to STDOUT (than
everything is caught and redirected to a log concentrator).

I could be wrong, but I think there is no appropriate way of doing it
with our standard AccessLogValve. I first thought to add a flag but then
noticed that the biggest part of the code of AccessLogValve is about
file management. Furthermore adding the STDOUT feature to it means we
would either produce lots of warnings for attributes that get ignored
once the feature is used, or risking that people might not understand
what they actually configure when enabling STDOUT but still setting file
and directory attributes.

I did a little experiment by stripping the existing AccessLogValve down
to just use STDOUT but still allow buffer and encoding configuration. I
ended up with 220 lines, less than 100 lines with actual code.

I would like to add it, but don't know whether there is enough demand
for that use case and whether people agree on using a separate class as
the right solution. Of course it also extends our base
AbstractAccessLogValve.

Opinion?

Thanks and regards,

Rainer


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Principal user names and JAAS

[Rainer Jung]

Hi again,

in the various realm implementations we generate our own 
GenericPrincipal nearly always using the user name supplied by the 
authenticate() call. In the JAAS case there could be implementations 
which provide their own "real" user names in the userPrincipal they 
return. We do make the userPrincipal available in our GenericPrincipal 
but I wonder whether it would be nicer to set the name in the 
GenericPrincipal the the name from the userPrincipal if a userPrincipal 
was found and it's name is not null or empty.


We noticed the situation when looking at the session list in Tomcat 
manager. In our case the user name in authenticate() is empty (and 
retrieved via a different mechanism) but the JAAS realm gets a usable 
userPrincipal back which would contain the real user name. So this is 
not about how an application can access that name, it is more about what 
Tomcat provides as the user name (including the session list in manager).


Note that getRemoteUser() also returns the name from the GenericPrincipal.

Would such a change make sense in the JAAS realm, maybe only optionally? 
Can we think of other realm, that might want to change the user name? I 
have no immediate candidates except JAAS.


Regards,

Rainer



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: AccessLogValve using STDOUT

[Romain Manni-Bucau]

I think the perf point is likely wrong. Default JUL handlers are "slow"
(the hypothesis being you log a lot but this is more comparatively to
others frameworks)
but if you switch the handler impl you can even bit log4j2 disruptor setup
so JUL is not slow by itself but only its configuration can be so I think
it is ok
to use JUL. Also Tomcat allows to switch the Log impl so you can use any
impl, including a stdout/stderr implementation which can be something worth
making configurable more than limiting it to the
access log IMO (like -Dtomcat.log.usestdstreams=true in LogFactory or just
a jar to add implementing Log SPI).


Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book



Le dim. 11 nov. 2018 à 11:48, Rainer Jung  a
écrit :

> Hi Romain,
>
> Am 11.11.2018 um 11:16 schrieb Romain Manni-Bucau:
> > Hi Rainer,
> >
> > There is an abstract access valve do providing a log impl (like [1]) can
> > enable that - plus other standard stuff like pushing on kafka accesses -
> > without hardcoding an stdout stream which can not work in docker in some
> > setup (where tomcat is launched by another process and redirects only
> > configured logs).
>
> I know, that there's the AbstractAccessLogValve, that's why I wrote "Of
> course it also extends our base AbstractAccessLogValve", so I am using
> it in the STDOUT variant just like the LoggingAccessLogPattern you
> mentioned does.
>
> Yes, another way would be to have a variant that writes via JULI and let
> people configure that (combining with what log framework they like).
> That would be more flexible, but also less performant. The reason, that
> the AccessLogValve doesn't simply use a log framework for the request
> protocol is (I think) only performance. If that is correct and still
> relevant, then using a less performing approach for a seemingly simpler
> STDOUT output would be surprising.
>
> Regards,
>
> Rainer
>
> > [1]
> >
> https://github.com/apache/meecrowave/blob/trunk/meecrowave-core/src/main/java/org/apache/meecrowave/tomcat/LoggingAccessLogPattern.java
> >
> > Le dim. 11 nov. 2018 11:06, Rainer Jung  a
> écrit :
> >
> >> Hi all,
> >>
> >> I don't like it but in managed container environments application
> >> instances tend to get configured to write any log output to STDOUT (than
> >> everything is caught and redirected to a log concentrator).
> >>
> >> I could be wrong, but I think there is no appropriate way of doing it
> >> with our standard AccessLogValve. I first thought to add a flag but then
> >> noticed that the biggest part of the code of AccessLogValve is about
> >> file management. Furthermore adding the STDOUT feature to it means we
> >> would either produce lots of warnings for attributes that get ignored
> >> once the feature is used, or risking that people might not understand
> >> what they actually configure when enabling STDOUT but still setting file
> >> and directory attributes.
> >>
> >> I did a little experiment by stripping the existing AccessLogValve down
> >> to just use STDOUT but still allow buffer and encoding configuration. I
> >> ended up with 220 lines, less than 100 lines with actual code.
> >>
> >> I would like to add it, but don't know whether there is enough demand
> >> for that use case and whether people agree on using a separate class as
> >> the right solution. Of course it also extends our base
> >> AbstractAccessLogValve.
> >>
> >> Opinion?
> >>
> >> Thanks and regards,
> >>
> >> Rainer
>

Re: AccessLogValve using STDOUT

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Romain,

On 11/11/18 05:57, Romain Manni-Bucau wrote:
> I think the perf point is likely wrong. Default JUL handlers are
> "slow" (the hypothesis being you log a lot but this is more
> comparatively to others frameworks) but if you switch the handler
> impl you can even bit log4j2 disruptor setup so JUL is not slow by
> itself but only its configuration can be so I think it is ok to use
> JUL. Also Tomcat allows to switch the Log impl so you can use any 
> impl, including a stdout/stderr implementation which can be
> something worth making configurable more than limiting it to the 
> access log IMO (like -Dtomcat.log.usestdstreams=true in LogFactory
> or just a jar to add implementing Log SPI).

Writing directly to stdout instead of through String (or
StringBuilder) buffers (to a logging framework) is always going to be
faster.

Honestly, maintainability here is more important to me than
performance. I'd be surprised if the performance difference is
significant. Writing to stdout versus a logging framework? Probably
similar in the amount of code necessary.

- -chris

> Le dim. 11 nov. 2018 à 11:48, Rainer Jung 
> a écrit :
> 
>> Hi Romain,
>> 
>> Am 11.11.2018 um 11:16 schrieb Romain Manni-Bucau:
>>> Hi Rainer,
>>> 
>>> There is an abstract access valve do providing a log impl (like
>>> [1]) can enable that - plus other standard stuff like pushing
>>> on kafka accesses - without hardcoding an stdout stream which
>>> can not work in docker in some setup (where tomcat is launched
>>> by another process and redirects only configured logs).
>> 
>> I know, that there's the AbstractAccessLogValve, that's why I
>> wrote "Of course it also extends our base
>> AbstractAccessLogValve", so I am using it in the STDOUT variant
>> just like the LoggingAccessLogPattern you mentioned does.
>> 
>> Yes, another way would be to have a variant that writes via JULI
>> and let people configure that (combining with what log framework
>> they like). That would be more flexible, but also less
>> performant. The reason, that the AccessLogValve doesn't simply
>> use a log framework for the request protocol is (I think) only
>> performance. If that is correct and still relevant, then using a
>> less performing approach for a seemingly simpler STDOUT output
>> would be surprising.
>> 
>> Regards,
>> 
>> Rainer
>> 
>>> [1]
>>> 
>> https://github.com/apache/meecrowave/blob/trunk/meecrowave-core/src/m
ain/java/org/apache/meecrowave/tomcat/LoggingAccessLogPattern.java
>>>
>>>
>> 
Le dim. 11 nov. 2018 11:06, Rainer Jung  a
>> écrit :
>>> 
 Hi all,
 
 I don't like it but in managed container environments
 application instances tend to get configured to write any log
 output to STDOUT (than everything is caught and redirected to
 a log concentrator).
 
 I could be wrong, but I think there is no appropriate way of
 doing it with our standard AccessLogValve. I first thought to
 add a flag but then noticed that the biggest part of the code
 of AccessLogValve is about file management. Furthermore
 adding the STDOUT feature to it means we would either produce
 lots of warnings for attributes that get ignored once the
 feature is used, or risking that people might not understand 
 what they actually configure when enabling STDOUT but still
 setting file and directory attributes.
 
 I did a little experiment by stripping the existing
 AccessLogValve down to just use STDOUT but still allow buffer
 and encoding configuration. I ended up with 220 lines, less
 than 100 lines with actual code.
 
 I would like to add it, but don't know whether there is
 enough demand for that use case and whether people agree on
 using a separate class as the right solution. Of course it
 also extends our base AbstractAccessLogValve.
 
 Opinion?
 
 Thanks and regards,
 
 Rainer
>> 
> 
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvoP30ACgkQHPApP6U8
pFiFGBAAg17UeLU+W4lM9aqPpXDfoMD/BTmg0Pk7rBPuqMBtBrWPct3Db/kaaVmR
sJxhn5EFQnBOPSt5oNtnPxhX4XrJiFoTMtNGJLrsMxft+6DX/2GkrprL/DflWYJ1
X5Ox6mrlb3JlnYnmsqrNF5CygTT4LaU3e6/cLiIRBHiKvZXNO0OIEbojPlKAvTEL
K5GxO43UTDXOY5CUKrzvsvoDGiUP53sglJaJ1RbLxd91QnrGwEOc8bLK5pjVrVeq
u3TohC+sLCTJwuSkvvxM77wM9wFTkbeUBeB1qxVqnt/tr0+LGOqBzldjNRldAB61
qn72JrQNadK2ZcP/Meri4x0Esskr0cAKSs4EZEtN//galK5Q0KTHRNQMzg/HkS3p
mlBus4zaZwvHJZFZy4BmP6aEE0dEoWUkSHNFxS/0gjtoeMqwL7gAipNK39SLMis4
l/a/zEvd/BHsNg9XNorLR3tnrg2ppNdkw/7KerthN/R15Ia0CDFKu4vrvPfJO7lY
+NumDhuqoK04iLXVHIwmwSqTX2YoceY8FKaREf2yiIjyFmyBZ0yRN0Lo51Ttg/ng
CoDdbbSG6rtOH262a2obcd2dfHFeo/cB2bZUvIVf7KcECXKokf+FT9gx2ZZ4Zc7y
SiS3s/G89f2r/CjkyXAQPEZPM0eIzLmFVVM6eHWFakL4e2khG34=
=112N
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For 

Re: AccessLogValve using STDOUT

[Romain Manni-Bucau]

Le dim. 11 nov. 2018 à 15:41, Christopher Schultz <
ch...@christopherschultz.net> a écrit :

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Romain,
>
> On 11/11/18 05:57, Romain Manni-Bucau wrote:
> > I think the perf point is likely wrong. Default JUL handlers are
> > "slow" (the hypothesis being you log a lot but this is more
> > comparatively to others frameworks) but if you switch the handler
> > impl you can even bit log4j2 disruptor setup so JUL is not slow by
> > itself but only its configuration can be so I think it is ok to use
> > JUL. Also Tomcat allows to switch the Log impl so you can use any
> > impl, including a stdout/stderr implementation which can be
> > something worth making configurable more than limiting it to the
> > access log IMO (like -Dtomcat.log.usestdstreams=true in LogFactory
> > or just a jar to add implementing Log SPI).
>
> Writing directly to stdout instead of through String (or
> StringBuilder) buffers (to a logging framework) is always going to be
> faster.
>

Point is not if it is faster or slower by itself - we don't need to dig
until the point javac replaces stringbuilders by direct constant in the
constant pool or the logrecord creation which can be hardcoded in the impl
to bypass the stacktrace capture.
My point is more the perf comparatively to the cost of the other parts.
Compared to a request it is negligible - assuming the handler(s) is(are)
well written - and not impacting the overall perf so it is ok as a solution.


>
> Honestly, maintainability here is more important to me than
> performance. I'd be surprised if the performance difference is
> significant. Writing to stdout versus a logging framework? Probably
> similar in the amount of code necessary.
>

Agree it is exactly the same (overriding 1 method with 1 call - or 2 if you
want to handle error cases on stderr/log.severe).
However, stdout creates a particular case of a particular container
deployment instead of reusing logging framework which are here to be
portable, even in docker they fulfill this part (interacting directly with
a driver or avoiding stdout which can be slower than alternatives in some
environments).

That said the cost of having both is still very light (once again it is max
1 class with 2 line of owned logic) so maybe Tomcat can get both?


>
> - -chris
>
> > Le dim. 11 nov. 2018 à 11:48, Rainer Jung 
> > a écrit :
> >
> >> Hi Romain,
> >>
> >> Am 11.11.2018 um 11:16 schrieb Romain Manni-Bucau:
> >>> Hi Rainer,
> >>>
> >>> There is an abstract access valve do providing a log impl (like
> >>> [1]) can enable that - plus other standard stuff like pushing
> >>> on kafka accesses - without hardcoding an stdout stream which
> >>> can not work in docker in some setup (where tomcat is launched
> >>> by another process and redirects only configured logs).
> >>
> >> I know, that there's the AbstractAccessLogValve, that's why I
> >> wrote "Of course it also extends our base
> >> AbstractAccessLogValve", so I am using it in the STDOUT variant
> >> just like the LoggingAccessLogPattern you mentioned does.
> >>
> >> Yes, another way would be to have a variant that writes via JULI
> >> and let people configure that (combining with what log framework
> >> they like). That would be more flexible, but also less
> >> performant. The reason, that the AccessLogValve doesn't simply
> >> use a log framework for the request protocol is (I think) only
> >> performance. If that is correct and still relevant, then using a
> >> less performing approach for a seemingly simpler STDOUT output
> >> would be surprising.
> >>
> >> Regards,
> >>
> >> Rainer
> >>
> >>> [1]
> >>>
> >> https://github.com/apache/meecrowave/blob/trunk/meecrowave-core/src/m
> ain/java/org/apache/meecrowave/tomcat/LoggingAccessLogPattern.java
> 
> >>>
> >>>
> >>
> Le dim. 11 nov. 2018 11:06, Rainer Jung  a
> >> écrit :
> >>>
>  Hi all,
> 
>  I don't like it but in managed container environments
>  application instances tend to get configured to write any log
>  output to STDOUT (than everything is caught and redirected to
>  a log concentrator).
> 
>  I could be wrong, but I think there is no appropriate way of
>  doing it with our standard AccessLogValve. I first thought to
>  add a flag but then noticed that the biggest part of the code
>  of AccessLogValve is about file management. Furthermore
>  adding the STDOUT feature to it means we would either produce
>  lots of warnings for attributes that get ignored once the
>  feature is used, or risking that people might not understand
>  what they actually configure when enabling STDOUT but still
>  setting file and directory attributes.
> 
>  I did a little experiment by stripping the existing
>  AccessLogValve down to just use STDOUT but still allow buffer
>  and enco