Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.
Hello Christopher., Thanks for the Response. Firstly thanks for adding a point me in asking me to check, if the annotations are reflecting in the Java process, which opened me a door to add the concerned annotations in correct place., by adding in java_tool_options in stead of Java_opts. yeah they are reflecting and creating a Java Process. but I am facing a problem here., while i am checking JSP, Thats : the port i am using here to enable JMX is been opening a process with the mentioned port and at the same time shows port is being used.: root@lab1workflow4scalsvc2zus1-deployment-577d856494-ftb22:/# jps Picked up JAVA_TOOL_OPTIONS: -Xms2048M -Xmx10240M -XX:+UseStringDeduplication -XX:+UseContainerSupport -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port=49151 -Djava.rmi.server.hostname=tomcat.default.svc.cluster.local -javaagent:/datadog-lib/dd-java-agent.jar -XX:OnError=/datadog-lib/continuousprofiler/tmp/dd_crash_uploader.sh -XX:ErrorFile=/datadog-lib/continuousprofiler/tmp/hs_err_pid_%p.log Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 49151; nested exception is: java.net.BindException: Address already in use (Bind failed) jdk.internal.agent.AgentConfigurationError: java.rmi.server.ExportException: Port already in use: 49151; nested exception is: java.net.BindException: Address already in use (Bind failed) at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:491) at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:447) at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:599) Caused by: java.rmi.server.ExportException: Port already in use: 49151; nested exception is: java.net.BindException: Address already in use (Bind failed) at java.rmi/sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:335) at java.rmi/sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:243) at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:412) at java.rmi/sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) at java.rmi/sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:234) at java.rmi/sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:220) at java.rmi/sun.rmi.registry.RegistryImpl.(RegistryImpl.java:180) at jdk.management.agent/sun.management.jmxremote.SingleEntryRegistry.(SingleEntryRegistry.java:49) at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.exportMBeanServer(ConnectorBootstrap.java:836) at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:479) ... 2 more Caused by: java.net.BindException: Address already in use (Bind failed) at java.base/java.net.PlainSocketImpl.socketBind(Native Method) at java.base/java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:452) at java.base/java.net.ServerSocket.bind(ServerSocket.java:395) at java.base/java.net.ServerSocket.(ServerSocket.java:257) at java.base/java.net.ServerSocket.(ServerSocket.java:149) at java.rmi/sun.rmi.transport.tcp.TCPDirectSocketFactory.createServerSocket(TCPDirectSocketFactory.java:45) at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:670) at java.rmi/sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:324) ... 11 more Java process is been referred as Default JAva process: root@lab1workflow4scalsvc2zus1-deployment-577d856494-ftb22:/# ps -aux | grep java root 1 5.3 1.9 20332668 2525004 ?Ssl 07:42 3:41 java -jar /app/workflow-service.jar root 928 0.0 0.0 6328 684 pts/0S+ 08:50 0:00 grep --color=auto java and port usage is as : root@lab1workflow4scalsvc2zus1-deployment-577d856494-ftb22:/# netstat -tulnp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp6 0 0 :::10109:::*LISTEN 1/java tcp6 0 0 :::9109 :::*LISTEN 1/java tcp6 0 0 :::49151:::*LISTEN 1/java tcp6 0 0 :::42965:::*LISTEN 1/java I have used an extreme port range, to make sure that the concerned port should not be in listening list on the application end. Please let me know if you have any suggestions me to refer from my end! Thanks Thanks & Regard
setenv.sh tomcat8 changelog
The tomcat8 changelog shows the following remark among others:
General
• Tighten up the default file permissions for the .tar.gz distribution
so no files or directories are world readable by default. Configure Tomcat to
run with a default umask of 0027 which may be overridden by setting UMASK in
setenv.sh. (markt)
Where do I find "setenv.sh" in my tomcat9 (Ubuntu) distribution?
root@mail:/var/lib# grep "^Umask:" /proc/`ps ax| grep tomcat | grep -v grep |
awk '{print $1}'`/status
Umask: 0027
--
Christoph
smime.p7s
Description: S/MIME cryptographic signature
Re: Weird CSRF prevention behavior
Lasse,
On 12/1/23 15:45, Lasse Lindqvist wrote:
Well, one thing that could be wrong is that Log4j2 does not have FINE or
FINEST levels. It does have TRACE. If that does not fix things, you could
always change tour log.trace to log.error if you only care about debugging
the original issue.
I'm not expecting to use log4j2 at the server level. I mentioned it was
in the application in case there would be some kind of confusion.
logging.properties, as I understand it, always uses the JULI conventions
for things like log-level spelling.
Thanks,
-chris
pe 1. jouluk. 2023 klo 22.28 Christopher Schultz (
ch...@christopherschultz.net) kirjoitti:
All,
I'm experimenting with the CsrfPreventionFilter in Tomcat 8.5. I've had
issues with it in the past so I haven't actually enabled it in any of my
applications, but I'm sufficiently motivated at this point to get it done.
My "application" is actually split up into two applications, each
running in a separate JVM but in the same URL space. I have a
reverse-proxy that figures this all out and it's been working for years.
I don't see why this wouldn't work.
I have enabled CSRF prevention in Application A which is the "primary
application" and the secondary application (Application B) is capable of
mimicking/proxying the csrf token back to Application A.
Application B has a feature where we present a web form to the user.
It's fairly simple (paraphrasing):
When I submit this form, I get an HTTP 403 response. Our application
doesn't send 403 responses. When I remove the CsrfPreventionFilter from
the configuration) by commenting-out the in
WEB-INF/web.xml, I do not get the 403 response and the form submission
is successful. I'm sure that the CSRF token is *NOT* in the POST
request: the browser shows me what is sent and it's not there. I have
hacked the form and added the token, submitted it, and it /works/.
But this is an HTTP POST and should be ignored by the filter.
So I figure I'll enable logging and see what's happening. There isn't
much logging in CsrfPreventionFilter, so I add this line to the
beginning of the skipNonceCheck method:
log.trace("skipNonceCheck(" + request.getMethod() + " " +
request.getRequestURI() + ")");
I build-from-source and launch my custom-build Tomcat with my
application in it. No logging. Oh, right... logging.properties. So I add
this to my conf/logging.properties file:
org.apache.catalina.filters.CsrfPreventionFilter.level = FINEST
To be sure there's no funny business, I use "catalina.sh run" and wait
for the console log to settle down. I make a few requests. No logs. Hmm.
Oh, the ConsoleAppender is set to FINE and not FINEST.
java.util.logging.ConsoleHandler.level = FINEST
Done. CTRL-C, catalina.sh run. Make some requests.
Nothing. Okay maybe the Filter is just ignoring these for some
reason. So I add this line to the beginning of doFilter, before anything
else happens:
log.trace("doFilter(" + request + ")");
Re-build. CTRL-C. catalina.sh run. Make some requests.
Nothing.
The Filter is absolutely running. If I reload a page, the csrf tokens on
all the links are changing. What's going on?
You'd think a Tomcat committer could figure out how to make logging work.
My application is using log4j2, but that library is only used by the
application and the JAR file is in WEB-INF/lib/. I wouldn't expect that
it would interfere with server-level logging.
Any ideas? About EITHER issue? If anyone can help with logging, maybe I
can figure out what's happening in the Filter. If you have any
suggestions about the Filter, I'm al ears. HTTP POST should not be
prohibited unless I'm reading both the code and the CSRF specs incorrectly.
Thanks,
-chris
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: setenv.sh tomcat8 changelog
4 Dec 2023 15:10:13 Christoph Kukulies : The tomcat8 changelog shows the following remark among others: General • Tighten up the default file permissions for the .tar.gz distribution so no files or directories are world readable by default. Configure Tomcat to run with a default umask of 0027 which may be overridden by setting UMASK in setenv.sh. (markt) Where do I find "setenv.sh" in my tomcat9 (Ubuntu) distribution? You won't. setenv.sh is a file you need to add. Where your distro looks for it would be a question for Ubuntu support. In a standard Tomcat distribution you would create it alongside the catalina.sh file. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.
Sai Vamsi, On 12/4/23 03:53, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote: Firstly thanks for adding a point me in asking me to check, if the annotations are reflecting in the Java process, which opened me a door to add the concerned annotations in correct place., by adding in java_tool_options in stead of Java_opts. You will probably want to use CATALINA_OPTS instead of any of the other ones. JAVA_TOOL_OPTS isn't an environment variable regognized by Tomcat. You certainly don't want to use JAVA_OPTS, because Tomcat uses JAVA_OPTS any time it invokes a JVM. For example, running bind/digest.sh doesn't need to have the JMX subsystem starting-up and trying to grab a port. JAVA_TOOL_OPTS is an environment variable used by JVM-launching processes, like jps for example... yeah they are reflecting and creating a Java Process. but I am facing a problem here., while i am checking JSP, Thats : the port i am using here to enable JMX is been opening a process with the mentioned port and at the same time shows port is being used.: root@lab1workflow4scalsvc2zus1-deployment-577d856494-ftb22:/# jps Picked up JAVA_TOOL_OPTIONS: -Xms2048M -Xmx10240M -XX:+UseStringDeduplication -XX:+UseContainerSupport -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port=49151 -Djava.rmi.server.hostname=tomcat.default.svc.cluster.local -javaagent:/datadog-lib/dd-java-agent.jar -XX:OnError=/datadog-lib/continuousprofiler/tmp/dd_crash_uploader.sh -XX:ErrorFile=/datadog-lib/continuousprofiler/tmp/hs_err_pid_%p.log Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 49151; nested exception is: java.net.BindException: Address already in use (Bind failed) Yes: you have set JAVA_TOOL_OPTS and then run jps. jps is trying to bind to your port which is aready bound by your Tomcat process. The solution is to use only CATALINA_OPTS to set these options. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.
Hi, Also newer jvm do have XX:+UseContainerSupport set as default, also when using XX:+UseContainerSupport setting -Xms and -Xmx do not really make sense at all, you either want the JVM to deduce the max heap size from the memory cgroup or not Mfg Thomas Am 4. Dezember 2023 18:52:13 MEZ schrieb Christopher Schultz : >Sai Vamsi, > >On 12/4/23 03:53, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote: >> Firstly thanks for adding a point me in asking me to check, if the >> annotations are reflecting in the Java process, which opened me a door to >> add the concerned annotations in correct place., by adding in >> java_tool_options in stead of Java_opts. > >You will probably want to use CATALINA_OPTS instead of any of the other ones. >JAVA_TOOL_OPTS isn't an environment variable regognized by Tomcat. You >certainly don't want to use JAVA_OPTS, because Tomcat uses JAVA_OPTS any time >it invokes a JVM. For example, running bind/digest.sh doesn't need to have the >JMX subsystem starting-up and trying to grab a port. > >JAVA_TOOL_OPTS is an environment variable used by JVM-launching processes, >like jps for example... > >> yeah they are reflecting and creating a Java Process. >> but I am facing a problem here., while i am checking JSP, Thats : the port >> i am using here to enable JMX is been opening a process with the mentioned >> port and at the same time shows port is being used.: >> >> root@lab1workflow4scalsvc2zus1-deployment-577d856494-ftb22:/# jps >> Picked up JAVA_TOOL_OPTIONS: -Xms2048M -Xmx10240M >> -XX:+UseStringDeduplication -XX:+UseContainerSupport >> -Dcom.sun.management.jmxremote >> -Dcom.sun.management.jmxremote.authenticate=false >> -Dcom.sun.management.jmxremote.ssl=false >> -Dcom.sun.management.jmxremote.local.only=false >> -Dcom.sun.management.jmxremote.port=49151 >> -Djava.rmi.server.hostname=tomcat.default.svc.cluster.local >> -javaagent:/datadog-lib/dd-java-agent.jar >> -XX:OnError=/datadog-lib/continuousprofiler/tmp/dd_crash_uploader.sh >> -XX:ErrorFile=/datadog-lib/continuousprofiler/tmp/hs_err_pid_%p.log >> Error: Exception thrown by the agent : java.rmi.server.ExportException: Port >> already in use: 49151; nested exception is: >> java.net.BindException: Address already in use (Bind failed) > >Yes: you have set JAVA_TOOL_OPTS and then run jps. jps is trying to bind to >your port which is aready bound by your Tomcat process. > >The solution is to use only CATALINA_OPTS to set these options. > >-chris > >- >To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >For additional commands, e-mail: users-h...@tomcat.apache.org > -- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.
Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.
Hello Chris. Thanks for the response.. as you suggested., I tried adding my annotations in CATALINA_OPTS., Instead of Java_Opts and Java_tool_options., But its not even working out., Java process is not getting created., and port is not bound to any process. Let me know if I am missing anything. Thanks & Regards, -- SAI VAMSI .B Senior DevOps Engineer From: Christopher Schultz Sent: Monday, December 4, 2023 23:22 To: users@tomcat.apache.org Subject: Re: [EXT] Re: Datadog _ JMX Integration facing connection issues. Sai Vamsi, On 12/4/23 03:53, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote: > Firstly thanks for adding a point me in asking me to check, if the > annotations are reflecting in the Java process, which opened me a door to add > the concerned annotations in correct place., by adding in java_tool_options > in stead of Java_opts. You will probably want to use CATALINA_OPTS instead of any of the other ones. JAVA_TOOL_OPTS isn't an environment variable regognized by Tomcat. You certainly don't want to use JAVA_OPTS, because Tomcat uses JAVA_OPTS any time it invokes a JVM. For example, running bind/digest.sh doesn't need to have the JMX subsystem starting-up and trying to grab a port. JAVA_TOOL_OPTS is an environment variable used by JVM-launching processes, like jps for example... > yeah they are reflecting and creating a Java Process. > but I am facing a problem here., while i am checking JSP, Thats : the port i > am using here to enable JMX is been opening a process with the mentioned port > and at the same time shows port is being used.: > > root@lab1workflow4scalsvc2zus1-deployment-577d856494-ftb22:/# jps > Picked up JAVA_TOOL_OPTIONS: -Xms2048M -Xmx10240M -XX:+UseStringDeduplication > -XX:+UseContainerSupport -Dcom.sun.management.jmxremote > -Dcom.sun.management.jmxremote.authenticate=false > -Dcom.sun.management.jmxremote.ssl=false > -Dcom.sun.management.jmxremote.local.only=false > -Dcom.sun.management.jmxremote.port=49151 > -Djava.rmi.server.hostname=tomcat.default.svc.cluster.local > -javaagent:/datadog-lib/dd-java-agent.jar > -XX:OnError=/datadog-lib/continuousprofiler/tmp/dd_crash_uploader.sh > -XX:ErrorFile=/datadog-lib/continuousprofiler/tmp/hs_err_pid_%p.log > Error: Exception thrown by the agent : java.rmi.server.ExportException: Port > already in use: 49151; nested exception is: > java.net.BindException: Address already in use (Bind failed) Yes: you have set JAVA_TOOL_OPTS and then run jps. jps is trying to bind to your port which is aready bound by your Tomcat process. The solution is to use only CATALINA_OPTS to set these options. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
