JAVA -tomcat- Request header is too large

[Ivano Luberti]

Hi, since a few days these errors started showing in my log files:

06-Dec-2023 07:39:56.082 INFO [http-nio-8080-exec-5826] 
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP 
request header
 Note: further occurrences of HTTP request parsing errors will be 
logged at DEBUG level.

    java.lang.IllegalArgumentException: Request header is too large
        at 
org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:790)
        at 
org.apache.coyote.http11.Http11InputBuffer.parseHeader(Http11InputBuffer.java:975)
        at 
org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuffer.java:604)
        at 
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:294)
        at 
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at 
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
        at 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1789)
        at 
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at 
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at 
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at 
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

        at java.lang.Thread.run(Thread.java:750)


I know I can increase the tolerated header size using sometjing like this:



Bu my question is how can i debug the issue?

For example: how can i find the page requested when the exception was 
raised?





--

Archimede Informatica tratta i dati personali in conformità a quanto
stabilito dal Regolamento UE n. 2016/679 (GDPR) e dal D. Lgs. 30 giugno 
2003 n. 196

per come modificato dal D.Lgs. 10 agosto 2018 n. 101.
Informativa completa 



dott. Ivano Mario Luberti

Archimede Informatica società cooperativa a r. l.
Via Gereschi 36, 56127 Pisa

tel.: +39 050/580959 | fax: +39 050/8932061

web: www.archicoop.it
linkedin: www.linkedin.com/in/ivanoluberti
facebook: www.facebook.com/archimedeinformaticapisa/

Re: JAVA -tomcat- Request header is too large

[Mark Thomas]

On 07/12/2023 15:37, Ivano Luberti wrote:

Hi, since a few days these errors started showing in my log files:

06-Dec-2023 07:39:56.082 INFO [http-nio-8080-exec-5826] 
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP 
request header
 Note: further occurrences of HTTP request parsing errors will be 
logged at DEBUG level.

    java.lang.IllegalArgumentException: Request header is too large
        at 
org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:790)
        at 
org.apache.coyote.http11.Http11InputBuffer.parseHeader(Http11InputBuffer.java:975)
        at 
org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuffer.java:604)
        at 
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:294)
        at 
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at 
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
        at 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1789)
        at 
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at 
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at 
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at 
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

        at java.lang.Thread.run(Thread.java:750)


I know I can increase the tolerated header size using sometjing like this:



Bu my question is how can i debug the issue?

For example: how can i find the page requested when the exception was 
raised?


Match the timestamp in the logs with the timestamp in the access logs of 
the associated 400 response.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.

[Christopher Schultz]

Sai Vamsi,

On 12/6/23 11:04, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote:

Hello Christopher.
Thanks for the response!

Let me walk you through the Environment.,
I have a cluster and I am installing multiple microservices in the cluster. And 
one of the microservices I am integrating the JMX on of the microservices.,
and the mentioned IP of the pod is being changed dynamically after each 
deployment .

Here are the annotations that I am using in my deployment:
- name: JAVA_OPTS
 #   value: >-
 # -Dcom.sun.management.jmxremote
 # -Dcom.sun.management.jmxremote.authenticate=false
 # -Dcom.sun.management.jmxremote.ssl=false
 # -Dcom.sun.management.jmxremote.local.only=false
 # -Dcom.sun.management.jmxremote.port=49151
 # -Dcom.sun.management.jmxremote.rmi.port=49151
 # 
-Djava.rmi.server.hostname=lab1workflow4scalsvc2zus1-service.hqm-lab1.svc.cluster.local


If that's YAML, then all of the above is comments. Those lines do nothing.


as i said , I have implemented these in three scenarios., Catalina_opts / 
Java_opts / Java_Tool_opts


You should definitely use CATALINA_OPTS and none of the other things you 
mention, here. Specifically because of the port issues. JAVA_TOOL_OPTS 
don't affect Tomcat at all are useless for you. JAVA_OPTS affects all 
Tomcat-related scripts, and you'll find that even "shutdown" causes an 
error... because it's trying to initialize JMX when shutting-down, which 
is again useless.



I have tried deploying catalina_ots and Java_opts as in the case of above 
mentioned.
I also tried other way, mentioning all these in the values.yaml file ( I am 
using Helm Deployment to upgrade my deployments) and referring these 
annotations as , and mentioning them in the Value
env:
 - name: CATALINA_OPTS
   value: {{ .Values.deployment.javaVMMemoryArgument }}


getting netstat output from my pod:
root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address   Foreign Address State   
PID/Program name
tcp6   0  0 :::10109:::*LISTEN  
1/java
tcp6   0  0 :::9109 :::*LISTEN  
1/java

root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# ps aux | grep 
tomcat
root 563  0.0  0.0   6328   748 pts/0S+   15:55   0:00 grep 
--color=auto tomcat


Don't look for 'tomcat'. Look for 'java'. Or 'catalina'. The word 
'tomcat' does not appear in a normal JVM launch command for Tomcat.



right now , I am passing my args with Catalina_opts in my deployment
and I am not able to get any tomcat process pid while i check with
netstat -apn , I can see multiple Java Process but not single tomcat process

My Jmx port is being configured as a service., so that it could be
bounded with Tcp protocol externally to the container., and allow all
tcp connections.

Hope I am able to provide you some detailed information., Please let
me know., if I can help you with anything in further!


I think maybe you are both configuring your YAML file incorrectly and 
also looking for the process incorrectly.


-chris



From: Christopher Schultz 
Sent: Wednesday, December 6, 2023 20:40
To: users@tomcat.apache.org 
Subject: Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.

Sai Vamsi,

On 12/5/23 08:57, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote:

I have added my annotations in Catalina_opts ., and I am not able to see any 
Java Process with the JMX, neither the port is being listened.
It came back to previous stage where Annotations are not on the JVM .,
Came back to previous stage, where connection refused error.,  as follows.

Could not initialize instance: tomcat-10.75.12.136-49151:
java.util.concurrent.ExecutionException: java.io.IOException: Failed to 
retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root 
exception is java.rmi.ConnectException: Connection refused to host: 
10.75.12.136; nested exception is:
  java.net.ConnectException: Connection refused (Connection refused)]
  at 
java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
  at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
  at org.datadog.jmxfetch.App.processRecoveryResults(App.java:953)
  at org.datadog.jmxfetch.App$5.invoke(App.java:927)
  at 
org.datadog.jmxfetch.tasks.TaskProcessor.processTasks(TaskProcessor.java:63)
  at org.datadog.jmxfetch.App.init(App.java:917)
  at org.datadog.jmxfetch.App.start(App.java:413)
  at org.datadog.jmxfetch.App.run(App.java:173)
  at org.datadog.jmxfetch.JmxFetch.main(JmxFetch.java:57)
  at org.datadog.jmxfetch.App.main(App.java:91)


Right: the DataDog

Re: 9.0.83 addSslHostConfig failures?

[Christopher Schultz]

Daniel,

On 12/6/23 23:19, Daniel Skiles wrote:

Please disregard that, I got overexcited.

It looks like you can query the operation, and it will return the correct
name and the correct parameters, but when you try to run it, you get
the following stack trace:

Caused by: javax.management.ServiceNotFoundException: Cannot find operation
[addSslHostConfig]
at
org.apache.tomcat.util.modeler.ManagedBean.getInvoke(ManagedBean.java:525)
~[tomcat-coyote.jar:9.0.83]
at
org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:286)
~[tomcat-coyote.jar:9.0.83]
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:814)
~[?:?]
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:802)
~[?:?]

Any ideas here?


Can you please post:

1. The JMX complete object name
2. The operation you are trying to perform (addSslHostConfig)
3. The list of parameters you are providing along with the call. 
Obviously, you can mask anything that is a secret


-chris


On Wed, Dec 6, 2023 at 4:45 PM Daniel Skiles  wrote:


I found the source of the problem

9.0.82 accepts the simple class name as a type parameter for the operation.

9.0.83 requires the canonical class name.

On Wed, Dec 6, 2023 at 3:06 PM Daniel Skiles 
wrote:


I've had no success digging into this.

Is this a regression, or was this an intentional change?

On Mon, Nov 27, 2023 at 8:56 AM Daniel Skiles 
wrote:


Thanks for taking a look.  My lightly scrubbed connector example is
attached.

On Tue, Nov 21, 2023 at 6:45 AM Michael Osipov 
wrote:


On 2023/11/21 11:25:11 Michael Osipov wrote:

On 2023/11/20 22:14:14 Daniel Skiles wrote:

Was there a change to the addSslHostConfig JMX mbean operation

between

9.0.82 and 9.0.83?  I have some code that works in 82, but fails

with an

MBeanException: Cannot find operation [addSslHostConfig] in 9.0.83.

When I attempt to look at the available operations on

ProtocolHandler in

jconsole, it throws an exception in 83 that opens a new window, but

works

in 82.


I have the following with 8.5.x:

Error setting Operation panel :org.apache.coyote.Request
Error setting Operation panel :org.apache.tomcat.util.net

.SSLHostConfig


addSslHostConfig is greyed out for me...let me go back a patch

version...

Tried on 8.5.92, same behavior. You should share your connector config.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org






-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

9.0.83 addSslHostConfig JMX Operation Regression (Sample Code Attached)

[Daniel Skiles]

All,
I've been doing some testing, and I'm pretty sure the addSslHostConfig
operation on ProtocolHandler is busted in 9.0.83.

In versions prior to 9.0.82, you can call the operation with a single
argument of type SSLHostConfig.

In 9.0.82, that contract seems to have been broken, and you had to call it
with two arguments:  an SSLHostConfig and a boolean.

In 9.0.83, it seems as though both operations are present, but which one is
actually accessible at runtime is non-deterministic.

This behavior presents through a direct invoke(...) call and via a JMX
Proxy object instantiated through JMX.newMBeanProxy.

I have attached a sample file that reproduces the behavior (sometimes, as
it is nondeterministic).

Is this a bug, or am I simply using the available feature incorrectly?

If it is the former, how do I formally report this?  If it is the latter,
what is the "correct" way to call this operation from JMX?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Virtual Thread with Http11Nio2Protocol

[Nicolas BONAMY]

Hi,

I try to use virtual thread on Apache Tomcat 10.1.16 with this configuration on 
macOS or on Linux:




But when I make a request, I'm not on a virtual thread : 
Thread[#76,Thread-14,5,main] . I profiled my application too but no virtual 
threads are used.

If I use a Http11NioProtocol instead of Http11Nio2Protocol, all requests are on 
virtual thread : 
VirtualThread[#65,http-nio-8080-virt-0]/runnable@ForkJoinPool-1-worker-1





Http11Nio2Protocol is not working with virtual threads? Has anyone encountered 
this problem before?

Re: Virtual Thread with Http11Nio2Protocol

[William Crowell]

Nicolas,

Which JDK version?

Regards,

William Crowell

From: Nicolas BONAMY 
Date: Thursday, December 7, 2023 at 2:55 PM
To: users@tomcat.apache.org 
Subject: Virtual Thread with Http11Nio2Protocol
Hi,

I try to use virtual thread on Apache Tomcat 10.1.16 with this configuration on 
macOS or on Linux:




But when I make a request, I'm not on a virtual thread : 
Thread[#76,Thread-14,5,main] . I profiled my application too but no virtual 
threads are used.

If I use a Http11NioProtocol instead of Http11Nio2Protocol, all requests are on 
virtual thread : 
VirtualThread[#65,http-nio-8080-virt-0]/runnable@ForkJoinPool-1-worker-1





Http11Nio2Protocol is not working with virtual threads? Has anyone encountered 
this problem before?


CAUTION: This email originated from outside of the organization. Do not click 
on links or open attachments unless you recognize the sender and know the 
content is safe.


This e-mail may contain information that is privileged or confidential. If you 
are not the intended recipient, please delete the e-mail and any attachments 
and notify us immediately.

Re: Virtual Thread with Http11Nio2Protocol

[Nicolas BONAMY]

William,

I’m using Adoptium Temurin 21.0.1+12

Regards,

Nicolas 

> Le 7 déc. 2023 à 21:00, William Crowell  a 
> écrit :
> 
> Nicolas,
> 
> Which JDK version?
> 
> Regards,
> 
> William Crowell
> 
> From: Nicolas BONAMY 
> Date: Thursday, December 7, 2023 at 2:55 PM
> To: users@tomcat.apache.org 
> Subject: Virtual Thread with Http11Nio2Protocol
> Hi,
> 
> I try to use virtual thread on Apache Tomcat 10.1.16 with this configuration 
> on macOS or on Linux:
> 
> class="org.apache.catalina.core.StandardVirtualThreadExecutor"/>
> 
> protocol="org.apache.coyote.http11.Http11Nio2Protocol"
>   connectionTimeout="2"
>   redirectPort="8443"
>   maxParameterCount="1000"
>   useVirtualThreads="true"
>   />
> But when I make a request, I'm not on a virtual thread : 
> Thread[#76,Thread-14,5,main] . I profiled my application too but no virtual 
> threads are used.
> 
> If I use a Http11NioProtocol instead of Http11Nio2Protocol, all requests are 
> on virtual thread : 
> VirtualThread[#65,http-nio-8080-virt-0]/runnable@ForkJoinPool-1-worker-1
> 
> class="org.apache.catalina.core.StandardVirtualThreadExecutor"/>
> 
> 
> protocol="org.apache.coyote.http11.Http11NioProtocol"
>   connectionTimeout="2"
>   redirectPort="8443"
>   maxParameterCount="1000"
>   useVirtualThreads="true"
>   />
> Http11Nio2Protocol is not working with virtual threads? Has anyone 
> encountered this problem before?
> 
> 
> CAUTION: This email originated from outside of the organization. Do not click 
> on links or open attachments unless you recognize the sender and know the 
> content is safe.
> 
> 
> This e-mail may contain information that is privileged or confidential. If 
> you are not the intended recipient, please delete the e-mail and any 
> attachments and notify us immediately.
> 

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Failing to decode the url correctly in tomcat 9.

[Kalaivani Sengottaiyan]

On Thu, Dec 7, 2023 at 2:34 PM Kalaivani Sengottaiyan <
kalaivani.sengottai...@veeva.com> wrote:

> In one of our sample case, this is the url recorded by ngnix
>
> "-" 127.0.0.1 - - [07/Dec/2023:21:59:30 +] "GET
> /233.0.100?event=autoproc&fn=F%D0%A9%E4%B8%AD%E7%A2%BA%C5%98%C3%86W%C5%A0d%27%C3%9C%CE%94%C3%A1%21%C3%A8%E3%81%AB%EB%AC%B8%C5%84n%C2%B0%C8%99%D0%B4%C4%BE%C3%B3%C3%A1%C3%A5%E0%B8%84%E0%B9%89%C3%A7%D0%B6%C4%90.zip&cksum=1225205368&addparms=Name%7C%7C%7CF%D0%A9%E4%B8%AD%E7%A2%BA%C5%98%C3%86W%C5%A0d%27%C3%9C%CE%94%C3%A1%21%C3%A8%E3%81%AB%EB%AC%B8%C5%84n%C2%B0%C8%99%D0%B4%C4%BE%C3%B3%C3%A1%C3%A5%E0%B8%84%E0%B9%89%C3%A7%D0%B6%C4%90%3B%3B%3B
> HTTP/1.1" 200 0 "-" "curl/7.79.1"
>
>
> When the request is received by the application running within tomcat, url
> is decoded as
>
>
> URL=http://localhost:8080/233.0.100?event=autoproc&fn=F
> Щ中確ŘÆWŠd'ÜΔá!èに문ńn°șдľóáåค้çжĐ.zip&cksum=1225205368&addparms=Name|||FЩ中確ŘÆWŠd'ÜΔá!èに문ńn°șдľóáåค้çжĐ;;;
>
>
>
> If I decode the fn param in the url, it should
> be FЩ中確ŘÆWŠd'ÜΔá!èに문ńn°șдľóáåค้çжĐ.zip rather 
> FЩ中確ŘÆWŠd'ÜΔá!èに문ńn°șдľóáåค้çжĐ.zip.
> Notice the character after d. It is ' (quote), but tomcat decodes as "39;".
> Default LANG is en_US.UTF-8 and connector in conf/server.xml.
>
>
> why is tomcat not url decoding correctly?
>
>
>
> Thanks
>
> Kalai
>

Re: Virtual Thread with Http11Nio2Protocol

[Han Li]

Hi Nicolas,

I took a quick look that Tomcat's VirtualThreadExecutor does not implement the 
ExecutorService interface, which leads to this result.

So I think this is a Tomcat bug.

Han

> On Dec 8, 2023, at 03:55, Nicolas BONAMY  wrote:
> 
> Hi,
> 
> I try to use virtual thread on Apache Tomcat 10.1.16 with this configuration 
> on macOS or on Linux:
> 
> class="org.apache.catalina.core.StandardVirtualThreadExecutor"/>
> 
> protocol="org.apache.coyote.http11.Http11Nio2Protocol"
>   connectionTimeout="2"
>   redirectPort="8443"
>   maxParameterCount="1000"
>   useVirtualThreads="true"
>   />
> But when I make a request, I'm not on a virtual thread : 
> Thread[#76,Thread-14,5,main] . I profiled my application too but no virtual 
> threads are used.
> 
> If I use a Http11NioProtocol instead of Http11Nio2Protocol, all requests are 
> on virtual thread : 
> VirtualThread[#65,http-nio-8080-virt-0]/runnable@ForkJoinPool-1-worker-1
> 
> class="org.apache.catalina.core.StandardVirtualThreadExecutor"/>
> 
> 
> protocol="org.apache.coyote.http11.Http11NioProtocol"
>   connectionTimeout="2"
>   redirectPort="8443"
>   maxParameterCount="1000"
>   useVirtualThreads="true"
>   />
> Http11Nio2Protocol is not working with virtual threads? Has anyone 
> encountered this problem before?


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.

[Bodavula, Sai Vamsi Mohan Krishna (TR Technology)]

Hey Christopher.,
Greetings of the day.


  1.
Might I have confused you with posting the arguments directly ., Yeah as i just 
shared you the annotations with comments , to state you the stuff i am using., 
But in my deployment ., I am using them in catalina opts., and trying to call 
them from values.yaml., which looks like this :

 javaVMMemoryArgument: "-Xms2048M -Xmx10240M -XX:+UseStringDeduplication 
-XX:+UseContainerSupport -Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.local.only=false 
-Dcom.sun.management.jmxremote.port=49151 
-Djava.rmi.server.hostname=lab1workflow4scalsvc2zus1-service.hqm-lab1.svc.cluster.local
 "

  and I am referring the word "javaVMMemoryArgument" from values yaml and 
calling it in Catalina_opts, so that it would fetch all these 
annotations as mentioned above, during the deployment. This is my deployment 
part., where I am referring to the above values from values.yaml
env:
- name: CATALINA_OPTS
  value: {{ .Values.deployment.javaVMMemoryArgument }}





  1.
Coming to Process., I have searched for Java process that listens on my 
mentioned port ie., 49151, but none of the process is listening to that process.

I even tried with
root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn | 
grep LISTEN
tcp6   0  0 :::10109:::*LISTEN  
1/java
tcp6   0  0 :::9109 :::*LISTEN  
1/java
root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn | 
more
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address   Foreign Address State   
PID/Program name
tcp6   0  0 :::10109:::*LISTEN  
1/java
tcp6   0  0 :::9109 :::*LISTEN  
1/java
root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn | 
grep ':443'netstat -tulpn | grep ':443'^C
root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn | 
grep ':49151'
root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/#

which confirms me that , any of the process is being listening on the port 
49151.

  2.
I would like to request you to suggest me with a better approach ., where i am 
missing anything in this process!


 



Thanks & Regards,
--

SAI VAMSI .B
Senior DevOps Engineer



From: Christopher Schultz 
Sent: Thursday, December 7, 2023 23:33
To: users@tomcat.apache.org 
Subject: Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.

Sai Vamsi,

On 12/6/23 11:04, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote:
> Hello Christopher.
> Thanks for the response!
>
> Let me walk you through the Environment.,
> I have a cluster and I am installing multiple microservices in the cluster. 
> And one of the microservices I am integrating the JMX on of the 
> microservices.,
> and the mentioned IP of the pod is being changed dynamically after each 
> deployment .
>
> Here are the annotations that I am using in my deployment:
> - name: JAVA_OPTS
>  #   value: >-
>  # -Dcom.sun.management.jmxremote
>  # -Dcom.sun.management.jmxremote.authenticate=false
>  # -Dcom.sun.management.jmxremote.ssl=false
>  # -Dcom.sun.management.jmxremote.local.only=false
>  # -Dcom.sun.management.jmxremote.port=49151
>  # -Dcom.sun.management.jmxremote.rmi.port=49151
>  # 
> -Djava.rmi.server.hostname=lab1workflow4scalsvc2zus1-service.hqm-lab1.svc.cluster.local

If that's YAML, then all of the above is comments. Those lines do nothing.

> as i said , I have implemented these in three scenarios., Catalina_opts / 
> Java_opts / Java_Tool_opts

You should definitely use CATALINA_OPTS and none of the other things you
mention, here. Specifically because of the port issues. JAVA_TOOL_OPTS
don't affect Tomcat at all are useless for you. JAVA_OPTS affects all
Tomcat-related scripts, and you'll find that even "shutdown" causes an
error... because it's trying to initialize JMX when shutting-down, which
is again useless.

> I have tried deploying catalina_ots and Java_opts as in the case of above 
> mentioned.
> I also tried other way, mentioning all these in the values.yaml file ( I am 
> using Helm Deployment to upgrade my deployments) and referring these 
> annotations as , and mentioning them in the Value
> env:
>  - name: CATALINA_OPTS
>value: {{ .Values.deployment.javaVMMemoryArgument }}
>
>
> getting netstat output from my pod:
> root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn
> Active Internet connections (only servers)
> Proto Recv-Q Send-Q Local Address   Foreign Address S