[
https://issues.apache.org/jira/browse/SOLR-14896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris M. Hostetter updated SOLR-14896:
--------------------------------------
Description:
I'm getting some offline reports from users who recently upgraded to solr 8.6
of non-reliably reproducing errors from Jetty when doing the HTTP header
parsing. these errors result in a jetty response that will typically look
something like...
{noformat}
HTTP/1.1 400 Illegal character VCHAR='='
Content-Type: text/html;charset=iso-8859-1
Content-Length: 70
Connection: close
<h1>Bad Message 400</h1><pre>reason: Illegal character VCHAR='='</pre>
{noformat}
...the exact "Illegal character" can vary. Based on some review of the
solr/jetty code bases these errors come from Jetty's HttpParser class when
unexpected character types are encountered while parsing the HTTP METHOD or
Header *NAMES* (VCHAR is a legal token type in Header _VALUES_)
----
These errors sometimes manifest in log files as RemoteSolrExceptions due to
intra-node communication, but I've been told they can also be returned directly
by solr/jetty when clients make requests – _even when the client is not using
SolrJ_ – suggesting that whatever is causing the "malformed" request is not
specifically a bug in solr/solrj – this seems like a jetty bug.
I'm suspect that the underlying cause is jetty buffers being re-used/shared
between multiple requests, and suspicious that this is caused by / related to
[jetty issue#4936|https://github.com/eclipse/jetty.project/issues/4936] / aka
[jetty bug#564984|https://bugs.eclipse.org/bugs/show_bug.cgi?id=564984] .. -BUT
... that issue suggests that the buffer re-use situation *ONLY* arises due to a
prior "500: Response header too large" error – but i have not been able to
confirm that that situation has ever existed on any of the affected servers
where this "400 Illegal character" error occurs.-
was:
I'm getting some offline reports from users who recently upgraded to solr 8.6
of non-reliably reproducing errors from Jetty when doing the HTTP header
parsing. these errors result in a jetty response that will typically look
something like...
{noformat}
HTTP/1.1 400 Illegal character VCHAR='='
Content-Type: text/html;charset=iso-8859-1
Content-Length: 70
Connection: close
<h1>Bad Message 400</h1><pre>reason: Illegal character VCHAR='='</pre>
{noformat}
...the exact "Illegal character" can vary. Based on some review of the
solr/jetty code bases these errors come from Jetty's HttpParser class when
unexpected character types are encountered while parsing the HTTP METHOD or
Header *NAMES* (VCHAR is a legal token type in Header _VALUES_)
----
These errors sometimes manifest in log files as RemoteSolrExceptions due to
intra-node communication, but I've been told they can also be returned directly
by solr/jetty when clients make requests -- _even when the client is not using
SolrJ_ -- suggesting that whatever is causing the "malformed" request is not
specifically a bug in solr/solrj -- this seems like a jetty bug.
I'm suspect that the underlying cause is jetty buffers being re-used/shared
between multiple requests, and suspicious that this is caused by / related to
[jetty issue#4936|https://github.com/eclipse/jetty.project/issues/4936] / aka
[jetty bug#564984|https://bugs.eclipse.org/bugs/show_bug.cgi?id=564984] .. BUT
... that issue suggests that the buffer re-use situation *ONLY* arises due to a
prior "500: Response header too large" error -- but i have not been able to
confirm that that situation has ever existed on any of the affected servers
where this "400 Illegal character" error occurs.
I've revised the description of this jira to strike out my concerns that this
may not be directly caused by [jetty
issue#4936|https://github.com/eclipse/jetty.project/issues/4936] – i was
missinformed, and have confirmed several situations of "500: Response header
too large" errors in the solr log files at some point prior to these "400 Bad
Message" errors being returned by the same node.
> jetty "Bad Message 400" / "Illegal character" responses to sporadic requests
> ----------------------------------------------------------------------------
>
> Key: SOLR-14896
> URL: https://issues.apache.org/jira/browse/SOLR-14896
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Chris M. Hostetter
> Priority: Major
>
> I'm getting some offline reports from users who recently upgraded to solr 8.6
> of non-reliably reproducing errors from Jetty when doing the HTTP header
> parsing. these errors result in a jetty response that will typically look
> something like...
> {noformat}
> HTTP/1.1 400 Illegal character VCHAR='='
> Content-Type: text/html;charset=iso-8859-1
> Content-Length: 70
> Connection: close
> <h1>Bad Message 400</h1><pre>reason: Illegal character VCHAR='='</pre>
> {noformat}
> ...the exact "Illegal character" can vary. Based on some review of the
> solr/jetty code bases these errors come from Jetty's HttpParser class when
> unexpected character types are encountered while parsing the HTTP METHOD or
> Header *NAMES* (VCHAR is a legal token type in Header _VALUES_)
> ----
> These errors sometimes manifest in log files as RemoteSolrExceptions due to
> intra-node communication, but I've been told they can also be returned
> directly by solr/jetty when clients make requests – _even when the client is
> not using SolrJ_ – suggesting that whatever is causing the "malformed"
> request is not specifically a bug in solr/solrj – this seems like a jetty bug.
> I'm suspect that the underlying cause is jetty buffers being re-used/shared
> between multiple requests, and suspicious that this is caused by / related to
> [jetty issue#4936|https://github.com/eclipse/jetty.project/issues/4936] / aka
> [jetty bug#564984|https://bugs.eclipse.org/bugs/show_bug.cgi?id=564984] ..
> -BUT ... that issue suggests that the buffer re-use situation *ONLY* arises
> due to a prior "500: Response header too large" error – but i have not been
> able to confirm that that situation has ever existed on any of the affected
> servers where this "400 Illegal character" error occurs.-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
