[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689352137 The import error was fixed already. It was not caused by this PR, but a change in master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14439) Upgrade to Tika 1.24.1
[ https://issues.apache.org/jira/browse/SOLR-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192690#comment-17192690 ] Andras Salamon commented on SOLR-14439: --- [~tallison] Can you please change the status to patch available, to start the precommit checks. > Upgrade to Tika 1.24.1 > -- > > Key: SOLR-14439 > URL: https://issues.apache.org/jira/browse/SOLR-14439 > Project: Solr > Issue Type: Task > Components: contrib - DataImportHandler >Reporter: Tim Allison >Assignee: Tim Allison >Priority: Major > Attachments: SOLR-14339.patch > > > We recently released 1.24.1 with several fixes for DoS vulnerabilities we > found via fuzzing: CVE-2020-9489 https://seclists.org/oss-sec/2020/q2/69 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9513) Use seconds instead of millisecs
Frank Zhu created LUCENE-9513: - Summary: Use seconds instead of millisecs Key: LUCENE-9513 URL: https://issues.apache.org/jira/browse/LUCENE-9513 Project: Lucene - Core Issue Type: Improvement Components: core/other Affects Versions: 8.6.2, master (9.0) Reporter: Frank Zhu In RangeFacetsExample, there is a trivial mistake where private final long nowSec is supposed to be secs. But it uses as millisecs. It doesn't affect the output due to the scale change (It will just make PAST_HOUR, PAST_SIX_HOURS not as intended). Still it's better to be correct. I've created a pull request on github with the simple fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9513) Use seconds instead of millisecs
[ https://issues.apache.org/jira/browse/LUCENE-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Zhu updated LUCENE-9513: -- Description: In RangeFacetsExample, there is a trivial mistake where private final long nowSec is supposed to be secs. But it uses as millisecs. It doesn't affect the output due to the scale change (It will just make PAST_HOUR, PAST_SIX_HOURS not as intended). Still it's better to be correct. I've created a pull request on github with the simple fix. https://github.com/apache/lucene-solr/pull/1846 was: In RangeFacetsExample, there is a trivial mistake where private final long nowSec is supposed to be secs. But it uses as millisecs. It doesn't affect the output due to the scale change (It will just make PAST_HOUR, PAST_SIX_HOURS not as intended). Still it's better to be correct. I've created a pull request on github with the simple fix. > Use seconds instead of millisecs > > > Key: LUCENE-9513 > URL: https://issues.apache.org/jira/browse/LUCENE-9513 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Affects Versions: master (9.0), 8.6.2 >Reporter: Frank Zhu >Priority: Trivial > > In RangeFacetsExample, there is a trivial mistake where > private final long nowSec is supposed to be secs. > But it uses as millisecs. > It doesn't affect the output due to the scale change (It will just make > PAST_HOUR, PAST_SIX_HOURS not as intended). Still it's better to be correct. > > I've created a pull request on github with the simple fix. > https://github.com/apache/lucene-solr/pull/1846 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9513) Use seconds instead of millisecs
[ https://issues.apache.org/jira/browse/LUCENE-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Zhu updated LUCENE-9513: -- Fix Version/s: master (9.0) > Use seconds instead of millisecs > > > Key: LUCENE-9513 > URL: https://issues.apache.org/jira/browse/LUCENE-9513 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Affects Versions: master (9.0), 8.6.2 >Reporter: Frank Zhu >Priority: Trivial > Fix For: master (9.0) > > > In RangeFacetsExample, there is a trivial mistake where > private final long nowSec is supposed to be secs. > But it uses as millisecs. > It doesn't affect the output due to the scale change (It will just make > PAST_HOUR, PAST_SIX_HOURS not as intended). Still it's better to be correct. > > I've created a pull request on github with the simple fix. > https://github.com/apache/lucene-solr/pull/1846 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9514) Include TermVectorsWriter in DWPT accounting
Simon Willnauer created LUCENE-9514: --- Summary: Include TermVectorsWriter in DWPT accounting Key: LUCENE-9514 URL: https://issues.apache.org/jira/browse/LUCENE-9514 Project: Lucene - Core Issue Type: Improvement Reporter: Simon Willnauer As a followup of LUCENE-9511 we should also include the TermVectorsWriter into the DWPT accounting since it might hold on to RAM in it's byte buffer recycling etc. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9511) Include StoredFieldsWriter in DWPT accounting
[
https://issues.apache.org/jira/browse/LUCENE-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192701#comment-17192701
]
Simon Willnauer commented on LUCENE-9511:
-
{quote}I guess this would mean applications using more threads to index would
have seen RAM consumed but not tracked properly by {{IndexWriter}} until this
fix.
{quote}
That is true. I think it's insignificant for most of our users since it only
really comes into play when you are tons of flushes and a large number of
blocked DWPT that are otherwise flush pending. That situation requires slow
flushes and slow commits as well as the situation that the DWPTs that are
flushing or pending are very small by itself ie. only a couple of small docs.
Otherwise stall control would kick in earlier and prevent adding more docs.
Still lucene is used in so many environments and having large amounts of
threads concurrently might add up here.
> Include StoredFieldsWriter in DWPT accounting
> -
>
> Key: LUCENE-9511
> URL: https://issues.apache.org/jira/browse/LUCENE-9511
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> StoredFieldsWriter might consume some heap space memory that can have a
> significant impact on decisions made in the IW if writers should be stalled
> or DWPTs should be flushed if memory settings are small in IWC and flushes
> are frequent. We should add some accounting to the StoredFieldsWriter since
> it's part of the DWPT lifecycle and not just present during flush.
> Our nightly builds ran into some OOMs due to the large chunk size used in the
> CompressedStoredFieldsFormat. The reason are very frequent flushes due to
> small maxBufferedDocs which causes 300+ DWPTs to be blocked for flush causing
> ultimately an OOM exception.
> {noformat}
>
> NOTE: reproduce with: ant test -Dtestcase=TestIndexingSequenceNumbers
> -Dtests.method=testStressConcurrentCommit -Dtests.seed=A04943A98C8E2954
> -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=vo-001 -Dtests.timezone=Africa/Ouagadougou
> -Dtests.asserts=true -Dtests.file.encoding=UTF8*06:06:15*[junit4] ERROR
> 107s J3 | TestIndexingSequenceNumbers.testStressConcurrentCommit
> <<<*06:06:15*[junit4]> Throwable #1:
> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is
> closed*06:06:15*[junit4]>at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:876)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:890)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3727)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit(TestIndexingSequenceNumbers.java:228)*06:06:15*
> [junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)*06:06:15*[junit4]>at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)*06:06:15*
> [junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)*06:06:15*
> [junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)*06:06:15*
> [junit4]>at
> java.base/java.lang.Thread.run(Thread.java:834)*06:06:15*[junit4]>
> Caused by: java.lang.OutOfMemoryError: Java heap space*06:06:15*[junit4]
> > at
> __randomizedtesting.SeedInfo.seed([A04943A98C8E2954]:0)*06:06:15*[junit4]
>> at
> org.apache.lucene.store.GrowableByteArrayDataOutput.(GrowableByteArrayDataOutput.java:46)*06:06:15*
> [junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.(CompressingStoredFieldsWriter.java:111)*06:06:15*
> [junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:130)*06:06:15*
> [junit4]> at
> org.apache.lucene.codecs.lucene87.Lucene87StoredFieldsFormat.fieldsWriter(Lucene87StoredFieldsFormat.java:141)*06:06:15*
> [junit4]>at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:39)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:46)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIn
[GitHub] [lucene-solr] s1monw opened a new pull request #1847: LUCENE-9514: Include TermVectorsWriter in DWPT accounting
s1monw opened a new pull request #1847: URL: https://github.com/apache/lucene-solr/pull/1847 TermVectorsWriter might consume some heap space memory that can have a significant impact on decisions made in the IW if writers should be stalled or DWPTs should be flushed if memory settings are small in IWC and flushes are frequent. This change adds RAM accounting to the TermVectorsWriter since it's part of the DWPT lifecycle and not just present during flush. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-9511) Include StoredFieldsWriter in DWPT accounting
[
https://issues.apache.org/jira/browse/LUCENE-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192701#comment-17192701
]
Simon Willnauer edited comment on LUCENE-9511 at 9/9/20, 7:35 AM:
--
{quote}I guess this would mean applications using more threads to index would
have seen RAM consumed but not tracked properly by {{IndexWriter}} until this
fix.
{quote}
That is true. I think it's insignificant for most of our users since it only
really comes into play when you are tons of flushes and a large number of
blocked DWPT that are otherwise flush pending. That situation requires slow
flushes and slow commits as well as the situation that the DWPTs that are
flushing or pending are very small by itself ie. only a couple of small docs.
Otherwise stall control would kick in earlier and prevent adding more docs.
Still lucene is used in so many environments and having large amounts of
threads concurrently might add up here.
What I am more concerned about is that if we have a bug in the code causing
large amounts of memory allocated will likely be undetected unless we see some
slowdowns due to flushing and it will help here to detect this, I hope :)
was (Author: simonw):
{quote}I guess this would mean applications using more threads to index would
have seen RAM consumed but not tracked properly by {{IndexWriter}} until this
fix.
{quote}
That is true. I think it's insignificant for most of our users since it only
really comes into play when you are tons of flushes and a large number of
blocked DWPT that are otherwise flush pending. That situation requires slow
flushes and slow commits as well as the situation that the DWPTs that are
flushing or pending are very small by itself ie. only a couple of small docs.
Otherwise stall control would kick in earlier and prevent adding more docs.
Still lucene is used in so many environments and having large amounts of
threads concurrently might add up here.
> Include StoredFieldsWriter in DWPT accounting
> -
>
> Key: LUCENE-9511
> URL: https://issues.apache.org/jira/browse/LUCENE-9511
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> StoredFieldsWriter might consume some heap space memory that can have a
> significant impact on decisions made in the IW if writers should be stalled
> or DWPTs should be flushed if memory settings are small in IWC and flushes
> are frequent. We should add some accounting to the StoredFieldsWriter since
> it's part of the DWPT lifecycle and not just present during flush.
> Our nightly builds ran into some OOMs due to the large chunk size used in the
> CompressedStoredFieldsFormat. The reason are very frequent flushes due to
> small maxBufferedDocs which causes 300+ DWPTs to be blocked for flush causing
> ultimately an OOM exception.
> {noformat}
>
> NOTE: reproduce with: ant test -Dtestcase=TestIndexingSequenceNumbers
> -Dtests.method=testStressConcurrentCommit -Dtests.seed=A04943A98C8E2954
> -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=vo-001 -Dtests.timezone=Africa/Ouagadougou
> -Dtests.asserts=true -Dtests.file.encoding=UTF8*06:06:15*[junit4] ERROR
> 107s J3 | TestIndexingSequenceNumbers.testStressConcurrentCommit
> <<<*06:06:15*[junit4]> Throwable #1:
> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is
> closed*06:06:15*[junit4]>at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:876)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:890)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3727)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit(TestIndexingSequenceNumbers.java:228)*06:06:15*
> [junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)*06:06:15*[junit4]>at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)*06:06:15*
> [junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)*06:06:15*
> [junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)*06:06:15*
> [junit4]>at
> java.base/java.lang.Thread.run(Thread.java:834)*06:06:15*[junit4]>
> Caused by: java.lang.OutOfMemoryError: Java heap space*06:06:15*[junit4]
> > at
> __randomizedtesting.SeedInfo.seed([A04943A98C8E2954]:0)*06:06:15*[junit4]
>> at
> org.apache.lucene.store
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689369526 I merged from master to retrigger the check. Clicking on "Rerun checks" did not work, as it merged into same master commit as before. Makes no sense to me (maybe because of reproducibility). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14768) Error 500 on PDF extraction: java.lang.NoClassDefFoundError: org/eclipse/jetty/server/MultiParts
[
https://issues.apache.org/jira/browse/SOLR-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192711#comment-17192711
]
Markus Kalkbrenner commented on SOLR-14768:
---
I just want to mention that I created this issue because it has been caught by
automated tests ;)
[https://github.com/solariumphp/solarium/actions] and
[https://github.com/mkalkbrenner/search_api_solr/actions]
Beside PDF extraction we test things like the Collections API, Streaming
Expressions, Facets, JSON Facets, ...
Therefore we run Solr docker containers. Maybe we can add a test that includes
a build of a docker container using the latest Solr dev version?
> Error 500 on PDF extraction: java.lang.NoClassDefFoundError:
> org/eclipse/jetty/server/MultiParts
>
>
> Key: SOLR-14768
> URL: https://issues.apache.org/jira/browse/SOLR-14768
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - Solr Cell (Tika extraction)
>Affects Versions: 8.6, 8.6.1
>Reporter: Markus Kalkbrenner
>Assignee: David Smiley
>Priority: Major
> Attachments: Solr v8.6.x fails with multipart MIME in commands.eml,
> Solr v8.6.x fails with multipart MIME in commands.eml, Solr v8.6.x fails with
> multipart MIME in commands.eml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> See [https://www.mail-archive.com/solr-user@lucene.apache.org/msg152182.html]
> The integration tests of the solarium PHP library and the integration tests
> of the Search API Solr Drupal module both fail on PDF extraction if executed
> on Solr 8.6.
> They still work on Solr 8.5.1 an earlier versions.
> {quote}2020-08-20 12:30:35.279 INFO (qtp855700733-19) [ x:5f3e6ce2810ef]
> o.a.s.u.p.LogUpdateProcessorFactory [5f3e6ce2810ef] webapp=/solr
> path=/update/extract
> params=\{json.nl=flat&commitWithin=0&omitHeader=false&resource.name=testpdf.pdf&literal.id=extract-test&commit=true&extractOnly=false&uprefix=attr_&wt=json}{add=[extract-test
> (1675547519474466816)],commit=} 0 957
> solr8_1 | 2020-08-20 12:30:35.280 WARN (qtp855700733-19) [ ]
> o.e.j.s.HttpChannel /solr/5f3e6ce2810ef/update/extract =>
> java.lang.NoClassDefFoundError: org/eclipse/jetty/server/MultiParts
> solr8_1 | at
> org.apache.solr.servlet.SolrRequestParsers.cleanupMultipartFiles(SolrRequestParsers.java:624)
> solr8_1 | java.lang.NoClassDefFoundError: org/eclipse/jetty/server/MultiParts
> solr8_1 | at
> org.apache.solr.servlet.SolrRequestParsers.cleanupMultipartFiles(SolrRequestParsers.java:624)
> ~[?:?]
> solr8_1 | at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:443)
> ~[?:?]
> solr8_1 | at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
> ~[?:?]
> solr8_1 | at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596)
> ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
> ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590)
> ~[jetty-security-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
> ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedH
[jira] [Comment Edited] (SOLR-14768) Error 500 on PDF extraction: java.lang.NoClassDefFoundError: org/eclipse/jetty/server/MultiParts
[
https://issues.apache.org/jira/browse/SOLR-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192711#comment-17192711
]
Markus Kalkbrenner edited comment on SOLR-14768 at 9/9/20, 7:54 AM:
I just want to mention that I created this issue because it has been caught by
automated tests ;)
[https://github.com/solariumphp/solarium/actions] and
[https://github.com/mkalkbrenner/search_api_solr/actions]
Beside PDF extraction we test things like the Collections API, Streaming
Expressions, Facets, JSON Facets, ...
Therefore we run Solr docker containers. Maybe we can add a test that includes
a build of a docker container using the latest Solr dev version?
BTW both test suites allways use the latest versions of Solr 7 and 8. At the
moment we had to break this rule and force Solr 8.5 instead of the latest
version to get the tests to pass again to not interrupt the development of
these frameworks.
was (Author: mkalkbrenner):
I just want to mention that I created this issue because it has been caught by
automated tests ;)
[https://github.com/solariumphp/solarium/actions] and
[https://github.com/mkalkbrenner/search_api_solr/actions]
Beside PDF extraction we test things like the Collections API, Streaming
Expressions, Facets, JSON Facets, ...
Therefore we run Solr docker containers. Maybe we can add a test that includes
a build of a docker container using the latest Solr dev version?
> Error 500 on PDF extraction: java.lang.NoClassDefFoundError:
> org/eclipse/jetty/server/MultiParts
>
>
> Key: SOLR-14768
> URL: https://issues.apache.org/jira/browse/SOLR-14768
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - Solr Cell (Tika extraction)
>Affects Versions: 8.6, 8.6.1
>Reporter: Markus Kalkbrenner
>Assignee: David Smiley
>Priority: Major
> Attachments: Solr v8.6.x fails with multipart MIME in commands.eml,
> Solr v8.6.x fails with multipart MIME in commands.eml, Solr v8.6.x fails with
> multipart MIME in commands.eml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> See [https://www.mail-archive.com/solr-user@lucene.apache.org/msg152182.html]
> The integration tests of the solarium PHP library and the integration tests
> of the Search API Solr Drupal module both fail on PDF extraction if executed
> on Solr 8.6.
> They still work on Solr 8.5.1 an earlier versions.
> {quote}2020-08-20 12:30:35.279 INFO (qtp855700733-19) [ x:5f3e6ce2810ef]
> o.a.s.u.p.LogUpdateProcessorFactory [5f3e6ce2810ef] webapp=/solr
> path=/update/extract
> params=\{json.nl=flat&commitWithin=0&omitHeader=false&resource.name=testpdf.pdf&literal.id=extract-test&commit=true&extractOnly=false&uprefix=attr_&wt=json}{add=[extract-test
> (1675547519474466816)],commit=} 0 957
> solr8_1 | 2020-08-20 12:30:35.280 WARN (qtp855700733-19) [ ]
> o.e.j.s.HttpChannel /solr/5f3e6ce2810ef/update/extract =>
> java.lang.NoClassDefFoundError: org/eclipse/jetty/server/MultiParts
> solr8_1 | at
> org.apache.solr.servlet.SolrRequestParsers.cleanupMultipartFiles(SolrRequestParsers.java:624)
> solr8_1 | java.lang.NoClassDefFoundError: org/eclipse/jetty/server/MultiParts
> solr8_1 | at
> org.apache.solr.servlet.SolrRequestParsers.cleanupMultipartFiles(SolrRequestParsers.java:624)
> ~[?:?]
> solr8_1 | at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:443)
> ~[?:?]
> solr8_1 | at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
> ~[?:?]
> solr8_1 | at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596)
> ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
> ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590)
> ~[jetty-security-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
> ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
> solr8_1 | at
> org.eclipse.jetty.server.handler.ScopedHand
[GitHub] [lucene-solr] dweiss commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
dweiss commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689410243 I've tried to clean up things a bit and ended up implementing both forked and in-JVM mode. It's not super beautiful (those intermediate interfaces could probably be simplified) but it works and allows for some interesting combinations of mixed forked-and-local workers. I committed this in to your branch, Uwe, but it's really subject to criticism, modifications and/or reversal - your decision. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9515) Detach DWPT from DefaultIndexingChain
Simon Willnauer created LUCENE-9515: --- Summary: Detach DWPT from DefaultIndexingChain Key: LUCENE-9515 URL: https://issues.apache.org/jira/browse/LUCENE-9515 Project: Lucene - Core Issue Type: Improvement Affects Versions: master (9.0), 8.7 Reporter: Simon Willnauer DefaultIndexingChain or rather its super class DocConsumer is still tightly coupled to DocumentsWriterPerThread. In oder to guarantee better code isolation and hopefully independent testing down the road we can expand the IndexingChain interface to take all the primitives needed rather than the complex DWPT class. It's unclear anyway how this API (DocConsumer and IndexingChain) should be implemented with all the classes it depends on being package private. I will open a follow-up to discuss if we can remove that abstraction. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689410718 > I don't want to block the changes here, but just think we should make a followup to implement the same test, but with clients in the same JVM. > > There are two separate cases for NIO locks (held by same VM vs held by another process), so we should stress test them both. > Past experience has shown that whatever lock testing we have, it is somehow never enough. I think this could be a separate issue. Basically, this just "restores" the test in master, which was done in Ant only. There are no functional changes. But this new code is easy to adapt, there are several possibilities: - We can clone the whole test and just replaces `List processes` by a thread pool and instead of spawning a JVM we just call main() in a thread. Cleanup is more or less the same. - We can modify the current test to not only spawn multiple JVMs, but aditionally also add a thread. We can even randomize the number of clients/threads (currently fixed to 2 clients, but it's just a change in test config). In both cases, the policy also needs the "connect" permission. An alternative would be to allow client/server to alternatively communicate through a PipedInput/OutputStream (if its in same JVM). Some questions before I commit this: - Should I rename the test to TestStressLockFactories? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] s1monw opened a new pull request #1848: LUCENE-9515: Detach DWPT from DefaultIndexingChain
s1monw opened a new pull request #1848: URL: https://github.com/apache/lucene-solr/pull/1848 This change removes the DWPT dependency from DefaultIndexingChain and rather passes on the primitives needed for creating the chain. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9516) Remove DocConsumer and IndexingChain from Lucene
Simon Willnauer created LUCENE-9516: --- Summary: Remove DocConsumer and IndexingChain from Lucene Key: LUCENE-9516 URL: https://issues.apache.org/jira/browse/LUCENE-9516 Project: Lucene - Core Issue Type: Wish Reporter: Simon Willnauer Disclaimer: This is a breaking change! We allow today to replace the entire indexing chain which is a fundamental part of our software. I personally don't know if there are any users of this API. but given the complexity I personally don't think we should further support this. If you are willing to implement this entire thing yourself I really wonder if you are better off building lucene from the source. An option like this on IWC might make users look into it while I am convinced they shouldn't. It's too complex and nothing is made for reuse down there. I wonder what others think but from my perspective it's time to remove it in 9.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9511) Include StoredFieldsWriter in DWPT accounting
[
https://issues.apache.org/jira/browse/LUCENE-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192727#comment-17192727
]
ASF subversion and git services commented on LUCENE-9511:
-
Commit 16a703fa14f6bab8a6852a82dddadd48a63df732 in lucene-solr's branch
refs/heads/branch_8x from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=16a703f ]
LUCENE-9511: Increase RAM buffer in test since now we account more ram that
get's allocated
> Include StoredFieldsWriter in DWPT accounting
> -
>
> Key: LUCENE-9511
> URL: https://issues.apache.org/jira/browse/LUCENE-9511
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> StoredFieldsWriter might consume some heap space memory that can have a
> significant impact on decisions made in the IW if writers should be stalled
> or DWPTs should be flushed if memory settings are small in IWC and flushes
> are frequent. We should add some accounting to the StoredFieldsWriter since
> it's part of the DWPT lifecycle and not just present during flush.
> Our nightly builds ran into some OOMs due to the large chunk size used in the
> CompressedStoredFieldsFormat. The reason are very frequent flushes due to
> small maxBufferedDocs which causes 300+ DWPTs to be blocked for flush causing
> ultimately an OOM exception.
> {noformat}
>
> NOTE: reproduce with: ant test -Dtestcase=TestIndexingSequenceNumbers
> -Dtests.method=testStressConcurrentCommit -Dtests.seed=A04943A98C8E2954
> -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=vo-001 -Dtests.timezone=Africa/Ouagadougou
> -Dtests.asserts=true -Dtests.file.encoding=UTF8*06:06:15*[junit4] ERROR
> 107s J3 | TestIndexingSequenceNumbers.testStressConcurrentCommit
> <<<*06:06:15*[junit4]> Throwable #1:
> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is
> closed*06:06:15*[junit4]>at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:876)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:890)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3727)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit(TestIndexingSequenceNumbers.java:228)*06:06:15*
> [junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)*06:06:15*[junit4]>at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)*06:06:15*
> [junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)*06:06:15*
> [junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)*06:06:15*
> [junit4]>at
> java.base/java.lang.Thread.run(Thread.java:834)*06:06:15*[junit4]>
> Caused by: java.lang.OutOfMemoryError: Java heap space*06:06:15*[junit4]
> > at
> __randomizedtesting.SeedInfo.seed([A04943A98C8E2954]:0)*06:06:15*[junit4]
>> at
> org.apache.lucene.store.GrowableByteArrayDataOutput.(GrowableByteArrayDataOutput.java:46)*06:06:15*
> [junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.(CompressingStoredFieldsWriter.java:111)*06:06:15*
> [junit4]> at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:130)*06:06:15*
> [junit4]> at
> org.apache.lucene.codecs.lucene87.Lucene87StoredFieldsFormat.fieldsWriter(Lucene87StoredFieldsFormat.java:141)*06:06:15*
> [junit4]>at
> org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:39)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:46)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:426)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:462)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:233)*06:06:15*
> [junit4]> at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:419)*06:06:15*
> [jun
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689463433 Hi @dweiss indeed this looks now very complicated (before the test was so simple after my changes from last night). Indeed I want to remove some abstraction, the factory is unneeded. I always try to use the default functional interfaces provided by Java 8, and just creating one for exactly one methods is code bloat. A simple if/else in the factory method would be as fine. We should maybe also change the number of client to 3 instead of 2. Also during nightly tests we can hammer with even more clients. What I committed in: I disabled all custom Mock Filesystems, as this is important to run on native OS without any wrapping. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9517) BugfixDeflater_JDK8252739 causes Java security issues in JDk11
Ignacio Vera created LUCENE-9517:
Summary: BugfixDeflater_JDK8252739 causes Java security issues in
JDk11
Key: LUCENE-9517
URL: https://issues.apache.org/jira/browse/LUCENE-9517
Project: Lucene - Core
Issue Type: Bug
Reporter: Ignacio Vera
We are running into issues when running Elasticsearch CI with java security
turned on and using JDK11 (only for the ones that contains the jdk bug ). The
errors look like:
{{}}
{{}}
{code:java}
java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "accessDeclaredMembers") {code}
{{}}
The issue seems to be here:
[http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l989]
As we now have a subclass that wants to run this code. Note that this code has
been removed in JDK12 and above.
We might need to wrap the creation of this object in a doPriviledged Block or
find a different solution that does not need to subclass the Deflater class.
cc: [~uschindler]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9517) BugfixDeflater_JDK8252739 causes Java security issues in JDk11
[
https://issues.apache.org/jira/browse/LUCENE-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ignacio Vera updated LUCENE-9517:
-
Description:
We are running into issues when running Elasticsearch CI with java security
turned on and using JDK11 (only for the ones that contains the jdk bug ). The
errors look like:
{code:java}
java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "accessDeclaredMembers") {code}
The issue seems to be here:
[http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l989]
As we now have a subclass that wants to run this code. Note that this code has
been removed in JDK12 and above.
We might need to wrap the creation of this object in a doPriviledged Block or
find a different solution that does not need to subclass the Deflater class.
cc: [~uschindler]
was:
We are running into issues when running Elasticsearch CI with java security
turned on and using JDK11 (only for the ones that contains the jdk bug ). The
errors look like:
{{}}
{{}}
{code:java}
java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "accessDeclaredMembers") {code}
{{}}
The issue seems to be here:
[http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l989]
As we now have a subclass that wants to run this code. Note that this code has
been removed in JDK12 and above.
We might need to wrap the creation of this object in a doPriviledged Block or
find a different solution that does not need to subclass the Deflater class.
cc: [~uschindler]
> BugfixDeflater_JDK8252739 causes Java security issues in JDk11
> --
>
> Key: LUCENE-9517
> URL: https://issues.apache.org/jira/browse/LUCENE-9517
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Ignacio Vera
>Priority: Major
>
> We are running into issues when running Elasticsearch CI with java security
> turned on and using JDK11 (only for the ones that contains the jdk bug ).
> The errors look like:
>
>
> {code:java}
> java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessDeclaredMembers") {code}
>
> The issue seems to be here:
> [http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l989]
> As we now have a subclass that wants to run this code. Note that this code
> has been removed in JDK12 and above.
> We might need to wrap the creation of this object in a doPriviledged Block or
> find a different solution that does not need to subclass the Deflater class.
>
> cc: [~uschindler]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1847: LUCENE-9514: Include TermVectorsWriter in DWPT accounting
mikemccand commented on a change in pull request #1847:
URL: https://github.com/apache/lucene-solr/pull/1847#discussion_r485497049
##
File path: lucene/core/src/java/org/apache/lucene/util/Accountable.java
##
@@ -41,4 +41,8 @@
return Collections.emptyList();
}
+ /**
+ * An accountable that always returns 0
+ */
+ Accountable NULL_ACCOUNTABLE = () -> 0;
Review comment:
Interface members are always `static final` right?
##
File path:
lucene/codecs/src/java/org/apache/lucene/codecs/simpletext/SimpleTextTermVectorsWriter.java
##
@@ -193,4 +189,9 @@ private void write(BytesRef bytes) throws IOException {
private void newLine() throws IOException {
SimpleTextUtil.writeNewline(out);
}
+
+ @Override
+ public long ramBytesUsed() {
+return scratch.get().bytes.length;
Review comment:
Ha.
##
File path:
lucene/core/src/java/org/apache/lucene/index/StoredFieldsConsumer.java
##
@@ -42,6 +44,7 @@
protected void initStoredFieldsWriter() throws IOException {
if (writer == null) { // TODO can we allocate this in the ctor? we call
start document for every doc anyway
this.writer = codec.storedFieldsFormat().fieldsWriter(directory, info,
IOContext.DEFAULT);
+ accountable = writer;
Review comment:
Hmm what is happening here? Are we using this new `accountable` member
somewhere? I don't see it in the diffs? Oh, I see, it is returned in
`DefaultIndexingChain.getChildResources()`, ok! Maybe add comment above its
declaration explaining that our parent/owning `DefaultIndexingChain`
returns/uses it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1848: LUCENE-9515: Detach DWPT from DefaultIndexingChain
mikemccand commented on a change in pull request #1848:
URL: https://github.com/apache/lucene-solr/pull/1848#discussion_r485498630
##
File path:
lucene/core/src/java/org/apache/lucene/index/DefaultIndexingChain.java
##
@@ -73,21 +78,38 @@
// Holds fields seen in each document
private PerField[] fields = new PerField[1];
private final InfoStream infoStream;
-
- DefaultIndexingChain(DocumentsWriterPerThread docWriter) {
-this.docWriter = docWriter;
-this.fieldInfos = docWriter.getFieldInfosBuilder();
-this.infoStream = docWriter.getIndexWriterConfig().getInfoStream();
+ private final ByteBlockPool.Allocator byteBlockAllocator;
+ private final LiveIndexWriterConfig indexWriterConfig;
+ private final int indexCreatedVersionMajor;
+ private final Consumer abortingExceptionConsumer;
+ private boolean hasHitAbortingException = false;
Review comment:
Don't need the `= false` -- it is java's default already.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689467186 I'd also change the test name now to `TestStressLockFactories`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1848: LUCENE-9515: Detach DWPT from DefaultIndexingChain
mikemccand commented on a change in pull request #1848:
URL: https://github.com/apache/lucene-solr/pull/1848#discussion_r485499853
##
File path:
lucene/core/src/java/org/apache/lucene/index/DefaultIndexingChain.java
##
@@ -73,21 +78,38 @@
// Holds fields seen in each document
private PerField[] fields = new PerField[1];
private final InfoStream infoStream;
-
- DefaultIndexingChain(DocumentsWriterPerThread docWriter) {
-this.docWriter = docWriter;
-this.fieldInfos = docWriter.getFieldInfosBuilder();
-this.infoStream = docWriter.getIndexWriterConfig().getInfoStream();
+ private final ByteBlockPool.Allocator byteBlockAllocator;
+ private final LiveIndexWriterConfig indexWriterConfig;
+ private final int indexCreatedVersionMajor;
+ private final Consumer abortingExceptionConsumer;
+ private boolean hasHitAbortingException = false;
+
+ DefaultIndexingChain(int indexCreatedVersionMajor, SegmentInfo segmentInfo,
Directory directory, FieldInfos.Builder fieldInfos, LiveIndexWriterConfig
indexWriterConfig,
+ Consumer abortingExceptionConsumer) {
+this.indexCreatedVersionMajor = indexCreatedVersionMajor;
+byteBlockAllocator = new ByteBlockPool.DirectTrackingAllocator(bytesUsed);
+IntBlockPool.Allocator intBlockAllocator = new
IntBlockAllocator(bytesUsed);
+this.indexWriterConfig = indexWriterConfig;
+assert segmentInfo.getIndexSort() == indexWriterConfig.getIndexSort();
+this.fieldInfos = fieldInfos;
+this.infoStream = indexWriterConfig.getInfoStream();
+this.abortingExceptionConsumer = abortingExceptionConsumer;
final TermsHash termVectorsWriter;
-if (docWriter.getSegmentInfo().getIndexSort() == null) {
- storedFieldsConsumer = new StoredFieldsConsumer(docWriter.codec,
docWriter.directory, docWriter.getSegmentInfo());
- termVectorsWriter = new TermVectorsConsumer(docWriter);
+if (segmentInfo.getIndexSort() == null) {
+ storedFieldsConsumer = new
StoredFieldsConsumer(indexWriterConfig.getCodec(), directory, segmentInfo);
+ termVectorsWriter = new TermVectorsConsumer(intBlockAllocator,
byteBlockAllocator, directory, segmentInfo, indexWriterConfig.getCodec());
} else {
- storedFieldsConsumer = new SortingStoredFieldsConsumer(docWriter.codec,
docWriter.directory, docWriter.getSegmentInfo());
- termVectorsWriter = new SortingTermVectorsConsumer(docWriter);
+ storedFieldsConsumer = new
SortingStoredFieldsConsumer(indexWriterConfig.getCodec(), directory,
segmentInfo);
+ termVectorsWriter = new SortingTermVectorsConsumer(intBlockAllocator,
byteBlockAllocator, directory, segmentInfo, indexWriterConfig.getCodec());
}
-termsHash = new FreqProxTermsWriter(docWriter, bytesUsed,
termVectorsWriter);
+termsHash = new FreqProxTermsWriter(intBlockAllocator, byteBlockAllocator,
bytesUsed, termVectorsWriter);
+ }
+
+ private void onAbortingException(Throwable th) {
Review comment:
This was moved from `DWPT`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1848: LUCENE-9515: Detach DWPT from DefaultIndexingChain
mikemccand commented on a change in pull request #1848:
URL: https://github.com/apache/lucene-solr/pull/1848#discussion_r485499853
##
File path:
lucene/core/src/java/org/apache/lucene/index/DefaultIndexingChain.java
##
@@ -73,21 +78,38 @@
// Holds fields seen in each document
private PerField[] fields = new PerField[1];
private final InfoStream infoStream;
-
- DefaultIndexingChain(DocumentsWriterPerThread docWriter) {
-this.docWriter = docWriter;
-this.fieldInfos = docWriter.getFieldInfosBuilder();
-this.infoStream = docWriter.getIndexWriterConfig().getInfoStream();
+ private final ByteBlockPool.Allocator byteBlockAllocator;
+ private final LiveIndexWriterConfig indexWriterConfig;
+ private final int indexCreatedVersionMajor;
+ private final Consumer abortingExceptionConsumer;
+ private boolean hasHitAbortingException = false;
+
+ DefaultIndexingChain(int indexCreatedVersionMajor, SegmentInfo segmentInfo,
Directory directory, FieldInfos.Builder fieldInfos, LiveIndexWriterConfig
indexWriterConfig,
+ Consumer abortingExceptionConsumer) {
+this.indexCreatedVersionMajor = indexCreatedVersionMajor;
+byteBlockAllocator = new ByteBlockPool.DirectTrackingAllocator(bytesUsed);
+IntBlockPool.Allocator intBlockAllocator = new
IntBlockAllocator(bytesUsed);
+this.indexWriterConfig = indexWriterConfig;
+assert segmentInfo.getIndexSort() == indexWriterConfig.getIndexSort();
+this.fieldInfos = fieldInfos;
+this.infoStream = indexWriterConfig.getInfoStream();
+this.abortingExceptionConsumer = abortingExceptionConsumer;
final TermsHash termVectorsWriter;
-if (docWriter.getSegmentInfo().getIndexSort() == null) {
- storedFieldsConsumer = new StoredFieldsConsumer(docWriter.codec,
docWriter.directory, docWriter.getSegmentInfo());
- termVectorsWriter = new TermVectorsConsumer(docWriter);
+if (segmentInfo.getIndexSort() == null) {
+ storedFieldsConsumer = new
StoredFieldsConsumer(indexWriterConfig.getCodec(), directory, segmentInfo);
+ termVectorsWriter = new TermVectorsConsumer(intBlockAllocator,
byteBlockAllocator, directory, segmentInfo, indexWriterConfig.getCodec());
} else {
- storedFieldsConsumer = new SortingStoredFieldsConsumer(docWriter.codec,
docWriter.directory, docWriter.getSegmentInfo());
- termVectorsWriter = new SortingTermVectorsConsumer(docWriter);
+ storedFieldsConsumer = new
SortingStoredFieldsConsumer(indexWriterConfig.getCodec(), directory,
segmentInfo);
+ termVectorsWriter = new SortingTermVectorsConsumer(intBlockAllocator,
byteBlockAllocator, directory, segmentInfo, indexWriterConfig.getCodec());
}
-termsHash = new FreqProxTermsWriter(docWriter, bytesUsed,
termVectorsWriter);
+termsHash = new FreqProxTermsWriter(intBlockAllocator, byteBlockAllocator,
bytesUsed, termVectorsWriter);
+ }
+
+ private void onAbortingException(Throwable th) {
Review comment:
This was moved from `DWPT`?
Ahh, no, we forked it from there, ok.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689479947 Hi, I reordered the methods a bit in the class and applied some changes with number of clients: - Default is now 3. Actually this does not change the test runtime (on my machine) - On nightly we create 8 clients I would still like to get rid of the Supplier. IMHO, we should remove one randomization level. As we now have 3 clients (or 8), a randomization could always have a 2:1 chance to produce a forked client. What do you think, @dweiss ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689480950 FYI, the limit to 2 clients was caused by ANT's xml. Every client used another XML copypaste. But actually the number of clients is not really a performance factor. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
dweiss commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689486069 I don't mind. Reshape any way you like. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] iverase opened a new pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
iverase opened a new pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849 see https://issues.apache.org/jira/browse/LUCENE-9517 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] iverase commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
iverase commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689503221 A couple of notes: * Not sure how this change can be tested. * We know this is only an issue for JDK11, should be add a check and only perform privilege action in that case? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14843) Define strongly-typed cluster configuration API
[
https://issues.apache.org/jira/browse/SOLR-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192801#comment-17192801
]
Andrzej Bialecki commented on SOLR-14843:
-
{quote}This is not a user facing api
{quote}
Correct, the purpose of this Jira is to define an internal API. Although in
order to be useful as a read-write facade we will likely need a user-facing
CRUD API too.
I created SIP-11 to have a place where we outline the proposal in greater
detail.
> Define strongly-typed cluster configuration API
> ---
>
> Key: SOLR-14843
> URL: https://issues.apache.org/jira/browse/SOLR-14843
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Andrzej Bialecki
>Priority: Major
> Labels: clean-api
> Fix For: master (9.0)
>
>
> Current cluster-level configuration uses a hodgepodge of traditional Solr
> config sources (solr.xml, system properties) and the new somewhat arbitrary
> config files kept in ZK ({{/clusterprops.json, /security.json,
> /packages.json, /autoscaling.json}} etc...). There's no uniform
> strongly-typed API to access and manage these configs - currently each config
> source has its own CRUD, often relying on direct access to Zookeeper. There's
> also no uniform method for monitoring changes to these config sources.
> This issue proposes a uniform config API facade with the following
> characteristics:
> * Using a single hierarchical (or at least key-based) facade for accessing
> any global config.
> * Using strongly-typed sub-system configs instead of opaque Map-s:
> components would no longer deal with JSON parsing/writing, instead they would
> use properly annotated Java objects for config CRUD. Config objects would
> include versioning information (eg. lastModified timestamp).
> * Isolating access to the underlying config persistence layer: components
> would no longer directly interact with Zookeeper or files. Most likely the
> default implementation would continue using different ZK files per-subsystem
> in order to limit the complexity of file formats and to reduce the cost of
> notifications for unmodified parts of the configs.
> * Providing uniform way to register listeners for monitoring changes in
> specific configs: components would no longer need to interact with ZK
> watches, they would instead be notified about modified configs that they are
> interested in.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
dweiss commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689507031 This is a hideous implementation detail inside the JDK but I think the privileged action should just be there - I think this is the correct way to dodge the problem. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke merged pull request #1790: Rename TestDirectoryFactory to DirectoryFactoriesTest
cpoerschke merged pull request #1790: URL: https://github.com/apache/lucene-solr/pull/1790 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke merged pull request #1832: SOLR-14831: remove deprecated-and-unused "facet.distrib.mco" constant
cpoerschke merged pull request #1832: URL: https://github.com/apache/lucene-solr/pull/1832 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14831) remove deprecated-and-unused "facet.distrib.mco" constant
[
https://issues.apache.org/jira/browse/SOLR-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192814#comment-17192814
]
ASF subversion and git services commented on SOLR-14831:
Commit 04def9449261d78038e8c34662300791a372bfaa in lucene-solr's branch
refs/heads/master from Christine Poerschke
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=04def94 ]
SOLR-14831: remove deprecated-and-unused "facet.distrib.mco" constant in
FacetParams.java (#1832)
> remove deprecated-and-unused "facet.distrib.mco" constant
> -
>
> Key: SOLR-14831
> URL: https://issues.apache.org/jira/browse/SOLR-14831
> Project: Solr
> Issue Type: Task
> Components: SolrJ
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is ready for removal e.g. as per the
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.2/solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java#L139-L144
> comment:
> {code}
>* @deprecated
>* This option is no longer used nor will if affect any queries as the fix
> has been built in. (SOLR-11711)
>* This will be removed entirely in 8.0.0
>*/
> @Deprecated
> public static final String FACET_DISTRIB_MCO = FACET_DISTRIB + ".mco";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14831) remove deprecated-and-unused "facet.distrib.mco" constant
[
https://issues.apache.org/jira/browse/SOLR-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192818#comment-17192818
]
ASF subversion and git services commented on SOLR-14831:
Commit b40d6e0509c2352ec6cc44637988a937f070c33a in lucene-solr's branch
refs/heads/branch_8x from Christine Poerschke
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b40d6e0 ]
SOLR-14831: remove deprecated-and-unused "facet.distrib.mco" constant in
FacetParams.java (#1832)
> remove deprecated-and-unused "facet.distrib.mco" constant
> -
>
> Key: SOLR-14831
> URL: https://issues.apache.org/jira/browse/SOLR-14831
> Project: Solr
> Issue Type: Task
> Components: SolrJ
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This is ready for removal e.g. as per the
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.2/solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java#L139-L144
> comment:
> {code}
>* @deprecated
>* This option is no longer used nor will if affect any queries as the fix
> has been built in. (SOLR-11711)
>* This will be removed entirely in 8.0.0
>*/
> @Deprecated
> public static final String FACET_DISTRIB_MCO = FACET_DISTRIB + ".mco";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14831) remove deprecated-and-unused "facet.distrib.mco" constant
[
https://issues.apache.org/jira/browse/SOLR-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192820#comment-17192820
]
Christine Poerschke commented on SOLR-14831:
The {{FacetParams.FACET_DISTRIB}} constant is now unused then too. I'll go mark
it as deprecated in branch_8x and remove it on master branch only.
> remove deprecated-and-unused "facet.distrib.mco" constant
> -
>
> Key: SOLR-14831
> URL: https://issues.apache.org/jira/browse/SOLR-14831
> Project: Solr
> Issue Type: Task
> Components: SolrJ
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This is ready for removal e.g. as per the
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.2/solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java#L139-L144
> comment:
> {code}
>* @deprecated
>* This option is no longer used nor will if affect any queries as the fix
> has been built in. (SOLR-11711)
>* This will be removed entirely in 8.0.0
>*/
> @Deprecated
> public static final String FACET_DISTRIB_MCO = FACET_DISTRIB + ".mco";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14831) remove deprecated-and-unused "facet.distrib.mco" constant
[
https://issues.apache.org/jira/browse/SOLR-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192829#comment-17192829
]
ASF subversion and git services commented on SOLR-14831:
Commit 12dab0f80e76e618c5a086b3664323d85dd0ce00 in lucene-solr's branch
refs/heads/master from Christine Poerschke
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=12dab0f ]
SOLR-14831: remove deprecated FacetParams.FACET_DISTRIB constant
> remove deprecated-and-unused "facet.distrib.mco" constant
> -
>
> Key: SOLR-14831
> URL: https://issues.apache.org/jira/browse/SOLR-14831
> Project: Solr
> Issue Type: Task
> Components: SolrJ
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This is ready for removal e.g. as per the
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.2/solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java#L139-L144
> comment:
> {code}
>* @deprecated
>* This option is no longer used nor will if affect any queries as the fix
> has been built in. (SOLR-11711)
>* This will be removed entirely in 8.0.0
>*/
> @Deprecated
> public static final String FACET_DISTRIB_MCO = FACET_DISTRIB + ".mco";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14831) remove deprecated-and-unused "facet.distrib.mco" constant
[
https://issues.apache.org/jira/browse/SOLR-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192828#comment-17192828
]
ASF subversion and git services commented on SOLR-14831:
Commit 4716a0b2fa917365d5c8d7d1d29cda902a9d45a7 in lucene-solr's branch
refs/heads/master from Christine Poerschke
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4716a0b ]
SOLR-14831: mark FacetParams.FACET_DISTRIB as deprecated
> remove deprecated-and-unused "facet.distrib.mco" constant
> -
>
> Key: SOLR-14831
> URL: https://issues.apache.org/jira/browse/SOLR-14831
> Project: Solr
> Issue Type: Task
> Components: SolrJ
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This is ready for removal e.g. as per the
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.2/solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java#L139-L144
> comment:
> {code}
>* @deprecated
>* This option is no longer used nor will if affect any queries as the fix
> has been built in. (SOLR-11711)
>* This will be removed entirely in 8.0.0
>*/
> @Deprecated
> public static final String FACET_DISTRIB_MCO = FACET_DISTRIB + ".mco";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14831) remove deprecated-and-unused "facet.distrib.mco" constant
[
https://issues.apache.org/jira/browse/SOLR-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192831#comment-17192831
]
ASF subversion and git services commented on SOLR-14831:
Commit e07816ed110959009ed728b6bc30d202bf24d807 in lucene-solr's branch
refs/heads/branch_8x from Christine Poerschke
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e07816e ]
SOLR-14831: mark FacetParams.FACET_DISTRIB as deprecated
> remove deprecated-and-unused "facet.distrib.mco" constant
> -
>
> Key: SOLR-14831
> URL: https://issues.apache.org/jira/browse/SOLR-14831
> Project: Solr
> Issue Type: Task
> Components: SolrJ
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This is ready for removal e.g. as per the
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.2/solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java#L139-L144
> comment:
> {code}
>* @deprecated
>* This option is no longer used nor will if affect any queries as the fix
> has been built in. (SOLR-11711)
>* This will be removed entirely in 8.0.0
>*/
> @Deprecated
> public static final String FACET_DISTRIB_MCO = FACET_DISTRIB + ".mco";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14831) remove deprecated-and-unused "facet.distrib.mco" constant
[
https://issues.apache.org/jira/browse/SOLR-14831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke resolved SOLR-14831.
Fix Version/s: 8.7
master (9.0)
Resolution: Fixed
> remove deprecated-and-unused "facet.distrib.mco" constant
> -
>
> Key: SOLR-14831
> URL: https://issues.apache.org/jira/browse/SOLR-14831
> Project: Solr
> Issue Type: Task
> Components: SolrJ
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Fix For: master (9.0), 8.7
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This is ready for removal e.g. as per the
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.2/solr/solrj/src/java/org/apache/solr/common/params/FacetParams.java#L139-L144
> comment:
> {code}
>* @deprecated
>* This option is no longer used nor will if affect any queries as the fix
> has been built in. (SOLR-11711)
>* This will be removed entirely in 8.0.0
>*/
> @Deprecated
> public static final String FACET_DISTRIB_MCO = FACET_DISTRIB + ".mco";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9484) Allow index sorting to happen after the fact
[ https://issues.apache.org/jira/browse/LUCENE-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Poerschke updated LUCENE-9484: Fix Version/s: 8.7 master (9.0) > Allow index sorting to happen after the fact > > > Key: LUCENE-9484 > URL: https://issues.apache.org/jira/browse/LUCENE-9484 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Simon Willnauer >Priority: Major > Fix For: master (9.0), 8.7 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > I did look into sorting an index after it was created and found that with > some smallish modifications we can actually allow that by piggibacking on > SortingLeafReader and addIndices in a pretty straight-forward and simple way. > With some smallish modifications / fixes to SortingLeafReader we can just > merge and unsorted index into a sorted index using a fresh index writer. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689546656 Hi, I refactored the code further and removed the randomization: - If uses `(clientId % 4)==0` to produce a thread client and otherwise a process client - In default mode, it creates 3 clients, so one of them is always in-process - In nightly mode, it creates 8 clients, so two of them are in-process I think that covers all modes. I think it's ready to commit. Question: Also backport to 8.x and remove the additional ANT task? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14843) Define strongly-typed cluster configuration API
[
https://issues.apache.org/jira/browse/SOLR-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192855#comment-17192855
]
Jan Høydahl commented on SOLR-14843:
[~dsmiley] Feel free to adapt the SIP page instructinos to fit our needs. It
should probably mention that a VOTE is optional but should be held if the
dev-list discussion is clearly not conclusive. IMO a [VOTE] thread is
light-weight, and a successful VOTE is a good foundation to start a huge work
effort from, and likewise a failed VOTE will get you back to the design phase
instead of wasting days of effort which ends up in a -1 on a Jira or worse a
revert further down the road.
> Define strongly-typed cluster configuration API
> ---
>
> Key: SOLR-14843
> URL: https://issues.apache.org/jira/browse/SOLR-14843
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Andrzej Bialecki
>Priority: Major
> Labels: clean-api
> Fix For: master (9.0)
>
>
> Current cluster-level configuration uses a hodgepodge of traditional Solr
> config sources (solr.xml, system properties) and the new somewhat arbitrary
> config files kept in ZK ({{/clusterprops.json, /security.json,
> /packages.json, /autoscaling.json}} etc...). There's no uniform
> strongly-typed API to access and manage these configs - currently each config
> source has its own CRUD, often relying on direct access to Zookeeper. There's
> also no uniform method for monitoring changes to these config sources.
> This issue proposes a uniform config API facade with the following
> characteristics:
> * Using a single hierarchical (or at least key-based) facade for accessing
> any global config.
> * Using strongly-typed sub-system configs instead of opaque Map-s:
> components would no longer deal with JSON parsing/writing, instead they would
> use properly annotated Java objects for config CRUD. Config objects would
> include versioning information (eg. lastModified timestamp).
> * Isolating access to the underlying config persistence layer: components
> would no longer directly interact with Zookeeper or files. Most likely the
> default implementation would continue using different ZK files per-subsystem
> in order to limit the complexity of file formats and to reduce the cost of
> notifications for unmodified parts of the configs.
> * Providing uniform way to register listeners for monitoring changes in
> specific configs: components would no longer need to interact with ZK
> watches, they would instead be notified about modified configs that they are
> interested in.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14701) Deprecate Schemaless Mode (Discussion)
[ https://issues.apache.org/jira/browse/SOLR-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192858#comment-17192858 ] Alexandre Rafalovitch commented on SOLR-14701: -- So, I am doing some quick review of actual code/configuration to better understand the issue. And a couple of questions come up: # Type widening. Like when the first pass thinks it is a plong and then it turns out to be a double. Do we actually need a multi-level type widening or just fall-back to default is sufficient if there is mismatch (plong or default, pdouble or default, pdate or default). The only example for multi-level is plong->pdouble->string if I am not mistaken. # Single/Multi-valued. Currently we only support multiValue. If we are tracking, can the field definitions be single-valued (plong, pdouble) and then actual fields declare multiValued? What is the value of having double field type definition, I find it quite confusing and hard to track actual multiplicity. # Do we only look at new fields or do we compare with existing schema definitions and note the mismatch by name/mulltiplicity? # Do we have to split out Managed Schema JSON? Or can we just do structured output of name:type:multiplicity and let users compose the commands themselves? Or have the output format and add more formats later. Because maybe they want to hand-edit the schema or (as it will likely to be) the learning schema with this URP chain is not actually the production schema. Or because we are detecting mismatches in point above and the precise JSON can get messy. Either way, the JSON command generation needs to be in an independent tool I am guessing to be reusable. Which is nice, but is it in scope? My own answers seem to be that the right approach seems to be: * Widen to default * Use single-value FieldTypes and only assign multiValued on fields themselves * Look at only new fields (for now), especially since the learning schema may not be matching production one * Output structured advice and leave room to do JSON generation for future Jira Bonus questions: * Do the field type definitions actually need to exist in schema if we never create the real fields that use them (we check right now). Or does this just become an abstract parsing and mapping exercise? * Do we still need copyField instructions if we are not creating the fields ourselves? Other notes to self: * We could probably track rough cardinality as well and/or field size for string (e.g. keep first 10 values/hashes and check length) and give recommendation based on that (for Enum, Faceting, Large field attribute, Lazy Loading in solrconfig.xml, etc). That's again based on focusing on advice, rather than pure schema generation. > Deprecate Schemaless Mode (Discussion) > -- > > Key: SOLR-14701 > URL: https://issues.apache.org/jira/browse/SOLR-14701 > Project: Solr > Issue Type: Improvement > Components: Schema and Analysis >Reporter: Marcus Eagan >Priority: Major > Attachments: image-2020-08-04-01-35-03-075.png > > > I know this won't be the most popular ticket out there, but I am growing more > and more sympathetic to the idea that we should rip many of the freedoms out > that cause users more harm than not. One of the freedoms I saw time and time > again to cause issues was schemaless mode. It doesn't work as named or > documented, so I think it should be deprecated. > If you use it in production reliably and in a way that cannot be accomplished > another way, I am happy to hear from more knowledgeable folks as to why > deprecation is a bad idea. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler edited a comment on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler edited a comment on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689546656 Hi, *correction* Actually we need at least two in process clients, otherwise it's not really testing anything with in-process (explanation: the in-porcess client is also "external" to the spawned process; to actually test that two lock factories in same process work, you need 2 local ones. My misunderstanding wa sthat the LockVerifyServer does not take place in locking). I refactored the code further and removed the randomization: - If uses `(clientId % 2)==0` to produce a thread client and otherwise a process client - In default mode, it creates 4 clients, so two of them is always in-process - In nightly mode, it creates 12 clients, so six of them are in-process I think that covers all modes. I think it's ready to commit. Question: Also backport to 8.x and remove the additional ANT task? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler edited a comment on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler edited a comment on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689546656 Hi, *correction* Actually we need at least two in process clients, otherwise it's not really testing anything with in-process (explanation: the in-porcess client is also "external" to the spawned process; to actually test that two lock factories in same process work, you need 2 local ones. My misunderstanding was that the LockVerifyServer does not take place in locking). I refactored the code further and removed the randomization: - If uses `(clientId % 2)==0` to produce a thread client and otherwise a process client - In default mode, it creates 4 clients, so two of them is always in-process - In nightly mode, it creates 12 clients, so six of them are in-process I think that covers all modes. I think it's ready to commit. Question: Also backport to 8.x and remove the additional ANT task? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler edited a comment on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler edited a comment on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689562340 Hi again, after reading the code of the LockStressTest: It tries to re-obtain a lock in the same JVM based on randomization. So actually the original test did everything right and we were also testing "smae JVM case". @rmuir Can you verify this? So IMHO, we can remove the inprocess stuff and rever back to the state from this morning. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689562340 Hi again, after reading the code of the LockStressTest: It tries to re-obtain a lock in the same JVM based on randomization. So actually the original test did everything right. @rmuir Can you verify this? So IMHO, we can remove the inprocess stuff and rever back to the state from this morning. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842:
URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689563876
Code is here:
```java
try (final Lock l = verifyLF.obtainLock(lockDir, LOCK_FILE_NAME)) {
if (rnd.nextInt(10) == 0) {
if (rnd.nextBoolean()) {
verifyLF = new
VerifyingLockFactory(getNewLockFactory(lockFactoryClassName), in, out);
}
try (final Lock secondLock = verifyLF.obtainLock(lockDir,
LOCK_FILE_NAME)) {
throw new IOException("Double obtain");
} catch (LockObtainFailedException loe) {
// pass
}
}
Thread.sleep(sleepTimeMS);
} catch (LockObtainFailedException loe) {
// obtain failed
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler edited a comment on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler edited a comment on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689562340 Hi again, after reading the code of the LockStressTest: It tries to re-obtain a lock in the same JVM based on randomization. So actually the original test did everything right and we were also testing "same JVM case" since long time. @rmuir Can you verify this? So IMHO, we can remove the inprocess stuff and revert back to the state from this morning. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689574638 I was talking with Robert. Let's revert the in-process stuff. Makes not much difference. We actually have a very good test for in-process logs, part of BaseLockFactoryTestCase. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] s1monw commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
s1monw commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689577005 change LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14701) Deprecate Schemaless Mode (Discussion)
[ https://issues.apache.org/jira/browse/SOLR-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192884#comment-17192884 ] Jan Høydahl commented on SOLR-14701: Thanks for doing this Alexandre! bq. * Widen to default Sure, but we should support int->float->string bq. * Use single-value FieldTypes and only assign multiValued on fields themselves Agree bq. * Look at only new fields (for now), especially since the learning schema may not be matching production one Yep bq. * Output structured advice and leave room to do JSON generation for future Jira I'd say update schema at the end. But I'm not fully clear on how you intend this tool to be used? Is it a standalone cmd tool, or just an update-chain you specify on http POST to your existing collection? Or an implicit handler /update/learning that has this update-chain preconfigured? bq. Do the field type definitions actually need to exist in schema if we never create the real fields that use them (we check right now). Or does this just become an abstract parsing and mapping exercise? I have earlier proposed that the major field types be made implicit and need not be part of schema. I think I proposed a new tag or something, which would bring 'int', 'long' etc with it. That would let you do it explicitly if you don't include that tag... Wrt the choice between analyzed text or string, it is impossible to know just by looking at text length or content whether the intention is to search as analyzed text or use for soring/faceting or matching. Likewise whether docvalues are needed, vectors, norms etc. I still think the approach from the AddFieldsToSchema processor approach makes sense, to index as text and string (capped at 256 and with _str suffix) as a default you know will be there. > Deprecate Schemaless Mode (Discussion) > -- > > Key: SOLR-14701 > URL: https://issues.apache.org/jira/browse/SOLR-14701 > Project: Solr > Issue Type: Improvement > Components: Schema and Analysis >Reporter: Marcus Eagan >Priority: Major > Attachments: image-2020-08-04-01-35-03-075.png > > > I know this won't be the most popular ticket out there, but I am growing more > and more sympathetic to the idea that we should rip many of the freedoms out > that cause users more harm than not. One of the freedoms I saw time and time > again to cause issues was schemaless mode. It doesn't work as named or > documented, so I think it should be deprecated. > If you use it in production reliably and in a way that cannot be accomplished > another way, I am happy to hear from more knowledgeable folks as to why > deprecation is a bad idea. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689584918 OK, reverted most of the changes in the stress tester. I kept the changes done by @dweiss about the magic numbers like starting gun "43" or access/release in the wire protocol. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9470) TestXYMultiPolygonShapeQueries reproducing failure
[
https://issues.apache.org/jira/browse/LUCENE-9470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192887#comment-17192887
]
ASF subversion and git services commented on LUCENE-9470:
-
Commit 7da15706da6a117ca56fdef36a2e5c3a9a714cd3 in lucene-solr's branch
refs/heads/master from Ignacio Vera
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7da1570 ]
LUCENE-9470: make TestXYMultiPolygonShapeQueries more resilient for CONTAINS
queries (#1776)
> TestXYMultiPolygonShapeQueries reproducing failure
> --
>
> Key: LUCENE-9470
> URL: https://issues.apache.org/jira/browse/LUCENE-9470
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: master (9.0)
>Reporter: Michael McCandless
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I hit this while beasting tests against [~simonw]'s PR to add "merge segments
> during getReader":
> It reproduces for me on current
> ({{77a4d495cc553ec80001346376fd87d6b73a6059}}) mainline:
> {noformat}
> [junit4:pickseed] Seed property 'tests.seed' already defined:
> 50C3891F4E0641A4
> [junit4] says g'day! Master seed: 50C3891F4E0641A4
> [junit4] Executing 1 suite with 1 JVM.
> [junit4]
> [junit4] Started J0 PID(1993475@localhost).
> [junit4] Suite: org.apache.lucene.document.TestXYMultiPolygonShapeQueries
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestXYMultiPolygonShapeQueries -Dtests.method=testRandomBig
> -Dtests.seed=50C3891F4E0641A4 -Dtests.nightly=true -Dtests.slow=true
> -Dtests.badap\
> ples=true
> -Dtests.linedocsfile=/l/simon/lucene/test-framework/src/resources/org/apache/lucene/util/2000mb.txt.gz
> -Dtests.locale=en-IM -Dtests.timezone=America/Lima -Dtests.asserts=true
> -Dtests.file.enco\
> ding=UTF-8
> [junit4] FAILURE 51.3s | TestXYMultiPolygonShapeQueries.testRandomBig <<<
> [junit4] > Throwable #1: java.lang.AssertionError: wrong hit (first of
> possibly more):
> [junit4] > FAIL: id=8233 should match but did not
> [junit4] > relation=CONTAINS
> [junit4] > query=XYShapeQuery: field=shape:[XYLINE([3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38, 3.4028235E38][3.40282\
> 3E38, 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.40282\
> 35E38][3.402823E38, 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4\
> 028229E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229\
> E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3\
> .4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.40282\
> 35E38][3.4028229E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028233E38][3.4028229E38, 3.4028233E38]\
> [3.4028229E38, 3.4028233E38][3.4028229E38, 3.4028233E38][3.4028229E38,
> 3.4028233E38][3.4028229E38, 3.4028233E38][3.4028229E38,
> 3.4028233E38][3.4028229E38, 3.4028233E38][3.4028229E38, 3.4028233E38][3.402\
> 8227E38, 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E3\
> 8, 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4\
> 028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233\
> ...
> 3E38][3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][\
> 3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.402
[GitHub] [lucene-solr] iverase merged pull request #1776: LUCENE-9470: make TestXYMultiPolygonShapeQueries more resilient for CONTAINS queries
iverase merged pull request #1776: URL: https://github.com/apache/lucene-solr/pull/1776 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9470) TestXYMultiPolygonShapeQueries reproducing failure
[
https://issues.apache.org/jira/browse/LUCENE-9470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192888#comment-17192888
]
ASF subversion and git services commented on LUCENE-9470:
-
Commit 8ac53aa6bc21d460b69866f64dd24f65f304ced6 in lucene-solr's branch
refs/heads/branch_8x from Ignacio Vera
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8ac53aa ]
LUCENE-9470: make TestXYMultiPolygonShapeQueries more resilient for CONTAINS
queries (#1776)
> TestXYMultiPolygonShapeQueries reproducing failure
> --
>
> Key: LUCENE-9470
> URL: https://issues.apache.org/jira/browse/LUCENE-9470
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: master (9.0)
>Reporter: Michael McCandless
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I hit this while beasting tests against [~simonw]'s PR to add "merge segments
> during getReader":
> It reproduces for me on current
> ({{77a4d495cc553ec80001346376fd87d6b73a6059}}) mainline:
> {noformat}
> [junit4:pickseed] Seed property 'tests.seed' already defined:
> 50C3891F4E0641A4
> [junit4] says g'day! Master seed: 50C3891F4E0641A4
> [junit4] Executing 1 suite with 1 JVM.
> [junit4]
> [junit4] Started J0 PID(1993475@localhost).
> [junit4] Suite: org.apache.lucene.document.TestXYMultiPolygonShapeQueries
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestXYMultiPolygonShapeQueries -Dtests.method=testRandomBig
> -Dtests.seed=50C3891F4E0641A4 -Dtests.nightly=true -Dtests.slow=true
> -Dtests.badap\
> ples=true
> -Dtests.linedocsfile=/l/simon/lucene/test-framework/src/resources/org/apache/lucene/util/2000mb.txt.gz
> -Dtests.locale=en-IM -Dtests.timezone=America/Lima -Dtests.asserts=true
> -Dtests.file.enco\
> ding=UTF-8
> [junit4] FAILURE 51.3s | TestXYMultiPolygonShapeQueries.testRandomBig <<<
> [junit4] > Throwable #1: java.lang.AssertionError: wrong hit (first of
> possibly more):
> [junit4] > FAIL: id=8233 should match but did not
> [junit4] > relation=CONTAINS
> [junit4] > query=XYShapeQuery: field=shape:[XYLINE([3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38, 3.4028235E38][3.40282\
> 3E38, 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.40282\
> 35E38][3.402823E38, 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4\
> 028229E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229\
> E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3\
> .4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.40282\
> 35E38][3.4028229E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028233E38][3.4028229E38, 3.4028233E38]\
> [3.4028229E38, 3.4028233E38][3.4028229E38, 3.4028233E38][3.4028229E38,
> 3.4028233E38][3.4028229E38, 3.4028233E38][3.4028229E38,
> 3.4028233E38][3.4028229E38, 3.4028233E38][3.4028229E38, 3.4028233E38][3.402\
> 8227E38, 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E3\
> 8, 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4\
> 028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233\
> ...
> 3E38][3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][\
> 3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.
[jira] [Resolved] (LUCENE-9470) TestXYMultiPolygonShapeQueries reproducing failure
[
https://issues.apache.org/jira/browse/LUCENE-9470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ignacio Vera resolved LUCENE-9470.
--
Fix Version/s: 8.7
Assignee: Ignacio Vera
Resolution: Fixed
> TestXYMultiPolygonShapeQueries reproducing failure
> --
>
> Key: LUCENE-9470
> URL: https://issues.apache.org/jira/browse/LUCENE-9470
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: master (9.0)
>Reporter: Michael McCandless
>Assignee: Ignacio Vera
>Priority: Major
> Fix For: 8.7
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I hit this while beasting tests against [~simonw]'s PR to add "merge segments
> during getReader":
> It reproduces for me on current
> ({{77a4d495cc553ec80001346376fd87d6b73a6059}}) mainline:
> {noformat}
> [junit4:pickseed] Seed property 'tests.seed' already defined:
> 50C3891F4E0641A4
> [junit4] says g'day! Master seed: 50C3891F4E0641A4
> [junit4] Executing 1 suite with 1 JVM.
> [junit4]
> [junit4] Started J0 PID(1993475@localhost).
> [junit4] Suite: org.apache.lucene.document.TestXYMultiPolygonShapeQueries
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestXYMultiPolygonShapeQueries -Dtests.method=testRandomBig
> -Dtests.seed=50C3891F4E0641A4 -Dtests.nightly=true -Dtests.slow=true
> -Dtests.badap\
> ples=true
> -Dtests.linedocsfile=/l/simon/lucene/test-framework/src/resources/org/apache/lucene/util/2000mb.txt.gz
> -Dtests.locale=en-IM -Dtests.timezone=America/Lima -Dtests.asserts=true
> -Dtests.file.enco\
> ding=UTF-8
> [junit4] FAILURE 51.3s | TestXYMultiPolygonShapeQueries.testRandomBig <<<
> [junit4] > Throwable #1: java.lang.AssertionError: wrong hit (first of
> possibly more):
> [junit4] > FAIL: id=8233 should match but did not
> [junit4] > relation=CONTAINS
> [junit4] > query=XYShapeQuery: field=shape:[XYLINE([3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38, 3.4028235E38][3.40282\
> 3E38, 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.40282\
> 35E38][3.402823E38, 3.4028235E38][3.402823E38, 3.4028235E38][3.402823E38,
> 3.4028235E38][3.402823E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4\
> 028229E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229\
> E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3\
> .4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.40282\
> 35E38][3.4028229E38, 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028235E38][3.4028229E38,
> 3.4028235E38][3.4028229E38, 3.4028233E38][3.4028229E38, 3.4028233E38]\
> [3.4028229E38, 3.4028233E38][3.4028229E38, 3.4028233E38][3.4028229E38,
> 3.4028233E38][3.4028229E38, 3.4028233E38][3.4028229E38,
> 3.4028233E38][3.4028229E38, 3.4028233E38][3.4028229E38, 3.4028233E38][3.402\
> 8227E38, 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E3\
> 8, 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4\
> 028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233E38][3.4028227E38,
> 3.4028233E38][3.4028227E38, 3.4028233\
> ...
> 3E38][3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][\
> 3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028\
> 235E38, 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.4028233E38][3.4028235E38, 3.4028233E38][3.4028235E38,
> 3.40
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1848: LUCENE-9515: Detach DWPT from DefaultIndexingChain
mikemccand commented on a change in pull request #1848:
URL: https://github.com/apache/lucene-solr/pull/1848#discussion_r485644701
##
File path: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
##
@@ -272,7 +273,9 @@ static int getActualMaxDocs() {
* and a message is printed to infoStream, if set (see {@link
* IndexWriterConfig#setInfoStream(InfoStream)}).
*/
- public final static int MAX_TERM_LENGTH =
DocumentsWriterPerThread.MAX_TERM_LENGTH_UTF8;
+ // if you increase this, you must fix field cache impl for
+ // getTerms/getTermsIndex requires <= 32768
Review comment:
Maybe remove this stale comment :) FieldCache is long gone!
##
File path: lucene/core/src/test/org/apache/lucene/index/TestIndexWriter.java
##
@@ -1533,7 +1533,7 @@ public void testWickedLongTerm() throws IOException {
Directory dir = newDirectory();
RandomIndexWriter w = new RandomIndexWriter(random(), dir, new
StringSplitAnalyzer());
-char[] chars = new char[DocumentsWriterPerThread.MAX_TERM_LENGTH_UTF8];
+char[] chars = new char[IndexWriter.MAX_TERM_LENGTH];
Review comment:
Ooh good catch (`char[]` vs `byte[]`).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9516) Remove DocConsumer and IndexingChain from Lucene
[ https://issues.apache.org/jira/browse/LUCENE-9516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192896#comment-17192896 ] Michael McCandless commented on LUCENE-9516: +1 > Remove DocConsumer and IndexingChain from Lucene > > > Key: LUCENE-9516 > URL: https://issues.apache.org/jira/browse/LUCENE-9516 > Project: Lucene - Core > Issue Type: Wish >Reporter: Simon Willnauer >Priority: Major > > Disclaimer: This is a breaking change! > We allow today to replace the entire indexing chain which is a fundamental > part of our software. I personally don't know if there are any users of this > API. but given the complexity I personally don't think we should further > support this. If you are willing to implement this entire thing yourself I > really wonder if you are better off building lucene from the source. An > option like this on IWC might make users look into it while I am convinced > they shouldn't. It's too complex and nothing is made for reuse down there. I > wonder what others think but from my perspective it's time to remove it in 9.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob merged pull request #1843: SOLR-14846 Remove Optional.ofNullable.orElse pattern
madrob merged pull request #1843: URL: https://github.com/apache/lucene-solr/pull/1843 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9513) Use seconds instead of millisecs
[ https://issues.apache.org/jira/browse/LUCENE-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192898#comment-17192898 ] Michael McCandless commented on LUCENE-9513: Thanks [~frankmanzhu] – fix looks great; I'll push soon. > Use seconds instead of millisecs > > > Key: LUCENE-9513 > URL: https://issues.apache.org/jira/browse/LUCENE-9513 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Affects Versions: master (9.0), 8.6.2 >Reporter: Frank Zhu >Priority: Trivial > Fix For: master (9.0) > > > In RangeFacetsExample, there is a trivial mistake where > private final long nowSec is supposed to be secs. > But it uses as millisecs. > It doesn't affect the output due to the scale change (It will just make > PAST_HOUR, PAST_SIX_HOURS not as intended). Still it's better to be correct. > > I've created a pull request on github with the simple fix. > https://github.com/apache/lucene-solr/pull/1846 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1846: LUCENE-9513 Use seconds instead of millisecs
mikemccand commented on a change in pull request #1846: URL: https://github.com/apache/lucene-solr/pull/1846#discussion_r485650485 ## File path: lucene/demo/src/java/org/apache/lucene/demo/facet/RangeFacetsExample.java ## @@ -46,7 +46,7 @@ private final Directory indexDir = new ByteBuffersDirectory(); private IndexSearcher searcher; - private final long nowSec = System.currentTimeMillis(); + private final long nowSec = System.currentTimeMillis()/1000L; Review comment: Good catch! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14846) Avoid use of Optional.ofNullable.orElse in the same line
[
https://issues.apache.org/jira/browse/SOLR-14846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192899#comment-17192899
]
ASF subversion and git services commented on SOLR-14846:
Commit c902837bb237e579f9fdd0db0a8b35680b294ad1 in lucene-solr's branch
refs/heads/master from Mike Drob
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c902837 ]
SOLR-14846 Clean up Optional use (#1843)
* Remove Optional.ofNullable.orElse pattern
* Remove use of Optional as method parameter
> Avoid use of Optional.ofNullable.orElse in the same line
>
>
> Key: SOLR-14846
> URL: https://issues.apache.org/jira/browse/SOLR-14846
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Minor
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Optional was meant to be used in public APIs to convey to the caller that
> they really need to null-check, not as an internal class to avoid doing our
> own null checks. There are better methods in {{Objects}} for that if we want
> them.
> See also Brian Goetz at JavaOne - https://youtu.be/MFzlgAuanU0?t=2719
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9517) BugfixDeflater_JDK8252739 causes Java security issues in JDk11
[
https://issues.apache.org/jira/browse/LUCENE-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192902#comment-17192902
]
Uwe Schindler commented on LUCENE-9517:
---
Sorry, I don't understand the issue.
Lucene's code does not do any reflection, so what's your issue exactly?
Uwe
> BugfixDeflater_JDK8252739 causes Java security issues in JDk11
> --
>
> Key: LUCENE-9517
> URL: https://issues.apache.org/jira/browse/LUCENE-9517
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Ignacio Vera
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We are running into issues when running Elasticsearch CI with java security
> turned on and using JDK11 (only for the ones that contains the jdk bug ).
> The errors look like:
>
>
> {code:java}
> java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessDeclaredMembers") {code}
>
> The issue seems to be here:
> [http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l989]
> As we now have a subclass that wants to run this code. Note that this code
> has been removed in JDK12 and above.
> We might need to wrap the creation of this object in a doPriviledged Block or
> find a different solution that does not need to subclass the Deflater class.
>
> cc: [~uschindler]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9517) BugfixDeflater_JDK8252739 causes Java security issues in JDk11
[
https://issues.apache.org/jira/browse/LUCENE-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192905#comment-17192905
]
Uwe Schindler commented on LUCENE-9517:
---
This is more a bug in this jdk code. So please report it there.
Lucene is not the only code that subclasses Deflater. Many code is doing this.
> BugfixDeflater_JDK8252739 causes Java security issues in JDk11
> --
>
> Key: LUCENE-9517
> URL: https://issues.apache.org/jira/browse/LUCENE-9517
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Ignacio Vera
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We are running into issues when running Elasticsearch CI with java security
> turned on and using JDK11 (only for the ones that contains the jdk bug ).
> The errors look like:
>
>
> {code:java}
> java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessDeclaredMembers") {code}
>
> The issue seems to be here:
> [http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l989]
> As we now have a subclass that wants to run this code. Note that this code
> has been removed in JDK12 and above.
> We might need to wrap the creation of this object in a doPriviledged Block or
> find a different solution that does not need to subclass the Deflater class.
>
> cc: [~uschindler]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9517) BugfixDeflater_JDK8252739 causes Java security issues in JDk11
[
https://issues.apache.org/jira/browse/LUCENE-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192906#comment-17192906
]
Uwe Schindler commented on LUCENE-9517:
---
Sorry, this is bullshit.
Please don't do this. Fix your CI.
> BugfixDeflater_JDK8252739 causes Java security issues in JDk11
> --
>
> Key: LUCENE-9517
> URL: https://issues.apache.org/jira/browse/LUCENE-9517
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Ignacio Vera
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We are running into issues when running Elasticsearch CI with java security
> turned on and using JDK11 (only for the ones that contains the jdk bug ).
> The errors look like:
>
>
> {code:java}
> java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessDeclaredMembers") {code}
>
> The issue seems to be here:
> [http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l989]
> As we now have a subclass that wants to run this code. Note that this code
> has been removed in JDK12 and above.
> We might need to wrap the creation of this object in a doPriviledged Block or
> find a different solution that does not need to subclass the Deflater class.
>
> cc: [~uschindler]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689603748 Sorry, this is bullshit. Fix your CI and don't apply such stuff in Lucene. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14846) Avoid use of Optional.ofNullable.orElse in the same line
[
https://issues.apache.org/jira/browse/SOLR-14846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192908#comment-17192908
]
Erick Erickson commented on SOLR-14846:
---
[~mdrob] Is this something that should never be allowed? If so, WDYT about
adding it to the forbiddenApis check?
> Avoid use of Optional.ofNullable.orElse in the same line
>
>
> Key: SOLR-14846
> URL: https://issues.apache.org/jira/browse/SOLR-14846
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Minor
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Optional was meant to be used in public APIs to convey to the caller that
> they really need to null-check, not as an internal class to avoid doing our
> own null checks. There are better methods in {{Objects}} for that if we want
> them.
> See also Brian Goetz at JavaOne - https://youtu.be/MFzlgAuanU0?t=2719
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689606665 Actually the bug is in JDK and not in our code. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9517) BugfixDeflater_JDK8252739 causes Java security issues in JDk11
[
https://issues.apache.org/jira/browse/LUCENE-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler resolved LUCENE-9517.
---
Assignee: Uwe Schindler
Resolution: Won't Fix
> BugfixDeflater_JDK8252739 causes Java security issues in JDk11
> --
>
> Key: LUCENE-9517
> URL: https://issues.apache.org/jira/browse/LUCENE-9517
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Ignacio Vera
>Assignee: Uwe Schindler
>Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> We are running into issues when running Elasticsearch CI with java security
> turned on and using JDK11 (only for the ones that contains the jdk bug ).
> The errors look like:
>
>
> {code:java}
> java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessDeclaredMembers") {code}
>
> The issue seems to be here:
> [http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l989]
> As we now have a subclass that wants to run this code. Note that this code
> has been removed in JDK12 and above.
> We might need to wrap the creation of this object in a doPriviledged Block or
> find a different solution that does not need to subclass the Deflater class.
>
> cc: [~uschindler]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9513) Use seconds instead of millisecs
[ https://issues.apache.org/jira/browse/LUCENE-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-9513. Fix Version/s: 8.7 Resolution: Fixed Thanks [~frankmanzhu]! > Use seconds instead of millisecs > > > Key: LUCENE-9513 > URL: https://issues.apache.org/jira/browse/LUCENE-9513 > Project: Lucene - Core > Issue Type: Improvement > Components: core/other >Affects Versions: master (9.0), 8.6.2 >Reporter: Frank Zhu >Priority: Trivial > Fix For: master (9.0), 8.7 > > Time Spent: 10m > Remaining Estimate: 0h > > In RangeFacetsExample, there is a trivial mistake where > private final long nowSec is supposed to be secs. > But it uses as millisecs. > It doesn't affect the output due to the scale change (It will just make > PAST_HOUR, PAST_SIX_HOURS not as intended). Still it's better to be correct. > > I've created a pull request on github with the simple fix. > https://github.com/apache/lucene-solr/pull/1846 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689611169 Actually I remember that this same issue happened also at other places in the JDK, luckily it was fixed on their side, because I complained on the mailing list. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689611752 If this an issue at your CI, just use StackWalker and let this go through. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] sigram commented on pull request #1684: SOLR-14613: strongly typed placement plugin interface and implementation
sigram commented on pull request #1684: URL: https://github.com/apache/lucene-solr/pull/1684#issuecomment-689612257 Thanks @murblanc for taking the lead on this issue, and for your patience! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9444) Need an API to easily fetch facet labels for a field in a document
[
https://issues.apache.org/jira/browse/LUCENE-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192918#comment-17192918
]
Michael McCandless commented on LUCENE-9444:
Overall the patch looks good! Some small feedback:
* Shouldn't {{FacetLabelReader}} be public so callers can interact with it?
This is the danger of putting unit tests in the same package as the classes
they are testing...
* I think {{FacetLabelReader}} is not thread safe (since {{decodedOrds}} is
reused)? So, each thread must pull its own instance? Can you add javadocs
explaining the thread safety?
* Instead of returning {{null}} if caller calls {{next()}} when {{hasNext()}}
had returned {{false}}, can you throw an exception?
* I think the caller must provide {{docId}} in non-decreasing order every
time? Can you 1) add unit test confirming you get an exception if you break
that? And 2) advertise this requirement in the javadocs (for both
{{lookupLabels}} methods)?
* Could you add {{@lucene.experimental}} to the class level javadocs? This
notifies the caller that these APIs are allowed to suddenly change, even in
feature release.
* Shouldn't the comparison be {{== INVALID_ORDINAL}} not {{<=}}?
* I think {{hasNext}} might incorrectly return {{true}} when there are in fact
no more labels, for the variant of {{lookupLabels}} taking a
{{facetDimension}}, when all remaining ordinals are from a different dimension?
Maybe 1) add a unit test showing this issue (failing on the current patch),
then 2) fix it, and 3) confirm the test then passes? {{Iterator}} is annoying
because of this {{hasNext}}/{{next}} dynamic!
* Maybe, instead of {{Iterator}}, we should simply return
{{List}}? Or, maybe we just do {{nextFacetLabel}} (and it can
return {{null}} when done)?
> Need an API to easily fetch facet labels for a field in a document
> --
>
> Key: LUCENE-9444
> URL: https://issues.apache.org/jira/browse/LUCENE-9444
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/facet
>Affects Versions: 8.6
>Reporter: Ankur
>Priority: Major
> Labels: facet
> Attachments: LUCENE-9444.patch
>
>
> A facet field may be included in the list of fields whose values are to be
> returned for each hit.
> In order to get the facet labels for each hit we need to
> # Create an instance of _DocValuesOrdinalsReader_ and invoke
> _getReader(LeafReaderContext context)_ method to obtain an instance of
> _OrdinalsSegmentReader()_
> # _OrdinalsSegmentReader.get(int docID, IntsRef ordinals)_ method is then
> used to fetch and decode the binary payload in the document's BinaryDocValues
> field. This provides the ordinals that refer to facet labels in the
> taxonomy.**
> # Lastly TaxonomyReader.getPath(ord) is used to fetch the labels to be
> returned.
>
> Ideally there should be a simple API - *String[] getLabels(docId)* that hides
> all the above details and gives us the string labels. This can be part of
> *TaxonomyFacets* but that's just one idea.
> I am opening this issue to get community feedback and suggestions.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #1844: SOLR-14847 Create Solr Server TGZ
HoustonPutman commented on a change in pull request #1844:
URL: https://github.com/apache/lucene-solr/pull/1844#discussion_r485683230
##
File path: solr/packaging/build.gradle
##
@@ -109,6 +109,21 @@ task toDir(type: Sync) {
}
}
+task toTgz(type: Tar) {
Review comment:
I just ran this in powershell on a fresh install of Solr. It worked fine
for me. Is there some other way I should test it on windows to see the file
permission issue?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14749) Provide a clean API for cluster-level event processing
[
https://issues.apache.org/jira/browse/SOLR-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192921#comment-17192921
]
Andrzej Bialecki commented on SOLR-14749:
-
[~ichattopadhyaya] as I explained in the comments that follow yours in the PR,
we actually need such coordination in Solr core. The concept of cluster
singleton instances is important, there are quite a few components currently
that work like this, and they all need this functionality. If Solr core doesn't
provide this then each plugin author will have to come up with their own hacky
and messy and buggy scheme.
If the package / plugin infra becomes more robust to support transitive
dependencies then we can extract the actual implementation of ClusterSingleton
management into its own plugin - but still, the interface needs to be in the
core, with a default implementation.
Again, this is not just for autoscaling - autoscaling will be just one user of
the ClusterSingleton concept.
I removed the Scheduler part from PR-1758. Do you still strongly object to the
current state of the PR, Ishan? If so, could you please outline technical
reasons for your objections?
> Provide a clean API for cluster-level event processing
> --
>
> Key: SOLR-14749
> URL: https://issues.apache.org/jira/browse/SOLR-14749
> Project: Solr
> Issue Type: Improvement
> Components: AutoScaling
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Labels: clean-api
> Time Spent: 9h
> Remaining Estimate: 0h
>
> This is a companion issue to SOLR-14613 and it aims at providing a clean,
> strongly typed API for the functionality formerly known as "triggers" - that
> is, a component for generating cluster-level events corresponding to changes
> in the cluster state, and a pluggable API for processing these events.
> The 8x triggers have been removed so this functionality is currently missing
> in 9.0. However, this functionality is crucial for implementing the automatic
> collection repair and re-balancing as the cluster state changes (nodes going
> down / up, becoming overloaded / unused / decommissioned, etc).
> For this reason we need this API and a default implementation of triggers
> that at least can perform automatic collection repair (maintaining the
> desired replication factor in presence of live node changes).
> As before, the actual changes to the collections will be executed using
> existing CollectionAdmin API, which in turn may use the placement plugins
> from SOLR-14613.
> h3. Division of responsibility
> * built-in Solr components (non-pluggable):
> ** cluster state monitoring and event generation,
> ** simple scheduler to periodically generate scheduled events
> * plugins:
> ** automatic collection repair on {{nodeLost}} events (provided by default)
> ** re-balancing of replicas (periodic or on {{nodeAdded}} events)
> ** reporting (eg. requesting additional node provisioning)
> ** scheduled maintenance (eg. removing inactive shards after split)
> h3. Other considerations
> These plugins (unlike the placement plugins) need to execute on one
> designated node in the cluster. Currently the easiest way to implement this
> is to run them on the Overseer leader node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14701) Deprecate Schemaless Mode (Discussion)
[ https://issues.apache.org/jira/browse/SOLR-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192924#comment-17192924 ] Alexandre Rafalovitch commented on SOLR-14701: -- Thanks Jan, these are good questions. To address it from the middle-out, I guess the main question is whether this should end up in the default schema or not. If it ends up in default schema as non-default URP chain, then the usage would be: * bin/post -params "update-chain=guess-schema" documents (to update schema, may require to have commit command to support that) * bin/post ..documents (to actually index) In that case, it makes sense to create schema at the end and to have copyField commands and so on. On the other hand, I was envisaging (in general) to have a learning schema and a bunch of learning examples that layer on top of that. Schemaless mode could be one of the examples. Then, you would create a separate core and it could be a default chain there. And then it would echo recommendations at the end back to the user. Thinking about it, this would be a bit of a demotion for schemaless mode. Perhaps too much of a demotion. And perhaps too much core handling. So, maybe it should be a learning/guessing URP chain, not a learning schema after all. Or maybe it can be combined somehow (still in main config) with some advice given in URP's finish() and schema created in commit(). So, a user could run the guess-schema several times, accumulating the changes (with commit off). And if they are happy with them, then they run commit. Or rollback. And hope for now autoCommit configured in schema... And actually, I am not sure how the output/advice actually gets back to the user. So, this may also be in the "too hard" category, but I capture it as a thought point. As to text vs. string, I think if we see text entries longer than 256 characters, we kind of know they make no sense as indexed strings. If we see much longer strings, that could trigger a warning about marking fields as long. But that's not something that could be created automatically. *int->float->string* requires configuration to recognize that and to decide on whether we support just that one level of extra mapping or have to do a full tree-walking implementation. Or hardcode the knowledge that Int/Long can widen to double. > Deprecate Schemaless Mode (Discussion) > -- > > Key: SOLR-14701 > URL: https://issues.apache.org/jira/browse/SOLR-14701 > Project: Solr > Issue Type: Improvement > Components: Schema and Analysis >Reporter: Marcus Eagan >Priority: Major > Attachments: image-2020-08-04-01-35-03-075.png > > > I know this won't be the most popular ticket out there, but I am growing more > and more sympathetic to the idea that we should rip many of the freedoms out > that cause users more harm than not. One of the freedoms I saw time and time > again to cause issues was schemaless mode. It doesn't work as named or > documented, so I think it should be deprecated. > If you use it in production reliably and in a way that cannot be accomplished > another way, I am happy to hear from more knowledgeable folks as to why > deprecation is a bad idea. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14439) Upgrade to Tika 1.24.1
[ https://issues.apache.org/jira/browse/SOLR-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Allison updated SOLR-14439: --- Status: Patch Available (was: Open) > Upgrade to Tika 1.24.1 > -- > > Key: SOLR-14439 > URL: https://issues.apache.org/jira/browse/SOLR-14439 > Project: Solr > Issue Type: Task > Components: contrib - DataImportHandler >Reporter: Tim Allison >Assignee: Tim Allison >Priority: Major > Attachments: SOLR-14339.patch > > > We recently released 1.24.1 with several fixes for DoS vulnerabilities we > found via fuzzing: CVE-2020-9489 https://seclists.org/oss-sec/2020/q2/69 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689635578 The problem with the fix here is: - I have no idea how the stack trace looks like. And under which circumstances you see this bug. Can you post a link to your CI, to see if it is causeed by the CI infrastrcuture? Does it also break with production ready software? How? - Have you opened a bug report in JDK 10, 11 (it's caused by this change: https://bugs.openjdk.java.net/browse/JDK-8185582)? Should I do? Are there any other ways how to workaround this bug? Why does this PR help for this bug (I don't know the stack trace)? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689636897 Does it help, because this code is only called from Deflater's ctor? http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l207 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler edited a comment on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler edited a comment on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689635578 The problem with the fix here is: - I have no idea how the stack trace looks like. And under which circumstances you see this bug. Can you post a link to your CI, to see if it is causeed by the CI infrastrcuture? Does it also break with production ready software? How? - Have you opened a bug report in JDK 10, 11 (it's caused by this change: https://bugs.openjdk.java.net/browse/JDK-8185582)? Should I do? - The workaround may only work for Elasticsearch, because lucene-core.jar has the accessDeclaredMembers permission given due to the MMapDirectory hack. But the problem is that we take now the responsibility for any bad stuff the Deflater internals are doing. If you remove the permission from Lucene's code, it won't work anymore. IMHO, as a workaround for this bug, you should add a method to your security manager which explicitely allows this access, if it finds the JDK class in the stack trace (similarily implemented like the checkExit() method in Lucene's test SecurityManager ando also the code that disallows calling System.exit() in Elasticsearch). So the workaround should really be in the consuming code. Are there any other ways how to workaround this bug? Why does this PR help for this bug (I don't know the stack trace)? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #1844: SOLR-14847: Create Solr Server TGZ
HoustonPutman commented on a change in pull request #1844:
URL: https://github.com/apache/lucene-solr/pull/1844#discussion_r485708813
##
File path: solr/packaging/build.gradle
##
@@ -109,6 +109,21 @@ task toDir(type: Sync) {
}
}
+task toTgz(type: Tar) {
Review comment:
To be clear, I tested creating the tgz, not using it. I can try that out
in a bit and see if the issues arise.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler commented on a change in pull request #1849:
URL: https://github.com/apache/lucene-solr/pull/1849#discussion_r485711771
##
File path:
lucene/core/src/java/org/apache/lucene/codecs/lucene87/BugfixDeflater_JDK8252739.java
##
@@ -40,7 +42,9 @@ public static Deflater createDeflaterInstance(int level,
boolean nowrap, int dic
throw new IllegalArgumentException("dictLength must be >= 0");
}
if (IS_BUGGY_JDK) {
- return new BugfixDeflater_JDK8252739(level, nowrap, dictLength);
+ // Some JDKs need "accessDeclaredMembers" privileges to create a
Deflater object
+ return AccessController.doPrivileged(
+ (PrivilegedAction) () -> new
BugfixDeflater_JDK8252739(level, nowrap, dictLength));
Review comment:
At least this should be `PrivilegedAction`, because that's
what we return.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689648275 Maybe we should do another fix for the original bug: As only setDictionary is affected, my proposal would be to not subclass `Deflater` at all and instead just return an interface only having the Deflater#setDictionary() method. In buggy jdks it returns a interface pointing to the original Deflater's method, otherwise our own one. I can create a new PR for that. I like this more. This may also speedup Java 11, because it's not doing crazy reflection on every new Deflater instance. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Reopened] (LUCENE-9517) BugfixDeflater_JDK8252739 causes Java security issues in JDk11
[
https://issues.apache.org/jira/browse/LUCENE-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler reopened LUCENE-9517:
---
Maybe we should do a completely different fix for the original issue. I will
provide a PR soon.
> BugfixDeflater_JDK8252739 causes Java security issues in JDk11
> --
>
> Key: LUCENE-9517
> URL: https://issues.apache.org/jira/browse/LUCENE-9517
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Ignacio Vera
>Assignee: Uwe Schindler
>Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> We are running into issues when running Elasticsearch CI with java security
> turned on and using JDK11 (only for the ones that contains the jdk bug ).
> The errors look like:
>
>
> {code:java}
> java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessDeclaredMembers") {code}
>
> The issue seems to be here:
> [http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/zip/Deflater.java#l989]
> As we now have a subclass that wants to run this code. Note that this code
> has been removed in JDK12 and above.
> We might need to wrap the creation of this object in a doPriviledged Block or
> find a different solution that does not need to subclass the Deflater class.
>
> cc: [~uschindler]
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1842: LUCENE-9512: Move LockFactory stress test to be a unit/integration test
uschindler commented on pull request #1842: URL: https://github.com/apache/lucene-solr/pull/1842#issuecomment-689649444 @dweiss Are you fine with this? I would like to commit and backport this soon. Unless you have some minor comments about the cleanup process, as I had to remove your abstraction on top of that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14846) Avoid use of Optional.ofNullable.orElse in the same line
[
https://issues.apache.org/jira/browse/SOLR-14846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192970#comment-17192970
]
Mike Drob commented on SOLR-14846:
--
Individually calling ofNullable and calling orElse* on an Optional is fine,
it's the immediate combination of the two that is unnecessary. I'm not sure
that forbiddenAPI is expressive enough to select for that, but I also haven't
spent a ton of time investigating it and would prefer to defer to an expert in
the library.
Can forbiddenAPI keep us from declaring methods with Optional parameters?
IntelliJ has an inspection/warning/error for that, btw.
> Avoid use of Optional.ofNullable.orElse in the same line
>
>
> Key: SOLR-14846
> URL: https://issues.apache.org/jira/browse/SOLR-14846
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Minor
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Optional was meant to be used in public APIs to convey to the caller that
> they really need to null-check, not as an internal class to avoid doing our
> own null checks. There are better methods in {{Objects}} for that if we want
> them.
> See also Brian Goetz at JavaOne - https://youtu.be/MFzlgAuanU0?t=2719
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14846) Avoid use of Optional.ofNullable.orElse in the same line
[
https://issues.apache.org/jira/browse/SOLR-14846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob resolved SOLR-14846.
--
Fix Version/s: master (9.0)
Resolution: Fixed
> Avoid use of Optional.ofNullable.orElse in the same line
>
>
> Key: SOLR-14846
> URL: https://issues.apache.org/jira/browse/SOLR-14846
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Minor
> Fix For: master (9.0)
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Optional was meant to be used in public APIs to convey to the caller that
> they really need to null-check, not as an internal class to avoid doing our
> own null checks. There are better methods in {{Objects}} for that if we want
> them.
> See also Brian Goetz at JavaOne - https://youtu.be/MFzlgAuanU0?t=2719
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14749) Provide a clean API for cluster-level event processing
[
https://issues.apache.org/jira/browse/SOLR-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192974#comment-17192974
]
Ishan Chattopadhyaya commented on SOLR-14749:
-
I'd prefer if this were to be completely isolated away from core. Transitive
dependencies are supported today if all modules are in contrib and their jars
are in classpath. Anyway, let's refactor it after this is done. I can live with
this for now. +1 to dealing with triggers totally inside packages and not
making the concept first class. Thanks for your work on this and patience.
> Provide a clean API for cluster-level event processing
> --
>
> Key: SOLR-14749
> URL: https://issues.apache.org/jira/browse/SOLR-14749
> Project: Solr
> Issue Type: Improvement
> Components: AutoScaling
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Labels: clean-api
> Time Spent: 9h
> Remaining Estimate: 0h
>
> This is a companion issue to SOLR-14613 and it aims at providing a clean,
> strongly typed API for the functionality formerly known as "triggers" - that
> is, a component for generating cluster-level events corresponding to changes
> in the cluster state, and a pluggable API for processing these events.
> The 8x triggers have been removed so this functionality is currently missing
> in 9.0. However, this functionality is crucial for implementing the automatic
> collection repair and re-balancing as the cluster state changes (nodes going
> down / up, becoming overloaded / unused / decommissioned, etc).
> For this reason we need this API and a default implementation of triggers
> that at least can perform automatic collection repair (maintaining the
> desired replication factor in presence of live node changes).
> As before, the actual changes to the collections will be executed using
> existing CollectionAdmin API, which in turn may use the placement plugins
> from SOLR-14613.
> h3. Division of responsibility
> * built-in Solr components (non-pluggable):
> ** cluster state monitoring and event generation,
> ** simple scheduler to periodically generate scheduled events
> * plugins:
> ** automatic collection repair on {{nodeLost}} events (provided by default)
> ** re-balancing of replicas (periodic or on {{nodeAdded}} events)
> ** reporting (eg. requesting additional node provisioning)
> ** scheduled maintenance (eg. removing inactive shards after split)
> h3. Other considerations
> These plugins (unlike the placement plugins) need to execute on one
> designated node in the cluster. Currently the easiest way to implement this
> is to run them on the Overseer leader node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1844: SOLR-14847: Create Solr Server TGZ
dweiss commented on a change in pull request #1844:
URL: https://github.com/apache/lucene-solr/pull/1844#discussion_r485726236
##
File path: solr/packaging/build.gradle
##
@@ -109,6 +109,21 @@ task toDir(type: Sync) {
}
}
+task toTgz(type: Tar) {
Review comment:
Create a tgz in Windows and unpack it on Linux. Shell scripts should
have the executable bit set.
This can be verified programmatically too, although the above is the
ultimate check. :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14849) Investigate test failures in TestReRankQParserPlugin.testMinExactCount
Tomas Eduardo Fernandez Lobbe created SOLR-14849:
Summary: Investigate test failures in
TestReRankQParserPlugin.testMinExactCount
Key: SOLR-14849
URL: https://issues.apache.org/jira/browse/SOLR-14849
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Tomas Eduardo Fernandez Lobbe
I've seen a couple failures in Jenkins, such as:
{noformat}
FAILED: org.apache.solr.search.TestReRankQParserPlugin.testMinExactCount
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at
__randomizedtesting.SeedInfo.seed([D74824A60D443A78:A2C81E3EE299C40F]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:1016)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:976)
at
org.apache.solr.search.TestReRankQParserPlugin.testMinExactCount(TestReRankQParserPlugin.java:675)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.car
[GitHub] [lucene-solr] iverase commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
iverase commented on pull request #1849:
URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689663895
Thanks Uwe for taking the time reviewing this issue.
>I have no idea how the stack trace looks like. And under which
circumstances you see this bug. Can you post a link to your CI, to see if it is
causeed by the CI infrastrcuture? Does it also break with production ready
software? How?
Stack-trace looks like:
```
java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "accessDeclaredMembers") |
at __randomizedtesting.SeedInfo.seed([7E21C06936079E14:4A7EB7FED693B2BB]:0)
|
-- | --
| | at
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
|
| | at
java.security.AccessController.checkPermission(AccessController.java:897) |
| | at
java.lang.SecurityManager.checkPermission(SecurityManager.java:322) |
| | at java.lang.Class.checkMemberAccess(Class.java:2847) |
| | at java.lang.Class.getDeclaredMethod(Class.java:2471) |
| | at java.util.zip.Deflater$DeflaterZStreamRef.get(Deflater.java:991)
|
| | at java.util.zip.Deflater.(Deflater.java:207) |
| | at
org.apache.lucene.codecs.lucene87.BugfixDeflater_JDK8252739.(BugfixDeflater_JDK8252739.java:53)
|
| | at
org.apache.lucene.codecs.lucene87.BugfixDeflater_JDK8252739.createDeflaterInstance(BugfixDeflater_JDK8252739.java:43)
|
```
I believe It happens every time the runtime environment is set to JDK11 na
dit is a affected by the Deflator bug. Here is a link to one of the logs:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+matrix-java-periodic-fips/ES_RUNTIME_JAVA=adoptopenjdk11,nodes=general-purpose/212/consoleFull
This is un-released code so it is not production ready.
> Have you opened a bug report in JDK 10, 11 (it's caused by this change:
https://bugs.openjdk.java.net/browse/JDK-8185582)? Should I do?
It would be great if you can report it.
> The workaround may only work for Elasticsearch, because lucene-core.jar
has the accessDeclaredMembers permission given due to the MMapDirectory hack.
This is a good point and I agree this fix only works if you give special
permissions to the jar.
> Maybe we should do another fix for the original bug: As only setDictionary
is affected, my proposal would be to not subclass Deflater at all and instead
just return an interface only having the Deflater#setDictionary() method.
That is another option I considered and it now makes more sense than this
approach. +1 not to subclass Deflater.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler opened a new pull request #1850: LUCENE-9517: Don't subclass Deflater and instead create a patch for setDictionary() using a functional interface
uschindler opened a new pull request #1850: URL: https://github.com/apache/lucene-solr/pull/1850 Bugfix for LUCENE-9517: Instead of subclassing Deflater like in LUCENE-9500, we just provide a method reference (using a functional interface), so code can call Deflater#setDictionary. The issue was caused by a missing AccessController#setPrivileged() in JDK's code when Deflater was subclassed. This workaround does not subclass, so the missing doPrivileged is not needed. This also adds the faulty methods to forbiddenapis. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler commented on pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849#issuecomment-689670688 I have a better fix without hacks, just not subclassing Deflater! See PR #1850. I will close this one. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler closed pull request #1849: LUCENE-9517: Add doPrivileged block when creating BugfixDeflater_JDK8252739 objects
uschindler closed pull request #1849: URL: https://github.com/apache/lucene-solr/pull/1849 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] sigram commented on pull request #1831: SOLR-14749 the scheduler part
sigram commented on pull request #1831: URL: https://github.com/apache/lucene-solr/pull/1831#issuecomment-689671370 @noblepaul in this PR the Scheduler depends on ClusterSingleton, that's why I imported it into this PR. Whatever the ClusterSingleton becomes it will be the version from PR-1758. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
