[jira] [Commented] (SOLR-13863) payload query function support for string encoded payloads (delimited_payloads_string)
[
https://issues.apache.org/jira/browse/SOLR-13863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984218#comment-16984218
]
Vincenzo D'Amore commented on SOLR-13863:
-
Ok, now it’s clearer. But this doesn’t not scale out if you have many stores.
I’m even been discouraged to do so by many in the Solr forum. on the other
hand,
I was asked by Erick Erickson to contribute with this solution.
https://lucene.472066.n3.nabble.com/Solr-Payload-example-td4451173.html
> payload query function support for string encoded payloads
> (delimited_payloads_string)
> --
>
> Key: SOLR-13863
> URL: https://issues.apache.org/jira/browse/SOLR-13863
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers, search, SearchComponents - other
>Reporter: Vincenzo D'Amore
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> I have the per-store pricing problem but payload function handles only
> numbers.
> Stores can also be in different countries, for example would be useful having
> with the price also the currency and other attributes related to the store.
> We could have add support for string encoded payloads fixing function query
> named *payload*.
> Example document:
> {
> {{ "id":"my sample doc",}}
> {{ "currencyPayload":[}}
> {{ "store1|EUR",}}
> {{ "store1|USD",}}
> {{ "store3|GBP"}}
> {{ ]}}
> }
> Querying Solr with
> {{fl=payload(currencyPayload,store3)}}
> would generate a response like the following:
> {
> {{ "response": {}}
> {{ "docs":[}}{
> {{ "id":"my sample doc", }}
> {{ "payload(currencyPayload,store3)":"GBP"}}
> {{ }]}}
> }
> {{}}}
> And executing `payload(payloadCurrency,store2)` returns `EUR`, and so on.
> You can use `payload` even as sorting function.
> {{sort=payload(payloadField,value) asc}}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8983) PhraseWildcardQuery - new query to control and optimize wildcard expansions in phrase
[ https://issues.apache.org/jira/browse/LUCENE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984226#comment-16984226 ] Adrien Grand commented on LUCENE-8983: -- This change seems to be causing test failures, e.g. https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Solaris/431/ > PhraseWildcardQuery - new query to control and optimize wildcard expansions > in phrase > - > > Key: LUCENE-8983 > URL: https://issues.apache.org/jira/browse/LUCENE-8983 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Fix For: 8.4 > > Time Spent: 3h > Remaining Estimate: 0h > > A generalized version of PhraseQuery, built with one or more MultiTermQuery > that provides term expansions for multi-terms (one of the expanded terms must > match). > Its main advantage is to control the total number of expansions across all > MultiTermQuery and across all segments. > This query is similar to MultiPhraseQuery, but it handles, controls and > optimizes the multi-term expansions. > > This query is equivalent to building an ordered SpanNearQuery with a list of > SpanTermQuery and SpanMultiTermQueryWrapper. > But it optimizes the multi-term expansions and the segment accesses. > It first resolves the single-terms to early stop if some does not match. > Then it expands each multi-term sequentially, stopping immediately if one > does not match. It detects the segments that do not match to skip them for > the next expansions. This often avoid expanding the other multi-terms on some > or even all segments. And finally it controls the total number of expansions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy merged pull request #1041: SOLR-13969: Clean up and document AuditEvent API
janhoy merged pull request #1041: SOLR-13969: Clean up and document AuditEvent API URL: https://github.com/apache/lucene-solr/pull/1041 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13969) Clean up and document AuditEvent API
[
https://issues.apache.org/jira/browse/SOLR-13969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984237#comment-16984237
]
ASF subversion and git services commented on SOLR-13969:
Commit 9f78482295819ba10a383d3027c1d6313c6e2e7d in lucene-solr's branch
refs/heads/master from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9f78482 ]
SOLR-13969: Clean up and document AuditEvent API (#1041)
> Clean up and document AuditEvent API
>
>
> Key: SOLR-13969
> URL: https://issues.apache.org/jira/browse/SOLR-13969
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Auditlogging
>Affects Versions: 8.1, 8.2, 8.3
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.4
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The AuditEvent bean has an awkward API for the request URL. It exposes
> {{public StringBuffer getRequestUrl()}} which returns a StringBuffer of the
> url up to the query part. This patch will replace that with a {{public String
> getBaseUrl()}}. We will also add a {{public String getUrl()}} method to get
> the full request URL including query parameters.
> Also need to add JavaDocs to all getters in {{AuditEvent}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13969) Clean up and document AuditEvent API
[
https://issues.apache.org/jira/browse/SOLR-13969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984240#comment-16984240
]
ASF subversion and git services commented on SOLR-13969:
Commit ee39b15b864cc98a954d2b413e5c460382bd191f in lucene-solr's branch
refs/heads/branch_8x from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ee39b15 ]
SOLR-13969: Clean up and document AuditEvent API (#1041)
(cherry picked from commit 9f78482295819ba10a383d3027c1d6313c6e2e7d)
> Clean up and document AuditEvent API
>
>
> Key: SOLR-13969
> URL: https://issues.apache.org/jira/browse/SOLR-13969
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Auditlogging
>Affects Versions: 8.1, 8.2, 8.3
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.4
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The AuditEvent bean has an awkward API for the request URL. It exposes
> {{public StringBuffer getRequestUrl()}} which returns a StringBuffer of the
> url up to the query part. This patch will replace that with a {{public String
> getBaseUrl()}}. We will also add a {{public String getUrl()}} method to get
> the full request URL including query parameters.
> Also need to add JavaDocs to all getters in {{AuditEvent}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-13969) Clean up and document AuditEvent API
[
https://issues.apache.org/jira/browse/SOLR-13969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Høydahl resolved SOLR-13969.
Resolution: Fixed
> Clean up and document AuditEvent API
>
>
> Key: SOLR-13969
> URL: https://issues.apache.org/jira/browse/SOLR-13969
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Auditlogging
>Affects Versions: 8.1, 8.2, 8.3
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.4
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The AuditEvent bean has an awkward API for the request URL. It exposes
> {{public StringBuffer getRequestUrl()}} which returns a StringBuffer of the
> url up to the query part. This patch will replace that with a {{public String
> getBaseUrl()}}. We will also add a {{public String getUrl()}} method to get
> the full request URL including query parameters.
> Also need to add JavaDocs to all getters in {{AuditEvent}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-8749) Proposal: Pluggable Interface for Slice Allocation Algorithm
[ https://issues.apache.org/jira/browse/LUCENE-8749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Atri Sharma resolved LUCENE-8749. - Resolution: Won't Fix Not needed post the smarter slice allocation algorithm > Proposal: Pluggable Interface for Slice Allocation Algorithm > > > Key: LUCENE-8749 > URL: https://issues.apache.org/jira/browse/LUCENE-8749 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Atri Sharma >Priority: Major > > The slice allocation method allocates one thread per segment today. If a user > wishes to use a different slice allocation algorithm, there is no way except > to make a change in IndexSearcher. This Jira proposes an interface to > decouple the slice allocation mechanism from IndexSearcher and allow plugging > in the method from an external factory (like Collectors). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13863) payload query function support for string encoded payloads (delimited_payloads_string)
[
https://issues.apache.org/jira/browse/SOLR-13863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984249#comment-16984249
]
Mikhail Khludnev commented on SOLR-13863:
-
How many stores that chain has? I suppose after it reaches reasonable amount
(my ballpark is 10K) it rather can be handled with nesting docs. I skim through
then thread its' about really payload, and giving your comment above you
already figure it out.
> payload query function support for string encoded payloads
> (delimited_payloads_string)
> --
>
> Key: SOLR-13863
> URL: https://issues.apache.org/jira/browse/SOLR-13863
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers, search, SearchComponents - other
>Reporter: Vincenzo D'Amore
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> I have the per-store pricing problem but payload function handles only
> numbers.
> Stores can also be in different countries, for example would be useful having
> with the price also the currency and other attributes related to the store.
> We could have add support for string encoded payloads fixing function query
> named *payload*.
> Example document:
> {
> {{ "id":"my sample doc",}}
> {{ "currencyPayload":[}}
> {{ "store1|EUR",}}
> {{ "store1|USD",}}
> {{ "store3|GBP"}}
> {{ ]}}
> }
> Querying Solr with
> {{fl=payload(currencyPayload,store3)}}
> would generate a response like the following:
> {
> {{ "response": {}}
> {{ "docs":[}}{
> {{ "id":"my sample doc", }}
> {{ "payload(currencyPayload,store3)":"GBP"}}
> {{ }]}}
> }
> {{}}}
> And executing `payload(payloadCurrency,store2)` returns `EUR`, and so on.
> You can use `payload` even as sorting function.
> {{sort=payload(payloadField,value) asc}}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9070) TestLRUCache failures
[
https://issues.apache.org/jira/browse/LUCENE-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984251#comment-16984251
]
Adrien Grand commented on LUCENE-9070:
--
[~atri] FYI more LRUQueryCache failures keep coming, see e.g.
https://builds.apache.org/job/Lucene-Solr-Tests-master/3925/.
> TestLRUCache failures
> -
>
> Key: LUCENE-9070
> URL: https://issues.apache.org/jira/browse/LUCENE-9070
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Major
>
> {noformat}
> 02:11:11[junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedAgreesWithRamUsageTester
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedConstantEntryOverhead
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testDetectMutatedQueries
> 02:11:11[junit4]> Assumption #1: LUCENE-7604: For some unknown reason
> the non-constant BadQuery#hashCode() does not trigger
> ConcurrentModificationException on Java 9 b150
> 02:11:11[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestLRUQueryCache -Dtests.method=testLRUConcurrentLoadsOfSameQuery
> -Dtests.seed=205F9EFCF29BA4D6 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=so-ET -Dtests.timezone=Europe/Ljubljana -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> 02:11:11[junit4] ERROR 0.02s J3 |
> TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery <<<
> 02:11:11[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 1 open files: {_0.cfs=1}
> 02:11:11[junit4]> at
> __randomizedtesting.SeedInfo.seed([205F9EFCF29BA4D6:5E7DB1C91CF54294]:0)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:807)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:465)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 02:11:11[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 02:11:11[junit4]> at
> java.base/java.lang.Thread.run(Thread.java:834)
> 02:11:11[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _0.cfs
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:725)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:768)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:79)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:84)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
> 02:11:11[junit4]> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
> 02:11:11[junit4]> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:526)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:442)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:418)
> 02:11:11[junit4]> ... 35 more
> 02:11:11[junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/workspace/apache+lucene-solr+master/lucene/build/core/test/J3/temp/lucene.search.TestLRUQueryCache_205F9EFCF29BA4D6-001
> 02:11:11[juni
[jira] [Created] (LUCENE-9071) Speed up computation of BM25 scores
Adrien Grand created LUCENE-9071: Summary: Speed up computation of BM25 scores Key: LUCENE-9071 URL: https://issues.apache.org/jira/browse/LUCENE-9071 Project: Lucene - Core Issue Type: Improvement Reporter: Adrien Grand We changed the way BM25 scores are computed in LUCENE-7997 in order to guarantee monotonicity of scores, but this translated to a small decrease of throughput, see annotation CC (October 2017) on Mike's nightly benchmarks. Even though the total number of score computations has decreased since we introduced block-max WAND, its relative cost is not negligible since we not only compute scores on collected documents, but also when decoding skip lists in order to compute the maximum score per block, or group of blocks. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz opened a new pull request #1043: LUCENE-9071: Speed up BM25 scores.
jpountz opened a new pull request #1043: LUCENE-9071: Speed up BM25 scores. URL: https://github.com/apache/lucene-solr/pull/1043 The approach of this change is to try to detect when computing the tf component of the BM25 score would return the same result in float and double space. In order to make this condition likely, it trades little precision from the norm by rounding to the nearest float that has 10 trailing zeros. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9071) Speed up computation of BM25 scores
[
https://issues.apache.org/jira/browse/LUCENE-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984267#comment-16984267
]
Adrien Grand commented on LUCENE-9071:
--
I'm getting a small but consistently reproducible speedup for boolean/term
queries, so I believe it is not noise. Here is the output of one run on
wikibigall for instance:
{noformat}
TaskQPS baseline StdDev QPS patch StdDev
Pct diff
SpanNear1.37 (20.9%)1.36 (20.7%)
-0.5% ( -34% - 51%)
PKLookup 195.89 (3.5%) 195.36 (3.8%)
-0.3% ( -7% -7%)
Prefix3 59.17 (6.7%) 59.13 (6.6%)
-0.1% ( -12% - 14%)
HighTermDayOfYearSort 46.46 (5.8%) 46.44 (6.1%)
-0.1% ( -11% - 12%)
HighTermMonthSort 65.80 (12.4%) 65.80 (12.3%)
-0.0% ( -22% - 28%)
Fuzzy1 160.52 (12.7%) 160.58 (13.2%)
0.0% ( -22% - 29%)
IntervalsOrdered 10.76 (3.3%) 10.76 (3.2%)
0.1% ( -6% -6%)
Wildcard 101.27 (3.7%) 101.33 (4.2%)
0.1% ( -7% -8%)
SloppyPhrase6.32 (7.2%)6.33 (7.2%)
0.1% ( -13% - 15%)
Fuzzy2 80.13 (8.3%) 80.42 (9.2%)
0.4% ( -15% - 19%)
AndHighOrMedMed 37.69 (2.2%) 37.90 (1.9%)
0.6% ( -3% -4%)
Phrase 10.95 (2.5%) 11.03 (2.4%)
0.8% ( -4% -5%)
AndMedOrHighHigh 28.77 (2.7%) 29.14 (3.1%)
1.3% ( -4% -7%)
IntNRQ 92.24 (3.1%) 94.13 (3.4%)
2.1% ( -4% -8%)
AndHighMed 51.84 (3.0%) 52.92 (3.3%)
2.1% ( -4% -8%)
Term 1375.94 (2.3%) 1405.17 (3.1%)
2.1% ( -3% -7%)
OrHighMed 71.51 (2.3%) 73.32 (2.9%)
2.5% ( -2% -7%)
OrHighHigh 86.33 (2.2%) 89.26 (3.0%)
3.4% ( -1% -8%)
AndHighHigh 36.11 (2.9%) 37.34 (3.9%)
3.4% ( -3% - 10%)
{noformat}
> Speed up computation of BM25 scores
> ---
>
> Key: LUCENE-9071
> URL: https://issues.apache.org/jira/browse/LUCENE-9071
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We changed the way BM25 scores are computed in LUCENE-7997 in order to
> guarantee monotonicity of scores, but this translated to a small decrease of
> throughput, see annotation CC (October 2017) on Mike's nightly benchmarks.
> Even though the total number of score computations has decreased since we
> introduced block-max WAND, its relative cost is not negligible since we not
> only compute scores on collected documents, but also when decoding skip lists
> in order to compute the maximum score per block, or group of blocks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13904) Make Analytics component aware of timeAllowed
[ https://issues.apache.org/jira/browse/SOLR-13904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated SOLR-13904: Attachment: (was: SOLR-13904.patch) > Make Analytics component aware of timeAllowed > - > > Key: SOLR-13904 > URL: https://issues.apache.org/jira/browse/SOLR-13904 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mikhail Khludnev >Assignee: Houston Putman >Priority: Major > Attachments: SOLR-13904.patch, SOLR-13904.patch, SOLR-13904.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13904) Make Analytics component aware of timeAllowed
[ https://issues.apache.org/jira/browse/SOLR-13904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated SOLR-13904: Attachment: SOLR-13904.patch Status: Open (was: Open) > Make Analytics component aware of timeAllowed > - > > Key: SOLR-13904 > URL: https://issues.apache.org/jira/browse/SOLR-13904 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mikhail Khludnev >Assignee: Houston Putman >Priority: Major > Attachments: SOLR-13904.patch, SOLR-13904.patch, SOLR-13904.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8983) PhraseWildcardQuery - new query to control and optimize wildcard expansions in phrase
[
https://issues.apache.org/jira/browse/LUCENE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984286#comment-16984286
]
Bruno Roustant commented on LUCENE-8983:
{quote}This change seems to be causing test failures
{quote}
Looking into this.
> PhraseWildcardQuery - new query to control and optimize wildcard expansions
> in phrase
> -
>
> Key: LUCENE-8983
> URL: https://issues.apache.org/jira/browse/LUCENE-8983
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Bruno Roustant
>Assignee: Bruno Roustant
>Priority: Major
> Fix For: 8.4
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> A generalized version of PhraseQuery, built with one or more MultiTermQuery
> that provides term expansions for multi-terms (one of the expanded terms must
> match).
> Its main advantage is to control the total number of expansions across all
> MultiTermQuery and across all segments.
> This query is similar to MultiPhraseQuery, but it handles, controls and
> optimizes the multi-term expansions.
>
> This query is equivalent to building an ordered SpanNearQuery with a list of
> SpanTermQuery and SpanMultiTermQueryWrapper.
> But it optimizes the multi-term expansions and the segment accesses.
> It first resolves the single-terms to early stop if some does not match.
> Then it expands each multi-term sequentially, stopping immediately if one
> does not match. It detects the segments that do not match to skip them for
> the next expansions. This often avoid expanding the other multi-terms on some
> or even all segments. And finally it controls the total number of expansions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-13977) solr create -c not working unter Windows 10
Ana Oleski created SOLR-13977: - Summary: solr create -c not working unter Windows 10 Key: SOLR-13977 URL: https://issues.apache.org/jira/browse/SOLR-13977 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: scripts and tools Affects Versions: 8.3 Reporter: Ana Oleski Hi, I've unpacked sol 8.3.1 under Windows 10. I can successfully start the server with bin/solr start but trying to create a core via command line solr create -c mycore fails with The system cannot find the batch label specified parse_create_args This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Reopened] (LUCENE-8983) PhraseWildcardQuery - new query to control and optimize wildcard expansions in phrase
[ https://issues.apache.org/jira/browse/LUCENE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Roustant reopened LUCENE-8983: > PhraseWildcardQuery - new query to control and optimize wildcard expansions > in phrase > - > > Key: LUCENE-8983 > URL: https://issues.apache.org/jira/browse/LUCENE-8983 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Fix For: 8.4 > > Time Spent: 3h > Remaining Estimate: 0h > > A generalized version of PhraseQuery, built with one or more MultiTermQuery > that provides term expansions for multi-terms (one of the expanded terms must > match). > Its main advantage is to control the total number of expansions across all > MultiTermQuery and across all segments. > This query is similar to MultiPhraseQuery, but it handles, controls and > optimizes the multi-term expansions. > > This query is equivalent to building an ordered SpanNearQuery with a list of > SpanTermQuery and SpanMultiTermQueryWrapper. > But it optimizes the multi-term expansions and the segment accesses. > It first resolves the single-terms to early stop if some does not match. > Then it expands each multi-term sequentially, stopping immediately if one > does not match. It detects the segments that do not match to skip them for > the next expansions. This often avoid expanding the other multi-terms on some > or even all segments. And finally it controls the total number of expansions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8983) PhraseWildcardQuery - new query to control and optimize wildcard expansions in phrase
[ https://issues.apache.org/jira/browse/LUCENE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984356#comment-16984356 ] Bruno Roustant commented on LUCENE-8983: The randomization made only one segment while I thought I ensured 2 segments even with randomization. To make the test more robust, I improved it to skip special segment test counters and just focus on query results and scores if there are not exactly 2 segments. > PhraseWildcardQuery - new query to control and optimize wildcard expansions > in phrase > - > > Key: LUCENE-8983 > URL: https://issues.apache.org/jira/browse/LUCENE-8983 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Fix For: 8.4 > > Time Spent: 3h > Remaining Estimate: 0h > > A generalized version of PhraseQuery, built with one or more MultiTermQuery > that provides term expansions for multi-terms (one of the expanded terms must > match). > Its main advantage is to control the total number of expansions across all > MultiTermQuery and across all segments. > This query is similar to MultiPhraseQuery, but it handles, controls and > optimizes the multi-term expansions. > > This query is equivalent to building an ordered SpanNearQuery with a list of > SpanTermQuery and SpanMultiTermQueryWrapper. > But it optimizes the multi-term expansions and the segment accesses. > It first resolves the single-terms to early stop if some does not match. > Then it expands each multi-term sequentially, stopping immediately if one > does not match. It detects the segments that do not match to skip them for > the next expansions. This often avoid expanding the other multi-terms on some > or even all segments. And finally it controls the total number of expansions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9070) TestLRUCache failures
[
https://issues.apache.org/jira/browse/LUCENE-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984413#comment-16984413
]
Adrien Grand commented on LUCENE-9070:
--
Another one https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/2031/
> TestLRUCache failures
> -
>
> Key: LUCENE-9070
> URL: https://issues.apache.org/jira/browse/LUCENE-9070
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Major
>
> {noformat}
> 02:11:11[junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedAgreesWithRamUsageTester
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedConstantEntryOverhead
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testDetectMutatedQueries
> 02:11:11[junit4]> Assumption #1: LUCENE-7604: For some unknown reason
> the non-constant BadQuery#hashCode() does not trigger
> ConcurrentModificationException on Java 9 b150
> 02:11:11[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestLRUQueryCache -Dtests.method=testLRUConcurrentLoadsOfSameQuery
> -Dtests.seed=205F9EFCF29BA4D6 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=so-ET -Dtests.timezone=Europe/Ljubljana -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> 02:11:11[junit4] ERROR 0.02s J3 |
> TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery <<<
> 02:11:11[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 1 open files: {_0.cfs=1}
> 02:11:11[junit4]> at
> __randomizedtesting.SeedInfo.seed([205F9EFCF29BA4D6:5E7DB1C91CF54294]:0)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:807)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:465)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 02:11:11[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 02:11:11[junit4]> at
> java.base/java.lang.Thread.run(Thread.java:834)
> 02:11:11[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _0.cfs
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:725)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:768)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:79)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:84)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
> 02:11:11[junit4]> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
> 02:11:11[junit4]> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:526)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:442)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:418)
> 02:11:11[junit4]> ... 35 more
> 02:11:11[junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/workspace/apache+lucene-solr+master/lucene/build/core/test/J3/temp/lucene.search.TestLRUQueryCache_205F9EFCF29BA4D6-001
> 02:11:11[junit4] 2> Nov 27, 2019 12:10:48 AM
> com.carr
[jira] [Commented] (LUCENE-8983) PhraseWildcardQuery - new query to control and optimize wildcard expansions in phrase
[ https://issues.apache.org/jira/browse/LUCENE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984418#comment-16984418 ] ASF subversion and git services commented on LUCENE-8983: - Commit 8bd5d7dd2edacc096805e9519656504f29ebd04e in lucene-solr's branch refs/heads/master from Bruno Roustant [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8bd5d7d ] LUCENE-8983: TestPhraseWildcardQuery more robust wrt randomization. > PhraseWildcardQuery - new query to control and optimize wildcard expansions > in phrase > - > > Key: LUCENE-8983 > URL: https://issues.apache.org/jira/browse/LUCENE-8983 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Fix For: 8.4 > > Time Spent: 3h > Remaining Estimate: 0h > > A generalized version of PhraseQuery, built with one or more MultiTermQuery > that provides term expansions for multi-terms (one of the expanded terms must > match). > Its main advantage is to control the total number of expansions across all > MultiTermQuery and across all segments. > This query is similar to MultiPhraseQuery, but it handles, controls and > optimizes the multi-term expansions. > > This query is equivalent to building an ordered SpanNearQuery with a list of > SpanTermQuery and SpanMultiTermQueryWrapper. > But it optimizes the multi-term expansions and the segment accesses. > It first resolves the single-terms to early stop if some does not match. > Then it expands each multi-term sequentially, stopping immediately if one > does not match. It detects the segments that do not match to skip them for > the next expansions. This often avoid expanding the other multi-terms on some > or even all segments. And finally it controls the total number of expansions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8983) PhraseWildcardQuery - new query to control and optimize wildcard expansions in phrase
[ https://issues.apache.org/jira/browse/LUCENE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984422#comment-16984422 ] ASF subversion and git services commented on LUCENE-8983: - Commit e35de979916774937d854009387208c200f35584 in lucene-solr's branch refs/heads/branch_8x from Bruno Roustant [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e35de97 ] LUCENE-8983: TestPhraseWildcardQuery more robust wrt randomization. > PhraseWildcardQuery - new query to control and optimize wildcard expansions > in phrase > - > > Key: LUCENE-8983 > URL: https://issues.apache.org/jira/browse/LUCENE-8983 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Fix For: 8.4 > > Time Spent: 3h > Remaining Estimate: 0h > > A generalized version of PhraseQuery, built with one or more MultiTermQuery > that provides term expansions for multi-terms (one of the expanded terms must > match). > Its main advantage is to control the total number of expansions across all > MultiTermQuery and across all segments. > This query is similar to MultiPhraseQuery, but it handles, controls and > optimizes the multi-term expansions. > > This query is equivalent to building an ordered SpanNearQuery with a list of > SpanTermQuery and SpanMultiTermQueryWrapper. > But it optimizes the multi-term expansions and the segment accesses. > It first resolves the single-terms to early stop if some does not match. > Then it expands each multi-term sequentially, stopping immediately if one > does not match. It detects the segments that do not match to skip them for > the next expansions. This often avoid expanding the other multi-terms on some > or even all segments. And finally it controls the total number of expansions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-8983) PhraseWildcardQuery - new query to control and optimize wildcard expansions in phrase
[ https://issues.apache.org/jira/browse/LUCENE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Roustant resolved LUCENE-8983. Resolution: Fixed > PhraseWildcardQuery - new query to control and optimize wildcard expansions > in phrase > - > > Key: LUCENE-8983 > URL: https://issues.apache.org/jira/browse/LUCENE-8983 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Fix For: 8.4 > > Time Spent: 3h > Remaining Estimate: 0h > > A generalized version of PhraseQuery, built with one or more MultiTermQuery > that provides term expansions for multi-terms (one of the expanded terms must > match). > Its main advantage is to control the total number of expansions across all > MultiTermQuery and across all segments. > This query is similar to MultiPhraseQuery, but it handles, controls and > optimizes the multi-term expansions. > > This query is equivalent to building an ordered SpanNearQuery with a list of > SpanTermQuery and SpanMultiTermQueryWrapper. > But it optimizes the multi-term expansions and the segment accesses. > It first resolves the single-terms to early stop if some does not match. > Then it expands each multi-term sequentially, stopping immediately if one > does not match. It detects the segments that do not match to skip them for > the next expansions. This often avoid expanding the other multi-terms on some > or even all segments. And finally it controls the total number of expansions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8983) PhraseWildcardQuery - new query to control and optimize wildcard expansions in phrase
[ https://issues.apache.org/jira/browse/LUCENE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984430#comment-16984430 ] Adrien Grand commented on LUCENE-8983: -- Thanks for looking so quickly [~bruno.roustant]. > PhraseWildcardQuery - new query to control and optimize wildcard expansions > in phrase > - > > Key: LUCENE-8983 > URL: https://issues.apache.org/jira/browse/LUCENE-8983 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Major > Fix For: 8.4 > > Time Spent: 3h > Remaining Estimate: 0h > > A generalized version of PhraseQuery, built with one or more MultiTermQuery > that provides term expansions for multi-terms (one of the expanded terms must > match). > Its main advantage is to control the total number of expansions across all > MultiTermQuery and across all segments. > This query is similar to MultiPhraseQuery, but it handles, controls and > optimizes the multi-term expansions. > > This query is equivalent to building an ordered SpanNearQuery with a list of > SpanTermQuery and SpanMultiTermQueryWrapper. > But it optimizes the multi-term expansions and the segment accesses. > It first resolves the single-terms to early stop if some does not match. > Then it expands each multi-term sequentially, stopping immediately if one > does not match. It detects the segments that do not match to skip them for > the next expansions. This often avoid expanding the other multi-terms on some > or even all segments. And finally it controls the total number of expansions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9070) TestLRUCache failures
[
https://issues.apache.org/jira/browse/LUCENE-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984433#comment-16984433
]
Atri Sharma commented on LUCENE-9070:
-
[~jpountz] I investigated the two that you pointed a few hours back -- could
not reproduce them
Looking at the latest one now
> TestLRUCache failures
> -
>
> Key: LUCENE-9070
> URL: https://issues.apache.org/jira/browse/LUCENE-9070
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Major
>
> {noformat}
> 02:11:11[junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedAgreesWithRamUsageTester
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedConstantEntryOverhead
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testDetectMutatedQueries
> 02:11:11[junit4]> Assumption #1: LUCENE-7604: For some unknown reason
> the non-constant BadQuery#hashCode() does not trigger
> ConcurrentModificationException on Java 9 b150
> 02:11:11[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestLRUQueryCache -Dtests.method=testLRUConcurrentLoadsOfSameQuery
> -Dtests.seed=205F9EFCF29BA4D6 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=so-ET -Dtests.timezone=Europe/Ljubljana -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> 02:11:11[junit4] ERROR 0.02s J3 |
> TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery <<<
> 02:11:11[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 1 open files: {_0.cfs=1}
> 02:11:11[junit4]> at
> __randomizedtesting.SeedInfo.seed([205F9EFCF29BA4D6:5E7DB1C91CF54294]:0)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:807)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:465)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 02:11:11[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 02:11:11[junit4]> at
> java.base/java.lang.Thread.run(Thread.java:834)
> 02:11:11[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _0.cfs
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:725)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:768)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:79)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:84)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
> 02:11:11[junit4]> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
> 02:11:11[junit4]> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:526)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:442)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:418)
> 02:11:11[junit4]> ... 35 more
> 02:11:11[junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/workspace/apache+lucene-solr+master/lucene/build/core/test/J3/temp/lucene.search.TestLRUQueryCache_205F9EFCF29BA4D6-001
> 02:11:11[jun
[jira] [Commented] (LUCENE-9070) TestLRUCache failures
[
https://issues.apache.org/jira/browse/LUCENE-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984452#comment-16984452
]
Adrien Grand commented on LUCENE-9070:
--
I believe some of these failures don't reproduce because they depend on timing
(I haven't looked in details), but we should still work on fixing them. If this
doesn't look like an easy fix, then let's revert asynchronous caching in order
to not get flooded by test failures?
> TestLRUCache failures
> -
>
> Key: LUCENE-9070
> URL: https://issues.apache.org/jira/browse/LUCENE-9070
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Major
>
> {noformat}
> 02:11:11[junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedAgreesWithRamUsageTester
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedConstantEntryOverhead
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testDetectMutatedQueries
> 02:11:11[junit4]> Assumption #1: LUCENE-7604: For some unknown reason
> the non-constant BadQuery#hashCode() does not trigger
> ConcurrentModificationException on Java 9 b150
> 02:11:11[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestLRUQueryCache -Dtests.method=testLRUConcurrentLoadsOfSameQuery
> -Dtests.seed=205F9EFCF29BA4D6 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=so-ET -Dtests.timezone=Europe/Ljubljana -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> 02:11:11[junit4] ERROR 0.02s J3 |
> TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery <<<
> 02:11:11[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 1 open files: {_0.cfs=1}
> 02:11:11[junit4]> at
> __randomizedtesting.SeedInfo.seed([205F9EFCF29BA4D6:5E7DB1C91CF54294]:0)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:807)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:465)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 02:11:11[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 02:11:11[junit4]> at
> java.base/java.lang.Thread.run(Thread.java:834)
> 02:11:11[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _0.cfs
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:725)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:768)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:79)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:84)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
> 02:11:11[junit4]> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
> 02:11:11[junit4]> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:526)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:442)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:418)
> 02:11:11[junit4]> ... 35 more
> 02:11:11[junit4] 2> NOTE: leaving temporary files on disk at:
> /va
[jira] [Commented] (LUCENE-9070) TestLRUCache failures
[
https://issues.apache.org/jira/browse/LUCENE-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984459#comment-16984459
]
Andrzej Bialecki commented on LUCENE-9070:
--
Sometimes I have a better luck reproducing elusive jenkins failures by running
tests while generating heavy load, eg. video encoding, {{find / -exec grep foo
{}\;}} or something like that...
> TestLRUCache failures
> -
>
> Key: LUCENE-9070
> URL: https://issues.apache.org/jira/browse/LUCENE-9070
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Major
>
> {noformat}
> 02:11:11[junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedAgreesWithRamUsageTester
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedConstantEntryOverhead
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testDetectMutatedQueries
> 02:11:11[junit4]> Assumption #1: LUCENE-7604: For some unknown reason
> the non-constant BadQuery#hashCode() does not trigger
> ConcurrentModificationException on Java 9 b150
> 02:11:11[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestLRUQueryCache -Dtests.method=testLRUConcurrentLoadsOfSameQuery
> -Dtests.seed=205F9EFCF29BA4D6 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=so-ET -Dtests.timezone=Europe/Ljubljana -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> 02:11:11[junit4] ERROR 0.02s J3 |
> TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery <<<
> 02:11:11[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 1 open files: {_0.cfs=1}
> 02:11:11[junit4]> at
> __randomizedtesting.SeedInfo.seed([205F9EFCF29BA4D6:5E7DB1C91CF54294]:0)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:807)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:465)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 02:11:11[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 02:11:11[junit4]> at
> java.base/java.lang.Thread.run(Thread.java:834)
> 02:11:11[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _0.cfs
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:725)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:768)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:79)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:84)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
> 02:11:11[junit4]> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
> 02:11:11[junit4]> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:526)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:442)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:418)
> 02:11:11[junit4]> ... 35 more
> 02:11:11[junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/workspace/apache+lucene-solr+master/lucene/build/core/test/J3/t
[jira] [Commented] (LUCENE-9070) TestLRUCache failures
[
https://issues.apache.org/jira/browse/LUCENE-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984462#comment-16984462
]
Atri Sharma commented on LUCENE-9070:
-
I am +1 for reverting, though trying to identify if it is more of a test issue
rather than a functionality issue. I beasted the test suite using the seeds
that were mentioned in the stack trace of the failing tests but to no avail.
The last failure seems more of an edge case of how a List is printed in Java (I
believe there is no guarantee to the order in which elements would be printed
when printing a list, and the test seems to be taking a dependency on that).
> TestLRUCache failures
> -
>
> Key: LUCENE-9070
> URL: https://issues.apache.org/jira/browse/LUCENE-9070
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Major
>
> {noformat}
> 02:11:11[junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedAgreesWithRamUsageTester
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedConstantEntryOverhead
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testDetectMutatedQueries
> 02:11:11[junit4]> Assumption #1: LUCENE-7604: For some unknown reason
> the non-constant BadQuery#hashCode() does not trigger
> ConcurrentModificationException on Java 9 b150
> 02:11:11[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestLRUQueryCache -Dtests.method=testLRUConcurrentLoadsOfSameQuery
> -Dtests.seed=205F9EFCF29BA4D6 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=so-ET -Dtests.timezone=Europe/Ljubljana -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> 02:11:11[junit4] ERROR 0.02s J3 |
> TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery <<<
> 02:11:11[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 1 open files: {_0.cfs=1}
> 02:11:11[junit4]> at
> __randomizedtesting.SeedInfo.seed([205F9EFCF29BA4D6:5E7DB1C91CF54294]:0)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:807)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:465)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 02:11:11[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 02:11:11[junit4]> at
> java.base/java.lang.Thread.run(Thread.java:834)
> 02:11:11[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _0.cfs
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:725)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:768)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:79)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:84)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
> 02:11:11[junit4]> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
> 02:11:11[junit4]> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:526)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:442)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQ
[jira] [Commented] (LUCENE-9070) TestLRUCache failures
[
https://issues.apache.org/jira/browse/LUCENE-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984467#comment-16984467
]
Adrien Grand commented on LUCENE-9070:
--
Can you revert for now then?
> TestLRUCache failures
> -
>
> Key: LUCENE-9070
> URL: https://issues.apache.org/jira/browse/LUCENE-9070
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Major
>
> {noformat}
> 02:11:11[junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedAgreesWithRamUsageTester
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedConstantEntryOverhead
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testDetectMutatedQueries
> 02:11:11[junit4]> Assumption #1: LUCENE-7604: For some unknown reason
> the non-constant BadQuery#hashCode() does not trigger
> ConcurrentModificationException on Java 9 b150
> 02:11:11[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestLRUQueryCache -Dtests.method=testLRUConcurrentLoadsOfSameQuery
> -Dtests.seed=205F9EFCF29BA4D6 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=so-ET -Dtests.timezone=Europe/Ljubljana -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> 02:11:11[junit4] ERROR 0.02s J3 |
> TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery <<<
> 02:11:11[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 1 open files: {_0.cfs=1}
> 02:11:11[junit4]> at
> __randomizedtesting.SeedInfo.seed([205F9EFCF29BA4D6:5E7DB1C91CF54294]:0)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:807)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:465)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 02:11:11[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 02:11:11[junit4]> at
> java.base/java.lang.Thread.run(Thread.java:834)
> 02:11:11[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _0.cfs
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:725)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:768)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:79)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:84)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
> 02:11:11[junit4]> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
> 02:11:11[junit4]> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:526)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:442)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:418)
> 02:11:11[junit4]> ... 35 more
> 02:11:11[junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/workspace/apache+lucene-solr+master/lucene/build/core/test/J3/temp/lucene.search.TestLRUQueryCache_205F9EFCF29BA4D6-001
> 02:11:11[junit4] 2> Nov 27, 2019 12:10:48 AM
> com.carrotsearch.randomizedtesting.ThreadLeakControl checkT
[GitHub] [lucene-solr] atris opened a new pull request #1044: Revert "LUCENE-8213: Asynchronous Caching in LRUQueryCache"
atris opened a new pull request #1044: Revert "LUCENE-8213: Asynchronous Caching in LRUQueryCache" URL: https://github.com/apache/lucene-solr/pull/1044 Reverts apache/lucene-solr#916 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] atris merged pull request #1044: Revert "LUCENE-8213: Asynchronous Caching in LRUQueryCache"
atris merged pull request #1044: Revert "LUCENE-8213: Asynchronous Caching in LRUQueryCache" URL: https://github.com/apache/lucene-solr/pull/1044 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8213) Cache costly subqueries asynchronously
[ https://issues.apache.org/jira/browse/LUCENE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984472#comment-16984472 ] ASF subversion and git services commented on LUCENE-8213: - Commit 09cd82a6fc51b90275ed326812e1d947c863952e in lucene-solr's branch refs/heads/revert-916-revised_LUCENE-8213 from Atri Sharma [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=09cd82a ] Revert "LUCENE-8213: Asynchronous Caching in LRUQueryCache (#916)" This reverts commit 2d98a619dd96abfa716098bbc5cac2bc3a1b7d49. > Cache costly subqueries asynchronously > -- > > Key: LUCENE-8213 > URL: https://issues.apache.org/jira/browse/LUCENE-8213 > Project: Lucene - Core > Issue Type: Improvement > Components: core/query/scoring >Affects Versions: 7.2.1 >Reporter: Amir Hadadi >Priority: Minor > Labels: performance > Attachments: > 0001-Reproduce-across-segment-caching-of-same-query.patch, > thetaphi_Lucene-Solr-master-Linux_24839.log.txt > > Time Spent: 21h 10m > Remaining Estimate: 0h > > IndexOrDocValuesQuery allows to combine costly range queries with a selective > lead iterator in an optimized way. However, the range query at some point > gets cached by a querying thread in LRUQueryCache, which negates the > optimization of IndexOrDocValuesQuery for that specific query. > It would be nice to see an asynchronous caching implementation in such cases, > so that queries involving IndexOrDocValuesQuery would have consistent > performance characteristics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8213) Cache costly subqueries asynchronously
[ https://issues.apache.org/jira/browse/LUCENE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984473#comment-16984473 ] ASF subversion and git services commented on LUCENE-8213: - Commit 165f6b276c021b3328e3a37dc125f666ece458d4 in lucene-solr's branch refs/heads/master from Atri Sharma [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=165f6b2 ] Revert "LUCENE-8213: Asynchronous Caching in LRUQueryCache (#916)" (#1044) This reverts commit 2d98a619dd96abfa716098bbc5cac2bc3a1b7d49. > Cache costly subqueries asynchronously > -- > > Key: LUCENE-8213 > URL: https://issues.apache.org/jira/browse/LUCENE-8213 > Project: Lucene - Core > Issue Type: Improvement > Components: core/query/scoring >Affects Versions: 7.2.1 >Reporter: Amir Hadadi >Priority: Minor > Labels: performance > Attachments: > 0001-Reproduce-across-segment-caching-of-same-query.patch, > thetaphi_Lucene-Solr-master-Linux_24839.log.txt > > Time Spent: 21.5h > Remaining Estimate: 0h > > IndexOrDocValuesQuery allows to combine costly range queries with a selective > lead iterator in an optimized way. However, the range query at some point > gets cached by a querying thread in LRUQueryCache, which negates the > optimization of IndexOrDocValuesQuery for that specific query. > It would be nice to see an asynchronous caching implementation in such cases, > so that queries involving IndexOrDocValuesQuery would have consistent > performance characteristics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9070) TestLRUCache failures
[
https://issues.apache.org/jira/browse/LUCENE-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984474#comment-16984474
]
Atri Sharma commented on LUCENE-9070:
-
[~jpountz] Done
> TestLRUCache failures
> -
>
> Key: LUCENE-9070
> URL: https://issues.apache.org/jira/browse/LUCENE-9070
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Major
>
> {noformat}
> 02:11:11[junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedAgreesWithRamUsageTester
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testRamBytesUsedConstantEntryOverhead
> 02:11:11[junit4]> Assumption #1: LUCENE-7595: RamUsageTester does not
> work exact in Java 9 (estimations for maps and lists)
> 02:11:11[junit4] IGNOR/A 0.00s J3 |
> TestLRUQueryCache.testDetectMutatedQueries
> 02:11:11[junit4]> Assumption #1: LUCENE-7604: For some unknown reason
> the non-constant BadQuery#hashCode() does not trigger
> ConcurrentModificationException on Java 9 b150
> 02:11:11[junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestLRUQueryCache -Dtests.method=testLRUConcurrentLoadsOfSameQuery
> -Dtests.seed=205F9EFCF29BA4D6 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=so-ET -Dtests.timezone=Europe/Ljubljana -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> 02:11:11[junit4] ERROR 0.02s J3 |
> TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery <<<
> 02:11:11[junit4]> Throwable #1: java.lang.RuntimeException:
> MockDirectoryWrapper: cannot close: there are still 1 open files: {_0.cfs=1}
> 02:11:11[junit4]> at
> __randomizedtesting.SeedInfo.seed([205F9EFCF29BA4D6:5E7DB1C91CF54294]:0)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:807)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:465)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 02:11:11[junit4]> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 02:11:11[junit4]> at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 02:11:11[junit4]> at
> java.base/java.lang.Thread.run(Thread.java:834)
> 02:11:11[junit4]> Caused by: java.lang.RuntimeException: unclosed
> IndexInput: _0.cfs
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:725)
> 02:11:11[junit4]> at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:768)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:79)
> 02:11:11[junit4]> at
> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:101)
> 02:11:11[junit4]> at
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:84)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:177)
> 02:11:11[junit4]> at
> org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:219)
> 02:11:11[junit4]> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
> 02:11:11[junit4]> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:526)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:442)
> 02:11:11[junit4]> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
> 02:11:11[junit4]> at
> org.apache.lucene.search.TestLRUQueryCache.testLRUConcurrentLoadsOfSameQuery(TestLRUQueryCache.java:418)
> 02:11:11[junit4]> ... 35 more
> 02:11:11[junit4] 2> NOTE: leaving temporary files on disk at:
> /var/lib/jenkins/workspace/apache+lucene-solr+master/lucene/build/core/test/J3/temp/lucene.search.TestLRUQueryCache_205F9EFCF29BA4D6-001
> 02:11:11[junit4] 2> Nov 27, 2019 12:10:48 AM
> com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
> 02
[GitHub] [lucene-solr] asfgit closed pull request #1039: SOLR-13968: Support postingsFormat and docValuesFormat in schema fields
asfgit closed pull request #1039: SOLR-13968: Support postingsFormat and docValuesFormat in schema fields URL: https://github.com/apache/lucene-solr/pull/1039 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13968) Support postingsFormat and docValuesFormat in schema fields
[ https://issues.apache.org/jira/browse/SOLR-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984486#comment-16984486 ] ASF subversion and git services commented on SOLR-13968: Commit b4cc3c6ad2db6555f7a1be89adb79397e5c76232 in lucene-solr's branch refs/heads/master from Bruno Roustant [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b4cc3c6 ] SOLR-13968: Support postingsFormat and docValuesFormat in schema fields. Closes #1039 > Support postingsFormat and docValuesFormat in schema fields > --- > > Key: SOLR-13968 > URL: https://issues.apache.org/jira/browse/SOLR-13968 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Minor > Time Spent: 1h 10m > Remaining Estimate: 0h > > Currently it is only possible to declare the postingsFormat and/or > docValuesFormat in a FieldType of the schema. > This PR changes SchemaField to also allow us to declare (and override) in > each Field individually. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
mkhludnev commented on a change in pull request #1034: SOLR-13863: payload
query function now handles string encoded payload field
(delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r351837061
##
File path:
solr/core/src/java/org/apache/solr/search/StringPayloadValueSource.java
##
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.IOException;
+import java.util.Map;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.PostingsEnum;
+import org.apache.lucene.index.Terms;
+import org.apache.lucene.index.TermsEnum;
+import org.apache.lucene.queries.function.FunctionValues;
+import org.apache.lucene.queries.function.ValueSource;
+import org.apache.lucene.queries.function.docvalues.StrDocValues;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.FieldComparator;
+import org.apache.lucene.search.FieldComparatorSource;
+import org.apache.lucene.search.IndexSearcher;
+import org.apache.lucene.search.SimpleFieldComparator;
+import org.apache.lucene.search.SortField;
+import org.apache.lucene.util.BytesRef;
+
+public class StringPayloadValueSource extends ValueSource {
+ protected final String field;
+ protected final String val;
+ protected final String indexedField;
+ protected final BytesRef indexedBytes;
+ protected final ValueSource defaultValueSource;
+
+ public StringPayloadValueSource(String field, String val, String
indexedField, BytesRef indexedBytes, ValueSource defaultValueSource) {
+this.field = field;
+this.val = val;
+this.indexedField = indexedField;
+this.indexedBytes = indexedBytes;
+this.defaultValueSource = defaultValueSource;
+ }
+
+ public SortField getSortField(boolean reverse) {
+return new StringPayloadValueSourceSortField(reverse);
+ }
+
+ @Override
+ public FunctionValues getValues(Map context, LeafReaderContext
readerContext) throws IOException {
+
+final Terms terms = readerContext.reader().terms(indexedField);
+
+FunctionValues defaultValues = defaultValueSource.getValues(context,
readerContext);
+
+// copied the bulk of this from TFValueSource - TODO: this is a very
repeated pattern - base-class this advance logic stuff?
+return new StrDocValues(this) {
+ PostingsEnum docs;
+ int atDoc;
+ int lastDocRequested = -1;
+ String docValue = null;
+
+ {
+reset();
+ }
+
+ public void reset() throws IOException {
+// no one should call us for deleted docs?
+
+if (terms != null) {
+ final TermsEnum termsEnum = terms.iterator();
+ if (termsEnum.seekExact(indexedBytes)) {
+docs = termsEnum.postings(null, PostingsEnum.ALL);
+ } else {
+docs = null;
+ }
+} else {
+ docs = null;
+}
+
+if (docs == null) {
+ // dummy PostingsEnum so floatVal() can work
+ // when would this be called? if field/val did not match? this is
called for every doc? create once and cache?
+ docs = new PostingsEnum() {
+@Override
+public int freq() {
+ return 0;
+}
+
+@Override
+public int nextPosition() throws IOException {
+ return -1;
+}
+
+@Override
+public int startOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public int endOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public BytesRef getPayload() throws IOException {
+ return null;
+}
+
+@Override
+public int docID() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int nextDoc() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int advance(int target) {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public lo
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
mkhludnev commented on a change in pull request #1034: SOLR-13863: payload
query function now handles string encoded payload field
(delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r351837357
##
File path:
solr/core/src/java/org/apache/solr/search/StringPayloadValueSource.java
##
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.IOException;
+import java.util.Map;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.PostingsEnum;
+import org.apache.lucene.index.Terms;
+import org.apache.lucene.index.TermsEnum;
+import org.apache.lucene.queries.function.FunctionValues;
+import org.apache.lucene.queries.function.ValueSource;
+import org.apache.lucene.queries.function.docvalues.StrDocValues;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.FieldComparator;
+import org.apache.lucene.search.FieldComparatorSource;
+import org.apache.lucene.search.IndexSearcher;
+import org.apache.lucene.search.SimpleFieldComparator;
+import org.apache.lucene.search.SortField;
+import org.apache.lucene.util.BytesRef;
+
+public class StringPayloadValueSource extends ValueSource {
+ protected final String field;
+ protected final String val;
+ protected final String indexedField;
+ protected final BytesRef indexedBytes;
+ protected final ValueSource defaultValueSource;
+
+ public StringPayloadValueSource(String field, String val, String
indexedField, BytesRef indexedBytes, ValueSource defaultValueSource) {
+this.field = field;
+this.val = val;
+this.indexedField = indexedField;
+this.indexedBytes = indexedBytes;
+this.defaultValueSource = defaultValueSource;
+ }
+
+ public SortField getSortField(boolean reverse) {
+return new StringPayloadValueSourceSortField(reverse);
+ }
+
+ @Override
+ public FunctionValues getValues(Map context, LeafReaderContext
readerContext) throws IOException {
+
+final Terms terms = readerContext.reader().terms(indexedField);
+
+FunctionValues defaultValues = defaultValueSource.getValues(context,
readerContext);
+
+// copied the bulk of this from TFValueSource - TODO: this is a very
repeated pattern - base-class this advance logic stuff?
+return new StrDocValues(this) {
+ PostingsEnum docs;
+ int atDoc;
+ int lastDocRequested = -1;
+ String docValue = null;
+
+ {
+reset();
+ }
+
+ public void reset() throws IOException {
+// no one should call us for deleted docs?
+
+if (terms != null) {
+ final TermsEnum termsEnum = terms.iterator();
+ if (termsEnum.seekExact(indexedBytes)) {
+docs = termsEnum.postings(null, PostingsEnum.ALL);
+ } else {
+docs = null;
+ }
+} else {
+ docs = null;
+}
+
+if (docs == null) {
+ // dummy PostingsEnum so floatVal() can work
+ // when would this be called? if field/val did not match? this is
called for every doc? create once and cache?
+ docs = new PostingsEnum() {
+@Override
+public int freq() {
+ return 0;
+}
+
+@Override
+public int nextPosition() throws IOException {
+ return -1;
+}
+
+@Override
+public int startOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public int endOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public BytesRef getPayload() throws IOException {
+ return null;
+}
+
+@Override
+public int docID() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int nextDoc() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int advance(int target) {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public lo
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
mkhludnev commented on a change in pull request #1034: SOLR-13863: payload
query function now handles string encoded payload field
(delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r351838742
##
File path:
solr/core/src/java/org/apache/solr/search/StringPayloadValueSource.java
##
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.IOException;
+import java.util.Map;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.PostingsEnum;
+import org.apache.lucene.index.Terms;
+import org.apache.lucene.index.TermsEnum;
+import org.apache.lucene.queries.function.FunctionValues;
+import org.apache.lucene.queries.function.ValueSource;
+import org.apache.lucene.queries.function.docvalues.StrDocValues;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.FieldComparator;
+import org.apache.lucene.search.FieldComparatorSource;
+import org.apache.lucene.search.IndexSearcher;
+import org.apache.lucene.search.SimpleFieldComparator;
+import org.apache.lucene.search.SortField;
+import org.apache.lucene.util.BytesRef;
+
+public class StringPayloadValueSource extends ValueSource {
+ protected final String field;
+ protected final String val;
+ protected final String indexedField;
+ protected final BytesRef indexedBytes;
+ protected final ValueSource defaultValueSource;
+
+ public StringPayloadValueSource(String field, String val, String
indexedField, BytesRef indexedBytes, ValueSource defaultValueSource) {
+this.field = field;
+this.val = val;
+this.indexedField = indexedField;
+this.indexedBytes = indexedBytes;
+this.defaultValueSource = defaultValueSource;
+ }
+
+ public SortField getSortField(boolean reverse) {
+return new StringPayloadValueSourceSortField(reverse);
+ }
+
+ @Override
+ public FunctionValues getValues(Map context, LeafReaderContext
readerContext) throws IOException {
+
+final Terms terms = readerContext.reader().terms(indexedField);
+
+FunctionValues defaultValues = defaultValueSource.getValues(context,
readerContext);
+
+// copied the bulk of this from TFValueSource - TODO: this is a very
repeated pattern - base-class this advance logic stuff?
+return new StrDocValues(this) {
+ PostingsEnum docs;
+ int atDoc;
+ int lastDocRequested = -1;
+ String docValue = null;
+
+ {
+reset();
+ }
+
+ public void reset() throws IOException {
Review comment:
As you said, there are numeric payload function. Can't these dragging
functions unified and reused across them both?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
mkhludnev commented on a change in pull request #1034: SOLR-13863: payload
query function now handles string encoded payload field
(delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r351835703
##
File path:
solr/core/src/java/org/apache/solr/search/StringPayloadValueSource.java
##
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.IOException;
+import java.util.Map;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.PostingsEnum;
+import org.apache.lucene.index.Terms;
+import org.apache.lucene.index.TermsEnum;
+import org.apache.lucene.queries.function.FunctionValues;
+import org.apache.lucene.queries.function.ValueSource;
+import org.apache.lucene.queries.function.docvalues.StrDocValues;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.FieldComparator;
+import org.apache.lucene.search.FieldComparatorSource;
+import org.apache.lucene.search.IndexSearcher;
+import org.apache.lucene.search.SimpleFieldComparator;
+import org.apache.lucene.search.SortField;
+import org.apache.lucene.util.BytesRef;
+
+public class StringPayloadValueSource extends ValueSource {
+ protected final String field;
+ protected final String val;
+ protected final String indexedField;
+ protected final BytesRef indexedBytes;
+ protected final ValueSource defaultValueSource;
+
+ public StringPayloadValueSource(String field, String val, String
indexedField, BytesRef indexedBytes, ValueSource defaultValueSource) {
+this.field = field;
+this.val = val;
+this.indexedField = indexedField;
+this.indexedBytes = indexedBytes;
+this.defaultValueSource = defaultValueSource;
+ }
+
+ public SortField getSortField(boolean reverse) {
+return new StringPayloadValueSourceSortField(reverse);
+ }
+
+ @Override
+ public FunctionValues getValues(Map context, LeafReaderContext
readerContext) throws IOException {
+
+final Terms terms = readerContext.reader().terms(indexedField);
+
+FunctionValues defaultValues = defaultValueSource.getValues(context,
readerContext);
+
+// copied the bulk of this from TFValueSource - TODO: this is a very
repeated pattern - base-class this advance logic stuff?
+return new StrDocValues(this) {
+ PostingsEnum docs;
+ int atDoc;
+ int lastDocRequested = -1;
+ String docValue = null;
+
+ {
+reset();
+ }
+
+ public void reset() throws IOException {
+// no one should call us for deleted docs?
+
+if (terms != null) {
+ final TermsEnum termsEnum = terms.iterator();
+ if (termsEnum.seekExact(indexedBytes)) {
+docs = termsEnum.postings(null, PostingsEnum.ALL);
+ } else {
+docs = null;
+ }
+} else {
+ docs = null;
+}
+
+if (docs == null) {
+ // dummy PostingsEnum so floatVal() can work
+ // when would this be called? if field/val did not match? this is
called for every doc? create once and cache?
+ docs = new PostingsEnum() {
+@Override
+public int freq() {
+ return 0;
+}
+
+@Override
+public int nextPosition() throws IOException {
+ return -1;
+}
+
+@Override
+public int startOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public int endOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public BytesRef getPayload() throws IOException {
+ return null;
+}
+
+@Override
+public int docID() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int nextDoc() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int advance(int target) {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public lo
[jira] [Commented] (SOLR-13968) Support postingsFormat and docValuesFormat in schema fields
[ https://issues.apache.org/jira/browse/SOLR-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984520#comment-16984520 ] ASF subversion and git services commented on SOLR-13968: Commit 1927e850c89c17e067d3bad0404a0c636767cc21 in lucene-solr's branch refs/heads/branch_8x from Bruno Roustant [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=1927e85 ] SOLR-13968: Support postingsFormat and docValuesFormat in schema fields. Closes #1039 > Support postingsFormat and docValuesFormat in schema fields > --- > > Key: SOLR-13968 > URL: https://issues.apache.org/jira/browse/SOLR-13968 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently it is only possible to declare the postingsFormat and/or > docValuesFormat in a FieldType of the schema. > This PR changes SchemaField to also allow us to declare (and override) in > each Field individually. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-13968) Support postingsFormat and docValuesFormat in schema fields
[ https://issues.apache.org/jira/browse/SOLR-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Roustant resolved SOLR-13968. --- Fix Version/s: 8.4 Resolution: Fixed > Support postingsFormat and docValuesFormat in schema fields > --- > > Key: SOLR-13968 > URL: https://issues.apache.org/jira/browse/SOLR-13968 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Bruno Roustant >Assignee: Bruno Roustant >Priority: Minor > Fix For: 8.4 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently it is only possible to declare the postingsFormat and/or > docValuesFormat in a FieldType of the schema. > This PR changes SchemaField to also allow us to declare (and override) in > each Field individually. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-13978) Remove bloat from default configset
Ishan Chattopadhyaya created SOLR-13978: --- Summary: Remove bloat from default configset Key: SOLR-13978 URL: https://issues.apache.org/jira/browse/SOLR-13978 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Reporter: Ishan Chattopadhyaya Fix For: 8.4 We need to review and remove all components that are not essential for search, indexing and other core functionality. Velocity, DIH, etc. should be reviewed. (Marking this as a 8.4 release blocker). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13977) solr create -c not working unter Windows 10
[ https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-13977: Attachment: image-2019-11-28-11-30-20-758.png > solr create -c not working unter Windows 10 > - > > Key: SOLR-13977 > URL: https://issues.apache.org/jira/browse/SOLR-13977 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Affects Versions: 8.3 >Reporter: Ana Oleski >Priority: Major > Attachments: image-2019-11-28-11-30-20-758.png > > > Hi, > I've unpacked sol 8.3.1 under Windows 10. I can successfully start the > server with bin/solr start but trying to create a core via command line > solr create -c mycore > fails with > The system cannot find the batch label specified parse_create_args > > This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13977) solr create -c not working unter Windows 10
[ https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984595#comment-16984595 ] Shawn Heisey commented on SOLR-13977: - Problem replicated on 8.3.0. I removed the "echo off" from the batch file and ran it again. This is what happened: !image-2019-11-28-11-30-20-758.png! I do not know what that might mean. > solr create -c not working unter Windows 10 > - > > Key: SOLR-13977 > URL: https://issues.apache.org/jira/browse/SOLR-13977 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Affects Versions: 8.3 >Reporter: Ana Oleski >Priority: Major > Attachments: image-2019-11-28-11-30-20-758.png > > > Hi, > I've unpacked sol 8.3.1 under Windows 10. I can successfully start the > server with bin/solr start but trying to create a core via command line > solr create -c mycore > fails with > The system cannot find the batch label specified parse_create_args > > This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13905) Make findRequestType in AuditEvent more robust
[ https://issues.apache.org/jira/browse/SOLR-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984627#comment-16984627 ] ASF subversion and git services commented on SOLR-13905: Commit 48010ffc9f636831f2b931ba0c099f587e8e3850 in lucene-solr's branch refs/heads/master from Jan Høydahl [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=48010ff ] Add 8.3.1 CHANGES entry for SOLR-13465, SOLR-13941 and SOLR-13905 > Make findRequestType in AuditEvent more robust > -- > > Key: SOLR-13905 > URL: https://issues.apache.org/jira/browse/SOLR-13905 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Auditlogging >Affects Versions: 8.3 >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Fix For: 8.4, 8.3.1 > > Time Spent: 20m > Remaining Estimate: 0h > > In SOLR-13941 we fixed the root cause for a NullPointer exception in > findRequestType for certain AuditEvents. > In this issue we make it even more robust and make the pattern matching more > performant at the same time as detecting some more patterns for ADMIN > requests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13941) Tests configure Jetty differently than when running via start.jar
[
https://issues.apache.org/jira/browse/SOLR-13941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984626#comment-16984626
]
ASF subversion and git services commented on SOLR-13941:
Commit 48010ffc9f636831f2b931ba0c099f587e8e3850 in lucene-solr's branch
refs/heads/master from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=48010ff ]
Add 8.3.1 CHANGES entry for SOLR-13465, SOLR-13941 and SOLR-13905
> Tests configure Jetty differently than when running via start.jar
> -
>
> Key: SOLR-13941
> URL: https://issues.apache.org/jira/browse/SOLR-13941
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: 8.4, 8.3.1
>
> Attachments: SOLR-13941.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Spinoff from SOLR-13905.
> There seems to be a slightly different configuration of our servlets when
> Solr is run from command line and through Test-runner for our tests.
> This causes different behavior of {{httpRequest.getPathInfo}} and
> {{httpRequest.getServletPath()}} in the two environments, making it hard to
> depend on these in critical code paths, such as
> [SolrDispatchFilter|https://github.com/apache/lucene-solr/blob/f07998fc234c81ff956a84ee508b85f8d573ef38/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java#L494-L499]
> and AuditEvent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13465) CoreContainer.auditloggerPlugin should be volatile
[ https://issues.apache.org/jira/browse/SOLR-13465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984625#comment-16984625 ] ASF subversion and git services commented on SOLR-13465: Commit 48010ffc9f636831f2b931ba0c099f587e8e3850 in lucene-solr's branch refs/heads/master from Jan Høydahl [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=48010ff ] Add 8.3.1 CHANGES entry for SOLR-13465, SOLR-13941 and SOLR-13905 > CoreContainer.auditloggerPlugin should be volatile > -- > > Key: SOLR-13465 > URL: https://issues.apache.org/jira/browse/SOLR-13465 > Project: Solr > Issue Type: Bug >Affects Versions: 8.1 >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Minor > Fix For: 8.4, 8.3.1 > > Time Spent: 20m > Remaining Estimate: 0h > > CoreContainer.auditloggerPlugin needs to be declared as volatile, see Hoss' > comment in SOLR-12120 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13977) solr create -c not working unter Windows 10
[
https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984630#comment-16984630
]
Jan Høydahl commented on SOLR-13977:
The problem is that {{bin/solr.cmd}} needs to be in CRLF line ending format,
but in 8.3.0 commit of SOLR-13682 converted the file to LF format.
[~noble.paul] can you convert it back to CRLF and schedule for 8.3.1? This
warrants a re-spin!
> solr create -c not working unter Windows 10
> -
>
> Key: SOLR-13977
> URL: https://issues.apache.org/jira/browse/SOLR-13977
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Affects Versions: 8.3
>Reporter: Ana Oleski
>Priority: Major
> Attachments: image-2019-11-28-11-30-20-758.png
>
>
> Hi,
> I've unpacked sol 8.3.1 under Windows 10. I can successfully start the
> server with bin/solr start but trying to create a core via command line
> solr create -c mycore
> fails with
> The system cannot find the batch label specified parse_create_args
>
> This works in 7.7.2. I haven't testes the versions in between
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13977) solr create -c not working unter Windows 10
[ https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-13977: --- Fix Version/s: 8.3.1 > solr create -c not working unter Windows 10 > - > > Key: SOLR-13977 > URL: https://issues.apache.org/jira/browse/SOLR-13977 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Affects Versions: 8.3 >Reporter: Ana Oleski >Priority: Major > Fix For: 8.3.1 > > Attachments: image-2019-11-28-11-30-20-758.png > > > Hi, > I've unpacked sol 8.3.1 under Windows 10. I can successfully start the > server with bin/solr start but trying to create a core via command line > solr create -c mycore > fails with > The system cannot find the batch label specified parse_create_args > > This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13977) solr create -c not working unter Windows 10
[ https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-13977: --- Priority: Blocker (was: Major) > solr create -c not working unter Windows 10 > - > > Key: SOLR-13977 > URL: https://issues.apache.org/jira/browse/SOLR-13977 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Affects Versions: 8.3 >Reporter: Ana Oleski >Priority: Blocker > Fix For: 8.3.1 > > Attachments: image-2019-11-28-11-30-20-758.png > > > Hi, > I've unpacked sol 8.3.1 under Windows 10. I can successfully start the > server with bin/solr start but trying to create a core via command line > solr create -c mycore > fails with > The system cannot find the batch label specified parse_create_args > > This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13977) solr create -c not working unter Windows 10
[
https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984633#comment-16984633
]
Jan Høydahl commented on SOLR-13977:
My git setting which I believe avoids this is on a Mac is
{noformat}
git config --global core.autocrlf input
{noformat}
> solr create -c not working unter Windows 10
> -
>
> Key: SOLR-13977
> URL: https://issues.apache.org/jira/browse/SOLR-13977
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Affects Versions: 8.3
>Reporter: Ana Oleski
>Priority: Blocker
> Fix For: 8.3.1
>
> Attachments: image-2019-11-28-11-30-20-758.png
>
>
> Hi,
> I've unpacked sol 8.3.1 under Windows 10. I can successfully start the
> server with bin/solr start but trying to create a core via command line
> solr create -c mycore
> fails with
> The system cannot find the batch label specified parse_create_args
>
> This works in 7.7.2. I haven't testes the versions in between
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13977) solr create -c not working unter Windows 10
[ https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-13977: --- Attachment: SOLR-13977.patch > solr create -c not working unter Windows 10 > - > > Key: SOLR-13977 > URL: https://issues.apache.org/jira/browse/SOLR-13977 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Affects Versions: 8.3 >Reporter: Ana Oleski >Priority: Blocker > Fix For: 8.3.1 > > Attachments: SOLR-13977.patch, image-2019-11-28-11-30-20-758.png > > > Hi, > I've unpacked sol 8.3.1 under Windows 10. I can successfully start the > server with bin/solr start but trying to create a core via command line > solr create -c mycore > fails with > The system cannot find the batch label specified parse_create_args > > This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13977) solr create -c not working unter Windows 10
[
https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984686#comment-16984686
]
Jan Høydahl commented on SOLR-13977:
Instead of setting such configs for each dev, there is something called
{{.gitattributes}} that can be checked into the repo and set rules for what
files must be checked in as CRLF etc. it also has much more power such as
smarter diffs for Java or Python - have not tested it yet but I attach a patch
[^SOLR-13977.patch] with such a file that I believe would do the trick. But I'm
not on Windows so cannot test.
> solr create -c not working unter Windows 10
> -
>
> Key: SOLR-13977
> URL: https://issues.apache.org/jira/browse/SOLR-13977
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Affects Versions: 8.3
>Reporter: Ana Oleski
>Priority: Blocker
> Fix For: 8.3.1
>
> Attachments: SOLR-13977.patch, image-2019-11-28-11-30-20-758.png
>
>
> Hi,
> I've unpacked sol 8.3.1 under Windows 10. I can successfully start the
> server with bin/solr start but trying to create a core via command line
> solr create -c mycore
> fails with
> The system cannot find the batch label specified parse_create_args
>
> This works in 7.7.2. I haven't testes the versions in between
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] freedev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
freedev commented on a change in pull request #1034: SOLR-13863: payload query
function now handles string encoded payload field (delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r351947694
##
File path:
solr/core/src/java/org/apache/solr/search/StringPayloadValueSource.java
##
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.IOException;
+import java.util.Map;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.PostingsEnum;
+import org.apache.lucene.index.Terms;
+import org.apache.lucene.index.TermsEnum;
+import org.apache.lucene.queries.function.FunctionValues;
+import org.apache.lucene.queries.function.ValueSource;
+import org.apache.lucene.queries.function.docvalues.StrDocValues;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.FieldComparator;
+import org.apache.lucene.search.FieldComparatorSource;
+import org.apache.lucene.search.IndexSearcher;
+import org.apache.lucene.search.SimpleFieldComparator;
+import org.apache.lucene.search.SortField;
+import org.apache.lucene.util.BytesRef;
+
+public class StringPayloadValueSource extends ValueSource {
+ protected final String field;
+ protected final String val;
+ protected final String indexedField;
+ protected final BytesRef indexedBytes;
+ protected final ValueSource defaultValueSource;
+
+ public StringPayloadValueSource(String field, String val, String
indexedField, BytesRef indexedBytes, ValueSource defaultValueSource) {
+this.field = field;
+this.val = val;
+this.indexedField = indexedField;
+this.indexedBytes = indexedBytes;
+this.defaultValueSource = defaultValueSource;
+ }
+
+ public SortField getSortField(boolean reverse) {
+return new StringPayloadValueSourceSortField(reverse);
+ }
+
+ @Override
+ public FunctionValues getValues(Map context, LeafReaderContext
readerContext) throws IOException {
+
+final Terms terms = readerContext.reader().terms(indexedField);
+
+FunctionValues defaultValues = defaultValueSource.getValues(context,
readerContext);
+
+// copied the bulk of this from TFValueSource - TODO: this is a very
repeated pattern - base-class this advance logic stuff?
+return new StrDocValues(this) {
+ PostingsEnum docs;
+ int atDoc;
+ int lastDocRequested = -1;
+ String docValue = null;
+
+ {
+reset();
+ }
+
+ public void reset() throws IOException {
+// no one should call us for deleted docs?
+
+if (terms != null) {
+ final TermsEnum termsEnum = terms.iterator();
+ if (termsEnum.seekExact(indexedBytes)) {
+docs = termsEnum.postings(null, PostingsEnum.ALL);
+ } else {
+docs = null;
+ }
+} else {
+ docs = null;
+}
+
+if (docs == null) {
+ // dummy PostingsEnum so floatVal() can work
+ // when would this be called? if field/val did not match? this is
called for every doc? create once and cache?
+ docs = new PostingsEnum() {
+@Override
+public int freq() {
+ return 0;
+}
+
+@Override
+public int nextPosition() throws IOException {
+ return -1;
+}
+
+@Override
+public int startOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public int endOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public BytesRef getPayload() throws IOException {
+ return null;
+}
+
+@Override
+public int docID() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int nextDoc() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int advance(int target) {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public long
[jira] [Commented] (SOLR-13977) solr create -c not working unter Windows 10
[ https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984688#comment-16984688 ] ASF subversion and git services commented on SOLR-13977: Commit 936f4b6ee9cd8c7f9a17800aadc8c5a91bdf74f6 in lucene-solr's branch refs/heads/master from Jan Høydahl [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=936f4b6 ] SOLR-13977: solr create -c not working under Windows 10 > solr create -c not working unter Windows 10 > - > > Key: SOLR-13977 > URL: https://issues.apache.org/jira/browse/SOLR-13977 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Affects Versions: 8.3 >Reporter: Ana Oleski >Priority: Blocker > Fix For: 8.3.1 > > Attachments: SOLR-13977.patch, image-2019-11-28-11-30-20-758.png > > > Hi, > I've unpacked sol 8.3.1 under Windows 10. I can successfully start the > server with bin/solr start but trying to create a core via command line > solr create -c mycore > fails with > The system cannot find the batch label specified parse_create_args > > This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] freedev commented on a change in pull request #1034: SOLR-13863: payload query function now handles string encoded payload field (delimited_payloads_string)
freedev commented on a change in pull request #1034: SOLR-13863: payload query
function now handles string encoded payload field (delimited_payloads_string)
URL: https://github.com/apache/lucene-solr/pull/1034#discussion_r351947694
##
File path:
solr/core/src/java/org/apache/solr/search/StringPayloadValueSource.java
##
@@ -0,0 +1,306 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.IOException;
+import java.util.Map;
+
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.PostingsEnum;
+import org.apache.lucene.index.Terms;
+import org.apache.lucene.index.TermsEnum;
+import org.apache.lucene.queries.function.FunctionValues;
+import org.apache.lucene.queries.function.ValueSource;
+import org.apache.lucene.queries.function.docvalues.StrDocValues;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.FieldComparator;
+import org.apache.lucene.search.FieldComparatorSource;
+import org.apache.lucene.search.IndexSearcher;
+import org.apache.lucene.search.SimpleFieldComparator;
+import org.apache.lucene.search.SortField;
+import org.apache.lucene.util.BytesRef;
+
+public class StringPayloadValueSource extends ValueSource {
+ protected final String field;
+ protected final String val;
+ protected final String indexedField;
+ protected final BytesRef indexedBytes;
+ protected final ValueSource defaultValueSource;
+
+ public StringPayloadValueSource(String field, String val, String
indexedField, BytesRef indexedBytes, ValueSource defaultValueSource) {
+this.field = field;
+this.val = val;
+this.indexedField = indexedField;
+this.indexedBytes = indexedBytes;
+this.defaultValueSource = defaultValueSource;
+ }
+
+ public SortField getSortField(boolean reverse) {
+return new StringPayloadValueSourceSortField(reverse);
+ }
+
+ @Override
+ public FunctionValues getValues(Map context, LeafReaderContext
readerContext) throws IOException {
+
+final Terms terms = readerContext.reader().terms(indexedField);
+
+FunctionValues defaultValues = defaultValueSource.getValues(context,
readerContext);
+
+// copied the bulk of this from TFValueSource - TODO: this is a very
repeated pattern - base-class this advance logic stuff?
+return new StrDocValues(this) {
+ PostingsEnum docs;
+ int atDoc;
+ int lastDocRequested = -1;
+ String docValue = null;
+
+ {
+reset();
+ }
+
+ public void reset() throws IOException {
+// no one should call us for deleted docs?
+
+if (terms != null) {
+ final TermsEnum termsEnum = terms.iterator();
+ if (termsEnum.seekExact(indexedBytes)) {
+docs = termsEnum.postings(null, PostingsEnum.ALL);
+ } else {
+docs = null;
+ }
+} else {
+ docs = null;
+}
+
+if (docs == null) {
+ // dummy PostingsEnum so floatVal() can work
+ // when would this be called? if field/val did not match? this is
called for every doc? create once and cache?
+ docs = new PostingsEnum() {
+@Override
+public int freq() {
+ return 0;
+}
+
+@Override
+public int nextPosition() throws IOException {
+ return -1;
+}
+
+@Override
+public int startOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public int endOffset() throws IOException {
+ return -1;
+}
+
+@Override
+public BytesRef getPayload() throws IOException {
+ return null;
+}
+
+@Override
+public int docID() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int nextDoc() {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public int advance(int target) {
+ return DocIdSetIterator.NO_MORE_DOCS;
+}
+
+@Override
+public long
[jira] [Commented] (SOLR-13977) solr create -c not working unter Windows 10
[
https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984689#comment-16984689
]
Jan Høydahl commented on SOLR-13977:
Ok, I pushed the fix to master but without the {{.gitattributes}} for now. If
someone on Windows can do a pull and verify that this works, you can merge back
to 8x and 8_3 branches.
> solr create -c not working unter Windows 10
> -
>
> Key: SOLR-13977
> URL: https://issues.apache.org/jira/browse/SOLR-13977
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Affects Versions: 8.3
>Reporter: Ana Oleski
>Priority: Blocker
> Fix For: 8.3.1
>
> Attachments: SOLR-13977.patch, image-2019-11-28-11-30-20-758.png
>
>
> Hi,
> I've unpacked sol 8.3.1 under Windows 10. I can successfully start the
> server with bin/solr start but trying to create a core via command line
> solr create -c mycore
> fails with
> The system cannot find the batch label specified parse_create_args
>
> This works in 7.7.2. I haven't testes the versions in between
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13977) solr create -c not working unter Windows 10
[ https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya updated SOLR-13977: Priority: Major (was: Blocker) > solr create -c not working unter Windows 10 > - > > Key: SOLR-13977 > URL: https://issues.apache.org/jira/browse/SOLR-13977 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Affects Versions: 8.3 >Reporter: Ana Oleski >Priority: Major > Fix For: 8.3.1 > > Attachments: SOLR-13977.patch, image-2019-11-28-11-30-20-758.png > > > Hi, > I've unpacked sol 8.3.1 under Windows 10. I can successfully start the > server with bin/solr start but trying to create a core via command line > solr create -c mycore > fails with > The system cannot find the batch label specified parse_create_args > > This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13977) solr create -c not working unter Windows 10
[ https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984702#comment-16984702 ] Ishan Chattopadhyaya commented on SOLR-13977: - Downgrading to Major from Blocker. There are various other ways (Collections APIs) etc. to create a core/collection. This bug doesn't affect core functionality in the least. > solr create -c not working unter Windows 10 > - > > Key: SOLR-13977 > URL: https://issues.apache.org/jira/browse/SOLR-13977 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Affects Versions: 8.3 >Reporter: Ana Oleski >Priority: Major > Fix For: 8.3.1 > > Attachments: SOLR-13977.patch, image-2019-11-28-11-30-20-758.png > > > Hi, > I've unpacked sol 8.3.1 under Windows 10. I can successfully start the > server with bin/solr start but trying to create a core via command line > solr create -c mycore > fails with > The system cannot find the batch label specified parse_create_args > > This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13977) solr create -c not working unter Windows 10
[ https://issues.apache.org/jira/browse/SOLR-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984704#comment-16984704 ] Ishan Chattopadhyaya commented on SOLR-13977: - bq. This warrants a re-spin! No, it doesn't. > solr create -c not working unter Windows 10 > - > > Key: SOLR-13977 > URL: https://issues.apache.org/jira/browse/SOLR-13977 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Affects Versions: 8.3 >Reporter: Ana Oleski >Priority: Major > Fix For: 8.3.1 > > Attachments: SOLR-13977.patch, image-2019-11-28-11-30-20-758.png > > > Hi, > I've unpacked sol 8.3.1 under Windows 10. I can successfully start the > server with bin/solr start but trying to create a core via command line > solr create -c mycore > fails with > The system cannot find the batch label specified parse_create_args > > This works in 7.7.2. I haven't testes the versions in between -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-13979) Expose separate metrics for distributed and non-distributed requests
Shalin Shekhar Mangar created SOLR-13979: Summary: Expose separate metrics for distributed and non-distributed requests Key: SOLR-13979 URL: https://issues.apache.org/jira/browse/SOLR-13979 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: metrics Reporter: Shalin Shekhar Mangar Fix For: master (9.0), 8.4 Currently we expose metrics such as count, rate and latency on a per handler level however for search requests there is no distinction made for distrib vs non-distrib requests. This means that there is no way to find the count, rate or latency of only user-sent queries. I propose that we expose distrib vs non-distrib metrics separately. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13669) [CVE-2019-0193] Remote Code Execution via DataImportHandler
[ https://issues.apache.org/jira/browse/SOLR-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984789#comment-16984789 ] Jan Høydahl commented on SOLR-13669: Guess fix-version should be changed to 8.2.0 and then resolve? > [CVE-2019-0193] Remote Code Execution via DataImportHandler > --- > > Key: SOLR-13669 > URL: https://issues.apache.org/jira/browse/SOLR-13669 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - DataImportHandler >Reporter: Michael Stepankin >Assignee: David Smiley >Priority: Major > Fix For: 8.1.2 > > > The DataImportHandler, an optional but popular module to pull in data from > databases and other sources, has a feature in which the whole DIH > configuration can come from a request's "dataConfig" parameter. The debug > mode of the DIH admin screen uses this to allow convenient debugging / > development of a DIH config. Since a DIH config can contain scripts, this > parameter is a security risk. Starting with version 8.2.0 of Solr, use of > this parameter requires setting the Java System property > "enable.dih.dataConfigParam" to true. > Mitigations: > * Upgrade to 8.2.0 or later, which is secure by default. > * or, edit solrconfig.xml to configure all DataImportHandler usages with an > "invariants" section listing the "dataConfig" parameter set to am empty > string. > * Ensure your network settings are configured so that only trusted traffic > communicates with Solr, especially to the DIH request handler. This is a > best practice to all of Solr. > Credits: > * Michael Stepankin > References: > * https://issues.apache.org/jira/browse/SOLR-13669 > * https://cwiki.apache.org/confluence/display/solr/SolrSecurity -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
