[GitHub] [lucene-solr] iverase commented on issue #1318: LUCENE-9263: Fix wrong transformation of distance in meters to radians in Geo3DPoint
iverase commented on issue #1318: LUCENE-9263: Fix wrong transformation of distance in meters to radians in Geo3DPoint URL: https://github.com/apache/lucene-solr/pull/1318#issuecomment-596068243 @nknize, I had another iteration and decided to go back to one constructor that defines the semiMajorAxis and semiMinorAxis only (The fattening factor was not used and can be calculated anyway from those two values). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13199) NPE due to unexpected null return value from QueryBitSetProducer.getBitSet
[
https://issues.apache.org/jira/browse/SOLR-13199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17053987#comment-17053987
]
Lucene/Solr QA commented on SOLR-13199:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 45m
53s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13199 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12995868/SOLR-13199.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 0c261f42154 |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/702/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/702/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> NPE due to unexpected null return value from QueryBitSetProducer.getBitSet
> --
>
> Key: SOLR-13199
> URL: https://issues.apache.org/jira/browse/SOLR-13199
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: master (9.0)
> Environment: h1. Steps to reproduce
> * Use a Linux machine.
> * Build commit {{ea2c8ba}} of Solr as described in the section below.
> * Build the films collection as described below.
> * Start the server using the command {{./bin/solr start -f -p 8983 -s
> /tmp/home}}
> * Request the URL given in the bug description.
> h1. Compiling the server
> {noformat}
> git clone https://github.com/apache/lucene-solr
> cd lucene-solr
> git checkout ea2c8ba
> ant compile
> cd solr
> ant server
> {noformat}
> h1. Building the collection
> We followed [Exercise
> 2|http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html#exercise-2] from
> the [Solr
> Tutorial|http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html]. The
> attached file ({{home.zip}}) gives the contents of folder {{/tmp/home}} that
> you will obtain by following the steps below:
> {noformat}
> mkdir -p /tmp/home
> echo '' >
> /tmp/home/solr.xml
> {noformat}
> In one terminal start a Solr instance in foreground:
> {noformat}
> ./bin/solr start -f -p 8983 -s /tmp/home
> {noformat}
> In another terminal, create a collection of movies, with no shards and no
> replication, and initialize it:
> {noformat}
> bin/solr create -c films
> curl -X POST -H 'Content-type:application/json' --data-binary '{"add-field":
> {"name":"name", "type":"text_general", "multiValued":false, "stored":true}}'
> http://localhost:8983/solr/films/schema
> curl -X POST -H 'Content-type:application/json' --data-binary
> '{"add-copy-field" : {"source":"*","dest":"_text_"}}'
> http://localhost:8983/solr/films/schema
> ./bin/post -c films example/films/films.json
> {noformat}
>Reporter: Johannes Kloos
>Assignee: Munendra S N
>Priority: Minor
> Labels: diffblue, newde
[jira] [Commented] (SOLR-13944) CollapsingQParserPlugin throws NPE instead of bad request
[
https://issues.apache.org/jira/browse/SOLR-13944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054012#comment-17054012
]
Lucene/Solr QA commented on SOLR-13944:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 46m
13s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 50m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13944 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12995970/SOLR-13944.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 0c261f42154 |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/703/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/703/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> CollapsingQParserPlugin throws NPE instead of bad request
> -
>
> Key: SOLR-13944
> URL: https://issues.apache.org/jira/browse/SOLR-13944
> Project: Solr
> Issue Type: Bug
>Affects Versions: 7.3.1
>Reporter: Stefan
>Assignee: Munendra S N
>Priority: Minor
> Attachments: SOLR-13944.patch, SOLR-13944.patch
>
>
> I noticed the following NPE:
> {code:java}
> java.lang.NullPointerException at
> org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1021)
> at
> org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1081)
> at
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:230)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1602)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1419)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:584)
> {code}
> If I am correct, the problem was already addressed in SOLR-8807. The fix does
> was not working in this case though, because of a syntax error in the query
> (I used the local parameter syntax twice instead of combining it). The
> relevant part of the query is:
> {code:java}
> &fq={!tag=collapser}{!collapse field=productId sort='merchantOrder asc, price
> asc, id asc'}
> {code}
> After discussing that on the mailing list, I was asked to open a ticket,
> because this situation should result in a bad request instead of a
> NullpointerException (see
> [https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201911.mbox/%3CCAMJgJxTuSb%3D8szO8bvHiAafJOs08O_NMB4pcaHOXME4Jj-GO2A%40mail.gmail.com%3E])
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9259) NGramFilter use wrong argument name for preserve option
[ https://issues.apache.org/jira/browse/LUCENE-9259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054025#comment-17054025 ] ASF subversion and git services commented on LUCENE-9259: - Commit 320578274be74a18ce150b604d28a740545fde48 in lucene-solr's branch refs/heads/master from Paul Pazderski [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3205782 ] LUCENE-9259: Fix wrong NGramFilterFactory argument name for preserveOriginal option > NGramFilter use wrong argument name for preserve option > --- > > Key: LUCENE-9259 > URL: https://issues.apache.org/jira/browse/LUCENE-9259 > Project: Lucene - Core > Issue Type: Bug > Components: modules/analysis >Affects Versions: 7.4, 8.0 >Reporter: Paul Pazderski >Priority: Minor > Attachments: LUCENE-9259.patch > > > LUCENE-7960 added the possibility to preserve the original term when using > NGram filters. The documentation says to enable it with 'preserveOriginal' > and it works for EdgeNGram filter. But NGram filter requires the initial > planned option 'keepShortTerms' to enable this feature. > This inconsistency is confusing. I'll provide a patch with a possible fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9259) NGramFilter use wrong argument name for preserve option
[ https://issues.apache.org/jira/browse/LUCENE-9259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054042#comment-17054042 ] ASF subversion and git services commented on LUCENE-9259: - Commit 7904b44f9da90edf546c521c17605d8d363dd0d6 in lucene-solr's branch refs/heads/branch_8x from Paul Pazderski [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7904b44 ] LUCENE-9259: Fix wrong NGramFilterFactory argument name for preserveOriginal option > NGramFilter use wrong argument name for preserve option > --- > > Key: LUCENE-9259 > URL: https://issues.apache.org/jira/browse/LUCENE-9259 > Project: Lucene - Core > Issue Type: Bug > Components: modules/analysis >Affects Versions: 7.4, 8.0 >Reporter: Paul Pazderski >Priority: Minor > Attachments: LUCENE-9259.patch > > > LUCENE-7960 added the possibility to preserve the original term when using > NGram filters. The documentation says to enable it with 'preserveOriginal' > and it works for EdgeNGram filter. But NGram filter requires the initial > planned option 'keepShortTerms' to enable this feature. > This inconsistency is confusing. I'll provide a patch with a possible fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13893) BlobRepository looks at the wrong system variable (runtme.lib.size)
[
https://issues.apache.org/jira/browse/SOLR-13893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054047#comment-17054047
]
Lucene/Solr QA commented on SOLR-13893:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 2m 33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 2m 33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 2m 33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 76m
0s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13893 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12995971/SOLR-13893.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-170-generic #199-Ubuntu SMP
Thu Nov 14 01:45:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 0c261f4 |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/704/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/704/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> BlobRepository looks at the wrong system variable (runtme.lib.size)
> ---
>
> Key: SOLR-13893
> URL: https://issues.apache.org/jira/browse/SOLR-13893
> Project: Solr
> Issue Type: Bug
>Reporter: Erick Erickson
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-13893.patch, SOLR-13893.patch, SOLR-13893.patch
>
>
> Tim Swetland on the user's list pointed out this line in BlobRepository:
> private static final long MAX_JAR_SIZE =
> Long.parseLong(System.getProperty("runtme.lib.size", String.valueOf(5 * 1024
> * 1024)));
> "runtme" can't be right.
> [~ichattopadhyaya][~noblepaul] what's your opinion?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9259) NGramFilter use wrong argument name for preserve option
[ https://issues.apache.org/jira/browse/LUCENE-9259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomoko Uchida updated LUCENE-9259: -- Attachment: LUCENE-9259-master.patch > NGramFilter use wrong argument name for preserve option > --- > > Key: LUCENE-9259 > URL: https://issues.apache.org/jira/browse/LUCENE-9259 > Project: Lucene - Core > Issue Type: Bug > Components: modules/analysis >Affects Versions: 7.4, 8.0 >Reporter: Paul Pazderski >Priority: Minor > Attachments: LUCENE-9259-master.patch, LUCENE-9259.patch > > > LUCENE-7960 added the possibility to preserve the original term when using > NGram filters. The documentation says to enable it with 'preserveOriginal' > and it works for EdgeNGram filter. But NGram filter requires the initial > planned option 'keepShortTerms' to enable this feature. > This inconsistency is confusing. I'll provide a patch with a possible fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9259) NGramFilter use wrong argument name for preserve option
[
https://issues.apache.org/jira/browse/LUCENE-9259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054062#comment-17054062
]
Tomoko Uchida commented on LUCENE-9259:
---
Since the old option {{keepShortTerm}} was added accidentally, we may be able
to go further for the master branch (9.0 release). I attached
[^LUCENE-9259-master.patch] to completely remove the wrongly named option and
replace it with the correct one, {{preserveOriginal,}} in accordance with its
Javadoc.
If there is no objection, I will commit it to the master after waiting a few
more days.
> NGramFilter use wrong argument name for preserve option
> ---
>
> Key: LUCENE-9259
> URL: https://issues.apache.org/jira/browse/LUCENE-9259
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
>Affects Versions: 7.4, 8.0
>Reporter: Paul Pazderski
>Priority: Minor
> Attachments: LUCENE-9259-master.patch, LUCENE-9259.patch
>
>
> LUCENE-7960 added the possibility to preserve the original term when using
> NGram filters. The documentation says to enable it with 'preserveOriginal'
> and it works for EdgeNGram filter. But NGram filter requires the initial
> planned option 'keepShortTerms' to enable this feature.
> This inconsistency is confusing. I'll provide a patch with a possible fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9259) NGramFilter use wrong argument name for preserve option
[ https://issues.apache.org/jira/browse/LUCENE-9259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054064#comment-17054064 ] Tomoko Uchida commented on LUCENE-9259: --- cc [~rcmuir] [~elyograg] > NGramFilter use wrong argument name for preserve option > --- > > Key: LUCENE-9259 > URL: https://issues.apache.org/jira/browse/LUCENE-9259 > Project: Lucene - Core > Issue Type: Bug > Components: modules/analysis >Affects Versions: 7.4, 8.0 >Reporter: Paul Pazderski >Priority: Minor > Attachments: LUCENE-9259-master.patch, LUCENE-9259.patch > > > LUCENE-7960 added the possibility to preserve the original term when using > NGram filters. The documentation says to enable it with 'preserveOriginal' > and it works for EdgeNGram filter. But NGram filter requires the initial > planned option 'keepShortTerms' to enable this feature. > This inconsistency is confusing. I'll provide a patch with a possible fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054068#comment-17054068
]
ASF subversion and git services commented on LUCENE-8962:
-
Commit 27b4fee074437a892c511898e9e86d62c127f1d7 in lucene-solr's branch
refs/heads/branch_8_5 from Michael Sokolov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=27b4fee ]
Revert "LUCENE-8962: Fix intermittent test failures"
This reverts commit a5475de57fed6b339cd5565bd1bd2650f265a537.
Revert "Add CHANGES entry for LUCENE-8962"
This reverts commit fdac6d866344611290c45c164112277581328bc9.
Revert "LUCENE-8962: Add ability to selectively merge on commit (#1155)"
This reverts commit a1791e77143aa8087c0b5ee0e8eb57422e59a09a.
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054066#comment-17054066
]
ASF subversion and git services commented on LUCENE-8962:
-
Commit 27b4fee074437a892c511898e9e86d62c127f1d7 in lucene-solr's branch
refs/heads/branch_8_5 from Michael Sokolov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=27b4fee ]
Revert "LUCENE-8962: Fix intermittent test failures"
This reverts commit a5475de57fed6b339cd5565bd1bd2650f265a537.
Revert "Add CHANGES entry for LUCENE-8962"
This reverts commit fdac6d866344611290c45c164112277581328bc9.
Revert "LUCENE-8962: Add ability to selectively merge on commit (#1155)"
This reverts commit a1791e77143aa8087c0b5ee0e8eb57422e59a09a.
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054067#comment-17054067
]
ASF subversion and git services commented on LUCENE-8962:
-
Commit 27b4fee074437a892c511898e9e86d62c127f1d7 in lucene-solr's branch
refs/heads/branch_8_5 from Michael Sokolov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=27b4fee ]
Revert "LUCENE-8962: Fix intermittent test failures"
This reverts commit a5475de57fed6b339cd5565bd1bd2650f265a537.
Revert "Add CHANGES entry for LUCENE-8962"
This reverts commit fdac6d866344611290c45c164112277581328bc9.
Revert "LUCENE-8962: Add ability to selectively merge on commit (#1155)"
This reverts commit a1791e77143aa8087c0b5ee0e8eb57422e59a09a.
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054075#comment-17054075
]
ASF subversion and git services commented on LUCENE-8962:
-
Commit 89083d43ae6c884e498e40f92053c689925abd66 in lucene-solr's branch
refs/heads/branch_8x from Michael Sokolov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=89083d4 ]
Revert "LUCENE-8962: Split test case (#1313)"
This reverts commit 90aced5a51f92ffd6e97449eb7c44aacc643c8a3.
Revert "LUCENE-8962: woops, remove leftover accidental copyright (darned IDEs)"
This reverts commit 3dbfd102794419551f2ba4b43344cf9e6242a2b8.
Revert "LUCENE-8962: Fix intermittent test failures"
This reverts commit a5475de57fed6b339cd5565bd1bd2650f265a537.
Revert "LUCENE-8962: Add ability to selectively merge on commit (#1155)"
This reverts commit a1791e77143aa8087c0b5ee0e8eb57422e59a09a.
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054077#comment-17054077
]
ASF subversion and git services commented on LUCENE-8962:
-
Commit 89083d43ae6c884e498e40f92053c689925abd66 in lucene-solr's branch
refs/heads/branch_8x from Michael Sokolov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=89083d4 ]
Revert "LUCENE-8962: Split test case (#1313)"
This reverts commit 90aced5a51f92ffd6e97449eb7c44aacc643c8a3.
Revert "LUCENE-8962: woops, remove leftover accidental copyright (darned IDEs)"
This reverts commit 3dbfd102794419551f2ba4b43344cf9e6242a2b8.
Revert "LUCENE-8962: Fix intermittent test failures"
This reverts commit a5475de57fed6b339cd5565bd1bd2650f265a537.
Revert "LUCENE-8962: Add ability to selectively merge on commit (#1155)"
This reverts commit a1791e77143aa8087c0b5ee0e8eb57422e59a09a.
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054074#comment-17054074
]
ASF subversion and git services commented on LUCENE-8962:
-
Commit 89083d43ae6c884e498e40f92053c689925abd66 in lucene-solr's branch
refs/heads/branch_8x from Michael Sokolov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=89083d4 ]
Revert "LUCENE-8962: Split test case (#1313)"
This reverts commit 90aced5a51f92ffd6e97449eb7c44aacc643c8a3.
Revert "LUCENE-8962: woops, remove leftover accidental copyright (darned IDEs)"
This reverts commit 3dbfd102794419551f2ba4b43344cf9e6242a2b8.
Revert "LUCENE-8962: Fix intermittent test failures"
This reverts commit a5475de57fed6b339cd5565bd1bd2650f265a537.
Revert "LUCENE-8962: Add ability to selectively merge on commit (#1155)"
This reverts commit a1791e77143aa8087c0b5ee0e8eb57422e59a09a.
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054076#comment-17054076
]
ASF subversion and git services commented on LUCENE-8962:
-
Commit 89083d43ae6c884e498e40f92053c689925abd66 in lucene-solr's branch
refs/heads/branch_8x from Michael Sokolov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=89083d4 ]
Revert "LUCENE-8962: Split test case (#1313)"
This reverts commit 90aced5a51f92ffd6e97449eb7c44aacc643c8a3.
Revert "LUCENE-8962: woops, remove leftover accidental copyright (darned IDEs)"
This reverts commit 3dbfd102794419551f2ba4b43344cf9e6242a2b8.
Revert "LUCENE-8962: Fix intermittent test failures"
This reverts commit a5475de57fed6b339cd5565bd1bd2650f265a537.
Revert "LUCENE-8962: Add ability to selectively merge on commit (#1155)"
This reverts commit a1791e77143aa8087c0b5ee0e8eb57422e59a09a.
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054078#comment-17054078
]
Michael Sokolov commented on LUCENE-8962:
-
I'm leaving on master for now until we can sort out how to move forward
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-9140) Replace some state polling with CollectionStateWatchers
[
https://issues.apache.org/jira/browse/SOLR-9140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054093#comment-17054093
]
David Smiley commented on SOLR-9140:
It'd be nice to get back to this sometime. It's painful looking at retry/sleep
loops like {{ZkController.waitForShardId.}}
> Replace some state polling with CollectionStateWatchers
> ---
>
> Key: SOLR-9140
> URL: https://issues.apache.org/jira/browse/SOLR-9140
> Project: Solr
> Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Fix For: 6.2, 7.0
>
> Attachments: SOLR-9140.patch, SOLR-9140.patch, SOLR-9140.patch
>
>
> There are a few places in ZkController and the collection processing code
> that directly query ZK for collection state, and then wait and poll for
> expected state changes. We can now replace these with
> CollectionStateWatchers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054111#comment-17054111
]
Simon Willnauer commented on LUCENE-8962:
-
Wouldn’t it be easier to sort it all out in a branch rather on master?
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[
https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054114#comment-17054114
]
Kevin Watters commented on SOLR-13749:
--
What's the timeline for ASAP? depending on how busy we are this week, we can
try to accommodate some changes...
> Implement support for joining across collections with multiple shards ( XCJF )
> --
>
> Key: SOLR-13749
> URL: https://issues.apache.org/jira/browse/SOLR-13749
> Project: Solr
> Issue Type: New Feature
>Reporter: Kevin Watters
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> This ticket includes 2 query parsers.
> The first one is the "Cross collection join filter" (XCJF) parser. This is
> the "Cross-collection join filter" query parser. It can do a call out to a
> remote collection to get a set of join keys to be used as a filter against
> the local collection.
> The second one is the Hash Range query parser that you can specify a field
> name and a hash range, the result is that only the documents that would have
> hashed to that range will be returned.
> This query parser will do an intersection based on join keys between 2
> collections.
> The local collection is the collection that you are searching against.
> The remote collection is the collection that contains the join keys that you
> want to use as a filter.
> Each shard participating in the distributed request will execute a query
> against the remote collection. If the local collection is setup with the
> compositeId router to be routed on the join key field, a hash range query is
> applied to the remote collection query to only match the documents that
> contain a potential match for the documents that are in the local shard/core.
>
>
> Here's some vocab to help with the descriptions of the various parameters.
> ||Term||Description||
> |Local Collection|This is the main collection that is being queried.|
> |Remote Collection|This is the collection that the XCJFQuery will query to
> resolve the join keys.|
> |XCJFQuery|The lucene query that executes a search to get back a set of join
> keys from a remote collection|
> |HashRangeQuery|The lucene query that matches only the documents whose hash
> code on a field falls within a specified range.|
>
>
> ||Param ||Required ||Description||
> |collection|Required|The name of the external Solr collection to be queried
> to retrieve the set of join key values ( required )|
> |zkHost|Optional|The connection string to be used to connect to Zookeeper.
> zkHost and solrUrl are both optional parameters, and at most one of them
> should be specified.
> If neither of zkHost or solrUrl are specified, the local Zookeeper cluster
> will be used. ( optional )|
> |solrUrl|Optional|The URL of the external Solr node to be queried ( optional
> )|
> |from|Required|The join key field name in the external collection ( required
> )|
> |to|Required|The join key field name in the local collection|
> |v|See Note|The query to be executed against the external Solr collection to
> retrieve the set of join key values.
> Note: The original query can be passed at the end of the string or as the
> "v" parameter.
> It's recommended to use query parameter substitution with the "v" parameter
> to ensure no issues arise with the default query parsers.|
> |routed| |true / false. If true, the XCJF query will use each shard's hash
> range to determine the set of join keys to retrieve for that shard.
> This parameter improves the performance of the cross-collection join, but
> it depends on the local collection being routed by the toField. If this
> parameter is not specified,
> the XCJF query will try to determine the correct value automatically.|
> |ttl| |The length of time that an XCJF query in the cache will be considered
> valid, in seconds. Defaults to 3600 (one hour).
> The XCJF query will not be aware of changes to the remote collection, so
> if the remote collection is updated, cached XCJF queries may give inaccurate
> results.
> After the ttl period has expired, the XCJF query will re-execute the join
> against the remote collection.|
> |_All others_| |Any normal Solr parameter can also be specified as a local
> param.|
>
> Example Solr Config.xml changes:
>
> {{<}}{{cache}} {{name}}{{=}}{{"hash_vin"}}
> {{ }}{{class}}{{=}}{{"solr.LRUCache"}}
> {{ }}{{size}}{{=}}{{"128"}}
> {{ }}{{initialSize}}{{=}}{{"0"}}
> {{ }}{{regenerator}}{{=}}{{"solr.NoOpRegenerator"}}{{/>}}
>
> {{<}}{{queryParser}} {{name}}{{=}}{{"xcjf"}}
> {{class}}{{=}}{{"org.apache.solr.search.join.XCJFQueryParserPlugin"}}{{>}}
> {{ }}{{<}}{{str}} {{name}}{{=}}{{"routerFie
[jira] [Created] (LUCENE-9267) The documentation of getQueryBuildTime function reports a wrong time unit.
Pierre-Luc Perron created LUCENE-9267: - Summary: The documentation of getQueryBuildTime function reports a wrong time unit. Key: LUCENE-9267 URL: https://issues.apache.org/jira/browse/LUCENE-9267 Project: Lucene - Core Issue Type: Task Components: modules/other Affects Versions: 8.4, 8.3, 8.2 Reporter: Pierre-Luc Perron As per documentation, the [MatchingQueries|https://lucene.apache.org/core/8_4_1/monitor/org/apache/lucene/monitor/MatchingQueries.html] class returns both getQueryBuildTime and getSearchTime in milliseconds. The code shows [searchTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/CandidateMatcher.java#L112] returning milliseconds. However, the code shows [buildTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/QueryIndex.java#L280] returning nanoseconds. The patch changes the documentation of getQueryBuildTime to report nanoseconds instead of milliseconds. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9267) The documentation of getQueryBuildTime function reports a wrong time unit.
[ https://issues.apache.org/jira/browse/LUCENE-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre-Luc Perron updated LUCENE-9267: -- Attachment: LUCENE-9267.patch > The documentation of getQueryBuildTime function reports a wrong time unit. > -- > > Key: LUCENE-9267 > URL: https://issues.apache.org/jira/browse/LUCENE-9267 > Project: Lucene - Core > Issue Type: Task > Components: modules/other >Affects Versions: 8.2, 8.3, 8.4 >Reporter: Pierre-Luc Perron >Priority: Trivial > Labels: documentation, newbie, pull-request-available > Attachments: LUCENE-9267.patch > > > As per documentation, the > [MatchingQueries|https://lucene.apache.org/core/8_4_1/monitor/org/apache/lucene/monitor/MatchingQueries.html] > class returns both getQueryBuildTime and getSearchTime in milliseconds. The > code shows > [searchTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/CandidateMatcher.java#L112] > returning milliseconds. However, the code shows > [buildTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/QueryIndex.java#L280] > returning nanoseconds. > The patch changes the documentation of getQueryBuildTime to report > nanoseconds instead of milliseconds. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nhat Nguyen updated LUCENE-8962:
Attachment: failed-tests.patch
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png, failed-tests.patch
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] plperron opened a new pull request #1330: LUCENE-9267 Replace getQueryBuildTime time unit from ms to ns
plperron opened a new pull request #1330: LUCENE-9267 Replace getQueryBuildTime time unit from ms to ns URL: https://github.com/apache/lucene-solr/pull/1330 # Description Change the documentation of getQueryBuildTime to report nanoseconds instead of milliseconds. # Solution Update the javadoc. # Tests N/A # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [x] I have developed this patch against the `master` branch. - [x] I have run `ant precommit` and the appropriate test suite. - [x] I have added tests for my changes. - [x] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8962) Can we merge small segments during refresh, for faster searching?
[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054163#comment-17054163
]
Nhat Nguyen commented on LUCENE-8962:
-
I have translated some Elasticsearch tests (see [^failed-tests.patch]) to
Lucene. The test _testMergeOnCommitKeepFullyDeletedSegments_ consistently fails
because _mergeFinished_ was called twice. This happens because we forget to set
_success_ to _true_ when the new segment does not have any document (i.e.,
_SegmentMerger#shouldMerge_ is false).
Even with that fix, both _testRandomOperations_ and
_testRandomOperationsWithSoftDeletes_ still randomly fail after a few thousand
iterations. Since both are multi-threaded tests, we will need more time to
figure out the root cause.
> Can we merge small segments during refresh, for faster searching?
> -
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
>Reporter: Michael McCandless
>Priority: Major
> Fix For: 8.5
>
> Attachments: LUCENE-8962_demo.png, failed-tests.patch
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9267) The documentation of getQueryBuildTime function reports a wrong time unit.
[ https://issues.apache.org/jira/browse/LUCENE-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre-Luc Perron updated LUCENE-9267: -- Description: As per documentation, the [MatchingQueries|https://lucene.apache.org/core/8_4_1/monitor/org/apache/lucene/monitor/MatchingQueries.html] class returns both getQueryBuildTime and getSearchTime in milliseconds. The code shows [searchTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/CandidateMatcher.java#L112] returning milliseconds. However, the code shows [buildTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/QueryIndex.java#L280] returning nanoseconds. The patch changes the documentation of getQueryBuildTime to report nanoseconds instead of milliseconds. was: As per documentation, the [MatchingQueries|https://lucene.apache.org/core/8_4_1/monitor/org/apache/lucene/monitor/MatchingQueries.html] class returns both getQueryBuildTime and getSearchTime in milliseconds. The code shows [searchTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/CandidateMatcher.java#L112] returning milliseconds. However, the code shows [buildTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/QueryIndex.java#L280] returning nanoseconds. The patch changes the documentation of getQueryBuildTime to report nanoseconds instead of milliseconds. Edit: add reference to github [pull request|https://github.com/apache/lucene-solr/pull/1330] > The documentation of getQueryBuildTime function reports a wrong time unit. > -- > > Key: LUCENE-9267 > URL: https://issues.apache.org/jira/browse/LUCENE-9267 > Project: Lucene - Core > Issue Type: Task > Components: modules/other >Affects Versions: 8.2, 8.3, 8.4 >Reporter: Pierre-Luc Perron >Priority: Trivial > Labels: documentation, newbie, pull-request-available > Attachments: LUCENE-9267.patch > > Time Spent: 10m > Remaining Estimate: 0h > > As per documentation, the > [MatchingQueries|https://lucene.apache.org/core/8_4_1/monitor/org/apache/lucene/monitor/MatchingQueries.html] > class returns both getQueryBuildTime and getSearchTime in milliseconds. The > code shows > [searchTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/CandidateMatcher.java#L112] > returning milliseconds. However, the code shows > [buildTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/QueryIndex.java#L280] > returning nanoseconds. > The patch changes the documentation of getQueryBuildTime to report > nanoseconds instead of milliseconds. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9267) The documentation of getQueryBuildTime function reports a wrong time unit.
[ https://issues.apache.org/jira/browse/LUCENE-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre-Luc Perron updated LUCENE-9267: -- Description: As per documentation, the [MatchingQueries|https://lucene.apache.org/core/8_4_1/monitor/org/apache/lucene/monitor/MatchingQueries.html] class returns both getQueryBuildTime and getSearchTime in milliseconds. The code shows [searchTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/CandidateMatcher.java#L112] returning milliseconds. However, the code shows [buildTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/QueryIndex.java#L280] returning nanoseconds. The patch changes the documentation of getQueryBuildTime to report nanoseconds instead of milliseconds. Edit: add reference to github [pull request|https://github.com/apache/lucene-solr/pull/1330] was: As per documentation, the [MatchingQueries|https://lucene.apache.org/core/8_4_1/monitor/org/apache/lucene/monitor/MatchingQueries.html] class returns both getQueryBuildTime and getSearchTime in milliseconds. The code shows [searchTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/CandidateMatcher.java#L112] returning milliseconds. However, the code shows [buildTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/QueryIndex.java#L280] returning nanoseconds. The patch changes the documentation of getQueryBuildTime to report nanoseconds instead of milliseconds. > The documentation of getQueryBuildTime function reports a wrong time unit. > -- > > Key: LUCENE-9267 > URL: https://issues.apache.org/jira/browse/LUCENE-9267 > Project: Lucene - Core > Issue Type: Task > Components: modules/other >Affects Versions: 8.2, 8.3, 8.4 >Reporter: Pierre-Luc Perron >Priority: Trivial > Labels: documentation, newbie, pull-request-available > Attachments: LUCENE-9267.patch > > Time Spent: 10m > Remaining Estimate: 0h > > As per documentation, the > [MatchingQueries|https://lucene.apache.org/core/8_4_1/monitor/org/apache/lucene/monitor/MatchingQueries.html] > class returns both getQueryBuildTime and getSearchTime in milliseconds. The > code shows > [searchTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/CandidateMatcher.java#L112] > returning milliseconds. However, the code shows > [buildTime|https://github.com/apache/lucene-solr/blob/320578274be74a18ce150b604d28a740545fde48/lucene/monitor/src/java/org/apache/lucene/monitor/QueryIndex.java#L280] > returning nanoseconds. > The patch changes the documentation of getQueryBuildTime to report > nanoseconds instead of milliseconds. > Edit: add reference to github [pull > request|https://github.com/apache/lucene-solr/pull/1330] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9268) Add some random tests for IndexWriter
Nhat Nguyen created LUCENE-9268: --- Summary: Add some random tests for IndexWriter Key: LUCENE-9268 URL: https://issues.apache.org/jira/browse/LUCENE-9268 Project: Lucene - Core Issue Type: Test Components: core/index Affects Versions: master (9.0), 8.5, 8.6 Reporter: Nhat Nguyen Add some tests that perform a set of operations randomly and concurrently on IndexWriter. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dnhatn opened a new pull request #1331: LUCENE-9268: Add some random tests to IndexWriter
dnhatn opened a new pull request #1331: LUCENE-9268: Add some random tests to IndexWriter URL: https://github.com/apache/lucene-solr/pull/1331 Add some tests that perform a set of operations randomly and concurrently on IndexWriter. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9259) NGramFilter use wrong argument name for preserve option
[
https://issues.apache.org/jira/browse/LUCENE-9259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054171#comment-17054171
]
Lucene/Solr QA commented on LUCENE-9259:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} common in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 2m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-9259 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12996003/LUCENE-9259-master.patch
|
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 320578274be |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-LUCENE-Build/256/testReport/ |
| modules | C: lucene lucene/analysis/common U: lucene |
| Console output |
https://builds.apache.org/job/PreCommit-LUCENE-Build/256/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> NGramFilter use wrong argument name for preserve option
> ---
>
> Key: LUCENE-9259
> URL: https://issues.apache.org/jira/browse/LUCENE-9259
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
>Affects Versions: 7.4, 8.0
>Reporter: Paul Pazderski
>Priority: Minor
> Attachments: LUCENE-9259-master.patch, LUCENE-9259.patch
>
>
> LUCENE-7960 added the possibility to preserve the original term when using
> NGram filters. The documentation says to enable it with 'preserveOriginal'
> and it works for EdgeNGram filter. But NGram filter requires the initial
> planned option 'keepShortTerms' to enable this feature.
> This inconsistency is confusing. I'll provide a patch with a possible fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] msokolov commented on a change in pull request #1331: LUCENE-9268: Add some random tests to IndexWriter
msokolov commented on a change in pull request #1331: LUCENE-9268: Add some
random tests to IndexWriter
URL: https://github.com/apache/lucene-solr/pull/1331#discussion_r389304104
##
File path: lucene/core/src/test/org/apache/lucene/index/TestIndexWriter.java
##
@@ -3775,4 +3777,120 @@ public void testRefreshAndRollbackConcurrently()
throws Exception {
IOUtils.close(sm, dir);
}
}
+
+ public void testRandomOperations() throws Exception {
+Directory dir = newDirectory();
+IndexWriterConfig iwc = newIndexWriterConfig();
+iwc.setMergePolicy(new FilterMergePolicy(newMergePolicy()) {
+ boolean keepFullyDeletedSegment = random().nextBoolean();
+ @Override
+ public boolean keepFullyDeletedSegment(IOSupplier
readerIOSupplier) {
+return keepFullyDeletedSegment;
+ }
+});
+IndexWriter writer = new IndexWriter(dir, iwc);
+SearcherManager sm = new SearcherManager(writer, new SearcherFactory());
+Semaphore permits = new Semaphore(100 + random().nextInt(1000));
+boolean singleDoc = random().nextBoolean();
+Thread[] threads = new Thread[1 + random().nextInt(4)];
+CountDownLatch latch = new CountDownLatch(threads.length);
+for (int i = 0; i < threads.length; i++) {
+ threads[i] = new Thread(() -> {
+latch.countDown();
+try {
+ latch.await();
+ while (permits.tryAcquire()) {
+String id = singleDoc ? "1" :
Integer.toString(random().nextInt(10));
+Document doc = new Document();
+doc.add(new StringField("id", id, Field.Store.YES));
+if (random().nextInt(10) <= 2) {
+ writer.updateDocument(new Term("id", id), doc);
+} else if (random().nextInt(10) <= 2) {
+ writer.deleteDocuments(new Term("id", id));
+} else {
+ writer.addDocument(doc);
+}
+if (random().nextInt(100) < 10) {
+ sm.maybeRefreshBlocking();
+}
+if (random().nextInt(100) < 5) {
+ writer.commit();
+}
+if (random().nextInt(100) < 1) {
+ writer.forceMerge(1 + random().nextInt(10),
random().nextBoolean());
+}
+ }
+} catch (Exception e) {
+ throw new AssertionError(e);
+}
+ });
+ threads[i].start();
+}
+for (Thread thread : threads) {
+ thread.join();
+}
+sm.maybeRefreshBlocking();
+IOUtils.close(writer, sm, dir);
+ }
+
+ public void testRandomOperationsWithSoftDeletes() throws Exception {
+Directory dir = newDirectory();
Review comment:
can we use try-with-resources to ensure proper closure of Directory,
IndexWriter, and SearcherManager on failure?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] msokolov commented on a change in pull request #1331: LUCENE-9268: Add some random tests to IndexWriter
msokolov commented on a change in pull request #1331: LUCENE-9268: Add some
random tests to IndexWriter
URL: https://github.com/apache/lucene-solr/pull/1331#discussion_r389304196
##
File path: lucene/core/src/test/org/apache/lucene/index/TestIndexWriter.java
##
@@ -3775,4 +3777,120 @@ public void testRefreshAndRollbackConcurrently()
throws Exception {
IOUtils.close(sm, dir);
}
}
+
+ public void testRandomOperations() throws Exception {
+Directory dir = newDirectory();
+IndexWriterConfig iwc = newIndexWriterConfig();
+iwc.setMergePolicy(new FilterMergePolicy(newMergePolicy()) {
+ boolean keepFullyDeletedSegment = random().nextBoolean();
+ @Override
+ public boolean keepFullyDeletedSegment(IOSupplier
readerIOSupplier) {
+return keepFullyDeletedSegment;
+ }
+});
+IndexWriter writer = new IndexWriter(dir, iwc);
+SearcherManager sm = new SearcherManager(writer, new SearcherFactory());
+Semaphore permits = new Semaphore(100 + random().nextInt(1000));
+boolean singleDoc = random().nextBoolean();
+Thread[] threads = new Thread[1 + random().nextInt(4)];
+CountDownLatch latch = new CountDownLatch(threads.length);
+for (int i = 0; i < threads.length; i++) {
+ threads[i] = new Thread(() -> {
+latch.countDown();
+try {
+ latch.await();
+ while (permits.tryAcquire()) {
+String id = singleDoc ? "1" :
Integer.toString(random().nextInt(10));
+Document doc = new Document();
+doc.add(new StringField("id", id, Field.Store.YES));
+if (random().nextInt(10) <= 2) {
+ writer.updateDocument(new Term("id", id), doc);
+} else if (random().nextInt(10) <= 2) {
+ writer.deleteDocuments(new Term("id", id));
+} else {
+ writer.addDocument(doc);
+}
+if (random().nextInt(100) < 10) {
+ sm.maybeRefreshBlocking();
+}
+if (random().nextInt(100) < 5) {
+ writer.commit();
+}
+if (random().nextInt(100) < 1) {
+ writer.forceMerge(1 + random().nextInt(10),
random().nextBoolean());
+}
+ }
+} catch (Exception e) {
+ throw new AssertionError(e);
+}
+ });
+ threads[i].start();
+}
+for (Thread thread : threads) {
+ thread.join();
+}
+sm.maybeRefreshBlocking();
+IOUtils.close(writer, sm, dir);
+ }
+
+ public void testRandomOperationsWithSoftDeletes() throws Exception {
+Directory dir = newDirectory();
Review comment:
Hmm I don't see where to officially +1 this in github, but thanks for
sharing these tests!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dnhatn commented on issue #1331: LUCENE-9268: Add some random tests to IndexWriter
dnhatn commented on issue #1331: LUCENE-9268: Add some random tests to IndexWriter URL: https://github.com/apache/lucene-solr/pull/1331#issuecomment-596128237 @msokolov Thanks for looking. I pushed https://github.com/apache/lucene-solr/pull/1331/commits/9e9b53271372d6a884b8de019c4aec7fed7dd97f to use try-with-resources for them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389306880
##
File path:
solr/core/src/test/org/apache/solr/handler/admin/ZookeeperStatusHandlerTest.java
##
@@ -74,6 +78,39 @@ public void tearDown() throws Exception {
super.tearDown();
}
+ @Test
+ public void testZkread() throws Exception {
+URL baseUrl = cluster.getJettySolrRunner(0).getBaseUrl();
+String basezk = baseUrl.toString().replace("/solr", "/api") +
"/cluster/zk";
+
+try( HttpSolrClient client = new
HttpSolrClient.Builder(baseUrl.toString()).build()) {
Review comment:
nit. Fix spaces
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389306592
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
+ private final CoreContainer coreContainer;
Review comment:
Do we need to keep the CoreContainer? is the zkClient enough?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389306574
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
Review comment:
The name of this class is not very representative. Also, all ZooKeeper
handlers are called `ZooKeeper...`, we should keep that for consistency
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389307304
##
File path:
solr/core/src/test/org/apache/solr/handler/admin/ZookeeperStatusHandlerTest.java
##
@@ -74,6 +78,39 @@ public void tearDown() throws Exception {
super.tearDown();
}
+ @Test
+ public void testZkread() throws Exception {
+URL baseUrl = cluster.getJettySolrRunner(0).getBaseUrl();
+String basezk = baseUrl.toString().replace("/solr", "/api") +
"/cluster/zk";
+
+try( HttpSolrClient client = new
HttpSolrClient.Builder(baseUrl.toString()).build()) {
+ Object o = Utils.executeGET(client.getHttpClient(),
+ basezk + "/security.json",
+ Utils.JSONCONSUMER );
+ assertNotNull(o);
+ o = Utils.executeGET(client.getHttpClient(),
+ basezk + "/configs",
+ Utils.JSONCONSUMER );
+ assertEquals("0", String.valueOf(getObjectByPath(o,true,
split(":/configs:_default:dataLength",':';
+ assertEquals("0", String.valueOf(getObjectByPath(o,true,
split(":/configs:conf:dataLength",':';
+ byte[] bytes = new byte[1024*5];
+ for (int i = 0; i < bytes.length; i++) {
+bytes[i] = (byte) random().nextInt(128);
Review comment:
why make things so complicated? cant you just put some static text? what are
you trying to achieve with this randomization?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389306716
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
+ private final CoreContainer coreContainer;
+
+ public ZkRead(CoreContainer coreContainer) {
+this.coreContainer = coreContainer;
+ }
+
+ @Command
+ public void get(SolrQueryRequest req, SolrQueryResponse rsp) {
+String path = req.getPathTemplateValues().get("*");
+if (path == null || path.isEmpty()) path = "/";
+byte[] d = null;
+try {
+ List l =
coreContainer.getZkController().getZkClient().getChildren(path, null, false);
+ if (l != null && !l.isEmpty()) {
+String prefix = path.endsWith("/") ? path : path + "/";
+
+rsp.add(path, (MapWriter) ew -> {
+ for (String s : l) {
+try {
+ Stat stat =
coreContainer.getZkController().getZkClient().exists(prefix + s, null, false);
+ ew.put(s, (MapWriter) ew1 -> {
+ew1.put("version", stat.getVersion());
+ew1.put("aversion", stat.getAversion());
+ew1.put("children", stat.getNumChildren());
+ew1.put("ctime", stat.getCtime());
+ew1.put("cversion", stat.getCversion());
+ew1.put("czxid", stat.getCzxid());
+ew1.put("ephemeralOwner", stat.getEphemeralOwner());
+ew1.put("mtime", stat.getMtime());
+ew1.put("mzxid", stat.getMzxid());
+ew1.put("pzxid", stat.getPzxid());
+ew1.put("dataLength", stat.getDataLength());
+ });
+} catch (Exception e) {
Review comment:
Why are you catching `Exception` here? instead of a subclass? Also, `exists`
throws interrupted exception, you should interrupt (and break)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389306787
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
+ private final CoreContainer coreContainer;
+
+ public ZkRead(CoreContainer coreContainer) {
+this.coreContainer = coreContainer;
+ }
+
+ @Command
+ public void get(SolrQueryRequest req, SolrQueryResponse rsp) {
+String path = req.getPathTemplateValues().get("*");
+if (path == null || path.isEmpty()) path = "/";
+byte[] d = null;
+try {
+ List l =
coreContainer.getZkController().getZkClient().getChildren(path, null, false);
+ if (l != null && !l.isEmpty()) {
+String prefix = path.endsWith("/") ? path : path + "/";
+
+rsp.add(path, (MapWriter) ew -> {
+ for (String s : l) {
+try {
+ Stat stat =
coreContainer.getZkController().getZkClient().exists(prefix + s, null, false);
+ ew.put(s, (MapWriter) ew1 -> {
+ew1.put("version", stat.getVersion());
+ew1.put("aversion", stat.getAversion());
+ew1.put("children", stat.getNumChildren());
+ew1.put("ctime", stat.getCtime());
+ew1.put("cversion", stat.getCversion());
+ew1.put("czxid", stat.getCzxid());
+ew1.put("ephemeralOwner", stat.getEphemeralOwner());
+ew1.put("mtime", stat.getMtime());
+ew1.put("mzxid", stat.getMzxid());
+ew1.put("pzxid", stat.getPzxid());
+ew1.put("dataLength", stat.getDataLength());
+ });
+} catch (Exception e) {
+ ew.put("s", Collections.singletonMap("error", e.getMessage()));
+}
+ }
+});
+
+ } else {
+d = coreContainer.getZkController().getZkClient().getData(path, null,
null, false);
+if (d == null || d.length == 0) {
+ rsp.add(path, null);
+ return;
+}
+
+Map map = new HashMap<>(1);
+map.put(WT, "raw");
+map.put(OMIT_HEADER, "true");
+req.setParams(SolrParams.wrapDefaults(new MapSolrParams(map),
req.getParams()));
+
+
+rsp.add(CONTENT, new ContentStreamBase.ByteArrayStream(d, null,
+d[0] == '{' ? CommonParams.JSON_MIME :
BinaryResponseParser.BINARY_CONTENT_TYPE));
+
+ }
+
+} catch (Exception e) {
Review comment:
Same as abo
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389306764
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
+ private final CoreContainer coreContainer;
+
+ public ZkRead(CoreContainer coreContainer) {
+this.coreContainer = coreContainer;
+ }
+
+ @Command
+ public void get(SolrQueryRequest req, SolrQueryResponse rsp) {
+String path = req.getPathTemplateValues().get("*");
+if (path == null || path.isEmpty()) path = "/";
+byte[] d = null;
+try {
+ List l =
coreContainer.getZkController().getZkClient().getChildren(path, null, false);
+ if (l != null && !l.isEmpty()) {
+String prefix = path.endsWith("/") ? path : path + "/";
+
+rsp.add(path, (MapWriter) ew -> {
+ for (String s : l) {
+try {
+ Stat stat =
coreContainer.getZkController().getZkClient().exists(prefix + s, null, false);
+ ew.put(s, (MapWriter) ew1 -> {
+ew1.put("version", stat.getVersion());
+ew1.put("aversion", stat.getAversion());
+ew1.put("children", stat.getNumChildren());
+ew1.put("ctime", stat.getCtime());
+ew1.put("cversion", stat.getCversion());
+ew1.put("czxid", stat.getCzxid());
+ew1.put("ephemeralOwner", stat.getEphemeralOwner());
+ew1.put("mtime", stat.getMtime());
+ew1.put("mzxid", stat.getMzxid());
+ew1.put("pzxid", stat.getPzxid());
+ew1.put("dataLength", stat.getDataLength());
+ });
+} catch (Exception e) {
+ ew.put("s", Collections.singletonMap("error", e.getMessage()));
+}
+ }
+});
+
+ } else {
+d = coreContainer.getZkController().getZkClient().getData(path, null,
null, false);
+if (d == null || d.length == 0) {
+ rsp.add(path, null);
+ return;
+}
+
+Map map = new HashMap<>(1);
+map.put(WT, "raw");
+map.put(OMIT_HEADER, "true");
+req.setParams(SolrParams.wrapDefaults(new MapSolrParams(map),
req.getParams()));
Review comment:
Why is this needed?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389306852
##
File path:
solr/core/src/test/org/apache/solr/handler/admin/ZookeeperStatusHandlerTest.java
##
@@ -74,6 +78,39 @@ public void tearDown() throws Exception {
super.tearDown();
}
+ @Test
+ public void testZkread() throws Exception {
Review comment:
This test doesn't belong in `ZookeeperStatusHandlerTest`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389306671
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
+ private final CoreContainer coreContainer;
+
+ public ZkRead(CoreContainer coreContainer) {
+this.coreContainer = coreContainer;
+ }
+
+ @Command
+ public void get(SolrQueryRequest req, SolrQueryResponse rsp) {
+String path = req.getPathTemplateValues().get("*");
+if (path == null || path.isEmpty()) path = "/";
+byte[] d = null;
+try {
+ List l =
coreContainer.getZkController().getZkClient().getChildren(path, null, false);
+ if (l != null && !l.isEmpty()) {
+String prefix = path.endsWith("/") ? path : path + "/";
+
+rsp.add(path, (MapWriter) ew -> {
+ for (String s : l) {
+try {
+ Stat stat =
coreContainer.getZkController().getZkClient().exists(prefix + s, null, false);
+ ew.put(s, (MapWriter) ew1 -> {
+ew1.put("version", stat.getVersion());
+ew1.put("aversion", stat.getAversion());
+ew1.put("children", stat.getNumChildren());
+ew1.put("ctime", stat.getCtime());
+ew1.put("cversion", stat.getCversion());
+ew1.put("czxid", stat.getCzxid());
+ew1.put("ephemeralOwner", stat.getEphemeralOwner());
+ew1.put("mtime", stat.getMtime());
+ew1.put("mzxid", stat.getMzxid());
+ew1.put("pzxid", stat.getPzxid());
+ew1.put("dataLength", stat.getDataLength());
+ });
+} catch (Exception e) {
+ ew.put("s", Collections.singletonMap("error", e.getMessage()));
Review comment:
you mean `"s"` or `s`? Also, the idea here is that we'll add an error per
child?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] tflobbe commented on a change in pull request #1327: SOLR-13942: /api/cluster/zk/* to fetch raw ZK data
tflobbe commented on a change in pull request #1327: SOLR-13942:
/api/cluster/zk/* to fetch raw ZK data
URL: https://github.com/apache/lucene-solr/pull/1327#discussion_r389307090
##
File path: solr/core/src/java/org/apache/solr/handler/admin/ZkRead.java
##
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.handler.admin;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.solr.api.Command;
+import org.apache.solr.api.EndPoint;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.BinaryResponseParser;
+import org.apache.solr.common.MapWriter;
+import org.apache.solr.common.params.CommonParams;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.zookeeper.data.Stat;
+
+import static org.apache.solr.common.params.CommonParams.OMIT_HEADER;
+import static org.apache.solr.common.params.CommonParams.WT;
+import static org.apache.solr.response.RawResponseWriter.CONTENT;
+import static
org.apache.solr.security.PermissionNameProvider.Name.COLL_READ_PERM;
+
+/**Exposes the content of the Zookeeper
+ * This is an expert feature that exposes the data inside the back end
zookeeper.This API may change or
+ * be removed in future versions.
+ * This is not a public API. The data that is returned is not guaranteed to
remain same
+ * across releases, as the data stored in Zookeeper may change from time to
time.
+ */
+@EndPoint(path = "/cluster/zk/*",
+method = SolrRequest.METHOD.GET,
+permission = COLL_READ_PERM)
+public class ZkRead {
+ private final CoreContainer coreContainer;
+
+ public ZkRead(CoreContainer coreContainer) {
+this.coreContainer = coreContainer;
+ }
+
+ @Command
+ public void get(SolrQueryRequest req, SolrQueryResponse rsp) {
+String path = req.getPathTemplateValues().get("*");
+if (path == null || path.isEmpty()) path = "/";
+byte[] d = null;
+try {
+ List l =
coreContainer.getZkController().getZkClient().getChildren(path, null, false);
+ if (l != null && !l.isEmpty()) {
+String prefix = path.endsWith("/") ? path : path + "/";
+
+rsp.add(path, (MapWriter) ew -> {
+ for (String s : l) {
+try {
+ Stat stat =
coreContainer.getZkController().getZkClient().exists(prefix + s, null, false);
+ ew.put(s, (MapWriter) ew1 -> {
+ew1.put("version", stat.getVersion());
+ew1.put("aversion", stat.getAversion());
+ew1.put("children", stat.getNumChildren());
+ew1.put("ctime", stat.getCtime());
+ew1.put("cversion", stat.getCversion());
+ew1.put("czxid", stat.getCzxid());
+ew1.put("ephemeralOwner", stat.getEphemeralOwner());
+ew1.put("mtime", stat.getMtime());
+ew1.put("mzxid", stat.getMzxid());
+ew1.put("pzxid", stat.getPzxid());
+ew1.put("dataLength", stat.getDataLength());
+ });
+} catch (Exception e) {
+ ew.put("s", Collections.singletonMap("error", e.getMessage()));
+}
+ }
+});
+
+ } else {
Review comment:
Should we consider the case where znodes have data and children?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[
https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054207#comment-17054207
]
Ishan Chattopadhyaya commented on SOLR-13749:
-
I think by ASAP, the urgency implied is Monday when [~romseygeek] will build
the 8.5 RC1. I think we should either fix this by then or revert it by then and
raise a fixed PR subsequently.
> Implement support for joining across collections with multiple shards ( XCJF )
> --
>
> Key: SOLR-13749
> URL: https://issues.apache.org/jira/browse/SOLR-13749
> Project: Solr
> Issue Type: New Feature
>Reporter: Kevin Watters
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> This ticket includes 2 query parsers.
> The first one is the "Cross collection join filter" (XCJF) parser. This is
> the "Cross-collection join filter" query parser. It can do a call out to a
> remote collection to get a set of join keys to be used as a filter against
> the local collection.
> The second one is the Hash Range query parser that you can specify a field
> name and a hash range, the result is that only the documents that would have
> hashed to that range will be returned.
> This query parser will do an intersection based on join keys between 2
> collections.
> The local collection is the collection that you are searching against.
> The remote collection is the collection that contains the join keys that you
> want to use as a filter.
> Each shard participating in the distributed request will execute a query
> against the remote collection. If the local collection is setup with the
> compositeId router to be routed on the join key field, a hash range query is
> applied to the remote collection query to only match the documents that
> contain a potential match for the documents that are in the local shard/core.
>
>
> Here's some vocab to help with the descriptions of the various parameters.
> ||Term||Description||
> |Local Collection|This is the main collection that is being queried.|
> |Remote Collection|This is the collection that the XCJFQuery will query to
> resolve the join keys.|
> |XCJFQuery|The lucene query that executes a search to get back a set of join
> keys from a remote collection|
> |HashRangeQuery|The lucene query that matches only the documents whose hash
> code on a field falls within a specified range.|
>
>
> ||Param ||Required ||Description||
> |collection|Required|The name of the external Solr collection to be queried
> to retrieve the set of join key values ( required )|
> |zkHost|Optional|The connection string to be used to connect to Zookeeper.
> zkHost and solrUrl are both optional parameters, and at most one of them
> should be specified.
> If neither of zkHost or solrUrl are specified, the local Zookeeper cluster
> will be used. ( optional )|
> |solrUrl|Optional|The URL of the external Solr node to be queried ( optional
> )|
> |from|Required|The join key field name in the external collection ( required
> )|
> |to|Required|The join key field name in the local collection|
> |v|See Note|The query to be executed against the external Solr collection to
> retrieve the set of join key values.
> Note: The original query can be passed at the end of the string or as the
> "v" parameter.
> It's recommended to use query parameter substitution with the "v" parameter
> to ensure no issues arise with the default query parsers.|
> |routed| |true / false. If true, the XCJF query will use each shard's hash
> range to determine the set of join keys to retrieve for that shard.
> This parameter improves the performance of the cross-collection join, but
> it depends on the local collection being routed by the toField. If this
> parameter is not specified,
> the XCJF query will try to determine the correct value automatically.|
> |ttl| |The length of time that an XCJF query in the cache will be considered
> valid, in seconds. Defaults to 3600 (one hour).
> The XCJF query will not be aware of changes to the remote collection, so
> if the remote collection is updated, cached XCJF queries may give inaccurate
> results.
> After the ttl period has expired, the XCJF query will re-execute the join
> against the remote collection.|
> |_All others_| |Any normal Solr parameter can also be specified as a local
> param.|
>
> Example Solr Config.xml changes:
>
> {{<}}{{cache}} {{name}}{{=}}{{"hash_vin"}}
> {{ }}{{class}}{{=}}{{"solr.LRUCache"}}
> {{ }}{{size}}{{=}}{{"128"}}
> {{ }}{{initialSize}}{{=}}{{"0"}}
> {{ }}{{regenerator}}{{=}}{{"solr.NoOpRegenerator"}}{{/>}}
>
> {{<}}{{queryParser}} {{name}}{{=}}{{"xcjf"}}
> {{class}}{{=}}{{"org.apache.solr
[jira] [Commented] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[ https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054212#comment-17054212 ] Gus Heck commented on SOLR-13749: - [~kwatters] Release branch has been cut already. RC1 is meant to be Monday. This is one of 2 remaining blockers. [https://marc.info/?l=lucene-dev&m=158333972314773&w=2] I see the following that has come up: # Use of existing shardsWhiteList which after some research looks suitable, but also looks like it will be difficult to obtain a reference to without changes to other classes. # Development of logic by which to detect the user's cross-core intent and switch to XCJF under the covers # merging the documentation into the join query parser documentation, with special attention to the fine details of when scoring is and isn't available. # Understand the implications of SOLR-13350 and ensure that this "plays nicely with that" # Can scoring of some sort be applied in the XCJF case. This might involve fiddling with StatsCaches and distributed IDF. I think 1-4 are necessary for the functionality to be released, and #5 ought to be a separate enhancement to this functionality (new Jira ticket). [~kwatters] If by some chance you think you can resolve 1-4 by noon on Monday, super cool, but otherwise I think there's enough (late breaking) feedback to be addressed here that we need to push this out to future versions. If you reply here that you'll try to meet that deadline I'll leave things and check back on monday. If I don't hear by Noon on Sunday I'll remove this from 8.5 and adjust the fix version to 8.6. It seems sort of unclean to me to put things out in an undocumented state, though I know [~dsmiley] favors that route. I favor C with the caveat that I don't think its necessary to remove from 8x now that we have an 8.5 branch. However I don't feel terribly strongly about it. If others chime in and A or B are more popular I'm happy to do that for 8.5 too. For reference his list of options is: - Option A: document in an obvious way (i.e. some call-out box) that the name & parameters will likely change without back-compat. In the project we sometimes throw out the word "experimental" a lot but here I'm just claiming the syntax/way it's invoked will change; I'm making no quality judgement on what's underneath. - Option B: comment it out making it invisible - Option C: remove from 8x/8.5; leave in master > Implement support for joining across collections with multiple shards ( XCJF ) > -- > > Key: SOLR-13749 > URL: https://issues.apache.org/jira/browse/SOLR-13749 > Project: Solr > Issue Type: New Feature >Reporter: Kevin Watters >Assignee: Gus Heck >Priority: Blocker > Fix For: 8.5 > > Time Spent: 1.5h > Remaining Estimate: 0h > > This ticket includes 2 query parsers. > The first one is the "Cross collection join filter" (XCJF) parser. This is > the "Cross-collection join filter" query parser. It can do a call out to a > remote collection to get a set of join keys to be used as a filter against > the local collection. > The second one is the Hash Range query parser that you can specify a field > name and a hash range, the result is that only the documents that would have > hashed to that range will be returned. > This query parser will do an intersection based on join keys between 2 > collections. > The local collection is the collection that you are searching against. > The remote collection is the collection that contains the join keys that you > want to use as a filter. > Each shard participating in the distributed request will execute a query > against the remote collection. If the local collection is setup with the > compositeId router to be routed on the join key field, a hash range query is > applied to the remote collection query to only match the documents that > contain a potential match for the documents that are in the local shard/core. > > > Here's some vocab to help with the descriptions of the various parameters. > ||Term||Description|| > |Local Collection|This is the main collection that is being queried.| > |Remote Collection|This is the collection that the XCJFQuery will query to > resolve the join keys.| > |XCJFQuery|The lucene query that executes a search to get back a set of join > keys from a remote collection| > |HashRangeQuery|The lucene query that matches only the documents whose hash > code on a field falls within a specified range.| > > > ||Param ||Required ||Description|| > |collection|Required|The name of the external Solr collection to be queried > to retrieve the set of join key values ( required )| > |zkHost|Optional|The connection string to be used to connect to Zookeeper. > zkHost and solrUrl are b
[jira] [Commented] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[
https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054215#comment-17054215
]
Ishan Chattopadhyaya commented on SOLR-13749:
-
{quote} * Option C: remove from 8x/8.5; leave in master{quote}
+1 to Option C.
> Implement support for joining across collections with multiple shards ( XCJF )
> --
>
> Key: SOLR-13749
> URL: https://issues.apache.org/jira/browse/SOLR-13749
> Project: Solr
> Issue Type: New Feature
>Reporter: Kevin Watters
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> This ticket includes 2 query parsers.
> The first one is the "Cross collection join filter" (XCJF) parser. This is
> the "Cross-collection join filter" query parser. It can do a call out to a
> remote collection to get a set of join keys to be used as a filter against
> the local collection.
> The second one is the Hash Range query parser that you can specify a field
> name and a hash range, the result is that only the documents that would have
> hashed to that range will be returned.
> This query parser will do an intersection based on join keys between 2
> collections.
> The local collection is the collection that you are searching against.
> The remote collection is the collection that contains the join keys that you
> want to use as a filter.
> Each shard participating in the distributed request will execute a query
> against the remote collection. If the local collection is setup with the
> compositeId router to be routed on the join key field, a hash range query is
> applied to the remote collection query to only match the documents that
> contain a potential match for the documents that are in the local shard/core.
>
>
> Here's some vocab to help with the descriptions of the various parameters.
> ||Term||Description||
> |Local Collection|This is the main collection that is being queried.|
> |Remote Collection|This is the collection that the XCJFQuery will query to
> resolve the join keys.|
> |XCJFQuery|The lucene query that executes a search to get back a set of join
> keys from a remote collection|
> |HashRangeQuery|The lucene query that matches only the documents whose hash
> code on a field falls within a specified range.|
>
>
> ||Param ||Required ||Description||
> |collection|Required|The name of the external Solr collection to be queried
> to retrieve the set of join key values ( required )|
> |zkHost|Optional|The connection string to be used to connect to Zookeeper.
> zkHost and solrUrl are both optional parameters, and at most one of them
> should be specified.
> If neither of zkHost or solrUrl are specified, the local Zookeeper cluster
> will be used. ( optional )|
> |solrUrl|Optional|The URL of the external Solr node to be queried ( optional
> )|
> |from|Required|The join key field name in the external collection ( required
> )|
> |to|Required|The join key field name in the local collection|
> |v|See Note|The query to be executed against the external Solr collection to
> retrieve the set of join key values.
> Note: The original query can be passed at the end of the string or as the
> "v" parameter.
> It's recommended to use query parameter substitution with the "v" parameter
> to ensure no issues arise with the default query parsers.|
> |routed| |true / false. If true, the XCJF query will use each shard's hash
> range to determine the set of join keys to retrieve for that shard.
> This parameter improves the performance of the cross-collection join, but
> it depends on the local collection being routed by the toField. If this
> parameter is not specified,
> the XCJF query will try to determine the correct value automatically.|
> |ttl| |The length of time that an XCJF query in the cache will be considered
> valid, in seconds. Defaults to 3600 (one hour).
> The XCJF query will not be aware of changes to the remote collection, so
> if the remote collection is updated, cached XCJF queries may give inaccurate
> results.
> After the ttl period has expired, the XCJF query will re-execute the join
> against the remote collection.|
> |_All others_| |Any normal Solr parameter can also be specified as a local
> param.|
>
> Example Solr Config.xml changes:
>
> {{<}}{{cache}} {{name}}{{=}}{{"hash_vin"}}
> {{ }}{{class}}{{=}}{{"solr.LRUCache"}}
> {{ }}{{size}}{{=}}{{"128"}}
> {{ }}{{initialSize}}{{=}}{{"0"}}
> {{ }}{{regenerator}}{{=}}{{"solr.NoOpRegenerator"}}{{/>}}
>
> {{<}}{{queryParser}} {{name}}{{=}}{{"xcjf"}}
> {{class}}{{=}}{{"org.apache.solr.search.join.XCJFQueryParserPlugin"}}{{>}}
> {{ }}{{<}}{{str}} {{name}}{{=}}{{"routerField"}}{{>vin}}
> {{}}
[jira] [Commented] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[
https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054217#comment-17054217
]
Ishan Chattopadhyaya commented on SOLR-13749:
-
{quote} * Understand the implications of SOLR-13350 and ensure that this "plays
nicely with that"{quote}
Please don't consider it a blocker. Essentially, in JoinQParserPlugin, the
fromCore and fromSearcher were being created once, but the core closing was
happening for every thread. Concern is to handle refcounted resources properly,
as I did in the patch/PR for SOLR-13350. Conceptually, I don't think this will
be much of an issue here (because I can't think of refcounted resources like
core or searcher would be opened on a per query basis, but I could be wrong
too), but I still haven't looked closely.
> Implement support for joining across collections with multiple shards ( XCJF )
> --
>
> Key: SOLR-13749
> URL: https://issues.apache.org/jira/browse/SOLR-13749
> Project: Solr
> Issue Type: New Feature
>Reporter: Kevin Watters
>Assignee: Gus Heck
>Priority: Blocker
> Fix For: 8.5
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> This ticket includes 2 query parsers.
> The first one is the "Cross collection join filter" (XCJF) parser. This is
> the "Cross-collection join filter" query parser. It can do a call out to a
> remote collection to get a set of join keys to be used as a filter against
> the local collection.
> The second one is the Hash Range query parser that you can specify a field
> name and a hash range, the result is that only the documents that would have
> hashed to that range will be returned.
> This query parser will do an intersection based on join keys between 2
> collections.
> The local collection is the collection that you are searching against.
> The remote collection is the collection that contains the join keys that you
> want to use as a filter.
> Each shard participating in the distributed request will execute a query
> against the remote collection. If the local collection is setup with the
> compositeId router to be routed on the join key field, a hash range query is
> applied to the remote collection query to only match the documents that
> contain a potential match for the documents that are in the local shard/core.
>
>
> Here's some vocab to help with the descriptions of the various parameters.
> ||Term||Description||
> |Local Collection|This is the main collection that is being queried.|
> |Remote Collection|This is the collection that the XCJFQuery will query to
> resolve the join keys.|
> |XCJFQuery|The lucene query that executes a search to get back a set of join
> keys from a remote collection|
> |HashRangeQuery|The lucene query that matches only the documents whose hash
> code on a field falls within a specified range.|
>
>
> ||Param ||Required ||Description||
> |collection|Required|The name of the external Solr collection to be queried
> to retrieve the set of join key values ( required )|
> |zkHost|Optional|The connection string to be used to connect to Zookeeper.
> zkHost and solrUrl are both optional parameters, and at most one of them
> should be specified.
> If neither of zkHost or solrUrl are specified, the local Zookeeper cluster
> will be used. ( optional )|
> |solrUrl|Optional|The URL of the external Solr node to be queried ( optional
> )|
> |from|Required|The join key field name in the external collection ( required
> )|
> |to|Required|The join key field name in the local collection|
> |v|See Note|The query to be executed against the external Solr collection to
> retrieve the set of join key values.
> Note: The original query can be passed at the end of the string or as the
> "v" parameter.
> It's recommended to use query parameter substitution with the "v" parameter
> to ensure no issues arise with the default query parsers.|
> |routed| |true / false. If true, the XCJF query will use each shard's hash
> range to determine the set of join keys to retrieve for that shard.
> This parameter improves the performance of the cross-collection join, but
> it depends on the local collection being routed by the toField. If this
> parameter is not specified,
> the XCJF query will try to determine the correct value automatically.|
> |ttl| |The length of time that an XCJF query in the cache will be considered
> valid, in seconds. Defaults to 3600 (one hour).
> The XCJF query will not be aware of changes to the remote collection, so
> if the remote collection is updated, cached XCJF queries may give inaccurate
> results.
> After the ttl period has expired, the XCJF query will re-execute the join
> against the remote collection.|
> |_All others_| |Any normal Sol
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task
mocobeta commented on a change in pull request #1304: LUCENE-9242: generate
javadocs by calling Ant javadoc task
URL: https://github.com/apache/lucene-solr/pull/1304#discussion_r389316120
##
File path: gradle/invoke-javadoc.gradle
##
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// invoke javadoc tool
+
+allprojects {
+
+ ext {
Review comment:
moved `ext{...}` under plugins.withType().
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task
mocobeta commented on a change in pull request #1304: LUCENE-9242: generate
javadocs by calling Ant javadoc task
URL: https://github.com/apache/lucene-solr/pull/1304#discussion_r389316224
##
File path: gradle/invoke-javadoc.gradle
##
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// invoke javadoc tool
+
+allprojects {
+
+ ext {
+javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
+javadocDestDir = "${javadocRoot}/${project.name}"
+ }
+
+ plugins.withType(JavaPlugin) {
+def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
+def title = "${libName} ${project.version} ${project.name} API".toString()
+def srcDirs = sourceSets.main.java.srcDirs.findAll { dir -> dir.exists() }
Review comment:
moved `def srcDirs = ...` into doFirst.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task
mocobeta commented on a change in pull request #1304: LUCENE-9242: generate
javadocs by calling Ant javadoc task
URL: https://github.com/apache/lucene-solr/pull/1304#discussion_r389316429
##
File path: gradle/invoke-javadoc.gradle
##
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// invoke javadoc tool
+
+allprojects {
+
+ ext {
+javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
+javadocDestDir = "${javadocRoot}/${project.name}"
+ }
+
+ plugins.withType(JavaPlugin) {
+def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
+def title = "${libName} ${project.version} ${project.name} API".toString()
+def srcDirs = sourceSets.main.java.srcDirs.findAll { dir -> dir.exists() }
+
+task invokeJavadoc {
Review comment:
changed the name to `renderJavadoc`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task
mocobeta commented on a change in pull request #1304: LUCENE-9242: generate
javadocs by calling Ant javadoc task
URL: https://github.com/apache/lucene-solr/pull/1304#discussion_r389316645
##
File path: gradle/invoke-javadoc.gradle
##
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// invoke javadoc tool
+
+allprojects {
+
+ ext {
+javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
+javadocDestDir = "${javadocRoot}/${project.name}"
+ }
+
+ plugins.withType(JavaPlugin) {
+def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
+def title = "${libName} ${project.version} ${project.name} API".toString()
+def srcDirs = sourceSets.main.java.srcDirs.findAll { dir -> dir.exists() }
+
+task invokeJavadoc {
+ description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ group "documentation"
+
+ dependsOn sourceSets.main.compileClasspath
+
+ inputs.property("linksource", "no")
Review comment:
removed the inputs properties and defined them as the task ext properties.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task
mocobeta commented on a change in pull request #1304: LUCENE-9242: generate
javadocs by calling Ant javadoc task
URL: https://github.com/apache/lucene-solr/pull/1304#discussion_r389316672
##
File path: gradle/invoke-javadoc.gradle
##
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// invoke javadoc tool
+
+allprojects {
+
+ ext {
+javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
+javadocDestDir = "${javadocRoot}/${project.name}"
+ }
+
+ plugins.withType(JavaPlugin) {
+def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
+def title = "${libName} ${project.version} ${project.name} API".toString()
+def srcDirs = sourceSets.main.java.srcDirs.findAll { dir -> dir.exists() }
+
+task invokeJavadoc {
+ description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ group "documentation"
+
+ dependsOn sourceSets.main.compileClasspath
+
+ inputs.property("linksource", "no")
+ inputs.property("linkJUnit", false)
+ inputs.property("linkHref", [])
+
+ inputs.files sourceSets.main.java.asFileTree
Review comment:
changed to `inputs.files { sourceSets.main.java.asFileTree }`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task
mocobeta commented on a change in pull request #1304: LUCENE-9242: generate
javadocs by calling Ant javadoc task
URL: https://github.com/apache/lucene-solr/pull/1304#discussion_r389317438
##
File path: gradle/invoke-javadoc.gradle
##
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// invoke javadoc tool
+
+allprojects {
+
+ ext {
+javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
+javadocDestDir = "${javadocRoot}/${project.name}"
+ }
+
+ plugins.withType(JavaPlugin) {
+def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
+def title = "${libName} ${project.version} ${project.name} API".toString()
+def srcDirs = sourceSets.main.java.srcDirs.findAll { dir -> dir.exists() }
+
+task invokeJavadoc {
+ description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ group "documentation"
+
+ dependsOn sourceSets.main.compileClasspath
+
+ inputs.property("linksource", "no")
+ inputs.property("linkJUnit", false)
+ inputs.property("linkHref", [])
+
+ inputs.files sourceSets.main.java.asFileTree
+ outputs.dir project.javadocRoot
+
+ doFirst {
+srcDirs.each { srcDir ->
Review comment:
Instead of repeatly invoking ant.javadoc, I added (possibly multiple)
`packageset`s to the javadoc task. For now, all projects have only one main
source directory, so only one packageset element will be created by this code.
```
srcDirs.collect { srcDir ->
packageset(dir: srcDir)
}
```
Does this change make sense?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task
mocobeta commented on a change in pull request #1304: LUCENE-9242: generate
javadocs by calling Ant javadoc task
URL: https://github.com/apache/lucene-solr/pull/1304#discussion_r389318421
##
File path: gradle/invoke-javadoc.gradle
##
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// invoke javadoc tool
+
+allprojects {
+
+ ext {
+javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
+javadocDestDir = "${javadocRoot}/${project.name}"
+ }
+
+ plugins.withType(JavaPlugin) {
+def libName = project.path.startsWith(":lucene") ? "Lucene" : "Solr"
+def title = "${libName} ${project.version} ${project.name} API".toString()
+def srcDirs = sourceSets.main.java.srcDirs.findAll { dir -> dir.exists() }
+
+task invokeJavadoc {
+ description "Generates Javadoc API documentation for the main source
code. This invokes Ant Javadoc Task."
+ group "documentation"
+
+ dependsOn sourceSets.main.compileClasspath
+
+ inputs.property("linksource", "no")
+ inputs.property("linkJUnit", false)
+ inputs.property("linkHref", [])
+
+ inputs.files sourceSets.main.java.asFileTree
+ outputs.dir project.javadocRoot
+
+ doFirst {
+srcDirs.each { srcDir ->
+ ant.javadoc(
+ overview: file("${srcDir}/overview.html"),
+ packagenames: "org.apache.lucene.*,org.apache.solr.*",
+ destDir: project.javadocDestDir,
+ access: "protected",
+ encoding: "UTF-8",
+ charset: "UTF-8",
+ docencoding: "UTF-8",
+ noindex: "true",
+ includenosourcepackages: "true",
+ author: "true",
+ version: "true",
+ linksource: inputs.properties.linksource,
+ use: "true",
+ failonerror: "true",
+ locale: "en_US",
+ windowtitle: title,
+ doctitle: title,
+ maxmemory: "512m",
+ classpath: sourceSets.main.compileClasspath.asPath,
+ bottom: "Copyright © 2000-${buildYear} Apache Software
Foundation. All Rights Reserved."
+ ) {
+packageset(dir: srcDir)
+
+tag(name: "lucene.experimental", description: "WARNING: This API
is experimental and might change in incompatible ways in the next release.")
+tag(name: "lucene.internal", description: "NOTE: This API is for
internal purposes only and might change in incompatible ways in the next
release.")
+tag(name: "lucene.spi", description: "SPI Name (Note: This is
case-insensitive. e.g., if the name is 'htmlStrip', 'htmlstrip' can be used
when looking up the service):", scope: "types")
+
+// resolve links to JavaSE and JUnit API
+link(offline: "true", href:
"https://docs.oracle.com/en/java/javase/11/docs/api/";, packageListLoc:
project(":lucene").file("tools/javadoc/java11/").toString())
+if (inputs.properties.get("linkJUnit")) {
+ link(offline: "true", href:
"https://junit.org/junit4/javadoc/4.12/";, packageListLoc:
project(":lucene").file("tools/javadoc/junit").toString())
+}
+// resolve inter-module links if 'linkHref' property is specified
+inputs.properties.get("linkHref").each { href ->
+ link(href: href)
+}
+
+arg(line: "--release 11")
+arg(line: "-Xdoclint:all,-missing")
+
+// force locale to be "en_US" (fix for:
https://bugs.openjdk.java.net/browse/JDK-8222793)
+arg(line: "-J-Duser.language=en -J-Duser.country=US")
+ }
+}
+
+// append some special table css, prettify css
+ant.concat(destfile: "${javadocDestDir}/stylesheet.css", append:
"true", fixlastline: "true", encoding: "UTF-8") {
+ filelist(dir: project(":lucene").file("tools/javadoc"), files:
"table_padding.css")
+ filelist(dir: project(":lucene").file("tools/prettify"), files:
"prettify.css")
+}
+// append prettify to scripts
+ant.concat(destfile: "${javadocDest
[GitHub] [lucene-solr] mocobeta commented on a change in pull request #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task
mocobeta commented on a change in pull request #1304: LUCENE-9242: generate
javadocs by calling Ant javadoc task
URL: https://github.com/apache/lucene-solr/pull/1304#discussion_r389318980
##
File path: gradle/invoke-javadoc.gradle
##
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// invoke javadoc tool
+
+allprojects {
+
+ ext {
+javadocRoot = project.path.startsWith(':lucene') ?
project(':lucene').file("build/docs") : project(':solr').file("build/docs")
Review comment:
yes we will be able to fix this problem in a follow-up issue...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta commented on issue #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task
mocobeta commented on issue #1304: LUCENE-9242: generate javadocs by calling Ant javadoc task URL: https://github.com/apache/lucene-solr/pull/1304#issuecomment-596146747 Hi @dweiss, I updated the scripts and added some replies to your comments. Can you please take a look again? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-13842) Remove wt=json from Implicit API definition's defaults
[
https://issues.apache.org/jira/browse/SOLR-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Munendra S N reassigned SOLR-13842:
---
Assignee: Munendra S N
> Remove wt=json from Implicit API definition's defaults
> --
>
> Key: SOLR-13842
> URL: https://issues.apache.org/jira/browse/SOLR-13842
> Project: Solr
> Issue Type: Task
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Minor
> Labels: newdev
> Attachments: SOLR-13842.patch
>
>
> From solr 7, {{json}} is the default response writer. So, {{wt=json}} can be
> removed from implicit API definitions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13842) Remove wt=json from Implicit API definition's defaults
[
https://issues.apache.org/jira/browse/SOLR-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Munendra S N updated SOLR-13842:
Status: Patch Available (was: Open)
> Remove wt=json from Implicit API definition's defaults
> --
>
> Key: SOLR-13842
> URL: https://issues.apache.org/jira/browse/SOLR-13842
> Project: Solr
> Issue Type: Task
>Reporter: Munendra S N
>Priority: Minor
> Labels: newdev
> Attachments: SOLR-13842.patch
>
>
> From solr 7, {{json}} is the default response writer. So, {{wt=json}} can be
> removed from implicit API definitions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13842) Remove wt=json from Implicit API definition's defaults
[
https://issues.apache.org/jira/browse/SOLR-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054306#comment-17054306
]
Munendra S N commented on SOLR-13842:
-
[^SOLR-13842.patch]
initial patch
> Remove wt=json from Implicit API definition's defaults
> --
>
> Key: SOLR-13842
> URL: https://issues.apache.org/jira/browse/SOLR-13842
> Project: Solr
> Issue Type: Task
>Reporter: Munendra S N
>Priority: Minor
> Labels: newdev
> Attachments: SOLR-13842.patch
>
>
> From solr 7, {{json}} is the default response writer. So, {{wt=json}} can be
> removed from implicit API definitions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13842) Remove wt=json from Implicit API definition's defaults
[
https://issues.apache.org/jira/browse/SOLR-13842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Munendra S N updated SOLR-13842:
Attachment: SOLR-13842.patch
> Remove wt=json from Implicit API definition's defaults
> --
>
> Key: SOLR-13842
> URL: https://issues.apache.org/jira/browse/SOLR-13842
> Project: Solr
> Issue Type: Task
>Reporter: Munendra S N
>Priority: Minor
> Labels: newdev
> Attachments: SOLR-13842.patch
>
>
> From solr 7, {{json}} is the default response writer. So, {{wt=json}} can be
> removed from implicit API definitions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
