[jira] [Updated] (LUCENE-9328) SortingGroupHead to reuse DocValues
[ https://issues.apache.org/jira/browse/LUCENE-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated LUCENE-9328: - Attachment: LUCENE-9328.patch Status: Patch Available (was: Patch Available) > SortingGroupHead to reuse DocValues > --- > > Key: LUCENE-9328 > URL: https://issues.apache.org/jira/browse/LUCENE-9328 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/grouping >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Minor > Attachments: LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, > LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > That's why > https://issues.apache.org/jira/browse/LUCENE-7701?focusedCommentId=17084365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17084365 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9328) SortingGroupHead to reuse DocValues
[ https://issues.apache.org/jira/browse/LUCENE-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated LUCENE-9328: - Attachment: (was: LUCENE-9328.patch) > SortingGroupHead to reuse DocValues > --- > > Key: LUCENE-9328 > URL: https://issues.apache.org/jira/browse/LUCENE-9328 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/grouping >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Minor > Attachments: LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, > LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > That's why > https://issues.apache.org/jira/browse/LUCENE-7701?focusedCommentId=17084365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17084365 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9328) SortingGroupHead to reuse DocValues
[ https://issues.apache.org/jira/browse/LUCENE-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated LUCENE-9328: - Attachment: LUCENE-9328.patch > SortingGroupHead to reuse DocValues > --- > > Key: LUCENE-9328 > URL: https://issues.apache.org/jira/browse/LUCENE-9328 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/grouping >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Minor > Attachments: LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, > LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > That's why > https://issues.apache.org/jira/browse/LUCENE-7701?focusedCommentId=17084365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17084365 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-14423) static caches in StreamHandler ought to move to CoreContainer lifecycle
[ https://issues.apache.org/jira/browse/SOLR-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Bialecki reassigned SOLR-14423: --- Assignee: Andrzej Bialecki > static caches in StreamHandler ought to move to CoreContainer lifecycle > --- > > Key: SOLR-14423 > URL: https://issues.apache.org/jira/browse/SOLR-14423 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: streaming expressions >Reporter: David Smiley >Assignee: Andrzej Bialecki >Priority: Major > > StreamHandler (at "/stream") has several statically declared caches. I think > this is problematic, such as in testing wherein multiple nodes could be in > the same JVM. One of them is more serious -- SolrClientCache which is > closed/cleared via a SolrCore close hook. That's bad for performance but > also dangerous since another core might want to use one of these clients! > CC [~jbernste] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14457) SolrClient leaks connections on compressed responses if the response is malformed
[ https://issues.apache.org/jira/browse/SOLR-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099650#comment-17099650 ] Roger Lehmann commented on SOLR-14457: -- I can confirm this issue happening even with Solr 8.5.0 and standalone mode. Also I came to the same conclusion, the exception is fired but handled silently, i.e. simply skipped. Properly closing the connection even when the exception occurs is the preferred solution form my point of view. > SolrClient leaks connections on compressed responses if the response is > malformed > - > > Key: SOLR-14457 > URL: https://issues.apache.org/jira/browse/SOLR-14457 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Affects Versions: 7.7.2 > Environment: Solr version: 7.7.2 > Solr cloud enabled > Cluster topology: 6 nodes, 1 single collection, 10 shards and 3 replicas. 1 > HTTP LB using > Round Robin over all nodes > All cluster nodes have gzip enabled for all paths, all HTTP verbs and all > MIME types. > Solr client: HttpSolrClient targeting the HTTP LB >Reporter: Samuel García Martínez >Priority: Major > > h3. Summary > When the SolrJ receives a malformed response Entity, for example like the one > described in SOLR-14456, the client leaks the connection forever as it's > never released back to the pool. > h3. Problem description > HttpSolrClient should have compression enabled, so it uses the compression > interceptors. > When the request is marked with "Content-Encoding: gzip" but for whatever > reason the response is not in GZIP format, when HttpSolrClient tries to > close the connection using Utils.consumeFully(), it tries to create the > GzipInputStream failing and throwing an Exception. The exception thrown makes > it impossible to access the underlying InputStream to be closed, therefore > the connection is leaked. > Despite that the content in the response should honour the headers specified > for it, SolrJ should be reliable enough to prevent the connection leak when > this scenario happens. On top of the bug itself, not being able to set a > timeout while waiting for a connection to be available, makes any application > unresponsive as it will run out of threads eventually. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection
[ https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Endika Posadas updated SOLR-14458: -- Attachment: image-2020-05-05-09-47-27-854.png > Solr Replica locked in recovering state after a Zookeeper disconnection > --- > > Key: SOLR-14458 > URL: https://issues.apache.org/jira/browse/SOLR-14458 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.4.1 > Environment: A Solr cluster with 2 replicas that each has 2 shards > split across 2 Windows VMS. > They use a 3 replica zookeeper across 3 vms. >Reporter: Endika Posadas >Priority: Major > Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, > solr-thread-dump.log, solr.log, solrrecovering.png > > > In a solr cluster, a Solr instance containing two shards has lost connection > with zookeeper. Upon reconnecting, it has checked the status with the leader > and start a recovery. However, it's stuck in recovering status without making > further progress (has been like that for days now). > > Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is > trying to acquire the lock to createa new Index Writer: `at > org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)` > ( > after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the > ReentrantLock it's waiting for is never released. Moreover, no thread can be > found holding the lock, leaving restarting Solr as the only solution. > There is no Error in the logs that can help with the issue. I have attached > solr.log and a grep with node 7 lines, as well as a thread dump. > > There is also no other recovery currently running. In Solr metrics, 4 > recoveries have started, 3 have completed and 1 is running (forever). > > My hypothesis is that > org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore, > boolean) was called once but for some reason openIndexWriter was skipped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection
[ https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099673#comment-17099673 ] Endika Posadas commented on SOLR-14458: --- Further investigating, it seems like the readLock hasn't been released, so the write lock is unable to acquire the lock. !image-2020-05-05-09-47-27-854.png! > Solr Replica locked in recovering state after a Zookeeper disconnection > --- > > Key: SOLR-14458 > URL: https://issues.apache.org/jira/browse/SOLR-14458 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.4.1 > Environment: A Solr cluster with 2 replicas that each has 2 shards > split across 2 Windows VMS. > They use a 3 replica zookeeper across 3 vms. >Reporter: Endika Posadas >Priority: Major > Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, > solr-thread-dump.log, solr.log, solrrecovering.png > > > In a solr cluster, a Solr instance containing two shards has lost connection > with zookeeper. Upon reconnecting, it has checked the status with the leader > and start a recovery. However, it's stuck in recovering status without making > further progress (has been like that for days now). > > Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is > trying to acquire the lock to createa new Index Writer: `at > org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)` > ( > after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the > ReentrantLock it's waiting for is never released. Moreover, no thread can be > found holding the lock, leaving restarting Solr as the only solution. > There is no Error in the logs that can help with the issue. I have attached > solr.log and a grep with node 7 lines, as well as a thread dump. > > There is also no other recovery currently running. In Solr metrics, 4 > recoveries have started, 3 have completed and 1 is running (forever). > > My hypothesis is that > org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore, > boolean) was called once but for some reason openIndexWriter was skipped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14457) SolrClient leaks connections on compressed responses if the response is malformed
[ https://issues.apache.org/jira/browse/SOLR-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099650#comment-17099650 ] Roger Lehmann edited comment on SOLR-14457 at 5/5/20, 8:58 AM: --- I can confirm this issue happening even with Solr 8.5.0 and standalone mode. Also I came to the same conclusion, the exception is fired but handled silently, i.e. simply skipped. This happens for us also for seemingly correctly formed responses, our application could handle the data and also the unzipped data from a curl call seemed to be correct. Properly closing the connection even when the exception occurs is the preferred solution form my point of view. was (Author: roger.lehmann): I can confirm this issue happening even with Solr 8.5.0 and standalone mode. Also I came to the same conclusion, the exception is fired but handled silently, i.e. simply skipped. Properly closing the connection even when the exception occurs is the preferred solution form my point of view. > SolrClient leaks connections on compressed responses if the response is > malformed > - > > Key: SOLR-14457 > URL: https://issues.apache.org/jira/browse/SOLR-14457 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Affects Versions: 7.7.2 > Environment: Solr version: 7.7.2 > Solr cloud enabled > Cluster topology: 6 nodes, 1 single collection, 10 shards and 3 replicas. 1 > HTTP LB using > Round Robin over all nodes > All cluster nodes have gzip enabled for all paths, all HTTP verbs and all > MIME types. > Solr client: HttpSolrClient targeting the HTTP LB >Reporter: Samuel García Martínez >Priority: Major > > h3. Summary > When the SolrJ receives a malformed response Entity, for example like the one > described in SOLR-14456, the client leaks the connection forever as it's > never released back to the pool. > h3. Problem description > HttpSolrClient should have compression enabled, so it uses the compression > interceptors. > When the request is marked with "Content-Encoding: gzip" but for whatever > reason the response is not in GZIP format, when HttpSolrClient tries to > close the connection using Utils.consumeFully(), it tries to create the > GzipInputStream failing and throwing an Exception. The exception thrown makes > it impossible to access the underlying InputStream to be closed, therefore > the connection is leaked. > Despite that the content in the response should honour the headers specified > for it, SolrJ should be reliable enough to prevent the connection leak when > this scenario happens. On top of the bug itself, not being able to set a > timeout while waiting for a connection to be available, makes any application > unresponsive as it will run out of threads eventually. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9328) SortingGroupHead to reuse DocValues
[ https://issues.apache.org/jira/browse/LUCENE-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099747#comment-17099747 ] Lucene/Solr QA commented on LUCENE-9328: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 24s{color} | {color:red} lucene_grouping generated 1 new + 100 unchanged - 0 fixed = 101 total (was 100) {color} | | {color:green}+1{color} | {color:green} Release audit (RAT) {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Check forbidden APIs {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Validate source patterns {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 8s{color} | {color:green} grouping in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 13s{color} | {color:red} join in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 33s{color} | {color:green} test-framework in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 52s{color} | {color:red} core in the patch failed. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 53m 24s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | lucene.search.join.TestBlockJoinSelector | | | solr.TestGroupingSearch | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | LUCENE-9328 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/13002039/LUCENE-9328.patch | | Optional Tests | compile javac unit ratsources checkforbiddenapis validatesourcepatterns | | uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | ant | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh | | git revision | master / 6f775bfa69d | | ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 | | Default Java | LTS | | javac | https://builds.apache.org/job/PreCommit-LUCENE-Build/269/artifact/out/diff-compile-javac-lucene_grouping.txt | | unit | https://builds.apache.org/job/PreCommit-LUCENE-Build/269/artifact/out/patch-unit-lucene_join.txt | | unit | https://builds.apache.org/job/PreCommit-LUCENE-Build/269/artifact/out/patch-unit-solr_core.txt | | Test Results | https://builds.apache.org/job/PreCommit-LUCENE-Build/269/testReport/ | | modules | C: lucene/grouping lucene/join lucene/test-framework solr/core U: . | | Console output | https://builds.apache.org/job/PreCommit-LUCENE-Build/269/console | | Powered by | Apache Yetus 0.7.0 http://yetus.apache.org | This message was automatically generated. > SortingGroupHead to reuse DocValues > --- > > Key: LUCENE-9328 > URL: https://issues.apache.org/jira/browse/LUCENE-9328 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/grouping >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Minor > Attachments: LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, > LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > > That's why > https://issues.apache.org/jira/browse/LUCENE-7701?focusedCommentId=17084365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17084365 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9278) Make javadoc folder structure follow Gradle project path
[ https://issues.apache.org/jira/browse/LUCENE-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099748#comment-17099748 ] ASF subversion and git services commented on LUCENE-9278: - Commit 9ee8aa61de4f3157e40cd7af6f767b454ac46ccf in lucene-solr's branch refs/heads/master from Uwe Schindler [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9ee8aa6 ] LUCENE-9278: Fix passing of Java properties for locale: The arguments must be separated. > Make javadoc folder structure follow Gradle project path > > > Key: LUCENE-9278 > URL: https://issues.apache.org/jira/browse/LUCENE-9278 > Project: Lucene - Core > Issue Type: Task > Components: general/build >Reporter: Tomoko Uchida >Assignee: Tomoko Uchida >Priority: Major > Fix For: master (9.0) > > Time Spent: 7h 10m > Remaining Estimate: 0h > > Current javadoc folder structure is derived from Ant project name. e.g.: > [https://lucene.apache.org/core/8_4_1/analyzers-icu/index.html] > [https://lucene.apache.org/solr/8_4_1/solr-solrj/index.html] > For Gradle build, it should also follow gradle project structure (path) > instead of ant one, to keep things simple to manage [1]. Hence, it will look > like this: > [https://lucene.apache.org/core/9_0_0/analysis/icu/index.html] > [https://lucene.apache.org/solr/9_0_0/solr/solrj/index.html] > [1] The change was suggested at the conversation between Dawid Weiss and I on > a github pr: [https://github.com/apache/lucene-solr/pull/1304] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9321) Port documentation task to gradle
[ https://issues.apache.org/jira/browse/LUCENE-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099752#comment-17099752 ] Uwe Schindler commented on LUCENE-9321: --- bq. There's one thing I'm not fully understand. According to this policy, how do we handle links in Solr javadocs to Lucene javadocs. To be strictly consistent, those should be relative links for website, but absolute links for tar.gz (because Lucene and Solr archives are separately distributed). ? Sorry for not having a full answer to this on the weekend. The proposal to split Lucene and Solr on the development mailinglist was something I did not want to make publish. With that background, I can now confirm my previous statements: - If solr splits off lucene, the links between both projects need to be absolute anyways. Or - removed alltogether from Javadocs, as it's to fragile - The recent changes also confirm my previous problem with absolute links: Once projects split, the current Javadoc repository needs to be moved possibly to another *.apache.org subdomain. Of course we will have redirects, but nevertheless I don't want to introduce a redirect for every link anywhere in the Javadocs. So relative links that link between modules of Solr and/or Lucene should be relative, otherwise it's a nightmare to maintain afterwards. These two reasons are my main arguments for having only relative links between modules in the "overall documentation". For individual Maven artifacts I don't care, but there the links should be absolute (with configurable location) or removed altogether. Somebody using the "javadoc" maven artifact just needs docs for the current method heshe is looking at, so we could possibly also remove all links there. > Port documentation task to gradle > - > > Key: LUCENE-9321 > URL: https://issues.apache.org/jira/browse/LUCENE-9321 > Project: Lucene - Core > Issue Type: Sub-task > Components: general/build >Reporter: Tomoko Uchida >Assignee: Uwe Schindler >Priority: Major > Fix For: master (9.0) > > Attachments: screenshot-1.png > > Time Spent: 2.5h > Remaining Estimate: 0h > > This is a placeholder issue for porting ant "documentation" task to gradle. > The generated documents should be able to be published on lucene.apache.org > web site on "as-is" basis. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-9321) Port documentation task to gradle
[ https://issues.apache.org/jira/browse/LUCENE-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099752#comment-17099752 ] Uwe Schindler edited comment on LUCENE-9321 at 5/5/20, 10:19 AM: - bq. There's one thing I'm not fully understand. According to this policy, how do we handle links in Solr javadocs to Lucene javadocs. To be strictly consistent, those should be relative links for website, but absolute links for tar.gz (because Lucene and Solr archives are separately distributed). ? Sorry for not having a full answer to this on the weekend. The proposal to split Lucene and Solr on the development mailinglist was something I did not want to make public. With that background, I can now confirm my previous statements: - If solr splits off lucene, the links between both projects need to be absolute anyways. Or - removed alltogether from Javadocs, as it's to fragile - The recent changes also confirm my previous problem with absolute links: Once projects split, the current Javadoc repository needs to be moved possibly to another *.apache.org subdomain. Of course we will have redirects, but nevertheless I don't want to introduce a redirect for every link anywhere in the Javadocs. So relative links that link between modules of Solr and/or Lucene should be relative, otherwise it's a nightmare to maintain afterwards. These two reasons are my main arguments for having only relative links between modules in the "overall documentation". For individual Maven artifacts I don't care, but there the links should be absolute (with configurable location) or removed altogether. Somebody using the "javadoc" maven artifact just needs docs for the current method heshe is looking at, so we could possibly also remove all links there. was (Author: thetaphi): bq. There's one thing I'm not fully understand. According to this policy, how do we handle links in Solr javadocs to Lucene javadocs. To be strictly consistent, those should be relative links for website, but absolute links for tar.gz (because Lucene and Solr archives are separately distributed). ? Sorry for not having a full answer to this on the weekend. The proposal to split Lucene and Solr on the development mailinglist was something I did not want to make publish. With that background, I can now confirm my previous statements: - If solr splits off lucene, the links between both projects need to be absolute anyways. Or - removed alltogether from Javadocs, as it's to fragile - The recent changes also confirm my previous problem with absolute links: Once projects split, the current Javadoc repository needs to be moved possibly to another *.apache.org subdomain. Of course we will have redirects, but nevertheless I don't want to introduce a redirect for every link anywhere in the Javadocs. So relative links that link between modules of Solr and/or Lucene should be relative, otherwise it's a nightmare to maintain afterwards. These two reasons are my main arguments for having only relative links between modules in the "overall documentation". For individual Maven artifacts I don't care, but there the links should be absolute (with configurable location) or removed altogether. Somebody using the "javadoc" maven artifact just needs docs for the current method heshe is looking at, so we could possibly also remove all links there. > Port documentation task to gradle > - > > Key: LUCENE-9321 > URL: https://issues.apache.org/jira/browse/LUCENE-9321 > Project: Lucene - Core > Issue Type: Sub-task > Components: general/build >Reporter: Tomoko Uchida >Assignee: Uwe Schindler >Priority: Major > Fix For: master (9.0) > > Attachments: screenshot-1.png > > Time Spent: 2.5h > Remaining Estimate: 0h > > This is a placeholder issue for porting ant "documentation" task to gradle. > The generated documents should be able to be published on lucene.apache.org > web site on "as-is" basis. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] romseygeek commented on pull request #1467: LUCENE-9350: Don't hold references to large automata on FuzzyQuery
romseygeek commented on pull request #1467: URL: https://github.com/apache/lucene-solr/pull/1467#issuecomment-623988037 I think this is ready @madrob ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #1467: LUCENE-9350: Don't hold references to large automata on FuzzyQuery
uschindler commented on a change in pull request #1467: URL: https://github.com/apache/lucene-solr/pull/1467#discussion_r420025857 ## File path: lucene/core/src/java/org/apache/lucene/search/FuzzyTermsEnum.java ## @@ -88,43 +89,44 @@ * @throws IOException if there is a low-level IO error */ public FuzzyTermsEnum(Terms terms, Term term, int maxEdits, int prefixLength, boolean transpositions) throws IOException { -this(terms, term, stringToUTF32(term.text()), maxEdits, prefixLength, transpositions); - } - - private FuzzyTermsEnum(Terms terms, Term term, int[] codePoints, int maxEdits, int prefixLength, boolean transpositions) throws IOException { -this(terms, new AttributeSource(), term, codePoints.length, maxEdits, -buildAutomata(term.text(), codePoints, prefixLength, transpositions, maxEdits)); +this(terms, new AttributeSource(), term, () -> new FuzzyAutomatonBuilder(term.text(), maxEdits, prefixLength, transpositions)); } /** * Constructor for enumeration of all terms from specified reader which share a prefix of * length prefixLength with term and which have at most {@code maxEdits} edits. * - * After calling the constructor the enumeration is already pointing to the first - * valid term if such a term exists. - * + * After calling the constructor the enumeration is already pointing to the first + * valid term if such a term exists. + * * @param terms Delivers terms. - * @param atts {@link AttributeSource} created by the rewrite method of {@link MultiTermQuery} - * that contains information about competitive boosts during rewrite + * @param atts An AttributeSource used to share automata between segments * @param term Pattern term. * @param maxEdits Maximum edit distance. - * @param automata An array of levenshtein automata to match against terms, - * see {@link #buildAutomata(String, int[], int, boolean, int)} + * @param prefixLength the length of the required common prefix + * @param transpositions whether transpositions should count as a single edit * @throws IOException if there is a low-level IO error */ - public FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, int termLength, - final int maxEdits, CompiledAutomaton[] automata) throws IOException { + FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, int maxEdits, int prefixLength, boolean transpositions) throws IOException { +this(terms, atts, term, () -> new FuzzyAutomatonBuilder(term.text(), maxEdits, prefixLength, transpositions)); + } + + private FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, Supplier automatonBuilder) throws IOException { -this.maxEdits = maxEdits; this.terms = terms; -this.term = term; this.atts = atts; -this.termLength = termLength; +this.term = term; this.maxBoostAtt = atts.addAttribute(MaxNonCompetitiveBoostAttribute.class); this.boostAtt = atts.addAttribute(BoostAttribute.class); -this.automata = automata; +atts.addAttributeImpl(new AutomatonAttributeImpl()); Review comment: I think that's fine here. There is only one implementation that's all internally used. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #1467: LUCENE-9350: Don't hold references to large automata on FuzzyQuery
uschindler commented on a change in pull request #1467: URL: https://github.com/apache/lucene-solr/pull/1467#discussion_r420028347 ## File path: lucene/core/src/java/org/apache/lucene/search/FuzzyTermsEnum.java ## @@ -88,43 +89,44 @@ * @throws IOException if there is a low-level IO error */ public FuzzyTermsEnum(Terms terms, Term term, int maxEdits, int prefixLength, boolean transpositions) throws IOException { -this(terms, term, stringToUTF32(term.text()), maxEdits, prefixLength, transpositions); - } - - private FuzzyTermsEnum(Terms terms, Term term, int[] codePoints, int maxEdits, int prefixLength, boolean transpositions) throws IOException { -this(terms, new AttributeSource(), term, codePoints.length, maxEdits, -buildAutomata(term.text(), codePoints, prefixLength, transpositions, maxEdits)); +this(terms, new AttributeSource(), term, () -> new FuzzyAutomatonBuilder(term.text(), maxEdits, prefixLength, transpositions)); } /** * Constructor for enumeration of all terms from specified reader which share a prefix of * length prefixLength with term and which have at most {@code maxEdits} edits. * - * After calling the constructor the enumeration is already pointing to the first - * valid term if such a term exists. - * + * After calling the constructor the enumeration is already pointing to the first + * valid term if such a term exists. + * * @param terms Delivers terms. - * @param atts {@link AttributeSource} created by the rewrite method of {@link MultiTermQuery} - * that contains information about competitive boosts during rewrite + * @param atts An AttributeSource used to share automata between segments * @param term Pattern term. * @param maxEdits Maximum edit distance. - * @param automata An array of levenshtein automata to match against terms, - * see {@link #buildAutomata(String, int[], int, boolean, int)} + * @param prefixLength the length of the required common prefix + * @param transpositions whether transpositions should count as a single edit * @throws IOException if there is a low-level IO error */ - public FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, int termLength, - final int maxEdits, CompiledAutomaton[] automata) throws IOException { + FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, int maxEdits, int prefixLength, boolean transpositions) throws IOException { +this(terms, atts, term, () -> new FuzzyAutomatonBuilder(term.text(), maxEdits, prefixLength, transpositions)); + } + + private FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, Supplier automatonBuilder) throws IOException { -this.maxEdits = maxEdits; this.terms = terms; -this.term = term; this.atts = atts; -this.termLength = termLength; +this.term = term; this.maxBoostAtt = atts.addAttribute(MaxNonCompetitiveBoostAttribute.class); this.boostAtt = atts.addAttribute(BoostAttribute.class); -this.automata = automata; +atts.addAttributeImpl(new AutomatonAttributeImpl()); +AutomatonAttribute aa = atts.addAttribute(AutomatonAttribute.class); Review comment: GetAttribute is also fine, because you added the impl a line before. If the attribute wouldn't be there, adding it won't work, because it's all private. So all: That's just a matter of preference. I'd check how we do this at other places. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14426) forbidden api error during precommit DateMathFunction
[ https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099798#comment-17099798 ] Alan Woodward commented on SOLR-14426: -- I've also run into this, it seems like it's due to DateMathValueFunction being defined as an old-style package-private class after the main class, rather than as a static nested class. We should probably try and find all instances of these in the codebase and clear them up. > forbidden api error during precommit DateMathFunction > - > > Key: SOLR-14426 > URL: https://issues.apache.org/jira/browse/SOLR-14426 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mike Drob >Priority: Major > > When running `./gradlew precommit` I'll occasionally see > {code} > * What went wrong: > Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'. > > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API > > calls failed while scanning class > > 'org.apache.solr.analytics.function.mapping.DateMathFunction' > > (DateMathFunction.java): java.lang.ClassNotFoundException: > > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while > > looking up details about referenced class > > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction') > {code} > `./gradlew clean` fixes this, but I don't understand what or why this > happens. Feels like a gradle issue? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14426) forbidden api error during precommit DateMathFunction
[ https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099808#comment-17099808 ] Alan Woodward commented on SOLR-14426: -- OK, from a quick grep it appears that there are 290 old-style package-private classes in the Solr code base. It seems odd that it's always this one that fails forbiddenApis though - [~uschindler] any ideas? > forbidden api error during precommit DateMathFunction > - > > Key: SOLR-14426 > URL: https://issues.apache.org/jira/browse/SOLR-14426 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mike Drob >Priority: Major > > When running `./gradlew precommit` I'll occasionally see > {code} > * What went wrong: > Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'. > > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API > > calls failed while scanning class > > 'org.apache.solr.analytics.function.mapping.DateMathFunction' > > (DateMathFunction.java): java.lang.ClassNotFoundException: > > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while > > looking up details about referenced class > > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction') > {code} > `./gradlew clean` fixes this, but I don't understand what or why this > happens. Feels like a gradle issue? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection
[ https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099970#comment-17099970 ] Endika Posadas commented on SOLR-14458: --- I found the problem: It happens that in SegmentsInfoRequestHandler, when the information is requested, it gets the Index Writer, holding a read lock on it. However, the read lock is only released when core Info is requested too, which is not the default. So in most cases, the instance will be unable to perform write after requesting segments info. > Solr Replica locked in recovering state after a Zookeeper disconnection > --- > > Key: SOLR-14458 > URL: https://issues.apache.org/jira/browse/SOLR-14458 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.4.1 > Environment: A Solr cluster with 2 replicas that each has 2 shards > split across 2 Windows VMS. > They use a 3 replica zookeeper across 3 vms. >Reporter: Endika Posadas >Priority: Major > Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, > solr-thread-dump.log, solr.log, solrrecovering.png > > > In a solr cluster, a Solr instance containing two shards has lost connection > with zookeeper. Upon reconnecting, it has checked the status with the leader > and start a recovery. However, it's stuck in recovering status without making > further progress (has been like that for days now). > > Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is > trying to acquire the lock to createa new Index Writer: `at > org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)` > ( > after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the > ReentrantLock it's waiting for is never released. Moreover, no thread can be > found holding the lock, leaving restarting Solr as the only solution. > There is no Error in the logs that can help with the issue. I have attached > solr.log and a grep with node 7 lines, as well as a thread dump. > > There is also no other recovery currently running. In Solr metrics, 4 > recoveries have started, 3 have completed and 1 is running (forever). > > My hypothesis is that > org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore, > boolean) was called once but for some reason openIndexWriter was skipped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection
[ https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099972#comment-17099972 ] Endika Posadas commented on SOLR-14458: --- As I was creating a patch I just found out that it got fixed yesterday as part of SOLR-14431. SO proceeding to mark the issue as fixed. > Solr Replica locked in recovering state after a Zookeeper disconnection > --- > > Key: SOLR-14458 > URL: https://issues.apache.org/jira/browse/SOLR-14458 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.4.1 > Environment: A Solr cluster with 2 replicas that each has 2 shards > split across 2 Windows VMS. > They use a 3 replica zookeeper across 3 vms. >Reporter: Endika Posadas >Priority: Major > Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, > solr-thread-dump.log, solr.log, solrrecovering.png > > > In a solr cluster, a Solr instance containing two shards has lost connection > with zookeeper. Upon reconnecting, it has checked the status with the leader > and start a recovery. However, it's stuck in recovering status without making > further progress (has been like that for days now). > > Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is > trying to acquire the lock to createa new Index Writer: `at > org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)` > ( > after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the > ReentrantLock it's waiting for is never released. Moreover, no thread can be > found holding the lock, leaving restarting Solr as the only solution. > There is no Error in the logs that can help with the issue. I have attached > solr.log and a grep with node 7 lines, as well as a thread dump. > > There is also no other recovery currently running. In Solr metrics, 4 > recoveries have started, 3 have completed and 1 is running (forever). > > My hypothesis is that > org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore, > boolean) was called once but for some reason openIndexWriter was skipped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection
[ https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Endika Posadas resolved SOLR-14458. --- Resolution: Duplicate > Solr Replica locked in recovering state after a Zookeeper disconnection > --- > > Key: SOLR-14458 > URL: https://issues.apache.org/jira/browse/SOLR-14458 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 8.4.1 > Environment: A Solr cluster with 2 replicas that each has 2 shards > split across 2 Windows VMS. > They use a 3 replica zookeeper across 3 vms. >Reporter: Endika Posadas >Priority: Major > Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, > solr-thread-dump.log, solr.log, solrrecovering.png > > > In a solr cluster, a Solr instance containing two shards has lost connection > with zookeeper. Upon reconnecting, it has checked the status with the leader > and start a recovery. However, it's stuck in recovering status without making > further progress (has been like that for days now). > > Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is > trying to acquire the lock to createa new Index Writer: `at > org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)` > ( > after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the > ReentrantLock it's waiting for is never released. Moreover, no thread can be > found holding the lock, leaving restarting Solr as the only solution. > There is no Error in the logs that can help with the issue. I have attached > solr.log and a grep with node 7 lines, as well as a thread dump. > > There is also no other recovery currently running. In Solr metrics, 4 > recoveries have started, 3 have completed and 1 is running (forever). > > My hypothesis is that > org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore, > boolean) was called once but for some reason openIndexWriter was skipped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on pull request #1480: SOLR-14456: Fix Content-Type header forwarding on compressed requests
HoustonPutman commented on pull request #1480: URL: https://github.com/apache/lucene-solr/pull/1480#issuecomment-624103813 Yeah, I think it would be worthwhile for the two classes to act similarly in this regard. I say go ahead and refactor. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14426) forbidden api error during precommit DateMathFunction
[ https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1714#comment-1714 ] Uwe Schindler commented on SOLR-14426: -- This is the usual problem with the old-style package-private classes which are in same source file than the main class. I fixed some of them already, but they are bad Java style. The only workaround is to do ant clean. The problem is that Java/Javac/Gradle/Ant can't figure out where the class file is coming from. When it gets recompiled, the leftover class without source file is a leftover. We can only fix that at some point. Eclipse is also getting angry about those quite often. To fix this, the classes should be moverd as static inner class into the main class. In most cases this can be done with copypaste, but there's no automated refactoring in Eclipse. Maybe other IDEs have better refactoring tools. > forbidden api error during precommit DateMathFunction > - > > Key: SOLR-14426 > URL: https://issues.apache.org/jira/browse/SOLR-14426 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mike Drob >Priority: Major > > When running `./gradlew precommit` I'll occasionally see > {code} > * What went wrong: > Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'. > > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API > > calls failed while scanning class > > 'org.apache.solr.analytics.function.mapping.DateMathFunction' > > (DateMathFunction.java): java.lang.ClassNotFoundException: > > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while > > looking up details about referenced class > > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction') > {code} > `./gradlew clean` fixes this, but I don't understand what or why this > happens. Feels like a gradle issue? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14426) forbidden api error during precommit DateMathFunction
[ https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1714#comment-1714 ] Uwe Schindler edited comment on SOLR-14426 at 5/5/20, 3:30 PM: --- This is the usual problem with the old-style package-private classes which are in same source file than the main class. I fixed some of them already, but they are bad Java style. The only workaround is to do ant clean. The problem is that Java/Javac/Gradle/Ant can't figure out where the class file is coming from. When it gets recompiled, the leftover class without source file is a relic leading to those failures. It's not really forbiddenapis fault, because the class file should not be there at all. We can only fix that at some point. Eclipse is also getting angry about those quite often. To fix this, the classes should be moverd as static inner class into the main class. In most cases this can be done with copypaste, but there's no automated refactoring in Eclipse. Maybe other IDEs have better refactoring tools. was (Author: thetaphi): This is the usual problem with the old-style package-private classes which are in same source file than the main class. I fixed some of them already, but they are bad Java style. The only workaround is to do ant clean. The problem is that Java/Javac/Gradle/Ant can't figure out where the class file is coming from. When it gets recompiled, the leftover class without source file is a leftover. We can only fix that at some point. Eclipse is also getting angry about those quite often. To fix this, the classes should be moverd as static inner class into the main class. In most cases this can be done with copypaste, but there's no automated refactoring in Eclipse. Maybe other IDEs have better refactoring tools. > forbidden api error during precommit DateMathFunction > - > > Key: SOLR-14426 > URL: https://issues.apache.org/jira/browse/SOLR-14426 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mike Drob >Priority: Major > > When running `./gradlew precommit` I'll occasionally see > {code} > * What went wrong: > Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'. > > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API > > calls failed while scanning class > > 'org.apache.solr.analytics.function.mapping.DateMathFunction' > > (DateMathFunction.java): java.lang.ClassNotFoundException: > > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while > > looking up details about referenced class > > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction') > {code} > `./gradlew clean` fixes this, but I don't understand what or why this > happens. Feels like a gradle issue? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-10932) install solr service service command fails
[ https://issues.apache.org/jira/browse/SOLR-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100014#comment-17100014 ] Markus Mandalka commented on SOLR-10932: Bad news: Sorry, i created a dupe of this problem in https://issues.apache.org/jira/browse/SOLR-11853 Goog news: was fixed 20. jan 2019, so this ticket can be closed. > install solr service service command fails > -- > > Key: SOLR-10932 > URL: https://issues.apache.org/jira/browse/SOLR-10932 > Project: Solr > Issue Type: Bug >Affects Versions: 6.6 > Environment: Suse linux >Reporter: Susheel Kumar >Priority: Minor > Labels: easyfix, newbie, patch > Attachments: SOLR-10932.patch > > > In SUSE distribution, "service --version" commands always fail and abort the > solr installation with printing the error "Script requires the 'service' > command" > We can fix it by changing "service --version" to "service --help" command. > Shawn's test results > == > This is what I get with OS versions that I have access to when running > "service --version": > CentOS 7: > service ver. 1.1 > Ubuntu 16: > service ver. 0.91-ubuntu1 > Ubuntu 14: > service ver. 0.91-ubuntu1 > CentOS 6: > service ver. 0.91 > Debian 6: > service ver. 0.91-ubuntu1 > Sparc Solaris 10: > bash: service: command not found > = -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10932) install solr service service command fails
[ https://issues.apache.org/jira/browse/SOLR-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100014#comment-17100014 ] Markus Mandalka edited comment on SOLR-10932 at 5/5/20, 3:40 PM: - Bad news: Sorry, i created a dupe of this problem in https://issues.apache.org/jira/browse/SOLR-11853 Goog news: was fixed 2. jan 2019, so this ticket can be closed. was (Author: mandalka): Bad news: Sorry, i created a dupe of this problem in https://issues.apache.org/jira/browse/SOLR-11853 Goog news: was fixed 20. jan 2019, so this ticket can be closed. > install solr service service command fails > -- > > Key: SOLR-10932 > URL: https://issues.apache.org/jira/browse/SOLR-10932 > Project: Solr > Issue Type: Bug >Affects Versions: 6.6 > Environment: Suse linux >Reporter: Susheel Kumar >Priority: Minor > Labels: easyfix, newbie, patch > Attachments: SOLR-10932.patch > > > In SUSE distribution, "service --version" commands always fail and abort the > solr installation with printing the error "Script requires the 'service' > command" > We can fix it by changing "service --version" to "service --help" command. > Shawn's test results > == > This is what I get with OS versions that I have access to when running > "service --version": > CentOS 7: > service ver. 1.1 > Ubuntu 16: > service ver. 0.91-ubuntu1 > Ubuntu 14: > service ver. 0.91-ubuntu1 > CentOS 6: > service ver. 0.91 > Debian 6: > service ver. 0.91-ubuntu1 > Sparc Solaris 10: > bash: service: command not found > = -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9362) ExpressionValueSource has buggy rewrite method
Dmitry Emets created LUCENE-9362: Summary: ExpressionValueSource has buggy rewrite method Key: LUCENE-9362 URL: https://issues.apache.org/jira/browse/LUCENE-9362 Project: Lucene - Core Issue Type: Bug Components: modules/expressions Reporter: Dmitry Emets ExpressionValuesSource does not actually rewrites himself due to small mistake in check of inner rewrites. {code:java} changed |= (rewritten[i] == variables[i]); {code} should be changed to {code:java} changed |= (rewritten[i] != variables[i]); {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9362) ExpressionValueSource has buggy rewrite method
[ https://issues.apache.org/jira/browse/LUCENE-9362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Emets updated LUCENE-9362: - Description: ExpressionValuesSource does not actually rewrite himself due to small mistake in check of inner rewrites. {code:java} changed |= (rewritten[i] == variables[i]); {code} should be changed to {code:java} changed |= (rewritten[i] != variables[i]); {code} was: ExpressionValuesSource does not actually rewrites himself due to small mistake in check of inner rewrites. {code:java} changed |= (rewritten[i] == variables[i]); {code} should be changed to {code:java} changed |= (rewritten[i] != variables[i]); {code} > ExpressionValueSource has buggy rewrite method > -- > > Key: LUCENE-9362 > URL: https://issues.apache.org/jira/browse/LUCENE-9362 > Project: Lucene - Core > Issue Type: Bug > Components: modules/expressions >Reporter: Dmitry Emets >Priority: Minor > > ExpressionValuesSource does not actually rewrite himself due to small mistake > in check of inner rewrites. > {code:java} > changed |= (rewritten[i] == variables[i]); > {code} > should be changed to > {code:java} > changed |= (rewritten[i] != variables[i]); > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9362) ExpressionValueSource has buggy rewrite method
[ https://issues.apache.org/jira/browse/LUCENE-9362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Emets updated LUCENE-9362: - Description: ExpressionValuesSource does not actually rewrite itself due to small mistake in check of inner rewrites. {code:java} changed |= (rewritten[i] == variables[i]); {code} should be changed to {code:java} changed |= (rewritten[i] != variables[i]); {code} was: ExpressionValuesSource does not actually rewrite himself due to small mistake in check of inner rewrites. {code:java} changed |= (rewritten[i] == variables[i]); {code} should be changed to {code:java} changed |= (rewritten[i] != variables[i]); {code} > ExpressionValueSource has buggy rewrite method > -- > > Key: LUCENE-9362 > URL: https://issues.apache.org/jira/browse/LUCENE-9362 > Project: Lucene - Core > Issue Type: Bug > Components: modules/expressions >Reporter: Dmitry Emets >Priority: Minor > > ExpressionValuesSource does not actually rewrite itself due to small mistake > in check of inner rewrites. > {code:java} > changed |= (rewritten[i] == variables[i]); > {code} > should be changed to > {code:java} > changed |= (rewritten[i] != variables[i]); > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] emetsds opened a new pull request #1485: LUCENE-9362: Fix rewriting check in ExpressionValueSource
emetsds opened a new pull request #1485: URL: https://github.com/apache/lucene-solr/pull/1485 # Description [LUCENE-9362](https://issues.apache.org/jira/browse/LUCENE-9362): ExpressionValuesSource does not actually rewrite itself due to small mistake in check of inner rewrites. # Solution `changed |= (rewritten[i] == variables[i]);` should be changed to `changed |= (rewritten[i] != variables[i]);` # Tests I added a test that checks that the ExpressionValueSource was rewritten when the inner ValueSource was rewritten, and vice versa # Checklist Please review the following and check all that apply: - [+] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [+] I have created a Jira issue and added the issue ID to my pull request title. - [+] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [+] I have developed this patch against the `master` branch. - [+] I have run `gradlew precommit` and the appropriate test suite. - [+] I have added tests for my changes. - [] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] sigram opened a new pull request #1486: SOLR-14423: Use SolrClientCache instance managed by CoreContainer
sigram opened a new pull request #1486: URL: https://github.com/apache/lucene-solr/pull/1486 See Jira for more details. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14423) static caches in StreamHandler ought to move to CoreContainer lifecycle
[ https://issues.apache.org/jira/browse/SOLR-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100106#comment-17100106 ] Andrzej Bialecki commented on SOLR-14423: - I created a PR with the initial attempt at fixing this: * {{CoreContainer.solrClientCache}} is initialized on {{load()}} and closed on {{shutdown()}} * I modified the StreamHandler, GraphHandler, ReindexCollectionCmd, ColStatus, SolrReporter and a few other places to use this instance. * probably the most controversial change is in the {{org.apache.solr.handler.sql.CalciteSolrDriver}} and {{SolrSchema}} / {{SolrTable}}. These objects are initialized in a way that makes it very difficult to pass an existing instance of SolrClientCache. For this reason I decided to create a single instance of the cache in {{SolrSchema}} and manage its lifecycle through the {{Connection}} - which required adding a wrapper to intercept the {{Connection.close()}} call. [~jbernste] I would appreciate your feedback. * I also added an instance of {{CoreContainer.objectCache}} - this is not strictly required by the other changes but it illustrates how we could use a single generic object cache at the CC level instead of adding many specialized accessors like the one for SolrClientCache. We can decide which way to go in the future. > static caches in StreamHandler ought to move to CoreContainer lifecycle > --- > > Key: SOLR-14423 > URL: https://issues.apache.org/jira/browse/SOLR-14423 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: streaming expressions >Reporter: David Smiley >Assignee: Andrzej Bialecki >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > StreamHandler (at "/stream") has several statically declared caches. I think > this is problematic, such as in testing wherein multiple nodes could be in > the same JVM. One of them is more serious -- SolrClientCache which is > closed/cleared via a SolrCore close hook. That's bad for performance but > also dangerous since another core might want to use one of these clients! > CC [~jbernste] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-10932) install solr service service command fails
[ https://issues.apache.org/jira/browse/SOLR-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cassandra Targett resolved SOLR-10932. -- Resolution: Duplicate > install solr service service command fails > -- > > Key: SOLR-10932 > URL: https://issues.apache.org/jira/browse/SOLR-10932 > Project: Solr > Issue Type: Bug >Affects Versions: 6.6 > Environment: Suse linux >Reporter: Susheel Kumar >Priority: Minor > Labels: easyfix, newbie, patch > Attachments: SOLR-10932.patch > > > In SUSE distribution, "service --version" commands always fail and abort the > solr installation with printing the error "Script requires the 'service' > command" > We can fix it by changing "service --version" to "service --help" command. > Shawn's test results > == > This is what I get with OS versions that I have access to when running > "service --version": > CentOS 7: > service ver. 1.1 > Ubuntu 16: > service ver. 0.91-ubuntu1 > Ubuntu 14: > service ver. 0.91-ubuntu1 > CentOS 6: > service ver. 0.91 > Debian 6: > service ver. 0.91-ubuntu1 > Sparc Solaris 10: > bash: service: command not found > = -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14425) Fix ZK sync usage to be synchronous (blocking)
[ https://issues.apache.org/jira/browse/SOLR-14425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley resolved SOLR-14425. - Resolution: Not A Problem Not actually a problem; it was my misunderstanding. > Fix ZK sync usage to be synchronous (blocking) > -- > > Key: SOLR-14425 > URL: https://issues.apache.org/jira/browse/SOLR-14425 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Reporter: David Smiley >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > As of this writing, we only use one call to ZK's "sync" method. It's related > to collection aliases -- I added this. I discovered I misunderstood the > semantics of the API; it syncs in the background and thus returns > immediately. Looking at ZK's sync CLI command and Curator both made me > realize my folly. I'm considering this only a "minor" issue because I'm not > sure I've seen a bug from this; or maybe I did in spooky test failures over a > year ago -- I'm not sure. And we don't use this pervasively (yet). > It occurred to me that if Solr embraced the Curator framework abstraction > over ZooKeeper, I would not have fallen into that trap. I'll file a separate > issue for that. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14423) static caches in StreamHandler ought to move to CoreContainer lifecycle
[ https://issues.apache.org/jira/browse/SOLR-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100212#comment-17100212 ] Joel Bernstein commented on SOLR-14423: --- [~ab], I'll take a look at the PR this week. > static caches in StreamHandler ought to move to CoreContainer lifecycle > --- > > Key: SOLR-14423 > URL: https://issues.apache.org/jira/browse/SOLR-14423 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: streaming expressions >Reporter: David Smiley >Assignee: Andrzej Bialecki >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > StreamHandler (at "/stream") has several statically declared caches. I think > this is problematic, such as in testing wherein multiple nodes could be in > the same JVM. One of them is more serious -- SolrClientCache which is > closed/cleared via a SolrCore close hook. That's bad for performance but > also dangerous since another core might want to use one of these clients! > CC [~jbernste] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14426) forbidden api error during precommit DateMathFunction
[ https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100221#comment-17100221 ] Mike Drob commented on SOLR-14426: -- Thanks for explaining what is going on, [~uschindler]. IntelliJ has a report for this, we have 437 instances of this issue apparently. I'll try to tackle a few of them and see how well it works. > forbidden api error during precommit DateMathFunction > - > > Key: SOLR-14426 > URL: https://issues.apache.org/jira/browse/SOLR-14426 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mike Drob >Priority: Major > > When running `./gradlew precommit` I'll occasionally see > {code} > * What went wrong: > Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'. > > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API > > calls failed while scanning class > > 'org.apache.solr.analytics.function.mapping.DateMathFunction' > > (DateMathFunction.java): java.lang.ClassNotFoundException: > > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while > > looking up details about referenced class > > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction') > {code} > `./gradlew clean` fixes this, but I don't understand what or why this > happens. Feels like a gradle issue? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design
[ https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100244#comment-17100244 ] ASF subversion and git services commented on SOLR-14173: Commit 3f227e6b5502a256a187bd48760603e36f8461ea in lucene-solr's branch refs/heads/branch_8x from Cassandra Targett [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3f227e6 ] SOLR-14173: add entry to CHANGES.txt > Ref Guide Redesign Phase 1: Page Design > --- > > Key: SOLR-14173 > URL: https://issues.apache.org/jira/browse/SOLR-14173 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, > gray-left-nav.png > > > The current design of the Ref Guide was essentially copied from a > Jekyll-based documentation theme > (https://idratherbewriting.com/documentation-theme-jekyll/), which had a > couple important benefits for that time: > * It was well-documented and since I had little experience with Jekyll and > its Liquid templates and since I was the one doing it, I wanted to make it as > easy on myself as possible > * It was designed for documentation specifically so took care of all the > things like inter-page navigation, etc. > * It helped us get from Confluence to our current system quickly > It had some drawbacks, though: > * It wasted a lot of space on the page > * The theme was built for Markdown files, so did not take advantage of the > features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one > big example - the plugin could create it at build time, but the theme > included JS to do it as the page loads, so we use the JS) > * It had a lot of JS and overlapping CSS files. While it used Bootstrap it > used a customized CSS on top of it for theming that made modifications > complex (it was hard to figure out how exactly a change would behave) > * With all the stuff I'd changed in my bumbling way just to get things to > work back then, I broke a lot of the stuff Bootstrap is supposed to give us > in terms of responsiveness and making the Guide usable even on smaller screen > sizes. > After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF > (SOLR-13782), I wanted to try to set us up for a more flexible system. We > need it for things like Joel's work on the visual guide for streaming > expressions (SOLR-13105), and in order to implement other ideas we might have > on how to present information in the future. > I view this issue as a phase 1 of an overall redesign that I've already > started in a local branch. I'll explain in a comment the changes I've already > made, and will use this issue to create and push a branch where we can > discuss in more detail. > Phase 1 here will be under-the-hood CSS/JS changes + overall page layout > changes. > Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of > the Guide. > Phase 3 (issue TBD) will explore moving us from Jekyll to another static site > generator that is better suited for our content format, file types, and build > conventions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design
[ https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100242#comment-17100242 ] ASF subversion and git services commented on SOLR-14173: Commit c74f951c1c210d112c19577b44a39eaeaaf126a5 in lucene-solr's branch refs/heads/branch_8x from Cassandra Targett [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c74f951 ] SOLR-14173: Change left nav item highlighting to fix menu jumpiness when hovering/selecting > Ref Guide Redesign Phase 1: Page Design > --- > > Key: SOLR-14173 > URL: https://issues.apache.org/jira/browse/SOLR-14173 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, > gray-left-nav.png > > > The current design of the Ref Guide was essentially copied from a > Jekyll-based documentation theme > (https://idratherbewriting.com/documentation-theme-jekyll/), which had a > couple important benefits for that time: > * It was well-documented and since I had little experience with Jekyll and > its Liquid templates and since I was the one doing it, I wanted to make it as > easy on myself as possible > * It was designed for documentation specifically so took care of all the > things like inter-page navigation, etc. > * It helped us get from Confluence to our current system quickly > It had some drawbacks, though: > * It wasted a lot of space on the page > * The theme was built for Markdown files, so did not take advantage of the > features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one > big example - the plugin could create it at build time, but the theme > included JS to do it as the page loads, so we use the JS) > * It had a lot of JS and overlapping CSS files. While it used Bootstrap it > used a customized CSS on top of it for theming that made modifications > complex (it was hard to figure out how exactly a change would behave) > * With all the stuff I'd changed in my bumbling way just to get things to > work back then, I broke a lot of the stuff Bootstrap is supposed to give us > in terms of responsiveness and making the Guide usable even on smaller screen > sizes. > After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF > (SOLR-13782), I wanted to try to set us up for a more flexible system. We > need it for things like Joel's work on the visual guide for streaming > expressions (SOLR-13105), and in order to implement other ideas we might have > on how to present information in the future. > I view this issue as a phase 1 of an overall redesign that I've already > started in a local branch. I'll explain in a comment the changes I've already > made, and will use this issue to create and push a branch where we can > discuss in more detail. > Phase 1 here will be under-the-hood CSS/JS changes + overall page layout > changes. > Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of > the Guide. > Phase 3 (issue TBD) will explore moving us from Jekyll to another static site > generator that is better suited for our content format, file types, and build > conventions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design
[ https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100243#comment-17100243 ] ASF subversion and git services commented on SOLR-14173: Commit ea07244e353da95c01552a1ade800f05d5edb7dd in lucene-solr's branch refs/heads/branch_8x from Cassandra Targett [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ea07244 ] SOLR-14173: Don't use JQuery-Slim as it breaks the sidebar sub-menu system. > Ref Guide Redesign Phase 1: Page Design > --- > > Key: SOLR-14173 > URL: https://issues.apache.org/jira/browse/SOLR-14173 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, > gray-left-nav.png > > > The current design of the Ref Guide was essentially copied from a > Jekyll-based documentation theme > (https://idratherbewriting.com/documentation-theme-jekyll/), which had a > couple important benefits for that time: > * It was well-documented and since I had little experience with Jekyll and > its Liquid templates and since I was the one doing it, I wanted to make it as > easy on myself as possible > * It was designed for documentation specifically so took care of all the > things like inter-page navigation, etc. > * It helped us get from Confluence to our current system quickly > It had some drawbacks, though: > * It wasted a lot of space on the page > * The theme was built for Markdown files, so did not take advantage of the > features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one > big example - the plugin could create it at build time, but the theme > included JS to do it as the page loads, so we use the JS) > * It had a lot of JS and overlapping CSS files. While it used Bootstrap it > used a customized CSS on top of it for theming that made modifications > complex (it was hard to figure out how exactly a change would behave) > * With all the stuff I'd changed in my bumbling way just to get things to > work back then, I broke a lot of the stuff Bootstrap is supposed to give us > in terms of responsiveness and making the Guide usable even on smaller screen > sizes. > After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF > (SOLR-13782), I wanted to try to set us up for a more flexible system. We > need it for things like Joel's work on the visual guide for streaming > expressions (SOLR-13105), and in order to implement other ideas we might have > on how to present information in the future. > I view this issue as a phase 1 of an overall redesign that I've already > started in a local branch. I'll explain in a comment the changes I've already > made, and will use this issue to create and push a branch where we can > discuss in more detail. > Phase 1 here will be under-the-hood CSS/JS changes + overall page layout > changes. > Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of > the Guide. > Phase 3 (issue TBD) will explore moving us from Jekyll to another static site > generator that is better suited for our content format, file types, and build > conventions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design
[ https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100241#comment-17100241 ] ASF subversion and git services commented on SOLR-14173: Commit 6a01cb4a21974b56b35afd1c83d74b1407892da6 in lucene-solr's branch refs/heads/branch_8x from Cassandra Targett [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6a01cb4 ] SOLR-14173: Ref Guide Redesign: upgrade bootstrap; change layout; consolidate CSS. See issue for list of changes. > Ref Guide Redesign Phase 1: Page Design > --- > > Key: SOLR-14173 > URL: https://issues.apache.org/jira/browse/SOLR-14173 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, > gray-left-nav.png > > > The current design of the Ref Guide was essentially copied from a > Jekyll-based documentation theme > (https://idratherbewriting.com/documentation-theme-jekyll/), which had a > couple important benefits for that time: > * It was well-documented and since I had little experience with Jekyll and > its Liquid templates and since I was the one doing it, I wanted to make it as > easy on myself as possible > * It was designed for documentation specifically so took care of all the > things like inter-page navigation, etc. > * It helped us get from Confluence to our current system quickly > It had some drawbacks, though: > * It wasted a lot of space on the page > * The theme was built for Markdown files, so did not take advantage of the > features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one > big example - the plugin could create it at build time, but the theme > included JS to do it as the page loads, so we use the JS) > * It had a lot of JS and overlapping CSS files. While it used Bootstrap it > used a customized CSS on top of it for theming that made modifications > complex (it was hard to figure out how exactly a change would behave) > * With all the stuff I'd changed in my bumbling way just to get things to > work back then, I broke a lot of the stuff Bootstrap is supposed to give us > in terms of responsiveness and making the Guide usable even on smaller screen > sizes. > After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF > (SOLR-13782), I wanted to try to set us up for a more flexible system. We > need it for things like Joel's work on the visual guide for streaming > expressions (SOLR-13105), and in order to implement other ideas we might have > on how to present information in the future. > I view this issue as a phase 1 of an overall redesign that I've already > started in a local branch. I'll explain in a comment the changes I've already > made, and will use this issue to create and push a branch where we can > discuss in more detail. > Phase 1 here will be under-the-hood CSS/JS changes + overall page layout > changes. > Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of > the Guide. > Phase 3 (issue TBD) will explore moving us from Jekyll to another static site > generator that is better suited for our content format, file types, and build > conventions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design
[ https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100245#comment-17100245 ] ASF subversion and git services commented on SOLR-14173: Commit d4dbd0b9e75cb0bd5b0188a0f70070b6867fd94b in lucene-solr's branch refs/heads/master from Cassandra Targett [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d4dbd0b ] SOLR-14173: Add entry in CHANGES.txt > Ref Guide Redesign Phase 1: Page Design > --- > > Key: SOLR-14173 > URL: https://issues.apache.org/jira/browse/SOLR-14173 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, > gray-left-nav.png > > > The current design of the Ref Guide was essentially copied from a > Jekyll-based documentation theme > (https://idratherbewriting.com/documentation-theme-jekyll/), which had a > couple important benefits for that time: > * It was well-documented and since I had little experience with Jekyll and > its Liquid templates and since I was the one doing it, I wanted to make it as > easy on myself as possible > * It was designed for documentation specifically so took care of all the > things like inter-page navigation, etc. > * It helped us get from Confluence to our current system quickly > It had some drawbacks, though: > * It wasted a lot of space on the page > * The theme was built for Markdown files, so did not take advantage of the > features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one > big example - the plugin could create it at build time, but the theme > included JS to do it as the page loads, so we use the JS) > * It had a lot of JS and overlapping CSS files. While it used Bootstrap it > used a customized CSS on top of it for theming that made modifications > complex (it was hard to figure out how exactly a change would behave) > * With all the stuff I'd changed in my bumbling way just to get things to > work back then, I broke a lot of the stuff Bootstrap is supposed to give us > in terms of responsiveness and making the Guide usable even on smaller screen > sizes. > After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF > (SOLR-13782), I wanted to try to set us up for a more flexible system. We > need it for things like Joel's work on the visual guide for streaming > expressions (SOLR-13105), and in order to implement other ideas we might have > on how to present information in the future. > I view this issue as a phase 1 of an overall redesign that I've already > started in a local branch. I'll explain in a comment the changes I've already > made, and will use this issue to create and push a branch where we can > discuss in more detail. > Phase 1 here will be under-the-hood CSS/JS changes + overall page layout > changes. > Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of > the Guide. > Phase 3 (issue TBD) will explore moving us from Jekyll to another static site > generator that is better suited for our content format, file types, and build > conventions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design
[ https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cassandra Targett updated SOLR-14173: - Fix Version/s: 8.6 > Ref Guide Redesign Phase 1: Page Design > --- > > Key: SOLR-14173 > URL: https://issues.apache.org/jira/browse/SOLR-14173 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Fix For: 8.6 > > Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, > gray-left-nav.png > > > The current design of the Ref Guide was essentially copied from a > Jekyll-based documentation theme > (https://idratherbewriting.com/documentation-theme-jekyll/), which had a > couple important benefits for that time: > * It was well-documented and since I had little experience with Jekyll and > its Liquid templates and since I was the one doing it, I wanted to make it as > easy on myself as possible > * It was designed for documentation specifically so took care of all the > things like inter-page navigation, etc. > * It helped us get from Confluence to our current system quickly > It had some drawbacks, though: > * It wasted a lot of space on the page > * The theme was built for Markdown files, so did not take advantage of the > features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one > big example - the plugin could create it at build time, but the theme > included JS to do it as the page loads, so we use the JS) > * It had a lot of JS and overlapping CSS files. While it used Bootstrap it > used a customized CSS on top of it for theming that made modifications > complex (it was hard to figure out how exactly a change would behave) > * With all the stuff I'd changed in my bumbling way just to get things to > work back then, I broke a lot of the stuff Bootstrap is supposed to give us > in terms of responsiveness and making the Guide usable even on smaller screen > sizes. > After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF > (SOLR-13782), I wanted to try to set us up for a more flexible system. We > need it for things like Joel's work on the visual guide for streaming > expressions (SOLR-13105), and in order to implement other ideas we might have > on how to present information in the future. > I view this issue as a phase 1 of an overall redesign that I've already > started in a local branch. I'll explain in a comment the changes I've already > made, and will use this issue to create and push a branch where we can > discuss in more detail. > Phase 1 here will be under-the-hood CSS/JS changes + overall page layout > changes. > Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of > the Guide. > Phase 3 (issue TBD) will explore moving us from Jekyll to another static site > generator that is better suited for our content format, file types, and build > conventions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design
[ https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cassandra Targett resolved SOLR-14173. -- Resolution: Fixed This is backported to branch_8x for use with 8.6. I think it's going to be jarring for 8.6 to be radically different from earlier 8.x versions at least, but if I decide I have time to backport to earlier versions & republish all those branches, I'll file another issue for that effort. > Ref Guide Redesign Phase 1: Page Design > --- > > Key: SOLR-14173 > URL: https://issues.apache.org/jira/browse/SOLR-14173 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Fix For: 8.6 > > Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, > gray-left-nav.png > > > The current design of the Ref Guide was essentially copied from a > Jekyll-based documentation theme > (https://idratherbewriting.com/documentation-theme-jekyll/), which had a > couple important benefits for that time: > * It was well-documented and since I had little experience with Jekyll and > its Liquid templates and since I was the one doing it, I wanted to make it as > easy on myself as possible > * It was designed for documentation specifically so took care of all the > things like inter-page navigation, etc. > * It helped us get from Confluence to our current system quickly > It had some drawbacks, though: > * It wasted a lot of space on the page > * The theme was built for Markdown files, so did not take advantage of the > features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one > big example - the plugin could create it at build time, but the theme > included JS to do it as the page loads, so we use the JS) > * It had a lot of JS and overlapping CSS files. While it used Bootstrap it > used a customized CSS on top of it for theming that made modifications > complex (it was hard to figure out how exactly a change would behave) > * With all the stuff I'd changed in my bumbling way just to get things to > work back then, I broke a lot of the stuff Bootstrap is supposed to give us > in terms of responsiveness and making the Guide usable even on smaller screen > sizes. > After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF > (SOLR-13782), I wanted to try to set us up for a more flexible system. We > need it for things like Joel's work on the visual guide for streaming > expressions (SOLR-13105), and in order to implement other ideas we might have > on how to present information in the future. > I view this issue as a phase 1 of an overall redesign that I've already > started in a local branch. I'll explain in a comment the changes I've already > made, and will use this issue to create and push a branch where we can > discuss in more detail. > Phase 1 here will be under-the-hood CSS/JS changes + overall page layout > changes. > Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of > the Guide. > Phase 3 (issue TBD) will explore moving us from Jekyll to another static site > generator that is better suited for our content format, file types, and build > conventions. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob opened a new pull request #1487: SOLR-14426 Move top level classes to nested classes
madrob opened a new pull request #1487: URL: https://github.com/apache/lucene-solr/pull/1487 https://issues.apache.org/jira/browse/SOLR-14426 This is as far as I could get before needing to give up. It was partly automated by IntelliJ but still painful manual in some respects. I think I got everything in the analytics module, and made a dent in core. Probably want to view this comparison ignoring whitespace changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14457) SolrClient leaks connections on compressed responses if the response is malformed
[ https://issues.apache.org/jira/browse/SOLR-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100277#comment-17100277 ] Samuel García Martínez commented on SOLR-14457: --- Just for me to understand, I have several questions: * In your scenario, are you using the HTTP1 or HTTP2 client? * When do you observe the leaks? * How a connection is leaked with proper HTTP responses? Do you know where it happens? In this bug, the exception happens like the following: * HTTPSolrClient receives an error code or a malformed response * The client throws an error * The client, on a finally block, ensures the connection is closed with Utils.consumeFully(HttpEntity) * Utils.consumeFully uses HttpEntity#getContent * When compression is enabled and response is compressed, the interceptor returns a new GzipInputStream * The GzipInputStream expects the header and it's not there because it's not really compressed. Does your scenario look like the one I explained above? > SolrClient leaks connections on compressed responses if the response is > malformed > - > > Key: SOLR-14457 > URL: https://issues.apache.org/jira/browse/SOLR-14457 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ >Affects Versions: 7.7.2 > Environment: Solr version: 7.7.2 > Solr cloud enabled > Cluster topology: 6 nodes, 1 single collection, 10 shards and 3 replicas. 1 > HTTP LB using > Round Robin over all nodes > All cluster nodes have gzip enabled for all paths, all HTTP verbs and all > MIME types. > Solr client: HttpSolrClient targeting the HTTP LB >Reporter: Samuel García Martínez >Priority: Major > > h3. Summary > When the SolrJ receives a malformed response Entity, for example like the one > described in SOLR-14456, the client leaks the connection forever as it's > never released back to the pool. > h3. Problem description > HttpSolrClient should have compression enabled, so it uses the compression > interceptors. > When the request is marked with "Content-Encoding: gzip" but for whatever > reason the response is not in GZIP format, when HttpSolrClient tries to > close the connection using Utils.consumeFully(), it tries to create the > GzipInputStream failing and throwing an Exception. The exception thrown makes > it impossible to access the underlying InputStream to be closed, therefore > the connection is leaked. > Despite that the content in the response should honour the headers specified > for it, SolrJ should be reliable enough to prevent the connection leak when > this scenario happens. On top of the bug itself, not being able to set a > timeout while waiting for a connection to be available, makes any application > unresponsive as it will run out of threads eventually. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14454) support for UTF-8 (string) types with DocValuesType.BINARY
[ https://issues.apache.org/jira/browse/SOLR-14454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100284#comment-17100284 ] David Smiley commented on SOLR-14454: - I just took a look at both -- my old patch and what you've done here. I agree that the approach & emphasis (motivating use-case) differs considerably. Nonetheless I want to ensure that the chosen solution is a workable basis for both use-cases. I generally like the direction you've gone with here more. I should spend more time giving a more proper review. I still think you should pause the compression aspect for another time because it's its own discussion. It's fine to add hooks to allow a subclass to do compression, as I see you have done. > support for UTF-8 (string) types with DocValuesType.BINARY > -- > > Key: SOLR-14454 > URL: https://issues.apache.org/jira/browse/SOLR-14454 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: Schema and Analysis >Affects Versions: master (9.0) >Reporter: Michael Gibney >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > The goal is to add support for string fields with arbitrarily large values in > the {{/export}} handler and streaming expressions. > {{StrField}} values are currently limited to 32766 bytes for the case where > {{indexed=true}} or {{docValues=true}}. Exceeding this value triggers an > "immense field" warning, and causes indexing to fail for the associated input > doc. > Configuring a {{StrField}} field as "{{indexed=false docValues=false}}" > removes this size limitation, so it is already possible to have large > _stored_ {{StrField}} values. But the "{{docValues=true}}" prerequisite for > the {{/export}} handler (and consequently for streaming expressions) limits > the size of field that can be used in conjunction with these features. > Adding support for UTF-8/string field types with {{DocValuesType.BINARY}} > would address this limitation and allow considerable flexibility in the > implementation of custom field types. N.b.: this would address field value > retrieval use cases only (e.g., {{/export}} and {{useDocValuesAsStored}}); > neither sorting nor faceting would be supported. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] samuelgmartinez commented on pull request #1480: SOLR-14456: Fix Content-Type header forwarding on compressed requests
samuelgmartinez commented on pull request #1480: URL: https://github.com/apache/lucene-solr/pull/1480#issuecomment-624334841 I just pushed the refactor. precommit is failing for an unused import I need to clean up. Also, fixed related tests because it no longer throws ParseException and now it always will throw RemoteSolrException when Content-Type is empty. Tomorrow I’ll build the servers to do a manual round of testing with the cluster setup explained in the issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] samuelgmartinez edited a comment on pull request #1480: SOLR-14456: Fix Content-Type header forwarding on compressed requests
samuelgmartinez edited a comment on pull request #1480: URL: https://github.com/apache/lucene-solr/pull/1480#issuecomment-624334841 I just pushed the refactor. precommit is failing for an unused import I need to clean up. Also, fixed related tests because it no longer throws ParseException and now it always will throw RemoteSolrException when Content-Type is empty. Tomorrow evening (CEST) I’ll build the server to run some checks with the cluster setup explained in the issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on pull request #1487: SOLR-14426 Move top level classes to nested classes
HoustonPutman commented on pull request #1487: URL: https://github.com/apache/lucene-solr/pull/1487#issuecomment-624345497 @uschindler have you seen this bug with other applications that use that plugin? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman commented on pull request #1487: SOLR-14426 Move top level classes to nested classes
HoustonPutman commented on pull request #1487: URL: https://github.com/apache/lucene-solr/pull/1487#issuecomment-624345348 This looks good to me. Kind of annoying that the gradle precommit plugin has this bug. I looked around in that repo to try and find something, but I couldn't find anything. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman removed a comment on pull request #1487: SOLR-14426 Move top level classes to nested classes
HoustonPutman removed a comment on pull request #1487: URL: https://github.com/apache/lucene-solr/pull/1487#issuecomment-624345497 @uschindler have you seen this bug with other applications that use that plugin? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman edited a comment on pull request #1487: SOLR-14426 Move top level classes to nested classes
HoustonPutman edited a comment on pull request #1487: URL: https://github.com/apache/lucene-solr/pull/1487#issuecomment-624345348 This looks good to me. Didn't know those protected classes were so frowned upon. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] anshumg commented on a change in pull request #1470: SOLR-14354: Async or using threads in better way for HttpShardHandler
anshumg commented on a change in pull request #1470: URL: https://github.com/apache/lucene-solr/pull/1470#discussion_r420457814 ## File path: solr/core/src/java/org/apache/solr/handler/component/HttpShardHandler.java ## @@ -130,77 +134,64 @@ public void submit(final ShardRequest sreq, final String shard, final Modifiable final Tracer tracer = GlobalTracer.getTracer(); final Span span = tracer != null ? tracer.activeSpan() : null; -Callable task = () -> { +params.remove(CommonParams.WT); // use default (currently javabin) +params.remove(CommonParams.VERSION); +QueryRequest req = makeQueryRequest(sreq, params, shard); +req.setMethod(SolrRequest.METHOD.POST); - ShardResponse srsp = new ShardResponse(); - if (sreq.nodeName != null) { -srsp.setNodeName(sreq.nodeName); - } - srsp.setShardRequest(sreq); - srsp.setShard(shard); - SimpleSolrResponse ssr = new SimpleSolrResponse(); - srsp.setSolrResponse(ssr); - long startTime = System.nanoTime(); +LBSolrClient.Req lbReq = httpShardHandlerFactory.newLBHttpSolrClientReq(req, urls); + +ShardResponse srsp = new ShardResponse(); +if (sreq.nodeName != null) { + srsp.setNodeName(sreq.nodeName); +} +srsp.setShardRequest(sreq); +srsp.setShard(shard); +SimpleSolrResponse ssr = new SimpleSolrResponse(); +srsp.setSolrResponse(ssr); + +pending.incrementAndGet(); +// if there are no shards available for a slice, urls.size()==0 +if (urls.size() == 0) { + // TODO: what's the right error code here? We should use the same thing when Review comment: Service unavailable error code seems reasonable to me here. Do you have something else you were debating to switch this out to? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1486: SOLR-14423: Use SolrClientCache instance managed by CoreContainer
madrob commented on a change in pull request #1486: URL: https://github.com/apache/lucene-solr/pull/1486#discussion_r420459575 ## File path: solr/core/src/java/org/apache/solr/handler/sql/CalciteSolrDriver.java ## @@ -59,11 +81,346 @@ public Connection connect(String url, Properties info) throws SQLException { if(schemaName == null) { throw new SQLException("zk must be set"); } -rootSchema.add(schemaName, new SolrSchema(info)); +SolrSchema solrSchema = new SolrSchema(info); +rootSchema.add(schemaName, solrSchema); // Set the default schema calciteConnection.setSchema(schemaName); -return connection; +return new SolrCalciteConnectionWrapper(calciteConnection, solrSchema); + } + + // the sole purpose of this class is to be able to invoke SolrSchema.close() + // when the connection is closed. + private static final class SolrCalciteConnectionWrapper implements CalciteConnection { Review comment: Can we have a new top level class that is a no-frills Wrapper, and then extend that here overriding just close? I think it is easier for people to see what is different. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness
[ https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100316#comment-17100316 ] Chris M. Hostetter commented on SOLR-13132: --- [~mgibney]- i haven't had a chance to review 6338b327d30d0c1d5fdcb8168baf8398b02787d4 in depth, but at first glance I'm a fan of what i see. your MultiAcc fix in 9ab4baef4a95a90e080d9118608619855f2e2759 seems to have addressed both of the failures I mentioned before, but it trying to beef up the MultiAcc testing i uncovered a new type of failure ...i don't really have any ideas (or even guesses) as to waht exactly is the source of the problem, but what i'm seeing is that in the MultiAcc situation _non-sweeping_ stats don't seem to be getting merged properly in some cases (even w/o refinement) ...BUT... the discrepency doesn't actually manifest when comparing "sweep vs non-sweep" – it happens when comparing the "default" behavior for faceting on a multivalued string field (which should be using ArrayUIF for these fields, and implicitly sweeping) compared to explicitly using ArrayDV w/sweeping ... the buckets, counts, "skg" results, and bucket orderings are all consistent regardless of sort, but we get different values for the non-sweeping "min" stat that's used in the same facets (or in some cases, the "min" stat is completely missing) You can see this when testing against the ec5f3a451a0be1a07a9ca37bf2cce33b8548b245 i just pushed to your branch with something like... {noformat} ant test -Dtestcase=TestCloudJSONFacetSKGSweep -Dtests.seed=838E28E3EBC3B3B3:66DB4D3457DDE2F7 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=rof-TZ -Dtests.timezone=Etc/GMT+3 -Dtests.asserts=true -Dtests.file.encoding=UTF-8 {noformat} Here's how the new testBespokeStructures i just added fails with that seed combination... {noformat} [junit4]> Throwable #1: java.lang.AssertionError: rows=0&q=(field_11_multi_sdsS:55+OR+field_0_multi_ss:46)&fore=field_5_multi_sdsS:9&back=*:*&json.facet={xxx1+:+{+type:terms,+method:${method_val:smart},+field:field_0_multi_ss,+limit:-1,+overrequest:0,+sort:+'count+desc',+refine:+false,+facet:{skg+:+{+"type":+"func",+++"func":+"relatedness($fore,$back)",+++"min_popularity":+0.001,+++${sweep_key:xxx}:+${sweep_val:yyy}+}+,min+:+"min(field_3_solo_i)"+}}+} ===> Mismatch: .xxx1.buckets[2][min]:32!=40 using method_val=dv&sweep_key=sweep_collection&sweep_val=true [junit4]>at __randomizedtesting.SeedInfo.seed([838E28E3EBC3B3B3:66DB4D3457DDE2F7]:0) [junit4]>at org.apache.solr.search.facet.TestCloudJSONFacetSKGSweep.assertFacetSKGsAreConsistent(TestCloudJSONFacetSKGSweep.java:635) [junit4]>at org.apache.solr.search.facet.TestCloudJSONFacetSKGSweep.testBespokeStructures(TestCloudJSONFacetSKGSweep.java:513) {noformat} And here's the first bit of the "expected" (ie: ArrayUIF i believe) vs "actual" (ArrayDV) output, reformated for easier reading... {noformat} expected = {count=11, xxx1={buckets=[ {val=39, count=3, skg={relatedness=0.0063, foreground_popularity=0.00833, background_popularity=0.08333}, min=10}, {val=46, count=3, skg={relatedness=-Infinity, foreground_popularity=0.0, background_popularity=0.025}, min=37}, {val=10, count=2, skg={relatedness=-Infinity, foreground_popularity=0.0, background_popularity=0.04167}, min=32}, ... actual = {count=11, xxx1={buckets=[ {val=39, count=3, skg={relatedness=0.0063, foreground_popularity=0.00833, background_popularity=0.08333}, min=10}, {val=46, count=3, skg={relatedness=-Infinity, foreground_popularity=0.0, background_popularity=0.025}, min=37}, {val=10, count=2, skg={relatedness=-Infinity, foreground_popularity=0.0, background_popularity=0.04167}, min=40}, ... {noformat} testBespoke w/same seed shows a "null" value for min for the first buckets - but other buckets have inconsistent min values ... {noformat} [junit4]> Throwable #1: java.lang.AssertionError: rows=0&q=(field_13_multi_sds:26+OR+field_6_multi_ss:33+OR+field_9_multi_ss:24)&fore=(field_4_multi_sds:27+OR+field_12_multi_ss:18+OR+field_2_multi_sdsS:28+OR+field_13_multi_sds:50)&back=*:*&json.facet={xxx+:+{+type:terms,+method:${method_val:smart},+field:field_12_multi_ss,+limit:-1,+overrequest:0,+sort:+'count+asc',+refine:+false,+facet:{skg+:+{+"type":+"func",+++"func":+"relatedness($fore,$back)",+++"min_popularity":+0.001,+++${sweep_key:xxx}:+${sweep_val:yyy}+}+,min+:+"min(field_4_multi_ids)"+}}+}&_stateVer_=org.apache.solr.search.facet.TestCloudJSONFacetSKGSweep_collection:4 ===> Mismatch: .xxx.buckets[0][min]==null using method_val=dv&sweep_key=sweep_collection&sweep_val=true expected = {count=23, xxx={buckets=[ {val=17, count=1, skg={relatedness=-0.00387, foreground_popularity=0.00833, background_popularity=0.05833}, min=16}, {val=19,
[jira] [Commented] (SOLR-13289) Support for BlockMax WAND
[ https://issues.apache.org/jira/browse/SOLR-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100329#comment-17100329 ] Tomas Eduardo Fernandez Lobbe commented on SOLR-13289: -- bq. Grouping Grouping uses a different collector. It’s not included as part of this PR. Maybe there are things that can be done to allow WAND in some cases for Grouping bq. Custom scoring via value sources bq. Payloads The optimization only applies if the natural score is the first thing to be sorted on. i.e. {{sort=score desc, id asc}}. If there is anything than “score” there, then the optimization doesn’t apply, not OOTB at least. bq. Is TwoPhaseIterator or custom ranking in collapse/expand mode or in response writer the way? or have to implement a custom query overriding the createWeight using ScoreMode.TOP_SCORES? I don’t know exactly what you want to do. In general, block-Max/WAND requires that the DISI can calculate the maximum contribution for a query in a block of documents with the indexed {{impacts}}. If you want to have more signals for calculating the score I guess those could be added as Impacts using a custom codec and then you could have a custom scorer to use those ([~jpountz], let me know if this is wrong). If you just need a custom similarity that uses TF/IDF, then you should be able to use it. > Support for BlockMax WAND > - > > Key: SOLR-13289 > URL: https://issues.apache.org/jira/browse/SOLR-13289 > Project: Solr > Issue Type: New Feature >Reporter: Ishan Chattopadhyaya >Assignee: Tomas Eduardo Fernandez Lobbe >Priority: Major > Attachments: SOLR-13289.patch, SOLR-13289.patch > > Time Spent: 3h > Remaining Estimate: 0h > > LUCENE-8135 introduced BlockMax WAND as a major speed improvement. Need to > expose this via Solr. When enabled, the numFound returned will not be exact. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9321) Port documentation task to gradle
[ https://issues.apache.org/jira/browse/LUCENE-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100365#comment-17100365 ] Tomoko Uchida commented on LUCENE-9321: --- Thank you, for elaborating. > Port documentation task to gradle > - > > Key: LUCENE-9321 > URL: https://issues.apache.org/jira/browse/LUCENE-9321 > Project: Lucene - Core > Issue Type: Sub-task > Components: general/build >Reporter: Tomoko Uchida >Assignee: Uwe Schindler >Priority: Major > Fix For: master (9.0) > > Attachments: screenshot-1.png > > Time Spent: 2.5h > Remaining Estimate: 0h > > This is a placeholder issue for porting ant "documentation" task to gradle. > The generated documents should be able to be published on lucene.apache.org > web site on "as-is" basis. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness
[ https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100425#comment-17100425 ] Michael Gibney commented on SOLR-13132: --- Ouch, yeah. I ran into that too and was trying to fix it earlier today. I hadn't yet realized that the discrepancy was related to UIF vs. DV ... with that clue it should be solved with 74e7ce3f0e65cc8abebcfcf4deef9da1ee73a656 (just pushed). For sweeping, the doc iterator includes "hits" for docs that aren't in the base DocSet; but collection should only happen for docs in the base DocSet, for purpose of stats, etc. This was in the initial patch, then for some reason I thought it was only relevant to the caching aspect of the initial patch (SOLR-13807) and excised it from this patch. Subsequently realized it was still relevant here and reintroduced it with 8a06d4c5d691dbf3e224223d7a3a0c84dcbfb87b, but mistakenly only for ArrayDV. Thanks for catching this! > Improve JSON "terms" facet performance when sorted by relatedness > -- > > Key: SOLR-13132 > URL: https://issues.apache.org/jira/browse/SOLR-13132 > Project: Solr > Issue Type: Improvement > Components: Facet Module >Affects Versions: 7.4, master (9.0) >Reporter: Michael Gibney >Priority: Major > Attachments: SOLR-13132-with-cache-01.patch, > SOLR-13132-with-cache.patch, SOLR-13132.patch, SOLR-13132_testSweep.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > When sorting buckets by {{relatedness}}, JSON "terms" facet must calculate > {{relatedness}} for every term. > The current implementation uses a standard uninverted approach (either > {{docValues}} or {{UnInvertedField}}) to get facet counts over the domain > base docSet, and then uses that initial pass as a pre-filter for a > second-pass, inverted approach of fetching docSets for each relevant term > (i.e., {{count > minCount}}?) and calculating intersection size of those sets > with the domain base docSet. > Over high-cardinality fields, the overhead of per-term docSet creation and > set intersection operations increases request latency to the point where > relatedness sort may not be usable in practice (for my use case, even after > applying the patch for SOLR-13108, for a field with ~220k unique terms per > core, QTime for high-cardinality domain docSets were, e.g.: cardinality > 1816684=9000ms, cardinality 5032902=18000ms). > The attached patch brings the above example QTimes down to a manageable > ~300ms and ~250ms respectively. The approach calculates uninverted facet > counts over domain base, foreground, and background docSets in parallel in a > single pass. This allows us to take advantage of the efficiencies built into > the standard uninverted {{FacetFieldProcessorByArray[DV|UIF]}}), and avoids > the per-term docSet creation and set intersection overhead. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14454) support for UTF-8 (string) types with DocValuesType.BINARY
[ https://issues.apache.org/jira/browse/SOLR-14454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100429#comment-17100429 ] Michael Gibney commented on SOLR-14454: --- Thanks for taking a closer look! And yes I'm perfectly happy to abandon the compression aspect altogether, esp. with hooks there to provide it as a plugin if desired (indeed I was kind of anticipating that, part of why I separated it out into its own commit). {quote}Nonetheless I want to ensure that the chosen solution is a workable basis for both use-cases. {quote} That sounds great, however you think it might make sense to proceed. > support for UTF-8 (string) types with DocValuesType.BINARY > -- > > Key: SOLR-14454 > URL: https://issues.apache.org/jira/browse/SOLR-14454 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: Schema and Analysis >Affects Versions: master (9.0) >Reporter: Michael Gibney >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > The goal is to add support for string fields with arbitrarily large values in > the {{/export}} handler and streaming expressions. > {{StrField}} values are currently limited to 32766 bytes for the case where > {{indexed=true}} or {{docValues=true}}. Exceeding this value triggers an > "immense field" warning, and causes indexing to fail for the associated input > doc. > Configuring a {{StrField}} field as "{{indexed=false docValues=false}}" > removes this size limitation, so it is already possible to have large > _stored_ {{StrField}} values. But the "{{docValues=true}}" prerequisite for > the {{/export}} handler (and consequently for streaming expressions) limits > the size of field that can be used in conjunction with these features. > Adding support for UTF-8/string field types with {{DocValuesType.BINARY}} > would address this limitation and allow considerable flexibility in the > implementation of custom field types. N.b.: this would address field value > retrieval use cases only (e.g., {{/export}} and {{useDocValuesAsStored}}); > neither sorting nor faceting would be supported. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13289) Support for BlockMax WAND
[ https://issues.apache.org/jira/browse/SOLR-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100516#comment-17100516 ] Adrien Grand commented on SOLR-13289: - bq. TwoPhaseIterator LUCENE-8806 explores supporting two-phase iterators with block-max WAND. bq. more signals for calculating the score We have been recommending using FeatureField for this use-case: https://lucene.apache.org/core/8_5_0/core/org/apache/lucene/document/FeatureField.html. It encode float values as term freqs to be able to work with block-max WAND. bq. If you just need a custom similarity that uses TF/IDF, then you should be able to use it. Indeed, we only require that scores do not decrease when tf increases and do not increase when norm decreases. > Support for BlockMax WAND > - > > Key: SOLR-13289 > URL: https://issues.apache.org/jira/browse/SOLR-13289 > Project: Solr > Issue Type: New Feature >Reporter: Ishan Chattopadhyaya >Assignee: Tomas Eduardo Fernandez Lobbe >Priority: Major > Attachments: SOLR-13289.patch, SOLR-13289.patch > > Time Spent: 3h > Remaining Estimate: 0h > > LUCENE-8135 introduced BlockMax WAND as a major speed improvement. Need to > expose this via Solr. When enabled, the numFound returned will not be exact. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org