[jira] [Updated] (LUCENE-9328) SortingGroupHead to reuse DocValues

2020-05-05 Thread Mikhail Khludnev (Jira)


 [ 
https://issues.apache.org/jira/browse/LUCENE-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated LUCENE-9328:
-
Attachment: LUCENE-9328.patch
Status: Patch Available  (was: Patch Available)

> SortingGroupHead to reuse DocValues
> ---
>
> Key: LUCENE-9328
> URL: https://issues.apache.org/jira/browse/LUCENE-9328
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/grouping
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Minor
> Attachments: LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, 
> LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> That's why 
> https://issues.apache.org/jira/browse/LUCENE-7701?focusedCommentId=17084365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17084365



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (LUCENE-9328) SortingGroupHead to reuse DocValues

2020-05-05 Thread Mikhail Khludnev (Jira)


 [ 
https://issues.apache.org/jira/browse/LUCENE-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated LUCENE-9328:
-
Attachment: (was: LUCENE-9328.patch)

> SortingGroupHead to reuse DocValues
> ---
>
> Key: LUCENE-9328
> URL: https://issues.apache.org/jira/browse/LUCENE-9328
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/grouping
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Minor
> Attachments: LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, 
> LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> That's why 
> https://issues.apache.org/jira/browse/LUCENE-7701?focusedCommentId=17084365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17084365



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (LUCENE-9328) SortingGroupHead to reuse DocValues

2020-05-05 Thread Mikhail Khludnev (Jira)


 [ 
https://issues.apache.org/jira/browse/LUCENE-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated LUCENE-9328:
-
Attachment: LUCENE-9328.patch

> SortingGroupHead to reuse DocValues
> ---
>
> Key: LUCENE-9328
> URL: https://issues.apache.org/jira/browse/LUCENE-9328
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/grouping
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Minor
> Attachments: LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, 
> LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> That's why 
> https://issues.apache.org/jira/browse/LUCENE-7701?focusedCommentId=17084365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17084365



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Assigned] (SOLR-14423) static caches in StreamHandler ought to move to CoreContainer lifecycle

2020-05-05 Thread Andrzej Bialecki (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrzej Bialecki reassigned SOLR-14423:
---

Assignee: Andrzej Bialecki

> static caches in StreamHandler ought to move to CoreContainer lifecycle
> ---
>
> Key: SOLR-14423
> URL: https://issues.apache.org/jira/browse/SOLR-14423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: David Smiley
>Assignee: Andrzej Bialecki
>Priority: Major
>
> StreamHandler (at "/stream") has several statically declared caches.  I think 
> this is problematic, such as in testing wherein multiple nodes could be in 
> the same JVM.  One of them is more serious -- SolrClientCache which is 
> closed/cleared via a SolrCore close hook.  That's bad for performance but 
> also dangerous since another core might want to use one of these clients!
> CC [~jbernste]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14457) SolrClient leaks connections on compressed responses if the response is malformed

2020-05-05 Thread Roger Lehmann (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099650#comment-17099650
 ] 

Roger Lehmann commented on SOLR-14457:
--

I can confirm this issue happening even with Solr 8.5.0 and standalone mode.
Also I came to the same conclusion, the exception is fired but handled 
silently, i.e. simply skipped.
Properly closing the connection even when the exception occurs is the preferred 
solution form my point of view.

> SolrClient leaks connections on compressed responses if the response is 
> malformed
> -
>
> Key: SOLR-14457
> URL: https://issues.apache.org/jira/browse/SOLR-14457
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: 7.7.2
> Environment: Solr version: 7.7.2
> Solr cloud enabled
> Cluster topology: 6 nodes, 1 single collection, 10 shards and 3 replicas. 1 
> HTTP LB using
> Round Robin over all nodes
> All cluster nodes have gzip enabled for all paths, all HTTP verbs and all 
> MIME types.
> Solr client: HttpSolrClient targeting the HTTP LB
>Reporter: Samuel García Martínez
>Priority: Major
>
> h3. Summary
> When the SolrJ receives a malformed response Entity, for example like the one 
> described in SOLR-14456, the client leaks the connection forever as it's 
> never released back to the pool.
> h3. Problem description
> HttpSolrClient should have compression enabled, so it uses the compression 
> interceptors.
> When the request is marked with "Content-Encoding: gzip" but for whatever 
> reason the response is not in GZIP format, when  HttpSolrClient tries to 
> close the connection using Utils.consumeFully(), it tries to create the 
> GzipInputStream failing and throwing an Exception. The exception thrown makes 
> it impossible to access the underlying InputStream to be closed, therefore 
> the connection is leaked.
> Despite that the content in the response should honour the headers specified 
> for it, SolrJ should be reliable enough to prevent the connection leak when 
> this scenario happens. On top of the bug itself, not being able to set a 
> timeout while waiting for a connection to be available, makes any application 
> unresponsive as it will run out of threads eventually.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection

2020-05-05 Thread Endika Posadas (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Endika Posadas updated SOLR-14458:
--
Attachment: image-2020-05-05-09-47-27-854.png

> Solr Replica locked in recovering state after a Zookeeper disconnection
> ---
>
> Key: SOLR-14458
> URL: https://issues.apache.org/jira/browse/SOLR-14458
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 8.4.1
> Environment: A Solr cluster with 2 replicas that each has 2 shards 
> split across 2 Windows VMS.
> They use a 3 replica zookeeper across 3 vms.
>Reporter: Endika Posadas
>Priority: Major
> Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, 
> solr-thread-dump.log, solr.log, solrrecovering.png
>
>
> In a solr cluster, a Solr instance containing two shards has lost connection 
> with zookeeper. Upon reconnecting, it has checked the status with the leader 
> and start a recovery. However, it's stuck in recovering status without making 
> further progress (has been like that for days now).
>  
> Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is  
> trying to acquire the lock to createa new Index Writer: `at 
> org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)`
>  (
> after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the 
> ReentrantLock it's waiting for is never released. Moreover, no thread can be 
> found holding the lock, leaving restarting Solr as the only solution.
> There is no Error in the logs that can help with the issue. I have attached 
> solr.log and a grep with node 7 lines, as well as a thread dump.
>  
> There is also no other recovery currently running. In Solr metrics, 4 
> recoveries have started, 3 have completed and 1 is running (forever).
>  
> My hypothesis is that 
> org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore,
>  boolean) was called once but for some reason openIndexWriter was skipped.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection

2020-05-05 Thread Endika Posadas (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099673#comment-17099673
 ] 

Endika Posadas commented on SOLR-14458:
---

Further investigating, it seems like the readLock hasn't been released, so the 
write lock is unable to acquire the lock.

!image-2020-05-05-09-47-27-854.png!

> Solr Replica locked in recovering state after a Zookeeper disconnection
> ---
>
> Key: SOLR-14458
> URL: https://issues.apache.org/jira/browse/SOLR-14458
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 8.4.1
> Environment: A Solr cluster with 2 replicas that each has 2 shards 
> split across 2 Windows VMS.
> They use a 3 replica zookeeper across 3 vms.
>Reporter: Endika Posadas
>Priority: Major
> Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, 
> solr-thread-dump.log, solr.log, solrrecovering.png
>
>
> In a solr cluster, a Solr instance containing two shards has lost connection 
> with zookeeper. Upon reconnecting, it has checked the status with the leader 
> and start a recovery. However, it's stuck in recovering status without making 
> further progress (has been like that for days now).
>  
> Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is  
> trying to acquire the lock to createa new Index Writer: `at 
> org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)`
>  (
> after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the 
> ReentrantLock it's waiting for is never released. Moreover, no thread can be 
> found holding the lock, leaving restarting Solr as the only solution.
> There is no Error in the logs that can help with the issue. I have attached 
> solr.log and a grep with node 7 lines, as well as a thread dump.
>  
> There is also no other recovery currently running. In Solr metrics, 4 
> recoveries have started, 3 have completed and 1 is running (forever).
>  
> My hypothesis is that 
> org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore,
>  boolean) was called once but for some reason openIndexWriter was skipped.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-14457) SolrClient leaks connections on compressed responses if the response is malformed

2020-05-05 Thread Roger Lehmann (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099650#comment-17099650
 ] 

Roger Lehmann edited comment on SOLR-14457 at 5/5/20, 8:58 AM:
---

I can confirm this issue happening even with Solr 8.5.0 and standalone mode.
Also I came to the same conclusion, the exception is fired but handled 
silently, i.e. simply skipped.
This happens for us also for seemingly correctly formed responses, our 
application could handle the data and also the unzipped data from a curl call 
seemed to be correct.

Properly closing the connection even when the exception occurs is the preferred 
solution form my point of view.


was (Author: roger.lehmann):
I can confirm this issue happening even with Solr 8.5.0 and standalone mode.
Also I came to the same conclusion, the exception is fired but handled 
silently, i.e. simply skipped.
Properly closing the connection even when the exception occurs is the preferred 
solution form my point of view.

> SolrClient leaks connections on compressed responses if the response is 
> malformed
> -
>
> Key: SOLR-14457
> URL: https://issues.apache.org/jira/browse/SOLR-14457
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: 7.7.2
> Environment: Solr version: 7.7.2
> Solr cloud enabled
> Cluster topology: 6 nodes, 1 single collection, 10 shards and 3 replicas. 1 
> HTTP LB using
> Round Robin over all nodes
> All cluster nodes have gzip enabled for all paths, all HTTP verbs and all 
> MIME types.
> Solr client: HttpSolrClient targeting the HTTP LB
>Reporter: Samuel García Martínez
>Priority: Major
>
> h3. Summary
> When the SolrJ receives a malformed response Entity, for example like the one 
> described in SOLR-14456, the client leaks the connection forever as it's 
> never released back to the pool.
> h3. Problem description
> HttpSolrClient should have compression enabled, so it uses the compression 
> interceptors.
> When the request is marked with "Content-Encoding: gzip" but for whatever 
> reason the response is not in GZIP format, when  HttpSolrClient tries to 
> close the connection using Utils.consumeFully(), it tries to create the 
> GzipInputStream failing and throwing an Exception. The exception thrown makes 
> it impossible to access the underlying InputStream to be closed, therefore 
> the connection is leaked.
> Despite that the content in the response should honour the headers specified 
> for it, SolrJ should be reliable enough to prevent the connection leak when 
> this scenario happens. On top of the bug itself, not being able to set a 
> timeout while waiting for a connection to be available, makes any application 
> unresponsive as it will run out of threads eventually.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9328) SortingGroupHead to reuse DocValues

2020-05-05 Thread Lucene/Solr QA (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099747#comment-17099747
 ] 

Lucene/Solr QA commented on LUCENE-9328:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 24s{color} 
| {color:red} lucene_grouping generated 1 new + 100 unchanged - 0 fixed = 101 
total (was 100) {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  0m 33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  0m 24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  0m 24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m  
8s{color} | {color:green} grouping in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  0m 13s{color} 
| {color:red} join in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
33s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 52s{color} 
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 24s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | lucene.search.join.TestBlockJoinSelector |
|   | solr.TestGroupingSearch |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-9328 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/13002039/LUCENE-9328.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / 6f775bfa69d |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| javac | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/269/artifact/out/diff-compile-javac-lucene_grouping.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/269/artifact/out/patch-unit-lucene_join.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/269/artifact/out/patch-unit-solr_core.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/269/testReport/ |
| modules | C: lucene/grouping lucene/join lucene/test-framework solr/core U: . 
|
| Console output | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/269/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> SortingGroupHead to reuse DocValues
> ---
>
> Key: LUCENE-9328
> URL: https://issues.apache.org/jira/browse/LUCENE-9328
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/grouping
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Minor
> Attachments: LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, 
> LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch, LUCENE-9328.patch
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> That's why 
> https://issues.apache.org/jira/browse/LUCENE-7701?focusedCommentId=17084365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17084365



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9278) Make javadoc folder structure follow Gradle project path

2020-05-05 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099748#comment-17099748
 ] 

ASF subversion and git services commented on LUCENE-9278:
-

Commit 9ee8aa61de4f3157e40cd7af6f767b454ac46ccf in lucene-solr's branch 
refs/heads/master from Uwe Schindler
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9ee8aa6 ]

LUCENE-9278: Fix passing of Java properties for locale: The arguments must be 
separated.


> Make javadoc folder structure follow Gradle project path
> 
>
> Key: LUCENE-9278
> URL: https://issues.apache.org/jira/browse/LUCENE-9278
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build
>Reporter: Tomoko Uchida
>Assignee: Tomoko Uchida
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>
> Current javadoc folder structure is derived from Ant project name. e.g.:
> [https://lucene.apache.org/core/8_4_1/analyzers-icu/index.html]
>  [https://lucene.apache.org/solr/8_4_1/solr-solrj/index.html]
> For Gradle build, it should also follow gradle project structure (path) 
> instead of ant one, to keep things simple to manage [1]. Hence, it will look 
> like this:
> [https://lucene.apache.org/core/9_0_0/analysis/icu/index.html]
>  [https://lucene.apache.org/solr/9_0_0/solr/solrj/index.html]
> [1] The change was suggested at the conversation between Dawid Weiss and I on 
> a github pr: [https://github.com/apache/lucene-solr/pull/1304]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9321) Port documentation task to gradle

2020-05-05 Thread Uwe Schindler (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099752#comment-17099752
 ] 

Uwe Schindler commented on LUCENE-9321:
---

bq. There's one thing I'm not fully understand. According to this policy, how 
do we handle links in Solr javadocs to Lucene javadocs. To be strictly 
consistent, those should be relative links for website, but absolute links for 
tar.gz (because Lucene and Solr archives are separately distributed). ?

Sorry for not having a full answer to this on the weekend. The proposal to 
split Lucene and Solr on the development mailinglist was something I did not 
want to make publish.

With that background, I can now confirm my previous statements:
- If solr splits off lucene, the links between both projects need to be 
absolute anyways. Or - removed alltogether from Javadocs, as it's to fragile
- The recent changes also confirm my previous problem with absolute links: Once 
projects split, the current Javadoc repository needs to be moved possibly to 
another *.apache.org subdomain. Of course we will have redirects, but 
nevertheless I don't want to introduce a redirect for every link anywhere in 
the Javadocs. So relative links that link between modules of Solr and/or Lucene 
should be relative, otherwise it's a nightmare to maintain afterwards.

These two reasons are my main arguments for having only relative links between 
modules in the "overall documentation". For individual Maven artifacts I don't 
care, but there the links should be absolute (with configurable location) or 
removed altogether. Somebody using the "javadoc" maven artifact just needs docs 
for the current method heshe is looking at, so we could possibly also remove 
all links there.

> Port documentation task to gradle
> -
>
> Key: LUCENE-9321
> URL: https://issues.apache.org/jira/browse/LUCENE-9321
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: general/build
>Reporter: Tomoko Uchida
>Assignee: Uwe Schindler
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: screenshot-1.png
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> This is a placeholder issue for porting ant "documentation" task to gradle. 
> The generated documents should be able to be published on lucene.apache.org 
> web site on "as-is" basis.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Comment Edited] (LUCENE-9321) Port documentation task to gradle

2020-05-05 Thread Uwe Schindler (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099752#comment-17099752
 ] 

Uwe Schindler edited comment on LUCENE-9321 at 5/5/20, 10:19 AM:
-

bq. There's one thing I'm not fully understand. According to this policy, how 
do we handle links in Solr javadocs to Lucene javadocs. To be strictly 
consistent, those should be relative links for website, but absolute links for 
tar.gz (because Lucene and Solr archives are separately distributed). ?

Sorry for not having a full answer to this on the weekend. The proposal to 
split Lucene and Solr on the development mailinglist was something I did not 
want to make public.

With that background, I can now confirm my previous statements:
- If solr splits off lucene, the links between both projects need to be 
absolute anyways. Or - removed alltogether from Javadocs, as it's to fragile
- The recent changes also confirm my previous problem with absolute links: Once 
projects split, the current Javadoc repository needs to be moved possibly to 
another *.apache.org subdomain. Of course we will have redirects, but 
nevertheless I don't want to introduce a redirect for every link anywhere in 
the Javadocs. So relative links that link between modules of Solr and/or Lucene 
should be relative, otherwise it's a nightmare to maintain afterwards.

These two reasons are my main arguments for having only relative links between 
modules in the "overall documentation". For individual Maven artifacts I don't 
care, but there the links should be absolute (with configurable location) or 
removed altogether. Somebody using the "javadoc" maven artifact just needs docs 
for the current method heshe is looking at, so we could possibly also remove 
all links there.


was (Author: thetaphi):
bq. There's one thing I'm not fully understand. According to this policy, how 
do we handle links in Solr javadocs to Lucene javadocs. To be strictly 
consistent, those should be relative links for website, but absolute links for 
tar.gz (because Lucene and Solr archives are separately distributed). ?

Sorry for not having a full answer to this on the weekend. The proposal to 
split Lucene and Solr on the development mailinglist was something I did not 
want to make publish.

With that background, I can now confirm my previous statements:
- If solr splits off lucene, the links between both projects need to be 
absolute anyways. Or - removed alltogether from Javadocs, as it's to fragile
- The recent changes also confirm my previous problem with absolute links: Once 
projects split, the current Javadoc repository needs to be moved possibly to 
another *.apache.org subdomain. Of course we will have redirects, but 
nevertheless I don't want to introduce a redirect for every link anywhere in 
the Javadocs. So relative links that link between modules of Solr and/or Lucene 
should be relative, otherwise it's a nightmare to maintain afterwards.

These two reasons are my main arguments for having only relative links between 
modules in the "overall documentation". For individual Maven artifacts I don't 
care, but there the links should be absolute (with configurable location) or 
removed altogether. Somebody using the "javadoc" maven artifact just needs docs 
for the current method heshe is looking at, so we could possibly also remove 
all links there.

> Port documentation task to gradle
> -
>
> Key: LUCENE-9321
> URL: https://issues.apache.org/jira/browse/LUCENE-9321
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: general/build
>Reporter: Tomoko Uchida
>Assignee: Uwe Schindler
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: screenshot-1.png
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> This is a placeholder issue for porting ant "documentation" task to gradle. 
> The generated documents should be able to be published on lucene.apache.org 
> web site on "as-is" basis.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] romseygeek commented on pull request #1467: LUCENE-9350: Don't hold references to large automata on FuzzyQuery

2020-05-05 Thread GitBox


romseygeek commented on pull request #1467:
URL: https://github.com/apache/lucene-solr/pull/1467#issuecomment-623988037


   I think this is ready @madrob ?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] uschindler commented on a change in pull request #1467: LUCENE-9350: Don't hold references to large automata on FuzzyQuery

2020-05-05 Thread GitBox


uschindler commented on a change in pull request #1467:
URL: https://github.com/apache/lucene-solr/pull/1467#discussion_r420025857



##
File path: lucene/core/src/java/org/apache/lucene/search/FuzzyTermsEnum.java
##
@@ -88,43 +89,44 @@
* @throws IOException if there is a low-level IO error
*/
   public FuzzyTermsEnum(Terms terms, Term term, int maxEdits, int 
prefixLength, boolean transpositions) throws IOException {
-this(terms, term, stringToUTF32(term.text()), maxEdits, prefixLength, 
transpositions);
-  }
-
-  private FuzzyTermsEnum(Terms terms, Term term, int[] codePoints, int 
maxEdits, int prefixLength, boolean transpositions) throws IOException {
-this(terms, new AttributeSource(), term, codePoints.length, maxEdits,
-buildAutomata(term.text(), codePoints, prefixLength, transpositions, 
maxEdits));
+this(terms, new AttributeSource(), term, () -> new 
FuzzyAutomatonBuilder(term.text(), maxEdits, prefixLength, transpositions));
   }
 
   /**
* Constructor for enumeration of all terms from specified 
reader which share a prefix of
* length prefixLength with term and which have at 
most {@code maxEdits} edits.
* 
-   * After calling the constructor the enumeration is already pointing to the 
first 
-   * valid term if such a term exists. 
-   * 
+   * After calling the constructor the enumeration is already pointing to the 
first
+   * valid term if such a term exists.
+   *
* @param terms Delivers terms.
-   * @param atts {@link AttributeSource} created by the rewrite method of 
{@link MultiTermQuery}
-   *  that contains information about competitive boosts during 
rewrite
+   * @param atts An AttributeSource used to share automata between segments
* @param term Pattern term.
* @param maxEdits Maximum edit distance.
-   * @param automata An array of levenshtein automata to match against terms,
-   * see {@link #buildAutomata(String, int[], int, boolean, 
int)}
+   * @param prefixLength the length of the required common prefix
+   * @param transpositions whether transpositions should count as a single edit
* @throws IOException if there is a low-level IO error
*/
-  public FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, int 
termLength,
-  final int maxEdits, CompiledAutomaton[] automata) throws IOException {
+  FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, int maxEdits, 
int prefixLength, boolean transpositions) throws IOException {
+this(terms, atts, term, () -> new FuzzyAutomatonBuilder(term.text(), 
maxEdits, prefixLength, transpositions));
+  }
+
+  private FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, 
Supplier automatonBuilder) throws IOException {
 
-this.maxEdits = maxEdits;
 this.terms = terms;
-this.term = term;
 this.atts = atts;
-this.termLength = termLength;
+this.term = term;
 
 this.maxBoostAtt = 
atts.addAttribute(MaxNonCompetitiveBoostAttribute.class);
 this.boostAtt = atts.addAttribute(BoostAttribute.class);
 
-this.automata = automata;
+atts.addAttributeImpl(new AutomatonAttributeImpl());

Review comment:
   I think that's fine here. There is only one implementation that's all 
internally used.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] uschindler commented on a change in pull request #1467: LUCENE-9350: Don't hold references to large automata on FuzzyQuery

2020-05-05 Thread GitBox


uschindler commented on a change in pull request #1467:
URL: https://github.com/apache/lucene-solr/pull/1467#discussion_r420028347



##
File path: lucene/core/src/java/org/apache/lucene/search/FuzzyTermsEnum.java
##
@@ -88,43 +89,44 @@
* @throws IOException if there is a low-level IO error
*/
   public FuzzyTermsEnum(Terms terms, Term term, int maxEdits, int 
prefixLength, boolean transpositions) throws IOException {
-this(terms, term, stringToUTF32(term.text()), maxEdits, prefixLength, 
transpositions);
-  }
-
-  private FuzzyTermsEnum(Terms terms, Term term, int[] codePoints, int 
maxEdits, int prefixLength, boolean transpositions) throws IOException {
-this(terms, new AttributeSource(), term, codePoints.length, maxEdits,
-buildAutomata(term.text(), codePoints, prefixLength, transpositions, 
maxEdits));
+this(terms, new AttributeSource(), term, () -> new 
FuzzyAutomatonBuilder(term.text(), maxEdits, prefixLength, transpositions));
   }
 
   /**
* Constructor for enumeration of all terms from specified 
reader which share a prefix of
* length prefixLength with term and which have at 
most {@code maxEdits} edits.
* 
-   * After calling the constructor the enumeration is already pointing to the 
first 
-   * valid term if such a term exists. 
-   * 
+   * After calling the constructor the enumeration is already pointing to the 
first
+   * valid term if such a term exists.
+   *
* @param terms Delivers terms.
-   * @param atts {@link AttributeSource} created by the rewrite method of 
{@link MultiTermQuery}
-   *  that contains information about competitive boosts during 
rewrite
+   * @param atts An AttributeSource used to share automata between segments
* @param term Pattern term.
* @param maxEdits Maximum edit distance.
-   * @param automata An array of levenshtein automata to match against terms,
-   * see {@link #buildAutomata(String, int[], int, boolean, 
int)}
+   * @param prefixLength the length of the required common prefix
+   * @param transpositions whether transpositions should count as a single edit
* @throws IOException if there is a low-level IO error
*/
-  public FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, int 
termLength,
-  final int maxEdits, CompiledAutomaton[] automata) throws IOException {
+  FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, int maxEdits, 
int prefixLength, boolean transpositions) throws IOException {
+this(terms, atts, term, () -> new FuzzyAutomatonBuilder(term.text(), 
maxEdits, prefixLength, transpositions));
+  }
+
+  private FuzzyTermsEnum(Terms terms, AttributeSource atts, Term term, 
Supplier automatonBuilder) throws IOException {
 
-this.maxEdits = maxEdits;
 this.terms = terms;
-this.term = term;
 this.atts = atts;
-this.termLength = termLength;
+this.term = term;
 
 this.maxBoostAtt = 
atts.addAttribute(MaxNonCompetitiveBoostAttribute.class);
 this.boostAtt = atts.addAttribute(BoostAttribute.class);
 
-this.automata = automata;
+atts.addAttributeImpl(new AutomatonAttributeImpl());
+AutomatonAttribute aa = atts.addAttribute(AutomatonAttribute.class);

Review comment:
   GetAttribute is also fine, because you added the impl a line before. If 
the attribute wouldn't be there, adding it won't work, because it's all 
private. So all: That's just a matter of preference. I'd check how we do this 
at other places. 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14426) forbidden api error during precommit DateMathFunction

2020-05-05 Thread Alan Woodward (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099798#comment-17099798
 ] 

Alan Woodward commented on SOLR-14426:
--

I've also run into this, it seems like it's due to DateMathValueFunction being 
defined as an old-style package-private class after the main class, rather than 
as a static nested class.  We should probably try and find all instances of 
these in the codebase and clear them up.

> forbidden api error during precommit DateMathFunction
> -
>
> Key: SOLR-14426
> URL: https://issues.apache.org/jira/browse/SOLR-14426
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mike Drob
>Priority: Major
>
> When running `./gradlew precommit` I'll occasionally see
> {code}
> * What went wrong:
> Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'.
> > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API 
> > calls failed while scanning class 
> > 'org.apache.solr.analytics.function.mapping.DateMathFunction' 
> > (DateMathFunction.java): java.lang.ClassNotFoundException: 
> > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while 
> > looking up details about referenced class 
> > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction')
> {code}
> `./gradlew clean` fixes this, but I don't understand what or why this 
> happens. Feels like a gradle issue?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14426) forbidden api error during precommit DateMathFunction

2020-05-05 Thread Alan Woodward (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099808#comment-17099808
 ] 

Alan Woodward commented on SOLR-14426:
--

OK, from a quick grep it appears that there are 290 old-style package-private 
classes in the Solr code base.  It seems odd that it's always this one that 
fails forbiddenApis though - [~uschindler] any ideas?

> forbidden api error during precommit DateMathFunction
> -
>
> Key: SOLR-14426
> URL: https://issues.apache.org/jira/browse/SOLR-14426
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mike Drob
>Priority: Major
>
> When running `./gradlew precommit` I'll occasionally see
> {code}
> * What went wrong:
> Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'.
> > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API 
> > calls failed while scanning class 
> > 'org.apache.solr.analytics.function.mapping.DateMathFunction' 
> > (DateMathFunction.java): java.lang.ClassNotFoundException: 
> > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while 
> > looking up details about referenced class 
> > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction')
> {code}
> `./gradlew clean` fixes this, but I don't understand what or why this 
> happens. Feels like a gradle issue?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection

2020-05-05 Thread Endika Posadas (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099970#comment-17099970
 ] 

Endika Posadas commented on SOLR-14458:
---

I found the problem: 
It happens that in SegmentsInfoRequestHandler, when the information is 
requested, it gets the Index Writer, holding a read lock on it. However, the 
read lock is only released when core Info is requested too, which is not the 
default.

So in most cases, the instance will be unable to perform write  after 
requesting segments info.

 

 

> Solr Replica locked in recovering state after a Zookeeper disconnection
> ---
>
> Key: SOLR-14458
> URL: https://issues.apache.org/jira/browse/SOLR-14458
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 8.4.1
> Environment: A Solr cluster with 2 replicas that each has 2 shards 
> split across 2 Windows VMS.
> They use a 3 replica zookeeper across 3 vms.
>Reporter: Endika Posadas
>Priority: Major
> Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, 
> solr-thread-dump.log, solr.log, solrrecovering.png
>
>
> In a solr cluster, a Solr instance containing two shards has lost connection 
> with zookeeper. Upon reconnecting, it has checked the status with the leader 
> and start a recovery. However, it's stuck in recovering status without making 
> further progress (has been like that for days now).
>  
> Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is  
> trying to acquire the lock to createa new Index Writer: `at 
> org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)`
>  (
> after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the 
> ReentrantLock it's waiting for is never released. Moreover, no thread can be 
> found holding the lock, leaving restarting Solr as the only solution.
> There is no Error in the logs that can help with the issue. I have attached 
> solr.log and a grep with node 7 lines, as well as a thread dump.
>  
> There is also no other recovery currently running. In Solr metrics, 4 
> recoveries have started, 3 have completed and 1 is running (forever).
>  
> My hypothesis is that 
> org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore,
>  boolean) was called once but for some reason openIndexWriter was skipped.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection

2020-05-05 Thread Endika Posadas (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099972#comment-17099972
 ] 

Endika Posadas commented on SOLR-14458:
---

As I was creating a patch I just found out that it got fixed yesterday as part 
of SOLR-14431.

SO proceeding to mark the issue as fixed.

> Solr Replica locked in recovering state after a Zookeeper disconnection
> ---
>
> Key: SOLR-14458
> URL: https://issues.apache.org/jira/browse/SOLR-14458
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 8.4.1
> Environment: A Solr cluster with 2 replicas that each has 2 shards 
> split across 2 Windows VMS.
> They use a 3 replica zookeeper across 3 vms.
>Reporter: Endika Posadas
>Priority: Major
> Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, 
> solr-thread-dump.log, solr.log, solrrecovering.png
>
>
> In a solr cluster, a Solr instance containing two shards has lost connection 
> with zookeeper. Upon reconnecting, it has checked the status with the leader 
> and start a recovery. However, it's stuck in recovering status without making 
> further progress (has been like that for days now).
>  
> Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is  
> trying to acquire the lock to createa new Index Writer: `at 
> org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)`
>  (
> after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the 
> ReentrantLock it's waiting for is never released. Moreover, no thread can be 
> found holding the lock, leaving restarting Solr as the only solution.
> There is no Error in the logs that can help with the issue. I have attached 
> solr.log and a grep with node 7 lines, as well as a thread dump.
>  
> There is also no other recovery currently running. In Solr metrics, 4 
> recoveries have started, 3 have completed and 1 is running (forever).
>  
> My hypothesis is that 
> org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore,
>  boolean) was called once but for some reason openIndexWriter was skipped.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Resolved] (SOLR-14458) Solr Replica locked in recovering state after a Zookeeper disconnection

2020-05-05 Thread Endika Posadas (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Endika Posadas resolved SOLR-14458.
---
Resolution: Duplicate

> Solr Replica locked in recovering state after a Zookeeper disconnection
> ---
>
> Key: SOLR-14458
> URL: https://issues.apache.org/jira/browse/SOLR-14458
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 8.4.1
> Environment: A Solr cluster with 2 replicas that each has 2 shards 
> split across 2 Windows VMS.
> They use a 3 replica zookeeper across 3 vms.
>Reporter: Endika Posadas
>Priority: Major
> Attachments: image-2020-05-05-09-47-27-854.png, replica7.log, 
> solr-thread-dump.log, solr.log, solrrecovering.png
>
>
> In a solr cluster, a Solr instance containing two shards has lost connection 
> with zookeeper. Upon reconnecting, it has checked the status with the leader 
> and start a recovery. However, it's stuck in recovering status without making 
> further progress (has been like that for days now).
>  
> Upon checking a thread dump, `recoveryExecutor-7-thread-3-processing-n` is  
> trying to acquire the lock to createa new Index Writer: `at 
> org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:179)`
>  (
> after lock(iwLock.writeLock()){color:#cc7832};{color}). However, the 
> ReentrantLock it's waiting for is never released. Moreover, no thread can be 
> found holding the lock, leaving restarting Solr as the only solution.
> There is no Error in the logs that can help with the issue. I have attached 
> solr.log and a grep with node 7 lines, as well as a thread dump.
>  
> There is also no other recovery currently running. In Solr metrics, 4 
> recoveries have started, 3 have completed and 1 is running (forever).
>  
> My hypothesis is that 
> org.apache.solr.update.DefaultSolrCoreState#closeIndexWriter(org.apache.solr.core.SolrCore,
>  boolean) was called once but for some reason openIndexWriter was skipped.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] HoustonPutman commented on pull request #1480: SOLR-14456: Fix Content-Type header forwarding on compressed requests

2020-05-05 Thread GitBox


HoustonPutman commented on pull request #1480:
URL: https://github.com/apache/lucene-solr/pull/1480#issuecomment-624103813


   Yeah, I think it would be worthwhile for the two classes to act similarly in 
this regard. I say go ahead and refactor.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14426) forbidden api error during precommit DateMathFunction

2020-05-05 Thread Uwe Schindler (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1714#comment-1714
 ] 

Uwe Schindler commented on SOLR-14426:
--

This is the usual problem with the old-style package-private classes which are 
in same source file than the main class. I fixed some of them already, but they 
are bad Java style. The only workaround is to do ant clean. The problem is that 
Java/Javac/Gradle/Ant can't figure out where the class file is coming from. 
When it gets recompiled, the leftover class without source file is a leftover.

We can only fix that at some point. Eclipse is also getting angry about those 
quite often.

To fix this, the classes should be moverd as static inner class into the main 
class. In most cases this can be done with copypaste, but there's no automated  
refactoring in Eclipse. Maybe other IDEs have better refactoring tools.

> forbidden api error during precommit DateMathFunction
> -
>
> Key: SOLR-14426
> URL: https://issues.apache.org/jira/browse/SOLR-14426
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mike Drob
>Priority: Major
>
> When running `./gradlew precommit` I'll occasionally see
> {code}
> * What went wrong:
> Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'.
> > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API 
> > calls failed while scanning class 
> > 'org.apache.solr.analytics.function.mapping.DateMathFunction' 
> > (DateMathFunction.java): java.lang.ClassNotFoundException: 
> > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while 
> > looking up details about referenced class 
> > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction')
> {code}
> `./gradlew clean` fixes this, but I don't understand what or why this 
> happens. Feels like a gradle issue?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-14426) forbidden api error during precommit DateMathFunction

2020-05-05 Thread Uwe Schindler (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1714#comment-1714
 ] 

Uwe Schindler edited comment on SOLR-14426 at 5/5/20, 3:30 PM:
---

This is the usual problem with the old-style package-private classes which are 
in same source file than the main class. I fixed some of them already, but they 
are bad Java style. The only workaround is to do ant clean. The problem is that 
Java/Javac/Gradle/Ant can't figure out where the class file is coming from. 
When it gets recompiled, the leftover class without source file is a relic 
leading to those failures. It's not really forbiddenapis fault, because the 
class file should not be there at all.

We can only fix that at some point. Eclipse is also getting angry about those 
quite often.

To fix this, the classes should be moverd as static inner class into the main 
class. In most cases this can be done with copypaste, but there's no automated  
refactoring in Eclipse. Maybe other IDEs have better refactoring tools.


was (Author: thetaphi):
This is the usual problem with the old-style package-private classes which are 
in same source file than the main class. I fixed some of them already, but they 
are bad Java style. The only workaround is to do ant clean. The problem is that 
Java/Javac/Gradle/Ant can't figure out where the class file is coming from. 
When it gets recompiled, the leftover class without source file is a leftover.

We can only fix that at some point. Eclipse is also getting angry about those 
quite often.

To fix this, the classes should be moverd as static inner class into the main 
class. In most cases this can be done with copypaste, but there's no automated  
refactoring in Eclipse. Maybe other IDEs have better refactoring tools.

> forbidden api error during precommit DateMathFunction
> -
>
> Key: SOLR-14426
> URL: https://issues.apache.org/jira/browse/SOLR-14426
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mike Drob
>Priority: Major
>
> When running `./gradlew precommit` I'll occasionally see
> {code}
> * What went wrong:
> Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'.
> > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API 
> > calls failed while scanning class 
> > 'org.apache.solr.analytics.function.mapping.DateMathFunction' 
> > (DateMathFunction.java): java.lang.ClassNotFoundException: 
> > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while 
> > looking up details about referenced class 
> > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction')
> {code}
> `./gradlew clean` fixes this, but I don't understand what or why this 
> happens. Feels like a gradle issue?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-10932) install solr service service command fails

2020-05-05 Thread Markus Mandalka (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100014#comment-17100014
 ] 

Markus Mandalka commented on SOLR-10932:


Bad news: Sorry, i created a dupe of this problem in 
https://issues.apache.org/jira/browse/SOLR-11853

Goog news: was fixed 20. jan 2019, so this ticket can be closed.

> install solr service service command fails
> --
>
> Key: SOLR-10932
> URL: https://issues.apache.org/jira/browse/SOLR-10932
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 6.6
> Environment: Suse linux
>Reporter: Susheel Kumar
>Priority: Minor
>  Labels: easyfix, newbie, patch
> Attachments: SOLR-10932.patch
>
>
> In SUSE distribution,  "service --version" commands always fail and abort the 
> solr installation with printing the error  "Script requires the 'service' 
> command" 
> We can fix it by changing "service --version" to "service --help" command. 
> Shawn's test results
> ==
> This is what I get with OS versions that I have access to when running
> "service --version":
> CentOS 7:
> service ver. 1.1
> Ubuntu 16:
> service ver. 0.91-ubuntu1
> Ubuntu 14:
> service ver. 0.91-ubuntu1
> CentOS 6:
> service ver. 0.91
> Debian 6:
> service ver. 0.91-ubuntu1
> Sparc Solaris 10:
> bash: service: command not found
> =



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-10932) install solr service service command fails

2020-05-05 Thread Markus Mandalka (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100014#comment-17100014
 ] 

Markus Mandalka edited comment on SOLR-10932 at 5/5/20, 3:40 PM:
-

Bad news: Sorry, i created a dupe of this problem in 
https://issues.apache.org/jira/browse/SOLR-11853

Goog news: was fixed 2. jan 2019, so this ticket can be closed.


was (Author: mandalka):
Bad news: Sorry, i created a dupe of this problem in 
https://issues.apache.org/jira/browse/SOLR-11853

Goog news: was fixed 20. jan 2019, so this ticket can be closed.

> install solr service service command fails
> --
>
> Key: SOLR-10932
> URL: https://issues.apache.org/jira/browse/SOLR-10932
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 6.6
> Environment: Suse linux
>Reporter: Susheel Kumar
>Priority: Minor
>  Labels: easyfix, newbie, patch
> Attachments: SOLR-10932.patch
>
>
> In SUSE distribution,  "service --version" commands always fail and abort the 
> solr installation with printing the error  "Script requires the 'service' 
> command" 
> We can fix it by changing "service --version" to "service --help" command. 
> Shawn's test results
> ==
> This is what I get with OS versions that I have access to when running
> "service --version":
> CentOS 7:
> service ver. 1.1
> Ubuntu 16:
> service ver. 0.91-ubuntu1
> Ubuntu 14:
> service ver. 0.91-ubuntu1
> CentOS 6:
> service ver. 0.91
> Debian 6:
> service ver. 0.91-ubuntu1
> Sparc Solaris 10:
> bash: service: command not found
> =



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Created] (LUCENE-9362) ExpressionValueSource has buggy rewrite method

2020-05-05 Thread Dmitry Emets (Jira)
Dmitry Emets created LUCENE-9362:


 Summary: ExpressionValueSource has buggy rewrite method
 Key: LUCENE-9362
 URL: https://issues.apache.org/jira/browse/LUCENE-9362
 Project: Lucene - Core
  Issue Type: Bug
  Components: modules/expressions
Reporter: Dmitry Emets


ExpressionValuesSource does not actually rewrites himself due to small mistake 
in check of inner rewrites.
{code:java}
changed |= (rewritten[i] == variables[i]);
{code}
should be changed to
{code:java}
changed |= (rewritten[i] != variables[i]);
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (LUCENE-9362) ExpressionValueSource has buggy rewrite method

2020-05-05 Thread Dmitry Emets (Jira)


 [ 
https://issues.apache.org/jira/browse/LUCENE-9362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Emets updated LUCENE-9362:
-
Description: 
ExpressionValuesSource does not actually rewrite himself due to small mistake 
in check of inner rewrites.
{code:java}
changed |= (rewritten[i] == variables[i]);
{code}
should be changed to
{code:java}
changed |= (rewritten[i] != variables[i]);
{code}

  was:
ExpressionValuesSource does not actually rewrites himself due to small mistake 
in check of inner rewrites.
{code:java}
changed |= (rewritten[i] == variables[i]);
{code}
should be changed to
{code:java}
changed |= (rewritten[i] != variables[i]);
{code}


> ExpressionValueSource has buggy rewrite method
> --
>
> Key: LUCENE-9362
> URL: https://issues.apache.org/jira/browse/LUCENE-9362
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/expressions
>Reporter: Dmitry Emets
>Priority: Minor
>
> ExpressionValuesSource does not actually rewrite himself due to small mistake 
> in check of inner rewrites.
> {code:java}
> changed |= (rewritten[i] == variables[i]);
> {code}
> should be changed to
> {code:java}
> changed |= (rewritten[i] != variables[i]);
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (LUCENE-9362) ExpressionValueSource has buggy rewrite method

2020-05-05 Thread Dmitry Emets (Jira)


 [ 
https://issues.apache.org/jira/browse/LUCENE-9362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Emets updated LUCENE-9362:
-
Description: 
ExpressionValuesSource does not actually rewrite itself due to small mistake in 
check of inner rewrites.
{code:java}
changed |= (rewritten[i] == variables[i]);
{code}
should be changed to
{code:java}
changed |= (rewritten[i] != variables[i]);
{code}

  was:
ExpressionValuesSource does not actually rewrite himself due to small mistake 
in check of inner rewrites.
{code:java}
changed |= (rewritten[i] == variables[i]);
{code}
should be changed to
{code:java}
changed |= (rewritten[i] != variables[i]);
{code}


> ExpressionValueSource has buggy rewrite method
> --
>
> Key: LUCENE-9362
> URL: https://issues.apache.org/jira/browse/LUCENE-9362
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/expressions
>Reporter: Dmitry Emets
>Priority: Minor
>
> ExpressionValuesSource does not actually rewrite itself due to small mistake 
> in check of inner rewrites.
> {code:java}
> changed |= (rewritten[i] == variables[i]);
> {code}
> should be changed to
> {code:java}
> changed |= (rewritten[i] != variables[i]);
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] emetsds opened a new pull request #1485: LUCENE-9362: Fix rewriting check in ExpressionValueSource

2020-05-05 Thread GitBox


emetsds opened a new pull request #1485:
URL: https://github.com/apache/lucene-solr/pull/1485


   
   
   
   # Description
   
   [LUCENE-9362](https://issues.apache.org/jira/browse/LUCENE-9362): 
ExpressionValuesSource does not actually rewrite itself due to small mistake in 
check of inner rewrites.
   
   # Solution
   
   `changed |= (rewritten[i] == variables[i]);`
   should be changed to
   `changed |= (rewritten[i] != variables[i]);`
   
   # Tests
   
   I added a test that checks that the ExpressionValueSource was rewritten when 
the inner ValueSource was rewritten, and vice versa
   
   # Checklist
   
   Please review the following and check all that apply:
   
   - [+] I have reviewed the guidelines for [How to 
Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms 
to the standards described there to the best of my ability.
   - [+] I have created a Jira issue and added the issue ID to my pull request 
title.
   - [+] I have given Solr maintainers 
[access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork)
 to contribute to my PR branch. (optional but recommended)
   - [+] I have developed this patch against the `master` branch.
   - [+] I have run `gradlew precommit` and the appropriate test suite.
   - [+] I have added tests for my changes.
   - [] I have added documentation for the [Ref 
Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) 
(for Solr changes only).
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] sigram opened a new pull request #1486: SOLR-14423: Use SolrClientCache instance managed by CoreContainer

2020-05-05 Thread GitBox


sigram opened a new pull request #1486:
URL: https://github.com/apache/lucene-solr/pull/1486


   See Jira for more details.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14423) static caches in StreamHandler ought to move to CoreContainer lifecycle

2020-05-05 Thread Andrzej Bialecki (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100106#comment-17100106
 ] 

Andrzej Bialecki commented on SOLR-14423:
-

I created a PR with the initial attempt at fixing this:
 * {{CoreContainer.solrClientCache}} is initialized on {{load()}} and closed on 
{{shutdown()}}
 * I modified the StreamHandler, GraphHandler, ReindexCollectionCmd, ColStatus, 
SolrReporter and a few other places to use this instance.
 * probably the most controversial change is in the 
{{org.apache.solr.handler.sql.CalciteSolrDriver}} and {{SolrSchema}} / 
{{SolrTable}}. These objects are initialized in a way that makes it very 
difficult to pass an existing instance of SolrClientCache. For this reason I 
decided to create a single instance of the cache in {{SolrSchema}} and manage 
its lifecycle through the {{Connection}} - which required adding a wrapper to 
intercept the {{Connection.close()}} call. [~jbernste] I would appreciate your 
feedback.
 * I also added an instance of {{CoreContainer.objectCache}} - this is not 
strictly required by the other changes but it illustrates how we could use a 
single generic object cache at the CC level instead of adding many specialized 
accessors like the one for SolrClientCache. We can decide which way to go in 
the future.

> static caches in StreamHandler ought to move to CoreContainer lifecycle
> ---
>
> Key: SOLR-14423
> URL: https://issues.apache.org/jira/browse/SOLR-14423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: David Smiley
>Assignee: Andrzej Bialecki
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> StreamHandler (at "/stream") has several statically declared caches.  I think 
> this is problematic, such as in testing wherein multiple nodes could be in 
> the same JVM.  One of them is more serious -- SolrClientCache which is 
> closed/cleared via a SolrCore close hook.  That's bad for performance but 
> also dangerous since another core might want to use one of these clients!
> CC [~jbernste]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Resolved] (SOLR-10932) install solr service service command fails

2020-05-05 Thread Cassandra Targett (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett resolved SOLR-10932.
--
Resolution: Duplicate

> install solr service service command fails
> --
>
> Key: SOLR-10932
> URL: https://issues.apache.org/jira/browse/SOLR-10932
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 6.6
> Environment: Suse linux
>Reporter: Susheel Kumar
>Priority: Minor
>  Labels: easyfix, newbie, patch
> Attachments: SOLR-10932.patch
>
>
> In SUSE distribution,  "service --version" commands always fail and abort the 
> solr installation with printing the error  "Script requires the 'service' 
> command" 
> We can fix it by changing "service --version" to "service --help" command. 
> Shawn's test results
> ==
> This is what I get with OS versions that I have access to when running
> "service --version":
> CentOS 7:
> service ver. 1.1
> Ubuntu 16:
> service ver. 0.91-ubuntu1
> Ubuntu 14:
> service ver. 0.91-ubuntu1
> CentOS 6:
> service ver. 0.91
> Debian 6:
> service ver. 0.91-ubuntu1
> Sparc Solaris 10:
> bash: service: command not found
> =



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Resolved] (SOLR-14425) Fix ZK sync usage to be synchronous (blocking)

2020-05-05 Thread David Smiley (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved SOLR-14425.
-
Resolution: Not A Problem

Not actually a problem; it was my misunderstanding.

> Fix ZK sync usage to be synchronous (blocking)
> --
>
> Key: SOLR-14425
> URL: https://issues.apache.org/jira/browse/SOLR-14425
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Minor
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> As of this writing, we only use one call to ZK's "sync" method.  It's related 
> to collection aliases -- I added this.  I discovered I misunderstood the 
> semantics of the API; it syncs in the background and thus returns 
> immediately.  Looking at ZK's sync CLI command and Curator both made me 
> realize my folly.  I'm considering this only a "minor" issue because I'm not 
> sure I've seen a bug from this; or maybe I did in spooky test failures over a 
> year ago -- I'm not sure.  And we don't use this pervasively (yet).
> It occurred to me that if Solr embraced the Curator framework abstraction 
> over ZooKeeper, I would not have fallen into that trap.  I'll file a separate 
> issue for that.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14423) static caches in StreamHandler ought to move to CoreContainer lifecycle

2020-05-05 Thread Joel Bernstein (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100212#comment-17100212
 ] 

Joel Bernstein commented on SOLR-14423:
---

[~ab], I'll take a look at the PR this week.

> static caches in StreamHandler ought to move to CoreContainer lifecycle
> ---
>
> Key: SOLR-14423
> URL: https://issues.apache.org/jira/browse/SOLR-14423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: David Smiley
>Assignee: Andrzej Bialecki
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> StreamHandler (at "/stream") has several statically declared caches.  I think 
> this is problematic, such as in testing wherein multiple nodes could be in 
> the same JVM.  One of them is more serious -- SolrClientCache which is 
> closed/cleared via a SolrCore close hook.  That's bad for performance but 
> also dangerous since another core might want to use one of these clients!
> CC [~jbernste]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14426) forbidden api error during precommit DateMathFunction

2020-05-05 Thread Mike Drob (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100221#comment-17100221
 ] 

Mike Drob commented on SOLR-14426:
--

Thanks for explaining what is going on, [~uschindler].

IntelliJ has a report for this, we have 437 instances of this issue apparently. 
I'll try to tackle a few of them and see how well it works.

> forbidden api error during precommit DateMathFunction
> -
>
> Key: SOLR-14426
> URL: https://issues.apache.org/jira/browse/SOLR-14426
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mike Drob
>Priority: Major
>
> When running `./gradlew precommit` I'll occasionally see
> {code}
> * What went wrong:
> Execution failed for task ':solr:contrib:analytics:forbiddenApisMain'.
> > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API 
> > calls failed while scanning class 
> > 'org.apache.solr.analytics.function.mapping.DateMathFunction' 
> > (DateMathFunction.java): java.lang.ClassNotFoundException: 
> > org.apache.solr.analytics.function.mapping.DateMathValueFunction (while 
> > looking up details about referenced class 
> > 'org.apache.solr.analytics.function.mapping.DateMathValueFunction')
> {code}
> `./gradlew clean` fixes this, but I don't understand what or why this 
> happens. Feels like a gradle issue?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design

2020-05-05 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100244#comment-17100244
 ] 

ASF subversion and git services commented on SOLR-14173:


Commit 3f227e6b5502a256a187bd48760603e36f8461ea in lucene-solr's branch 
refs/heads/branch_8x from Cassandra Targett
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3f227e6 ]

SOLR-14173: add entry to CHANGES.txt


> Ref Guide Redesign Phase 1: Page Design
> ---
>
> Key: SOLR-14173
> URL: https://issues.apache.org/jira/browse/SOLR-14173
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
> Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, 
> gray-left-nav.png
>
>
> The current design of the Ref Guide was essentially copied from a 
> Jekyll-based documentation theme 
> (https://idratherbewriting.com/documentation-theme-jekyll/), which had a 
> couple important benefits for that time:
> * It was well-documented and since I had little experience with Jekyll and 
> its Liquid templates and since I was the one doing it, I wanted to make it as 
> easy on myself as possible
> * It was designed for documentation specifically so took care of all the 
> things like inter-page navigation, etc.
> * It helped us get from Confluence to our current system quickly
> It had some drawbacks, though:
> * It wasted a lot of space on the page
> * The theme was built for Markdown files, so did not take advantage of the 
> features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one 
> big example - the plugin could create it at build time, but the theme 
> included JS to do it as the page loads, so we use the JS)
> * It had a lot of JS and overlapping CSS files. While it used Bootstrap it 
> used a customized CSS on top of it for theming that made modifications 
> complex (it was hard to figure out how exactly a change would behave)
> * With all the stuff I'd changed in my bumbling way just to get things to 
> work back then, I broke a lot of the stuff Bootstrap is supposed to give us 
> in terms of responsiveness and making the Guide usable even on smaller screen 
> sizes.
> After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF 
> (SOLR-13782), I wanted to try to set us up for a more flexible system. We 
> need it for things like Joel's work on the visual guide for streaming 
> expressions (SOLR-13105), and in order to implement other ideas we might have 
> on how to present information in the future.
> I view this issue as a phase 1 of an overall redesign that I've already 
> started in a local branch. I'll explain in a comment the changes I've already 
> made, and will use this issue to create and push a branch where we can 
> discuss in more detail.
> Phase 1 here will be under-the-hood CSS/JS changes + overall page layout 
> changes.
> Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of 
> the Guide.
> Phase 3 (issue TBD) will explore moving us from Jekyll to another static site 
> generator that is better suited for our content format, file types, and build 
> conventions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design

2020-05-05 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100242#comment-17100242
 ] 

ASF subversion and git services commented on SOLR-14173:


Commit c74f951c1c210d112c19577b44a39eaeaaf126a5 in lucene-solr's branch 
refs/heads/branch_8x from Cassandra Targett
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c74f951 ]

SOLR-14173: Change left nav item highlighting to fix menu jumpiness when 
hovering/selecting


> Ref Guide Redesign Phase 1: Page Design
> ---
>
> Key: SOLR-14173
> URL: https://issues.apache.org/jira/browse/SOLR-14173
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
> Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, 
> gray-left-nav.png
>
>
> The current design of the Ref Guide was essentially copied from a 
> Jekyll-based documentation theme 
> (https://idratherbewriting.com/documentation-theme-jekyll/), which had a 
> couple important benefits for that time:
> * It was well-documented and since I had little experience with Jekyll and 
> its Liquid templates and since I was the one doing it, I wanted to make it as 
> easy on myself as possible
> * It was designed for documentation specifically so took care of all the 
> things like inter-page navigation, etc.
> * It helped us get from Confluence to our current system quickly
> It had some drawbacks, though:
> * It wasted a lot of space on the page
> * The theme was built for Markdown files, so did not take advantage of the 
> features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one 
> big example - the plugin could create it at build time, but the theme 
> included JS to do it as the page loads, so we use the JS)
> * It had a lot of JS and overlapping CSS files. While it used Bootstrap it 
> used a customized CSS on top of it for theming that made modifications 
> complex (it was hard to figure out how exactly a change would behave)
> * With all the stuff I'd changed in my bumbling way just to get things to 
> work back then, I broke a lot of the stuff Bootstrap is supposed to give us 
> in terms of responsiveness and making the Guide usable even on smaller screen 
> sizes.
> After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF 
> (SOLR-13782), I wanted to try to set us up for a more flexible system. We 
> need it for things like Joel's work on the visual guide for streaming 
> expressions (SOLR-13105), and in order to implement other ideas we might have 
> on how to present information in the future.
> I view this issue as a phase 1 of an overall redesign that I've already 
> started in a local branch. I'll explain in a comment the changes I've already 
> made, and will use this issue to create and push a branch where we can 
> discuss in more detail.
> Phase 1 here will be under-the-hood CSS/JS changes + overall page layout 
> changes.
> Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of 
> the Guide.
> Phase 3 (issue TBD) will explore moving us from Jekyll to another static site 
> generator that is better suited for our content format, file types, and build 
> conventions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design

2020-05-05 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100243#comment-17100243
 ] 

ASF subversion and git services commented on SOLR-14173:


Commit ea07244e353da95c01552a1ade800f05d5edb7dd in lucene-solr's branch 
refs/heads/branch_8x from Cassandra Targett
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ea07244 ]

SOLR-14173: Don't use JQuery-Slim as it breaks the sidebar sub-menu system.


> Ref Guide Redesign Phase 1: Page Design
> ---
>
> Key: SOLR-14173
> URL: https://issues.apache.org/jira/browse/SOLR-14173
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
> Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, 
> gray-left-nav.png
>
>
> The current design of the Ref Guide was essentially copied from a 
> Jekyll-based documentation theme 
> (https://idratherbewriting.com/documentation-theme-jekyll/), which had a 
> couple important benefits for that time:
> * It was well-documented and since I had little experience with Jekyll and 
> its Liquid templates and since I was the one doing it, I wanted to make it as 
> easy on myself as possible
> * It was designed for documentation specifically so took care of all the 
> things like inter-page navigation, etc.
> * It helped us get from Confluence to our current system quickly
> It had some drawbacks, though:
> * It wasted a lot of space on the page
> * The theme was built for Markdown files, so did not take advantage of the 
> features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one 
> big example - the plugin could create it at build time, but the theme 
> included JS to do it as the page loads, so we use the JS)
> * It had a lot of JS and overlapping CSS files. While it used Bootstrap it 
> used a customized CSS on top of it for theming that made modifications 
> complex (it was hard to figure out how exactly a change would behave)
> * With all the stuff I'd changed in my bumbling way just to get things to 
> work back then, I broke a lot of the stuff Bootstrap is supposed to give us 
> in terms of responsiveness and making the Guide usable even on smaller screen 
> sizes.
> After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF 
> (SOLR-13782), I wanted to try to set us up for a more flexible system. We 
> need it for things like Joel's work on the visual guide for streaming 
> expressions (SOLR-13105), and in order to implement other ideas we might have 
> on how to present information in the future.
> I view this issue as a phase 1 of an overall redesign that I've already 
> started in a local branch. I'll explain in a comment the changes I've already 
> made, and will use this issue to create and push a branch where we can 
> discuss in more detail.
> Phase 1 here will be under-the-hood CSS/JS changes + overall page layout 
> changes.
> Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of 
> the Guide.
> Phase 3 (issue TBD) will explore moving us from Jekyll to another static site 
> generator that is better suited for our content format, file types, and build 
> conventions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design

2020-05-05 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100241#comment-17100241
 ] 

ASF subversion and git services commented on SOLR-14173:


Commit 6a01cb4a21974b56b35afd1c83d74b1407892da6 in lucene-solr's branch 
refs/heads/branch_8x from Cassandra Targett
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6a01cb4 ]

SOLR-14173: Ref Guide Redesign: upgrade bootstrap; change layout; consolidate 
CSS. See issue for list of changes.


> Ref Guide Redesign Phase 1: Page Design
> ---
>
> Key: SOLR-14173
> URL: https://issues.apache.org/jira/browse/SOLR-14173
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
> Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, 
> gray-left-nav.png
>
>
> The current design of the Ref Guide was essentially copied from a 
> Jekyll-based documentation theme 
> (https://idratherbewriting.com/documentation-theme-jekyll/), which had a 
> couple important benefits for that time:
> * It was well-documented and since I had little experience with Jekyll and 
> its Liquid templates and since I was the one doing it, I wanted to make it as 
> easy on myself as possible
> * It was designed for documentation specifically so took care of all the 
> things like inter-page navigation, etc.
> * It helped us get from Confluence to our current system quickly
> It had some drawbacks, though:
> * It wasted a lot of space on the page
> * The theme was built for Markdown files, so did not take advantage of the 
> features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one 
> big example - the plugin could create it at build time, but the theme 
> included JS to do it as the page loads, so we use the JS)
> * It had a lot of JS and overlapping CSS files. While it used Bootstrap it 
> used a customized CSS on top of it for theming that made modifications 
> complex (it was hard to figure out how exactly a change would behave)
> * With all the stuff I'd changed in my bumbling way just to get things to 
> work back then, I broke a lot of the stuff Bootstrap is supposed to give us 
> in terms of responsiveness and making the Guide usable even on smaller screen 
> sizes.
> After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF 
> (SOLR-13782), I wanted to try to set us up for a more flexible system. We 
> need it for things like Joel's work on the visual guide for streaming 
> expressions (SOLR-13105), and in order to implement other ideas we might have 
> on how to present information in the future.
> I view this issue as a phase 1 of an overall redesign that I've already 
> started in a local branch. I'll explain in a comment the changes I've already 
> made, and will use this issue to create and push a branch where we can 
> discuss in more detail.
> Phase 1 here will be under-the-hood CSS/JS changes + overall page layout 
> changes.
> Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of 
> the Guide.
> Phase 3 (issue TBD) will explore moving us from Jekyll to another static site 
> generator that is better suited for our content format, file types, and build 
> conventions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design

2020-05-05 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100245#comment-17100245
 ] 

ASF subversion and git services commented on SOLR-14173:


Commit d4dbd0b9e75cb0bd5b0188a0f70070b6867fd94b in lucene-solr's branch 
refs/heads/master from Cassandra Targett
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d4dbd0b ]

SOLR-14173: Add entry in CHANGES.txt


> Ref Guide Redesign Phase 1: Page Design
> ---
>
> Key: SOLR-14173
> URL: https://issues.apache.org/jira/browse/SOLR-14173
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
> Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, 
> gray-left-nav.png
>
>
> The current design of the Ref Guide was essentially copied from a 
> Jekyll-based documentation theme 
> (https://idratherbewriting.com/documentation-theme-jekyll/), which had a 
> couple important benefits for that time:
> * It was well-documented and since I had little experience with Jekyll and 
> its Liquid templates and since I was the one doing it, I wanted to make it as 
> easy on myself as possible
> * It was designed for documentation specifically so took care of all the 
> things like inter-page navigation, etc.
> * It helped us get from Confluence to our current system quickly
> It had some drawbacks, though:
> * It wasted a lot of space on the page
> * The theme was built for Markdown files, so did not take advantage of the 
> features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one 
> big example - the plugin could create it at build time, but the theme 
> included JS to do it as the page loads, so we use the JS)
> * It had a lot of JS and overlapping CSS files. While it used Bootstrap it 
> used a customized CSS on top of it for theming that made modifications 
> complex (it was hard to figure out how exactly a change would behave)
> * With all the stuff I'd changed in my bumbling way just to get things to 
> work back then, I broke a lot of the stuff Bootstrap is supposed to give us 
> in terms of responsiveness and making the Guide usable even on smaller screen 
> sizes.
> After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF 
> (SOLR-13782), I wanted to try to set us up for a more flexible system. We 
> need it for things like Joel's work on the visual guide for streaming 
> expressions (SOLR-13105), and in order to implement other ideas we might have 
> on how to present information in the future.
> I view this issue as a phase 1 of an overall redesign that I've already 
> started in a local branch. I'll explain in a comment the changes I've already 
> made, and will use this issue to create and push a branch where we can 
> discuss in more detail.
> Phase 1 here will be under-the-hood CSS/JS changes + overall page layout 
> changes.
> Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of 
> the Guide.
> Phase 3 (issue TBD) will explore moving us from Jekyll to another static site 
> generator that is better suited for our content format, file types, and build 
> conventions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Updated] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design

2020-05-05 Thread Cassandra Targett (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-14173:
-
Fix Version/s: 8.6

> Ref Guide Redesign Phase 1: Page Design
> ---
>
> Key: SOLR-14173
> URL: https://issues.apache.org/jira/browse/SOLR-14173
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
> Fix For: 8.6
>
> Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, 
> gray-left-nav.png
>
>
> The current design of the Ref Guide was essentially copied from a 
> Jekyll-based documentation theme 
> (https://idratherbewriting.com/documentation-theme-jekyll/), which had a 
> couple important benefits for that time:
> * It was well-documented and since I had little experience with Jekyll and 
> its Liquid templates and since I was the one doing it, I wanted to make it as 
> easy on myself as possible
> * It was designed for documentation specifically so took care of all the 
> things like inter-page navigation, etc.
> * It helped us get from Confluence to our current system quickly
> It had some drawbacks, though:
> * It wasted a lot of space on the page
> * The theme was built for Markdown files, so did not take advantage of the 
> features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one 
> big example - the plugin could create it at build time, but the theme 
> included JS to do it as the page loads, so we use the JS)
> * It had a lot of JS and overlapping CSS files. While it used Bootstrap it 
> used a customized CSS on top of it for theming that made modifications 
> complex (it was hard to figure out how exactly a change would behave)
> * With all the stuff I'd changed in my bumbling way just to get things to 
> work back then, I broke a lot of the stuff Bootstrap is supposed to give us 
> in terms of responsiveness and making the Guide usable even on smaller screen 
> sizes.
> After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF 
> (SOLR-13782), I wanted to try to set us up for a more flexible system. We 
> need it for things like Joel's work on the visual guide for streaming 
> expressions (SOLR-13105), and in order to implement other ideas we might have 
> on how to present information in the future.
> I view this issue as a phase 1 of an overall redesign that I've already 
> started in a local branch. I'll explain in a comment the changes I've already 
> made, and will use this issue to create and push a branch where we can 
> discuss in more detail.
> Phase 1 here will be under-the-hood CSS/JS changes + overall page layout 
> changes.
> Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of 
> the Guide.
> Phase 3 (issue TBD) will explore moving us from Jekyll to another static site 
> generator that is better suited for our content format, file types, and build 
> conventions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Resolved] (SOLR-14173) Ref Guide Redesign Phase 1: Page Design

2020-05-05 Thread Cassandra Targett (Jira)


 [ 
https://issues.apache.org/jira/browse/SOLR-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett resolved SOLR-14173.
--
Resolution: Fixed

This is backported to branch_8x for use with 8.6.

I think it's going to be jarring for 8.6 to be radically different from earlier 
8.x versions at least, but if I decide I have time to backport to earlier 
versions & republish all those branches, I'll file another issue for that 
effort.

> Ref Guide Redesign Phase 1: Page Design
> ---
>
> Key: SOLR-14173
> URL: https://issues.apache.org/jira/browse/SOLR-14173
> Project: Solr
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
> Fix For: 8.6
>
> Attachments: SOLR-14173.patch, SOLR-14173.patch, blue-left-nav.png, 
> gray-left-nav.png
>
>
> The current design of the Ref Guide was essentially copied from a 
> Jekyll-based documentation theme 
> (https://idratherbewriting.com/documentation-theme-jekyll/), which had a 
> couple important benefits for that time:
> * It was well-documented and since I had little experience with Jekyll and 
> its Liquid templates and since I was the one doing it, I wanted to make it as 
> easy on myself as possible
> * It was designed for documentation specifically so took care of all the 
> things like inter-page navigation, etc.
> * It helped us get from Confluence to our current system quickly
> It had some drawbacks, though:
> * It wasted a lot of space on the page
> * The theme was built for Markdown files, so did not take advantage of the 
> features of the {{jekyll-asciidoc}} plugin we use (the in-page TOC being one 
> big example - the plugin could create it at build time, but the theme 
> included JS to do it as the page loads, so we use the JS)
> * It had a lot of JS and overlapping CSS files. While it used Bootstrap it 
> used a customized CSS on top of it for theming that made modifications 
> complex (it was hard to figure out how exactly a change would behave)
> * With all the stuff I'd changed in my bumbling way just to get things to 
> work back then, I broke a lot of the stuff Bootstrap is supposed to give us 
> in terms of responsiveness and making the Guide usable even on smaller screen 
> sizes.
> After upgrading the Asciidoctor components in SOLR-12786 and stopping the PDF 
> (SOLR-13782), I wanted to try to set us up for a more flexible system. We 
> need it for things like Joel's work on the visual guide for streaming 
> expressions (SOLR-13105), and in order to implement other ideas we might have 
> on how to present information in the future.
> I view this issue as a phase 1 of an overall redesign that I've already 
> started in a local branch. I'll explain in a comment the changes I've already 
> made, and will use this issue to create and push a branch where we can 
> discuss in more detail.
> Phase 1 here will be under-the-hood CSS/JS changes + overall page layout 
> changes.
> Phase 2 (SOLR-1) will be a wholesale re-organization of all the pages of 
> the Guide.
> Phase 3 (issue TBD) will explore moving us from Jekyll to another static site 
> generator that is better suited for our content format, file types, and build 
> conventions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] madrob opened a new pull request #1487: SOLR-14426 Move top level classes to nested classes

2020-05-05 Thread GitBox


madrob opened a new pull request #1487:
URL: https://github.com/apache/lucene-solr/pull/1487


   https://issues.apache.org/jira/browse/SOLR-14426
   
   This is as far as I could get before needing to give up. It was partly 
automated by IntelliJ but still painful manual in some respects. I think I got 
everything in the analytics module, and made a dent in core.
   
   Probably want to view this comparison ignoring whitespace changes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14457) SolrClient leaks connections on compressed responses if the response is malformed

2020-05-05 Thread Jira


[ 
https://issues.apache.org/jira/browse/SOLR-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100277#comment-17100277
 ] 

Samuel García Martínez commented on SOLR-14457:
---

Just for me to understand, I have several questions:
 * In your scenario, are you using the HTTP1 or HTTP2 client?
 * When do you observe the leaks?
 * How a connection is leaked with proper HTTP responses? Do you know where it 
happens?

In this bug, the exception happens like the following:
 * HTTPSolrClient receives an error code or a malformed response
 * The client throws an error
 * The client, on a finally block, ensures the connection is closed with 
Utils.consumeFully(HttpEntity)
 * Utils.consumeFully uses HttpEntity#getContent
 * When compression is enabled and response is compressed, the interceptor 
returns a new GzipInputStream
 * The GzipInputStream expects the header and it's not there because it's not 
really compressed.

Does your scenario look like the one I explained above?

> SolrClient leaks connections on compressed responses if the response is 
> malformed
> -
>
> Key: SOLR-14457
> URL: https://issues.apache.org/jira/browse/SOLR-14457
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: 7.7.2
> Environment: Solr version: 7.7.2
> Solr cloud enabled
> Cluster topology: 6 nodes, 1 single collection, 10 shards and 3 replicas. 1 
> HTTP LB using
> Round Robin over all nodes
> All cluster nodes have gzip enabled for all paths, all HTTP verbs and all 
> MIME types.
> Solr client: HttpSolrClient targeting the HTTP LB
>Reporter: Samuel García Martínez
>Priority: Major
>
> h3. Summary
> When the SolrJ receives a malformed response Entity, for example like the one 
> described in SOLR-14456, the client leaks the connection forever as it's 
> never released back to the pool.
> h3. Problem description
> HttpSolrClient should have compression enabled, so it uses the compression 
> interceptors.
> When the request is marked with "Content-Encoding: gzip" but for whatever 
> reason the response is not in GZIP format, when  HttpSolrClient tries to 
> close the connection using Utils.consumeFully(), it tries to create the 
> GzipInputStream failing and throwing an Exception. The exception thrown makes 
> it impossible to access the underlying InputStream to be closed, therefore 
> the connection is leaked.
> Despite that the content in the response should honour the headers specified 
> for it, SolrJ should be reliable enough to prevent the connection leak when 
> this scenario happens. On top of the bug itself, not being able to set a 
> timeout while waiting for a connection to be available, makes any application 
> unresponsive as it will run out of threads eventually.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14454) support for UTF-8 (string) types with DocValuesType.BINARY

2020-05-05 Thread David Smiley (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100284#comment-17100284
 ] 

David Smiley commented on SOLR-14454:
-

I just took a look at both -- my old patch and what you've done here.  I agree 
that the approach & emphasis (motivating use-case) differs considerably.  
Nonetheless I want to ensure that the chosen solution is a workable basis for 
both use-cases.  I generally like the direction you've gone with here more.  I 
should spend more time giving a more proper review.

I still think you should pause the compression aspect for another time because 
it's its own discussion.  It's fine to add hooks to allow a subclass to do 
compression, as I see you have done.

> support for UTF-8 (string) types with DocValuesType.BINARY
> --
>
> Key: SOLR-14454
> URL: https://issues.apache.org/jira/browse/SOLR-14454
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Affects Versions: master (9.0)
>Reporter: Michael Gibney
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The goal is to add support for string fields with arbitrarily large values in 
> the {{/export}} handler and streaming expressions.
> {{StrField}} values are currently limited to 32766 bytes for the case where 
> {{indexed=true}} or {{docValues=true}}. Exceeding this value triggers an 
> "immense field" warning, and causes indexing to fail for the associated input 
> doc.
> Configuring a {{StrField}} field as "{{indexed=false docValues=false}}" 
> removes this size limitation, so it is already possible to have large 
> _stored_ {{StrField}} values. But the "{{docValues=true}}" prerequisite for 
> the {{/export}} handler (and consequently for streaming expressions) limits 
> the size of field that can be used in conjunction with these features.
> Adding support for UTF-8/string field types with {{DocValuesType.BINARY}} 
> would address this limitation and allow considerable flexibility in the 
> implementation of custom field types. N.b.: this would address field value 
> retrieval use cases only (e.g., {{/export}} and {{useDocValuesAsStored}}); 
> neither sorting nor faceting would be supported.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] samuelgmartinez commented on pull request #1480: SOLR-14456: Fix Content-Type header forwarding on compressed requests

2020-05-05 Thread GitBox


samuelgmartinez commented on pull request #1480:
URL: https://github.com/apache/lucene-solr/pull/1480#issuecomment-624334841


   I just pushed the refactor. precommit is failing for an unused import I need 
to clean up. 
   
   Also, fixed related tests because it no longer throws ParseException and now 
it always will throw RemoteSolrException when Content-Type is empty.
   
   Tomorrow I’ll build the servers to do a manual round of testing with the 
cluster setup explained in the issue.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] samuelgmartinez edited a comment on pull request #1480: SOLR-14456: Fix Content-Type header forwarding on compressed requests

2020-05-05 Thread GitBox


samuelgmartinez edited a comment on pull request #1480:
URL: https://github.com/apache/lucene-solr/pull/1480#issuecomment-624334841


   I just pushed the refactor. precommit is failing for an unused import I need 
to clean up. 
   
   Also, fixed related tests because it no longer throws ParseException and now 
it always will throw RemoteSolrException when Content-Type is empty.
   
   Tomorrow evening (CEST) I’ll build the server to run some checks with the 
cluster setup explained in the issue.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] HoustonPutman commented on pull request #1487: SOLR-14426 Move top level classes to nested classes

2020-05-05 Thread GitBox


HoustonPutman commented on pull request #1487:
URL: https://github.com/apache/lucene-solr/pull/1487#issuecomment-624345497


   @uschindler have you seen this bug with other applications that use that 
plugin?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] HoustonPutman commented on pull request #1487: SOLR-14426 Move top level classes to nested classes

2020-05-05 Thread GitBox


HoustonPutman commented on pull request #1487:
URL: https://github.com/apache/lucene-solr/pull/1487#issuecomment-624345348


   This looks good to me. Kind of annoying that the gradle precommit plugin has 
this bug. I looked around in that repo to try and find something, but I 
couldn't find anything.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] HoustonPutman removed a comment on pull request #1487: SOLR-14426 Move top level classes to nested classes

2020-05-05 Thread GitBox


HoustonPutman removed a comment on pull request #1487:
URL: https://github.com/apache/lucene-solr/pull/1487#issuecomment-624345497


   @uschindler have you seen this bug with other applications that use that 
plugin?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] HoustonPutman edited a comment on pull request #1487: SOLR-14426 Move top level classes to nested classes

2020-05-05 Thread GitBox


HoustonPutman edited a comment on pull request #1487:
URL: https://github.com/apache/lucene-solr/pull/1487#issuecomment-624345348


   This looks good to me. Didn't know those protected classes were so frowned 
upon. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] anshumg commented on a change in pull request #1470: SOLR-14354: Async or using threads in better way for HttpShardHandler

2020-05-05 Thread GitBox


anshumg commented on a change in pull request #1470:
URL: https://github.com/apache/lucene-solr/pull/1470#discussion_r420457814



##
File path: 
solr/core/src/java/org/apache/solr/handler/component/HttpShardHandler.java
##
@@ -130,77 +134,64 @@ public void submit(final ShardRequest sreq, final String 
shard, final Modifiable
 final Tracer tracer = GlobalTracer.getTracer();
 final Span span = tracer != null ? tracer.activeSpan() : null;
 
-Callable task = () -> {
+params.remove(CommonParams.WT); // use default (currently javabin)
+params.remove(CommonParams.VERSION);
+QueryRequest req = makeQueryRequest(sreq, params, shard);
+req.setMethod(SolrRequest.METHOD.POST);
 
-  ShardResponse srsp = new ShardResponse();
-  if (sreq.nodeName != null) {
-srsp.setNodeName(sreq.nodeName);
-  }
-  srsp.setShardRequest(sreq);
-  srsp.setShard(shard);
-  SimpleSolrResponse ssr = new SimpleSolrResponse();
-  srsp.setSolrResponse(ssr);
-  long startTime = System.nanoTime();
+LBSolrClient.Req lbReq = 
httpShardHandlerFactory.newLBHttpSolrClientReq(req, urls);
+
+ShardResponse srsp = new ShardResponse();
+if (sreq.nodeName != null) {
+  srsp.setNodeName(sreq.nodeName);
+}
+srsp.setShardRequest(sreq);
+srsp.setShard(shard);
+SimpleSolrResponse ssr = new SimpleSolrResponse();
+srsp.setSolrResponse(ssr);
+
+pending.incrementAndGet();
+// if there are no shards available for a slice, urls.size()==0
+if (urls.size() == 0) {
+  // TODO: what's the right error code here? We should use the same thing 
when

Review comment:
   Service unavailable error code seems reasonable to me here. Do you have 
something else you were debating to switch this out to?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[GitHub] [lucene-solr] madrob commented on a change in pull request #1486: SOLR-14423: Use SolrClientCache instance managed by CoreContainer

2020-05-05 Thread GitBox


madrob commented on a change in pull request #1486:
URL: https://github.com/apache/lucene-solr/pull/1486#discussion_r420459575



##
File path: solr/core/src/java/org/apache/solr/handler/sql/CalciteSolrDriver.java
##
@@ -59,11 +81,346 @@ public Connection connect(String url, Properties info) 
throws SQLException {
 if(schemaName == null) {
   throw new SQLException("zk must be set");
 }
-rootSchema.add(schemaName, new SolrSchema(info));
+SolrSchema solrSchema = new SolrSchema(info);
+rootSchema.add(schemaName, solrSchema);
 
 // Set the default schema
 calciteConnection.setSchema(schemaName);
 
-return connection;
+return new SolrCalciteConnectionWrapper(calciteConnection, solrSchema);
+  }
+
+  // the sole purpose of this class is to be able to invoke SolrSchema.close()
+  // when the connection is closed.
+  private static final class SolrCalciteConnectionWrapper implements 
CalciteConnection {

Review comment:
   Can we have a new top level class that is a no-frills Wrapper, and then 
extend that here overriding just close? I think it is easier for people to see 
what is different.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness

2020-05-05 Thread Chris M. Hostetter (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100316#comment-17100316
 ] 

Chris M. Hostetter commented on SOLR-13132:
---

[~mgibney]- i haven't had a chance to review 
6338b327d30d0c1d5fdcb8168baf8398b02787d4 in depth, but at first glance I'm a 
fan of what i see.

your MultiAcc fix in 9ab4baef4a95a90e080d9118608619855f2e2759 seems to have 
addressed both of the failures I mentioned before, but it trying to beef up the 
MultiAcc testing i uncovered a new type of failure ...i don't really have any 
ideas (or even guesses) as to waht exactly is the source of the problem, but 
what i'm seeing is that in the MultiAcc situation _non-sweeping_ stats don't 
seem to be getting merged properly in some cases (even w/o refinement)

...BUT...

the discrepency doesn't actually manifest when comparing "sweep vs non-sweep" – 
it happens when comparing the "default" behavior for faceting on a multivalued 
string field (which should be using ArrayUIF for these fields, and implicitly 
sweeping) compared to explicitly using ArrayDV w/sweeping ... the buckets, 
counts, "skg" results, and bucket orderings are all consistent regardless of 
sort, but we get different values for the non-sweeping "min" stat that's used 
in the same facets (or in some cases, the "min" stat is completely missing)

You can see this when testing against the 
ec5f3a451a0be1a07a9ca37bf2cce33b8548b245 i just pushed to your branch with 
something like...
{noformat}
ant test  -Dtestcase=TestCloudJSONFacetSKGSweep  
-Dtests.seed=838E28E3EBC3B3B3:66DB4D3457DDE2F7 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=rof-TZ -Dtests.timezone=Etc/GMT+3 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
{noformat}

Here's how the new testBespokeStructures i just added fails with that seed 
combination...
{noformat}
   [junit4]> Throwable #1: java.lang.AssertionError: 
rows=0&q=(field_11_multi_sdsS:55+OR+field_0_multi_ss:46)&fore=field_5_multi_sdsS:9&back=*:*&json.facet={xxx1+:+{+type:terms,+method:${method_val:smart},+field:field_0_multi_ss,+limit:-1,+overrequest:0,+sort:+'count+desc',+refine:+false,+facet:{skg+:+{+"type":+"func",+++"func":+"relatedness($fore,$back)",+++"min_popularity":+0.001,+++${sweep_key:xxx}:+${sweep_val:yyy}+}+,min+:+"min(field_3_solo_i)"+}}+}
 ===> Mismatch: .xxx1.buckets[2][min]:32!=40 using 
method_val=dv&sweep_key=sweep_collection&sweep_val=true
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([838E28E3EBC3B3B3:66DB4D3457DDE2F7]:0)
   [junit4]>at 
org.apache.solr.search.facet.TestCloudJSONFacetSKGSweep.assertFacetSKGsAreConsistent(TestCloudJSONFacetSKGSweep.java:635)
   [junit4]>at 
org.apache.solr.search.facet.TestCloudJSONFacetSKGSweep.testBespokeStructures(TestCloudJSONFacetSKGSweep.java:513)
{noformat}
And here's the first bit of the "expected" (ie: ArrayUIF i believe) vs "actual" 
(ArrayDV) output, reformated for easier reading...
{noformat}
expected = {count=11, xxx1={buckets=[
  {val=39, count=3, 
skg={relatedness=0.0063, foreground_popularity=0.00833, 
background_popularity=0.08333}, 
min=10}, 
  {val=46, count=3, 
skg={relatedness=-Infinity, foreground_popularity=0.0, 
background_popularity=0.025}, 
min=37}, 
  {val=10, count=2, 
skg={relatedness=-Infinity, foreground_popularity=0.0, 
background_popularity=0.04167}, 
min=32},
...

actual = {count=11, xxx1={buckets=[
  {val=39, count=3, 
skg={relatedness=0.0063, foreground_popularity=0.00833, 
background_popularity=0.08333}, 
min=10}, 
  {val=46, count=3, 
skg={relatedness=-Infinity, foreground_popularity=0.0, 
background_popularity=0.025}, 
min=37}, 
  {val=10, count=2, 
skg={relatedness=-Infinity, foreground_popularity=0.0, 
background_popularity=0.04167}, 
min=40}, 
...
{noformat}

testBespoke w/same seed shows a "null" value for min for the first buckets - 
but other buckets have inconsistent min values ...
{noformat}
   [junit4]> Throwable #1: java.lang.AssertionError: 
rows=0&q=(field_13_multi_sds:26+OR+field_6_multi_ss:33+OR+field_9_multi_ss:24)&fore=(field_4_multi_sds:27+OR+field_12_multi_ss:18+OR+field_2_multi_sdsS:28+OR+field_13_multi_sds:50)&back=*:*&json.facet={xxx+:+{+type:terms,+method:${method_val:smart},+field:field_12_multi_ss,+limit:-1,+overrequest:0,+sort:+'count+asc',+refine:+false,+facet:{skg+:+{+"type":+"func",+++"func":+"relatedness($fore,$back)",+++"min_popularity":+0.001,+++${sweep_key:xxx}:+${sweep_val:yyy}+}+,min+:+"min(field_4_multi_ids)"+}}+}&_stateVer_=org.apache.solr.search.facet.TestCloudJSONFacetSKGSweep_collection:4
 ===> Mismatch: .xxx.buckets[0][min]==null using 
method_val=dv&sweep_key=sweep_collection&sweep_val=true




expected = {count=23, xxx={buckets=[
  {val=17, count=1, 
skg={relatedness=-0.00387, foreground_popularity=0.00833, 
background_popularity=0.05833}, 
min=16}, 
  {val=19,

[jira] [Commented] (SOLR-13289) Support for BlockMax WAND

2020-05-05 Thread Tomas Eduardo Fernandez Lobbe (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100329#comment-17100329
 ] 

Tomas Eduardo Fernandez Lobbe commented on SOLR-13289:
--

bq. Grouping
Grouping uses a different collector. It’s not included as part of this PR. 
Maybe there are things that can be done to allow WAND in some cases for Grouping
bq. Custom scoring via value sources
bq. Payloads
The optimization only applies if the natural score is the first thing to be 
sorted on. i.e. {{sort=score desc, id asc}}. If there is anything than “score” 
there, then the optimization doesn’t apply, not OOTB at least.

bq. Is TwoPhaseIterator or custom ranking in collapse/expand mode or in 
response writer the way? or have to implement a custom query overriding the 
createWeight using ScoreMode.TOP_SCORES?
I don’t know exactly what you want to do. In general, block-Max/WAND requires 
that the DISI can calculate the maximum contribution for a query in a block of 
documents with the indexed {{impacts}}. If you want to have more signals for 
calculating the score  I guess those could be added as Impacts using a custom 
codec and then you could have a custom scorer to use those ([~jpountz], let me 
know if this is wrong). If you just need a custom similarity that uses TF/IDF, 
then you should be able to use it.

> Support for BlockMax WAND
> -
>
> Key: SOLR-13289
> URL: https://issues.apache.org/jira/browse/SOLR-13289
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Attachments: SOLR-13289.patch, SOLR-13289.patch
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> LUCENE-8135 introduced BlockMax WAND as a major speed improvement. Need to 
> expose this via Solr. When enabled, the numFound returned will not be exact.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (LUCENE-9321) Port documentation task to gradle

2020-05-05 Thread Tomoko Uchida (Jira)


[ 
https://issues.apache.org/jira/browse/LUCENE-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100365#comment-17100365
 ] 

Tomoko Uchida commented on LUCENE-9321:
---

Thank you, for elaborating.

> Port documentation task to gradle
> -
>
> Key: LUCENE-9321
> URL: https://issues.apache.org/jira/browse/LUCENE-9321
> Project: Lucene - Core
>  Issue Type: Sub-task
>  Components: general/build
>Reporter: Tomoko Uchida
>Assignee: Uwe Schindler
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: screenshot-1.png
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> This is a placeholder issue for porting ant "documentation" task to gradle. 
> The generated documents should be able to be published on lucene.apache.org 
> web site on "as-is" basis.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness

2020-05-05 Thread Michael Gibney (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100425#comment-17100425
 ] 

Michael Gibney commented on SOLR-13132:
---

Ouch, yeah. I ran into that too and was trying to fix it earlier today. I 
hadn't yet realized that the discrepancy was related to UIF vs. DV ... with 
that clue it should be solved with 74e7ce3f0e65cc8abebcfcf4deef9da1ee73a656 
(just pushed). For sweeping, the doc iterator includes "hits" for docs that 
aren't in the base DocSet; but collection should only happen for docs in the 
base DocSet, for purpose of stats, etc. This was in the initial patch, then for 
some reason I thought it was only relevant to the caching aspect of the initial 
patch (SOLR-13807) and excised it from this patch. Subsequently realized it was 
still relevant here and reintroduced it with 
8a06d4c5d691dbf3e224223d7a3a0c84dcbfb87b, but mistakenly only for ArrayDV. 
Thanks for catching this!

> Improve JSON "terms" facet performance when sorted by relatedness 
> --
>
> Key: SOLR-13132
> URL: https://issues.apache.org/jira/browse/SOLR-13132
> Project: Solr
>  Issue Type: Improvement
>  Components: Facet Module
>Affects Versions: 7.4, master (9.0)
>Reporter: Michael Gibney
>Priority: Major
> Attachments: SOLR-13132-with-cache-01.patch, 
> SOLR-13132-with-cache.patch, SOLR-13132.patch, SOLR-13132_testSweep.patch
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> When sorting buckets by {{relatedness}}, JSON "terms" facet must calculate 
> {{relatedness}} for every term. 
> The current implementation uses a standard uninverted approach (either 
> {{docValues}} or {{UnInvertedField}}) to get facet counts over the domain 
> base docSet, and then uses that initial pass as a pre-filter for a 
> second-pass, inverted approach of fetching docSets for each relevant term 
> (i.e., {{count > minCount}}?) and calculating intersection size of those sets 
> with the domain base docSet.
> Over high-cardinality fields, the overhead of per-term docSet creation and 
> set intersection operations increases request latency to the point where 
> relatedness sort may not be usable in practice (for my use case, even after 
> applying the patch for SOLR-13108, for a field with ~220k unique terms per 
> core, QTime for high-cardinality domain docSets were, e.g.: cardinality 
> 1816684=9000ms, cardinality 5032902=18000ms).
> The attached patch brings the above example QTimes down to a manageable 
> ~300ms and ~250ms respectively. The approach calculates uninverted facet 
> counts over domain base, foreground, and background docSets in parallel in a 
> single pass. This allows us to take advantage of the efficiencies built into 
> the standard uninverted {{FacetFieldProcessorByArray[DV|UIF]}}), and avoids 
> the per-term docSet creation and set intersection overhead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-14454) support for UTF-8 (string) types with DocValuesType.BINARY

2020-05-05 Thread Michael Gibney (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-14454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100429#comment-17100429
 ] 

Michael Gibney commented on SOLR-14454:
---

Thanks for taking a closer look! And yes I'm perfectly happy to abandon the 
compression aspect altogether, esp. with hooks there to provide it as a plugin 
if desired (indeed I was kind of anticipating that, part of why I separated it 
out into its own commit).
{quote}Nonetheless I want to ensure that the chosen solution is a workable 
basis for both use-cases.
{quote}
That sounds great, however you think it might make sense to proceed.

> support for UTF-8 (string) types with DocValuesType.BINARY
> --
>
> Key: SOLR-14454
> URL: https://issues.apache.org/jira/browse/SOLR-14454
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Affects Versions: master (9.0)
>Reporter: Michael Gibney
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The goal is to add support for string fields with arbitrarily large values in 
> the {{/export}} handler and streaming expressions.
> {{StrField}} values are currently limited to 32766 bytes for the case where 
> {{indexed=true}} or {{docValues=true}}. Exceeding this value triggers an 
> "immense field" warning, and causes indexing to fail for the associated input 
> doc.
> Configuring a {{StrField}} field as "{{indexed=false docValues=false}}" 
> removes this size limitation, so it is already possible to have large 
> _stored_ {{StrField}} values. But the "{{docValues=true}}" prerequisite for 
> the {{/export}} handler (and consequently for streaming expressions) limits 
> the size of field that can be used in conjunction with these features.
> Adding support for UTF-8/string field types with {{DocValuesType.BINARY}} 
> would address this limitation and allow considerable flexibility in the 
> implementation of custom field types. N.b.: this would address field value 
> retrieval use cases only (e.g., {{/export}} and {{useDocValuesAsStored}}); 
> neither sorting nor faceting would be supported.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org



[jira] [Commented] (SOLR-13289) Support for BlockMax WAND

2020-05-05 Thread Adrien Grand (Jira)


[ 
https://issues.apache.org/jira/browse/SOLR-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100516#comment-17100516
 ] 

Adrien Grand commented on SOLR-13289:
-

bq. TwoPhaseIterator

LUCENE-8806 explores supporting two-phase iterators with block-max WAND.

bq. more signals for calculating the score

We have been recommending using FeatureField for this use-case: 
https://lucene.apache.org/core/8_5_0/core/org/apache/lucene/document/FeatureField.html.
 It encode float values as term freqs to be able to work with block-max WAND.

bq. If you just need a custom similarity that uses TF/IDF, then you should be 
able to use it.

Indeed, we only require that scores do not decrease when tf increases and do 
not increase when norm decreases.

> Support for BlockMax WAND
> -
>
> Key: SOLR-13289
> URL: https://issues.apache.org/jira/browse/SOLR-13289
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Tomas Eduardo Fernandez Lobbe
>Priority: Major
> Attachments: SOLR-13289.patch, SOLR-13289.patch
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> LUCENE-8135 introduced BlockMax WAND as a major speed improvement. Need to 
> expose this via Solr. When enabled, the numFound returned will not be exact.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org