[GitHub] [lucene-solr] noblepaul opened a new pull request #1980: SOLR-14930: Deprecate rulebased replica placement starategy
noblepaul opened a new pull request #1980: URL: https://github.com/apache/lucene-solr/pull/1980 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14575) Solr restore is failing when basic authentication is enabled
[
https://issues.apache.org/jira/browse/SOLR-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213717#comment-17213717
]
Jan Høydahl commented on SOLR-14575:
I also tried with 8.2 and no problems. I'm not asking you to install 8.6 in
production. I'm asking you to install a clean new setup with 2 nodes on 8.6 and
with a minimum of custom configuration, try to replicate your issue. My attempt
above is the simplest setup possible. Do do not need to use Docker to reproduce
if you are not familiar with it, but it is important that you try to do a clean
reproduction with steps that any of us can follow.
> Solr restore is failing when basic authentication is enabled
>
>
> Key: SOLR-14575
> URL: https://issues.apache.org/jira/browse/SOLR-14575
> Project: Solr
> Issue Type: Bug
> Components: Backup/Restore
>Affects Versions: 8.2
>Reporter: Yaswanth
>Priority: Major
>
> Hi Team,
> I was testing backup / restore for solrcloud and its failing exactly when I
> am trying to restore a successfully backed up collection.
> I am using solr 8.2 with basic authentication enabled and then creating a 2
> replica collection. When I run the backup like
> curl -u xxx:xxx -k
> '[https://x.x.x.x:8080/solr/admin/collections?action=BACKUP&name=test&collection=test&location=/solrdatabkup'|https://x.x.x.x:8080/solr/admin/collections?action=BACKUP&name=test&collection=test&location=/solrdatabkup%27]
> it worked fine and I do see a folder was created with the collection name
> under /solrdatabackup
> But now when I deleted the existing collection and then try running restore
> api like
> curl -u xxx:xxx -k
> '[https://x.x.x.x:8080/solr/admin/collections?action=RESTORE&name=test&collection=test&location=/solrdatabkup'|https://x.x.x.x:8080/solr/admin/collections?action=BACKUP&name=test&collection=test&location=/solrdatabkup%27]
> its throwing an error like
> {
> "responseHeader":{
> "status":500,
> "QTime":457},
> "Operation restore caused
> *exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> ADDREPLICA failed to create replica",*
> "exception":{
> "msg":"ADDREPLICA failed to create replica",
> "rspCode":500},
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"ADDREPLICA failed to create replica",
> "trace":"org.apache.solr.common.SolrException: ADDREPLICA failed to create
> replica\n\tat
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:280)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:252)\n\tat
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:820)\n\tat
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:786)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:546)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)\n\tat
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Context
[jira] [Updated] (SOLR-14066) Deprecate DIH and migrate to a community supported package
[ https://issues.apache.org/jira/browse/SOLR-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Pimentel updated SOLR-14066: Attachment: image-2020-10-14-10-31-36-518.png > Deprecate DIH and migrate to a community supported package > -- > > Key: SOLR-14066 > URL: https://issues.apache.org/jira/browse/SOLR-14066 > Project: Solr > Issue Type: Improvement > Components: contrib - DataImportHandler >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Blocker > Fix For: 8.6 > > Attachments: image-2019-12-14-19-58-39-314.png, > image-2020-10-13-21-01-48-401.png, image-2020-10-13-21-18-43-877.png, > image-2020-10-14-10-31-36-518.png > > Time Spent: 1h 10m > Remaining Estimate: 0h > > DIH doesn't need to remain inside Solr anymore. Plan is to deprecate DIH in > 8.6, remove from 9.0. A community supported version of DIH (which can be used > with Solr's package manager) can be found here > https://github.com/rohitbemax/dataimporthandler. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14066) Deprecate DIH and migrate to a community supported package
[ https://issues.apache.org/jira/browse/SOLR-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213765#comment-17213765 ] Rui Pimentel commented on SOLR-14066: - Hi Jan Høydahl, Thanks for the information and my apologies. Best regards Rui h1. > Deprecate DIH and migrate to a community supported package > -- > > Key: SOLR-14066 > URL: https://issues.apache.org/jira/browse/SOLR-14066 > Project: Solr > Issue Type: Improvement > Components: contrib - DataImportHandler >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Blocker > Fix For: 8.6 > > Attachments: image-2019-12-14-19-58-39-314.png, > image-2020-10-13-21-01-48-401.png, image-2020-10-13-21-18-43-877.png, > image-2020-10-14-10-31-36-518.png > > Time Spent: 1h 10m > Remaining Estimate: 0h > > DIH doesn't need to remain inside Solr anymore. Plan is to deprecate DIH in > 8.6, remove from 9.0. A community supported version of DIH (which can be used > with Solr's package manager) can be found here > https://github.com/rohitbemax/dataimporthandler. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14931) Macros in appends/invariants parameters not getting expanded in Solrcloud
Johannes Baiter created SOLR-14931:
--
Summary: Macros in appends/invariants parameters not getting
expanded in Solrcloud
Key: SOLR-14931
URL: https://issues.apache.org/jira/browse/SOLR-14931
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: SolrCloud
Affects Versions: 8.6.3, 7.6
Reporter: Johannes Baiter
Macros in a request handler's {{appends}} or {{invariants}} parameter sets are
not getting expanded when running in a SolrCloud setup.
In my case, I have a handler like the following:
{code:xml}
some_field:"$${my_term}"
term_defaults:'$${my_term},freq_defaults:docfreq(some_field,
'$${my_term}')
term_appends:'$${my_term}',freq_appends:docfreq(some_field,
'$${my_term}')
{code}
When querying the handler with `?my_term=foobar&echoParams=all`, something like
this is the result:
{code:json}
{
"response": {
"docs": [
{
"term_defaults": "foobar",
"term_appends": "${my_term}",
"freq_appends": 0,
"freq_defaults": 1
}
],
"maxScore": 13.244947,
"numFound": 1,
"start": 0
},
"responseHeader": {
"QTime": 262091,
"params": {
"echoParams": "all",
"fl": [
"term_defaults:'foobar',freq_defaults:docfreq(some_field,
'foobar')",
"term_appends:'foobar',freq_appends:docfreq(some_field,
'foobar')"
],
"my_term": "foobar",
"q": "some_field:\"foobar\"",
"rows": "1"
},
"status": 0,
"zkConnected": true
}
}
{code}
As you can see, the macro in the {{appends}} {{fl}} parameter gets expanded in
the {{responseHeader.params.fl}} field, but not in the actual document.
I believe the reason for this is the following:
* Setting of defaults/invariants/appends happens once before the request gets
sent to the shards
* Here, macro expansion happens, i.e. the shards get the fully expanded
parameters
* On the shards, once again, defaults/invariants/appends are set
(https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java#L209)
* However, this time *macros are not expanded*
(https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/request/json/RequestUtil.java#L151-L161)
* For {{defaults}} parameters, this doesn't break anything, since the
(expanded) parameter from the request takes precedence over the server-supplied
value.
* However, for {{appends}}, both the expanded and the unexpanded version are
now set, while for {{invariants}} the expanded value is removed from the params
I think the easy/obvious fix for this would be not to set
{{defaults}}/{{invariants}}/{{appends}} parameters twice, i.e. disable it when
{{isShard=true}}, just like macro expansion, but I don't have the full picture
if that might not break something else.
Here's the Twitter thread where I rubber-ducked through this issue:
https://twitter.com/jbaiter_/status/1316023733576843272
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14932) Broken "Add replica" button in solr admin UI
[ https://issues.apache.org/jira/browse/SOLR-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sayan Das updated SOLR-14932: - Attachment: image-2020-10-14-16-57-15-275.png > Broken "Add replica" button in solr admin UI > > > Key: SOLR-14932 > URL: https://issues.apache.org/jira/browse/SOLR-14932 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Admin UI >Reporter: Sayan Das >Priority: Minor > Attachments: image-2020-10-14-16-57-15-275.png > > > Not able to list live nodes in admin UI > !image-2020-10-14-16-56-41-579.png|width=401,height=85! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14932) Broken "Add replica" button in solr admin UI
[ https://issues.apache.org/jira/browse/SOLR-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sayan Das updated SOLR-14932: - Description: Not able to list live nodes in admin UI, when trying to add new replica. !image-2020-10-14-16-57-15-275.png|width=388,height=82! was:Not able to list live nodes in admin UI !image-2020-10-14-16-56-41-579.png|width=401,height=85! > Broken "Add replica" button in solr admin UI > > > Key: SOLR-14932 > URL: https://issues.apache.org/jira/browse/SOLR-14932 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Admin UI >Reporter: Sayan Das >Priority: Minor > Attachments: image-2020-10-14-16-57-15-275.png > > > Not able to list live nodes in admin UI, when trying to add new replica. > !image-2020-10-14-16-57-15-275.png|width=388,height=82! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14932) Broken "Add replica" button in solr admin UI
Sayan Das created SOLR-14932: Summary: Broken "Add replica" button in solr admin UI Key: SOLR-14932 URL: https://issues.apache.org/jira/browse/SOLR-14932 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: Admin UI Reporter: Sayan Das Attachments: image-2020-10-14-16-57-15-275.png Not able to list live nodes in admin UI !image-2020-10-14-16-56-41-579.png|width=401,height=85! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14932) Broken "Add replica" button in solr admin UI
[ https://issues.apache.org/jira/browse/SOLR-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213788#comment-17213788 ] Sayan Das commented on SOLR-14932: -- If no one else is working on it already. I am willing to fix this up as part of my first contribution to Solr community. > Broken "Add replica" button in solr admin UI > > > Key: SOLR-14932 > URL: https://issues.apache.org/jira/browse/SOLR-14932 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Admin UI >Reporter: Sayan Das >Priority: Minor > Attachments: image-2020-10-14-16-57-15-275.png > > > Not able to list live nodes in admin UI, when trying to add new replica. > !image-2020-10-14-16-57-15-275.png|width=388,height=82! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14933) Ability to add T and P type replica from admin UI
Sayan Das created SOLR-14933: Summary: Ability to add T and P type replica from admin UI Key: SOLR-14933 URL: https://issues.apache.org/jira/browse/SOLR-14933 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: Admin UI Reporter: Sayan Das Attachments: image-2020-10-14-17-07-08-159.png As of now we can already add NRT replica from admin UI. Would look to add drop down listing all types of replicas along with the live nodes drop down. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14483) AdminUI - Create replica - empty drop-down
[
https://issues.apache.org/jira/browse/SOLR-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213791#comment-17213791
]
Thomas Wöckinger commented on SOLR-14483:
-
This is only one part of the issue, because using the rest API to add a new
replica is also not working.
> AdminUI - Create replica - empty drop-down
> --
>
> Key: SOLR-14483
> URL: https://issues.apache.org/jira/browse/SOLR-14483
> Project: Solr
> Issue Type: Bug
> Components: Admin UI
>Affects Versions: 8.5.1
> Environment: Debian-10 based
> Solr: 8.5.1
> Firefox/google-chrome
>Reporter: Morten Bøgeskov
>Priority: Minor
>
> When you wish to add a replica using the Admin UI, and press the "add
> replica" button (under a shard):
> * A json document is fetched:
> /solr/admin/zookeeper?_=xxx&path=%2Flive_nodes&wt=json
> ** payload:
> ** {{{"tree":[{
> "text":"/live_nodes","a_attr":{
> "href":"admin/zookeeper?detail=true&path=%2Flive_nodes"},
> "children":[{
> "text":"host-1.example.com:8983_solr","a_attr":{
>
> "href":"admin/zookeeper?detail=true&path=%2Flive_nodes%2Fhost-1.example.com%3A8983_solr"},
> "ephemeral":true,
> "version":0},...}}
> * A JavaScript console error:
> ** Firefox:
> ** {{TypeError: "children[child].data is undefined"}}
> Angular 15
> jQuery 8
> Angular 24{{ Possibly unhandled rejection: {}
> [angular.min.js:146:303|http://socl-p01:18003/solr/libs/angular.min.js]}}
> Angular 15
> jQuery 8
> {{Angular 24}}
> ** Google-chrome:
> ** TypeError: Cannot read property 'title' of undefined
> at collections.js:219
> at I (angular-resource.min.js:31)
> at angular.min.js:159
> at m.$digest (angular.min.js:170)
> at m.$apply (angular.min.js:174)
> at k (angular.min.js:125)
> at v (angular.min.js:130)
> at XMLHttpRequest.y.onload (angular.min.js:131) "Possibly unhandled
> rejection: {}"
> Resulting in an empty drop-down.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14933) Ability to add T and P type replica from admin UI
[ https://issues.apache.org/jira/browse/SOLR-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213793#comment-17213793 ] Sayan Das commented on SOLR-14933: -- I would like to work on it, once I get green flag for development. > Ability to add T and P type replica from admin UI > - > > Key: SOLR-14933 > URL: https://issues.apache.org/jira/browse/SOLR-14933 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Admin UI >Reporter: Sayan Das >Priority: Minor > Attachments: image-2020-10-14-17-07-08-159.png > > > As of now we can already add NRT replica from admin UI. Would look to add > drop down listing all types of replicas along with the live nodes drop down. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14933) Ability to add T and P type replica from admin UI
[ https://issues.apache.org/jira/browse/SOLR-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213806#comment-17213806 ] David Eric Pugh commented on SOLR-14933: There are quite a few Collection API commands that aren't exposed in the UI, and this would be a nice one to add. > Ability to add T and P type replica from admin UI > - > > Key: SOLR-14933 > URL: https://issues.apache.org/jira/browse/SOLR-14933 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Admin UI >Reporter: Sayan Das >Priority: Minor > Attachments: image-2020-10-14-17-07-08-159.png > > > As of now we can already add NRT replica from admin UI. Would look to add > drop down listing all types of replicas along with the live nodes drop down. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14549) Listing of Files in a Directory on Solr Admin is Broken
[ https://issues.apache.org/jira/browse/SOLR-14549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213807#comment-17213807 ] David Eric Pugh commented on SOLR-14549: Thanks [~krisden] for the update. Now that you laid out what you think is happening, in my looking at it, it makes sense to me as well that the recursion doesn't happen properly. Why? Dunno. > Listing of Files in a Directory on Solr Admin is Broken > --- > > Key: SOLR-14549 > URL: https://issues.apache.org/jira/browse/SOLR-14549 > Project: Solr > Issue Type: Bug > Components: Admin UI >Affects Versions: master (9.0), 8.5.1, 8.5.2 >Reporter: David Eric Pugh >Assignee: Kevin Risden >Priority: Major > Attachments: Screenshot at Jun 09 07-40-06.png > > > The Admin interface for showing files only lets you see the top level files, > no nested files in a directory: > http://localhost:8983/solr/#/gettingstarted/files?file=lang%2F > Choosing a nested directory doesn't generate any console errors, but the tree > doesn't open. > I believe this was introduced during SOLR-14209 upgrade in Jquery. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14282) /get handler doesn't return copied fields
[
https://issues.apache.org/jira/browse/SOLR-14282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213814#comment-17213814
]
David Eric Pugh commented on SOLR-14282:
I'm not sure [~ami...@intellective.com] on if a update log ever expires I
know that with a fully committed Solr index, I still don't get copyFields
returned, which makes sense the way I read the code.
I poked around, and there are a bunch of magic parameters that aren't
documented related to how SolrCloud works, like `sync` and `getVersions` etc
that go between DistributedUpdateProcessor and the RealTimeGetComponent.So,
I'm thinking that we could add a flag, if needed, to disable the copyField.
Something like "skipCopyFields" (following the wonky parameter names being sent
around ;-) ) to DistributedUpdateProcessor and RealTimeGetComponent.
So a `/get?ids=123` would include copyFields, but a
`/get?ids=123&skipCopyFields` would not.
Since this is touching DistributedUpdateProcessor, I'm going to need some
suggestions/ideas from someone else who knows this code much deeper than I!
> /get handler doesn't return copied fields
> -
>
> Key: SOLR-14282
> URL: https://issues.apache.org/jira/browse/SOLR-14282
> Project: Solr
> Issue Type: Bug
> Components: search, SolrJ
>Affects Versions: 8.4
> Environment: SOLR 8.4.0, SOLRJ, Oracle Java 8
>Reporter: Andrei Minin
>Priority: Major
> Attachments: SOLR-14282-test-update.patch, copied_fields_test.zip,
> managed-schema.xml
>
>
> We are using /get handler to retrieve documents by id in our Java application
> (SolrJ)
> I found that copied fields are missing in documents returned by /get handler
> but same documents returned by query contain copied (by schema) fields.
> Attached documents:
> # Integration test project archive
> # Managed schema file for SOLR
> SOLR schema details:
> # Unique field name "d_ida_s"
> # Lowecase text type definition:
> {code:java}
> positionIncrementGap="100">
>
>
>
>
> {code}
> 3. Copy field instruction sample:
> {code:java}
> stored="true" multiValued="false"/>
> />
> {code}
> ConcurrenceUserNamea_s is string type field and ConcurrenceUserNameu_lca_s is
> lower case text type field.
> Integration test uploads document to SOLR server and makes 2 requests: one
> using /get rest point to fetch document by id and one using query field name>:.
> Document returned by /get rest, doesn't have copied fields while document
> returned by query, contains copied fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14933) Ability to add T and P type replica from admin UI
[ https://issues.apache.org/jira/browse/SOLR-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213838#comment-17213838 ] Sayan Das commented on SOLR-14933: -- sure [~epugh] may be if you can list it down, I will try to add them as well. With my personal experience replica type, is utmost crucial and it is really annoying to add replica from API every time. > Ability to add T and P type replica from admin UI > - > > Key: SOLR-14933 > URL: https://issues.apache.org/jira/browse/SOLR-14933 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Admin UI >Reporter: Sayan Das >Priority: Minor > Attachments: image-2020-10-14-17-07-08-159.png > > > As of now we can already add NRT replica from admin UI. Would look to add > drop down listing all types of replicas along with the live nodes drop down. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-14483) AdminUI - Create replica - empty drop-down
[
https://issues.apache.org/jira/browse/SOLR-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Eric Pugh reassigned SOLR-14483:
--
Assignee: David Eric Pugh
> AdminUI - Create replica - empty drop-down
> --
>
> Key: SOLR-14483
> URL: https://issues.apache.org/jira/browse/SOLR-14483
> Project: Solr
> Issue Type: Bug
> Components: Admin UI
>Affects Versions: 8.5.1
> Environment: Debian-10 based
> Solr: 8.5.1
> Firefox/google-chrome
>Reporter: Morten Bøgeskov
>Assignee: David Eric Pugh
>Priority: Minor
>
> When you wish to add a replica using the Admin UI, and press the "add
> replica" button (under a shard):
> * A json document is fetched:
> /solr/admin/zookeeper?_=xxx&path=%2Flive_nodes&wt=json
> ** payload:
> ** {{{"tree":[{
> "text":"/live_nodes","a_attr":{
> "href":"admin/zookeeper?detail=true&path=%2Flive_nodes"},
> "children":[{
> "text":"host-1.example.com:8983_solr","a_attr":{
>
> "href":"admin/zookeeper?detail=true&path=%2Flive_nodes%2Fhost-1.example.com%3A8983_solr"},
> "ephemeral":true,
> "version":0},...}}
> * A JavaScript console error:
> ** Firefox:
> ** {{TypeError: "children[child].data is undefined"}}
> Angular 15
> jQuery 8
> Angular 24{{ Possibly unhandled rejection: {}
> [angular.min.js:146:303|http://socl-p01:18003/solr/libs/angular.min.js]}}
> Angular 15
> jQuery 8
> {{Angular 24}}
> ** Google-chrome:
> ** TypeError: Cannot read property 'title' of undefined
> at collections.js:219
> at I (angular-resource.min.js:31)
> at angular.min.js:159
> at m.$digest (angular.min.js:170)
> at m.$apply (angular.min.js:174)
> at k (angular.min.js:125)
> at v (angular.min.js:130)
> at XMLHttpRequest.y.onload (angular.min.js:131) "Possibly unhandled
> rejection: {}"
> Resulting in an empty drop-down.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14933) Ability to add T and P type replica from admin UI
[ https://issues.apache.org/jira/browse/SOLR-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213865#comment-17213865 ] Erick Erickson commented on SOLR-14933: --- [~sayan.das] All the commands and their options are already listed in the collections API documentation: https://lucene.apache.org/solr/guide/8_4/collection-management.html#collection-management If I may suggest, you might want to triage which commands (and their options) are most useful most widely, "progress not perfection". For instance, MIGRATECOLLECTION seems like something that's rare enough that it should have a lower priority. Choose wisely, because this is one of those things that could go on forever ;). Wouldn't it be cool, for instance, to have a context menu open up when you hovered over a shard in the graph to allow you to add a replica > Ability to add T and P type replica from admin UI > - > > Key: SOLR-14933 > URL: https://issues.apache.org/jira/browse/SOLR-14933 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Admin UI >Reporter: Sayan Das >Priority: Minor > Attachments: image-2020-10-14-17-07-08-159.png > > > As of now we can already add NRT replica from admin UI. Would look to add > drop down listing all types of replicas along with the live nodes drop down. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley closed pull request #1970: SOLR-14869: do not add deleted docs in child doc transformer
dsmiley closed pull request #1970: URL: https://github.com/apache/lucene-solr/pull/1970 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14869) [child] transformer includes deleted documents
[
https://issues.apache.org/jira/browse/SOLR-14869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213869#comment-17213869
]
ASF subversion and git services commented on SOLR-14869:
Commit fa3e1ad71fd97200b98f9e7d1e461ddfa78e357c in lucene-solr's branch
refs/heads/master from Bar Rotstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=fa3e1ad ]
SOLR-14869: ChildDocTransformer should have omitted deleted child documents.
Closes #1970
> [child] transformer includes deleted documents
> --
>
> Key: SOLR-14869
> URL: https://issues.apache.org/jira/browse/SOLR-14869
> Project: Solr
> Issue Type: Bug
>Reporter: Chris M. Hostetter
>Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have nested documents that include "deleted" docs (ie: using "delete
> by query") the {{\[child\]}} transformer still include those deleted docs i
> it's output
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14869) [child] transformer includes deleted documents
[
https://issues.apache.org/jira/browse/SOLR-14869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213872#comment-17213872
]
ASF subversion and git services commented on SOLR-14869:
Commit 1bbea05ac774cc925a106ec5c3c070412620315b in lucene-solr's branch
refs/heads/branch_8x from Bar Rotstein
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=1bbea05 ]
SOLR-14869: ChildDocTransformer should have omitted deleted child documents.
Closes #1970
(cherry picked from commit fa3e1ad71fd97200b98f9e7d1e461ddfa78e357c)
> [child] transformer includes deleted documents
> --
>
> Key: SOLR-14869
> URL: https://issues.apache.org/jira/browse/SOLR-14869
> Project: Solr
> Issue Type: Bug
>Reporter: Chris M. Hostetter
>Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have nested documents that include "deleted" docs (ie: using "delete
> by query") the {{\[child\]}} transformer still include those deleted docs i
> it's output
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14869) [child] transformer includes deleted documents
[
https://issues.apache.org/jira/browse/SOLR-14869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley resolved SOLR-14869.
-
Fix Version/s: 8.7
Assignee: David Smiley
Resolution: Fixed
> [child] transformer includes deleted documents
> --
>
> Key: SOLR-14869
> URL: https://issues.apache.org/jira/browse/SOLR-14869
> Project: Solr
> Issue Type: Bug
>Reporter: Chris M. Hostetter
>Assignee: David Smiley
>Priority: Major
> Fix For: 8.7
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have nested documents that include "deleted" docs (ie: using "delete
> by query") the {{\[child\]}} transformer still include those deleted docs i
> it's output
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14869) [child] transformer includes deleted documents
[
https://issues.apache.org/jira/browse/SOLR-14869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213874#comment-17213874
]
David Smiley commented on SOLR-14869:
-
Thanks for contributing again Bar!
> [child] transformer includes deleted documents
> --
>
> Key: SOLR-14869
> URL: https://issues.apache.org/jira/browse/SOLR-14869
> Project: Solr
> Issue Type: Bug
>Reporter: Chris M. Hostetter
>Assignee: David Smiley
>Priority: Major
> Fix For: 8.7
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have nested documents that include "deleted" docs (ie: using "delete
> by query") the {{\[child\]}} transformer still include those deleted docs i
> it's output
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-10059) In SolrCloud, every fq added via is computed twice.
[
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213880#comment-17213880
]
Christine Poerschke commented on SOLR-10059:
Just adding a note here to cross-reference SOLR-10059 and SOLR-14931 as
potentially having things in common; haven't looked too closely though, only
going by _"SolrCloud" and "appends" and "twice"_ mentionings.
> In SolrCloud, every fq added via is computed twice.
>
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 6.4
>Reporter: Marc Morissette
>Priority: Major
> Labels: performance
> Attachments: SOLR-10059_7x.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While researching another issue, I noticed that parameters appended to a
> query via SearchHandler's are added to the query twice
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field
> queries added in this fashion are however computed twice and that hinders
> performance on filter queries that aren't simple bitsets such as those
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough
> collection, then replace "appends" with "defaults". You'll notice significant
> performance improvements.
> {code}
>
>
> {!collapse field=routingKey hint=top_fc}
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14931) Macros in appends/invariants parameters not getting expanded in Solrcloud
[
https://issues.apache.org/jira/browse/SOLR-14931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213881#comment-17213881
]
Christine Poerschke commented on SOLR-14931:
Just adding a note here to cross-reference SOLR-10059 and SOLR-14931 as
potentially having things in common; haven't looked too closely though, only
going by _"SolrCloud" and "appends" and "twice"_ mentionings.
> Macros in appends/invariants parameters not getting expanded in Solrcloud
> -
>
> Key: SOLR-14931
> URL: https://issues.apache.org/jira/browse/SOLR-14931
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: 7.6, 8.6.3
>Reporter: Johannes Baiter
>Priority: Major
>
> Macros in a request handler's {{appends}} or {{invariants}} parameter sets
> are not getting expanded when running in a SolrCloud setup.
> In my case, I have a handler like the following:
> {code:xml}
>
>
> some_field:"$${my_term}"
> name="fl">term_defaults:'$${my_term},freq_defaults:docfreq(some_field,
> '$${my_term}')
>
>
> name="fl">term_appends:'$${my_term}',freq_appends:docfreq(some_field,
> '$${my_term}')
>
>
> {code}
> When querying the handler with `?my_term=foobar&echoParams=all`, something
> like this is the result:
> {code:json}
> {
> "response": {
> "docs": [
> {
> "term_defaults": "foobar",
> "term_appends": "${my_term}",
> "freq_appends": 0,
> "freq_defaults": 1
> }
> ],
> "maxScore": 13.244947,
> "numFound": 1,
> "start": 0
> },
> "responseHeader": {
> "QTime": 262091,
> "params": {
> "echoParams": "all",
> "fl": [
> "term_defaults:'foobar',freq_defaults:docfreq(some_field,
> 'foobar')",
> "term_appends:'foobar',freq_appends:docfreq(some_field,
> 'foobar')"
> ],
> "my_term": "foobar",
> "q": "some_field:\"foobar\"",
> "rows": "1"
> },
> "status": 0,
> "zkConnected": true
> }
> }
> {code}
> As you can see, the macro in the {{appends}} {{fl}} parameter gets expanded
> in the {{responseHeader.params.fl}} field, but not in the actual document.
> I believe the reason for this is the following:
> * Setting of defaults/invariants/appends happens once before the request gets
> sent to the shards
> * Here, macro expansion happens, i.e. the shards get the fully expanded
> parameters
> * On the shards, once again, defaults/invariants/appends are set
> (https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java#L209)
> * However, this time *macros are not expanded*
> (https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/request/json/RequestUtil.java#L151-L161)
> * For {{defaults}} parameters, this doesn't break anything, since the
> (expanded) parameter from the request takes precedence over the
> server-supplied value.
> * However, for {{appends}}, both the expanded and the unexpanded version are
> now set, while for {{invariants}} the expanded value is removed from the
> params
> I think the easy/obvious fix for this would be not to set
> {{defaults}}/{{invariants}}/{{appends}} parameters twice, i.e. disable it
> when {{isShard=true}}, just like macro expansion, but I don't have the full
> picture if that might not break something else.
> Here's the Twitter thread where I rubber-ducked through this issue:
> https://twitter.com/jbaiter_/status/1316023733576843272
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul commented on pull request #1975: Include missing commands in package tool help section
noblepaul commented on pull request #1975: URL: https://github.com/apache/lucene-solr/pull/1975#issuecomment-708382169 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
gus-asf commented on pull request #1979: URL: https://github.com/apache/lucene-solr/pull/1979#issuecomment-708382753 Oh lovely, git hub decided to convert my ( originally all lower case) @ since to a user's name... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14930) Deprecate rulebased replica placement strategy
[ https://issues.apache.org/jira/browse/SOLR-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-14930: -- Summary: Deprecate rulebased replica placement strategy (was: Deprecate rulebased replica placement starategy) > Deprecate rulebased replica placement strategy > -- > > Key: SOLR-14930 > URL: https://issues.apache.org/jira/browse/SOLR-14930 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This should be removed from 9.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14915) Prometheus-exporter should not depend on Solr-core
[ https://issues.apache.org/jira/browse/SOLR-14915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213888#comment-17213888 ] David Smiley commented on SOLR-14915: - Thanks for doing some build experimentation to show me your proposal. I do not think that the Solr distribution should be bloated by duplicated dependencies of SolrJ, which are quite some number of megabytes. Duplication of the other libs seems more manageable (e.g. logging libs + Caffeine). I think I prefer that SolrJ libs be externally referenced, but the rest be local to the lib directory of this contrib. I also think this contrib ought to have its own log4j2.xml. I think I could use your proposal as a guide on how I could accomplish this... though I don't plan to get to this until next week. > Prometheus-exporter should not depend on Solr-core > -- > > Key: SOLR-14915 > URL: https://issues.apache.org/jira/browse/SOLR-14915 > Project: Solr > Issue Type: Improvement > Components: contrib - prometheus-exporter >Reporter: David Smiley >Assignee: David Smiley >Priority: Minor > Attachments: patch.patch > > Time Spent: 40m > Remaining Estimate: 0h > > I think it's *crazy* that our Prometheus exporter depends on Solr-core -- > this thing is a _client_ of Solr; it does not live within Solr. The exporter > ought to be fairly lean. One consequence of this dependency is that, for > example, security vulnerabilities reported against Solr (e.g. Jetty) can (and > do, where I work) wind up being reported against this module even though > Prometheus isn't using Jetty. > From my evaluation today of what's going on, it appears the crux of the > problem is that the prometheus exporter uses some utility mechanisms in > Solr-core like XmlConfig (which depends on SolrResourceLoader and the rabbit > hole goes deeper...) and DOMUtils (further depends on PropertiesUtil). It > can easy be made to not use XmlConfig. DOMUtils & PropertiesUtil could move > to SolrJ which already has lots of little dependency-free utilities needed by > SolrJ and Solr-core alike. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on pull request #1870: SOLR-14865: 'Index Merge Metrics' documentation correction
cpoerschke commented on pull request #1870: URL: https://github.com/apache/lucene-solr/pull/1870#issuecomment-708388622 @ctargett and/or @sigram would you have any thoughts on this documentation change, with a view towards it perhaps being included in the upcoming 8.7 Solr Reference Guide? Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke edited a comment on pull request #1870: SOLR-14865: 'Index Merge Metrics' documentation correction
cpoerschke edited a comment on pull request #1870: URL: https://github.com/apache/lucene-solr/pull/1870#issuecomment-708388622 @ctargett and/or @sigram would you have any thoughts on this documentation change, with a view towards it perhaps being included in the upcoming 8.7 Solr Reference Guide? Thanks! edit: github won't let me add both of you as reviewers, for some reason. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
uschindler commented on a change in pull request #1979:
URL: https://github.com/apache/lucene-solr/pull/1979#discussion_r504664083
##
File path:
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/DropIfFlaggedFilter.java
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.analysis.miscellaneous;
+
+import org.apache.lucene.analysis.FilteringTokenFilter;
+import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.tokenattributes.FlagsAttribute;
+
+/**
+ * Allows Tokens with a given combination of flags to be dropped. If all flags
specified are present
+ * the token is dropped, otherwise it is retained.
+ *
+ * @see DropIfFlaggedFilterFactory
+ * @since 8.8.0
+ */
+public class DropIfFlaggedFilter extends FilteringTokenFilter {
Review comment:
You should make the class final, too. That's not a requirement, but
would be in line with other filters. The reason is: We don't want subclasses,
as we prefer delegation over inheritance.
##
File path:
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/DropIfFlaggedFilter.java
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.analysis.miscellaneous;
+
+import org.apache.lucene.analysis.FilteringTokenFilter;
+import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.tokenattributes.FlagsAttribute;
+
+/**
+ * Allows Tokens with a given combination of flags to be dropped. If all flags
specified are present
+ * the token is dropped, otherwise it is retained.
+ *
+ * @see DropIfFlaggedFilterFactory
+ * @since 8.8.0
+ */
+public class DropIfFlaggedFilter extends FilteringTokenFilter {
+ private final FlagsAttribute flagsAtt = addAttribute(FlagsAttribute.class);
+ private final int dropFlags;
+
+ /**
+ * Construct a token stream filtering the given input.
+ *
+ * @param input the source stream
+ * @param dropFlags a combination of flags that indicates that the token
should be dropped.
+ */
+ @SuppressWarnings("WeakerAccess")
+ protected DropIfFlaggedFilter(TokenStream input, int dropFlags) {
Review comment:
Can you make the class final and make the constructor public? Why is
this protected?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
uschindler commented on a change in pull request #1979: URL: https://github.com/apache/lucene-solr/pull/1979#discussion_r504668474 ## File path: lucene/CHANGES.txt ## @@ -19,7 +19,7 @@ API Changes * LUCENE-8474: RAMDirectory and associated deprecated classes have been removed. (Dawid Weiss) -* LUCENE-3041: The deprecated Weight#extractTerms() method has been Review comment: Hi could you please undo all the formaating changes in the CHANGES.txt? This makes merging of other pull requests hard. Maybe your editor reformatted the whole file. I'd just copy the new changes entry into the file. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14282) /get handler doesn't return copied fields
[
https://issues.apache.org/jira/browse/SOLR-14282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213910#comment-17213910
]
Andrei Minin commented on SOLR-14282:
-
David, if /get is used in distributed processor (as I guess, to synchronize
updates across nodes), may be it makes sense to create separate rest point to
sync data from update log and return SolrInputDocument instead of SolrDocument?
It sounds like you need input document to write it into index, not
solrDocument, on remote nodes.
1) /get that returns SolrInputDocument from update log,there is no need to
convert to Lucene Document and then to SolrDocument, just return
SolrInputDocument and write it to index on each node - avoiding non needed
transformation will save CPU / memory resources.
2) /get that returns SolrDocument matching SolrDocument returned from Lucene
searcher (Lucene Document -> SolrDocument), but without writing document to
index (real time get before index commit). This document must contain all
fields and values processed by analyzers, etc - exact match of SolrDocument
returned by searcher by doc id after index commit.
> /get handler doesn't return copied fields
> -
>
> Key: SOLR-14282
> URL: https://issues.apache.org/jira/browse/SOLR-14282
> Project: Solr
> Issue Type: Bug
> Components: search, SolrJ
>Affects Versions: 8.4
> Environment: SOLR 8.4.0, SOLRJ, Oracle Java 8
>Reporter: Andrei Minin
>Priority: Major
> Attachments: SOLR-14282-test-update.patch, copied_fields_test.zip,
> managed-schema.xml
>
>
> We are using /get handler to retrieve documents by id in our Java application
> (SolrJ)
> I found that copied fields are missing in documents returned by /get handler
> but same documents returned by query contain copied (by schema) fields.
> Attached documents:
> # Integration test project archive
> # Managed schema file for SOLR
> SOLR schema details:
> # Unique field name "d_ida_s"
> # Lowecase text type definition:
> {code:java}
> positionIncrementGap="100">
>
>
>
>
> {code}
> 3. Copy field instruction sample:
> {code:java}
> stored="true" multiValued="false"/>
> />
> {code}
> ConcurrenceUserNamea_s is string type field and ConcurrenceUserNameu_lca_s is
> lower case text type field.
> Integration test uploads document to SOLR server and makes 2 requests: one
> using /get rest point to fetch document by id and one using query field name>:.
> Document returned by /get rest, doesn't have copied fields while document
> returned by query, contains copied fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14654) Remove plugin loading from .system collection (for 9.0)
[
https://issues.apache.org/jira/browse/SOLR-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213911#comment-17213911
]
Cassandra Targett commented on SOLR-14654:
--
[~noble.paul], This commit broke the Ref Guide build on master:
https://github.com/apache/lucene-solr/commit/03fe8e52609a76572a1e39084a2f75471c65614e
The page {{blob-store-api.adoc}} was removed, and while the link to the page
was removed from its parent {{configuration-apis.adoc}}, the definition of the
blob-store-api.adoc page as its child was not. At the top of the page there is
a section that defines all of the pages that are considered it's children:
{code}
= Configuration APIs
:page-children: blob-store-api, config-api, request-parameters-api,
managed-resources
{code}
When a page is removed, it must also be removed from the page-children list or
the build will fail.
Since this change was on master only, it's very confusing how this sort of
thing can happen again with this issue. Gradle has a task to build the Ref
Guide that requires *no* local dependencies - {{gradlew buildSite}}. It takes
only 2-3 minutes. It would have failed and shown you that this parent page
still contained a reference to a child that no longer exists. Since you were
doing a commit only for Ref Guide changes, it becomes even stranger that you
wouldn't check that you'd made all the required changes successfully.
> Remove plugin loading from .system collection (for 9.0)
> ---
>
> Key: SOLR-14654
> URL: https://issues.apache.org/jira/browse/SOLR-14654
> Project: Solr
> Issue Type: Task
>Reporter: Noble Paul
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This code must go from master
> all places where "runtimeLib" can be used will be removed from 9.0 . With
> the new package system in place we don;t need this anymore
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] sarowe commented on pull request #1965: LUCENE-9572 - TypeAsSynonymFilter gains selective flag transfer and an ignore list.
sarowe commented on pull request #1965: URL: https://github.com/apache/lucene-solr/pull/1965#issuecomment-708400010 > @sarowe do you have any comments? I'll take a look later today. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14654) Remove plugin loading from .system collection (for 9.0)
[
https://issues.apache.org/jira/browse/SOLR-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213920#comment-17213920
]
Noble Paul commented on SOLR-14654:
---
I ran
{{gradlew precommit}}
and everything went fine
I was not aware of the command
{{gradlew buildSite}}
I shall fix it
> Remove plugin loading from .system collection (for 9.0)
> ---
>
> Key: SOLR-14654
> URL: https://issues.apache.org/jira/browse/SOLR-14654
> Project: Solr
> Issue Type: Task
>Reporter: Noble Paul
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This code must go from master
> all places where "runtimeLib" can be used will be removed from 9.0 . With
> the new package system in place we don;t need this anymore
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14654) Remove plugin loading from .system collection (for 9.0)
[ https://issues.apache.org/jira/browse/SOLR-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213924#comment-17213924 ] ASF subversion and git services commented on SOLR-14654: Commit a7a6757afffee9e9a541c80d0c14947645c0d410 in lucene-solr's branch refs/heads/master from Noble Paul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a7a6757 ] SOLR-14654: ref guide error > Remove plugin loading from .system collection (for 9.0) > --- > > Key: SOLR-14654 > URL: https://issues.apache.org/jira/browse/SOLR-14654 > Project: Solr > Issue Type: Task >Reporter: Noble Paul >Priority: Major > Fix For: master (9.0) > > Time Spent: 0.5h > Remaining Estimate: 0h > > This code must go from master > all places where "runtimeLib" can be used will be removed from 9.0 . With > the new package system in place we don;t need this anymore -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-10059) In SolrCloud, every fq added via is computed twice.
[
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213937#comment-17213937
]
Johannes Baiter commented on SOLR-10059:
I believe SOLR-14931 describes exactly the issue [~tboeghk] described in the
above comment, thank you [~cpoerschke]!
> In SolrCloud, every fq added via is computed twice.
>
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 6.4
>Reporter: Marc Morissette
>Priority: Major
> Labels: performance
> Attachments: SOLR-10059_7x.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While researching another issue, I noticed that parameters appended to a
> query via SearchHandler's are added to the query twice
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field
> queries added in this fashion are however computed twice and that hinders
> performance on filter queries that aren't simple bitsets such as those
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough
> collection, then replace "appends" with "defaults". You'll notice significant
> performance improvements.
> {code}
>
>
> {!collapse field=routingKey hint=top_fc}
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10059) In SolrCloud, every fq added via is computed twice.
[
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213937#comment-17213937
]
Johannes Baiter edited comment on SOLR-10059 at 10/14/20, 2:01 PM:
---
I believe SOLR-14931 describes exactly the same issue in the above comment by
[~tboeghk], thank you [~cpoerschke]!
was (Author: jbaiter):
I believe SOLR-14931 describes exactly the issue [~tboeghk] described in the
above comment, thank you [~cpoerschke]!
> In SolrCloud, every fq added via is computed twice.
>
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 6.4
>Reporter: Marc Morissette
>Priority: Major
> Labels: performance
> Attachments: SOLR-10059_7x.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While researching another issue, I noticed that parameters appended to a
> query via SearchHandler's are added to the query twice
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field
> queries added in this fashion are however computed twice and that hinders
> performance on filter queries that aren't simple bitsets such as those
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough
> collection, then replace "appends" with "defaults". You'll notice significant
> performance improvements.
> {code}
>
>
> {!collapse field=routingKey hint=top_fc}
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14282) /get handler doesn't return copied fields
[
https://issues.apache.org/jira/browse/SOLR-14282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213938#comment-17213938
]
Erik Hatcher commented on SOLR-14282:
-
{quote}Now document from /get contains copied field but field value is not
correct because not changed by field index analyzer.
{quote}
[~ami...@intellective.com] - Field index analyzer? You're asking /get to
return back results from index-time analysis - like tokens? You won't get
this back from /select either.
Also - stored field values can come from not just the input document or a
copyField, but also update processors. IMO, for /get to return stored fields
like /select would return is asking too much... or at the very least deserves a
new parameter (or handler?) that would go to the effort of pulling a raw doc
from TLOG, and running all the update chain except actually adding the
document, in order to return stored fields.
> /get handler doesn't return copied fields
> -
>
> Key: SOLR-14282
> URL: https://issues.apache.org/jira/browse/SOLR-14282
> Project: Solr
> Issue Type: Bug
> Components: search, SolrJ
>Affects Versions: 8.4
> Environment: SOLR 8.4.0, SOLRJ, Oracle Java 8
>Reporter: Andrei Minin
>Priority: Major
> Attachments: SOLR-14282-test-update.patch, copied_fields_test.zip,
> managed-schema.xml
>
>
> We are using /get handler to retrieve documents by id in our Java application
> (SolrJ)
> I found that copied fields are missing in documents returned by /get handler
> but same documents returned by query contain copied (by schema) fields.
> Attached documents:
> # Integration test project archive
> # Managed schema file for SOLR
> SOLR schema details:
> # Unique field name "d_ida_s"
> # Lowecase text type definition:
> {code:java}
> positionIncrementGap="100">
>
>
>
>
> {code}
> 3. Copy field instruction sample:
> {code:java}
> stored="true" multiValued="false"/>
> />
> {code}
> ConcurrenceUserNamea_s is string type field and ConcurrenceUserNameu_lca_s is
> lower case text type field.
> Integration test uploads document to SOLR server and makes 2 requests: one
> using /get rest point to fetch document by id and one using query field name>:.
> Document returned by /get rest, doesn't have copied fields while document
> returned by query, contains copied fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
gus-asf commented on a change in pull request #1979: URL: https://github.com/apache/lucene-solr/pull/1979#discussion_r504710181 ## File path: lucene/CHANGES.txt ## @@ -19,7 +19,7 @@ API Changes * LUCENE-8474: RAMDirectory and associated deprecated classes have been removed. (Dawid Weiss) -* LUCENE-3041: The deprecated Weight#extractTerms() method has been Review comment: Yeah apparently the file has a bazillion lines that end in spaces. I typically want this on in the editor to avoid silly extra white-space in code, I'll have to figure out how to turn it off temporarily. Might not be bad to have a commit at some point that fixes this up, but we don't have to do it here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14930) Deprecate rulebased replica placement strategy
[ https://issues.apache.org/jira/browse/SOLR-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213945#comment-17213945 ] Ilan Ginzburg commented on SOLR-14930: -- I think this Jira should be called "remove" and not "deprecate" > Deprecate rulebased replica placement strategy > -- > > Key: SOLR-14930 > URL: https://issues.apache.org/jira/browse/SOLR-14930 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This should be removed from 9.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1980: SOLR-14930: Deprecate rulebased replica placement strategy
murblanc commented on a change in pull request #1980: URL: https://github.com/apache/lucene-solr/pull/1980#discussion_r504718678 ## File path: solr/core/src/java/org/apache/solr/cloud/api/collections/Assign.java ## @@ -556,19 +500,8 @@ public static AssignStrategy createAssignStrategy(SolrCloudManager solrCloudMana // If a cluster wide placement plugin is configured (and that's the only way to define a placement plugin), it overrides // per collection configuration (i.e. rules are ignored) Review comment: Need to update comment as with rules gone, there's nothing per collection anymore This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9579) Update Unit to latest 4.x release
Mike Drob created LUCENE-9579: - Summary: Update Unit to latest 4.x release Key: LUCENE-9579 URL: https://issues.apache.org/jira/browse/LUCENE-9579 Project: Lucene - Core Issue Type: Test Reporter: Mike Drob Assignee: Mike Drob -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9579) Update JUnit to latest 4.x release
[ https://issues.apache.org/jira/browse/LUCENE-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated LUCENE-9579: -- Summary: Update JUnit to latest 4.x release (was: Update Unit to latest 4.x release) > Update JUnit to latest 4.x release > -- > > Key: LUCENE-9579 > URL: https://issues.apache.org/jira/browse/LUCENE-9579 > Project: Lucene - Core > Issue Type: Test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
gus-asf commented on a change in pull request #1979:
URL: https://github.com/apache/lucene-solr/pull/1979#discussion_r504726501
##
File path:
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/DropIfFlaggedFilter.java
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.analysis.miscellaneous;
+
+import org.apache.lucene.analysis.FilteringTokenFilter;
+import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.tokenattributes.FlagsAttribute;
+
+/**
+ * Allows Tokens with a given combination of flags to be dropped. If all flags
specified are present
+ * the token is dropped, otherwise it is retained.
+ *
+ * @see DropIfFlaggedFilterFactory
+ * @since 8.8.0
+ */
+public class DropIfFlaggedFilter extends FilteringTokenFilter {
Review comment:
That said if it's a standard for the project I'll do it. I have my
disagreements but we can hash that out elsewhere.
##
File path:
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/DropIfFlaggedFilter.java
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.analysis.miscellaneous;
+
+import org.apache.lucene.analysis.FilteringTokenFilter;
+import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.tokenattributes.FlagsAttribute;
+
+/**
+ * Allows Tokens with a given combination of flags to be dropped. If all flags
specified are present
+ * the token is dropped, otherwise it is retained.
+ *
+ * @see DropIfFlaggedFilterFactory
+ * @since 8.8.0
+ */
+public class DropIfFlaggedFilter extends FilteringTokenFilter {
+ private final FlagsAttribute flagsAtt = addAttribute(FlagsAttribute.class);
+ private final int dropFlags;
+
+ /**
+ * Construct a token stream filtering the given input.
+ *
+ * @param input the source stream
+ * @param dropFlags a combination of flags that indicates that the token
should be dropped.
+ */
+ @SuppressWarnings("WeakerAccess")
+ protected DropIfFlaggedFilter(TokenStream input, int dropFlags) {
Review comment:
Not sure why that wound up protected. Thx for noticing this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14282) /get handler doesn't return copied fields
[
https://issues.apache.org/jira/browse/SOLR-14282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213958#comment-17213958
]
Andrei Minin commented on SOLR-14282:
-
Erik, you are right about tokens / analyzers - tokens not returned and it is
"too much", no need to return it, my fault, I forgot it.
Returning copied fields is sufficient. May be using additional parameter in
request, as David suggested, will work?
> /get handler doesn't return copied fields
> -
>
> Key: SOLR-14282
> URL: https://issues.apache.org/jira/browse/SOLR-14282
> Project: Solr
> Issue Type: Bug
> Components: search, SolrJ
>Affects Versions: 8.4
> Environment: SOLR 8.4.0, SOLRJ, Oracle Java 8
>Reporter: Andrei Minin
>Priority: Major
> Attachments: SOLR-14282-test-update.patch, copied_fields_test.zip,
> managed-schema.xml
>
>
> We are using /get handler to retrieve documents by id in our Java application
> (SolrJ)
> I found that copied fields are missing in documents returned by /get handler
> but same documents returned by query contain copied (by schema) fields.
> Attached documents:
> # Integration test project archive
> # Managed schema file for SOLR
> SOLR schema details:
> # Unique field name "d_ida_s"
> # Lowecase text type definition:
> {code:java}
> positionIncrementGap="100">
>
>
>
>
> {code}
> 3. Copy field instruction sample:
> {code:java}
> stored="true" multiValued="false"/>
> />
> {code}
> ConcurrenceUserNamea_s is string type field and ConcurrenceUserNameu_lca_s is
> lower case text type field.
> Integration test uploads document to SOLR server and makes 2 requests: one
> using /get rest point to fetch document by id and one using query field name>:.
> Document returned by /get rest, doesn't have copied fields while document
> returned by query, contains copied fields.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1974: SOLR-14914: Add option to disable metrics collection
madrob commented on a change in pull request #1974:
URL: https://github.com/apache/lucene-solr/pull/1974#discussion_r504766830
##
File path: solr/core/src/java/org/apache/solr/metrics/SolrMetricManager.java
##
@@ -110,19 +110,22 @@
public static final int DEFAULT_CLOUD_REPORTER_PERIOD = 60;
- private MetricRegistry.MetricSupplier counterSupplier;
- private MetricRegistry.MetricSupplier meterSupplier;
- private MetricRegistry.MetricSupplier timerSupplier;
- private MetricRegistry.MetricSupplier histogramSupplier;
+ private final MetricsConfig metricsConfig;
+ private final MetricRegistry.MetricSupplier counterSupplier;
+ private final MetricRegistry.MetricSupplier meterSupplier;
+ private final MetricRegistry.MetricSupplier timerSupplier;
+ private final MetricRegistry.MetricSupplier histogramSupplier;
public SolrMetricManager() {
+metricsConfig = new MetricsConfig.MetricsConfigBuilder().build();
counterSupplier = MetricSuppliers.counterSupplier(null, null);
meterSupplier = MetricSuppliers.meterSupplier(null, null);
Review comment:
@Muse-Dev is not _obviously_ wrong, because there is a path where if
info is not null, but loader is, then we get an NPE.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1974: SOLR-14914: Add option to disable metrics collection
madrob commented on a change in pull request #1974:
URL: https://github.com/apache/lucene-solr/pull/1974#discussion_r504767202
##
File path: solr/core/src/java/org/apache/solr/metrics/SolrMetricManager.java
##
@@ -110,19 +110,22 @@
public static final int DEFAULT_CLOUD_REPORTER_PERIOD = 60;
- private MetricRegistry.MetricSupplier counterSupplier;
- private MetricRegistry.MetricSupplier meterSupplier;
- private MetricRegistry.MetricSupplier timerSupplier;
- private MetricRegistry.MetricSupplier histogramSupplier;
+ private final MetricsConfig metricsConfig;
+ private final MetricRegistry.MetricSupplier counterSupplier;
+ private final MetricRegistry.MetricSupplier meterSupplier;
+ private final MetricRegistry.MetricSupplier timerSupplier;
+ private final MetricRegistry.MetricSupplier histogramSupplier;
public SolrMetricManager() {
+metricsConfig = new MetricsConfig.MetricsConfigBuilder().build();
counterSupplier = MetricSuppliers.counterSupplier(null, null);
meterSupplier = MetricSuppliers.meterSupplier(null, null);
Review comment:
(But it is still wrong in this case)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14654) Remove plugin loading from .system collection (for 9.0)
[ https://issues.apache.org/jira/browse/SOLR-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214004#comment-17214004 ] ASF subversion and git services commented on SOLR-14654: Commit 3fbde34f40f416ee1d5578448c18d35e0b331be4 in lucene-solr's branch refs/heads/reference_impl_dev from markrmil...@gmail.com [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3fbde34 ] SOLR-14654: remove ref to blob-store-api > Remove plugin loading from .system collection (for 9.0) > --- > > Key: SOLR-14654 > URL: https://issues.apache.org/jira/browse/SOLR-14654 > Project: Solr > Issue Type: Task >Reporter: Noble Paul >Priority: Major > Fix For: master (9.0) > > Time Spent: 0.5h > Remaining Estimate: 0h > > This code must go from master > all places where "runtimeLib" can be used will be removed from 9.0 . With > the new package system in place we don;t need this anymore -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
gus-asf commented on a change in pull request #1979:
URL: https://github.com/apache/lucene-solr/pull/1979#discussion_r504726501
##
File path:
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/DropIfFlaggedFilter.java
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.analysis.miscellaneous;
+
+import org.apache.lucene.analysis.FilteringTokenFilter;
+import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.tokenattributes.FlagsAttribute;
+
+/**
+ * Allows Tokens with a given combination of flags to be dropped. If all flags
specified are present
+ * the token is dropped, otherwise it is retained.
+ *
+ * @see DropIfFlaggedFilterFactory
+ * @since 8.8.0
+ */
+public class DropIfFlaggedFilter extends FilteringTokenFilter {
Review comment:
If it's a standard for the project I'll do it. I have my disagreements
but we can hash that out elsewhere.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob opened a new pull request #1981: LUCENE-9579 Update to JUnit 4.13.1
madrob opened a new pull request #1981: URL: https://github.com/apache/lucene-solr/pull/1981 @dweiss I didn't see any compat issues that this caused with carrot search runner, but will give you a chance to look before I merge this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] sigram commented on a change in pull request #1974: SOLR-14914: Add option to disable metrics collection
sigram commented on a change in pull request #1974:
URL: https://github.com/apache/lucene-solr/pull/1974#discussion_r504781919
##
File path: solr/core/src/java/org/apache/solr/metrics/SolrMetricManager.java
##
@@ -110,19 +110,22 @@
public static final int DEFAULT_CLOUD_REPORTER_PERIOD = 60;
- private MetricRegistry.MetricSupplier counterSupplier;
- private MetricRegistry.MetricSupplier meterSupplier;
- private MetricRegistry.MetricSupplier timerSupplier;
- private MetricRegistry.MetricSupplier histogramSupplier;
+ private final MetricsConfig metricsConfig;
+ private final MetricRegistry.MetricSupplier counterSupplier;
+ private final MetricRegistry.MetricSupplier meterSupplier;
+ private final MetricRegistry.MetricSupplier timerSupplier;
+ private final MetricRegistry.MetricSupplier histogramSupplier;
public SolrMetricManager() {
+metricsConfig = new MetricsConfig.MetricsConfigBuilder().build();
counterSupplier = MetricSuppliers.counterSupplier(null, null);
meterSupplier = MetricSuppliers.meterSupplier(null, null);
Review comment:
@madrob well spotted, though the message is somewhat misleading. I'll
fix this in `MetricSuppliers`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] nknize commented on pull request #1940: LUCENE-9552: Adds a LatLonPoint query that accepts an array of LatLonGeometries
nknize commented on pull request #1940: URL: https://github.com/apache/lucene-solr/pull/1940#issuecomment-708491077 Circle approximation vs distance aside I think the real benefit of this PR is that it creates a single query for multiple geometry types whereas right now we have to create a Boolean query for handling GeometryCollections? I like the idea of handling the query logic in a single query vs having to build a Boolean query of many LatLon queries. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
gus-asf commented on pull request #1979: URL: https://github.com/apache/lucene-solr/pull/1979#issuecomment-708492412 > I am not so happy about the tests using the "obsolete" Token class. I'd use a MockTokenizer with a string instead. I don't see how this would allow me to set flags on the individual tokens? Do you have an example of that? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
uschindler commented on a change in pull request #1979:
URL: https://github.com/apache/lucene-solr/pull/1979#discussion_r504798160
##
File path:
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/DropIfFlaggedFilter.java
##
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.analysis.miscellaneous;
+
+import org.apache.lucene.analysis.FilteringTokenFilter;
+import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.tokenattributes.FlagsAttribute;
+
+/**
+ * Allows Tokens with a given combination of flags to be dropped. If all flags
specified are present
+ * the token is dropped, otherwise it is retained.
+ *
+ * @see DropIfFlaggedFilterFactory
+ * @since 8.8.0
+ */
+public class DropIfFlaggedFilter extends FilteringTokenFilter {
Review comment:
Thanks. All fine.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
uschindler commented on pull request #1979: URL: https://github.com/apache/lucene-solr/pull/1979#issuecomment-708504330 > I don't see how this would allow me to set flags on the individual tokens? Do you have an example of that? Sorry I missed that. The Token class ha bit of bad history, so we try to avoid it where possible. But for your use case a CannedTokenStream using a token array is perfectly fine. It's also the Token class of test-framework. All fine, sorry. We should mabe create some better "tester" method so you can create a tokenstream for tests using a builder interface like: `CannedTokenStream.builder.addToken(a,b,c,d).addToken(x,y,zw).build()` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler edited a comment on pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
uschindler edited a comment on pull request #1979: URL: https://github.com/apache/lucene-solr/pull/1979#issuecomment-708504330 > I don't see how this would allow me to set flags on the individual tokens? Do you have an example of that? Sorry I missed that. The Token class ha bit of bad history, so we try to avoid it where possible. But for your use case a CannedTokenStream using a token array is perfectly fine. It's also the Token class of test-framework. All fine, sorry. We should mabe create some better "tester" method so you can create a tokenstream for tests using a builder interface like: `CannedTokenStream.builder().addToken(a,b,c,d).addToken(x,y,z,w).build()` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14647) Importing Gradle Projects into Eclipse pollutes checkout
[
https://issues.apache.org/jira/browse/SOLR-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214040#comment-17214040
]
James Dyer commented on SOLR-14647:
---
I am an eclipse user, not an expert but my belief is that every eclipse project
will have a .project file and .settings directory in its root. Additionally,
java projects will have a .classpath file in its root. These should be in
.gitignore .
As for bin, this is controlled by a line in .classpath. When I imported the
gradle project back in July using the standard "import > gradle existing
project" option in eclipse, by default we had this line in .classpath:
{code:xml}
{code}
However, a maven project will have this:
{code:xml}
{code}
I would hope this is somehow configurable, should we not want to have such a
common directory name to .gitignore. Additionally, the workaround for the
circular reference problem probably fixes this also but (I think) eclipse users
will need to be instructed to import and sync the project in a non-standard
way. If I understand the nature of the work there correctly, we are
constructing the eclipse files by hand like the old "ant eclipse" did,
bypassing the usual import mechanisms, right? (As an aside, I worked around
the circular reference problem simply by reducing the severity of that
particular message from "error" to "warning". I could be wrong, but I think
that message is more an annoyance and does not affect functionality)
> Importing Gradle Projects into Eclipse pollutes checkout
>
>
> Key: SOLR-14647
> URL: https://issues.apache.org/jira/browse/SOLR-14647
> Project: Solr
> Issue Type: Bug
> Components: Build
>Affects Versions: master (9.0)
>Reporter: James Dyer
>Assignee: James Dyer
>Priority: Minor
> Fix For: master (9.0)
>
> Attachments: SOLR-14647.patch
>
>
> Switch to master branch, then open Eclipse IDE and select "file > import >
> existing gradle project" to import lucene/solr. Afterwards, "git status"
> shows unstaged ".project", ".classpath", ".settings" and "bin"
> files/directories.
> Adjust the .gitignore file to correctly filter these out.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] munendrasn merged pull request #1975: Include missing commands in package tool help section
munendrasn merged pull request #1975: URL: https://github.com/apache/lucene-solr/pull/1975 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14915) Prometheus-exporter should not depend on Solr-core
[ https://issues.apache.org/jira/browse/SOLR-14915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214045#comment-17214045 ] Dawid Weiss commented on SOLR-14915: I personally think there should be no single binary Solr distribution with everything thrown in. Rather, tools that are independent should just ship as independent binaries (with all dependencies allowing their execution). Partial jar subsets with links across the binary distribution will be very difficult to manage automatically... > Prometheus-exporter should not depend on Solr-core > -- > > Key: SOLR-14915 > URL: https://issues.apache.org/jira/browse/SOLR-14915 > Project: Solr > Issue Type: Improvement > Components: contrib - prometheus-exporter >Reporter: David Smiley >Assignee: David Smiley >Priority: Minor > Attachments: patch.patch > > Time Spent: 40m > Remaining Estimate: 0h > > I think it's *crazy* that our Prometheus exporter depends on Solr-core -- > this thing is a _client_ of Solr; it does not live within Solr. The exporter > ought to be fairly lean. One consequence of this dependency is that, for > example, security vulnerabilities reported against Solr (e.g. Jetty) can (and > do, where I work) wind up being reported against this module even though > Prometheus isn't using Jetty. > From my evaluation today of what's going on, it appears the crux of the > problem is that the prometheus exporter uses some utility mechanisms in > Solr-core like XmlConfig (which depends on SolrResourceLoader and the rabbit > hole goes deeper...) and DOMUtils (further depends on PropertiesUtil). It > can easy be made to not use XmlConfig. DOMUtils & PropertiesUtil could move > to SolrJ which already has lots of little dependency-free utilities needed by > SolrJ and Solr-core alike. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9574) Add a token filter to drop tokens based on flags.
[ https://issues.apache.org/jira/browse/LUCENE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214082#comment-17214082 ] ASF subversion and git services commented on LUCENE-9574: - Commit ab5671d367eb0099269bb91696c5b8c84090485f in lucene-solr's branch refs/heads/master from Gus Heck [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ab5671d ] LUCENE-9574 Add DropIfFlaggedFilterFactory (#1979) > Add a token filter to drop tokens based on flags. > - > > Key: LUCENE-9574 > URL: https://issues.apache.org/jira/browse/LUCENE-9574 > Project: Lucene - Core > Issue Type: New Feature > Components: modules/analysis >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Time Spent: 8h 10m > Remaining Estimate: 0h > > (Breaking this off of SOLR-14597 for independent review) > A filter that tests flags on tokens vs a bitmask and drops tokens that have > all specified flags. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf merged pull request #1979: LUCENE-9574 Add DropIfFlaggedFilterFactory
gus-asf merged pull request #1979: URL: https://github.com/apache/lucene-solr/pull/1979 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9568) FuzzyTermEnums sets negative boost for fuzzy search & highlight
[
https://issues.apache.org/jira/browse/LUCENE-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214084#comment-17214084
]
Michael McCandless commented on LUCENE-9568:
Whoa, thanks [~rcmuir] – that is indeed a very important reason to keep using
{{minTermLength}}! I forgot about that :)
Maybe we could just do {{Math.max(0, similarity)}} to ensure we never try to
set negative boost?
Let's also add a comment above where we set {{minTermLength}} indicating the
importance?
> FuzzyTermEnums sets negative boost for fuzzy search & highlight
> ---
>
> Key: LUCENE-9568
> URL: https://issues.apache.org/jira/browse/LUCENE-9568
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/highlighter
>Affects Versions: 8.5.1
>Reporter: Juraj Jurčo
>Priority: Minor
> Labels: highlighting, newbie
> Attachments: FindSqlHighlightTest.java
>
>
> *Description*
> When user indexes a word with an apostrophe and constructs a fuzzy query for
> highlighter, it throws an exception with set negative boost for a query.
> *Repro Steps*
> # Index a text with apostrophe. E.g. doesn't
> # Parse a fuzzy query e.g.: se~, se~2, se~3
> # Try to highlight a text with apostrophe
> # The exception is thrown (for details see attached test test with repro
> steps)
> *Actual Result*
> {{java.lang.IllegalArgumentException: boost must be a positive float, got
> -1.0}}
> *Expected Result*
> * No exception.
> * Highlighting marks are inserted into a text.
> *Workaround*
> - not known.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14930) Deprecate rulebased replica placement strategy
[ https://issues.apache.org/jira/browse/SOLR-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214087#comment-17214087 ] Cassandra Targett commented on SOLR-14930: -- bq. I think this Jira should be called "remove" and not "deprecate" It should be both, since Rule-based Replica Placement has AFAIK *never* been formally deprecated to date. It's not enough to simply remove it from master/9.0 - it also needs 8.x changes to mark the code as deprecated and update the Ref Guide docs for the feature and the Guide's Upgrade Notes. We should be sure to include it with the official Release Notes and update the Deprecations page in the wiki. It would be also be nice if some of this messaging could also describe what's going to replace it (the new autoscaling plugin? nothing?) from the people doing the removing so the rest of us don't have to try to figure it out from some Slack channel or other indirect non-Jira sources when we get asked privately what someone is supposed to do for the same functionality in 9.0. > Deprecate rulebased replica placement strategy > -- > > Key: SOLR-14930 > URL: https://issues.apache.org/jira/browse/SOLR-14930 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > This should be removed from 9.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14930) Deprecate rulebased replica placement strategy
[ https://issues.apache.org/jira/browse/SOLR-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214087#comment-17214087 ] Cassandra Targett edited comment on SOLR-14930 at 10/14/20, 5:32 PM: - bq. I think this Jira should be called "remove" and not "deprecate" It should be both, since Rule-based Replica Placement has AFAIK *never* been formally deprecated to date. It's not enough to simply remove it from master/9.0 - it also needs 8.x changes to mark the code as deprecated and update the Ref Guide docs for the feature and the Guide's Upgrade Notes. We should be sure to include it with the official Release Notes and update the Deprecations page in the wiki. It would be also be nice if some of this messaging could also describe what's going to replace it (the new autoscaling plugin? nothing?) from the people doing the removing so the rest of us don't have to try to figure it out from some Slack channel or other indirect non-Jira sources when we get asked privately what someone is supposed to do for the same functionality in 9.0. *To be very clear, I have *no* problem with removing it. But things need to be done completely and openly. was (Author: ctargett): bq. I think this Jira should be called "remove" and not "deprecate" It should be both, since Rule-based Replica Placement has AFAIK *never* been formally deprecated to date. It's not enough to simply remove it from master/9.0 - it also needs 8.x changes to mark the code as deprecated and update the Ref Guide docs for the feature and the Guide's Upgrade Notes. We should be sure to include it with the official Release Notes and update the Deprecations page in the wiki. It would be also be nice if some of this messaging could also describe what's going to replace it (the new autoscaling plugin? nothing?) from the people doing the removing so the rest of us don't have to try to figure it out from some Slack channel or other indirect non-Jira sources when we get asked privately what someone is supposed to do for the same functionality in 9.0. > Deprecate rulebased replica placement strategy > -- > > Key: SOLR-14930 > URL: https://issues.apache.org/jira/browse/SOLR-14930 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > This should be removed from 9.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14930) Deprecate rulebased replica placement strategy
[ https://issues.apache.org/jira/browse/SOLR-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214087#comment-17214087 ] Cassandra Targett edited comment on SOLR-14930 at 10/14/20, 5:32 PM: - bq. I think this Jira should be called "remove" and not "deprecate" It should be both, since Rule-based Replica Placement has AFAIK *never* been formally deprecated to date. It's not enough to simply remove it from master/9.0 - it also needs 8.x changes to mark the code as deprecated and update the Ref Guide docs for the feature and the Guide's Upgrade Notes. We should be sure to include it with the official Release Notes and update the Deprecations page in the wiki. It would be also be nice if some of this messaging could also describe what's going to replace it (the new autoscaling plugin? nothing?) from the people doing the removing so the rest of us don't have to try to figure it out from some Slack channel or other indirect non-Jira sources when we get asked privately what someone is supposed to do for the same functionality in 9.0. -- To be very clear, I have *no* problem with removing it. But things need to be done completely and openly. was (Author: ctargett): bq. I think this Jira should be called "remove" and not "deprecate" It should be both, since Rule-based Replica Placement has AFAIK *never* been formally deprecated to date. It's not enough to simply remove it from master/9.0 - it also needs 8.x changes to mark the code as deprecated and update the Ref Guide docs for the feature and the Guide's Upgrade Notes. We should be sure to include it with the official Release Notes and update the Deprecations page in the wiki. It would be also be nice if some of this messaging could also describe what's going to replace it (the new autoscaling plugin? nothing?) from the people doing the removing so the rest of us don't have to try to figure it out from some Slack channel or other indirect non-Jira sources when we get asked privately what someone is supposed to do for the same functionality in 9.0. *To be very clear, I have *no* problem with removing it. But things need to be done completely and openly. > Deprecate rulebased replica placement strategy > -- > > Key: SOLR-14930 > URL: https://issues.apache.org/jira/browse/SOLR-14930 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > This should be removed from 9.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9574) Add a token filter to drop tokens based on flags.
[ https://issues.apache.org/jira/browse/LUCENE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214091#comment-17214091 ] Uwe Schindler commented on LUCENE-9574: --- Thanks! > Add a token filter to drop tokens based on flags. > - > > Key: LUCENE-9574 > URL: https://issues.apache.org/jira/browse/LUCENE-9574 > Project: Lucene - Core > Issue Type: New Feature > Components: modules/analysis >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Time Spent: 8h 10m > Remaining Estimate: 0h > > (Breaking this off of SOLR-14597 for independent review) > A filter that tests flags on tokens vs a bitmask and drops tokens that have > all specified flags. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #1965: LUCENE-9572 - TypeAsSynonymFilter gains selective flag transfer and an ignore list.
uschindler commented on a change in pull request #1965:
URL: https://github.com/apache/lucene-solr/pull/1965#discussion_r504855884
##
File path:
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/TypeAsSynonymFilter.java
##
@@ -64,9 +82,15 @@ public boolean incrementToken() throws IOException {
}
termAtt.append(typeAtt.type());
posIncrAtt.setPositionIncrement(0);
+ // control what flags transfer to synonym
+ int flags = getAttribute(FlagsAttribute.class).getFlags();
Review comment:
This should not call getAttribute and instead use the instance
`flagsAtt` above (instance field). In the next line it's correct.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #1965: LUCENE-9572 - TypeAsSynonymFilter gains selective flag transfer and an ignore list.
uschindler commented on a change in pull request #1965: URL: https://github.com/apache/lucene-solr/pull/1965#discussion_r504856970 ## File path: lucene/CHANGES.txt ## @@ -15,7 +15,7 @@ API Changes * LUCENE-8474: RAMDirectory and associated deprecated classes have been removed. (Dawid Weiss) -* LUCENE-3041: The deprecated Weight#extractTerms() method has been +* LUCENE-3041: The deprecated Weight#extractTerms() method has been Review comment: same issue like in previous PR! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9575) Add PatternTypingFilter
[
https://issues.apache.org/jira/browse/LUCENE-9575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214100#comment-17214100
]
Uwe Schindler commented on LUCENE-9575:
---
One you have a pull request, I'm happy to review the tokenfilter!
> Add PatternTypingFilter
> ---
>
> Key: LUCENE-9575
> URL: https://issues.apache.org/jira/browse/LUCENE-9575
> Project: Lucene - Core
> Issue Type: New Feature
> Components: modules/analysis
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Major
>
> One of the key asks when the Library of Congress was asking me to develop the
> Advanced Query Parser was to be able to recognize arbitrary patterns that
> included punctuation such as POW/MIA or 401(k) or C++ etc. Additionally they
> wanted 401k and 401(k) to match documents with either style reference, and
> NOT match documents that happen to have isolated 401 or k tokens (i.e. not
> documents about the http status code) And of course we wanted to give up as
> little of the text analysis features they were already using.
> This filter in conjunction with the filters from LUCENE-9572, LUCENE-9574 and
> one solr specific filter in SOLR-14597 that re-analyzes tokens with an
> arbitrary analyzer defined for a type in the solr schema, combine to achieve
> this.
> This filter has the job of spotting the patterns, and adding the intended
> synonym as at type to the token (from which minimal punctuation has been
> removed). It also sets flags on the token which are retained through the
> analysis chain, and at the very end the type is converted to a synonym and
> the original token(s) for that type are dropped avoiding the match on 401
> (for example)
> The pattern matching is specified in a file that looks like:
> {code}
> 2 (\d+)\(?([a-z])\)? ::: legal2_$1_$2
> 2 (\d+)\(?([a-z])\)?\(?(\d+)\)? ::: legal3_$1_$2_$3
> 2 C\+\+ ::: c_plus_plus
> {code}
> That file would match match legal reference patterns such as 401(k), 401k,
> 501(c)3 and C++ The format is:
> :::
> and groups in the pattern are substituted into the replacement so the first
> line above would create synonyms such as:
> {code}
> 401k --> legal2_401_k
> 401(k) --> legal2_401_k
> 503(c) --> legal2_503_c
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9568) FuzzyTermEnums sets negative boost for fuzzy search & highlight
[
https://issues.apache.org/jira/browse/LUCENE-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214102#comment-17214102
]
Robert Muir commented on LUCENE-9568:
-
I'm not sure its so simple: today with TopTermsRewrite we may give it
negatives, and everything is happily sorted and top-n'd correctly, and it will
just truncate them at the very end (when it goes to create the actual Query
object). So it still gives a correct top-N. See Adrien's link.
If we were to do it any "sooner" (e.g. in fuzzytermsenum itself), it may
compute an incorrect top-N.
I think the problem is just that only TopTermsRewrite does this (honestly, it
is really the only one that should be used with this crazy query). The other
more exotic rewrite methods that might be used by spans or highlighters don't
do it, so there will be problems today in those cases.
> FuzzyTermEnums sets negative boost for fuzzy search & highlight
> ---
>
> Key: LUCENE-9568
> URL: https://issues.apache.org/jira/browse/LUCENE-9568
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/highlighter
>Affects Versions: 8.5.1
>Reporter: Juraj Jurčo
>Priority: Minor
> Labels: highlighting, newbie
> Attachments: FindSqlHighlightTest.java
>
>
> *Description*
> When user indexes a word with an apostrophe and constructs a fuzzy query for
> highlighter, it throws an exception with set negative boost for a query.
> *Repro Steps*
> # Index a text with apostrophe. E.g. doesn't
> # Parse a fuzzy query e.g.: se~, se~2, se~3
> # Try to highlight a text with apostrophe
> # The exception is thrown (for details see attached test test with repro
> steps)
> *Actual Result*
> {{java.lang.IllegalArgumentException: boost must be a positive float, got
> -1.0}}
> *Expected Result*
> * No exception.
> * Highlighting marks are inserted into a text.
> *Workaround*
> - not known.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob merged pull request #1981: LUCENE-9579 Update to JUnit 4.13.1
madrob merged pull request #1981: URL: https://github.com/apache/lucene-solr/pull/1981 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #1965: LUCENE-9572 - TypeAsSynonymFilter gains selective flag transfer and an ignore list.
uschindler commented on a change in pull request #1965:
URL: https://github.com/apache/lucene-solr/pull/1965#discussion_r504863498
##
File path:
lucene/test-framework/src/java/org/apache/lucene/analysis/BaseTokenStreamTestCase.java
##
@@ -190,7 +196,8 @@ public static void assertTokenStreamContents(TokenStream
ts, String[] output, in
if (posLengthAtt != null) posLengthAtt.setPositionLength(45987653);
if (keywordAtt != null) keywordAtt.setKeyword((i&1) == 0);
if (payloadAtt != null) payloadAtt.setPayload(new BytesRef(new byte[] {
0x00, -0x21, 0x12, -0x43, 0x24 }));
-
+ if (flagsAtt != null) flagsAtt.setFlags(Integer.MAX_VALUE); // all 1's
Review comment:
that's all bits except the first 1 (`0...`). To invert
all bits use `-1` or better `~0`
##
File path:
lucene/test-framework/src/java/org/apache/lucene/analysis/BaseTokenStreamTestCase.java
##
@@ -125,7 +125,7 @@ public void reflectWith(AttributeReflector reflector) {
// arriving to pos Y have the same endOffset)
public static void assertTokenStreamContents(TokenStream ts, String[]
output, int startOffsets[], int endOffsets[], String types[], int
posIncrements[],
Review comment:
maybe we should not change this method signature and just add another
one, so we do not need to change unrelated files like MinHashFilter?
##
File path:
lucene/test-framework/src/java/org/apache/lucene/analysis/BaseTokenStreamTestCase.java
##
@@ -294,6 +304,7 @@ public static void assertTokenStreamContents(TokenStream
ts, String[] output, in
if (posLengthAtt != null) posLengthAtt.setPositionLength(45987653);
if (keywordAtt != null) keywordAtt.setKeyword(true);
if (payloadAtt != null) payloadAtt.setPayload(new BytesRef(new byte[] {
0x00, -0x21, 0x12, -0x43, 0x24 }));
+if (flagsAtt != null) flagsAtt.setFlags(Integer.MAX_VALUE); // all 1's
Review comment:
same here
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9579) Update JUnit to latest 4.x release
[ https://issues.apache.org/jira/browse/LUCENE-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214107#comment-17214107 ] ASF subversion and git services commented on LUCENE-9579: - Commit 9805b125dc85681fe1d55bb99748deaa03999b3b in lucene-solr's branch refs/heads/master from Mike Drob [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9805b12 ] LUCENE-9579 Update to JUnit 4.13.1 (#1981) > Update JUnit to latest 4.x release > -- > > Key: LUCENE-9579 > URL: https://issues.apache.org/jira/browse/LUCENE-9579 > Project: Lucene - Core > Issue Type: Test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #1965: LUCENE-9572 - TypeAsSynonymFilter gains selective flag transfer and an ignore list.
gus-asf commented on a change in pull request #1965: URL: https://github.com/apache/lucene-solr/pull/1965#discussion_r504865817 ## File path: lucene/CHANGES.txt ## @@ -15,7 +15,7 @@ API Changes * LUCENE-8474: RAMDirectory and associated deprecated classes have been removed. (Dawid Weiss) -* LUCENE-3041: The deprecated Weight#extractTerms() method has been +* LUCENE-3041: The deprecated Weight#extractTerms() method has been Review comment: Well actually this PR predates other :), but yes same issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gus-asf commented on a change in pull request #1965: LUCENE-9572 - TypeAsSynonymFilter gains selective flag transfer and an ignore list.
gus-asf commented on a change in pull request #1965:
URL: https://github.com/apache/lucene-solr/pull/1965#discussion_r504867934
##
File path:
lucene/test-framework/src/java/org/apache/lucene/analysis/BaseTokenStreamTestCase.java
##
@@ -190,7 +196,8 @@ public static void assertTokenStreamContents(TokenStream
ts, String[] output, in
if (posLengthAtt != null) posLengthAtt.setPositionLength(45987653);
if (keywordAtt != null) keywordAtt.setKeyword((i&1) == 0);
if (payloadAtt != null) payloadAtt.setPayload(new BytesRef(new byte[] {
0x00, -0x21, 0x12, -0x43, 0x24 }));
-
+ if (flagsAtt != null) flagsAtt.setFlags(Integer.MAX_VALUE); // all 1's
Review comment:
oops yeah realized that partway through (when I switched to ~0 for the
other file), didn't come back and fix it here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob opened a new pull request #1982: LUCENE-9579 Update to JUnit 4.13.1
madrob opened a new pull request #1982: URL: https://github.com/apache/lucene-solr/pull/1982 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #1948: LUCENE-9536: Optimize OrdinalMap when one segment contains all distinct values.
mikemccand commented on a change in pull request #1948:
URL: https://github.com/apache/lucene-solr/pull/1948#discussion_r504863887
##
File path: lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java
##
@@ -271,13 +273,26 @@ protected boolean lessThan(TermsEnumIndex a,
TermsEnumIndex b) {
globalOrd++;
}
-this.firstSegments = firstSegments.build();
-this.globalOrdDeltas = globalOrdDeltas.build();
+long ramBytesUsed = BASE_RAM_BYTES_USED + segmentMap.ramBytesUsed();
+this.valueCount = globalOrd;
+
+// If the first segment contains all of the global ords, then we can apply
a small optimization
+// and hardcode the first segments and global ord deltas as all zeroes.
Review comment:
Insert possessive quote (`first segment's`)?
##
File path: lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java
##
@@ -271,13 +273,26 @@ protected boolean lessThan(TermsEnumIndex a,
TermsEnumIndex b) {
globalOrd++;
}
-this.firstSegments = firstSegments.build();
-this.globalOrdDeltas = globalOrdDeltas.build();
+long ramBytesUsed = BASE_RAM_BYTES_USED + segmentMap.ramBytesUsed();
+this.valueCount = globalOrd;
+
+// If the first segment contains all of the global ords, then we can apply
a small optimization
+// and hardcode the first segments and global ord deltas as all zeroes.
+if (ordDeltaBits.length > 0 && ordDeltaBits[0] == 0L &&
ordDeltas[0].size() == this.valueCount) {
Review comment:
Hmm why only the first segment? Couldn't it be the 3rd segment, in
addition, that matches the global ords?
Edit: ahh OK I understand now -- this opto is indeed specific to the first
segment, so we can store `this.firstSegments` as all 0s. Good!
Do we (somewhere, couldn't find it here) pre-sort all segments by the
cardinality descending? Then we could know all segments that meet this
optimization are at the start of the segments list, and possibly building the
ordinal map is faster (not sure). But then we would need to un-sort in the end
to return the final `OrdinalMap`. But it might enable this opto to apply more
often, except, I think we would then need an additional dereference on lookup,
hmm.
Does our `PackedLongValues.monotonicBuilder` already optimize for the case
where it is all 0s, for the case where another segment (not the first) has all
the global values as well?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jtibshirani commented on a change in pull request #1948: LUCENE-9536: Optimize OrdinalMap when one segment contains all distinct values.
jtibshirani commented on a change in pull request #1948:
URL: https://github.com/apache/lucene-solr/pull/1948#discussion_r504888212
##
File path: lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java
##
@@ -271,13 +273,26 @@ protected boolean lessThan(TermsEnumIndex a,
TermsEnumIndex b) {
globalOrd++;
}
-this.firstSegments = firstSegments.build();
-this.globalOrdDeltas = globalOrdDeltas.build();
+long ramBytesUsed = BASE_RAM_BYTES_USED + segmentMap.ramBytesUsed();
+this.valueCount = globalOrd;
+
+// If the first segment contains all of the global ords, then we can apply
a small optimization
+// and hardcode the first segments and global ord deltas as all zeroes.
Review comment:
I don't think the possessive quote gives the right meaning? Perhaps I
could say 'first segment indices' here to be more clear.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jtibshirani commented on a change in pull request #1948: LUCENE-9536: Optimize OrdinalMap when one segment contains all distinct values.
jtibshirani commented on a change in pull request #1948:
URL: https://github.com/apache/lucene-solr/pull/1948#discussion_r504899193
##
File path: lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java
##
@@ -271,13 +273,26 @@ protected boolean lessThan(TermsEnumIndex a,
TermsEnumIndex b) {
globalOrd++;
}
-this.firstSegments = firstSegments.build();
-this.globalOrdDeltas = globalOrdDeltas.build();
+long ramBytesUsed = BASE_RAM_BYTES_USED + segmentMap.ramBytesUsed();
+this.valueCount = globalOrd;
+
+// If the first segment contains all of the global ords, then we can apply
a small optimization
+// and hardcode the first segments and global ord deltas as all zeroes.
+if (ordDeltaBits.length > 0 && ordDeltaBits[0] == 0L &&
ordDeltas[0].size() == this.valueCount) {
Review comment:
> Do we (somewhere, couldn't find it here) pre-sort all segments by the
cardinality descending?
We do in fact -- the segments are sorted by 'weight', which in all call
sites corresponds to the number of unique terms. This was added in
[LUCENE-5782](https://issues.apache.org/jira/browse/LUCENE-5782).
> Does our PackedLongValues.monotonicBuilder already optimize for the case
where it is all 0s, for the case where another segment (not the first) has all
the global values as well?
It does look like it -- when constructing the individual `PackedInts.Reader`
instances, we identify the all 0s case and use the lightweight
`PackedInts.NullReader`. It's great we optimize that case, but it does mean
this PR doesn't make an enormous space difference.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jtibshirani commented on a change in pull request #1948: LUCENE-9536: Optimize OrdinalMap when one segment contains all distinct values.
jtibshirani commented on a change in pull request #1948:
URL: https://github.com/apache/lucene-solr/pull/1948#discussion_r504899193
##
File path: lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java
##
@@ -271,13 +273,26 @@ protected boolean lessThan(TermsEnumIndex a,
TermsEnumIndex b) {
globalOrd++;
}
-this.firstSegments = firstSegments.build();
-this.globalOrdDeltas = globalOrdDeltas.build();
+long ramBytesUsed = BASE_RAM_BYTES_USED + segmentMap.ramBytesUsed();
+this.valueCount = globalOrd;
+
+// If the first segment contains all of the global ords, then we can apply
a small optimization
+// and hardcode the first segments and global ord deltas as all zeroes.
+if (ordDeltaBits.length > 0 && ordDeltaBits[0] == 0L &&
ordDeltas[0].size() == this.valueCount) {
Review comment:
> Do we (somewhere, couldn't find it here) pre-sort all segments by the
cardinality descending?
We do in fact -- the segments are sorted by 'weight', which in all call
sites corresponds to the number of unique terms. This was added in
[LUCENE-5782](https://issues.apache.org/jira/browse/LUCENE-5782).
> Does our PackedLongValues.monotonicBuilder already optimize for the case
where it is all 0s, for the case where another segment (not the first) has all
the global values as well?
When constructing the individual `PackedInts.Reader` instances, we do
identify the all 0s case and use the lightweight `PackedInts.NullReader`. It's
great we optimize that case, but it does mean this PR doesn't make an enormous
space difference.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob merged pull request #1982: LUCENE-9579 Update to JUnit 4.13.1
madrob merged pull request #1982: URL: https://github.com/apache/lucene-solr/pull/1982 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9579) Update JUnit to latest 4.x release
[ https://issues.apache.org/jira/browse/LUCENE-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214200#comment-17214200 ] ASF subversion and git services commented on LUCENE-9579: - Commit 25f36c0c3bc669bd7fefb899eaef04757baa96b9 in lucene-solr's branch refs/heads/branch_8x from Mike Drob [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=25f36c0 ] LUCENE-9579 Update to JUnit 4.13.1 (#1982) > Update JUnit to latest 4.x release > -- > > Key: LUCENE-9579 > URL: https://issues.apache.org/jira/browse/LUCENE-9579 > Project: Lucene - Core > Issue Type: Test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ctargett commented on a change in pull request #1870: SOLR-14865: 'Index Merge Metrics' documentation correction
ctargett commented on a change in pull request #1870: URL: https://github.com/apache/lucene-solr/pull/1870#discussion_r504903798 ## File path: solr/solr-ref-guide/src/metrics-reporting.adoc ## @@ -534,15 +534,34 @@ These metrics are available only on a per-core basis. Metrics can be aggregated These metrics are collected in respective registries for each core (e.g., `solr.core.collection1`), under the `INDEX` category. -Basic metrics are always collected - collection of additional metrics can be turned on using boolean parameters in the `/config/indexConfig/metrics` section of `solrconfig.xml`: +Metrics collection is controlled by boolean parameters in the `/config/indexConfig/metrics` section of `solrconfig.xml`: + Review comment: I was a little confused by the fully qualified XML path here, we don't do that in other places in the docs so it jumped out at me. If we think maybe we should do that more often, this would be fine and I can make an issue to update other places in the Guide to someday be consistent but otherwise I only wonder if it would be jarring for others also (I might be alone in my impression). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9579) Update JUnit to latest 4.x release
[ https://issues.apache.org/jira/browse/LUCENE-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob resolved LUCENE-9579. --- Fix Version/s: 8.7 master (9.0) Resolution: Fixed > Update JUnit to latest 4.x release > -- > > Key: LUCENE-9579 > URL: https://issues.apache.org/jira/browse/LUCENE-9579 > Project: Lucene - Core > Issue Type: Test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: master (9.0), 8.7 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on pull request #1949: Jinja2_autoescape_false set to True
madrob commented on pull request #1949: URL: https://github.com/apache/lucene-solr/pull/1949#issuecomment-708600777 Thanks for bringing this to our attention, John! This is for preventing XSS if there is some nasty string in... a GitHub PR title? Trying to make sure I understand the scope and impact. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9574) Add a token filter to drop tokens based on flags.
[ https://issues.apache.org/jira/browse/LUCENE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214201#comment-17214201 ] Gus Heck commented on LUCENE-9574: -- Actually, I had expected when I started this, that 8.7 branch might have been cut already by the time I committed, and certainly the rest of the AQP changes won't make 8.7. Do we want to include it in 8.7 even so? > Add a token filter to drop tokens based on flags. > - > > Key: LUCENE-9574 > URL: https://issues.apache.org/jira/browse/LUCENE-9574 > Project: Lucene - Core > Issue Type: New Feature > Components: modules/analysis >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Time Spent: 8h 10m > Remaining Estimate: 0h > > (Breaking this off of SOLR-14597 for independent review) > A filter that tests flags on tokens vs a bitmask and drops tokens that have > all specified flags. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-10321) Unified highlighter returns empty fields when using glob
[
https://issues.apache.org/jira/browse/SOLR-10321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214207#comment-17214207
]
Mitchell Kotler commented on SOLR-10321:
I just ran into this issue and it is not just a nuisance for me, but a complete
non-starter. I store text in one field per page (page_no_1, page_no_2, etc,
set hl.fl to page_no_*). This allows me to re-use the same index to find which
page a query appears on as well as searching across documents. This works fine
with the original highlighter, until we had a very large document which the
original highlighter was much too slow for. The unified highlighter is much
quicker to highlight snippets (with the proper indexes set up), but it will
return a field for the highest page in the index, not in the document. So if I
have one document with 60,000 pages, the unified highlighter will return up to
60,000 empty arrays per document, which makes the response multiple orders of
magnitude larger than it needs to be. Fixing this bug or having a work around
would be very helpful for us. I am not sure if using the fastvector
highlighter in the mean time would be our best bet.
> Unified highlighter returns empty fields when using glob
>
>
> Key: SOLR-10321
> URL: https://issues.apache.org/jira/browse/SOLR-10321
> Project: Solr
> Issue Type: Bug
> Components: highlighter
>Affects Versions: 6.4.2
>Reporter: Markus Jelsma
>Priority: Minor
> Fix For: 7.0
>
>
> {code}
> q=lama&hl.method=unified&hl.fl=content_*
> {code}
> returns:
> {code}
>name="http://www.nu.nl/weekend/3771311/dalai-lama-inspireert-westen.html";>
>
>
> Nobelprijs Voorafgaand aan zijn bezoek aan Nederland is de dalai
> lama in Noorwegen om te vieren dat 25 jaar geleden de
> Nobelprijs voor de Vrede aan hem werd toegekend. Anders dan in Nederland
> wordt de dalai lama niet ontvangen in het Noorse
> parlement.
>
>
>
>
>
>
>
>
>
>
>
> {code}
> FastVector and original do not emit:
> {code}
>
>
>
>
>
>
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] wojtek2kdev opened a new pull request #1983: [LUCENE-9410] German/French stemmers fail for common forms maux, gegrüßt, grüßend, schlummert
wojtek2kdev opened a new pull request #1983: URL: https://github.com/apache/lucene-solr/pull/1983 https://issues.apache.org/jira/browse/LUCENE-9410 I don't know if it's just an acceptable exception or something serious? I wrote test cases for these words. Should them be mapped by hand to simple forms? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9576) IndexWriter::commit() hangs when the server has a stale NFS mount.
[ https://issues.apache.org/jira/browse/LUCENE-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214219#comment-17214219 ] Robert Muir commented on LUCENE-9576: - I propose to remove this janky SSD detection logic, and instead simply default as if haveSpinningDisks = false. It is 2020! And it would remove a bunch of hairy code that will never be correct in all cases and may even cause trouble. If a user has spinning disks, it is true it may cause lower performance, requiring them to set the boolean themselves. But it should not be a surprise... and I think it wouldn't be unexpected either, to notice that indexing is slow on a spinning disk. cc [~mikemccand][~uschindler] > IndexWriter::commit() hangs when the server has a stale NFS mount. > -- > > Key: LUCENE-9576 > URL: https://issues.apache.org/jira/browse/LUCENE-9576 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Affects Versions: 8.5.2 >Reporter: Girish Nayak >Priority: Major > Attachments: IndexWriter-commit.PNG > > > Noticed IndexWriter::commit() hangs when the server has one or more stale NFS > mounts. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9576) IndexWriter::commit() hangs when the server has a stale NFS mount.
[ https://issues.apache.org/jira/browse/LUCENE-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214222#comment-17214222 ] Robert Muir commented on LUCENE-9576: - Also I'm willing to guess, simply wiring the default will *improve* index performance for all the folks where detection is wrong or unsupported today. Probably more people with fast disks and bad auto-detection than there are people really trying to tweak out max performance on a spinning disk. > IndexWriter::commit() hangs when the server has a stale NFS mount. > -- > > Key: LUCENE-9576 > URL: https://issues.apache.org/jira/browse/LUCENE-9576 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Affects Versions: 8.5.2 >Reporter: Girish Nayak >Priority: Major > Attachments: IndexWriter-commit.PNG > > > Noticed IndexWriter::commit() hangs when the server has one or more stale NFS > mounts. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jtibshirani commented on a change in pull request #1948: LUCENE-9536: Optimize OrdinalMap when one segment contains all distinct values.
jtibshirani commented on a change in pull request #1948:
URL: https://github.com/apache/lucene-solr/pull/1948#discussion_r504927238
##
File path: lucene/core/src/java/org/apache/lucene/index/OrdinalMap.java
##
@@ -271,13 +273,26 @@ protected boolean lessThan(TermsEnumIndex a,
TermsEnumIndex b) {
globalOrd++;
}
-this.firstSegments = firstSegments.build();
-this.globalOrdDeltas = globalOrdDeltas.build();
+long ramBytesUsed = BASE_RAM_BYTES_USED + segmentMap.ramBytesUsed();
+this.valueCount = globalOrd;
+
+// If the first segment contains all of the global ords, then we can apply
a small optimization
+// and hardcode the first segments and global ord deltas as all zeroes.
+if (ordDeltaBits.length > 0 && ordDeltaBits[0] == 0L &&
ordDeltas[0].size() == this.valueCount) {
+ this.firstSegments = LongValues.ZEROES;
+ this.globalOrdDeltas = LongValues.ZEROES;
+ ramBytesUsed += RamUsageEstimator.shallowSizeOf(LongValues.ZEROES);
Review comment:
I added this to address a failure in `TestOrdinalMap`. But now I see it
makes more sense to modify the test !
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9576) IndexWriter::commit() hangs when the server has a stale NFS mount.
[ https://issues.apache.org/jira/browse/LUCENE-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214229#comment-17214229 ] Uwe Schindler commented on LUCENE-9576: --- I can confirm that any type of stale file mount causes issues: - Elasticsearch won't start as it does the spinning disk check on startup - the first Lucene commit hangs. The bad thing is: it can be caused by any mount, also ones not actually used by indexing at all. And that's very bad. I recognized this problem already on PANGAEA where we have some Tape-backed mounts with huge amounts of data. This affects Lucene, which was completed unrelated (e.g. when tape roboter changing tapes in background, those mounts may hang). > IndexWriter::commit() hangs when the server has a stale NFS mount. > -- > > Key: LUCENE-9576 > URL: https://issues.apache.org/jira/browse/LUCENE-9576 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Affects Versions: 8.5.2 >Reporter: Girish Nayak >Priority: Major > Attachments: IndexWriter-commit.PNG > > > Noticed IndexWriter::commit() hangs when the server has one or more stale NFS > mounts. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9576) IndexWriter::commit() hangs when the server has a stale NFS mount.
[ https://issues.apache.org/jira/browse/LUCENE-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214231#comment-17214231 ] Uwe Schindler commented on LUCENE-9576: --- We also had a question on Java-User about this today: https://lists.apache.org/x/thread.html/reecbc4ba44f3e5625612482d3a0bf93a1e6e795de95b9ddf81331e6c@%3Cjava-user.lucene.apache.org%3E > IndexWriter::commit() hangs when the server has a stale NFS mount. > -- > > Key: LUCENE-9576 > URL: https://issues.apache.org/jira/browse/LUCENE-9576 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Affects Versions: 8.5.2 >Reporter: Girish Nayak >Priority: Major > Attachments: IndexWriter-commit.PNG > > > Noticed IndexWriter::commit() hangs when the server has one or more stale NFS > mounts. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9576) IndexWriter::commit() hangs when the server has a stale NFS mount.
[ https://issues.apache.org/jira/browse/LUCENE-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214233#comment-17214233 ] Uwe Schindler commented on LUCENE-9576: --- The detection is also often wrong: on policeman jenkins are two NVMe disks, combined to raid1 using devidemapper. The virtual raid device emulated by Linux device mapper returns rotational=true. So just adding raid harms performance. > IndexWriter::commit() hangs when the server has a stale NFS mount. > -- > > Key: LUCENE-9576 > URL: https://issues.apache.org/jira/browse/LUCENE-9576 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Affects Versions: 8.5.2 >Reporter: Girish Nayak >Priority: Major > Attachments: IndexWriter-commit.PNG > > > Noticed IndexWriter::commit() hangs when the server has one or more stale NFS > mounts. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9576) IndexWriter::commit() hangs when the server has a stale NFS mount.
[ https://issues.apache.org/jira/browse/LUCENE-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214234#comment-17214234 ] Uwe Schindler commented on LUCENE-9576: --- In addition, virtual disks by vmware or kvm often return rotational=true, although they are backed by completely different hardware. On windows we always assume rotational, which is also bad. > IndexWriter::commit() hangs when the server has a stale NFS mount. > -- > > Key: LUCENE-9576 > URL: https://issues.apache.org/jira/browse/LUCENE-9576 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Affects Versions: 8.5.2 >Reporter: Girish Nayak >Priority: Major > Attachments: IndexWriter-commit.PNG > > > Noticed IndexWriter::commit() hangs when the server has one or more stale NFS > mounts. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9576) IndexWriter::commit() hangs when the server has a stale NFS mount.
[
https://issues.apache.org/jira/browse/LUCENE-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214238#comment-17214238
]
Chris Marzullo commented on LUCENE-9576:
{quote}The bad thing is: it can be caused by any mount, also ones not actually
used by indexing at all.
{quote}
This is what we saw. We were writing to a local xfs ssd filesystem. There was
stale NFS mount. IMO stale mounts are a common bugbear, but can't really
comment on what is best for lucene.
We were able to doge the issue with Robert's suggestion.
> IndexWriter::commit() hangs when the server has a stale NFS mount.
> --
>
> Key: LUCENE-9576
> URL: https://issues.apache.org/jira/browse/LUCENE-9576
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 8.5.2
>Reporter: Girish Nayak
>Priority: Major
> Attachments: IndexWriter-commit.PNG
>
>
> Noticed IndexWriter::commit() hangs when the server has one or more stale NFS
> mounts.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12182) Can not switch urlScheme in 7x if there are any cores in the cluster
[ https://issues.apache.org/jira/browse/SOLR-12182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214273#comment-17214273 ] Timothy Potter commented on SOLR-12182: --- I think with Solr 9, we shouldn't store the scheme in state.json and instead resolve it at runtime based on what is stored in ZK. I'm going to take that approach for master vs. a migration tool. Also going to look into using the scheme that's currently active for a node vs. cluster-wide property, not sure how far I can get with that but will investigate as part of this work. > Can not switch urlScheme in 7x if there are any cores in the cluster > > > Key: SOLR-12182 > URL: https://issues.apache.org/jira/browse/SOLR-12182 > Project: Solr > Issue Type: Bug >Affects Versions: 7.0, 7.1, 7.2 >Reporter: Anshum Gupta >Priority: Major > Attachments: SOLR-12182.patch, SOLR-12182_20200423.patch > > > I was trying to enable TLS on a cluster that was already in use i.e. had > existing collections and ended up with down cores, that wouldn't come up and > the following core init errors in the logs: > *org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: > replica with coreNodeName core_node4 exists but with a different name or > base_url.* > What is happening here is that the core/replica is defined in the > clusterstate with the urlScheme as part of it's base URL e.g. > *"base_url":"http:hostname:port/solr"*. > Switching the urlScheme in Solr breaks this convention as the host now uses > HTTPS instead. > Actually, I ran into this with an older version because I was running with > *legacyCloud=false* and then realized that we switched that to the default > behavior only in 7x i.e while most users did not hit this issue with older > versions, unless they overrode the legacyCloud value explicitly, users > running 7x are bound to run into this more often. > Switching the value of legacyCloud to true, bouncing the cluster so that the > clusterstate gets flushed, and then setting it back to false is a workaround > but a bit risky one if you don't know if you have any old cores lying around. > Ideally, I think we shouldn't prepend the urlScheme to the base_url value and > use the urlScheme on the fly to construct it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
