[
https://issues.apache.org/jira/browse/SOLR-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195048#comment-17195048
]
Alexandre Rafalovitch commented on SOLR-14701:
----------------------------------------------
Eric, these are great thoughts and I look forward to others commenting on
those. This issue did start as a higher-level discussion actually, so we can
still adjust/redo/drop the code part.
My thoughts specifically on what you are saying are:
* The crux of the issue. I think the real issue is that we chased somebody
else's approach because "mindshare" and then paid a similar price they did on
technical level. But now we cannot just rip out what we have because it is kind
of half-interesting. So, we are cleaning it up and side-lining it into learning
schema. Where it has a chance to be actually useful. So, that's the context of
why I feel it was worth cleaning it up. A use case I see is somebody throwing
their data into it, and being able to do some faceting/analytics/sorting on it
to understand what they actually have.
* ??Anything is a multivalued string??.
You can do that already with {{<dynamicField name="*"...>}}. I did [a
presentation around
that|https://www.slideshare.net/arafalov/rapid-solr-schema-development-phone-directory]
a while ago . By doing this, you don't need this guessing chain and it would
actually not do anything because it skips any existing field or dynamicField.
But it could stop skipping dynamic fields and create specific fields while
still leaving the fallback. Or it could have a section of which dynamic fields
to override, if you still want something_i to map directly but any others to be
type-guessed. I think this could be an interesting approach even for production
schema.
* Additionally, the current implementation accumulates data for all the fields
and only skips them in commit. Theoretically, it could do comparison of types
and issue some advice like "currently this field matches a multiValued String
glob. But it could actually be a single-valued date, as long as you keep the
date-parsing URP in default chain. It would be easy enough to spit that into a
log under Warning/Info level.
* Modify across multiple commits. We have a general issue with modifying field
definition. Because the lucene-level data structures cannot go from
single-valued to multi-valued and hand-editing definitions can throw scary
exception. And because the deleted values are still in the index until it is
empty/segment is purged. There had been a discussion about "rewriting segment
filter" that does some of that cleanup/refactoring, but I am not sure if that
happened on lucene level yet. There are two ways this solution is a bit better
than existing one however. First is that it does not create definitions until
commit, so you can feed it all your different spaCy outputs and then do a
commit on joint definitions. Of course, then you have to rerun the queries to
get the data in, but that's the price. The second is the corollary of that
awkwardness; after the schema is created, the data is not in yet, so you can go
and review the configurations and adjust them as desired.
> Deprecate Schemaless Mode (Discussion)
> --------------------------------------
>
> Key: SOLR-14701
> URL: https://issues.apache.org/jira/browse/SOLR-14701
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Reporter: Marcus Eagan
> Assignee: Alexandre Rafalovitch
> Priority: Major
> Attachments: image-2020-08-04-01-35-03-075.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I know this won't be the most popular ticket out there, but I am growing more
> and more sympathetic to the idea that we should rip many of the freedoms out
> that cause users more harm than not. One of the freedoms I saw time and time
> again to cause issues was schemaless mode. It doesn't work as named or
> documented, so I think it should be deprecated.
> If you use it in production reliably and in a way that cannot be accomplished
> another way, I am happy to hear from more knowledgeable folks as to why
> deprecation is a bad idea.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
