[
https://issues.apache.org/jira/browse/SOLR-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022523#comment-17022523
]
Andy Vuong commented on SOLR-14213:
-----------------------------------
I’m thinking of adding a config option in solrconfig.xml such as
<sharedStorage>…</sharedStorage>
when present indicates the client’s intention to use the shared storage
feature. On solr startup we’ll then check if the property is present before
initializing anything related to shared storage. This removes the need to lazy
initialize anything as we currently do and we’ll need to add a configuration
section down the line anyway for a few currently hard coded tunable params.
Initially the sharedStorage options might look like:
{code:java}
<sharedStorage>
<lst name=“clientConfiguration”>
<str name=“provider”>s3</str>
<str name=“bucketName”></str>
<str name=“region”></str>
<str name=“credentialsPath”></str>
</lst>
</sharedStorage>
{code}
The initialization flow might go such as -> user configures solrconfig.xml with
the above configuration. Solr starts up and the presence of the configuration
acts as a feature flag, initializing the shared storage components. If a
storage client is configured, does an initial request to the provider to see if
the bucket exists and credentials work (what we currently do on very first
shared storage access by default today).
Other configurations can later be added under the same section.
A few limitations/challenges:
* Configuration is in solrconfig.xml but is a cluster level configuration (one
bucket for the whole cluster and no plans to allow specifying multiple buckets
if we wanted too)
* Config API allows updating solrconfig and it’s not clear how much work
should be done to support that api in context of this config or if we even
should (for a later issue?)
* We'll need a "shared storage enabled" cluster-level flag (we're currently
gating the feature via the sharedIndex flag at a collection level) that gets
set when this config is read. This adds another layer preventing collections
being created with sharedIndex=true (see command in desc)
> Configuring Solr Cloud to use Shared Storage
> --------------------------------------------
>
> Key: SOLR-14213
> URL: https://issues.apache.org/jira/browse/SOLR-14213
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
> Reporter: Andy Vuong
> Priority: Minor
>
> Clients can currently create shared collections by sending a collection
> admin command such as
> *_solr/admin/collections?action=CREATE&name=gettingstarted&sharedIndex=true&numShards=1_*
>
> There are a set of shared storage specific classes such as
> SharedStorageManager that get initialized on startup when the CoreContainer
> loads. There are also components that are lazily loaded when shared storage
> functionality is needed. This was initially written this way because a Solr
> Cloud cluster could spin up and not used shared collections in which case
> shared store components wouldn’t need to be loaded. There is also no support
> for configuring Solr Cloud to use shared storage via config files. Lazy
> loading leads to some poor code and initialization flow that should be
> revisited.
> This JIRA is for designing the configuration of Solr Cloud to use shared
> storage and initializing shared storage components based on this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
