Re: Dynamic schema design: feedback requested

On Mon, Mar 11, 2013 at 5:51 PM, Chris Hostetter wrote: > : I guess my main point is, we shouldn't decide a priori that using the > : API means you can no longer hand edit. > > and my point is we should build a feature where solr has the ability to > read/write some piece of information, we should

Re: Dynamic schema design: feedback requested

: we needed to, we could just assert that the schema file is the : persistence mechanism, as opposed to the system of record, hence if : you hand edit it and then use the API to change it, your hand edit may : be lost. Or we may decide to do away with "local FS" mode altogether. presuming that i

Re: Dynamic schema design: feedback requested

On Mon, Mar 11, 2013 at 2:50 PM, Chris Hostetter wrote: > > : > 2) If you wish to use the /schema REST API for read and write operations, > : > then schema information will be persisted under the covers in a data store > : > whose format is an implementation detail just like the index file format.

Re: Dynamic schema design: feedback requested

To revisit sarowes comment about how/when to decide if we are using the "config file" version of schema info (and hte API is read only) vs "internal managed state data" version of schema info (and the API is read/write)... On Wed, 6 Mar 2013, Steve Rowe wrote: : Two possible approaches: : :

Re: Dynamic schema design: feedback requested

: > 2) If you wish to use the /schema REST API for read and write operations, : > then schema information will be persisted under the covers in a data store : > whose format is an implementation detail just like the index file format. : : This really needs to be driven by costs and benefits... :

Re: Dynamic schema design: feedback requested

On Wed, Mar 6, 2013 at 7:50 PM, Chris Hostetter wrote: > 2) If you wish to use the /schema REST API for read and write operations, > then schema information will be persisted under the covers in a data store > whose format is an implementation detail just like the index file format. This really n

Re: Dynamic schema design: feedback requested

On Mar 8, 2013, at 2:57 PM, Steve Rowe wrote: >> multiple collections may share the same config set and thus schema, so what >> happens if someone does not know this and hits PUT >> localhost:8983/solr/collection1/schema and it affects also the schema for >> collection2? > > Hmm, that's a grea

Re: Dynamic schema design: feedback requested

On Mar 6, 2013, at 7:50 PM, Chris Hostetter wrote: > I think it would make a lot of sense -- not just in terms of > implementation but also for end user clarity -- to have some simple, > straightforward to understand caveats about maintaining schema > information... > > 1) If you want to keep

Re: Dynamic schema design: feedback requested

Hi Jan, On Mar 6, 2013, at 4:50 PM, Jan Høydahl wrote: > Will ZK get pushed the serialized monolithic schema.xml / schema.json from > the node which changed it, and then trigger an update to the rest of the > cluster? Yes. > I was kind of hoping that once we have introduced ZK into the mix as

Re: Dynamic schema design: feedback requested

On Mar 6, 2013, at 4:50 PM, Chris Hostetter wrote: > i don't think it's > unreasable to say "if you would like to manipulate the schema using an > API, then you give up the ability to manipulate it as a config file on > disk" As long as you can initially work with an easily editable file, I

Re: Dynamic schema design: feedback requested

: As far as a user editing the file AND rest API access, I think that : seems fine. Yes, the user is in trouble if they break the file, but that Ignoring for a moment what format is used to persist schema information, I think it's important to have a conceptual distinction between "data" that

Re: Dynamic schema design: feedback requested

How will this all work with ZooKeeper and cloud? Will ZK get pushed the serialized monolithic schema.xml / schema.json from the node which changed it, and then trigger an update to the rest of the cluster? I was kind of hoping that once we have introduced ZK into the mix as our centralized conf

Re: Dynamic schema design: feedback requested

On Mar 6, 2013, at 3:33 PM, Mark Miller wrote: > On Mar 6, 2013, at 12:08 PM, Steve Rowe wrote: >> I'm not sure what pieces you might be missing, sorry. > > My main confusion is around this: > > bq. When schema.xml is present, schema.json (if any) will be ignored. > > Basically, why have schem

Re: Dynamic schema design: feedback requested

On Mar 6, 2013, at 12:08 PM, Steve Rowe wrote: > I'm not sure what pieces you might be missing, sorry. My main confusion is around this: bq. When schema.xml is present, schema.json (if any) will be ignored. Basically, why have schema.json? Perhaps it's just me, but a json schema seems like a

Re: Dynamic schema design: feedback requested

I'm not sure what pieces you might be missing, sorry. I had thought about adding a web UI for schema composition, but that would be a major effort, and not in scope here. I agree, though, especially without a full schema modification REST API, that hand editing will have to be supported. Steve

Re: Dynamic schema design: feedback requested

Hmm…I think I'm missing some pieces. I agree with Erick that you should be able to load a schema from any object - a DB, a file in ZooKeeper, you name it. But it seems by default, having that object be schema.xml seems nicest to me. That doesn't mean you have to use DOM or XML internally - just

Re: Dynamic schema design: feedback requested

In response to my thoughts about using DOM as an intermediate representation for schema elements, for use in lazy re-loading on schema change, Erik Hatcher argued against (solely) using XML for schema serialization (

Re: Dynamic schema design: feedback requested

bq. Change Solr schema serialization from XML to JSON, and provide an XML->JSON conversion tool. What is the motivation for the change? I think if you are sitting down and looking to design a schema, working with the XML is fairly nice and fast. I picture that a lot of people would start by wor

Dynamic schema design: feedback requested

I'm working on SOLR-3251 , to dynamically add fields to the Solr schema. I posted a rough outline of how I propose to do this: