So far, this project has been an experimental project, mainly something that's being used and developed by Apple committers. I would be hesitant to support an official Apache release for the same without testing or interest by the broader community.
Towards that, can we invite community members to try it out from the sandbox repo itself, and graduate it to a main repository based on feedback or interest? On Sat, 9 Sept, 2023, 4:48 am Mark Miller, <markrmil...@gmail.com> wrote: > I think the main motivation would be cost savings. > > The main thing I like about keeping it separate is the ability to have an > independent release cycle. I initially preferred a separation due to that. > > But the cost for what it actually is, is high. > > It essentially consists of two fairly simple parts. An update processor > that puts updates on a Kafka queue (the queue system could be pluggable > given sufficient need, but now its Kafka) and a simple application that > polls the Kafka queue and sends updates to Solr via SolrJ. > > As a separate component, it needs its own little build system. It needs its > own CI setup. It needs its own documentation setup. It's own independent > release testing and voting and announcements, etc. And it will miss out on > various helpers, checkers, smoke testers, etc from the Solr build. > > It's a pretty heavy ongoing burden for something that could reasonably be > packed into two classes (it's not two classes, because it's much nicer not > to be). > > On Fri, Sep 8, 2023 at 5:57 PM Houston Putman <hous...@apache.org> wrote: > > > I always assumed after the CDCR stuff was removed from Solr, the idea was > > to provide a better first-party solution one day. > > To me, first party means not "experimental" or "sandbox", so it makes > sense > > to live in the main Solr repo. > > > > In general, I agree with Eric, nothing should "live" in the solr-sandbox > > repo forever. It should be developed there, and either be promoted or > > abandoned. > > The same goes for the file encryption stuff that lives there. Once it > > reaches a certain level of stability, it should be moved in. > > > > I think as a module, it's much easier to develop as a part of Solr > instead > > of separately. This was certainly the case for the Docker image. > > > > - Houston > > > > On Fri, Sep 8, 2023 at 6:04 PM Eric Pugh > > <ep...@opensourceconnections.com.invalid> wrote: > > > > > My perspective of Solr-sandbox is that it’s an area for ideas to be > > worked > > > on, but no real promises…. They might be abandoned at any moment, or > > have > > > issues.. No real expectation of docs or any kind of support. It’s > > meant > > > for solr committers to collaborate with other solr committers on new > > things… > > > > > > So if we are going to ref guide it etc, seems like it should live in > the > > > main Solr repo. Or, be promoted to some repo that doesn’t have > > “sandbox” > > > in the name, like the Solr-operator project…. > > > > > > > > > > > > > On Sep 8, 2023, at 4:50 PM, David Smiley <david.w.smi...@gmail.com> > > > wrote: > > > > > > > > Hi Houston, > > > > > > > > Can you please elaborate on the purpose: > > > >> and moving it into Solr would allow others to use and collaborate on > > it > > > > easier. > > > > > > > > How is that? > > > > > > > > I am guessing another motivation may be visibility / awareness. If > > that > > > is > > > > a motivation, I think that can be addressed with prominent references > > in > > > > the Solr Reference Guide and website news updates. > > > > > > > > ~ David > > > > > > > > > > > > On Thu, Sep 7, 2023 at 12:33 PM Houston Putman <hous...@apache.org> > > > wrote: > > > > > > > >> Hey everyone, > > > >> > > > >> I think it's a good time to talk about the future of the cross DC > work > > > >> that's been going on in the Solr Sandbox repo. > > > >> > > > >> Mark (and others) have been doing amazing work there, and I think > it's > > > >> close to a state that we can bring it out of the sandbox and into > Solr > > > >> itself. It's been running successfully in Apple, and moving it into > > Solr > > > >> would allow others to use and collaborate on it easier. > > > >> > > > >> There are two parts to this cross DC tool. > > > >> > > > >> - A cross-dc-producer > > > >> < > https://github.com/apache/solr-sandbox/tree/main/crossdc-producer > > >, > > > >> which is a Solr plugin that has a Update-request-processor that > adds > > > >> documents to a Kafka Queue to be indexed asynchronously. > > > >> - A cross-dc-consumer > > > >> < > https://github.com/apache/solr-sandbox/tree/main/crossdc-consumer > > >, > > > >> which is a separate application that handles consuming documents > > from > > > >> Kafka > > > >> and sending it to Solr to be indexed. > > > >> > > > >> In my mind, I envision when we move the cross-dc-consumer into the > > Solr > > > >> codebase, it would live much like the Prometheus Exporter. Where > it's > > > >> released as a part of the same binary package, but not in the slim > > > >> distribution. > > > >> > > > >> As for the cross-dc-producer, that can live as a Solr module, much > > like > > > the > > > >> other modules that Solr ships with. Once again, this would not be > > > included > > > >> with the Slim distribution. > > > >> > > > >> This ties the cross-dc packages to specific Solr versions, much like > > > other > > > >> Modules or SolrJ. I think this is an acceptable thing, but others > > might > > > >> want it to be more version agnostic or have a separate release > > schedule > > > >> from Solr. > > > >> > > > >> Overall this thread is to just get the conversation started so that > we > > > can > > > >> throw all the ideas out there before we make a decision on one. > > > >> > > > >> - Houston > > > >> > > > > > > _______________________ > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 > | > > > http://www.opensourceconnections.com < > > > http://www.opensourceconnections.com/> | My Free/Busy < > > > http://tinyurl.com/eric-cal> > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > > > > > > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > > > > > > > > > This e-mail and all contents, including attachments, is considered to > be > > > Company Confidential unless explicitly stated otherwise, regardless of > > > whether attachments are marked as such. > > > > > > > > >
