Hi Houston, Can you please elaborate on the purpose: > and moving it into Solr would allow others to use and collaborate on it easier.
How is that? I am guessing another motivation may be visibility / awareness. If that is a motivation, I think that can be addressed with prominent references in the Solr Reference Guide and website news updates. ~ David On Thu, Sep 7, 2023 at 12:33 PM Houston Putman <hous...@apache.org> wrote: > Hey everyone, > > I think it's a good time to talk about the future of the cross DC work > that's been going on in the Solr Sandbox repo. > > Mark (and others) have been doing amazing work there, and I think it's > close to a state that we can bring it out of the sandbox and into Solr > itself. It's been running successfully in Apple, and moving it into Solr > would allow others to use and collaborate on it easier. > > There are two parts to this cross DC tool. > > - A cross-dc-producer > <https://github.com/apache/solr-sandbox/tree/main/crossdc-producer>, > which is a Solr plugin that has a Update-request-processor that adds > documents to a Kafka Queue to be indexed asynchronously. > - A cross-dc-consumer > <https://github.com/apache/solr-sandbox/tree/main/crossdc-consumer>, > which is a separate application that handles consuming documents from > Kafka > and sending it to Solr to be indexed. > > In my mind, I envision when we move the cross-dc-consumer into the Solr > codebase, it would live much like the Prometheus Exporter. Where it's > released as a part of the same binary package, but not in the slim > distribution. > > As for the cross-dc-producer, that can live as a Solr module, much like the > other modules that Solr ships with. Once again, this would not be included > with the Slim distribution. > > This ties the cross-dc packages to specific Solr versions, much like other > Modules or SolrJ. I think this is an acceptable thing, but others might > want it to be more version agnostic or have a separate release schedule > from Solr. > > Overall this thread is to just get the conversation started so that we can > throw all the ideas out there before we make a decision on one. > > - Houston >
