It's notable that Solr-core in main already includes OpenTelemetry dependencies. I expect this will be increasingly common, like SLF4J has become.
Docker: definitely a separate topic altogether. See https://issues.apache.org/jira/browse/SOLR-17653 which I'm likely to pursue. I'm very familiar with running Solr in Docker in integration tests. On Fri, Apr 18, 2025 at 10:47 AM Gus Heck <gus.h...@gmail.com> wrote: > This sounds good to me because Dropwizard metrics is used by the Dropwizard > framework. Recent experiences where Dropwizard is used as an intermediary > layer and integration tests using our test framework have shown it's a path > to jar hell with Solr wanting one version and the Dropwizard based code > wanting another... What I don't know is the extent to which OTel might be > adopted by other similar frameworks (what does spring boot usually work > with?) The issue of us having several hundred dependencies and folks using > our test framework is always going to have some clash but maybe this will > help? > > Side topic: I'm aware of at least two places that have come to the > conclusion that integrated tests should spin up a separate JVM for solr, > and part of me wonders if there's a way to push our test framework to do > that natively so that folks using solr don't need to do a lot of > engineering to create integration tests. This might also benefit our tests > by having a separate log for the server and the client side of > interactions. Currently, you can't just scroll through and look for the > stack trace trivially because often you have one reported from the server > and one reported from the test itself. We might even be able to set a "no > stack trace in the logs" general standard for the server side of things > this way and fail any tests that generate a stack trace to ensure that a > healthy solr never logs stack traces. Such a thing might involve having a > testing module (not loaded in production) that allows/simplifies > configuration without security hurdles... > > The obvious question of course is speed.... But maybe a solidification of > what's an integration test and what's a unit test helps there, and > integration tests are intended for build infra, or final checks rather than > developer testing during development. > > -Gus > > On Thu, Apr 17, 2025 at 11:45 PM David Smiley <dsmi...@apache.org> wrote: > > > That's a nice start! I'd like to work with you on making the SIP better. > > Maybe a 1:1 (others invited too!) video might be most productive. I'd > love > > to involve AB more here. I think the SIP template that you clearly > started > > from isn't great; I'm tempted to replace it with a list of considerations > > and not a proposed format. People should structure their SIP however the > > topic best presents itself. > > > > On Thu, Apr 17, 2025 at 2:34 PM Matthew Biscocho (BLOOMBERG/ 919 3RD A) < > > mbisco...@bloomberg.net> wrote: > > > > > Based on the discussion from yesterdays meetup, created SIP-23 for > moving > > > to Open Telemetry > > > > > > https://cwiki.apache.org/confluence/display/SOLR/SIP-23%3A+Switch+from+DropWizard+to+OpenTelemetry > > > > > > With Solr 10 around the corner, hoping this work can get done before > > then. > > > > > > From: dev@solr.apache.org At: 04/14/25 01:04:13 UTC-4:00To: > > > dev@solr.apache.org > > > Subject: Re: SIP proposal: Switch from DropWizard to OpenTelemetry > > > > > > I'm excited about the change! Where I work, it would *significantly* > > > simplify our metrics pipeline if Solr were to embrace OTel for metrics, > > as > > > we could then use company-provided OTel plugins. The broad industry > > > adoption of OTel points to this being the least friction. With > > DropWizard, > > > it appears we hacked attributes onto it, in a sense. > > > > > > The primary criteria/requirement that comes to mind is to have > > > strong/sophisticated ways to filter the right metrics to publish. Solr > > has > > > that today. > > > > > > On Fri, Apr 11, 2025 at 5:00 PM Matthew Biscocho (BLOOMBERG/ 919 3RD > A) < > > > mbisco...@bloomberg.net> wrote: > > > > > > > Hey everyone, > > > > > > > > Making this thread because I was interested in writing up a SIP for > > > > SOLR-17458 https://issues.apache.org/jira/browse/SOLR-17458 and > > starting > > > > a discussion around this. The proposed change involves migrating > Solr's > > > > metrics framework from DropWizard to OpenTelemetry (OTel). This will > > move > > > > Solr to an attribute based metric framework and will also help > > > > future-proofing by keeping it from vendor lock-in, given > > OpenTelemetry's > > > > stance on being vendor-neutral and working with many different > > pipelines. > > > > Since Solr already packages an Open Telemetry SDK as a module for > > > exporting > > > > spans through OTLP, we can use that foundation to have it push > metrics > > as > > > > well. There will be a significant change here which will probably > break > > > > many components and no longer be backwards compatible so this will > > > probably > > > > need to be a major release version change but we can potentially > keep a > > > few > > > > things backwards compatible if needed. Just some things off the top > of > > my > > > > head: > > > > > > > > * The Prometheus Exporter (Maybe should be deprecated?) > > > > * GET /admin/metrics endpoint > > > > * JVM metrics collected from OTel > > > > * Metric reporters? > > > > > > > > Anyone have thoughts? > > > > > > > > > > > > > > -- > http://www.needhamsoftware.com (work) > https://a.co/d/b2sZLD9 (my fantasy fiction book) >
