This sounds good to me because Dropwizard metrics is used by the Dropwizard framework. Recent experiences where Dropwizard is used as an intermediary layer and integration tests using our test framework have shown it's a path to jar hell with Solr wanting one version and the Dropwizard based code wanting another... What I don't know is the extent to which OTel might be adopted by other similar frameworks (what does spring boot usually work with?) The issue of us having several hundred dependencies and folks using our test framework is always going to have some clash but maybe this will help?
Side topic: I'm aware of at least two places that have come to the conclusion that integrated tests should spin up a separate JVM for solr, and part of me wonders if there's a way to push our test framework to do that natively so that folks using solr don't need to do a lot of engineering to create integration tests. This might also benefit our tests by having a separate log for the server and the client side of interactions. Currently, you can't just scroll through and look for the stack trace trivially because often you have one reported from the server and one reported from the test itself. We might even be able to set a "no stack trace in the logs" general standard for the server side of things this way and fail any tests that generate a stack trace to ensure that a healthy solr never logs stack traces. Such a thing might involve having a testing module (not loaded in production) that allows/simplifies configuration without security hurdles... The obvious question of course is speed.... But maybe a solidification of what's an integration test and what's a unit test helps there, and integration tests are intended for build infra, or final checks rather than developer testing during development. -Gus On Thu, Apr 17, 2025 at 11:45 PM David Smiley <dsmi...@apache.org> wrote: > That's a nice start! I'd like to work with you on making the SIP better. > Maybe a 1:1 (others invited too!) video might be most productive. I'd love > to involve AB more here. I think the SIP template that you clearly started > from isn't great; I'm tempted to replace it with a list of considerations > and not a proposed format. People should structure their SIP however the > topic best presents itself. > > On Thu, Apr 17, 2025 at 2:34 PM Matthew Biscocho (BLOOMBERG/ 919 3RD A) < > mbisco...@bloomberg.net> wrote: > > > Based on the discussion from yesterdays meetup, created SIP-23 for moving > > to Open Telemetry > > > https://cwiki.apache.org/confluence/display/SOLR/SIP-23%3A+Switch+from+DropWizard+to+OpenTelemetry > > > > With Solr 10 around the corner, hoping this work can get done before > then. > > > > From: dev@solr.apache.org At: 04/14/25 01:04:13 UTC-4:00To: > > dev@solr.apache.org > > Subject: Re: SIP proposal: Switch from DropWizard to OpenTelemetry > > > > I'm excited about the change! Where I work, it would *significantly* > > simplify our metrics pipeline if Solr were to embrace OTel for metrics, > as > > we could then use company-provided OTel plugins. The broad industry > > adoption of OTel points to this being the least friction. With > DropWizard, > > it appears we hacked attributes onto it, in a sense. > > > > The primary criteria/requirement that comes to mind is to have > > strong/sophisticated ways to filter the right metrics to publish. Solr > has > > that today. > > > > On Fri, Apr 11, 2025 at 5:00 PM Matthew Biscocho (BLOOMBERG/ 919 3RD A) < > > mbisco...@bloomberg.net> wrote: > > > > > Hey everyone, > > > > > > Making this thread because I was interested in writing up a SIP for > > > SOLR-17458 https://issues.apache.org/jira/browse/SOLR-17458 and > starting > > > a discussion around this. The proposed change involves migrating Solr's > > > metrics framework from DropWizard to OpenTelemetry (OTel). This will > move > > > Solr to an attribute based metric framework and will also help > > > future-proofing by keeping it from vendor lock-in, given > OpenTelemetry's > > > stance on being vendor-neutral and working with many different > pipelines. > > > Since Solr already packages an Open Telemetry SDK as a module for > > exporting > > > spans through OTLP, we can use that foundation to have it push metrics > as > > > well. There will be a significant change here which will probably break > > > many components and no longer be backwards compatible so this will > > probably > > > need to be a major release version change but we can potentially keep a > > few > > > things backwards compatible if needed. Just some things off the top of > my > > > head: > > > > > > * The Prometheus Exporter (Maybe should be deprecated?) > > > * GET /admin/metrics endpoint > > > * JVM metrics collected from OTel > > > * Metric reporters? > > > > > > Anyone have thoughts? > > > > > > > -- http://www.needhamsoftware.com (work) https://a.co/d/b2sZLD9 (my fantasy fiction book)
