I would recommend getting a heap snapshot and posting it to https://pprof.me .
curl -o heap.pprof http://localhost:9090/debug/pprof/heap Also including http://localhost:9090/tsdb-status would help. On Thu, Nov 7, 2024 at 9:53 AM Bhanu Prakash <[email protected]> wrote: > Hello Prometheus Community, > > I’m encountering a memory usage issue with my Prometheus server, > particularly during and after startup, and I’m hoping to get some insights > on optimizing it. > > *Problem:* Upon startup, my Prometheus instance consumes a large amount > of memory, primarily due to the WAL (Write-Ahead Log) replay. To address > this, I enabled --enable-feature=memory-snapshot-on-shutdown, expecting it > to reduce the startup memory spike by eliminating the need for full WAL > replay. However, I’m still seeing the memory usage spike to around 5GB on > startup. Once started, Prometheus continues to hold this high memory, > without releasing it back to the system. > > Is there a recommended way to configure Prometheus to release memory > post-startup? > Are there additional configurations or optimizations for large WAL files > or memory management that could help? > > Any guidance or suggestions would be greatly appreciated! > > Thank you, > BhanuPrakash. > > > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/prometheus-users/99e6725f-f757-45f4-80a4-98c60e2b5063n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/99e6725f-f757-45f4-80a4-98c60e2b5063n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmowoD8mORtkAuOrf_Z4cd%2BbHWXgswACO1cmEwNsOFgQEQ%40mail.gmail.com.
