*Hi everyone,*
I'm relatively new to Go and even newer to *CGO*, so I’d really appreciate
any guidance on an issue I’ve been facing.
*Problem Overview*
I noticed that when using CGO, my application's *memory usage keeps
increasing*, and threads do not seem to be properly cleaned up.
The Go runtime (pprof) does not indicate any leaks, but when I monitor the
process using *Activity Monitor*, I see a growing number of threads and
increasing memory consumption.
*Observations*
- Memory usage *keeps increasing* over time when using CGO, even after
forcing garbage collection.
- Threads spawned for CGO calls appear to *not be released*, causing a
large number of lingering threads.
- The issue is *not present* when using pure Go time.Sleep() instead of
the CGO function.
*Reproducible Example*
I created a minimal program that reproduces the issue. The following
*CGO-based* code keeps allocating memory and does not free threads properly:
package main
import (
"fmt"
"runtime"
"runtime/debug"
"sync"
"time"
)
/*
#include <unistd.h>
void cgoSleep() {
sleep(1);
}
*/
import "C"
func main() {
start := time.Now()
var wg sync.WaitGroup
for i := 0; i < 5000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
C.cgoSleep()
}()
}
wg.Wait()
end := time.Now()
// Force GC and free OS memory
runtime.GC()
debug.FreeOSMemory()
time.Sleep(10 * time.Second)
var m runtime.MemStats
runtime.ReadMemStats(&m)
fmt.Printf("Alloc = %v MiB", m.Alloc/1024/1024)
fmt.Printf("\tTotalAlloc = %v MiB", m.TotalAlloc/1024/1024)
fmt.Printf("\tSys = %v MiB", m.Sys/1024/1024)
fmt.Printf("\tNumGC = %v\n", m.NumGC)
fmt.Printf("Total time: %v\n", end.Sub(start))
select {}
}
*Expected Behavior*
- The memory usage should *not* continue rising indefinitely.
- Threads should be properly cleaned up when they finish executing.
- The behavior should be similar to the following *pure Go* equivalent,
which does *not* exhibit the issue:
*Actual Results* *With CGO (cgoSleep()):*
- *Memory Usage:* *296 MB*
- *Threads:* *5,003*
- *System Memory (Sys from runtime.MemStats)*:
*205 MB *
*With Pure Go (time.Sleep()):*
- *Memory Usage:* *14 MB*
- *Threads:* *14*
- *System Memory (Sys from runtime.MemStats)*: *24 MB*
*Additional Attempt*
I tried forcing thread cleanup using runtime.LockOSThread() and
runtime.Goexit(), but *while the number of threads decreases, memory is
still never fully released*:
go func() { runtime.LockOSThread() defer wg.Done() C.cgoSleep()
runtime.Goexit() }() *Questions*
1. *Why is memory increasing indefinitely with CGO?*
2. *Why are threads not getting properly cleaned up after CGO calls?*
3. *Is there a way to force the Go runtime to reclaim memory allocated
for CGO threads?*
4. *Is there a better approach to handling CGO calls that spawn
short-lived threads?*
5. *Would using runtime.UnlockOSThread() help in this case, or is this
purely a CGO threading issue?*
6. *Is there a way to track down where the memory is being held? Since
pprof does not show high memory usage, what other tools can I use?*
*Go Version*
go1.23.5 darwin/arm64
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/golang-nuts/5e8c1622-e6f0-47a8-a869-e78797800fb5n%40googlegroups.com.
