Hi Konstantin, Thank you for the hit. I'll look into it.
On Monday, March 6, 2017 at 5:25:47 PM UTC+8, Konstantin Khomoutov wrote: > > On Mon, 6 Mar 2017 12:01:43 +0300 > Konstantin Khomoutov <[email protected] <javascript:>> wrote: > > > > The application is working right now. Current work for me is to > > > found a way to reduce it's memory footprint as it will take at > > > least 1GB memory to hold only C10K idle connections. > > > > > > I know it's mainly me causing such memory usage, so I just want to > > > know if there are a better way to get around my problem without > > > changing too many of those already existing design. That's is when > > > the epoll-style idea came out. > > > > > > + epoll-styled design is seems suits better for my application by > > > nature. > > > > May be you could get away using an approach sported by certain > > networking servers (namely, the Dovecot IMAP server): they have a > > dedicated process managing idling connections, and pass socket > > descriptors back and forth between such "hibernation" process and the > > "master" process. The idea is that when a master process decides a > > particular client is idling, it discards or persists the client's > > state and passes its socket descriptor to the hibernation process; > > when the hibernation process detects a sensible incoming data on the > > socket, it sends it back to the master process which "un-hibernates" > > the client's state and resumes normal operation. > > > > This approach uses Unix-domain sockets which allow ownership transfers > > of file (and socket) descriptors between processes. > > > > Supposedly you could implement such a hibernation process yourself > > in C using libevent of libav with the master process still being > > written in Go, and see whether that would work out. > > > > Note though that the hibernation process may need to be made somewhat > > smart about the data it detects on idling connections: if your > > application-level protocol supports some sort of "keepalives" (in the > > form of "pinging" or otherwise), their support should supposedly be > > implemented directly in the hibernation process itself. > > As of Go 1.7, the code which implements FD passing can be found in > {go}/src/syscall/syscall_unix_test.go file. > > You might also take a look at [1] which directly uses epoll via > syscalls and basically allows you to wait on any number of sockets in > a single goroutine. This looks like you could implement such a > "hibernation process" using a dedicated goroutine: when a worker > goroutine thinks the connection it's serving starts idling, it passes > it to the "hibernation" goroutine and exits; when a hibernation > goroutine thinks there's an activity on a socket it manages, it simply > sends the socket to a goroutine which manages a pool of workers. > > Unfortunately, the package only supports epoll, but at least you could > try to put up a PoC to see if it worth the effort of adding more > OS-specific select/poll backends. > > 1. https://godoc.org/github.com/hkwi/fdset > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
