I just wrote this a bit over a month ago:
https://git.parallelcoin.io/dev/pod/src/branch/master/pkg/util/cl
It was brutal simple before (only one 600 line source lots of type
switches) but it now also has a registration system for setting up
arbitrary subsystem log levels.
By the way, I could be wrong in my thinking about this, but it was central
to the design to use interface{} channels and meaningful type names and the
variables and inputs for, such as a printf type function, are this way only
bundled into a struct literal, no function is called, the execution thread
just forks and log messages pass through directly without any function call.
I even have a type for closures to pass straight through and they also
don't execute until the last step of composing the output, so I have 6 core
goroutines and 6 for each subsystem I set up, and most of the work is done
in the end of two passes through channels (subsystems drop them if so
configured).
I think it also has a somewhat crude shutdown handling system in it too.
It could definitely do with more work and actual performance overhead in
throughput and latency to be profiled and optimised, but being simple I
think it's not got much room to improve in the general design.
The messages pass through two 'funnels', with one per subsystem gated by
its level setting, and then the actual function calls to splice the strings
and output the result is deferred until the final 6 channel level main
thread. I have just used them mainly on a package level basis but you can
declare several in a package they just have to have different string labels.
Anyway, I'm kinda proud of it because my specific need is for avoiding such
in-thread overheads as possible since during replay of the database log it
only runs one thread due to the dependencies between data elements, and I
think this does this well but I can't say I have any measurements either
way, except I do know that especially permanent allocated channels only
have a startup cost and their scheduling is very low cost, and also
explicates to the compiler that the processes are concurrent and ideally
should not be overlapping each other.
The print functions in fmt and log all have the per-call overhead cost,
whereas channels have initialisation and lower context switch cost, but
yes, more memory use but mainly because it funnels messages from another
couple hundred other threads. So that's also my theory and why this post
caught my eye. My logger is unlicenced so do whatever you like except
pretend it's not prior art.
On Saturday, 16 March 2019 17:02:06 UTC+1, Thomas S wrote:
>
> Hello,
>
> I have a software needing a lot of logs when processing.
>
> I need to :
> 1- Improve the processing time by doing non-blocking logs
> 2- Be safe with goroutines parallel logs
>
> fmt package doesn't match with (1) & (2)
> log package doesn't match with (1)
>
> *Here is my proposal. What do you think ?*
>
> *Design :*
>
> [Log functions] -----channel---->[Buffering function (goroutine)]
> ----channel----> [Printing function (goroutine)]
>
>
> package glog
>
> import (
> "container/list"
> "encoding/json"
> "fmt"
> )
>
> /*
> ** Public
> */
>
> func Println(data ...interface{}) {
> bufferChan <- fmt.Sprintln(data...)
> }
>
> func Print(data ...interface{}) {
> bufferChan <- fmt.Sprint(data...)
> }
>
> func Printf(s string, data ...interface{}) {
> go func() {
> r := fmt.Sprintf(s, data...)
> bufferChan <- r
> }()
> }
>
> /*
> ** Private
> */
>
> var bufferChan chan string
> var outChan chan string
>
> func init() {
> bufferChan = make(chan string)
> outChan = make(chan string)
> go centrale()
> go buffout()
> }
>
> func centrale() {
> var buff *list.List
> buff = list.New()
> for {
> if buff.Len() > 0 {
> select {
> case outChan <- buff.Front().Value.(string):
> buff.Remove(buff.Front())
> case tmp := <-bufferChan:
> buff.PushBack(tmp)
> }
>
> } else {
> tmp := <-bufferChan
> buff.PushBack(tmp)
> }
> }
> }
>
> func buffout() {
> for {
> data := <-outChan
> fmt.Print(data)
> }
> }
>
>
>
> It works well for now, I want to be sure to not miss anything as it's a
> very important part of the code.
>
> Thank you for your review.
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
