>From times to times I write a scraper or some other tool that would
authenticate to a service and then use the auth result to do stuff
concurrently. But when auth expires, I need to synchronize all my
goroutines and have a single one do the re-auth process, check the status,
etc. and then arrange for all goroutines to go back to work using the new
auth result.
To generalize the problem: multiple goroutines read a cached value that
expires at some point. When it does, they all should block and some I/O
operation has to be performed by a single goroutine to renew the cached
value, then unblock all other goroutines and have them use the new value.
I solved this in the past in a number of ways: having a single goroutine
that handles the cache by asking it for the value through a channel, using
sync.Cond (which btw every time I decide to use I need to carefully re-read
its docs and do lots of tests because I never get it right at first). But
what I came to do lately is to implement an upgradable lock and have every
goroutine do:
*<code>*
func (x implem) getProtectedValue() (someType, error) {
// acquires a read lock that can be upgraded
lock := x.upgradableLock.UpgradableRLock()
// the Unlock method of the returned lock does the right thing
// even if we later upgrade the lock
defer lock.Unlock()
// here we have read access to x.protectedValue
// if the cached value is no longer valid, upgrade the lock
// and update it
if !isValid(x.protectedValue) && lock.TryUpgrade() {
// within this if, we know we *do* have write access
// to x.protectedValue
x.protectedValue, x.protectedValueError = newProtectedValue()
}
// here we can only say everyone has read access to x.protectedValue
// (the goroutine that got the lock upgrade could still write
// here but as this code is shared, we should check the result
// of the previous lock.TryUpgrade() again)
return x.protectedValue, x.protectedValueError
}
*</code>*
The implementation is quite simple:
*<code>*
// Upgradable implements all methods of sync.RWMutex, plus a new
// method to acquire a read lock that can optionally be upgraded
// to a write lock.
type Upgradable struct {
readers sync.RWMutex
writers sync.RWMutex
}
func (m *Upgradable) RLock() { m.readers.RLock() }
func (m *Upgradable) TryRLock() bool { return m.readers.TryRLock() }
func (m *Upgradable) RUnlock() { m.readers.RUnlock() }
func (m *Upgradable) RLocker() sync.Locker { return m.readers.RLocker() }
func (m *Upgradable) Lock() {
m.writers.Lock()
m.readers.Lock()
}
func (m *Upgradable) TryLock() bool {
if m.writers.TryLock() {
if m.readers.TryLock() {
return true
}
m.writers.Unlock()
}
return false
}
func (m *Upgradable) Unlock() {
m.readers.Unlock()
m.writers.Unlock()
}
// UpgradableRLock returns a read lock that can optionally be
// upgraded to a write lock.
func (m *Upgradable) UpgradableRLock() *UpgradableRLock {
m.readers.RLock()
return &UpgradableRLock{
m: m,
unlockFunc: m.RUnlock,
}
}
// UpgradableRLock is a read lock that can be upgraded to a write
// lock. This is acquired by calling (*Upgradable).
// UpgradableRLock().
type UpgradableRLock struct {
mu sync.Mutex
m *Upgradable
unlockFunc func()
}
// TryUpgrade will attempt to upgrade the acquired read lock to
// a write lock, and return whether it succeeded. If it didn't
// succeed then it will block until the goroutine that succeeded
// calls Unlock(). After unblocking, the read lock will still be
// valid until calling Unblock().
//
// TryUpgrade panics if called more than once or if called after
// Unlock.
func (u *UpgradableRLock) TryUpgrade() (ok bool) {
u.mu.Lock()
defer u.mu.Unlock()
if u.m == nil {
panic("TryUpgrade can only be called once and not after Unlock")
}
if ok = u.m.writers.TryLock(); ok {
u.m.readers.RUnlock()
u.m.readers.Lock()
u.unlockFunc = u.m.Unlock
} else {
u.m.readers.RUnlock()
u.m.writers.RLock()
u.unlockFunc = u.m.writers.RUnlock
}
u.m = nil
return
}
// Unlock releases the lock, whether it was a read lock or a write
// lock acquired by calling Upgrade.
//
// Unlock panics if called more than once.
func (u *UpgradableRLock) Unlock() {
u.mu.Lock()
defer u.mu.Unlock()
if u.unlockFunc == nil {
panic("Unlock can only be called once")
}
u.unlockFunc()
u.unlockFunc = nil
u.m = nil
}
*</code>*
I obviously try to avoid using it for other than protecting values that
require potentially long I/O operations (like in my case re-authenticating
to a web service) and also having a lot of interested goroutines. The good
thing is that most of the time this pattern only requires one RLock, but
the (*UpgradableRLock).Unlock requires an additional lock/unlock to prevent
misusage of the upgradable lock (I could potentially get rid of it but
preferred to keep it in the side of caution). Another thing I don't like is
that I need to allocate for each UpgradableRLock. I'm thinking to re-define
UpgradableRLock to be a defined type of Upgradable and convert the pointer
type of the receiver to *Upgradable whenever needed (also getting rid of
the lock that prevents misusage).
I wanted to ask the community what do you think of this pattern, pros and
cons, and whether this is overkill and could be better solved with
something obvious I'm missing (even though I did my best searching).
The reason why I decide to use this pattern is that all other options seem
to make my code more complex and less readable. I generally just implement
one method that does all the value-getting and it ends up being quite
readable and easy to follow:
1. Get an upgradable lock
2. If the value is stale and I get to upgrade the lock, then update the
value and store the error as well. The "update" part is generally
implemented as a separate method or func called "updateThingLocked".
3. Return the value and the error.
Kind regards.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/0fe9c0f3-a703-45f3-88b7-bbfc29111b2en%40googlegroups.com.
