Ah interesting, I guess that could mean I would need to switch to using
reflect.Value as the "value" type in the Lua runtime. I am unclear about
the performance consequences, but I guess I could try to measure that.
Also, looking at the implementation of reflect, its seems like the Value
type I suggested in my reply to Ben [1] is a "special purpose" version of
reflect.Value - if you squint at it from the right angle!
--
Arnaud
[1]
type Value struct {
scalar uint64
iface interface{}
}
On Wednesday, 16 December 2020 at 00:56:52 UTC Keith Randall wrote:
> Unfortunately for you, interfaces are immutable. We can't provide a means
> to create an interface from a pointer, because then the user can modify the
> interface using the pointer they constructed it with (as you were planning
> to do).
>
> You could use a modifiable reflect.Value for this.
>
> var i int64 = 77
> v := reflect.ValueOf(&i).Elem()
>
> At this point, v now has .Type() of int64, and is settable.
>
> Note that to get the value you can't do v.Interface().(int64), as that
> allocates. You need to use v.Int().
> Of course, reflection has its own performance gotchas. It will solve this
> problem but may surface others.
> On Tuesday, December 15, 2020 at 12:04:54 PM UTC-8 [email protected] wrote:
>
>> Nice project!
>>
>> It's a pity Go doesn't have C-like unions for cases like this (though I
>> understand why). In my implementation of AWK in Go, I modelled the value
>> type as a pseudo-union struct, passed by value:
>>
>> type value struct {
>> typ valueType // Type of value (Null, Str, Num, NumStr)
>> s string // String value (for typeStr)
>> n float64 // Numeric value (for typeNum and typeNumStr)
>> }
>>
>> Code here:
>> https://github.com/benhoyt/goawk/blob/22bd82c92461cedfd02aa7b8fe1fbebd697d59b5/interp/value.go#L22-L27
>>
>> Initially I actually used "type Value interface{}" as well, but I
>> switched to the above primarily to model the funky AWK "numeric string"
>> concept. However, I seem to recall that it had a significant performance
>> benefit too, as passing everything by value avoided a number of allocations.
>>
>> Lua has more types to deal with, but you could try something similar. Or
>> maybe include int64 (for bool as well) and string fields, and everything
>> else falls back to interface{}? It'd be a fairly large struct, so not sure
>> it would help ... you'd have to benchmark it. But I'm thinking something
>> like this:
>>
>> type Value struct {
>> typ valueType
>> i int64 // for typ = bool, integer
>> s string // for typ = string
>> v interface{} // for typ = float, other
>> }
>>
>> -Ben
>>
>> On Wednesday, December 16, 2020 at 6:50:05 AM UTC+13 [email protected]
>> wrote:
>>
>>> Hi
>>>
>>> The context for this question is that I am working on a pure Go
>>> implementation of Lua [1] (as a personal project). Now that it is more or
>>> less functionally complete, I am using pprof to see what the main CPU
>>> bottlenecks are, and it turns out that they are around memory management.
>>> The first one was to do with allocating and collecting Lua "stack frame"
>>> data, which I improved by having add-hoc pools for such objects.
>>>
>>> The second one is the one that is giving me some trouble. Lua is a
>>> so-called "dynamically typed" language, i.e. values are typed but variables
>>> are not. So for easy interoperability with Go I implemented Lua values
>>> with the type
>>>
>>> // Go code
>>> type Value interface{}
>>>
>>> The scalar Lua types are simply implemented as int64, float64, bool,
>>> string with their type "erased" by putting them in a Value interface. The
>>> problem is that the Lua runtime creates a great number of short lived Value
>>> instances. E.g.
>>>
>>> -- Lua code
>>> for i = 0, 1000000000 do
>>> n = n + i
>>> end
>>>
>>> When executing this code, the Lua runtime will put the values 0 to 1
>>> billion into the register associated with the variable "i" (say, r_i). But
>>> because r_i contains a Value, each integer is converted to an interface
>>> which triggers a memory allocation. The critical functions in the Go
>>> runtime seem to be convT64 and mallocgc.
>>>
>>> I am not sure how to deal with this issue. I cannot easily create a
>>> pool of available values because Go presents say Value(int64(1000)) as an
>>> immutable object to me, so I cannot keep it around for later use to hold
>>> the integer 1001. To be more explicit
>>>
>>> // Go code
>>> i := int64(1000)
>>> v := Value(i) // This triggers an allocation (because the interface
>>> needs a pointer)
>>> // Here the Lua runtime can work with v (containing 1000)
>>> j := i + 1
>>> // Even though v contains a pointer to a heap location, I cannot
>>> modify it
>>> v := Value(j) // This triggers another allocation
>>> // Here the Lua runtime can work with v (containing 1001)
>>>
>>>
>>> I could perhaps use a pointer to an integer to make a Value out of.
>>> This would allow reuse of the heap location.
>>>
>>> // Go code
>>> p :=new(int64) // Explicit allocation
>>> vp := Value(p)
>>> i :=int64(1000)
>>> *p = i // No allocation
>>> // Here the Lua runtime can work with vp (contaning 1000)
>>> j := i + 1
>>> *p = j // No allocation
>>> // Here the Lua runtime can work with vp (containing 1001)
>>>
>>> But the issue with this is that Go interoperability is not so good, as
>>> Go int64 now map to (interfaces holding) *int64 in the Lua runtime.
>>>
>>> However, as I understand it, in reality interfaces holding an int64 and
>>> an *int64 both contain the same thing (with a different type annotation): a
>>> pointer to an int64.
>>>
>>> Imagine that if somehow I had a function that can turn an *int64 to a
>>> Value holding an int64 (and vice-versa):
>>>
>>> func Int64PointerToInt64Iface(p *int16) interface{} {
>>> // returns an interface that has concrete type int64, and points
>>> at p
>>> }
>>>
>>> func int64IfaceToInt64Pointer(v interface{}) *int64 {
>>> // returns the pointer that v holds
>>> }
>>>
>>> then I would be able to "pool" the allocations as follows:
>>>
>>> func NewIntValue(n int64) Value {
>>> v = getFromPool()
>>> if p == nil {
>>> return Value(n)
>>> }
>>> *p = n
>>> return Int64PointerToint64Iface(p)
>>> }
>>>
>>> func ReleaseIntValue(v Value) {
>>> addToPool(Int64IPointerFromInt64Iface(v))
>>> }
>>>
>>> func getFromPool() *int64 {
>>> // returns nil if there is no available pointer in the pool
>>> }
>>>
>>> func addToPool(p *int64) {
>>> // May add p to the pool if there is spare capacity.
>>> }
>>>
>>> I am sure that this must leak an abstraction and that there are good
>>> reasons why this may be dangerous or impossible, but I don't know what the
>>> specific issues are. Could someone enlighten me?
>>>
>>> Or even better, would there be a different way of modelling Lua values
>>> that would allow good Go interoperability and allow controlling heap
>>> allocations?
>>>
>>> If you got to this point, thank you for reading!
>>>
>>> Arnaud Delobelle
>>>
>>> [1] https://github.com/arnodel/golua
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/3bb7d415-1be3-4b5d-9379-558b2d59b2ban%40googlegroups.com.
