Hi Ben, that's an interesting idea. I considered it at the start but
didn't go for it in the end (I can't remember why exactly, probably
because that would make it quite a big struct for Lua). There is a
possibility that I could adapt it a bit and have something like
type Value struct {
scalar uint64
iface interface{}
}
The type could be always obtained from the iface field (it would be
its concrete type), but the value could be encoded in the scalar field
for a few types such as int64, float64, bool. There would be no
storage overhead for int64 and floa64, as the extra 8 bytes used for
the scalar field are saved by having a "constant" iface field. The
overhead for other non-scalar values would be only 8 bytes.
I would need some reusable "dummy" interface values for the types
encoded in the scalar:
var (
dummyInt64 interface{} = int64(0)
dummyFloat64 interface{} = float64(0)
dummyBool interface{} = false
)
Then I could create Value instances like this:
func IntValue(n int64) Value {
return Value{uint64(n), dummyInt64}
}
func FloatValue(f float64) Value {
return Value{*(*uint64)(unsafe.Pointer(&f)), dummyFloat64}
}
func BoolValue(b bool) Value {
var s uint64
if b {
s = 1
}
return Value{s, dummyBool}
}
func StringValue(s string) Value {
return Value{iface: s}
}
func TableValue(t Table) Value {
return Value{iface: t}
}
We could obtain the type of Values like this:
type ValueType uint8
const (
IntType ValueType = iota
FloatType
BoolType
StringType
TableType
)
func (v Value) Type() ValueType {
switch v.iface.(type) {
case int64:
return IntType
case float64:
return FloatType
case bool:
return BoolType
case string:
return StringType
case Table:
return TableType
default:
panic("invalid type")
}
}
Methods like this could extract the concrete value out a Value instance:
func (v Value) AsInt() int64 {
return int64(v.scalar)
}
func (v Value) AsFloat() float64 {
return *(*float64)(unsafe.Pointer(&v.scalar))
}
func (v Value) AsBool() bool {
return v.scalar != 0
}
func (v Value) AsString() string {
return v.iface.(string)
}
func (v Value) AsTable() Table {
return v.iface.(Table)
}
Interoperability with Go code is not as good but still OK. There is
no need to maintain a pool of reusable values, which is a bonus. I'll
have to see how much modification to the codebase it requires, but
that sounds interesting.
--
Arnaud
On Tue, 15 Dec 2020 at 20:06, [email protected] <[email protected]> wrote:
>
> Nice project!
>
> It's a pity Go doesn't have C-like unions for cases like this (though I
> understand why). In my implementation of AWK in Go, I modelled the value type
> as a pseudo-union struct, passed by value:
>
> type value struct {
> typ valueType // Type of value (Null, Str, Num, NumStr)
> s string // String value (for typeStr)
> n float64 // Numeric value (for typeNum and typeNumStr)
> }
>
> Code here:
> https://github.com/benhoyt/goawk/blob/22bd82c92461cedfd02aa7b8fe1fbebd697d59b5/interp/value.go#L22-L27
>
> Initially I actually used "type Value interface{}" as well, but I switched to
> the above primarily to model the funky AWK "numeric string" concept. However,
> I seem to recall that it had a significant performance benefit too, as
> passing everything by value avoided a number of allocations.
>
> Lua has more types to deal with, but you could try something similar. Or
> maybe include int64 (for bool as well) and string fields, and everything else
> falls back to interface{}? It'd be a fairly large struct, so not sure it
> would help ... you'd have to benchmark it. But I'm thinking something like
> this:
>
> type Value struct {
> typ valueType
> i int64 // for typ = bool, integer
> s string // for typ = string
> v interface{} // for typ = float, other
> }
>
> -Ben
>
> On Wednesday, December 16, 2020 at 6:50:05 AM UTC+13 [email protected] wrote:
>>
>> Hi
>>
>> The context for this question is that I am working on a pure Go
>> implementation of Lua [1] (as a personal project). Now that it is more or
>> less functionally complete, I am using pprof to see what the main CPU
>> bottlenecks are, and it turns out that they are around memory management.
>> The first one was to do with allocating and collecting Lua "stack frame"
>> data, which I improved by having add-hoc pools for such objects.
>>
>> The second one is the one that is giving me some trouble. Lua is a so-called
>> "dynamically typed" language, i.e. values are typed but variables are not.
>> So for easy interoperability with Go I implemented Lua values with the type
>>
>> // Go code
>> type Value interface{}
>>
>> The scalar Lua types are simply implemented as int64, float64, bool, string
>> with their type "erased" by putting them in a Value interface. The problem
>> is that the Lua runtime creates a great number of short lived Value
>> instances. E.g.
>>
>> -- Lua code
>> for i = 0, 1000000000 do
>> n = n + i
>> end
>>
>> When executing this code, the Lua runtime will put the values 0 to 1 billion
>> into the register associated with the variable "i" (say, r_i). But because
>> r_i contains a Value, each integer is converted to an interface which
>> triggers a memory allocation. The critical functions in the Go runtime seem
>> to be convT64 and mallocgc.
>>
>> I am not sure how to deal with this issue. I cannot easily create a pool of
>> available values because Go presents say Value(int64(1000)) as an immutable
>> object to me, so I cannot keep it around for later use to hold the integer
>> 1001. To be more explicit
>>
>> // Go code
>> i := int64(1000)
>> v := Value(i) // This triggers an allocation (because the interface
>> needs a pointer)
>> // Here the Lua runtime can work with v (containing 1000)
>> j := i + 1
>> // Even though v contains a pointer to a heap location, I cannot modify
>> it
>> v := Value(j) // This triggers another allocation
>> // Here the Lua runtime can work with v (containing 1001)
>>
>>
>> I could perhaps use a pointer to an integer to make a Value out of. This
>> would allow reuse of the heap location.
>>
>> // Go code
>> p :=new(int64) // Explicit allocation
>> vp := Value(p)
>> i :=int64(1000)
>> *p = i // No allocation
>> // Here the Lua runtime can work with vp (contaning 1000)
>> j := i + 1
>> *p = j // No allocation
>> // Here the Lua runtime can work with vp (containing 1001)
>>
>> But the issue with this is that Go interoperability is not so good, as Go
>> int64 now map to (interfaces holding) *int64 in the Lua runtime.
>>
>> However, as I understand it, in reality interfaces holding an int64 and an
>> *int64 both contain the same thing (with a different type annotation): a
>> pointer to an int64.
>>
>> Imagine that if somehow I had a function that can turn an *int64 to a Value
>> holding an int64 (and vice-versa):
>>
>> func Int64PointerToInt64Iface(p *int16) interface{} {
>> // returns an interface that has concrete type int64, and points at p
>> }
>>
>> func int64IfaceToInt64Pointer(v interface{}) *int64 {
>> // returns the pointer that v holds
>> }
>>
>> then I would be able to "pool" the allocations as follows:
>>
>> func NewIntValue(n int64) Value {
>> v = getFromPool()
>> if p == nil {
>> return Value(n)
>> }
>> *p = n
>> return Int64PointerToint64Iface(p)
>> }
>>
>> func ReleaseIntValue(v Value) {
>> addToPool(Int64IPointerFromInt64Iface(v))
>> }
>>
>> func getFromPool() *int64 {
>> // returns nil if there is no available pointer in the pool
>> }
>>
>> func addToPool(p *int64) {
>> // May add p to the pool if there is spare capacity.
>> }
>>
>> I am sure that this must leak an abstraction and that there are good reasons
>> why this may be dangerous or impossible, but I don't know what the specific
>> issues are. Could someone enlighten me?
>>
>> Or even better, would there be a different way of modelling Lua values that
>> would allow good Go interoperability and allow controlling heap allocations?
>>
>> If you got to this point, thank you for reading!
>>
>> Arnaud Delobelle
>>
>> [1] https://github.com/arnodel/golua
>
> --
> You received this message because you are subscribed to a topic in the Google
> Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/163s0WdXYIU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/dcd07f38-1ead-4359-90f3-f6b514c7d541n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CAJ6cK1bpOETKSJv4%2BEEvyMtX-FVBG2PvxzEgv5Wi_sK%3Dgm95rg%40mail.gmail.com.
