[issue19904] Add 128-bit integer support to struct
New submission from Fil Mackay: I've been looking at adding 128-bit support to the struct module. Currently only named integer types are supported, which vary in implementation. These include: short int long long long Depending on the platform, none may translate to 128-bit integer (the case with all platforms today?). One approach would be to make a new type that relates specifically to 128-bit integer, side-stepping the naming approaches to integer in C. The other, would be to make new types for all integer sizes that relate to specific sizes, instead of relying on C namings. Much bigger implications? I propose creating new types: "o": __int128_t "O": __uint128_t "t": __int256_t (why not?) "T": __uint256_t "v": __int512_t (what, too far?) "V": __int512_t What implications are there here in killing the connection between a C named type and a specific size? -- components: ctypes messages: 205344 nosy: fil priority: normal severity: normal status: open title: Add 128-bit integer support to struct type: enhancement versions: Python 2.7, Python 3.4, Python 3.5 ___ Python tracker <http://bugs.python.org/issue19904> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19905] Add 128-bit integer support to ctypes
New submission from Fil Mackay: This depends on struct issue #19904, and involves adding the following types: c_int128 c_uint128 c_int256 (maybe?) c_uint256 c_int512 (too much?) c_uint512 After resolution of the struct issue, this implementation will become clearer. -- components: ctypes messages: 205345 nosy: fil priority: normal severity: normal status: open title: Add 128-bit integer support to ctypes type: enhancement versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5 ___ Python tracker <http://bugs.python.org/issue19905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19904] Add 128-bit integer support to struct
Fil Mackay added the comment: The use case is interacting with C structures that have 128/256/512 bit integers in them. These require correct memory alignment, and can't reliably be hacked with multiple int64's. These size ints come from the fact that CPU's now have registers of these sizes, and can't be serviced with int.from/to_bytes for performance reasons. The same reason int64 being supported with this approach would be very inconvenient and terrible for performance since they are an intrinsic type in modern hardware, not a software construction. Two key use cases I can think of are: - any form of integer SIMD operation (vectors) - hosting and maintaining hash values which are routinely 128-bit and greater -- ___ Python tracker <http://bugs.python.org/issue19904> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19904] Add 128-bit integer support to struct
Fil Mackay added the comment: I noticed that python 2.7 has been removed - why would this not be appropriate for this version? Given the current level of use of this version, I see it's inclusion as very important - without having to be relegated to 3.x only. -- ___ Python tracker <http://bugs.python.org/issue19904> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19904] Add 128-bit integer support to struct
Fil Mackay added the comment: Stefan, performance is not the principle motivator here: the intention is that it is just sensible to support this integer type, just like supporting int64 since it is an intrinsic type supported by CPU's. Of course any integer length could be handled by memoryview unpacker, but this would not really make sense for int64. My view is that any intrinsic type (ie. register type) is the key point at which it should be supported by struct, and not a custom unpacker. 128/256/512 bit integers are in this category.. Principally this is so that it can be used by ctypes (see #19905). -- ___ Python tracker <http://bugs.python.org/issue19904> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19904] Add 128-bit integer support to struct
Fil Mackay added the comment: Antoine, No, the SIMD vector should be treated as a int128 (for eg.), not as 4 python ints. It is the unpacking process that unpacks into 4 python ints. The two shouldn't be confused - you definitely want to be able to treat the vector in both states (packed and unpacked). I'm not saying that Python should necessarily make use of SIMD instructions (don't think performance is critical there in py), but that it should at least be access to a raw 128/256/512 bit integer within a C structure. Part of this is interacting with SIMD'able structures - at least in my use cases :) -- ___ Python tracker <http://bugs.python.org/issue19904> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19904] Add 128-bit integer support to struct
Fil Mackay added the comment: Antoine, > I don't think "register types" matter here. The struct module provides > interoperability with low-level *data types* (in C and other common > languages), not CPU *registers*. OK, see where you're coming from. I guess my thinking was such that struct (and ctypes) support is required for data types that are common within C structures. The fact that 128-bit ints are now appearing within these structures is due to the fact that CPU support has dramatically improved, and are supported natively in register types. So it was kind of an indirect connection. So the question I guess is whether you think these types are "common enough" in C structs, which for me they certainly are for my data processing applications.. (pretty please :) > So IMHO the question is whether there is an use case for 128-bit > integers; *not* for arrays of four 32-bit integers packed in a 128-bit > register. Agree completely - just stick to thinking about atomic 128-bit integers. A pack/unpack function should do the conversion, and have nothing to do with this struct support. -- ___ Python tracker <http://bugs.python.org/issue19904> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19904] Add 128-bit integer support to struct
Fil Mackay added the comment: So where do we go from here - is there enough support for this to proceed, or vote the idea off the island :) -- ___ Python tracker <http://bugs.python.org/issue19904> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
