Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> Josiah Carlson wrote:
>
> > It would basically be something along the lines of cPickle, but would
> > only support the basic types of: int, long, float, str, unicode, tuple,
> > list, dictionary.
>
> if you're aware of a way to do that faster than the current marshal
> implementation, maybe you could work on speeding up marshal instead?
The current implementation uses a fixed resize semantic (1024 bytes at a
time) that makes large marshal operations slow. If we were to switch to
a list resize-like or cStringIO semantic (overallocate by ~size>>3,
or at least double, respectively), it would likely increase the speed
for large resize operations. (see the w_more definition) This should
make it significantly faster than cPickle in basically all cases.
w_object uses a giant if/else if block to handle all of the possible
cases, both for identity checks against None, False, True, etc., as well
as with the various Py*_Check(). This is necessary due to marshal
supporting subclasses (the Py*_Check() calls) and the dynamic layout of
memory during Python startup. The identity checks may be able to be
replaced with a small array-like thing if we were to statically allocate
them from a single array to guarantee that their addresses are a fixed
distance apart...
char base_objects[320];
PyObject* IDENTITY[8];
int cases[8];
/*
64 bytes per object is overkill, and we may want to allocate enough room
for 15 objects, to make sure that IDENTITY[0] = NULL;
*/
p = 0
for obj_init in objs_to_init:
init_object(base_objects+p, obj_init)
x = ((base_objects+p)>>6)&7
IDENTITY[x] = (PyObject*)(base_objects+p)
cases[x] = p//64
p += 64
Then we could use the following in w_object...
x = (v>>6)&7
if v == IDENTITY[x] {
switch (cases[x]) {
case 0: /* should be null */
...
case 1: /* based on the order of objs_to_init */
}
}
The Py*_Check() stuff isn't so amenable to potential speedup, but in a
custom no-subclasses only base types version, we ccould use a variant of
the above mechanism to look directly at types, then use a second
switch/case statement, which should be significantly faster than the
if/else if tests that it currently uses. An identity check, then a fast
type check, otherwise fail.
- Josiah
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
