On Wed, Jul 25, 2012 at 6:12 PM, Yaroslav Halchenko
<li...@onerussian.com> wrote:
> actually I have not stated alternative variant since I thought it would
> not be wise to 'waste' memory : just store association between a
> particular build and target module_name but now I have mentioned that
> such code is pretty much there ... but incorrect and not used:
>
> $> grep -e '\Wkey\W' -e '^def cython_inline' -e 'code_cache'
> Cython/Build/Inline.py
> _code_cache = {}
> def cython_inline(code,
> key = orig_code, arg_sigs, sys.version_info, sys.executable,
> Cython.__version__
> module_name = "_cython_inline_" +
> hashlib.md5(str(key).encode('utf-8')).hexdigest()
> for key, value in literals.items():
> module_code = module_code.replace(key, value)
> _code_cache[key] = module_name
>
>
> so
> 1. key in for loop overrides the key tuple identifying the module_path
> 2. _code_cache is not used anywhere (but does waste memory although probably
> not much since there is not that many values of key I guess which
> it would get)
>
> thus I wondered if it is ok to waste memory ( ;) ) should following patch be
> used? or should _code_cache be removed altogether (or am I missing its role?)
>
> $> quilt diff
> --- a/Cython/Build/Inline.py
> +++ b/Cython/Build/Inline.py
> @@ -138,13 +138,11 @@ def cython_inline(code,
> arg_sigs = tuple([(get_type(kwds[arg], ctx), arg) for arg in arg_names])
> key = orig_code, arg_sigs, sys.version_info, sys.executable,
> Cython.__version__
> module_name = "_cython_inline_" +
> hashlib.md5(str(key).encode('utf-8')).hexdigest()
> -
> - so_ext = [ ext for ext,_,mod_type in imp.get_suffixes() if mod_type ==
> imp.C_EXTENSION ][0]
> - module_path = os.path.join(lib_dir, module_name+so_ext)
> + module_path = _code_cache.get(module_name, None)
>
> if not os.path.exists(lib_dir):
> os.makedirs(lib_dir)
> - if force or not os.path.isfile(module_path):
> + if force or module_path is None or not os.path.isfile(module_path):
> cflags = []
> c_include_dirs = []
> qualified = re.compile(r'([.\w]+)[.]')
> @@ -189,7 +187,9 @@ def __invoke(%(params)s):
> build_extension.build_temp = os.path.dirname(pyx_file)
> build_extension.build_lib = lib_dir
> build_extension.run()
> - _code_cache[key] = module_name
> + # Should we check if module_path is not None either it still matches?
> + module_path = os.path.join(lib_dir,
> build_extension.get_ext_filename(module_name))
> + _code_cache[module_name] = module_path
>
> module = imp.load_dynamic(module_name, module_path)
> arg_list = [kwds[arg] for arg in arg_names]
Compiled modules can persist between sessions as well.
I like your solution of caching the module name, what if we computed
the module name iff it wasn't in the cache, then compiled the file iff
the .so file didn't exist (with a check that the module name was OK).
Alternatively, could we just rename the compiled library to be what we
expect if it wasn't already?
- Robert
_______________________________________________
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
