[issue7846] Fnmatch cache is never cleared during usage

Tom Rini added the comment: So yes, we have some code that was doing, roughly: if any(fnmatchcase(key, pat) for pat in pattern): refs.add(key) on 200-300 elements, a lot. That said, many of them were not globs so we've worked around this to only use fnmatchcase on globs and just key in set

[issue7846] Fnmatch cache is never cleared during usage

R. David Murray added the comment: Grr. "it is *not* true that nothing is ever cached". -- ___ Python tracker ___ ___ Python-bugs-lis

[issue7846] Fnmatch cache is never cleared during usage

R. David Murray added the comment: Well, it's unnoficially configurable pre-3.2 (set _MAXCACHE on the module). The lru_cache implementation doesn't have such an undocumented way of tweaking the limit. In neither case, however, is it true that "nothing is ever cached". You are saying that y

[issue7846] Fnmatch cache is never cleared during usage

Tom Rini added the comment: Did a change later make this user-configurable? I've got some code here that now runs so slow as to be unusable because nothing is ever cached anymore. Thanks! -- nosy: +Tom.Rini ___ Python tracker

[issue7846] Fnmatch cache is never cleared during usage

Raymond Hettinger added the comment: For Py3.2, applying the new functools.lru_cache(). See r83871. -- nosy: +rhettinger ___ Python tracker ___ __

[issue7846] Fnmatch cache is never cleared during usage

R. David Murray added the comment: Committed to py3k in r82730, 3.1 in r82733, 2.7 in r82732, and, since 2.6.6 is still to come, 2.6 in r82740. Andrew: FYI I changed the assertTrue to assertLessEqual to take advantage of the better failure messages provided by the newer unittest methods. Tha

[issue7846] Fnmatch cache is never cleared during usage

Changes by R. David Murray : -- assignee: -> r.david.murray ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue7846] Fnmatch cache is never cleared during usage

Andrew Clegg added the comment: Updated py3k patch to use the more concise bytes syntax, cheers. -- Added file: http://bugs.python.org/file17918/py3k-fnmatch.patch ___ Python tracker ___

[issue7846] Fnmatch cache is never cleared during usage

Antoine Pitrou added the comment: Instead of writing `bytes('foo', encoding='ascii')`, you can just write `b'foo'`. Otherwise, patch looks ok. -- nosy: +pitrou ___ Python tracker __

[issue7846] Fnmatch cache is never cleared during usage

Changes by Andrew Clegg : Added file: http://bugs.python.org/file17917/py3k-fnmatch.patch ___ Python tracker ___ ___ Python-bugs-list mailing l

[issue7846] Fnmatch cache is never cleared during usage

Andrew Clegg added the comment: OK, regenerated both patches from the top level of the checkout (paths within the patch are 'Lib/fnmatch.py' and 'Lib/test/test_fnmatch.py'. Sorry about the corrupted patch before - I've tested this version by reverting my changes and reapplying it, and it seem

[issue7846] Fnmatch cache is never cleared during usage

Changes by Andrew Clegg : Removed file: http://bugs.python.org/file17902/py27-fnmatch.patch ___ Python tracker ___ ___ Python-bugs-list mailing

[issue7846] Fnmatch cache is never cleared during usage

Changes by Andrew Clegg : Removed file: http://bugs.python.org/file17901/py3k-fnmatch.patch ___ Python tracker ___ ___ Python-bugs-list mailing

[issue7846] Fnmatch cache is never cleared during usage

R. David Murray added the comment: Thanks for working on this. I tried the py3k patch but it doesn't apply cleanly. Patch says "patch: malformed patch at line 57:" followed by an apparently blank line. Also, could you please generate the patches from the top level of the checkout? It

[issue7846] Fnmatch cache is never cleared during usage

Changes by Andrew Clegg : Removed file: http://bugs.python.org/file16116/fnmatch.patch ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue7846] Fnmatch cache is never cleared during usage

Andrew Clegg added the comment: Attached is a patch and unit test against release27-maint. -- Added file: http://bugs.python.org/file17902/py27-fnmatch.patch ___ Python tracker _

[issue7846] Fnmatch cache is never cleared during usage

Andrew Clegg added the comment: Sorry, messed up indentation on last patch, trying again... -- Added file: http://bugs.python.org/file17901/py3k-fnmatch.patch ___ Python tracker

[issue7846] Fnmatch cache is never cleared during usage

Changes by Andrew Clegg : Removed file: http://bugs.python.org/file17900/py3k-fnmatch.patch ___ Python tracker ___ ___ Python-bugs-list mailing

[issue7846] Fnmatch cache is never cleared during usage

Andrew Clegg added the comment: Attached is the Py3K version of the patch, and a unit test (Py3K only). -- Added file: http://bugs.python.org/file17900/py3k-fnmatch.patch ___ Python tracker

[issue7846] Fnmatch cache is never cleared during usage

R. David Murray added the comment: OK, I've looked at the patch now. (1) it needs unit tests. (2) a py3k port would be helpful, since the code is very different in py3k, and we are now applying patches first to the py3k branch and then backporting. I think this can be considered a bug (the

[issue7846] Fnmatch cache is never cleared during usage

Andrew Clegg added the comment: Hi, This bug seems to have stalled - how can I get it moved forward? There don't seem to be any objections with the patch as-is, and the problem seems clear. Cheers -- ___ Python tracker

[issue7846] Fnmatch cache is never cleared during usage

Andrew Clegg added the comment: "If you give something an _, then it is not considered part of the public API and it (the internal API, not the value) is subject to change, which means you should *not* suggest that users change it. If they find it and want to change it anyway, that's their l

[issue7846] Fnmatch cache is never cleared during usage

Éric Araujo added the comment: Hi Your rationale makes perfect sense. Perhaps add a comment suggesting it’s ok to change the value. Cheers -- ___ Python tracker ___ __

[issue7846] Fnmatch cache is never cleared during usage

Andrew Clegg added the comment: Hi, I used a global name for a couple of reasons: it is consistent with the cache itself and the size of the cache being defined in the same place; and because I wanted to allow the value to be modified by users. I used the leading underscore to give a weak in

[issue7846] Fnmatch cache is never cleared during usage

Éric Araujo added the comment: Hello The feature seems useful to me and the patch straightforward. I wonder why you used a global name for the max length (“_MAXCACHE”) instead of inlining the value where it is used. Does it feel more constanty? ;) I fear that the leading underscore won’t dis

[issue7846] Fnmatch cache is never cleared during usage

Changes by Brian Curtin : -- keywords: +needs review priority: -> normal stage: -> test needed type: -> behavior versions: +Python 2.6, Python 2.7, Python 3.1, Python 3.2 -Python 2.5 ___ Python tracker __

[issue7846] Fnmatch cache is never cleared during usage

New submission from Andrew Clegg : The fnmatch module has a cache of translation between glob patterns and compiled regular expressions. However this cache is never emptied; only added to. I am writing a python program which as part of its execution checks millions of unique globs - this cause