[issue28743] test_choices_algorithms() in test_random uses lots of memory

Martin Panter added the comment: Thanks Raymond, now the test is noticeably faster and fine with my memory situation :) -- stage: -> resolved ___ Python tracker ___ ___

[issue28743] test_choices_algorithms() in test_random uses lots of memory

Raymond Hettinger added the comment: A smaller value suffices for this test. It was trying to make sure the underlying algorithms are as in-sync as possible without going to extremes. -- resolution: -> fixed status: open -> closed ___ Python tracke

[issue28743] test_choices_algorithms() in test_random uses lots of memory

Roundup Robot added the comment: New changeset 3551fca2c6ae by Raymond Hettinger in branch '3.6': Issue #28743: Reduce memory consumption for random module tests https://hg.python.org/cpython/rev/3551fca2c6ae -- nosy: +python-dev ___ Python tracker

[issue28743] test_choices_algorithms() in test_random uses lots of memory

Changes by Raymond Hettinger : -- assignee: -> rhettinger ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http

[issue28743] test_choices_algorithms() in test_random uses lots of memory

Martin Panter added the comment: Gareth, are you sure that fixes the main memory problem? Did you see Serhiy’s cum_weights list? Looking at the code, a list of every number from one to 13 million will use more memory (to hold each unique integer) than the initial list of repeated ones. I thin

[issue28743] test_choices_algorithms() in test_random uses lots of memory

Gareth Rees added the comment: In order for this to work, the __getitem__ method needs to be: def __getitem__(self, key): if 0 <= key < self.n: return self.elem else: raise IndexError(key) But unfortunately this is very bad for the performance of the

[issue28743] test_choices_algorithms() in test_random uses lots of memory

Serhiy Storchaka added the comment: The main memory consumer is a list of cumulated weights created inside choices(). -- ___ Python tracker ___ _

[issue28743] test_choices_algorithms() in test_random uses lots of memory

Gareth Rees added the comment: Couldn't the test case use something like this to avoid allocating so much memory? from collections.abc import Sequence class RepeatedSequence(Sequence): """Immutable sequence of n repeats of elem.""" def __init__(self, elem, n):

[issue28743] test_choices_algorithms() in test_random uses lots of memory

Serhiy Storchaka added the comment: This is a problem to me too. -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-lis

[issue28743] test_choices_algorithms() in test_random uses lots of memory

New submission from Martin Panter: Revision 32bfc8b6 added test.test_random.MersenneTwister_TestBasicOps.test_choices_algorithms(), which runs the following code: n = 13132817 # 13 million self.gen.choices(range(n), [1]*n, k=1) When I build Python with the “--with-pydebug” configure