Re: Test optimizations (2-5x as fast)

2011-06-06 Thread Erik Rose
On Jun 5, 2011, at 10:18 AM, Ned Batchelder wrote: > When I try this on a PostgreSQL database, I have problems relating to > violated uniqueness constraints, sometimes from tests themselves, sometimes > from setUpClass, sometimes from tearDownClass. In the latter two cases, it's > the sites ta

Re: Test optimizations (2-5x as fast)

2011-05-18 Thread Erik Rose
> I suspect he was thinking of PostgreSQL's support for template > databases. It skips parsing overhead, so that creating a copy of a > template is roughly disk-bound. Ah yes. I've been away from my old friend Postgres for a few years. :-) -- You received this message because you are subscribed

Re: Test optimizations (2-5x as fast)

2011-05-17 Thread Erik Rose
> Is there a sensible to way "copy" databases in SQL? SQL 2003 introduced CREATE TABLE x LIKE y for cloning the schema of a table. It's supported in MySQL at least. You could then do a bunch of INSERT INTO ... SELECTs if you deferred foreign key checks first. Wait, why do you want to? -- You

Re: Test optimizations (2-5x as fast)

2011-05-17 Thread Erik Rose
On May 17, 2011, at 7:16 AM, Jonas H. wrote: > 3) Hash the SQL generated for setup/fixtures. (step in right before the SQL > is sent to the database) > Advantages: No false-positives, simple > Disadvantages: Does not eliminate the need for SQL generation and fixture > parsing + model creation, s

Re: Test optimizations (2-5x as fast)

2011-05-17 Thread Erik Rose
> I would be very happy to test this against Oracle database to see is how > much patch improves speed since previously running tests against Oracle > has been a real pain specially all db recreate stuff took a long long > time. Great! I'll post again to this thread when the patch is ready. Or, if

Re: Test optimizations (2-5x as fast)

2011-05-17 Thread Erik Rose
> I declare myself bike-shedding. Given the 3 options, I'm: > > +1 on #1 > +0 on #2 > -0 on #3 Heh, I was just going to quietly sit here and do that while everybody else kept mailing. :-) -- You received this message because you are subscribed to the Google Groups "Django developers" group.

Re: Test optimizations (2-5x as fast)

2011-05-16 Thread Erik Rose
> Regarding the signals, basically we have a bunch of post_save type > things, which tend to store aggregate data for certain conditions. > These get populated (in some cases) in our tests, but don't correspond > to a fixture or a model in the same app. Ah, gotcha. So, a couple solutions off the t

Re: Test optimizations (2-5x as fast)

2011-05-16 Thread Erik Rose
Woo, thanks for the constructive suggestions! > Also, one thing I'm quickly noticing (I'm a bit confused why its > setup_class and not setUpClass as well), I was writing to nose's hooks; didn't realize Django used unittest2 now! > but this wont work with > postgres without changing the DELETE co

Re: Test optimizations (2-5x as fast)

2011-05-16 Thread Erik Rose
> How about caching the test databases? The database state could be cached > after model setup (which takes some time if you've got lots of them) + > initial data fixture setup, and after the setup for each test case (fixtures > + setUp() method). > > So, in the best case, no database setup is

Re: Test optimizations (2-5x as fast)

2011-05-16 Thread Erik Rose
Ahoy, Alex! Thanks for the quick response. > 1. Class-level fixture setup > >> This is the one I'm most interested. I did a patch a number of months ago >> to do the fixture parsing, but not DB insertion on a per-class basis. I >> didn't find that to be a big win. However, I'm going to be wo

Test optimizations (2-5x as fast)

2011-05-13 Thread Erik Rose
optimization #1 into core, as well as any accusations of insanity re: #2 and #3. FWIW, everything works great without touching any of the tests on 3 of our Django sites, totaling over 2000 tests. Best regards and wishes for a happy weekend, Erik Rose support.mozilla.com -- You received this me