Re: [tor-dev] Testing in Tor

2012-12-23 Thread Damian Johnson
> ... but running the Stem tests to learn about Tor code would > not be helpful. I have mixed feelings on this. Stem's tests are chiefly to test Stem, but the integration tests are for checking how our library interacts with a live Tor instance. There's a lot of value to Stem for us to prevent bug

Re: [tor-dev] Testing in Tor

2012-12-23 Thread Sean Robinson
On Sat, Dec 22, 2012 at 7:42 PM, Damian Johnson wrote: > > On Sat, Dec 22, 2012 at 6:16 PM, Simon wrote: > > So what's the difference between Stem tests and 'Chutney'? > > Stem is a controller library with integration tests to check its > interaction with a live tor instance. Its tests focus on t

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-22 Thread Simon
On Thu, Dec 20, 2012 at 11:25 AM, Nick Mathewson wrote: > On Wed, Dec 19, 2012 at 8:57 PM, Simon wrote: >> On Wed, Dec 19, 2012 at 4:35 PM, Nick Mathewson wrote: >>> What's your favorite C mocking solution for integrating with existing >>> codebases without much disruption? >> >> This could be w

Re: [tor-dev] Testing in Tor

2012-12-22 Thread Damian Johnson
> Yes :-) I've seen projects which have tests which take nearly 10 hours > to run. However, the longer the tests take to run then the less likely > that developers will run them. Agreed. I run stem's unit tests more often than the integ tests since those have the runtime of around five seconds. Fo

Re: [tor-dev] Testing in Tor

2012-12-22 Thread Simon
On Thu, Dec 20, 2012 at 2:43 PM, Damian Johnson wrote: >>> You want to point at your tor binary, I think, not just the path >>> (i.e. something like "--tor ../tor-2.3.*/src/or/tor") >> >> That did the trick, thanks: > > Oops, I'm sorry about not being clearer about that. No problem. >> Why do th

Re: [tor-dev] Testing in Tor

2012-12-21 Thread Damian Johnson
Hi Simon, I've made some fixes to hopefully avoid confusion about this in the future (admittedly it was a really bad error message). Better sanity checking for the tor command... https://gitweb.torproject.org/stem.git/commitdiff/e22a753058a2b9c61301c62e2d2123ccea046f29 Reporting testing as having

Re: [tor-dev] Testing in Tor

2012-12-20 Thread Damian Johnson
>> You want to point at your tor binary, I think, not just the path >> (i.e. something like "--tor ../tor-2.3.*/src/or/tor") > > That did the trick, thanks: Oops, I'm sorry about not being clearer about that. > Why do the tests take so long to run? I noticed that most of the time > almost no CPU

Re: [tor-dev] Testing in Tor

2012-12-20 Thread Simon
On Thu, Dec 20, 2012 at 12:31 PM, wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > You want to point at your tor binary, I think, not just the path > (i.e. something like "--tor ../tor-2.3.*/src/or/tor") That did the trick, thanks: $ time ./run_tests.py --integ --tor ../tor-0.2.3.25

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-20 Thread Damian Johnson
> It looks like the data directory was created as expected: > > $ find test/data/ > test/data/ > test/data/log > test/data/torrc > > Unfortunately ./run_tests.py --help isn't telling me about e.g. a > verbosity level option to help give more of a clue as to what is going > wrong :-( > > Any more id

Re: [tor-dev] Testing in Tor

2012-12-20 Thread meejah
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 You want to point at your tor binary, I think, not just the path (i.e. something like "--tor ../tor-2.3.*/src/or/tor") If you're liking Python, I'd also love some feedback on txtorcon's API (it has 98% unit-test coverage, but doesn't depend on a runn

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-20 Thread Simon
On Thu, Dec 20, 2012 at 11:48 AM, Damian Johnson wrote: >> Is that a bug that it says 'TESTING PASSED' even though it failed to >> start Tor? Is the permission denied error because I need to be root to >> run the tests? If so, how to change the tests so that being root isn't >> necessary? > > Yea,

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-20 Thread Damian Johnson
> Is that a bug that it says 'TESTING PASSED' even though it failed to > start Tor? Is the permission denied error because I need to be root to > run the tests? If so, how to change the tests so that being root isn't > necessary? Yea, saying 'PASSED' rather than 'FAILED' would be a bug. No, the t

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-20 Thread Nick Mathewson
On Wed, Dec 19, 2012 at 8:57 PM, Simon wrote: > On Wed, Dec 19, 2012 at 4:35 PM, Nick Mathewson wrote: >> What's your favorite C mocking solution for integrating with existing >> codebases without much disruption? > > This could be worth a separate thread. I'm not aware of really good > solutions

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-20 Thread Simon
On Thu, Dec 20, 2012 at 10:50 AM, Damian Johnson wrote: >> So I have a built and make tested tor-0.2.3.25/ folder and stem/ in a >> parallel folder. When I run the command then I get the following: >> >> $ ./run_tests.py --integ >> Unable to start tor, 'tor' does not exists. >> >> How can I run th

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-20 Thread Damian Johnson
> So I have a built and make tested tor-0.2.3.25/ folder and stem/ in a > parallel folder. When I run the command then I get the following: > > $ ./run_tests.py --integ > Unable to start tor, 'tor' does not exists. > > How can I run the tests on the Tor built in the tor-0.2.3.25/ folder? Just run

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-20 Thread Simon
On Wed, Dec 19, 2012 at 6:31 PM, Damian Johnson wrote: >> [...] >>> Maybe there is no automated testing for any Tor projects? At least a >>> quick search on the wiki only found [1] which lists possible ways to >>> test (but was created 7 months ago and apparently not updated since >>> and collect

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-19 Thread Damian Johnson
> [...] >> Maybe there is no automated testing for any Tor projects? At least a >> quick search on the wiki only found [1] which lists possible ways to >> test (but was created 7 months ago and apparently not updated since >> and collecting dust) and [2] discussing a manual test procedure for >> T

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-19 Thread Simon
On Wed, Dec 19, 2012 at 4:35 PM, Nick Mathewson wrote: > On Wed, Dec 19, 2012 at 5:45 PM, Simon wrote: >> On Wed, Dec 19, 2012 at 1:49 PM, Nick Mathewson wrote: >>> On Wed, Dec 19, 2012 at 2:29 PM, Simon wrote: > [...] >>> * Large parts of the codebase have been written in a tightly coupled

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-19 Thread Nick Mathewson
On Wed, Dec 19, 2012 at 5:45 PM, Simon wrote: > On Wed, Dec 19, 2012 at 1:49 PM, Nick Mathewson wrote: >> On Wed, Dec 19, 2012 at 2:29 PM, Simon wrote: [...] >> * Large parts of the codebase have been written in a tightly coupled >> style that needs refactoring before it can be tested without

Re: [tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-19 Thread Simon
On Wed, Dec 19, 2012 at 1:49 PM, Nick Mathewson wrote: > On Wed, Dec 19, 2012 at 2:29 PM, Simon wrote: > [...] >> Tor seems to have good planning compared to most open source projects. >> So I would be interested in hearing why testing is apparently 'falling >> between the cracks'. Why isn't the

[tor-dev] Testing in Tor [was Re: Brainstorming a Tor censorship analysis tool]

2012-12-19 Thread Nick Mathewson
On Wed, Dec 19, 2012 at 2:29 PM, Simon wrote: [...] > Maybe there is no automated testing for any Tor projects? At least a > quick search on the wiki only found [1] which lists possible ways to > test (but was created 7 months ago and apparently not updated since > and collecting dust) and [2] di