[tor-dev] Stem and Nyx bug tracking moved

2019-12-21 Thread Damian Johnson
Hi all. Stem and Nyx's issue tracking has moved from Trac to GitHub. In the future please file tickets at... https://github.com/torproject/stem/issues/ https://github.com/torproject/nyx/issues/ Thanks! PS. All existing tickets have moved. These trac components will be disabled to cut down on con

Re: [tor-dev] Stem ORPort protocol support

2018-02-07 Thread teor
> On 8 Feb 2018, at 07:23, Damian Johnson wrote: > > Hi all. Over the last few months Tim and I have been collaborating on > Python support for the ORPort protocol. With it you can download > descriptors without a DirPort, and possibly fancier things in the > future like full circuit constructio

Re: [tor-dev] Stem ORPort protocol support

2018-02-07 Thread meejah
Damian Johnson writes: > Thanks meejah! Took a peek but they both look pretty old and it's > unclear to me how complete either got. If there's something in > particular you think is worthwhile integrating I'm all ears. I haven't looked closely enough to know the answer to that ;) but as I unders

Re: [tor-dev] Stem ORPort protocol support

2018-02-07 Thread Damian Johnson
> So, there is this already -- not sure how "complete" it is though (and > looks like hasn't seen commits for 2+ years) but might have useful code: Thanks meejah! Took a peek but they both look pretty old and it's unclear to me how complete either got. If there's something in particular you think

Re: [tor-dev] Stem ORPort protocol support

2018-02-07 Thread meejah
Damian Johnson writes: > they'd care to go (potentially all the way up to a Python Tor client, > similar to Orchid). So, there is this already -- not sure how "complete" it is though (and looks like hasn't seen commits for 2+ years) but might have useful code: https://github.com/pycepa/pycep

[tor-dev] Stem ORPort protocol support

2018-02-07 Thread Damian Johnson
Hi all. Over the last few months Tim and I have been collaborating on Python support for the ORPort protocol. With it you can download descriptors without a DirPort, and possibly fancier things in the future like full circuit construction. Tim put together a wonderful proof of concept called Endos

Re: [tor-dev] stem support for v3 ephemeral onion services

2018-01-31 Thread Damian Johnson
Great! Thanks David. Juggling some other things but I'll try to give this a pass in the next week or two. On Wed, Jan 31, 2018 at 12:56 PM, David Goulet wrote: > On 25 Jan (08:10:00), David Goulet wrote: >> On 25 Jan (06:50:30), teor wrote: >> > >> > > On 25 Jan 2018, at 05:14, Micah Lee wrote:

Re: [tor-dev] stem support for v3 ephemeral onion services

2018-01-31 Thread David Goulet
On 25 Jan (08:10:00), David Goulet wrote: > On 25 Jan (06:50:30), teor wrote: > > > > > On 25 Jan 2018, at 05:14, Micah Lee wrote: > > > > > > Now that Tor Browser 7.5 is released and includes the tor 0.3.2 series, > > > which supports next generation onion services, I would love to make > > > O

Re: [tor-dev] stem support for v3 ephemeral onion services

2018-01-25 Thread Micah Lee
On 01/25/2018 05:10 AM, David Goulet wrote: > On 25 Jan (06:50:30), teor wrote: >> >>> On 25 Jan 2018, at 05:14, Micah Lee wrote: >>> >>> Now that Tor Browser 7.5 is released and includes the tor 0.3.2 series, >>> which supports next generation onion services, I would love to make >>> OnionShare u

Re: [tor-dev] stem support for v3 ephemeral onion services

2018-01-25 Thread David Goulet
On 25 Jan (06:50:30), teor wrote: > > > On 25 Jan 2018, at 05:14, Micah Lee wrote: > > > > Now that Tor Browser 7.5 is released and includes the tor 0.3.2 series, > > which supports next generation onion services, I would love to make > > OnionShare use these by default. Here is the issue [1]. >

Re: [tor-dev] stem support for v3 ephemeral onion services

2018-01-24 Thread teor
> On 25 Jan 2018, at 05:14, Micah Lee wrote: > > Now that Tor Browser 7.5 is released and includes the tor 0.3.2 series, > which supports next generation onion services, I would love to make > OnionShare use these by default. Here is the issue [1]. > > OnionShare is written in python3 and relie

[tor-dev] stem support for v3 ephemeral onion services

2018-01-24 Thread Micah Lee
Now that Tor Browser 7.5 is released and includes the tor 0.3.2 series, which supports next generation onion services, I would love to make OnionShare use these by default. Here is the issue [1]. OnionShare is written in python3 and relies on stem to communicate with the Tor controller. Although t

Re: [tor-dev] Stem Release 1.5

2016-11-30 Thread isis agora lovecruft
Damian Johnson transcribed 1.9K bytes: > Damn this was long overdue. I'm delighted to announce Stem 1.5.2, the > accumulation of seventeen months of improvements. > > What is *Stem *, you ask? For those who > aren't familiar with it Stem is a Python library for intera

[tor-dev] Stem Release 1.5

2016-11-22 Thread Damian Johnson
Damn this was long overdue. I'm delighted to announce Stem 1.5.2, the accumulation of seventeen months of improvements. What is *Stem *, you ask? For those who aren't familiar with it Stem is a Python library for interacting with Tor. With it you can script against yo

Re: [tor-dev] stem is now available through Guix

2016-08-16 Thread ng0
Hi, Damian Johnson writes: > Hi ng0, that's neat! Is there a stem-specific guix page to link to? > The general package page is large enough it hosed my browser for quite > a while, and its link is for a multi-thousand line file... > > http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/p

Re: [tor-dev] stem is now available through Guix

2016-08-16 Thread Damian Johnson
Hi ng0, that's neat! Is there a stem-specific guix page to link to? The general package page is large enough it hosed my browser for quite a while, and its link is for a multi-thousand line file... http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/python.scm#n9855 Not the end of the wor

[tor-dev] stem is now available through Guix

2016-08-16 Thread ng0
Hi, stem is now available as python-stem on all systems which are able to run GNU Guix: https://www.gnu.org/software/guix/packages/ "guix package -i python-stem" installs the python3 variant (binary substitute), "guix package -i python2-stem" installs the python2 variant (binary substitute). See

[tor-dev] Stem Release 1.4

2015-05-13 Thread Damian Johnson
Greetings wonderful carbon-based residents of the Internet. I'm pleased to announce the 1.4.0 release of Stem! What is Stem, you ask? For those who aren't familiar with it Stem is a Python library for interacting with Tor. With it you can script against your relay, descriptor data, or even write a

[tor-dev] Stem descriptor lazy loading

2015-01-25 Thread Damian Johnson
I've been wanting to do this for years. Just a quick announcement that Stem now lazy-loads descriptors, giving it a nice performance boost... Server descriptors: 27% faster Extrainfo descriptors: 71% faster Microdescriptors: 43% faster Consensus: 37% faster For details see... https://g

Re: [tor-dev] stem on windows 7

2014-12-31 Thread Sherief Alaa
On 12/31/2014 07:47 PM, Tony Willingham wrote: > Run what exactly from the command line?Currently I'm running the > "Russia" stem example within Eclipse. > > Assuming it is not a stem issue, how do I troubleshoot? You need to tell tor to use some obfs3 bridges. You can do that by editing th

Re: [tor-dev] stem on windows 7

2014-12-31 Thread Tony Willingham
Run what exactly from the command line?Currently I'm running the "Russia" stem example within Eclipse. Assuming it is not a stem issue, how do I troubleshoot? Thanks Tony On Wed, Dec 31, 2014 at 12:29 PM, Damian Johnson wrote: > Hi Tony. If you run tor directly from the commandline does it

Re: [tor-dev] stem on windows 7

2014-12-31 Thread Damian Johnson
Hi Tony. If you run tor directly from the commandline does it finish bootstrapping? This doesn't sound like a Stem issue, but rather that you're unable to finish bootstrapping tor. On Wed, Dec 31, 2014 at 5:41 AM, Tony Willingham wrote: > Any suggestions on how to deal with stem on Windows 7 gett

[tor-dev] stem on windows 7

2014-12-31 Thread Tony Willingham
Any suggestions on how to deal with stem on Windows 7 getting stuck at: [notice] Bootstrapped 45%: Asking for relay descriptors. Occasionally it will make it to: [notice] Bootstrapped 50%: Asking for relay descriptors. If I run Vidalia in client mode only, it works fine. I see in the documentati

[tor-dev] Stem Release 1.3

2014-12-22 Thread Damian Johnson
Greetings wonderful people of the world! After months down in the engine room I'm delighted to announce the 1.3.0 release of Stem. For those who aren't familiar with it, Stem is a Python library for interacting with Tor. With it you can script against your relay, descriptor data, or even write app

[tor-dev] Stem roadmap, and ideas for the interpreter?

2014-11-21 Thread Damian Johnson
Hi Nick. While sipping my morning coffee I just realized that you might not be aware of my sinister long schemes, and that might be to the detriment of us both. The roadmap I have for myself goes something like... Step 1: Write Stem so we have a well tested, stable controller library. (Done [1])

[tor-dev] Stem Release 1.2

2014-06-01 Thread Damian Johnson
Hi all. After months of work I'm please to announce the release of Stem 1.2.0! For those who aren't familiar with it, Stem is a Python library for interacting with Tor. With it you can script against your relay, descriptor data, or even write applications similar to arm and Vidalia. https://ste

[tor-dev] Stem Release 1.1

2013-10-14 Thread Damian Johnson
Hi all. After seven months of work I'm pleased to announce Stem's 1.1.0 release! For those who aren't familiar with it, Stem is a Python library for interacting with Tor. With it you can script against your relay, descriptor data, or even write applications similar to arm and Vidalia. https://ste

Re: [tor-dev] stem

2013-06-10 Thread Damian Johnson
> Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep > off spam, which some of my questions at first may be, essentially Qs. But, > if you think they would be of interest to tor-dev, or others could help, > just let me know, and i will sign up for it. They certainly are!

Re: [tor-dev] Stem Integ tests for Tor

2013-05-01 Thread Nick Mathewson
On Tue, Apr 30, 2013 at 10:29 AM, Ashish Patil wrote: > On Mon, Apr 29, 2013 at 17:44 PM, Nick Mathewson > wrote: >> Right now, all chutney >> does is configure and launch a small Tor network, but the small >> network requires a little babysitting to make it bootstrap. (For >> example, you someti

Re: [tor-dev] Stem Integ tests for Tor

2013-05-01 Thread Damian Johnson
> Hello! > As part of the Stem Integ tests, nickm suggested integrating > gcov into Stem. I did not get that part. Gcov is for code > coverage of C/C++ source code. We could integrate it to Stem, > but then it would involve executing command line statements > on a gcov installed machine. Hi Ashish

[tor-dev] Stem Integ tests for Tor

2013-05-01 Thread Ashish Patil
Hello! As part of the Stem Integ tests, nickm suggested integrating gcov into Stem. I did not get that part. Gcov is for code coverage of C/C++ source code. We could integrate it to Stem, but then it would involve executing command line statements on a gcov installed machine. Also, we could add si

Re: [tor-dev] Stem Integ tests for Tor

2013-04-30 Thread Ashish Patil
On Mon, Apr 29, 2013 at 17:44 PM, Nick Mathewson wrote: > Right now, all chutney > does is configure and launch a small Tor network, but the small > network requires a little babysitting to make it bootstrap. (For > example, you sometimes need to send the clients a SIGHUP once the > authorities ha

Re: [tor-dev] Stem Integ tests for Tor

2013-04-29 Thread Nick Mathewson
On Sun, Apr 28, 2013 at 11:00 PM, Damian Johnson wrote: > 3. Once we have a decently reliable set of tests to check basic tor > functionality move on to chutney. Nick would be able to best advise on > the work needed there. What we'd need most for Chutney would be to make it a bit more robust, an

Re: [tor-dev] Stem Integ tests for Tor

2013-04-29 Thread Ashish Patil
Hi! Thanks atagar, your reply cleared up a lot of things. I am going through the present integ tests in Stem & would contact you if I have any doubts. As for chutney, I suppose nickm's copy of chutney would be the one with most updates (if different from the default copy). I will go through that a

Re: [tor-dev] Stem GSoC 2013

2013-04-28 Thread Damian Johnson
> Interesting idea, after some light tinkering today I managed to run it with > stem onboard, I'm unsure how much patching this will need (or rather how > much can be done), but I'm on it. Thanks! Personally I'm not at all familiar with IronPython - what kind of modifications do CPython apps usual

Re: [tor-dev] Stem Integ tests for Tor

2013-04-28 Thread Damian Johnson
Hi Asis, glad you want to improve our testing of tor! That's one area that could certainly use some love. ;) > I would like to improve on Tor's codebase by writing Integ tests for it > in Stem. I would like to know what kind of tests should be included > in this. Also, atagar was talking about smo

Re: [tor-dev] Stem GSoC 2013

2013-04-28 Thread tomasz . kunikowski
Hey, > talked quite a bit about iron python for stem. Considering that you > have some windows experience maybe a tutorial for making stem work > with iron python and/or Visual Studio would be a good fit? I'm adding > Lee to the cc in case he has some thoughts. Interesting idea, after some light

[tor-dev] Stem Integ tests for Tor

2013-04-28 Thread Ashish Patil
Hello everyone! I would like to improve on Tor's codebase by writing Integ tests for it in Stem. I would like to know what kind of tests should be included in this. Also, atagar was talking about smoke tests for Tor as a whole. I would love to know everyone's ideas on the same. Also, I would like t

Re: [tor-dev] Stem GSoC 2013

2013-04-27 Thread Damian Johnson
> Speaking of connections.py, if you improved it during the port to stem > to also support Window's netstat then that would greatly help in > porting arm to windows... > > https://trac.torproject.org/projects/tor/wiki/doc/arm#Windows Oh. Speaking of connection.py improvements, that module presentl

Re: [tor-dev] Stem GSoC 2013

2013-04-27 Thread Damian Johnson
> connections.py looks interesting, doing *nix specific tasks is nice for a > change, > since I'm dealing with win32api much more often. I just finished attending Linux Fest Northwest during which Lee and I talked quite a bit about iron python for stem. Considering that you have some windows expe

Re: [tor-dev] Stem GSoC 2013

2013-04-27 Thread tomasz . kunikowski
Hey Damian, (I hope I got replying to digest right this time around). Thanks for all the feedback and assistance so far, it's an immense help. > The right way is to add this capability to tor, and the hacky way is > to construct a normal circuit, truncate the exit, then replace it. The closest

Re: [tor-dev] Stem GSoC 2013

2013-04-26 Thread Damian Johnson
Hi Tomasz. I gave this some more thought throughout the day and here's a few other tasks that would be nicely suited for GSoC. Together with some tutorial and test expansions this should make for a pretty full summer. Honestly expanding the tor controller interface (and the corresponding stem chan

Re: [tor-dev] Stem GSoC 2013

2013-04-26 Thread Damian Johnson
> As far as tutorials go, I was thinking about dealing with manual stream > attachment (as a follow up to #8728), > exploring more operations on circuits i.e. tearing down, forcing hops > through specified > countries or address ranges (maybe a more robust example as an excuse to > come up with goo

Re: [tor-dev] Stem GSoC 2013

2013-04-26 Thread Tomasz Kunikowski
Hey, I favor GSoC projects that are collections of small items rather than a single big one (that way the incremental pieces are helpful, even if the project isn't quite fit into the summer). > We're on the same page here, I think more tasks of a smaller scope is the way to go. The next area

Re: [tor-dev] Stem GSoC 2013

2013-04-23 Thread Damian Johnson
Hi Tom. Sorry for not getting back to you earlier, been juggling lots of email. Glad you want to get involved with stem and *many* thanks for the patches so far! They've been a big help. > I'm mainly interested in expanding stem's tutorial base as it could really > use some more real-world applica

[tor-dev] Stem GSoC 2013

2013-04-22 Thread Tomasz Kunikowski
Hey, My name is Tomasz Kunikowski, I'm a production engineering student at Wroclaw University of Economics branching heavily into IT. My handle on IRC and trac is ragwater. I'm mainly interested in expanding stem's tutorial base as it could really use some more real-world applications and example

[tor-dev] Stem Release 1.0

2013-03-30 Thread Damian Johnson
Hi all. After eighteen months of work and a number of delays rivaling that of the Big Dig I'm pleased to announce the initial release of stem! For those who aren't familiar with it, stem is a python controller library for tor. With it you can write scripts and applications that interact with your

Re: [tor-dev] Stem Tutorials

2013-03-19 Thread Damian Johnson
Hi Lunar, thanks for the feedback! > * I wonder if the use of the fancy functools.partial should not be >replaced by a two lines 'def'. YMMV. Hmm, I'm not sure which is more confusing: scoping rules of an inline function or bundling references via a partial. Personally I find the partial to

Re: [tor-dev] Stem Tutorials

2013-03-19 Thread Jérémy Bobbio
Damian Johnson: > Hi all. Stem is having its initial release later this month, and in > anticipation of it I've been expanding its tutorials... > > https://stem.torproject.org/tutorial.html > > Are there any rough spots in these tutorials? In Tortoise and the Hare: * I wonder if the use of the

Re: [tor-dev] Stem Tutorials

2013-03-11 Thread Abel Luck
Damian Johnson: > Hi all. Stem is having its initial release later this month, and in > anticipation of it I've been expanding its tutorials... > > https://stem.torproject.org/tutorial.html > > Are there any rough spots in these tutorials? Other categories that we > should cover? > > Suggestions

[tor-dev] Stem Tutorials

2013-03-10 Thread Damian Johnson
Hi all. Stem is having its initial release later this month, and in anticipation of it I've been expanding its tutorials... https://stem.torproject.org/tutorial.html Are there any rough spots in these tutorials? Other categories that we should cover? Suggestions welcome! -Damian

Re: [tor-dev] Stem code review 2013-01-03

2013-01-05 Thread Damian Johnson
Hi Sean. As always, thanks for the code review! > I did find the 'assert extended == "EXTENDED"' test after is_ok() to be > strange... Oops, thanks for catching that. I don't mind a bit of paranoia in making sure that the response is giving us a circuit id but I've avoided using the assert keywo

[tor-dev] Stem code review 2013-01-03

2013-01-05 Thread Sean Robinson
Stem devs, This is a review of commits made to Stem from 2012-12-22 through 2013-01-03. Happy new year. I'm a fan of the new Controller.get_circuits() method. It's a nice and clever re-use of internal code. And it makes circuit-oriented method tests much more readable. Controller.get_circuit(

Re: [tor-dev] Stem code review 2012-12-21

2012-12-22 Thread Damian Johnson
Hi Sean, thanks for the code review! > Would GETINFO/GETCONF cache hit logging[1] benefit from log_once? Or even > something between log and log_once (e.g. rate_limited_log)? Interesting thought, but I don't think so. When I look for cache hits it's usually to answer the question of 'did doing

[tor-dev] Stem code review 2012-12-21

2012-12-22 Thread Sean Robinson
Stem devs, This is a review of Stem commits from 2012-12-11 through 2012-12-21. Damian, you are very good at documentation, concise and informative. I tend to hit one or the other, but not both. I recognize when someone gets the right mix. Good work. I, too, have found logging.basicConfig[0]

Re: [tor-dev] Stem code review 2012-12-10

2012-12-14 Thread Damian Johnson
Hi Sean, thanks for the code review! > As to re-attaching event listeners[1], I agree that putting a specialized > hook into BaseController.msg seems bad. I have an alternate idea[2] that > puts the re-attachment in an authenticate method. I am not proposing this > as the solution, but I hope th

[tor-dev] Stem code review 2012-12-10

2012-12-14 Thread Sean Robinson
Stem devs, This is a review of recent commits to Stem. It begins where my last review ended[0] and finishes at the "Adding a close_stream..." merge. The pydoc changes to Controller.extend_circuit are good additions. I do not understand much of the context for changes regarding network status do

Re: [tor-dev] Stem code review 2012-12-04

2012-12-08 Thread Damian Johnson
> I must not be reasoning about Event._parse_standard_attr() correctly. I > think it is already looking for _QUOTED positional args, but is working at > it backwards from _KEYWORD_ARGS parsing. Right. The thing that I'm talking about is *new* quoted arguments (ie, things not presently in the spec

Re: [tor-dev] Stem code review 2012-12-04

2012-12-08 Thread Sean Robinson
On Fri, Dec 7, 2012 at 8:01 PM, Damian Johnson wrote: > > The quoted key/value mapping is more readable, now. Good work. Why not > look for quoted positional args before non-quoted positional args? Why not > do just like kwarg handling? > MY_EVENT "arg1 arg2" > > ... where the spec says we rea

Re: [tor-dev] Stem code review 2012-12-04

2012-12-07 Thread Damian Johnson
Hi Sean. Thanks for code reviewing my recent commits! > 1) I do not like the new _get_event() with assert_class and assert_content. > There are transformations and tests and returned values all within what is a > mock object builder, meaning it works via side-effect. This could be > surprisin

Re: [tor-dev] stem mock_method usage

2012-12-07 Thread Damian Johnson
Hi Sean. You're using the mock_method() function correctly... class Foo: def greeting(self): return 'hi' f = Foo() print f.greeting() mocking.mock_method(Foo, 'greeting', mocking.return_value('bye')) print f.greeting() ===

[tor-dev] Stem code review 2012-12-04

2012-12-07 Thread Sean Robinson
Stem devs, I did a code review of recent Stem commits ("Tor event handling" merge (commit 42872dd08e81d6b3) through "Checking for None by identity" (commit 69f72efc9367092c)). My comments and questions follow. I will skip my own contributions (they were great 8-) in that range, as I am biased.

[tor-dev] stem mock_method usage

2012-12-07 Thread Sean Robinson
Damian, I am attempting to write a test using test.mocking.mock_method, but I do not understand how to use it correctly. Could you give me pointers on the following smallest (non-)working test case. import stem from stem.control import Controller import test.mocking as mocking socket = stem.soc

Re: [tor-dev] Stem Descriptor Parsers

2012-07-10 Thread Damian Johnson
> After looking at possible use cases, wouldn't it make sense to allow the > caller to specify a file to be written to? Make sense, though via a convenience method. Libraries should provide basic building blocks (such as 'give me the csv string for these descriptors') in addition to less flexible

Re: [tor-dev] Stem Descriptor Parsers

2012-07-10 Thread Megan Chang
Hi Damian, After looking at possible use cases, wouldn't it make sense to allow the caller to specify a file to be written to? Regardless, we were thinking of creating two methods, one that takes a list of descriptors, and one that takes a single descriptor. This would remove the need to check fo

Re: [tor-dev] Stem Descriptor Parsers

2012-07-09 Thread Damian Johnson
On Mon, Jul 9, 2012 at 8:40 AM, Erik I Islo wrote: > Hello, > > Megan and I have been working on the CSV export functionality that was being > discussed a little over a week ago, and given the recent discussion, we > would like to clarify the expected/desired implementation of this feature. > > We

Re: [tor-dev] Stem Descriptor Parsers

2012-07-09 Thread Erik I Islo
Hello, Megan and I have been working on the CSV export functionality that was being discussed a little over a week ago, and given the recent discussion, we would like to clarify the expected/desired implementation of this feature. We have created an export.py module within /stem/descriptor, which

Re: [tor-dev] Stem Descriptor Parsers

2012-07-06 Thread Damian Johnson
> So is export intended to be an instance method of descriptor, one that just > dumps a single csv line of the instance attributes (maybe subject to some > selection of those attributes)? Or a static method that takes a collection? Either would work fine. I was envisioning the former, though on

Re: [tor-dev] Stem Descriptor Parsers

2012-07-06 Thread Norman Danner
Yes, I was wondering whether there would be something simpler than Django after I wrote that message. Megan and Erik: take a look through the websites for Django, Tornado, and Cyclone/Twisted to get a sense as to what each does. - Norman On 7/6/12 10:34 AM, Sathyanarayanan Gunasekar

Re: [tor-dev] Stem Descriptor Parsers

2012-07-06 Thread Sathyanarayanan Gunasekaran
> OK; Megan and Erik, after you incorporate the export function into > Descriptor in stem, please start reading through the Django tutorial. I'm not sure if Django is a good choice for this project. We don't require such a heavy web framework with a templating engine, auth, etc. I'd rather use Tor

Re: [tor-dev] Stem Descriptor Parsers

2012-07-06 Thread Norman Danner
On 7/6/12 4:10 AM, Karsten Loesing wrote: On Fri, Jul 6, 2012 at 4:25 AM, Norman Danner wrote: Do I understand Onionoo correctly to be basically a small webservice that returns a JSON formatted description of data read from a file based on the HTTP request parameters, along with a program that

Re: [tor-dev] Stem Descriptor Parsers

2012-07-06 Thread Karsten Loesing
On Fri, Jul 6, 2012 at 4:25 AM, Norman Danner wrote: > Do I understand Onionoo correctly to be basically a small webservice that > returns a JSON formatted description of data read from a file based on the > HTTP request parameters, along with a program that presumably runs with some > frequency t

Re: [tor-dev] Stem Descriptor Parsers

2012-07-05 Thread Norman Danner
OK, bringing back to tor-dev... On 7/5/12 1:44 PM, Damian Johnson wrote: Hi Norman. (Taking this off tor-dev@ for the moment until we get things straightened out...) Actually, this would have been an interesting discussion for the list. Feel free to add it back in. The TorExport functiona

Re: [tor-dev] Stem Descriptor Parsers

2012-07-03 Thread Damian Johnson
>> Which descriptor parsers are you referring to, as some are already >> implemented? A lot of what we're working on in Tor Export overlaps your work >> regarding descriptor parsers. > > According to my proposal, I was to implement the parsers for V3 > network status documents and microdescriptor d

Re: [tor-dev] Stem Descriptor Parsers

2012-07-02 Thread Ravi Chandra Padmala
On Mon, Jul 2, 2012 at 9:24 PM, Megan Chang wrote: > Which descriptor parsers are you referring to, as some are already > implemented? A lot of what we're working on in Tor Export overlaps your work > regarding descriptor parsers. According to my proposal, I was to implement the parsers for V3 net

Re: [tor-dev] Stem Proc Integration Tests

2012-06-29 Thread Damian Johnson
> Should we wait for Ravi to implement this, or can we go ahead and do it > ourselves as needed? Feel free, Ravi will be busy with other controller functionality for quite a while. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torpr

Re: [tor-dev] Stem Proc Integration Tests

2012-06-29 Thread Norman Danner
(Sorry, wasn't subscribed to tor-dev@, so these last couple of messages didn't go there...) On 6/29/12 4:09 PM, Damian Johnson wrote: I think the question here is that it seems like one should be able to get all the descriptors from the running Tor process via GETINFO desc/all-recent

Re: [tor-dev] Stem Proc Integration Tests

2012-06-29 Thread Damian Johnson
> Keep in mind that metrics tarballs can be huge. stem's tests probably > shouldn't download one or more of these tarballs in an automatic integ > test run. Oops yup. Should have mentioned that. We're just picking out a descriptor that seems to exercise most of the parsing. This is just for a san

Re: [tor-dev] Stem Proc Integration Tests

2012-06-29 Thread Michele Orrù
2012/6/28 Damian Johnson : >> Erik and I have finished the proc.py integration tests > > Great! I'm looking forward to checking them out. > >> Tomorrow, we will be starting the next Stem task regarding the Tor Export >> project. Do you have any recommendations on where we should start exploring >

Re: [tor-dev] Stem Proc Integration Tests

2012-06-29 Thread Karsten Loesing
On 6/28/12 11:37 PM, Damian Johnson wrote: > - integ tests are pretty short, and just run the parser against some > test data from the metrics archive and the cached consensus Keep in mind that metrics tarballs can be huge. stem's tests probably shouldn't download one or more of these tarballs

Re: [tor-dev] Stem Proc Integration Tests

2012-06-28 Thread Damian Johnson
> Erik and I have finished the proc.py integration tests Great! I'm looking forward to checking them out. > Tomorrow, we will be starting the next Stem task regarding the Tor Export > project. Do you have any recommendations on where we should start exploring > in the Stem documentation? Hmm,

Re: [tor-dev] stem test output

2012-06-27 Thread Karsten Loesing
On 6/25/12 3:45 AM, Damian Johnson wrote: > Karsten, should the 'dirreq-v3-share' percentage values be able to go > above 100%? The extra-info descriptors that I just fetched has one > such entry, which makes stem's descriptor parser complain... > > extra-info siltornado 995D0FE5A89563D79A383CCC24

Re: [tor-dev] stem test output

2012-06-24 Thread Damian Johnson
Hi Karsten, hi Sathyanarayanan. Over the last couple weeks I've been making a variety of fixes to stem and its tests. It now runs with python 2.5 and doesn't require tor to be in your path, among other things. However, the hanging issue remains and I'm pretty stuck there. It isn't strictly stem re

Re: [tor-dev] Stem Testing Mocking Issue

2012-06-19 Thread Damian Johnson
> The way the os module seems to reference posix, I don't believe we will run > into any platform dependencies... Ahhh, gotcha. My understanding of the usage was backwards (I thought that you planned to provide posix as the new argument). In that case looks good to merge, will do tomorrow morning

Re: [tor-dev] Stem Testing Mocking Issue

2012-06-19 Thread Erik I Islo
The way the os module seems to reference posix, I don't believe we will run into any platform dependencies. Since os determines what environment it is in then references either itself or an appropriate external module (such as posix) in __dict__, it should always work. With os.readlink, the issue

Re: [tor-dev] Stem Testing Mocking Issue

2012-06-19 Thread Damian Johnson
Hi Eric. > First, to clarify our github repository situation... gotcha > Next, in continuing work on the unit tests for proc.py, we ran into another > issue with the mocking code. Nice catch, though for your example (os.readlink) won't this make the tests platform dependent? Currently Beck (an

Re: [tor-dev] Stem Testing Mocking Issue

2012-06-18 Thread Erik I Islo
Hi Damian, First, to clarify our github repository situation, both Megan and I will be maintaining remote repos in github per Professor Danner's request so that we each gain experience with version control systems. As we have been working so far, I have been maintaining the mocking revisions in m

Re: [tor-dev] Stem Testing Mocking Issue

2012-06-15 Thread Damian Johnson
> The changes look great! I see no issues with merging to the master branch. Great, pushed. > Sorry about the reStructuredText inconvenience. I actually didn't even think > of it -- I just meant to clean up the comments a bit, but I'll definitely be > more careful in the future. No worries. Ag

Re: [tor-dev] STEM: Tor2csv / Tor2xml / Tor2json ?

2012-06-15 Thread Damian Johnson
We discussed this on irc. I like the idea of providing csv import/export functionality and Fabio is filing a ticket for it. This would be a nice project for one of the new people hacking on stem, though I'm happy to do it if someone has a pressing need for it. I'm less interested in xml though if

[tor-dev] STEM: Tor2csv / Tor2xml / Tor2json ?

2012-06-15 Thread Fabio Pietrosanti (naif)
Hi, i just would like to provide a suggestion for Stem use (maybe already done), now that it has a powerful cached-consensus/descriptors parsers. Would it possible to provide easy to use command line tools to access Tor's data in the following formats: - csv - xml - json So that anyone requirin

Re: [tor-dev] Stem Testing Mocking Issue

2012-06-14 Thread Erik I Islo
The changes look great! I see no issues with merging to the master branch. Sorry about the reStructuredText inconvenience. I actually didn't even think of it -- I just meant to clean up the comments a bit, but I'll definitely be more careful in the future. - Erik & Megan On Thu, Jun 14, 2012 at 4

Re: [tor-dev] Stem Testing Mocking Issue

2012-06-14 Thread Damian Johnson
> We also updated your adaptation to our patch so that no code is repeated. > This should make the function cleaner and more readable. This new code can > be found at: > https://github.com/jacthinman/Tor-Stem/blob/mocking/test/mocking.py Ah ha, makes much more sense now - thanks. After some refle

Re: [tor-dev] Stem Testing Mocking Issue

2012-06-13 Thread Erik I Islo
Hi Damian, To answer your first question, we ran into the trouble when mocking time.time(). This came up for us, as type(time.time) is 'builtin_function_or_method', which is the same as type(open) -> 'builtin_function_or_method'. We also updated your adaptation to our patch so that no code is re

Re: [tor-dev] Stem Testing Mocking Issue

2012-06-12 Thread Damian Johnson
Hi Megan, thanks for the patch! What is an example of a standard library function with a builtin type? I'd like to exercise the use case that has been causing you trouble. This change has a couple issues, for instance it treats anything with the same name as a builtin like a builtin. It also calls

Re: [tor-dev] Stem Sphinx Documentation

2012-06-10 Thread Damian Johnson
Pushed the suggested changes... https://gitweb.torproject.org/stem.git/commitdiff/7947d69 https://gitweb.torproject.org/stem.git/commitdiff/737018b https://gitweb.torproject.org/stem.git/commitdiff/860cd87 https://gitweb.torproject.org/stem.git/commitdiff/9654beb Thanks! -Damian __

Re: [tor-dev] Stem Testing Code Question - Wesleyan Interns

2012-06-08 Thread Damian Johnson
Hi Erik. Glad to hear that you're digging into the code. When you find confusing bits feel free to revise the comments (if you find it to be confusing then it's quite likely others will too - admittedly the dance with the settings.cfg isn't the most straight forward code). > In run_tests.py, under

[tor-dev] Stem Testing Code Question - Wesleyan Interns

2012-06-08 Thread Erik I Islo
Hi Damian, Having gotten started looking at some of the testing code from Stem over the last day or so, Megan and I have come up with a question. While both of us are currently on #tor and #tor-dev (as ErikI and MeganC), we figured an email would be best for getting started as we have a bit more

Re: [tor-dev] Stem Sphinx Documentation

2012-06-07 Thread Karsten Loesing
On 6/6/12 7:32 PM, Damian Johnson wrote: >> - Why does digest() return the base64-encoded digest, not the >> hex-formatted one? Network statuses are the only documents in Tor using >> base64 (or rather, a variant of it without trailing ='s), so it's easier >> to convert those to hex than to conver

Re: [tor-dev] Stem Sphinx Documentation

2012-06-06 Thread Damian Johnson
Thanks, Karsten. > - Does ExtraInfoDescriptor support bridge descriptors yet? Those don't > contain a signature, which means that the signature variable shouldn't > be marked as required. Also, there should be a digest() method for > RelayExtraInfoDescriptor and a digest variable for > BridgeExt

Re: [tor-dev] Stem Sphinx Documentation

2012-06-06 Thread Karsten Loesing
On 6/6/12 6:48 AM, Damian Johnson wrote: > Hi Ravi, Beck, and everyone else hacking on stem. I just finished and > merged a rewrite of our documentation into reStructuredText. The > results are... very pretty. Looks really cool! > Karsten > http://www.atagar.com/transfer/tmp/stem_html_12_06_05/st

  1   2   >