Hi Karsten, hi Philipp, added these benchmarks to our site...
https://stem.torproject.org/tutorials/mirror_mirror_on_the_wall.html#are-there-any-other-parsing-libraries
Cheers! -Damian
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.
Yikes, thanks for getting these Karsten! I don't think we should omit
the earlier results since the python community is still very much
split between 2.7 and 3.x. I'll include both so users know they can
upgrade their interpreter to get a nice little speed boost.
Thanks!
On Fri, Jan 15, 2016 at
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 14/01/16 17:22, Damian Johnson wrote:
> Oh, forgot to talk about compression. You can run the stem script
> against compressed tarballs but python didn't add lzma support
> until python 3.3...
>
> https://stem.torproject.org/faq.html#how-do-i-read
> What you could try, though, is extend Zoossh to parse tarballs rather
> than directories. This is more than 2 times faster in metrics-lib,
> and it doesn't clutter your hard disk with thousands or millions of
> tiny files.
For what it's worth processing tarballs rather than flat files made a
hu
Oh, forgot to talk about compression. You can run the stem script
against compressed tarballs but python didn't add lzma support until
python 3.3...
https://stem.torproject.org/faq.html#how-do-i-read-tar-xz-descriptor-archives
I suppose we could run over bz2 or gz tarballs, or upgrade python. But
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 13/01/16 21:01, Philipp Winter wrote:
> On Wed, Jan 13, 2016 at 05:47:03PM +0100, Karsten Loesing wrote:
>> Do the Zoossh results there look plausible?
>
> I'm surprised that descriptor parsing is so slow, but I think the
> results are plausible,
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 13/01/16 20:42, Damian Johnson wrote:
>> This was Stem commit c01a9cda4e7699c7f4bd642c8e81ed45aab7a29b
>> and Python version 2.7.10.
>
> Great, thanks! Also what was the metrics-lib and zoossh commits?
metrics-lib: 8767f3e3bb8f6c9aa8cdb4c9fb0e9f2b
On Wed, Jan 13, 2016 at 05:47:03PM +0100, Karsten Loesing wrote:
> Do the Zoossh results there look plausible?
I'm surprised that descriptor parsing is so slow, but I think the
results are plausible, yes. I should look into it.
Thanks,
Philipp
___
tor-
> This was Stem commit c01a9cda4e7699c7f4bd642c8e81ed45aab7a29b and
> Python version 2.7.10.
Great, thanks! Also what was the metrics-lib and zoossh commits?
> Or should we add these performance tests for metrics-lib, Stem, and
> Zoossh to their own repository that also comes with scripts to fetc
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 12/01/16 17:19, Damian Johnson wrote:
> Thanks! Yup, those results look reasonable. I was expecting a
> smaller delta with server/extrainfo descriptors and larger one
> with microdescriptors due to the lazy loading but oh well. What
> stem commit an
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 13/01/16 16:28, Philipp Winter wrote:
> On Tue, Jan 12, 2016 at 09:40:35AM +0100, Karsten Loesing wrote:
>> Philipp, would you be able to write the Zoossh counterpart for
>> the descriptor types supported by it?
>
> I attached a small tool that sho
On Tue, Jan 12, 2016 at 09:40:35AM +0100, Karsten Loesing wrote:
> Philipp, would you be able to write the Zoossh counterpart for the
> descriptor types supported by it?
I attached a small tool that should do the same thing Damian's script
does for consensuses and server descriptors. Note, howeve
On Tue, Jan 12, 2016 at 09:40:35AM +0100, Karsten Loesing wrote:
> Philipp, would you be able to write the Zoossh counterpart for the
> descriptor types supported by it? I'm even more curious now how those
> numbers compare to metrics-lib and Stem.
I'd love to, but I cannot promise when I'll be d
Thanks! Yup, those results look reasonable. I was expecting a smaller
delta with server/extrainfo descriptors and larger one with
microdescriptors due to the lazy loading but oh well. What stem commit
and python version was this with?
Any thoughts on when you'll have time to clean up the metrics-l
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 12/01/16 05:38, Damian Johnson wrote:
> Hi Karsten, implemented Stem counterparts of these (see attached).
> On one hand the code is delightfully simple, but on the other
> measurements I got were quite a bit slower. Curious to see what
> you get w
Hi Karsten, implemented Stem counterparts of these (see attached). On
one hand the code is delightfully simple, but on the other
measurements I got were quite a bit slower. Curious to see what you
get when running at the same place you took your measurements.
Cheers! -Damian
On Thu, Jan 7, 2016
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 03/01/16 21:25, Damian Johnson wrote:
> Nice! Few questions...
>
> * Where are your metrics-lib scripts used for the benchmarks?
> Should be easy for me to write stem counterparts once I know what
> we're running. I'll later be including our demo s
Nice! Few questions...
* Where are your metrics-lib scripts used for the benchmarks? Should
be easy for me to write stem counterparts once I know what we're
running. I'll later be including our demo scripts with the benchmarks
later so if possible comments would be nice so they're good examples
fo
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi Damian,
I'm digging out this old thread, because I think it's still relevant.
I started writing some performance evaluations for metrics-lib and got
some early results. All examples read a monthly tarball from
CollecTor and do something trivial w
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 18/10/15 23:50, Damian Johnson wrote:
>> Damian and I sat down yesterday at the dev meeting to talk about
>> doing a comparison of the various descriptor-parsing libraries
>> with respect to capabilities, run-time performance, memory usage,
>> etc.
> The only thing that's wrong is that zoossh can detect types by looking
> at @type.
Thanks! Fix pushed, it'll show up in a few minutes.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On Sun, Oct 18, 2015 at 02:50:47PM -0700, Damian Johnson wrote:
> > Damian and I sat down yesterday at the dev meeting to talk about doing
> > a comparison of the various descriptor-parsing libraries with respect
> > to capabilities, run-time performance, memory usage, etc.
>
> Hi Karsten, started
> Damian and I sat down yesterday at the dev meeting to talk about doing
> a comparison of the various descriptor-parsing libraries with respect
> to capabilities, run-time performance, memory usage, etc.
Hi Karsten, started moving this forward with the easy bit: a table
comparing capabilities. Mi
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hello Philipp and iwakeh, hello list,
Damian and I sat down yesterday at the dev meeting to talk about doing
a comparison of the various descriptor-parsing libraries with respect
to capabilities, run-time performance, memory usage, etc.
We put togeth
24 matches
Mail list logo
