Re: [tor-dev] Visualization of the lizard Sybil attack

[David Fifield]

On Thu, Jan 08, 2015 at 10:52:17AM -0800, Arlo Breault wrote:
> It seems like the dark (tangent-like) line running through the middle
> of the uptimes (not the conspicuous vertical stripe you describe) is
> just a consequence of the sorting.

Yes, how it works is we first sort by center of mass, then by total
uptime. The center of mass moves from left to right as you look from top
to bottom.

It turns out that there are many descriptors that are only up for one or
two hours out of an entire month. Those are what make the dark stripe.

The big surprise is how many descriptors there are. In November, there
were 13,242 unique descriptors seen, more than double the steady-state
total number of relays. Many relays appear for a short time and then
disappear forever. The black band at the center consists of the actual
stable relays. Above and below that is a lot of churn.

> Are the vertical white lines periods where a consensus wasn’t reached?

Yes, there are some missing hours in the input files.

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Is it time to drop support for the v1/v2 protos?

[David Fifield]

On Mon, Jan 12, 2015 at 04:25:56PM +0100, Philipp Winter wrote:
> On Sat, Dec 27, 2014 at 03:38:28PM +0100, Tom van der Woerdt wrote:
> > After reading the Tor spec [1] I did some digging and realized that
> > the old handshakes and link protocols (v1 (certs up-front) and v2
> > (renegotiation)) are not used anymore as of 0.2.3.6-alpha which
> > introduced link proto v3.
> > 
> > Supporting v1 and v2 requires (among other things) supporting SSLv3
> > which (imho) should be deprecated everywhere.
> 
> I was curious about how many relays and clients still want to speak
> version 1 and 2.  I patched one of my guard relays to keep statistics
> about the content of VERSIONS cells.  Here's the result after almost
> three days.  The numbers include relays as well as clients.
> 
>   Versions |  Amount total | Amount w/o duplicate hosts
>   -+---+---
>1 and 2 |  34,648  (9%) | 21,552 (23%)
>  3 |  73,202 (18%) | 54,307 (59%)
>3 and 4 | 291,807 (73%) | 16,235 (18%)
>  4 |   3  (0%) |  2  (0%)

Does "1 and 2" contradict section 4.1 or tor-spec.txt?

Since the version 1 link protocol does not use the
"renegotiation" handshake, implementations MUST NOT list version
1 in their VERSIONS cell.

Is it a bug that tor is including 1 in its VERSIONS cell?

I found this nice log message in the source code:
  } else if (highest_supported_version == 1) {
/* Negotiating version 1 makes no sense, since version 1 has no VERSIONS
 * cells. */
log_fn(LOG_PROTOCOL_WARN, LD_OR,
   "Used version negotiation protocol to negotiate a v1 connection. "
   "That's crazily non-compliant. Closing connection.");
connection_or_close_for_error(chan->conn, 0);
return;
  }

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Is it time to drop support for the v1/v2 protos?

[David Fifield]

On Mon, Jan 12, 2015 at 06:26:14PM +0100, Tom van der Woerdt wrote:
> > On 12 Jan 2015, at 16:25, Philipp Winter  wrote:
> >  Versions |  Amount total | Amount w/o duplicate hosts
> >  -+---+---
> >   1 and 2 |  34,648  (9%) | 21,552 (23%)
> 
> We debugged this last week on IRC, as 1,2 is an invalid combination
> according to the specification. After correlating the ip addresses, we
> concluded that this is GFW scanning and not actual client usage.

I'm sure some of the 1+2 is GFW scanning, but probably not all of it.
Mainstream tor definitely sends 1+2 when using a v2 handshake.

https://gitweb.torproject.org/tor.git/tree/src/or/connection_or.c?id=b0c32106b3559b4ee9fabfb1a49e2e328c850305#n2122

/** Array of recognized link protocol versions. */
static const uint16_t or_protocol_versions[] = { 1, 2, 3, 4 };
/** Number of versions in or_protocol_versions. */
static const int n_or_protocol_versions =
  (int)( sizeof(or_protocol_versions)/sizeof(uint16_t) );

/** Send a VERSIONS cell on conn, telling the other host about the
 * link protocol versions that this Tor can support.
 *
 * If v3_plus, this is part of a V3 protocol handshake, so only
 * allow protocol version v3 or later.  If not v3_plus, this is
 * not part of a v3 protocol handshake, so don't allow protocol v3 or
 * later.
 **/
int
connection_or_send_versions(or_connection_t *conn, int v3_plus)
{
  var_cell_t *cell;
  int i;
  int n_versions = 0;
  const int min_version = v3_plus ? 3 : 0;
  const int max_version = v3_plus ? UINT16_MAX : 2;
  tor_assert(conn->handshake_state &&
 !conn->handshake_state->sent_versions_at);
  cell = var_cell_new(n_or_protocol_versions * 2);
  cell->command = CELL_VERSIONS;
  for (i = 0; i < n_or_protocol_versions; ++i) {
uint16_t v = or_protocol_versions[i];
if (v < min_version || v > max_version)
  continue;
set_uint16(cell->payload+(2*n_versions), htons(v));
++n_versions;
  }
  cell->payload_len = n_versions * 2;

  connection_or_write_var_cell_to_buf(cell, conn);
  conn->handshake_state->sent_versions_at = time(NULL);

  var_cell_free(cell);
  return 0;
}

> Are you sure you are deduplicating correctly? That's a lot of hosts.

Even if it were only GFW probing, GFW rarely uses duplicate IPs, except
for a few. Most IPs you will only see once or twice over the course of
months.

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Pluggable transports meeting tomorrow (16:00 UTC Wednesday 14th of January 2015)

[David Fifield]

Just wanted to remind you that the regular biweekly pluggable transports
meeting is going to occur tomorrow at 16:00 UTC. Place is the #tor-dev
IRC channel in the OFTC network.

https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports#PluggableTransportIRCmeetings

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Is it time to drop support for the v1/v2 protos?

[David Fifield]

On Thu, Jan 15, 2015 at 02:29:28PM +0100, Philipp Winter wrote:
> On Mon, Jan 12, 2015 at 08:24:58PM +0100, Tom van der Woerdt wrote:
> > Interestingly, that paints a completely different picture. I added
> > that line to two machines (guard+exit) and after a few minutes :
> > 
> > # cat /var/lib/tor/node*/infolog | grep Negotiated | awk '{ print $8
> > }' | sort | uniq -dc
> >  40 2
> >  76 3
> >3811 4
> > 
> > # cat /var/lib/tor/node*/infolog | grep Negotiated | awk '{ print $8
> > }' | sort | uniq -dc
> >  50 2
> > 122 3
> >6269 4
> > 
> > I'll let it run a bit longer but these two machines (which are both
> > exits as well - probably relevant) get almost solely v4 handshakes.
> 
> I now did the same for my two relays and I get two different
> distributions, which I found surprising.  That's the relay [0] whose
> results I showed earlier.
> 
>   Negotiated version | Per connection | Per host
>   ---++-
>2 |   12,236  (8%) |  9,292 (21%)
>3 |   29,768 (20%) | 23,393 (52%)
>4 |  108,884 (72%) | 12,051 (27%)
> 
> And here's a relay [1] on the same physical machine with almost the same
> configuration.  The major difference is that this relay is not
> configured to run a directory service whereas the other one is.  The
> numbers are close to yours, Tom.
> 
>   Negotiated version | Per connection | Per host
>   ---++
>2 |  761  (1%) |   279 (3%)
>3 |4,468  (5%) | 1,301 (14%)
>4 |   82,811 (94%) | 7,494 (83%)
> 
> [0] 
> 
> [1] 
> 

Is there an easy way to tell which connections are from clients and
which are from other relays? One of the relays has a higher guard
probability; maybe that has something to do with it. Probably there is a
different distribution of versions among relays than among client,
especially when you account for bandwidth weighting (any relay
connecting to you is more likely to be fast than slow, which may mean
it's more likely to be newer than older).

David
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] meek works again in 4.5-alpha-3

[David Fifield]

On Sat, Dec 06, 2014 at 02:57:19PM -0800, David Fifield wrote:
> What's a good way to inform support of issues that users might run into?
> Should I just send email to h...@rt.torproject.org, or is there a better
> way?
> 
> I was thinking about this today because meek is broken in the 4.5-alpha
> series, but not in the main 4.0 series. I don't know if anyone told
> support workers about it, even though it might matter to them.
> 
> https://trac.torproject.org/projects/tor/ticket/13788
> https://blog.torproject.org/blog/tor-browser-45-alpha-1-released#comment-79244
> https://blog.torproject.org/blog/tor-browser-45-alpha-2-released#comment-81358

Dear support, the meek pluggable transport works again in 4.5-alpha-3
(and 4.0.3).

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Estimating censorship lag by obfs4 blocking

[David Fifield]

According to a couple of blog comments on February 1, 2015, the default
obfs4 bridges are blocked by GFW.

https://blog.torproject.org/blog/tor-browser-45a3-released#comment-86416
https://blog.torproject.org/blog/tor-browser-45a3-released#comment-86907

The obfs4 bridges were introduced on November 17, 2014.
https://blog.torproject.org/blog/tor-browser-45-alpha-1-released
I have a message from Yawning Angel verifying that the obfs4 bridges
were reachable on December 2, 2014.

With this information, we can put rough bounds on how long it took the
GFW to react to a new circumvention system: between 2 and 10 weeks.

Does anyone know information that could tighten up the bounds, a date
between December 2 and February 1 when obfs4 bridges were known to be
either accessible or inaccessible?

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Bridge users by transport is broken

[David Fifield]

On Sat, Jan 31, 2015 at 05:38:28PM +0100, Karsten Loesing wrote:
> On 28/01/15 22:37, isis wrote:
> > Karsten Loesing transcribed 2.5K bytes:
> >> Hi Kevin,
> >> 
> >> manually updated them once more.  Automated updates are still
> >> broken.
> >> 
> >>> On Wed, Dec 31, 2014 at 8:01 AM, Karsten Loesing 
> >>>  wrote:
> >>> 
> >>> Turns out that all user-number graphs were broken.  I manually 
> >>> added the missing two weeks since December 12, but graphs are
> >>> not updating automatically anymore.  I know what the problem is
> >>> (we turned off the rsync service to serve descriptors and I
> >>> need to implement a replacement), but I need to find time to
> >>> fix this.
> > 
> > Just checking: this breakage isn't a side-effect of the changes we
> > made in how BridgeDB syncs bridge descriptors to Metrics, or is it?
> > Is there anything that I should do/fix/check?
> 
> No, this is not related to BridgeDB/ponticum, it's about syncing data
> between CollecTor/yatei and the EC2 instance that crunches numbers for
> user stats.  But thanks for asking!

Karsten, could I ask you to manually update the graph once more? I would
like to include the numbers in the next monthly meek report.

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] How to set up meek on a cooperating web server

[David Fifield]

Roger asked me what it would take to run a meek bridge on an existing
hard-to-block HTTPS web site that wants to help, Wikipedia for example.
The difference with our existing deployments on CDNs is that we don't
expect any special cooperation by the CDN, whereas an interested site
could make minor changes to make running a bridge easier.

It's actually not hard to do. For example, this is the meek-server port
on the Tor bridge behind meek-azure:
https://meek.bamsoftware.com/
That URL is actually totally usable directly in a bridge line:
Bridge meek 0.0.2.0:3 url=https://meek.bamsoftware.com/
The reason we don't just distribute the above bridge line, (and why we
involve a CDN at all), is because meek.bamsoftware.com is trivial to
block by domain or IP--it is used for nothing but circumvention, so the
collateral damage of blocking it is zero. That's why we have to do
domain fronting and put a domain with some unblockability in front of
it.

With an important web site, the unblockability comes for free, so you
don't need domain fronting. Wikipedia could set up their own Tor bridge
running meek-server, then set up a special URL path to reverse-proxy to
the bridge. A bridge line might look like this:
Bridge meek 0.0.2.0:4 url=https://en.wikipedia.org/magic-meek-path/
They would configure their web server to forward every request for the
path /magic-meek-path to their Tor bridge running meek-server, and
that's it.

You could do the same thing with a vhost rather than special path. The
domain name doesn't even have to exist--its only purpose is to tag the
requests that should get forwarded to meek-server. The bridge line would
be like this:
Bridge meek 0.0.2.0:4 url=https://magic.meek.bogus/ 
front=en.wikipedia.org
They would set up the vhost magic.meek.bogus to forward to a meek-server
as before. Here we are technically using domain fronting, but not to
gain unblockability, merely to tag meek requests. (The requests arrive
at the server for en.wikipedia.org, but they have a Host header of
magic.meek.bogus.)

This is how to run meek-server:
https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-serverbridge
And this is an example of doing vhost forwarding on Nginx:
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/nginx/README
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/nginx/nginx.conf.example

An even lighter-weight alternative that doesn't require running
meek-server is to host one of the "reflector" apps. For example, they
can host this file: 
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/php/index.php
at a path like /meek/index.php. Then the bridge line will be
Bridge meek 0.0.2.0:4 url=https://en.wikipedia.org/meek/index.php
All the PHP file does is forward to a meek-server running elsewhere.
This way is less efficient than running your own meek-server, but it's
the first thing I would try for testing. We have reflector apps for PHP
and Python WSGI:
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/php
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/wsgi

In summary, it's not hard to do. A quick test with index.php is less
than a day of work. Setting up meek-server on your own bridge is more
involved, but not more than other pluggable transports.

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Fwd: Orbot v15-alpha-3 with VPN and Meek!

[David Fifield]

On Sat, Feb 14, 2015 at 02:58:28AM -0500, Nathan Freitas wrote:
> More progress on Orbot VPN support, and now, thanks to our new PLUTO
> library (https://github.com/guardianproject/pluto), support for Meek
> (https://trac.torproject.org/projects/tor/wiki/doc/meek) and soon Obfs4
> as well.
> 
> Currently you can use Meek or you can use VPN, but you can't use both
> together... still working on that, as I can't get Meek to talk to the
> passthrough HTTP proxy I use to allow socket connections out of the VPN
> filter.

An upstream HTTP proxy should work, either through torrc HTTPProxy or
the --proxy option.

I just remembered that an upstream SOCKS proxy won't work if you are
using naked meek-client, because when I looked there wasn't SOCKS client
support in the Go standard library. This function determines what types
of proxy are acceptable:
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/meek-client/meek-client.go?id=0.15#n321

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Fwd: Orbot v15-alpha-3 with VPN and Meek!

[David Fifield]

On Sat, Feb 14, 2015 at 03:22:38AM -0500, Roger Dingledine wrote:
> On Sat, Feb 14, 2015 at 12:08:10AM -0800, David Fifield wrote:
> > An upstream HTTP proxy should work, either through torrc HTTPProxy or
> > the --proxy option.
> 
> Careful! The torrc "HTTPProxy" line is only for non-tunneled directory
> fetches.
> 
> It's the "HTTPSProxy" line that most people want -- that causes Tor to
> send its TLS connections via the proxy using the CONNECT command.

Okay, whatever it is that sets TOR_PT_PROXY, because that's what
meek-client is reading.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] bittorrent based pluggable transport

[David Fifield]

On Sat, Feb 28, 2015 at 10:46:03AM -0800, Dan Cristian Octavian wrote:
> My name is Dan, I've been working on a pluggable transport for Tor based on
> bittorrent as cover traffic and wanted to let you know about it.
> 
> https://github.com/danoctavian/bit-smuggler
> 
> In a nutshell, I'm tunnelling a data stream through a bittorrent peer
> connection that is created by real bittorrent clients (uTorrent for this
> implementation) - to avoid "parroting" traffic pitfalls and active probing.
> This made the implementation quite tricky to get right, so my reasoning is 
> that
> it's a worthy trade-off.

People reading this should look at the documentation, there's thoughtful
information there.

https://github.com/danoctavian/bit-smuggler/blob/master/README.md
https://github.com/danoctavian/bit-smuggler/blob/master/DESIGN.md
https://github.com/danoctavian/bit-smuggler/blob/master/docs/system-components.png

I don't know anything about BitTorrent. What parts of the protocol are
easily visible to the censor, without expensive reconstruction? I guess
it includes at least: file names, file sizes, peer IP addresses.

About active probing: it's true that if the censor probes you, you look
like a BitTorrent client. Is there anything weird about how you use the
protocol that could make you stand out anyway? At
https://github.com/danoctavian/bit-smuggler/blob/master/README.md#security,
you say that a network monitor would have to reconstruct a stream in
order to detect anomalies. Could a censor acting as an ordinary peer
detect them more easily, just by participating in the file transfer?
(I'm thinking of how the movie studios would run their own BitTorrent
clients in order to find other downloaders.)

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] bittorrent based pluggable transport

[David Fifield]

Also interesting is that BitTorrent has its own family of obfuscation
transports. I think they are designed to evade throttling by ISPs, which
is a threat model similar to the censorship one.

https://en.wikipedia.org/wiki/BitTorrent_protocol_encryption

MSE (Message Stream Encryption) is a little weird, but not entirely
dissimilar to obfs3.

https://wiki.vuze.com/w/Message_Stream_Encryption
http://www.tcs.hut.fi/Publications/bbrumley/nordsec08_brumley_valkonen.pdf

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] bittorrent based pluggable transport

[David Fifield]

On Mon, Mar 02, 2015 at 07:10:55PM -0800, Dan Cristian Octavian wrote:
> If I understand correctly, you are arguing that my assumption that bittorrent
> is unlikely to be blocked is faulty. I don't have a strong argument against
> this, other than that it would be a very drastic move since for that part of
> the world bittorrent is the main way to get access to media files. As we've
> seen they've blocked already major things such as facebook or google so it
> would not be surprising.

I think it's reasonable to just state in your threat model that
BitTorrent is not blocked. Even though there will be censors for which
that is not true, I'm sure there are enough where it is true for it to
be interesting. It's a mistake to say that if something doesn't work in
China (or any other single concrete threat environment), then it's
useless. It's a question of motivation, and technical capability, and
resources, all of which vary under different censors. BitTorrent is
interesting because I would guess, at least in the U.S., that you're
more likely to get blocked by your ISP than by a firewall further out.

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Tor Project proposal for GSoC 2015

[David Fifield]

On Mon, Mar 02, 2015 at 02:51:51PM +0100, Juro P. Doi wrote:
> 
> I would like to participate to the GSoC 2015 (my first one) and contribute to 
> the Tor project. I am registered as timide on the OFTC IRC network.
> 
> I am currently student in master of computer science, and working with LibFTE 
> for a school project.
> Actually, fteproxy can be used as a Tor bridge in order to hide a Tor 
> connection. To use this type of transport, one have to know the address of a 
> remote fte proxy server or use an hardcoded one. I would like to improve the 
> use of fteproxy in order that a client could use his own remote server or at 
> the opposite could ask for one. The main idea is that server nodes announce 
> themselves to a distributed service that clients could query.
> I already had some mail exchanges with the maintainer of the FTE tools, who 
> advised me to post my proposal to the mailing list.
>  
> It can be done at two different points:
>   1. into fteproxy so that it's Tor independant and any service that use 
> fteproxy could benefit of it (as Tor)
>   2. into Tor so that all type of bridges could be shared
>  
> Anouncements should be done over multiple protocols an queried with fallback 
> by the clients.
> These methods could be over services known by hardcoded IP/URL or set by the 
> user.
> Types of services could be:
>   1. services like https://gitweb.torproject.org/bridgedb.git
>   2. IRC
>   3. mail
>   etc.
> 
> What do you think about it?

Distributing bridges over different channels is a good idea. But the
hardest part of the problem, it seems to me, is this: how do you prevent
the censor from learning all your bridge addresses simply by pretending
to be a client? For example, if you have an IRC service that provides
bridge addresses, what stops the censor from polling this service
thousands of times, and adding all the addresses to a firewall
blacklist? It is a very interesting problem. If you have any ideas, they
could be very useful.

BridgeDB has some reasonable answers. It makes the bridge addresses
keyed on your source address, so you have to control a lot of source
addresses (IP addresses or email addresses) in order to learn a lot of
bridges. It also divides bridges into different pools, so if you find
some way to exploit the HTTP pool, for example, and learn all the
bridges, you do not automatically learn all the bridges in the email
pool, too. These answers are not perfect, but they are good enough that
BridgeDB works in most places where people need to use bridges.

There's a separate question, which is whether the interaction with
BridgeDB or something like it could be automatic, rather than manual. It
would be very cool if you could get some personal FTE bridges just by
running Tor Browser. But then you have a challenge: how do you do the
automatic step in a way that the censor cannot just easily block it?
(And if you have a bootstrapping step that cannot be easily blocked, why
not use it for *all* your communication, not just bootstrapping?)

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Tor Browser sha256 checksums for old versions?

[David Fifield]

On Sat, Mar 07, 2015 at 06:30:09PM +0100, Griffin Boyce wrote:
> Hey all,
> 
>   I was just wondering if it's possible to get a gpg-signed list of sha256
> checksums for the Tor Browser.  The website only shows the current version's
> list of hashes.  Which is really useful, but it would be great to have them
> all if possible.

I don't know if there's a place where they're all in a single file, but
you can get them for historical releases here:

https://archive.torproject.org/tor-package-archive/torbrowser/

For example,

https://archive.torproject.org/tor-package-archive/torbrowser/4.0.3/sha256sums.txt
https://archive.torproject.org/tor-package-archive/torbrowser/4.0.3/sha256sums.txt-gk.asc
https://archive.torproject.org/tor-package-archive/torbrowser/4.0.3/sha256sums.txt-mikeperry.asc
https://archive.torproject.org/tor-package-archive/torbrowser/4.0.3/sha256sums.txt.asc

David
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Preliminary Debian packages for meek

[David Fifield]

On Thu, Mar 05, 2015 at 12:53:52AM +0100, Ximin Luo wrote:
> Source package: https://mentors.debian.net/package/meek
> Binary packages: https://people.torproject.org/~infinity0/bin/
> 
> From the binaries, one should install both meek-client and 
> xul-ext-meek-http-helper; the latter depends on xfvb and xauth.
> 
> meek-client has been modified slightly to run meek-http-helper in a headless 
> firefox using Xfvb(1), so that it works even when run as a system service. 
> Example torrc:
> 
> 
> UseBridges 1
> Bridge meek 0.0.2.0:1 url=https://meek-reflect.appspot.com/ 
> front=www.google.com
> ClientTransportPlugin meek exec /usr/bin/meek-client-wrapper --log 
> /var/log/tor/meek-client-wrapper.log --helper /usr/bin/meek-browser-helper -- 
> /usr/bin/meek-client --log /var/log/tor/meek-client.log
> 

This is great. Thanks a log. The xfvb idea is really clever.

Someone asked this question on Tor Stack Exchange:
https://tor.stackexchange.com/questions/3620/how-to-install-tor-with-meek-support-on-ubuntu-debian
Seems like this package will soon be the answer?

I'll suggest the meek-client-wrapper program from #12716 to the
maintainer of the FreeBSD port.

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Summary of meek's costs, February 2015

[David Fifield]

Here's the summary of meek's CDN fees for February 2015. Earlier reports:
https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html
https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html
https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html
https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html
https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html
https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html

App Engine +   Amazon + Azure = total by month
February 2014$0.09 +   -- +-- =$0.09
March 2014   $0.00 +   -- +-- =$0.00
April 2014   $0.73 +   -- +-- =$0.73
May 2014 $0.69 +   -- +-- =$0.69
June 2014$0.65 +   -- +-- =$0.65
July 2014$0.56 +$0.00 +-- =$0.56
August 2014  $1.56 +$3.10 +-- =$4.66
September 2014   $4.02 +$4.59 + $0.00 =$8.61
October 2014$40.85 +  $130.29 + $0.00 =  $171.14
November 2014  $224.67 +  $362.60 + $0.00 =  $587.27
December 2014  $326.81 +  $417.31 + $0.00 =  $744.12
January 2015   $464.37 +  $669.02 + $0.00 = $1133.39
February 2015  $650.53 +  $604.83 + $0.00 = $1255.36
--
total by CDN  $1715.53 + $2191.74 + $0.00 = $3907.27 grand total

My motivation behind making these reports is to provide transparency and
to contribute to the body of scientific knowledge. I think it's useful
to document these kinds of numbers, for the benefit of others who want
to experiment with this kind of system.

The number of simultaneous users was up a little bit in February
relative to December, hovering around 1250.

https://metrics.torproject.org/userstats-bridge-transport.html?graph=userstats-bridge-transport&start=2014-12-01&end=2015-02-28&transport=meek

The bill for Amazon actually went down last month, for the first time
ever. Okay, that is partially because February is about 10% shorter than
January. Actually, if you account for the shorter month, the bill is
eerily almost exactly the same:
$669.02 * 28/31 = $604.28 ≈ $604.83 (actual Feb. cost)


== App Engine a.k.a. meek-google ==

Bandwidth was up about 30% and instance hours were up about 45%.

At the end of February, I tried tweaking some performance parameters as
an experiment to cause fewer instances to be created. This is to reduce
costs, and also because I guess that extra instances don't really help
much with performance in this case. The creation of instances is
triggered by the application's response latency, which for us is usually
caused by some temporary network issue, rather than a lack of CPU or
something like that. We'll see next month whether it has an effect.

Here is how the Google costs broke down:
4114 GB $492.71
3226 instance hours $157.82
Compared to the previous month:
2944 GB $353.31
2221 instance hours $111.06

https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4534


== Amazon a.k.a. meek-amazon ==

meek-amazon was pretty much the same as last month.

I am thinking about disabling the public Amazon backend, just because
it's a bit more expensive and I haven't been able to get free credits
for it. And when I say "disabling," I mean just removing it as a default
option in Tor Browser; it wouldn't just stop working immediately. My
idea is that we could publish a guide on setting up your own Amazon
CloudFront instance, if CloudFront is a backend that works for you.

Asia Pacific (Singapore)115M requests   $138.07 1018 GB $147.04
Asia Pacific (Sydney)76K requests $0.101 GB   $0.06
Asia Pacific (Tokyo) 29M requests$34.26  193 GB  $24.99
EU (Ireland)111M requests   $133.22  868 GB  $68.23
South America (Sao Paulo) 2M requests $4.25   11 GB   $2.61
US East (Northern Virginia)  30M requests$29.90  278 GB  $22.12
--
total   363M requests   $339.80 2369 GB $265.05

https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A4F3


== Azure a.k.a. meek-azure ==

I haven't been able to get estimated costs out of Azure. I didn't get a
reply to my request last month to be added to whatever plan lets you see
usage data.

I did, however, recover the bandwidth history for the meek-azure bridge
from Onionoo, so we can estimate what the cost would be. If we estimate
a traffic mix similar to that of meek-amazon, with 40% coming from North
America and Europe, and 60% coming from elsewhere, then the costs would
be:

https://onionoo.torproject.org/bandwidth?fingerprint=AA033EEB61601B2B7312D89B62AAA23DC3ED8A34
2014-09   47 GB   $5.53
2014-10  298 GB  $35.04
2014-11  500 GB  $58.80
2014-12  512 GB  $60.21
2015-01  638 GB  $75.03
2015-02  614 GB  $72.21

https://globe.tor

[tor-dev] What's the explanation for weekly cycles in user graphs?

[David Fifield]

When you look at the user graphs, many of them show a weekly cycle.
What's our explanation for why this occurs?

I notice it strongly when I look at the graphs for the meek pluggable
transport, where usage is high on weekdays and lower on weekends. The
same thing happens in some per-country graphs. (In all these graphs, the
light white vertical lines are Mondays.)

https://metrics.torproject.org/userstats-bridge-transport.html?graph=userstats-bridge-transport&start=2015-01-01&end=2015-03-18&transport=meek
https://metrics.torproject.org/userstats-relay-country.html?graph=userstats-relay-country&start=2015-01-01&end=2015-03-18&country=ao&events=off
https://metrics.torproject.org/userstats-relay-country.html?graph=userstats-relay-country&start=2015-01-01&end=2015-03-18&country=py&events=off

You can eyeball more examples in the omni-graph:
https://people.torproject.org/~dcf/graphs/relays-all.pdf

But it doesn't look like that everywhere. Here are graphs for obfs3 and
the United States:

https://metrics.torproject.org/userstats-bridge-transport.html?graph=userstats-bridge-transport&start=2015-01-01&end=2015-03-18&transport=obfs3
https://metrics.torproject.org/userstats-relay-country.html?graph=userstats-relay-country&start=2015-01-01&end=2015-03-18&country=us&events=off

And there is perhaps even the opposite pattern, where there are small
peaks on the weekends, like in Germany:

https://metrics.torproject.org/userstats-relay-country.html?graph=userstats-relay-country&start=2015-01-01&end=2015-03-18&country=de&events=off

Is there a usual story we tell to explain what's happening? A few
hypotheses:
 * People use Tor at work to get their job done (work firewall blocks
   sites they need).
 * People use Tor at work to goof off.
 * People are relaxing and partying on the weekends, not sitting in
   front of a computer.
 * People don't have good Internet at home, so they use it more at work
   (and Tor use just correlates with Internet use).

George Danezis's tech report on discovering censorship events describes
the weekly patterns but doesn't offer a cause.
https://research.torproject.org/techreports/detector-2011-09-09.pdf
"The deployed model considers a time interval of seven (7) days to model
connection rates... The key reason for a weekly model is our observation
that some jurisdictions exhibit weekly patterns. A 'previous day' model
would then raise alarms every time weekly patterns emerge"

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] What's the explanation for weekly cycles in user graphs?

[David Fifield]

On Wed, Mar 18, 2015 at 12:41:55PM +0100, Philipp Winter wrote:
> On Tue, Mar 17, 2015 at 06:09:00PM -0700, David Fifield wrote:
> > You can eyeball more examples in the omni-graph:
> > https://people.torproject.org/~dcf/graphs/relays-all.pdf
> 
> That's a really useful overview!  It would be great if we could include
> that on the metrics page.

Here is the source code:
https://lists.torproject.org/pipermail/tor-dev/2014-October/007697.html

> > Is there a usual story we tell to explain what's happening? A few
> > hypotheses:
> >  * People use Tor at work to get their job done (work firewall blocks
> >sites they need).
> >  * People use Tor at work to goof off.
> >  * People are relaxing and partying on the weekends, not sitting in
> >front of a computer.
> >  * People don't have good Internet at home, so they use it more at work
> >(and Tor use just correlates with Internet use).
> 
> It looks like many of these patterns started emerging after the big
> botnet spike.  It might be caused by infected office computers whose
> owners don't know that Tor is running and who tend to turn off their
> computers over the weekend.  There are probably also infected home
> computers that tend to be used only over the weekend.  That wouldn't
> explain the meek-specific pattern, though, because the botnet only used
> vanilla Tor as far as I know.

That's a good observation about the botnet. But I agree, it seems like
too much at this point for a malware author to start building in
pluggable transports, especially one that's only easily usable with Tor
Browser at this point.

> Apparently several countries such as Ethiopia and Uzbekistan had these
> weekly patterns for a long time, even before the botnet.  These
> countries have a rather small user base and the few users might only use
> Tor in an office setting, like you said.

I wonder if it correlates with censored-ness. I.e., people using Tor for
circumvention more than anonymity. Uzbekistan and Ethiopia are both "not
free" in the Freedom House 2014 summary:
https://freedomhouse.org/sites/default/files/resources/FOTN%202014%20Summary%20of%20Findings.pdf

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Remove "*" from pluggable transports spec?

[David Fifield]

I was going to write an email advocating for the removal of the wildcard
'*' transport specification in pt-spec.txt. But then I saw that the
current version of the spec doesn't mention the wildcard anymore; it was
replace with a TODO in
https://gitweb.torproject.org/torspec.git/commit/pt-spec.txt?id=4dcd7e94f17c072e771119ec90d7cbce4a4788a4
"TODO: Document '*' transport"

So how about we remove the TODO and just act like it never existed?

Normally tor tells the transport plugin what transports to activate by
giving it a string like "obfs3,obfs4"; but you're supposed to also be
able to pass "*", which means "activate all transports you are capable of."
Apparently this feature, though pyptlib and goptlib support it, has
never been implemented in tor itself:
Implement the wildcard "*" protocol in {Client,Server}TransportPlugin 
lines 
https://trac.torproject.org/projects/tor/ticket/3725

A wildcard specification doesn't really make sense, anyway. For one
example, flashproxy-client knows two transport names, "flashproxy" and
"websocket", which are synonyms. But if tor asks for "*",
flashproxy-client shouldn't open up two listeners. Another case is fog
(a dynamic transport combiner), which constructs a chain of transports
from a transport name like "obfs3_websocket". The transports can be
combined in infinite ways; it doesn't make sense to say, "activate all
of them."
https://lists.torproject.org/pipermail/tor-dev/2013-December/005966.html

Let's just drop this part of the spec, and delete some underspecified
and unused code?

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Fwd: [guardian-dev] Orbot v15 alpha 5 is out: MeekObfs4VPNQRCodez!

[David Fifield]

On Thu, Mar 19, 2015 at 11:24:57AM -0400, Nathan Freitas wrote:
> * Meek and Obfs4 pluggable transports are now included and working quite
> well, even with the VPN mode

Great, Guardian and everyone!

What would it take to get some screenshots that show how to turn on
pluggable transports? I would like to add a guide to
https://trac.torproject.org/projects/tor/wiki/doc/meek#Quickstart and
instructions to
https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports#Howtousepluggabletransports
 .

Or do you have such a guide I can just link to?

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Call for a big fast bridge (to be the meek backend)

[David Fifield]

On Thu, Sep 18, 2014 at 08:41:20AM -0700, David Fifield wrote:
> On Thu, Sep 18, 2014 at 02:02:42PM +0100, Ximin Luo wrote:
> > On 18/09/14 03:31, David Fifield wrote:
> > > Currently in the bundles we're not setting a bridge fingerprint, so
> > > relays wouldn't have to share a key.
> > > 
> > 
> > This is something to be *fixed*, not to build future components on top of.
> > 
> > Previously you mentioned that "the user could set their circuits to 4
> > hops" but I think this is a hack of a solution and we can do better,
> > by authenticating the Bridge.
> 
> I really disagree with you here :( I don't understand your point of
> view. Let's try and assume good faith.
> 
> Do you remember a couple of days ago, when I had to separate the tor
> processes for flash proxy and meek because the metrics were getting
> mixed up? That would have been *impossible* to do if there were
> hardcoded fingerprints out there in bundles. And how I recently put out
> a call for someone else to run the meek bridge? How is that transition
> supposed to work if changing the fingerprint means we suddenly and
> inexplicably break every existing client installation?
> 
> The answer surely isn't "make sure the bridge's private key never
> changes" and it isn't "anticipate every possible eventuality
> indefinitely into the future."
> 
> Can you explain what you don't like about four hops? To me it feels like
> the right thing. It wouldn't just be for meek, you know, but for all
> bridge circuits (including ordinary plain-vanilla bridges). When you're
> using a bridge you treat the first hop as unauthenticated and
> unencrypted, as if it were a SOCKS proxy or third-party VPN or any other
> circumvention proxy. You treat the first hop as not chosen by you,
> because it's not: even with BridgeDB you're just pasting in some bytes
> the web site chose for you. After your first circumvention hop, then you
> add your own three hops, notably including your own chosen guard.
>   bridge → guard → middle → exit

Mike talked to me about this and made me understand that adding a fourth
hop is not sufficient to prevent certain attacks. In other words, you
need to TLS to be good, because Tor's current crypto doesn't have a
per-hop MAC. Namely,
https://trac.torproject.org/projects/tor/ticket/14937#comment:17 .

So we'll add fingerprint for the meek bridges after all. Upgrading or
migrating bridges will require more care, basically the same as what
exists for obfs3 etc. now (when you want to change a bridge, you need to
keep the old one running for a while in order to give people time to
upgrade, and in case of a major error like private key disclosure, just
accept that many users will be cut off when you change the fingerprint).

David
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] What's going on with websocket user numbers?

[David Fifield]

I noticed there's a big increase in reported users of the websocket
transport in the past few weeks, from about 5 to over 50.

https://metrics.torproject.org/userstats-bridge-transport.html?graph=userstats-bridge-transport&start=2014-12-31&end=2015-03-31&transport=websocket

"websocket" is the transport name that flash proxy uses. I checked the
bridge for flash proxy, and it doesn't seem to be used notably more
often in the last weeks than before.

https://globe.torproject.org/#/bridge/9DD7D759971D03E70F0181B4E6D5F7735E7BFDED

There are other bridges out there that also support the websocket
transport. I don't know who's running them. I appended a list at the
end of this message. It comes from grepping for "websocket" in

https://onionoo.torproject.org/details?type=bridge&fields=nickname,hashed_fingerprint,or_addresses,platform,transports

Most of them seem to be related; namely, there are a lot with these
characteristics:
80 <= ORPort <= 105
platform = Tor 0.2.7.0-alpha-dev on Linux
transports = ["websocket","fte","obfs3","scramblesuit"]
Morlock-sounding nicknames: ashrak, bashaak, delmac, gonach...
The only bridges not belonging to this set are C3L (ORPort = 8118),
giygas (my bridge), and nerds (ORPort = 9052, platform = Tor 0.2.5.11 on
Linux").

It's possible that the new websocket users are using one of these other
bridges. It's also possible, since the bridges run more than one
transport, that the users are actually using some other transport, and
they are mistakenly being counted for websocket:

https://lists.torproject.org/pipermail/tor-dev/2014-September/007483.html

A lot of them have last_restarted of 2015-03-14 or 2014-03-25, which
fall roughly where the user increase is on the metrics graphs. Maybe
they were recently rebooted with support for the websocket transport.

David Fifield


{"nickname":"ashrak","hashed_fingerprint":"F1C465D38F18091F6FED9CAC924DA50F2813C0AE","or_addresses":["10.219.160.68:80"],"platform":"Tor
 0.2.7.0-alpha-dev on 
Linux","transports":["websocket","fte","obfs3","scramblesuit"]},
{"nickname":"bashaak1","hashed_fingerprint":"196FC9A50B1F3828730C11C4E1DD5B55F979F548","or_addresses":["10.152.136.2:80"],"platform":"Tor
 0.2.7.0-alpha-dev on 
Linux","transports":["obfs3","scramblesuit","websocket","fte"]},
{"nickname":"bashaak2","hashed_fingerprint":"F76FB49B6B1CC1BB23618A20A2D6E8B1509BF7BC","or_addresses":["10.128.21.28:81"],"platform":"Tor
 0.2.7.0-alpha-dev on 
Linux","transports":["obfs3","scramblesuit","websocket","fte"]},
{"nickname":"benna2","hashed_fingerprint":"3665A4A74A6554CA03122AFBC9C3C6C5AD2F9BC6","or_addresses":["10.35.205.157:83"],"platform":"Tor
 0.2.7.0-alpha-dev on 
Linux","transports":["obfs3","scramblesuit","websocket","fte"]},
{"nickname":"C3L","hashed_fingerprint":"E1F69F3CE3107F740646127527F8B21BA8BCDFC4","or_addresses":["10.226.202.166:8118","[fd9f:2e19:3bcf::73:c310]:8118"],"platform":"Tor
 0.2.7.0-alpha-dev on 
Linux","transports":["obfs3","scramblesuit","obfs4","websocket","fte"]},
{"nickname":"delmac1","hashed_fingerprint":"16EBD0A3DAD2833FDA9323F89ED9B2EFF484D748","or_addresses":["10.223.29.80:84"],"platform":"Tor
 0.2.7.0-alpha-dev on 
Linux","transports":["obfs3","scramblesuit","websocket","fte"]},
{"nickname":"delmac2","hashed_fingerprint":"5B300273F583602780746191ADF17AC96190CC92","or_addresses":["10.12.236.225:85"],"platform":"Tor
 0.2.7.0-alpha-dev on 
Linux","transports":["obfs3","scramblesuit","websocket","fte"]},
{"nickname":"giygas","hashed_fingerprint":"9DD7D759971D03E70F0181B4E6D5F7735E7BFDED","or_addresses":["10.233.17.187:9001","[fd9f:2e19:3bcf::df:d557]:9001"],"platform":"Tor
 0.2.5.10 on Linux","transports":["websocket","obfs3_websocket"]},
{"nickname":"gonach1","hashed_fingerprint":"F532D7DC394BAE539D530EB9EC5BA22450

Re: [tor-dev] What's going on with websocket user numbers?

[David Fifield]

On Mon, Mar 30, 2015 at 11:01:31PM -0700, David Fifield wrote:
> It's possible that the new websocket users are using one of these other
> bridges. It's also possible, since the bridges run more than one
> transport, that the users are actually using some other transport, and
> they are mistakenly being counted for websocket:
> 
> https://lists.torproject.org/pipermail/tor-dev/2014-September/007483.html
> 
> A lot of them have last_restarted of 2015-03-14 or 2014-03-25, which
> fall roughly where the user increase is on the metrics graphs. Maybe
> they were recently rebooted with support for the websocket transport.

I guess this is what's happening. I made a graph with websocket, fte,
obfs4, and scramblesuit, and it looks like there is a correlated
increase in scramblesuit at the same time as the increase in websocket.
Probably some new scramblesuit bridges started being used, and some of
their users are being counted for websocket.

https://metrics.torproject.org/userstats-bridge-transport.html?graph=userstats-bridge-transport&start=2014-12-31&end=2015-03-31&transport=obfs4&transport=websocket&transport=fte&transport=scramblesuit

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] PT-themed stuffed animals: huggable transports

[David Fifield]

In an effort to improve pluggable transports' visual identity, we should
introduce a line of stuffed animals. The only rule that the the animal's
name has to be made out of the transport name.

The meek meerkat! [1]
The obfs blobfish! [2]
The FTE eft! [3]
The flash proxy banded quail (Philortyx fasciatus)! [4]
The ScrambleSuit bumblebee (Bombus centralis)! [5]

Call now, operators are standing by.

[1] https://en.wikipedia.org/wiki/Meerkat
[2] https://en.wikipedia.org/wiki/Blobfish
[3] https://en.wikipedia.org/wiki/Eft
[4] https://en.wikipedia.org/wiki/Philortyx_fasciatus
[5] https://en.wikipedia.org/wiki/Bombus_centralis
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Big performance improvement for meek-azure

[David Fifield]

I found and fixed a big performance limitation in the meek-azure
transport. If you tried meek-azure before, but it was too slow, give it
another try!

meek-azure has always seemed slower than the other two backends, and I
recently spent the time to figure out why. It was a lack of persistent
HTTP connections between the Azure host and the Tor bridge. Every single
HTTP request was doing a complete TCP and TLS handshake. This of course
increased latency, but the bottleneck was actually CPU on the bridge, as
it tried to cope with all those TLS handshakes. Here is the commit that
causes connections to be shared and reused:
https://gitweb.torproject.org/pluggable-transports/meek.git/commit/?id=e06bcf2c849075cf241addf2182dfc0125a35c92

This issue did not affect meek-google because the App Engine URL fetch
API reuses HTTP connections transparently. It did not affect meek-amazon
because the CloudFront CDN does its own connection reuse.

Usage on meek-azure has already increased in the past few days since I
started testing the change. I attached a graph that shows the all-time
bandwidth history of the bridge.
https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8A34

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Release: obfs4proxy-0.0.5

[David Fifield]

On Thu, Apr 16, 2015 at 06:48:53PM +, Yawning Angel wrote:
> On Thu, 16 Apr 2015 14:03:58 -0400 (EDT)
> "Steve Snyder"  wrote:
> > The v0.5 binary is substantially larger than the v0.4 binary (7204KB
> > vs.5408KB, both built with Go v1.4.2).
> > 
> > Is this expected behavior with the newer version of obfs4proxy?
> 
> Building on linux/amd64 with 1.4.2 I get:
> 
>  * 0.0.4 7441792
>  * 0.0.5 7424544
> 
> If I strip either binary I get something in the 5 MiB range, so that
> might be where the difference is coming from, though I would strongly
> discourage doing so as it is not something that is supported by the
> toolchain developers[0], and if a binary that has been stripped breaks,
> I'm not going to do anything about it beyond saying "don't do that".

In Tor Browser we build with
go build -ldflags '-s'

https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/gitian/descriptors/linux/gitian-pluggable-transports.yml?id=d4e10e98af5237ed21796a4d48c3e09db6994959#n209

Check discussion here:
https://trac.torproject.org/projects/tor/ticket/12649

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Fwd: Games Without Frontiers: Investigating Video Games as a Covert Channel

[David Fifield]

On Fri, Mar 20, 2015 at 08:24:49AM -0400, Rishab Nithyanand wrote:
> Hey all,
> 
> I just thought I'd share and get feedback about some recent work from our team
> at Stony Brook University. I posted this to the tor-talk list earlier and it
> was suggested to x-post here, too.
> 
> Title: Games Without Frontiers: Investigating Video Games as a Covert Channel
> [ http://arxiv.org/pdf/1503.05904v1.pdf ]

Thanks for posting this! I just saw this Wired article and associated
demo video:

https://www.wired.com/2015/04/app-hides-secret-messages-starcraft-style-games/
https://www.youtube.com/watch?v=lQX5HpdNZ64

I want to thank you for making the demo video. This is exactly the kind
of visualization that I think aids understanding. It is one thing to
read about it; it's another to see the buildings laid out in a grid and
a textual representation of the game commands. I added a link to it in
the Child's Garden of Pluggable Transports.

https://trac.torproject.org/projects/tor/wiki/doc/AChildsGardenOfPluggableTransports#Castle

Feel free to edit it and add what you like. I think it would be great to
have an example of an encoded message.

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Urdu & Hindi translations of Tor browser ?

[David Fifield]

On Sun, Apr 19, 2015 at 12:50:52AM -0400, Griffin Boyce wrote:
>   Both populations also have a large number of speakers: ~300M for Hindi
> and ~66M for Urdu.

I was really surprised; Hindi is the third-most spoken language in the
world, trailing only Mandarin and English. Of the top 10 languages in
this Wikipedia list, Tor Browser is missing localizations only for
Hindi, Bengali, and Malay. (Urdu is #11.)

https://en.wikipedia.org/wiki/List_of_languages_by_total_number_of_speakers

David
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Urdu & Hindi translations of Tor browser ?

[David Fifield]

On Sun, Apr 19, 2015 at 02:26:22AM -0400, Griffin Boyce wrote:
>   I guess what I'm really asking is what's the process for supporting
> additional languages?  Would adding additional languages add too much
> overhead to release cycles?

It's also partly a matter of the file size of additional packages. But
see:

Support a multi-lingual TBB that can switch between localizations
https://trac.torproject.org/projects/tor/ticket/12967

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Possible anomaly in meek user graph circa April 15, 2015

[David Fifield]

The latest meek user graph shows two recent large increases. The first
increase from 2000 to 3000 is around April 9. The second from 3000 to
5000 is all on April 15. The first increase makes sense; it corresponds
with the removal of a bottleneck on meek-azure:
https://lists.torproject.org/pipermail/tor-dev/2015-April/008637.html
The second one happens all in one day and I don't have an explanation
for it.

Here are the day-by-day numbers, the same numbers from which the graph
is derived:
↓↓
2015-04-09,bridge,,meek2584,54
2015-04-10,bridge,,meek3363,54
2015-04-11,bridge,,meek3392,51
2015-04-12,bridge,,meek3531,52
2015-04-13,bridge,,meek3344,49 ←
2015-04-14,bridge,,meek5385,53 ←
2015-04-15,bridge,,meek5113,52
2015-04-16,bridge,,meek4990,52
2015-04-17,bridge,,meek5173,51
2015-04-18,bridge,,meek5422,33

I at first suspected conflation with the statistics of some other
transport, like we observed with meek and websocket back in September
2014.
https://lists.torproject.org/pipermail/tor-dev/2014-September/007483.html
There is a nearly simultaneous increase in obfs4 (also attached); I
didn't see anything similar in any other transports. However I checked
Onionoo and the only bridges running meek are ones I know about, and
they aren't running any other transports.

https://onionoo.torproject.org/details?type=bridge
{"nickname":"UtahMeekBridge","hashed_fingerprint":"88F745840F47CE0C6A4FE61D827950B06F9E4534","or_addresses":["10.45.162.225:8000"],"last_seen":"2015-04-20
 19:48:46","first_seen":"2014-10-24 
20:37:04","running":true,"flags":["Fast","Guard","Running","Stable","Valid"],"last_restarted":"2015-04-13
 19:14:56","advertised_bandwidth":6517470,"platform":"Tor 0.2.5.12 on 
Linux","transports":["meek","meek"]},
{"nickname":"TorLandMeek","hashed_fingerprint":"55CD4494A9EC14B74A677A7B7DD356E3417F9E69","or_addresses":["10.60.60.122:443"],"last_seen":"2015-04-16
 17:45:30","first_seen":"2015-04-15 
18:44:44","running":false,"flags":["Valid"],"last_restarted":"2015-04-15 
18:24:47","advertised_bandwidth":3540992,"platform":"Tor 0.2.5.10 on 
Linux","transports":["meek"]},
{"nickname":"starman","hashed_fingerprint":"AA033EEB61601B2B7312D89B62AAA23DC3ED8A34","or_addresses":["10.243.156.101:9002","[fd9f:2e19:3bcf::24:d1d2]:9002"],"last_seen":"2015-04-20
 19:48:46","first_seen":"2014-09-16 
06:37:04","running":true,"flags":["Fast","Guard","Running","Stable","Valid"],"last_restarted":"2015-04-09
 16:19:58","advertised_bandwidth":2434220,"platform":"Tor 0.2.5.12 on 
Linux","transports":["meek","meek"]},
{"nickname":"TorLandMeek","hashed_fingerprint":"3FD131B74D9A96190B1EE5D31E91757FADA1A4F3","or_addresses":["10.116.43.196:443"],"last_seen":"2015-04-20
 19:48:46","first_seen":"2014-10-14 
06:37:04","running":true,"flags":["Fast","Guard","Running","Stable","Valid"],"last_restarted":"2015-04-15
 19:02:22","advertised_bandwidth":4125225,"platform":"Tor 0.2.6.7 on 
Linux","transports":["meek"]},

David
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Summary of meek's costs, April 2015

[David Fifield]

On Tue, May 05, 2015 at 06:22:47PM -0700, Mike Perry wrote:
> David Fifield:
> > Here's the summary of meek's CDN fees for April 2015.
> > 
> > total by CDN  $3292.25 + $3792.79 + $0.00 = $7085.04 grand total
> > https://metrics.torproject.org/userstats-bridge-transport.html?graph=userstats-bridge-transport&start=2015-02-01&end=2015-04-30&transport=meek
> 
> Yikes! Are these costs covered by a grant or anything? Should we be
> running a donations campaign?

It's partly covered by grants but not fully.

I'd be happy with donations but I don't want to handle any money. We
also need to think about long-term sustainability: usage and costs will
continue to increase(at least until the world changes), and donations
will need to increase too.

Look at the "1 year" bandwidth graph for meek-google. It's pretty close
to linear since October 2014, increasing 400 KB/s/month.
https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4534

> > If you want to help reduce costs, you can
> >  1. Use meek-azure; it's still covered through a grant for the next four
> > months.
> >  2. Set up your own App Engine or CDN account. Then you can pay for your
> > own usage (it might even be free depending on how much you use).
> > Here are instructions on how to set up your own:
> >   
> > https://gitweb.torproject.org/pluggable-transports/meek.git/tree/appengine/README
> >   
> > https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront
> >   https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure
> > Then you will have to enter a bridge line manually. Follow the
> > instructions at
> >   
> > https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtochangethefrontdomain
> > but instead of changing the "front=" part, change the "url=" part.
> > For example,
> >   bridge meek 0.0.2.0:1 url=https://.appspot.com/ 
> > front=www.google.com
> 
> Please let me know if anyone takes you up on this!
> 
> I am happy to add the meek bridges of anyone who does this as an option
> in Tor Browser. We can add logic to round robin or randomly select
> between the set of meek providers for a given meek type upon first
> install, or even for every browser startup. 

Thanks.

In recommending that people run their own reflectors, I actually had a
different use case in mind: that they would run one for themself or for
their friends, and not announce it publicly. So basically like setting
up any other private proxy, except it works in more places.

> Given your costs, it also seems worthwhile for us to fund development to
> improve this situation, so that meek remains a transport of last resort
> rather than people's first choice.

I don't have the feeling that it's people's first choice. Rather I think
we're seeing new users who were not being served by any of the other
transports. It's going to be slower than other transports. On the other
hand, not needing to find bridges is a big distinction, and once you
have something that works there's little incentive to change it.

> Here's a couple options:
> 
> 1. We can add a browser notification box for meek users that either
> tells them about meek-azure, or tells them that now that Tor Browser
> works, they can use it to visit https://bridges.torproject.org to get a
> bridge type that doesn't cost so much money.

I don't want to lean too hard on meek-azure, because its grant runs out
in four months and I don't have a plan to keep it going.

I wouldn't want people to feel guilty when they manage to circumvent
censorship, especially if nothing else works for them. But yes, we can
probably make some UI and backend changes that make the default options
less costly.

> 2. Perhaps cleaner: if BridgeDB itself were accessible through a domain
> front, we could export its captcha and bridge distribution through an
> API on this domain front. Once your IP forwarding in
> https://trac.torproject.org/projects/tor/ticket/13171 is solved,
> BridgeDB even could still make use of its IP-based hashring logic.

For this purpose you wouldn't even need the full power of BridgeDB. The
list of bridges doesn't need to be kept secret for blocking resistance,
so you could even just put the list on a web page and domain-front to
download it. (It still might make sense to keep the list secret to
hinder financial DoS on the operators, but unless there are a ton of
operators, they'll still be enumerable and vulnerable.)

I gave a talk about domin fronting at Stanford and that's what the
audience suggested: use a centrally paid-for account only to get a
bridge on someone else&#

Re: [tor-dev] Summary of meek's costs, April 2015

[David Fifield]

On Tue, May 05, 2015 at 11:04:58PM -0400, Griffin Boyce wrote:
> Mike Perry wrote:
> >David Fifield:
> >>Here's the summary of meek's CDN fees for April 2015.
> >>
> >>total by CDN  $3292.25 + $3792.79 + $0.00 = $7085.04 grand total
> >>https://metrics.torproject.org/userstats-bridge-transport.html?graph=userstats-bridge-transport&start=2015-02-01&end=2015-04-30&transport=meek
> >
> >Yikes! Are these costs covered by a grant or anything? Should we be
> >running a donations campaign?
> >
> >>If you want to help reduce costs, you can
> >> 1. Use meek-azure; it's still covered through a grant for the next four
> >>months.
> >> 2. Set up your own App Engine or CDN account. Then you can pay for your
> >>own usage (it might even be free depending on how much you use).
> >>Here are instructions on how to set up your own:
> >>https://gitweb.torproject.org/pluggable-transports/meek.git/tree/appengine/README
> >>https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront
> >>https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure
> >>Then you will have to enter a bridge line manually. Follow the
> >>instructions at
> >>https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtochangethefrontdomain
> >>but instead of changing the "front=" part, change the "url=" part.
> >>For example,
> >>  bridge meek 0.0.2.0:1 url=https://.appspot.com/
> >>front=www.google.com
> >
> >Please let me know if anyone takes you up on this!
> >
> >I am happy to add the meek bridges of anyone who does this as an option
> >in Tor Browser. We can add logic to round robin or randomly select
> >between the set of meek providers for a given meek type upon first
> >install, or even for every browser startup.
> 
>   If there were some randomization logic included, I'd be happy to
> contribute an App Engine or Amazon meek access point.  If a few people did
> that, the costs might be more manageable.  But also the stats might be a bit
> harder to aggregate (which might be important if David is writing a
> thesis/paper/etc).

Thanks Griffin. At this point we'd need, what, 60 operators in order to
cost on average $30/month? At current usage rates.

I think we already have plenty of aggregated stats. It's been nice to be
able to separate cleanly amazon/azure/google, but we shouldn't try to
keep that forever at the expense of becoming more scalable.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Summary of meek's costs, April 2015

[David Fifield]

On Wed, May 06, 2015 at 04:36:48AM +, isis wrote:
> But just to be clear — since it sounds like you've asked for several new
> things in that last paragraph :) — which do you want:
> 
>   1. Tor Browser users use meek to get to BridgeDB, to get non-meek bridges 
> by:
>1.a. Retrieving and solving a CAPTCHA inside Tor Launcher.
>1.b. Solving a CAPTCHA on a BridgeDB web page.
> 
>   2. Tor Browser users use BridgeDB's domain front, to get non-meek bridges 
> by:
>2.a. Retrieving and solving a CAPTCHA inside Tor Launcher.
>2.b. Solving a CAPTCHA on a BridgeDB web page.
> 
> If you want #2, then we're essentially transferring the domain-fronting costs
> (and the DDoS risks) from meek to BridgeDB, and we'd need to decide who is
> going to maintain that service, and who is going to pay for it.  Could The
> Tor Project fund BridgeDB domain fronting?

You still have the DoS risk, but in normal usage the costs will be way
way less because you're only paying for bootstrapping and not for
GNU/Linux ISO downloads or whatever it is people do with Tor. Bandwidth
costs across all CDNs are between $0.10 and $0.20 per GB. To reach even
one GB would take a million 1K bootstraps.

> As far as maintenance goes, the threat to any of our domain fronts, including
> meek and any BridgeDB domain fronts, from China's Great Cannon waging economic
> counter-counter-warfare by attacking us (like they did to GreatFire.org) is
> something which must be taken into account.  Will the maintainer of this
> service need to wake up to emergency, the-request-rate-is-skyrocketing, emails
> at 4AM to shut the service down?  Or do we already have technical measures to
> detect DDoS and prevent $30,000+/day CDN bills?  Further, what happens when #2
> is being DDoS-ed?  Should we fallback to #1?  Should we have both, and some
> strategy for balancing between the two?

App Engine is nice because you can set a daily cost limit, and the
service shuts down after that. It's currently set at $45/day (after we
bumped into the previous $40/day limit one day last week :/). It's nice
because the maximum damage a DoS can cause (besides shutting down the
service) is O(1).

Amazon sucks and they don't have any automatic way to shut down a
service. I emailed them and they were very clear about that. The best
you can do is set up an email alert at different cost threshold (which I
have done). But that requires someone with credentials to be awake and
online when it happens. This is the main reason I want to drop Amazon.
(Apart from the billing concerns, Amazon's CDN, technically, is nice and
fast and reliable.)
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Summary of meek's costs, April 2015

[David Fifield]

On Wed, May 06, 2015 at 11:56:36AM -0700, Arthur D. Edelstein wrote:
> > Amazon sucks and they don't have any automatic way to shut down a
> > service. I emailed them and they were very clear about that. The best
> > you can do is set up an email alert at different cost threshold (which I
> > have done). But that requires someone with credentials to be awake and
> > online when it happens. This is the main reason I want to drop Amazon.
> > (Apart from the billing concerns, Amazon's CDN, technically, is nice and
> > fast and reliable.)
> 
> Would it make sense to add some code to your meek server to monitor
> bandwidth usage and automatically shut off if a limit is reached?

I don't think that helps because I think you will still get charged for
requests+bandwidth even if the origin server is unresponsive or returns
an error. I could be wrong about this. Yawning wrote some such code a
while back.

Even if you cut off all abusive use of bandwidth, if the adversary can
figure out how to charge you for requests, they cost $1 per million on
Amazon.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Summary of meek's costs, April 2015

[David Fifield]

On Wed, May 06, 2015 at 12:56:04PM -0700, Arthur D. Edelstein wrote:
> Maybe you could rig up something that shuts down the instance? Or does
> Amazon charge you even then?

That might work. I found some documentation on an API for CloudFront web
distributions:
https://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/Actions_Dist.html
https://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/PutConfig.html

It looks like you can PUT an true.

It would be nice if this were a separate process apart from meek-server
that counts requests and bytes and keeps track of estimated costs. It
should be able to send a message to another process somewhere that
controls the AWS API keys.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Paper on domain fronting (meek)

[David Fifield]

I and my colleagues at Berkeley, Lantern, and Psiphon wrote a paper on
domain fronting, the censorship circumvention technology that underlies
the meek pluggable transport among other systems. It's going to appear
at the PET Symposium on June 30, 2015. If you've been following along,
you know most of the story for Tor, but the paper also includes the
story of deployment on Lantern and Psiphon.

I'm trying an experiment. Normally this kind of paper is published as a
PDF. I find it annoying to read PDF papers on the web, where you have to
scroll to the end to find URLs for references (if you're lucky). So I
made an HTML version of what's going to get published in the
proceedings.

https://www.bamsoftware.com/papers/fronting/

I have to admit, though, that PDF is nice for printing out and reading
offline, so here is the PDF version:

https://www.bamsoftware.com/papers/fronting.pdf
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] CollecTor data: mapping bridge-network-status to bridge-server-descriptor to bridge-extra-info

[David Fifield]

I'm trying to use CollecTor data to find out how much bandwidth is
offered by different pluggable transports over time. I.e., I want to be
able to say something like, "On July 1, bridges with obfs3 offered X MB/s,
bridges with obfs4 offered Y MB/s," etc. To do this, I'm mapping through
three types of CollecTor documents:
bridge-network-status (where the bandwidth is and which links to router 
digests)
bridge-server-descriptor (which links to extra-info digests)
bridge-extra-info (where the transports are)
I'm having trouble because sometimes, a router digest listed in a
bridge-network-status document is not found in the same tarball.

https://collector.torproject.org/archive/bridge-descriptors/bridge-descriptors-2015-07.tar.xz
Here is an example of what I'm doing, using the above tarball.
bridge-descriptors-2015-07/statuses/04/20150704-000350-4A0CCD2DDC7995083D73F5D667100C8A5831F16D
This is a bridge-network-status document. One of its entries is:
r starman qgM+62FgGytzEtibYqqiPcPtijQ 
mdOOBxVOTpw8loBezhSDZxLIcXs 2015-07-03 21:39:31 10.174.163.60 9002 0
s Fast Guard Running Stable Valid
w Bandwidth=2646
p reject 1-65535
The second base64-encoded string is the router digest.
base64decode("mdOOBxVOTpw8loBezhSDZxLIcXs") = 
99D38E07154E4E9C3C96805ECE14836712C8717B
bridge-descriptors-2015-07/server-descriptors/9/9/99d38e07154e4e9c3c96805ece14836712c8717b
Now we go looking for a bridge-server-descriptor with router
digest 99D38E07154E4E9C3C96805ECE14836712C8717B, which is in the
above file. It has a line:
extra-info-digest D69106C8BAF5C0044F7331F24DF77E85BBF84027
bridge-descriptors-2015-07/extra-infos/d/6/d69106c8baf5c0044f7331f24df77e85bbf84027
Now we find a bridge-extra-info with digest
D69106C8BAF5C0044F7331F24DF77E85BBF84027 in the above file. It
tells us what transports the bridge supports (there are two, one
for IPv4 and one for IPv6):
transport meek
transport meek

Here's an example of where it goes wrong.
bridge-descriptors-2015-07/statuses/01/20150701-060138-4A0CCD2DDC7995083D73F5D667100C8A5831F16D
r Unnamed ABk0wg4j6BLCdZKleVtmNrfzJGI 
eGIOW1mGM/Dbw+t5bXnR8jdnsoY 2015-07-01 05:56:14 10.123.124.91 443 0
s Fast Running Stable Valid
w Bandwidth=156
p reject 1-65535
We are looking for router digest 
78620E5B598633F0DBC3EB796D79D1F23767B286:
base64decode("eGIOW1mGM/Dbw+t5bXnR8jdnsoY") = 
78620E5B598633F0DBC3EB796D79D1F23767B286
But there is no file 
bridge-descriptors-2015-07/server-descriptors/7/8/78620e5b598633f0dbc3eb796d79d1f23767b286.
However, I did find it in the previous month's tarball,
https://collector.torproject.org/archive/bridge-descriptors/bridge-descriptors-2015-06.tar.xz
bridge-descriptors-2015-06/server-descriptors/8/3/835a43ff89db9c1be8ddf7536d759875878620e7

It seems rare that the bridge-server-descriptor is missing. In the
2015-07 tarball, it happened for 5891/477496 relays (1.2%). An
additional 4/477496 (0.0%) had a bridge-server-descriptor but were
missing bridge-extra-info.

How do you handle cases like this? I had a browse through the Onionoo
source code, but did not quickly understand it. Should I just always
include the month preceding the earliest month I want to process?
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] CollecTor data: mapping bridge-network-status to bridge-server-descriptor to bridge-extra-info

[David Fifield]

On Wed, Jul 08, 2015 at 11:39:54PM -0400, Roger Dingledine wrote:
> > It seems rare that the bridge-server-descriptor is missing. In the
> > 2015-07 tarball, it happened for 5891/477496 relays (1.2%).
> [snip]
> > How do you handle cases like this? I had a browse through the Onionoo
> > source code, but did not quickly understand it. Should I just always
> > include the month preceding the earliest month I want to process?
> 
> How many of the 5891 cases does that resolve?

Peeking into 2015-06 resolves all 5891 cases of missing
bridge-server-descriptor in the 2015-07 tarball. (There are still 4
cases of missing bridge-extra-info.)
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Per-transport bridge bandwidth and bridge counts

[David Fifield]

I made some graphs that show the count and total bandwidth of all
bridges, broken down by transport.

https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwidth.png

https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-count.png
The top part of the graph is non-default bridges, and the bottom is
default bridges (the ones that ship with Tor Browser). Note the varying
vertical scales. Source code and data are here:
https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11.tar.gz

https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwidth.csv

I have a question about getting bandwidth numbers. What I want is
something like available bandwidth capacity; i.e., how much headroom
transports have given their level of use. I think that the bandwidth
numbers I'm using are partially conflated with usage. Notice how parts
of pt-count.png are flat while pt-bandwidth.png has a weekly pattern.

The bandwidth number I'm using is the "w Bandwidth=" line from a
bridge-network-status document. It looks like this:
r starman qgM+62FgGytzEtibYqqiPcPtijQ mdOOBxVOTpw8loBezhSDZxLIcXs 
2015-07-03 21:39:31 10.174.163.60 9002 0
s Fast Guard Running Stable Valid
w Bandwidth=2646
p reject 1-65535
dir-spec.txt says:
An estimate of the bandwidth of this relay, in an arbitrary unit
(currently kilobytes per second).  Used to weight router
selection. See section 3.4.2 for details on how the value of
Bandwidth is determined in a consensus."
Section 3.4.2 says:
When we speak of a router's bandwidth in this section, we mean
either its measured bandwidth, or its advertised bandwidth
The bandwidth in a "w" line should be taken as the best estimate
of the router's actual capacity that the authority has.  For
now, this should be the lesser of the observed bandwidth and
bandwidth rate limit from the server descriptor.  It is given in
kilobytes per second, and capped at some arbitrary value
(currently 10 MB/s).
I don't know where observed bandwidth comes from. Is there some kind of
external test that measures it, or does it come from measuring user
traffic?

Another option for getting bandwidth is the "bandwidth" line in a
bridge-server-descriptor document. It looks like this:
bandwidth 1073741824 1073741824 2646895
The three fields are bandwidth-avg bandwidth-burst bandwidth-observed.
dir-spec.txt says:
Estimated bandwidth for this router, in bytes per second.  The
"average" bandwidth is the volume per second that the OR is
willing to sustain over long periods; the "burst" bandwidth is
the volume that the OR is willing to sustain in very short
intervals.  The "observed" value is an estimate of the capacity
this relay can handle.  The relay remembers the max bandwidth
sustained output over any ten second period in the past day, and
another sustained input.  The "observed" value is the lesser of
these two numbers.
So it seems that bandwidth-observed is not what we want, because it's
derived from current usage. bandwidth-avg and bandwidth-burst aren't
useful because they are operator-controlled and often left at the
default of 1 GB/s.

Yet another option is the speed of serving consensuses, the
"dirreq-v3-tunneled-dl" line in a bridge-extra-info document:
dirreq-v3-tunneled-dl 
complete=13624,timeout=1456,running=4,min=1938,d1=21008,d2=43181,q1=53397,d3=63519,d4=88911,md=117048,d6=146982,d7=180109,q3=199979,d8=223313,d9=298860,max=1558627
You can take the "md=" field, for example, to get median B/s for serving
consensuses.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Per-transport bridge bandwidth and bridge counts

[David Fifield]

On Sat, Jul 11, 2015 at 12:18:07PM -0700, David Fifield wrote:
> I made some graphs that show the count and total bandwidth of all
> bridges, broken down by transport.
>   
> https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwidth.png
>   
> https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-count.png
> The top part of the graph is non-default bridges, and the bottom is
> default bridges (the ones that ship with Tor Browser). Note the varying
> vertical scales. Source code and data are here:
>   https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11.tar.gz
>   
> https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwidth.csv
> 
> I have a question about getting bandwidth numbers. What I want is
> something like available bandwidth capacity; i.e., how much headroom
> transports have given their level of use. I think that the bandwidth
> numbers I'm using are partially conflated with usage. Notice how parts
> of pt-count.png are flat while pt-bandwidth.png has a weekly pattern.
> 
> Yet another option is the speed of serving consensuses, the
> "dirreq-v3-tunneled-dl" line in a bridge-extra-info document:
>   dirreq-v3-tunneled-dl 
> complete=13624,timeout=1456,running=4,min=1938,d1=21008,d2=43181,q1=53397,d3=63519,d4=88911,md=117048,d6=146982,d7=180109,q3=199979,d8=223313,d9=298860,max=1558627
> You can take the "md=" field, for example, to get median B/s for serving
> consensuses.

I tried extracting extracting the rate of serving consensuses as a
proxy for bridge bandwidth. The "bandwidth" one is the one you saw
before, summing observed bandwidth of bridges. "dl.md" is the median
download rate of consensuses, and "dl.max" is the maximum.

https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-count.png
https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-bandwidth.png
https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-dl.md.png
https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-dl.max.png

Source code:
https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20.tar.gz

These plots omit all bridges that did not have a "dirreq-v3-tunneled-dl"
line.

The new graphs have a different shape than the observed-bandwidth one,
at least. They still seem somewhat tied to usage. It looks like there is
a weekly cycle in the dl.max one--it could be simply that with more
users connecting during the week, the bridge has a higher chance of
seeing a fast one that pushes up the maximum.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Two new Onionoo versions 2.4 and 2.5 add new "effective_family" and "measured" fields

[David Fifield]

On Wed, Aug 19, 2015 at 02:27:10PM +0200, Karsten Loesing wrote:
> The new version 2.5 that I deployed this week adds the optional
> "measured" field to details documents.  The main idea behind this new
> field is that relay operators and Tor network debuggers can now figure
> out easily whether a relay is affected by not being measured by a
> sufficient number of bandwidth authorities and as a result has lower
> usage than it could handle.  This field is not yet displayed by the
> Onionoo clients Atlas or Globe, but it's accessible via Onionoo's API
> [2].  More details are on ticket #16020 [3].
> 
> [2] https://onionoo.torproject.org/protocol.html#details
> 
> [3] https://trac.torproject.org/projects/tor/ticket/16020

Onionoo is great. Here are a couple of graphs that have to do with the
"Measured" flag.

The first one is a CDF of consensus weight (which maps to e.g. the
probability of using a relay in a circuit). Check out the big rise at a
consensus weight of 20. That's caused by the default bandwidth setting
of 20 KB/s when a relay is unmeasured:

https://gitweb.torproject.org/tor.git/tree/src/or/dirvote.h?id=tor-0.2.6.10#n89
/** Default bandwidth to clip unmeasured bandwidths to using method >=
 * MIN_METHOD_TO_CLIP_UNMEASURED_BW.  (This is not a consensus method; 
do not
 * get confused with the above macros.) */
#define DEFAULT_MAX_UNMEASURED_BW_KB 20

The second graph shows the age of all currently running relays (first
seen date) with the consensus weight. The unmeasured relays are colored
differently. See how they all fall on the 20 KB/s line.
library(ggplot2)
library(rjson)

details <- fromJSON(file="https://onionoo.torproject.org/details?type=relay";)
# Convert JSON into a data.frame.
relays <- Filter(function(x) { x[["running"]] }, details[["relays"]])
relays <- do.call(rbind, lapply(relays, function(x) {
as.data.frame(c(
x[c("nickname", "measured", "consensus_weight", "first_seen")],
exit=(("Exit" %in% unlist(x["flags"])) & !("BadExit" %in% 
x["flags"]))
))
}))
relays$first_seen <- as.POSIXct(relays$first_seen, tz="GMT")

p <- ggplot(relays, aes(consensus_weight))
p <- p + stat_ecdf()
p <- p + scale_x_log10()
p <- p + labs(title=NULL, x="Consensus weight (KB/s)", y="Fraction")
ggsave("consensus_weight.png", p, width=7, height=4, dpi=120)

p <- ggplot(relays, aes(first_seen, consensus_weight, color=measured))
p <- p + geom_point(alpha=0.2)
p <- p + scale_y_log10()
p <- p + labs(title=NULL, x="First seen date", y="Consensus weight (KB/s)")
p <- p + guides(color=guide_legend(override.aes=c(alpha=1)))
ggsave("first_seen.png", p, width=7, height=4, dpi=120)
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Video of my presentation on domain fronting (PETS 2015)

[David Fifield]

My presentation on domain fronting (the meek pluggable transport in
Tor), which I gave at PETS 2015 on June 30, 2015, is online. This page
has the video and a copy of the slides. It is about 17 minutes long.

https://www.bamsoftware.com/talks/fronting-pets2015/

Here is the paper on which the talk is based, by Chang Lan, Rod Hynes,
Percy Wegmann, Vern Paxson, and me:

https://www.bamsoftware.com/papers/fronting/
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] patch to improve consensus download decompress performance

[David Fifield]

On Fri, Aug 28, 2015 at 11:52:22AM +1000, Tim Wilson-Brown - teor wrote:
> To proceed with the patch, we need to know / decide:
> 
> What is the range of compression ratios on recent microdescriptors and
> microdescriptor consensuses? Do they vary much?
> (Does someone have an archive somewhere?)

Recent:
https://collector.torproject.org/recent/relay-descriptors/microdescs/consensus-microdesc/
Archived:
https://collector.torproject.org/archive/relay-descriptors/microdescs/
Format info:
https://collector.torproject.org/formats.html#relay-descriptors
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Where are the GetTor downloads (e.g. GitHub)?

[David Fifield]

Someone IRL asked me about mirrored downloads of Tor Browser on services
like GitHub. I know that such services are planned (or already
implemented?) for GetTor, and I know about these tickets:

Integrate cloud services that are not blocked in mainland China
https://bugs.torproject.org/14114

Automate upload of latest Tor Browser to cloud services
https://bugs.torproject.org/14744

Script to upload Tor Browser to Github
https://bugs.torproject.org/14835

My question is, is there a static URL on GitHub or similar that has the
latest downloads? That is, one that people can access even without
having used GetTor? Such a URL would be more useful than a typical
mirror for many users.

I was referred to Lantern's GitHub download page for comparison:
https://github.com/getlantern/lantern/releases/tag/latest
It has direct links to downloads under githubusercontent.com:
https://raw.githubusercontent.com/getlantern/lantern-binaries/master/lantern-installer.exe
https://raw.githubusercontent.com/getlantern/lantern-binaries/master/lantern-installer.dmg
https://raw.githubusercontent.com/getlantern/lantern-binaries/master/lantern-installer-64.deb
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Where are the GetTor downloads (e.g. GitHub)?

[David Fifield]

On Mon, Aug 31, 2015 at 10:07:17PM -0300, ilv wrote:
> Hi David,
> 
> > 
> > My question is, is there a static URL on GitHub or similar that has the
> > latest downloads? That is, one that people can access even without
> > having used GetTor? Such a URL would be more useful than a typical
> > mirror for many users.
> > 
> 
> Yes, you can find the latest Tor Browser versions here:
> 
> https://github.com/gettorbrowser/dl
> 
> Although you will need to know which is the latest version in order to
> construct a direct download link. I might work on that by the end of my
> SoP if people find it useful.

Thanks, that's great!

> We are also trying to get an official account for it, but it's still on
> progress (https://trac.torproject.org/projects/tor/ticket/10692).

I think this is an important job, so I hope you get whatever support you
need to get official accounts.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Reproducibility of Pluggable Transports python.msi

[David Fifield]

On Sun, Sep 06, 2015 at 11:26:16PM +, Jeremy Rand wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
> 
> I was looking at the Gitian descriptor for the pluggable transports at
> https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/gitia
> n/descriptors/windows/gitian-pluggable-transports.yml
> , and I noticed that it has an input file called "python.msi".
> Furthermore, I noticed the following line in
> https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/gitia
> n/versions
> :
> 
> PYTHON_MSI_URL=https://www.python.org/ftp/python/${PYTHON_VER}/${PYTHON_
> MSI_PACKAGE}
> 
> - From this, I conclude that Python is not being built in Gitian, and
> the download from www.python.org is assumed to be safe / not
> backdoored.  Is this correct?
> 
> If I'm correct, is there a reason that Python is not being built in
> Gitian?  Was it attempted and found that Python cannot easily be built
> for Windows in Gitian?  Or was it not attempted and just still on the
> to-do list?  I don't see any relevant ticket on Trac.

Way way back when pluggable transports were first integrated into Tor
Browser, we tried compiling Python and it was too problematic to be
worth it. Here is the comment you want to read:

https://trac.torproject.org/projects/tor/ticket/9444#comment:18
https://trac.torproject.org/projects/tor/ticket/9444#comment:20

Those comments are two years old now. Maybe things have changed and it's
easier to cross-compile for Windows now. If it's something you have
expertise with, it'd be great if you tried it!
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Reproducibility of Pluggable Transports python.msi

[David Fifield]

On Wed, Sep 09, 2015 at 03:33:24PM -0400, Brandon Wiley wrote:
> I am in favor of standardizing on the Go codebase for pluggable transports 
> that
> ship with Tor. This is something we talked about at the last developer 
> meeting.
> The reason I favor this is not for reproducible build reasons, but because
> maintaining four implementations (C, Python, C++, and Go) is confusing for PT
> developers. As far as I know, since the last developer meeting all Tor 
> products
> have been migrating towards shipping the Go PT implementation so that they can
> get obfs4 support. Last I checked, some of Tor products are also shipping 
> other
> PT implementations in order to maintain access to transports not available in
> Go. I imagine that there is some time in the future where there will no longer
> be any bridges available for the older transports and so bundling clients for
> them will no longer be necessary. However, I don't know what the current level
> of use for non-Go transports is. I'd love to know if someone has those stats.

You can see the usage of each transport here:

https://metrics.torproject.org/userstats-bridge-transport.html?graph=userstats-bridge-transport

obfs2 - Go
obfs3 - Go
obfs4 - Go
meek - Go
ScrambleSuit - Go
flash proxy - Python
FTE - Python
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Hello, I am a Tor Browser user in China. Currently, many obfs4 bridges are blocked by China's firewall.

[David Fifield]

On Thu, Oct 01, 2015 at 08:55:33PM +0800, Li Xiaodong wrote:
> Hello, I am a Tor Browser user in China. Currently, many obfs4 bridges are
> blocked by China's firewall. When will SkypeMorph Pluggable Transports and
> Dust Pluggable Transports be deployed in Tor Browser? There are no directory
> servers in I2P network. Can Tor learn from I2P? If Tor user have to access
> directory servers by bridges, I feel that Tor will be easily blocked by 
> China's
> firewall. Can Tor no longer use directory servers in future? Since 2010 
> China's
> firewall has blocked all of the IP addresses of Tor directory servers. Thank
> you very much for your help. I really appreciate it. All the best with your
> work. Good luck in all that you will achieve.

Tor does not use directory servers when you are using bridges (including
obfs4 bridges). The bridge has a copy of the directory information.
Blocking the directory servers does not cause bridges to be blocked. But
the IP address of the bridge itself could be blocked. It seems that this
is what has happened in your case.

I don't know about a schedule for deploying SkypeMorph and Dust. They
may not help in your case anyway. The GFW is probably blocking the IP
addresses of your bridges, not detecting the obfs4 protocol itself.

You might have luck with the meek-amazon or meek-azure transports. They
are not as likely to have their IP addresses blocked. With a little
effort, you can customize it to use domains that you choose.

https://trac.torproject.org/projects/tor/wiki/doc/meek#Quickstart
https://program-think.blogspot.com/2014/10/gfw-tor-meek.html
http://www.atgfw.org/2015/02/torgfwpk1-meektor.html
https://plus.google.com/+GhostAssassin/posts/26zCmDmjYXP
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Hello, thank you very much for your help. I really appreciate it.

[David Fifield]

On Fri, Oct 02, 2015 at 12:05:49AM +0800, Li Xiaodong wrote:
> Hello, thank you very much for your help. I really appreciate it. In this
> afternoon of China Time, I found a obfs4 bridge which is usable in China. The
> speed of Tor Browser connecting with obfs4 bridge, and the speed of Tor 
> Browser
> connecting with Meek Azure, which is faster? It is more difficult for China's
> firewall to block I2P. If Tor can learn some strong points of I2P in future,
> Tor can work better in China. Thank you very much for your help. I really
> appreciate it. All the best with your work. Good luck in all that you will
> achieve.

Most likely, obfs4 will be faster than meek-azure. Especially after
tomorrow, when I will slow down meek-azure to reduce costs. See:
https://lists.torproject.org/pipermail/tor-dev/2015-September/009533.html

If you know some details of how I2P resists blocking, please add them to
this wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports
To edit the wiki you can create an account:
https://trac.torproject.org/projects/tor/register
Or use the shared cypherpunks/writecode account.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Networks Blocking Tor's SSL Connections

[David Fifield]

On Wed, Oct 07, 2015 at 10:06:00AM +1100, Tim Wilson-Brown - teor wrote:
> Hi All,
> 
> This morning I observed a “free wifi” network blocking tor’s SSL connections.
> While other SSL connections from my machine went through, I observed multiple
> network traces of tor completing a TCP 3-way handshake, and then getting no
> reply to the first SSL packet it sent.
> 
> I think they may have been blocking unknown or untested certificates, but I
> can’t be sure.
> 
> Still, I was able to use meek(-google) to access tor.
> 
> Has anyone else seen this kind of blocking behaviour?
> (Is this the right list?)

I don't know about specific instances in the free wifi scenario, but
some national censorship systems work that way, observing something
about the handshake rather than blacklisting the IP addresses of
directory authorities or relays.

Iran filters Tor by ssl handshake, Sept 2011
https://bugs.torproject.org/4014

GFW probes based on Tor's SSL cipher list (Dec 2011)
https://bugs.torproject.org/4744

Ethiopia blocks Tor based on ServerHello (Jun 2012)
https://bugs.torproject.org/6045

Kazakhstan uses DPI to block Tor (Jun 2012)
https://bugs.torproject.org/6140

UAE uses DPI to block Tor (Jun 2012)
https://bugs.torproject.org/6246

The Philippines are blocking Tor? (Jun 2012)
https://bugs.torproject.org/6258

How is Iran blocking Tor? (Oct 2012)
https://bugs.torproject.org/7141

SSL handshake filtered when MAX_SSL_KEY_LIFETIME_ADVERTISED is 365 days (Mar 
2013)
https://bugs.torproject.org/8443
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Getting meek covered by a CDN for free

[David Fifield]

On Mon, Oct 12, 2015 at 07:06:05PM +, Virgil Griffith wrote:
> I met with some CDNs today and they have expressed interest in doing meek for
> us.
> 
> Is there someone at Tor Project I can forward the CDNs to who are more serious
> about hosting meek?

Sending them to me (da...@bamsoftware.com or d...@torproject.org) is a
good idea. I have handled all of Tor's meek deployments so far.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Feedback on CollecTor web redesign

[David Fifield]

On Tue, Oct 20, 2015 at 09:31:38PM +0200, Karsten Loesing wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Hi devs,
> 
> I just finished a redesign of the CollecTor website and would
> appreciate your feedback:
> 
>   https://metrics.torproject.org/index2.html

This URL is 404 for me.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Hello, after about one hour since my Tor browser successfully connected to Tor network with a obfs4 bridge, I couldn't open any webpage through Tor proxy.

[David Fifield]

On Thu, Oct 22, 2015 at 11:53:15PM +0800, Li Xiaodong wrote:
> Hello, after about one hour since my Tor browser successfully connected to Tor
> network with a obfs4 bridge, I couldn't open any webpage through Tor proxy. 
> But
> after I restarted my Tor browser, Tor browser can work normally again. Does
> China's firewall disturb obfs4 bridges? Thank you very much for your help. I
> really appreciate it.

I'm not aware of any disruption of obfs4 bridges, other than IP blocking
of some bridges.

If this continues to happen, you can help us by starting a ticket on the
bug tracker.
https://trac.torproject.org/projects/tor/register
https://trac.torproject.org/projects/tor/newticket
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] A layered transport

[David Fifield]

On Mon, Oct 26, 2015 at 03:44:59PM +0800, Da Feng wrote:
> Hi:
>I've discovered that the GFW normally doesn't block https
> protocols. We can use a https front tier to distribute connections to
> actual bridges.

This is a good idea. HTTPS is a good cover protocol.

> The front tier encrypts an internal address identifier with its
> private key (no matching public key or public algorithm) and returns
> to user the encrypted identifier, part of which also includes the
> user's chosen password. Then when submitting requests, the user
> encrypt again with his password the items such as his timestamp,
> broswer headers.

If I understand this correctly, this is a defense against active
probing. You are saying that every HTTPS bridge has a password, and the
client must provide the correct password in order to use the bridge.
(Like ScrambleSuit or obfs4.) If the GFW tries to connect to the bridge,
it does not know the password, and therefore it cannot use the bridge.

> The request line to https server is no different from an ordinary one
> and include both the user encrypted item and front tier encrypted
> item. After the connection is established, data is relayed inside
> https between bridge and user.

You can already do something like this, except for the active probing
defense, using a PHP script and Tor Browser. First install this file on
an HTTP server:

https://gitweb.torproject.org/pluggable-transports/meek.git/tree/php/index.php

(Let's say it is installed at https://example.com/meek/index.php.) Then,
in the Tor Browser bridge configuration, enter the bridge line:

meek 0.0.2.0:1 url=https://example.com/meek/index.php

See "What to do if meek gets blocked" for some more options:
https://lists.torproject.org/pipermail/tor-talk/2015-January/036410.html
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] [tor-commits] [tor-messenger-build/master] Bug 17492: Include default bridges configuration

[David Fifield]

On Fri, Nov 06, 2015 at 08:29:19PM -0500, Roger Dingledine wrote:
> On Sat, Nov 07, 2015 at 12:13:45AM +, bo...@torproject.org wrote:
> > commit 7a1c6fd121dd001eb999ef03ebbbed264da37026
> > Author: Nicolas Vigier 
> > Date:   Sat Nov 7 00:45:48 2015 +0100
> > 
> > Bug 17492: Include default bridges configuration
> > 
> > However, we exclude meek from the bridges configuration, as it currently
> > doesn't work in Tor Messenger because it requires firefox.
> 
> This could be a great opportunity for Yawning's recent meek variant that
> doesn't use an actual browser. (It won't work as well as the real meek,
> but maybe no actual adversaries care about the difference quite yet?)

You can also just run meek-client without the Firefox helper. It has a
standalone mode. Just change the torrc line

ClientTransportPlugin meek exec 
./TorBrowser/Tor/PluggableTransports/meek-client-torbrowser -- 
./TorBrowser/Tor/PluggableTransports/meek-client

to

ClientTransportPlugin meek exec ./TorBrowser/Tor/PluggableTransports/meek-client

There's a man page for meek-client:
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/doc/meek-client.1.txt
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Summary of meek's costs, October 2015

[David Fifield]

Here's the summary of meek's CDN fees for October 2015.

App Engine +   Amazon +   Azure = total by month
February 2014$0.09 +   -- +  -- = $0.09
March 2014   $0.00 +   -- +  -- = $0.00
April 2014   $0.73 +   -- +  -- = $0.73
May 2014 $0.69 +   -- +  -- = $0.69
June 2014$0.65 +   -- +  -- = $0.65
July 2014$0.56 +$0.00 +  -- = $0.56
August 2014  $1.56 +$3.10 +  -- = $4.66
September 2014   $4.02 +$4.59 +   $0.00 = $8.61
October 2014$40.85 +  $130.29 +   $0.00 =   $171.14
November 2014  $224.67 +  $362.60 +   $0.00 =   $587.27
December 2014  $326.81 +  $417.31 +   $0.00 =   $744.12
January 2015   $464.37 +  $669.02 +   $0.00 =  $1133.39
February 2015  $650.53 +  $604.83 +   $0.00 =  $1255.36
March 2015 $690.29 +  $815.68 +   $0.00 =  $1505.97
April 2015 $886.43 +  $785.37 +   $0.00 =  $1671.80
May 2015   $871.64 +  $896.39 +   $0.00 =  $1768.03
June 2015  $601.83 +  $820.00 +   $0.00 =  $1421.83
July 2015  $732.01 +  $837.08 +   $0.00 =  $1569.09
August 2015$656.76 +  $819.59 + $154.89 =  $1631.24
September 2015 $617.08 +  $710.75 + $490.58 =  $1818.41
October 2015   $672.01 +  $110.72 + $300.64 =  $1083.37
--
total by CDN  $7443.58 + $7987.32 + $946.11 = $16377.01 grand total

https://metrics.torproject.org/userstats-bridge-transport.html?graph=userstats-bridge-transport&start=2015-08-01&end=2015-10-31&transport=meek

In October we had about 4,000 simultaneous users all month.

There was an unfortunate outage of meek-amazon (not the result of
censorship, just operations failure). Between 30 September and 9 October
the bridge had an expired HTTPS certificate. 
[tor-talk] Outage of meek-amazon
https://lists.torproject.org/pipermail/tor-talk/2015-October/039231.html
https://lists.torproject.org/pipermail/tor-talk/2015-October/039234.html
And then, as a side effect of installing a new certificate, the bridge's
fingerprint changed, which caused Tor Browser to refuse to connect. It
used to be that we didn't include fingerprints for the meek bridges, but
now we do, so we didn't anticipate this error and didn't notice it
quickly.
Update the meek-amazon fingerprint to 
B9E7141C594AF25699E0079C1F0146F409495296
https://trac.torproject.org/projects/tor/ticket/17473
[tor-talk] Changed fingerprint for meek-amazon bridge (attn support)

https://lists.torproject.org/pipermail/tor-talk/2015-November/039397.html
Interestingly, the meek-amazon bridge still had about 400 simultaneous
users (not as much as normal) during the time when the fingerprint
didn't match. I would have expected it to go almost to zero. Maybe it's
people using an old version of Tor Browser (from before March 2015) or
some non–Tor Browser installation.

Our grant for meek-azure ran out and now it costs money. Accordingly
I've rate-limited it to limit costs. I set it to 1.1 MB/s on 2 October
and to 0.8 MB/s on 30 October.
[tor-talk] meek-azure now rate-limited
https://lists.torproject.org/pipermail/tor-talk/2015-October/039169.html

meek-google was also out for a few days around 30 October because I
messed up the app upload and pointed it to the wrong bridge.

If you want to set up your own bridge and CDN instance without rate
limiting, I can help you do it. Here are some docs to look at:
https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-serverbridge
https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine
https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront
https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure


== App Engine a.k.a. meek-google ==

Here is how the Google costs broke down:
2842 GB $341.06
6619 instance hours $330.95
Compared to the previous month:
2871 GB $344.53
5451 instance hours $272.55

https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4534


== Amazon a.k.a. meek-amazon ==

Usage of meek-amazon was quite low this month because of bridge outages
having to do with an expired HTTPS certificate and a changed relay
fingerprint.

Asia Pacific (Singapore)   6M requests  $8.24   36 GB  $5.05
Asia Pacific (Sydney)635K requests  $0.791 GB  $0.21
Asia Pacific (Tokyo)   1M requests  $1.715 GB  $0.71
EU (Ireland)  39M requests $46.81  130 GB  $9.97
South America (Sao Paulo)  2M requests  $4.985 GB  $1.23
US East (Northern Virginia)   24M requests $24.36   86 GB  $6.64
--
total 74M requests $86.89  266 GB $23.81


https://globe.torproject.org/#/bridge/F4AD82B2032EDEF6C02C5A529C42CFAFE516564D
(Note new fingerprint.)


== Azure a.k.a. meek-azure ==

Zone 1  1652 GB  $202.51
Zone 2   586 GB   $98.13
--
total   2238 GB  $300.64

https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62A

Re: [tor-dev] Summary of meek's costs, October 2015

[David Fifield]

On Fri, Nov 20, 2015 at 05:50:51PM -0600, Tom Ritter wrote:
> On 18 November 2015 at 16:32, David Fifield  wrote:
> > There was an unfortunate outage of meek-amazon (not the result of
> > censorship, just operations failure). Between 30 September and 9 October
> > the bridge had an expired HTTPS certificate.
> > [tor-talk] Outage of meek-amazon
> > 
> > https://lists.torproject.org/pipermail/tor-talk/2015-October/039231.html
> > 
> > https://lists.torproject.org/pipermail/tor-talk/2015-October/039234.html
> > And then, as a side effect of installing a new certificate, the bridge's
> > fingerprint changed, which caused Tor Browser to refuse to connect. It
> > used to be that we didn't include fingerprints for the meek bridges, but
> > now we do, so we didn't anticipate this error and didn't notice it
> > quickly.
> > Update the meek-amazon fingerprint to 
> > B9E7141C594AF25699E0079C1F0146F409495296
> > https://trac.torproject.org/projects/tor/ticket/17473
> > [tor-talk] Changed fingerprint for meek-amazon bridge (attn support)
> > 
> > https://lists.torproject.org/pipermail/tor-talk/2015-November/039397.html
> > Interestingly, the meek-amazon bridge still had about 400 simultaneous
> > users (not as much as normal) during the time when the fingerprint
> > didn't match. I would have expected it to go almost to zero. Maybe it's
> > people using an old version of Tor Browser (from before March 2015) or
> > some non–Tor Browser installation.
> 
> It seems like it would be better to use the SPKI rather than the cert
> fingerprint, this would allow you to reissue the same key and keep
> things working for older clients.

The fingerprint I'm talking about is the relay fingerprint, not the
HTTPS/X.509 one. The HTTPS certificate and the relay identity
fingerprint are completely independent. It just happened that in this
case, the relay was so configured, that when it rebooted to start using
the new HTTPS cert, it also generated a new identity key.

We're not pinning the HTTPS cert and in fact we can't; it's just used
for confidentiality on the CDN↔meek-server link.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Graphs - Estimated Traffic Capacity

[David Fifield]

On Fri, Nov 20, 2015 at 01:38:56PM -0500, David Goulet wrote:
> Anyway, if you think this algorithm could be improved, please respond. If you
> think this algorithm is wrong, please respond. If you can reproduce the result
> on your own with this algo, omg please respond! :) The above could be totally
> wrong but hopefully we did a fairly good job. Please remember, this is an
> estimate. :)

I would be interested to see the capacity by pluggable transport: how
much headroom each PT has and which ones need more bridges.

I tried it a few months ago but I didn't have the idea of looking at the
read/write bytes history. I tried using measured/advertised bandwidth
and using the speed of consensus downloads from the
dirreq-v3-tunneled-dl line.

[tor-dev] Per-transport bridge bandwidth and bridge counts
https://lists.torproject.org/pipermail/tor-dev/2015-July/009136.html
Using measured bandwidth:
https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-bandwidth.png
Using median dirreq-v3-tunneled-dl rate:
https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20/pt-dl.md.png
Source code:
https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-20.tar.gz
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Better relay uptime visualisation

[David Fifield]

On Mon, Dec 07, 2015 at 02:51:23PM -0500, Philipp Winter wrote:
> I spent some time improving the existing relay uptime visualisation [0].
> Inspired by a research paper [1], the new algorithm uses single-linkage
> clustering with Pearson's correlation coefficient as distance function.
> The idea is that relays are grouped next to each other if their uptime
> (basically a binary sequence) is highly correlated.  Check out the
> following gallery.  It contains monthly relay uptime images, dating back
> to 2007:
> 

How about just taking the XOR of two sequences as the distance?

It would be interesting to know if there are any near-perfect
anticorrelations; i.e., one relay starts when another stops.

> Another practical problem is that it's cumbersome to learn the relay
> fingerprint of a given column.  I'm looking into JavaScript/HTML tricks
> that can show text when you hover over a region in the image.  Perhaps
> somebody knows more?

One way is to set an onmousemove handler that inserts text into a
preexisting element. For example (untested):




var OUTPUT_ELEM = document.getElementById("output");
/* Get an event's coordinates relative to a given element. */
function elem_coords(event, elem) {
var rect = elem.getBoundingClientRect();
/* http://stackoverflow.com/a/872537 */
if (typeof pageXOffset !== "undefined") {
scrollLeft = pageXOffset;
scrollTop = pageYOffset;
} else if (document.documentElement !== undefined && 
document.documentElement.clientHeight !== undefined) {
scrollLeft = document.documentElement.scrollLeft;
scrollTop = document.documentElement.scrollTop;
} else {
scrollLeft = document.body.scrollLeft;
scrollTop = document.body.scrollTop;
}
var x = event.pageX - (scrollLeft + rect.left);
var y = event.pageY - (scrollTop + rect.top);
return { x: x, y: y };
}
function onmousemove_callback(event) {
var c = elem_coords(event, img_element);
OUTPUT_ELEM.innerText = get_text_for_coordinates(c.x, c.y);
}
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Better relay uptime visualisation

[David Fifield]

On Tue, Dec 08, 2015 at 10:47:08AM +1100, Tim Wilson-Brown - teor wrote:
> 
> On 8 Dec 2015, at 10:43, Tom Ritter <[1]t...@ritter.vg> wrote:
> 
> On 7 December 2015 at 13:51, Philipp Winter <[2]p...@nymity.ch> wrote:
> 
> I spent some time improving the existing relay uptime visualisation
> [0].
> Inspired by a research paper [1], the new algorithm uses 
> single-linkage
> clustering with Pearson's correlation coefficient as distance 
> function.
> The idea is that relays are grouped next to each other if their uptime
> (basically a binary sequence) is highly correlated.  Check out the
> following gallery.  It contains monthly relay uptime images, dating
> back
> to 2007:
> <[3]https://nymity.ch/sybilhunting/uptime-visualisation/>
> 
> If you aren't familiar with this type of visualisation: Every image
> shows the uptime of all Tor relays that were online in a given month.
> Every row is a consensus and every column is a relay.  White pixels
> mean
> that a relay was offline and black pixels means that a relay was
> online.  Red pixels are used to highlight suspiciously similar
> clusters.
> 
> 
> That's really cool.  It seems to imply that the majority of the tor
> network stop operating halfway through the month though... Do the
> other tor graphs take into account hibernating relays?  For example, I
> would expect the time-to-download graph would be somewhat affected:
> [4]https://metrics.torproject.org/torperf.html?graph=torperf&start=
> 2015-10-01&end=2015-10-31&source=all&filesize=5mb
> 
> 
> Hibernating relays run from the start of their first period to gauge load.
> Then they start at a random time during the day/month, but early enough that
> they think they'll still use all their bandwidth.
> 
> I wonder if we're seeing another phenomenon? (daily / monthly server 
> restarts?)
> Or we could be seeing hibernation failing to work as intended.

Relays turn on or off all the time. Of all the descriptors seen in a
year, less than 10% are continuously running the whole time. The rest
either started at some time or stopped at some time or both. See an
example here for 2014:

https://people.torproject.org/~dcf/graphs/microdescs/microdescs-2014-short.png
All we're seeing is the distributions of the dates at which the subset
of relays that stopped during the month actually stopped, which seems
pretty uniform. I'll bet that if you look at those relays in the
previous month, they are running at the end of the month, not
hibernating.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Bitcoin-paid hidden meek relays?

[David Fifield]

On Thu, Dec 10, 2015 at 12:09:36PM +0100, Jacek Wielemborek wrote:
> List,
> 
> I'm starting this thread in relation to the following discussion:
> 
> https://security.stackexchange.com/questions/107490/how-will-france-block-tor
> 
> I definitely don't have a perfect knowledge of Tor, so correct me if I'm
> wrong at any point. I heard that Tor entry nodes right now are
> distributed to users as IP addresses, which are scarce resources that
> could be blocked.
> 
> In order to make this more difficult, pluggable transports were
> introduced - those try to use other protocols to define communication
> with Tor network, lowering the cost of introducing new way to connect
> with it.
> 
> One that particularly caught my attention is Meek, which uses CDNs to
> evade censorship. CDNs share same IPs for many critical Internet
> serices, so blocking them is not an option and having them behind TLS
> makes it even more complicated.
> 
> The problem I noticed though is that the costs of Meek go up and if I
> read the reports from David Fifield (the maintainer of Meek), the
> bandwidth has to be limited to avoid abuse. This slows the transport
> down and I thought of another approach.
> 
> Instead of using a limited amount of Meek nodes, we could encourage
> users to run those on their own and distribute them in a way similar to
> regular Tor entry nodes. The catch would be that those nodes would
> require a small Bitcoin pre-payment that covers the cost of bandwidth
> used. The node would require the user to pass some proof of payment
> during the connection and not respond at all if there was no payment
> made. Here's an example way this could work:
> 
> 1. User sends a message to brid...@bridges.torproject.org with a request
> for a meek bridge and a GPG public key attached,
> 2. The bridge service replies with a Bitcoin address created just for
> the user,
> 3. As soon as the user sends any Bitcoins, it receives another e-mail
> with the hostname of the Meek relay and a token that one should use
> 
> Together with Amazon AWS, this could be an automated solution to the
> bandwidth and payment problem that would result in short-lived relays
> that are difficult to detect.

I like the idea of bridges that users pay for according to their use.
There are a lot of unanswered questions about how exactly the payments
and accounting work. It seems to me that this is a case where the
details are very important (I could be wrong).

I know of some research along these lines. "Proof-of-Work as Anonymous
Micropayment: Rewarding a Tor Relay" by Alex Biryukov and Ivan
Pustogarov (they worked out some of the protocol details):
In this paper we propose a new micropayments scheme which can be
used to reward Tor relay operators. Tor clients do not pay Tor
relays with electronic cash directly but submit proof of work
shares which the relays can resubmit to a crypto-currency mining
pool. Relays credit users who submit shares with tickets that
can later be used to purchase improved service.

One misunderstood thing about meek: it doesn't require a CDN to work.
You can fairly easily upload your own meek proxy (as a PHP or Python
script) to any HTTPS web host. The CDN and its collateral damage are
only required when you make the proxies widely known, as we do with
meek-google, meek-amazon, and meek-azure in Tor Browser. If you run your
own and do not share it widely, then you do not need a CDN.

For example, here is the PHP version:
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/php
Upload the index.php file to a web host service that supports HTTPS.
Then configure its URL as a bridge line in Tor Browser:
meek 0.0.2.0:4 url=https://mysite.example.com/index.php

It's conceivable that we could even distribute such bridge lines in
BridgeDB. People could upload their PHP files and we could have a lot of
meek bridges. But BridgeDB is not currently set up for that. I don't
know exactly how it works, but BridgeDB assumes that the bridge is
running at the same place as a full Tor relay, which isn't the case for
a PHP forwarder like this. We also don't have a way for such PHP proxies
to automatically self-publish their URLs for others to use.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Go version in Gitian descriptors

[David Fifield]

On Sun, Jan 03, 2016 at 11:01:25PM -0600, Jeremy Rand wrote:
> I noticed that it looks like Tor Project is using Go 1.4.2 to build
> the pluggable transports in Gitian.  I'm curious why a newer version
> of Go isn't used.  My understanding is that Go 1.4.2 (or earlier) is
> needed to build Go 1.5 because 1.5's source code is itself in Go.
> Would using Go 1.5 be as simple as building 1.4.2 in Gitian (as is
> done now), and then using 1.4.2 to build 1.5, and then placing 1.5 in
> PATH instead of 1.4.2 as is done now?  Have obstacles been identified
> in such a configuration, or is it just that no one tried it?

It's just that nobody has tried it. Here's the ticket for the most
recent update (to 1.4.2) if you want to make a new patch and file a new
ticket:
https://bugs.torproject.org/15448
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Summary of meek's costs, December 2015

[David Fifield]

On Mon, Jan 11, 2016 at 01:19:22PM -0900, Jesse V wrote:
> On 01/11/2016 12:47 PM, David Fifield wrote:
> > December 2015  $561.29 +  $603.27 +  $172.60 =  $1337.16
> > ...
> > The number of users increased by about 1,000 in December 2015.
> 
> Thanks for the report, David.
> 
> There are now ~1800 users on meek-google sharing 8 Mbits. Of course,
> it's also possible that many are lurking. I wonder if we should think of
> a way to load-balance the CDNs a bit more, assuming Azure is accessible
> from China. Perhaps we could re-order or at least clarify the bridge
> options in the Tor Browser. Since it's logistically challenging to get
> bridges from inside a censored country, it's possible that users try the
> various bridge options using the built-in bridges (which are likely
> blocked) before discovering that meek works, so they use that. I wonder
> how we could encourage users to use other transports that are likely
> faster, then use meek as a last resort.

Azure is accessible in China but it's even more severely rate limited
because it's more expensive than Google. We still have support from
Google, so that $561.29 actually costs about $61.29.

I think you're right that people are choosing meek because it works
without any setup. We have a kind of long-term goal to do a
domain-fronted interface to BridgeDB, so that people could get obfs4
bridges automatically and directly within Tor Launcher.
https://bugs.torproject.org/16550
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Summary of meek's costs, December 2015

[David Fifield]

On Mon, Jan 11, 2016 at 02:51:12PM -0900, Jesse V wrote:
> On 01/11/2016 02:42 PM, David Fifield wrote:
> > We still have support from
> > Google, so that $561.29 actually costs about $61.29.
> 
> Oh, I was not aware of this. When does the support expire, and how much
> would it cost (in terms of Tor's budget) to double the bandwidth to 16
> Mbits, for instance? I'm just thinking about the implications if there
> were donations for meek.

It expires pretty soon. March, I think?

I don't know how much usage will increase with a higher rate limit, but
I'm sure that we could saturate 24 Mbps, because that's about where we
were before instituting rate limiting. I think of it the other way:
rather than picking a speed and figuring out what it will cost, set a
cost and then adjust your speed to meet it.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] How many exits exit from an IP address different than their OR address? (10.7%)

[David Fifield]

I wanted to know how many exits exit from an address that is different
from their OR address. The answer is about 10.7%, 109/1018 exits. The
interesting part is that of those 109 mismatches, 87 have an exit
address that differs from the OR address in all four octets; i.e., the
IP addresses used by the exit are not even in the same /8.

Also, there are several groups of exits whose OR addresses *are* in a
related subnet, but which all exit through the same, unrelated, IP
address. See for example 109.236.82.* (all exiting through
185.108.128.7), 178.17.171.* (all exiting through 37.48.65.71), and
217.23.11.* (all exiting through 85.17.31.120) in the table below.

I wrote an exitmap module that hits an IP-checker website. The module
wrote a CSV file where each row included the exit's OR address and the
address as reported by the remote web site. I ignored IPv6 OR addresses.
I did not find any fingerprint that had more than one IPv4 address
(considering the .fingerprint and .or_addresses members from Stem).
Percentages are just raw counts, not weighted by exit probability.


Total number of exits: 1018 (100.0%)
Number of failures: 18 (1.8%)
Number of matches: 891 (87.5%)
Total number of mismatches: 109 (10.7%)
Number of 1-octet mismatches: 19 (1.9%)
Number of 2-octet mismatches: 3 (0.3%)
Number of 3-octet mismatches: 0 (0.0%)
Number of 4-octet mismatches: 87 (8.5%)

Table of mismatches
   * = mismatch 1 octet
  ** = mismatch 2 octets
 *** = mismatch 3 octets
 = mismatch 4 octets
  fingerprint  or_address  remote_address mark
 B060482C784788B8A564DECD904E14CB305C8B38  176.10.104.241  176.10.104.240*
 DC41244B158D1420C98C66F7B5E569C09DCE98FE  176.10.104.241  176.10.104.240*
 487092BA36F4675F2312AA09AC0393D85DAD6145  176.10.104.244  176.10.104.243*
 DE7DE889E0D1A5F397AE35642060B84999581203  176.10.104.244  176.10.104.243*
 7A65CC8F45134F3393A0295EFFD2980DD885E8E2   176.123.6.157   176.123.6.155*
 A7E78D9880BB0793409D386D15E83B3B1236B19F  178.175.128.54  178.175.128.50*
 8BF22BEA5F854F2BB2E09C71260ADF590D355E36  213.61.149.125  213.61.149.100*
 80D73E75A30BEEF627604B7014753304764E0723  213.61.149.126  213.61.149.100*
 5D263037FC175596B3A344132B0B755EB8FB1D1C   31.185.27.203 31.185.27.1*
 C974508A98446F36169FB248655BCD50DF17F14C  37.130.227.134  37.130.227.133*
 170EC06D58E9094A027F4169514ADD98D30983A6   46.235.226.27  46.235.226.226*
 D3BE73ECE50DA9243DC0DF9DA6ED25027B82D38550.7.178.10150.7.178.100*
 FEDEF82551BB49DB54BE6C77D27AD0E20A8D6FD650.7.178.10250.7.178.100*
 10E13E340651D0EF66B4DEBF610B3C0981168107  77.247.181.164  77.247.181.162*
 06E123865C590189B3181114F23F0F13A7BC0E69  77.247.181.166  77.247.181.162*
 664BDC0344771122FC6C4F577BF0AEC3F4ED5456   80.73.242.142   80.73.242.130*
 9E3E47D7B92144CBAB0D487CCE192531F6A55833   81.209.35.111   81.209.35.112*
 410C286061266C562049178C6BF1E04060F51F7C   94.103.175.85   94.103.175.86*
 F4B72EA6FD0EACF652B6C200611F37244F2B31F3  94.242.228.107  94.242.228.108*
 62BEE61EB88D4A81C3BA3931D6FA999D706AC4D562.149.13.57   62.149.12.153   **
 1BC17B72F90998F0BAEE25E36FFC140C9D7A8D7A 85.119.82.485.119.84.57   **
 AA60D088E8317BBA3D2CF96C706AB4452FB2F57D  90.182.128.100   90.182.235.46   **
 4E915A5CCDBBDD4BD4B7F09A5AFB581AB1AD7EF2 107.181.178.204  104.156.228.72 
 03B4386E579BEBCF7605D0FB17A688B35C342D5D   109.236.82.50   185.108.128.7 
 C3B1E12BCA2F307B64737AB84993E7B414FD3D09   109.236.82.50   185.108.128.7 
 B9DC592CBC10EC7428730BF658F9211B8773800F   109.236.82.51   185.108.128.7 
 DE89B3B6954FFC0F76F6B37A05E2AD6EF3B046F1   109.236.82.51   185.108.128.7 
 CF9F5B88C0F95EB6C5547CE5FB38503049DF784D   109.236.82.52   185.108.128.7 
 57275A9356CFB1D36E64B117D44AB9F67711E483   109.236.82.53   185.108.128.7 
 D1E52741A7DCB4D51C7EFC82A654E2527516E1B0   109.236.82.53   185.108.128.7 
 7837F5B740223D19D8A0D3127D2A8EB9FDA59F67  148.163.73.112 206.217.208.162 
 B6045286F21E7078203E1B03C8E00669A2154724  148.163.73.112 206.217.208.162 
 D225C2EC81E044E3947BC49929E46003ED62C89F  151.80.164.147  46.166.186.238 
 23A69F271ACBFACB20467559D1C04F13DEF74B4B  153.92.126.135  128.127.105.94 
 4236DF494DC91E1245234AA676F1F993ECB067F5  153.92.126.135  128.127.105.94 
 27CE487BABE6C2128FB2D1A801C7CB715B48EC6B   153.92.126.19  168.1.6.34 
 C369EB6BFDFB996864746944544B82C9684C46EA   153.92.126.19  168.1.6.34 
 5453CAF677B8E754C4D5EFA1FEDD8ABB16950FCB  153.92.127.143168.1.10.226 
 756F68024C6E55FF7142AD933C135F88B552A932  153.92.127.143168.1.10.226 
 B0E7C83FA0C728997BB60CB0C758A3A3FDC3C1DB 173.243.112.148  162.216.46.165 
 E6BE2BEAAF4A543F9D94CD20CB459ECFEA3AD1B1 173.243.112.148  162.216.46.165 
 7585DDCEB14783C69960E777BA258F5B3948F1C9  176.126.85.175 159.122.133.198 
 CBBCD92AD9479C8FBB8AA17FF22C7EC206FF5B1A  176.126.85.175 159.122.133.198 
 9634A0F1FBE02EBF6E

Re: [tor-dev] Questions about censorship detection paper

[David Fifield]

On Tue, Jan 12, 2016 at 07:21:39AM +, John wrote:
> I ran into the technical report from George Danezis about an
> anomaly-based censorship-detection system for Tor. I have a few
> questions that I hope you can help me with.
> 
> Is there an implementation available of the approach described in the paper?

Go to
https://metrics.torproject.org/userstats-relay-country.html
and check "Show possible censorship events if available".

I think the source code is here:
https://gitweb.torproject.org/karsten/metrics-web.git/tree/detector

> The paper talks about finding anomalies in the time series of the number
> of requests to directory servers from certain jurisdictions. Where can I
> find this data about the requests to directory servers?

https://collector.torproject.org/

I think you want the "dirreq-v3-reqs" lines. For example, here is a
recently archived extra-info consensus:
https://collector.torproject.org/recent/relay-descriptors/extra-infos/2016-01-12-07-05-27-extra-infos
(The file will expire after a few days; after that go to
https://collector.torproject.org/archive/relay-descriptors/extra-infos/.)

Here's a line from the file showing the number of requests by country
for one of the relays:
dirreq-v3-reqs 
us=7808,ru=4400,de=4112,fr=2584,gb=1848,it=1352,es=1208,jp=1192,br=1184,nl=1024,ua=1008,ca=912,pl=728,au=696,md=680,in=584,se=464,ar=448,mx=448,id=384,be=336,at=320,tr=288,tw=288,ch=280,cz=264,il=264,kr=248,ir=240,th=232,kz=224,pt=224,gr=216,sg=216,fi=208,my=208,ro=192,dk=184,hu=176,ae=168,ve=168,by=160,cl=160,ie=160,vn=160,za=152,ph=144,no=136,co=128,hk=128,lv=128,nz=120,sk=120,bg=112,eg=112,pk=112,rs=104,bd=88,ma=80,ec=72,ng=72,dz=64,hr=64,pe=64,si=64,ee=56,lt=56,do=48,tn=48,az=40,ba=40,cr=40,is=40,jo=40,kw=40,qa=40,uy=40,af=32,am=32,bo=32,ci=32,cy=32,gh=32,gt=32,iq=32,ke=32,lu=32,pr=32,uz=32,et=24,ge=24,lb=24,mk=24,mn=24,sa=24,sd=24,sv=24,sy=24,tm=24,al=16,bh=16,bj=16,cu=16,gd=16,kg=16,kh=16,ly=16,ni=16,np=16,om=16,ps=16,re=16,sn=16,tt=16,??=8,ad=8,ao=8,as=8,aw=8,ax=8,bb=8,bf=8,bm=8,bn=8,bs=8,bw=8,cd=8,cg=8,cm=8,cn=8,cv=8,dm=8,gf=8,gm=8,gp=8,gq=8,gw=8,gy=8,hn=8,ht=8,im=8,jm=8,km=8,ky=8,li=8,lk=8,me=8,mg=8,mq=8,mr=8,mt=8,mu=8,mz=8,na=8,nc=8,ne=8,pa=8,pf=8,pg=8,pm=8,p
 y=8,rw=8,sc=8,sm=8,sr=8,tj=8,tz=8,ug=8,vi=8,vu=8,ye=8,yt=8,zm=8,zw=8
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Questions about censorship detection paper

[David Fifield]

On Tue, Jan 12, 2016 at 11:49:19PM +, John wrote:
> Hi David,
> 
> Thank you, these pointers were very helpful. Do you know if there is
> some kind of resource that lists known censorship events? I'd like to
> see how good the approach from the paper does at identifying them.

For Tor-specific censorship, a list that is reasonably complete and up
to date is: http://www.eecs.berkeley.edu/~sa499/tor_timeline.pdf. It was
composed by mining the Tor issue tracker and blog for events.

For general censorship, I'm not aware of any systematic or complete list
of censorship events. There are quite a lot of research papers that
study one particular place for a period of time, and from them we know
of some incidents.

Here's a list of some measurement papers I'm aware of that might discuss
some events. You might also look at OpenNet and Freedom House reports.


Internet censorship in Iran: A first look
https://www.usenix.org/conference/foci13/workshop-program/presentation/aryan

The Anatomy of Web Censorship in Pakistan
https://www.usenix.org/conference/foci13/workshop-program/presentation/Nabi

Dimming the Internet: Detecting throttling as a mechanism of censorship in Iran
http://arxiv.org/abs/1306.4361

Global Network Interference Detection Over the RIPE Atlas Network
https://www.usenix.org/system/files/conference/foci14/foci14-anderson.pdf

Analysis of Country-wide Internet Outages Caused by Censorship
http://conferences.sigcomm.org/imc/2011/docs/p1.pdf

Characterizing Web Censorship Worldwide: Another Look at the OpenNet Initiative 
Data
http://censorbib.nymity.ch/pdf/Gill2015a.pdf
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] How many exits exit from an IP address different than their OR address? (10.7%)

[David Fifield]

On Sun, Jan 17, 2016 at 10:24:47PM +, cacahuatl wrote:
> On Sun, Jan 17, 2016 at 01:01:03PM +0100, coderman wrote:
> > misguided because it won't work as you expect, the right way to check
> > is to build circuits and see where they exit from. you can do this
> > yourself!
>
> Tor Project already does it for you with archived data and everything:
> https://collector.torproject.org/#exit-lists
> "The exit list service TorDNSEL publishes exit lists containing the IP
> addresses of relays that it found when exiting through them."

Oh, wow, thanks. I did not know that TorDNSEL was already providing this
service. Combined with 
https://collector.torproject.org/#type-network-status-consensus-3
it should be possible to do what I did with my exitmap scan for any time
in the past.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Does Orbot use default obfs4 bridges?

[David Fifield]

Does Orbot have a list of default built-in obfs4 bridges? Or do users
fetch them dynamically? I looked in the source code and found default
meek bridges but not default obfs4.

I'm asking because we recently added a few new high-capacity default
obfs4 bridges.
https://bugs.torproject.org/18071 (1 bridge)
https://bugs.torproject.org/18091 (2 bridges)
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Does Orbot use default obfs4 bridges?

[David Fifield]

On Tue, Jan 19, 2016 at 03:29:38PM -0500, Nathan Freitas wrote:
> 
> On Tue, Jan 19, 2016, at 02:52 PM, David Fifield wrote:
> > Does Orbot have a list of default built-in obfs4 bridges? Or do users
> > fetch them dynamically? I looked in the source code and found default
> > meek bridges but not default obfs4.
> 
> No we have no default Obfs4 bridges built-in. It has been on my list to
> get done, because we really should, obviously to make the user
> experience better, and to alleviate meek traffic.
> 
> > I'm asking because we recently added a few new high-capacity default
> > obfs4 bridges.
> > https://bugs.torproject.org/18071 (1 bridge)
> > https://bugs.torproject.org/18091 (2 bridges)
> 
> I will get this built-in ASAP.

The list of Tor Browser built-in bridges is here:
https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/Bundle-Data/PTConfigs/bridge_prefs.js
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Introducing Snowflake (webrtc pt)

[David Fifield]

On Mon, Jan 25, 2016 at 02:34:42PM -0800, Serene wrote:
> Snowflake is a webrtc pluggable transport inspired by flashproxy.
> (https://gitweb.torproject.org/pluggable-transports/snowflake.git)
> Arlo, David, and I have made lots of progress on it lately, and it now
> appears to have reached minimum viability.

I started a wiki page.
https://trac.torproject.org/projects/tor/wiki/doc/Snowflake
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Configuration of tor relay using setup files (use of API via Tor Expert Bundle)

[David Fifield]

On Sun, Feb 07, 2016 at 03:44:35PM +, Nathan Bliss wrote:
> Is there a way to configure a bridge in tor (e.g. meek) via the config files
> from the command line without having to use the GUI in the Tor browser? I've
> been searching for documentation on this, so if I've missed it I would be
> grateful for a pointer to where this is in the docs... I found something about
> the torrc but this was an option for configuring a tor server node as a bridge
> whereas I want to configure the bridge as a client (in client mode).

Here is an example torrc for configuring a client:

https://gitweb.torproject.org/pluggable-transports/meek.git/tree/meek-client/torrc
Just comment out the "Bridge" lines other than the one you need. Then to
run it, you can do
tor -f torrc
Depending on your setup, you might have to provide the full paths to tor
or torrc. For example,
C:\path\whatever\tor.exe -f C:\path\whatever\torrc

If you need the web browser TLS camouflage (see
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/README),
then it'll be easiest to run tor manually from inside the Tor Browser
directory tree with its torrc-defaults file. That will look something
like this:
Browser\TorBrowser\Tor\tor.exe --defaults-torrc 
Browser\TorBrowser\Data\Tor\torrc-defaults -f your-own-torrc
where "your-own-torrc" is a file containing the bridge you are using
(like in the previous paragraph).

> I need to do this in order to have automation scripts working with search
> engines such as google. Google always seems to be able to detect http
> connection via a script or via the tor interface running in standard mode. 
> Only
> a client connection via google-meek bridge seems to work and I'm looking for a
> way to do this via the tor stem API or via the config files.

This doesn't make sense to me. Are you saying that you're making client
connections through the Tor network to Google, and Google behaves
differently depending on whether you connect using a pluggable transport
or not? That seems unlikely. Web sites have no way of knowing what
pluggable transport you're using. It's more likely that you just got
lucky with your exit nodes while you were using meek-google. I suggest
you run some more experiments to confirm your guess.

In any event, if you are using meek for purposes other than
circumventing censorship, please take the time to set up your own App
Engine instance (it's easy) and pay for it yourself, otherwise you are
taking away bandwidth and capacity from actual censored users. To set up
your own instance, you will have to follow these instructions. Download
the "appengine" files from the meek source code, edit the file
"app.yaml" to use your App Engine app name, and then upload it with
"goapp deploy".
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/appengine
https://gitweb.torproject.org/pluggable-transports/meek.git/tree/appengine/README
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Summary of meek's costs moved to tor-project list

[David Fifield]

I decided to move the meek cost emails to the tor-project list, because
they are more project-y than dev-y. Here is the email for January 2016:

https://lists.torproject.org/pipermail/tor-project/2016-February/000101.html

There's a table of all previous summaries here:

https://trac.torproject.org/projects/tor/wiki/doc/meek#Costs
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Request for feedback/victims: cfc

[David Fifield]

During the OONI survey to find instances of server-side Tor blocking, we
found a few variations on CloudFlare captcha pages. They don't all say
"Attention Required!". Apparently there is an option to customize the
page, but few sites make use of it. Here are the regexes we used
(excerpted from https://www.bamsoftware.com/git/ooni-tor-blocks.git):
if status == 403:
if server == "cloudflare-nginx" and re.search("Attention 
Required! \\| CloudFlare|One more step to access", body):
return True, "403-CLOUDFLARE"
if server == "cloudflare-nginx" and re.search("|", body):
# A customized captcha page.
return True, "403-CLOUDFLARE"
if server == "cloudflare-nginx" and re.search("Access denied \\| 
[^ ]* used CloudFlare to restrict access", body):
# With this one you don't get a captcha. May be controlled by the
# site operator.
return True, "403-CLOUDFLARE"
if status == 503:
if re.search("", body):
return True, "503-CLOUDFLARE"
I now think the 'server == "cloudflare-nginx"' tests are unnecessary.
The last two patterns above don't even give you a captcha to solve, just
deny access. You might want to limit your detection to 403 and 503
responses (or maybe exempt 200-series and 300-series responses).

These are a couple of sites that used customized CloudFlare:
https://4chan.org/ ("Verification Required")
https://yelp.com/ ("You're not barking up the wrong tree...")
yelp.com only started using CloudFlare a little while ago. It's a funny
case, because they *also* implement a hard Tor blacklist. Once you get
through the CloudFlare captcha 403, you get a 503 from a different
system.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Using Let's Encrypt for meek bridges

[David Fifield]

I'm looking for ideas of good ways to handle TLS certificates and their
renewal for meek bridges. I want to use Let's Encrypt for this process,
and I hope that someone who knows Let's Encrypt well can contribute some
ideas.

All three of the meek bridges use HTTPS to receive connections from the
CDN, so they need TLS certificates. For example, when you use
meek-azure, your traffic is forwarded to the bridge at
https://meek.bamsoftware.com/. How it works now is I do the usual domain
validation procedure with a CA, receive an email to show that I control
the domain, install the cert and key, and then run the server like this:
meek-server --cert /etc/meek/cert.pem --key /etc/meek/key.pem

When I used Let's Encrypt in the past, using the --webroot option, it
wanted to write a file to the URL path "/.well-known/acme-challenge/{token}".
That won't work for meek-server as it exists today, because meek-server
never serves files from the filesystem. But it could. Perhaps we could
add an option like --acme-webroot that would allow serving files from a
single whitelisted directory.

I notice that there are other ways of proving domain ownership (HTTP,
TLS SNI, DNS). Maybe we could use one of those?
https://ietf-wg-acme.github.io/acme/#identifier-validation-challenges

I also note that there are third-party plugins:
https://github.com/letsencrypt/letsencrypt/wiki/Plugins
Maybe there could be a plugin for meek-server (ideally not requiring new
complicated code in meek-server itself).

Currently you have to restart meek-server in order to make it notice a
changed certificate and key file. It would be better if that were not
necessary--maybe we could periodically stat the files, and re-load them
if they have changed?

This is going to be an issue for Snowflake as well, because we will want
to use WebSocket over TLS for the server component.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] iObfs: obfs4proxy on iOS

[David Fifield]

On Mon, Apr 04, 2016 at 12:04:45AM -0400, Mike Tigas wrote:
> [again, cross-posted to tor-dev and guardian-dev.]
> 
> A quick status report on this: it works! Hit a big epiphany, figured out
> how to get `gomobile` to emit the necessary bits, then went wild.
> 
> Some example stdout from Onion Browser connecting to Tor via obfs4,
> meek_lite (google), and scramblesuit:
> https://gist.github.com/mtigas/f1b9a3a8befa6f60d517eb2340f3cdd4
> 
> There are trivial forks of obfs4[1] and goptlib[2] that simply hard-code
> some options that are normally sent as environment variables because
> obfs4proxy runs in managed mode[3]. (It's the best I have right now
> until I can figure out a better way to communicate between obfs4proxy
> and the iOS bits.) I’ve tacked a few other quick thoughts at the bottom
> of the iObfs readme[4]. As a quick test I've started building it into
> Onion Browser (iobfs branch[5]), which is what got the output linked above.
> 
> [1]:
> https://github.com/mtigas/obfs4/compare/1df5c8ffe8f4aa2614323698e8008f1ab1fb7a18...mtigas:iObfs-201604-dev
> [2]:
> https://github.com/mtigas/goptlib/compare/f17a5f239f705d7e39a8bccbebdf9927cc99dbeb...mtigas:iObfs-201604-dev

This is radical. Maybe you don't need the fork of goptlib if you do
os.Setenv on the relevant variables before calling pt.ClientSetup in
obfs4?
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Using Let's Encrypt for meek bridges

[David Fifield]

On Fri, Apr 08, 2016 at 05:28:45PM -0700, George Tankersley wrote:
> > I'm looking for ideas of good ways to handle TLS certificates and their
> > renewal for meek bridges. I want to use Let's Encrypt for this process,
> > and I hope that someone who knows Let's Encrypt well can contribute some
> > ideas.
> 
> > ideally not requiring new complicated code in meek-server itself
> 
> If you're OK with some amount of new code, there are Go client libraries that
> might be sufficiently flexible:
> 
> - https://github.com/xenolf/lego
> - https://ericchiang.github.io/go/tls/lets/encrypt/letsencrypt/2015/11/13/
> a-letsencrypt-client-for-go.html
> 
> I'll give this a try soon if you haven't already.

Thanks. You are welcome to try. I haven't done anything yet.

My ideal situation would be that I can run an acme client *outside* of
meek-server, the same way I can do for Apache. Something like this:
letsencrypt-auto certonly [some other arguments, maybe --webroot] 
meek.bamsoftware.com

Adding acme code to meek-server seems less desirable to me. I am trying
to keep the feature count low to keep the bug count low. meek-server is
under 500 lines and I wouldn't want to add more than, say, 200 lines for
acme support. It might have to write new certificate files to the
filesystem, which means worrying about privileges. It's possible,
though, that I don't properly understand how an acme library should work
and these concerns aren't appropriate.

My current inclination is to add an --acme-webroot option to
meek-server, and install a restricted http.Handler that allows serving
files from that directory. It will only serve files whose names are in
the specific format that acme requires. Maybe it can only serve files
whose creation time is recent, as an additional safeguard against
accidentally serving non-acme files.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] meek-server performance improvements?

[David Fifield]

I saw you say on IRC that you had an idea for improving the efficiency
of meek-server. What's your idea? The server hosting meek-azure is
passing 90% CPU at times.

One idea I've seen is using one connection for upstream data
(data-carrying POSTs, emptry responses), and one connection for
downstream data (GET, one big chunked download). As I understand it,
this is what Lantern's enproxy does:
https://github.com/getlantern/enproxy
So far I've avoided this idea because of the possibly weird traffic
patterns.

I've also done some preliminary work on allowing the server to send data
in larger chunks, which should reduce polling:
https://trac.torproject.org/projects/tor/ticket/12857#comment:4
All that's really needed here is a decision on how to let the client
indicate that it supports such a server, as the change is backward
incompatible.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] meek-server performance improvements?

[David Fifield]

On Mon, Apr 25, 2016 at 06:30:03AM +, Yawning Angel wrote:
> On Sun, 24 Apr 2016 22:02:23 -0700
> David Fifield  wrote:
> 
> > I saw you say on IRC that you had an idea for improving the efficiency
> > of meek-server. What's your idea? The server hosting meek-azure is
> > passing 90% CPU at times.
> 
> Hmm, you're building meek-server with Go 1.6.x right?  CloudFlare of
> all companies contributed some of the optimizations I wanted to do
> courtesy of V. Krasnov, formerly of Intel...[0]

We're using 1.6. I suspect (but I am not sure) that most of the CPU is
used by TLS handshakes. I thought of trying to get an ECDSA cert to see
if it goes faster than the current RSA cert.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] [Proposal] Obfuscating the Tor Browser Bundle initial download

[David Fifield]

On Mon, May 09, 2016 at 09:23:20PM +, William Waites wrote:
> 
> Blake Hadley  writes:
> >
> > The environment requires an HTTPS proxy to reach the World Web Web.
> >
> > Do HTTP proxies inherently create a situation similar to MITM?
> 
> Yes, that is exactly what they do. If your web browser isn't nagging you
> all the time with "hey this certificate is untrusted" then a signing
> certificate will have had to be installed in your computer. This lets
> your employer decrypt all of your HTTPS traffic, inspect it, and
> reencrypt it. Unless you understand exactly what is happening and what
> to do about it, best to avoid using the Internet from there altogether.

Well, an "HTTPS proxy" doesn't have to be a TLS interception device.
When I think "HTTPS proxy", I think of an ordinary HTTP proxy using the
CONNECT method to support TLS. In that case, the proxy does not get to
see plaintext, which includes the path part of the URL. The proxy can
tell what domain name you are connecting to, but not what document you
are requesting.

In either case, there's no benefit to obfuscating the download names.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Request for feedback/victims: cfc-0.0.2

[David Fifield]

On Fri, Apr 01, 2016 at 06:06:18PM +, Yawning Angel wrote:
> I'll probably add support for other (user-configurable?) cached content
> providers when I have time.  The archive.is person doesn't seem to want
> to respond to e-mail, so asking them to optionally not set X-F-F, seems
> like it'll go absolutely nowhere.

This is some kind of meta-archive service. Their about page lists many
web archives (some of the specialized):
http://timetravel.mementoweb.org/about/
http://www.mementoweb.org/guide/quick-intro/
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] GreatFire Circumvention Central: tests of speed and stability of circumvention tools in China

[David Fifield]

https://en.greatfire.org/blog/2016/jul/greatfireorg-now-testing-vpn-speed-and-stability-china
https://cc.greatfire.org/en
"Our newest website, Circumvention Central (CC), aims to provide
real-time information and data about circumvention solutions
that work in China. Since 2011, we have been collecting data
about blocked websites in China and now we will add data about
the effectiveness of VPNs and other circumvention tools."

They list various VPNs and other tools. Tor is on the list. (This page
has JavaScript charts that aren't working for me in Tor Browser.)
https://cc.greatfire.org/en/provider/Tor

At the moment, these are the figures I see:
Speed (kbps)Stability
1. HideMyAss504 (+39%)  98% (-1%)
2. VyprVPN  445 (−33%)  98% (+11%)
3. FreeBrowser  440 (+13%)  88% (+1%)
4. XX-Net   356 (New)   97%
5. Astrill  294 (−32%)  97% (+1%)
6. Lantern  262 (+15%)  95% (+3%)
7. VPNSecure259 (New)   87%
8. Psiphon  218 (+6%)   89% (+2%)
9. Ultrasurf206 (+460%) 86% (−4%)
10. Freegate188 (−29%)  91% (+1%)
11. PureVPN 151 (−25%)  94% (+1%)
12. Tor 112 (+237%) 96% (+1%)

The test details have interesting values under the "Settings" header:
https://cc.greatfire.org/en/provider/Tor#http-tests
"0", "socks5", "是", "lantern-proxy", "lantern-proxy,no bridge",
"lantern-proxy,meek-arzue", "proxy,no bridge", "lantern-proxy,obfs4",
"obfs4", "否", "meek-arzue", "obfs4,sock5 proxy".

There's a tool on the list I didn't know, XX-Net. Its GitHub page says
"It uses google app engine (GAE) as a proxy server through the
firewall" and "A Reborn GoAgent."
https://github.com/XX-net/XX-Net
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Some information about Tor relays

[David Fifield]

On Fri, Aug 26, 2016 at 01:42:38AM +, Liu, Zhuotao wrote:
> This is Sky from University of Illinois. Currently we are working on research
> project related with Tor. 
> 
> To help us to better design and evaluation our proposal, we need some
> information about the Tor relays that is currently unavailable from the Atlas.
> Thus, if someone who operates Tor relays could provide us such information,
> your help will be greatly appreciated. :) 
> 
> We hope to have an estimate about computation capacity of Tor relays. For
> instance, how many circuits a relay can maintain when its CPU is driven to
> about 100%? On average, how many circuits are maintained by a busy guard and
> what the CPU utilization is. These kinds of information would be really
> helpful. 

I don't know about CPU usage, but as for circuits, I think you can get
them from @extra-info and @bridge-extra-info descriptors:
https://collector.torproject.org/#type-extra-info
https://collector.torproject.org/#type-bridge-extra-info
For example, see some of the files at
https://collector.torproject.org/recent/relay-descriptors/extra-infos/

The "cell-circuits-per-decile" lines might be interesting to you.
https://spec.torproject.org/dir-spec:
"cell-circuits-per-decile" num NL
Mean number of circuits that are included in any of the deciles,
rounded up to the next integer.

For parsing the descriptor files, you can use Stem:

https://stem.torproject.org/_modules/stem/descriptor/extrainfo_descriptor.html
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Some information about Tor relays

[David Fifield]

On Fri, Aug 26, 2016 at 04:46:45AM +, Liu, Zhuotao wrote:
> Thanks for that info, David. That seems valuable to me. :) 
> 
> However, I am a bit confused about the definition 
> 
> "cell-circuits-per-decile": Mean number of circuits that are included in any 
> of the deciles, 
> rounded up to the next integer. 
> 
> What is the exact meaning of 'decile'? Is it one tenth of a hour? Or 
> something else? 

I don't know. My reading of dir-spec says it is probably the 0%–10%,
10%–20%, 20%–30%, etc. circuits counting by number of cells.

https://spec.torproject.org/dir-spec

"cell-processed-cells" num,...,num NL
Mean number of processed cells per circuit, subdivided into
deciles of circuits by the number of cells they have processed in
descending order from loudest to quietest circuits.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] HTTPS Everywhere

[David Fifield]

On Mon, Sep 05, 2016 at 10:28:26PM +0530, AKASH DAS wrote:
> Can I know the issues that are currently in https everywhere.

I don't know if this is what you're looking for, but here are some open
bug tracker tickets.

https://trac.torproject.org/projects/tor/query?status=!closed&component=HTTPS+Everywhere%2FEFF-HTTPS+Everywhere
https://trac.torproject.org/projects/tor/query?status=!closed&component=HTTPS+Everywhere%2FHTTPS+Everywhere%3A+Chrome
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Pluggable transport idea: TLS session resumption

[David Fifield]

Here's an idea for a new pluggable transport. It's just a TLS tunnel,
but with a twist that allows the server's certificate to be omitted,
depriving the censor of many classification features, such as whether
the certificate is signed by a CA, the certificate's lifetime, and
whether the commonName matches the server's IP address. I got the idea
from ShadowsocksR:
https://github.com/breakwa11/shadowsocks-rss/blob/master/ssr.md#2%E6%B7%B7%E6%B7%86%E6%8F%92%E4%BB%B6

The trick that makes it work is RFC 5077 session tickets. How it's
supposed to work is, the client makes a TLS connection as usual, and the
server sends back a session ticket before beginning the flow of
application data. The session ticket is just a random blob from the
client's point of view; from the server's point of view it's an
authenticated ciphertext that encodes session parameters such as the
ciphersuite and master secret. The next time the client connects to the
same server, it sends the session ticket along with its ClientHello and
the server can skip sending its certificate.

Here's a simple way to adapt session tickets for circumvention. The
client *always* sends a session ticket when making a connection, even
the very first time. The ticket doesn't come from a previous
communication with the server; the client generates the ticket itself.
The ticket could for instance be a random string--the censor wouldn't be
able to distinguish--but we can also use those bits for other purposes.
When the server receives the ClientHello, it *pretends* that the ticket
was a valid one, and finishes the handshake using hardcoded session
parameters, just statically configuring what it would have encoded into
a session ticket.

The protocol as just described would be vulnerable to active probing;
the censor could test for servers by sending them garbage session
tickets and seeing how they respond. But that's easy to fix. We can, for
example, let the client and server have a shared secret, and have the
server treat the session ticket as the client part of an obfs4
handshake--which conveniently resembles a random blob. If the session
ticket doesn't pass obfs4 authentication, then the TLS server can
respond as it naturally would if a client sent an invalid session
ticket; i.e., issue a new ticket and do a full handshake (then send
dummy data, I guess). The server can also honor its own legitimately
issued tickets, but still send dummy data in response. Only clients who
know the shared secret will be able to access the proxy functionality of
the server.

In order to block such a transport, the censor will have to look at
features other than the server certificate. It could, for example:
 * block all session tickets (dunno how costly that would be)
 * statefully track which tickets servers have issued, and block
   connections that use an unknown ticket.
 * track the fraction of connections that use session tickets on each
   TLS server.
 * active-probe the server in order to get a certificate, and then look
   at certificate features.
We'd have to do some research to find out the distribution of session
ticket sizes in the wild. (RFC 5077 recommends a specific format:
https://tools.ietf.org/html/rfc5077#section-4.)

I didn't think of this idea; it comes from ShadowsocksR and its
tls1.2_ticket_auth plugin.
https://github.com/breakwa11/shadowsocks-rss/blob/master/ssr.md#2%E6%B7%B7%E6%B7%86%E6%8F%92%E4%BB%B6
The documentation is in Chinese. Here is what Google Translate says in
English:
Analog TLS1.2 handshake at client has a session ticket situation
is connected. Currently complete analog implementation, software
testing through capture perfectly disguised as TLS1.2. Because
there is no ticket so send certificates and other complicated
steps, so the firewall can not make a judgment based on the
certificate. At the same time it comes with the ability of
certain anti-replay attacks. In case of a replay attack is
searched end log in to the service, you can grep "replay attack"
search, you can use this plug-in found in your area there is no
line of interference for TLS. TLS relatively powerless firewall,
anti-blocking ability than other plug-ins should be strong, but
may also encounter a lot of interference, but the agreement
itself will check out any interference, then disconnect
encounter interference, to avoid long waits, so that customers
end browser or reconnect themselves. This plug-in is compatible
with the original agreement (the server is configured to require
tls1.2_ticket_auth_compatible), one more than the original
protocol handshake causes the connection will take longer, use C
# client open automatically reconnect when performed better than
the other plug-ins. Support for custom parameters, parameters
SNI, it sends the host name field, this function is very similar
to the TOR 

Re: [tor-dev] Tor Browser downloads and updates graphs

[David Fifield]

On Sun, Sep 11, 2016 at 04:13:00PM +, Georg Koppen wrote:
> Here are the graphs showing initial downloads, update pings and update
> requests over time:
> 
> https://people.torproject.org/~karsten/volatile/torbrowser-annotated-2016-09-11.pdf
> 
> The update pings are made by Tor Browser instances roughly twice a day
> and they indicate the number of active Tor Browser users. More
> importantly, one can see the decrease or increase of Tor Browser usage
> over time.
> 
> * We don't know what (8) or (9) is but it seems to us we are losing
> users over time and are only getting them back slowly if at all. A
> weekday/weekend pattern is visible there as well.

Does Tor Browser continue checking for further updates in the span of
time between when it downloads an update and when it is restarted? For
example, you are running 6.0, the browser downloads the 6.0.1 update and
stages it and asks you to restart; does the browser check for updates
until you actually restart? If not, then the decreases in update pings
might be people being tardy in restarting their browser.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Tor Browser downloads and updates graphs

[David Fifield]

On Mon, Sep 12, 2016 at 11:12:15AM -0400, Mark Smith wrote:
> On 9/11/16 3:45 PM, David Fifield wrote:
> >> * We don't know what (8) or (9) is but it seems to us we are losing
> >> users over time and are only getting them back slowly if at all. A
> >> weekday/weekend pattern is visible there as well.
> > 
> > Does Tor Browser continue checking for further updates in the span of
> > time between when it downloads an update and when it is restarted? For
> > example, you are running 6.0, the browser downloads the 6.0.1 update and
> > stages it and asks you to restart; does the browser check for updates
> > until you actually restart? If not, then the decreases in update pings
> > might be people being tardy in restarting their browser.
> 
> That is a good theory, but I don't think update checks occur if there is
> a pending update. The code that checks and returns early is here:
> 
> https://gitweb.torproject.org/tor-browser.git/tree/toolkit/mozapps/update/nsUpdateService.js?h=tor-browser-45.4.0esr-6.0-1#n2388

Oh, thanks for finding that source code link. I looked for that code and
didn't find it.

But that's exactly what I'm saying: once someone has downloaded an
update, they stop sending update pings until their next restart, which
might explain the decreases in update pings at (8) and (9) in the graphs.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] uProxy adds Tor support

[David Fifield]

https://blog.uproxy.org/2016/09/uproxy-adds-tor-support.html

This blog post says that uProxy gained support for proxying others'
traffic through Tor.

uProxy client → censor → uProxy server → Tor → destination

In the classic uProxy deployment scenario, the client and server are
people who know each other. (I say "classic" because there are other
deployment modalities now.) They each run a browser extension. The
client encapsulates its web traffic and sends it to the server over
WebRTC. The server then issues the web requests and sends the responses
back over the WebRTC channel.

A potential problem is that the uProxy server's own web browsing gets
mixed up with that of the client. The client could, for example, get the
server in trouble by accessing sketchy web sites. That's what the Tor
integration is meant to solve.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal 274: A Name System API for Tor Onion Services

[David Fifield]

On Fri, Oct 07, 2016 at 04:06:51PM -0400, George Kadianakis wrote:
>In particular, onion addresses are currently composed of 16 random base32
>characters, and they look like this:
> 
>   3g2upl4pq6kufc4m.onion
>   vwakviie2ienjx7t.onion
>   idnxcnkne4qt76tg.onion
>   vwakviie2ienjx6t.onion

Is there any meaning intended by the near-collision in the 2nd and 4th
addresses?

>If all plugins fail to successfuly perform the name resolution, Tor SHOULD
>default to using the exit node for name resolution.
>XXX or not?  because of leaks?

It would be really nice if it were possible for the browser to show some
kind of UI indication that a name I type into the URL bar is going to be
handled specially. Even now, the URL bar is magical: if you typo a DNS
name, the browser does a web search and leaks whatever you just typed. I
think in the protocol as proposed, there's no way for the browser to
predict whether a name will be looked up in a special way using a
plugin, or even to learn what the mapping was after it has occurred.

How do you think this will interact with attacker-controlled names in
HTML? Let's say someone has a local petname database where
"bank.petname" points to the onion site of their bank. What happens when
HTML says
http://bank.petname/favicon.ico";>
Will it show the icon of the user's bank, whatever bank it happens to
be? I guess my question is if attackers are allowed to query the user's
local naming setup, even in an indirect way, or does the magic only
happen for names that are typed directly into the URL bar? (Because the
mapping happens below the SOCKS layer, I guess it could not apply only
to the URL bar.)

>Hence, the contribution of this proposal is a modular Name System API
>(NSA)

This acronym might cause confusion :)

>|
> $~~$
>|  1.$  4. GNS magic!! 
>  $
>|  User: SOCKS CONNECT to$ debian.zkey -> 
> sejnfjrq6szgca7v.onion$
>|http://debian.zkey/ 
> $~~$~~~$
>|   $
>  +-|-+ $
>  |+v-+ 2. +-+|   3.$
>  ||Tor   | debian.zkey|Tor  ||   debian.zkey 
> +-$---+
>  ||Networking->Naming   ->
>  |
>  ||Submodule ||Submodule||  Tor Name System API  |  
> GNS|
>  ||  <- <-  
> wrapper|
>  ||  | 6. | ||5. |
>  |
>  |+|-+ sejnfjrq6szgca7v.onion +-+|sejnfjrq6szgca7v.onion 
> +-+
>  +-|-+
>|  7.
>|  Tor: Connect to
>|   http://sejnfjrq6szgca7v.onion/
>v

So here, the browser thinks it is connecting to debian.zkey (the URL bar
says "debian.zkey"). But Tor is really connecting to sejnfjrq6szgca7v.onion
in the background. What name does the browser put in its Host header? It
can't be the onion name, because there's no feedback from the naming
module back to the user application layer. It must be "debian.zkey"
then. If that is a petname, then it just got leaked to the server. I can
imagine this might also cause a problem with some virtual hosting setups
(though I suppose those are not very common for onion services). If the
user uses HTTPS, e.g. https://facebook.zkey/, then they'll get a TLS
name mismatch error, even if https://facebookcorewwwi.onion/ exists and
has a valid certificate--so using the naming system is not a transparent
replacement for memorizing the onion address. Maybe non-HTTP protocols
will also have problems.

It would be nice if there were some standard external way to just do a
name→onion query. As specified, there's no easy way for e.g. the browser
to query how a name will be mapped, short of launching its own instance
of the GNS wrapper and communicating with the over the Name System API.
I wonder, perhaps there's a way to extend the Tor RESOLVE cell to
support this feature? So you could do, for example
$ tor-resolve debian.zkey
sejnfjrq6szgca7v.onion

>When launching name plugins, Tor sets various environment variables to pass
>data to the name plugin (e.g. NS API version, state directory, etc.). More
>information on the environment variables at [INITENVVARS].
> 
>After a name plugin initializes and parses all needed environment
>variables, it communicates with Tor using its stdin/stdout.

I worry that repeating the envvar and stdin/stdout design of the PT spec
will cause problems for p

Re: [tor-dev] Tor Relays on Whonix Gateway

[David Fifield]

On Wed, Oct 19, 2016 at 10:35:16PM +0200, ban...@openmailbox.org wrote:
> On 2016-10-17 10:24, isis agora lovecruft wrote:
> > 
> > You're planning to enable "ServerTransportPlugin snowflake" on Whonix
> > Gateways
> > by default?  And then "ClientTransportPluging snowflake" on workstations
> > behind the gateway?
> 
> I was planning to enable the server by default (I thought WebRTC was P2P
> though) but after looking at it some more I don't think it's a good idea.

It doesn't make sense to run the Snowflake server on a lot of bridges
anyway. It's not like the obfs* model where you need lots of bridges in
order to get IP diversity. Snowflake gets IP diversity by routing
through web browsers. The bridge itself may even be blocked by the
censor; it doesn't matter.

The server component of Snowflake isn't even WebRTC. Snowflake is WebRTC
between the client and the browser proxy, then WebSocket (which is
easier to program) between the browser proxy and the bridge. The server
component is actually just a WebSocket server, borrowed from flash
proxy.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Call for help on testing core tor releases

[David Fifield]

On Thu, Oct 13, 2016 at 02:29:19PM -0400, isab...@riseup.net wrote:
> Hello Tor community!
> 
> The Core Tor Team would like to improve our release process by getting
> it more tested so bugs are found earlier, so stable releases can get out
> faster and without any big bugs.
> 
> During Tor's Meeting in Seattle a couple of weeks ago, we discussed this
> topic and decided to organize a QA team (with the help of the community)
> for core tor.

There are some people on the tor-qa who test Tor Browser before release;
maybe some of them will help? Or maybe you want to do core tor QA on
this list?
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-qa
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Using fingerprint of cached relay bypasses bridge?

[David Fifield]

Someone on #tor-project IRC reported that you can bypass your pluggable
transport if you use the fingerprint of an ordinary relay already known
to Tor in your bridge line. I would file a ticket but I haven't been
able to reproduce it.

The example the IRC user gave was this, meant to be pasted into Tor
Browser as a custom bridge:
Bridge meek 0.0.2.0:2 3C3A6134E4B5B7D1C18AD4E86EE23FAC63866554 
url=https://d2zfqthxsdq309.cloudfront.net/ front=a0.awsstatic.com
The fingerprint is the wrong one for this bridge. It should be
B9E7141C594AF25699E0079C1F0146F409495296 for the bridge nicknamed
TorLandMeek. It is instead for the relay nicknamed traffic70 at
188.138.1.166:9001.

The claim is that if tor has already cached a descriptor with
fingerprint 3C3A6134E4B5B7D1C18AD4E86EE23FAC63866554, then it will make
a direct connection for the purpose of making a one-hop circuit. "it's
about one hop tunnel when exit is entry" says the IRC user. They point
to these parts of the source code:

https://gitweb.torproject.org/tor.git/tree/src/or/circuituse.c?id=tor-0.2.8.9#n2010
r = node_get_by_nickname(conn->chosen_exit_name, 1);
https://gitweb.torproject.org/tor.git/tree/src/or/circuituse.c?id=tor-0.2.8.9#n2015
extend_info = extend_info_from_node(r, conn->want_onehop ? 1 : 0);

I wasn't able to reproduce it. I used this torrc file:

DataDirectory datadir
UseBridges 1
Bridge meek 0.0.2.0:2 3C3A6134E4B5B7D1C18AD4E86EE23FAC63866554 
url=https://d2zfqthxsdq309.cloudfront.net/ front=a0.awsstatic.com
ClientTransportPlugin meek exec ./meek-client --log meek-client.log

First I tried with a cold cache, and got a fingerprint mismatch:

Nov 01 18:00:53.000 [notice] Bootstrapped 10%: Finishing handshake with 
directory server
Nov 01 18:00:54.000 [warn] Tried connecting to router at 0.0.2.0:2, but 
identity key was not as expected: wanted 
3C3A6134E4B5B7D1C18AD4E86EE23FAC63866554 but got 
B9E7141C594AF25699E0079C1F0146F409495296.
Nov 01 18:00:54.000 [warn] Problem bootstrapping. Stuck at 10%: Finishing 
handshake with directory server. (Unexpected identity in router certificate; 
IDENTITY; count 1; recommendation warn; host 
3C3A6134E4B5B7D1C18AD4E86EE23FAC63866554 at 0.0.2.0:2)

Then I commented out "UseBridges 1", let the bootstrap finish, and
uncommented "Use Bridges 1" again. I got the same output:

Nov 01 18:05:06.000 [notice] Bootstrapped 10%: Finishing handshake with 
directory server
Nov 01 18:05:09.000 [warn] Tried connecting to router at 0.0.2.0:2, but 
identity key was not as expected: wanted 
3C3A6134E4B5B7D1C18AD4E86EE23FAC63866554 but got 
B9E7141C594AF25699E0079C1F0146F409495296.
Nov 01 18:05:09.000 [warn] Problem bootstrapping. Stuck at 10%: Finishing 
handshake with directory server. (Unexpected identity in router certificate; 
IDENTITY; count 1; recommendation warn; host 
3C3A6134E4B5B7D1C18AD4E86EE23FAC63866554 at 0.0.2.0:2)

I used tcpdump to check for connections to 188.138.1.166, and didn't see
any.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] automatically detect many new identical/similar bridges

[David Fifield]

On Wed, Dec 14, 2016 at 10:09:00AM +, nusenu wrote:
> in the context of [1] I'm wondering if it makes sense to add bridge
> support to ornetradar.
> 
> If there is any value to automatically detect multiple new bridges:
> 
> - Do bridges publish ContactInfo in their descriptor? If not: Why not?
> (it shouldn't disclose their bridge location)

From my understanding, bridge descriptors may include ContactInfo, but
Collector removes it before publishing. So you can distinguish
descriptors that originally had ContactInfo set from those that did not,
but you can't tell what the ContactInfo was.

https://collector.torproject.org/#bridge-descriptors
> 5. Replace contact information: If there is contact information in a
> descriptor, the contact line is changed to somebody.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Flashproxy has been Deactivated by Stanford? Why?

[David Fifield]

On Tue, Dec 20, 2016 at 01:21:04AM +0800, to...@riseup.net wrote:
> It turned out that the entire code has been commented out and apparently
> Flashproxy became
> out of service. Why? Has the project discontinued, or just down for
> maintenance?

Flash proxy is basically retired now. It was removed from Tor Browser a
year ago (https://bugs.torproject.org/17428) after it had been
supplanted by more effective transports. I don't know why there was a
blog post on December 16 promoting flash proxy, because it's no longer
used.

Even when flash proxy was part of Tor Browser, it had very few users
(less than 100; see 
https://metrics.torproject.org/userstats-bridge-transport.html?start=2013-01-01&transport=websocket),
probably because of the difficulty of running it as a client
(https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports/FlashProxy/Howto).
Compare those user numbers to meek (current about 10K) or obfs4 (30K).

The reason I haven't asked people to stop running the flash proxy badge
is we're working on a new pluggable transport along the same lines but
without the usability challenges:
https://trac.torproject.org/projects/tor/wiki/doc/Snowflake. I was
thinking about adapting existing flash proxy badges to provide capacity
to Snowflake instead. This would go for Cupcake as well. The need to get
the badge running again hasn't been pressing because Snowflake isn't
deployed yet, but we're getting close.

The badge was deactivated by Stanford (without my knowledge, but I found
out a while ago). I arranged with them to move it to alternate hosting
and have them install a redirect, but that has been a low priority
behind other work on Snowflake.

I'm sorry about the confusion. If I get some time I'll add a notice to
the flash proxy main page saying that it's been retired.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] GAEuploader

[David Fifield]

On Sun, Jan 22, 2017 at 03:53:16PM -0800, Katherine Li wrote:
> I would really appreciate user testing on GAEuploader. You can download it 
> at: 
> https://github.com/katherinelitor/GAEuploader/releases
> README: https://github.com/katherinelitor/GAEuploader
> Tor wiki page, containing step-by-step screenshots of how to use:
> https://trac.torproject.org/projects/tor/wiki/doc/GAEuploader

I would like to echo this call for testing. GAEuploader automates the
process of uploading the meek code to App Engine. If you already have a
Google account, it's pretty easy: the program will open a browser so you
can authenticate and accept the App Engine terms of service, then upload
the code under a domain name you choose. At the end, GAEuploader prints
out a bridge line to paste into Tor Browser.

The difference between this and the old meek-google is that the App
Engine code you upload is just for you, not shared with anyone else.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] OnionGatherer: evaluating status of hidden services

[David Fifield]

On Fri, Mar 10, 2017 at 12:58:55PM +0100, Massimo La Morgia wrote:
> we are a research group at Sapienza University, Rome, Italy. We do research on
> distributed systems, Tor, and the Dark Web. As part of our work, we have
> developed OnionGatherer, a service that gives up-to-date information about 
> Dark
> Web hidden services to Tor users.

...and presumably helps you build a crowdsourced list of onion services
that you plan to use for some other research purpose?

If you're planning a research project on Tor users, you should write to
the research safety board and get ideas about how ot do it in a way that
minimizes risk.
https://research.torproject.org/safetyboard.html

This idea seems, to me, to have a lot of privacy problems. You're asking
people to use Chrome instead of Tor Browser, which means they will be
vulnerable to a lot of fingerprinting and trivial deanonymization
attacks. Your extension reports not only the onion domains that it
finds, but also the URL of the page you were browsing at the time:
var onionsJson = JSON.stringify({onions:onions, website: 
window.location.href});
You need to at least inform your research subjects/users what of their
private data you are storing and what you are doing with it.

You're using two different regexes for onion URLs that aren't the same.
The one used during replacement doesn't match "https", so I guess it
will fail on URLs like https://facebookcorewwwi.onion/.
/^(http(s)?:\/\/)?.{16}(\.onion)\/?.*$/
/(http:\/\/)?\b[\w\d]{16}\.onion(\/[\S]*|)/
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Flag blocked websites

[David Fifield]

On Fri, Mar 10, 2017 at 08:27:01AM -0500, Boter42 wrote:
> I think it would be important to have a way to flag/report those websites that
> can't be access by the users while they're using the tor browser. 
> 
> Is there already a solution to do this? Do you think it would be a good tool? 
> 
> It would be great to have an updated records of this kind of websites so that
> we can push website owners to make the Tor user-experience as smooth as
> possible. 

There's an informal yet quite large list here:
https://trac.torproject.org/projects/tor/wiki/org/doc/ListOfServicesBlockingTor
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev