On Mon, Jun 30, 2014 at 2:15 AM, coderman <coder...@gmail.com> wrote: > 1) compute the cost of global traffic analysis. we have big data mark
> specifically UPSTREAM model collection at backbone peering points. > this is just one part of a series of costs; how much raw DPI capacity > (it is finite)? how much memory/storage for backtrace to some hours > window? 30day window? how much engineering time (earth human hours) to > implement the collection, classification, and analysis of all flows in > daily time? in near-real-time (<60sec)? how is accuracy beyond doubt > identified? how much does additional accuracy in shorter time cost? Along your three posts relative to the above... Netflow at scale is a challenge but not an impossibility. First address creating, recording and searching the flows. Then induce a client/server to [regularly] create traffic you can spot and search for it [1]. You don't need full take / DPI for that. And once a tap is in place you can use it for both at once. Excluding the secret tap itself [2], estimating costs of netflow per bandwidth is a matter of common commercial parts. (Storage is fixed, but there are probably some speedups to be had in creating, filtering and search with custom gear.) ISP's routinely utilize netflow for engineering metrics and security things. [1] Tor, I2P, high latency, low latency, store-forward, etc... perhaps with any non busy / non full of chaff / non fixed cell size system this could be recognizably induced. And the list of relays in these networks is known which allows you to select and handoff those flows for dedicated analysis. [2] Getting enough taps out there and at the right places to ensure that your searches have some useful hit rate... now that seems the hard and expensive part if you don't have some cooperation/force with the Tier-n's. For this induced purpose it's probably cheaper and easier to Sybil up a bunch of nodes than to tap the internet. Yet I'd not discount the possibility and value of some larger attempt at global analysis like that. Especially since ISP's and researchers already do it on their own scales. -- tor-talk mailing list - tor-talk@lists.torproject.org To unsubscribe or change other settings go to https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk
