[RFC] apt-zeroconf 0.1
Dear Debian Developers,
we are working on a distributed apt-cacher for local networks
implemented in Python. We call it apt-zeroconf since we use avahi for
automatically finding other apt-zeroconf instances on the LAN, similar
to Apple's Rendezvous/Bonjour/Zeroconf technology.
It is still in an early development stage, so we wouldn't recommend it
for everyday use. This release is solely for the purpose of getting
feedback and further ideas from you. And bugreports, of course.
Please send all comments, ideas or insults either as replies to this
email or to our trac at
http://trac.phidev.org/trac/wiki/AptZeroconf
You can also reach us via IRC in #apt-zeroconf on irc.debian.org.
Features
* implemented in Python
* easy and fast to set up
* share your apt-cache with others on your LAN transparently using
avahi
* automatic file-list-updates using inotify
* acts as a http server and proxy for apt
* no root privileges required
* active development
Concept
---
When running more than one Debian box in a LAN with slow internet access
the most often used solution to prevent downloading the same Debian
package twice is running apt-cacher or the like on one single machine,
a centralized server.
The problem with this solution is, that every machine in the network
needs to be configured for this proxy. If it is down, no one can update.
If you go to another network, you must tweak your sources.list or
apt.conf in order to upgrade, which is quite common nowadays with
notebooks and wifi technology.
We would therefore be interested in an automatic and peer-to-peer
solution. When there is no apt caching daemon in the LAN, we want to
fetch the packages directly from the internet. But if there is one or
even more available, apt should automatically use it, without any
configuration from the user. This is by the way the reason we called
it apt-zeroconf in the first place.
The next question is: Who runs an apt caching daemon? In our first
implementation, everyone running apt-zeroconf also shares his apt
package cache in /var/cache/apt/archives to everyone in the network.
Now, one might think this could potentially pose a security threat as
everyone can offer and distribute debs without any authentication
whatsoever. This is not the case as we are not yet caching the package
lists or pdiffs, which are PGP-signed and contain MD5, SHA1 and SHA256
checksums of the packages. But due to the trusted PGP signatures,
caching package lists shouldn't be an issue.
Technical
-
Apt-zeroconf consists of two parts: a http server and the avahi-related
stuff. Avahi is used to let everyone else in the network know that you
have an apt-zeroconf caching service installed and running. The httpd
does act as http server if used in fake-mirror-mode, but also as http
proxy if you're using it as such with apt. It uses port 1618 by
default.
In every mode, apt-zeroconf is provided with a fallback mirror by apt,
so if there are no apt-zeroconf instances on the LAN offering your deb,
your regular Debian mirror will be used to provide apt with the
necessary files. See Configuration for details.
Configuration
-
Yes, there is a little configuration, despite the name, but only
because the user needs to be aware that he is actually using it and
that the daemon has to be running when performing an update, upgrade
or install.
There are two ways to use it:
* Proxy Mode
Add the following line to your /etc/apt/apt.conf:
Acquire { http { Proxy "http://127.0.0.1:1618";; } }
This way every mirror listed in your sources.list will be cached.
If you don't want this use the second method.
* Fake Mirror Mode
Change in your /etc/apt/sources.list
deb http://your.mirror.debian.org/debian sid main
to
deb http://127.0.0.1:1618/your.mirror.debian.org/debian sid main
e.g. deb http://127.0.0.1:1618/ftp.de.debian.org/debian sid main
With this method, you can select only specific mirrors to be patched
through apt-zeroconf.
TODO
* Configuration
* Support for resuming (HTTP header pass-through)
* Proxy support
* Documentation
* Use debconf for even easier configuration
Where to get it
---
Grab it while it's hot from:
* source: http://phidev.org/azc/apt-zeroconf_0.1.tar.gz
* deb: http://phidev.org/azc/apt-zeroconf_0.1_all.deb
* dsc: http://phidev.org/azc/apt-zeroconf_0.1.dsc
Cheers,
Florian Ludwig
Franz Pletz
--
Franz Pletz \ eat Depends: cook | eat-out
www: http://franz-pletz.org/ \ But eat-out is non-free so that's out
email: [EMAIL PROTECTED] \ And cook Recommends: clean-pans
signature.asc
Description: Digital signature
Re: [RFC] apt-zeroconf 0.1
On Sun, Nov 19, 2006 at 07:07:03PM +0100, Reinhard Tartler wrote: > Why don't you install a line like > > >e.g. deb http://127.0.0.1:1618/debian main contrib non-free > > in /etc/apt/sources.list.d, and be done with it? That's what I'd call > zero config. > > You would have to convince apt to prefer apt-zeroconfig sources over > 'regular' HTTP sources, so that it only uses the slow line when the > package couldn't be found in the neighbourhood. I believe this could be > tweaked be tuning /etc/apt/preferences. If not, you would have to hack > up apt a bit. We also pondered with that idea, but with this configuration modifying apt is definitely required. Apt associates a server to every package and version where to get the debs from. We are not building package lists from all hosts in the network because this would compromise the security of the whole network. To understand this, let me show you what is happening behind the curtains if you're adding a line like this deb http://127.0.0.1:1618/ftp.de.debian.org/debian main to your sources.list: 1. When you perform an update the package lists and PGP signatures will be fetched from ftp.de.debian.org. Apt-zeroconf effectively acts as a proxy to apt. 2. If you install software and this mirror has the current version and/or is pinned, apt will query apt-zeroconf for the file. Apt-zeroconf looks in its file-lists if a host in the network has this file and fetches it in this case. If not, ftp.de.debian.org is also used by proxying the request. Now, without a package list, we can't just easily pin our source. When we generate a package list from all the hosts in the network we can't authenticate the source of the packages and the list through the checksums as they might be forged. If we can get the package list and the corresponding PGP signature from another host in the network, however, we can check if we are in a secure environment. This is also a feature which we would like to implement in future versions. So, what modification would apt need for this cool feature to work? We would need to be able to define some kind of master-server where every request for debs would go to. We try to emulate this by associating the package list from a mirror to our apt-zeroconf proxy with our scheme of changing sources.list. Yes, it's a hack, but it works. What would also be nice is some kind of download-hook where we can either change the URL before it is downloaded or the download mechanism itself. We didn't want to go through the trouble of modifying apt itself initially, though. > >With this method, you can select only specific mirrors to be patched > >through apt-zeroconf. > > In this scenario, I'd personally don't care at all which mirror I get my > packages from but from the apt-suite and the archive I get the packages > from. This can be determined by examining the current apt config, so it > is possible to create such a config in postinst. But if we do this in postinst, everytime a user changes his apt config, our postinst-script needs to be run somehow. I think we can't do this easily and reliably. > As you already said, authentication is already ensured by apt itself, > which is nice. Unfortunately, this is only the case if we have a correctly authenticated package list, see my comments on this above. > > TODO > > > > * Proxy support > > why proxy support? I thought you want to serve packages from the local > neighbourhood. No, currently, apt-zeroconf might also fetch packages from an external HTTP source itself and therefore act as a proxy to apt. This won't work if you need a proxy to connect to the internet. Thanks for your comments. Cheers, Franz -- Franz Pletz \ eat Depends: cook | eat-out www: http://franz-pletz.org/ \ But eat-out is non-free so that's out email: [EMAIL PROTECTED] \ And cook Recommends: clean-pans signature.asc Description: Digital signature
Bug#368966: ITP: pympd -- Frontend for mpd in the style of rhythmbox and itunes
Package: wnpp Severity: wishlist Owner: Franz Pletz <[EMAIL PROTECTED]> * Package name: pympd Version : 0.06.1 Upstream Author : pympd Dev Team * URL : http://http://pympd.sourceforge.net/ * License : GPL Programming Lang: Python Description : Frontend for mpd in the style of rhythmbox and itunes Pympd is a frontend for mpd in the style of rhythmbox and itunes, written in python, with pygtk. Pympd itself is not a music player, but is actually a frontend to mpd, a daemon for playing and managing music. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Real Life hits: need to give up packages for adoption
On Mon, May 29, 2006 at 09:29:34PM +0200, wrote: > * festival, speech-tools > (some clean-up work, new major upstream release pending) [..] > * gnupg2 > (some clean-up work) I'd be interested in taking these three, especially gnupg2 as I'm using it on a daily basis. Thanks, Franz -- Franz Pletz \ Lots of the basis of the open source movement www: http://franz-pletz.org/ \ come from procrastinating students. email: [EMAIL PROTECTED] \ -- Andrew Tridgell signature.asc Description: Digital signature
Bug#369932: ITP: gplanarity -- simple puzzle game involving untangling planar graphs
Package: wnpp Severity: wishlist Owner: Franz Pletz <[EMAIL PROTECTED]> * Package name: gplanarity Version : 11496 Upstream Author : Monty <[EMAIL PROTECTED]> * URL : http://web.mit.edu/xiphmont/Public/gPlanarity.html * License : GPL Programming Lang: C Description : simple puzzle game involving untangling planar graphs gPlanarity is a simple puzzle game involving untangling planar graphs for fun and prizes. If you tend to get addicted to cute little math puzzles, this one is a doozy. gPlanarity is a super-clone of the flash Planarity game written by John Tantalo. The original Planarity ran well in IE and Firefox on other platforms, but was slow and liked to lock up or abort under Linux browsers. gPlanarity implements gameplay identical to the original Planarity but adds some UI and game extras around the basic game such as multiple board generation algorithms, puzzle boards, complete backing state, group select/drag and so on. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#375441: ITP: python-stemmer -- Python bindings for Snowball stemming algorithms
Package: wnpp Severity: wishlist Owner: Franz Pletz <[EMAIL PROTECTED]> * Package name: python-stemmer Version : 1.0.1 Upstream Author : Richard Boulton <[EMAIL PROTECTED]> * URL : http://snowball.tartarus.org/ * License : BSD, MIT Programming Lang: C, Python Description : Python bindings for Snowball stemming algorithms PyStemmer provides access to efficient algorithms for calculating a "stemmed" form of a word. This is a form with most of the common morphological endings removed; hopefully representing a common linguistic base form. This is most useful in building search engines and information retrieval software; for example, a search with stemming enabled should be able to find a document containing "cycling" given the query "cycles". PyStemmer provides algorithms for several (mainly european) languages, by wrapping the libstemmer library from the Snowball project in a Python module. It also provides access to the classic Porter stemming algorithm for english: although this has been superceded by an improved algorithm, the original algorithm may be of interest to information retrieval researchers wishing to reproduce results of earlier experiments. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#435678: ITP: cheese -- take pictures and videos from your webcam
Package: wnpp Severity: wishlist Owner: Franz Pletz <[EMAIL PROTECTED]> * Package name: cheese Version : 0.1.4 Upstream Author : Daniel G. Siegel <[EMAIL PROTECTED]> * URL : http://live.gnome.org/Cheese * License : GPL-2 Programming Lang: C Description : take pictures and videos from your webcam Photobooth-inspired GNOME application for taking pictures and videos from a webcam. It also includes fancy graphical effects based on the gstreamer-backend. . Future releases will be able to interface with IM clients, youtube and flickr. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#456815: ITP: apt-zeroconf -- Caching peer-to-peer apt proxy for local networks
Package: wnpp Severity: wishlist Owner: Franz Pletz <[EMAIL PROTECTED]> * Package name: apt-zeroconf Version : 0.4.1 Upstream Author : Franz Pletz <[EMAIL PROTECTED]> * URL : http://trac.phidev.info/trac/wiki/AptZeroconf * License : GPL Programming Lang: Python Description : Caching peer-to-peer apt proxy for local networks Apt-zeroconf is an extension to apt which enables it to find other instances on the LAN in order to exchange debian package files and not to fetch them from the internet. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
