Joe Zeff posted on Mon, 02 Apr 2012 12:03:24 -0700 as excerpted: > On 04/02/2012 11:46 AM, Duncan wrote: >> On gentoo I build from sources, but I've not had dependency hell since >> my Mandrake days, as there's usually at least a masked version of the >> ebuild that I can unmask somewhere, that has all the deps listed. > > I know that apt-get, the Debian/Ubuntu package manager, will resolve > deps and bring them in as needed, as I've seen it happen on my sister's > Ubuntu box. And, yum, the Fedora equivalent, does the same. I'm not > familiar with gentoo, so I have to ask: doesn't your package manager > take care of such things, or are you talking about building packages > that are not from the gentoo repos?
Umm... somewhere along the line, one or both of us got off track. I /thought/ what I wrote was perfectly clear, but obviously not, at least not to you, or your question either wouldn't be needed or would itself be clear to me as well. So I'll try backing up a bit and do a retrace. Let's see if that helps. 1) Bob is trying to build pan from current git sources (so presumably he has a pan package and its deps are installed, but any new deps in the git version are not). It builds fine without SSL support (thus the original question, since he didn't know the ssl support wasn't built), but won't build with it as his gnutls is too old (that's a new dep, since the ssl support is new, no released version with it, yet). 2) So he needs a new gnutls to build the new pan with ssl support. Again, the package he has would take care of deps as of the old gnutls, but he needs a newer one for pan, with newer deps. 3) As it happens, one of those newer gnutls deps is nettle, and in gentoo (what I use here) as well, the "stable" gnutls was too old for pan. They hadn't stabilized the newer gnutls due to API changes related to its new nettle dependencies. I suspect that we're all clear to that point. 4) This is where I think the confusion may have started so I'll include a bit more detail this time around. I don't know what distro bob is on but I'm presuming it's not the Gentoo ~arch (aka testing) that I run. Whatever it is, tho, they seem to be in about the same position Gentoo itself was a couple months ago when I went thru the gnutls upgrade in ordered to satisfy pan's new deps, myself. Originally I left out a bit of gentoo detail as I doubted it mattered to him, but eh... throw it in this time and see if it clears things up. 5) On gentoo the in-tree available versions of gnutls and their stability level has changed since March 16: 2.10.x (2.10.5 currently, might have been 2.10.4 or whatever then): No nettle required, stable for quite some time. (This is what Dave S says is in Ubuntu latest, also.) 2.12.x (2.12.18 added in gentoo on March 16): Requires nettle AND p11- kit. In theory, it's still 2.x so should maintain API, but apparently there were some issues, as gentoo, and now it seems bob's distro... and dave shocket's (which he said is Ubuntu latest)... stuck at 2.10.x stable for quite some time. Since March 16's gentoo addition of 2.12.18, various archs appear to be stabilizing on it, with about a third of them (including the big ones, amd64 and x86) now stable on 2.12.18, while the rest are still stable on 2.10.5. 3.0.x (3.0.16 also gentoo-added on March 16): 3.x major version. Gentoo has this available in-tree, but it's not stable yet, only ~arch (testing). Before March 16's 3.0.16, 3.x was in-gentoo-tree but I believe hard-masked, not visible for installation even for ~arch users, unless they queried for it specifically, and deliberately unmasked it. >From what I can see both from gentoo, and now from this thread I see the same in other distros, the 2.12 version is where a lot of the BIG changes occurred, including nettle and p11-kit dependencies, and really SHOULD have been 3.0. The 2.x series should have remained stable at 2.10.x, without the nettle and p11-kit deps, which only should have been added for the 3.x series. Oh, well, upstreams decide, distros (and end users advanced enough to care) have to cope with it. 6) Now, it should be mentioned that gentoo is both automated-from-source (end-user builds, but using an automated dependency tracking and build- script system) and rolling distribution (no real distro version upgrades after installation, individual package upgrades are introduced and unmasked to ~arch, then to stable, over time). Additionally, unlike a lot of distros but as can be seen from #4 above, it's the norm for multiple versions of a particular package to be in-tree at the same time, with various users on various versions, depending on whether they're ~arch or stable (or even have unmasked hard-masked versions), their upgrade schedule and whether they religiously deep-upgrade (as I do), or only pull in dependency upgrades when some leaf package they've installed ups its dependency to require it, or the old versions get pruned from the tree. 7) On gentoo, ~arch (testing) is generally defined as upstream-stable, but gentoo-package testing, tho in practice, there's a bit of variance on that per individual package. So with some exceptions, unless upstream defines it as stable, it's not even a candidate for ~arch and later stable. Upstream development versions, betas and rcs are often available in-tree, but they normally remain hard-masked, only visible to those who deliberately unmask them for their own reasons. Additionally, in many cases especially for major projects such as kde/gnome/xorg/mozilla, upstream unstable/development versions are not added to the main tree, but can be found in various overlays. Often, the overlays have live-vcs- version builds available as well, for the REALLY adventurous. There's a few of them (hard-masked, of course) in the main tree, but not many. ***>>> 8) Dependency-wise, gentoo's ebuilds (the build-scripts for individual packages) specify all the usual dependency information, which is then automatically handled by the package manager. That's why I said I'd not seen dependency hell since back when I was on Mandrake. That's I guess the short and direct answer to your question, but it's not complete. ***<<< 9) Here, I normally run ~arch, but in addition to the main tree, have the kde, x11/xorg and mozilla (for firefox) overlays added, and routinely unmask various packages before they're unmasked in-tree. For example, due to gentoo's build-from source, gcc is a critical system package, and it doesn't tend to be unmasked even to ~arch until it can build nearly everything at that stability level (so ~arch for me). gcc 4.6 still remains hard-masked, for instance, due to the fact that gentoo's still stable on grub-legacy (0.97 with a whole slew of patches), with grub2 packages in-tree but hard-masked, and there's apparently some blocking boot-bugs when grub-legacy is built with gcc-4.6. But I unmasked gcc 4.6.2 a few months ago here and did a full rebuild. Everything I'm running (I've unmasked grub2 here) builds and appears to run fine with gcc 4.6.2. Similarly, I unmasked and ran the kde 4.8 prereleases, betas 1 and 2, and rcs 1 and 2. (FWIW, kde 4.8.0 was released at the end of January, with 4.8.1 a month later. 4.8.2 should be out this week. So the pre-releases were late last fall and early winter.) They were never in-tree at all as they're pre-releases and weren't stable candidates, but I'm running the kde overlay, which had them available. ***>>> (to end) 10) Because I'm always running at least ~arch, which by definition is gentoo testing level ebuilds for upstream stable, and am often deliberately unmasking the even less mature hard-masked versions, not even ~arch, while gentoo's ebuilds DO specify dependency information, at the immaturity level I run them, that dependency information isn't always 100% current. Thus, a more complete answer to your dependency question is yes, gentoo's package management deals with it in general, but at the immature package testing level I routinely run, those dependencies aren't 100% guaranteed to be current and upto date. So I do very occasionally see too-low-a- version errors when I try to build something. However, because I DO religiously run deep upgrades (I don't ever NOT do deep upgrades and my own helper scripts all assume deep), in practice, I very seldom see such errors, because generally I've long since done that upgrade anyway. Instead, I much more often see errors because I'm trying to run versions too new as I've unmasked something that's hard-masked precisely for that reason, an incompatible API change or the like, and either the deps are specified in the ebuild and I have to try to figure out how to get everything playing on either the old or new version, or they're not yet high-side version limited or patches applied to build with the new versions (I unmasked the still-masked version, not yet expected by the currently specified dependencies, after all), and I have to go looking for patches or unmasking sometimes live-vcs versions of the depending packages, in ordered to get everything working with the new versions. But that's not dependency hell, since (1) it's almost always due to a deliberate unmask on my part, and (2) it's almost always only a single level of dependency changes I need to figure out. Beyond that, generally I'm already updated, and/or the existing dependency specs are correct, so I don't get into the sort of recursive "but I can't build this because it depends on that, and I can't build that because it depends on something else, N levels deep" sort of situation that is normally referred to as "dependency hell". 11) Another contrast between gentoo and most distributions that helps with this sort of thing, is that since gentoo assumes build-from-source at the user end, it really doesn't make sense at all to try to split packages into run-time and development sub-packages. IOW, there's no libXXXX-dev packages on gentoo, there's only the libXXXX package, including both the binary library and the related headers, *.pc files, etc, that are normally only required if you're building a package depending on that library. Since in gentoo, EVERY package is assumed to be built from source, (almost, servant-ware binary-only packages are an exception) ANY package depending on that library would require the build- time bits to build and install anyway, and why would someone only install the library, if they weren't going to be using it as a dependency for some other package. On a source-based distro, it really doesn't make sense to split up packages into runtime and -dev sub-packages, so gentoo doesn't do so. That alone seriously diminishes dependency hell issues, since it roughly halves the number of package dependencies required. No more having to track down libXXX-dev, even when one *IS* doing manual builds outside the package system, because if the library is installed, so are the bits of it normally found in the -dev package on binary-based distros. =:^) 12) So much for the general gentoo case, now we're back to the specific case at hand, pan from live-git-repo sources. =:^) As mentioned above, only git-pan requires gnutls at all, and that only when the --with-gnutls configure option is enabled in ordered to handle secure connections. FWIW, gentoo does have a live-git pan ebuild (pan-9999, in gentoo, -9999 versions are used for live-vcs packages) available, but it doesn't have a gentoo developer following things closely enough to keep up with deps like this. As a result, I've customized my pan-9999, keeping it in my own personal overlay, where I can manage the deps changes myself. I'd submit the changes back to gentoo, with optional deps changes hooked up to gentoo USE flags as necessary, and do occasionally do so (usually in relation to a release version bump), but generally only test the side of the option I'm interested in (in this case, --with-gnutls), not both sides, and sometimes don't even know how to properly define the mapping for both sides, so I'm afraid I'm not ready to be a proper proxy-maintainer of the gentoo pan-9999 ebuild yet -- my fixes only help on the one side, and require real gentoo developer help to get them setup for the other. So of course, when git-pan's deps change, I'm making the changes to my ebuild to reflect them, myself. The package management system obviously can't do that by itself, tho as with most distros, there's various maintainer tools available to help with the process. But a lot of the time, I'm not even aware of minimum version requirement changes, because as I said, I routinely do deep upgrades myself, and with occasional exceptions like this gnutls thing, generally have well above minimal versions long since installed. So usually, the only time I actually bump the in-ebuild deps is when Heinrich or someone mentions current deps on list, and I go back and compare what's in the ebuild against them, or when I sync to changes made in the gentoo ebuild since I last checked it, sometimes in preparation for submitting a gentoo pan package version-bump request, etc. Of course right now, with gnutls being such a new dep, when the new pan version comes out this summer (assuming a continuation of the annual bump pattern, on or about August 1), if I file the version-bump request with gentoo, I'll probably have to say I don't know exactly how to specify the new deps and how to hook them up to the relevant SSL USE flag (there's no old one yet to pattern after). Meanwhile, I haven't modified my ebuild at all for that, but instead, have a bashrc file setup that portage (gentoo's main PM and the one I use) will source before reading the ebuild, that adds the --with-gnutls option via an available user extension syntax. ( EXTRA_ECONF='--with-gnutls' ) So the ebuild doesn't specify gnutls dependencies at all, I use the extension to inject the configure script option, and with that the configure script simply picks up the gnutls (3.0.16) I've manually unmasked and upgraded to. 13) But of course, most of that, being gentoo specific, was irrelevant to bob and dave, as dave's running ubuntu and bob's presumably not running gentoo either. So I had skipped all that, apparently leaving you with the impression that gentoo's ebuilds don't handle deps automatically, even tho I specifically stated that I hadn't experienced dependency hell since my mandrake era, before I switched to gentoo. I still don't quite understand how you got from that specific statement that I had NOT had dependency hell on gentoo, that it didn't handle the deps well, but since I had skipped the details as irrelevant in that context, I guess it must have been due to omitted information. I trust that my usual information overload approach has corrected that problem now? =:^) 14) Fully back to the thread topic, now. Based on data from multiple distros now, it seems that gnutls 2.10.x has been where many stabilized, because 2.12.x and above require nettle and p11-kit. Meanwhile, pan's new secure-connections feature requires the --with-gnutls configure option, which requires gnutls 2.12+ as earlier versions apparently didn't work so well in Heinrich's testing. So it is that users trying that feature in git-pan are having to build a newer gnutls from sources, since their distros are still stabilized on 2.10, and end up running into both the nettle and p11-kit recursive dependency issues when they do so. Those in turn may recurse into further necessary updates, thus triggering "dependency hell", the seeming unending series of manual updates required just to build what was originally thought to be a single package update. Luckily for me, most of the work was already done for me in gentoo and because I run quite new versions of everything anyway; I only had a couple unmasks to do, plus of course rebuilding a couple other packages that depended on gnutls after the upgrade, due to the ABI issues. But I'd say about 95% of the complexity was already handled, and what wasn't, was a level of issue that I'm used to dealing with relatively routinely anyway, in my normal course of early unmaskings, running kde betas, etc. People on the more ordinary binary distros, however, will be far more likely to have to deal with at least 20% of the issues manually (unless they're running redhat 5 or something, the packaging system will still deal with the majority of the problem, but it'll still look like dependency hell, especially to the uninitiated), where I had to deal with only perhaps 5%, and on average, they'll be less accustomed to dealing with that level of dependency resolution issue in the first place. So they'll have perhaps 4X the exposed problem, but only be perhaps a third as well prepared to deal with it, resulting in an experience that's likely 4*3=12 times more difficult for them to deal with! But I've seen suitably determined people bring even a redhat 5 system up to current kde, taking days of recursive dependency builds to even get kdelibs to build, but getting it done. If a suitably determined person can do that, then surely, a suitably determined person can do the much less difficult as it's far closer to current, upgrade from pan 0.133+ to git-pan, and with it from gnutls 2.10 to at minimum 2.12, along with its recursive dependencies in turn. There, there's only the two of them, nettle and p11-kit, the others are likely to be upto date, instead of the several dozen just to build kdelibs, that the poor guy upgrading that redhat system was dealing with! -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman _______________________________________________ Pan-users mailing list Pan-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/pan-users
