Re: [gentoo-dev] preserve_old_lib and I'm even more lazy
On Fri, Feb 24, 2012 at 1:43 PM, Alexis Ballier wrote: > moreover the && wont delete the lib if revdep-rebuild failed i think, > so it should be even safer to copy/paste :) Am I the only paranoid person who moves them rather than unlinking them? Oh, if only btrfs were stable... Rich
Re: [gentoo-dev] btrfs status and/was: preserve_old_lib
On Fri, Feb 24, 2012 at 8:06 PM, Richard Yao wrote: > Have you tried ZFS? Yes - but not terribly interested in doing that on linux. I do appreciate that it can be done, but still lacks raid-z reshaping, which means it isn't quite flexible enough. > On 02/24/12 18:26, Duncan wrote: >> FWIW, in the rare event it breaks revdep-rebuild or the underlying >> rebuilding itself, I rely on my long set FEATURES=buildpkg and >> emerge -K. I also use buildpkg, but I don't keep them around forever. >> I'm not sure if that's a reference to the btrfs snapshots allowing >> rollbacks feature, or a hint that you're running it and worried >> about its stability underneath you... That would be the former. I'm QUITE aware of its stability. I've played around with it on a VM - I posted on my blog an experience with it around a year ago as well. It has come quite a way, but it is definitely not production quality. Xfs-tools is useful if you want to try breaking it - I think I posted on my blog an article about capturing linux kernel core dumps for debugging purposes - it panics quite readily. If you do want to mess with it I'd recommend using the git kernel maintained by the btrfs team. It is obviously bleeding-edge, but due to the high pace of fixes it tends to be more stable than the version in the mainline kernel. Rich
Re: [gentoo-dev] preserve_old_lib and I'm even more lazy
On Fri, Feb 24, 2012 at 10:44 PM, Mike Gilbert wrote: > > I've been using btrfs exclusively for about 6 months, and I don't > *think* I've lost anything... :) > >From what I've seen as long as you keep things simple, and don't have heavy loads, you're at least reasonably likely to get by unscathed. I'd definitely keep good backups though. Just read the mailing lists, or for kicks run xfs-test on your server. Xfs-test doesn't do any direct disk access or anything like that - it is no different than running bazillions of cat's, mv's, rm's, cp's, etc. It most likely will panic your system if you try it on btrfs - on ext4 it will probably load the living daylights out of it but you should be fine. The issues with btrfs at this point are the ones that aren't so easy to spot, like race conditions, issues when you use more unusual configurations, and so on. Oh, and go ahead and try filling up your disk some time. If your kernel is recent enough it might not panic when you get down to a few GB left. I'm eager for the rise of btrfs - it IS the filesystem of the future. However, that cuts both ways right now. Rich
Re: [gentoo-dev] preserve_old_lib and I'm even more lazy
On Sat, Feb 25, 2012 at 10:02 AM, Doug Goldstein wrote: > FWIW, I'll second the ZFS > btrfs suggestion. Oh, if you need a safe COW filesystem today I'd definitely recommend ZFS over btrfs for sure, although I suspect the people who are most likely to take this sort of advice are also the sort of people who are most likely to not be running Gentoo. There are a bazillion problems with btrfs as it stands. However, fundamentally there is no reason to think that ZFS will remain better in the future, once the bugs are worked out. They're still focusing on keeping btrfs from hosing your data - tuning performance is not a priority yet. However, the b-tree design of btrfs should scale very well once the bugs are worked out. Rich
Re: [gentoo-dev] preserve_old_lib and I'm even more lazy
On Sat, Feb 25, 2012 at 3:52 PM, Richard Yao wrote: > ZFSOnLinux performance tuning is not a priority either, but there have > been a few patches and the performance is good. btrfs might one day > outperform ZFS in terms of single disk performance, assuming that it > does not already, but I question the usefulness of single disk > performance as a performance metric. Why would btrfs be inferior to ZFS on multiple disks? I can't see how its architecture would do any worse, and the planned features are superior to ZFS (which isn't to say that ZFS can't improve either). Beyond the licensing issues ZFS also does not support reshaping of raid-z, which is the only n+1 redundancy solution it offers. Btrfs of course does not yet support n+1 at all aside from some experimental patches floating around, but it plans to support reshaping at some point in time. Of course, there is no reason you couldn't implement reshaping for ZFS, it just hasn't happened yet. Right now the competition for me is with ext4+lvm+mdraid. While I really would like to have COW soon, I doubt I'll implement anything that doesn't support reshaping as mdraid+lvm does. I do realize that you can add multiple raid-zs to a zpool, but that isn't quite enough. If I have 4x1TB disks I'd like to be able to add a single 1TB disk and end up with 5TB of space. I'd rather not have to find 3 more 1TB hard drives to hold the data on while I redo my raid and then try to somehow sell them again. Rich
Re: [gentoo-dev] preserve_old_lib and I'm even more lazy
On Sat, Feb 25, 2012 at 4:56 PM, Richard Yao wrote: > raidz has 3 varieties, which are single parity, double parity and > triple parity. As for reshaping, ZFS is a logical volume manager. You > can set and resize limits on ZFS datasets as you please. That isn't my understanding as far as raidz reshaping goes. You can create raidz's and add them to a zpool. You can add individual drives/partitions to zpools. You can remove any of these from a zpool at any time and have it move data into other storage areas. However, you can't reshape a raidz. Suppose I have a system with 5x1TB hard drives. They're merged into a single raidz with single-parity, so I have 4TB of space. I want to add one 1TB drive to the array and have 5TB of single-parity storage. As far as I'm aware you can't do that with raidz. What you could do is set up some other 4TB storage area (raidz or otherwise), remove the original raidz, recycle those drives into the new raidz, and then move the data back onto it. However, doing this requires 4TB of storage space. With mdadm you could do this online without the need for additional space as a holding area. ZFS is obviously a capable filesystem, but unless Oracle re-licenses it we'll never see it take off on Linux. For good or bad everybody seems to like the monolithic kernel. Btrfs obviously has a ways to go before it is a viable replacement, but I doubt Oracle would be sinking so much money into it if they intended to ever re-license ZFS. Rich
Re: [gentoo-dev] preserve_old_lib and I'm even more lazy
On Sat, Feb 25, 2012 at 5:47 PM, Richard Yao wrote: > If you have proper backups, you should be able to destroy the pool, > make a new one and restore the backup. If you do not have backups, > then I think there are more important things to consider than your > ability to do this without them. I wouldn't have pointed it out if the solution were this simple in my case. Not everything is worth backing up - I'd rather take a 2% chance of losing everything but maybe the 0.1% of my storage that I back up, than wipe the drive and have a 100% chance of losing everything but the 0.1% of my storage that I back up. My data isn't worth the cost of a proper backup solution, but it isn't worthless either so if I can have my cake and eat it too so much the better. That said, it is true that reshaping often isn't practical for other reasons, such as having 4 1TB drives, and by the time you want to add another one the best price point is on 500TB drives. Thanks for your comments just the same - they are informative. My licensing concern is more of wanting to promote GPL software than being compliant, so FreeBSD isn't much of a help. You may be right about Oracle wanting to keep btrfs for the low end, although where they are already aiming is already high enough for me, and once btrfs becomes mainstream nobody is really going to be able to hold it back - it isn't like Oracle actually has any control over it beyond contributing the most code. Rich
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Thu, Mar 8, 2012 at 11:51 AM, Ulrich Mueller wrote: >> On Thu, 08 Mar 2012, Michael Orlitzky wrote: > >> There's also libbash now: > > Looks like complete overkill to me, considering the simple task at > hand. > Plus, wasn't the whole point that we can't guarantee that the bash installed on a user's system can parse the ebuild until we've checked the EAPI? If we use libbash doesn't that just keep the same constraint but on a different package? Is the current stable libbash guaranteed to be able to parse a bash v12 script? If you just parse the file with a defined set of rules without regard to how bash might parse the file, then you can determine the EAPI and then decide how to source it. For all we know EAPI G will turn ebuilds into python scripts so trying to read the thing with bash or libbash will be doomed to failure. Rich
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Fri, Mar 9, 2012 at 12:47 PM, Zac Medico wrote: > Anyway, lets focus on our main goal, which is to decide on a way to > obtain the EAPI _without_ sourcing the ebuild. Agreed. Plus, an approach that either uses the filename or something like a comment line is also going to be much more flexible if bash syntax changes substantially, or if we want to support ebuilds-in-python or some other approach. That is the main merit I see to the shebang approach - you could define some kind of API that actually involves executing the ebuild. Sticking the EAPI in the filename is a little less complicated and it gives you the same flexibility - I'm not aware of any file format that is fussy about the content of the filename. Even approaches like putting EAPI=5 in the file (even inside a comment) might break if the file is meant to be generally interpreted by some program that has rigid syntax rules.Most scripting languages could probably handle this somehow, but if you ever wanted ELF ebuilds that wouldn't be so likely to fly unless you could embed \nEAPI=ELF8\n in something near the start of the file. Sure, I doubt we'll ever want ELF ebuilds (I'd consider non-programatic ebuilds for more trivial packages more likely), but the point is that we should prefer options that offer more flexibility down the road over ones that solve the immediate need but just leave us with the same debate two years from now. Rich
[gentoo-dev] Re: newsitem: unmasking udev-181
On Sat, Mar 10, 2012 at 9:27 PM, William Hubbs wrote: > here is the udev 181 unmasking news item. > > If all goes well, this will be committed to the tree on 3/14 UTC. I guess this might be OK for unstable, but before this goes stable we really need to improve the docs around this. As far as I can tell neither the genkernel nor dracut docs have specific instructions about how to handle mounting /usr. Since having a separate /usr is often the result of having a more complex configuration (nfs, lvm, mdraid, etc), instructions explaining how things work and how to handle variations is pretty important. Perhaps genkernel automagically does the right thing in some cases, but I know that dracut does not unless you properly configure it. I doubt either tool will handle more complex situations without some scripting. I'm not really asking for automation here - just documentation and reasonable stability in how things work. Again, this is likely more of a concern before this is stabilized. However, knowing what I went through to get my bind-mounted /usr on LVM+mdraid working with dracut, I can imagine that any unstable users with tricky setups could face a fun weekend. Perhaps a suggestion for the news item. I'd recommend that anybody who needs an initramfs to mount /usr get that working BEFORE they upgrade udev. This situation is a heck of a lot easier to figure out if the system still can be booted when the initramfs doesn't work. Rich
Re: [gentoo-dev] Re: newsitem: unmasking udev-181
On Sat, Mar 10, 2012 at 10:28 PM, Luca Barbato wrote: > I guess we could pour more effort in getting dracut more easy to use and/or > try to figure out which are the items that should be moved to / and fix the > remaining ones... > Well, I'm trying not to take sides in that war. Unless we want to patch the living daylights out of upsteam it seems almost moot. I'm not even so much interested in making dracut easy to use. I'd just appreciate it if there were even a single sentence written on each of its modules and options, let alone some kind of coherent guide to how to make it all work. There is a guide out there, but it hasn't really kept pace with the very rapidly evolving tool, and it doesn't really cover all the nuances. Another useful thing would be to update our official RAID+LVM guide so that when you're done following it your system will boot. Rich
Re: [gentoo-dev] Deprecate EAPI1?
On Sun, Mar 11, 2012 at 9:28 AM, Patrick Lauer wrote: > I'd deprecate eapi2 too, we don't need 5 flavours around when we > effectively only want to support one (and eapi0 in a few places) > > I wouldn't mind having a deprecation timeline for eapi3 too (now +6 > months maybe?), but there's no need to rush things. Is there really much of a benefit to this? I guess for anybody who runs scripts to mass-manipulate ebuilds it might be helpful, but I think all the package managers planned on supporting all the EAPIs for quite a while longer. I can imagine that this will lead to quite a bit of churn with updating ebuilds and especially eclasses. If a package doesn't require a feature in a newer EAPI, what is the point? Why not deprecate the x86 arch while we're at it... :) Rich
Re: [gentoo-dev] Deprecate EAPI1?
On Sun, Mar 11, 2012 at 12:18 PM, Chí-Thanh Christopher Nguyễn wrote: > Assume a new version 13.37 of your package manager drops EAPI=1 support. > So package-manager-13.37.ebuild checks in pkg_pretend() if any EAPI=1 > package is installed on the system. If yes, then it aborts, telling the > user to get rid of the package first. > > That way, the situation where the package manager does not know how to > uninstall a package is avoided. I take it that removing a package without using a package manager is left as an exercise to the reader? Rich
Re: [gentoo-dev] Deprecate EAPI1?
On Sun, Mar 11, 2012 at 8:15 PM, Francesco Riosa wrote: > To be able to upgrade a gentoo installation as old as five years is > interesting and valuable but require an effort that has yet to be > made. I suspect it shouldn't be difficult IF you have access to a binary package respository for the system set. Then you can do an emerge -K system and get all your core packages updated. In theory you should be able to install a gentoo stage3 in a chroot and do a quickpkg @system to generate such a repository. I'm open to comments as to why this wouldn't work, but it would seem to be the easiest solution to me. Of course, this won't get you whatever CFLAGS/USE settings you prefer - but once you have a working updated system set you could always do an emerge -e system to rebuild them with your own settings. Rich
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Sun, Mar 11, 2012 at 10:03 PM, Brian Harring wrote: > Pragmatic reality, the eapi function actually would work fine. As > pointed out elsewhere, bash parses as it goes- which isn't going to > change. Unless the ebuild isn't written in bash... How do you source the ebuild if you don't know what to use to source it? How do you know what to use to source it if you don't know the EAPI? Right now all the existing EAPIs use bash, but there is no reason the file couldn't be xml, or python, or just about anything else. If we want to allow for that kind of flexibility, then it might make sense to go ahead and state that our convention is to stick EAPI=5 in one of the first few lines of the ebuild, or inside a comment, but also go a step further and state that the text "EAPI=" cannot appear elsewhere in the ebuild (or perhaps within the first 10 lines). Just about any file format we might use would allow us to make "EAPI=" appear in it, but not all could guarantee that it would occur at the start of a line, or at the start of a line immediately after a #. In any case, I can really see the KISS value in a very rigid syntax that is trivial to parse. Stuff like this almost makes me wish our ebuilds already were xml files or such, with bash embedded inside sections. Finding a particular tag in an xml file is trivial as the fundamentals haven't changed in 15 years. Rich
Re: [gentoo-dev] Re: newsitem: unmasking udev-181
On Sun, Mar 11, 2012 at 7:15 PM, William Hubbs wrote: > > I was thinking of another news item once we are ready to go stable. > > What do you think? > I think that makes the most sense. That news item can include links to the documentation that gets written over the next few months. Rich
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Mon, Mar 12, 2012 at 6:12 AM, Kent Fredric wrote: > > There's the obvious case of compiled-binaries where that might not be > possible, but thats definately strawman stuff and I wouldn't support > that sort of nonsense anyway. Compiled binaries for ebuilds can gtfo. > Why do I feel like a similar debate must have happened on some list back when the SMTP message terminator was worked out? The problem is that we're running into is the use of in-band control information. An advantage of sticking the EAPI in the filename is that it is completely out-of-band. Other options like an external file or POSIX attributes have the same advantage (with the disadvantage that they are harder to keep in-sync or support in general). If you want to stick the EAPI inside the file then we have to go with one of two routes: 1. Make the file directly interpret-able by some existing tool like bash, xml parsers, or (gtfo) ld-linux.so. This requires making the EAPI in-band info, and thus if the file format changes there is risk that any existing code that interprets EAPIs could break, to the degree that we weren't infinitely clever in designing the spec. 2. Make the file require pre-processing before being fed to another interpreter. This breaks syntax-highlighting and such, and is generally more of a pita. However, this allows in-file EAPI info to remain out-of-band. An ebuild might be an xml file with EAPI and other meta-data, and might contain sections that get ripped out and handed over to bash, python, or ld-linux.so to the extent that you want to be told to gtfo. In my thinking the option with the fewest drawbacks and the greatest extent of future-proofing is to go with GLEP 55 and stick it in the filename. Every filesystem and rsync tool handles filenames, they're impossible to separate from file contents, and every interpreter out there ignores them. If we don't want to stick it in the filename then either we have to decide that whatever we come up with could break, or that we really have to change what ebuilds look like. Rich
Re: [gentoo-dev] RFD : .ebuild is only bash
On Mon, Mar 12, 2012 at 11:59 AM, Ciaran McCreesh wrote: > On Tue, 13 Mar 2012 04:57:04 +1300 > Kent Fredric wrote: >> I think this notion should be concluded before we continue debating as >> to how best to implement EAPI declarations. >> >> Is it really so fixed that ".ebuild" will only ever be bash ? > > What version of bash are we talking about here? It's not the case that > ebuilds will always be bash 3, which is enough to make GLEP 55 the safe > option. Well, we do always have the option of keeping the EAPI= syntax but making it more strict per the proposals, and then grepping it out rather than sourcing the ebuild. That seems likely to always work with bash. Then if we ever switched to some other format we'd have to reconsider whether we want to tweak this approach further or adopt GLEP 55. If you envision a future where big changes are inevitable over the long term, then just going with GLEP 55 is probably the cleanest solution. If you envision a future where we are likely to never move away from bash, or if we do it is so far off that we're content to let our children deal with it, then other approaches may seem nicer. I guess the question is whether we need to future-proof against a future that may never come. Then again, as we're seeing from systemd a lot of stuff that "always" was done in bash doesn't necessarily have to stay that way. Rich
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Mon, Mar 12, 2012 at 1:01 PM, Zac Medico wrote: > It would be very fragile without the sanity check / feedback mechanism > that's already been suggested. Another obvious check is to have repoman run a grep with the regexp and give an error if there is not exactly one match. Rich
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Mon, Mar 12, 2012 at 1:46 PM, Zac Medico wrote: > If we want to handle every possible screwup, including stray EAPI > assignments inside inherited eclasses, we still need to compare the > probed value to the value that's obtained from bash. Well, I wasn't intending to suggest that the repoman check need be the only one. However, preventing problems is at least as useful as detecting them. That said, it would probably best to have exactly one way to determine the official EAPI. If that is to parse the filename, then parse the filename. If it is to grep for a regexp and expect exactly one hit and parse it in a certain way, then do that. It might be that the "one" official way is to grep for a regexp and if you find it, use it, otherwise assume 0, 1, 2, 3, or 4 and source the ebuild for it. That still gives you only a single answer (well, except in situations where the current way is already broken). If people want to abuse the EAPI syntax I suppose we can generate an error, but ignoring it might be just as valid behavior. I'm not sure what happens if you define PN/etc in ebuild besides things breaking in a horrible manner. I'd put changing EAPI in the same category. Rich
Re: [gentoo-dev] RFD : .ebuild is only bash
On Tue, Mar 13, 2012 at 2:29 AM, Richard Yao wrote: > To make XML a viable substitute for bash, you will need to implement a > turing complete language in XML, which should probably preclude its use > in ebuilds. You would likely have better luck with a functional > programming language, although you are more than welcome to demonstrate > otherwise. Well, a trivial solution to that is to embed bash code (or some other language) into the content of the xml file. As I and others posted earlier the advantage is that it makes all the key/value stuff easier to manage than doing it in bash, but it makes editing the scripting content harder and requires pre-processing before being fed into an interpreter. If you look at metadata.xml you could argue that we're already using xml-based ebuilds to an extent, but we split the metadata across two different files in different formats and call them different things. In any case, my point in bringing up xml was that the whole point of GLEP 55 was to future-proof the interpretation of ebuild files, and xml is just one example of what the future could conceivably look like. Rich
Re: [gentoo-dev] Let's redesign the entire filesystem! [was newsitem: unmasking udev-181]
On Tue, Mar 13, 2012 at 8:20 PM, Joshua Kinard wrote: > The trend now seems to be to modularize everything these days, even stuff > like the core disk drivers, then build those core modules into an initramfs > that the kernel cherrypicks from at boot. That's the perception, anyways, > and one which I don't really get. Well, on most distros the kernel is just another package that is the same on every box. If you want one kernel for every PC, then it needs to support every piece of hardware in existence. So, either it is highly modular, or it is going to suck up a ton of RAM. The solution is a one-size-fits-all kernel, combined with a one-size-fits-all initramfs. For Gentoo where people build their own kernels, it doesn't make as much sense. Rich
Re: [gentoo-dev] Re: newsitem: unmasking udev-181
On Tue, Mar 13, 2012 at 8:29 PM, Joshua Kinard wrote: > > My contention is that I shouldn't need an initramfs loaded into my kernel to > get my system into a minimally-usable state. I've been running separate > /usr setups for 10+ years, and only now, such a setup breaks, hence my beef > with Fedora's assertion that such a setup is wrong. I was thinking about this and here is another way to think about it. Right now you can't boot a linux kernel without a whole bunch of c/asm code in linux. That code is necessary to do arch-specific setup, locate the root device, mount it, and run init. The new model is that you can't boot a linux kernel without a whole bunch of c/asm code in linux, and a bunch of scripts and userspace code in a blob (that can potentially be part of the kernel image). You could view this as a simple refactoring of code. Instead of all the boot logic being in c/asm which is hard to tweak, now some of it is written in bash and a bunch of userspace tools. All of this can just be viewed as part of the kernel - it can even be part of the same file if you want. Obviously this isn't a perfect analogy, as a bunch of userspace tools already existed but now require the extra glue code to work (mounting /usr). Once upon a time you didn't even need grub or lilo to boot - you could just stick the kernel at the start of your boot disk and the first 512 bytes of the kernel conveniently contained a boot sector. That code actually still exists but simply tells the user to bugger_off. So, you really could just view this as another step in the evolution of the linux boot process. After seeing some of the more exotic boot processes used in ARM/etc stuff like this just doesn't throw me for much of a loop. And, if you setup dracut/genkernel appropriately it really is just one extra step to make your system bootable. Rich
Re: [gentoo-dev] Re: Let's redesign the entire filesystem!
On Wed, Mar 14, 2012 at 1:29 PM, Zac Medico wrote: > On 03/14/2012 10:11 AM, Maxim Kammerer wrote: >> What's wrong with: >> * having an "early mounts" list file >> * having an "early modules" list file >> * init system in early boot (e.g., OpenRC in init.sh) loading "early >> modules" and mounting "early mounts" from /etc/fstab > > You're assuming that the /sbin/init hasn't migrated to /usr/sbin/init. > Other that that, it sounds like a perfect solution if you're in the "I'd > rather die than use an initramfs" camp. Well, anybody is welcome to create any replacement/addition for (/usr)/sbin/init or (/usr)/sbin/rc that they wish. If you make it good enough, perhaps others will even use it. Beyond that, anything else is just a suggestion. In the end the folks writing udev and systemd are writing code, which gets them a lot further than emails do... :) Rich
Re: [gentoo-dev] Re: Let's redesign the entire filesystem!
On Wed, Mar 14, 2012 at 7:51 PM, Richard Yao wrote: > > I proposed a way that this could work with no effort on the part of the > Gentoo developers in one of my earlier emails: > Then go ahead and make it happen. If as you say no dev participation is needed there is nothing Gentoo needs to do to support this. On Wed, Mar 14, 2012 at 6:49 PM, Greg KH wrote: > > We aren't Debian here people, we don't support "everything" :) > > If you want to support both, great, feel free to step up and do the > work. > Gentoo is about choice, but it is largely about the choices that people are willing to step up and maintain. A few months ago there was a big thread and lots of devs said that systemd isn't supported on Gentoo. Some devs stepped up and decided to maintain it and now I'd say systemd is about as supported on Gentoo as Prefix, FreeBSD, Sparc, or MIPS. That didn't happen because of mailing list persuasion - it happened because a few people interested in making it happen wrote a bunch of ebuilds. How do systemd units end up in various packages? The people interested in seeing them write good-quality patches and submit bugs, or otherwise work with the maintainers to commit them. For those who don't like the current direction, by all means create an overlay called udev-root, mdev-boot, noinitramfs, or whatever. You don't need anybody's permission to do it - just go on github and make it happen. Write some good code. There are several devs here who might even help you out with it, and nobody here is going to object to packages going into the main tree as long as they're maintained in accordance with Gentoo QA. Create some USE flags where you need tie-ins to other system packages and as long as everything behaves nicely and patches are good and maintained, I'm sure the package maintainers will accept them. Gentoo already gives its users a lot of choice, but it can only offer the choices that people are willing to maintain. Right now I see a lot of complaining and not a lot of maintaining. When I see a package lastrited I don't moan about it - I either sigh or sign up to maintain it. By all means make suggestions to improve the transition or write docs, but simply posting on this list isn't likely to change the direction the linux winds are blowing. The forces involved are much larger than Gentoo. Rich
Re: [gentoo-dev] Re: Let's redesign the entire filesystem!
On Thu, Mar 15, 2012 at 7:04 AM, Joshua Kinard wrote: > This does lead me to wonder if a light-weight udev could exist that lacks > half or more of the functionality of the current udev. I'll be honest, I've > only edited my udev rules file once, and that was only when I installed a > Sun Happy Meal quad ethernet card in which all four ports utilize the same > MAC address and udev doesn't handle this very gracefully (if I had Solaris, > I could edit the card's firmware and change this setting). You know - I had a similar issue, but with a pair of PL2303 USB RS232 interfaces. That makes me wonder if there is a possible way to enhance udev to better handle situations where devices have no unique ID and thus tend to be difficult to access consistently across reboots. In my case I had to hack a rule so that I got a symlink if the device was in a specific USB slot. Use case is controlling tuners for mythtv. No doubt a simpler 80% solution could be created for udev, and likely it would be easier to cut down on its dependencies as a result. However, the other 20% of users will still need the more complete solution. Big distros that want to support lots of hardware with a one-size-fits-all configuration will just deploy that complete solution everywhere, which means that the only people maintaining the simple solution would be people who like to tailor each system. For most of the more enterprise-y OS providers (ie the ones with money to pay devs), one-size-fits-all is a lot more sustainable. You won't find an edition of MS Windows that works only on PCs without scanners and sound but uses 50MB less RAM, for example. Sure, we don't have the same constraints as the enterprise-y distros, but we do have the constraint that if we want to do things differently we will spend a lot of time patching what we could otherwise simply reuse as-is. I don't think that split filesystem installs are going away anytime soon. In fact, when btrfs is finally mature we might see them blossum. Using subvolumes you could have more granular snapshotting and mount options, while still maintaining a shared disk space pool (with granular quotas). If everything the distro is likely to mangle is in a few subvolumes you can reverts snapshots on those without losing changes in other subvolumes if you ran in production before deciding to revert. That gets you a lot more flexibility than a single snapshot on root - especially in terms of recovery time (you can still copy files between snapshots if you only snapshotted root - in fact with reflinks this is very fast). Rich
Re: [gentoo-dev] Re: Let's redesign the entire filesystem!
On Thu, Mar 15, 2012 at 10:42 AM, Greg KH wrote: > > Why not use the links in /dev/serial/ which are there for this specific > reason? > # ls -l /dev/serial ls: cannot access /dev/serial: No such file or directory Something in a newer version of udev perhaps? Or would my defining my own symlinks end up overriding some rule elsewhere. I just added these lines to /etc/udev/rules.d: SUBSYSTEM=="tty", DRIVERS=="pl2303", KERNELS=="4-1:1.0", KERNEL=="ttyUSB*", SYMLINK="mythser/rca1" SUBSYSTEM=="tty", DRIVERS=="pl2303", KERNELS=="3-3:1.0", KERNEL=="ttyUSB*", SYMLINK="mythser/rca2" Rich
Re: [gentoo-dev] gentoo-news repository migrated to git
On Thu, Mar 15, 2012 at 2:28 PM, Ulrich Mueller wrote: > The gentoo-news repository has moved from subversion to git some time > ago. New news items should be committed to git only, because the > master rsync doesn't pull from the svn repository any more. > Is there a link to the repository anywhere? I don't see one on g.o, but I might not be looking in the right place. Might not hurt to include instructions on how to push to it as well... Rich
Re: [gentoo-dev] /dev/serial/ (was "Let's redesign the entire filesystem!")
On Thu, Mar 15, 2012 at 3:17 PM, Greg KH wrote: > On Thu, Mar 15, 2012 at 03:04:36PM -0400, Rich Freeman wrote: >> # ls -l /dev/serial >> ls: cannot access /dev/serial: No such file or directory > > Do you have your serial device plugged in? If not, it will not show up. > Yup - it is plugged in, and the links in /dev/mythser show up fine. Since I've been recording my TV shows on the correct channels I have to assume the devices are working fine too. > You do know that USB buses can be dynamically renumbered depending on > the phase of the moon, right? Be careful here... Hmm - this has been stable for me for years, compared to just using /dev/ttyUSBn. In any case, I have no idea why nothing shows up in /dev/serial. The only device nodes I can find for serial are /dev/ttyUSBn and /dev/mythser/n (the latter being from my own rules). Rich
Re: [gentoo-dev] Undocumented and unused USE variables
On Fri, Mar 16, 2012 at 8:49 AM, Christoph Niethammer wrote: > Here the euse command is realy handy. :-) > However the sysfs USE flag is still hiding its documentation. > So lets see if this is a bug or a feature. ;-) Yup, euse is helpful, or you can grep /usr/portage/profiles/use.* (including the local version). I suspect we'll find 3 categories here: 1. Stuff like amd64 which isn't really a USE flag proper, but which is required due to how arch support works. As far as I know amd64 is the only one in this category. 2. Stuff like pango which really are documented. 3. Stuff like sysfs which are neither. File a bug... :) Rich
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Sun, Mar 18, 2012 at 3:23 AM, Ralph Sennhauser wrote: > The ebuild extensions for GLEP 55 > would likely always be ebuild- as integers are reserved for > future use by Gentoo. Looking at GLEP 55 the proposal was ebuild- - not ebuild-. I don't believe there is any restriction that EAPIs be integers or incremental in nature. Don't mean to nitpick, but perhaps not all realize this... Rich
Re: [gentoo-dev] RFC: License problem
On Wed, Mar 21, 2012 at 10:34 AM, Richard Yao wrote: > On 03/21/12 10:18, Justin wrote: >> I will not extract part of the software, e.g. subroutines, for use in >> other contexts without permission of the author. > > Portage could be considered to be one of these contexts. Unless portage is just installing a small part of the application then I don't really see an issue here. If we keep it as a whole I don't see the issue. The first clause really suggests that we need RESTRICT=mirror. I don't see any clear need for RESTRICT=fetch. Obviously the license needs to go into the tree/etc. Rich
Re: [gentoo-dev] About suggesting to create a separate partition for portage tree in handbook
On Tue, Mar 27, 2012 at 2:34 PM, Alexandre Rostovtsev > The partitioning scheme is something that the user needs to decide on > *before* getting Gentoo up and running. After the user had finished > installing the operating system, it's too late to inform him about the > advantages of a separate /usr/portage. Yes and no (if you have free space, you could easily move /usr/portage - some other changes are harder). However, you could extend this line of argument to raid, lvm, and even stuff like the use of systemd or an alternative package manager. All of those things are much easier to implement if you just start out with them. I'm all for creating a wiki to talk about some alternative options. Perhaps even link to it at the start of the handbook in the intro (if you're not in a rush and want to read about more advanced configurations, check out ...). However, I tend to agree that the handbook should be a nearly-foolproof no-frills Gentoo installation. > 1. ext4, not ext3, needs to be recommended as the default filesystem. We > have kernel 3.2 marked stable, there is no need to keep talking about > ext4 as if it's something experimental. I tend to agree here. Not sure we need the full discussion of filesystems either. Ext4 is probably good enough for everybody, and mention ext3/2 as more established alternatives. I tend to feel the same way about stuff like LILO. Then again, Gentoo is about choice. It just seems like we're presenting users with more choices than makes sense for a newbie. If there is a choice between something that 99.99% of users will want, and some ancient piece of cruft that still works and is better for 0.01% of the userbase, does that really have to be in the handbook? Rich
Re: [gentoo-dev] About suggesting to create a separate partition for portage tree in handbook
On Wed, Mar 28, 2012 at 3:16 AM, Brian Dolbec wrote: > On Tue, 2012-03-27 at 19:16 +0100, Ciaran McCreesh wrote: >> But that's ok, because extensive studies have shown that the only possible >> reasons for putting /usr/portage on its own partition are historical, >> since everyone has an SSD now. >> > > Yeah, right. Since I must be the only one out there that doesn't yet > have an SSD, you'll give me (and anyone else that still doesn't) one? Woosh...
Re: [gentoo-dev] About suggesting to create a separate partition for portage tree in handbook
On Wed, Mar 28, 2012 at 10:59 AM, Richard Yao wrote: > On 03/28/12 03:16, Brian Dolbec wrote: >> On Tue, 2012-03-27 at 19:16 +0100, Ciaran McCreesh wrote: >>> But that's ok, because extensive studies have shown that the only possible >>> reasons for putting /usr/portage on its own partition are historical, >>> since everyone has an SSD now. >> >> Yeah, right. Since I must be the only one out there that doesn't yet >> have an SSD, you'll give me (and anyone else that still doesn't) one? > > In response to the people who don't like what Brian had to say, I would > like to say that we can't start making assumptions about what hardware > people have and ignore anyone who does not fit those assumptions. Nobody doesn't like what Brian had to say. Most everybody around here including Ciaran likely agrees with him. The issue is that Ciaran said the complete opposite of what he was trying to communicate (sarcasm), and that likely due to language/culture/etc that might not have been clear to somebody who isn't a native English speaker in a western culture. The allusion was clearly to the larger udev/systemd/usr issues and the point he was making is that many of these boil down to disagreements about what use cases you consider important. So, just take everything Ciaran said in that particular post, assume he meant the exact opposite, and now you'll see where he is coming from. Yes, I do agree that sarcasm tends to cause problems on international email lists, but his post did at least make me smile. :) Rich
Re: [gentoo-dev] rfc: location of portage tree
On Wed, Mar 28, 2012 at 2:53 PM, Christoph Mende wrote: > > I believe it's /var/lib/. Here's what FHS says: > /var/cache is intended for cached data from applications. Such data is > locally generated as a result of time-consuming I/O or calculation. > The application must be able to regenerate or restore the data. Unlike > /var/spool, the cached files can be deleted without data loss. > I can do rm -rf /usr/portage ; mkdir /usr/portage ; emerge --sync and it will work just fine, I think. That really does point to cache. The only thing different from a browser cache is that portage doesn't automatically refresh it. distfiles and packages are the same (well, depending on where you get your binpackages from, that might or might not be a cache per-se - if you're just using FEATURES=buildpkg then you can do an emerge -e world and get it back). > And: > /var/lib/ is the location that must be used for all distribution > packaging support. > I think that things like the local list of installed packages belongs in this category. It is a bit debatable how the tree fits into this. However, this really is bikeshedding. Sure, /usr isn't ideal, but unless we actually start supporting some use case where it doesn't work so well in the future, I doubt we'll ever see it move. Plus, there is even a case for keeping it in /usr in the Fedora-envisioned /usr-is-ro world. You could have a complete installation and a portage tree that it was generated from all snapshotted there. Sure, maybe /usr/lib or /usr/share might make more sense then, but again, I don't see it changing unless it actually results in a real benefit to users. Rich
Re: [gentoo-dev] Relicensing sys-freebsd/* under the BSD-2 license
On Fri, Mar 30, 2012 at 2:47 PM, Ulrich Mueller wrote: > Ebuilds in the Portage tree must be licensed under the GPL. This is > part of the Gentoo Social Contract [1], so I guess it won't change > anytime soon. > > And IMHO, we would be ill-advised to allow different licenses for > ebuilds in the tree, because that would imply that we cannot copy code > from one ebuild to another (or from ebuild to eclass) any more. > Speaking as an individual trustee, I tend to agree. If there are specific pains associated with not being able to submit patches upstream or such, please do point them out and I'm sure we'll consider what can be done to accommodate this. However, if this really is a one-off situation then we're probably better-off if we just ask individual contributors to re-license when needed. I'd think the only thing in the portage tree upstream would be interested in would be patches (including sed operations). Rich
Re: [gentoo-dev] Happy 10th birthday (in advance)
On Sat, Mar 31, 2012 at 7:44 AM, Samuli Suominen wrote: > (or hand me powers to remove people from ML :-) > That sounds like a great idea. We could create a code of conduct, and then designate individuals to enforce it. Maybe we should call them proctors: http://www.gentoo.org/proj/en/council/coc.xml What could go wrong? Seriously though, this debate like many others recently probably shouldn't be viewed as for-Gentoo and against-Gentoo. Lots of people care about Gentoo, we just don't always agree on what is best. In this case the issue is pragmatism vs idealism, and both have their place. What is important is that we go ahead and share our views, debate points within reason, don't obsess over getting in the last word, and then work together to support the decisions that get made. My two cents in this debate is that I'm willing to accept Ciaran's suggestion that Portage 2.2's approach has its limitations, but it is the best thing we have implemented now, and thus I'll take the 98% solution over the 20% solution (which is what we get if all we do is argue over how to get to 100%(. If somebody wants to write the code to get us from 98->100%, I'm sure we'll all be for it. Rich
Re: [gentoo-dev] Re: [gentoo-dev-announce] Council meeting summary for 3 April 2012
On Sun, Apr 8, 2012 at 6:04 PM, Greg KH wrote: > On Sun, Apr 08, 2012 at 04:30:01PM +0200, Ulrich Mueller wrote: >> The council has voted in favour of a separate /usr being supported >> (5 yes, 1 no vote). > > What? Perhaps the council should be the ones to clarify, but I think the vote only was for separate /usr being supported. The irc log seemed a bit more nuanced than perhaps came out in the summary. Maybe I'm misreading it. I didn't see anything in the log about a decision that newer versions of udev are not able to be stabled. So, as to what "a separate /usr being supported" means, the impression I got was "don't worry if you're running it, you'll have an upgrade path." Right now it sounds like the proposed upgrade path is that some devs will fork udev and keep it running more like the current one (presumably breaking in the same situations that it already does today). > And udev isn't even the problem, all you need is to mount your /usr from > initramfs. So, the original proposal wasn't even a correct/valid > proposal in the first place. Well, as far as I can tell the proposal that was voted on didn't even mention udev at all, or initramfs. But, as you point out using an initramfs is likely to be more reliable. I'm sure the same arguments were going around back when people were advocating for dropping bootloader support in the kernel and telling people to bugger_off_msg. An initramfs creates more flexibility, at the cost of an extra layer of software, just like grub. The main downside to it is that it tends to require more maintenance, though if you build the necessary drivers to mount /usr into the kernel I imagine that an initramfs would probably work across differing kernel versions. In any case, we should still be updating documentation/etc regardless. A better guide to dracut/genkernel would be useful no matter how this turns out. I'd like to see stable Gentoo stay current with udev in any case, but I don't mind using a forked version as long as it is of similar quality to the original. As you've pointed out already, that may not actually help people with a separate /usr, so I'd encourage people to get an initramfs working. Rich
Re: [gentoo-dev] Are we allowed to provide valid SRC_URI for fetch restricted packages?
On Tue, Apr 10, 2012 at 3:38 PM, Mike Frysinger wrote: > > i don't see a problem here. although this is something that you'd prob want > to post to the trustees and let them make a decision. > -mike > This is a bit of a stale thread. This particular package was modified to remove the SRC_URI, so it is no longer outstanding. The Trustees have discussed, and i'd say a majority would prefer not to have SRC_URI for fetch-restricted files. However, I don't believe we put it to a vote. Personally, I tend to not see an issue here, but am open-minded to specific precedents/etc that raise concerns. I think others are a bit more conservative. Rich
Re: [gentoo-dev] Re: Re: Re: [gentoo-dev-announce] Council meeting summary for 3 April 2012
On Tue, Apr 10, 2012 at 10:28 PM, Steven J Long
wrote:
> As for the burden of ensuring that binaries installed to /{s,}bin don't link
> to libs in /usr, why not just automate a QA check for that, and let
> developers decide whether a fix is necessary? After all, core packages that
> do that even when configured with prefix and execprefix = /, aren't so
> portable, and Gentoo has always championed "doing the right thing" wrt
> helping upstream fix portability issues.
The only issue with that logic is that upstream is perfectly aware of
what they're doing already, and bugs are likely to be closed as
WONTFIX. So, all the QA checks would do is ensure that we slowly
start maintaining forks of more and more packages. Right now the
problem is probably manageable, but I'm not convinced it will stay
that way. Once upstream developers consider all constraints to be
removed on their dependencies you could see a lot of stuff getting
pulled into root if you tried to enforce this rule.
In any case, it sounds like the directive is to keep limping along for
a while longer, and that makes sense anyway until docs/etc are
improved. I agree with Ralph's suggestion that the newer initramfs
tools should be stabilized as well.
Rich
Re: [gentoo-dev] Re: Re: Re: Re: [gentoo-dev-announce] Council meeting summary for 3 April 2012
On Wed, Apr 11, 2012 at 11:09 AM, Steven J Long wrote: > That might be true for some Linux-only packages, but I really find it hard > to believe that any upstream targetting more than one OS (just adding a BSD > is enough) with software that could be considered critical (I for one would > include all POSIX utilities as well as basic stuff like mount, fsck and > lvm2) would want to ignore this kind of thing; if the build-system is > ignoring configuration, it's a bug. The issue is that if udev requires libfoo, then libfoo must not be in /usr. If libfoo is libx11 or dbus or some other complex library, that will pull in a bunch of other stuff as well. However, I doubt that the maintainers of all those libraries would consider them boot-essential, so they may not be inclined to make the move. Obviously we're not there now, but it is a possible consequence of moving in this direction. Keep in mind that systemd in particular does not aim to be cross-platform - they advertise this as one of their core features. Since tight integration is their goal that could slowly bring in a lot of other stuff. The main platform pushing it along is Fedora, and they're aiming to move everything into /usr, so this is a non-issue for them. > I read the decision from the Council to be "we will continue to support the > traditional split /usr eg with lvm, and no initramfs" and a Council member > put himself forward to maintain patches to udev to ensure that going > forward, since it is needed in his employment. > > Given that we can do it with initscripts, and don't need to fork udev at > all, what's the problem? I can't really comment on what the decision from the Council actually was. However, maintaining patches to udev is effectively the same thing as forking it. Right now it probably isn't hard, and over time that could change. The only time patches != fork is if the patches have been submitted upstream and are likely to be merged. In any case, it isn't a crisis now and waiting a little to see which way the wind ends up blowing probably makes sense. Rich
Re: [gentoo-dev] >= udev-182 tracker
On Wed, Apr 11, 2012 at 3:53 PM, William Hubbs wrote: > To allay any fears of that happening, I have opened a tracker [1] for > issues which need to be resolved before newer udevs can go stable. > Probably wouldn't hurt for the initramfs maintainers to open a stablization tracking bug for whatever version they think will best address the /usr mounting issue and make them blockers. Rich
Re: [gentoo-dev] Re: Re: Re: Re: Re: [gentoo-dev-announce] Council meeting summary for 3 April 2012
On Sat, Apr 21, 2012 at 10:43 PM, Steven J Long wrote: > The Council has voted that Gentoo continue to support that subset, without > an initramfs. Citation, please? Rich
Re: [gentoo-dev] Proposal to move use.local.desc somewhere in /var
On Tue, Apr 24, 2012 at 1:32 PM, Mike Frysinger wrote: > it's also a bit of a speed issue. i often want to look at what flags get used > across the tree. what's faster: loading + parsing 15000 xml files, or loading > 1 file ? shifting it to metadata/ as a cache of all the xml files is probably > fine, but i'm not sure dropping it completely is an improvement. ++ Any time I add a new use flag/etc the first thing I do is look for how other packages accomplish the same thing. No sense writing scripts when we already have one that does the job. Rich
Re: [gentoo-dev] Re: New license: yEd Software License Agreement
On Fri, Apr 27, 2012 at 9:45 AM, Amadeusz Żołnowski wrote: > Excerpts from Duncan's message of 2012-04-27 15:38:20 +0200: >> No distribution allowed. You're going to be doing restrict=mirror, >> correct? > > Why RESTRICT=mirror? I'd put RESTRICT=fetch, actually. > I don't see much point in using RESTRICT=fetch unless there is no URL available to fetch. I think RESTRICT=mirror should be sufficient in most cases to cover situations where we are not allowed to redistribute, and is consistent with what we do across the tree. Consistency is very important where there are legal concerns. Otherwise our actions with one ebuild will be used to argue that our actions on another ebuild are wrong. Rich
Re: [gentoo-dev] Feature request: package.use.stable.mask and package.use.stable.force
On Fri, Apr 27, 2012 at 7:35 AM, Chí-Thanh Christopher Nguyễn wrote: > I agree that the ~arch ebuilds should be tested in the same > configuration in which they will end up in arch. However in this case, > the possible configurations for arch are a subset of those in ~arch, so > the testing covers those too. Just because a stable system brings in fewer use flags doesn't necessarily mean that it is less likely to break. Use flags can have all kinds of effects, some applied when they are present, and others applied when they are absent. The value of testing comes from testing things in the anticipated future production environment. Of course, the fact that we stabilize individual packages and not all their libraries/etc at the same time already weakens this. Rich
Re: [gentoo-dev] Re: New license: yEd Software License Agreement
On Fri, Apr 27, 2012 at 10:54 AM, Duncan <1i5t5.dun...@cox.net> wrote: > Tho FWIW I think restrict=fetch applies to stuff like cd/dvd-based game > data as well, where the agreement is on the cd not a website, but still > requires specific click-thru. I think the real determiner should almost always be whether there is a URL that you can reliably fetch or not. If you're talking about a CD there is nothing to fetch. If you want somebody to copy a file off of a CD and stick it in distfiles then RESTRICT=fetch makes perfect sense, but it has nothing to do with licensing, but the fact that no consistent URL exists. Legally the basic issue is whether linking to something is the same as distributing it. I'm open to a lawyer's summary, but my sense is that the case law around this is very messy, especially on an international scale. Rich
Re: [gentoo-dev] Re: New license: yEd Software License Agreement
On Fri, Apr 27, 2012 at 11:33 AM, Amadeusz Żołnowski wrote: > > And this is probably the case when user has to accept a license on the > website. This is URL for zip archive of yEd-3.9.1: > > http://www.yworks.com/en/products_download.php?file=yEd-3.9.1.zip > > It directs to website with license text, check-box for accept and > download button. If check-box is not set, following message is shown: > > "In order to download yEd, it is necessary that you first accept the > license terms." > > If check-box is set, client is redirected to the page with actual link to > zip archive. It turns out the vendor is lying - you can download it fine without accepting the license from: http://www.yworks.com/products/yed/demo/yEd-3.9.1.zip No doubt the vendor WANTS users to accept the license first, but it is not "necessary" from a technical standpoint. > > Moreover, I have had email conversation with yWorks representative and > he says that installation files need to be obtained manually by the end > users from their website. > Again, they likely intend for them to be obtained in this manner, but the word "need" is not true from a technical perspective. This brings up a debate that was recently held over deep-linking in bugzilla over a math library. The trustees never took a final vote since the maintainer decided to just implement RESTRICT=fetch. The issue there was about more than just copyright, however, and the trade regulations around munitions do not apply in this case. I don't think we have clear policy around this situation. I see our options as: 1. Set RESTRICT=fetch because upstream wants us to and we like to cooperate. 2. Set RESTRICT=fetch because even if legally they're on shaky ground upstream could probably waste a lot of our time and money. 3. Set RESTRICT=mirror because legally we think we have the right to do so, and want to stand for our principles and make life easier for our users. The potential upstream responses to doing #3 might be to do nothing, to not like us, to sue us, or to not use a fixed URL to distribute the file so that we have to restrict fetching for technical reasons. Do we as a matter of policy want to respect broken click-through download implementations? Rich
Re: [gentoo-dev] Re: New license: yEd Software License Agreement
On Fri, Apr 27, 2012 at 2:04 PM, Nikos Chantziaras wrote: > Didn't the user already accept the license by putting it in ACCEPT_LICENSE? > If not, portage will not download it. > Well, I'd argue that it is impossible to "accept a license" in the first place. It is possible to agree to a contract if there is consideration on both sides and a meeting of the minds. Copyright says you can't copy something. A license says you might be able to. You don't have to "accept" a license to benefit it. A license does not restrict what a user can do, it restricts what the person issuing the license can do (I can't sue you for redistributing my code if I licensed it to you under the GPL). Some licenses are conditional - I only limit my own ability to sue you if you give people a copy of the source for any binary you give them, and if you don't do that I am now free to sue you. Ultimately the foundation for licenses is copyright law, and other forms of IP law. Copyright says we can't distribute anything we don't create except under very specific circumstances. A license says that we can distribute stuff without fear of lawsuit under some conditions. The yEd license says we can't distribute anything, so as far as I can see, we might as well not have any license at all. We're not protected at all from a lawsuit, except to the degree that we don't do anything that we can be sued for (like distributing their software). But yes, from a technical standpoint you can only install a package if its license is contained in ACCEPT_LICENSE. Whether this has any legal meaning is up to you or a court with jurisdiction to decide. Rich
Re: [gentoo-dev] Re: New license: yEd Software License Agreement
On Sat, Apr 28, 2012 at 2:53 PM, Alec Warner wrote: > That doesn't mean you didn't / cannot accept, merely that some (all?) > provisions are likely unenforceable in a court of law. I don't think > EULAs have been ruled illegal yet. I doubt that my proclamation that you aren't allowed to eat breakfast has been ruled illegal yet either. Fortunately that has no bearing on whether you need to listen to me. :) Oh, and I also proclaim that you accept my proclamation by choosing to eat your next meal. Fortunately in reality that has no bearing on whether you accept my agreement either. Just because a publisher says that the terms of a contract of adhesion are binding on you by virtue of your taking some action does not make it so. Courts have ruled inconsistently on whether EULAs can be enforced. Then again, Missouri is one of those places where courts have ruled that software is not sold but licensed, and the Foundation is incorporated there (as well as in New Mexico). So, perhaps there is some element of risk here, though I'd have to read the court decisions to see whether the fact that the software is free impacts the enforcability of the EULA. That makes me wonder whether we should consider more carefully where we incorporate - if it makes us more subject to local jurisdiction it probably isn't a good idea to incorporate in multiple jurisdictions since it allows a potential plaintiff to venue shop. Rich
Re: [gentoo-dev] busybox[sep-usr] support for mounting /usr w/out hassle
On Mon, Apr 30, 2012 at 4:05 AM, Tony "Chainsaw" Vroon wrote: > Binaries that are essential for system boot, and must be available in > single user mode go in /bin and /sbin, with their libraries in /lib. > This allows for /usr to be: > 1) marked read-only for NFS mounts, which some of us rely on > 2) inside of an LVM2 container, allowing for / to be (very) small > 3) on a squashfs filesystem, in order to save space These are all things easily supported with an initramfs. In fact, initramfs-based solutions allow the same sorts of things to be done with all the other filesystems and not just /usr. > Trying to second-guess my motivation, and trying to undo unanimous > council votes simply because your opinion is different, really has to > stop. I don't think anybody is trying to undo council votes - people are just speculating as to what they voted on. The easiest solution is for somebody to say "I'm John Smith, and I am speaking officially for the council, and we agree that what was decided upon is X." It seems pretty clear that everybody wants to support a separate /usr. We even have multiple supported solutions, including an initramfs, a use flag on busybox, and I believe somebody posted a script that can be run during early boot to mount /usr. It sounds like the only thing that isn't supported is "doing nothing" - but with Gentoo if you "do nothing" you don't get an installed system that works on any configuration. > > I feel a lot better about vapier's pragmatic approach then I do about > udev/systemd upstream's ability and motivation to support current > systems. If you had any doubts about whether udev was part of the > problem, consider what tarball you will have to extract it from in future. Well, if others feel differently about the direction udev is taking, they can of course just fork it. I can't say I'm terribly excited about the amount of vertical integration going on. I don't run Gnome, and I don't run Unity. I really do prefer the unix way. However, I don't contribute much to those upstream projects, and I don't see much value in telling a bunch of people who do that they are doing it wrong. I don't like how Google develops Android in the dark, or that they bundle 1GB of third-party stuff in their Chromium source and distribute a favored binary-only derivative. However, I do like that they're giving me all of that stuff essentially for free, and so beyond the odd blog post I try not to give them too hard a time. In the same way I think we need to give the maintainers of these projects in Gentoo some slack, or join those projects and help them to address your needs. It is a lot easier to tell others what to do than to help make it happen, but a volunteer-based project like Gentoo needs the latter more than the former. Rich
Re: [gentoo-dev] Chromium bundled code
On Mon, Apr 30, 2012 at 12:11 PM, Mike Frysinger wrote: > On Monday 30 April 2012 12:00:59 Rich Freeman wrote: >> doing it wrong. I don't like how Google develops Android in the dark, >> or that they bundle 1GB of third-party stuff in their Chromium source >> and distribute a favored binary-only derivative. > > err, they distribute a Chromium source tarball, and their build system > includes flags to use the system versions of those bundled libs if you so > choose. i think this is a perfectly fine compromise. Must be a fairly new compromise - but glad to hear that they've come along a bit. Maybe their future Google Earth install package for Linux won't bundle wine. :) In any case, I think this really just gets to my point - people who develop FOSS aren't doing it with the express goal of making your life difficult - they just don't always have the same itches. If you help them out or ask NICELY sometimes they'll even help scratch yours. Rich
Re: [gentoo-dev] Chromium bundled code
On Thu, May 3, 2012 at 5:34 AM, "Paweł Hajdan, Jr." wrote: > > Right, and the source tarball is ~175 MB. It's not perfect, but it's > also far from 1 GB. The bundled libraries are included in the tarball. I was thinking about the unpacked size - which is 1001M according to du for chromium-18.0.1025.168.tar.bz2. That's a lot of source code... However, my whole point wasn't to throw stones at the chromium team - I think that they've been doing a great job of fixing this problem, and will continue to do so. I just was pointing out that Google's practice of bundling dependencies wasn't to my liking, but that I wasn't going to give them too much of a hard time precisely because I'm not being part of the solution. Rich
Re: [gentoo-dev] Chromium bundled code
On Fri, May 4, 2012 at 9:50 PM, Chí-Thanh Christopher Nguyễn wrote: > I'd say that Android is an operating system based on Linux. It is not > 'the Linux "stack"'. > I think he was wondering whether the lock-in between dbus, udev and the > Linux kernel will reach proportions where they will be distributed in > one source tree. Well, I think that all of this is at the heart of the debate. I don't see whether the source trees are merged as being terribly important - the issue is whether the level of integration makes them difficult to peel apart. This is just the vertical integration debate all over again. If you want to use gnome, thou shalt use systemd. If you want to use Chromium, thou shalt use udev, and so on. I can't wait until we hit the point where if you want to use your favorite browser you have to install the maintainer's favorite DE, sysvinit, etc. The only solution I see to getting back to the "unix way" is to agree upon standard APIs, but I think that is many years off. Most of these technologies are very new, and rapidly evolving. Google themselves have shown with platforms like Android that compatibility is not of terrible importance to them beyond the application API (how many times have they broken the camera driver binary interface?), and of course Linux is famous for breaking driver-level binary interfaces, though this is mitigated by the fact that outside of Android almost everybody makes their drivers open-source, or they actually support their drivers for more than six months. The next year or two will be an interesting experiment for the Linux community. Will the "unix way" prevail, or does the future lie in highly integrated tools, that generally do not interoperate, but which provide much greater functionality? Rich
Re: [gentoo-dev] RFC: remove ldap from desktop profiles use flags
On Sat, May 5, 2012 at 3:55 PM, Dale wrote: > Jesus Rivero (Neurogeek) wrote: >> I don't like this change much. There are valid use cases for an ldap use >> flag in the desktop profile that could break easily with this change. >> There are valid use cases for every USE flag in portage, otherwise they wouldn't be there. The fact that SOME people find ldap useful on a desktop does not mean that it should be the default. > > Not to mention, you add the possibility that the user may miss the > change since they are not expecting it. I would expect it when I was > changing profiles but not so much just coming out of the blue. It has already been suggested that this be prefaced by a news item, which would be completely sensible for just about any profile USE change. Rich
Re: [gentoo-dev] RFC: remove ldap from desktop profiles use flags
On Sun, May 6, 2012 at 5:37 AM, Michał Górny wrote: > > I don't think even heavyweight DE/WM usually needs ldap... > Tend to agree. I don't think we want to create a new profile every time we want to change one of the flags. Some other questionable ones: emboss - Adds support for the European Molecular Biology Open Software Suite firefox - probably OK for what it does now, but not everybody uses it xulrunner - not even used now There will always be some level of variation if you are looking at single flags. What matters isn't coming up with profiles that exactly match all of our users, but rather ones that are good for 80+% of them. As far as ldap goes, if we wanted an "enterprise desktop" profile that might be a good fit for such a configuration. I agree that -ldap isn't really a lightweight desktop so much as a normal one. If you really wanted "lightweight" then you'd probably not be running desktop at all, or the regular desktop vs kde/gnome. The bottom line is that we don't need 47 different profile targets - there will always be a "use" for 1 more. That's why we all run Gentoo - we aren't bound by the decisions made for us by the package maintainers. Rich
Re: [gentoo-dev] RFC: remove ldap from desktop profiles use flags
On Mon, May 7, 2012 at 7:17 PM, Walter Dnes wrote: > There's a "server" profile which could be the answer. I've never seen that as being a terribly useful profile. Servers tend to be very minimal configurations. Maybe if we ever ripped sshd out of the default profile we might put it there, but beyond that what would you run on EVERY server? If I were to build a server I'd just stick with the default profile, and then add to it. Rich
Re: [gentoo-dev] Tightly-coupled core distro [was: Council meeting summary for 3 April 2012]
On Wed, May 9, 2012 at 9:08 PM, Patrick Lauer wrote: > On 05/10/12 06:36, Greg KH wrote: >> On Wed, May 09, 2012 at 08:51:37PM +0200, Fabio Erculiani wrote: >>> Please stop throwing lennartware at people. FailAudio has been enough, >>> thanks. >> The use of these terms is both rude and totally uncalled for. You >> should be ashamed of yourself. > It's reactive. I've been called stupid, conservative, behind the times, > user of obsolete software that will go the way of the dinosaurs. Why > should we be ashamed of not agreeing with these funny pranksters? Look, I have pretty mixed feelings about all the vertical integration. However, let's at least do each other the professional courtesy of not resorting to name-calling. We're allowed to disagree, and that's OK. By all means voice your opinion. However, let's talk about the issues, and not the people advocating them. This is just polite behavior. It is also the rules for posting on this list, especially if you hold a g.o address. > So why would I give up my freedom to tinker just because someone else is > writing more code than I do? I understand your frustration. Really, I do - I often find myself sharing it. However, in the end people working on FOSS are basically free to do what they want, and everybody is free to use or support what they want. I don't like the fact that most people contributing to Android tend/aspire to be associated with the commercial market for smartphones, and as a result they tend to embrace pro-developer / anti-consumer solutions (like not allowing easy blocking of ads, or randomizing calls to read the IMEI, etc). However, the market is what it is. The only thing that is really any different today is that companies are at least releasing the source for the stuff they do - in the past they'd have just closed it all off so that there wouldn't even be the option of forking. If I want to I can at least find the API call to read my IMEI and tamper with it. I think part of the community frustration is the increasing level of commercial support around Linux. That has given us much more robust stuff to play with, but it also has resulted in a loss of control and change in general atmosphere. In the world of 1999 Linux market share took a back seat to hackability. In the world of the Canonicals, market share matters a great deal, and appealing to open source contributors matters a lot less. Rich
Re: [gentoo-dev] pushing fixes to stable before closing bugs
On Sat, May 12, 2012 at 6:34 AM, "Paweł Hajdan, Jr." wrote: > The idea is that if you only fix in ~arch, you risk a serious and > _known_ regression in stable, which could be easily avoided. How can you have a regression in stable if stable has the bug already? A regression is when you have a stable package with a bug, then you introduce a new stable package that doesn't have that bug, and then you introduce yet another new stable package that has the same bug back again. While I can see some value in tracking whether bugs have made it to stable yet, I think we need better tools if we want to do that. Otherwise it is a big housekeeping mess tracking what is and isn't in stable yet. I see the value when we're talking about security bugs, or very critical bugs, but for the run-of-the-mill minor issues that are the majority of bug reports I don't see the value in keeping bugs open for a month or two just to track that the inevitable hasn't happened yet. Rich
Re: [gentoo-dev] pushing fixes to stable before closing bugs
On Sat, May 12, 2012 at 4:01 PM, "Paweł Hajdan, Jr." wrote: > Agreed. I'm talking about compile errors or crashes here. I've really > seen arches stabilizing packages with known problems, just having closed > bugs because "the fix is in ~arch". We might already be on the same page, but I think the bar to set for stable is that the new build must be overall at least as good as the previous one. Having known issues isn't necessarily a problem as long as overall the level of quality is maintained (all software has bugs - some teams are just better about knowing about them). If foo-3 is stable, and foo-4.1 introduces a bug, and foo-4.2 fixes that bug, then I agree that stablizing foo-4.1 might be a mistake. On the other hand, if foo-3 has 30 bugs, and foo-4.1 has 20 bugs, and foo-4.2 has 25 bugs (just not that one in particular), then it might still make sense to go with 4.1 in the short term. Obviously not all bugs are equal. This is why maintainers in general should be controlling what packages get STABLEREQ'ed, and for important packages it is best to make this decision as a team. Is there really a sense that this is a big problem? I'm sure it happens - but AFAICT the effort required to prevent this might not be worth it except in isolated cases. Rich
Re: [gentoo-dev] Re: RFC: Enable FEATURES=config-protect-if-modified by default?
On Wed, May 16, 2012 at 5:02 AM, Fabio Erculiani wrote: > I implemented this feature in Entropy long time ago (2009 iirc) and > enabled it by default as well. > We never had a single issue. Users seem quite happy about it. > This is also the default behavior with the cfg-update alternative to dispatch-conf - unmodified files are just replaced. Never had a problem with it. 90% of the time these are files I've never even looked at. Will the new option remove files automatically as well? I've been getting warnings on boot that I suspect are the result of ancient udev rules files that never got removed due to protection. I've been meaning to investigate, but I need to figure out which if any I did actually modify. Rich
Re: [gentoo-dev] Re: Remove eclass/ChangeLog
On Wed, May 23, 2012 at 4:02 AM, Samuli Suominen wrote: > On 05/23/2012 05:36 AM, Ryan Hill wrote: >> One person doesn't do entries. OMG let's remove it! >> > > absolutely because inconsitency renderess the file useless > Well, for now the solution is to enforce following policy. For the future, perhaps the policy's time has ended. Sure, Changelogs can be handy, but they are increasingly redundant and scm comments have the potential to be far more useful. By all means require meaningful scm commit comments, but if Changelog files are holding us back either auto-generate them or ditch them. And, to add my own to cents to this chain about the only thing worse than failing to directly inherit an eclass dependency is to break the tree fixing the problem on personal initiative. When a dev messes up the solution is to talk to them or if necessary talk to devrel/QA - not to break end-user systems. Rich
Re: [gentoo-dev] Portage Git migration - clean cut or git-cvsserver
On Wed, May 23, 2012 at 10:43 AM, Anthony G. Basile wrote: > > Looks like the bloodbath begins. I too am in favor of killing cvs. Please, let it die. I'll miss my scripts, but I'll gladly deal with that over whatever breakage comes along every time some cvs plugin messes up the tree, or we can't use some useful git feature because cvs amazingly enough behaves like an scm invented 20 years ago. Rich
Re: [gentoo-dev] Portage Git migration - clean cut or git-cvsserver
On Wed, May 23, 2012 at 1:22 PM, Alexey Shvetsov wrote: > > That isnt true =) you can commit from shallow clone if and only if original > repo doesnt have a branching and merging points before and after shallow > clone point respectively > Is that going to be a practical condition to maintain? And how big will the repository actually be? Are we talking a GB or two, or something orders of magnitude larger? Rich
Re: [gentoo-dev] Portage Git migration - clean cut or git-cvsserver
On Wed, May 23, 2012 at 4:00 PM, Ezequiel Garcia wrote: > On Wed, May 23, 2012 at 4:37 PM, Arun Raghavan > wrote: >> I, for one, think we should stay with CVS and leave all this git >> Linusware to the new-fangled Fedora kids with their fancy init systems >> and tight coupling. CVS was good enough for my grandfather, and it's >> good enough for you. > > Perhaps it would be instructive if you could tell us one advantage of > cvs over git. > Sure. The slow commit rate encourages careful deliberation before hitting the enter key, which therefore improves quality. Then, if you do make a mistake the slow commit rate means that fixing that mistake can take a long time, which increases the amount of pain our end-users run into due to the mistake, which leads to lots of flame wars on -dev. That means that the guy who made the mistake is subjected to more public ridicule, and is less likely to do it again, That improves quality too. Since cvs doesn't tie together tree-wide changes in a nice way or allow them to be transactionally completed, individual package maintainers don't need to be as concerned with the big picture view. Now as the maintainer of libfoo the fact that somebody changed my ebuild without making a corresponding change in some profile is completely hidden from me, and I can go to sleep peacefully without realizing that my users are all going to have horribly broken systems in the morning. Blissful ignorance of end-user suffering improves developer morale, and helps get rid of pesky users at the same time. cvs also makes more more aware of what is going on around me. Anytime I want to work on something in parallel with the main development branch I get to manually merge changes in, which keeps me aware of my place in the world. That means that I'm less likely to build nice new features, which means fewer bugs, which improves quality, and may even drive away users as an added bonus! See, cvs is really the wave of the future! Rich
Re: [gentoo-dev] Portage Git migration - clean cut or git-cvsserver
On Thu, May 24, 2012 at 11:02 AM, Ralph Sennhauser wrote: > Can we keep the master on Gentoo hardware please. > > Also, there still should be a bug at b.g.o and git format-patch works > just fine for that. Maybe it's only github now but how many places is a > developer supposed to monitor? I'm actually a little torn on this one. I'm fine with keeping the "master" on Gentoo in the sense that this is where the rsync tree gets generated. However, gitbub has a lot of tools like pull requests that could potentially improve workflow, especially for things like proxy maintainers. So, letting those teams work more outside of Gentoo and just push their changes into Gentoo might make sense. Perhaps github should be viewed as a widely-shared overlay that gets automatic updates from the main tree in the master branch (or whatever we call it). You can work on a branch in github, get it where you want it to be, and then push it to Gentoo pretty easily. When I don't have access to an upstream repository I often just push a copy to a fork on Github just to make my own life easier. Rich
Re: [gentoo-dev] Portage Git migration - Handling Pull Requests (Was: clean cut or git-cvsserver)
On Thu, May 24, 2012 at 6:01 PM, Dan Douglas wrote: > But ok it's a good point. Github isn't a good central point of contact. People > have to use their discression. It's just uncommon these days for a project as > big as Gentoo to have ultra-centralized corporate-style procedures where > everything happens exclusively though official channels. And anyway you > couldn't > "enforce" that sort of thing if you tried. > ++ There is no policy that every commit in cvs needs to be referenced to a bug, and I doubt we're going to change that. So, if I call up another dev on the phone and tell them they have a typo in an ebuild, and they fix it and commit it, nothing has gone wrong. That isn't the most efficient way to work usually, but there is no policy against it. In general users should file bugs and not contact devs directly, but if somebody has a private arrangement otherwise, no harm no foul. Github is just another overlay. If in the course of working on the next kde release the kde team makes 385 changes to their overlay, we don't make them log and close 385 bugs on b.g.o before they merge in the files from the overlay. So, if a team wants to use non-official tools, by all means go ahead. The QA standards apply to anything hitting the main tree, and must be followed at that point in time. We should keep our tools working well, but if in some case there is a better way of working, or a small team has some other preference, well, have at it! Rich
Re: [gentoo-dev] Re: review required by herd? (new ebuild)
On Fri, May 25, 2012 at 11:23 PM, Ryan Hill wrote: > Some projects are territorial. Games is one. I imagine adding something > to kde, gnome, or xfce categories without contacting those guys would get you > an email. If in doubt I usually file a bug when I'm adding a package and CC > them. If they don't respond, I add it. Perhaps, but in general devs should be fairly free to add new packages to the tree as long as they properly maintain them. If a dev makes a mistake with a newly-added package, then file a bug and they can fix it. If there is a dispute over whether something is a bug then we have QA/Devrel/council/etc. That said, things work best when everybody cooperates, so if the maintainer and a project can get along that is fine. If not, then that's what Devrel/Council is for. In that bug it was suggested that the Council doesn't get involved with projects. I'm not quite sure in that case why we'd even bother to have a Council. I view the Council like the Supreme Court - the lower courts (projects) can do things however seems best, but if things get out of hand you can always appeal. Most of the time the Council will just let the projects run things, but in general I think they should have fairly absolute control over the distro, since they are an elected body (alongside the Trustees which has a different but perhaps overlapping domain). Rich
Re: [gentoo-dev] RFC: Enable FEATURES="userpriv usersandbox" by default?
On Mon, May 28, 2012 at 9:09 PM, Maxim Kammerer wrote: > Ditto, ~2 years with regular full @world rebuild. > Yup, been years since the last time I even saw a bug for this. Probably wouldn't hurt to announce in news if it will impact existing users. I doubt anybody would actually remove the portage user, but never hurts just to make people aware... Rich
Re: [gentoo-dev] RFC: Enable FEATURES="userpriv usersandbox" by default?
On Tue, May 29, 2012 at 10:11 AM, Michał Górny wrote: > > Wouldn't that break users who sync using a regular user? And then break > again, and again every time portage is merged? Yup, unless that regular user is the same one used for userpriv (if I'm correctly understanding the problem that you're pointing at). I don't see this as a show-stopper - just a reason to have a news item. Those not using userpriv can always disable it and run as root as they are already doing. Those who are using a regular user to sync could ensure that their make.conf uses the same user for userpriv. Rich
Re: [gentoo-dev] RFC: Enable FEATURES="userpriv usersandbox" by default?
On Tue, May 29, 2012 at 10:57 AM, hasufell wrote: > I am against too many defaults. It's documented and people can > activate it. > I'm already annoyed by pre-set stuff like "cups" in > releases/make.defaults. While universal agreement is a bit much to hope for, I just wanted to point out that fewer defaults is really just an illusion. There is ALWAYS a default, anytime you have an option. The default might be one thing, or it might be another, but there is ALWAYS a default. My thinking is that our defaults should generally reflect the most mainstream or least-surprising behavior, especially where there are upstream projects. in the case of portage, we are the upstream, so we should do whatever is most useful and least obnoxious to our users. If you're running something other than a generic desktop/server, there will always be a need to tweak things. Rich
Re: [gentoo-dev] Portage Git migration - clean cut or git-cvsserver
On Sat, May 26, 2012 at 12:18 PM, Alexey Shvetsov wrote: > Since most of us want "clean cut" solution so i will close bug #333699 as > WONTFIX Along these lines, I'm looking at 333531 (the git migration tracker), and it seems like there isn't actually much to do: 333685 - Seems like no action, and not sure how critical 333687 - while the bug is still open, as far as I can tell it seems good enough to me to move forward 333697 - ditto 333701 - paused since there are other tasks to do first, though it seems to me that little remains 333705 - not sure how critical this actually is - do we care if in some obscure case history is lost? Nothing says that we have to completely destroy the old cvs repo. Maybe we should just do a mock migration now and post copies of the before/after somewhere public and let people have at them. 333709 - seems like there is legitimate work to be done here, but again nothing that big So, what is the big issue? Is there something not being tracked, or is one of those items a lot harder than it looks? Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman wrote: > Yeah... this is why I was asking about access to infra to test the > conversion; so far, I haven't had any replies, though. > A mock conversion would probably help with creating procedures/docs/etc as well. It is nice to say that we're "just going to use git" but I think everybody has a slightly different picture of how that is going to work. If we could set up an "official unofficial" portage tree in git based on a one-time migration (maybe refreshing it from time to time) that could be a sandbox used to work things out, and it would then be replaced with the official tree. When the official migration comes along we'd already be experts in doing it. All we need to do is execute the migration, and just not point the rsync generation process at it. Maybe it won't be perfectly right at first, and that would basically be the point of doing it. Devs could update tools to work against it, and the docs could be written alongside. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson wrote: > 1. > Discussion on merge policy. Originally I thought we would disallow merge > commits, so that we would get a cleaner history. However, it turns out that if > the repo ends up being pushed to different places with slightly different > histories, merges are absolutely going to be required to prevent somebody from > having to rebase at least one of their sets of commits that are already > pushed. Not sure I'm following, but I will be the first to admit that I'm a git novice. Would this be aided by a convention, like only committing to master on the gentoo official repository, and any on-the-side work on places like github/etc stays in branches? Those repositories would just keep getting fed commits on master from the official repository. > > 2. > Git-SVN breakage. Why does this matter you're wondering? > We need the newer Git for the commit signing, but it comes with a > price, the git-svn binary has some major failures with SVN 1.7. > Git since 1.7.8 has been broken this way. To clarify - these won't be issues for gentoo per se, but there is a sense that we can't stabilize the latest git because it will break it for people using git-svn on non-gentoo work? I think the general conclusion was that we would not be supporting any mixed git+cvs/svn/etc workflows for gentoo itself. If that is the case, what is our sense of how important this feature even is to gentoo developers? They're the only ones who really have to have the latest git to commit to the official tree. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 3:13 PM, Robin H. Johnson wrote: > - You have a commit, that you want to put into the Gentoo tree. > - You have already pushed it to your github, signed > - It needs to be merged/rebased so that it applies on the Gentoo tree. > - If you force it to be a rebase so it applies on the tip, then you may > have changed the history of your github tree, and broken any further > forks. > - If you permit a merge instead, nobody gets broken. Maybe the best compromise is to tell people that if you push to "master" on other repositories, you get to deal with the mess. If we try to keep side overlays/etc working on branches and not on master then there will be no history to rewrite, as the merge will be rebased when it hits the official master, and from there it will get pulled by other repositories. We can perhaps allow merge commits on other branches, where the continuity of history is less important. Does that make sense? > You'd be excluding me entirely, I need to use git-svn for other work > projects, and emerging between two different versions of git would be > very annoying (I switch constantly between the sides of work as they > overlap). I'm a big proponent of letting the people doing the work scratch their own itches first! However, this does make us dependent on upstream - is there any sense of when they'll be ready, or what their own priority is for this issue. If this is becoming a deprecated feature then I'm not sure we can tie our future to it. I wasn't sure if any of the existing git-svn bugs pertained to this issue. Either way we should add this as a blocker to the git migration tracker. I think that even if we made a big push it would still take us a month or two to be ready with docs/infra/etc, and that might be optimistic. So, this might not be rate-limiting if upstream is actively working on it. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 3:33 PM, Michał Górny wrote: > What would git signing work with rebased commits? Would all of them > have to be signed once again? > The whole point of rebasing is to throw away history (which is either good or bad based on your perspective). So, if 14 devs spend 3 years and 2000 commits working on something in a branch, and I commit it to master using a rebase, then all you'll see in the master history is that rich0 committed 20k lines of code to master on May 31st, and that would be signed by me. I think that rebasing before merging is a pretty typical workflow anyway - when you submit a patch to Linus, he doesn't really care that you spent six months tweaking it - he just is getting a big blob of code that either works or doesn't. Does all that sub-history really matter? You could always push the branch to the repository if you wanted to keep it on the side. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Thu, May 31, 2012 at 5:52 PM, Kent Fredric wrote: > Just I haven't worked out what happens when the SHA1 of the 'parent' > header changes, which *will* change if the rebase is anything other > than a fast-forward. > > If that SHA1 changes, the gpg signature will surely fail? Rebasing doesn't modify past commits - it creates new ones and the past ones are no longer in the history of the current head. So, it doesn't break the old signatures so much as discard them. You would need to create new signatures in their place, presumably from whoever performed the rebase. I'm trying to glean what I can from the little info out there about how the new commit signatures work, but I don't think that you can use signatures to convey authorship if later authors are going to rebase the branch. The situation is not unlike what we have now with manifests. As far as I can tell if you want to do work outside of master, and then get those commits into master but preserve all the past signatures in the history of master, then you'd need to do a merge commit, so that all the deltas to do the merge are in a separate commit which is then signed by the person doing the merge. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 12:05 AM, Michał Górny wrote: > Yes, it isn't but such kind of work flow was presented in the message I > replied to. Yup, I wasn't aware that when rebasing you have the option to squash commits or not. They all get rewritten as if they were applied to the current head in the first place, but apparently unless you squash them you still get all the individual commits. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 12:55 AM, Kent Fredric wrote: > > Hmm, thats annoying. Almost makes me wish it was the trees that were > signed, not the commits. I think it is the tree that is signed, but that changes too. Rebasing re-applies the same diff to the new head to give you a new set of commits. When you apply the same diff to a different parent you end up with a different tree, so the tree signature won't be the same either. Keep in mind that git does not store a long train of diffs. It stores a long chain of complete trees, and the diffs get calculated if you ask for them. Since it is COW you only re-store files that actually change, and incorporate others by reference. However, if you have a 1MB file that you change 1 line on 100x, you'll end up with 100MB of files. Of course, when they get packed I'd hope that they'd compress well. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 7:23 AM, Kent Fredric wrote: > > Nope, at least not as far as I can tell, and I just implemented commit > signature verification >_> I've been trying to find an example of a signed commit, but can't find one anywhere, so it is hard to tell what it is doing without going through the git source carefully. If you have an example of a signed commit feel free to send it to me. Note that I am NOT interested in the output of any git operation (such as git log --show-signature, git show, etc) - these are all processed and do not reveal what is actually going on. I want a copy of the actual file containing the signature. If the signature is embedded in the commit then I want the file in the objects directory tree that contains the commit. They're just compressed text files (though it is a bit of a pain to decompress them). > Not nessecarily. Given that : > > a file with a given content has a fixed SHA > A tree is just a list of these SHA's , and that in turn is referenced > by SHA, so if 2 commits have identical file content, their 'tree' sha > will be the same ( in theory ). > > So that means, if you perform a rebase, assuming the filesystem looks > the same as it did before the rebase, it will have the same SHA1 for > the tree, regardless of the process of how it got to be that way. The filesystem WON'T look the same after a rebase. quick example (operations done in this order): file in commit A in master: 1 2 3 4 5 file in commit B in a branch off of master: 1 2a 3 4 5 file in commit C in master: 1 2 3 4a 5 if you merge master into the branch you'll end up with a new commit D whose parent is B that looks like: 1 2a 3 4a 5 if instead you rebase master into the branch you'll end up with a new commit D whose parent is C that looks like: 1 2a 3 4a 5 The history for the branch will no longer contain B. If there were 14 commits B1..14 you'd end up with 14 commits D1.14 that each contain the line 4a. None of them would use the same trees as B1..14, and they can't use the same signatures as a result, even if only the tree was signed. As far as the new history was concerned, line 4a was there before the branch was started. At least, that is my understanding of rebasing. Again, I'm a bit of a git novice, but I never really grokked git until I saw a presentation that didn't start with commands, but instead started out with just walking through the actual structure of the repository. Once you grok the COW model I find it all clicks into place. Rich
Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 11:12 AM, Andreas K. Huettel wrote: > Now, does the "signed data" also contain the parent sha? > So, I was working on a lengthy email which now would be fairly repetitive of what Kent posted. Suffice it to say I managed to rip out a commit from the kde overlay, deflate it, and compared that the signature: -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.18 (GNU/Linux) iQEcBAABCgAGBQJPx+mcAAoJEO+t9ga+3I3aqLoH/0OrRA1+NPRHGfbbLoQrqMwl sB+2It2Pb9LfPjEme+lrQu5WgFY4j7k0qd2ZYdnXM7JdQjsqmpfAMloHh5JN4TAS 4vk8+u2GJCYgzL/SY5XlPl2l8dT91PhQJSN0yVt4Q9TsoN3nzVpFBjACJCy9R6j2 HrXvz/g3+MqY/9VesV8IiVgvQUTVgCdh8zBJ2rVyWAVH0bErsn518aiwEyfzNOxA 1qJxxgGJLMpXp+nI8rnmhqTAAKiNA+byAKAsTEl3LS7OvQZ51aOCwa4A2GLOn2ef 5JmuYQG5/FsS0RfXrqk72PiStTBWa3TakHYrgNXOXlslIR5AIB2tYnXqZcdEqYQ= =fucY -END PGP SIGNATURE- does in fact verify for the payload: --start-- tree 7d7f97cded40158d0f580ca6fbe97398d5c867f8 parent 14d7d9cb2219f64c7a715d8da0bbe48a32c9dad8 author Johannes Huber 1338501525 +0200 committer Johannes Huber 1338501525 +0200 [kde-base/kdelibs] Sync with live. (Portage version: 2.2.0_alpha108/git/Linux i686, unsigned Manifest commit) --end-- Dump those into a text file and run gpg for yourself... The full commit contains the gpg signature in a field as already posted by Kent. And while I appreciate the performance boost and space savings provided by all the compression/packing/etc, I've learned to almost hate those features with a passion this morning... Getting a cloned repo unpacked, and the commit decompressed was a bit pita. The other issue is that the header in the commit file is stripped before it is signed, the actual start of the commit is "commit 830tree..." Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 9:25 AM, Nicolas Sebrecht wrote: > So, like explained before your concern is clearly out of the current > discussion. Importing commit history from Overlays is not supported and > will probably never be. Gentoo doesn't forces (and doesn't want to) > overlays maintainers to use Git. I'm not sure that git even supports this, unless the overlay is a complete clone of the entire gentoo-x86.git repository. I think the way git operations work is by finding common parents in the history of two heads, and then moving forward from there. If you have two completely different repositories then they never will have a common parent. Plus, even if it did work, to rebase the overlay on the gentoo-x86 repository you'd have to import the full gentoo-x86 tree into it. Then you'd have to push EVERYTHING in your overlay into gentoo-x86. I don't think you can just push individual files from one tree to another. >From what I've seen the various methods out there which do involve moving only part of one branch into another basically involve a LOT of history manipulation or are really no different than just copying the files into the branch and adding them, losing all history in the process. I'm not sure how important all that history preservation actually is - the overlay would still possess it. If we did want to preserve it then the only practical option I see is to have the overlay start out as a clone of gentoo-x86 and keep around all the non-overlay packages, which then means that anybody using the overlay will get all those old gentoo-x86 packages as part of their portage tree. Plus you still have the rebase+gpg-signatures=fail issue. Rich
Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 11:36 AM, Andreas K. Huettel wrote: >> On 2 June 2012 03:12, Andreas K. Huettel wrote: >> Yes. Which basically means, you *cannot* have both >> >> a) rebase only merges >> and >> b) every commit must be signed >> >> as policies. >> > > I would say that this is a very strong argument in favour of allowing merge > commits. One advantage of merge commits with signatures is that the history really does reflect who signed what. Proxy maintainer signs a bunch of ebuilds. I merge them in. The commits show that the proxy maintainer signed a bunch of work done against an old tree, and I signed a bunch of merge diffs that basically synced them up to the new tree. However, this is missing another issue. What is the value of preserving all those original signatures in the first place? I'd think that they'd mainly be used as some kind of web-of-trust. Well, would such a web-of-trust include proxy maintainers in the first place? If you want the tree to be traceable to Gentoo devs, then rewriting the signatures is probably a good thing. However, Kent did point out the rebase function doesn't actually apply new signatures to the "new old" commits anyway, so you'd end up with unsigned commits in the history. git-rebase is just a shell script, that ultimately just calls git-commit as far as I can see, which means that implementing re-signing is just a matter of adding the appropriate parameters, or use default configuration (assuming it doesn't already do this). Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 12:33 PM, Robin H. Johnson wrote: > What about overlay repositories that elect to be a branch of the main > tree via git? > > Do we call that forbidden usage? I think that branches off of the main tree are mainly going to be useful for more eclass/profile/etc-related work. Work on a package or small group of packages will almost always go better in a completely independent overlay. If you try to do that kind of work in a branch then if you create an rsync tree from that branch it will contain all the other packages that you aren't working on, and they'll get stale very quickly. Anybody using that as an overlay will get a bunch of old ebuilds for who-knows-what in their tree. Now, branches in the main tree are going to be really useful for stuff like package moves, new virtuals, eclass api changes, or other messy changes that have big tree-wide impacts. You can stage the change and fix the 300 impacted ebuilds in a branch, get lots of people to test them, and then merge those in with a single transaction, pulling in updates from master all the while. That is more about portage tree maintenance than package maintenance per-se. All that said, having the tree in git is still a big help to proxy maintainers and others even with all these issues. I've worked as an outside contributor to other projects and it is a lot easier with git. I can easily work in my own PM, rebase against their master, and then easily submit a nice clean diff as a patch, even without doing any pushing at all. I don't have to have push rights to anything official to be more efficient in my contributions. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Fri, Jun 1, 2012 at 4:49 PM, Michael Weber wrote: > > Cloning the repo [1] takes 200seconds on 8cores (it's 2GB of data and > 22 minutes of 3.4GHz cpus). > As others have pointed out, probably the best way to bootstrap this is to offer tarballs of a shallow repository and a full repository. Perhaps we'd offer the latter as a torrent. The shallow repository should be light on the CPU too. This would be a lot easier on the server than having everybody and their uncle doing a full 2GB clone. Devs could then do a pull to get the latest and greatest, and that would only transfer the delta. I imagine our mirror network can handle the bandwidth compared to what we're already doing with distfiles. Worst case we could take a one-time hit and use S3 or whatever to do the distribution (they even support bittorrent to cut down on the bill). Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Sat, Jun 2, 2012 at 12:04 AM, Robin H. Johnson wrote: > please look up git-bundle before suggesting things like tarballs of > repos/checkouts. > Looks useful. Wasn't aware that a bundle was something other than a tarball. We'll probably need to spell out the preferred process in the docs, and reference it frequently in communications. Otherwise you'll get quite a few clones instead. It appears that devs will have to add the remote for the live repository after they've cloned the bundle - otherwise they'll just keep pulling from the bundle which isn't all that convenient. Rich
Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver
On Sat, Jun 2, 2012 at 8:03 AM, Dirkjan Ochtman wrote: > On Sat, Jun 2, 2012 at 12:59 PM, Rich Freeman wrote: >> It appears that devs will have to add the remote for the live >> repository after they've cloned the bundle - otherwise they'll just >> keep pulling from the bundle which isn't all that convenient. > > I think you still misunderstand. As I understand Robin, we wouldn't > even offer up a clone of the full-history bundle, it would only be > offered as a normal download. The default workflow is cloning from the > shallow version, which will obviously give you the desired remote. I wasn't talking about full-history. I was talking about the fact that we're distributing a bundle. If you clone a bundle, you won't have a remote for the live repository. You would need to add it, unless you plan to never push a commit back to the gentoo repository, and you plan to manually download bundles anytime you want to update your local repository. I'm not sure how exactly Robin was planning on making the full history available, but it sounded like it would also be distributed as a bundle. That means that you can certainly clone it - just type git clone path-to-locally-saved-bundle-file . If it is in some other format like a pack file then you would import it into a repository via a different command. Rich
Re: [gentoo-dev] Git braindump: 2 of N: developer interaction (merge co-ordinators)
On Sun, Jun 3, 2012 at 2:21 PM, Dirkjan Ochtman wrote: > That makes a lot of sense. There are probably a bunch of > projects/teams where having their own branches makes some amount of > sense; but we should really try the free-for-all at first. So, it sounds like we have a migrated tree already. Why not just plan a few days where we tell everybody to just mirror their work as an experiment (it would be strongly encouraged but not mandatory if for some reason a few can't go along with it). So, devs would do their work in cvs, and then just do the exact same commit in git and push it. The rsync mirrors would of course stay with cvs (this is just a test). Perhaps we might even make it a complete free-for-all with no rules at first. Then we can circle back on the list and discuss what did and didn't work well. That could include rebase-vs-merge, problems with concurrency, and so on. A test of some sort would cut down the risk of the unexpected when we do the real migration. Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Mon, Jun 4, 2012 at 2:50 AM, Dirkjan Ochtman wrote: > On Sun, Jun 3, 2012 at 9:35 PM, Andreas K. Huettel > wrote: >> However, then the "committer" of the contributed commits before the merge is >> then the user, I guess? >> >> (The rule meaning as suggested by Robin >>> - if you include a commit from a user: >>> author := non-@gentoo >>> committer := @gentoo >>> signer := $committer > > I guess, I'm not sure how the committer thing works in git. > Well, only Robin can explain exactly what he meant, but it sounds like we don't want the committer field to ever have a non-gentoo email in it, and signatures should be gentoo as well. So, if a dev just applies a patch to their tree/etc then there is no issue (just set author). If a dev wants to actually pull in a commit they'd need to edit the fields accordingly and re-sign it. Not sure offhand how to best do that (I assume it is possible - probably with some variation on rebase or something rebase calls). I don't think the intent is to snub non-devs. The issue is what is the purpose of the signatures and committers field in the first place. The signature verifies that the commit is intact, and you can only do that if you have a key to check it with, and you can trust that key. If the signer is a dev then we already have policy that the keys need to be published, and we have a list of key IDs on our website. I'm sure that could be improved on. If we stick non-dev signatures in the tree then that becomes more of a problem (though it clearly is possible - maybe something to think about). I assume the committer denotes a layer of accountability, and having a dev in that spot makes sense (devs who are proxies are accountable for oversight at some level - though I'd personally give them the benefit of the doubt since we want to encourage the proxy role). I think the key with git is to not let the perfect be the enemy of the good. We don't have an unbroken signature chain on our current portage tree, so I don't think we need one to move to git. As long as git is at least as good as what we have now, then we should accept it. We should of course strive to improve, but let's not keep the almost completely unsigned cvs around for another 10 years while we argue about signatures. Rich
Re: [gentoo-dev] Git braindump: 2 of N: developer interaction (merge co-ordinators)
On Mon, Jun 4, 2012 at 2:48 AM, Dirkjan Ochtman wrote: > IMO we should try to be cutting down barriers from the git migration, > not throwing up more. The process has taken long enough already; the > desire to control everything about the migration is part of why it's > taken so long. Fair enough. We should also be asking ourselves what happens if something goes really wrong and there are no commits for a whole two days. I think the answer is, not much. As long as there are a few devs who can tend to security issues or whatever I don't see it as an issue if we have the odd glitch. We do need to get the docs up so that we're at least somewhat harmonized on conventions (authors vs committers, and so on). Let's not burn the first person to make a mistake at the stake either... Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Mon, Jun 4, 2012 at 8:45 AM, Dirkjan Ochtman wrote: > > Well, it doesn't seem like a big deal IF there's an explicit merge > commit that's signed by a dev. I'm not sure about that. If you were verifying a tree, how would you identify which commits were merged in by what dev, using an automated algorithm? The only thing the merge commit contains is a list of two parents, and a tree. It doesn't say which one is which, unless we can rely on their order. Now, all those intermediate commits were never actually published via rsync, so their integrity isn't a direct issue. However, I'm not sure how easy automated verification would be. Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Mon, Jun 4, 2012 at 9:48 AM, Dirkjan Ochtman wrote: > > You simply walk the tree from root to tip. When you encounter an > unsigned changeset, the nearest signed descendant is responsible for > merging that changeset. > How do you KNOW that the nearest signed descendant actually merged it? How do you know it wasn't added by a hacker? Also, when walking the tree keep in mind that there isn't just one path in it (with merge commits), and the links are from any particular HEAD going back. I'm not convinced that this is impossible, but it isn't as trivial as it might seem at first glance. Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Mon, Jun 4, 2012 at 10:26 AM, Dirkjan Ochtman wrote: > On Mon, Jun 4, 2012 at 4:18 PM, Rich Freeman wrote: >> How do you KNOW that the nearest signed descendant actually merged it? >> >> How do you know it wasn't added by a hacker? > > Because then the signature for the nearest signed descendant wouldn't > check out (unless it got hacked before he signed it, of course, but in > that case hopefully he wouldn't sign it...). When I do a cvs commit, I don't check the logs to make sure the last 25 commits all look valid. So, why would I expect others to do any differently in git. I make my changes, I run a git pull (bringing in the hacked commit on gentoo-x86 master), and then merge/rebase in my changes, signing my commit (which indicates that what _I_ just commited is good, not that everything before is good). I am not the one commiting in hacked files - they were there before I got there. > > Of course, we'd have to make sure the tip of whatever is pushed is > always signed, but the hook for that should be trivial. Yup, but the hacker wouldn't run the hook. Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Mon, Jun 4, 2012 at 11:02 AM, Dirkjan Ochtman wrote: > If the tree was bad before you pushed, then it's not your fault the > tree is bad. You're only responsible for the commits you bring into > the tree, so if you're merging contributor's unsigned changesets, you > merge them with a signature of your own. Yup, but the fact that the tree is bad is still a problem, even if it isn't my fault. > If the hacker has unfettered access to the server where the repository > lives, we probably have bigger problems, as they can get whatever > rsynced to all our users. I guess we could have rsync process check > that the cset it's about to push out to mirrors is signed? So, the whole point of signing is that it lets you prove that the repository is uncompromised. If we're going to assume that the server is secure, then we don't need signatures - whatever is on the server is by definition correct. A robust security infrastructure is already spelled out in a GLEP (though that one is dated). Ideally it should be verifiable from end to end, so that when you run emerge if a package has been tampered with it will just refuse to install it. Since we don't distribute the whole git repository the git commits only get us part of the way there. However, if every step of the distribution assumes that the previous step could have been compromised that would be a good start. Again, we don't need to be there 100% to go live. However, I think that was the whole point of signing commits. If we aren't going to add any assurance at all with our signing practices, then there isn't much point in having them. Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Mon, Jun 4, 2012 at 12:19 PM, Dirkjan Ochtman wrote: > So to prevent your scenario, we'd > have to get everyone to check the signature of the tip of tree they > pulled before committing/merging. How can we be sure this has happened? This is the problem with signed manifests today. I can sign a manifest, but I didn't actually check all the files inside it, and the file might or might not have been signed before I modified it, and most likely I didn't even check the signature even if it was there. Anything we do has to be automated to be of any real value. Ideally if something goes wrong it should be as detectable as possible. Warts and all the current system hasn't broken down yet. However, if we ever did find out about an intrusion in our cvs repository, we'd essentially have to do a 100% code review to be sure it was OK, and that includes checking all tarballs on mirrors. With signed commits we could verify that the tree was intact, and if anything bad was found we could pinpoint exactly whose key was compromised and do a focused check on their commits. Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Mon, Jun 4, 2012 at 3:10 PM, Brian Harring wrote: > One thing people need to keep in mind here is that when you sign the > commit, you're signing off on the history implicitly. Directly > addressing freeman's comment about "people sign the manifest but don't > look at what they're signing", when it comes to git signage, bluntly, > people doing that shouldn't have access- if they can't be arsed to > validate what they're signing, then trusting them w/ the tree is > probably questionable. I suspect that you're missing my point. The argument was made that as long as merge commits are signed you know that unsigned commits referenced by them are OK. However, some of those commits might have been already in gentoo-x86 and I doubt that anybody is going to check those. If I have a perfect commit, I do a git pull and a git push and the result is a merge that references whatever was in gentoo-x86 before, whether placed there by dev, or hacker, or whatever. Unless I go back and review the existing gentoo-x86 history (and likely have to repeat the process when somebody else does a push before I do), I can't vouch for what was in there already - just what I'm adding. The reason I mentioned maifests is that they have the same issue. If I keyword an arch on foo-1.4.5, I sign the manifest. That doesn't mean that I checked every file in the package's directory tree for issues. At most I checked foo-1.4.5, but I can't sign off on just 1.4.5 - I have to sign off on everything. Also, when I sign off on 1.4.5, I'm really just signing off for the keyword change, not the piece of buggy code I didn't write on line 37 of the ebuild. Of course when merging a pile of commits into the tree you should check all of them to make sure they're fine (or rather that the end result of them is fine - no absolute need to squash together bug introductions and fixes even if that is nicer). However, I'm not sure I'd extend that to checking commits ALREADY in gentoo-x86 made by some other dev. The general principle is that if you change something in the tree, you should be responsible for what you changed, and that makes sense. Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Mon, Jun 4, 2012 at 4:41 PM, Brian Harring wrote: > > If that doesn't answer your question/concerns, be more explicit > please. How about a scenario: 1. Gentoo dev commits a bunch of stuff to the tree. Top of tree is signed. 2. Hacker commits something to the tree. Top of tree is not signed. No need for preimage attacks or whatever on sha1 - they just log into the server and do a git commit or whatever right into the tree. 3. Gentoo dev commits a bunch of stuff to the tree. Top of tree is signed. 4. Rsync mirror update happens - top of tree is signed, so update proceeds normally. If you go back and look at the tree you see a bunch of signed and unsigned commits. How do you easily detect how the unsigned ones got there (via a dev with a merge commit, or via other means)? Either way they'll be parents of merge commits - since merge commits have two parents - the pre-commit gentoo-x86 tree, and the incoming commits. Andreas - I'm pretty sure a merge commit still includes a tree. Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Tue, Jun 5, 2012 at 2:50 AM, Michał Górny wrote: > On Mon, 4 Jun 2012 16:57:42 -0400 > Rich Freeman wrote: > >> If you go back and look at the tree you see a bunch of signed and >> unsigned commits. How do you easily detect how the unsigned ones got >> there (via a dev with a merge commit, or via other means)? > > Well, that's not a very good solution but the server-side hooks could > also verify the tree state before applying new commits. The obvious problem with this is that it makes the git server a single point of failure - if it is compromised the hooks will not help. Hooks should nevertheless be there to eliminate mistakes. Note that in no way are any of these git flaws any worse than the status quo. I just want to avoid any false sense of security. I think these are flaws that are worth fixing, and I think that was why many have labored to get the signing enabled in git in the first place. My suggestion is to keep working on this, but it shouldn't be considered a blocker for adoption, since these are not new security flaws, and if anything despite its holes git is an improvement. Rich
Re: [gentoo-dev] Git braindump: 1 of N: merging & git signing
On Fri, Jun 8, 2012 at 7:01 AM, W. Trevor King wrote: > When the breach is discovered, you can then isolate the dev (or devs) > who implicitly signed the hack (2) by pulling the ToT without checking > for a valid signature (3). Then you yell at them for sloppy security, > and tell them to install your signature-checking post-receive hook. Well, if devs are supposed to do this, we should probably write this down as a policy somewhere. Probably wouldn't hurt if the post-receive hook actually existed, and it was designed to only work on the official tree otherwise everybody will just uninstall it since people don't just pull from the official tree. I doubt any dev checks the signatures on manifest files before they overwrite them with a new signature. If they did it wouldn't matter since those signatures aren't even mandatory anyway. Certainly it isn't intuitive to me that when I perform a signature on changes I make that I'm also vouching for work committed by somebody else before me. Process can be as effective as technology in achieving security, but only if those processes are clear, and unintrusive enough to ensure they are followed. I wouldn't count on being able to yell at developers - first it does nothing to solve the mess that you'd be in at that point, and second you can only yell at volunteers so much. Rich
