Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.

2013-07-02 Thread Sergei Trofimovich 
[ sorry, a lot to quote ]

What upstream do you plan to report bugs when user possibly mixes
all of it in one mess? Each of those is known to be unstable. The mix
is just a disaster.

Is gentoo's kernel team able to resolve user's OOpsen?

> ### ... and configuration. ###
> 
> This problem is not only visible for patches, but also in the config.

Insane :]

> Meet CONFIG_DEVTMPFS; forget to enable it, greet a failing boot. We're
> telling users to enable it in some places, in the handbook it's a single
> line you must read, on the Wiki it's kind of missing unless you are
> luckily on the right page, on the Quick Install book it is missing too.

Forbid users install udev to ROOT=/ if running kernel does not support devtmpfs
(easy to check by /proc/filesystems)

-- 

  Sergei


signature.asc
Description: PGP signature

Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.

2013-07-02 Thread Fabio Erculiani 
On Tue, Jul 2, 2013 at 9:36 AM, Sergei Trofimovich  wrote:
> [ sorry, a lot to quote ]
>
> What upstream do you plan to report bugs when user possibly mixes
> all of it in one mess? Each of those is known to be unstable. The mix
> is just a disaster.
>
> Is gentoo's kernel team able to resolve user's OOpsen?
>
>> ### ... and configuration. ###
>>
>> This problem is not only visible for patches, but also in the config.
>
> Insane :]
>
>> Meet CONFIG_DEVTMPFS; forget to enable it, greet a failing boot. We're
>> telling users to enable it in some places, in the handbook it's a single
>> line you must read, on the Wiki it's kind of missing unless you are
>> luckily on the right page, on the Quick Install book it is missing too.
>
> Forbid users install udev to ROOT=/ if running kernel does not support 
> devtmpfs
> (easy to check by /proc/filesystems)

No. As explained multiple times, this check is not reliable and
doesn't work (chroot, binpkgs, containers without kernel, and so
on...).
Making sure that the user doesn't build an unbootable kernel is the way to go.

>
> --
>
>   Sergei



-- 
Fabio Erculiani

gentoo-checkconf script Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.

2013-07-02 Thread Michael Weber 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 07/01/2013 11:53 PM, Anthony G. Basile wrote:
> Now I'm confused because gentoo-sources is gentoo specific.  It
> contains stuff that we need in gentoo but other distros do not
> need, like our end-to-end support for certain xattr namespaces.  If
> you remove these then we must either 1) maintain a userland which
> is not in line with other distros or 2) give up on critical
> features we want in gentoo, like markings on elf object in
> user.pax.flags and certain caps, as well as in the future
> preserving selinux labels through emerge.  Upstream will not accept
> them because of "who needs that crap" and we can't give them up 
> without loosing core functionality.  Feel free to review those
> patches but don't ask us to drop them from gentoo-sources because
> their not in upstream.

What about a check-kernel-config-for-gentoo-compliance script for
starterts?

I manage a handfull of kernel configs over some years (laptop vs.
server, graphics, firewalling capabilities) and was always tempted to
write an script to check if the config meets a certain set of
requirements. I think of "xattr", "selinux", "gentoo-boot" and so on,
that can be expanded by users demand, like, "CONFIG_CMDLINE should
include" and "CONFIG_DEFAULT_HOSTNAME=x" and "all iptables target on".

A additional make target in gentoo-sources could the warn about any
missing feature, and ask for "yes" or wait some seconds.
(I remember reaging some funny note about my kernel supporting x32 but
by userland not, like that kernel build would run on that userland)

==> Merging a certain source does always imply to run it on that system.

(diff-ing configs is really nasty since sub*module=N drops lines from
the config)

(and i got lazy on reading all the added features in subsystems [1])


   Michael "I can live with a lot of things, as long as I can
configure/compile/update my kernel and the out-of-tree drivers when i
want" Weber

[1]
http://3.bp.blogspot.com/_rtOXMZlMTkg/RZWVjP3f49I/ADs/YpHlSwXpiUg/s400/drinking_bird.jpg


- -- 
Michael Weber
Gentoo Developer
web: https://xmw.de/
mailto: Michael Weber 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlHSj+IACgkQknrdDGLu8JD65AD+NHyGeFNQw4GceLp0g9ypik5j
NzoEwKYztMCOwKcjbO4A/A1e/KQv4DabFoZA41kdPBH8DMOITWL7Jb3OHqewwpPL
=OOdc
-END PGP SIGNATURE-

Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.

2013-07-02 Thread Michael Weber 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 07/02/2013 10:21 AM, Fabio Erculiani wrote:
> On Tue, Jul 2, 2013 at 9:36 AM, Sergei Trofimovich
>  wrote:
>> Forbid users install udev to ROOT=/ if running kernel does not
>> support devtmpfs (easy to check by /proc/filesystems)
> 
> No. As explained multiple times, this check is not reliable and 
> doesn't work (chroot, binpkgs, containers without kernel, and so 
> on...). Making sure that the user doesn't build an unbootable
> kernel is the way to go.
Nah, as I just wrote on another sub-thread of this, this assumption is
as-bad-as the other one.

Imagine users to cross-compile kernels for other
userlands/hardwares/whatever, like compile a gentoo-sources on a fast
x86 for a ARM RaspberryPi with a Debian userland.

Two thoughts, out package manager does not handle reboots (and I like
it), and kernel builds as well as installs/bootloader are a bit out of
this scope.

- -- 
Michael Weber
Gentoo Developer
web: https://xmw.de/
mailto: Michael Weber 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlHSkVgACgkQknrdDGLu8JDn5AD/bgSoPzVvyajlh8K2vDGWiwhR
Nbjxr8rwPvTl/RW6LE8A/j7QUy1QbSkqOXSmkuXbUNbplSRhVamwDQ+XFMK1jgMI
=HjGE
-END PGP SIGNATURE-

Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.

2013-07-02 Thread Michael Weber 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Sorry for that GPG-traffic and me needing 3 mails to formulate my
thoughts.

I am not against providing a gentoo-sources binary kernel package with
a sound setup to suite gentoo-needs and `make allmodconfig`
to give the less enthusiastic `make nconfig` button pushers something
to boot their fresh installs with -- and never move away from it.

As long as it's slot-ed and proper CONFIG_LOCALVERSION'ed* aka having
more than one version installed to do downgrades and tests.

(*) Meaning, the modules dir should change with every release to avoid
loading outdated out of tree modules, and the kernel files in /boot
should not clash.

Suggesting to run `grub2-mkconfig -o /boot/grub/grub.cfg` would be
nice, too.

*enough*

- -- 
Michael Weber
Gentoo Developer
web: https://xmw.de/
mailto: Michael Weber 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iF4EAREIAAYFAlHSlOcACgkQknrdDGLu8JCRhAEAhpl7vsTVmo9CPARie4kWznTd
cam1MmdcRejlpkiOeiUA/0Nn0Q7cERzPyXkiwFO5bbprB9OUMFNrXlygp5Hb8KVn
=UjWq
-END PGP SIGNATURE-

Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.

2013-07-02 Thread Tom Wijsman 
On Tue, 2 Jul 2013 10:36:34 +0300
Sergei Trofimovich  wrote:

> What upstream do you plan to report bugs when user possibly mixes
> all of it in one mess? Each of those is known to be unstable. The mix
> is just a disaster.

They used to do this to us and to kernel upstream before. Why? Because
we are unaware. The users just applied their patches, forget about them
when they experience problems; then they come and file a bug without
us knowing they do use these. Thus we'll ask for a .config and any
separate patches the user applied from now on.

The mix is just a disaster, but that's up to the user whether he wants
to go for such disaster; the patches are guarded using an experimental
USE flag and menu config option, if you enable both, you can't expect
everything to work fine. Well, maybe it does, if you sparingly enable
one or two patches but not all of them.

> Is gentoo's kernel team able to resolve user's OOpsen?

When patches are applied, we can ask them to try to reproduce it without
patches; unless, it is clear from the BUG that an experimental patch is
at cause. If they consistently get a BUG with a patch, we can send these
users to report this at the patch author; basically we act as a filter
before sending these reports upstream. And yes, we do try to resolve
some of these on our own.

> > Meet CONFIG_DEVTMPFS; forget to enable it, greet a failing boot.
> > We're telling users to enable it in some places, in the handbook
> > it's a single line you must read, on the Wiki it's kind of missing
> > unless you are luckily on the right page, on the Quick Install book
> > it is missing too.
> 
> Forbid users install udev to ROOT=/ if running kernel does not
> support devtmpfs (easy to check by /proc/filesystems)

But udev comes as part of stage 3 tarball; so, we would have to change
the whole way of installing a system. I think opting to enable it by
default for Gentoo kernel sources is a better / less invasive approach.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : tom...@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D


signature.asc
Description: PGP signature

Re: [gentoo-dev] Re: Gentoo Hangouts

2013-07-02 Thread Donnie Berkholz 
On 09:21 Mon 24 Jun , Mike Pagano wrote:
> Sometimes it helps to realize that the people on the other end of the
> wire are just that: people.
> 
> I've seen behaviors change among team members for the better.

^^ This.

Seeing people as close to "in person" as we can get without a conference 
really does help to improve the quality of our interactions. It's a lot 
harder to be a jerk when you can picture the person you're writing to as 
a living, breathing person.

Given recent, and more distant, actions by some of our community with 
DevRel consequences, it should be clear that treating our fellow 
developers decently is a problem. And seriously, hopping on a video chat 
once or twice a year should be doable for most of us.

I don't necessarily care a lot about the PR value of any replays, I see 
it as much more important to strengthening Gentoo than to do outreach. 

That said, I've gotten over 10,000 views of an intro talk on Gentoo 
that's posted on YouTube despite its pretty bad audio quality, and 
nearly 2,000 views of a Gentoo talk targeted at developers, so there's 
clearly people looking for this stuff.

-- 
Thanks,
Donnie

Donnie Berkholz
Council Member / Sr. Developer, Gentoo Linux 
Analyst, RedMonk 


pgphu4DSZRfgK.pgp
Description: PGP signature

Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.

2013-07-02 Thread Sergei Trofimovich 
On Tue, 2 Jul 2013 10:21:53 +0200
Fabio Erculiani  wrote:

> On Tue, Jul 2, 2013 at 9:36 AM, Sergei Trofimovich  wrote:
> > [ sorry, a lot to quote ]
> >
> > What upstream do you plan to report bugs when user possibly mixes
> > all of it in one mess? Each of those is known to be unstable. The mix
> > is just a disaster.
> >
> > Is gentoo's kernel team able to resolve user's OOpsen?
> >
> >> ### ... and configuration. ###
> >>
> >> This problem is not only visible for patches, but also in the config.
> >
> > Insane :]
> >
> >> Meet CONFIG_DEVTMPFS; forget to enable it, greet a failing boot. We're
> >> telling users to enable it in some places, in the handbook it's a single
> >> line you must read, on the Wiki it's kind of missing unless you are
> >> luckily on the right page, on the Quick Install book it is missing too.
> >
> > Forbid users install udev to ROOT=/ if running kernel does not support 
> > devtmpfs
> > (easy to check by /proc/filesystems)
> 
> No. As explained multiple times, this check is not reliable and
> doesn't work (chroot, binpkgs, containers without kernel, and so
> on...).
> Making sure that the user doesn't build an unbootable kernel is the way to go.

udev's case:
It's _not_ the kernel which upgrade breaks user's system. Why do you try to 
"fix" it?
If you want to save user's box - make check at pre-install time. Otherwise it 
will break.

containers, etc. case:
If you build gentoo(and, what really matters _install_) in container
- you can run gentoo in container. Yes, with it's stock broken kernel.

Who cares what you have built if the host is xen-3.0.6 with CONFIG_DEVTMPFS=n?
If you really want to proceed unsafe action you need to do it explicitely.

kernel's case:
CONFIG_DEVTMPFS needs 'default yes', right? One-liner to gentoo-sources.
Why 'hide' it? It's very counterintuitive.

What will you do with all-those-required-to-boot-properly
  - root filesystem
  - disk controller
  - USB keyboard drivers
 ?
Include them all unless CONFIG_DONT_REMOVER_OR_WONT_BOOT
option?

I'm afraid I don't see how it's a solution. 
Suppose, tomorrow's udev will require CONFIG_foo, and glibc will require
CONFIG_bar. How will you save user with your mechanism?

-- 

  Sergei


signature.asc
Description: PGP signature

Re: [gentoo-dev] Re: [gentoo-kernel] Proper distribution integration of kernel *-sources, patches and configuration.

2013-07-02 Thread Greg KH 
Almost all of this portion of the thread is off-topic for gentoo-dev, so
I'll leave it alone, and will be more than willing to take it up
somewhere else it is on-topic for, like linux-kernel, if you want to.

But, there is one thing I do want to ask/comment on, as it is relevant
to users of Gentoo:

On Mon, Jul 01, 2013 at 11:29:39PM -0400, Richard Yao wrote:
> >> 4. Risk of patent trolls
> >
> > I know of no kernel patches outside of the tree because of this, do you?
> 
> There is compatibility code for DOS long filenames in fat that is no
> longer in-tree because of this.

I remember the conversations that a number of us had a few years ago
about this potential issue, but do not see any in-kernel changes that
were ever made because of that.  Could you point me at the changes that
were made that were accepted into the kernel.org tree?

thanks,

gerg k-h

Re: [gentoo-dev] Proper distribution integration of kernel *-sources, patches and configuration.

2013-07-02 Thread Tomáš Pružina 
This reminds me of: http://imgs.xkcd.com/comics/standards.png
It sure sounds nicely, however I would not want to be the guy who maintains
the whole mess of (often) incompatible patchsets.
Given the fact that some patches lag 1-3 stable versions behind Linuses
tree (grsec used by hardened for example),
this might be better idea on paper than in reality.

If you feel up for this, go for it. Be ready to meet resistance though.

// I personally maintain my own tree, based on stable from gregkh and few
patches

On Mon, Jul 1, 2013 at 4:41 PM, Tom Wijsman  wrote:

> Hello
>
>
> Please reply to gentoo-dev in case ML daemon changes Reply-To.
>
>
> ### TL; DR ###
>
> By introducing feature patches which menu options are disabled by
> default to genpatches, we can deduplicate *-sources maintainers as well
> as large groups of users work. By introducing a distribution section
> in the menuconfig, we can provide options that enable minimal Gentoo
> requirements by default (DEVTMPFS, making Gentoo systems bootable since
> an udev release a long time ago) and other distribution stuff.
>
>
> ### Proper distribution integration of kernel *-sources, patches ... ###
>
> Gentoo is a distribution; but what is a distribution that doesn't
> properly integrate what it provides, but instead expects its users to
> do so, needlessly duplicating work amongst its maintainers and users.
>
> Let's say I want genpatches, aufs and TuxOnIce; closest candidates:
>
>  - sys-kernel/aufs-sources: genpatches, aufs
>  - sys-kernel/pf-sources: genpatches, CK, BFQ, BFQ, TuxOnIce, UKSM
>  - sys-kernel/tuxonice-sources: genpatches, TuxOnIce
>
> What do I do? Take one (eg. aufs) and apply the other (eg. TuxOnIce)?
> What if I want to add another common patchset to it? Hardened perhaps?
>
> You can see, some of these sources (like pf-sources) already attempt to
> do so; with pf-sources in mind, why do we even have ck-sources,
> tuxonice-sources in the Portage tree? Just to duplicate work?
>
> I think we should trim down on the amount of *-sources and combine
> multiple patches into genpatches. Take an example of geek-sources
> which does all this without a problem; I don't really like the approach
> with USE flags used there, as the menuconfig can just cover that.
>
> https://github.com/init6/init_6/wiki/geek-sources
>
> What does a patch introducing new features really do? Or rather, what
> should it do when we add it? Let me summarize:
>
>  1) The features should be disabled by default.
>  2) These feature should depend on a non-vanilla / experimental option.
>  3) The patch should not affect the build by default.
>  4) The user can optionally enable the feature.
>
> So, in genpatches, since 3.9.7, BFQ was added to try this out (except
> 2.). Ensured it to be disabled by default, did not affect the build for
> anyone that does not enable it and the users have been enabling and
> using it on their own. So, does it differentiate more from vanilla? No.
>
> This helps users as well as maintainers to not have to apply BFQ on
> their own, they simply have to tick a config option and they are set.
> If all patches that introduce new features are added in this fashion,
> it spares large groups of users from having to do this on their own; we
> can also deduplicate the work in the Portage tree this way.
>
>
> ### ... and configuration. ###
>
> This problem is not only visible for patches, but also in the config.
>
> Meet CONFIG_DEVTMPFS; forget to enable it, greet a failing boot. We're
> telling users to enable it in some places, in the handbook it's a single
> line you must read, on the Wiki it's kind of missing unless you are
> luckily on the right page, on the Quick Install book it is missing too.
>
> So, we are currently providing a configuration that expects anyone to
> enable CONFIG_DEVTMPFS. But you have to remember that it need to make
> sure you read about it or enable it from the udev upgrade a while ago
> if you decide to start from a fresh config or are installing without
> that handbook you kind of have memorized.
>
> Searching for CONFIG_DEVTMPFS in the forums and #gentoo logs shows that
> this is quite often suggested as a fix and quite often actually fixes
> an user's boot. Why duplicate telling users to do that if we can do it?
>
> There are a small set of other variables in this nature, mostly *TMPFS*.
>
> This is why I think it would be handy to add a Gentoo section to the
> kernel, along the lines as described by Linus.
>
> https://lkml.org/lkml/2012/7/13/369
>
> The same goes for asking the user to enable configuration options in the
> kernel, why can't we just tell him to enable all option that toggles
> support for the user. For example; in the Gentoo section, there would be
> a "Init System Support" under which you can toggle an option to support
> the minimal requirements for some init system.
>
> Feedback is very welcome.
>
> P.S.: Not everything in this mail has been acked by the kernel lead;
> only some thoughts, I was suggested