I'd vote for option 5. Thanks so much for bringing this up. Devin Prater r.d.t.pra...@gmail.com
On Mon, Apr 18, 2022 at 7:28 PM Steve McIntyre <st...@einval.com> wrote: > TL;DR: firmware support in Debian sucks, and we need to change this. See > the > "My preference, and rationale" Section below. > > In my opinion, the way we deal with (non-free) firmware in Debian is a > mess, > and this is hurting many of our users daily. For a long time we've been > pretending that supporting and including (non-free) firmware on Debian > systems > is not necessary. We don't want to have to provide (non-free) firmware to > our > users, and in an ideal world we wouldn't need to. However, it's very > clearly no > longer a sensible path when trying to support lots of common current > hardware. > > Background - why has (non-free) firmware become an issue? > ========================================================= > > Firmware is the low-level software that's designed to make hardware devices > work. Firmware is tightly coupled to the hardware, exposing its features, > providing higher-level functionality and interfaces for other software to > use. > For a variety of reasons, it's typically not Free Software. > > For Debian's purposes, we typically separate firmware from software by > considering where the code executes (does it run on a separate processor? > Is it > visible to the host OS?) but it can be difficult to define a single > reliable > dividing line here. Consider the Intel/AMD CPU microcode packages, or the > U-Boot firmware packages as examples. > > In times past, all necessary firmware would normally be included directly > in > devices / expansion cards by their vendors. Over time, however, it has > become > more and more attractive (and therefore more common) for device > manufacturers > to not include complete firmware on all devices. Instead, some devices just > embed a very simple set of firmware that allows for upload of a more > complete > firmware "blob" into memory. Device drivers are then expected to provide > that > blob during device initialisation. > > There are a couple of key drivers for this change: > > • Cost: it's typically cheaper to fit smaller flash memory (or no flash > at > all) onto a device. The cost difference may seem small in many cases, > but > reducing the bill of materials (BOM) even by a few cents can make a > substantial difference to the economics of a product. For most vendors, > they will have to implement device drivers anyway and it's not > difficult to > include firmware in that driver. > > • Flexibility: it's much easier to change the behaviour of a device by > simply > changing to a different blob. This can potentially cover lots of > different > use cases: > > □ separating deadlines for hardware and software in manufacturing > (drivers and firmware can be written and shipped later); > □ bug fixes and security updates (e.g. CPU microcode changes); > □ changing configuration of a device for different users or products > (e.g. potentially different firmware to enable different > frequencies on > a radio product); > □ changing fundamental device operation (e.g. switching between RAID > and > JBOD functionality on a disk controller). > > Due to these reasons, more and more devices in a typical computer now need > firmware to be uploaded at runtime for them to function correctly. This has > grown: > > • Going back 10 years or so, most computers only needed firmware uploads > to > make WiFi hardware work. > > • A growing number of wired network adapters now demand firmware uploads. > Some will work in a limited way but depend on extra firmware to allow > advanced features like TCP segmentation offload (TSO). Others will > refuse > to work at all without a firmware upload. > > • More and more graphics adapters now also want firmware uploads to > provide > any non-basic functions. A simple basic (S)VGA-compatible framebuffer > is > not enough for most users these days; modern desktops expect 3D > acceleration, and a lot of current hardware will not provide that > without > extra firmware. > > • Current generations of common Intel-based laptops also need firmware > uploads to make audio work (see the firmware-sof-signed package). > > At the beginning of this timeline, a typical Debian user would be able to > use > almost all of their computer's hardware without needing any firmware > blobs. It > might have been inconvenient to not be able to use the WiFi, but most > laptops > had wired ethernet anyway. The WiFi could always be enabled and configured > after installation. > > Today, a user with a new laptop from most vendors will struggle to use it > at > all with our firmware-free Debian installation media. Modern laptops > normally > don't come with wired ethernet now. There won't be any usable graphics on > the > laptop's screen. A visually-impaired user won't get any audio prompts. > These > experiences are not acceptable, by any measure. There are new computers > still > available for purchase today which don't need firmware to be uploaded, but > they > are growing less and less common. > > Current state of firmware in Debian > =================================== > > For clarity: obviously not all devices need extra firmware uploading like > this. > There are many devices that depend on firmware for operation, but we never > have > to think about them in normal circumstances. The code is not likely to be > Free > Software, but it's not something that we in Debian must spend our time on > as > we're not distributing that code ourselves. Our problems come when our user > needs extra firmware to make their computer work, and they need/expect us > to > provide it. > > We have a small set of Free firmware binaries included in Debian main, and > these are included on our installation and live media. This is great - we > all > love Free Software and this works. > > However, there are many more firmware binaries that are not Free. If we are > legally able to redistribute those binaries, we package them up and include > them in the non-free section of the archive. As Free Software developers, > we > don't like providing or supporting non-free software for our users, but we > acknowledge that it's sometimes a necessary thing for them. This tension is > acknowledged in the Debian Free Software Guidelines. > > This tension extends to our installation and live media. As non-free is > officially not considered part of Debian, our official media cannot include > anything from non-free. This has been a deliberate policy for many years. > Instead, we have for some time been building a limited parallel set of > "unofficial non-free" images which include non-free firmware. These > non-free > images are produced by the same software that we use for the official > images, > and by the same team. > > There are a number of issues here that make developers and users unhappy: > > 1. Building, testing and publishing two sets of images takes more effort. > 2. We don't really want to be providing non-free images at all, from a > philosophy point of view. So we mainly promote and advertise the > preferred > official free images. That can be a cause of confusion for users. We do > link to the non-free images in various places, but they're not so easy > to > find. > 3. Using non-free installation media will cause more installations to use > non-free software by default. That's not a great story for us, and we > may > end up with more of our users using non-free software and believing > that > it's all part of Debian. > 4. A number of users and developers complain that we're wasting their > time by > publishing official images that are just not useful for a lot (a > majority?) > of users. > > We should do better than this. > > Options > ======= > > The status quo is a mess, and I believe we can and should do things > differently. > > I see several possible options that the images team can choose from here. > However, several of these options could undermine the principles of > Debian. We > don't want to make fundamental changes like that without the clear backing > of > the wider project. That's why I'm writing this... > > 1. Keep the existing setup. It's horrible, but maybe it's the best we can > do? > (I hope not!) > > 2. We could just stop providing the non-free unofficial images altogether. > That's not really a promising route to follow - we'd be making it even > harder for users to install our software. While ideologically pure, > it's > not going to advance the cause of Free Software. > > 3. We could stop pretending that the non-free images are unofficial, and > maybe > move them alongside the normal free images so they're published > together. > This would make them easier to find for people that need them, but is > likely to cause users to question why we still make any images without > firmware if they're otherwise identical. > > 4. The images team technically could simply include non-free into the > official > images, and add firmware packages to the input lists for those images. > However, that would still leave us with problem 3 from above (non-free > generally enabled on most installations). > > 5. We could split out the non-free firmware packages into a new > non-free-firmware component in the archive, and allow a specific > exception > only to allow inclusion of those packages on our official media. We > would > then generate only one set of official media, including those non-free > firmware packages. > > (We've already seen various suggestions in recent years to split up the > non-free component of the archive like this, for example into > non-free-firmware, non-free-doc, non-free-drivers, etc. Disagreement > (bike-shedding?) about the split caused us to not make any progress on > this. I believe this project should be picked up and completed. We > don't > have to make a perfect solution here immediately, just something that > works > well enough for our needs today. We can always tweak and improve the > setup > incrementally if that's needed.) > > These are the most likely possible options, in my opinion. If you have a > better > suggestion, please let us know! > > I'd like to take this set of options to a GR, and do it soon. I want to > get a > clear decision from the wider Debian project as to how to organise > firmware and > installation images. If we do end up changing how we do things, I want a > clear > mandate from the project to do that. > > My preference, and rationale > ============================ > > Mainly, I want to see how the project as a whole feels here - this is a big > issue that we're overdue solving. > > What would I choose to do? My personal preference would be to go with > option 5: > split the non-free firmware into a special new component and include that > on > official media. > > Does that make me a sellout? I don't think so. I've been passionately > supporting and developing Free Software for more than half my life. My > philosophy here has not changed. However, this is a complex and nuanced > situation. I firmly believe that sharing software freedom with our users > comes > with a responsibility to also make our software useful. If users can't > easily > install and use Debian, that helps nobody. > > By splitting things out here, we would enable users to install and use > Debian > on their hardware, without promoting/pushing higher-level non-free > software in > general. I think that's a reasonable compromise. This is simply a change to > recognise that hardware requirements have moved on over the years. > > Further work > ============ > > If we do go with the changes in option 5, there are other things we could > do > here for better control of and information about non-free firmware: > > 1. Along with adding non-free firmware onto media, when the installer (or > live > image) runs, we should make it clear exactly which firmware packages > have > been used/installed to support detected hardware. We could link to docs > about each, and maybe also to projects working on Free > re-implementations. > > 2. Add an option at boot to explicitly disable the use of the non-free > firmware packages, so that users can choose to avoid them. > > Acknowledgements > ================ > > Thanks to people who reviewed earlier versions of this document and/or made > suggestions for improvement, in particular: > > • Cyril Brulebois > • Matthew Garrett > • David Leggett > • Martin Michlmayr > • Andy Simpkins > • Neil Williams > > -- > Steve McIntyre, Cambridge, UK. > st...@einval.com > Who needs computer imagery when you've got Brian Blessed? >
