(Edit: Please read my next blog post after or in stead of reading this one.)
Here we are, with working images, and we’ve begun to look into how to better make our images and the contents within more like (and simultaneously more compatible with) official Gentoo. A lot of that has involved us using tools provided by official Gentoo, such as catalyst and eselect. A lot of things, however, cannot be accomplished with plain Gentoo, such as catching portage before it tries to emerge something to make sure the file-system is set up correctly. It took us over half a year to finally “catch up” with Gentoo Embedded and actually reach the limits of what Gentoo provides officially.
Let’s step back a month or so, and go down another path: Gentoo Prefix. This excellent project gives people like me hope for Gentoo where there is none. I installed this to OSX, and I have had all kinds of fun hacking my programs into existence, as I’m sure my readers know from previous posts about Pidgin. In fact, it is because of Prefix that I have begun to scratch the surface of the ebuild world with many a Prefix bug report which often (I think) ended up in an ebuild/patch submission.
My experience with each project has been somewhat different, and somewhat disturbing too. What I have found with both projects, I believe I can trace straight up into Gentoo, and may be what is driving more than a few users and developers away from the once-very-popular distribution. I will attempt to describe what I have found, but please keep in mind that, as authoritative as I may sound, all of what follows is largely my own opinion based on what I’ve seen.
The first thing that struck me when I joined the channel for Gentoo Embedded and asked a few questions, was not the lack of response, because I did receive responses. It was a lack of will to help newcomers understand. In other words, they appeared unfriendly because they failed to explain the concepts around cross-compiling, which is a major, if not critical, component of Gentoo Embedded. Their documentation, which they might and have pointed people to, does not help much because it is very vague about the best methods to use to cross-compile. It all contributes towards that feeling of unfriendliness: both their documentation and the people in the channel fail to explain fully to newcomers how exactly to get started on building cross-compiled software. The Gentoo Embedded developers themselves acknowledge that the world of cross-compiling is “a black art, requiring a lot of research,” and part of the issue is “a shortage of documentation”. It’s too bad, then, that they fail to fill this need completely. (Edit: see my next blog post.)
(My experience with Gentoo Prefix was marginally better. Their documentation, while scraggly and rough, nevertheless got me started on the right foot with some solid instructions.)
It wasn’t until I saw heavy redundancy in Gentoo Prefix that I noticed a pattern in what I had been experiencing. Suddenly the very unofficial Gentoo Wiki made sense. Suddenly the very reason that Gentoo Prefix even existed, and existed as an official Gentoo “project”, made sense. The pattern I discovered is not a lack of new ideas, but a resistance to new ideas. Gentoo itself seems to only run after what other distributions have long been developing.
Think about it: Gentoo Prefix, in reality, ought not to be its own project. It is actually a mirror of Gentoo, and has its own version of portage and its own version of the portage tree. Gentoo Prefix is really Gentoo with the added ability to install to any directory, rather than just /. However, it continues to be on its own as a separate branch. Why? Both because official Gentoo is adverse to accepting Gentoo Prefix, since it requires quite a lot of patching, and maybe also because Gentoo Prefix developers are busy enough as it is just trying to keep up.
Another example: Gentoo Embedded’s purpose is to simply to build stage3’s and maintain stability across all architectures as best as possible. Each day, their battle is to fight bugs that appear in new versions of software, and to try and make building these stage3’s as easy as possible so their automated builders can do it. Everything I have discovered about this project always falls into this category entirely. But wait! Isn’t main Gentoo already doing that? Why does this project exist outside of Gentoo? Why is it difficult to merge support for these architectures back into main Gentoo?
Is it because, as I suspect, Gentoo is locked down a bit too much, and requires too much effort to get new ideas implemented, compared to other distributions? Is it because of the long procedures required to release new documentation that the Gentoo Wiki exists?
My last example: In my recent attempt to implement a (relatively) small feature into portage, I received responses that would’ve driven me away from the idea and towards hackish solutions had I not planted my feet firmly on the ground of reasons which are so far uncontested.
In all my experiences since I first started pulling back the layers of the Gentoo development world, this is what I have discovered: the amount of work required to get people to accept new ideas in Gentoo is enormous. I wonder if it is because of this that Gentoo is not innovating as fast as other distributions, or attracting developers as I heard it once did. I wonder if Gentoo is trying protect themselves from bad ideas, and thus also protecting themselves from good ideas. My experiences, and my speculation, may be entirely off-base, but at the same time I have a hard time believing that all the above difficulties of developing for Gentoo are merely coincidental.
What makes this whole thing even worse, for me anyway, is that I cannot be the one to volunteer and make all this happen. And anyway, in the end, I don’t think it’ll come down to one person, but the entire community of developers. One of the lead developers for Gentoo Embedded (solar) seems to understand this issue and is himself pushing for an easier process to become a Gentoo developer. Hopefully that’s only the beginning for an evolution in Gentoo policies. It’s pretty important it happens. After all, the future of Gentoo development depends on it.