Ever closed and opened a shell because you needed to refresh the history to pull up a command you used before? Nevermore! Read this excellent post on briancarper.net to make your terminal life much easier.

Duplicity has a wonderful design feature: it’s really really simple.

Duplicity has an annoying design flaw: it’s really really simple.

In case you didn’t notice, Duplicity’s simplicity is both helpful and inefficient. It’s extremely easy to start using Duplicity because its usage is so very simple. Want to back something up? Just write “duplicity /some/directory ftp://user@host.com/some/other/directory” for FTP backup, or “duplicity /some/directory file://some/other/directory” for local backup, or any other protocol out of the 11+ protocols it supports. All it does is write files to the location of your choosing, and then it can recover and list files (either latest or at a date of your choosing) from the backup directory you give it.

This simplicity is really great for the simple use-cases, like backing up a home directory. It’s when you get into databases, exclusion and inclusion rules, and other such fine print that you have to plan a little.

I’ve got three different “things” I have to back up: my web development stage, my remote shell (mainly for irssi), and my home directory. Each one presents its own challenges, which I break down below.

Backing up a Web Development Stage

There are two parts to my stage: a database, and a public_html directory. (The configuration of the server is handled by OSX. Otherwise, I’d be backing up configuration files as well.) Backing up directories is what Duplicity is designed to do, so that’s no issue. Backing up raw /var/mysql/ files is generally considered to be a bad idea, so what I do instead is generate a mysql dump file (“mysqldump -uroot -p”) and stash that into a temporary directory.

Note that the temporary directory is an entirely different place than the public_html directory, so I need to somehow tell Duplicity to back up two different locations at the same time. It’s actually not too difficult to figure out:

duplicity / --include /path/to/public_html --include /path/to/temp_directory --exclude '**' file://some/other/directory

Duplicity decides whether or not to include or exclude files by simply finding the first matching rule on the command-line. In this example, /path/to/temp_directory/mysql.dump matches, because everything in its directory (“temp_directory”) is getting backed up. Perfect. Also note that my public_html directory gets matched the same way. Any others that don’t match the first two inclusion rules will always match that ‘**’ exclusion rule, so that the only two things backed up are public_html and temp_directory.

The only bummer here is I have to re-dump the mysql.dump file to a temporary directory. For a long time I did this by hand, but it’s an extra step I’d rather be done automatically. Keep that in mind, because I’ll be coming back to this in a few sections.

Backing up a Remote Shell

Problem: Duplicity must back up from a local path. Solution: download the remote location to a local directory, and back that up. So, now backing up becomes a two step process, as the web development stage was:

rsync -zav user@host:/some/directory /path/to/temp_directory
duplicity /path/to/temp_directory file://some/other/directory

Again, I’d much rather send off a simple one-liner, and not have to worry about rsyncing and whatever else.

Backing up a Home Folder

Couldn’t get any simpler, so I’ll spend just a few lines on this topic:

duplicity /home/directory file://some/other/directory

And we’re done. This is the easiest yet. I personally also use a bunch of –exclude arguments to keep things like Trash, caches, and large ISO’s out.

Doing all the Above, Three Times a Week

Ugh. What a pain. I count at least five commands, if not more. How can I make this easier?

Easy! Write a bash script! That was the first thing I did. I wrote a script called “duplicity-home-folder.sh” and one called “duplicity-var-www.sh” and so on. Each one had a few simple copied-and-pasted checks, along with a usually-monstrous duplicity command, complete with verbosity and inclusion/exclusion arguments.

This worked on and off for a while. I was keeping up a reasonable habit of backing up, and it paid off quite a few times. When I moved to a new operating system, I simply backed up, installed, and “recovered”. The same applied to my move to OSX, though it was complicated by the fact that OSX is not Linux. The same will apply to my upgrade to Snow Leopard.

Now that I’m in OSX, though, I have a completely different set of data to back up. I began tweaking the simple bash scripts, but soon found I was actually rewriting them instead. Having to copy-and-paste code between them was a pain, so I tried to merge them. Then I realized I really ought to store the backup settings separate from this script so I could adjust and add settings without touching the script. That’s when I decided to use Python instead.

Here’s how my current configs/home-jacob directory looks, where I stash all my backup configuration: (generated by “find configs/home-jacob”)

configs/home-jacob
configs/home-jacob/args
configs/home-jacob/binary
configs/home-jacob/inclusion-rules.d
configs/home-jacob/inclusion-rules.d/00-always
configs/home-jacob/inclusion-rules.d/00-always/exclude
configs/home-jacob/inclusion-rules.d/05-duplicity
configs/home-jacob/inclusion-rules.d/05-duplicity/exclude
configs/home-jacob/inclusion-rules.d/06-gentoo-embedded-handbook
configs/home-jacob/inclusion-rules.d/06-gentoo-embedded-handbook/exclude
configs/home-jacob/inclusion-rules.d/06-gentoo-embedded-handbook/include
configs/home-jacob/inclusion-rules.d/07-gentoo-docs
configs/home-jacob/inclusion-rules.d/07-gentoo-docs/exclude
configs/home-jacob/inclusion-rules.d/10-documents
configs/home-jacob/inclusion-rules.d/10-documents/exclude
configs/home-jacob/inclusion-rules.d/10-documents/include
configs/home-jacob/inclusion-rules.d/15-chat
configs/home-jacob/inclusion-rules.d/15-chat/include
configs/home-jacob/inclusion-rules.d/40-odds-ends
configs/home-jacob/inclusion-rules.d/40-odds-ends/include
configs/home-jacob/inclusion-rules.d/50-gentoo
configs/home-jacob/inclusion-rules.d/50-gentoo/exclude
configs/home-jacob/inclusion-rules.d/50-gentoo/include
configs/home-jacob/inclusion-rules.d/60-security
configs/home-jacob/inclusion-rules.d/60-security/include
configs/home-jacob/inclusion-rules.d/70-settings
configs/home-jacob/inclusion-rules.d/70-settings/include
configs/home-jacob/inclusion-rules.d/75-gaming
configs/home-jacob/inclusion-rules.d/75-gaming/include
configs/home-jacob/inclusion-rules.d/99-everything
configs/home-jacob/inclusion-rules.d/99-everything/exclude
configs/home-jacob/location
configs/home-jacob/location/local
configs/home-jacob/location/remote
configs/home-jacob/post-run.d
configs/home-jacob/post-run.d/90-generate-checksums
configs/home-jacob/pre-run.d


As you can see, there’s quite a bit of flexibility now. All the include/exclude rules are broken out into separate, organized files. I can have my script run scripts before and after the backup occurs.

I’ll leave it to your imagination how I made the above three backup scenarios cut down into three simple one-liners.

Actually, I’ll help your imagination out a little bit by posting my “dirconfig.py” and “duplicity-quickstart.py” scripts. The first handles loading the above configuration, and the second handles executing duplicity and other scripts based on that configuration. These are my first attempts at Python, so be nice.

dirconfig.py

import os

class DirConfiguration:

	def __init__ (self, path, recursive=True, debug=False):
		if not os.path.exists(path):
			raise ValueError ('Path does not exist')
		elif not os.path.isdir(path):
			raise ValueError ('Path is not a directory')

		self.path = path
		self.recursive = recursive
		self.debug = debug
		self.config = self.__read_path(path, recursive=recursive)

	def __read_path (self, rootPath, config={}, recursive=True):
		paths = os.listdir(rootPath)
		for currPath in paths:
			currPath = rootPath + "/" + currPath
			key = os.path.basename(currPath)
			if self.debug: print 'this is for path', currPath

			if os.path.isdir(currPath) and recursive:
				value = self.__read_path(currPath, {}, recursive)
				if self.debug: print 'for key ' + key + ' storing child value:', value
				config[key] = value

			elif os.path.isfile(currPath):
				currFile = open(currPath, 'r')
				key = os.path.basename(currPath)

				if key == 'children':
					if self.debug: print 'Warning: illegal file name in "'+currPath+'":', key
				else:
					for currLine in currFile:
						if key not in config:
							config[key] = []

						if self.debug: print 'for key ' + key + ' storing:', currLine.strip()
						config[key].append(currLine.strip())

			else:
				if self.debug: print 'Notice: odd path exists in configuration directory:', currPath

		if self.debug: print 'returning this for path "' + rootPath + '":', config
		return config

duplicity-quickstart.py

#!/usr/bin/python

import sys
import subprocess
from optparse import OptionParser
from dirconfig import DirConfiguration

parser = OptionParser(usage='usage: %prog [options] config-dir', version='1.0')
parser.add_option('-v', '--verbose', dest='verbosity', help='verbosity level of duplicity (overrides args file)', type='int')
parser.add_option('-d', '--debug', dest='debug', help='turn on the debugger for the configuration parser', action='store_true', default=False)
parser.add_option('', '--force', dest='force', help='force duplicity to complete an action (most likely cleanup)', action='store_true', default=False)
parser.add_option('-a', '--action', dest='action', help='action duplicity ought to take', default='incremental', type='choice', choices=['full', 'incremental', 'restore', 'verify', 'collection-status', 'list-current-files', 'cleanup', 'remove-older-than', 'remove-all-but-n-full'])
parser.add_option('-f', '--file-to-restore', dest='restore_path', help='if restore is the action, this can determine which file specifically is restored', type='string')
parser.add_option('-r', '--restore-to', dest='restore_to', help='if restore is the action, this can determine where the restored files are stored', type='string')
parser.add_option('', '--allow-source-mismatch', dest='allow_mismatch', help='if the backup source changed, but you still want to use the same backup destination, and duplicity is complaining, use this', action='store_true', default=False)

(options, args) = parser.parse_args()

if len(args) < 1:
	raise ValueError('Error: only one extra argument required, to indicate the path for the backup configuration directory.')

configuration = DirConfiguration(args[0], True, options.debug).config

up_actions = ['full', 'incremental']
down_actions = ['restore', 'verify']
remote_actions = ['collection-status', 'list-current-files', 'cleanup', 'remove-older-than', 'remove-all-but-n-full']

duplicity_opts = []

if not configuration.has_key('binary'):
	raise ValueError('Error: backup directory has no binary file.')

duplicity_opts.extend(configuration.get('binary'))
duplicity_opts.append(options.action)

if configuration.has_key('args'):
	duplicity_opts.extend(configuration.get('args'))

if options.verbosity:
	duplicity_opts.extend(['-v', str(options.verbosity)])

if options.force:
	duplicity_opts.append('--force')

if options.allow_mismatch:
	duplicity_opts.append('--allow-source-mismatch')

if not configuration.has_key('location'):
	raise ValueError('Error: configuration directory has no location directory.')

elif not configuration.get('location').has_key('local'):
	raise ValueError("Error: configuration's location directory has no local file.")

elif not configuration.get('location').has_key('remote'):
	raise ValueError("Error: configuration's location directory has no remote file.")

local_path = configuration.get('location').get('local')[0]
remote_path = configuration.get('location').get('remote')[0]

if options.action == 'restore':
	if options.restore_to:
		local_path = options.restore_to
	if options.restore_path:
		duplicity_opts.extend(['--file-to-restore', options.restore_path])

if configuration.has_key('inclusion-rules.d') and options.action in up_actions or options.action == 'verify':
	for rule_cat, rules in sorted(configuration['inclusion-rules.d'].items()):
		for rule_type, values in sorted(rules.items()): # excludes always comes before includes
			if rule_type != 'include' and rule_type != 'exclude':
				sys.stderr.write('Warning: in category ' + str(rule_cat) + ', unkown rule type: ' + str(rule_type) + '\n')
				continue

			for value in values:
				if not value.startswith('#') and value.strip() != '':
					duplicity_opts.extend(['--'+rule_type, value.replace('%%local%%', local_path)])

if options.action in up_actions:
	duplicity_opts.append(local_path)
	duplicity_opts.append(remote_path)
elif options.action in down_actions:
	duplicity_opts.append(remote_path)
	duplicity_opts.append(local_path)
elif options.action in remote_actions:
	duplicity_opts.append(remote_path)
else:
	raise ValueError('Error: incorrect action. This is a bug: it should have been caught before now!')

if configuration.has_key('pre-run.d'):
	for script, script_contents in sorted(configuration['pre-run.d'].items()):
		print 'Calling pre-run script: ' + args[0]+'/pre-run.d/' + script
		return_code = subprocess.call(['sh', args[0]+'/pre-run.d/' + script, local_path, remote_path, options.action])
		if return_code != 0:
			raise Exception('subprocess returned non-zero code ' + str(return_code))

print 'Calling: ' + ' '.join(map(str, duplicity_opts))
return_code = subprocess.call(duplicity_opts)
if (return_code != 0):
	raise Exception('duplicity exited with a non-zero code: ' + str(return_code))

if configuration.has_key('post-run.d'):
	for script, script_contents in sorted(configuration['post-run.d'].items()):
		print 'Calling post-run script: ' + args[0]+'/post-run.d/' + script
		return_code = subprocess.call(['sh', args[0]+'/post-run.d/' + script, local_path, remote_path, options.action])
		if return_code != 0:
			raise Exception('subprocess returned non-zero code ' + str(return_code))

I don’t know whether or not Firefox even allows JavaScript access to the passwords automatically filled out when you visit a page, but I sure am not going to wait until some cracker finds a hole in whatever security Firefox provides and gets at my automatically-entered password.

So, this time around I’ve decided to require Firefox to wait until I’ve entered the username before the password is filled out. If I forget the username, I simply hit the down arrow in the username box and it gives me a selection.

The setting is located at about:config (for those who don’t know: type that into the address bar) and is called “signon.autofillForms”. I set it to false. Long may security prosper!

Hello readers! It’s been a while, hasn’t it, since I’ve made a good post about how to do something or other? That’s usually a good indication that I’ve been busy with something or other, and have been using my system as-is without much time for exploration.

Well, I want to let anyone who reads this to know I’m still alive and kicking. I’ve got two posts planned (one about how to compile Pidgin plugins in Gentoo Prefix properly, and another about how I use duplicity with 12+ options efficiently) so stay tuned.

What’s been keeping me busy? College, that’s what. This is my first year, so I’m still finding my feet. Even now I’m still experimenting with homework schedules, varying between solid week of work and lazy afternoons. At some point I will find a happy medium, I hope.

The Neuvoo project started out alive and strong, and I haven’t been able to check my idling IRC client in days, so I hope they’re doing well and well on their way to producing their first release.

Oh, and thanks for reading.

(I was so excited when I watched the Google Wave video that I had to write this while I watched. That’ll explain the overload of optimism and excitement.)

Google seems to do all the obvious things: make free e-mail with an excellent spam filter, take the existing phone system and digitize it a little, make an online word processor and collaboration, and provide a really nice search engine.

This year, Google is doing it again. Two engineers in Google’s forces stepped back and examined the world of communication we’ve built. We all know that e-mail is ancient and inefficient, and they knew that too. So, they decided to completely reinvent the entire communication system.

It’s called Google Wave. It’s really simple.

Instead of thinking of communication as messages that are literally shoved around from place to place, Google Wave thinks of messages as being part of a conversation, which doesn’t move anywhere. Rather, a conversation between two (or more!) people occurs in a single, shared location.

Messages can be very long or very short. You can send long paragraphs of text in an e-mail format, or you can communicate back and forth in a texting-like format. There is no limits to what you can do with these messages.

If two participants of a conversation are online at the same time, they can instantly see when someone’s sent a new message or started a new conversation. If two participants are viewing the same conversation, they can see messages update instantly on each other’s screens, the same way instant messaging does. Google Wave goes even further: if a participant is typing a reply, any other participant can see each character as it’s being typed.

Google Wave also allows people to reply to specific paragraphs of a message, so that threading is tracked very efficiently.

In a conversation with three or more people, but you want to reply privately to only one person so that only he/she can see it? Just let Google Wave know. It can change the permissions on any branch of the conversation.

Came late to a conversation, and now the conversation is flooded with messages? Use the playback feature to see each message come up one at a time in the order they were posted, so you can see the conversation evolve in real-time, even though you weren’t present when it happened!

Want to share photos? Drag and drop photos right into the wave. Thumbnails appear instantly on everyone’s screen while the full-sized images upload in the background. Suddenly a shared photo album is constructed. Effortlessly.

Now, embed a conversation on your blog. Whoa, wait a sec’! Google Wave is now in your blog! Remember how you can reply to a paragraph, etc., in a conversation? Guess what Wave can do for your blog? You guessed it! The comments system becomes instantly available through Wave’s interface on your blog. And not only that, but Wave considers this blog “post” as simply another conversation, so it appears in Google Wave as a conversation with normal replies as well!

Google Wave is now no longer just an e-mail client and instant messenger rolled into one, but now it’s your blog too.

Revolution, folks.

Now, go ahead and dream. Dream that this all runs on your iPhone. Or Google Android. All the above features, now available here. Sheer. Brilliance.

Did we forget something? Oh, yes. Right. We forgot editing. How about we let everyone edit any message in a conversation. That’s right. Any message.

Now Google Wave is all of the above and a wiki.

What about history? How do we see what someone originally wrote? Did you forget about the playback feature? Oh yea. Oh yea. That’s right. You can play back edits too!

Everyone can edit at the same time too! Take notes from a class together, have one person go to a meeting, and let others submit comments or questions from the outside, and so on.

Linking between waves. Tagging. Folders. Saved searches. Organization is simple with Google Waves.

Now, throw in extensions (like Firefox Add-ons) that can modify anything the internal code can, from characters you type to messages you send to conversations created, and you have yourself a revolution. Games, movies, and all other sorts of things can be done socially, recorded, for playback later, through extended Wave seamlessly.

Do you want to see this in action? Betcha you do. Go here:
http://wave.google.com/help/wave/about.html

https://docs.google.com, when used with Google Gears turned on and allowed, will result in some kind of unreported error in the Gears backend, leaving its users with a blank page. I have a feeling that Gears cannot handle HTTPS connections, but I’m not sure. It could be just a Safari and/or OSX issue.

Others have reported endless authentication redirection, which makes sense if you study the URL that the HTTPS version always gets stuck on. (There’s a “redirect” parameter which points to https://docs.google.com/?pli=1, which, if followed manually, simply circles back around through the authentication again and sticks back at the same old blank page.) Hopefully Google takes notice and fixes this soon.

This is very handy if you’re working on Gentoo documentation, which is composed of XML that gets translated into HTML.

Alright, enough with the introductions. I’m just going to write how I got gorg to work.

1. Install gorg. If you’re using Gentoo Prefix, and bug #281335 is not solved yet, use my Gentoo Prefix overlay to get it installed. (I only tested on OSX, though.)
2. Copy /etc/gorg/gorg.conf.sample to gorg.conf, and begin editing it.
3. Change the root variable to point to the directory where your copy of the Gentoo site will reside. I changed mine to:
   root = "/Users/jacob/Sites/gentoo"
4. I’m not sure if this is required, or what this even does, but I changed the two mount variables to match root:
   mount = /cgi-bin on /Users/jacob/Sites/gentoo/xml/cgi-bin
mount = /images on /Users/jacob/Sites/gentoo/xml/images
5. I left everything else to defaults. Save, exit. The other sample files should/can be left as samples. It doesn’t affect gorg in any way negative, at least as far as I can tell.
6. Now you’ll need to make a monster of a checkout:
   cvs -d :pserver:anonymous@anoncvs.gentoo.org:/var/cvsroot co gentoo/xml/htdocs
7. When that’s done, either move the htdocs folder to the place indicated by root, or make a symlink. I opted for a symlink. This allows my gorg environment and my programming environment to remain at least organizationally (that’s a word!) separate.
8. Run the following, replacing the “/User/jacob/Sites/gentoo” part with wherever you told root to point:
   xmlcatalog --create /Users/jacob/Sites/gentoo/index.xml >> /etc/xml/catalog
9. You should be viewing Gentoo’s home page, albeit with images and other things missing. Congrats!

Now, to finish off the work on Gentoo Embedded’s documentation…

I read a blog post recently on Planet Larry, which describes only one facet of how cell-phones are disgusting. It reminded me that I’ll have to get one soon. All the same, it’s nice to see I’m not the only one who hates it.

It feels like the cell-phone world is moving at a snails pace. Sure, the iPhone stepped it up a notch. It introduced a certain amount of flexibility never before seen in cell-phones. However, the price for it is unbelievable, it’s still using the ancient idea of calling numbers, which are hardly memorable, it’s still locked down so that you can’t do anything creative with it, and the service itself won’t work with any device except the phones provided.

Why can’t cell-phone companies open their API’s so that more hardware can be produced which can log into and use the cell-phone system? Why can’t the cell-phone system either take on more features, or back off and start over with a better foundation for generic data (like mobile broadband already does) so that more information than just calls can be sent over the network? It seems like a no-brainer, and a way to make your network very, very popular.

I wonder if the iPhone is such a success because people feel the limitations of their phones. I think more can be done to break us free of a locked down system that is rigid and hardly feature-rich anymore. Unfortunately, people find it hard to get out of certain ruts when it comes to technology and move on to better systems. The current system is easy enough to use and can do just enough to stay useful. That’s going to make it hard to move on, and harder for companies to see the point.

That’s the approach I’m taking when it comes to Gentoo right now. September, my deadline for all computer projects, is my hard deadline, because that’s when college starts, and that’s when I expect my free time will be dramatically shortened, if not entirely consumed, as far as my computer projects are concerned. Because of that, I can’t waste a lot of time patching and hacking. If it doesn’t work in Gentoo Prefix like I want it to, I’ll throw it into VirtualBox’s copy of Gentoo and use the Shared Folders feature to keep data synced in and out of the virtual machine.

First off, don’t even think about doing this if you don’t have x86 virtualization extensions. You may survive the experience, but you won’t be able to stand it for much longer after. On my ol’ 1.8GHz Pentium 4, the speeds were always at least twice as slow. With my new Core 2 Duo processor with VT-x, the virtual machine can let the real processor take on most of the processing needs. If I had a “Core i7″ Intel processor, or something else with Nehalem, I would also have Nested Paging which can give as much as a 1/3 increase in speeds.

But never-mind, because VT-x gives me near-native performance, which is absolutely necessary for Gentoo. Another good thing to have is a bigger-than-8G hard-drive. (In other words, the default Linux size is not good enough.) I gave the machine 512M out of my 4G total, which is more than enough for a Linux desktop. I turned on 3D acceleration and gave it 64M of VRAM, because I want to try KDE4.3. Everything else are defaults, pretty much.

The kernel is pretty easy to figure out. AC97 for audio, and Core 2 Duo as my processor works. Most of the info on the kernel can be found here.

The one snag I hit that I couldn’t figure out for a while was GRUB. Apparently GRUB v0.97 is extremely old and no longer supported. GRUB2 is a much better idea. I’m running grub-9999 no problems, but that’s probably more due to luck than anything.

Besides that, not a whole lot to talk about. I’ve got the desktop profile enabled, and am recompiling everything with that before I autounmask what remains of KDE4.3 after I used kde-testing’s package.unmask and package.keywords files. Things are compiling right now, which is why I had time to write this.

My last blog post was the first one I’ve really just started writing on a feeling, and expressed it in full. Unfortunately, what came off in the post was all negative, without much hope for resolution, or even without anything positive about the projects mentioned.

So, here is a continuation of the last blog post I’ve written, in which I go over again what was bad, but also what was good, and how the bad can be corrected. Probably what the first post should’ve been.

The first thing I discussed was Gentoo Embedded, and its perceived “unfriendliness”. A big component of this feeling, I pointed out earlier, was a lack of documentation. This was too strong, and even wrong. The documentation does in fact cover most of the concepts around cross-compiling that’s important for getting started, and discusses how to build a cross-compiler from scratch. It fails to mention crossdev-wrappers, but it is not required for success. This is where the documentation ends, and leaves the reader to decide where to go from there. Most Gentoo users probably ought to know what to do to get the system they want, but it could be helpful to provide pointers from there. The documentation also doesn’t exactly flow from page to page, requiring a little bit of experimentation and extra thought.

Beyond documentation, though, there’s another problem that Gentoo Embedded deals with on a daily basis: cross-compiling, currently, is a huge topic in Linux all to itself that someone could make a career out of. There are so many ins and outs to choosing a cross-compiler, to picking USE flags, versions, and whatever else that it’s difficult to know it all. And knowledge is what is required to point someone in a particular direction without throwing them into a pit. (If I were to try and help someone, it’d probably end up being blind leading the blind, and all that). Only a few in #gentoo-embedded, not including me, have that kind of knowledge, and they aren’t always online or available to assist. (Maybe they’re all hiding in the mailing lists… I ought to sign up for those!) Those who are online are likely busy with other things, like work, and can’t get heavily involved or give heavily involved answers.

So what can be done to fix this? Well, it’s hard to correct the personnel issue immediately. In reality, it’s completely out of Gentoo Embedded’s control. However, the documentation can, and will, be corrected, if not by other volunteers who have already stepped forward, then by me (or, both). By idling on the channel, and paying attention to which questions go unanswered, or the general trend of questions, it should be possible to further customize the documentation in that direction, and further lighten the load on the channel.

To summarize, Gentoo Embedded covers a huge topic that is perhaps too much for the amount of knowledgeable people idling in the channel, but hopefully a change in documentation, which is coming, and a continual evolution to best fit the needs of Gentoo Embedded users, will solve all that. (This is where my discussion of how hard it is to change documentation comes in: compared to a wiki, the procedure required to get changes included is very difficult. That was the primary thrust of my previous discussion of Gentoo Embedded.)

The next part of my blog was about Gentoo Prefix. They discuss a much smaller topic, so their documentation was easier to write, and it was a lot easier to get into. However, it was when I started with Gentoo Prefix that I noticed a redundancy in Gentoo projects as a whole, and not just in Gentoo Embedded or Gentoo Prefix. This is where I dove into the flaws of the project system. Now, it’s good to note that I really like the way Gentoo sets up projects: if you have an idea for a big new feature or addition to Gentoo, you can make a project and develop it. You get your own CVS, web space, channel, mailing list, and so on, all within the Gentoo umbrella. This is beautiful.

So where does it go wrong? In a very subtle area: integration into official Gentoo. It seems (remember, I speak in opinions, not in facts) that many Gentoo projects are excluded from pushing their ideas back into main Gentoo for everyone who doesn’t already know about the project to enjoy. The benefits of integrating the mothership with its projects are: the Gentoo projects no longer has to reinvent the wheel, or constantly work around it; the Gentoo projects get recognition from the main Gentoo devs, who can then work with, rather than instead of, the projects.

How can we fix this? Get Gentoo developers to work with each-other more. Get people to meet and discuss how Gentoo can be better consolidated and made more streamlined. Really this is what I’m pushing for in a sentence: Streamline Gentoo.

This problem, perhaps, isn’t as widespread as I might think. Maybe it’s really just decisions high up for the betterment of the community. For example, they might’ve decided that, since 95% of the community doesn’t use it, we ought to leave it separate. The Gentoo documentation process (and thus the Wiki) could exist for the necessities of Q/A, issues that I’m not knowledgeable about.

I will continue to look around and see if I’m right or wrong, and what I can do to solve the issues I do find. I’ll keep writing.