Installing and Running Software Locally

Lately I’ve been installing a lot of software by hand into my system. I don’t have the willpower to figure out how to create a custom ebuild for each piece of software, and some software has a really, really odd installation that would require some kind of custom script or fix to fit it in / somewhere.

So, I started simple. I created a Software directory in my $HOME/Documents/Personal directory, and stashed anything I wanted that wasn’t readily available in there. For example, duplicity is updated often, and some newer versions aren’t available. I download the latest 0.5.12 version into a newly created Documents/Personal/Software/duplicity folder, and extract it. I follow the README, and build it.

Here’s where things get interesting. Just about any software you download will want to “make install” in some way. However, the last thing I want is files, which, remember, portage knows nothing about, cluttering up my root file system. The next best thing is to install with a prefix, as the README states. It works like this:

export PREFIX="$HOME/opt/"
python setup.py install --prefix="$PREFIX"

Assuming you created the $PREFIX directory first, you should see duplicity in $PREFIX. The next problem is how to run duplicity. You can’t just execute it because it isn’t in any directories mentioned in $PATH. This is simply remedied:

export PATH="$PATH:$PREFIX/bin"
export PYTHONPATH="$PYTHONPATH:$PREFIX/lib/python2.5/site-packages"
export MANPATH="$MANPATH:$PREFIX/man:$PREFIX/share/man"
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH:$PREFIX/lib/pkgconfig"

Duplicity only actually needs $PATH and $PYTHONPATH, but these are the variables that I have found all of my locally-applications need.

I should now be able to execute duplicity --version and get 0.5.12.

I used this for a year or so, quite happy with the way it was working out. I had several applications pushed in there, and they all ran perfectly. I created a “q_add-path” alias for the above list of export commands so that I wouldn’t have to run them by hand every time. However, a day came when portage actually caught up and I wanted to use the portage method in place of my locally installed applications so that it’d be easier to manage. The problem was, duplicity was embedded into the $PREFIX directory, and there was no uninstall command.

I find duplicity’s lack of uninstall command disturbing, but I also find it’s more common than you’d think. To remedy this, I figured I could just install duplicity into its own directory and redirect $PREFIX to that directory. Simple.

Ugh, but that was a hack. I’d have to create an alias for every single new piece of software. What a pain! There had to be a better way. Then I remembered unionfs. If you recall, unionfs installation was no piece of cake in Gentoo, and probably not in other operating systems. But I got it to work at last!

So, once unionfs was working, I set to work on a script that would handle automatically loading in whatever software I wanted. The results is this:

#!/bin/bash

all_base="$HOME/opt"
config_base="$all_base/config"
mount_base="$all_base/merged"
software_base="$all_base/software"

union_mount()
{
	if [[ "$1" == "" ]]; then
		return 1
	fi

	if [[ "`grep "$mount_base" /proc/mounts`"  == "" ]]; then
		sudo mount -t unionfs -o "ro,dirs=$software_base/base" none "$mount_base" || return $?
	fi
	if [[ "$1" == "base" ]]; then return 0; fi # mount base once!

	sudo mount -t unionfs -o "ro,remount,add=:$software_base/$1" none "$mount_base"
	return $?
}

union_unmount()
{
	if [[ "$1" == "" ]]; then
		return 1
	fi

	if [[ "`grep $mount_base /proc/mounts`"  == "" ]]; then
		echo "The entire unionfs is already unmounted!"
		exit 1
	fi
	if [[ "$1" == "--all" ]]; then
		sudo umount "$mount_base"
	else
		sudo mount -t unionfs -o "remount,del=$software_base/$1" none "$mount_base"
	fi
	return $?
}

while [[ "$1" != "" ]]; do
	if [[ "$1" == "mount" ]]; then
		specificmount="$2"
		if [[ "$specificmount" == "" || "$specificmount" == "--all" ]]; then
			specificmount=".*";
			echo "(Mounting all)"
		fi

		if [[ "$2" != "" ]]; then shift; fi

		# thanks to http://mandrivausers.org/index.php?showtopic=21998&st=0&p=164692&#entry164692
		old_IFS=$IFS
		IFS=$'\n'
		lines=($(cat "$config_base/mounts" | grep "\($software_base/\)*$specificmount\$"))
		IFS=$old_IFS

		for line in ${lines[@]}; do
			echo "Mounting $line"
			if [[ "`grep "$line" /proc/mounts`" == "" ]]; then
				union_mount "$line"
			else
				echo "$line is already mounted!"
			fi
		done
	elif [[ "$1" == "unmount" ]]; then
		specificmount="$2"
		if [[ "$specificmount" == "" || "$specificmount" == "--all" ]]; then
			echo "(Unmounting all)"
			union_unmount "--all" && echo "Done"
			exit $?
		else
			shift
		fi

		# thanks to http://mandrivausers.org/index.php?showtopic=21998&st=0&p=164692&#entry164692
		old_IFS=$IFS
		IFS=$'\n'
		lines=($(cat "$config_base/mounts" | grep "\($software_base/\)*$specificmount\$"))
		IFS=$old_IFS

		count="0"
		for line in ${lines[@]}; do
			echo "Unmounting $line"
			if [[ "`grep "$line" /proc/mounts`" != "" ]]; then
				union_unmount "$line" && let 'count += 1'
			else
				echo "$line is not mounted!"
			fi
		done

		if [[ "$count" -lt 1 ]]; then
			echo "There were no mounts to unmount."
		fi
	elif [[ "$1" == "refresh-software" ]]; then
		old_wd="`pwd`"
		cd "$software_base" || exit $?
		ls -1 || exit $?
		echo -n "Overwrite current list with this one? "
		read -sn1 answer
		if [[ "$answer" == "y" || "$answer" == "Y" ]]; then
			echo "[y]"
			ls -1 > "$config_base/mounts" || exit $?
			echo "Done"
		else
			echo "[n]"
		fi
		cd "$old_wd"
	else
		echo "Command not understood: $1"
		echo "Usage: `basename $0` 

 ...
"
		echo
		echo "Actions:"
		echo "    mount"
		echo "               * If param is blank or --all, all mounts are mounted."
		echo "               * If param is specified, that mount is mounted."
		echo "               * In all cases, base mount will be automatically mounted if it is"
		echo "                 not already."
		echo
		echo "    unmount"
		echo "               * If param is blank or --all, all mounts, including base unionfs"
		echo "                 mount, are unmounted."
		echo "               * If param is specified, that mount is unmounted."
		echo
		echo "    refresh-software"
		echo "               * Doesn't take any parameters."
		echo "               * Will refresh the mounts configuration file with new software,"
		echo "                 after checking with user that the new list is OK."
		exit 1
	fi
	shift
done

So, the basic idea is this:

  1. Create the $HOME/opt/ directory.
  2. Stash the above script into a file called unionfs-local-mount.
  3. Create three directories in opt: config, merged, and software.
  4. In software, create a directory called base. Whenever unionfs goes to mount anything, it always needs a base mount to start layering things on top of. This folder will be that base. Now, I put my custom scripts and whatnot in a different software folder called “custom”, but you could put your scripts in base if you felt like it.
  5. Now in software, create a directory to contain the software (something like “duplicity-0.5.12”).
  6. Install the software to that directory like I did when I installed straight to opt before.
  7. Now that your software directory has been changed, you should update the config directory. The unionfs script will handle this for you. Just execute “unionfs-local-mount refresh-software“.

You’ll want to adjust $PREFIX to look to $HOME/opt/merged now, because when you run…

unionfs-local-mount mount

…it will put every piece of software available into $PREFIX. If you want to remove a particular piece of software, you can always use the unmount command, with a second parameter containing the name of the software (say, “duplicity0.5.12”) and away it goes! I’ve provided some usage information in the script, which you can access by running:

unionfs-local-mount --help

This is basically what I use to manage software that I installed. I also created a script that manages dependencies of local software that I had to install to get it to work, but it’s not worth sharing. It’s easier to track deps by hand, really, in the end.

Advertisements
Post a comment or leave a trackback: Trackback URL.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: