Lately I’ve been installing a lot of software by hand into my system. I don’t have the willpower to figure out how to create a custom ebuild for each piece of software, and some software has a really, really odd installation that would require some kind of custom script or fix to fit it in / somewhere.
So, I started simple. I created a Software directory in my $HOME/Documents/Personal directory, and stashed anything I wanted that wasn’t readily available in there. For example, duplicity is updated often, and some newer versions aren’t available. I download the latest 0.5.12 version into a newly created Documents/Personal/Software/duplicity folder, and extract it. I follow the README, and build it.
Here’s where things get interesting. Just about any software you download will want to “make install” in some way. However, the last thing I want is files, which, remember, portage knows nothing about, cluttering up my root file system. The next best thing is to install with a prefix, as the README states. It works like this:
export PREFIX="$HOME/opt/"
python setup.py install --prefix="$PREFIX"
Assuming you created the $PREFIX directory first, you should see duplicity in $PREFIX. The next problem is how to run duplicity. You can’t just execute it because it isn’t in any directories mentioned in $PATH. This is simply remedied:
export PATH="$PATH:$PREFIX/bin"
export PYTHONPATH="$PYTHONPATH:$PREFIX/lib/python2.5/site-packages"
export MANPATH="$MANPATH:$PREFIX/man:$PREFIX/share/man"
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH:$PREFIX/lib/pkgconfig"
Duplicity only actually needs $PATH and $PYTHONPATH, but these are the variables that I have found all of my locally-applications need.
I should now be able to execute duplicity --version and get 0.5.12.
I used this for a year or so, quite happy with the way it was working out. I had several applications pushed in there, and they all ran perfectly. I created a “q_add-path” alias for the above list of export commands so that I wouldn’t have to run them by hand every time. However, a day came when portage actually caught up and I wanted to use the portage method in place of my locally installed applications so that it’d be easier to manage. The problem was, duplicity was embedded into the $PREFIX directory, and there was no uninstall command.
I find duplicity’s lack of uninstall command disturbing, but I also find it’s more common than you’d think. To remedy this, I figured I could just install duplicity into its own directory and redirect $PREFIX to that directory. Simple.
Ugh, but that was a hack. I’d have to create an alias for every single new piece of software. What a pain! There had to be a better way. Then I remembered unionfs. If you recall, unionfs installation was no piece of cake in Gentoo, and probably not in other operating systems. But I got it to work at last!
So, once unionfs was working, I set to work on a script that would handle automatically loading in whatever software I wanted. The results is this:
#!/bin/bash all_base="$HOME/opt" config_base="$all_base/config" mount_base="$all_base/merged" software_base="$all_base/software" union_mount() { if [[ "$1" == "" ]]; then return 1 fi if [[ "`grep "$mount_base" /proc/mounts`" == "" ]]; then sudo mount -t unionfs -o "ro,dirs=$software_base/base" none "$mount_base" || return $? fi if [[ "$1" == "base" ]]; then return 0; fi # mount base once! sudo mount -t unionfs -o "ro,remount,add=:$software_base/$1" none "$mount_base" return $? } union_unmount() { if [[ "$1" == "" ]]; then return 1 fi if [[ "`grep $mount_base /proc/mounts`" == "" ]]; then echo "The entire unionfs is already unmounted!" exit 1 fi if [[ "$1" == "--all" ]]; then sudo umount "$mount_base" else sudo mount -t unionfs -o "remount,del=$software_base/$1" none "$mount_base" fi return $? } while [[ "$1" != "" ]]; do if [[ "$1" == "mount" ]]; then specificmount="$2" if [[ "$specificmount" == "" || "$specificmount" == "--all" ]]; then specificmount=".*"; echo "(Mounting all)" fi if [[ "$2" != "" ]]; then shift; fi # thanks to http://mandrivausers.org/index.php?showtopic=21998&st=0&p=164692&#entry164692 old_IFS=$IFS IFS=$'\n' lines=($(cat "$config_base/mounts" | grep "\($software_base/\)*$specificmount\$")) IFS=$old_IFS for line in ${lines[@]}; do echo "Mounting $line" if [[ "`grep "$line" /proc/mounts`" == "" ]]; then union_mount "$line" else echo "$line is already mounted!" fi done elif [[ "$1" == "unmount" ]]; then specificmount="$2" if [[ "$specificmount" == "" || "$specificmount" == "--all" ]]; then echo "(Unmounting all)" union_unmount "--all" && echo "Done" exit $? else shift fi # thanks to http://mandrivausers.org/index.php?showtopic=21998&st=0&p=164692&#entry164692 old_IFS=$IFS IFS=$'\n' lines=($(cat "$config_base/mounts" | grep "\($software_base/\)*$specificmount\$")) IFS=$old_IFS count="0" for line in ${lines[@]}; do echo "Unmounting $line" if [[ "`grep "$line" /proc/mounts`" != "" ]]; then union_unmount "$line" && let 'count += 1' else echo "$line is not mounted!" fi done if [[ "$count" -lt 1 ]]; then echo "There were no mounts to unmount." fi elif [[ "$1" == "refresh-software" ]]; then old_wd="`pwd`" cd "$software_base" || exit $? ls -1 || exit $? echo -n "Overwrite current list with this one? " read -sn1 answer if [[ "$answer" == "y" || "$answer" == "Y" ]]; then echo "[y]" ls -1 > "$config_base/mounts" || exit $? echo "Done" else echo "[n]" fi cd "$old_wd" else echo "Command not understood: $1" echo "Usage: `basename $0` ... " echo echo "Actions:" echo " mount" echo " * If param is blank or --all, all mounts are mounted." echo " * If param is specified, that mount is mounted." echo " * In all cases, base mount will be automatically mounted if it is" echo " not already." echo echo " unmount" echo " * If param is blank or --all, all mounts, including base unionfs" echo " mount, are unmounted." echo " * If param is specified, that mount is unmounted." echo echo " refresh-software" echo " * Doesn't take any parameters." echo " * Will refresh the mounts configuration file with new software," echo " after checking with user that the new list is OK." exit 1 fi shift done
So, the basic idea is this:
- Create the $HOME/opt/ directory.
- Stash the above script into a file called unionfs-local-mount.
- Create three directories in opt: config, merged, and software.
- In software, create a directory called base. Whenever unionfs goes to mount anything, it always needs a base mount to start layering things on top of. This folder will be that base. Now, I put my custom scripts and whatnot in a different software folder called “custom”, but you could put your scripts in base if you felt like it.
- Now in software, create a directory to contain the software (something like “duplicity-0.5.12”).
- Install the software to that directory like I did when I installed straight to opt before.
- Now that your software directory has been changed, you should update the config directory. The unionfs script will handle this for you. Just execute “
unionfs-local-mount refresh-software“.
You’ll want to adjust $PREFIX to look to $HOME/opt/merged now, because when you run…
unionfs-local-mount mount
…it will put every piece of software available into $PREFIX. If you want to remove a particular piece of software, you can always use the unmount command, with a second parameter containing the name of the software (say, “duplicity0.5.12”) and away it goes! I’ve provided some usage information in the script, which you can access by running:
unionfs-local-mount --help
This is basically what I use to manage software that I installed. I also created a script that manages dependencies of local software that I had to install to get it to work, but it’s not worth sharing. It’s easier to track deps by hand, really, in the end.
My Humble Corner 