Blog‎ > ‎

funkybackup - Mac Timemachine type backup for Linux

posted 26 Aug 2015, 07:53 by Ben Edwards   [ updated 6 Jun 2016, 10:25 by Ben Edwards ]
I am punting this blog post and the script up so people can benefit from it and to get feedback.  It comes with absolutely no greentree and is currently being developed.  It seems to work for me  and will be used shortly in a small charity but should be considered a alpha version.  Just need to say that as I am giving it away and not legally or morally responsible for its use. all feedback gladly received.  I also need I know to add more annotation to the code.

The latest version of the scripts are available through GitHub, the ones below are not up to date and are for illustration purposed only.

funkybackup is a set of Linux scripts that give you a  Mac Timemaching style backup i.e.:
  • Backup every hour and keep for a day
  • Backup every day and keep for a month
  • Backup every month and keep for a year
  • Backup every year
  • Is very efficient as (incremental backups):
    • Only hold each version of a file once (using the minimum amount of disk space)
    • Only copies files that have changes (using minimum resources, disk, CPU and network (if copying over net) )
  • Is extremely easy for users to restore

It does not have, or need, a funky GUI as it creates a directory structure that is easy for the user to understand. 
Each backup has a directory and there is also a 'current' directory for the latest backup. 

In a way it is actually more flexible than Timemachine as in addition to this you can backup as often as you like (i.e. every 10 minutes).  Obviously it does depend of the amount on data you are backing up, how often it changes and the speed of your connection (you can backup locally or remotely over a network or the Internet). 

The latest versions of the scripts are at https://github.com/funkytwig/funkybackup

Scripts

OK, lets dive in and go through the scripts to show you how it works. 

Configuration (vars.bash).

This script is where you setup what you want to backup and where, it has the following variables:

  • backup_name - chose a name for your backup
  • script_home - where the funkybackup scripts live
  • what_to_backup - path to what you want to backup
  • path_to_store_backups - path to the directory you want to store backups in

Frequent Backup (backup.bash).

This script is run as often as you like and each time it runs it creates a backup directory. 

The directories are names as such:

backupname_sequence_F_day_date_HHMM (e.g.'ben_1440601201_F_Wed_26Aug2015_1600'):

  • backupname - the name you assign to the backup (I chose ben as it is my home directory).
  • sequence - This is so when you look at the directories in a file browser they can easily be ordered so the most recent is at the top.  It actually uses Julian seconds (seconds since 1 January 1, 4713 BC).
  • day - The day using 3 characters to make it easy for users to find the backup they want
  • date - The data is a format easy for people to read
  • HHMM - time of the backup

The _F_ is to show this is a frequent backup (i.e. more frequent than an hour).  The scripts use this to help then delete old backups.  The folowing is also used in this position

  • _F_ - Frequent Sub houre backups
  • _H_ - hourly backups
  • _M_ - monthly backups
  • _Y_ - Yearly backups

The script uses the Linux utility rsync to do the actual work.  The script currently backs up over the computers file-system (i.e. it uses directory structures).  This mean that it can back up onto a on or connected to the computer, a local network mount  or NAS (mounted locally).  It will also work through a VPN.  rsync also sports ssh transfers.    Doing this would require some slight changes to this script only and may be added to the scripts later.

As well as doing the frequent backups the script also creates a symbolic link to the latest backup (ben_9999999999_current).

Create Hourly, Daily and Yearly Backups (backup_close_period.bash)

Now this is where the magic happens.  This scripts does two things.  It renames the Frequent backups (so they become Hourly, Daily, Monthly or Yearly Backups) and deletes old backups.  It is called with a parameter which indicates the type of backup you are doing:

  • H - Hourly  backups, the script is called with this parameter at the end of each hour.
  • D - Daily  backups, the script is called with this parameter at the end of each day.
  • M - Monthly  backups, the script is called with this parameter at the end of each month.
  • Y - Yearly  backups, the script is called with this parameter at the end of each year.

Here is the code:


So that is basically it it. This all called from a cronjob, mine looks like this:


Although the scripts are deigned to be run at the end of each period if they do not it does not really cause any problems as it is fairly robust in this respect.  If they are run at the wrong time your backups will still have the correct date/time.  The worst that can happen is the period backups may not be at the end of the period or  you may end up with more than on for a period.  however as the date/times and sequences are correct this is no a real problem. 

The scripts below are slightly updated version of the ones above and I have provided a updated cron and stub sxripts to run from cron so stderr goes into the scripts.

Any feedback, questions please get in touch.

Comments