Archiving Adobe Lightroom Back Ups with PowerShell

LightroomLogoIf you are an Adobe Lightroom user it is critical to have regular backups of your photo library catalogue. Luckily this is a simple task thanks to the fact that Lightroom has features built in to regularly taka a backup for you (which in effect means making a copy of your current catalogue file into a new location in the location you have specified in the user preferences of the application.

For information on how to configure the backup settings in Lightroom check out this Adobe link: https://helpx.adobe.com/lightroom/help/back-catalog.html

Lightroom unfortunately does nothing to clear out old backups and prior to Lightroom version 6 these backups were not even compressed, which together can mean the space required to store backups grows very quickly. It was always frustrating as the catalogue files can be compressed by a huge margin (80-90% in cases). Luckily newer versions of Lightroom now compress the backups into zip files which makes their size much less of an issue.

Anyway for those familiar with PowerShell I have a script that I use which after each backup to remove old backups, compress the new backups and move the backup to a new location (to a separate drive to guard against drive failure).

powershellLogo1The script is called LR_Zip_Tuck as it zips the backups and tucks them away. There are two versions of the script. V1 is for Lightroom versions before V6/CC as it includes the additional compression step which is no required since Lightroom V6. This still wo9rks with Lightroom V6 but is slower , and so V2 of the Script is recommended.

The script first waits until the Lightroom application is no longer running before proceeding. This means that you can run this script on exit of Lightroom as it is still backing up (if you have it set to backup on exit) and it will wait until Lightroom has finished (I run it from a desktop shortcut when I still in Lightroom or it is backing up on exit).

## check if Lightroom is running, and if so just wait for it to close
$target = "lightroom"

$process = Get-Process | Where-Object {$_.ProcessName -eq $target}

if($process)
{
	Write-Output "Waiting for Lightroom to exit..."
	$process.WaitForExit()
	start-sleep -s 2
}

It then loops each folder in the backup location looking for catalogue backups that Lightroom has created since the last time the script was run. It then copies it to the off drive backup location and then deletes local the file.

## loop each subfolder in backup location and process
foreach ($path in (Get-ChildItem -Path $LocalBkUpFolder -Directory))
{
	## find zip file in this folder and rename
	$path | Get-ChildItem | where {$_.extension -eq ".zip"} | Select-Object -first 1 | % { $_.FullName} | Rename-Item -NewName {$path.Name + ".zip"}

	## move file to parent folder (as dont need subfolders now)
	$SourceFilePath = $path.FullName + "\" + $path.Name + ".zip"
	Move-Item $SourceFilePath -Destination $LocalBkUpFolder

	## copy zip to remote share location
	Write-Output "Tucking backup away on remote share"
	Copy-Item $NewFileName -Destination $RemoteBkUpFolder

	## delete folder
	Remove-Item -Recurse -Force $path.FullName
}

It then does some house keeping ensuring that only the configured number of old backups exist in the local and remote locations (ensuring that the oldest are deleted first). This prevents the backups building up over time.

## cleanup zip files (local)
Remove-MostFiles $LocalBkUpFolder *.zip 8

## cleanup zip files (remote)
Remove-MostFiles $RemoteBkUpFolder *.zip 20

That’s about it. The scripts are available on my GitHub repo here (as LR_ZipTuck_V1.ps1 and LR_ZipTuck_V2.ps1).

Advertisement

Backing Up Your Blog Content Using HTTrack

I’m pretty strict on making sure I have my data backed up in numerous places and my blog content is no different. I would hate to lose all these years of babbling. In this post I cover how I back up this blog, and this will apply to any blog engine or indeed any website.

This blog is hosted on WordPress.com and I trust the guys at ‘Automatic’ to keep my data safe, but accidents do happen. Ideally I want an up to date backup of this blog together with any images used. Personally I?m not too concerned about having it in a WordPress format but rather actually prefer having the raw content that I could use to recreate the blog elsewhere.

The HTTrack Tool

The tool I use is HTTrack (http://www.httrack.com) which is a web site copying tool. It essentially re-creates a working copy of the site on the local disk (which you can navigate in a browser too). The tool has many features and includes command line interface which makes it easy to run via a scheduled task. The various command line switches are documented here http://www.httrack.com/html/fcguide.html , but I use this simple command below:

C:\HTTrack\httrack.exe http://richhewlett.com -O c:\TargetFolderPathHere

For interest the –O switch tells HTTrack to output the site to disk and hence produce a copy.

You can create a Windows Scheduled Task that periodically runs this command line and you have an automated backup. I personally go a bit further and wrap this command into a Windows PowerShell script. This script creates new folder each time with the current date and implements error handling which writes to the system eventlog.

This script is an example only and comes with no guarantees that it will work for you without modification:


#===============================================
# Backup blog to disk using HTTRACK
# (Created by Rich Hewlett, see blog at RichHewlett.com)
#==============================================
clear-host
write-output "---------------------Script Start--------------------"
write-output " HTTrack Site Backup Script"
write-output "-------------------------------------------------------"

# set file paths
$timestamp = Get-Date -format yyyy_MMM_dd_HHmmss
$TargetFolderPath="F:\MyBlogBackUp\$timestamp"
$HTTrackPath="C:\HTTrack\httrack.exe"

write-output "Backup target path is $TargetFolderPath"
write-output "HTTrack is at $HTTrackPath"

# set error action preference so errors don't stop and the trycatch
# kicks in to handle gracefully
$erroractionpreference = "Continue"

try
{
    write-output "Creating output folder $TargetFolderPath ..."
    New-Item $TargetFolderPath -type directory

    write-output "Download data ..."
    invoke-expression "$HTTrackPath http://MyBlog.com -O $TargetFolderPath"
    write-output "Done with downloading."    

    write-eventlog -LogName "Network" -Source "HTTrack" -EventId 1 -Message "Downloaded blog for backup"

}
catch
{
    # error occurred so lets report it
    write-output "ERROR OCCURRED DURING SCRIPT " $error

    # write an event to the event log
    write-output "Writing FAIL to EventLog"
    write-eventlog -LogName "Network" -Source "HTTrack" -EventId 1 -Message "Download blog for backup FAILED during execution. $error" -EntryType Error
}

write-output "------------------------------------Script end------------------------------------"

I run this job monthly via a Windows scheduled task.

UPDATE: A working Powershell script can be found on my GitHub site.

Using WordPress Export Feature

If you run a WordPress blog you can also do an export via the Dashboard which will export all site content (including comments) which is useful. In addition to the above raw HTML backup above I also use the export tool periodically manually (and therefore infrequently). For more information on this feature check out http://en.support.wordpress.com/export/

How to Backup To USB Drive Only If It’s Connected

A key part of most personal data backup strategies involves backing up data to an external USB drive but I don’t want to leave it constantly connected. In this post I cover how to backup to an external drive using a scheduled automated process but only if the external drive is connected at the time.

I don’t believe in leaving external backup USB drive always connected to my system (PC or Server) to avoid the data being corrupted or deleted. Also if the backup drive is for off-site storage then its not possible to be always connected anyway. Using the steps below I can perform a full data backup by simply physically connecting the drive (connecting a USB drive or docking the SATA drive into a USB dock for example), and then leaving it overnight. In the morning I can safely physically disconnect it and store it without even having to log onto the machine.

The basic flow:

1) Assuming that the machine is always on, which my server is, a scheduled task runs every night and executes a backup script (regardless of the backup drive being connected or not). If your machine is not always on then vary this by setting the scheduled task at a time when the machine is usually on.
2) The script checks for the existence of a specific folder on the connected drive (e.g: U:\backup\). If the drive isn’t connected then the folder path won’t exist and the script just exits happily. However, if the drive has been connected then the folder path will exist and the backup script will continue and copy over the data.
3) After a successful backup the script safely disconnects the USB drive. This step is technically optional as Windows supports pulling a USB drive out without doing a soft eject but its highly recommended to tell Windows first to avoid data corruption.
4) Optionally you can also output a backup log somewhere to enable you to check the logs periodically without having to reconnect the USB drive to verify the job worked.

More detailed steps:

Firstly connect the external USB drive and make a note of the drive letter it uses. Changing the drive letter to something memorable might help (B for backup, U for USB, O for Offsite etc). We’ll use U for this example. Next we need to write a DOS command script, a simple program or Powershell script to perform the backup of the data using the backup tool of your choice. I use Robocopy to copy the files via a Powershell script and below is a simplified version of my script.

#==========================================================================================================
# Checks for presence of offsite backup USB drive, and backs up relevant data to drive if present,
# exits gracefully if not present. Run script everynight and it only backs up data when offsite
# USB external drive is turned on.
#==========================================================================================================
clear-host

# set file paths and log file names
$timestamp = Get-Date -format yyyyMMdd_HHmmss
$LogBasePath="D:\Logs\OffSiteUSBBackup"
$LogFile="$LogBasePath\USBBkUp_$timestamp.txt"
$USBDriveLetter="U"
$USBDriveBackupPath="U:\Backup"

# set error action preference so errors don't stop and the trycatch kicks in to handle gracefully
$erroractionpreference = "Continue"

try
{
	# Check USB drive is on by verfiying the path
	if(Test-Path $USBDriveBackupPath)
	{
		# now copy the data folders to backup drive
		invoke-expression "Robocopy C:\Docs $USBDriveBackupPath\Docs /MIR /LOG:$LogFile /NP"
		invoke-expression "Robocopy C:\Stuff $USBDriveBackupPath\Stuff /MIR /LOG+:$LogFile /NP"

		# Copy the log file too
		invoke-expression "Robocopy $LogBasePath $USBDriveBackupPath\Logs /MIR /NP"

		# Sleep for 60 to ensure all transactions complete, then disconnect USB drive
		Start-Sleep -Seconds 60
		Invoke-Expression "c:\DevCon\USB_Disk_Eject /removeletter $USBDriveLetter"
	}
}
catch
{
	# Catch the error, log it somewhere, but make sure you still eject the drive using below
	Start-Sleep -Seconds 60
	Invoke-Expression "c:\DevCon\USB_Disk_Eject /removeletter $USBDriveLetter"
}

It is key to include in the script a check for the existence of a specific folder on the drive letter belonging to the external drive (U in our example). Only if its present do we continue with the backup.

I make sure that Robocopy logs the output to a file and that file is on the server and copied to the USB drive (as a record of the last backup date etc). I also report the running of the PowerShell script to the Eventlog for reporting purposes but this is outside the scope of this post.

It’s safer to tell Windows that you’re gonna pull the drive out and so I call  USB_Disk_Eject  from within my script, passing in the drive letter. I then wait 30 seconds to ensure the drive has had sufficient time to disconnect before I exit the script. There are a few tools available for ejecting USB drives such as Microsoft’s Device Console (DevCon.exe) but I use USB Disk Ejector (https://github.com/bgbennyboy/USB-Disk-Ejector).

Now set up a Scheduled Task in Windows to run the script every night at a set time.  As the script is scheduled to run every night all I have to do if I want to perform a back-up is connect my backup drive and leave it until the morning. The script will run overnight, find the drive, backup and disconnect. In the morning I can just physically disconnect the drive safely without having to log onto the machine. Periodically I’ll check the backup logs and the backup drive to make sure all is well and to check remaining drive space etc.

UPDATE: A working Powershell script can be found on my GitHub site.

Do you like this approach? Got a better idea? Let me know via the comments.