Cmder – A Better Windows Console

Whilst Linux treats console users as first rate citizens and provides many useful and powerful terminal emulators Windows has always lagged behind. This is evermore noticeable now that many developer and IT Ops workloads are done via the terminal. Modern web development and DevOps tooling requires at least some interaction with the terminal, and with the world moving to git for source control developers everywhere are having to embrace consoles.
Whilst Microsoft have traditionally neglected the Windows console they have started to add new features and improvements. For a background on the Windows Console and its architecture check out this blog series. Windows 10 has the best Windows console to date, but there are better out there from 3rd parties and I’ve really got into Cmder.
Cmder is a smart per-configured bundle of the ConEmu emulator software with some extras thrown in. To quote directly from their website:
 

Cmder is a software package created out of pure frustration over the absence of nice console emulators on Windows. It is based on amazing software, and spiced up with the Monokai color scheme and a custom prompt layout, looking sexy from the start.

It can be run portable on a USB Stick if you wish and it has full Git and Bash support. You can emulate the Windows Command Prompt or PowerShell, Bash, Windows SubSystem for Linux (WSL), even the VS Developer Command Prompt among others. All in a slick feature rich emulator.

cmder

It has hundreds of settings that can be tweaked to get everything just the way you like it and it also has the awesome Quake mode so it can slide down from the top of your display.
Cmder2
Support for Cmd, PowerShell, Bash and many more is included out the box, but if you are a Visual Studio user and want to emulate the Developer Command Prompt for VS2017 (reommended) then check out the simple instructions in this guide by Ricardo Serradas on Medium.
I’ve been using it for months and its been stable, performant and has also caught the eye of collegues due to those good looks which make it a pleasure to work in compared to the plain Windows console. Give it a try.
Advertisement

Calculate a file hash without 3rd party tools on Windows & Linux.

If you need to generate a hash of a file (e.g. MD5, SHA256 etc) then there are numerous 3rd party tools that you can download but if you are restricted to only built in tools or don’t need to do this often enough to install something then there are built in OS tools for Windows and Linux that can be used.

Windows:

For Windows there is “certUtil” which can be used from the command prompt console with  the “-hashfile” option to generate a hash for a supplied file:

CertUtil [Options] -hashfile filePath [HashAlgorithm]

The [HashAlgorithm] options are MD2, MD4, MD5, SHA1 (default), SHA256, SHA384 and SHA512.

For example to get an MD5 hash of a file use:

CertUtil -hashfile C:\ExampleFile1.txt MD5

More documentation for CertUtil can be seen here.

For those with access to PowerShell v4  and above (Windows 8.1 & Win Server 2012 R2) you can use the built in commandlet called get-filehash like this:

Get-FileHash C:\ExampleFile1.txt  -Algorithm MD5 | Format-List

The algorithms supported are SHA1, SHA256 (default), SHA384, SHA512, MACTripleDES, MD5 & RIPEMD160.

For Powershell versions prior to V4 there are numerous scripts available on the web that will work out the hash for you using various methods.

Linux:

For Linux use the correct  hashalgorithmSUM command in the terminal for the algorithm you are looking for, i.e. for an MD5 hash use md5sum or for SHA512 hash use sha512sum.

For example:

md5sum /home/rich/Documents/ExampleFile1.txt 
sha1sum /home/rich/Documents/ExampleFile1.txt
sha512sum /home/rich/Documents/ExampleFile1.txt

 

 

Archiving Adobe Lightroom Back Ups with PowerShell

LightroomLogoIf you are an Adobe Lightroom user it is critical to have regular backups of your photo library catalogue. Luckily this is a simple task thanks to the fact that Lightroom has features built in to regularly taka a backup for you (which in effect means making a copy of your current catalogue file into a new location in the location you have specified in the user preferences of the application.

For information on how to configure the backup settings in Lightroom check out this Adobe link: https://helpx.adobe.com/lightroom/help/back-catalog.html

Lightroom unfortunately does nothing to clear out old backups and prior to Lightroom version 6 these backups were not even compressed, which together can mean the space required to store backups grows very quickly. It was always frustrating as the catalogue files can be compressed by a huge margin (80-90% in cases). Luckily newer versions of Lightroom now compress the backups into zip files which makes their size much less of an issue.

Anyway for those familiar with PowerShell I have a script that I use which after each backup to remove old backups, compress the new backups and move the backup to a new location (to a separate drive to guard against drive failure).

powershellLogo1The script is called LR_Zip_Tuck as it zips the backups and tucks them away. There are two versions of the script. V1 is for Lightroom versions before V6/CC as it includes the additional compression step which is no required since Lightroom V6. This still wo9rks with Lightroom V6 but is slower , and so V2 of the Script is recommended.

The script first waits until the Lightroom application is no longer running before proceeding. This means that you can run this script on exit of Lightroom as it is still backing up (if you have it set to backup on exit) and it will wait until Lightroom has finished (I run it from a desktop shortcut when I still in Lightroom or it is backing up on exit).

## check if Lightroom is running, and if so just wait for it to close
$target = "lightroom"

$process = Get-Process | Where-Object {$_.ProcessName -eq $target}

if($process)
{
	Write-Output "Waiting for Lightroom to exit..."
	$process.WaitForExit()
	start-sleep -s 2
}

It then loops each folder in the backup location looking for catalogue backups that Lightroom has created since the last time the script was run. It then copies it to the off drive backup location and then deletes local the file.

## loop each subfolder in backup location and process
foreach ($path in (Get-ChildItem -Path $LocalBkUpFolder -Directory))
{
	## find zip file in this folder and rename
	$path | Get-ChildItem | where {$_.extension -eq ".zip"} | Select-Object -first 1 | % { $_.FullName} | Rename-Item -NewName {$path.Name + ".zip"}

	## move file to parent folder (as dont need subfolders now)
	$SourceFilePath = $path.FullName + "\" + $path.Name + ".zip"
	Move-Item $SourceFilePath -Destination $LocalBkUpFolder

	## copy zip to remote share location
	Write-Output "Tucking backup away on remote share"
	Copy-Item $NewFileName -Destination $RemoteBkUpFolder

	## delete folder
	Remove-Item -Recurse -Force $path.FullName
}

It then does some house keeping ensuring that only the configured number of old backups exist in the local and remote locations (ensuring that the oldest are deleted first). This prevents the backups building up over time.

## cleanup zip files (local)
Remove-MostFiles $LocalBkUpFolder *.zip 8

## cleanup zip files (remote)
Remove-MostFiles $RemoteBkUpFolder *.zip 20

That’s about it. The scripts are available on my GitHub repo here (as LR_ZipTuck_V1.ps1 and LR_ZipTuck_V2.ps1).

Using PowerShell for your VS Code Integrated Terminal

Microsoft’s superb Visual Studio Code editor has an integrated terminal which is accessed via the ‘View’menu or via the Ctrl+’ shortcut keys. On Windows by default the terminal used is the Windows Command Prompt (cmd.exe) terminal, however you can easily configure VS Code to use a different terminal such as Windows PowerShell.

Open the User Settings config file (the ‘settings.json’ file accessed via File > Preferences > User Settings) and modify the setting for which terminal to run on Windows:

The default setting is:

 “terminal.integrated.shell.windows”: “C:\\WINDOWS\\system32\\cmd.exe”,

To use the PowerShell terminal instead add this to your settings.json user settings file:

“terminal.integrated.shell.windows”: “C:\\Windows\\sysnative\\WindowsPowerShell\\v1.0\\Powershell.exe”,

Now PowerShell will be used instead of cmd.exe. Currently only one terminal can be configured in VS Code and so you can’t have both PowerShell and cmd.exe so you’ll have to choose your favourite for now. You can however access mutliple instances of the terminal via the drop down on the terminal window.

vscodeposhterminal2

Finally whilst on the subject of VS Code and PowerShell I recommend installing Microsoft’s PowerShell Extension which lets you code and debug PowerShell scripts directly within VS Code (and benefit from its features, e.g. git integration etc).

Interactive file reading with Powershell

Sometimes you want to see the contents of text file whilst it is still being updated, a common example is where you are outputting to a log file and need to see the output interactively without having to keep opening the file to check for progress, or to see if a job has complete.

Luckily there is the very useful Get-Content Powershell Command.

This can take a “-wait” parameter that will reread the file every second or so and check for updates, displaying it in the console (until you end the command with Ctrl&C as usual).

So for example the command below will read the file constantly until you tell it to stop:

Get-Content C:\Logs\Log.txt -wait

In addition there is the “-Tail” parameter which is the Powershell equivalent to the Unix tail command. This will read the last few lines of the file only and not the whole file. This can be used on its own like this:

Get-Content C:\Logs\Log.txt -tail 5

…which displays the last 5 lines of the file. Or you can combine -wait and -tail together to constantly read the last (specified) number of lines:

Get-Content C:\Logs\Log.txt -wait -tail 5

For more information, see Get-Content Powershell Command on MSDN.

Allow PowerShell Execution

By default PowerShell’s execution policy is very restrictive which is a good thing for security. If you are editing or running scripts on your machine you may want to relax it slightly. As I often forget to do this on new machines I’m making a note of the command in this post:

Open PowerShell prompt as Administrator, and then run:

set-executionpolicy remotesigned

Remotesigned means local scripts can be run but downloaded ones must be signed. You can remove all restriction via:

set-executionpolicy Unrestricted

To view the current setting on your machine use get* instead of set*:

get-executionpolicy

For more information see technet here:
https://technet.microsoft.com/en-us/library/ee176961.aspx

 

Using SlickRun For Fast PowerShell Commands

I’m a big fan of SlickRun and if you’ve not used it I recommend you download it and give it a try. I find it an invaluable tool not only for launching apps but also web sites, collections of applications and directories etc. It’s the simplicity of SlickRun that makes it so powerful. Sure there are newer, and certainly prettier launchers out there but SlickRun is still the best in my opinion. Whilst it’s usefulness is slightly diminished by the Windows 7 start menu search it still has its place for providing shortcuts of any name you like, and for running multiple apps off one command. Anyone reading this blog will no doubt have spotted that I also am a fan of Windows PowerShell, so I’ve now combined the two. The objective being to run simple PowerShell commands within SlickRun and have the results appear quickly in a console (with no coding).

PowerShell can be run with start up parameters (see the documentation here). The one’s we need to use are:
-NoExit to not exit the console window after running the command (so you can see the results)
-command to executes the specified command as though they were typed at the Windows PowerShell command prompt.

The SlickRun setup for the MagicWord (I’ve used ‘POSH’ in this example) is:
MagicWord: POSH
Filename: powershell
Parameters: powershell -NoExit -command "$I$"

The filename can be just "powershell" or if you feel better you can put in the full path to the PowerShell exe. Passing the command text as "$I$" just passed in what you typed in SlickRun after the magic word.

We can then just use the POSH keyword followed by the command we wish to run. For example this keyed into SlickRun…

Posh1

launches the PowerShell console and the results…

Posh2

Of course this approach can be used from other application launchers, directly from the command line or the Start-Run dialog but then of course you will have more to type as you will need to key the start-up parameters each time.

Android Remote Desktop Client

2XClient_LogonI find that I am increasingly relying on the computing power of my Android smartphone (a HTC Desire) and finding novel ways of using it to make my IT life easier. Sometimes I just want to connect to my PC that is in another room, or more often for me it’s my headless Windows Home Server, and so I scouted for a Remote Desktop client that I could run on my phone. The key requirement was for it to use the Windows native Remote Desktop protocol and therefore not require any software to be installed on my PC or Server, which ruled out a lot of the VNC based Apps. Luckily 2x.com have released an excellent FREE App that ticks all the boxes.

2XClient for Android can be found here or on Android Market here. It is dead easy to set up the target machines and there are several display optimisation options. The key thing though is that it’s actually very easy to navigate the target machines desktop via a custom keyboard and a nifty mouse icon that can be dragged around with a left and right mouse button attached (left image below).  In these images I’m logging onto my Windows Home Server (a Windows 2003 based OS) but I also use it with my Windows 7 PC too. One thing to note for Windows 7 though is that I needed to set my Remote Desktop settings (via My Computer > System Properties > Remote Settings)  to “Allow connections from computers running any version of Remote Desktop” as opposed to the default setting of enforcing Network Level Authentication.

 2XClient_Mouse  2XClient_Keyboard  2XClient_StartMenu

It is surprisingly easy to do simple tasks on the target machine, especially after a bit of practice. Here I am using PowerShell and checking my Home Server Console.

2XClient_WHS12XClient_POSH

A very powerful tool to have on your phone and ideal for those quick techy tasks when you can’t be bothered to get off the sofa.

Windows PowerShell Console Fun

1

Whilst watching Die Hard 4 the other day I noticed the funky transparent console windows that were being used to battle out cyber warfare, and being a traditional geek I immediately liked the idea of doing the same for my PowerShell console. Sure I know these guys are using Linux and transparent consoles have been around for years but still I fancied some of that slickness on my Windows.

POSh_PssGlassIt didn’t take long to find PSGlass on CodePlex (http://powershellglass.codeplex.com/) which is a neat little exe that runs in your system tray and hunts out any PowerShell console windows, and any it finds it converts to transparent using the Windows Aero effects. The peek into the source code shows that it’s checking for a window with a process name of “powershell” or “cmd” and then uses the DwmEnableBlurBehindWindow API to make it transparent.  The effect is shown on the left. It’s simple and effective and could be extended easily to do more. Not content with this I wanted to achieve the same result from within PowerShell itself so set about using the API in a script. Luckily for me Oisin Grehan had already written a script to achieve the same result and posted it on PoshCode.org, check it out at http://poshcode.org/2052. For this script the DwmExtendFrameIntoClientArea API is used to create a sheet of glass effect with no borders and the effect is much more striking (right screen shot). The fact that POSh_Glassyou don’t have to have an application running in the background is of course much better and as its a script you can add it to your profile to always take effect on Powershell start-up. I have found it useful to create a function in my profile to toggle the glass effect on/off depending on my mood and what actions I am trying to perform.

As the console was now transparent I quickly found myself wanting it to stay on the top of other windows and so set about looking for a script for that too. Again the excellent PoshCode.org came to the rescue with this script from Shay Levy at http://poshcode.org/1837. Again I downloaded the script and added a function to my profile to be able to toggle it on/off at will from within the console itself. In order to access the Set-TopMost() function that sits within the TopMost.ps1 script I used dot sourcing (described here by Jesse Hamrick) and my functions are shown below:

function OnTop
{
	. TopMost.ps1
	Set-TopMost (get-process -id $pid).mainwindowhandle
}

function NoOnTop
{
	. TopMost.ps1
	Set-TopMost (get-process -id $pid).mainwindowhandle -disable
}

PoshMatrixFor total geek heaven why not go the extra mile and put a Matrix style screensaver within the console itself. If you haven’t seen it I would recommend checking it out at http://www.nivot.org…. The video on the site shows how it works but it essentially runs a Matrix code screensaver inside the console (not the whole desktop).  Now that’s one clever PowerShell script!

2017 UPDATE:

As PoshCode.org site has been down a while I’ve added alternative links to be able to find the above mentioned scripts from the awesome web.archive.org instead of PoshCode.org:

Backing Up TFS 2010 Using PowerShell: Part 2

In my previous post (“Backing Up TFS 2010 Using PowerShell: Part 1”) I covered how to backup up the TFS SQL Server databases using Windows PowerShell and the SQL Server Management Objects (SMO) API. In this post I’m going to take it one stage further and add another layer of backup for the paranoid amongst you like me.

Backing up the databases is all well and good but I like to be able to see what I’m backing up too and there’s nothing quite like being able to see all your data on disk in its native format.  So to give me this extra warm fuzzy feeling I have written a PowerShell script to perform a ‘Get-Latest’ of all the source code in my ‘Team Project Collection’ in TFS and then copy those latest file versions to a backup location. By scheduling this activity to occur periodically through the week I see two key benefits. Firstly I can see that I have the data in a second place and that its in its native format (i.e. just plain files of source code, documents, images etc) which I can then backup and know that should I not be able to restore the SQL databases I will at least have the latest source files. The second benefit is that I have the ability to quickly access my source code (on the local machine or remotely) from another machine that may not have Visual Studio installed. This is useful if I just want to check something out and don’t need to modify the files.

I wanted this script to run on the server (in my case a Windows Home Server) and so to be able to perform a Get-Latest on TFS I needed to install ‘Team Explorer’ and setup a ‘Workspace’. Team Explorer is the standalone client required to interact with TFS Source code without Visual Studio. The Team Explorer setup package is included on the TFS installation media in the TeamExplorer folder. Browse to this and then run Setup.exe and keep the default values. This installs the Visual Studio 2010 shell, the TFS object model, Visual Studio Team Explorer 2010 and MS Help Viewer.

Next you need to setup a Workspace for the server using Team Explorer (via Start > All Programs > Microsoft Visual Studio 2010 > Microsoft Visual Studio 2010) and access the TFS server by adding the server in the normal way and go to ‘source control’. Select the top item in source control (in most cases ‘SERVERNAME\DefaulCollection’) and right click, “Map to local Folder”. Enter path of local folder on server and then it will ask if you want to get latest, say yes. You now have a workspace set up and you will be able to perform a manual or automated ‘Get-Latest’.

To automate the interaction with TFS and perform the ‘get’ you could use the TF.exe command line tool that installs with the Team Explorer setup. Personally as a PowerShell fan I’ chose to use the PowerShell cmdlets that come with the Team Foundation 2010 Power Tools. To use these you’ll need to download the power tools and run the tfpt.msi installation package. Choose custom and just choose the server specific items, in this case just the PowerShell cmdlets and the command line interface options (see screenshot).

powertolls

The basic flow of the script is configure the file paths required, import the Microsoft.TeamFoundation.PowerShell snap-in/module, and then call the ‘Update-TFSWorkspace’ cmdlet. This is the command to perform a Get-Latest on the workspace previously configured. The workspace folder you specify must match the local folder you specified earlier in Team Explorer. The next step is to use Robocopy to copy the files to the backup location. Robocopy is a superb file copy tool as it only copies changed files which significantly improves the speed of this procedure compared to using a standard file copy cmdlet. Once complete the log is passed to RoboCopyLogger (more about this in a future blog post) that scans the log for errors, and then we add an Event Log entry for success or failure.

Finally set up a Scheduled Task to configure this script to run automatically each week. If you were to use this script as is you would need to change the file paths, and use an Event Log Source that was relevant to your system (or just remove the Write-Eventlog call”).


clear-host
write-output "------------------------------------Script Start------------------------------------"
write-output " TFS Data Get-Latest Backup "
write-output "------------------------------------------------------------------------------------" 

# set file paths
$LocalWorkspaceFolder="c:\tempcode\tfsbackups\latestdata"
$RemoteBackupFolder="\\WinHomeSvr\Backup2\TFSBackups\LatestData"
$Robocopy="c:\scripts\Robocopy.exe"
$timestamp = Get-Date -format yyyyMMddHHmmss
$Log="c:\Scripts\Logs\TFSData\TFSLatestDataBackup_$timestamp.txt"
$RobocopyLogger="c:\Scripts\RoboCopyLoggerTweet_BETA\RobocopyLoggerTweet"

# set error action preference so errors stop and the trycatch kicks in
$erroractionpreference = "Continue"

try
{
    # add the TFS snapin to the session
    add-pssnapin Microsoft.TeamFoundation.Powershell

    # get latest on the top level workspace level
    Update-TfsWorkspace $LocalWorkspaceFolder -Force -Recurse

    # now robocopy the local cache to the final backup folder
    invoke-expression "$Robocopy $LocalWorkspaceFolder
                                        $RemoteBackupFolder /MIR /LOG:$Log /NP"

    # Copy the log files too
    invoke-expression "$Robocopy $Log $RemoteBackupFolder /MIR"

    # Check the log for errors etc and log to eventlog/twitter
    invoke-expression "$RobocopyLogger $Log 1"

    # write all events to the logs
    write-output "Writting SUCCESS to EventLog"
    write-eventlog -LogName "HomeNetwork" -Source "TFS Backups"
                              -EventId 1 -Message "TFS Data BackUp Script ran"
}
catch
{
    # error occurred so lets report it
    write-output "ERROR OCCURRED" $error 

    # write an event to the event log
    write-output "Writting FAIL to EventLog"
    write-eventlog -LogName "HomeNetwork" -Source "TFS Backups"
                               -EventId 1 -Message "TFS Data BackUp Script failed" -EntryType Error
}

write-output "------------------------------------Script end------------------------------------"