The HTML Agility Pack

For a current project I needed to perform a simple screen scrape action. The resulting solution was functional but a bit rough and ready. Luckily I stumbled upon this open-source HTML library project: The HTML Agility Pack, hosted on CodePlex at http://htmlagilitypack.codeplex.com.

It is an excellent little library that makes dealing with HTML a breeze, whether you are screen scraping or just manipulating HTML documents locally. It is very forgivable with regards to malformed HTML documents and supports loading pages directly from the web. You can just parse the HTML or modify it, and it even supports LINQ.  A key benefit of this library is that it doesn’t force you to learn a new object model but instead mirrors the System.XML object model – a huge help for getting up and running quickly, as well as making coding it feel natural.

Download HTML directly via a URL:

HtmlDocument htmlDoc = new HtmlDocument();
HtmlWeb webGet = new HtmlWeb();
htmlDoc = webGet.Load(url);

Or parse an HTML string:

HtmlDocument htmlDoc = new HtmlDocument();
htmlDoc.LoadHtml(htmlString);

Then you can use XPATH to query the HTML document as you would an XML document:           

// select a <li> where it has an element of <b> with a value of "Name:"
var nameItem = htmlDoc.DocumentNode.SelectSingleNode("//li[b='Name:']");
if (nameItem != null && nameItem.ChildNodes.Count > 1)
{
    name = nameItem.ChildNodes[1].InnerText;
}

You can download it via NuGet here : http://nuget.org/packages/HtmlAgilityPack.

For more examples of it’s use check out these posts: Parsing HTML Documents with the Html Agility Pack and Crawling a web sites with HtmlAgilityPack.

Enjoy.

Useful Web Based UML Drawing Tools

A basic sequence diagram can be a very powerful tool to explain the interactions in a system but drawing them can often be too time consuming to bother for disposable uses. I find that many people draw them out on rough paper to help explain their argument but less actually ever bother to build them in soft form unless for a formal document. There are a lot of powerful feature rich UML building tools but recently I found this: http://www.websequencediagrams.com.

It lets you build sequence diagrams like the one below in seconds by typing the object interactions in a short hand form, such as:

title Authentication Sequence
Alice->Bob: Authentication Request
Bob-->Alice: Authentication Response
Bob-->Jeff: Pass Request
Jeff-->Bob: Return Response

…which draws this in real time in the browser: 
WebSequenceDigrams 

And you can even choose the style and colouring too. There’s also the functionality to save diagrams and import saved diagram text. Check out the API page too for tips on embedding the drawing engine into your web pages allowing you to edit your diagrams as well as plugins for Confluence, Trac and xWiki. There are also example implementations for Ruby, Java and Python.

A similar online tool is http://yuml.me with which you can draw Class, Activity and Use Case Diagrams. Here is an example of a Use Case diagram definition:

[Customer]-(Make Cup of Tea)
(Make Cup of Tea)<(Add Milk)
(Make Cup of Tea)>(Add Tea Bag)

…which makes this:

YUMLUseCase

yUML.me also supports API integration with a whole host of things (Gmail, Android, .Net, PowerShell, Ruby and more).

Now there’s no reason to not use a quick UML diagram to explain what you mean!

Exporting an HTML Report From your VS/TFS Test Results

Exporting an HTML Report From your VS/TFS Test Results

Have you ever wanted to export your unit test results from Visual Studio or your TFS Build? Sometimes you may need to provide evidence of your unit testing position to a project stakeholder. This may be for an internal review or as part of a gateway check in a waterfall project. Alternatively you may just want to export your test results for storage later. Out the box Visual Studio provides the Test Results view which is a great tool for seeing the tests that passed/failed in a build, but it is in a custom file format (*.trx) and requires Visual Studio to view.

However I recently found this command line tool on CodePlex by ‘ridomin’ called trx2html which ‘does what it says on the tin’, it converts trx files to HTML. Using the command line, just pass the file path to your test results file for your solution or TFS Build (*.trx) to the trx2html.exe and out pops a fancy HTML report (example below).

trx2html

The tool is open source on CodePlex and so you can download the latest version from: http://trx2html.codeplex.com.

New WordPress.com Source Code Live Writer Plugin Version

New WordPress.com Source Code Live Writer Plugin Version

I’m pleased to announce the release of version 1.4.0 of my Windows Live Writer (WLW) Source Code Highlighter plugin for WordPress.com hosted blogs. For those not aware of this plugin it enables you to quickly insert a source code snippet into your hosted blog posts using the built in WordPress.com source code short codes. For more information and a run down of features checkout this page.

This is essentially a bug fixes release with some small new features:

  1. Bug Fix: The previous version did not handle XML, HTML very well due to problem with HTML encoding. This has been resolved and so HTML, XML and also strings such as “List<T>” are now handled correctly.
  2. Bug Fix: Opening a previously published post from your blog that contained Source Code would often clear the post’s title and categories. This took a bit of head scratching but it turned out it seems to be a bug in WLW, which I have now circumvented. Old posts can now be retrieved with titles intact.
  3. Bug Fix: Some minor miscellaneous stability and refactoring changes.
  4. New Feature: A requested feature was to have the ability to set a default programming language for the code snippets. After some prototyping I found it better to default the language to the one previously selected for the last inserted code snippet. This means that if you are, for example, normally inserting C# then you won’t need to change the language dropdown as it will default to the last one you inserted. Don’t forget you can customise the text file that accompanies the plugin to remove the languages you don’t use to make the dropdown smaller if you desire.
  5. New Feature: Since I released the previous version WordPress.com now supports these new language choices: ‘r’, ‘html’, and ‘clojure’. These have now been added to the language dropdown list for use via the plugin.

Still on the backlog list is an MSI installer, new version notifications and auto-update features. For now I hope you enjoy this new version, and thank you to all of you who have submitted feedback in the comments. The new version (1.4) is available here to download now.

No VS with Notepad++ and Programmers Notepad

Sometimes, despite Visual Studio being an excellent IDE, you want to go the No VS route and hack your code in notepad. Perhaps you just want to make a quick change and its not worth firing up the full VS experience. Maybe you are only checking the code as a background time-filler in between other tasks you don’t want VS eating up your workstations resources all day. Perhaps most likely you’ll need to edit your code on a PC without VS (e.g. Netbook). A while ago I came across this excellent post by SecretGeek where he’s used a batch  script to add compilation to the excellent Notepad++. I recommend taking 5 minutes to follow his instructions and get this working as it makes Notepad++ even more useful. Don’t forget you will need to change the .Net framework path in the batch file if you’re coding apps older than .Net 4.0.

I’ve also copied the batch file and then modified it to work for solution files too, so I can compile the whole solution if required instead of just the project file. I’ve then added a second entry in the Notepad++ file (as per SecretGeek’s instructions) and a second shortcut key for this batch file. The only change required is to amend the “*.*proj” to “*.*sln”.  This change assumes that the project and solution files are NOT in the same folder though as this confuses the .Net compiler.

As good as Notepad++ is as a notepad tool, if you need something a little more like a mini IDE then I recommend “Programmers Notepad”. Not only is it an excellent lightweight Notepad like tool but it also supports “projects” and tools. By far the best feature though is that it parses the output from the compiler and highlights the results. Also clicking on the error will take you straight to the offending line of code (see screenshots below).

CompileNetProject_ErrorAndGoToErrLineCompileNetProject_action

It is easy to integrate SecretGeek’s above batch files into Programmers Notepad.  Go to Tools > Add Tool, and set it up like this:

CompileNetProjectCompileNetSln

Now we’re cooking! So what’s left? Well what about being able to run your Unit Tests from this mini IDE? Let’s assume Visual Studio is installed on the machine in question (which negates the point of the exercise perhaps, but not always as sometimes you have it installed but just don’t want to run the full blown VS IDE) so we can run MSTest via a batch file. For this batch file to work though we have to make a few more assumptions:

1 You will always run the tests whilst the unit test project code is the open active document in Programmers Notepad.
2 The test assembly name is the same name as the test project file.

Using these assumptions we can code the RunTests.bat batch file to find the active document’s VS project file (*.*proj) and use its name to find the output assembly name in the bin\debug folder. Once we have the test assembly we can throw it at MSTest for it to run the tests. Assigning this new batch file to a Tool menu item in Programmers Notepad (as we did with the other batch files above) we can run the tests within Programmers Notepad and the test results will be output into the output window.

RunTests

The code for the RunTests.bat batch files is:

@ECHO OFF

Set MSTestPath="C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MStest.exe"

:findproj

Set current=%cd%

If not exist *.*proj goto tryFindProjAgain
REM get project file name
FOR %%f IN (*.*proj) DO Set dllname=%%f
REM Convert proj file name to test assembly name
set dllname=%dllname:csproj=dll%
Set fullpath=%cd%\bin\Debug\%dllname%

%MSTestPath% /testcontainer:%fullpath% /detail:errormessage

goto end

:tryFindProjAgain

ECHO No project file found in:
ECHO %cd%

cd ..

REM detect if we are at the root level
if "%current%"=="%cd%" goto end

goto findproj

:end

Using SlickRun For Fast PowerShell Commands

I’m a big fan of SlickRun and if you’ve not used it I recommend you download it and give it a try. I find it an invaluable tool not only for launching apps but also web sites, collections of applications and directories etc. It’s the simplicity of SlickRun that makes it so powerful. Sure there are newer, and certainly prettier launchers out there but SlickRun is still the best in my opinion. Whilst it’s usefulness is slightly diminished by the Windows 7 start menu search it still has its place for providing shortcuts of any name you like, and for running multiple apps off one command. Anyone reading this blog will no doubt have spotted that I also am a fan of Windows PowerShell, so I’ve now combined the two. The objective being to run simple PowerShell commands within SlickRun and have the results appear quickly in a console (with no coding).

PowerShell can be run with start up parameters (see the documentation here). The one’s we need to use are:
-NoExit to not exit the console window after running the command (so you can see the results)
-command to executes the specified command as though they were typed at the Windows PowerShell command prompt.

The SlickRun setup for the MagicWord (I’ve used ‘POSH’ in this example) is:
MagicWord: POSH
Filename: powershell
Parameters: powershell -NoExit -command "$I$"

The filename can be just "powershell" or if you feel better you can put in the full path to the PowerShell exe. Passing the command text as "$I$" just passed in what you typed in SlickRun after the magic word.

We can then just use the POSH keyword followed by the command we wish to run. For example this keyed into SlickRun…

Posh1

launches the PowerShell console and the results…

Posh2

Of course this approach can be used from other application launchers, directly from the command line or the Start-Run dialog but then of course you will have more to type as you will need to key the start-up parameters each time.

Android Remote Desktop Client

2XClient_LogonI find that I am increasingly relying on the computing power of my Android smartphone (a HTC Desire) and finding novel ways of using it to make my IT life easier. Sometimes I just want to connect to my PC that is in another room, or more often for me it’s my headless Windows Home Server, and so I scouted for a Remote Desktop client that I could run on my phone. The key requirement was for it to use the Windows native Remote Desktop protocol and therefore not require any software to be installed on my PC or Server, which ruled out a lot of the VNC based Apps. Luckily 2x.com have released an excellent FREE App that ticks all the boxes.

2XClient for Android can be found here or on Android Market here. It is dead easy to set up the target machines and there are several display optimisation options. The key thing though is that it’s actually very easy to navigate the target machines desktop via a custom keyboard and a nifty mouse icon that can be dragged around with a left and right mouse button attached (left image below).  In these images I’m logging onto my Windows Home Server (a Windows 2003 based OS) but I also use it with my Windows 7 PC too. One thing to note for Windows 7 though is that I needed to set my Remote Desktop settings (via My Computer > System Properties > Remote Settings)  to “Allow connections from computers running any version of Remote Desktop” as opposed to the default setting of enforcing Network Level Authentication.

 2XClient_Mouse  2XClient_Keyboard  2XClient_StartMenu

It is surprisingly easy to do simple tasks on the target machine, especially after a bit of practice. Here I am using PowerShell and checking my Home Server Console.

2XClient_WHS12XClient_POSH

A very powerful tool to have on your phone and ideal for those quick techy tasks when you can’t be bothered to get off the sofa.

Software KVMs

Recently I have acquired an additional desktop machine on my desk and quickly saw the potential to expand the amount of screen real estate at my disposal (you can never have enough screens). So imagine I have a laptop physically connected to two screens and a desktop PC, with one screen connected, on the same desk. I want to be able to seamlessly control that desktop PC with my main keyboard and mouse that it physically connected to my laptop. That way I get to have three screens and twice the processing power. Remote Desktop tools of course are not useful here as we can already see the desktop PC’s monitor and we don’t want to control the PC through a window on the laptop. Instead we need Software KVM Applications (in fact without the V for Video as we can see the screen). These work by sending your keyboard and mouse movements over a network connection to the additional PC. They also transfer your clipboard contents so you can copy paste easily between the machines.

First I tried Synergy from http://synergy-foss.org/ which is fairly unique as it’s a cross platform offering that runs on Windows, Mac and Linux which is incredibly powerful if you have a mixed environment. Sy1I Sy2tried the latest stable build which was 1.3.6. I found it functional but basic and not that robust. The Synergy client would stop running on on several occasions (usually after locking/unlocking the PC). The UI is also very basic. That said it did the job and I have since found that the UI has been completely overhauled for the current BETA version. Whilst I don’t think it is quiet solid enough yet it looks to be a big player in this space and there is no doubting that for those with a mixed environment it is great.

In the end I decided on InputDirector, found at http://www.inputdirector.com/. This is a Windows only offering but is more mature than Synergy with a host of additional options. It is easy to configure with one PC being the master and one being the slave. You can right click the icon in the system tray and choose to enable/disable it and also to lock/shutdown the Master and Slave PC’s, which I find useful when I want to lock both PCs in one go. The best feature though is it’s stability as I have not found one issue with it yet and am surprised how effortlessly it handles the docking/undocking of my laptop which is acting as Master. Once the laptop is docked a message pops up on the system tray to notify me that master and slave are in communication again and all is well.

InputDirector screenshots below:

 ID_1ID1ID3

XML Error in TFS Power Tools Backup Plan

XML Error in TFS Power Tools Backup Plan

Whilst checking the Windows Event Log on my server I found this worrying error being reported by the Team Foundation Server Power Tools Backup & Restore Tool :

Error: Tfpt Backup Restore : There is an error in XML document (499, 100).

I posted an article about using this tool to backup your TFS server in September, and it has been successfully backing up daily since that time. The above error has been reported after every daily backup for the past few weeks it seems. It immediately follows an information event from MSSQL$SQLEXPRESS stating that the “Database has been backed up”. To investigate more I opened the TFS Admin Console on the server and clicked “Take Full Backup Now” whereby I got the same error interactively – well at least it’s consistent.

TFS Backup There is an error in XML document

From here you can open the log, and in that I can see it’s reporting that the BackupSets.xml file in the root of the backup target location is invalid:

[Info   @20:30:15.977] Backup Set Name Tfs_DefaultCollection database Backup
[Info   @20:30:15.977] Backup Set Description Tfs_DefaultCollection database – Full Backup
[Info   @20:30:15.977] Adding database Tfs_DefaultCollection to the backupset
[Error  @20:30:16.352] System.InvalidOperationException: There is an error in XML document (499, 100). —> System.Xml.XmlException: There is an unclosed literal string. Line 499, position 100.

On opening the file there appears to be nothing wrong with the XML as such but in my case it didn’t seem complete, as the last successful backup only detailed one entry not the usual two (*.bak and *.trn files). This file seems to be record of the backups as opposed to being core to the actual SQL backup process so I deleted/renamed the BackupSets.xml file and ran another full backup. This time it worked fine and a new BackupSets.xml file was created. It’s now running fine again.

Backing up TFS 2010 with new Power Tools Backup Plan

Backing up TFS 2010 with new Power Tools Backup Plan

At last – backup is built into the TFS product, well via the Power Tools at least. Backing up TFS has always been difficult and non-intuitive without a SQL DBA in your pocket, even the documentation is at best extensive but at worse confusing. But all that’s history as the September 2010 TFS Power Tools now includes a Backup plan feature.

Recently I have posted a few articles on backing up TFS 2010 using Windows PowerShell. The first involved backing up the all TFS SQL databases using PowerShell and SMO and can be found here: ‘Backing Up TFS Part-1’. the second covered an alternative strategy of backing up the latest content of your TFS repository and can found here: ‘Backing Up TFS Part-2’.

Brian Harry recently posted this article ‘Backing up and restoring your TFS server’ describing the new Backup Plan feature of the new release of the TFS Power Tools and this week the September 2010 TFS Power Tools were released. Brian’s posts provide all the details you need on installing and configuring a backup plan together with some screenshots.

I installed the updated Power Tools on my Windows Home Server (you must remove any previous versions manually first) running TFS and easily setup a backup plan:

TFSBackupInstallOption

I did hit one problem with it though. It seems that the feature is a little fussy on the target location of the backup files. The target must be a network share and the tool will attempt to apply the relevant access privileges to the folder for the scheduled job to be able to write the backups successfully. The tool verifies that the information you have supplied is accurate and it checks that a backup can run. My initial attempts to verify my backup plan failed at this point with the following error in the log:

Microsoft.SqlServer.Management.Smo.FailedOperationException: Backup failed for Server ‘servername\SqlExpress’.  —> Microsoft.SqlServer.Management.Common.ExecutionFailureException: An exception occurred while executing a Transact-SQL statement or batch. —> System.Data.SqlClient.SqlException: Cannot open backup device ‘\\servername\backup\tfsbackups\Tfs_Configuration_20100911231343.bak’. Operating system error 5(failed to retrieve text for this error. Reason: 1815).

After some investigation and a helpful post from Dave Hunter I concluded that this was due to the permissions on the backup share on my Home Server. As that share is a WHS managed share it has specific security permissions that prevented the backup tool from asserting its authority and granting the relevant privileges. To circumvent the problem I created a new standard non-WHS share on my Home Server with minimal restrictions. Once I entered the new share’s details the backup tool verified the backup plan successfully and ran a backup fine. I then knocked up a simple RoboCopy script to copy the contents of the new backup share to my original intended WHS share target location on a daily basis via a scheduled task.

In summary I believe that this is a major step forward for TFS and will benefit many of the new users picked up since the introduction of the TFS Basic Configuration in TFS 2010, proving to be another nail in the coffin for Visual SourceSafe. I’d recommend any team still using SourceSafe or any other tool to take another look at TFS as it is definitely getting easier to manage than previously.