List all Transitive Dependencies in your .Net Core Project

Photo by cottonbro studio on Pexels.com

Sometimes you need to find out what packages a .Net Core project or solution depends on quickly and whilst you can find this information in the Visual Studio IDE or by opening the project files individually and reading them, there is a quick and easy way using the “dotnet list package” command. What’s more you can also list the packages that your dependencies also have dependencies on, which is really useful for tracking down what part of your solution has a requirement on a specific package.

dotnet list package

This outputs all NuGet package references for a specific project or a solution (which you can specify in the command parameters or just let dotnet find the nearest solution or project file in the current directory structure). Example below:

Project 'WebApplication1' has the following package references
   [net6.0]: 
   Top-level Package                                Requested   Resolved
   > Azure.Storage.Blobs                            12.14.1     12.14.1 
   > Microsoft.AspNet.Identity.EntityFramework      2.2.3       2.2.3  

To see which packages your project relies on and also which packages they in turn rely on then run the command with the –include-transitive flag.

dotnet list package --include-transitive

This outputs the full list of packages and their dependencies, like so:

Project 'WebApplication1' has the following package references
   [net6.0]: 
   Top-level Package                                Requested   Resolved
   > Azure.Storage.Blobs                            12.14.1     12.14.1 
   > Microsoft.AspNet.Identity.EntityFramework      2.2.3       2.2.3   

   Transitive Package                         Resolved
   > Azure.Core                               1.25.0  
   > Azure.Storage.Common                     12.13.0 
   > EntityFramework                          6.1.0   
   > Microsoft.AspNet.Identity.Core           2.2.3   
   > Microsoft.Bcl.AsyncInterfaces            1.1.1   
   > System.Diagnostics.DiagnosticSource      4.6.0   
   > System.IO.Hashing                        6.0.0   
   > System.Memory.Data                       1.0.2   
   > System.Numerics.Vectors                  4.5.0   
   > System.Text.Encodings.Web                4.7.2   
   > System.Text.Json                         4.7.2   
   > System.Threading.Tasks.Extensions        4.5.4   

I found this useful this week when a large solution’s build pipeline was erroring due to a missing dependency but it wasn’t clear which project needed it.

For more info on the dotnet package command see the Microsoft docs here.

Support multiple JS module formats with rollup

Having recently needed to produce a shared JavaScript npm package for internal sharing of functionality between applications I naively failed to consider the impact of the various competing module formats at large in the JS world, but luckily the solution is straight forward with the help of ‘rollup’.

The requirement was to make a shared JavaScript library to be shared between numerous React/Node applications, and make it available internally via an npm/yarn install by publishing it to an internal registry. All straight forward in theory, however there are competing module formats and you need to either pick one or support both.

The various formats are mainly CommonJS (CJS)(the original NodeJS module system), Async Module Definition (AMD), Universal Module Definition (UMD)(a combination of CommonJS & AMD) and ES6 Modules (ESM). Most require the use of “require” keyword to import modules, or in the case of ES6 modules the use of “import”. So why do I need to care about all this? Wouldn’t it be easier if just use the modern ES6 module format only. Well usually you would pick one to use for your app and mostly ignore the others, but when building a shared library you need to consider what’s used by the consuming application so that it can use your new library. Just using ES6 Modules makes it difficult for apps built using CommonJS to consume the library, and in my case I also found that consuming the library from a majority ES6 module based app was problematic and there is often something that requires the CommonJS format (I’m looking at you Node).

So whilst over time everything will standardise (maybe) on ES6 modules in the meantime I used rollup (https://rollupjs.org) to solve the problem. Rollup module bundler allows you to code in modern ES6 standard and then compile it down into other formats (e.g. CommonJS) in one bundle (or different bundles if you wish).

To make life even easier I found this excellent ts-library-template repo for a rollup based TypeScript JS library which fitted my needs perfectly. Rollup is configured to output both CJS and ESM formats.

output: [
  {
     dir: `dist/${dirname(pkg.main)}`,
     entryFileNames: '[name].js',
     format: 'cjs'
  },
  {
     dir: `dist/${dirname(pkg.module)}`,
     entryFileNames: '[name].mjs',
     format: 'esm'
  }
]

So when the library is compiled it will be able to be be consumed by both CJS and ESM based applications.

Window is undefined during SSR

If when server side rendering a React application (other JS frameworks are available) that makes use of the global window object during the initial render, or perhaps the global document object, then you may get an error stating “window is undefined”. This is because when the app is being rendered on the server side on the web server it is not within the browser environment and therefore it cannot access the global objects that the browser provides.

To resolve this issue you will need to check for the presence and validity of the window object before calling it. This could be done manually in your code but a simple widely used npm package can do the job for you – ‘global‘. A very small lightweight library tor require global variables. It is a common dependency on popular npm packages and so it may already be indirectly referenced in your app anyway, but install via ..

npm i global

Then in your code just add an import for window before using it like below. Now the lack of window object will be handled.

// import the global window variable
import window from 'global';

// you can now use window global object
const url = window && window.location.href ? window.location.href : ''

For more info checkout the npm page for global or the github repo LINK

Links:

https://npmjs.com/package/global
https://github.com/Raynos/global

NodeJS & HTTP Error 431

Photo by Vie Studio on Pexels.com

I recently found error responses from a Node JS microservice with HTTP error “431 Request Header Fields Too Large” but at first it seemed to be intermittent dependent on the test environment being used. Further investigations though found it to be a Node setting on the max header size combined with Node JS version changes and a few large cookies.

Error 431 Request Header Fields Too Large HTTP error indicates that the total size of the request headers (which includes cookies) is too large for the web server to accept. This often occurs where large cookies have built up maxing out the request size.

In 2018 Node (version 11.6.0) was updated to resolve a security vulnerability in this area – Denial of Service with large HTTP headers (CVE-2018-12121) and this resulted in the default max request headers size being reduced to 8kb (from 16kb), more info here (Interestingly 8kb was chosen as it was the NGINX default at the time). The default limit was eventually increased back to 16kb in v13.13.0 which means that if you happen to be running against a Node version between 11.6 and 13.13 then you will hit a 8kb limit but before or after those versions the limit won’t be hit until 16kb – which is the situation I was in recently.

If the default max header size for your node installation is not correct for you then it is easy to configure a new value using –max-http-header-size parameter.

--max-http-header-size=16250

Of course you shouldn’t set this value too high and should instead to configure it as low as feasible for your specific application.

Coding in Spectrum Basic Again

My first computer was a Sinclair ZX Spectrum (16k) with rubber keys which is an icon of the innovative 1980’s micro computer market.

Sinclair ZX Spectrum

On it I learned to code in Sinclair Basic either by reading the manual or typing in programs from the Spectrum magazines of the time. It wasn’t long before one Christmas my ultimate gift arrived…a ZX Spectrum 128k +2….now that was a machine!

ZX Spectrum 128k +2

It’s probably no surprise that the main purpose of my Spectrum was to play games and I played for many hours, but I also learned to program, create databases and do word processing on my Citizen 120D dot matrix printer. All this just seemed like fun at the time but turned out to be useful skills, which I guess is the benefit of combining gaming with utility uses in modern devices like the Raspberry Pi.

Citizen 120D Printer

Recently I’ve been getting nostalgic and rekindle my love of coding on for a Spectrum, but now in the modern PC era we have many many options for how to do this. We have emulators and IDEs and free resources to download quickly (no 10 minute load time 🙂 ). In this post I’m covering a few options to quickly and easily getting started on coding in Sinclair Basic, mostly so I don’t forget when I get nostalgic again in the future.

The most traditional and authentic approach would be to setup an original Spectrum machine and code directly on it, but this is not the easiest option, especially if you don’t have a working spectrum sitting around at home. So we are going down the emulator route here.

Essentially there are two key things we need to code and run our basic program. Firstly we need a compiler to compile our program down into binary and secondly we need an emulator to run the compiled, program. There are lots of options here, and I’ll post links later for where to look for alternatives but below are my current choices.

SpectNetIDE – An all in one solution that runs as a plugin in Visual Studio.

SpecNetIDE is a Visual Studio 2019 plugin enabling you to code your Spectrum program in the excellent Visual Studio IDE and it also includes an emulator so you can run and debug all in one place. Check it out here: https://dotneteer.github.io/spectnetide/

SpectNetIDE

If you already have Visual Studio installed then installing this trying it out is quick and easy. If you dont have Visual Studio then you download the Community edition free from Microsoft. Note this is a Windows only option. Assembly and Basic is supported with basic being implemented via the widely used Python based Boriel compiler which complies Basic down into Z80 Machine code.

After a few setup steps which are well documented then you’re away and coding. Make sure you install V2 if you plan on coding in Basic. Follow the simple documentation to get started with a ZX Basic program. There are some links at the end of this post for guides to coding in ZX Basic and many of the 1980’s manuals are available to download for free.

Once you’ve completed your program you can create TAP file (the equalivant of an old data tape) and play it on other emulators.

‘VS Code > Command Line > Emulator’ option

An alternative option that I have been using is to use VS Code (or any text editor, including Notepad) and then using Boriel compiler via the command line to compile the code to a TAP file and then loading the TAP file into one of the many available emulators. This option runs on Windows and Linux and Mac.

Ewhilst any text editor can be used to code your program, VS Code is argubly the best code editor there is, plus you can install ZX Spectrum plugins to make your programming easier. The one i use is ZX Basic which provides syntax highlighting.

Next we need to be able to compile this down so we install Boriel Compiler from the downloads section (this Python compiler is open source on github here). Check out the installation and quickstart guide on the GitHub page. Once its extracted you can call the compiler via the command line with something like this (which will create a tap file):

./zxb.exe ./code/helloworld.bas --tap --BASIC --autorun

Once compiled into a TAP file then we can run this on any Emultor you like that supports TAP files (which the majority do). My current favourite is the JavaScript based Qaop/JS as it runs in your browser. Click on the sides of the window to open the menu and choose to ‘Open’ your TAP file, and watch your amazing program run in all its 1980’s glory.

Now I just need to learn to code better programs than I did as a child….the links below should help.

Useful Links

Add Git Bash & VS Dev Cmd Prompt Profiles to Windows Terminal

I admit I was not too impressed with the early beta versions of Windows Terminal, maybe because I use Cmder as my daily terminal driver and its features are excellent. However since Windows Terminal reached RTM at v1.0 it does seem a better quality product and with the demo at MS Build 2021 of the new features coming soon I can see that my number one feature request will be released soon – Quake Mode! Quake Mode enables the terminal to drop down from the top of screen on a keypress and as it is a feature of Cmder I use it every day to show/hide the terminal for those quick actions.

Windows Terminal does need some tweaking to get it to look and behave to my tastes, and unfortunately it still doesnt have a proper opacity setting (we have to make do with the smudgy ‘arcylic’ setting instead, but with some it is highly customisable via the settings.json file and settings UI. One thing that you can do is to add your own profiles, so I added my own profile for GitBash.

Assuming you have GitBash installed (via Git) then the below config should work but you’ll need to check your paths. Add this section to your profiles list in the settings.json file which can be found by opening Windows Terminal, choose ‘settings’ from the dropdown, then click the settings icon, the json file will then open. It is usually located somewhere like C:\<YOURUSERNAME>\Rich\AppData\Local\Packages\Microsoft.WindowsTerminal_8wekyb3d8bbwe\LocalState.

{
 "closeOnExit": "always",
 "commandline": "C:\\Program Files\\Git\\bin\\bash.exe -I -l",
 "icon": "C:\\Program Files\\Git\\mingw64\\share\\git\\git-for-windows.ico",
 "name": "GitBash",
 "startingDirectory": "%USERPROFILE%",
 "tabTitle": "GitBash"
}

Its also useful to add a profile for the Developer Command Prompfor VS 2019, like this:

{
 "name": "Developer Command Prompt for VS 2019",
 "commandline": "%comspec%  /k \"%ProgramFiles(x86)%\\Microsoft Visual Studio\\2019\\Community\\Common7\\Tools\\VsDevCmd.bat\"",
 "icon": "%ProgramFiles(x86)%\\Microsoft Visual Studio\\2019\\Community\\Common7\\IDE\\Assets\\VisualStudio.70x70.contrast-standard_scale-180.png",
 "tabTitle": "Dev Cmd VS 2019"
}

This snippet assumes Visual Studio community edition is installed, if using Enterprise then correct the file paths to /Enterprise/ instead of /Community/. If you’re using Cmder and want to read how to add the Developer Command prompt to Cmder check out this post here.

To see the whole file check out my GitHub repo where I’ll keep my latest config file (whilst I remember to update) at https://github.com/RichHewlett/windows-terminal-config.

VS Code Keyboard Mappings in VS 2019

Quick post to let you and ‘future me’ know that Visual Studio 2019 includes the option to use VS Code keyboard mappings out of the box. As someone who has been using VS Code as my development environment much more often than full Visual Studio recently this was a really useful find for me, and one I wasn’t aware of. I had been recently fumbling about in Visual Studio using muscle memory to perform tasks before realising that I am in a different editor and so I need to use the ‘other’ keyboard mappings. I decided enough was enough and so I dug into the Visual Studio settings to tweak some of the keyboard shortcuts and found that you can just tell it to use VS Code mappings instead.

In Visual Studio 2019 > Tools > Options > Environment > Keyboard : Apply following Keyboard Mapping Scheme Visual Studio Code.

So I can now standardise my keyboard mappings across both IDEs and that is one less thing for me to remember.

Referencing External Controllers in ASP.Net Core 3.x

I recently had a situation where I needed to include a utility controller and set of operations into every .Net Core Web API that used a common in-house framework. The idea being that by baking this utility controller in to the framework then every API built that references it can take advantage of this common set of API operations for free. The types of operations that this might include are logging, registration, security or health check type API operations that every Micro Service might need to implement out of the box (some perhaps only in certain environments). Anyway I knew this was possible in .Net Core through the use of ApplicationPart and ApplicationPartManager so I thought I’d code it up and write a blog post to remind ‘future me’ of the approach, however it soon became clear that since .Net Core 3.x this is now automatic so this is the easiest blog post ever. That said there are times when you’ll want to NOT automatically include external controllers and so we’ll cover that too.

Photo by Luca Bravo on Unsplash

External Controllers Added Automatically in .Net 3.x

If you’re using .Net Core 3.x or upwards then ASP.net Core will traverse any referenced assemblies in the assembly hierarchy looking for those that reference the ASP.Net Core assembly and then look for controllers within those assemblies. Any assemblies found are automatically added to the controllers collection and you’ll be able to hit them from your client.

To test this, create a File > New ASP.Net Core Web Project in Visual Studio, then add class libraries to the solution and in those class libraries add Controller classes with the functionality you need. Add the [ApiController] attribute then follow the auto prompt to add the AspNetCore NuGet package. Code your new controller to return something and then add a reference to that new class library project from the original Web API project and you’re done. Run the Web API and try to hit the operation in your new external controller from the browser/postman. It should find the operation in the controller that lives inside the class library project.

This ability to include controllers from external assemblies into the main Web API project has always been very useful but now its just go easier which is great.

Removing automatically added controllers

I can see at least two issues with this automatic approach. Firstly it is a potential security risk as you may inadvertently include controllers in your project via package dependencies and these could be nefarious. This is discussed in more detail in this post along with a way clearing out all ApplicationParts from the collection and then adding the default one back in (reproduced below)

 
 public void ConfigureServices(IServiceCollection services)
{
  services.AddControllers().ConfigureApplicationPartManager(o =>
    {
      o.ApplicationParts.Clear();
      o.ApplicationParts.Add(new AssemblyPart(typeof(Startup).Assembly));
    });
} 

Secondly you may just want to control via configuration which ApplicationParts / Controllers are to be used by the API. In my use case I wanted to be able to allow API developers to be able to disable the built-in utility controllers if required. I can use configuration to drive this and remove the auto discovered controller by Assembly name (as below).

 
 public void ConfigureServices(IServiceCollection services)
{
  services.AddControllers().ConfigureApplicationPartManager(o =>
    {
      // custom local config check here
      if (!configuration.UseIntrinsicControllers)
      {
        // assuming the intrinsic controller is in an assembly named IntrinsicControllerLib
        ApplicationPart appPart = o.ApplicationParts.SingleOrDefault(s => s.Name == "IntrinsicControllerLib");
        if (appPart != null)
        {
         // remove it to stop it being used
         o.ApplicationParts.Remove(appPart);
        }
      }
} 

So as you can see its now easier than ever to add utility controllers into your API from externally referenced projects/assemblies but “with great power comes great responsibility” and so we need to ensure that only approved controllers (or other ApplicationParts such as views) are added at runtime.

Hope you found this useful.

References for further reading:

Creating Linux Desktop Shortcuts

In the previous post about how to hibernate your Ubuntu machine I touched on the concept of creating a desktop shortcut for the hibernate command to make it easy to run the command. That post can be found here. But since then I have been playing with desktop shortcuts some more and so in this post I go into some detail and outline how to create a shortcut icon with content menus too.

The basic concepts with examples…

The basic concept is to create the shortcut as a text file with the “.desktop” file extenstion and in it we define what we want to launch together with some other basic details about the application or script/command, such as icon, name, and whether it needs to run in the terminal etc. Below is an example “.desktop” file, saved to my desktop as “HibernateNow.desktop”. It is running a bash script via the terminal (/hibernatescript.sh).

 
 [Desktop Entry]
Type=Application
Terminal=true
Name=HibernateNow
Icon=utilities-terminal
Exec=gnome-terminal -e "bash -c './hibernatescript.sh;$SHELL'"
Categories=Application; 

From ubuntu 20.04 onwards you may need to right click on the file after creation and choose “Allow Launching”, which is a one time operation (see screenshot).

But lets say that we want to create a shortcut for an application instead of a script. Its the same concept except the Exec path would be the file path to the executable application instead of a script and it would be “terminal=false” as we don’t need to launch the command in the terminal.

In the below example we are launching the Visual Studio Code application which is executed from /usr/share/code/code.

Just save this file on your desktop with a name “.desktop” extension, eg. VSCode.desktop.

 
[Desktop Entry]
Type=Application
Terminal=false
Name=VSCode
Icon=utilities-terminal
Exec=/usr/share/code/code
Categories=Application; 

In a nutshell that’s it for creating application shortcuts in Ubuntu (and many other Linux distros as this is a shared standard). Just create the file and amend the Exec path to be the application you want to launch, then change the name and optionally the icon etc. The category property is for you to optionally choose which group of application types it sits with in the menus.

Taking it further…

But what else can we do with these “.desktop” files? The specification for the files can be found here. As well as the Type=Application option there is also Type=Link for web/document links that can be used with a URL property like this….

 
[Desktop Entry]
Encoding=UTF-8
Name=Google
Type=Link
URL=http://www.google.co.uk/
Icon=text-html 

But this doesnt work for me in Ubuntu 20.10 and it seems this has been removed from Gnome as per the discussion on this GitHub issue. If you want to add a web URL then its possible to use the Type=Application and point to your browser of choice (e.g. Firefox) and pass the URL as a parameter as per below:

 
[Desktop Entry]
Type=Application
Terminal=false
Name=Google
Icon=text-html
Exec=firefox -new-window www.google.co.uk
Categories=Application; 

Notice that the Icon property has been updated to be Icon=text-html so show its a web link. The icons spec can be found here . A good place to find new icons is to poke around in the “.Desktop” files already on your system. On Ubuntu checkout here: /usr/share/applications for a list of your shared system applications shortcuts or /home/.local/share/applications for your users application shortcuts list. If you add your custom “.desktop” file to one of the above file paths then you can see them in your applications menu, and then you can go further by adding it to your Favourites list (right click icon > Add to Favorties), thus making it appear in your launcher (depending on your distro’s launcher toolbar settings).

Adding Context Menu Actions to the shortcuts…

Check out this example below for a shortcut named Test that launches an application (VS Code in this case). Notice the Actions property and associated “Desktop Action” entry
which provides an additional menu item for action within the shortcut (in this case it uses Firefox to navigate to Google). If we create the textfile with the below contents and call it “test.Desktop”, then add it to the /home/.local/share/applications folder, it will appear in the Applications list and also now has a right click context menu which includes “Navigate to Google” (as per the screenshot).

 
[Desktop Entry]
Type=Application
Terminal=false
Name=Test-Shortcut
Icon=com.visualstudio.code
Exec=/usr/share/code/code 
Categories=Application;
Actions=shortcut-action-here;

[Desktop Action shortcut-action-here]
Name=Navigate to Google
Exec=firefox -new-window www.Google.co.uk
Icon=com.visualstudio.code 

And if I add it to my Dock Toolbar then I get the context menu there too.

A Useful GitHub Shortcut Example…

A better fleshed out and usable example is below where the shortcut is to GitHub (via firefox) and there are multiple actions for the context menu options for viewing Profile, Issues and Pull Requests.

 
[Desktop Entry]
Type=Application
Terminal=false
Name=GitHub
Icon=/usr/local/share/github.png
Exec=firefox www.github.com 
Categories=Application;
Actions=Profile;Issues;Pull-Requests

[Desktop Action Profile]
Name=Profile
Exec=firefox https://github.com/richhewlett
Icon=/home/Desktop/github.png

[Desktop Action Issues]
Name=Issues
Exec=firefox https://github.com/issues
Icon=/home/Desktop/github.png

[Desktop Action Pull-Requests]
Name=Pull Requests
Exec=firefox https://github.com/pulls
Icon=/home/Desktop/github.png
 

So as you can see there is quite a lot of useful functionality in the desktop shortcuts on Linux.

Enabling Hibernate on Ubuntu

Personally I prefer to hibernate all my machines instead of shut them down as I find it more efficient being able to carry on from where I left off than to start a fresh. One annoyance with my Ubuntu installation is that Hibernate is not a first class function and needs some tweaking to get it to work. For my own record this post records what I have done to be able to Hibernate my Ubuntu 20.04 install. All credit to the original post on AskUbuntu.com.

For the actual steps checkout the AskUbuntu post but essentially you need to find the Swap partition (assuming you have one), grabs it UUID and then edit the grub file to update the GRUB_CMDLINE_LINUX_DEFAULT entry to add the resume=UUID=XXXXX-XXX-XXXX-XXXX-YYYYYYYYYY value. Finally update the grub file with “sudo update-grub”.

Once setup you can enter hibernate from the terminal with:

sudo systemctl hibernate

Now you may want to add a shortcut to your desktop or to a hot key to run this command without entering the terminal.

For a desktop shortcut check out this post, where it explains how to create a *.desktop file that will launch the script. Create a text file named e.g. HibernateNow.desktop and then add the content of the file like below, assuming that you have created a bash script with the above “sudo systemctl hibernate” command in it called “hibernatescript.sh” :

[Desktop Entry]
Type=Application
Terminal=true
Name=HibernateNow
Icon=utilities-terminal
Exec=gnome-terminal -e "bash -c './hibernatescript.sh;$SHELL'"
Categories=Application;

Then you can double click the HibernateNow.desktop file and hibernate the machine. Or setup a hotkey for this instead.

Happy hibernating.