Support multiple JS module formats with rollup

Having recently needed to produce a shared JavaScript npm package for internal sharing of functionality between applications I naively failed to consider the impact of the various competing module formats at large in the JS world, but luckily the solution is straight forward with the help of ‘rollup’.

The requirement was to make a shared JavaScript library to be shared between numerous React/Node applications, and make it available internally via an npm/yarn install by publishing it to an internal registry. All straight forward in theory, however there are competing module formats and you need to either pick one or support both.

The various formats are mainly CommonJS (CJS)(the original NodeJS module system), Async Module Definition (AMD), Universal Module Definition (UMD)(a combination of CommonJS & AMD) and ES6 Modules (ESM). Most require the use of “require” keyword to import modules, or in the case of ES6 modules the use of “import”. So why do I need to care about all this? Wouldn’t it be easier if just use the modern ES6 module format only. Well usually you would pick one to use for your app and mostly ignore the others, but when building a shared library you need to consider what’s used by the consuming application so that it can use your new library. Just using ES6 Modules makes it difficult for apps built using CommonJS to consume the library, and in my case I also found that consuming the library from a majority ES6 module based app was problematic and there is often something that requires the CommonJS format (I’m looking at you Node).

So whilst over time everything will standardise (maybe) on ES6 modules in the meantime I used rollup (https://rollupjs.org) to solve the problem. Rollup module bundler allows you to code in modern ES6 standard and then compile it down into other formats (e.g. CommonJS) in one bundle (or different bundles if you wish).

To make life even easier I found this excellent ts-library-template repo for a rollup based TypeScript JS library which fitted my needs perfectly. Rollup is configured to output both CJS and ESM formats.

output: [
  {
     dir: `dist/${dirname(pkg.main)}`,
     entryFileNames: '[name].js',
     format: 'cjs'
  },
  {
     dir: `dist/${dirname(pkg.module)}`,
     entryFileNames: '[name].mjs',
     format: 'esm'
  }
]

So when the library is compiled it will be able to be be consumed by both CJS and ESM based applications.

Advertisement

Window is undefined during SSR

If when server side rendering a React application (other JS frameworks are available) that makes use of the global window object during the initial render, or perhaps the global document object, then you may get an error stating “window is undefined”. This is because when the app is being rendered on the server side on the web server it is not within the browser environment and therefore it cannot access the global objects that the browser provides.

To resolve this issue you will need to check for the presence and validity of the window object before calling it. This could be done manually in your code but a simple widely used npm package can do the job for you – ‘global‘. A very small lightweight library tor require global variables. It is a common dependency on popular npm packages and so it may already be indirectly referenced in your app anyway, but install via ..

npm i global

Then in your code just add an import for window before using it like below. Now the lack of window object will be handled.

// import the global window variable
import window from 'global';

// you can now use window global object
const url = window && window.location.href ? window.location.href : ''

For more info checkout the npm page for global or the github repo LINK

Links:

https://npmjs.com/package/global
https://github.com/Raynos/global

NodeJS & HTTP Error 431

Photo by Vie Studio on Pexels.com

I recently found error responses from a Node JS microservice with HTTP error “431 Request Header Fields Too Large” but at first it seemed to be intermittent dependent on the test environment being used. Further investigations though found it to be a Node setting on the max header size combined with Node JS version changes and a few large cookies.

Error 431 Request Header Fields Too Large HTTP error indicates that the total size of the request headers (which includes cookies) is too large for the web server to accept. This often occurs where large cookies have built up maxing out the request size.

In 2018 Node (version 11.6.0) was updated to resolve a security vulnerability in this area – Denial of Service with large HTTP headers (CVE-2018-12121) and this resulted in the default max request headers size being reduced to 8kb (from 16kb), more info here (Interestingly 8kb was chosen as it was the NGINX default at the time). The default limit was eventually increased back to 16kb in v13.13.0 which means that if you happen to be running against a Node version between 11.6 and 13.13 then you will hit a 8kb limit but before or after those versions the limit won’t be hit until 16kb – which is the situation I was in recently.

If the default max header size for your node installation is not correct for you then it is easy to configure a new value using –max-http-header-size parameter.

--max-http-header-size=16250

Of course you shouldn’t set this value too high and should instead to configure it as low as feasible for your specific application.

Referencing External Controllers in ASP.Net Core 3.x

I recently had a situation where I needed to include a utility controller and set of operations into every .Net Core Web API that used a common in-house framework. The idea being that by baking this utility controller in to the framework then every API built that references it can take advantage of this common set of API operations for free. The types of operations that this might include are logging, registration, security or health check type API operations that every Micro Service might need to implement out of the box (some perhaps only in certain environments). Anyway I knew this was possible in .Net Core through the use of ApplicationPart and ApplicationPartManager so I thought I’d code it up and write a blog post to remind ‘future me’ of the approach, however it soon became clear that since .Net Core 3.x this is now automatic so this is the easiest blog post ever. That said there are times when you’ll want to NOT automatically include external controllers and so we’ll cover that too.

Photo by Luca Bravo on Unsplash

External Controllers Added Automatically in .Net 3.x

If you’re using .Net Core 3.x or upwards then ASP.net Core will traverse any referenced assemblies in the assembly hierarchy looking for those that reference the ASP.Net Core assembly and then look for controllers within those assemblies. Any assemblies found are automatically added to the controllers collection and you’ll be able to hit them from your client.

To test this, create a File > New ASP.Net Core Web Project in Visual Studio, then add class libraries to the solution and in those class libraries add Controller classes with the functionality you need. Add the [ApiController] attribute then follow the auto prompt to add the AspNetCore NuGet package. Code your new controller to return something and then add a reference to that new class library project from the original Web API project and you’re done. Run the Web API and try to hit the operation in your new external controller from the browser/postman. It should find the operation in the controller that lives inside the class library project.

This ability to include controllers from external assemblies into the main Web API project has always been very useful but now its just go easier which is great.

Removing automatically added controllers

I can see at least two issues with this automatic approach. Firstly it is a potential security risk as you may inadvertently include controllers in your project via package dependencies and these could be nefarious. This is discussed in more detail in this post along with a way clearing out all ApplicationParts from the collection and then adding the default one back in (reproduced below)

 
 public void ConfigureServices(IServiceCollection services)
{
  services.AddControllers().ConfigureApplicationPartManager(o =>
    {
      o.ApplicationParts.Clear();
      o.ApplicationParts.Add(new AssemblyPart(typeof(Startup).Assembly));
    });
} 

Secondly you may just want to control via configuration which ApplicationParts / Controllers are to be used by the API. In my use case I wanted to be able to allow API developers to be able to disable the built-in utility controllers if required. I can use configuration to drive this and remove the auto discovered controller by Assembly name (as below).

 
 public void ConfigureServices(IServiceCollection services)
{
  services.AddControllers().ConfigureApplicationPartManager(o =>
    {
      // custom local config check here
      if (!configuration.UseIntrinsicControllers)
      {
        // assuming the intrinsic controller is in an assembly named IntrinsicControllerLib
        ApplicationPart appPart = o.ApplicationParts.SingleOrDefault(s => s.Name == "IntrinsicControllerLib");
        if (appPart != null)
        {
         // remove it to stop it being used
         o.ApplicationParts.Remove(appPart);
        }
      }
} 

So as you can see its now easier than ever to add utility controllers into your API from externally referenced projects/assemblies but “with great power comes great responsibility” and so we need to ensure that only approved controllers (or other ApplicationParts such as views) are added at runtime.

Hope you found this useful.

References for further reading:

Free SSL encryption on Azure with Cloudflare

I have a small Web Application site (running ASP.net Core) hosted on Microsoft Azure under a shared plan. Azure shared plans are budget friendly whilst providing features like custom domain names. Until recently the site was HTTP only which was initially fine for my use case but as HTTPS is becoming increasingly a minimum standard on most sites and even browsers are now warning users when navigating to unsecure sites, it was time to move to HTTPS.

Photo by James Sutton on Unsplash

The Problem

Whilst setting up HTTPs certificates has got easier and cheaper there is still usually a trade off between effort and cost. I could have added SSL to my Azure subscription but I couldn’t justify that cost for the site in question. So I needed an easy and cheap solution – and I definitely found it with Cloudflare!

The Solution

I am amazed how easy it was to set up SSL fronting of my existing site using Cloudflare’s SSL.

  1. Create an account at Cloudflare.com
  2. Enter your domain name where the site is hosted.
  3. Confirm the pricing plan (choose FREE)
  4. Cloudflare scans the domain’s DNS entries and asks you to confirm which entries you want proxying. Once done you’ll be given the new NameServers which you’ll need to update on your domain registar’s site.
  5. Once the NameServers have synced (could take minutes to 48 hours) you are done!

There are a few configuration modes for their SSL offering, e.g. full SSL (i.e. SSL from end users to Cloudflare to origin server) or Flexible (i.e. where the SSL stops at Cloudflare and traffic is not encrypted from your origin server to Cloudflare). The options you choose will depend on your setup and your requirements. As my site is hosted on Azure I got Full SSL encryption by default (using Azure’s SSL keys to encrypt traffic from my web server to Cloudflare).

You don’t need an Azure hosted site for this SSL goodness to work as they will front your service wherever its hosted.

This process was so easy I should have done it months ago. For more info checkout these resources:

IIS Express Launch Script

Usually during web development you want to run your web code locally via a local development web server and there are many options for this. In fact most development workflows provide this functionality. For example Visual Studio provides a local web server to run your code, and React/Webpack toolchains usually use NodeJS based solutions. Sometimes though you want to want to fire up something simple to run your code outside your development workflow.

Photo by Aryan Dhiman on Unsplash

The Problem

When developing ReactJS projects I often prefer to run my transpiled bundled code (produced via ‘NPM run build’) in a different way to my development code on a different local web server and, as I usually have IIS Express installed already, I prefer to use that than install new global Node based web server tools. Whilst the command line parameters for IIS Express are simple I have added them to this post to prevent me having to remember them in the future 🙂

The Solution

To run IIS Express from the commandline use:

iisexpress /path:<path_to_files>

You can also specify a port number and other options if you require (see the documentation here).

I prefer to wrap this into a RunOutputBuild.cmd batch file that I can store with the code in the repo and fire up when I want to host my transpiled build output files.

REM Script to launch IIS Express and host the build folder
echo "Launching IIS Express"
cmd /K "c:\program files (x86)\IIS Express\iisexpress.exe" /path:%~dp0build

This assumes the IIS Express installation location is the default one and that the script lives in root of the repo as does the \build folder. The useful %~dp0 key resolves to the local directory where the command script was run from.

I can then just run the command file and browse to the compiled site. Its also possible to add compilation steps and one to fire up the browser automatically if required. Alternatively you could write an NPM Script or Gulp/Grunt step that replicates this functionality to launch IIS Express.

Break your site out of Internet Explorer Compatibility View

Internet Explorer Compatibility mode is a feature of IE that allows you to choose to render sites that targeted older versions of IE when they were developed. It essentially pretends to be IE 8 during rendering which can correct many issues. Microsoft maintains a list of sites that require this compatibility mode and allows users to choose additional sites that should be rendered using this mode. For many large enterprises that were suck on IE 6 because of the need to old legacy systems built to IE6 standards this feature proved very valuable as it enabled them to move their workstations to newer supported version of IE whilst they built replacements for their legacy systems. You can choose to enable Compatibility mode for specific sites or all intranet sites.

alt text

The Problem

So what if you know that the majority of the users of your modern web application are using IE11 but with Compatibility mode on which makes their browser pretend its IE8 and thus unable to make use of new browser features? Unless you build in support for very old browsers via polyfills then those users will see errors and unexpected behaviors.

The Solution

If you want to ensure that users of IE11 or Edge are not Restricted by Compatibility mode then you need to disable it for your site and this is possible by adding a meta tag to your pages. Add the below meta tag in your head tag:

<head>
    <meta http-equiv="x-ua-compatible" content="ie=edge" />
</head>

The page will then disable the Compatibility mode and render the page as Edge compliant modern standards. This is a very useful way to target modern browser features without having to turn off compatibility settings on each client and possibly causing issues with other sites.

For more information checkout these links here and here

Razor Pages Fixes to Tag Helpers Issues

Recently when adding Razor Pages to an existing ASP.net Core MVC web application I had issues with the Tag Helpers not working. No markup was being produced. Not only were the tag helpers (i.e. asp-for) not doing their job of but I also noticed that the markup was not being formatted in bold in Visual Studio as it should be.

At this point I checked for a _ViewImports.cshtml file which I found before checking some other things (see the list below), however I failed to notice that as this was an MVC application with Razor Views there is already a _ViewImports.cshtml file but in the wrong place for my Razor Pages. The _ViewImports.cshtml file must be in the root of the Pages folder where your Razor pages reside, and so you will need one in both the Pages folder for your Razor Pages and also in the Views folder for your MVC Razor Views.

The _ViewImports.cshtml file must contain:

@addTagHelper *, Microsoft.AspNetCore.Mvc.TagHelpers

Adding a new _ViewImports.cshtml file under the Pages folder resolved my problem, but in the meantime here are some additional things to try if you have other Tag Helper problems:

  • Update all dependency Packages
  • Add/re-add Microsoft.AspNetCore.MVC.TagHelpers package if its missing
  • Check your _ViewImports.cshtml file is in the Pages folder or a parent folder of your Razor Pages. You may need one in the Areas folder if you have one.
  • Check that the _ViewImports.cshtml file includes a reference to Microsoft.AspNetCore.Mvc.TagHelpers. If it does try removing it, rebuilding the solution and then re-adding it, and rebuilding again.
  • Check the namespace in the _ViewImports.cshtml file is correct.
  • Should all these fail, try turning on the developer exception page and see if that helps to narrow down the problem.

Cheap Azure Hosting via Static Web Sites

Something that is pretty cool and not that well known is that you can now host your static web site in the cloud with Microsoft Azure just from your Azure storage account. The functionality is currently in preview only but its functional enough to get up and running quickly if you have an Azure account.

Why host a static site?

Whilst it does depend on your requirements many sites are quite capable of being static sites with no server side processing. The classic example is a blog site whereby the site could just serve up static html, images and JavaScript straight from disk as the content changes fairly infrequently.

The growth in JavaScript libraries and the functionality of frameworks like React.js make static sites even more viable. Using the power of JavaScript its possible to create rich powerful web applications that don’t need server side processing. There has been an explosion of static site generators over recent years that will take text or markdown files and generate a complete static site for you. Two very popular generators of note are Gatsby (React.js based) and Jekyll (Ruby) but there are literally hundreds of others as can be seen by this online directory: staticgen.com.

Hosting a static site in Azure

Of course you could always host a static site in Azure if you hosted it in a full featured web site (via a hosted VM or azure web site) but the beauty of a hosting a static only site is that you can host it straight out of storage area and so you don’t need to pay for any compute power which makes it extremely cheap (and even free). You just pay standard Azure storage rates which include a generous data transfer limit (about 5GB a month).

If you think about it hosting a static web site is just a natural extension for a cloud offering like Azure as they already host files and binary content on public URLs in Azure Storage. This new functionality though makes it more explicit and enables web site like functionality such as custom error pages. It is also possible to add your custom domain name to the site and link up SSL (although unfortunately at the moment SSL requires use of an Azure CDN which adds to the cost.)

So how do you host your site, well follow the official instructions here.

Once you have a web page being served by the default Azure storage URL you can proceed to add your own custom domain name using these steps.

Now you should have a fully working site, but to keep costs even lower we can utilise caching of our static content to encourage the client browser to cache the files thus reducing our data transfer costs. Luckily it is easy to set cache control settings on our Azure Blob storage items. This blog post by Alexandre Brisebois covers doing it in code but if you are just testing, or have a site that doesn’t change much you can do it manually via the Azure Portal. To do so enter your Azure Portal, browse to your Storage Account and then using Storage Explorer find the files you want to set caching for and go to their properties. In the Properties dialog you can set the Cache-control value in the HTTP header to something like…

 "public, max-age=86400". 

There are other alternatives to Azure for hosting static files and some offerings are very cheap or free. Some of these are more advanced than the current Azure offering and provide additional features such as integrated SSL and contact forms. One such vendor is netlify.com but there are others.

In summary, if you want to host a site cheaply and you dont really need server side processing then consider hosting a static site, and if you’re already using Azure then its a simple step to give it a go.

.

Developer Roadmaps

Something that’s proving popular on Medium these days are “development roadmaps” that outline a roadmap approach to choosing techniques and technologies for certain technical domains (for example Web development or Dev Ops). Some of these are particularly powerful for putting the many bewildering technologies all on one page with logical grouping and a visual representation of how they interact. Modern web development has seen so much change over recent times that it is very easy to get lost and become overwhelmed and these roadmaps can help clear the fog (a little).

My favourite is the Web Developer Roadmap in 2019 maintained by Kamran Ahmed over on GitHub.

I have shared this with several people who have also found it useful regardless of their level of expertise. The front end roadmap is a great guide to what the community are currently settling on as the standard choices for tooling and techniques. I have checked back to the roadmap a few times over the last 6 months to verify my approach when starting on a new project and I find that visualising the options makes decision making easier.

There are also Backend and DevOps Roadmaps included which are equally as useful.

For some more useful roadmaps check out this medium post.