Modern JavaScript essentials: Concepts and tools

If there's one thing modern JavaScript has it's variety. From the handful of JavaScript projects used by 99% of users, there are hundreds more that cover glaring omissions in these major projects. And from these broader JavaScript projects, there are thousands more that cover edge cases that only a small percentage of users or only their owners know about. This leads to what many in the JavaScript community call JavaScript fatigue: A daily occurrence of new JavaScript projects that creates an almost endless learning curve.

While learning curves are a natural part of technology, the reality is nobody has the time to keep up with thousands of projects of anything no matter how good or what they have to offer. If you've ever felt overwhelmed or intimidated by the amount of JavaScript projects, you're not alone. The good news is you don't need to give up on JavaScript just because you can't keep up with every JavaScript project that offers a better, greater or faster way of doing things. What you need to do is learn modern JavaScript essentials, to not only help you understand and write better JavaScript, but also to help you quickly weed out thousands of JavaScript projects that don't fit your needs.

What you'll read next is a combination of concepts and tools that are essential to working with modern JavaScript. It isn't a JavaScript language treatise or specification summary -- there are many other places you can read that -- it's more of an off the cuff discussion on essentials you really should have no excuse for not knowing if you plan work with JavaScript on a day to day basis.

ECMAScript: ES5, ES5.1, ES6 (or ECMAScript 2015), ES7 (ECMAScript 2016)

ECMAScript [1] is the specification on which JavaScript is based on. As a specification, it means ECMAScript is a blueprint to which JavaScript engines (implementations) must adhere to. JavaScript engines on the other hand are what's included in browsers or other environments to run JavaScript code.

ECMAScript or ES versions like any other language version are a big deal because they represent new features that make a language more powerful and easier to work with (e.g. PHP 5 to PHP 7, Python 2 to Python 3). ES had been relatively stagnant up until 2015 with the appearance of ES6 -- prior to that, ES3 was published in 1999, ES4 was abandoned, ES5 was published in 2009 and ES5.1 was published in 2011. This meant JavaScript had enjoyed very long periods of feature stability.

The release of ES6 in 2015 marked significant changes to address features required by the explosive growth of JavaScript. And the speed of feature changes was so great that ES7 became a reality in 2016. So unlike the first ES/JavaScript major version cycle that took 10 years, the latest ES/JavaScript major version cycle only took 1 year! Why is this important ? Because you'll constantly face situations where plain JavaScript is either ES5.1, ES6 or ES7 compliant. It will be JavaScript all the same, but it won't run on all JavaScript engines because these are in constant flux to support different ES versions. The following is a list of major JavaScript engines and their ES support:

  • SpiderMonkey[2].- Used by the Firefox browser and Adobe Acrobat. Supports ES5.1, and features from ES6 and ES7.
  • V8[3].- Used by the Google Chrome browser, Opera browser and Node.js. Supports ES6.
  • JavaScriptCore[4].- Used by the Apple Safari browser. Supports ES6.
  • Chakra[5].- Used by the Microsoft Edge browser. Supports ES5.1 and features from ES6.

In addition to different ES support across JavaScript engines, another important factor related to ES versioning you'll face is when/if you use languages that produce JavaScript. In such cases you have to produce JavaScript, but you'll always need to ponder the question of whether it will be JavaScript ES5.1, ES6 or ES7 ?

As a rule of thumb, the lower ES version you target the more likely it's that JavaScript will run everywhere. Although higher ES versions are more likely to include JavaScript with 'magical' like features compared to older versions, just be aware that when targeting higher ES versions you run the risk of JavaScript not working on older JavaScript engines, unless of course you use a shim.

Shims and Polyfills

A shim is a generic term used in both the technology and non-technology world. If you happen to sit at a wobbly table or chair, the wooden or paper artifact they use to level it off is called a shim. In the technology world, a shim is a piece of software that allows an old component or API to remain functional in the face of new demands -- so just like a wobbly table or chair, you don't throw it away, you use a shim to keep it working. A specific JavaScript example of a shim is the ES6 shim which allows older JavaScript engines to behave as closely as possible to the newer JavaScript ES6 standard. So if you have an application that uses ES6, it doesn't mean you have to scrap all older JavaScript engines, you can use the ES6 shim to allow older browsers to interpret ES6 features.

A pollyfill is a piece of software that fulfills a feature you expected a browser to support natively. Although very similar to a shim, a pollyfill provides functionality to fulfill something new because support for it isn't there yet, where as a shim provides functionality to fulfill something new for something that's old that you don't want to scrap. This is the 'why' behind the fill in pollyfill, it fills in missing functionality. A lot of modern JavaScript libraries (e.g. Angular, Web components) rely on pollyfills, to be able to run on browsers that don't yet support certain modern JavaScript functionalities natively.

Package managers

Package managers are an essential piece of software for most languages because they help install packages, resolve package dependencies and keep track of an application's requirements (e.g. Java uses maven, PHP uses compose, Python uses pip and Ruby uses gems). For many early or superificial JavaScript users, package managers can sound like overkill, since using a JavaScript package on a project can be as simple as adding a <script> tag that points to a project's CDN and jumping straight to creating JavaScript logic around the package.

Though you may still be able to get away using a couple of <script> tags that point to CDN resources and go straight to implementing JavaScript logic, this practice looks to have its days numbered. For modern JavaScript projects that only use JavaScript, it isn't uncommon to have to manage dozens or hundreds of JavaScript packages, at which point package managers become a necessity.

In JavaScript like in other languages, there are now various package managers to choose from, but the most popular JavaScript package managers are: npm and Bower.

Npm emerged from Node.js which is the dominant platform to run JavaScript on the server. With Node.js being bound to the server, it was only natural the amount and variety of JavaScript packages exploded like most server bound languages (e.g. Java, PHP, Python, Ruby) and so npm was born. There are now over 250,000 JavaScript packages available through npm, ranging from small utility packages for debugging, command line executables, to multiple full-fledged server MVC frameworks to build web applications. Npm has become so popular to manage JavaScript packages, that it's now even used to manage projects unrelated to Node.js (e.g. seeing npm used to manage standalone JavaScript React projects -- which are bound to the browser -- is not uncommon).

Bower is another JavaScript package manager with a wider scope than npm. Unlike npm which focuses on managing JavaScript packages, Bower is designed to manage packages that include JavaScript, HTML, CSS, fonts and even image files. This last focus makes Bower a better fit for JavaScript projects that are bound to the client (i.e. browser) as it automatically manages all these additional bits -- HTML, fonts, images -- that are important in UI (User Interface) development.

There's no hard rule to using npm over Bower, or viceversa. A project's type and the JavaScript packages it requires will usually end up determining which package manager is the best fit. Npm being the most popular is the most referenced package manager (e.g. if you see package 'X' it will have instructions : "To use 'X' use: npm..."), but many UI bound JavaScript packages have complementary Bower instructions (e.g. "To use 'X' you can also use: bower...").

Transpiling and transpilers

Transpiling is a term that refers to converting source code into another type of source code. Technically speaking, transpiling is a specific type of compiling process. Where as you typically compile source code to convert it to machine-code so it can be executed, transpiling just changes the source between programming languages. In JavaScript transpiling has become common with the emergence of languages that produce JavaScript.

Transpiling inevitably introduces another step to modern JavaScript development, that while not insurmountable, must be addressed. Transpilers can be used in various places. You can add a transpiler to a web page so the transpilation process is done on an end user's browser and you can also perform the transpilation process as part of an application's build process, so everything is deployed as plain JavaScript. In addition, there are also online transpilers to convert small snippets of non-JavaScript code into JavaScript code.

It's worth mentioning that loading a transpiler into a web page should only be done for development purposes, because it places additional load on a user's browser. For production environments, it's best to use a transpiler as part of the build process to deliver plain JavaScript to a user's browser, thus reducing load times because the browser is spared the transpilation process.

The dominant JavaScript transpiler is named Babel.

Modules, namespaces & module types

Have you ever noticed early JavaScript versions don't use import/export/include type statements to create modules and namespaces like it's done in other languages such as Java, PHP, Python or Ruby ? JavaScript doesn't use import/export/include statements because it doesn't have any, at least not until recently. The only support for modules and namespaces in JavaScript in the early years was through an ad hoc mechanism that enclosed constructs into a variable that created a pseudo namespace for all its contents, as shown in the following snippet:

    JSHACK = { // JavaScript fields,functions...
        function process() {
         ...
        }
    }
    JSHACK.process() // Call process() function inside JSHACK
  

Modules play an important role in programming languages because they decouple functionality into standalone units (e.g. a module for XML processing, another module for JSON processing), which in turn favors changing these units independently and plugging them into larger projects like building blocks. With modules also comes the concept of namespaces, which means every variable, function, class or construct in a module is given a unique identifier (e.g. xml.process() or json.read()). These unique identifiers or namespaces also play an important role in development because they avoid potential name clashes (e.g. using a common function name like load() without namespaces can be confusing, with a namespace it's unequivocally clear xml.load() or json.load()).

With the explosive growth of modern JavaScript, it was only a matter of time for JavaScript to get its own import/export/include like statements, similar and with the same purpose as those in other programming languages. But it turns out, JavaScript modules was such an important missing piece in modern JavaScript development, that there's not one, two or three ways to implement JavaScript modules, but four!

At this point, it's futile to moan about why there are so many JavaScript module types, the point is they are out there and you'll have to deal with it, at least for the time being. In no particular order, the different JavaScript module techniques are:

  • CommonJS.- Uses the exports and require keywords to reference modules. CommonJS is used extensively in Node.js packages.
  • AMD (Asynchronous Module Definition).- AMD loads JavaScript modules asynchronously (i.e. without blocking). Because an application can require multiple modules of varying size, AMD ensures JavaScript modules are loaded out-of-band (i.e. without the application 'freezing' while modules are loaded).
  • UMD (Universal Module Definition).- UMD appeared to support both AMD and CommonJS features.
  • ES6 modules (Native).- It wasn't until JavaScript's ECMAScript 6 version the language got its own native module syntax. Unfortunately, it came years after the other module techniques came into use. And as you can probably guess, ES6 modules offer the same features as CommonJS (compact and declarative syntax), AMD (asynchronous loading) and some additional features like the ability to deal with circular dependencies.

Module loaders and bundlers

As if having to transpile other languages into JavaScript or having four different JavaScript module systems to deal with wasn't enough, now you'll need to learn about module loaders & bundlers! Fortunately module loaders & bundlers -- which are often the same piece of software -- are used to bring a little sanity to the prospect of transpiling and dealing with various module types.

Let's take a text book example of what module loaders do. You found this exciting package that's a bit dated which uses plain JavaScript and CommonJS modules, but your application must use this other bleeding edge framework that's written in TypeScript and uses ES 6 modules, on top of this you need to use JSX because you want React components. The good is news is you can use a module loader, the bad news is you'll need to use a module loader.

A module loader takes care of unifying whatever discrepancies might exist between JavaScript modules. This means an application can use a CommonJS module & ES 6 module and the loader makes them work together. In addition, a module loader also takes care of the transpiling process, so if an application uses TypeScript or JSX code, it gets transformed into plain JavaScript so it's understandable to a JavaScript engine.

The same piece of software that works as a module loader, is also often a module bundler. A module bundler is useful because it groups an application's modules into a single module. Why would you even want to bundle modules ? In other languages whose logic runs on the server, using a dozen or a hundred modules is an afterthought because all modules are loaded locally. But in JavaScript where it's likely modules need to be transferred over a network to an end user's browser, making a single module can be the difference between quick load times (i.e. one module) and long load times (i.e. a dozen or a hundred modules).

Given the amount of work a module loader/bundler performs, they often require a lot of configuration effort and it can take some time to understand all their capabilities. On top of this, module loaders/bundlers are probably one of the most fragmented segments in modern JavaScript (i.e. there are a lot of options and they all use different techniques).

Some of the more popular module loaders/bundlers include: SystemJS, RequireJS, webpack and browserify.