The Tedium of Managing Code

There’s a place motivation goes to die for web developers. It’s when something we have to do is simultaneously very, very hard and very, very uninteresting. You know, the corner of Hard and Boring Streets in downtown Must-Ship-to-Clientsville. In my personal map of life, that’s the intersection where you’ll find anything that has to do with client-side JavaScript packaging and dependency management.

I have some slides about this in recent talks. They always cause a stir. On one occasion, I had to pause for people to finish cheering. Not at me, but at the notion. There was, I think, relief and amusement at the shared recognition that bundling up and managing our client JavaScript is challenging and, at least for some of us, not our favorite thing to endure.

What are we trying to do here?#section2

A first clue to the wide-flung nature of this beast is that I don’t even have a concise term for what I’m talking about. The summarized goal is that we need to get our various bits of JavaScript put together in a way that a browser can use it. This actually entails several things, as we’ll see, but it can feel like one general objective.

We have left behind the rickety old days, when we wrote and downloaded scripts, tossed them in a directory and stuck them in <script> tags in our web pages. We’re making For-Real Things with JavaScript nowadays, and that comes with the baggage of needing to package, manage, and maintain our code, third-party code, and dependencies.

Starting basic, our first need is to take our application code, package it into one (remember: staying basic here) file, and output it somewhere. Then we can reference the output package in a <script> tag.

This may sound like an exercise in concatentation. But to get distracted by concatenation is to miss the actual thrust of what we need to do: make our code modular and handle dependencies.

You can depend on this#section3

We write code, and as we write it, bits (I’m not going to say modules just yet because that comes in a minute—hang on) of our code need other bits of code from other places. Those needed things—dependencies—might be within our own codebase or external to it.

A primary task is not just to smoosh all of our code together in a package, but to resolve and load the dependencies it needs as part of that process.

That means we need to be able to reference the dependencies we need in a way the packaging tool understands, and the tool needs to know how to find them. Not only that, our code and the code of our dependencies needs to be modularized in the right shape or our tool will rage quit.

Keeping it contained#section4

That is, our code and its dependencies need to use the appropriate module syntax. This is fine when everyone is in the same universe and gets along well, as in the case of pairing npm modules with the popular browserify tool.

browserify can seem so simple and magical. require npm modules that you need in your code just like in node, then bundle it up and, whammo, it spits out a script that works in the browser. So far, so awesome.

But code is modularized and written in different ways—AMD, UMD, CommonJS—or not at all. Some of it might be ES6 (JS 2015 to the cool kids), which we need to transpile.

“Just shim it” and other things easier said than done#section5

There are methods for subduing or translating modules that are in the wrong shape for your packaging tool of choice—packaging tools can be extended or configured to shim wayward modules. But the overhead of managing for many different flavors of modules can be a tedious addition to an increasingly cumbersome process.

Meanwhile, we have additional things to deal with. We also have to manage the source of code and dependencies. Not everything we need for the web comes from npm. Browser-targeted JavaScript has many sources: bower, CDNs, application code from your own repository, third-party code that isn’t managed at all. Fun.

Another common scenario is including a core dependency from a CDN—a classic example is jQuery—within a <script> tag. We need to tell our tool that that dependency is already accounted for, and not to worry about it. And if we can get the config syntax right (grrrr, this one bites me every time), provide that jQuery dependency to our own modules as $ instead of jQuery. Et cetera.

Yes, it’s all very possible#section6

At this point some, maybe many of you are squinting and thinking Come on, it’s not that hard. And, in the grand scheme of things, it’s not. I guess. It’s doable.

But here’s the punchline. Or, at least, the point that makes me want to lie down on the floor for a while. Every single tool or system or combination of tools does each of the things I’ve talked about in a slightly different way. browserify, require.js, webpack, others. Whether that’s the expectation of AMD module syntax or a standalone config file or different command-line options, each solution has its own learning curve that proves remarkably unfun for me when I’d rather be, you know, implementing features. It sabotages my focus.

And then we add more#section7

Any single aspect, like shimming for non-conformant module syntax, can be trivial in isolation, but typically I am at least one layer removed from the packaging by way of a build workflow abstraction. I’m usually looking at my packaging config through a murky lens of gulp or grunt. And often there are other bits at play, like a watch task to spawn packaging builds when relevant code has changed.

It’s a telling sign that the browserify task in my most recent gulp workflows is the only one I don’t fully have a handle on—it’s sourced from a boilerplate example. At one point I went through the code, line by line, and added my own comments, as a learning exercise. For five minutes, I had the whole system glowing and complete in my head. Now I look at the code and it is, once again, soup.

But, ES6!#section8

ES6 (JS 2015) is a significant update to the JavaScript language and has its own, built-in module syntax. And that is great! Especially if we could now go blow up all of the existing code in the world and start over.

Just this morning I was pondering the readme on babel-loader, an ES6 module transpiler and loader. We’ve got a project using webpack and we want to write our own stuff for it in ES6. But now, here I am again. How do I configure webpack correctly vis-a-vis babel-loader? How can I be certain that I can import non-ES6 dependencies into my freshly-minted ES6 modules?

The reality is that even when ES6 support becomes more widespread, there are going to be multiple, co-existing module syntaxes and package managers and unmanaged third-party code. The complexity is a sign that JavaScript has really come of age as the programming language of the web, but mastering this stuff takes some effort. Excuse me while I go off to debug the Uncaught ReferenceError: ufSkpO1xuIl19 is not defined exception that browserify just barked at me.