A lone black Grackle bird sitting high in a lush green tree, with clear blue skies behind it.

Tooling, Dependencies, and You

Jake Robins - Dev

When I teach web development, a common theme which emerges is the concept of dependency management. Sometimes it's about bringing in little packages like dotenv to leverage an external environment file, sometimes it's larger libraries and frameworks like express, and sometimes it's tooling itself, like sass. Later in your career it can be even bigger decisions like what runtime or even programming language to use, not to mention the whole other world of infrastructure and hosting. It can be tough for a new developer to worry about the philosophy of dependency management when you're still learning fundamental concepts. How can you expect someone to give proper thought to what routing framework their web API projects needs when they hardly know what a route is? However, my philosophy is that bringing in dependencies to a project is a really big decision which has so many downstream effects that I think it's really important to start asking these questions of my students early and often.

Dependency management is a big topic, and many people smarter than me have written about it. I don't have a formal education in software development, so this kind of theoretical or philosophical thinking sometimes gets me into trouble. More than once, a new idea has coalesced before me in my career, and out of excitement over having figured something out, I rush to the internet to see if others have anything to say about it too. It's there that I discover that my "new idea" has been solved for decades and if I had just taken one computer science course in college I'd have already known about it. C'est la vie for a bootcamp grad like me. All this is to say that the philosophies outlined here on dependency management may not be novel or particularly innovative, but I find value in articulating them, and I figured someone out there may find it useful, too.

What is the virtue of a dependency?

Some of my colleagues and students have teased me in the past for being anti-dependency because I ask a lot of questions about the tools I use. It's been especially pronounced lately because in my personal development journey I'm in a sort of Back to Basics era where I'm returning to simple problems and trying to solve them really well. That pendulum swings both ways, of course, but for now it's probably fair to call me a dependency skeptic.

A meme of Taylor swift with the caption "I'm in my Back to Basics Era"

This doesn't mean that I think dependencies are bad! Even a vanilla JavaScript file running on a plain HTML file is dependent on a fortress of software stacked beneath it, from the runtime to the browser to the operating system all the way down to the ones and zeros flying in and out of the user's processor. It's dependencies all the way down. And that's good! For the same reason that you might abstract a piece of logic into a helper function so you can use it in two places, dependencies abstract solved problems away from your code so you can focus on what's new and innovative about your project.

For example, working with dates in software is a well-known swamp of wrong turns and disaster. If I need to work with dates on a project, whether its as simple as displaying a date value in a human-readable way, or if I need to do more complex things like add and subtract, work with ranges, or more, I almost always bring in a dependency to help me out. I'm fond of date-fns for my JavaScript projects (Day.js is another good one). This is because I just plainly do not want to write code that adds 24 hours to 2024-04-16T09:58:44Z and then shows you "April 17th, 2024" in the browser. It's dull, I'll probably mess it up, and someone has already taken the time to do it for me.

So I'm not anti-dependency, even when I ironically use hashtags like #VanillaRevolution. I have extracted and continue to extract a ton of value from software dependencies in my projects and my clients' projects. But what I do believe in is taking the decision to bring in a new dependency very seriously, asking a lot of questions, and putting in place some plans and practices to build resilience in your software.

A Hard Lesson

Being skeptical of dependencies is, as you may have guessed, a learned behaviour. I almost wrote trauma response, but I didn't want to be that dramatic. "Who hurt you, Jake?" you ask. Well, let me share a story that I think about a lot.

A year or so ago I took on a new client and inherited a React application that was bootstrapped with create-react-app (or CRA for short). CRA was the original framework for spinning up new React applications, made by Facebook itself. It is simple, has very few bells and whistles, and I've used it extensively in learning environments where you just wanna bootstrap a new React app to show off this, that, or the other thing. CRA has since been overshadowed by more capable systems, from Gatsby and Next.JS to Remix and Vite. CRA's last release was in April of 2022, now two years ago, and is officially in maintenance mode. And it's starting to show its age. There are over 1,700 open issues on their GitHub repository, and this is the message which greets you when you fire up the development server for the first time:

One of your dependencies, babel-preset-react-app, is importing the
"@babel/plugin-proposal-private-property-in-object" package without
declaring it in its dependencies. This is currently working because
"@babel/plugin-proposal-private-property-in-object" is already in your
node_modules folder for unrelated reasons, but it may break at any time.

babel-preset-react-app is part of the create-react-app project, which
is not maintianed anymore. It is thus unlikely that this bug will
ever be fixed. Add "@babel/plugin-proposal-private-property-in-object" to
your devDependencies to work around this error. This will make this message
go away.

I knew immediately that at some point soon I would have to convince the client to retire CRA and get them on to a modern toolset. The actual work to move it to something like Vite is actually not that difficult. A clean swap is a pull request I can probably put together in an hour or two. But for this client, it ended up being over a year before we could get it done. You see, this particular project had another dependency called CRACO, which is a configuration override tool for CRA. It allows you to change some things about CRA and customize it without "ejecting" it, which is a bit of a destructive operation that breaks open the pre-packaged nature of CRA and exposes its guts forever. CRACO was further extended with the craco-less plugin, which allows a CRA project to use LESS as a CSS tool. All this, so that the project could set up a configuration that looked a lot like the example code on the craco-less README:

module.exports = {
  plugins: [
    {
      plugin: CracoLessPlugin,
      options: {
        lessLoaderOptions: {
          lessOptions: {
            modifyVars: {
              "@primary-color": "#1DA57A",
              "@link-color": "#1DA57A",
              "@border-radius-base": "2px",
            },
            javascriptEnabled: true,
          },
        },
      },
    },
  ],
};

There, buried in the CRACO configuration file, I discovered a massive list, hundreds of lines long, of custom CSS overrides for the project's Ant Design Component library. The overrides touched multiple components that were used on virtually every page of the entire application. And just like that, I realized I was trapped. I couldn't remove CRA without removing CRACO, and I couldn't remove CRACO without removing craco-less, and I couldn't remove craco-less without refactoring the entire site's styling. The previous developers had made a fatal mistake of coupling the individual styles for their pages to their build toolset in a way that was incredibly onerous to fix.

Naturally, replacing CRA was deferred. Over the next year, I began working on various pages and one by one refactoring small amounts of these styles away into more localized overrides. It was laborious. Some pages never got updated in that year, and so the styles remained. My client, like any business, had to prioritize the things that were important to their operation, and so this got pushed and pushed and pushed. But then, one day, something finally came up that forced our hands.

I'll spare you the whole story here, but a combination of changes to our infrastructure and some new features coming out let us to the realization that CRA finally had to go. We needed some new things that Vite offered and we were upgrading our Ant Component library to version 5, which took styling in a different direction and didn't support LESS out of the box. So the client approved the work to clean this up. It was a slog. I wouldn't wish it on my worst enemy.

This story is one where a dependency chain goes bad, and there were a lot of bad decisions leading up to it. The selection of create-react-app in the beginning was probably suspect given more fully-featured frameworks were available at the time, especially since the default configuration of CRA wasn't sufficient for the project's needs. But instead of making a better choice, two more packages were added in just to allow LESS support. There was probably a better styling solution using SASS or even Tailwind at that point. And then the unholy marriage of styles and build tool was icing on this very poor-tasting cake. I vowed to internalize these lessons and ensure I never purposefully inflicted this pain on myself or another dev again.

When to Reach for a Dependency

As developers, we solve problems with code. Whether we use our own code, or we add a package to do it for us, we're using code. The only difference with a dependency, is that it is someone else's code.

I knew this since the beginning as a developer but I never really got it until later in my career. It's someone else's code. What do I know about this person or these persons who wrote it? Are they good developers? Did they use best practices when it comes to performance, security, and code quality? Do they maintain their package and respond to issues raised? Do they in turn have their own dependencies, and are those maintained as well? In the Node ecosystem at least, and in others as well, literally anyone can publish software to the registry. Hell, they even let me do it. Twice! And because of that, there are some hilarious and gnarly npm packages out there. Given then, I think very deeply about bringing this code into my project, because now I'm taking responsibility for someone else's code.

A completely unreadable web of dependencies, connected together by lines that cross in every direction.
Dependency Chart for create-react-app's 1231 dependencies by 765 maintainers

So when should you reach for a dependency? Here are a few common checkpoints I put a solution through before I bring it in.

Is the package substantial?

Can you write this code yourself in a short period of time, including testing it? If so, you should probably do that. There's a really famous example of a tiny npm package gone wrong that I love sharing with students: left-pad. You can read this piece on it for the whole story, but the short version is that after a disagreement between the developer and NPM, a small package called left-pad (just 11 lines of code) was suddenly unpublished, breaking everything upstream of it. And a lot of people used left-pad. It was a dependency of React, for example. The internet kind of broke that day. The memes that surfaced were incredible, like this person who made a left-pad API to query against to replace the lost package. XKCD has a famous comic along the same lines.

Someday ImageMagick will finally break for good and we'll have a long period of scrambling as we try to reassemble civilization from the rubble. Randall Monroe
Dependency

Anyone building JavaScript applications probably has the skill to have written left-pad themselves, and the choice to bring this in as a dependency certainly feels ludicrious, especially in hindsight!

Is the package being maintained?

This one is an obvious one but bears repeating. It's important to understand how much work is being done to maintain any package you bring in to your project. Some things I check include:

  • How many open issues are there in the repository? Are they being addressed by the authors? In issue conversations, are the authors communicating in receptive and productive ways, or are they dismissing problems or ignoring them? How long have issues been open?
  • When was the last release, and are releases happening at regular intervals?
  • Are there published plans for future releases or deprecation?
  • Are the maintainers backed by any kind of organization and does their funding appear stable?

You still end up stuck with a judgement call after checking in on all these pieces, and I don't exactly have specific advice on that. Sometimes, it's obvious that something is well-maintained or not, but sometimes its somewhere in the middle. But with practice you'll learn to start identifying red flags.

What features, opinions or choices of the package might constrain you?

This is maybe one of the hardest questions to answer, especially if you're new to developing. Every package you ever use will have made choices in its own development that will impact how you interact with it and use it in your project. It can be as small a choice as to what arguments can be passed into a function and in what order, to as large as a choice as what software paradigms are supported. Those choices might be fine for you, they might be inconvenient, or they might be show-stoppers. Here's an example I recently went through.

I maintain a React app that uses React Router to manage routing. The app is fairly large, and so I use Code Splitting in order to keep bundle sizes down. In the older versions of React Router (pre 6.4), the code splitting fit in really nicely and was in sync with how React intended it to work. Here's an example:

export default function Page() {
  return <div>My Page!</div>
}
const Page = React.lazy(() => import("./Page"));

function App() {
  return (
    <Routes>
      <Suspense fallback={<div>Loading...</div>}>
        <Route path="/page" element={<Page />} />
      </Suspense>
    </Routes>
  );
}

In this setup, I can just use the regular API that React offers for lazy loading (React.lazy targeting a component with a default export). Now the code associated with Page won't be sent to the user unless they visit /page .

In the newer versions (6.4 or later), React Router introduced a new kind of router that separates the routing from the rendering. In general, I like this approach, but when I migrated I discovered that the setup for lazy loading is a little clunky. The router config looks pretty benign:

const router = createBrowserRouter([
  {
    path: "/page",
    lazy: () => import("./Page"), // Lazy Loaded
  }
])

But in order for this to work, the Page.jsx file has to export a component named Component. This is because the router config file usually expects a Component property in a non-lazy load situation. The results of the lazy function are spread into the containing object, so the default export cannot be used.

import Page from './Page';

const router = createBrowserRouter([
  {
    path: "/page",
    Component: Page, // not Lazy Loaded
  }
])

So, in order to make this work, I had to go back to the Page.jsx file and alter it from a default export, to something that exports a component called Component :

export function Component() {
  return <div>My Page!</div>
}

I really don't like that the component name is now genericized. I have a number of lazy loaded routes and so now all my pages go from semantic names to all being called Component. There are a few workarounds you can do, like setting the displayName property on Component, so that it shows up in the dev tools with a proper name, or creating wrappers, like this:

// Set displayName

Component.displayName = "Page"

// Or, wrap it

function Page() {
  return <div>My Page!</div>
}

export function Component() {
  return <Page />
}

// or build an anonymous function in the router itself for each route

const router = createBrowserRouter([
  {
    path: "/page",
    async lazy() {
      const Page = await import("./Page")
      return { Component: Page }
    }
  }
])

In a half-hour or so of research I wasn't able to find a better solution. Maybe there is one and I can change it later, but this is just one of those compromises I have to make if I want to leverage the new React Router. In the end, I chose to make the compromise because it's not too burdensome, but this is the kind of decision you have to be aware of when bringing in packages. Sometimes it ends up more detrimental than an inconvenient API, and the worst thing ever is coding yourself into a corner that you can't get out of, forcing you to backtrack and remove a dependency all together.

Are you prepared to maintain your project to keep up with your dependency?

Dependencies are not just one time decisions. They are commitments. In the app I mentioned above, we have Dependenbot setup to alert us to security vulnerabilities that come up in our dependencies. In the last year, we've received 35 such alerts, which is roughly one every 10 days. Each of these alerts generally required us to update a package, run our test suite against it to ensure it doesn't cause any regression bugs, then deploy it. Even with our automated deployment pipeline, this usually costs me 15-30 minutes of touch time each, meaning that I may have spent upwards of 20 hours a year just keeping up with our dependencies. That's time that produces no new value for my clients.

Looking back, I still think that time was worth it - I know I've saved more than that amount of time in code I didn't have to write, and I continue to get value from these dependencies as the app is used and I develop new features on it. But your product might be built under different circumstances. For example, if I have a one-time relationship with a client rather than an ongoing one (such as building a static landing page, delivering it, then moving on), I may not be around to help them maintain it, or they may not have the budget for maintenance. Straddling them with a bunch of dependencies that later turn in to security vulnerabilities I won't be around to fix is not something I would take lightly.

Do you understand the solution it's offering?

This last one can be a little controversial, so I'll do my best to flesh it out. A mistake I see students making a lot is to use dependencies to solve problems they don't understand.

Consider this example: You need to create an interface that allows user to drag and drop something on the screen from one list to another. Any browser-based solution here will ultimately need to dig in to the core platform and understand the Drag and Drop API. But what if you've never used this API before? Or worse, what if you didn't know it existed? Many students find themselves here. They've used drag and drop features on other websites so they know it's possible, but they don't know where to start in implementing it. A package like react-dnd, which is a collection of hooks to help developers build drag and drop UI components in a React application, is an easy escape hatch in this situation that can get the job done and let you move on to the next thing.

My challenge to new developers is to resist this temptation, for a number of reasons.

  1. If you end up having to troubleshoot something in the package, you will have a bad time.
  2. You won't be able to make the sound decisions mentioned in the previous section about constraints.
  3. You'll miss an opportunity to learn core features of the Web Platform, which is knowledge that will outlive whatever package of the week you're exploring.

I think of this recommendation as controversial because some people can take away the wrong messages from it. It's ok if you don't understand a package's implementation completely. I don't understand end-to-end how React Router is implemented. But I understand the principles of client-side routing and the fundamentals of the History and Location APIs. If I really wanted to, I could waste a whole bunch of time implementing a minimal version of React Router that would have fewer features and be less well-tested. But that's not productive for me, and so I choose that path from convenience, not necessity. Similarly, I'd recommend something like react-dnd not because you can't figure out drag events, but because you recognize that complex UI elements like that have a ton of edge cases and you don't want to go through all the motions of building and testing them all.

There are also lots of scenarios where this recommendation just isn't realistic or actually counter to your objectives. Standing up something as a prototype under time constraints changes my decision making process, for example. Most importantly, I want to recognize that sometimes, you're just trying to get to the end of the day and you take a shortcut. We've all been there and incurred that technical debt to meet a deployment deadline or just to get to 5PM so you can get some tacos. Trust me, I get it, and if that's the path you gotta go down, no judgement from me. These are philosophies, not laws.

Conclusion

If you take anything away from this blog post, I want it to be that dependencies in important projects are something that should be carefully thought through. Every project, every dependency, and every developer brings a unique circumstance to the decision, and there is no one-size-fits-all answer to "should I npm install this?" But if we are thoughtful, diligent and critical, we can make better decisions as developers, and therefore make better products for our clients.