Dependence Day: The Power and Peril of Third-Party Solutions

“Why don’t we just use this plugin?” That’s a question I started hearing a lot in the heady days of the 2000s, when open-source CMSes were becoming really popular. We asked it optimistically, full of hope about the myriad solutions only a download away. As the years passed, we gained trustworthy libraries and powerful communities, but the graveyard of crufty code and abandoned services grew deep. Many solutions were easy to install, but difficult to debug. Some providers were eager to sell, but loath to support.

Article Continues Below

Years later, we’re still asking that same question—only now we’re less optimistic and even more dependent, and I’m scared to engage with anyone smart enough to build something I can’t. The emerging challenge for today’s dev shop is knowing how to take control of third-party relationships—and when to avoid them. I’ll show you my approach, which is to ask a different set of questions entirely.

A web of third parties#section2

I should start with a broad definition of what it is to be third party: If it’s a person and I don’t compensate them for the bulk of their workload, they’re third party. If it’s a company or service and I don’t control it, it’s third party. If it’s code and my team doesn’t grasp every line of it, it’s third party.

The third-party landscape is rapidly expanding. Github has grown to almost 7 million users and the WordPress plugin repo is approaching 1 billion downloads. Many of these solutions are easy for clients and competitors to implement; meanwhile, I’m still in the lab debugging my custom code. The idea of selling original work seems oddly…old-fashioned.

Yet with so many third-party options to choose from, there are more chances than ever to veer off-course.

What could go wrong?#section3

At a meeting a couple of years ago, I argued against using an external service to power a search widget on a client project. “We should do things ourselves,” I said. Not long after this, on the very same project, I argued in favor of a using a third party to consolidate RSS feeds into a single document. “Why do all this work ourselves,” I said, “when this problem has already been solved?” My inconsistency was obvious to everyone. Being dogmatic about not using a third party is no better than flippantly jumping in with one, and I had managed to do both at once!

But in one case, I believed the third party was worth the risk. In the other, it wasn’t. I just didn’t know how to communicate those thoughts to my team.

I needed, in the parlance of our times, a decision-making framework. To that end, I’ve been maintaining a collection of points to think through at various stages of engagement with third parties. I’ll tour through these ideas using the search widget and the RSS digest as examples.

The difference between a request and a goal#section4

This point often reveals false assumptions about what a client or stakeholder wants. In the case of the search widget, we began researching a service that our client specifically requested. Fitted with ajax navigation, full-text searching, and automated crawls to index content, it seemed like a lot to live up to. But when we asked our clients what exactly they were trying to do, we were surprised: they were entirely taken by the typeahead functionality; the other features were of very little perceived value.

In the case of the RSS “smusher,” we already had an in-house tool that took an array of feed URLs and looped through them in order, outputting x posts per feed in some bespoke format. They’re too good for our beloved multi-feed widget? But actually, the client had a distinctly different and worthwhile vision: they wanted x results from their array of sites in total, and they wanted them ordered by publication date, not grouped by site. I conceded.

It might seem like an obvious first step, but I have seen projects set off in the wrong direction because the end goal is unknown. In both our examples now, we’re clear about that and we’re ready to evaluate solutions.

To dev or to download#section5

Before deciding to use a third party, I find that I first need to examine my own organization, often in four particular ways: strengths, weaknesses, betterment, and mission.

Strengths and weaknesses#section6

The search task aligned well with our strengths because we had good front-end developers and were skilled at extending our CMS. So when asked to make a typeahead search, we felt comfortable betting on ourselves. Had we done it before? Not exactly, but we could think through it.

At that same time, backend infrastructure was a weakness for our team. We had happened to have a lot of turnover among our sysadmins, and at times it felt like we weren’t equipped to hire that sort of talent. As I was thinking through how we might build a feed-smusher of our own, I felt like I was tempting a weak underbelly. Maybe we’d have to set up a cron job to poll the desired URLs, grab feed content, and store that on our servers. Not rocket science, but cron tasks in particular were an albatross for us.

Betterment of the team#section7

When we set out to achieve a goal for a client, it’s more than us doing work: it’s an opportunity for our team to better themselves by learning new skills. The best opportunities for this are the ones that present challenging but attainable tasks, which create incremental rewards. Some researchers cite this effect as a factor in gaming addiction. I’ve felt this myself when learning new things on a project, and those are some of my favorite work moments ever. Teams appreciate this and there is an organizational cost in missing a chance to pay them to learn. The typeahead search project looked like it could be a perfect opportunity to boost our skill level.

Organizational mission#section8

If a new project aligns well with our mission, we’re going to resell it many times. It’s likely that we’ll want our in-house dev team to iterate on it, tailoring it to our needs. Indeed, we’ll have the budget to do so if we’re selling it a lot. No one had asked us for a feed-smusher before, so it didn’t seem reasonable to dedicate an R&D budget to it. In contrast, several other clients were interested in more powerful site search, so it looked like it would be time well spent.

We’ve now clarified our end goals and we’ve looked at how these projects align with our team. Based on that, we’re doing the search widget ourselves, and we’re outsourcing the feed-smusher. Now let’s look more closely at what happens next for both cases.

Evaluating the unknown#section9

The frustrating thing about working with third parties is that the most important decisions take place when we have the least information. But there are some things we can determine before committing. Familiarity, vitality, extensibility, branding, and Service Level Agreements (SLAs) are all observable from afar.

Familiarity: is there a provider we already work with?#section10

Although we’re going to increase the number of third-party dependencies, we’ll try to avoid increasing the number of third-party relationships.

Working with a known vendor has several potential benefits: they may give us volume pricing. Markup and style are likely to be consistent between solutions. And we just know them better than we’d know a new service.

Vitality: will this service stick around?#section11

The worst thing we could do is get behind a service, only to have it shut down next month. A service with high vitality will likely (and rightfully) brag about enterprise clients by name. If it’s open source, it will have a passionate community of contributors. On the other hand, it could be advertising a shutdown. More often, it’s somewhere in the middle. Noting how often the service is updated is a good starting point in determining vitality.

Extensibility: can this service adapt as our needs change?#section12

Not only do we have to evaluate the core service, we have to see how extensible it is by digging into its API. If a service is extensible, it’s more likely to fit for the long haul.

APIs can also present new opportunities. For example, imagine selecting an email-marketing provider with an API that exposes campaign data. This might allow us to build a dashboard for campaign performance in our CMS—a unique value-add for our clients, and a chance to keep our in-house developers invested and excited about the service.

Branding: is theirs strong, or can you use your own?#section13

White-labeling is the practice of reselling a service with your branding instead of that of the original provider. For some companies, this might make good sense for marketing. I tend to dislike white-labeling. Our clients trust us to make choices, and we should be proud to display what those choices are. Either way, you want to ensure you’re comfortable with the brand you’ll be using.

SLAs: what are you getting, beyond uptime?#section14

For client-side products, browser support is a factor: every external dependency represents another layer that could abandon older browsers before we’re ready. There’s also accessibility. Does this new third-party support users with accessibility needs to the degree that we require? Perhaps most important of all is support. Can we purchase a priority support plan that offers fast and in-depth help?

In the case of our feed-smusher service, there was no solution that ran the table. The most popular solution actually had a shutdown notice! There were a couple of smaller providers available, but we hadn’t worked with either before. Browser support and accessibility were moot since we’d be parsing the data and displaying it ourselves. The uptime concern was also diminished because we’d be sure to cache the results locally. Anyway, with viable candidates in hand, we can move on to more productive concerns than dithering between two similar solutions.

Relationship maintenance#section15

If someone else is going to do the heavy lifting, I want to assume as much of the remaining burden as possible. Piloting, data collection, documentation, and in-house support are all valuable opportunities to buttress this new relationship.

As exciting as this new relationship is, we don’t want to go dashing out of the gates just yet. Instead, we’ll target clients for piloting and quarantine them before unleashing it any further. Cull suggestions from team members to determine good candidates for piloting, garnering a mix of edge-cases and the norm.

If the third party happens to collect data of any kind, we should also have an automated way to import a copy of it—not just as a backup, but also as a cached version we can serve to minimize latency. If we are serving a popular dependency from a CDN, we want to send a local version if that call should fail.

If our team doesn’t have a well-traveled directory of provider relationships, the backstory can get lost. Let a few months pass, throw in some personnel turnover, and we might forget why we even use a service, or why we opted for a particular package. Everyone on our team should know where and how to learn about our third-party relationships.

We don’t need every team member to be an expert on the service, yet we don’t want to wait for a third-party support staff to respond to simple questions. Therefore, we should elect an in-house subject-matter expert. It doesn’t have to be a developer. We just need somebody tasked with monitoring the service at regular intervals for API changes, shutdown notices, or new features. They should be able to train new employees and route more complex support requests to the third party.

In our RSS feed example, we knew we’d read their output into our database. We documented this relationship in our team’s most active bulletin, our CRM software. And we made managing external dependencies a primary part of one team member’s job.

DIY: a third party waiting to happen?#section16

Stop me if you’ve heard this one before: a prideful developer assures the team that they can do something themselves. It’s a complex project. They make something and the company comes to rely on it. Time goes by and the in-house product is doing fine, though there is a maintenance burden. Eventually, the developer leaves the company. Their old product needs maintenance, no one knows what to do, and since it’s totally custom, there is no such thing as a community for it.

Once you decide to build something in-house, how can you prevent that work from devolving into a resented, alien dependency?

  • Consider pair-programming. What better way to ensure that multiple people understand a product, than to have multiple people build it?
  • “Job-switch Tuesdays.” When feasible, we have developers switch roles for an entire day. Literally, in our ticketing system, it’s as though one person is another. It’s a way to force cross-training without doubling the hours needed for a task.
  • Hold code reviews before new code is pushed. This might feel slightly intrusive at first, but that passes. If it’s not readable, it’s not deployable. If you have project managers with a technical bent, empower them to ask questions about the code, too.
  • Bring moldy code into light by displaying it as phpDoc, JSDoc, or similar.
  • Beware the big. Create hourly estimates in Fibonacci increments. As a project gets bigger, so does its level of uncertainty. The Fibonacci steps are biased against under-budgeting, and also provide a cue to opt out of projects that are too difficult to estimate. In that case, it’s likely better to toe-in with a third party instead of blazing into the unknown by yourself.

All of these considerations apply to our earlier example, the typeahead search widget. Most germane is the provision to “beware the big.” When I say “big,” I mean that relative to what usually works for a given team. In this case, it was a deliverable that felt very familiar in size and scope: we were being asked to extend an open-source CMS. If instead we had been asked to make a CMS, alarms would have gone off.

Look before you leap, and after you land#section17

It’s not that third parties are bad per se. It’s just that the modern web team strikes me as a strange place: not only do we stand on the shoulders of giants, we do so without getting to know them first—and we hoist our organizations and clients up there, too.

Granted, there are many things you shouldn’t do yourself, and it’s possible to hurt your company by trying to do them—NIH is a problem, not a goal. But when teams err too far in the other direction, developers become disenfranchised, components start to look like spare parts, and clients pay for solutions that aren’t quite right. Using a third party versus staying in-house is a big decision, and we need to think hard before we make it. Use my line of questions, or come up with one that fits your team better. After all, you’re your own best dependency.

About the Author

Scott Fennell

Scott Fennell is a front-end developer in Anchorage, Alaska, who worries about dependence on third-party meme generators when bickering with coworkers. He blogs about code and ski dogs, but you should probably run through a decision-making framework before reading him.

9 Reader Comments

  1. Great article, very sound advice. I’ve been a junior developer in this situation a few times, but with no say on which way the client goes.

    I can really relate to that feeling of not being able to articulate why we should go in one direction or the other.

  2. From my experience I can say only that third-party components come in very handy when we are talking about stuff like D3 and/or NVD3. These kind of libraries take years to develop and include a lot of specific knowledge.

    At that point these are the only ones we are using as third-party components, and we got to them after initially tried to develop software that draws chart and tables in canvas by ourselves.

    Time has proven that DYI libraries/components/classes or however you want to call them in the context of your application and programming language, is far better. Even a few lines of commented code on specific places can clear up a lot of confusion and experienced developers rarely struggle with understanding and maintaining them.

    On the subject of scale-ability and customization, in-house developed code is better; given that there is sound architecture behind it and it’s not developed solely by an intern.

    Bottom line, if you have skilled people in the team who know what and how to do it, you are better off with DYI solutions for most of the cases. Long-term that is the only solution proven effective in our company’s field*.

    *Sports betting and gambling software.

  3. @Iain Simmons,
    Thank you for the kind words!

    One thing I didn’t really have space for in the article, but I think is interesting, is the idea of dependancies that rise to the level of being “beyond reproach”. Stuff that’s remarkably stable, effectively impossible for your shop to reproduce, and also critically important to your core business. From your comment, it sounds like you do a lot of charts and data graphics, so it makes sense that something like NVD3 would be a candidate for that status. For me, jQuery and WordPress core are probably the closest. I’m not suggesting we abandon due diligence when updating or installing those libraries (never!), I mean it just as an interesting thought exercise: What are my most exalted dependencies, and what do they have in common?

  4. One other advantage to developing in-house is less bloat. Third-party libraries have all sorts of stuff you’ll never need. This is less the case with JavaScript, where developers are more careful about code size. But it’s also more dangerous, because the DOM has global scope and CSS rules often have unintended consequences.

    The flip side of this, of course, is the third-party libraries often have features you’ll ultimately need, that you’ll get before you need them.

  5. Great article.

    As the lone developer at a small creative agency, I’m faced with decisions involving third-party libraries or apps all the time. Often, it’s about saving time but also about solving problems that are over my head.

    But I’ve also been on the management side, making calls about whether we stay in-house or look outside for a given solution. I usually erred on the side of in-house for critical pieces of development, often citing the employees’ experience gained as part of my decision.

    I cannot stress enough how important code review and documentation are for in-house development. My first experience with team management involved an employee leaving under bad circumstances two months into my tenure, only for me to discover how little he documented code for business-critical applications.

  6. Great article and an interesting checklist.
    From a UX prospective, libraries make customization of a component extremely difficult. I’ve watched requirement watered down in order to support Dev teams “its good enough” mentality. Component breed laziness, and very few internal teams ever stand up and say “wow, I’d really like to learn something new.” Product management sees a deliverable met, and without concern for how the user will engage the system. The UX teams, revolt because the watered down requirement + component don’t support the full end goal. In the end, higher calls to Customer Service because ‘its good enough’ wasn’t good enough for the user, and half the teams half quit from either boredom or annoyance. Components are built by developers and developers aren’t focused on providing the best user experience.

  7. Really interesting article. There are so many pros and cons to using third party solutions and I think you summarised them nicely.

    Support is probably the biggest issue that needs considering, if a plugin or service doesn’t have a good community or support service then I probably wont use it.

  8. Interesting article. I find the easiest way to avoid the ambivalent nature of 3rd party applications is to speak to people. I use the community around any application to get an idea of how I may fare If I use the code. Through my work at boxChilli I have created a network of individuals who I can approach to discuss the potential dangers of using specific 3rd party applications before going ahead and wishing I hadn’t (which has happened to many of us I am sure).

  9. Nice article. A lot of projects are dependent on third party developments which has its own pros and cons. For searching a right solution provider or specifically a CMS development company, it is important to understand the business requirements and identify the key concerned areas. As there are number of players into enterprise content management, launch a search for the one which can fulfill the business needs and suits the processes that business follows. Select a platform that is secured, flexible and user friendly. Better if it can be used through mobile devices. For large businesses, it is advisable to have a CMS that supports multiple website in various languages.
    It is always better to plan out the requirements and search the most efficient and reliable third party solution.

Got something to say?

We have turned off comments, but you can see what folks had to say before we did so.

More from ALA