High Quality Integration Tests

Don't reach for end to end tests

Jan 17, 2023

It’s a common cycle. If you’ve been programming long enough, I’m sure you’ve seen it.

Should we write tests at all?

Turns into: eventually writing unit tests.

At this point, I see many teams move on to a tool like playwright.

Then we start to see complaints about how long tests are taking to complete. Running them in a CI environment can take a really long time with e2e tests.

At this point people either write less tests, or I’ve even seen tests deleted because they have had so many issues over time!

The Unit Path

For some reason, I’ve seen managers over the years assume that any test outside of a browser is a “unit” test. I’ve also found those same managers run teams where a common test looks like this:

test('converts weekly price to annual', () => {
  expect(convertToAnnualPrice('weekly', PlusPlan)).toEqual(728);
});

First, let me say. There is nothing wrong with this test. In fact, it’s a test I have in our own codebase at work.

The problem is, when this is all you have. Nonstop tests confirming function ouput.

When this is all you use over and over, it’s no wonder why teams start to believe they have to do end to end testing to get solid coverage.

The Integration Path

Instead of testing our low level functions. We can test the actual user interface. Triggering clicks and making sure the data displayed is correct, like so:

test(`Renders each plan's pricing for monthly, and changes to yearly`, async () => {
  let screen = render(
    <PlanGroup group={legalBettingToolPlans} cadence="month" />
  );

  screen.getByText('Now $39');
  screen.getByText('Now $199');
  screen.getByText('Now $999');

  await waitFor(() => screen.getByText('Yearly').click());

  screen.getByText('Now $33');
  screen.getByText('Now $166');
  screen.getByText('Now $833');
});

This tests represents a UI like this:

We have pricing cards, and you can change the billing period at the top. This is live on oddsjam.com if you’re interested.

In the test, we change the period, and we make sure the pricing is changed and calculated correctly.

Is this always enough?

In my experience, nearly 90% of the time, this type of user interface testing, using testing-library is enough!

You might be thinking, what if this pricing card is populated by a request to a server. How can we be sure this is working as expected?

Here’s a prime example.

If we structure our frontend and backend code correctly, we can have the same amount of safety without writing end to end, slowwwww tests.

Let’s start by looking at an example UI component. I’ll be using React, but it applies to any framework.

Smart data fetching libraries

type PricingResponse = {
  monthlyPrice: number
  yearlyPrice: number
}

const PlanPricing = () => {
  const {data, isValidating} = useSWR<PricingResponse>('/api/pricing')
  
  if(isValidating) return null

  return (
    <div>
      Monthly: {data?.monthlyPrice}
    </div>
  )
}

In this example, we are using a library called SWR. I chose it because its a simple example for using REST apis. If you’re using GraphQL or protobufs, this becomes even simpler and safer for tests, because you will have automatic TS types for your api responses, compared to my code above.

Mocking API Data

This is the magical part of using SWR, React Query, Apollo, etc:

let screen = render(
    <SWRConfig
      value={{
        fallback: {
          '/api/pricing': { monthlyPrice: 500, yearlyPrice: 1000 },
        },
      }}
    >
      <PlanPricing />
    </SWRConfig>
);

In our tests, we can pass in data as if it was loaded from an api.

Now, I know what you’re thinking. What if the api returns an error, or it doesn’t return the data in the shape we expect!

For the first part, we should have a global error catcher setup. If you do not like that approach, you can access any errors using the useSWR hook. I don’t want to dive into this too much, because every smart data fetching client lets you check for them. You can pass an error in, or trigger one in a test if you want to.

For the next part, I never test for the api returning data incorrectly!

Maybe I am just spoiled, but I make sure my backends are statically typed, and offer something like swagger, or GraphQL.

In either of those cases, our frontend can auto generate the backend response types.

That means, if the backend decided to remove the `monthlyPrice` field, we would know at build time, we don’t need a special test for this.

Backend

These same principles apply to backend code. However I feel it is even simpler and more “unit” like. If you do frontend integration tests. Generate static types for api responses in your frontend, and then write tests to ensure your backend responds correctly… you have really solid test coverage.

End to end isn’t doing anything for you except slowing you down and adding flakiness.

Where E2E is useful

If you have a special authentication system, which kicks out to a third party and you want to ensure users can login. A few tests end-to-end could be useful. There are some scenarios where this is a must. For most products, this is a rare test case and integration tests, by rendering high levels of your app / component code, you can have everything you need.

Conclusion

I would only put a few critical tests in your e2e environment, mainly between systems you don’t have full control over. Otherwise, if you are able to enforce static types between services, and you write integration tests in each service, I don’t how you can have bugs that cannot be tested for at the integration level.

I could dive into this at a much deeper level, and include backend examples of full static typing, but this post was already getting long. I think this gives a high level overview of my approach and it’s worked well for me over the years.

If you have any questions, please leave a comment.

zach.codes

Discussion about this post