Building a GraphQL Client For File Uploads

Doing normal GraphQL requests is straight forward, as documented on graphql.org. Anyone can use a normal fetch or curl to do basic requests. However, I haven't seen anyone doing what I am working on. File upload progress.

Since the spec is only in Apollo Server, and not the main GraphQL spec, it seems even harder to figure out how it should be done. (Please correct me if this is wrong). In order to get file progress in JS, you still must use XHR. All the current clients use fetch, which doesn't support this, and most do not have built in file uploading. In this post, I am going to detail my process for building react-use-upload, and how I made my code conform to the spec.

Getting started

When I set out building this library, I wanted to get file uploads working with a normal REST api. This helped me lay a solid foundation for my library. After, I first took a look at how graphql-tag works:

import gql from 'graphql-tag';

const uploadMutation = gql`
  mutation UploadFile($input: UploadFileInput!) {
    uploadFile(input: $input) {
      name
      randomField
    }
  }
`;

I figured since Apollo client uses graphql-tag, my library should as well. It seems like the nicest way to build our queries and mutations on the frontend. By console logging the uploadMutation variable, it was easy to see what we're working with. If we take a look at the github page for graphql-tag you can see what the object looks like that our template tag is transformed into.

Why is this useful? The next spot I had to read up on was the official graphql.org. This article helps us out somewhat, letting us know how a standard post request should look when communicating with a GraphQL server:

{
  "query": "...",
  "operationName": "...",
  "variables": { "myVariable": "someValue", ... }
}

We can get the query and operationName from graphql-tag. This gives us a starting point for our file upload client. At this point in my journey, my code looked similar to this:

var body = new FormData();
  body.append(
    'operations',
    JSON.stringify({
      query: options.mutation.loc.source.body,
      variables: options.variables,
    }),
  );

I knew that my formdata needed to look like this, because the graphql upload spec is used by Apollo Server. I had known this before hand, but funny enough, I didn't need to. If you try doing a normal file upload using formdata, and send it to an Apollo Server using 2.0 or greater, it will return an error mentioning that you need to conform to the spec, with a link to the above repo.

At this point, it should already be obvious how many different places the information is on how to use graphql-tag and building a frontend client. This isn't the fault of any one person or library. GraphQL file uploading seems like it was an after thought. Not being mentioned on the official graphql.org left it to other people to design a way to handle the process.

Creating a File Map

With that being said, the spec mentions creating a file map. What this does is send your variables and mutation info, THEN send the files. I think this was done so that the server can start processing the request before the files are being uploaded inside the formdata. What I created was a mess of code, but it did work:

/*
  This takes a normal graphql variables object, and turns it into the correct
  format for file uploads. The spec can be found here and is built in to 
  Apollo server: https://github.com/jaydenseric/graphql-multipart-request-spec
*/
let createGraphQLFileMap = (
  variables,
  path = 'variables',
  filesMap = {},
  files = [],
  count = -1,
) => {
  Object.keys(variables).forEach(key => {
    if (variables[key] instanceof FileList) {
      [...variables[key]].forEach(file => {
        filesMap[(count += 1)] = [path + '.' + key + '.' + count];
        files.push(file);
      });
    }
    // we need to recursively look through the variables object to find the file list, so we can create the map
    if (typeof variables[key] === 'object')
      return createGraphQLFileMap(
        variables[key],
        path + '.' + key,
        filesMap,
        files,
        count,
      );
  });

  //If there is a single file, we need to remove the .0 on the end.
  if (Object.keys(filesMap).length === 1)
    filesMap[0] = [filesMap[0][0].replace('.0', '')];
  return { filesMap, files };
};

export default createGraphQLFileMap;

This would take our variables object, like so:

{input: {files, name: 'test'}}

and convert it into a map similar to this curl request that the spec had suggested in the readme:

curl localhost:3001/graphql \
  -F operations='{ "query": "mutation($files: [Upload!]!) { multipleUpload(files: $files) { id } }", "variables": { "files": [null, null] } }' \
  -F map='{ "0": ["variables.files.0"], "1": ["variables.files.1"] }' \
  -F [email protected] \
  -F [email protected]

Because of the way FormData works, this is definitely a pain to do. You can't give it an object with variables.files[0] and have it name the FormData key variables.files.0.

My resulting code wasn't looking too bad, since I abstracted the file map logic above:

var body = new FormData();
  body.append(
    'operations',
    JSON.stringify({
      query: options.mutation.loc.source.body,
      variables: options.variables,
    }),
  );

  let { files, filesMap } = createFileMap(options.variables);

  body.append('map', JSON.stringify(filesMap));
  files.forEach((file, index) => body.append(index, file));

This would effectively take in a mutation and variables exactly like a normal Apollo Client mutation would work. I am aiming to keep this hook as simple as possible for end users, all of whom are probably using Apollo on the frontend.

A better way

Turns out, Jayden Seric, the guy behind apollo-upload-client and who helped get the upload spec into Apollo Server, is a really nice guy. He saw me comment on GitHub and left a reply with code samples helping me understand how it all works. It turns out, I wasn't too far off. Even though my code felt messy, it was on the right track.

Here's a snippet of his upload client

const { clone, files } = extractFiles(operation)
  const operationJSON = JSON.stringify(clone)

  if (files.size) {
    // See the GraphQL multipart request spec:
    // https://github.com/jaydenseric/graphql-multipart-request-spec

    const form = new FormData()

    form.append('operations', operationJSON)

    const map = {}
    let i = 0
    files.forEach(paths => {
      map[++i] = paths
    })
    form.append('map', JSON.stringify(map))

    i = 0
    files.forEach((paths, file) => {
      form.append(`${++i}`, file, file.name)
    })

    fetchOptions.body = form

Extract Files

This is definitely much cleaner, and thankfully he has made a package that handles all of the file mapping logic! After looking over this file, I went and checked that out. You can see it for yourself on npm: extract-files

This package handles what I was attempting in my mess of code. It pulls out File objects / lists from an object, and correctly transforms them to match the spec. This is extremely useful for anyone that wants to make their own GraphQL client. Here's my resulting code which builds the formdata after implementing extract-files

  const { clone, files } = extractFiles({
    query: options.mutation.loc.source.body,
    variables: options.variables,
  });

  var body = new FormData();
  body.append('operations', JSON.stringify(clone));

  const map = {};
  let i = 0;
  files.forEach(paths => {
    map[++i] = paths;
  });
  body.append('map', JSON.stringify(map));

  i = 0;
  files.forEach((paths, file) => {
    body.append(`${++i}`, file, file.name);
  });

Yes, it is very similar to his client. The main difference is that with my FormData, I will be using XHR, not fetch. In the end, here's how my React hook works, for those interested:

import React, { useState } from 'react';
import { useUpload } from 'react-use-upload';

import gql from 'graphql-tag';

const uploadMutation = gql`
  mutation UploadFile($input: UploadFileInput!) {
    uploadFile(input: $input) {
      name
      randomField
    }
  }
`;

const GraphQLUpload = () => {
  let [files, setFiles] = useState();

  let { loading, progress } = useUpload(files, {
    mutation: uploadMutation,
    variables: { input: { files, name: 'test' } },
  });

  return (
    <div>
      <div>GraphQL Test</div>
      {loading ? (
        <div>Progress: {progress}</div>
      : (
        <input type="file" onChange={e => setFiles(e.target.files)} multiple />
      )}
    </div>
  );
};

In this example, UploadFileInput takes an array of files, and an extra field called name, just so that I could test sending other data along with the files. If you're interested in making your own client, or playing with react-use-upload, I have full server and client examples in the repo. You can find the server here, which includes normal rest endpoints, and signed upload examples. Client side react code using the hook can also be seen here

I hope this helps understand that GraphQL is really just a special REST endpoint. All the normal interactions you make with a REST api can be done with GraphQL, you just need to conform to its different standards. Sometimes developers think GraphQL is a magical black box that operates outside of the realm of normal REST calls, when in fact, any network request over HTTP can be done with a basic client, no matter the language.