Handling Image Uploads With Gin and Svelte Dropzone

While I expect most of the file/photo operations to happen via a background sync between the web and archive server and a personal computer where the scanning of physical photographs is happening I also want the website to be used as an ongoing archive which means giving other family members not involved in the archiving process the ability to contribute. This means a simple web form where they can go and upload any digital photographs that they may have (e.g., from their phone). We’ll then process those photos and add them to the main archive (what that exact process looks like and how it will work is still TBD).

For the direct-upload form I’ve decided to use the excellent svelte-file-dropzone library to allow people to either drag-and-drop or click-to-upload their photos and then added some extra bits to show image previews and then handle the actual upload (with progress bars because it’s 2023!). It took me a while to get everything linked together because I couldn’t find anyone else who wrote about how to put it all together, though I’m sure there are others who have done so. Here’s how I got it all working together, I hope it can be useful to someone else in the future.

You can seem some screenshots of the final product under the demo header.

tl;dr

Here’s the svelte kit page altogether, a detailed explanation to follow:

<script lang="ts">
  // src/routes/upload/+page.svelte

  import Icon from 'svelte-awesome';
  import Dropzone from 'svelte-file-dropzone/Dropzone.svelte';
  import type DropzoneEvent from 'svelte-file-dropzone/Dropzone.svelte';
  import { filesize } from 'filesize';
  import PQueue from 'p-queue';
  import nopreview from '$lib/assets/nopreview.png';

  import {
    faCircleCheck,
    faCirclePause,
    faCircleXmark,
  } from '@fortawesome/free-solid-svg-icons';

  type Upload = {
    id: number;
    raw: File;
    preview: string | ArrayBuffer | null;
    upload: {
      started: boolean;
      completed: boolean;
      errored: boolean;
      speed: null;
      percent: number;
    };
  };

  let id = 0;
  let uploads: Upload[] = [];
  let uploadStarted = false;

  // currently browsers can only preview a limited number of image types
  const previewMimeTypes = ['image/gif', 'image/jpeg', 'image/png'];

  function handleFilesSelect(e: DropzoneEvent) {
    const { acceptedFiles } = e.detail;

    acceptedFiles.forEach((f: File) => {
      let file: Upload = {
        id: ++id, // this is just a svelte order tracker
        raw: f,
        preview: null,
        upload: {
          started: false,
          completed: false,
          errored: false,
          speed: null,
          percent: 0,
        },
      };

      if (previewMimeTypes.include(f.type)) {
        let reader = new FileReader();
        reader.readAsDataURL(f);
        reader.onload = (e) => {
          if (e.target !== null) {
            file.preview = e.target.result;
          }
          uploads = [...uploads, file];
        };
      } else {
        uploads = [...uploads, file];
      }
    });
  }

  function removeUpload(event: MouseEvent, photoId: number) {
    event.preventDefault();
    uploads = uploads.filter((u) => u.id !== photoId);
  }

  async function doTheUpload(): Promise<void> {
    uploadStarted = true;

    const queue = new PQueue({ concurrency: 2 });

    uploads.forEach(async (upload) => {
      (async () => {
        await queue.add(() =>
          (async () => {
            return new Promise((resolve, reject) => {
              const form = new FormData();
              form.append('photo', upload.raw);

              const xhr = new XMLHttpRequest();
              xhr.open('POST', '/api/upload');

              // must be _after_ xhr.open
              xhr.setRequestHeader('Authorization', '...');

              function progressHandler(event: ProgressEvent) {
                upload.upload.percent = Math.round(
                  (event.loaded / event.total) * 100
                );
                uploads = uploads;
              }
              xhr.upload.addEventListener('progress', progressHandler, false);

              xhr.onload = () => {
                if (xhr.status === 200) {
                  upload.upload.completed = true;
                  uploads = uploads;
                  resolve(true);
                } else {
                  upload.upload.errored = true;
                  uploads = uploads;
                  reject(xhr.status);
                }
              };

              upload.upload.started = true;
              uploads = uploads;
              xhr.send(form);
            });
          })()
        );
      })();
    });
  }
</script>

<template>
  <section class="section">
    <div class="container">
      <Dropzone accept="image/*", on:drop="{handleFilesSelect}" />
    </div>

    <div class="container">
      <div class="columns is-multiline mt-4">
        {#each uploads as upload (upload.id)}
          <div class="column is-3">
            <div class="box">
              <div class="columns">
                <div class="column has-text-centered upload-preview">
                  {#if upload.preview !== null}
                    <img src={upload.preview} alt={upload.raw.name} />
                  {:else}
                    <img src={nopreview} alt="no preview available" />
                  {/if}
                </div>
              <div class="columns is-vcentered">
                <div class="column is-three-quarters">
                  <p class="upload-title"><b>{upload.raw.name}</b></p>
                  <p class="upload-filesize">
                    <i class="is-size-7">{filesize(upload.raw.size, {round: 0})}</i>
                </div>
                <div class="column has-text-centered">
                  {#if !uploadStarted}
                    <button class="delete is-large" on:click={e=>removeUpload(e, upload.id)></button>
                  {:else}
                    {#if upload.upload.started && !upload.upload.completed}
                      <span class="bulma-loader-mixin"></span>
                    {:else if upload.upload.started && !upload.upload.errored}
                      <span class="icon is-large">
                        <Icon data={faCirclePause} scale=2 />
                      </span>
                    {:else if upload.upload.completed}
                      <span class="icon is-large has-text-success">
                        <Icon data={faCircleCheck} scale=2 />
                      </span>
                    {:else}
                      <span class="icon is-large has-text-danger">
                        <Icon data={faCircleXmark} scale=2 />
                      </span>
                    {/if}
                  {/if}
                </div>
                {#if uploadStarted}
                  <div class="columns">
                    <div class="column">
                      <progress class="progress" value={upload.upload.percent} max=100>
                        {upload.upload.percent}%
                      </progress>
                    </div>
                  </div>
                {/if}
              </div>
            </div>
          </div>
        {/each}
      </div>
      <div class="columns mt-4">
        <div class="column is-half is-offset-one-quarter has-text-centered">
          <button class="button" on:click={doTheUpload}>Start Upload</button>
      </div>
    </div>
  </section>
</template>

<style lang="scss">
  @import 'bulma/sass/utilities/_all';

  div.upload-preview {
    max-height: 200px;

    img {
      max-height: 100%;
      max-width: 100%;
    }
  }

  p.upload-title {
    white-space: nowrap;
    overflow: hidden;
    text-overflow: ellipsis;
  }

  @mixin delete-is-large-size {
    // size taken from the bulma delete button is-large
    height: 32px;
    max-height: 32px;
    max-width: 32px;
    min-height: 32px;
    min-width: 32px;
    width: 32px;
  }

  span.bulma-loader-mixin {
    @include loader;
    @include delete-is-large-size;
  }
</style>

Here’s the necessary backend code to handle the upload:

package routes

import (
        "log"
        "net/http"
        "github.com/gin-gonic/gin"
)

func HandleUpload(c *gin.Context) {
        file, err := c.FormFile("photo")
        if err != nil {
                // log or notify error handling service
                c.String(http.StatusInternalServiceError, "")
                return
        }

        // now you can do whatever you need to with the file like save it to
        // disk or a cloud storage
        log.Printf("received a file with name %s", file.Filename)

        // or c.JSON, etc...
        c.String(http.StatusOK, "ok")
}

Finally, later on where you mount your routes:

package main

import "github.com/gin-gonic/gin"
import "example.com/routes"

func main() {
        r := gin.Default()
        r.POST("/api/upload", routes.HandleUpload)
        r.Run()
}

detailed explanation

You’ll have noticed right away that I’m using Typescript for the frontend code so there will be type annotations. If you’re not using Typescript then you can just remove them.

To start, we import our dependencies. As mentioned previously I’m using svelte-file-dropzone to show and handle the actual drag-and-drop functionality. I’m also using svelte-awesome to easily show and inline font awesome icons. I’m using filesize to convert the filesize into human-readable bytes. And, finally, to handle concurrent uploads gracefully the p-queue package.

I also need to import the image that I’m using to show as a preview when we can’t compute a preview (this is because browsers can only handle data URLs for certain image types like GIF, JPEG, and PNG (probably among others like the newer WebP) but we accept all image types including, for example, TIFF. When the user attempts to upload an image in a format that we can’t accept then instead of an actual preview we’ll show them this image instead.

Finally we need to import the font awesome icons that we’ll be using.

import Icon from 'svelte-awesome';
import Dropzone from 'svelte-file-dropzone/Dropzone.svelte';
import type DropzoneEvent from 'svelte-file-dropzone/Dropzone.svelte';
import { filesize } from 'filesize';
import PQueue from 'p-queue';
import nopreview from '$lib/assets/nopreview.png';

import {
  faCircleCheck,
  faCirclePause,
  faCircleXmark,
} from '@fortawesome/free-solid-svg-icons';

Up next is the definition of the Upload type which we’ll be using to keep track of all of the files that the user is trying to upload. We keep an internal id which we can use to let svelte match the for each with the item from the array (more info here). We keep the raw File object that gets passed when selecting files to upload. We have an optional preview to let the user know (when possible as described above) what it is that they’re uploading. And, finally, we keep track of some information that we can use when actually processing the upload: if it’s started yet, it if has completed or errored, a computed upload speed (at the initial time of writing I haven’t implemented this yet, but I’ll update this post when I do), and a whole-number percent of the upload that has completed based on uploaded and total bytes.

type Upload = {
  id: number;
  raw: File;
  preview: string | ArrayBuffer | null;
  upload: {
    started: boolean;
    completed: boolean;
    errored: boolean;
    speed: null;
    percent: number;
  };
};

Now, we set some initial variables that we’ll be using to track state:

let id = 0;
let uploads: Upload[] = [];
let uploadStarted = false;

// currently browsers can only preview a limited number of image types
const previewMimeTypes = ['image/gif', 'image/jpeg', 'image/png'];

We use the id as mentioned before to calculate an internal ID to make use of keyed for each loops in svelte. The uploads array is used to keep a running total of all of the files that we’ll be uploading. The uploadStarted is used as a global marker for when we’ve started the upload process (so that we can show an empty progress bar on all of the uploads at once, for example).

Next we need to actually do something when the user selects or drops files on the dropzone:

function handleFilesSelect(e: DropzoneEvent) {
  const { acceptedFiles } = e.detail;

  acceptedFiles.forEach((f: File) => {
    let file: Upload = {
      id: ++id, // this is just a svelte order tracker
      raw: f,
      preview: null,
      upload: {
        started: false,
        completed: false,
        errored: false,
        speed: null,
        percent: 0,
      },
    };

    if (previewMimeTypes.include(f.type)) {
      let reader = new FileReader();
      reader.readAsDataURL(f);
      reader.onload = (e) => {
        if (e.target !== null) {
          file.preview = e.target.result;
        }
        uploads = [...uploads, file];
      };
    } else {
      uploads = [...uploads, file];
    }
  });
}

First, we get the acceptedFiles, the e.detail also returns a rejectedFiles if necessary, but I’m not exactly sure how to get it to populate or why it might be necessary. For my use-case it’s enough to deal only with accepted files.

We then need to loop over all of those accepted files creating a new Upload entry for each one and initializing all of the upload fields to be not-yet-started. If we’re dealing with an acceptable preview MIME type then we can calculate the preview and update the preview field with the data URL, otherwise we’ll leave the preview field empty which will show our “no preview” image instead. In any case (whether we support a preview or not) we then add the new upload to the uploads array: uploads = [...uploads, file].

Removing files once they’ve been selected is straight forward:

function removeUpload(event: MouseEvent, photoId: number) {
  event.preventDefault();
  uploads = uploads.filter((u) => u.id !== photoId);
}

Here’s the relevant template:

{#if !uploadStarted}
  <button class="delete is-large" on:click={e=>removeUpload(e, upload.id)></button>
{:else}
  <!-- we'll show some different buttons -->
{/if}

Finally, the most complicated part is to actually send the files to the server! We can’t use fetch because it doesn’t support upload progress (yet) so we need to stick with a classic Ajax request (XMLHttpRequest).

The code starts to get a little ugly so that we can handle concurrent uploads: we need to promise-ify the XMLHttpRequest so that we can async/await add it to the p-queue that we’re using to handle concurrency.

If you wanted to just send everything all at once you could do this instead, ignoring the async/await bits:

async function doTheUpload(): Promise<void> {
  uploads.forEach((upload) => {
    const form = new FormData();
    form.append('photo', upload.raw);

    const xhr = new XMLHttpRequest();
    xhr.open('POST', '/api/upload');

    xhr.onload = () => {
      if (xhr.status === 200) {
        upload.upload.completed = true;
        uploads = uploads;
      } else {
        upload.upload.errored = true;
        uploads = uploads;
      }
    };

    upload.upload.started = true;
    uploads = uploads;
    xhr.send(form);
  });
}

Instead as you can see we wrap that like so to make the concurrency work:

async function doTheUpload(): Promise<void> {
  const queue = new PQueue({ concurrency: 2 });

  uploads.forEach(async (upload) => {
    (async () => {
      await queue.add(() =>
        (async () => {
          return new Promise((resolve, reject) => {
            // the XMLHttpRequest happens in here
            // resolve() if it's successful, reject() if it's not
          });
        })()
      );
    })();
  });
}

Inside the loop we create a new FormData and then add the raw File that we got from the dropzone to it.

// "upload" comes from the forEach loop above
const form = new FormData();
form.append('photo', upload.raw);

We then need to create the actual XMLHTTPRequest (we’re doing one file per request, because it makes reporting progress easier, but you could also just send everything all at once).

After “opening” the request (but no data gets sent until you call send) we set an Authorization header (which needs to be done after opening the request but before sending it).

const xhr = new XMLHttpRequest();
xhr.open('POST', '/api/upload');

// must be _after_ xhr.open
xhr.setRequestHeader('Authorization', '...');

To enable progress reporting to the user we need to add an event listener for progress. We just calculate the percent that we’ve uploaded and then when we call uploads = uploads svelte will update the view (and the progress bar) for us.

function progressHandler(event: ProgressEvent) {
  upload.upload.percent = Math.round((event.loaded / event.total) * 100);
  uploads = uploads;
}
xhr.upload.addEventListener('progress', progressHandler, false);

Finally, we need to let the user know what happened once the upload is done. If we get back a 200 response code then we set the upload as completed, otherwise as errored. The frontend will then show the appropriate icon.

xhr.onload = () => {
  if (xhr.status === 200) {
    upload.upload.completed = true;
    uploads = uploads;
    resolve(true);
  } else {
    upload.upload.errored = true;
    uploads = uploads;
    reject(xhr.status);
  }
};

The last bit to do for each loop is to mark each file as upload started so that we show the upload spinner and then send off the request. Again, uploads = uploads is a bit strange but it’s necessary for the way that svelte reactivity works to get it to update the frontend for us.

I won’t spend any time on the template code as I hope it’s self-explanatory enough as the logic is fairly simple and only has a few conditions.

A few brief notes about the styling. In order avoid vertical images from enlarging the box undesirably we set a maximum height on the box itself and then we also need to set the maximum height on the img within.

div.upload-preview {
  max-height: 200px;

  img {
    max-height: 100%;
    max-width: 100%;
  }
}

To prevent long filenames from expanding the box or breaking onto a second line:

p.upload-title {
  white-space: nowrap;
  overflow: hidden;
  text-overflow: ellipsis;
}

And finally, to create a loading spinner we can use the builtin mixin from the bulma library as well as our own mixin to make it mimic the is-large size that we’re using everywhere else:

// provides the loader mixin
@import 'bulma/sass/utilities/_all';

@mixin delete-is-large-size {
  // size taken from the bulma delete button is-large
  height: 32px;
  max-height: 32px;
  max-width: 32px;
  min-height: 32px;
  min-width: 32px;
  width: 32px;
}

span.bulma-loader-mixin {
  @include loader;
  @include delete-is-large-size;
}

Finally, on the backend you can access the uploaded file like you normally would as an enctype="multipart/form-data" form. Using the gin-gonic library for example:

import "github.com/gin-gonic/gin"

func HandleUpload(c *gin.Context) {
        file, _ := c.FormFile("photo")
        // do something with the file
        c.String(200, file.Filename)
}

demo

It’s not a real demo, just a couple of screenshots to show what it looks like.

After selecting some images to upload (courtesy of unsplash): pre-upload preview

Once the upload starts and some images start uploading: upload in-progress

After the upload is complete: upload complete

Links to the images found in the screenshots:

tl;dr#

detailed explanation#

demo#

tl;dr

detailed explanation

demo