Annotate Images With JavaScript

In this tutorial we use Pintura, a JavaScript image editor, to quickly create a JavaScript image annotation solution to annotate and decorate user submitted images and photos.

Our goal is to allows users to capture a photo using their mobile phone camera and then add annotations to it.

We’ll start by setting up a basic HTML form, we’re not going to reinvent the wheel so let’s use Pintura a JavaScript image editor SDK to handle the image annotation process.

Setting Up The Initial Code

We’ll start with a file input field, as that’s the most common way to request an image file from the user. We add the accept attribute to limit the file selection to images.

<input type="file" accept="image/*" />

Now to catch the image we listen for the 'change' event to fire, when it fires we get the first image file from the file input element.

<input type="file" accept="image/*" />

<script>
    document.querySelector('input').addEventListener('change', (e) => {
        const imageFile = e.target.files[0];

        // No image file selected, so nothing to edit
        if (!imageFile) return;

        // We have an image file, but what to do now?
        // ...
    });
</script>

Note that when the user clicks the cancel button in the browse file system dialog, the change event will still fire but the files property will be an empty FileList, that’s why we check if imageFile is actually defined.

This is the result, it doesn’t do anything, yet.

With our HTML file input ready we can now add Pintura to handle the image annotation process.

Loading The Image Editor

We’ll only load the Pintura module after an image file has been selected. This speeds up the initial page load as the editor won’t be needed when a user skips our file upload field.

<input type="file" accept="image/*" />
<script>
    document.querySelector('input').addEventListener('change', (e) => {
        const imageFile = e.target.files[0];
        if (!imageFile) return;

        // We have an image file, let's dynamically load Pintura
        import('./pintura.js').then(({ openDefaultEditor }) => {
            // Pintura has loaded, let's edit the image
            const editor = openDefaultEditor({
                src: imageFile,
            });

            // Pintura returns the resulting image and we store it in the file input
            editor.on('process', ({ dest }) => {
                const dataTransfer = new DataTransfer();
                dataTransfer.items.add(dest);
                e.target.files = dataTransfer.files;
            });
        });
    });
</script>

We’re going to have to improve the file input UX a tiny bit.

Note that if you just want get on with annotating images skip ahead to the image annotation section

Improving The File Input UX

This is what currently happens:

  1. When an image file is selected the Pintura JavaScript module will be loaded.
  2. When the module is finished loading Pintura will open the image file in a modal.
  3. When the user is finished editing the file it is saved back to the file input.

This “works” but creates a problem where if the editor module takes a bit longer to load in step 1, the interface doesn’t give any feedback and the user is left wondering about the state of things.

To fix this we need to add an image loading indicator, it’ll let the user know what’s going on while waiting for the editor to load.

<input type="file" accept="image/*" />
<script>
    document.querySelector('input').addEventListener('change', (e) => {
        const imageFile = e.target.files[0];
        if (!imageFile) return;

        // Show loading indicator
        const loadingState = document.createElement('span');
        loadingState.className = 'loading';
        loadingState.textContent = 'Loading image…';
        e.target.before(loadingState);

        // We have an image file, let's dynamically load Pintura
        import('./pintura.js').then(({ openDefaultEditor }) => {
            // Now ready
            loadingState.remove();

            // Pintura has loaded, let's edit the image
            const editor = openDefaultEditor({
                src: imageFile,
            });

            // Hide loading indicator when the editor modal is fully visible
            editor.on('show', () => {
                loadingState.remove();
            });

            // Pintura returns the resulting image and we store it in the file input
            editor.on('process', ({ dest }) => {
                const dataTransfer = new DataTransfer();
                dataTransfer.items.add(dest);
                e.target.files = dataTransfer.files;
            });
        });
    });
</script>
<style>
    /* Hide the file input while loading */
    .loading ~ * {
        display: none;
    }
</style>

Excellent!

There’s one final UX thing to deal with.

When we save the image back to the file input nothing changes, the file name is the same, so the file input still looks the same.

We need to show that we’ve actually edited the image.

We’ll do this by adding a tiny image thumbnail before the file input. It’ll show the currently selected image and give the user confidence that the changes have been applied successfully.

<input type="file" accept="image/*" />
<script>
    document.querySelector('input').addEventListener('change', (e) => {
        const imageFile = e.target.files[0];
        if (!imageFile) return;

        // This adds or updates the current image preview
        const showPreview = () => {
            // Create a new preview or re-use existing one
            const image =
                e.target.parentNode.querySelector('.preview') || new Image();
            image.className = 'preview';

            // We don't want our preview to be too big
            image.width = 32;

            // If already loaded a preview, need to unload
            image.src && URL.revokeObjectURL(image.src);

            // Set the src of the preview image to the file object in the file input
            image.src = URL.createObjectURL(e.target.files[0]);

            // Attach/move the preview to before the file input
            e.target.before(image);
        };

        // Render a tiny image preview of the currently selected file
        showPreview();

        // Show loading indicator
        const loadingState = document.createElement('span');
        loadingState.className = 'loading';
        loadingState.textContent = 'Loading image…';
        e.target.before(loadingState);

        // We have an image file, let's dynamically load Pintura
        import('./pintura.js').then(({ openDefaultEditor }) => {
            // Pintura has loaded, let's edit the image
            const editor = openDefaultEditor({
                src: imageFile,
            });

            // Hide loading indicator when the editor modal is fully visible
            editor.on('show', () => {
                loadingState.remove();
            });

            // Pintura returns the resulting image and we store it in the file input
            editor.on('process', ({ dest }) => {
                const dataTransfer = new DataTransfer();
                dataTransfer.items.add(dest);
                e.target.files = dataTransfer.files;

                // Update our image preview with the edited file
                showPreview();
            });
        });
    });
</script>
<style>
    /* Styles the preview to be similar to the file input button */
    .preview {
        display: inline-block;
        width: 1.5rem;
        height: 1.5rem;
        object-fit: cover;
        margin-right: 0.3125rem;
        vertical-align: bottom;
        border-radius: 0.175rem;
    }

    /* Hide the file input and image preview while loading */
    .loading ~ * {
        display: none;
    }
</style>

While we side tracked quite a bit, it gave us some greate insight into what is needed to offer a good user experience.

Add an image to the field below to show the loading state and preview thumbnail.

That. Is. Nice. We’re finally ready to annotate images.

Annotating Images

While Pintura offers a wide range of tools to edit images we’re only interested in the annotation tool, in the next step we’ll limit the toolset to the annotation tool.

Note that starting from this code snippet we’re only altering the editor code, the rest of the code is hidden.

import('./pintura.js').then(({ openDefaultEditor }) => {
    const editor = openDefaultEditor({
        src: imageFile,

        // Only show the annotation tool
        utils: ['annotate'],
    });
});

The demo below reflects our changes. We’ve only selected one tool so the editor hides the toolbar creating more space for the active tool.

Try it out in the demo below, we can annotate our image with text, draw lines, and add shapes like circles and squares.

Adding Stickers

To make our annotation editor more powerful we can add stickers.

Stickers can be emoji, images, or shapes.

import('./pintura.js').then(({ openDefaultEditor }) => {
    openDefaultEditor({
        src: imageFile,
        utils: ['annotate'],

        // Set the stickers tool as the active tool
        annotateActiveTool: 'preset',

        // Add some stickers
        annotatePresets: [
            [
                // Add image based stickers
                'Numbers',
                ['sticker-one.svg', 'sticker-two.svg', 'sticker-three.svg'],
            ],
            [
                // Add emoji stickers
                'Emoji',
                ['👍', '👎'],
            ],
            [
                // Add shape stickers
                'Shapes',
                [
                    // A speech bubble
                    {
                        thumb: '💬',
                        alt: 'Speech cloud',
                        shape: {
                            width: 256,
                            height: 128,
                            fontSize: 64,
                            backgroundColor: [1, 1, 1, 0.85],
                            color: [0, 0, 0],
                            cornerRadius: 10,
                            text: 'Hello World',
                        },
                    },
                ],
            ],
        ],
    });
});

Let’s try out the stickers below.

Dynamic Annotation Shapes

Now for our final trick we’ll add a dynamic shape.

Imagine you need an annotation tool to measure the distance in pixels between two points. We can achieve that by adding a line shape that automatically expands to a text showing the distance between the start and end point.

In the code snippet below we first add the "measure" shape preset, this is the preset that we’ll expand in the next step.

import('./pintura.js').then(({ openDefaultEditor }) => {
    openDefaultEditor({
        src: imageFile,
        utils: ['annotate'],
        annotateActiveTool: 'preset',
        annotatePresets: [
            [
                'Numbers',
                ['sticker-one.svg', 'sticker-two.svg', 'sticker-three.svg'],
            ],
            ['Emoji', ['👍', '👎']],
            [
                'Shapes',
                [
                    {
                        thumb: '💬',
                        alt: 'Speech cloud',
                        shape: {
                            width: 256,
                            height: 128,
                            fontSize: 64,
                            backgroundColor: [1, 1, 1, 0.85],
                            color: [0, 0, 0],
                            cornerRadius: 10,
                            text: 'Hello World',
                        },
                    },
                    // Add our dynamic measure shape
                    {
                        thumb: '📏',
                        alt: 'Measure',
                        shape: {
                            style: 'measure',
                            x1: 0,
                            y1: 0,
                            x2: 100,
                            y2: 100,
                            strokeWidth: '.5%',
                            strokeColor: [0.5, 0.85, 1],
                            lineEnd: 'bar',
                            lineStart: 'bar',
                        },
                    },
                ],
            ],
        ],
    });
});

Now we need to automatically expand shapes with the 'measure' style property to show a text box, for this we’ll use a custom shape preprocessor.

For brevity the example below only shows the shapePreprocessor code.

import('./pintura.js').then(
    ({ openDefaultEditor, shapeGetCenter, shapeGetLength, shapeGetLevel }) => {
        openDefaultEditor({
            // ... other editor properties here

            // Set our custom shape preprocessor
            shapePreprocessor: [
                (shape, options) => {
                    // Should return undefined if shape is not a match
                    if (!shape.style || shape.style !== 'measure') return;

                    // Remove the `style` property from the cloned shaped so it no longer matches after processing
                    const { style, ...clone } = shape;

                    // Calculate the font size scalar so we can always read the font size info no matter the size of the image
                    const scalar = options.isPreview
                        ? Math.min(1, options.scale * 2)
                        : 1;

                    // Calculate line midpoint
                    const center = shapeGetCenter(clone);

                    // Calculate line length
                    const length = shapeGetLength(clone);

                    // Properties for the text label
                    const text = `${Math.round(length)}px`;
                    const fontSize = 24 / scalar;
                    const width = text.length * fontSize * 0.75;
                    const lineHeight = fontSize * 1.5;

                    // Return the cloned line shape and the added text shape
                    return [
                        clone,
                        {
                            width,
                            lineHeight,
                            text,
                            fontSize,
                            textAlign: 'center',
                            fontFamily: 'monospace',
                            color: [1, 1, 1],
                            backgroundColor: [0, 0, 0],
                            cornerRadius: 8,

                            // This places the text box in the middle of the line
                            x: center.x - width * 0.5,
                            y: center.y - lineHeight * 0.5,

                            // This makes sure the text box isn't flipped or rotated
                            ...shapeGetLevel(clone, options),
                        },
                    ];
                },
            ],
        });
    }
);

Let’s take a look at the final code, or skip to the final demo.

<input type="file" accept="image/*" />
<script>
    document.querySelector('input').addEventListener('change', (e) => {
        const imageFile = e.target.files[0];

        // No image file selected, so nothing to edit
        if (!imageFile) return;

        // This adds or updates the current image preview
        const showPreview = () => {
            // Create a new preview or re-use existing one
            const image =
                e.target.parentNode.querySelector('.preview') || new Image();
            image.className = 'preview';

            // We don't want our preview to be too big
            image.width = 32;

            // If already loaded a preview, need to unload
            image.src && URL.revokeObjectURL(image.src);

            // Set the src of the preview image to the file object in the file input
            image.src = URL.createObjectURL(e.target.files[0]);

            // Attach/move the preview to before the file input
            e.target.before(image);
        };

        // Render a tiny image preview of the currently selected file
        showPreview();

        // Show loading indicator
        const loadingState = document.createElement('span');
        loadingState.className = 'loading';
        loadingState.textContent = 'Loading image…';
        e.target.before(loadingState);

        // We have an image file, let's dynamically load Pintura
        import('./pintura.js').then(
            ({
                openDefaultEditor,
                shapeGetCenter,
                shapeGetLength,
                shapeGetLevel,
            }) => {
                // Pintura has loaded, let's edit the image
                const editor = openDefaultEditor({
                    src: imageFile,

                    // Set the stickers tool as the active tool
                    annotateActiveTool: 'preset',

                    // Add some stickers
                    annotatePresets: [
                        [
                            // Add image based stickers
                            'Numbers',
                            [
                                'sticker-one.svg',
                                'sticker-two.svg',
                                'sticker-three.svg',
                            ],
                        ],
                        [
                            // Add emoji stickers
                            'Emoji',
                            ['👍', '👎'],
                        ],
                        [
                            // Add shape stickers
                            'Shapes',
                            [
                                // A speech bubble
                                {
                                    thumb: '💬',
                                    alt: 'Speech cloud',
                                    shape: {
                                        width: 256,
                                        height: 128,
                                        fontSize: 64,
                                        backgroundColor: [1, 1, 1, 0.85],
                                        color: [0, 0, 0],
                                        cornerRadius: 10,
                                        text: 'Hello World',
                                    },
                                },
                                // Add our dynamic measure shape
                                {
                                    thumb: '📏',
                                    alt: 'Measure',
                                    shape: {
                                        style: 'measure',
                                        x1: 0,
                                        y1: 0,
                                        x2: 100,
                                        y2: 100,
                                        strokeWidth: '.5%',
                                        strokeColor: [0.5, 0.85, 1],
                                        lineEnd: 'bar',
                                        lineStart: 'bar',
                                    },
                                },
                            ],
                        ],
                    ],

                    // Set our custom shape preprocessor
                    shapePreprocessor: [
                        (shape, options) => {
                            // Should return undefined if shape is not a match
                            if (!shape.style || shape.style !== 'measure')
                                return;

                            // Remove the `style` property from the cloned shaped so it no longer matches after processing
                            const { style, ...clone } = shape;

                            // Calculate the font size scalar so we can always read the font size info no matter the size of the image
                            const scalar = options.isPreview
                                ? Math.min(1, options.scale * 2)
                                : 1;

                            // Calculate line midpoint
                            const center = shapeGetCenter(clone);

                            // Calculate line length
                            const length = shapeGetLength(clone);

                            // Properties for the text label
                            const text = `${Math.round(length)}px`;
                            const fontSize = 24 / scalar;
                            const width = text.length * fontSize * 0.75;
                            const lineHeight = fontSize * 1.5;

                            // Return the cloned line shape and the added text shape
                            return [
                                clone,
                                {
                                    width,
                                    lineHeight,
                                    text,
                                    fontSize,
                                    textAlign: 'center',
                                    fontFamily: 'monospace',
                                    color: [1, 1, 1],
                                    backgroundColor: [0, 0, 0],
                                    cornerRadius: 8,

                                    // This centers the text box to the line
                                    x: center.x - width * 0.5,
                                    y: center.y - lineHeight * 0.5,

                                    // This makes sure the text box isn't flipped or rotated
                                    ...shapeGetLevel(clone, options),
                                },
                            ];
                        },
                    ],
                });

                // Hide loading indicator when the editor modal is fully visible
                editor.on('show', () => {
                    loadingState.remove();
                });

                // Pintura returns the resulting image and we store it in the file input
                editor.on('process', ({ dest }) => {
                    const dataTransfer = new DataTransfer();
                    dataTransfer.items.add(dest);
                    e.target.files = dataTransfer.files;
                    showPreview();
                });
            }
        );
    });
</script>
<style>
    /* Styles the preview to be similar to the file input button */
    .preview {
        display: inline-block;
        width: 1.5rem;
        height: 1.5rem;
        object-fit: cover;
        margin-right: 0.3125rem;
        vertical-align: bottom;
        border-radius: 0.175rem;
    }

    /* Hide the file input and image preview while loading */
    .loading ~ * {
        display: none;
    }
</style>

The demo that implements the above code, try it out! 🚀

Conclusion

We’ve created a flexible and powerful image annotation solution. Along the way we’ve learned quite a bit about offering a good user experience.

We can extend the editor further with more stickers and custom shapes.

Additionally we can use the editor API to disable stickers or sticker groups, limit the amount of stickers that a user may apply, or customize the HTML of the sticker thumbnails.

I share web dev tips on Twitter, if you found this interesting and want to learn more, follow me there

More articles More articles