Annotate Images With JavaScript
In this tutorial we use Pintura, a JavaScript image editor, to quickly create a JavaScript image annotation solution to annotate and decorate user submitted images and photos.
Our goal is to allows users to capture a photo using their mobile phone camera and then add annotations to it.
We’ll start by setting up a basic HTML form, we’re not going to reinvent the wheel so let’s use Pintura a JavaScript image editor SDK to handle the image annotation process.
Setting Up The Initial Code
We’ll start with a file input field, as that’s the most common way to request an image file from the user. We add the accept attribute to limit the file selection to images.
<input type="file" accept="image/*" />Now to catch the image we listen for the 'change' event to fire, when it fires we get the first image file from the file input element.
<input type="file" accept="image/*" />
<script>
document.querySelector('input').addEventListener('change', (e) => {
const imageFile = e.target.files[0];
// No image file selected, so nothing to edit
if (!imageFile) return;
// We have an image file, but what to do now?
// ...
});
</script>Note that when the user clicks the cancel button in the browse file system dialog, the change event will still fire but the files property will be an empty FileList, that’s why we check if imageFile is actually defined.
This is the result, it doesn’t do anything, yet.
With our HTML file input ready we can now add Pintura to handle the image annotation process.
Loading The Image Editor
We’ll only load the Pintura module after an image file has been selected. This speeds up the initial page load as the editor won’t be needed when a user skips our file upload field.
<input type="file" accept="image/*" />
<script>
document.querySelector('input').addEventListener('change', (e) => {
const imageFile = e.target.files[0];
if (!imageFile) return;
// We have an image file, let's dynamically load Pintura
import('./pintura.js').then(({ openDefaultEditor }) => {
// Pintura has loaded, let's edit the image
const editor = openDefaultEditor({
src: imageFile,
});
// Pintura returns the resulting image and we store it in the file input
editor.on('process', ({ dest }) => {
const dataTransfer = new DataTransfer();
dataTransfer.items.add(dest);
e.target.files = dataTransfer.files;
});
});
});
</script>We’re going to have to improve the file input UX a tiny bit.
Note that if you just want get on with annotating images skip ahead to the image annotation section
Improving The File Input UX
This is what currently happens:
- When an image file is selected the Pintura JavaScript module will be loaded.
- When the module is finished loading Pintura will open the image file in a modal.
- When the user is finished editing the file it is saved back to the file input.
This “works” but creates a problem where if the editor module takes a bit longer to load in step 1, the interface doesn’t give any feedback and the user is left wondering about the state of things.
To fix this we need to add an image loading indicator, it’ll let the user know what’s going on while waiting for the editor to load.
<input type="file" accept="image/*" />
<script>
document.querySelector('input').addEventListener('change', (e) => {
const imageFile = e.target.files[0];
if (!imageFile) return;
// Show loading indicator
const loadingState = document.createElement('span');
loadingState.className = 'loading';
loadingState.textContent = 'Loading image…';
e.target.before(loadingState);
// We have an image file, let's dynamically load Pintura
import('./pintura.js').then(({ openDefaultEditor }) => {
// Now ready
loadingState.remove();
// Pintura has loaded, let's edit the image
const editor = openDefaultEditor({
src: imageFile,
});
// Hide loading indicator when the editor modal is fully visible
editor.on('show', () => {
loadingState.remove();
});
// Pintura returns the resulting image and we store it in the file input
editor.on('process', ({ dest }) => {
const dataTransfer = new DataTransfer();
dataTransfer.items.add(dest);
e.target.files = dataTransfer.files;
});
});
});
</script>
<style>
/* Hide the file input while loading */
.loading ~ * {
display: none;
}
</style>Excellent!
There’s one final UX thing to deal with.
When we save the image back to the file input nothing changes, the file name is the same, so the file input still looks the same.
We need to show that we’ve actually edited the image.
We’ll do this by adding a tiny image thumbnail before the file input. It’ll show the currently selected image and give the user confidence that the changes have been applied successfully.
<input type="file" accept="image/*" />
<script>
document.querySelector('input').addEventListener('change', (e) => {
const imageFile = e.target.files[0];
if (!imageFile) return;
// This adds or updates the current image preview
const showPreview = () => {
// Create a new preview or re-use existing one
const image =
e.target.parentNode.querySelector('.preview') || new Image();
image.className = 'preview';
// We don't want our preview to be too big
image.width = 32;
// If already loaded a preview, need to unload
image.src && URL.revokeObjectURL(image.src);
// Set the src of the preview image to the file object in the file input
image.src = URL.createObjectURL(e.target.files[0]);
// Attach/move the preview to before the file input
e.target.before(image);
};
// Render a tiny image preview of the currently selected file
showPreview();
// Show loading indicator
const loadingState = document.createElement('span');
loadingState.className = 'loading';
loadingState.textContent = 'Loading image…';
e.target.before(loadingState);
// We have an image file, let's dynamically load Pintura
import('./pintura.js').then(({ openDefaultEditor }) => {
// Pintura has loaded, let's edit the image
const editor = openDefaultEditor({
src: imageFile,
});
// Hide loading indicator when the editor modal is fully visible
editor.on('show', () => {
loadingState.remove();
});
// Pintura returns the resulting image and we store it in the file input
editor.on('process', ({ dest }) => {
const dataTransfer = new DataTransfer();
dataTransfer.items.add(dest);
e.target.files = dataTransfer.files;
// Update our image preview with the edited file
showPreview();
});
});
});
</script>
<style>
/* Styles the preview to be similar to the file input button */
.preview {
display: inline-block;
width: 1.5rem;
height: 1.5rem;
object-fit: cover;
margin-right: 0.3125rem;
vertical-align: bottom;
border-radius: 0.175rem;
}
/* Hide the file input and image preview while loading */
.loading ~ * {
display: none;
}
</style>While we side tracked quite a bit, it gave us some greate insight into what is needed to offer a good user experience.
Add an image to the field below to show the loading state and preview thumbnail.
That. Is. Nice. We’re finally ready to annotate images.
Annotating Images
While Pintura offers a wide range of tools to edit images we’re only interested in the annotation tool, in the next step we’ll limit the toolset to the annotation tool.
Note that starting from this code snippet we’re only altering the editor code, the rest of the code is hidden.
import('./pintura.js').then(({ openDefaultEditor }) => {
const editor = openDefaultEditor({
src: imageFile,
// Only show the annotation tool
utils: ['annotate'],
});
});The demo below reflects our changes. We’ve only selected one tool so the editor hides the toolbar creating more space for the active tool.
Try it out in the demo below, we can annotate our image with text, draw lines, and add shapes like circles and squares.
Adding Stickers
To make our annotation editor more powerful we can add stickers.
Stickers can be emoji, images, or shapes.
import('./pintura.js').then(({ openDefaultEditor }) => {
openDefaultEditor({
src: imageFile,
utils: ['annotate'],
// Set the stickers tool as the active tool
annotateActiveTool: 'preset',
// Add some stickers
annotatePresets: [
[
// Add image based stickers
'Numbers',
['sticker-one.svg', 'sticker-two.svg', 'sticker-three.svg'],
],
[
// Add emoji stickers
'Emoji',
['👍', '👎'],
],
[
// Add shape stickers
'Shapes',
[
// A speech bubble
{
thumb: '💬',
alt: 'Speech cloud',
shape: {
width: 256,
height: 128,
fontSize: 64,
backgroundColor: [1, 1, 1, 0.85],
color: [0, 0, 0],
cornerRadius: 10,
text: 'Hello World',
},
},
],
],
],
});
});Let’s try out the stickers below.
Dynamic Annotation Shapes
Now for our final trick we’ll add a dynamic shape.
Imagine you need an annotation tool to measure the distance in pixels between two points. We can achieve that by adding a line shape that automatically expands to a text showing the distance between the start and end point.
In the code snippet below we first add the "measure" shape preset, this is the preset that we’ll expand in the next step.
import('./pintura.js').then(({ openDefaultEditor }) => {
openDefaultEditor({
src: imageFile,
utils: ['annotate'],
annotateActiveTool: 'preset',
annotatePresets: [
[
'Numbers',
['sticker-one.svg', 'sticker-two.svg', 'sticker-three.svg'],
],
['Emoji', ['👍', '👎']],
[
'Shapes',
[
{
thumb: '💬',
alt: 'Speech cloud',
shape: {
width: 256,
height: 128,
fontSize: 64,
backgroundColor: [1, 1, 1, 0.85],
color: [0, 0, 0],
cornerRadius: 10,
text: 'Hello World',
},
},
// Add our dynamic measure shape
{
thumb: '📏',
alt: 'Measure',
shape: {
style: 'measure',
x1: 0,
y1: 0,
x2: 100,
y2: 100,
strokeWidth: '.5%',
strokeColor: [0.5, 0.85, 1],
lineEnd: 'bar',
lineStart: 'bar',
},
},
],
],
],
});
});Now we need to automatically expand shapes with the 'measure' style property to show a text box, for this we’ll use a custom shape preprocessor.
For brevity the example below only shows the shapePreprocessor code.
import('./pintura.js').then(
({ openDefaultEditor, shapeGetCenter, shapeGetLength, shapeGetLevel }) => {
openDefaultEditor({
// ... other editor properties here
// Set our custom shape preprocessor
shapePreprocessor: [
(shape, options) => {
// Should return undefined if shape is not a match
if (!shape.style || shape.style !== 'measure') return;
// Calculate the font size scalar so we can always read the font size info no matter the size of the image
const scalar = options.isPreview
? Math.min(1, options.scale * 2)
: 1;
// Calculate line midpoint
const center = shapeGetCenter(shape);
// Calculate line length
const length = shapeGetLength(shape);
// Properties for the text label
const text = `${Math.round(length)}px`;
const fontSize = 24 / scalar;
const width = text.length * fontSize * 0.75;
const lineHeight = fontSize * 1.5;
// Return the text shape containing the line length
return [
{
width,
lineHeight,
text,
fontSize,
textAlign: 'center',
fontFamily: 'monospace',
color: [1, 1, 1],
backgroundColor: [0, 0, 0],
cornerRadius: 8,
// This places the text box in the middle of the line
x: center.x - width * 0.5,
y: center.y - lineHeight * 0.5,
// This makes sure the text box isn't flipped or rotated
...shapeGetLevel(shape, options),
},
];
},
],
});
}
);Let’s take a look at the final code, or skip to the final demo.
<input type="file" accept="image/*" />
<script>
document.querySelector('input').addEventListener('change', (e) => {
const imageFile = e.target.files[0];
// No image file selected, so nothing to edit
if (!imageFile) return;
// This adds or updates the current image preview
const showPreview = () => {
// Create a new preview or re-use existing one
const image =
e.target.parentNode.querySelector('.preview') || new Image();
image.className = 'preview';
// We don't want our preview to be too big
image.width = 32;
// If already loaded a preview, need to unload
image.src && URL.revokeObjectURL(image.src);
// Set the src of the preview image to the file object in the file input
image.src = URL.createObjectURL(e.target.files[0]);
// Attach/move the preview to before the file input
e.target.before(image);
};
// Render a tiny image preview of the currently selected file
showPreview();
// Show loading indicator
const loadingState = document.createElement('span');
loadingState.className = 'loading';
loadingState.textContent = 'Loading image…';
e.target.before(loadingState);
// We have an image file, let's dynamically load Pintura
import('./pintura.js').then(
({
openDefaultEditor,
shapeGetCenter,
shapeGetLength,
shapeGetLevel,
}) => {
// Pintura has loaded, let's edit the image
const editor = openDefaultEditor({
src: imageFile,
// Set the stickers tool as the active tool
annotateActiveTool: 'preset',
// Add some stickers
annotatePresets: [
[
// Add image based stickers
'Numbers',
[
'sticker-one.svg',
'sticker-two.svg',
'sticker-three.svg',
],
],
[
// Add emoji stickers
'Emoji',
['👍', '👎'],
],
[
// Add shape stickers
'Shapes',
[
// A speech bubble
{
thumb: '💬',
alt: 'Speech cloud',
shape: {
width: 256,
height: 128,
fontSize: 64,
backgroundColor: [1, 1, 1, 0.85],
color: [0, 0, 0],
cornerRadius: 10,
text: 'Hello World',
},
},
// Add our dynamic measure shape
{
thumb: '📏',
alt: 'Measure',
shape: {
style: 'measure',
x1: 0,
y1: 0,
x2: 100,
y2: 100,
strokeWidth: '.5%',
strokeColor: [0.5, 0.85, 1],
lineEnd: 'bar',
lineStart: 'bar',
},
},
],
],
],
// Set our custom shape preprocessor
shapePreprocessor: [
(shape, options) => {
// Should return undefined if shape is not a match
if (!shape.style || shape.style !== 'measure')
return;
// Calculate the font size scalar so we can always read the font size info no matter the size of the image
const scalar = options.isPreview
? Math.min(1, options.scale * 2)
: 1;
// Calculate line midpoint
const center = shapeGetCenter(shape);
// Calculate line length
const length = shapeGetLength(shape);
// Properties for the text label
const text = `${Math.round(length)}px`;
const fontSize = 24 / scalar;
const width = text.length * fontSize * 0.75;
const lineHeight = fontSize * 1.5;
// Return the text shape
return [
{
width,
lineHeight,
text,
fontSize,
textAlign: 'center',
fontFamily: 'monospace',
color: [1, 1, 1],
backgroundColor: [0, 0, 0],
cornerRadius: 8,
// This centers the text box to the line
x: center.x - width * 0.5,
y: center.y - lineHeight * 0.5,
// This makes sure the text box isn't flipped or rotated
...shapeGetLevel(shape, options),
},
];
},
],
});
// Hide loading indicator when the editor modal is fully visible
editor.on('show', () => {
loadingState.remove();
});
// Pintura returns the resulting image and we store it in the file input
editor.on('process', ({ dest }) => {
const dataTransfer = new DataTransfer();
dataTransfer.items.add(dest);
e.target.files = dataTransfer.files;
showPreview();
});
}
);
});
</script>
<style>
/* Styles the preview to be similar to the file input button */
.preview {
display: inline-block;
width: 1.5rem;
height: 1.5rem;
object-fit: cover;
margin-right: 0.3125rem;
vertical-align: bottom;
border-radius: 0.175rem;
}
/* Hide the file input and image preview while loading */
.loading ~ * {
display: none;
}
</style>The demo that implements the above code, try it out! 🚀
Conclusion
We’ve created a flexible and powerful image annotation solution. Along the way we’ve learned quite a bit about offering a good user experience.
We can extend the editor further with more stickers and custom shapes.
Additionally we can use the editor API to disable stickers or sticker groups, limit the amount of stickers that a user may apply, or customize the HTML of the sticker thumbnails.
Or join my newsletter
