Video Editor exported methods

The functions, properties, and defaults related to video editing that are exported by the Pintura Video module.

These exports are available on the pinturavideo module.

createDefaultVideoWriter

The createDefaultVideoWriter function returns a default video writer array. This is an array of processes the editor runs to write the output video data.

The createDefaultImageWriter function shares the following properties with the createDefaultVideoWriter function:

• targetSize
• store
• renameFile
• outputProps

import { imageStateToCanvas } from '@pqina/pintura';

import {
    createDefaultVideoWriter,
    createMediaStreamEncoder,
} from '@pqina/pintura-video';

openDefaultEditor({
    imageWriter: createDefaultVideoWriter({
        // Resize image
        targetSize: {
            width: 512,
        },

        // Use media stream encoder
        encoder: createMediaStreamEncoder({
            // Required
            imageStateToCanvas,
        }),
    }),
});

createMediaStreamEncoder

This video encoder is based on the native browser MediaStream API. It's mostly there as a fallback for Safari and older browsers. In most cases we want to use the MuxerEncoder.

The MediaStream encoder works well for shorter video's but in most cases isn't an option for longer videos. For longer video's using the FFmpegEncoder or using server side FFmpeg is often a better idea.

The MediaStream encoder has some limitations:

1. The duration of the save process is equal to the trimmed video time.
2. The browser tab needs to stay active for video creation process to work.
3. If the user device is very slow the created video can have missing frames.
4. Safari can only start processing the video after the user has interacted with the page or has been on the page for about 7 seconds.
5. Chrome creates a WebM or MP4 video with no duration information, this can sometimes be corrected by passing the output video file through this third party library but it unfortunately doesn't work consistently.

import { openDefaultEditor, imageStateToCanvas } from './pintura.js';
import {
    createDefaultVideoWriter,
    createMediaStreamEncoder,
} from './pinturavideo.js';

openDefaultEditor({
    imageWriter: createDefaultVideoWriter({
        // Resize image
        targetSize: {
            width: 512,
        },

        // Use media stream encoder
        encoder: createMediaStreamEncoder({
            // Required
            imageStateToCanvas,

            // output quality (optional)
            videoBitrate: 2500000, // 2.5MBps
            audioBitrate: 192000, // 192KBps

            // Default FPS (optional)
            framesPerSecond: 24,

            // By default logging is disabled
            log: false,

            // Force a mime type, please note that this won't work on each browser.
            // By default this property is undefined and Pintura will search a mime type.
            mimeType: 'video/webm;codecs=vp9,opus',

            // If the `mimeType` property isn't set the `mimeTypes` property allows us to define a list of mime types to test in order.
            // These are the default values.
            mimeTypes: [
                // MP4 if at all possible
                'video/mp4;codecs=avc1,aac',
                'video/mp4;codecs=avc,aac',
                'video/mp4;codecs=mp4a,aac',
                'video/mp4;codecs=h264,aac',
                'video/mp4',

                // WEBM will usually work
                'video/webm;codecs=vp9,opus',
                'video/webm;codecs=vp8,opus',
                'video/webm;codecs=h264,opus',
                'video/webm',

                // Matroska fallback for Linux
                'video/x-matroska;codecs=avc1,opus',
                'video/x-matroska;codecs=vp9,opus',
                'video/x-matroska;codecs=vp8,opus',
                'video/x-matroska',
            ],
        }),
    }),
});

createMediabunnyEncoder

This video encoder is based on the native browser VideoEncoder API and has the third-party library Mediabunny as a dependency.

This encoder doesn't have the disadvantages listed for the MediaStream encoder.

In the example below we pass Mediabunny to createMediabunnyEncoder, this will set the outputFormat to MP4, everything will be handled by Mediabunny.

We can optionally pass videoBitrate and audioBitrate to control the output quality, these properties can be set to a number or to one of the QUALITY levels.

import {
    openDefaultEditor,
    imageStateToCanvas,
    createDefaultImageWriter,
    createDefaultMediaWriter,
} from './pintura.js';

import {
    createDefaultVideoWriter,
    createMediabunnyEncoder,
    createMediaStreamEncoder,
} from './pinturavideo.js';

import * as Mediabunny from 'mediabunny';

openDefaultEditor({
    // here we supply two video writers
    imageWriter: createDefaultMediaWriter(
        {
            // Generic options writer options
        },
        [
            // For writing images
            createDefaultImageWriter(),

            // Use Mediabunny to encode videos
            createDefaultVideoWriter({
                encoder: createMediabunnyEncoder({
                    // Pass the entire Mediabunny library and the encoder
                    mediabunny: Mediabunny,

                    // This draws the video frame
                    imageStateToCanvas,

                    // Defaults to `Mediabunny.QUALITY_HIGH`
                    // videoBitrate: 2500000,

                    // Defaults to `Mediabunny.QUALITY_HIGH`
                    // audioBitrate: 192000,

                    // Defaults to `new Mp4OutputFormat()`
                    // outputFormat: new WebMOutputFormat()

                    // Enable logging
                    log: true,
                }),
            }),

            // Media stream as fallback
            createDefaultVideoWriter({
                encoder: createMediaStreamEncoder({
                    imageStateToCanvas,
                }),
            }),
        ]
    ),
});

A more optimised build target can be achieved by only passing the needed modules inside Mediabunny.

import {
    openDefaultEditor,
    imageStateToCanvas,
    createDefaultImageWriter,
    createDefaultMediaWriter,
} from './pintura.js';

import {
    createDefaultVideoWriter,
    createMediabunnyEncoder,
    createMediaStreamEncoder,
} from './pinturavideo.js';

import {
    // The outputFormat to use
    WebMOutputFormat,
    Mp4OutputFormat,

    // Required for internal use
    ALL_FORMATS,
    QUALITY_HIGH,
    Input,
    Output,
    BlobSource,
    CanvasSource,
    VideoSampleSink,
    AudioBufferSink,
    BufferTarget,
    AudioBufferSource,
    getFirstEncodableVideoCodec,
    getFirstEncodableAudioCodec,
} from 'mediabunny';

openDefaultEditor({
    // here we supply two video writers
    imageWriter: createDefaultMediaWriter(
        {
            // Generic options writer options
        },
        [
            // For writing images
            createDefaultImageWriter(),

            // Use Mediabunny to encode videos
            createDefaultVideoWriter({
                encoder: createMediabunnyEncoder({
                    // Pass the entire Mediabunny library and the encoder
                    mediabunny: {
                        ALL_FORMATS,
                        QUALITY_HIGH,
                        Input,
                        Output,
                        BlobSource,
                        CanvasSource,
                        VideoSampleSink,
                        AudioBufferSink,
                        BufferTarget,
                        AudioBufferSource,
                        getFirstEncodableVideoCodec,
                        getFirstEncodableAudioCodec,
                    },

                    // Set the output format of the video
                    outputFormat: new Mp4OutputFormat(),

                    // This draws the video frame
                    imageStateToCanvas,

                    // Enable logging
                    log: true,
                }),
            }),

            // Media stream as fallback
            createDefaultVideoWriter({
                encoder: createMediaStreamEncoder({
                    imageStateToCanvas,
                }),
            }),
        ]
    ),
});

createMuxerEncoder

This video encoder is based on the native browser VideoEncoder API.

This encoder uses the MIT licensed webm-muxer or the mp4-muxer as a dependency.

This encoder doesn't have the disadvantages listed for the MediaStream encoder but currently only works on the latest versions of Chrome and Firefox.

See the Video Encoder and Audio Encoder API Browser Compatibility tables for an up to date list of browsers that support the API.

import * as Mp4Muxer from 'mp4-muxer';

import {
    openDefaultEditor,
    imageStateToCanvas,
    createDefaultImageWriter,
    createDefaultMediaWriter,
} from './pintura.js';

import {
    createMuxerEncoder,
    createDefaultVideoWriter,
    createMediaStreamEncoder,
} from './pinturavideo.js';

openDefaultEditor({
    // here we supply two video writers
    imageWriter: createDefaultMediaWriter(
        {
            // Generic options writer options
        },
        [
            // For writing images
            createDefaultImageWriter(),

            // Use muxer to encode videos
            createDefaultVideoWriter({
                encoder: createMuxerEncoder({
                    // when using the mp4 muxer we need to set video/mp4 mimetype
                    muxer: Mp4Muxer,
                    mimeType: 'video/mp4',

                    // video and audio bitrate to use (optional)
                    videoBitrate: 2500000, // 2.5MBps
                    audioBitrate: 192000, // 192KBps, should be either (96000, 128000, 160000, or 192000)
                    audioSampleRate: undefined, // will default to output device supported sample rate, 441000 is a common value to use instead

                    // this draws the image
                    imageStateToCanvas,

                    // enable logging
                    log: true,
                }),
            }),

            // Media stream as fallback
            createDefaultVideoWriter({
                encoder: createMediaStreamEncoder({
                    imageStateToCanvas,
                }),
            }),
        ]
    ),
});

codecs

The createMuxerEncoder factory also accepts a codecs property, this property is automatically set internally but you can override it with your own codecs configuration.

// for MP4
createMuxerEncoder({
    // when using the mp4 muxer we need to set video/mp4 mimetype
    muxer: Mp4Muxer,
    mimeType: 'video/mp4',

    // Mp4 codecs default values
    codecs: {
        muxer: {
            // can be set to: 'avc' | 'hevc' | 'vp9' | 'av1'
            video: 'avc',

            // can be set to: 'aac' | 'opus'
            audio: 'aac',
        },
        encoder: {
            // for video see: https://developer.mozilla.org/en-US/docs/Web/API/VideoEncoder/configure#codec
            video: 'avc1.640028',

            // for audio see: https://developer.mozilla.org/en-US/docs/Web/API/AudioEncoder/configure#codec
            audio: 'mp4a.40.2',
        },
    },

    // ... other options
});

createFFmpegEncoder

To use the FFmpeg encoder we need to download and install FFmpeg WASM.

Please make sure to read the FFmpeg GitHub page before proceeding.

Based on the example below the FFmpeg WASM package dist folder contents should be saved to ./ffmpeg/

The FFmpeg WASM Core package dist folder contents should be saved to ./ffmpeg/core/

Resulting in the following directory structure:

example
├─ index.html
├─ pintura
├─ pintura-video
└─ ffmpeg
   ├─ core
   │  ├─ ffmpeg-core.js
   │  ├─ ffmpeg-core.wasm
   │  └─ ffmpeg-core.worker.js
   ├─ ffmpeg.min.js
   └─ ffmpeg.min.js.map

import { openDefaultEditor, imageStateToCanvas } from './pintura/pintura.js';

import {
    createDefaultVideoWriter,
    createFFmpegEncoder,
} from './pintura-video/pinturavideo.js';

openDefaultEditor({
    imageWriter: createDefaultVideoWriter({
        // Resize image
        targetSize: {
            width: 512,
        },

        // Use FFmpeg encoder
        encoder: createFFmpegEncoder({
            // References to FFmpeg
            scriptPath: './ffmpeg/ffmpeg.min.js',
            corePath: './ffmpeg/core/ffmpeg-core.js',

            // Used to render annotations
            imageStateToCanvas,

            // Default output quality (optional)
            videoBitrate: '384k',
            audioBitrate: '48k',

            // Default FPS (optional)
            framesPerSecond: 24,

            // Toggle logging (optional)
            log: true,
        }),
    }),
});

FFmpeg WASM makes use of SharedArrayBuffer which is only available on webpages that are cross-origin isolated. Your server needs to return these two headers Cross-Origin-Embedder-Policy: require-corp and Cross-Origin-Opener-Policy: same-origin for SharedArrayBuffer to be available.

To enabled SharedArrayBuffer for local testing with Chrome:

Windows:

chrome --enable-features=SharedArrayBuffer

MacOS:

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 --enable-features=SharedArrayBuffer

The FFmpeg package clocks in at 25MB so it's wise to first consider the user network connection before downloading the package.

Alternatively we can generate the output video on the server, that way we bypass both the FFmpeg WASM download size and the MediaStream API limitations. Please see the FFmpeg encoder function contained in pinturavideo.js for the FFmpeg commands used.