Cryptograss:Video encoding pipeline

Video encoding pipeline describes how a video travels from a musician's laptop to a published Release on PickiPedia — and, as of September 2026, why none of that work leaves delivery-kid any more.

How it works

1. Upload. Special:DeliverVideo mints a draft, then the browser posts the video bytes directly to delivery-kid — they never pass through the wiki server. A ReleaseDraft: page is created first, so an upload that fails leaves an inspectable record instead of vanishing.
2. Analyse. delivery-kid probes the file for duration, resolution and codecs, and writes them onto the draft page.
3. Finalize. The video is transcoded to AV1 video and Opus audio, packaged as HLS, and pinned to IPFS.
4. Publish. A bot notices the finalized draft, creates the Release: page, and adds BitTorrent metadata so the release seeds as well as pins.

Steps 3 and 4 are automatic and typically complete within seconds of each other.

Why AV1 and Opus

Both are royalty-free. H.264 and AAC — what cameras and phones actually record — are covered by patent pools. Since cryptograss exists to distribute music freely and durably, the published artefact should not depend on a licensing regime that can change. Encoding is also DRM-free: nothing here is encrypted or access-controlled, and the file is downloadable from IPFS by anyone.

The practical cost is that Apple ships no software AV1 decoder. Safari plays AV1 only on M3-and-later Macs and A17 Pro-and-later iPhones. Chrome and Firefox both bundle the royalty-free dav1d decoder and play AV1 on any desktop, including older Intel Macs. So on Apple hardware without AV1 support, use Firefox or Chrome, or download the file and play it in VLC.

The September 2026 rework

Until early September 2026, transcoding was sent to Coconut, a paid cloud service. Long videos failed there consistently, and a chain of separate bugs made those failures nearly impossible to read. All of it surfaced from a single complaint: longer videos would not upload.

What was wrong

Uploads died instantly on large files

The upload handler read each file into memory in one allocation. On delivery-kid's 4 GB box a multi-gigabyte video exceeded available memory and raised MemoryError before copying a single byte — 13 milliseconds after the upload began. Because MemoryError stringifies to the empty string, the error surfaced as a bare "Upload error:" with nothing after it. Fixed by streaming to disk in 8 MB chunks: peak memory for a 200 MB file fell from 210 MB to 17 MB. (maybelle-config#108)

Coconut could not handle long videos

Two independent attempts on a 10.5 GB source failed 12m11s and 11m54s after submission — a fixed timeout, not a transient fault. Coconut's failure webhook carried no error field at all, so both the draft page and the container log reported only "Unknown error". (maybelle-config#110)

The local fallback used patented codecs

The "fallback" path was libx264 + aac — quietly undermining the royalty-free goal that sent the work to Coconut in the first place. It also could not be reached: once Coconut accepted a job, the finalize handler returned, and the job's later failure arrived by webhook with nothing left to fall back to.

delivery-kid could never write to the wiki

The wiki client was built with mwclient's default path of /w/, but PickiPedia serves its API at /api.php. Every write 404'd, so every diagnostics snapshot silently no-opped, and draft pages froze at whatever the browser last wrote.

What changed

Encoding moved onto delivery-kid using SVT-AV1, which has become fast enough to make the cloud service unnecessary. Measured on two threads at 1080p, extrapolated to a 66-minute source:

Encoder	Royalty-free	Speed	66-minute source
SVT-AV1 preset 12	yes	0.51× realtime	~34 min
SVT-AV1 preset 10	yes	0.81× realtime	~53 min
libx264 medium	no	1.32× realtime	~1.5 h
libvpx-VP9 speed 4	yes	2.69× realtime	~3.0 h

SVT-AV1 at preset 10 is faster than the patented libx264 profile it replaced. Coconut is now opt-in only; auto means local.

A detail worth recording, because it fails silently: AV1 and Opus cannot be carried in MPEG-TS segments. ffmpeg accepts such a command, exits zero, and writes a playlist whose streams probe back as bin_data — unplayable. HLS output here therefore uses fMP4/CMAF segments, with the GOP derived from source frame rate so keyframes land on segment boundaries. (maybelle-config#113)

Finally, encoding an hour-long video locally meant the finalize page sat frozen for the duration. ffmpeg's progress is now streamed to the page, which reports position, encoding speed and a running estimate:

Encoding 34% · 11m40s of 34m00s · 1.36x · ~16m left

(maybelle-config#114 and pickipedia#108)

Known issues

• maybelle-config#115 — A finalize that pins successfully but fails afterwards deletes the source and leaves no record, so it reports as total failure when the content is in fact published. Encountered on the 44th Bday Cicada Jam-Bash, which had been pinned and seeding for hours while the wiki showed nothing and the audit recommended deleting the page. Recovered by writing the CID back onto the draft.
• maybelle-config#110 — the original Coconut failure. Largely moot now that Coconut is out of the default path.
• Diagnostics snapshots to ReleaseDraft:<uuid>/diagnostics have never successfully written a page. The /w/api.php path bug was one cause; a second remains, most likely an unset bot password in the container.

First releases through the local pipeline

Release	Length	Finalized
Pickipedia Dispatch — Sombrero Edition	34 min	2 September 2026
44th Bday Cicada Jam-Bash	66 min	3 September 2026
Cicada 44th Bday Jam Set 1	13 min	4 September 2026

All three are AV1 1080p with Opus audio. The third was the first to run start to finish with no manual intervention.

Advice for uploaders

• Export around 8–12 Mbps for 1080p. Camera and editor defaults are often 21 Mbps or higher, which is far more than jam footage needs — that is the difference between a 5 GB upload and a 10.5 GB one, with no visible gain.
• Do not economise on audio. 256–320 kbps. It is a rounding error in file size and it is the part anyone actually listens to.
• Fill in venue and performers before finalizing. Those fields become the Release page, and the performer names are what connect a video to everyone's pages later.

See also

• Cryptograss:Delivery-kid — the server this all runs on
• Special:DeliverVideo — the upload form
• Special:Releases — everything published