The problem
A folder of media is opaque to anything that is not a media player. Ask an assistant "which of these clips is the interview" or "find the moment she turns to camera" and it has nothing to go on: a language model cannot open a video, listen to a recording or look at a photo. Even the models that can are slow, expensive, and start from zero every time you ask, so every question re-decodes the same files.
What we do about it
Analyse each file once, and leave a small text file next to it: IMG_1731.MOV gets IMG_1731.MOV.smartmedia. It says what the file is, when and where it was made, who is in it, what happens, what is said, what is written, what plays, with times. It plays the role an .srt plays for subtitles, for everything.
Anything can read it: grep, a shell loop, a Mac app, a language model with the file pasted in. A folder of sidecars is a folder of documents an assistant reads in one pass, and the hashes let it trust that each document is about the file beside it.
Three things, one name
- The protocol is the file format: open, versioned, six types on one envelope. The format and its rules, then a page per type.
- The API writes the file for you: a file in, a sidecar out, priced per minute or per image. Using the API.
- The Mac app runs the API over a folder: drop files, say what they are, get sidecars. Smartmedia Drop.
How to read the format pages
Every field carries a grade. Measured values are facts of the file's bytes, the same every time they are computed: durations, loudness, camera motion, colours. Observed values are a model's reading of the picture or the sound: who is in frame, what they do, what a sign says. They describe what is there, never what to do with it; no field anywhere holds an opinion. Derived values are recomputed from the rest.
Times are seconds from the start, positions are fractions of the frame, people are letters, ids count from 1. A missing section means not analysed; an empty one means analysed and nothing found. The format page has the whole list.