The file
- Name. The original's full name plus
.smartmedia:IMG_1731.MOV.smartmedia,memo.m4a.smartmedia,DSC_0412.HEIC.smartmedia. It sits next to the original, so file browsers keep the pair together. - Contents. One JSON object, UTF-8, no comments. The extension is
.smartmedia, never.json. Media typeapplication/vnd.betterapps.smartmedia+json; macOS UTIcom.sevensigmagroup.smartmedia. - Bound to the original.
source.sha256is the SHA-256 of the original's bytes andsource.quickHasha fast fingerprint. A reader compares hashes before trusting a sidecar: a renamed or moved original still matches; a different file with the same name does not. - Who writes it. Whoever has the original.
generatednames the writer and the models it used, the way a camera writes its firmware version; nothing else in the file says how it was made. - Which type. Declared by the writer in
type. A reader never second-guesses it. A 1.0 file has no type and isvideo/edit.
Six types
Every sidecar declares one type, medium/kind, and the type decides everything below the envelope. The six are separate formats that share an envelope and a set of rules; each has its own page, and a reader of one type needs nothing from another.
video/footageOne continuous take, as recorded.video/editCut together: scenes of shots, subtitles and titles, a soundtrack.audio/speechPeople talking: a memo, an interview, a call, a lecture.audio/musicA track, a stem, a mix. Measured only.image/sceneA view of something: a photograph, a painting, a render.image/graphicMade of marks: a screen, a document, a diagram, a chart.
The envelope
The five fields every file starts with, in this order, the same in every type. Everything after them belongs to the type.
| field | shape | meaning | |
|---|---|---|---|
| format | – | "smartmedia" | Always this string. |
| version | – | "1.1" | major.minor. Within a major, changes only add: a 1.0 reader reads a 1.4 file and ignores what it does not know. |
| type | – | video/footage · video/edit · audio/speech · audio/music · image/scene · image/graphic | Which of the six this file is. Declared by whoever wrote it; a reader never second-guesses it. A 1.0 file has no type and is video/edit. |
| generated | M | { at, by, models } | When it was written, by which writer, with which models (a map of role to model id: vision, speech, music, recognition…). Nothing per item. |
| source | MO | { name, bytes, sha256, quickHash, container, duration, video, audio, image, tags, camera, exposure, recorded, location } | The file and the recording. All measured except location.venue, which is a model's pick of the place the picture shows. |
MmeasuredOobservedDderivedMOmixed
source, by medium
| field | shape | meaning | |
|---|---|---|---|
| name, bytes, sha256, quickHash | M | string, integer, hex, hex | Every medium. quickHash is the SHA-256 of the size in bytes as decimal text, then the first MiB, then the last MiB: a fingerprint without reading the whole file. |
| container, duration | M | string, seconds | Video and audio. container as the demuxer names it. |
| video | M | { codec, width, height, fps, rotation, hdr, colorTransfer, colorPrimaries, pixelFormat, bitRate } | Video only. |
| audio | M | { codec, channels, sampleRate, bitRate } | null | Video and audio. null on a silent video. |
| image | M | { format, width, height, orientation, colorSpace, bitDepth, hdr, alpha } | Image only. width and height after orientation is applied, so they are what a viewer shows. |
| tags | M | { title, artist, album, albumArtist, track, disc, year, genre, composer, comment, artwork } | Audio, from the container's own tags, only when the file has them. artwork is { mime, bytes, width, height }, never the bytes. |
| camera | M | { make, model, lens, software } | Video and image, when the file names one. |
| exposure | M | { iso, shutter, aperture, focalLength, focalLength35, exposureBias, flash, whiteBalance } | Image, when the file carries it. shutter in seconds, aperture the f-number, focal lengths in millimetres. |
| recorded | M | { utc, local, offset, zoneSource, timeOfDay { bucket, sunrise, sunset, goldenHourStart, minutesToSunset, sunUp } } | When the file carries a capture time. timeOfDay needs a location. |
| location | MO | { lat, lon, altitude, accuracy, address { … }, nearby [ { name, type, distance } ], venue { name, reason } | null } | When the file carries a position. venue is the pick among nearby that the picture shows, so it is never in an audio file. |
Rules every reader can rely on
- TimeSeconds from the start of the file, a number with two decimals. Spans have s and e, instants have t. No frame numbers, no timecodes, no time strings. An image has no time at any level.
- PositionFractions of width and height from the top left, two decimals: x, y for a point, x, y, w, h for a box. Never pixels; source gives the pixel size when a reader needs it.
- IdentityIds are integers from 1, in time order. People are letters A, B, … in order of first appearance, in the picture or in the sound; songs are letters too. An id means the same thing everywhere in one file.
- OrderEvery array is sorted by its start time (or t, or id, or reading order in a graphic).
- Absent versus emptyA missing section means not analysed. An empty array or null means analysed, nothing found. A file with only the envelope is valid. A section the type never has is simply never there.
- Measured versus observedA property of the field, stated for every field. Measured values are facts of the bytes. Observed values describe what is there, never what to do with it. No field anywhere holds an opinion of quality, importance or intent.
- Types are declaredA file is the type it says it is. Nothing in a file explains the choice, and the standard defines no test for telling types apart.
- VocabulariesClosed lists, given on each type's page. Each includes other. A reader that meets an unknown value treats it as other and keeps going.
- Unknown fieldsAllowed at any level and preserved by tools that rewrite the file.
Sections by type
What each type can carry, at a glance. ● present when analysed, ○ a sentence rather than a block, – never present. The type pages define every section; names shared between types mean the same thing.
| section | footage | edit | speech | music | scene | graphic | what it holds |
|---|---|---|---|---|---|---|---|
| always | |||||||
| format, version, type, generated, source | ● | ● | ● | ● | ● | ● | The envelope, the same in every type. |
| summary | ● | ● | ● | – | ● | ● | The whole piece in one paragraph. A music file's one-line description is its recognition. |
| events[] | ● | ● | ● | ● | – | – | One time-ordered list of everything, derived. Images have no time. |
| the picture | |||||||
| camera | ● | ● | – | – | – | – | What the camera did (per shot in an edit). |
| picture | ● | ● | – | – | ● | ● | Colour and tone of the pixels. |
| form | ● | ● | – | – | ● | – | The shot as a cinematographer names it. |
| kind | – | – | – | – | ● | ● | What sort of image: a photograph or a painting; a screen or a chart. |
| composition | ○ | ○ | – | – | ● | – | A sentence in a video; a block with boxes and devices in a scene image. |
| scenes[], shots[] | – | ● | – | – | – | – | Only an edit has more than one shot. |
| layout[], depicts[] | – | – | – | – | – | ● | The parts of a graphic, and the pictures inside it. |
| who and what | |||||||
| people[] | ● | ● | ● | – | ● | – | The subjects; in speech audio, the voices. |
| bystanders[] | ● | ● | – | – | ● | – | People who are not subjects (background[] in an edit). |
| objects[], details[] | ● | ● | – | – | ● | – | Things worth naming; small things a viewer would miss. |
| activities[], moments[], actions[] | ● | ● | – | – | – | – | Continuous things, their turning points, a running description. |
| sound | |||||||
| speech | ● | ● | ● | – | – | – | The transcript: words, sentences, paragraphs, speakers, fillers, silences. |
| music | ● | ● | – | ● | – | – | Where music plays, which recording or cover, which part of it, tempo, key, beats. |
| signal | – | – | ● | ● | – | – | Whole-file levels: loudness, noise floor, clipping, a waveform overview. |
| written | |||||||
| text | ● | ● | – | – | ● | ● | Text in the picture. |
Columns are the six types in order: video/footage, video/edit, audio/speech, audio/music, image/scene, image/graphic.
Reconciliation and derived outputs
Given a media file and a sidecar: same sha256, the sidecar describes this file. Same quickHash and size but no full hash yet, very probably; compute the full hash when it matters. Neither, the sidecar is for another file, whatever its name. An export, a crop or a re-encode is a different file and gets its own sidecar; the format does not link versions of a file to each other.
Any reader can produce, without a model: an .srt or .vtt from speech.paragraphs[].sentences; a cue sheet from music.songs[]; alt text from an image's summary; a plain-text rendering of the whole file for a language model. None of these is authoritative; the .smartmedia file is.
History
- 1.0, 2026-09-13. Edited video only, without a type field. SMARTMEDIA.md and the JSON schema describe it.
- 1.1 draft, 2026-09-14.
typeas medium/kind with six values, each a separate format on its own page.