What it is for
Any recording whose point is the words. The transcript is four views of the same words: paragraphs for reading, words for alignment, fillers and silences for editing, speakers for who said what, with the voices as people and the levels as signal.
A recording whose point is the music, which is audio/music, even when the music has words.
music, even if a jingle plays
Every audio file starts with the same envelope: format, version, type, generated, source. What follows is this type's own.
Signal
Whole-file levels. Everything here is a fact of the samples.
| field | shape | meaning | |
|---|---|---|---|
| loudness | M | { integrated, truePeak, range } | LUFS, dBTP, LU, as broadcast meters report them. |
| noiseFloor | M | dBFS | The level of the quietest sustained stretch. |
| dynamicRange | M | dB | Peak minus noise floor. |
| clipping[] | M | [ { s, e } ] | Spans at full scale. Empty when clean. |
| waveform | M | { step, peaks [] } | The peak level per step seconds as fractions of full scale: enough to draw an overview without decoding the file. |
People
| field | shape | meaning | |
|---|---|---|---|
| people[] | O | [ { label, description } ] | The voices, lettered in order of first speaking. description is what the sound gives: a name if one is said, a role if one is stated, otherwise the voice itself in a few words. |
Speech
Always present in a speech file.
| field | shape | meaning | |
|---|---|---|---|
| language | M | BCP 47 | en, es-419. One per file. |
| speakers[] | MO | [ { id, person } ] | id is the voice's index in words and paragraphs, measured. person is the letter in people[]: whoever is seen (or heard) speaking, or "off camera" for a voice never seen. |
| paragraphs[] | M | [ { s, e, speaker, sentences [ { s, e, text } ] } ] | The reading view: one speaker per paragraph, punctuated sentences with their own spans. |
| words[] | M | [ { w, s, e, confidence, speaker } ] | Every word with its span, punctuation attached, confidence 0 to 1, speaker index. The alignment view. |
| fillers[] | M | [ { w, t } ] | The ums and uhs, also in words; listed again so an editor can cut them. |
| silences[] | M | [ { s, e } ] | Spans below the noise floor, 0.5 s and up. |
| loudness | M | { integrated, truePeak, range } | Of the track: LUFS, dBTP, LU, as broadcast meters report them. |
A speech file never has a music section. If a jingle plays, the words over it are still speech and the jingle is not described; signal.waveform shows where the sound is loud, nothing says what it is.
Summary and events
| field | shape | meaning | |
|---|---|---|---|
| summary | O | string | One paragraph: what the conversation is about and what happens in it. |
| events[] | D | [ { t, e, kind, text } ] | Paragraphs as speech, silences as silence, pauses as pause. |
A speech file answers "who said what, when, and how clean is the recording". A model reads summary and the paragraphs; an editor cuts on fillers, silences and the waveform.
Vocabularies
Closed lists. Each includes other; a reader that meets an unknown value treats it as other and keeps going.
Example
Illustrative and abbreviated.
{
"format": "smartmedia", "version": "1.1", "type": "audio/speech",
"generated": { "at": "2026-09-14T10:02:11Z", "by": "smartmedia-api 0.1", "models": { "speech": "deepgram nova-3", "text": "google/gemini-3.7-flash" } },
"source": { "name": "New Recording 14.m4a", "bytes": 3184402, "sha256": "9b1c…", "quickHash": "e07a…",
"container": "mov,mp4,m4a,3gp,3g2,mj2", "duration": 412.65, "audio": { "codec": "aac", "channels": 1, "sampleRate": 48000, "bitRate": 64000 },
"recorded": { "utc": "2026-09-12T16:40:03Z", "local": "2026-09-12T18:40:03", "offset": "+02:00" } },
"summary": "Two people plan a kitchen renovation: B lists what has to go, A pushes back on the budget, they agree to get two quotes.",
"signal": { "loudness": { "integrated": -21.4, "truePeak": -3.1, "range": 9.8 }, "noiseFloor": -52, "dynamicRange": 49,
"clipping": [], "waveform": { "step": 1, "peaks": [ 0.12, 0.31, 0.44 ] } },
"people": [ { "label": "A", "description": "Called Marc by B; a low, unhurried voice." },
{ "label": "B", "description": "Leads the conversation; quick, laughs often." } ],
"speech": { "language": "es", "speakers": [ { "id": 0, "person": "B" }, { "id": 1, "person": "A" } ],
"paragraphs": [ { "s": 3.20, "e": 9.85, "speaker": 0, "sentences": [ { "s": 3.20, "e": 6.10, "text": "Bueno, empecemos por la cocina." } ] } ],
"words": [ { "w": "Bueno,", "s": 3.20, "e": 3.55, "confidence": 0.99, "speaker": 0 } ],
"fillers": [ { "w": "eh", "t": 12.40 } ], "silences": [ { "s": 60.20, "e": 61.90 } ],
"loudness": { "integrated": -21.4, "truePeak": -3.1, "range": 9.8 } },
"events": [ { "t": 3.20, "e": 9.85, "kind": "speech", "text": "B: Bueno, empecemos por la cocina. …" } ]
}