What it is for
Any file whose point is the music: which recording it is or which song it covers, which part of that recording each stretch of the file plays, and the facts an editor cuts to: tempo, key, beats, an energy curve.
People talking, which is audio/speech.
speech, summary, people, sections, instruments, lyrics: nothing in a music file is a reading; the one-line description is the recognition
Every audio file starts with the same envelope: format, version, type, generated, source. What follows is this type's own.
Signal
Whole-file levels. Everything here is a fact of the samples.
| field | shape | meaning | |
|---|---|---|---|
| loudness | M | { integrated, truePeak, range } | LUFS, dBTP, LU, as broadcast meters report them. |
| noiseFloor | M | dBFS | The level of the quietest sustained stretch. |
| dynamicRange | M | dB | Peak minus noise floor. |
| clipping[] | M | [ { s, e } ] | Spans at full scale. Empty when clean. |
| waveform | M | { step, peaks [] } | The peak level per step seconds as fractions of full scale: enough to draw an overview without decoding the file. |
Music
Always present, and measured throughout: the catalogue says what the recording is, the samples say how it moves. The whole file is music, so there is no list of where it plays; a single track has one song, a mix has many.
| section | shape | meaning | |
|---|---|---|---|
| songs[] | M | [ { label, pieces [ { s, e, at } ], seconds, tempo, key, keyConfidence, recognition } ] | One per distinct recording, lettered. pieces are the spans of the file where it plays, each with at: the position in the recognised recording that plays at s, so a reader subtracts to know which part of the song any moment is. seconds is the total played. tempo in BPM, key as "F# minor" with keyConfidence 0 to 1. |
| songs[].recognition | M | { match, title, artists [], album, releaseDate, label, duration, language, genres [], isrc, upc, score, confidence, matchedAt, spotify, youtube, deezer, musicbrainz } | null | The recording as a catalogue names it. match is recording when the file holds that very recording, cover when another performance of the same song. duration is the recognised recording's full length. confidence is identified (score ≥ 70) or likely (40 to 69); null means not in the catalogue, common for library music. It names the recording; it says nothing about rights. |
| beats[] | M | [ t, … ] | Every beat, whole file. |
| energy | M | { step, values [] } | Short-term loudness per step seconds, 0 to 1 relative to the file's own maximum. The curve an editor cuts to. |
Events
| field | shape | meaning | |
|---|---|---|---|
| events[] | D | [ { t, e, kind, text } ] | One music event per piece, silences as silence. |
"Which fragment is playing" is answered by pieces[].at and recognition.duration. For a 3:34 recording whose file starts at 2:00, pieces is [ { s: 0, e: 94.20, at: 120.00 } ] and duration is 214.30: the file is the last 94 seconds of it. source.tags is kept even when recognition disagrees: one is what the file says about itself, the other what the sound is.
Vocabularies
Closed lists. Each includes other; a reader that meets an unknown value treats it as other and keeps going.
Example
Illustrative and abbreviated.
{
"format": "smartmedia", "version": "1.1", "type": "audio/music",
"generated": { "at": "2026-09-14T10:05:40Z", "by": "smartmedia-api 0.1", "models": { "music": "librosa", "recognition": "acrcloud" } },
"source": { "name": "low-tide-chorus.wav", "bytes": 16621344, "sha256": "41f0…", "quickHash": "c93d…", "container": "wav", "duration": 94.20,
"audio": { "codec": "pcm_s16le", "channels": 2, "sampleRate": 44100, "bitRate": 1411200 },
"tags": { "title": "Low Tide (chorus edit)", "artist": "Harbour Lights" } },
"signal": { "loudness": { "integrated": -9.6, "truePeak": -0.3, "range": 5.2 }, "noiseFloor": -71, "dynamicRange": 71, "clipping": [],
"waveform": { "step": 1, "peaks": [ 0.88, 0.91, 0.90 ] } },
"music": { "songs": [ { "label": "A", "pieces": [ { "s": 0, "e": 94.20, "at": 120.00 } ], "seconds": 94.20,
"tempo": 96, "key": "D major", "keyConfidence": 0.83,
"recognition": { "match": "recording", "title": "Low Tide", "artists": [ "Harbour Lights" ], "album": "Slack Water",
"releaseDate": "2024-05-17", "duration": 214.30, "isrc": "GBXXX2400123", "score": 92, "confidence": "identified", "matchedAt": 10.00 } } ],
"beats": [ 0.31, 0.94, 1.56 ], "energy": { "step": 1, "values": [ 0.91, 0.93, 0.95 ] } },
"events": [ { "t": 0, "e": 94.20, "kind": "music", "text": "Low Tide, Harbour Lights, from 2:00 of the recording" } ]
}