Music DNA — Product Specification

Version: 0.5.0 Last updated: 2026-07-15 Status: MVP in build. Stem separation, MusicAtlas (Song DNA), and ElevenLabs snippet APIs land in the next few days.

1. What the product is

Music DNA is songwriting intelligence for songwriters, producers, and topliners. Users upload a track (their own demo, a reference, or a released song), and the app produces an explainable, interactive "DNA" of that track — key, tempo, chord progression, sections, energy curve, harmonic complexity, mood, era, "sounds like" references, and plain-English why for every tag.

The user then talks to their music: a roster of specialist AI agents (Chord Coach, Producer, Lyricist, Reference Scout, Blueprint Architect) can be chatted with by text or, in hands-free mode, by voice — each agent has its own Deepgram voice and its own tools. Chat and voice are the primary interface; the UI is the transcript.

The design language is "specimen sheet, not dashboard": tabular numbers, dashed borders as a confidence signal, no chart gradients, no glow.

2. What the app does TODAY (pre-API)

2.1 Authentication

• Email/password and Google OAuth via Lovable Cloud auth.
• Route gate at /_authenticated/* — unauthenticated users are redirected to /auth.
• No anonymous sign-ups.

2.2 Track library

• Authenticated users see their own tracks at /tracks (RLS scoped to auth.uid()).
• Upload flow accepts audio files; a row is created in tracks with status pending.
• Track detail page at /tracks/:trackId renders the DNA specimen sheet once analysis lands.

2.3 Music DNA layer (mock provider for now)

All DNA reads go through MusicDNAProvider in src/lib/music-dna/. Providers are swappable via env flag:

• self-built — Fadr L1 + JS L2 + LLM L3 (MVP default when Fadr key is wired)
• musicatlas — Plan A, stubbed and flagged off
• cyanite — Plan C, interface only

The schema is ours; swapping providers is a one-line change.

Three-layer taxonomy:

• L1 Measured (confidence 1.0) — key, BPM, time signature, chord progression, section timestamps, loudness. Rendered with solid borders.
• L2 Derived (0.9) — energy curve (Web Audio RMS), section loudness, harmonic complexity, vocal presence, stereo width, chord movement classification (Roman numerals + common names like "Sensitive Female Progression"). Rendered with dashed borders.
• L3 Interpreted (0.7) — genre, sub-genre, mood adjectives, era feel, "sounds like" references, plain-English why per tag. Rendered with muted borders and a "why this tag?" affordance.

2.4 Chord chat (single agent, per-track)

• Timeline of chords aligned to sections.
• Chat panel that has the current DNA blob + selected chord range in context.
• Uses Lovable AI Gateway (no API key required from user).

2.5 Correction log

• Every DNA tag has a "this is wrong" affordance.
• Corrections are logged to dna_corrections and used to retune thresholds later. Bumps to METHODOLOGY_VERSION when L2 thresholds change.

2.6 Landing / sales page

• Marketing route at / describes the full product (including the APIs that ship in days).
• SEO metadata per route (title, description, og:*, twitter:card).

2.7 Spec page

• This page. Downloadable as Markdown.

3. What the app does WHEN THE 3 APIS LAND (in days)

3.1 Stem separation API

• User uploads a mix; we separate into vocals, drums, bass, other stems.
• Stems power:

- Vocal presence L2 metric (real, not RMS-approximated) - Hum-to-analyze — user hums a melody, we isolate voice and analyze pitch contour - Ref listening — Reference Scout agent listens to isolated drums to answer "what's the kick pattern doing here?" - Stem download for the songwriter to use in a DAW.

3.2 MusicAtlas Song DNA API (Plan A provider)

• Replaces the mock self-built provider as the primary L1 + L2 source when the flag flips.
• Adds: rhythmic density, groove micro-timing, MERT embeddings, Essentia mood — the Python-only metrics that were roadmap under the JS-only MVP.
• Adds nearest-neighbor "sounds like" search over an embedding index — powers the Reference Scout agent and the cross-track searchTracks / compareTracks tools.

3.3 ElevenLabs music snippet API

• Generate short (5–20s) musical snippets from a text prompt or from a DNA blob.
• Wired into the Blueprint Architect agent as a tool: user says "give me a bridge that departs from this vi–IV–I–V but keeps the tempo" → agent calls the snippet API with the extracted constraints → snippet plays inline in the chat transcript.
• Snippets are stored in snippets (RLS-scoped) and can be dragged back into a track's timeline as an idea marker.

3.4 Deepgram voices + VAD hands-free

• Each agent is bound to a Deepgram Aura-2 voice (Chord Coach → aura-2-luna-en, Producer → aura-2-orion-en, Lyricist → aura-2-andromeda-en, Reference Scout → aura-2-arcas-en, Blueprint Architect → aura-2-zeus-en). User can preview and change any agent's voice; choice persists per user per agent.
• VAD (voice-activity detection) loop with tunable sensitivity presets ("quiet room / normal / loud studio"): tune START_RMS, SILENCE_MS, and busyRef lock so the mic doesn't hear the app's own playback or a nearby guitar amp.
• Hands-free mode means the songwriter can keep both hands on the instrument. Sing, hum, play a chord — the agent responds by voice.

4. Agent roster (chat + voice)

Five specialist agents. Each is one config file: agent, voice, greeting, systemPrompt, tools bundled together.

Entry points (same VoiceBot component both places):

• Per-track: /tracks/:trackId — track DNA is in context.
• Global: /chat and /chat/:threadId — library-wide tools; a sidebar of sessions; + New picks an agent.

Threading: one thread per (user, track, agent). Switching agents swaps threads. Global chat threads have real URLs so reload works.

5. Data model

Public schema tables (all with RLS + explicit GRANTs):

• tracks — user's uploaded audio + status
• dna_blobs — L1/L2/L3 payload, provider, methodology version, provenance
• dna_corrections — user corrections keyed to (track_id, tag_path)
• chat_threads — one row per (user, scope, agent), scope is track:<id> or global
• chat_messages — role, content, tool_calls, tool_results, audio_url (for voice replies)
• agent_voice_prefs — (user_id, agent_id) → voice_id
• snippets — ElevenLabs-generated audio, prompt, DNA constraints, source thread
• user_roles — role enum stored separately from profiles (never on the profile table)

Auth is via the _authenticated route gate. Server functions that touch user-scoped data use requireSupabaseAuth middleware.

6. Architecture

• Framework: TanStack Start v1 on Vite 7, React 19. SSR-capable, deployed to Cloudflare Workers (edge).
• Styling: Tailwind v4 via src/styles.css, semantic tokens only — no hardcoded colors.
• Backend: Lovable Cloud (Supabase under the hood). App-internal server logic uses createServerFn. Webhooks and cron endpoints live under src/routes/api/public/*.
• AI: Lovable AI Gateway for chat completions and embeddings. Deepgram for STT + TTS. ElevenLabs for snippet generation. MusicAtlas for Song DNA. A stem-separation vendor for stems.
• DNA provider abstraction: MusicDNAProvider interface at src/lib/music-dna/provider.ts. Never hardcode a provider in feature code.

7. Aesthetics (locked)

• Voice: specimen sheet, not dashboard. Reference: Braun product manuals, ECM record inserts, Muji signage.
• Numbers: font-variant-numeric: tabular-nums. Values right-aligned in tables.
• Borders: solid = measured, dashed = derived, muted = interpreted. Border is the confidence signal — no colored badges for confidence.
• Charts: line only. No gradient fills, no glow, no drop shadow.
• Chips: DNA tags are chips with a thin border; hover reveals the why string.
• Chat surface: transcript is the UI. Tool calls render as inline pill-shaped cards with the tool name and a compact result — not as JSON.
• No generic AI aesthetic. No purple/indigo gradients. No Inter/Poppins as headline fonts.
• Motion encodes data, never decorates. Visuals that move (helix rotation, ribbon sweep, constellation arcs, confidence pulse) do so because the data is what's moving — animation speed comes from confidence, weight from duration, opacity from frequency. No idle glow, no ambient shimmer.

7b. DNA visuals (shipped)

Below the specimen sheet on every track page, five dynamic visualizations turn numbers into a living portrait:

1. DNA Helix — rotating double-helix, one rung per chord, rung weight = duration, stroke tone = major/minor.
2. Energy Ribbon — mirrored RMS ribbon with dashed section boundaries and a sweeping playhead glimmer.
3. Chord Constellation — unique-chord ring; arcs weighted by transition frequency; animated stroke offset.
4. Section Contact Sheet — one card per section with a mini sparkline of that section's energy, LUFS, and duration.
5. Confidence Weather — every DNA tag as a cell; solid/dashed/dotted border by confidence tier; low-confidence cells breathe.

Five more visuals are queued behind the incoming APIs: Reference Scout Nebula (MusicAtlas embeddings, 2D projection of your taste), Stem Cross-Section (per-stem waveform lanes), Snippet Sculpt (morphing waveform for iterated snippets), Live Waveform Sync (playhead-driven full-mix waveform), and Lyric Prosody River (syllable stress + rhyme arcs). See .lovable/plan.md §Phase 5 for the return-to plan.

8. Build order

1. Agent roster + tool-calling chat (Chord Coach, Producer, Lyricist) — mostly prompt + config work
2. Persistent per-track threads
3. searchTracks and compareTracks tools — unlock library-wide questions
4. Cross-track global chat page (/chat)
5. Wire stem separation, MusicAtlas, and ElevenLabs the day their keys arrive
6. Deepgram STT + TTS on the chat surface
7. Per-agent voice picker (persisted in agent_voice_prefs)
8. VAD hands-free loop with sensitivity presets
9. Hum-to-analyze + live reference listening — the songwriter-specific superpowers text chat cannot do

Steps 1–4 give the product real value with only what's shipped today. Steps 5–9 turn it into the hands-free studio companion.

9. Non-goals

• Not a DAW. We do not replace Logic, Ableton, or FL Studio.
• Not a distribution platform. We do not publish to Spotify.
• Not a stock-music library. Snippets are sketches for the songwriter, not licensed cues.
• Not a general-purpose chatbot. Every agent is scoped to songwriting.

10. Open questions

• Which stem-separation vendor (Demucs-hosted, LALAL.AI, or the vendor bundled with the incoming API)?
• Do we expose the raw MusicAtlas embedding vector to power users, or keep it internal?
• Should Blueprint Architect be allowed to call generateSnippet without user confirmation, or always ask first (credits cost real money)?

This spec is the source of truth. When it changes, bump `SPEC_VERSION` in `src/lib/spec-doc.ts` and the plan doc in `.lovable/plan.md`.