Skip to content
arXiv Game AI· Kyungsu Kim·· 2026-10-09PickAI score43

LiveSynth: A Streaming Neural Synthesizer for Instrument Cloning and Text-to-Instrument

LiveSynth: A Streaming Neural Synthesizer for Instrument Cloning and Text-to-Instrument

AI summary

LiveSynth is a streaming neural synthesizer that supports zero-shot instrument cloning and text-to-instrument generation, and can be played interactively in real time. It uses a feedback-free causal Transformer conditioned on MIDI events and CLAP timbre embeddings to output continuous audio latents at 100 Hz, which a causal variational autoencoder then decodes into audio; frame-level timbre conditioning enables continuous timbre morphing, and MIDI conditioning dropout allows autonomous continuation from a performance prefix.

Why it matters

The paper sets out the architecture and real-time performance data of a streaming neural synthesizer, useful for understanding low-latency generation for playable instruments.

Source: arXiv Game AI · arxiv.org