Run Fish Audio Drama 3 in Turkish on a Single 16 GB GPU

What it does

This project adapts the Fish Audio Drama 3 text-to-speech model for Turkish by replacing its full speaker head with a distilled 32-expert module. The key architectural trick is streaming: instead of loading the massive 41.5 GB checkpoint into RAM or VRAM, the engine pulls expert weights directly from an NVMe drive during inference. This allows the system to run on consumer hardware with just 16 GB of VRAM.

The output preserves Drama 3’s ability to interpret inline directions (like [whispering] or [excited]) written in Turkish within square brackets. Benchmarks show it runs at roughly 2.7x real-time speed for short sentences and up to 3.4x for paragraphs when using a "hot tier" cache that keeps frequently used experts in pinned RAM.

Who it helps

Developers and researchers working on Turkish audio generation who are constrained by memory limits. It’s particularly useful for local deployments where cloud TTS APIs are too expensive or privacy-sensitive. The project provides concrete performance metrics, including a Word Error Rate (WER) of 2.27% on the Freya-TR-Eval benchmark, making it a viable option for high-quality synthetic speech without needing enterprise-grade GPUs.

Getting started

Setup involves cloning the repository and installing dependencies via pip. You need to download three components:

1. The base Fish Audio Drama 3 model from Hugging Face.
2. The distilled Turkish speaker weights (speaker.safetensors).
3. The expert trace file (trace_experts.pt) which informs the hot tier selection.

To run inference, you use a Python script with specific flags. For example, --disk --hot 6000 enables disk streaming and caches the top 6,000 experts in RAM (requiring ~25 GB total system RAM). If your machine has less than 32 GB RAM, omit the --hot flag to rely entirely on disk reads, though this drops speed to ~1.2x real-time. A simple HTTP server (server.py) is also included for integration into applications.

Limits and costs

The software is open-source under Apache-2.0, but the underlying Fish Audio Drama 3 model may have separate licensing terms; check their official documentation before commercial use. Hardware requirements are strict: you need an NVIDIA GPU with at least 16 GB VRAM and fast NVMe storage. Peak VRAM usage hits 15.0 GiB during paragraph generation. System RAM needs vary significantly based on whether you enable the hot tier cache. An optional FP8 quantization mode exists but increases error rates slightly, so it is disabled by default.

Source links

• Official Repository
• Project Documentation