If you're building your own voice server, the first decision is which family of engines to install. There are two, and they don't differ in the quality of the result: they differ in how they treat the graphics card's memory. Choosing wrong doesn't give you worse audio — it gives you a lost afternoon and a GPU you can't use for anything else.
| Your VRAM | What to install |
|---|---|
| 8 – 24 GB | hot/cold |
| 32 GB or more | vLLM |
If you only want the answer, that's it. What follows is why.
It loads the model when it's needed and releases it when it isn't. It keeps one "hot" worker ready to answer quickly and spins up "cold" ones on demand.
The practical consequence is the one that matters: the GPU is still yours for other things. You can have the voice server and, on the same card, generate images, run a small language model, or simply play a game. On a personal machine or a home lab that isn't a detail, it's the whole reason it exists.
It's what you want with an RTX 4070, a 4080, or any consumer card, and when you transcribe occasionally rather than continuously.
It reserves the memory at startup — on the order of 22 to 29 GB depending on how you configure it — and doesn't release it. In exchange it does continuous batching: instead of serving requests one at a time, it puts many of them through the same compute step.
When you have real volume, the difference is enormous. When you don't, you've turned an expensive card into a service that sits idle most of the time occupying all the memory.
It's what you want if you transcribe hours of audio a day, with many concurrent streams, and you have cards dedicated to inference.
This is where people get it wrong, because vLLM works and the numbers from an isolated test come out fine.
The problem is what that test doesn't show: with 16–24 GB, vLLM reserves practically the whole card permanently. You've paid the full price — losing co-location — without receiving the full benefit, because with that much memory the context cache isn't big enough for continuous batching to shine.
Below 16 GB it simply doesn't pay off: the cache is so small that hot/cold wins.
If in doubt, start with hot/cold. Changing later is a reinstall, not a rewrite.
In hot/cold there are several interchangeable synthesis engines behind an environment
variable, TTS_BACKEND. One of them, voxcpm, is not for production in this family:
its torch.compile path doesn't get along with the subprocess pool and causes a CUDA
allocator race under concurrent load. It's confirmed by the model's own maintainers, who
recommend a single-process runtime.
For that engine in earnest, the repository is the vLLM family one. In hot/cold it stays available for development and testing, and it warns you on load.
We mention it because it's the kind of thing that, if nobody tells you, you discover under load and in production.
Everything above is criteria, not results. The results are published and reproducible in uttera-benchmarks: four load profiles — latency, burst up to N=1024, and sustained — over two corpora.
If your case doesn't look like any of the four, clone it and measure your own. That's exactly what it's for.
And if what you want is to stand the whole server up, the complete path is in how to run your own voice server.
Anything to add or correct? Write to support@uttera.ai. If you correct us, we edit the post and credit you.
Uttera