
mix of engineering and research | building voice @rumik_ai
Joined August 2023
- Tweets157
- Following204
- Followers128
- Likes805
spent a day optimizing seed-vc. assumed batching was the free lunch. It was 1.4x.
the real cost: the reference prompt is prepended to every chunk, diffused through all 15 steps, then thrown away. you can't cache it, because the DiT is bidirectional. you can only make it shorter.
1
10
238
Its been approx 20+ hrs and finally reduced my GRPO runtime from ~12.8 hrs to ~1.5 hrs. used vLLM inference on one GPU and DDP on the other 3, applied pretty much everything I had.
using 3 GPUs for DDP, hosted vLLM inference on one GPU, with 8 samples generating parallely, along with codec tokens, grad_accum=4, data sharding across ranks, micro-batching within the loss function, TF32 tensor cores, BF16 autocast for forward pass, Flash SDP + mem-efficient SDP, cuDNN benchmark mode, BF16 reduced precision reduction... works well for Hopper architecture.
also turned on gradient checkpointing, manual gradient all-reduce (instead of full DDP wrapping), plus a file-based vLLM watcher that restarts the inference server with fresh merged weights at every optimizer step from a clean process tree (to avoid NCCL conflicts with torchrun), with retry logic on generation calls during server.
biggest debugging rabbit holes:
vLLM V1 silently ignoring stop_token_ids (had to force V0 with VLLM_USE_V1=0), and merge_and_unload() with tie_word_embeddings=True dropping the trained lm_head during save, model generates infinite codec tokens and never stops. fix: untie before merge so both embed_tokens and lm_head are saved separately.
GRPO on TTS is a different beast from text!
listening to this audio generated from lstm music-generator that I mad 2 years back ^^.
I was generating abc notation autoregressivly and then converting that into midi file formate using abc2midi library
huggingface.co/spaces/nullHa…
3
143
Ablated refusal behaviour from BharatGen's Param-1 2.9B Instruct making it a uncensored model
huggingface.co/nullHawk/Para…
1
153







