San Francisco, CA
Joined August 2022
- Tweets58
- Following170
- Followers326
- Likes213
I wonder if this is why Elizabeth Holmes lowers her voice on purpose 😭
The voice you choose matters for voice agents. We studied how listener demographics affect which voice they'd rather speak with, using 27,000+ preference votes from 676 listeners across 440 speakers.
A few interesting findings:
- Male listeners preferred the lower-pitched voice in nearly 60% of comparisons.
- Female listeners leaned toward female voices, while male listeners showed little gender preference.
- Listeners tended to prefer accents from their own country.
This has gotta be one of the most interesting benchmarks we’ve made LOL
Introducing MAFIA-GAME-BENCH🔪
Current conversational AI still struggles with persuasion, an important capability in sales, collections, and support. Agents need to make arguments, respond to pushback, and change people’s minds.
Our new benchmark measures this through the game of Mafia: real-time AI models play with humans, and we measure what people actually do.
Across 123 games with 58 human players, humans were much better at getting others to follow their arguments.
Here’s what we found:
GPT-Live-1 lowk cooked here lol
Can you beat GPT-Live-1 in a game of Mafia?
We played a few rounds with it and it was surprisingly good. In this video, the AI convinced the group to vote Vax out and eventually won.
We’ve been thinking about persuasion as one way to measure social intelligence in full-duplex models.
link below to play!
I’m starting a new hiking group in SF. I named it “Nerd Hiking” 😭 So happy to see fellow AI researchers and voice friends pulling up today.
If you’re a nerd who wants to join, DM me!
(Unfortunately, I forgot to get most folks in the photo… next time 😔)
Voice matters WAY more than people think. Excited to finally launch Sona Library.
Your customer might stay on the line just because your voice agent sounds like Scarlett Johansson.
Change nothing but the voice profile, and call success rates can shift by as much as 20%. The voice matters more than you think.
That’s why we’re launching Sona Library for voice AI companies. (thread below)
Psyched to see full-duplex speech-to-speech models finally getting the recognition they deserve 😄
GPT-Live-1 feels like one of the first models that voice agent companies are actually deploying with confidence.
Reasoning is getting really strong. Tool calling is getting really strong.
Which makes me think we’re getting much closer to the era of building AI that is not just intelligent, but socially intelligent.
…AI that understands underlying human intent and knows how to act appropriately depending on the context.
If you’re building voice agents, would love to hear what you think!
openai.com/index/introducing…
Ashley retweeted
NVIDIA's Personaplex model is super impressive, enabling natural, human-like conversations while responding with nods and affirmations!
Super natural sounding + can run locally at 7B size.
- Inspired by Moshi from @kyutai_labs
- Full duplex, which means AI listens while talking (no more awkward AI pauses
- Handles interruptions, backchannels, turn-taking
Congrats on the team!!
Future work (6/6)
- They plan to introduce models like acoustic echo cancellation to improve stability
- May also add more generative models to get more flexible and expressive audio output
- Congrats to the MoE Key Lab of AI and X-LANCE Lab (SJTU), Shanghai AI Laboratory, BIGAI, Fudan University, Northwestern Polytechnical University, and SIAT-CAS for this paper
Ashley retweeted
The Smart Turn model is integrated into Pipecat and available in Pipecat Cloud. Blog (+ dataset, model weights, GitHub repo): daily.co/blog/improved-accur…
Ashley retweeted
Congrats to the @trydaily team on the Smart Turn v3.1 release!
Smart Turn intelligently detects when a user has finished speaking by analyzing linguistic cues and intonation patterns, enabling voice agents to respond at the right moment.
We’re proud that our team at Liva contributed high-quality, targeted human voice data to help advance Smart Turn. The improvement in model accuracy underscores the value of real, diverse voice data.
Smart Turn v3.1.
Smart Turn is a completely open source, open data, open training code turn detection model for voice AI, trained on audio data across 23 languages.
The model operates on the input audio in a voice agent pipeline. Each time the user pauses briefly, this model runs and returns a binary decision about whether the user has finished speaking or not.
The 3.1 release has two big improvements:
1. New data sets for English and Spanish, collected and labeled by contributors Liva AI, Midcentury, and MundoAI.
The majority of the training data for the Smart Turn model is synthetically generated. Using synthetic data makes it possible to scale up training for a model like this. We've done a lot of work on the synthetic data pipeline to emulate as much of the natural variability of human speech as possible. But accurately labeled human data is very valuable and has a measurable impact on model quality. The 3.1 training run incorporates three new human data sets.
2. An unquantized, GPU-oriented version of the model alongside the ONNX version intended to run on CPUs.
The Smart Turn ONNX quant delivers a result in 12ms on my laptop and 70ms on a typical cloud vCPU. That's fast! Because this is an audio model, you can run it in parallel with transcription and it will generally give you a result before the transcription final chunks are available.
But if you have GPUs in your fleet, you can run the model even faster. (Or, more to the point, very scalably.) Inference runs in ~2ms on an NVIDIA L40S.
Read the launch blog post if you're interested in more details. And if you're running this model yourself, see notes in the blog post about ONNX runtime optimization.
The Smart Turn model is fully integrated into Pipecat, and available in Pipecat Cloud.
