teaching computers to speak @dubguild || phd from @nagoyauniv
republika ng pilipinas
Joined July 2020
- Tweets358
- Following684
- Followers254
- Likes3K
Pinned Tweet
TTSモデルを1Bから3Bまでスケールさせました！
よかったら読んでみてね〜
テックブログを公開しました。
「Scaling Speech AI」の下、1Bから3Bへと音声言語モデルをスケールさせた際のTTS性能へ影響を検証しました。
日本語特有の読みや表記揺れ、表現の広がりがみられるに加え、現状の課題についても整理しています。
日本語音声生成・SpeechLM・TTSに関心のある方はぜひご覧ください。
blog.dubguild.com/melte/llm-…
1B/3Bモデルの構築にあたって実施した、データ前処理・事前学習・事後学習の詳細も、今後順次公開していく予定です。
続報もお待ちいただければ幸いです。
as a visual learner, i’ve always appreciated people who take the time to explain topics with visual aids (especially @3blue1brown!)
but this channel has helped me understand so many SOTA ML concepts. thank you for all the effort you put into making these videos 🙇♂️🙇♂️🙇♂️
Wow! Such a cool paper on fixing softmax and cross-entropy!
I enjoyed reading this paper today, so I made an explainer to make the ideas more accessible to a broader audience.
Check it out: youtu.be/dYxEmKbZfr4
here’s a summary of the amazing work done by Zhijie Huang during his time at @DubGuild !
Real-world noise makes speech tokens unstable.
We train a neural codec to recover clean speech from degraded audio through augment-and-denoise training.
Our new blog is here！！
blog.dubguild.com/melte/nois…
lester violeta retweeted
💫 Large Language Continuous Diffusion Models
Presenting Sigma, the first large-scale (3B/8B) continuous diffusion LM (dLM) trained with likelihood.
📈 Comparable with discrete dLMs and AR on math and coding
🎛️ Enables meaningful study of inference-time interventions
Jointly led with Wei Guo (@WeiGuo01), Jean-Marie Lemercier (@jm_lemercier), Simon Welker (@_cobalamin)
Team: @YongganFu @mmkamani7 Sajad Norouzi @julberner @tomasgeffner @karsten_kreis @YongxinChen1 @MoleiTaoMath @jwthickstun @PavloMolchanov Ante Jukic @ArashVahdat @MardaniMorteza
lester violeta retweeted
A nice presentation by @paveltyletski on
Diffusion Gemma: 4x faster text generation.
I think that text diffusion will not only be 4x faster and 10x more energy efficient, but eventually it will also be of higher quality, and supersede autoregressive LLMs.
youtu.be/3n5RN7i8h2Y?si=YkYz… via @YouTube
recently i've been complaining about low quality papers getting accepted in conferences
but now i realized that the only way to alleviate this is to actually review papers, so now im increasing my max papers to review
hoping a lot more people can do this
lester violeta retweeted
Always wanted to see this experiment! Interesting!!
arxiv.org/abs/2609.30975
“pewdiepie gets banned for trying to steal reasoning traces from gpt” sounds like a sentence generated by an llm trained on reddit
omg brings me back to watching @karpathy youtube tutorials
pretty great demo i wonder if it also knows the notations (eg just say do F R U R' U' F' to get the top cross)
a one-shot last layer would have also been great to see 😃
one thing that caught my eye is that the ASR in the dots demo removes fillers and rephrases it into a text format for your notes (was it always like that?)
openai.com/index/devday-2026…
being back on opus 5.5 token costs after using fable for a while basically feels like suddenly being granted unlimited usage
agentic coding was hard for me to embrace until recently, when i realized it’s literally what i wanted 15 years ago (i got really interested in engineering after seeing tony stark work with jarvis)
i’m naturally skeptical, but with how fast ai moves, i don’t think i can afford to let that skepticism turn into fear of change. finding the balance between staying critical and staying open is still difficult though
literally a turing test amiright
Two days ago, GPT-6 Astra broke a yet unsolved German Army Enigma message from 1941. Amazingly Astra was able to autonomously:
- Search historical archives
- Compare uncertain letters
- Find contextual clues
- Build an Enigma simulator
- Write cryptanalysis code
- Run parallel experiments
- Test competing keys
- Recover the plaintext
- Cross-check the results
1/n
i have such a soft spot for researchers who share what their training run looks like, its such an underrated piece of information but it actually helps a lot in letting other people reproduce the experiments
Nearly half a year of silence. We spent it studying one problem: how far RL can scale.
MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks.
Streaming the run: mimo.xiaomi.com/rl/
lester violeta retweeted
原因を述べてみた
medium.com/@unilight/%E9%9F%…
oh my god lmfao
BREAKING: The NBA has ruled on the Los Angeles Clippers in for salary cap circumvention investigations on Kawhi Leonard after yearlong probe -- stripping the franchise of 5 first-round picks, issuing a $30 million fine to owner Steve Ballmer and suspensions for Ballmer, Lawrence Frank and Gillian Zucker, sources tell ESPN.
Kawhi Leonard will have to pay $700,000 in restitution for improper benefits by the Clippers for his uncle and former business rep, Dennis Robertson. No contract void or suspension for Leonard. And Robertson -- who was fired by Leonard in June -- is being banned by the NBA from all business dealings.
永住許可の申請手数料20万円になってて鬱
在留手数料大幅値上げ10月から パブコメで変更なし 政府決定
mainichi.jp/articles/2026082…
在留期間が3カ月以下は1万円▽1年は3万3000円▽5年以上は7万5000円――など期間に応じた金額を徴収へ。永住許可の申請も現在の1万円から20万円に引き上げられます。
“claude-ish writing” actually reminds me of my writing style back when i was an undergrad. i went to a catholic university which meant taking lots of humanities classes even up until my senior year. interestingly having to write essays on philosophy/theology/psychology/etc also influenced how me and the other engineering students wrote our technical reports, where the use of sophisticated words and analogies was rewarded as “intellectual writing” instead of just being straight to the point
it took me a few years to shake this writing style off when writing technical papers. for claude i actually wonder if aside from rlvr checking the code, some other reward function is also used which rewards sophisticated vocabs & analogies == intellectual writing, causing it to become “claude-ish”?