MLLMs hallucinate bounding boxes as confidently as real ones!
MTLA catches them by reading the model's own attention: does it look inside the box it drew?
Training-free, one recipe for all modalities and model families.
arxiv.org/abs/2607.05978
github.com/TalRemez/MTLA
#AI #LLM
1
2
81
Tal Remez retweeted
MusicGen from FAIR:
Generates music from a text description.
Paper + open source code + models + demo on HuggingFace.
We've just released MusicGen, and there is a @huggingface demo now, here is a thread about me playing with it just right now. huggingface.co/spaces/facebo…
A 🧵👇
Tal Remez retweeted
Simple and Controllable Music Generation
paper page: huggingface.co/papers/2306.0…
introduce MusicGen, a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns, which eliminates the need for cascading several models, e.g., hierarchically or upsampling. Following this approach, we demonstrate how MusicGen can generate high-quality samples, while being conditioned on textual description or melodic features, allowing better controls over the generated output. We conduct extensive empirical evaluation, considering both automatic and human studies, showing the proposed approach is superior to the evaluated baselines on a standard text-to-music benchmark. Through ablation studies, we shed light over the importance of each of the components comprising MusicGen.
Tal Remez retweeted
We've just released MusicGen, and there is a @huggingface demo now, here is a thread about me playing with it just right now. huggingface.co/spaces/facebo…
A 🧵👇
Tal Remez retweeted
🤖 Demo available at: huggingface.co/spaces/facebo…
Tal Remez retweeted
Super excited to share that today we release MusicGen: a simple and controllable music generation.
🤖 🎵 🔊
📜 Paper: arxiv.org/pdf/2306.05284.pdf
🖥️ Code and models avail under: github.com/facebookresearch/….
🎵 Samples can be found here: ai.honu.io/papers/musicgen/
See more details👇
We present MusicGen: A simple and controllable music generation model. MusicGen can be prompted by both text and melody.
We release code (MIT) and models (CC-BY NC) for open research, reproducibility, and for the music community: github.com/facebookresearch/…
Tal Remez retweeted
We present MusicGen: A simple and controllable music generation model. MusicGen can be prompted by both text and melody.
We release code (MIT) and models (CC-BY NC) for open research, reproducibility, and for the music community: github.com/facebookresearch/…
Tal Remez retweeted
Pretrained LMs are fundamental in NLP. Can they benefit SpeechLMs?
We present TWIST, a SpeechLM training method using a textual LM warm-start. TWIST consistently outperforms cold-start baselines.
We also introduce the largest SpeechLM to date.
Paper: arxiv.org/abs/2305.13009
1/n
Textually Pretrained Speech Language Models
propose TWIST, a method for training SpeechLMs using a warm-start from a pretrained textual language models. We show using both automatic and human evaluations that TWIST outperforms a cold-start SpeechLM across the board. We empirically analyze the effect of different model design choices such as the speech tokenizer, the pretrained textual model, and the dataset size. We find that model and dataset scale both play an important role in constructing better-performing SpeechLMs. Based on our observations, we present the largest (to the best of our knowledge) SpeechLM both in terms of number of parameters and training data. We additionally introduce two spoken versions of the StoryCloze textual benchmark to further improve model evaluation and advance future research in the field
paper page: huggingface.co/papers/2305.1…
Tal Remez retweeted
Learn how a proof-of-concept visually-driven text-to-speech model, called VDTTS, generates speech that matches the exact timing of mouth movements in a video and can be used to replace audio that was recorded in noisy conditions. Read more ↓ goo.gle/37tUkuo
















