Great paper from AWS.
I use a similar setup where an agent orchestrator sits on top of a multi-agent system.
(bookmark it)
This work introduces one of the many approaches available to manage compound LLM systems.
Compound LLM systems usually solve coordination by adding a higher-level model.
That meta-agent reads worker outputs, writes the final answer, allocates later calls and decides when to stop, which concentrates three separate control decisions in one opaque, order-sensitive call.
UnitBoost investigates whether the manager needs to be generative at all.
A task-given unit map turns worker outputs into slot-value proposals, a constrained argmax assembles the output, and slots left unfilled or unsupported become an explicit residual that directs the next round.
On three held-out benchmarks it beats the best single candidate chosen with gold labels by 0.060 to 0.195 task-score points, and beats input-matched generative managers by 0.048 to 0.076.
Replacing only the management step improves six compound-system configurations. Residual-directed rounds raise FanOutQA cell F1 from 0.4778 to 0.5524.
Chat with Paper: academy.dair.ai/papers/unitb…
Dawei retweeted
It's exciting to see renewed interest in weakly supervised learning as well as its promises and limitations when applied to LMs. 9/N
cc @cs_dawei @andst_link
📢 Check out our new #ACL2023 paper! "Weaker Than You Think: A Critical Look at Weakly Supervised Learning"
⚠ Want to apply weak supervision to solve your real-world tasks? Wait a second! ⚠
arxiv.org/abs/2305.17442
🧵 1/N
We hope our findings and recommendations will spur more robust future work in WSL such that future methods are truly beneficial in realistic low-resource scenarios. 🙌🙌🙌
🧵11/N
This is joint work with Xiaoyu Shen, Marius Mosbach (@mariusmosbach), Andreas Stephan (@andst_link ), and Dietrich Klakow.
🧵 12/N
Dawei retweeted
We will present our latest work on learning with noisily labeled data at #AAAI2021😊 If you are interested in distant supervision and noisy labels, we invite you to visit our poster this weekend or take a look into the paper arxiv.org/abs/2101.09763 @cs_dawei







