Keep up with the latest machine learning datasets from @paperswithcode. Follow for daily updates.
Joined April 2021
- Tweets284
- Following1
- Followers16.4K
- Likes129
🦿QDax: a Quality-Diversity benchmark suite for Deep Neuroevolution in Reinforcement Learning domain for robot control.
It specifies different benchmarks based on the complexity of both the task and the agent controlled by a deep neural network.
paperswithcode.com/dataset/q…
🏞ImageNet-X: a set of annotations of factors like pose, background, or lighting of the ImageNet validation set and 12K training images for studying the types of mistakes as a function of a model's architecture, learning paradigm, and training procedures.
paperswithcode.com/dataset/i…
🎯ACES: a Translation Accuracy Challenge Set, consisting of 36,476 examples covering 146 language pairs and representing challenges from 68 phenomena, designed to evaluate a wide range of machine translation metrics. More details below…
Dataset: paperswithcode.com/dataset/a…
1/4
The increase in parameters for many models only showed minor improvements when assessed on MATH. Minerva 540B attains SOTA by pre-training on high quality scientific and mathematical data and has answers selected via majority voting.
paperswithcode.com/sota/math…
5/7
1
6
Currently, models evaluated on MATH are mostly general language models like GPT-2, GPT-3 and PaLM. Minerva (based on PaLM) is a recent model designed to perform quantitative reasoning tasks.
Check all the model results on the MATH Leaderboard: paperswithcode.com/sota/math…
4/7
There are more than 12K math problems in the MATH dataset, each are tagged by difficulty from 1 to 5 complete with a full step-by-step solution and they span seven subjects. Geometry is one of them where diagrams can be specified in text using Asymptote language.
3/7
1
4
🤔How good are language models at solving math problems?
Let’s review the progress of LMs on the MATH benchmark in this thread.
1/7
➗Lila: a mathematical reasoning benchmark consisting 23 tasks covering mathematical abilities, language format, language diversity and external knowledge.
It's an extension of 20 datasets by collecting Python programs’ task instructions and solutions.
paperswithcode.com/dataset/l…
{}Code Syntax: a dataset of programs annotated with the syntactic relationships in their syntax trees designed for code syntax understanding tasks.
It contains 30K+ code samples annotated with 2M+ relation edges in 43 relation types for Python and Java.
paperswithcode.com/dataset/c…
💻The Stack: a dataset for pre-training Code LLMs. It contains 3TB of permissively-licensed code in 30 programming languages.
It’s created as part of the BigCode Project, an open scientific collaboration working on responsible development of Code LLMs.
paperswithcode.com/dataset/t…
🏺Breaking Bad: a large-scale dataset of fractured objects. It has 10k meshes, each with 100 fractures mode, totalling 1M breakdown patterns.
It serves as a benchmark for fractured object reassembly and new challenges for geometric shape understanding.
paperswithcode.com/dataset/b…
🌄Diffusion DB: a large text-to-image prompt dataset containing 2M images generated by Stable Diffusion using user’s prompts and hyperparameters.
It opens up research in prompt engineering, deepfakes detection, and designing public AI tools.
paperswithcode.com/dataset/d…
1
10
👀Perception Test: a benchmark for evaluating the perception and reasoning skills of multimodal models.
It uses real-world videos to define tasks that require understanding of memory, patterns, physics, and semantics across visual, audio, and text.
paperswithcode.com/dataset/p…
🏞Avalon: a benchmark for Reinforcement Learning Generalization using procedurally generated worlds.
It consists of tasks where embodied agents in diverse 3D worlds must survive by navigating terrain, hunting or gathering food, and avoiding hazards.
paperswithcode.com/dataset/a…
🗣SpeechMatrix is a large-scale corpus designed for multilingual speech-to-speech translation.
The data is mined from real speech of European Parliament recordings. It contains a total of 418K hours of speech alignments in 136 language pairs.
paperswithcode.com/dataset/s…
👤UnrealEgo: a dataset that provides in-the-wild stereo images with a large variety of motions for 3D human pose estimation.
A total of 450k (900k images) are captured for the dataset and metadata is provided for each frame.
paperswithcode.com/dataset/u…
📚MTEB: a benchmark suite spanning 8 embedding tasks with 56 datasets and 112 languages.
Tasks include classification, clustering, retrieval and more. It opens up opportunities for the development of universal text embedding models in the ML field.
paperswithcode.com/dataset/m…
