CaptionEmporium/conceptual-captions-cc12m-llavanext
Dataset Card for conceptual-captions-cc12m-llavanext Dataset Summary This is a data of 21,930,344 synthetic captions for 10,965,172 images from conceptual_12m. In the interest of reproducibility, an archive found here on Huggingface was used (cc12m-wds). The captions were produced using llama3-llava-next-8b inferenced in float16, followed by cleanup and shortening with Meta-Llama-3-8B. Languages The captions are in English. Data… See the full description on the dataset page: https://huggingface.co/datasets/CaptionEmporium/conceptual-captions-cc12m-llavanext.
Thành tựu nổi bật
Những danh hiệu mới nhất của dự án
#1 Repository Của Tháng
#1 Repository Của Tháng
#1 Repository Của Tháng
Dataset Card for conceptual-captions-cc12m-llavanext
Dataset Description
- Point of Contact: Caption Emporium
Dataset Summary
This is a data of 21,930,344 synthetic captions for 10,965,172 images from conceptual_12m. In the interest of reproducibility, an archive found here on Huggingface was used (cc12m-wds). The captions were produced using llama3-llava-next-8b inferenced in float16, followed by cleanup and shortening with Meta-Llama-3-8B.
Languages
The captions are in English.
Data Instances
An example of a row:
{
"caption_llava":"Two loaves of golden brown Cuban bread, one slightly overlapping the other, resting on a white surface, with a focus on the crust's texture and the hint of a tropical setting.",
"caption_llava_short":"Golden brown Cuban bread loaves rest on a white surface, showcasing their textured crust and hinting at a tropical setting.\n",
"caption":"This is the best recipe I have ever tried for Cuban bread. I lived in Key West... Cuban Recipes, Bread Recipes, Cooking Recipes, Cuban Desserts, Pan Cubano Recipe, Cuban Bread, Cuban Sandwich, Sandwiches, Recipe From Scratch",
"url":"https://i.pinimg.com/originals/da/5e/76/da5e7622c119c4c96b9e42e7e2a667a0.jpg",
"key":"000000001",
"status":"success",
"error_message":"None",
"width":555,
"height":416,
"exif":"{}",
"original_width":555,
}
Data Splits
| train | |
|---|---|
| conceptual-captions-cc12m-llavanext | 10965172 |
Dataset Creation
Producing the Captions
https://huggingface.co/lmms-lab/llama3-llava-next-8b was prompted with the following to produce a caption:
prompt_gen = lambda txt :f"""
Please make a detailed but succinct caption of this image. If you see text or objects, be sure to describe them in detail along with any other aspects of the foreground and background. As a hint, here is the alt-text attribute of the image, which may or may not have to do with the image:
Hint:
\`\`\`
{txt}
\`\`\`
"""
This produced image captions with failures approximately 2.6% of the time. Failures were defined as either:
- Having one of the following repetitive pieces of text:
to_reformats = [' no text', ' other objects', ' additional objects', ' no objects ', 'alt-text']. - Having a repetitive sequence.
These captions were reformatted with Meta-Llama-3-8B to fix repetitions or remove the mention of these things. Then, prefixes were pruned as in anime-caption-danbooru-2021-sfw-5m-hq.
Short captions were produced from the resulting captions using the following prompt in Meta-Llama-3-8B:
prompt = lambda img_prompt: f"""
Please take the following image caption and attempt to distill it into a single sentence. Remove any redundant lines or descriptions and make it a maximum of 30 words in length.
\`\`\`
{img_prompt}
\`\`\`
Please only write the caption and no other text.
"""
Source Data
Discussion of Biases
Please refer to the original conceptual_12m repository. The captions are likely highly biased on the alt-text of the image and the training data of the vision-language model.
Known Limitations
There may still be a very small number of erroneous captions present, but the vast majority have been eliminated. A small percentage of images failed to caption (~0.0018%) and these images have the caption "An image".
Additional Information
Dataset Curators
Caption Emporium
Licensing Information
The dataset is available under the Creative Commons ShareAlike (CC BY-SA 4.0). As per the original dataset repository, Google LLC ("Google") is acknowledged as having aggregated the original dataset.
Special Thanks
The following person helped provide compute to assist with the generation of the captions:
Citation Information
@misc{conceptual-captions-cc12m-llavanext,
author = { Caption Emporium },
title = { conceptual-captions-cc12m-llavanext },
year = { 2024 },
publisher = { Huggingface },
journal = { Huggingface repository },
howpublished = {\url{https://huggingface.co/datasets/CaptionEmporium/conceptual-captions-cc12m-llavanext}},
}
Dự án tương tự
Khám phá các dự án và mô hình AI khác có sự tương đồng.
Infinity Instruct Beijing Academy of Artificial Intelligence (BAAI) [Paper][Code][🤗] The quality and scale of instruction data are crucial for model performance. Recently, open-source models have increasingly relied on fine-tuning datasets comprising millions of instances, necessitating both high quality and large scale. However, the open-source community has long been constrained by the high costs associated with building such extensive and high-quality instruction… See the full description on the dataset page: https://huggingface.co/datasets/BAAI/Infinity-Instruct.
OpenAssistant Conversations Dataset (OASST1) Dataset Summary In an effort to democratize research on large-scale alignment, we release OpenAssistant Conversations (OASST1), a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 fully annotated conversation trees. The corpus is a product of a worldwide crowd-sourcing effort… See the full description on the dataset page: https://huggingface.co/datasets/OpenAssistant/oasst1.
Summary databricks-dolly-15k is an open source dataset of instruction-following records generated by thousands of Databricks employees in several of the behavioral categories outlined in the InstructGPT paper, including brainstorming, classification, closed QA, generation, information extraction, open QA, and summarization. This dataset can be used for any purpose, whether academic or commercial, under the terms of the Creative Commons Attribution-ShareAlike 3.0 Unported… See the full description on the dataset page: https://huggingface.co/datasets/databricks/databricks-dolly-15k.
News [2025/04/22] We split the data and kept only the medical SFT dataset (medical_o1_sft.json). The file medical_o1_sft_mix.json contains a mix of medical and general instruction data. [2025/02/22] We released the distilled dataset from Deepseek-R1 based on medical verifiable problems. You can use it to initialize your models with the reasoning chain from Deepseek-R1. [2024/12/25] We open-sourced the medical reasoning dataset for SFT, built on medical verifiable problems and an… See the full description on the dataset page: https://huggingface.co/datasets/FreedomIntelligence/medical-o1-reasoning-SFT.
Dataset Card for GLUE Dataset Summary GLUE, the General Language Understanding Evaluation benchmark (https://gluebenchmark.com/) is a collection of resources for training, evaluating, and analyzing natural language understanding systems. Supported Tasks and Leaderboards The leaderboard for the GLUE benchmark can be found at this address. It comprises the following tasks: ax A manually-curated evaluation dataset for fine-grained… See the full description on the dataset page: https://huggingface.co/datasets/nyu-mll/glue.
GAIA dataset GAIA is a benchmark which aims at evaluating next-generation LLMs (LLMs with augmented capabilities due to added tooling, efficient prompting, access to search, etc). We added gating to prevent bots from scraping the dataset. Please do not reshare the validation or test set in a crawlable format. Data and leaderboard GAIA is made of more than 450 non-trivial question with an unambiguous answer, requiring different levels of tooling and autonomy to… See the full description on the dataset page: https://huggingface.co/datasets/gaia-benchmark/GAIA.
Dataset Card for SQuAD Dataset Summary Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. SQuAD 1.1 contains 100,000+ question-answer pairs on 500+ articles. Supported Tasks and Leaderboards Question… See the full description on the dataset page: https://huggingface.co/datasets/rajpurkar/squad.
Dataset Card for Alpaca-Cleaned Repository: https://github.com/gururise/AlpacaDataCleaned Dataset Description This is a cleaned version of the original Alpaca Dataset released by Stanford. The following issues have been identified in the original release and fixed in this dataset: Hallucinations: Many instructions in the original dataset had instructions referencing data on the internet, which just caused GPT3 to hallucinate an answer. "instruction":"Summarize… See the full description on the dataset page: https://huggingface.co/datasets/yahma/alpaca-cleaned.
中文基于满血DeepSeek-R1蒸馏数据集（Chinese-Data-Distill-From-R1） 🤗 Hugging Face | 🤖 ModelScope | 🚀 Github | 📑 Blog 注意：提供了直接SFT使用的版本，点击下载。将数据中的思考和答案整合成output字段，大部分SFT代码框架均可直接直接加载训练。 本数据集为中文开源蒸馏满血R1的数据集，数据集中不仅包含math数据，还包括大量的通用类型数据，总数量为110K。 为什么开源这个数据？ R1的效果十分强大，并且基于R1蒸馏数据SFT的小模型也展现出了强大的效果，但检索发现，大部分开源的R1蒸馏数据集均为英文数据集。 同时，R1的报告中展示，蒸馏模型中同时也使用了部分通用场景数据集。 为了帮助大家更好地复现R1蒸馏模型的效果，特此开源中文数据集。 该中文数据集中的数据分布如下：… See the full description on the dataset page: https://huggingface.co/datasets/Congliu/Chinese-DeepSeek-R1-Distill-data-110k.