Dự án này đã lâu không có cập nhật mới nên không còn hiển thị trên trang chủ, danh mục hay kết quả tìm kiếm — dữ liệu vẫn được giữ nguyên và trang này vẫn xem được bình thường.
Dataset
nlp-waseda

nlp-waseda/JMMLU

JMMLU is a four-choice question set consisting of Japanese-translated questions of a portion of MMLU (Translated questions) and questions based on unique Japanese cultural context (Japanese questions). It is designed to assess the performance of large language models in Japanese. JMMLU consists of 7,536 questions in the following 56 tasks (subjects).

14thích
1.2ktải xuống
0lượt xem

Thành tựu nổi bật

Những danh hiệu mới nhất của dự án

1
TINIX TRENDING·Chủ đề: Japanese

#1 Repository Của Tháng

1
TINIX TRENDING·Chủ đề: arxiv:2009.03300

#1 Repository Của Tháng

1
TINIX TRENDING·Chủ đề: license:cc-by-nc-nd-4.0

#1 Repository Của Tháng

JMMLU

Japanese Massive Multitask Language Understanding Benchmark

JMMLU is a four-choice question set consisting of Japanese-translated questions of a portion of MMLU (Paper, Github) (Translated questions) and questions based on unique Japanese cultural context (Japanese questions). It is designed to assess the performance of large language models in Japanese.

For the translated questions, a maximum of 150 questions from each of the 57 MMLU tasks (subjects) were selected and first machine-translated into Japanese. Next, the translators checked the machine translations and removed questions and tasks that were difficult to translate, irrelevant, or inconsistent with the Japanese culture. The remaining questions were modified to make them fluent.

The Japanese questions are based on school subjects, such as Japanese civics and history, and are manually created by Japanese teachers.

The format is the same as MMLU:

Question, Choice A, Choice B, Choice C, Choice D, Answer

Github

The JMMLU consists of 7,536 questions in the following 56 tasks (subjects).

Japanese Task NameEnglish Task NameNumber
専門医学professional_medicine150
専門心理学professional_psychology150
専門会計professional_accounting150
哲学philosophy150
雑学miscellaneous150
医学遺伝学medical_genetics99
形式論理formal_logic125
先史学prehistory150
天文学astronomy148
熟語japanese_idiom150
世界宗教world_religions147
世界事実global_facts97
世界史world_history150
社会学sociology150
栄養学nutrition149
日本史japanese_history150
日本地理japanese_geography139
人間の老化human_aging150
論理学logical_fallacies150
倫理的議論moral_disputes148
臨床知識clinical_knowledge150
経営学management102
解剖学anatomy132
計量経済学econometrics113
機械学習machine_learning111
国際法international_law120
公民japanese_civics150
公共関係public_relations109
高校心理学high_school_psychology150
高校物理high_school_physics150
高校統計学high_school_statistics150
高校数学high_school_mathematics150
高校生物学high_school_biology148
高校情報科学high_school_computer_science98
高校化学high_school_chemistry149
高校地理high_school_geography150
高校ヨーロッパ史high_school_european_history150
高校ミクロ経済学high_school_microeconomics149
高校マクロ経済学high_school_macroeconomics148
概念物理学conceptual_physics150
法理学jurisprudence107
電気工学electrical_engineering144
大学医学college_medicine150
大学物理college_physics100
大学数学college_mathematics99
大学生物学college_biology143
大学化学college_chemistry99
大学コンピュータ科学college_computer_science99
初等数学elementary_mathematics150
抽象代数abstract_algebra99
マーケティングmarketing150
ビジネス倫理business_ethics86
セクシュアリティhuman_sexuality130
セキュリティ研究security_studies150
コンピュータセキュリティcomputer_security99
ウイルス学virology150

The copyrights for Japanese and World History belongs to STEP Corporation. Commercial use other than for research and evaluation of language models is prohibited.

The copyrights for Japanese idioms, Japansese civics, and Japanese geography belong to New Style Cram School VIST. Commercial use is allowed only for research and evaluation of language models.

This work is licensed under CC BY-NC-ND 4.0

Acknowledgment

We express our gratitude to the RIKEN for their support in the translation of MMLU. We also acknowledge the contributions from Step Corporation, who provided materials on Japanese and World History, and from New Style Cram School VIST, who supplied resources on japanese_idioms, japansese_civics, and japanese_geography.

Dự án tương tự

Khám phá các dự án và mô hình AI khác có sự tương đồng.

Dataset
fka/prompts.chat

a.k.a. Awesome ChatGPT Prompts This is a Dataset Repository mirror of prompts.chat — a social platform for AI prompts. 📢 Notice This Hugging Face dataset is a mirror. For the latest prompts, features, and community contributions, please visit: 🌐 Website: prompts.chat 📦 GitHub: github.com/f/awesome-chatgpt-prompts About prompts.chat is an open-source platform where users can share, discover, and collect AI prompts from the community. The project can… See the full description on the dataset page: https://huggingface.co/datasets/fka/prompts.chat.

♥0
27.4k
Oct 26, 2026
Dataset
openai/MMMLU

Multilingual Massive Multitask Language Understanding (MMMLU) The MMLU is a widely recognized benchmark of general knowledge attained by AI models. It covers a broad range of topics from 57 different categories, covering elementary-level knowledge up to advanced professional subjects like law, physics, history, and computer science. We translated the MMLU’s test set into 14 languages using professional human translators. Relying on human translators for this evaluation increases… See the full description on the dataset page: https://huggingface.co/datasets/openai/MMMLU.

♥0
14.5k
Oct 16, 2024
Modelsentence-similarity
sentence-transformers/all-MiniLM-L6-v2

all-MiniLM-L6-v2 This is a sentence-transformers(https://www.SBERT.net) model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search. Usage (Sentence-Transformers) Using t...

♥0
190.2M
Jun 1, 2026
RepositoryTypeScript
firecrawl/firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

195.6k
Oct 26, 2026
RepositoryPython
Significant-Gravitas/AutoGPT

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

187.7k
Oct 24, 2026
RepositoryGo
ollama/ollama

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

184.9k
Oct 26, 2026
Dataset
cais/mmlu

Dataset Card for MMLU Dataset Summary Measuring Massive Multitask Language Understanding by Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt (ICLR 2021). This is a massive multitask test consisting of multiple-choice questions from various branches of knowledge. The test spans subjects in the humanities, social sciences, hard sciences, and other areas that are important for some people to learn. This covers 57… See the full description on the dataset page: https://huggingface.co/datasets/cais/mmlu.

♥0
546.5k
Mar 8, 2024
Dataset
TIGER-Lab/MMLU-Pro

MMLU-Pro Dataset MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines. |Github | 🏆Leaderboard | 📖Paper | 🚀 What's New [2026.03.11] Added more cutting-edge frontier models to the leaderboard, including the Claude-4.6 series, Seed2.0 series, Qwen3.5 series, and Gemini-3.1-Pro… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro.

♥0
300.9k
May 2, 2026
Dataset
Idavidrein/gpqa

Dataset Card for GPQA GPQA is a multiple-choice, Q&A dataset of very hard questions written and validated by experts in biology, physics, and chemistry. When attempting questions out of their own domain (e.g., a physicist answers a chemistry question), these experts get only 34% accuracy, despite spending >30m with full access to Google. We request that you do not reveal examples from this dataset in plain text or images online, to reduce the risk of leakage into foundation… See the full description on the dataset page: https://huggingface.co/datasets/Idavidrein/gpqa.

♥0
96.7k
Sep 21, 2026