RIKEN and Citadel AI Develop a Japanese Toxicity Dataset for LLMs

The Language Information Access Technology Team at RIKEN’s Center for Advanced Intelligence Project led by Satoshi Sekine, in collaboration with Citadel AI Inc. (HQ: Shibuya, Tokyo; CEO: Hironori “Rick” Kobayashi; hereinafter referred to as “Citadel AI”), and the LLM Study Group founded by the National Institute of Informatics, has developed the AnswerCarefully dataset for evaluating Japanese large language models (LLMs). This dataset aims to create safer and more reliable LLMs, and was released for both research and commercial use on April 30, 2024.

Challenges of Japanese LLMs

Toxicity is one of the core challenges in developing and utilizing LLMs. Toxic text may include discriminatory language, extreme opinions, or inappropriate content. If such toxic text is used as training data, the model may generate inappropriate or toxic outputs. Additionally, toxic input prompts further increase the risk of inappropriate model behavior. Thus, the selection and quality control of datasets are crucial in developing LLMs.

Another challenge is the shortage of Japanese training data compared to languages like English, as foundation models such as GPT-4 and Gemini are primarily developed outside of Japan. To improve the safety and reliability of LLMs, it is essential to construct datasets that provide appropriate responses to toxic content in Japanese and train the models with high quality Japanese datasets.

The AnswerCarefully Toxicity Dataset

The AnswerCarefully dataset, developed by RIKEN in collaboration with the LLM Study Group and Citadel AI, aims to address these challenges. This dataset contains human-written examples of toxic and harmful content in Japanese, along with appropriate responses expected from LLMs. It can be used for training and evaluating LLMs, enabling them to respond appropriately to real-world situations and provide safer and fairer services to people and society. 

By releasing AnswerCarefully as an open dataset, LLM developers may utilize it for both research and commercial purposes, and we aim to contribute these advancements widely to society.

For more details on AnswerCarefully, please visit the AnswerCarefully website.

About Citadel AI

Citadel AI provides software products that test and monitor the quality of AI systems. Our technology helps organizations minimize AI reliability risks and maximize AI performance from research to deployment. Citadel AI’s technology is built from our team’s first-hand experience deploying high-risk AI systems at world-leading companies such as Google, Waymo, Toyota, and more. 

About RIKEN