We’re excited to release Citadel AI’s new Model Cards and Data Cards in Citadel Lens, designed to help organizations maintain accurate and up-to-date records of their models and datasets. These new, qualitative reports expand the comprehensive library of quantitative reports in Lens.
Model Cards and Data Cards are not new concepts, and have been explored in landmark research at Google and Microsoft. While these papers helped to popularize documentation for open-source systems, many internal models and datasets within organizations remain underdocumented due to a lack of streamlined tools and motivation.
As technical documentation is a key requirement of the EU AI Act and most AI governance frameworks such as ISO 42001 and Japan’s AI Guidelines for Business, we’ve seen many organizations start to adopt Model Cards and Data Cards in their inventory of AI projects.
Citadel Model Cards and Citadel Data Cards allow organizations to use Citadel Lens as a centralized hub of both qualitative and quantitative documentation about AI systems, and is the first of several new, qualitative reports on our roadmap.
Citadel Model Card
The Citadel Model Card is designed to be a single report containing key information about an AI model, such as its development process, deployment requirements, and guidance for downstream users.
To create a Model Card, simply connect the target model and datasets to Citadel Lens. Lens will then automatically:
- Evaluate the quantitative performance metrics of the model
- Generate quantitative metrics about the model’s training and test datasets (if provided)
- Record technical specifications of the model and datasets
- Provide a short, easy-to-use template to fill out the remaining documentation
Citadel Model Card and Data Card provide an essential foundation for documenting and sharing the progress and status of various models and datasets within our organization in a detailed and accurate manner.
This enables all stakeholders to have a centralized, easily manageable reporting environment, making the process from data entry to information sharing efficient and effective.
Yuki Hamaguchi, Software Engineer, Suntory
Citadel Data Card
The Citadel Data Card is designed to provide key information about a dataset throughout its entire lifecycle, including how it was sourced, the types of sensitive data it contains, and guidance on downstream usage for model development.
Datasets can vary significantly in terms of their modality, composition, and intended usage. Therefore, the Data Card format is designed to strike a balance between being general enough to cover most use cases and specific enough to capture concrete information for AI developers and users.
We’re expanding Citadel Lens to cover other components of your AI inventory, such as risk assessments and impact assessments. To work with our team of AI experts and see how Citadel Lens can help you document your AI models and datasets, please contact us at any time.
