To view our extensive collection of tutorials, how-to's, explanation guides, and reference material, please visit our documentation on Read the Docs
DataEval analyzes datasets and models to give users the ability to train and test performant, unbiased, and reliable AI models and monitor data for impactful shifts to deployed models.
DataEval is an effective, powerful, and reliable set of tools for any T&E engineer. Throughout all stages of the machine learning lifecycle, DataEval supports model development, data analysis, and monitoring with state-of-the-art algorithms to help you solve difficult problems. With a focus on computer vision tasks, DataEval provides simple, but effective metrics for performance estimation, bias detection, and dataset linting.
DataEval is easy to install, supports a wide range of Python versions, and is compatible with many of the most popular packages in the scientific and T&E communities.
DataEval also has native interoperability between JATIC's suite of tools when using MAITE-compliant datasets and models.
Python versions: 3.9 - 3.12
Supported packages: NumPy, Pandas, Sci-kit learn, MAITE, NRTK
Choose your preferred method of installation below or follow our installation guide.
You can install DataEval directly from pypi.org using the following command.
The optional dependencies of DataEval are all.
pip install dataeval[all]DataEval can be installed in a Conda/Mamba environment using the provided
environment.yaml file. As some dependencies are installed from the pytorch
channel, the channel is specified in the below example.
micromamba create -f environment\environment.yaml -c pytorchTo install DataEval from source locally on Ubuntu, pull the source down and change to the DataEval project directory.
git clone https://github.com/aria-ml/dataeval.git
cd dataevalInstall DataEval with all extras.
poetry install --extras=allEnable Poetry's virtual environment.
poetry env activateInstall DataEval with all extras and dependencies for development.
uv sync --extra=allEnable uv's virtual environment.
source .venv/bin/activateIf you have any questions, feel free to reach out to us!
This material is based upon work supported by the Chief Digital and Artificial Intelligence Office under Contract No. W519TC-23-9-2033. The views and conclusions contained herein are those of the author(s) and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Government.