Skip to content

User Personas

CheckMAITE's Python API supports several personas with varying skill levels. The personas below represent the core users we support.

Data Scientist

Data Scientist: who they are

  • Researchers / analytics / data scientists working with dataset and models
  • Most comfortable using Jupyter Notebooks, but can read/write some Python code as needed
  • Focused on analysis and experimentation, not infrastructure

Data Scientist: key workflows

  • Complete T&E workflows using JATIC tools via CheckMAITE on models and datasets
  • Explore and analyze data using notebooks
  • Use cloud resources when local compute isn't enough
  • Needs to compute a standard set of analyses in a structured manner

Data Scientist: pain points

  • Struggles to understand all the details of the wide variety of JATIC tools, wants a low barrier to entry with sane defaults
  • No reproducibility or traceability on completed analyses
  • Limited by computational power of local machine
  • Collaboration with team members is difficult across local environments and operating systems
  • Unclear what models and datasets are available to them

Data Scientist: what they need

  • Low barrier to entry for new tooling
  • Jupyter Notebook workflows with clear, reusable Python examples
  • Reproducibility of workflows
  • Ability to search previously executed workflows
  • High level guidance on JATIC tool usage through reasonable defaults and documentation
  • Ability to run workflows on external resources (e.g. cloud)
  • Ability to discovery models and datasets available

ML Engineer

ML Engineer: who they are

  • ML Engineers / Software Engineers
  • Most comfortable working with code via IDEs and CLIs
  • Conducts T&E analyses
  • Assists infrastructure engineers with day-to-day platform maintenance regarding user-facing tooling

ML Engineer: key workflows

  • Analyses datasets and models using Python
  • Conducts their own T&E analyses using JATIC tools via CheckMAITE
  • Enables Data Scientists by writing code to enable new workflows and simplify tool interactions
  • Maintains model registry
  • Maintains model served on platform

ML Engineer: pain points

  • Occassionally needs high compute resources
  • Collaboration with team members is difficult across local environments and operating systems

ML Engineer: what they need

  • Ability to run workflows on external resources (e.g. cloud)
  • Ability to discovery models and datasets available
  • Deep understanding of JATIC tools in order to build interfaces for data scientists

T&E workflow

A typical CheckMAITE project curates a new dataset, then uses it to pick the most suitable of the available models. Each tutorial covers one stage of that workflow.

1. Bring your model and dataset

Led by: ML Engineer

Wrap the model and dataset so CheckMAITE can run them. Every later stage depends on this step. Tutorial: ONNX object detection wrapper.

2. Curate the dataset

Led by: Data Scientist

Check that the dataset is fit to evaluate models against before trusting any model result on it. Remove low-quality data, look for biases and correlations, compare it with operational data, and check that the task is achievable. Tutorials: Cleaning, Bias, Shift, Feasibility.

3. Evaluate the model

Led by: Data Scientist

Measure each candidate model's baseline performance, how it holds up under realistic perturbations, and what drives its predictions, so the team can choose which model to fine-tune. Tutorials: MAITE Evaluation, NRTK, XAITK.

4. Record and compare results

Led by: Data Scientist

Store results from every stage so they can be compared across runs, datasets, and models without re-running anything. Tutorial: Analytics Store.

5. Run at scale

Led by: ML Engineer, with Data Scientists using it when local compute isn't enough

Move stages 2–4 off the local machine when they need more compute or have to be shared, and serve models remotely. Tutorials: Ray Simple Job Submission, Ray Job Submission, Remote Inference with Ray Serve.