Skip to content

JSON Format

DICOM Metadata

{
  "id": "String",
  "createdAt": "Date",
  "updatedAt": "Date",
  "name": "String",
  "description": "String",
  "isPrivate": "Boolean",
  "datasets": [
    {
      "id": "String", // starts with `D_`
      "type": "String", // e.g., DICOM
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "studies": [
        {
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "number": "Number" // corresponds to assigned exam number in project
        }
      ],
      "dicomMetadata": [
        {
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "SeriesInstanceUID": "String", // corresponds to DICOM series unique identifier
          "SOPInstanceUID": "String", // corresponds to DICOM instance unique identifier
          "dicomTags": "Record<String,String>" // Key-Value dicom tags, e.g., {"Modality":"CT"}
        }
      ]
    }
  ]
}

Top Level

  • The top level consists of id, createdAt, updatedAt, name, description, isPrivate and datasets

Datasets

  • The datasets have an id, type, createdAt, updatedAt, name, description, studies and dicomMetadata.

Studies

  • The studies have a StudyInstanceUID (corresponds to DICOM study unique identifier), number (corresponds to assigned exam number in project), and findings.
  • datasets -> studies

DicomMetadata

  • The dicomMetadata has a StudyInstanceUID, SeriesInstanceUID, SOPInstanceUID and dicomTags
  • datasets -> dicomMetadata

Annotations

{
  "id": "String",
  "createdAt": "Date",
  "updatedAt": "Date",
  "name": "String",
  "description": "String",
  "isPrivate": "Boolean",
  "labelGroups": [
    {
      "id": "String", // starts with `G_`
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "type": "String", // STANDARD or METADATA
      "labels": [
        {
          "id": "String", // starts with `L_`
          "parentId": "String", // parent label id, if sublabel; starts with `L_`
          "createdAt": "Date",
          "updatedAt": "Date",
          "name": "String",
          "shortName": "String",
          "description": "String",
          "color": "String", // hexadecimal color code
          "type": "String", // GLOBAL or LOCAL
          "scope": "String", // STUDY, SERIES, or INSTANCE
          "annotationMode": "String", // e.g., bbox, freeform, polygon, etc.
          "radlexTagIds": ["String"] // array of RadLex ontology unique identifiers
        }
      ]
    }
  ],
  "datasets": [
    {
      "id": "String", // starts with `D_`
      "type": "String", // e.g., DICOM
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "studies": [
        {
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "number": "Number" // corresponds to assigned exam number in project
        }
      ],
      "annotations": [
        {
          "id": "String", // starts with `A_`
          "parentId": "String", // parent annotation id, if applied from a sublabel; starts with `A_`
          "isImported": "Boolean",
          "isInterpolated": "Boolean",
          "clonedFromModelOutputId": "String", // model output id, if annotation cloned from model output, starts with `O_`
          "createdAt": "Date",
          "createdById": "String", // user id; starts with `U_`
          "updatedAt": "Date",
          "updatedById": "String", // user id; starts with `U_`
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "SeriesInstanceUID": "String", // corresponds to DICOM series unique identifier
          "SOPInstanceUID": "String", // corresponds to DICOM instance unique identifier
          "frameNumber": "Number", // frame number of a multiframe DICOM pixel_array
          "labelId": "String", // matches with label id above; starts with `L_`
          "annotationNumber": "Number", // tracking number for multiple annotations of a label on a single image
          "height": "Number", // image height
          "width": "Number", // image width
          "data": "Object",
          "note": "String",
          "radlexTagIds": ["String"], // array of RadLex ontology unique identifiers
          "reviewsPositiveCount": "Number", // number of positive reviews
          "reviewsNegativeCount": "Number" // number of negative reviews
        }
      ]
    }
  ]
}

Top Level

  • The top level consists of id, createdAt, updatedAt, name, description, isPrivate, labelGroups and datasets

Label Groups

  • The labelGroups have an id, createdAt, updatedAt, name, description, type and labels

Labels

  • The labels have an id, parentId, createdAt, updatedAt, name, shortName, description, color, type, scope, annotationMode, and radlexTagIds
  • labelGroups -> labels

Datasets

  • The datasets have an id, type, createdAt, updatedAt, name, description, studies and annotations.

Studies

  • The studies have a StudyInstanceUID (corresponds to DICOM study unique identifier), number (corresponds to assigned exam number in project), and findings.
  • datasets -> studies

Annotations

  • The annotations have an id, parentId, isImported, createdAt, createById, updatedAt, updatedById, StudyInstanceUID, SeriesInstanceUID, SOPInstanceUID, frameNumber, labelId (corresponds to labels), annotationNumber, height (of image), width (of image), data, note, radlexTagIds, reviewsPositiveCounts , reviewsNegativeCounts
  • datasets -> annotations

Data depends on the annotationMode -

  1. Bounding box: x, y, height, width.
  2. Freeform, polygon, line: vertices - [ [x,y] ]
  3. Mask: mask - [[0,...,0],[0,...,0], ...]
  4. Location: x, y point

datasets -> annotations -> data

Options:

  • annotationMode - bounding box, freeform, polygon, line, location
  • type - local or global
  • scope - for global labels, does the label apply to the image, series, or entire exam
  • isPrivate - projects can be public or private

Model Outputs

{
  "id": "String",
  "createdAt": "Date",
  "updatedAt": "Date",
  "name": "String",
  "description": "String",
  "isPrivate": "Boolean",
  "labelGroups": [
    {
      "id": "String", // starts with "G_"
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "type": "String", // STANDARD or METADATA
      "labels": [
        {
          "id": "String", // starts with `L_`
          "parentId": "String", // parent label id, if sublabel; starts with `L_`
          "createdAt": "Date",
          "updatedAt": "Date",
          "name": "String",
          "shortName": "String",
          "description": "String",
          "color": "String", // hexadecimal color code
          "type": "String", // GLOBAL or LOCAL
          "scope": "String", // STUDY, SERIES, or INSTANCE
          "annotationMode": "String", // e.g., bbox, freeform, polygon, etc.
          "radlexTagIds": ["String"] // array of RadLex ontology unique identifiers
        }
      ]
    }
  ],
  "models": [
    {
      "id": "String", // starts with `M_`
      "createdAt": "Date",
      "createdById": "String", // userID of the user who created the model
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "scope": "String", // model input scope - STUDY, SERIES, or INSTANCE
      "labelClasses": [
        {
          "classIndex": "Number", // number of the predicted class as output by model
          "labelId": "String" // labelID on MD.ai corresponding to the class
        }
      ],
      "versions": [
        {
          "id": "Number", // unique model ID number
          "isDefault": "Boolean",
          "versionNumber": "Number", // model version number
          "createdAt": "Date",
          "dockerImageName": "String", // e.g. "gcr.io/mdai-models/public/model-1"
          "dockerImageTag": "String", // e.g. v1, v2, etc.
          "config": {
            "cpuLimit": "String", // max cpu limit requested e.g. 1
            "cpuRequest": "String", // cpu requested e.g. 0.1
            "memoryLimit": "String", // max memory limit requested e.g. 10G
            "memoryRequest": "String", // memory requested e.g. 1G
            "acceleratorType": "String" // cpu or gpu requested e.g 'gpu_nvidia-tesla-t4'
          },
          "args": [
            {
              "key": "String", // additional key-value pairs passed via UI
              "default": "String"
            }
          ]
        }
      ],
      "tasks": [
        {
          "id": "Number",
          "modelVersionId": "Number", // model version number used for this task
          "type": "String", // Inference or Testing
          "config": {
            "resourceIds": ["Number"], // array of resourceId numbers
            "resourceType": "String", // DATASET, EXAM, SERIES or IMAGE
            "dicomDatasetId": "Number"
          },
          "args": [
            {
              "key": "String", // key-value pairs passed via UI
              "value": "String"
            }
          ],
          "createdAt": "Date",
          "status": "String", // PREPARING INPUTS, PROVISIONING RESOURCES, RUNNING, FAILED or SUCCEEDED
          "startTime": "Date",
          "endTime": "Date",
          "progress": "Number", // task's progress percentage 0-100
          "summary": {
            "errorCount": "Number", // number of resource types with errors
            "successCount": "Number" // number of successful resource types
          }
        }
      ]
    }
  ],
  "datasets": [
    {
      "id": "String", // starts with `D_`
      "type": "String", // e.g., DICOM
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "studies": [
        {
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "number": "Number" // corresponds to assigned exam number in project
        }
      ],
      "modelOutputs": [
        {
          "id": "String", // model output id, start with `O_`
          "modelId": "Number",
          "modelVersionId": "Number", // matches with model versions above
          "modelTaskId": "Number", // matches with task ids above
          "type": "String", // e.g. ANNOTATION, NONE, etc.
          "createdAt": "Date",
          "labelId": "String", // matches with label id above; starts with `L_`
          "data": "Object",
          "probability": "Number",
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "SeriesInstanceUID": "String", // corresponds to DICOM series unique identifier
          "SOPInstanceUID": "String" // corresponds to DICOM instance unique identifier
        }
      ]
    }
  ]
}

Top Level

  • The top level consists of id, createdAt, updatedAt, name, description, isPrivate, labelGroups, models and datasets

Label Groups

  • The labelGroups have an id, createdAt, updatedAt, name, description, type and labels.

Labels

  • The labels have an id, parentId, createdAt, updatedAt, name, shortName, description, color, type, scope, annotationMode, and radlexTagIds
  • labelGroups -> labels

Models

  • The models have an id, createdAt, createdById, updatedAt, name, description, scope, labelClasses, versions and tasks

Label Classes

  • The labelClasses have a classIndex and labelId.
  • models -> labelClasses

Versions

  • The versions have an id, isDefault, versionNumber, createdAt, dockerImageName, dockerImageTag, config and args (key-value pair added via the UI)
  • models -> versions

Config (Versions)

  • The config has a cpuLimit, cpuRequest, memoryLimit, memoryRequest and acceleratorType
  • models -> versions -> config

Tasks

  • The tasks have an id, modelVersionId, type, config, args, createdAt, status, startTime, endTime, progress and summary
  • models -> tasks

Config (Tasks)

  • The config has a resourceIds, resourceType and dicomDatasetId
  • models -> tasks -> config

Summary

  • The summary has an errorCount and successsCount
  • models -> tasks -> summary

Datasets

  • The datasets have an id, type, createdAt, updatedAt, name, description, studies and modelOutputs.

Studies

  • The studies have a StudyInstanceUID (corresponds to DICOM study unique identifier), number (corresponds to assigned exam number in project), and findings.
  • datasets -> studies

Model Outputs

  • The modelOutputs have an id, modelId, modelVersionId, modelTaskId, type, createdAt, labelId, data, probability, StudyInstanceUID, SeriesInstanceUID and SOPInstanceUID
  • datasets -> modelOutputs

Data depends on the annotationMode -

  1. Bounding box: x, y, height, width.
  2. Freeform, polygon, line: vertices - [ [x,y] ]
  3. Mask: mask - [[0,...,0],[0,...,0], ...]
  4. Location: x, y point

datasets -> modelOutputs -> data

Models

The exported .zip model file with following structure:

model.zip/
  |__ source
      |__ v1.zip   # Uploaded model file when create model version, for versionNumber=1
      |__ v2.zip
      |__ v3.zip
  |__ model.json   # Model schema file

The model.json schema file with following structure:

{
 "model": {
    "id": "String",
    "name": "String",
    "description": "String",
    "scope": "String",
    "autoRunInference": "Boolean",
    "labelClasses": [{
      "classIndex": "Number",
      "labelId": "String"
    }],
    "versions": [{
      "id": "Number",
      "versionNumber": "Number",
      "dockerImageName": "String",
      "dockerImageTag": "String",
      "config": {
        "cpuLimit": "String",
        "cpuRequest": "String",
        "memoryLimit": "String",
        "memoryRequest": "String",
        "acceleratorType": "String",
        "gpuCount": "Number",
        "batchSize": "Number",
        "fileType": "String",
      },
      "isDefault": "Boolean",
      "args": [{
        "key": "String",
        "default": "String"
      }],
      "status": "String",
      "source": "String",   // Uploaded file name in 'source' folder, for example, 'v1.zip', if 'null', means no upload file
    }],
  },
  "labels": [
    {
      "id": "String",
      "parentId": "String",
      "name": "String",
      "shortName": "String",
      "description": "String",
      "color": "String",
      "type": "String",
      "scope": "String",
      "annotationMode": "String",
      "radlexTagIds": ["String"]
    }
  ]
}

Top Level

  • The top level consists of model and labels.

Model

  • The model has an id, name, description, scope, autoRunInference, labelClasses and versions

Label Classes

  • The labelClasses have a classIndex and labelId.
  • model -> labelClasses

Versions

  • The versions have an id, isDefault, versionNumber, dockerImageName, dockerImageTag, config, isDefault, args (key-value pair added via the UI), status and source.
  • model -> versions

Config

  • The config has a cpuLimit, cpuRequest, memoryLimit, memoryRequest, acceleratorType, gpuCount, batchSize and fileType
  • model -> versions -> config

Labels

  • The labels have an id, parentId, name, shortName, description, color, type, scope, annotationMode, and radlexTagIds