Skip to content

JSON Format

DICOM Metadata

{
  "id": "String",
  "createdAt": "Date",
  "updatedAt": "Date",
  "name": "String",
  "description": "String",
  "isPrivate": "Boolean",
  "datasets": [
    {
      "id": "String", // starts with `D_`
      "type": "String", // e.g., DICOM
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "studies": [
        {
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "number": "Number" // corresponds to assigned exam number in project
        }
      ],
      "dicomMetadata": [
        {
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "SeriesInstanceUID": "String", // corresponds to DICOM series unique identifier
          "SOPInstanceUID": "String", // corresponds to DICOM instance unique identifier
          "dicomTags": "Record<String,String>" // Key-Value dicom tags, e.g., {"Modality":"CT"}
        }
      ],
      "videoMetadata": [
        {
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "SeriesInstanceUID": "String", // corresponds to DICOM series unique identifier
          "origFileName": "String", // corresponds to file name of uploaded video file
          "fileExtension": "String", // corresponds to extension of uploaded video file or .mp4 if fps changed while uploading.
        }
      ]
    }
  ]
}

Top Level

  • The top level consists of id, createdAt, updatedAt, name, description, isPrivate and datasets

Datasets

  • The datasets have an id, type, createdAt, updatedAt, name, description, studies, dicomMetadata and videoMetadata.
  • The videoMetadata attribute is included only when the dataset contains video files.

Studies

  • The studies have a StudyInstanceUID (corresponds to DICOM study unique identifier), number (corresponds to assigned exam number in project), and findings.
  • datasets -> studies

DicomMetadata

  • The dicomMetadata has a StudyInstanceUID, SeriesInstanceUID, SOPInstanceUID and dicomTags
  • datasets -> dicomMetadata

VideoMetadata

  • The videoMetadata has a StudyInstanceUID, SeriesInstanceUID, origFileName and fileExtension
  • datasets -> videoMetadata

Annotations

{
  "id": "String",
  "createdAt": "Date",
  "updatedAt": "Date",
  "name": "String",
  "description": "String",
  "isPrivate": "Boolean",
  "labelGroups": [
    {
      "id": "String", // starts with `G_`
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "type": "String", // STANDARD or METADATA
      "labels": [
        {
          "id": "String", // starts with `L_`
          "parentId": "String", // parent label id, if sublabel; starts with `L_`
          "createdAt": "Date",
          "updatedAt": "Date",
          "name": "String",
          "shortName": "String",
          "description": "String",
          "color": "String", // hexadecimal color code
          "type": "String", // GLOBAL or LOCAL
          "scope": "String", // STUDY, SERIES, or INSTANCE
          "annotationMode": "String", // e.g., bbox, freeform, polygon, etc.
          "radlexTagIds": ["String"] // array of RadLex ontology unique identifiers
        }
      ]
    }
  ],
  "datasets": [
    {
      "id": "String", // starts with `D_`
      "type": "String", // e.g., DICOM
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "studies": [
        {
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "number": "Number" // corresponds to assigned exam number in project
        }
      ],
      "annotations": [
        {
          "id": "String", // starts with `A_`
          "parentId": "String", // parent annotation id, if applied from a sublabel; starts with `A_`
          "isImported": "Boolean",
          "isInterpolated": "Boolean",
          "clonedFromModelOutputId": "String", // model output id, if annotation cloned from model output, starts with `O_`
          "createdAt": "Date",
          "createdById": "String", // user id; starts with `U_`
          "updatedAt": "Date",
          "updatedById": "String", // user id; starts with `U_`
          "updateHistory": [
            {
              "updatedAt": "String", // Date
              "updatedById": "String" // User hash id
            }
          ],
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "SeriesInstanceUID": "String", // corresponds to DICOM series unique identifier
          "SOPInstanceUID": "String", // corresponds to DICOM instance unique identifier
          "frameNumber": "Number", // frame number of a multiframe DICOM pixel_array or video
          "timestamp": "NUmber", // timestamp of the frame in a video (in seconds). Available only for videos
          "labelId": "String", // matches with label id above; starts with `L_`
          "annotationNumber": "Number", // tracking number for multiple annotations of a LOCAL label on an image (null for GLOBAL labels)
          "height": "Number", // image height (null for GLOBAL labels)
          "width": "Number", // image width (null for GLOBAL labels)
          "data": "Object",
          "note": "String",
          "radlexTagIds": ["String"], // array of RadLex ontology unique identifiers
          "reviews": [
            {
              "userId": "String", // user id; starts with `U_`
              "review": "String", // `POSITIVE` or `NEGATIVE`
              "createdAt": "Date",
              "updatedAt": "Date"
            }
          ],
          "reviewsPositiveCount": "Number", // number of positive reviews
          "reviewsNegativeCount": "Number", // number of negative reviews
          "groupId": "String" // annotation group id
        }
      ]
    }
  ]
}

Top Level

  • The top level consists of id, createdAt, updatedAt, name, description, isPrivate, labelGroups and datasets

Label Groups

  • The labelGroups have an id, createdAt, updatedAt, name, description, type and labels

Labels

  • The labels have an id, parentId, createdAt, updatedAt, name, shortName, description, color, type, scope, annotationMode, and radlexTagIds
  • labelGroups -> labels

Datasets

  • The datasets have an id, type, createdAt, updatedAt, name, description, studies and annotations.

Studies

  • The studies have a StudyInstanceUID (corresponds to DICOM study unique identifier), number (corresponds to assigned exam number in project), and findings.
  • datasets -> studies

Annotations

  • The annotations have an id, parentId, isImported, isInterpolated, clonedFromModelOutputId, createdAt, createById, updatedAt, updatedById, updateHistory, StudyInstanceUID, SeriesInstanceUID, SOPInstanceUID, frameNumber, timestamp (only for videos), labelId (corresponds to labels), annotationNumber, height (of image), width (of image), data, note, radlexTagIds, reviews, reviewsPositiveCounts , reviewsNegativeCounts, groupId
  • datasets -> annotations

Data depends on the annotationMode -

  1. Bounding box: x, y, width, height
  2. Freeform, polygon, line: vertices - [ [x,y] ]
  3. Mask: mask - [[0,...,0],[0,...,0], ...]
  4. Location: x, y point

datasets -> annotations -> data

Options:

  • annotationMode - bounding box, freeform, polygon, line, location
  • type - local or global
  • scope - for global labels, does the label apply to the image, series, or entire exam
  • isPrivate - projects can be public or private

Model Outputs

{
  "id": "String",
  "createdAt": "Date",
  "updatedAt": "Date",
  "name": "String",
  "description": "String",
  "isPrivate": "Boolean",
  "labelGroups": [
    {
      "id": "String", // starts with "G_"
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "type": "String", // STANDARD or METADATA
      "labels": [
        {
          "id": "String", // starts with `L_`
          "parentId": "String", // parent label id, if sublabel; starts with `L_`
          "createdAt": "Date",
          "updatedAt": "Date",
          "name": "String",
          "shortName": "String",
          "description": "String",
          "color": "String", // hexadecimal color code
          "type": "String", // GLOBAL or LOCAL
          "scope": "String", // STUDY, SERIES, or INSTANCE
          "annotationMode": "String", // e.g., bbox, freeform, polygon, etc.
          "radlexTagIds": ["String"] // array of RadLex ontology unique identifiers
        }
      ]
    }
  ],
  "models": [
    {
      "id": "String", // starts with `M_`
      "createdAt": "Date",
      "createdById": "String", // userID of the user who created the model
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "scope": "String", // model input scope - STUDY, SERIES, or INSTANCE
      "labelClasses": [
        {
          "classIndex": "Number", // number of the predicted class as output by model
          "labelId": "String" // labelID on MD.ai corresponding to the class
        }
      ],
      "versions": [
        {
          "id": "Number", // unique model ID number
          "isDefault": "Boolean",
          "versionNumber": "Number", // model version number
          "createdAt": "Date",
          "dockerImageName": "String", // e.g. "gcr.io/mdai-models/public/model-1"
          "dockerImageTag": "String", // e.g. v1, v2, etc.
          "config": {
            "cpuLimit": "String", // max cpu limit requested e.g. 1
            "cpuRequest": "String", // cpu requested e.g. 0.1
            "memoryLimit": "String", // max memory limit requested e.g. 10G
            "memoryRequest": "String", // memory requested e.g. 1G
            "acceleratorType": "String" // cpu or gpu requested e.g 'gpu_nvidia-tesla-t4'
          },
          "args": [
            {
              "key": "String", // additional key-value pairs passed via UI
              "default": "String"
            }
          ]
        }
      ],
      "tasks": [
        {
          "id": "Number",
          "modelVersionId": "Number", // model version number used for this task
          "type": "String", // Inference or Testing
          "config": {
            "resourceIds": ["Number"], // array of resourceId numbers
            "resourceType": "String", // DATASET, EXAM, SERIES or IMAGE
            "dicomDatasetId": "Number"
          },
          "args": [
            {
              "key": "String", // key-value pairs passed via UI
              "value": "String"
            }
          ],
          "createdAt": "Date",
          "status": "String", // PREPARING INPUTS, PROVISIONING RESOURCES, RUNNING, FAILED or SUCCEEDED
          "startTime": "Date",
          "endTime": "Date",
          "progress": "Number", // task's progress percentage 0-100
          "summary": {
            "errorCount": "Number", // number of resource types with errors
            "successCount": "Number" // number of successful resource types
          }
        }
      ]
    }
  ],
  "datasets": [
    {
      "id": "String", // starts with `D_`
      "type": "String", // e.g., DICOM
      "createdAt": "Date",
      "updatedAt": "Date",
      "name": "String",
      "description": "String",
      "studies": [
        {
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "number": "Number" // corresponds to assigned exam number in project
        }
      ],
      "modelOutputs": [
        {
          "id": "String", // model output id, start with `O_`
          "modelId": "Number",
          "modelVersionId": "Number", // matches with model versions above
          "modelTaskId": "Number", // matches with task ids above
          "type": "String", // e.g. ANNOTATION, NONE, etc.
          "createdAt": "Date",
          "labelId": "String", // matches with label id above; starts with `L_`
          "data": "Object",
          "probability": "Number",
          "StudyInstanceUID": "String", // corresponds to DICOM study unique identifier
          "SeriesInstanceUID": "String", // corresponds to DICOM series unique identifier
          "SOPInstanceUID": "String" // corresponds to DICOM instance unique identifier
        }
      ]
    }
  ]
}

Top Level

  • The top level consists of id, createdAt, updatedAt, name, description, isPrivate, labelGroups, models and datasets

Label Groups

  • The labelGroups have an id, createdAt, updatedAt, name, description, type and labels.

Labels

  • The labels have an id, parentId, createdAt, updatedAt, name, shortName, description, color, type, scope, annotationMode, and radlexTagIds
  • labelGroups -> labels

Models

  • The models have an id, createdAt, createdById, updatedAt, name, description, scope, labelClasses, versions and tasks

Label Classes

  • The labelClasses have a classIndex and labelId.
  • models -> labelClasses

Versions

  • The versions have an id, isDefault, versionNumber, createdAt, dockerImageName, dockerImageTag, config and args (key-value pair added via the UI)
  • models -> versions

Config (Versions)

  • The config has a cpuLimit, cpuRequest, memoryLimit, memoryRequest and acceleratorType
  • models -> versions -> config

Tasks

  • The tasks have an id, modelVersionId, type, config, args, createdAt, status, startTime, endTime, progress and summary
  • models -> tasks

Config (Tasks)

  • The config has a resourceIds, resourceType and dicomDatasetId
  • models -> tasks -> config

Summary

  • The summary has an errorCount and successsCount
  • models -> tasks -> summary

Datasets

  • The datasets have an id, type, createdAt, updatedAt, name, description, studies and modelOutputs.

Studies

  • The studies have a StudyInstanceUID (corresponds to DICOM study unique identifier), number (corresponds to assigned exam number in project), and findings.
  • datasets -> studies

Model Outputs

  • The modelOutputs have an id, modelId, modelVersionId, modelTaskId, type, createdAt, labelId, data, probability, StudyInstanceUID, SeriesInstanceUID and SOPInstanceUID
  • datasets -> modelOutputs

Data depends on the annotationMode -

  1. Bounding box: x, y, height, width.
  2. Freeform, polygon, line: vertices - [ [x,y] ]
  3. Mask: mask - [[0,...,0],[0,...,0], ...]
  4. Location: x, y point

datasets -> modelOutputs -> data

Models

The exported .zip model file with following structure:

model.zip/
  |__ source
      |__ v1.zip   # Uploaded model file when create model version, for versionNumber=1
      |__ v2.zip
      |__ v3.zip
  |__ model.json   # Model schema file

The model.json schema file with following structure:

{
 "model": {
    "id": "String",
    "name": "String",
    "description": "String",
    "scope": "String",
    "autoRunInference": "Boolean",
    "labelClasses": [{
      "classIndex": "Number",
      "labelId": "String"
    }],
    "versions": [{
      "id": "Number",
      "versionNumber": "Number",
      "dockerImageName": "String",
      "dockerImageTag": "String",
      "config": {
        "cpuLimit": "String",
        "cpuRequest": "String",
        "memoryLimit": "String",
        "memoryRequest": "String",
        "acceleratorType": "String",
        "gpuCount": "Number",
        "batchSize": "Number",
        "fileType": "String",
      },
      "isDefault": "Boolean",
      "args": [{
        "key": "String",
        "default": "String"
      }],
      "status": "String",
      "source": "String",   // Uploaded file name in 'source' folder, for example, 'v1.zip', if 'null', means no upload file
    }],
  },
  "labels": [
    {
      "id": "String",
      "parentId": "String",
      "name": "String",
      "shortName": "String",
      "description": "String",
      "color": "String",
      "type": "String",
      "scope": "String",
      "annotationMode": "String",
      "radlexTagIds": ["String"]
    }
  ]
}

Top Level

  • The top level consists of model and labels.

Model

  • The model has an id, name, description, scope, autoRunInference, labelClasses and versions

Label Classes

  • The labelClasses have a classIndex and labelId.
  • model -> labelClasses

Versions

  • The versions have an id, isDefault, versionNumber, dockerImageName, dockerImageTag, config, isDefault, args (key-value pair added via the UI), status and source.
  • model -> versions

Config

  • The config has a cpuLimit, cpuRequest, memoryLimit, memoryRequest, acceleratorType, gpuCount, batchSize and fileType
  • model -> versions -> config

Labels

  • The labels have an id, parentId, name, shortName, description, color, type, scope, annotationMode, and radlexTagIds