How to participate ?

Registered participants must provide one of the two following documents:

  • A technical report detailing their method provided along with the docker container
  • A submission to the REMIA workshop of a paper detailing their report

Participants with submitted articles to the REMIA workshop and technical reports will be guaranteed to be cited in the future article summarizing the ATLAS challenge.

Using Docker containers

To avoid driver incompatibilities and memory limitations, all submitted Dockers must be able to perform inference on the CPU using a maximum memory of 32 GB. To prevent cheating, we will disable the Internet connection during inference on the test images. An example of a Dockerfile that can be used to build images with Python is available here (Dockerfile). In addition to installing Python, it creates a Python virtual environment and installs some Python packages (see requirements.txt). Then, it defines a working directory and copies some files (check_pytorch.py, my_network_infer.py, unet.py, my_network.pt) from the host (your machine) into the image. When the image is executed a bash shell is run. The directory structure should look like the following:

Get the Atlas Docker container

The Atlas Docker container is organized as follows:

  • atlas-docker
  • |____Dockerfile
  • |____requirements.txt
  • |____check_pytorch.py
  • |____unet.py
  • |____my_network_infer.py
  • |____my_network.pt
  • |____my_network_train.py
  • |____MRIImages
    • |____ im0.nii.gz
Only Registered participants can download the docker image.

Check that everything goes well

Unzip the archive and, using a terminal, go to the atlas-docker folder. Then run the following command to build the image:
docker build -t name_of_your_image .

To see your newly built image, run
docker images
You should see an image named name_of_your_image.

To create and run a new container from the newly built image, use the following command
docker run -it name_of_your_image

You should get inside the container in the /app folder. To check that Pytorch is installed correctly, you can run inside the container in the /app folder
python3 check_pytorch.py

Infer on the CPU using the Docker image

After creating and training a network (my_network_train.py) on GPU/CPU/TPU etc., save your model to a file (my_network.pt). Then, create a CPU inference script such as my_network_infer.py. Note that the files my_network.pt and my_network_infer.py have already been copied (see Dockerfile).

To be able to execute the inference script on an MRI image hosted on the local machine (your computer), you need to mount the folder containing the MRI images into the container. To do this, use the -v option when using docker run
docker run -v absolute_host_machine_path_where_are_the_MRI_images:absolute_path_inside_the_container -it name_of_your_image

For example,
docker run: docker run -v absolute_host_machine_path_where_are_the_MRI_images:/app/MRIImages -it name_of_your_image
As before, you should get inside the container in the /app folder.

To run your inference script on an image, you can do
python3 my_network_infer.py -i MRIImages/name_of_the_MRI_image.nii.gz -o MRIImages/output_network_image.nii.gz -s my_network.pt

It is the participant’s responsibility to create a development environment and other prerequisites besides GPU drivers, such as Conda, PyTorch, CUDA, CUDNN, and other tools.

Upload your docker

Participants are invited to submit their contributions by uploading a Docker archive. This archive should contain a Docker file that builds the image and a script that allows us to run the model on the test images.

Each participant is authorized to submit three times their models. The best performance over the two submissions will be retained.

Only Registered participants can upload a contribution.

Upload your technical report

Participants are invited to upload a technical report detailing their method provided along with the docker container if they do not submit a paper to to the REMIA workshop.

Metrics

Evaluation of the algorithms will be done on the test set using the provided docker container by the participants. Five different metrics will be used:

  • The average dice per structure
  • The 5mm surface Dice
  • The Average symmetric surface distance
  • The Hausdorff distance
  • The Root Mean Square Error on tumor burden calculation

Every metric except for the tumor burden will be calculated twice, once for the whole liver (healthy liver + unhealthy liver) and once for the tumor. The code that allows metric calculations is provided with the dataset.

Rank-then-aggregate voting system over the nine scores obtained by the participants will be used to compute the ranking.