Exercise 9a: Docker Universe
To use Docker containers within HTCondor jobs you need to specify the docker universe, the image you will be using and set "should_transfer_files" to YES in order to copy files to and from scratch space.
Please note that due to fair-use limits applied by Docker Hub, you should prefix Docker Hub image URLs (docker.io/...) with the CERN pull-through cache URL (registry.cern.ch/docker.io/...). The same applies for other registries (e.g. ghcr.io/... -> registry.cern.ch/ghcr.io/...).
Following the example from the user manual, create exercise9a.sub as shown below:
universe = docker
docker_image = registry.cern.ch/docker.io/ubuntu
executable = /bin/cat
arguments = /etc/hosts
should_transfer_files = YES
when_to_transfer_output = ON_EXIT
output = out.$(ClusterId).$(ProcId)
error = err.$(ClusterId).$(ProcId)
log = log.$(ClusterId).$(ProcId)
request_memory = 100M
+Requirements = OpSysAndVer =?= "CentOS7"
queue 1
Execute condor_submit exercise9a.sub to submit the job.
To run local applications or scripts for example test.sh within the container, specify the input files with "transfer_input_files" and as shown below:
universe = docker
docker_image = registry.cern.ch/docker.io/ubuntu
executable = ./test.sh
transfer_input_files = test.sh
should_transfer_files = YES
when_to_transfer_output = ON_EXIT
output = out.$(ClusterId).$(ProcId)
error = err.$(ClusterId).$(ProcId)
log = log.$(ClusterId).$(ProcId)
request_memory = 100M
+Requirements = OpSysAndVer =?= "CentOS7"
queue 1
The following job submission example further shows how to execute a ROOT macro in batch mode, running a benchmark example available within the container image:
universe = docker
docker_image = registry.cern.ch/docker.io/rootproject/root-ubuntu16
executable = /usr/local/bin/root.exe
arguments = -b -q /usr/local/share/doc/root/tutorials/benchmarks.C
should_transfer_files = YES
when_to_transfer_output = ON_EXIT
output = out.$(ClusterId).$(ProcId)
error = err.$(ClusterId).$(ProcId)
log = log.$(ClusterId).$(ProcId)
request_memory = 2000M
+Requirements = OpSysAndVer =?= "CentOS7"
queue 1
Note
HTCondor Docker Universe only supports pulling images from public repositories.