Exercise 9a: Docker Universe

To use Docker containers within HTCondor jobs you need to specify the docker universe, the image you will be using and set "should_transfer_files" to YES in order to copy files to and from scratch space. Following the example from the user manual, create exercise9a.sub as shown below:

universe                = docker
docker_image            = docker.io/ubuntu
executable              = /bin/cat
arguments               = /etc/hosts
should_transfer_files   = YES
when_to_transfer_output = ON_EXIT
output                  = out.$(ClusterId).$(ProcId)
error                   = err.$(ClusterId).$(ProcId)
log                     = log.$(ClusterId).$(ProcId)
request_memory          = 100M
+Requirements           = OpSysAndVer =?= "CentOS7"
queue 1

Execute condor_submit exercise9a.sub to submit the job.

To run local applications or scripts for example test.sh within the container, specify the input files with "transfer_input_files" and as shown below:

universe                = docker
docker_image            = docker.io/ubuntu
executable              = ./test.sh
transfer_input_files    = test.sh
should_transfer_files   = YES
when_to_transfer_output = ON_EXIT
output                  = out.$(ClusterId).$(ProcId)
error                   = err.$(ClusterId).$(ProcId)
log                     = log.$(ClusterId).$(ProcId)
request_memory          = 100M
+Requirements           = OpSysAndVer =?= "CentOS7"
queue 1

The following job submission example further shows how to execute a ROOT macro in batch mode, running a benchmark example available within the container image:

universe                = docker
docker_image            = docker.io/rootproject/root-ubuntu16
executable              = /usr/local/bin/root.exe
arguments               = -b -q /usr/local/share/doc/root/tutorials/benchmarks.C
should_transfer_files   = YES
when_to_transfer_output = ON_EXIT
output                  = out.$(ClusterId).$(ProcId)
error                   = err.$(ClusterId).$(ProcId)
log                     = log.$(ClusterId).$(ProcId)
request_memory          = 2000M
+Requirements           = OpSysAndVer =?= "CentOS7"
queue 1

Note

HTCondor Docker Universe only supports pulling images from public repositories.

---

Last update: March 20, 2024