Exercise 3a: Regular Expressions
In exercises 3a to 3e, more functionalities of queue command will be demonstrated. Sometimes it is easier to group a set of jobs and execute them as an array than execute them one by one. HTCondor provides the ability to match executables files using regular expressions and add a job to the queue for each file. In this exercise, all the executables which have the ending .sh will be queued. These executables are located in the current working directory in the submit machine. Create three simple executables based on the following code, replacing # with the numbers 1, 2 and 3 respectively.
#!/bin/bash echo "Hello i am the file number #"
Submit the executables using the following submit description file:
executable = $(filename) arguments = $(ClusterId)$(ProcId) output = output/$(ClusterId).$(ProcId).out error = error/$(ClusterId).$(ProcId).err log = log/$(ClusterId).log queue filename matching files *.sh
Note: In the submit description file, $(filename) is interpreted to be the file matching the regular expression.
The three jobs, even if they have different executables, have the same Cluster(21728) and different ProcIds(21728.0, 21728.1, 21728.2). By executing condor_q -nobatch, the same ClusterId and different ProcIds can be seen.
-- Schedd: bigbird04.cern.ch : <184.108.40.206:9618?... @ 03/28/17 10:24:36 ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD 21728.0 fprotops 3/28 10:24 0+00:00:00 I 0 0.0 file01.sh file01.sh 21728.1 fprotops 3/28 10:24 0+00:00:00 I 0 0.0 file02.sh file02.sh 21728.2 fprotops 3/28 10:24 0+00:00:00 I 0 0.0 file03.sh file03.sh 3 jobs; 0 completed, 0 removed, 3 idle, 0 running, 0 held, 0 suspended