Exercise 3a: Regular Expressions

In exercises 3a to 3e, more functionalities of queue command will be demonstrated. Sometimes it is easier to group a set of jobs and execute them as an array than execute them one by one. HTCondor provides the ability to match executables files using regular expressions and add a job to the queue for each file. In this exercise, all the executables which have the ending .sh will be queued. These executables are located in the current working directory in the submit machine. Create three simple executables based on the following code, replacing # with the numbers 1, 2 and 3 respectively.

#!/bin/bash

echo "Hello i am the file number #"

Submit the executables using the following submit description file:

    executable              = $(filename)
    arguments               = $(ClusterId)$(ProcId)
    output                  = output/$(ClusterId).$(ProcId).out
    error                   = error/$(ClusterId).$(ProcId).err
    log                     = log/$(ClusterId).log

    queue filename matching files *.sh

Note: In the submit description file, $(filename) is interpreted to be the file matching the regular expression.

The three jobs, even if they have different executables, have the same Cluster(21728) and different ProcIds(21728.0, 21728.1, 21728.2). By executing condor_q -nobatch, the same ClusterId and different ProcIds can be seen.

-- Schedd: bigbird04.cern.ch : <128.142.194.115:9618?... @ 03/28/17 10:24:36
 ID       OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
21728.0   fprotops        3/28 10:24   0+00:00:00 I  0    0.0 file01.sh file01.sh
21728.1   fprotops        3/28 10:24   0+00:00:00 I  0    0.0 file02.sh file02.sh
21728.2   fprotops        3/28 10:24   0+00:00:00 I  0    0.0 file03.sh file03.sh

3 jobs; 0 completed, 0 removed, 3 idle, 0 running, 0 held, 0 suspended

---

Last update: November 26, 2019