Skip to content

Python API

Why use the Python API

HTCondor has long had a Python API, but until recently it was not possible to us it to submit jobs to our HTCondor installation at CERN. Fortunately, the version that we are now running is capable of submitting jobs to run here at CERN with the credentials needed to interact with things like network filesystems.

The documentation for the API can be found here. There are tutorials, for example here or fully fledged binder tutorials here. We would urge you to consult them. The point to this guide is to give you tips on how to use those bindings here at CERN.

Python job submission at CERN

Python submission: token management

In a python shell, here's a method to get your token submitted to the schedd. It assumes that you are running on either lxplus or a machine setup like lxplus to have SEC_CREDENTIAL_PRODUCER in the condor config (if you are running on your own installation, if you can submit jobs to the CERN batch system from the command line, then it will also work in the api). It also assumes that you have valid kerberos tickets.

import htcondor

col = htcondor.Collector()
credd = htcondor.Credd()
credd.add_user_cred(htcondor.CredTypes.Kerberos, None)

It should be noted at this point that there is a credd.check_user_cred() method, but it currently does not work with kerberos tokens, but this should be fixed upstream soon. For the time being, it would be good practice to submit a token with each session. You can normally count on tokens being maintained for a few hours, but adding a token is relatively lightweight, so can be repeated as necessary.

Submitting a job

With the creds on the schedd, submitting with the submit object isn't a lot different.

>>> sub = htcondor.Submit()
>>> sub['Executable'] = "/afs/cern.ch/user/b/bejones/tmp/condor/hello.sh"
>>> sub['Error'] = "/afs/cern.ch/user/b/bejones/tmp/condor/error/hello-$(ClusterId).$(ProcId).err"
>>> sub['Output'] = "/afs/cern.ch/user/b/bejones/tmp/condor/output/hello-$(ClusterId).$(ProcId).out"
>>> sub['Log'] = "/afs/cern.ch/user/b/bejones/tmp/condor/log/hello-$(ClusterId).log"
>>> sub['MY.SendCredential'] = True
>>> sub['+JobFlavour'] = '"tomorrow"'
>>> sub['request_cpus'] = '1'
>>>
>>> schedd = htcondor.Schedd()
>>> with schedd.transaction() as txn:
...     cluster_id = sub.queue(txn)
...
>>> print(cluster_id)
1034

Querying jobs

>>> print(cluster_id)
1034
>>> q = schedd.query(constraint='ClusterId == 1034', projection=['ClusterId','ProcId','Cmd','Args', 'JobStatus'])
>>> q
[[ ClusterId = 1034; ProcId = 0; ServerTime = 1622815338; Cmd = "/afs/cern.ch/user/b/bejones/tmp/condor/hello.sh"; JobStatus = 1 ]]
>>> for job in q:
...     print(f'{job.get("ClusterId")}.{job.get("ProcId")} status: {htcondor.JobStatus(job.get("JobStatus")).name}')
...
1034.0 status: IDLE