Skip to content

EOS

This section covers known issues and recommendations related to EOS.

How to best access EOS from central services

The EOS team provides a KB with recommendations to best access EOS: KB0004244.

No EOS submission allowed

As of today, one should not place logs mentioned in the HTCondor submit file to EOS. This restriction may be lifted in future when a EOS fuse-bind library with a better performance becomes available.

In the meantime, please instruct your executable to stage-in from / stage-out to EOS, from within the executable script.

The following elements of the HTCondor submit file should not contain path pointing to EOS mount point (starting with /eos/):

  • executable
  • log
  • input
  • output
  • error
  • initialdir

Note

If you are unsure about what field is causing this error, you can use the -verbose flag to print all the details of your job: Example: condor_submit -verbose job.sub.

If either of these elements points to a file under /eos/, you will experience submission error ERROR: EOS Submission is not currently supported by the HTCondor Service.

There are several approaches available to avoid this error:

  • Use -spool option of HTCondor, and then asynchronously download data with condor_transfer_data. There is no network file system involved, the operation is asynchronous, and you would have to actively download the data.

    $ condor_submit -spool hello.jdl

    • The spooled data is stored within condor infrastructure, for certain period of time which is min of the following 3 options:

      • until you do condor_transfer_data (where $JOBID represents the condor job ID, e.g. JOBID=123456789.0 )

        $ condor_transfer_data $JOBID

      • until you do condor_rm.

      • 10 days (or less if the spool fills up).

  • Use AFS for your log files, provide AFS paths that are writeable for your user instead of EOS paths.

Tip

You can use EOS for your files stage-in/stage-out from within the batch executable script, without any restriction. The only restriction is to avoid using EOS for log files mentioned in the HTCondor submit file. See Big files for an example.

Last update: March 20, 2024