Knowledge Center         Contents    Previous  Next    Index  
Platform Computing Corp.

Job Dependencies

Contents

Job Dependency Terminology

Example: If job2 depends on job1 (meaning that job2 cannot start until job1 reaches a specific state), then job2 is the child job and job1 is the parent job.

Job Dependency Scheduling

About job dependency scheduling

Sometimes, whether a job should start depends on the result of another job. For example, a series of jobs could process input data, run a simulation, generate images based on the simulation output, and finally, record the images on a high-resolution film output device. Each step can only be performed after the previous step finishes successfully, and all subsequent steps must be aborted if any step fails.

Some jobs may not be considered complete until some post-job processing is performed. For example, a job may need to exit from a post-execution job script, clean up job files, or transfer job output after the job completes.

In LSF, any job can be dependent on other LSF jobs. When you submit a job, you use bsub -w to specify a dependency expression, usually based on the job states of preceding jobs.

LSF will not place your job unless this dependency expression evaluates to TRUE. If you specify a dependency on a job that LSF cannot find (such as a job that has not yet been submitted), your job submission fails.

Specify a job dependency

  1. To specify job dependencies, use bsub -w to specify a dependency expression for the job.
Syntax

bsub -w 'dependency_expression'

The dependency expression is a logical expression composed of one or more dependency conditions. For syntax of individual dependency conditions, see Dependency Conditions.

Multiple jobs with the same name

By default, if you use the job name to specify a dependency condition, and more than one of your jobs has the same name, all of your jobs that have that name must satisfy the test.

To change this behavior, set JOB_DEP_LAST_SUB in lsb.params to 1. Then, if more than one of your jobs has the same name, the test is done on the one submitted most recently.

Dependency Conditions

The following dependency conditions can be used with any job:

done

Syntax

done(job_ID | "job_name")

Description

The job state is DONE.

ended

Syntax

ended(job_ID | "job_name")

Description

The job state is EXIT or DONE.

exit

Syntax

exit(job_ID | "job_name"[,[operator] exit_code])

where operator represents one of the following relational operators:

>

>=

<

<=

==

!=

Description

The job state is EXIT, and the job's exit code satisfies the comparison test.

If you specify an exit code with no operator, the test is for equality (== is assumed).

If you specify only the job, any exit code satisfies the test.

Examples

exit (myjob)

The job named myjob is in the EXIT state, and it does not matter what its exit code was.

exit (678,0)

The job with job ID 678 is in the EXIT state, and terminated with exit code 0.

exit ("678",!=0)

The job named 678 is in the EXIT state, and terminated with any non-zero exit code.

external

Syntax

external(job_ID | "job_name", "status_text")

Specify the first word of the job status or message description (no spaces). Only the first word is evaluated.

Description

The job has the specified job status, or the text of the job's status begins with the specified word.

Job ID or job name

Syntax

job_ID | "job_name"

Description

If you specify a job without a dependency condition, the test is for the DONE state (LSF assumes the "done" dependency condition by default).

post_done

Syntax

post_done(job_ID | "job_name")

Description

The job state is POST_DONE (the post-processing of specified job has completed without errors).

post_err

Syntax

post_err(job_ID | "job_name")

Description

The job state is POST_ERR (the post-processing of specified job has completed with errors).

started

Syntax

started(job_ID | "job_name")

Description

The job state is:

Advanced dependency conditions

Job arrays

If you use job arrays, you can specify additional dependency conditions that only work with job arrays.

To use other dependency conditions with array jobs, specify elements of a job array in the usual way.

Job dependency examples

bsub -J "JobA" -w 'done(JobB)' command

The simplest kind of dependency expression consists of only one dependency condition. For example, if JobA depends on the successful completion of JobB, submit the job as shown.

-w 'done(312) && (started(Job2)||exit("99Job"))'

The submitted job will not start until the job with the job ID of 312 has completed successfully, and either the job named Job2 has started, or the job named 99Job has terminated abnormally.

-w "210"

The submitted job will not start unless the job named 210 is finished.

View Job Dependencies

The bjdepinfo command displays any dependencies that jobs have, either jobs that depend on a job or jobs that your job depends on.

By specifying -r, you get not only direct dependencies (job A depends on job B), but also indirect dependencies (job A depends on job B, job B depends on jobs C and D). You can also limit the number of levels returned using the -r option.

The -l option displays results in greater detail.

JOBID PARENT PARENT_STATUS PARENT_NAME  LEVEL 
123   32522  RUN           JOB32522     1 
JOBID CHILD CHILD_STATUS CHILD_NAME LEVEL 
300   310   PEND         JOB310     1  
300   311   PEND         JOB311     1  
300   312   PEND         JOB312     1  
JOBID PARENT PARENT_STATUS PARENT_NAME LEVEL 
100   99     PEND          JOB99       1 
100   98     PEND          JOB98       1 
100   97     PEND          JOB97       1 
100   30     PEND          JOB30       1 
Dependency condition of job <32522> is not satisfied: done(23455) 
JOBID PARENT PARENT_STATUS PARENT_NAME  LEVEL 
32522 23455  RUN           JOB23455     1 

Platform Computing Inc.
www.platform.com
Knowledge Center         Contents    Previous  Next    Index