GE_PE(5)                   Grid Engine File Formats                   GE_PE(5)



[1mNAME[0m
       ge_pe - Grid Engine parallel environment configuration file format

[1mDESCRIPTION[0m
       Parallel environments are parallel programming and runtime environments
       allowing for the execution of shared memory or distributed memory  par-
       allelized applications. Parallel environments usually require some kind
       of setup to  be  operational  before  starting  parallel  applications.
       Examples  for  common  parallel environments are shared memory parallel
       operating systems and the distributed memory environments Parallel Vir-
       tual Machine (PVM) or Message Passing Interface (MPI).

       [4mge_pe[24m  allows  for  the  definition of interfaces to arbitrary parallel
       environments.  Once a parallel environment is defined or modified  with
       the  [1m-ap  [22mor [1m-mp [22moptions to [4mqconf[24m(1) and linked with one or more queues
       via [4mpe_list[24m in [4mqueue_conf[24m(5) the environment can be requested for a job
       via  the  [1m-pe  [22mswitch to [4mqsub[24m(1) together with a request of a range for
       the number of parallel processes to be allocated by the job. Additional
       [1m-l  [22moptions  may  be  used  to  specify  the job requirement to further
       detail.

       Note, Grid Engine allows backslashes (\)  be  used  to  escape  newline
       (\newline)  characters. The backslash and the newline are replaced with
       a space (" ") character before any interpretation.

[1mFORMAT[0m
       The format of a [4mge_pe[24m file is defined as follows:

   [1mpe_name[0m
       The name  of  the  parallel  environment  as  defined  for  [4mpe_name[24m  in
       [4msge_types[24m(1).  To be used in the [4mqsub[24m(1) [1m-pe [22mswitch.

   [1mslots[0m
       The  number  of  parallel processes being allowed to run in total under
       the parallel environment concurrently.  Type is  number,  valid  values
       are 0 to 9999999.

   [1muser_lists[0m
       A  comma separated list of user access list names (see [4maccess_list[24m(5)).
       Each user contained in at least one of the enlisted  access  lists  has
       access  to the parallel environment. If the [1muser_lists [22mparameter is set
       to NONE (the default) any user has access being not explicitly excluded
       via  the [1mxuser_lists [22mparameter described below.  If a user is contained
       both in an access list enlisted in [1mxuser_lists [22mand [1muser_lists [22mthe  user
       is denied access to the parallel environment.

   [1mxuser_lists[0m
       The  [1mxuser_lists [22mparameter contains a comma separated list of so called
       user access lists as described in [4maccess_list[24m(5).  Each user  contained
       in  at  least one of the enlisted access lists is not allowed to access
       the parallel environment. If the [1mxuser_lists [22mparameter is set  to  NONE
       (the  default)  any  user has access. If a user is contained both in an
       access list enlisted in [1mxuser_lists [22mand [1muser_lists [22mthe user  is  denied
       access to the parallel environment.

   [1mstart_proc_args[0m
       The  invocation  command  line of a start-up procedure for the parallel
       environment. The start-up procedure is invoked by [4mge_shepherd[24m(8)  prior
       to executing the job script. Its purpose is to setup the parallel envi-
       ronment correspondingly to its needs.  An optional prefix "user@" spec-
       ifies  the user under which this procedure is to be started.  The stan-
       dard output of the start-up procedure is redirected to  the  file  [4mREQ-[0m
       [4mNAME[24m.po[4mJID[24m  in  the job's working directory (see [4mqsub[24m(1)), with [4mREQNAME[0m
       being the name of the job as displayed by [4mqstat[24m(1) and  [4mJID[24m  being  the
       job's  identification  number.   Likewise, the standard error output is
       redirected to [4mREQNAME[24m.pe[4mJID[0m
       The following special variables being expanded at runtime can  be  used
       (besides  any  other  strings which have to be interpreted by the start
       and stop procedures) to constitute a command line:

       [4m$pe_hostfile[0m
              The pathname of a file containing a detailed description of  the
              layout  of  the parallel environment to be setup by the start-up
              procedure. Each line of the file refers to a host on which  par-
              allel  processes  are  to  be  run. The first entry of each line
              denotes the hostname, the second entry the  number  of  parallel
              processes to be run on the host, the third entry the name of the
              queue, and the fourth entry a processor range to be used in case
              of a multiprocessor machine.

       [4m$host[24m  The  name  of  the host on which the start-up or stop procedures
              are started.

       [4m$job_owner[0m
              The user name of the job owner.

       [4m$job_id[0m
              Grid Engine's unique job identification number.

       [4m$job_name[0m
              The name of the job.

       [4m$pe[24m    The name of the parallel environment in use.

       [4m$pe_slots[0m
              Number of slots granted for the job.

       [4m$processors[0m
              The [1mprocessors [22mstring as contained in  the  queue  configuration
              (see  [4mqueue_conf[24m(5)) of the master queue (the queue in which the
              start-up and stop procedures are started).

       [4m$queue[24m The cluster queue of the master queue instance.

   [1mstop_proc_args[0m
       The invocation command line of a shutdown procedure  for  the  parallel
       environment.  The shutdown procedure is invoked by [4mge_shepherd[24m(8) after
       the job script has finished. Its purpose is to stop the parallel  envi-
       ronment  and  to remove it from all participating systems.  An optional
       prefix "user@" specifies the user under which this procedure is  to  be
       started.   The standard output of the stop procedure is also redirected
       to the file [4mREQNAME[24m.po[4mJID[24m in the job's working directory (see [4mqsub[24m(1)),
       with [4mREQNAME[24m being the name of the job as displayed by [4mqstat[24m(1) and [4mJID[0m
       being the job's identification number.  Likewise,  the  standard  error
       output is redirected to [4mREQNAME[24m.pe[4mJID[0m
       The  same  special variables as for [1mstart_proc_args [22mcan be used to con-
       stitute a command line.

   [1mallocation_rule[0m
       The allocation rule is interpreted by the scheduler  thread  and  helps
       the  scheduler to decide how to distribute parallel processes among the
       available machines. If, for instance, a parallel environment  is  built
       for  shared memory applications only, all parallel processes have to be
       assigned to a single machine, no matter how much suitable machines  are
       available.   If, however, the parallel environment follows the distrib-
       uted memory paradigm, an even distribution of processes among  machines
       may be favorable.
       The  current  version  of  the scheduler only understands the following
       allocation rules:

       [1m<int>[22m:    An integer number fixing the number of processes per host. If
                 the  number  is  1, all processes have to reside on different
                 hosts. If the special denominator [1m$pe_slots [22mis used, the full
                 range  of  processes as specified with the [4mqsub[24m(1) [1m-pe [22mswitch
                 has to be allocated on a single host (no matter  which  value
                 belonging  to  the  range is finally chosen for the job to be
                 allocated).

       [1m$fill_up[22m: Starting from the best  suitable  host/queue,  all  available
                 slots are allocated. Further hosts and queues are "filled up"
                 as long as a job still requires slots for parallel tasks.

       [1m$round_robin[22m:
                 From all suitable hosts a single slot is allocated until  all
                 tasks  requested  by the parallel job are dispatched. If more
                 tasks are requested than suitable hosts are found, allocation
                 starts  again  from  the  first  host.  The allocation scheme
                 walks through suitable hosts in a best-suitable-first  order.

   [1mcontrol_slaves[0m
       This  parameter can be set to TRUE or FALSE (the default). It indicates
       whether Grid Engine is the creator of the slave  tasks  of  a  parallel
       application  via  [4mge_execd[24m(8) and [4mge_shepherd[24m(8) and thus has full con-
       trol over all processes in a parallel application, which enables  capa-
       bilities  such  as resource limitation and correct accounting. However,
       to gain control over the slave  tasks  of  a  parallel  application,  a
       sophisticated  PE  interface  is required, which works closely together
       with Grid Engine facilities. Such PE interfaces are  available  through
       your local Grid Engine support office.

       Please  set  the  control_slaves  parameter  to  false for all other PE
       interfaces.

   [1mjob_is_first_task[0m
       The [1mjob_is_first_task [22mparameter can be set to TRUE or FALSE. A value of
       TRUE  indicates that the Grid Engine job script already contains one of
       the tasks of the parallel application (the number of slots reserved for
       the  job is the number of slots requested with the -pe switch), while a
       value of FALSE indicates that the job script (and its child  processes)
       is  not  part of the parallel program (the number of slots reserved for
       the job is the number of slots requested with the -pe switch + 1).

       If  wallclock  accounting  is  used  (execd_params  ACCT_RESERVED_USAGE
       and/or  SHARETREE_RESERVED_USAGE set to TRUE) and [1mcontrol_slaves [22mis set
       to FALSE, the [1mjob_is_first_task [22mparameter influences the accounting for
       the  job:  A  value of TRUE means that accounting for cpu and requested
       memory gets multiplied by the number of slots requested  with  the  -pe
       switch,  if  [1mjob_is_first_task [22mis set to FALSE, the accounting informa-
       tion gets multiplied by number of slots + 1.

   [1murgency_slots[0m
       For pending jobs with a slot range PE request the number  of  slots  is
       not  determined.  This  setting specifies the method to be used by Grid
       Engine to assess the number of slots such jobs might finally get.

       The  assumed  slot  allocation  has  a  meaning  when  determining  the
       resource-request-based  priority  contribution for numeric resources as
       described in [4msge_priority[24m(5) and is  displayed  when  [4mqstat[24m(1)  is  run
       without [1m-g t [22moption.

       The following methods are supported:

       [1m<int>[22m:    The  specified integer number is directly used as prospective
                 slot amount.

       [1mmin[22m:      The slot range minimum is used as prospective slot amount. If
                 no lower bound is specified with the range 1 is assumed.

       [1mmax[22m:      The  of  the  slot  range maximum is used as prospective slot
                 amount.  If no upper bound is specified with  the  range  the
                 absolute  maximum  possible  due to the PE's [1mslots [22msetting is
                 assumed.

       [1mavg[22m:      The average of all numbers  occurring  within  the  job's  PE
                 range request is assumed.

   [1maccounting_summary[0m
       This  parameter is only checked if [1mcontrol_slaves [22m(see above) is set to
       TRUE and thus Grid Engine is the creator of the slave tasks of a paral-
       lel  application  via  [4mge_execd[24m(8)  and  [4mge_shepherd[24m(8).  In this case,
       accounting information is available for every single slave task started
       by Grid Engine.

       The  [1maccounting_summary  [22mparameter can be set to TRUE or FALSE. A value
       of TRUE indicates that only a single accounting record  is  written  to
       the  [4maccounting[24m(5) file, containing the accounting summary of the whole
       job including all slave tasks, while a  value  of  FALSE  indicates  an
       individual  [4maccounting[24m(5)  record  is  written for every slave task, as
       well as for the master task.

[1mRESTRICTIONS[0m
       [1mNote[22m, that the functionality of the start-up,  shutdown  and  signaling
       procedures remains the full responsibility of the administrator config-
       uring the parallel environment.  Grid Engine  will  just  invoke  these
       procedures  and  evaluate  their  exit status. If the procedures do not
       perform their tasks properly or if the parallel environment or the par-
       allel  application  behave  unexpectedly,  Grid  Engine has no means to
       detect this.

[1mSEE ALSO[0m
       [4mge_intro[24m(1),  [4mge__types[24m(1),  [4mqconf[24m(1),   [4mqdel[24m(1),   [4mqmod[24m(1),   [4mqsub[24m(1),
       [4maccess_list[24m(5), [4mge_qmaster[24m(8), [4mge_shepherd[24m(8).

[1mCOPYRIGHT[0m
       See [4mge_intro[24m(1) for a full statement of rights and permissions.



GE 6.2u2                 $Date: 2008/12/29 08:09:36 $                 GE_PE(5)
