JSV(1)                     Grid Engine File Formats                     JSV(1)



[1mNAME[0m
       JSV - Grid Engine Job Submission Verifier

[1mDESCRIPTION[0m
       JSV  is  an abbreviation for Job Submission Verifier. A JSV is a script
       or binary that can be used to verify, modify or reject a job during the
       time of job submission.

       JSVs  will be triggered by submit clients like [4mqsub[24m, [4mqrsh[24m, [4mqsh[24m and [4mqmon[0m
       on submit hosts (Client JSV) or they verify incoming jobs on the master
       host (Server JSV) or both.

[1mCONFIGURATION[0m
       JSVs  can  be  configured on various locations. Either a [1mjsv_url [22mcan be
       provided with the [1m-jsv [22msubmit parameter during job submission, a corre-
       sponding  switch  can  be  added  to  one of the [1msge_request [22mfiles or a
       [1mjsv_url [22mcan be configured in the global cluster  configuration  of  the
       Grid Engine installation.

       All defined JSV instances will be executed in following order:

          1) qsub -jsv ...
          2) $cwd/.sge_request
          3) $HOME/.sge_request
          4) $SGE_ROOT/$SGE_CELL/common/sge_request
          5) Global configuration

       The  Client  JSVs (1-3) can be defined by Grid Engine end users whereas
       the client JSV defined in the  global  sge_request  file  (4)  and  the
       server JSV (5) can only be defined by the Grid Engine administrators.

       Due  to  the  fact  that (4) and (5) are defined and configured by Grid
       Engine administrators  and  because  they  are  executed  as  last  JSV
       instances in the sequence of JSV scripts, an administrator has an addi-
       tional way to define certain policies for a cluster.

       As soon as one JSV instance rejects a job the whole process of  verifi-
       cation  is stopped and the end user will get a corresponding error mes-
       sage that the submission of the job has failed.

       If a JSV accepts a job or accepts a job after it applied serveral modi-
       fications  then  the following JSV instance will get the job parameters
       including all modifications as  input  for  the  verification  process.
       This is done as long as either the job is accepted or rejected.

       Find  more information how to use Client JSVs in [4mqsub[24m(1) and for Server
       JSVs in [4msge_conf[24m(5)

[1mLIFETIME[0m
       A Client or Server JSV is started as own  UNIX  process.  This  process
       communicates  either  with  a  Grid Engine client process or the master
       daemon by exchanging  commands,  job  parameters  and  other  data  via
       [1mstdin[22m/[1mstdout [22mchannels.

       Client JSV instances are started by client applications before a job is
       sent to qmaster. This instance does the job verification for the job to
       be submitted. After that verification the JSV process is stopped.

       Server  JSV  instances  are  started for each worker thread part of the
       qmaster process (for version 6.2 of Grid Engine  this  means  that  two
       processes  are  started).  Each  of  those processes have to verify job
       parameters for multiple jobs as long as  the  master  is  running,  the
       underlaying JSV configuration is not changed and no error occurs.

[1mPROTOCOL[0m
       After  a  JSV  script or binary is started it will get commands through
       its [1mstdin [22mstream and it has to respond with  certain  commands  on  the
       [1mstdout  [22mstream.  Data  which  is  send  via  the [1mstderr [22mstream of a JSV
       instance is ignored. Each command which is send to/by a JSV script  has
       to  be terminated by a new line character ('[1m\n[22m') whereas new line char-
       acters are not allowed in the whole command string itself.

       In general commands which are exchanged between a JSV and  client/qmas-
       ter  have  following format. Commands and arguments are case sensitive.
       Find the EBNF command description below.

             [1mcommand [22m:= [1mcommand_name [22m' ' { [1margument [22m' ' } ;

       A [1mcommand [22mstarts with a [1mcommand_name [22mfollowed by a space character  and
       a space separated list of [1marguments[22m.

[1mPROTOCOL (JSV side)[0m
       Following  [1mcommands  [22mhave to be implemented by an JSV script so that it
       conforms to version 1.0 of the JSV protocol which was first implemented
       in Grid Engine 6.2u2:

       [1mbegin_command [22m:= '[1mBEGIN[22m' ;
              After   a   JSV  instance  has  received  all  [1menv_commands  [22mand
              [1mparam_commands  [22mof  a  job  which  should   be   verified,   the
              client/qmaster  will trigger the verification process by sending
              one [1mbegin_command[22m. After that it will  wait  for  [1mparam_commands[0m
              and  [1menv_commands  [22mwhich  are sent back from the JSV instance to
              modify the  job  specification.  As  part  of  the  verification
              process  a JSV script or binary has to use the [1mresult_command [22mto
              indicate that the verification process is finished for a job.

       [1menv_command [22m:= [1mENV [22m' ' [1mmodifier [22m' ' [1mname [22m' ' [1mvalue [22m;

       [1mmodifier [22m:= '[1mADD[22m' | '[1mMOD[22m' | '[1mDEL[22m' ;
              The [1menv_command [22mis an optional command  which  has  only  to  be
              implemented  by  a JSV instance if the [1msend_data_command [22mis sent
              by this JSV before a the [1mstarted_command [22mwas sent. Only in  that
              case  the client or master will use one or multiple [1menv_commands[0m
              to pass the environment variables ([1mname [22mand [1mvalue[22m)  to  the  JSV
              instance which would be exported to the job environment when the
              job would be started. Client and qmaster will only sent [1menv_com-[0m
              [1mmands [22mwith the modifier '[1mADD[22m'.

              JSV instances modify the set of environment variables by sending
              back [1menv_commands [22mand by using the [1mmodifiers ADD[22m, [1mMOD [22mand [1mDEL[22m.

       [1mparam_command [22m:= '[1mPARAM[22m' ' ' [1mparam_parameter [22m' ' [1mvalue [22m;

       [1mparam_parameter [22m:= [1msubmit_parameter [22m| [1mpseudo_parameter [22m;
              The [1mparam_command [22mhas two additional arguments which  are  sepa-
              rated  by  space characters. The first argument is either a [1msub-[0m
              [1mmit_parameter  [22mas  it  is  specified  in  [4mqsub[24m(1)  or  it  is  a
              [1mpseudo_parameters  [22mas documented below.  The second parameter is
              the [1mvalue [22mof the corresponding [1mparam_parameter[22m.

              Multiple [1mparam_commands [22mwill be sent to a JSV instance after the
              JSV  has  sent  a [1mstarted_command[22m. The sum of all [1mparam_commands[0m
              which is sent represents a job specification of that  job  which
              should be verified.

              [1msubmit_parameters  [22mare  for  example  [1mb  [22m(similar to the [1mqsub -b[0m
              switch) or [1mmasterq [22m(similar to [1mqsub  -masterq  [22mswitch).  Find  a
              complete  list  of  [1msubmit_parameters  [22min  the [4mqsub[24m(1) man page.
              Please note that not in all cases the [1mparam_parameter  [22mname  and
              the  corresponding  [1mvalue  [22mformat  is  equivalent  with the qsub
              switch name and its argument format.  E.g. the [1mqsub -pe  [22mparame-
              ters  will  by  available  as  a set of parameters with the name
              [1mpe_name[22m, [1mpe_min[22m, [1mpe_max [22mor the switch combination [1m-soft -l  [22mwill
              be  passed  to JSV scripts as [1ml_soft [22mparameter. For details con-
              cerning this differences consult also the [4mqsub[24m(1) man page.

       [1mstart_command [22m:= '[1mSTART[22m' ;
              The [1mstart_command [22mhas  no  additional  arguments.  This  command
              indicates  that  a new job verification should be started. It is
              the first command which will be sent to JSV script after it  has
              been  started  and it will initiate each new job verification. A
              JSV instance might trash cached values which  are  still  stored
              due  to  a previous job verification. The application which send
              the [1mstart_command [22mwill wait for a [1mstarted_command [22mbefore it con-
              tinues.

       [1mquit_command [22m:= '[1mQUIT[22m' ;
              The [1mquit_command [22mhas no additional arguments. If this command is
              sent to a JSV instance then it should terminate  itself  immedi-
              ately.

[1mPROTOCOL (client/qmaster side)[0m
       A  JSV  script or binary can send a set of commands to a client/qmaster
       process to indicate its state in the communication process,  to  change
       the  job  specification of a job which should be verified and to report
       messages or errors. Below you can find the commands  which  are  under-
       stood  by  the  client/qmaster  which will implement version 1.0 of the
       communication protocol which  was  first  implemented  in  Grid  Engine
       6.2u2:

       [1merror_command [22m:= '[1mERROR[22m' [1mmessage [22m;
              Any  time a JSV script encounters an error it might report it to
              the client/qmaster. If the error happens during a job  verifica-
              tion  the  job which is currently verified will be rejected. The
              JSV binary or script will also be restarted before it gets a new
              verification task.

       [1mlog_command [22m:= '[1mLOG[22m' [1mlog_level [22m;

       [1mlog_level [22m:= '[1mINFO[22m' | '[1mWARNING[22m' | '[1mERROR[22m'
              [1mlog_commands  [22mcan be used whenever the client or qmaster expects
              input from a JSV instance. This command can be  used  in  client
              JSVs  to  send  information  to  the user submitting the job. In
              client JSVs all messages, independed of the  [1mlog_level  [22mwill  be
              printed  to  the  [1mstdout  [22mstream of the used submit client. If a
              server JSV receives a [1mlog_command [22mit will add the received  mes-
              sage  to  the  message  file respecting the specified [1mlog_level[22m.
              Please note that [1mmessage [22mmight contain spaces but  no  new  line
              characters.

       [1mparam_command [22m(find definition above)
              By sending [1mparam_commands [22ma JSV script can change the job speci-
              fication of the job which should be verified. If a JSV  instance
              later  on  sends  a  [1mresult_command  [22mwhich  indicates that a JSV
              instance should be accepted with correction then the values pro-
              vided  with  these [1mparam_commands [22mwill be used to modify the job
              before it is accepted by the Grid Engine system.

       [1mresult_command [22m:= '[1mRESULT[22m' [1mresult_type [22m[ [1mmessage [22m] ;

       [1mresult_type [22m:= '[1mACCEPT[22m' | '[1mCORRECT[22m' | '[1mREJECT[22m' | '[1mREJECT_WAIT[22m' ;
              After the verification of a job is done a JSV script  or  binary
              has  to send a [1mresult_command [22mwhich indicates what should happen
              with the job.  If the [1mresult_type [22mis [1mACCEPTED [22mthe  job  will  be
              accepted  as  it  was  initially  submitted by the end user. All
              [1mparam_commands [22mand  [1menv_commands  [22mwhich  might  have  been  sent
              before  the  [1mresult_command  [22mare  ignored  in  this  case.   The
              [1mresult_type CORRECT [22mindicates that the job  should  be  accepted
              after all modifications sent via [1mparam_commands [22mand [1menv_commands[0m
              are applied to the job.  [1mREJECT [22mand [1mREJECT_WAIT [22mcause the client
              or qmaster instance to reject the job.

       [1msend_data_command [22m:= '[1mSEND[22m' [1mdata_name [22m;

       [1mdata_name [22m:= '[1mENV[22m';
              If  a  client/qmaster  receives  a  [1msend_env_command  [22mfrom a JSV
              instance before a [1mstarted_command [22mis sent, then it will not only
              pass  job  parameters  with [1mparam_commands [22mbut also [1menv_commands[0m
              which provide the JSV with  the  information  which  environment
              variables would be exported to the job environment if the job is
              accepted and started later on.

              The job environment is not passed to JSV  instances  as  default
              because  the  job environment of the end user might contain data
              which might be interpreted wrong in the JSV  context  and  might
              therefore cause errors or security issues.

       [1mstarted_command [22m:= '[1mSTARTED[22m' ;
              By  sending the [1mstarted_command [22ma JSV instance indicates that it
              is ready to receive [1mparam_commands [22mand [1menv_commands  [22mfor  a  new
              job  verification. It will only receive [1menv_commands [22mif it sends
              a [1msend_data_command [22mbefore the [1mstarted_command[22m.

[1mPSEUDO PARAMETERS[0m
       [1mCLIENT [22mThe corresponding value for  the  [1mCLIENT  [22mparameters  is  either
              '[1mqmaster[22m' or the name of a submit client like '[1mqsub[22m',
               '[1mqsh[22m', '[1mqrsh[22m', '[1mqlogin[22m' and so on This parameter value can't be
              changed by JSV instances.  It will always be sent as part  of  a
              job verification.

       [1mCMDARGS[0m
              Number  of  arguments  which will be passed to the job script or
              command  when the job execution is started.  It will  always  be
              sent  as  part of a job verification.  If no arguments should be
              passed to the job script or command it will have  the  value  0.
              This  parameter can be changed by JSV instances. If the value of
              [1mCMDARGS [22mis bigger than the number of available [1mCMDARG<id> [22mparam-
              eters  then  the missing parameters will be automatically passed
              as empty parameters to the job script.

       [1mCMDNAME[0m
              Either the path to the script or the command  name  in  case  of
              binary submission.  It will always be sent as part of a job ver-
              ification.

       [1mCONTEXT[0m
              Either '[1mclient[22m' if the JSV which receives this [1mparam_command [22mwas
              started  by a commandline client like [4mqsub[24m, [4mqsh[24m, ... or '[1mmaster[22m'
              if it was started by the [4msge_qmaster[24m process.  It will always be
              sent  as part of a job verification.  Changing the value of this
              parameters is not possible within JSV instances.

       [1mGROUP  [22mDefines Primary group of the user which tries to submit the  job
              which  should  be verifyed. This parameter cannot be changed but
              is always sent as part of the  verification  process.  The  user
              name is passed as parameters with the name [1mUSER[22m.

       [1mJOB_ID [22mNot  available in the client context (see [1mCONTEXT[22m). Otherwise it
              contains the job number of the job which will  be  submitted  to
              Grid Engine when the verification process is successfull. [1mJOB_ID[0m
              is  an  optional  parameter  which  can't  be  changed  by   JSV
              instances.

       [1mUSER   [22mUsername  of the user which tries to submit the job which should
              be verified. Cannot be changed but is always sent as part of the
              verification  process.  The  group  name  is passed as parameter
              with the name [1mGROUP[0m

       [1mVERSION[0m
              [1mVERSION [22mwill always be  sent  as  part  of  a  job  verification
              process and it will always be the first parameter which is sent.
              It will contain a version number of the format  <major>.<minor>.
              In  version 6.2u2 and higher the value will be '[1m1.0[22m'.  The value
              of this parameter can't be changed.

[1mEXAMPLE[0m
       Here is an example for  the  communication  of  a  client  with  a  JSV
       instance when following job is submitted:

       > qsub -pe p 3 -hard -l a=1,b=5 -soft -l q=all.q $SGE_ROOT/examples/jobs/sleeper.sh

       Data  in  the  first  column is sent from the client/qmaster to the JSV
       instance. That data contained in the second column is sent from the JSV
       script  to the client/qmaster. New line characters which terminate each
       line in the communication protocol are ommited.

          START
                                  SEND ENV
                                  STARTED
          PARAM VERSION 1.0
          PARAM CONTEXT client
          PARAM CLIENT qsub
          PARAM USER ernst
          PARAM GROUP staff
          PARAM CMDNAME /sge_root/examples/jobs/sleeper.sh
          PARAM CMDARGS 1
          PARAM CMDARG0 12
          PARAM l_hard a=1,b=5
          PARAM l_soft q=all.q
          PARAM M user@hostname
          PARAM N Sleeper
          PARAM o /dev/null
          PARAM pe_name pe1
          PARAM pe_min 3
          PARAM pe_max 3
          PARAM S /bin/sh
          BEGIN
                                  RESULT STATE ACCEPT


[1mSEE ALSO[0m
       [4mge_intro[24m(1), [4mqalter[24m(1), [4mqlogin[24m(1), [4mqmake[24m(1), [4mqrsh[24m(1), [4mqsh[24m(1),  [4mqsub[24m(1),
       [4mqtcsh[24m(1),

[1mCOPYRIGHT[0m
       See [4mge_intro[24m(1) for a full statement of rights and permissions.



GE 6.2u2                 $Date: 2009/02/11 12:33:12 $                   JSV(1)
