WebJob - Download and execute a program over HTTP/HTTPS
webjob {-e|--execute} {-f|--file} config program [options]
webjob {-v|--version}
WebJob downloads and executes a program in one unified operation. WebJob's runtime behavior is controlled through a config file identified by the --file argument. Among other things, this file identifies a Web resource willing to satisfy WebJob GET requests. Once initialized, WebJob downloads the target program, stores it in on the local file system, and executes it in a sub-process. The options, if any, are bound to the target program at invocation. By default, any output generated by the target program is directed to stdout/stderr. However, output may also be directed to a Web resource through a WebJob PUT request. When compiled with OpenSSL, WebJob supports HTTPS requests and certificate authentication.
WebJob's GET/PUT requests may be bound to any available outbound port. This increases the likelihood that distant Web resources can be reached in spite of firewalls and filtering routers.
WebJob is roughly split into three stages: GET, RUN, and PUT. Each stage provides an independent timer. In the event that a given stage doesn't complete within the alloted time, WebJob will attempt to abort gracefully. A timeout value of zero means there is no time limit.
The modes of operation described in this section are mutually exclusive. In other words, only one mode may be specified per invocation.
Use config file settings to download and execute the specified program. The config file, as specified by the --file argument, may be the name of a regular file or '-' which indicates that configuration information is to be read from stdin. Note: Path information in the program argument is not allowed. The options, if any, are bound to the target program at invocation. The various config file controls and syntax rules are described in the CONFIGURATION CONTROLS section of this document.
Display version information and exit.
This section describes the various controls that WebJob recognizes. In general, controls either shape runtime behavior or provide information needed by the application to perform a specific function. Controls and their values, one pair/line, are written to a file having the following format.
<control> = <value>
All controls are case insensitive, but, in general, their values are not. Comments may occur anywhere on a given line, and must begin with a pound character (i.e. '#'). In any given line, all text to the right of the first comment will be ignored. White space surrounding controls and values is ignored.
This section describes each control that may be specified, defines what values it may have, and states which modes of operation recognize the control.
Applies to execute.
GetTimeLimit is optional. The value of this control specifies how long the program is willing wait before giving up on download (i.e. GET) requests. If GetTimeLimit is non zero and the time limit expires, the program will abort. If GetTimeLimit is not set or is set to zero, no limit will be imposed.
Applies to execute.
Import is optional. When specified, the directives contained in the file referenced by this control are included in the current configuration. Multiple instances of this control are allowed per file, and recursion is permitted up to three levels deep. Imports may be specified using a relative path.
Applies to execute.
OverwriteExecutable is optional. When enabled (i.e. 'Y'), existing executables in the specified TempDirectory will be overwritten by subsequent GET requests. The default value is disabled.
Applies to execute.
PutTimeLimit is optional. The value of this control specifies how long the program is willing wait before giving up on upload (i.e. PUT) requests. If PutTimeLimit is non zero and the time limit expires, the program will abort. If PutTimeLimit is not set or is set to zero, no limit will be imposed.
Applies to execute.
RunTimeLimit is optional. The value of this control specifies how long the program is willing wait for the target application to finish. If RunTimeLimit is non zero and the time limit expires, the program will kill the spawned process and abort. If RunTimeLimit is not set or is set to zero, no limit will be imposed.
Applies to execute.
RunType is optional. This control sets a corresponding flag that classifies uploaded data as linktest or snapshot, The value of this control does not affect the format or content of WebJob output. When output is uploaded to a Web server, the value of this control may be used to determine how the data is processed. If not specified, RunType defaults to linktest.
Applies to execute.
SSLBundledCAsFile is required when SSLVerifyPeerCert is enabled. This control specifies the name of a PEM (Privacy Enhanced Mail) encoded file that contains a bundled set of Certificate Authority (CA) certificates. Any validated peer certificate that is signed by one of these CAs will be accepted provided that the SSLMaxChainLength and SSLExpectedPeerCN checks are also satisfied. SSLBundledCAsFile may be specified as a relative path.
Applies to execute.
SSLExpectedPeerCN is required when SSLVerifyPeerCert is enabled. The value of this control represents the peer's expected Common Name (CN). Conventionally, CNs are specified as fully qualified domain names. This control eliminates the need to perform a DNS lookup at the time of certificate validation. This, in turn, may help to prevent attacks involving DNS spoofing.
Applies to execute.
SSLMaxChainLength is optional when SSLVerifyPeerCert is enabled. The value of this control determines how deep a certificate chain may be before it is considered invalid. If not specified, this control defaults to a value of 2.
Applies to execute.
SSLPassPhrase is optional when SSLUseCertificate is enabled. Its value, if specified, is used to decrypt the contents of the client's private key file (see SSLPrivateKeyFile). If a passphrase is required to load the private key and this control has not been set, the user will be prompted for one. Naturally, this will cause problems for automated tasks, so keep that in mind.
Applies to execute.
SSLPrivateKeyFile is required when SSLUseCertificate is enabled. This control specifies the name of a PEM (Privacy Enhanced Mail) encoded key file that can be used to sign SSL certificates. SSLPrivateKeyFile may be specified as a relative path.
Applies to execute.
SSLPublicCertFile is required when SSLUseCertificate is enabled. This control specifies the name of a PEM (Privacy Enhanced Mail) encoded certificate to be used during SSL handshakes. SSLPublicCertFile may be specified as a relative path.
Applies to execute.
SSLUseCertificate is optional. When enabled, (i.e. 'Y') it instructs the application to provide client side certificate authentication, if necessary.
Applies to execute.
SSLVerifyPeerCert is optional. When enabled, (i.e. 'Y') it instructs the application to verify the credentials of the peer server.
Applies to execute.
TempDirectory is optional. This control specifies a directory in which all temporary files are to be created. If not specified, TempDirectory defaults to /tmp (\temp for NT/2K). TempDirectory may be specified as a relative path.
Applies to execute.
URLAuthType is optional. It identifies what authentication scheme to use when issuing HTTP/HTTPS requests. The value specified by this control applies to any requests involving URLGetURL or URLPutURL. When URLAuthType is set to basic, user credentials are base 64 encoded and incorporated into the request header. User credentials specified in the URL take precedence over credentials specified in the URLUsername and URLPassword controls. If not specified, URLAuthType defaults to none.
Applies to execute.
URLDownloadLimit is optional. The value of this control specifies how much data the client is willing to receive for any given HTTP/HTTPS request. If URLDownloadLimit is not set or is set to zero, no limit will be imposed.
Applies to execute.
URLGetURL is required. It defines the scheme, user credentials, host address, port, and CGI application to be used when making requests. If a username/password pair is specified in the URL, that pair takes precedence over the values specified by URLUsername/URLPassword, if any. URLs must use a scheme of http or https and satisfy the following regular expression:
scheme://(user(:pass)?@)?host(:port)?/(path(\?query)?)?
Applies to execute.
URLPassword is optional. It identifies the password to use when accessing a remote Web server. The value specified by this control is used in conjunction with URLGetURL and URLPutURL unless those controls supply their own username/password pair.
Applies to execute.
URLPutURL is optional. It defines the scheme, user credentials, host address, port, and CGI application to be used when making PUT requests. If a username/password pair is specified in the URL, that pair takes precedence over the values specified by URLUsername/URLPassword, if any. In any event, user credentials are only sent when basic authentication has been requested (See URLAuthType). URLPutURL uses the same syntax as URLGetURL.
Applies to execute.
URLUsername is optional. It identifies the username to use when accessing a remote Web server. The value specified by this control is used in conjunction with URLGetURL and URLPutURL unless those controls supply their own username/password pair.
Applies to execute.
UnlinkExecutable is optional. When enabled (i.e. 'Y'), the downloaded executable is deleted before the program exits.
Applies to execute.
UnlinkOutput is optional. When enabled (i.e. 'Y'), any output files are unlinked before the program exits. This control has no effect when output data is directed to stdout/stderr.
Upon successful completion, a value of XER_OK (i.e. 0) is returned. If the program encountered a critical error and had to abort immediately, the value XER_Abort (i.e. 1) is returned. If the command line does not conform to the required syntax, a value of XER_Usage (i.e. 2) is returned. Otherwise, one of the following error codes is returned. These codes indicate which subsystem encountered the fatal error.
XER_BootStrap, Value = 3
XER_ProcessArguments, Value = 4
XER_ReadProperties, Value = 5
XER_CheckDependencies, Value = 6
XER_Configure, Value = 7
XER_GetStage, Value = 8
XER_RunStage, Value = 9
XER_PutStage, Value = 10
The config file specified on the command line contains a set of controls that are used to configure WebJob's runtime behavior. For more details on what controls may be specified, see the CONFIGURATION CONTROLS section of this document.
The program specified on the command line is retrieved from the Web resource and written to the local file system. The exact location varies depending on the value of TempDirectory. When UnlinkExecutable is enabled, the program is deleted before WebJob exits. However, if the WebJob aborts, this file may not be deleted.
The sequence 'stdxxx' represents an output stream where 'xxx' is one of {out|err|env}. The sequence 'hhhhhhhh' is a random hex extension. When URLPutURL has been set, the sub-process writes its data to the provided stdout/stderr streams. Once the sub-process has finished, WebJob creates a third stream, stdenv, which contains information regarding environment settings and exit status. When UnlinkOutput is enabled, these files are deleted after they have been uploaded and before WebJob exits. However, if the WebJob aborts or the upload fails, these files may not be deleted.
When SSLUseCertificate has been enabled, WebJob expects to find certificate and key files in the locations specified by SSLPublicCert and SSLPrivateKey controls, respectively. If these files do not exist or have the proper format, WebJob will abort.
When SSLUseCertificate has been enabled, WebJob expects to find a bundled certificate authorities file in the location specified by SSLBundledCAs control. If this file does not exist or have the proper format, WebJob will abort.
The following examples are intended to demonstrate different ways of configuring and using WebJob. Any text encapsulated between '--- XXX ---' delimiters represents config file content.
This example demonstrates how to download and run ps(1) with
output being directed to stdout/stderr. It is assumed that a Web resource
has already been configured to serve the appropriate ps(1)
binary for the system in question. Also, assume that the following
information has been provided.
URL = http://trusted.server.net/cgi-client/nph-webjob.cgi
Username = client
Password = access
The simplest configuration file that will carry out this task is as follows:
--- example1.cfg ---
URLGetURL=http://client:access@trusted.server.net/cgi-client/nph-webjob.cgi
URLAuthType=basic
--- example1.cfg ---
or, alternatively
--- example1.cfg ---
URLGetURL=http://trusted.server.net/cgi-client/nph-webjob.cgi
URLUsername=client
URLPassword=access
URLAuthType=basic
--- example1.cfg ---
Once the config file has been created, WebJob may be invoked as follows:
webjob -e -f example1.cfg ps -aux
or
webjob -e -f - example1.cfg ps -aux < example1.cfg
One item to note is that WebJob will store the downloaded program in the directory specified by TempDirectory. In this case, TempDirectory hasn't been specified, so data will be written to the default directory (i.e. /tmp). To automatically remove this file after program completion, enable the UnlinkExecutable control.
Suppose that you want to direct the output of ps(1) to a Web
resource instead of stdout/stderr. Given that that the URL from example one
may be used for both GET and PUT requests, the config file becomes:
--- example2.cfg ---
URLGetURL=http://trusted.server.net/cgi-client/nph-webjob.cgi
URLPutURL=http://trusted.server.net/cgi-client/nph-webjob.cgi
URLUsername=client
URLPassword=access
URLAuthType=basic
RunType=snapshot
--- example2.cfg ---
This job may be started as follows:
webjob -e -f example2.cfg ps -aux
One item to note is that WebJob will create three files of the form webjob.stdxxx.hhhhhhhh where xxx is {out|err|env} and hhhhhhhh is a random hex extension. The exact location of the files depends on the value of TempDirectory. These files contain the output data that will be uploaded to the Web resource in a PUT request. To automatically remove these files after they have been successfully uploaded, enable the UnlinkOutput control.
Suppose that you want to run netstat(1) over a secure,
authenticated channel. Further, suppose you also want to delete the
downloaded
program and its associated output files upon job completion. Assuming that the Web
resource is properly configured, then the information provided below is
sufficient to create the following config file.
GET/PUT URL = https://secure.server.net:443/cgi-client/nph-webjob.cgi
SSL Public Cert File = /usr/local/integrity/ssl/mycrt.pem
SSL Private Key File = /usr/local/integrity/ssl/mykey.pem
SSL Bundled CAs File = /usr/local/integrity/ssl/CAs.pem
SSL PassPhrase = flimflam
Certificate Chain Length = 2
Web Resource Validates Client Certificates = Y
--- example3.cfg ---
URLGetURL=https://secure.server.net:443/cgi-client/nph-webjob.cgi
URLPutURL=https://secure.server.net:443/cgi-client/nph-webjob.cgi
URLUsername=client
URLPassword=access
URLAuthType=basic
RunType=snapshot
UnlinkOutput=Y
UnlinkExecutable=Y
SSLUseCertificate=Y
SSLPublicCertFile=/usr/local/integrity/ssl/mycrt.pem
SSLPrivateKeyFile=/usr/local/integrity/ssl/mykey.pem
SSLPassPhrase=flimflam
SSLVerifyPeerCert=Y
SSLBundledCAsFile=/usr/local/integrity/ssl/CAs.pem
SSLExpectedPeerCN=secure.server.net
SSLMaxChainLength=2
--- example3.cfg ---
This job may be started as follows:
webjob -e -f example3.cfg netstat -an
openssl(1)
Klayton Monroe, klm@ir.exodus.net
WebJob was initially written to assist Incident Response Handlers in their efforts to investigate potentially compromised systems. Often, these Handlers must work around the constraints imposed by the surrounding environment. For example, lack of physical or shell access, untrusted diagnostic programs, lack of encryption, many machines in need of investigation, etc. Therefore, I felt that Handlers, or their eyes and ears in the field, needed an efficient way to import and run known good diagnostic tools when investigating live systems. WebJob is lightweight, portable, and easy to use. This makes it a good candidate for establishing a foothold on the target system. Once there, all that is needed to begin diagnostics is a small configuration file and remote access to a secure Web resource. In the case where there are many machines to investigate, WebJob can be deployed in parallel. What's more, it may be the case that all output can be directed to and aggregated on a central Web resource. This can significantly reduce the amount time, effort, and resources needed to arrive at a determination.