StandOut - The Flexible Output Object

Logging and Output Control

Author: Michael Foord
Contact: fuzzyman@voidspace.org.uk
Version: 2.1.0
Date: 2005/01/06
License:BSD License [1]
Online Version:standout online
Support:Mailing List

StandOut Manual

Introduction

standout is a module that provides a single class StandOut - the flexible output object. It provides a simple way of adding logging to a program, and a way of adding verbosity controls.

By assigning a priority level to each message, it makes it trivially easy to allow your users to choose their own verbosity level. StandOut works with programs that use stdout or with GUI applications.

Alternatively you can just use StandOut for logging stdout and stderr to a file.

This module is a part of the pythonutils [2] package. Many of the modules in this package can be seen at the Voidspace Modules Page or the Voidspace Python Recipebook.

Downloading

As well as being included in the pythonutils package, you can download standout directly from :

StandOut for Logging

The most basic use of StandOut is for logging output to a file. You set it up to log stdout and/or stderr and give it a filename - and that's it. Because it intercepts the normal streams you don't need to do anything in your program to use it (it transparently logs the normal output of print statements and error messages).

Just close the output objects to restore normality and close your log file.

from standout import StandOut
stout = StandOut(filename='log.txt')
stout_err = StandOut(stream='error', share=True)

try:
    # body of program
    main()
finally:
    # these close the log file
    stout.close()
    stout_err.close()

If we add the above code to our program (obviously replacing the main() call with the appropriate thing) then everything printed (to sys.stdout) and tracebacks to sys.stderr will be logged to file. Everything on sys.stderr will have an [err] prefix added to each line.

What it does is create two instances of StandOut. The first one works on sys.stdout (the default behaviour) and tells it to use the filename log.txt.

The second instance (stout_err`) works with the error stream (using the stream='error' keyword argument). We also tell it to share the log file with the first instance - through the share=True keyword argument.

We keep the references to these two objects around so that we can explicitly close them when our program has finished. This closes the log file.

Uncaught Exceptions

There is a problem with the above code though. Tracebacks are normally only printed to sys.stderr when there is an uncaught exception. With the above code an uncaught exception would be handled by the try:...finally: block.

This means out output objects would be closed - and then the final exception re-raised. What if we want to log this exception to our file as well ?

There is a neat trick using traceback.print_exc and StringIO that allows us to print the traceback from an exception.

The following code catches all unhandled exceptions and prints the traceback to sys.stderr (which then gets logged). It also allows you to exit with a keyboard interrupt. Only after this are the output objects closed.

from StringIO import StringIO
from traceback import print_exc
from standout import StandOut

stout = StandOut(filename='log.txt')
stout_err = StandOut(stream='error', share=True)

try:
    # do the business
    main()
except KeyboardInterrupt:
    sys.stderr.write('Exited by Keyboard Interrupt.\n')
except Exception, e:
    # print any error without bombing out
    # (so we can display it, then close our files nicely)
    f = StringIO()
    print_exc(file = f)
    # write error to sys.stderr rather than printing to sys.stdout
    sys.stderr.write(f.getvalue() + '\n')
#
stout.close()
stout_err.close()

Verbosity and Priority

StandOut allows you to assign messages a priority level. You can give messages an individual priority level - and set general priority levels for all messages.

StandOut also has a verbosity setting. Only messages that have a priority equal or greater than (>=) the verbosity level are actually displayed.

This makes it is easy to implement several degrees of verbosity in your program [3].

Verbosity and priority levels vary from one to nine.

A verbosity of one means that every message is printed. A verbosity of nine means that only messages with a priority of nine are printed.

Verbosity level zero is special - it switches off printing altogether.

You set a verbosity level when you create the object by passing in the verbosity keyword argument. You an adjust it later by altering the verbosity attribute of your object.

from standout import StandOut

stout = StandOut(verbosity=5)
# now verbosity is 5

print stout.verbosity
5

stout.verbosity = 1
# guess what it is now :-)

The default verbosity and the default verbosity are both 5. This means that all messages will be printed.

Setting Priority of Messages

There are three ways of setting the priority of messages. The priority attribute, as a prefix to the message, or by using the write method of our output object.

To change the priority level of messages you can set the priority attribute or print a priority marker. A priority marker looks like &priority-n;, where 'n' is the priority level.

from standout import StandOut

stout = StandOut()
print stout.priority
5

stout.priority = 6
print stout.priority
6

print '&priority-5;'
print stout.priority
5

Printing a priority marker does not result in anything being printed - it sends a message to the output object. This means you can completely control your output object from print commands.

You can also set the priority level of a single message by prefixing the line with a priority marker, or by using the write method of the output object. The write method takes an optional second argument - an integer value which is the priority level of this message.

from standout import StandOut

stout = StandOut(verbosity=6)
print stout.priority
5
print stout.verbosity
6

stout.write('This won't get printed\n, 5)
print '&priority-4;Nor will this'
stout.write('But this will\n', 6)
But this will
print '&priority-7;And so will this'
And so will this

If for any reason you want to actually print a &priority-n marker at the start of a line then you can escape it with a &priority-e; : print '&priority-e;&priority-1;' actually prints : &priority-1;.

Caution!

Normal print statements make two calls to sys.stdout.write. Once for the text you are printing and another for the trailing '\n' or ' ' that the print statement always emits.

StandOut captures this, to make sure the trailing 'n' or ' ' is printed at the same priority as the original line. This means you mustn't use stout.write(line) with a priority marker.

This is because stout.write(line) only makes one call, not two. Either call stout.write(line, priority) to set a priority for that line - or set stout.priority directly.

Passing in a Print Function

What we've heard so far is fine if you are using normal print statements. What if you want to use StandOut in conjunction with a GUI application ? StandOut makes it possible to output to a GUI window using normal print statements.

You do this by passing in a function that actually does the printing. This should take a single argument - the string to be printed. Anything that is sent to stdout will also be passed to this function.

You can either pass in the function when you create your instance (using the keyword argument print_fun) or you can set it later using the set_print method. This means that the two code snippets show below are equivalent.

def print_function(inline):
    """
    If you define this function inside a GUI app, you will
    have access to it's namespace.
    """

    #trivial example
    window.output(inline)

#example 1
stout = StandOut(print_fun=print_function)

#example 2
stout = StandOut()
stout.set_print(print_function)

Using either technique, any output using the print command (or stout.write) will be printed to stdout and passed to your print function.

What if you want to switch off output to stdout altogether ? Or perhaps you only want to emit critical messages on stdout - with most messages being displayed in the GUI (and everything being logged to file).

Well, you can set different verbosity levels for stdout, your print function, and the log file.

Controlling Verbosity

You can set different verbosity levels for stdout, your print function, and the log file. This means that by setting a priority for a message you can control which of your output streams it appears on.

We've already seen how the stdout verbosity is controlled by the verbosity attribute.

The file stream is controlled through the file_verbosity attribute (or keyword argument).

The print function verbosity is controlled through the printfun_verbosity keyword and attribute.

The setall method is a quick way of changing the verbosity for all three output methods.

stout.setall(verbosity)

Setting verbosity, file_verbosity, or printfun_verbosity to 0 disables that ouput method.

Keyword Arguments

This section has a full list of keyword arguments - along with an explanation of their function. The values shown are the default values.

The keyword arguments all map to attributes of the same name on StandOut instances. Most of these you can modify directly to control the behaviour of your instance.

priority

priority = 5

This sets the priority for all messages. If priority is 5 - then only output methods with a 'verbosity' of 5 or lower will display them.

This value can later be set by adjusting the stout.priority attribute or using the priority markers.

You can also set the priority of individual messages through the stout.write method or using the priority markers.

verbosity

verbosity = 5

This is the verbosity level for messages to be printed to stdout. If the verbosity is 5 then only messages with a priority of 5 or higher will be output.

You can adjust this using stout.verbosity.

filename

filename = None

If you pass in a filename when you create the object it will be used as a logfile. It has it's own 'verbosity' level called 'file_verbosity'.

If you don't pass in a filename, you can later add one by setting stout.filename.

Changing stout.filename after you have already set one is a bad thing to do Evil or Very Mad .

file_verbosity

file_verbosity = 5

This is the verbosity level of the log file.

Only messages with a priority higher than this will be sent to the logfile.

print_fun

print_fun = None

If you pass in a function (that takes one parameter - the line to be printed) this will be used to print as well. This could be used for displaying to the output window of a GUI, for example - or for custom formatting of the output.

The function isn't stored at stout.print_fun - this value is just set to True to say we have a function.

If you want to pass in a function after object creation then use the stout.set_print(function) method.

..warning:

You musn't have print statements in your function or you will get stuck in
a loop. Call ``stout.output.write(line)`` instead.

printfun_verbosity

printfun_verbosity = 5

Any function you pass in also has it's own verbosity setting - printfun_verbosity.

stream

stream = 'output'

By default StandOut will divert the sys.stdout stream. Set to 'error' to divert sys.stderr.

Note

If you want to divert both, you will need to create two objects - using the share keyword for the second. See the examples in StandOut for Logging.

share

share = False

You can divert both sys.stdout and sys.stderr. You can also log both to the same file. Set a filename for your sys.stdout object and set share = True for your sys.stderr object.

Any lines sent to sys.stderr will have a prefix attached to them. See error_marker.

error_marker

error_marker = '[err] '

This is the marker put before every line logged from sys.stderr.

It only applies if share is on - this means both streams are logged to the same file.

Methods

close

stout.close()

When your program has finished with the obejct it should call stout.close() this restores the normal output streams and closes any logfile we have been using.

write

stout.write(line, priority=None)

This can be used as an alternative way of specifying a priority for an individual line. It leaves stout.priority unaffected.

Calls to stout.write must have 'n' at the end if you want it to end with a newline.

If you don't specify a priority then it behaves like sys.stdout.write would.

Don't use priority markers with this method, and you can't use priority = 0 (the priority setting will be ignored).

set_print

stout.set_print(function)

This is used to pass in an additional printing function after the object has been created.

setall

stout.setall(verbosity)

This is a quick way of changing the verbosity for all three output methods.

Notes

Setting verbosity, file_verbosity, or printfun_verbosity to 0 disables that ouput method.

Setting priority to 0 switches off all output.

If you want to print to stdout directly and bypass the stout object for any reason - it is saved at stout.output.

Calls to stout.output.write(line) have the same effect that sys.stdout.write(line) would have had.

CHANGELOG

See the source code for CHANGELOG details.

Footnotes

[1]Online at http://www.voidspace.org.uk/python/license.shtml
[2]Online at http://www.voidspace.org.uk/python/pythonutils.html
[3]See the module for examples of this in action.

Note

Rendering this document with docutils also needs the textmacros module and the PySrc CSS stuff. See http://www.voidspace.org.uk/python/firedrop2/textmacros.shtml


Certified Open Source