Guideline: Testing a Method Asset
This guideline describes how to test method assets created with Method Composer.
Main Description

1.0 Introduction

Testing method assets is important because errors tend to build up over time.  Some reasons for errors are:

  • parallel work on method assets that have dependencies
  • forgetting to make updates to all dependent plug-ins
  • inexperienced method authors

Before releasing a method asset, there are a number of types of tests that should be executed.  These are described in the following sections.

2.0 Tools

The following tools can be helpful for testing.

  • IBM(R) Rational(R) Method Composer - adds features useful for testing that aren't available in EPF Composer, such as BIRT reporting and spell checking.
  • Funduc Search and replace - excellent tool for global search and replace
  • Parasoft(R) WebKing(R) - checks for accessibility, spelling
  • Adobe(R) Dreamweaver(R) - fully-featured HTML editor
  • Scooter Software Beyond Compare - tool for comparing files and directories
  • IBM Automated Content Testing - scripts that check for a variety of issues and inconsistencies.  This is part of the Rational Method Authoring Quality Assessor packaged service asset.

3.0 Orphans/Invalid Files/Dependencies

Invalid files

Search and delete files left by Windows or the version control system.

*.keep - IBM(R) Rational(R) ClearCase(R)

thumbs - Microsoft(R) Windows(R)

Also, many file types are difficult to translate, and so we try to standardize on jpg for graphics and keep all text in xmi or xml formats.

IBM Automated Content Testing provides a script "Test for File Types" which lists non-standard file types.

All elements are categorized

Browse the "all practices" configuration for disciplines and domains and confirm that the "uncategorized" category is empty.
Work product slots are an exception.

Orphan files

Make a copy of the library and delete all packages, custom categories, processes. Do not delete the plug-ins, as that will delete all files including orphans.  Any remaining xmi files must be orphans, and should be removed from ClearCase. Caution – get the team to confirm before doing the delete – the problem may be that the file should be added back in rather than be removed.

Orphan resource files

Try RMC's "Edit>Clean Resources" in a copy of the library, then use beyond compare to determine what changes were made.

All tasks and work products have a responsible role

Browse the published web site looking at each role and work product.  Can create BIRT reports to detect this.

IBM Automated Content Testing includes a report "Check for Primary Performer" that reports on tasks that have no primary role and work products that have no responsible role.

All tasks have an input and output work product

Strictly speaking, a task might not have any input work products, but generally if a task is missing inputs and/or outputs, it is usually an oversight.  Browse the published web site looking at each task.  Can create BIRT reports to detect this.

IBM Automated Content Testing includes a report "Check for Input and Output Artifact" that reports on tasks that are missing inputs and outputs.

Unreferenced core elements

Every element in "core" should be referenced by at least once practice in the library.
A BIRT report can be used for this.  Unused elements should be moved out of the release.

External broken links

Choose the publishing option to check external hyperlinks.

Dependency Testing

Publish each practice on its own (can use the RMC Process Builder).

Confirm that there are no broken link errors (there may be errors if, for example, there is a link to a guideline, but that guideline is not explicitly attached to any elements in the plug-in).

Create a configuration that has no practices, just Core.  Confirm that there are no errors (that no core plug-ins depend on practice plug-ins).

Confirm that there are no unnecessary dependencies.  Publish a configuration with all practices, but no assign plug-ins or definition plug-ins for those assignments.

4.0 Method composer error reports and logs

Publication errors

Fix or explain every error in the publication log for each of the available configurations.

Configuration errors

Fix or explain every error and warning in the error log tab for each of the available configurations.

Library problem view

Fix or explain every error in the library problem view.
- open every configuration to make sure errors are loaded

Library validation

Enable Preferences/Method/Authoring/Debug/Enable method library validation.

Run "Validate Method Library".

View the logs and correct any errors.

5.0 Regression testing

When upgrading to a new version of Method Composer, check that contributes, extends, replace work as expected.  Find an example in the library, confirm it works as expected.

Use Beyond Compare to list the differences in the published sites. Confirm that the changes were expected.

6.0 Standards

The tests below describe how to test and correct the most common violations of authoring standards.

File naming conventions

Use windows search and scan for mixed-case file names, spaces in filenames, or long filenames.

Bad HTML

Microsoft fonts: Search for MsoNormal, Font-family, and Microsoft to detect Microsoft word originated HTML. Use word cleanup macro in Method Composer, Dreamweaver and manual means to clean it up.

Extra fonts on links (inserted by old version of Method Composer)

Search for font color="#0000ff and check if is part of a link. If lots, use search/replace script to do a global replace.

Bad character testing

Certain non-ASCII characters cause problems with some browsers and translation tools.  The most common problem characters are curly quotes and bullets inserted when pasting from Microsoft(R) Word(R).

Use search and replace – search for [~-ΓΏ]. For any unexpected characters, replace with correct characters.

  • curly single and double quotes - replace with straight quotes typed from the keyboard (" and ')
  • bullets and dash/emdash - replace with hyphen
  • guillemets - replace with double angle brackets << >>>

Remember that the XMI files contain escaped HTML, so be careful with characters that are defined with an ampersand, such as trademarks - in XMI you may need an additional ampersand.

Brief descriptions

Run a BIRT report to extract brief descriptions.  Everything except checklists should have brief descriptions.

Project and directory names

When using workspaces, it is easy for plug-in name, plug-in folder name, and plug-in project names to become inconsistent.  Use search-replace to find all plug-in names and project names.

  • search for methodplugin*name in all plugin.xmi files
  • search for <NAME> tags in all .project files

Copy results into a text file (including filenames), remove non-name text, sort in Excel, and look for differences between directories, project names, and plug-in names.

Version and copyright information

Every practice and the library as a whole should have a "release info" page that describes the latest release.

Confirm that these pages are included in the published configurations' views

  • In EPF Composer - you need to check the views by hand
  • In Rational Method Composer  - you need to check that every "release_info" page is tagged

Confirm that every plug-in has the appropriate copyright (open source has EPF copyright, company owned content has company copyright), and confirm it publishes correctly.

Confirm that licensing/legal information is attached and publishes.

Accessibility

Use Parasoft WebKing to check compliance with accessibility standards.

Spelling/Grammar/Readiness for translation

Spell check is provided by Rational Method Composer on a per element and per plug-in basis.  You can also check an entire library using WebKing.

IBM Automated Content Testing also has a script "Style Search" that checks for common style problems, such as words that are difficult to translate.

Content that is intended to be translated has to go through additional checks in order to ensure that it will work with translation memory tools.

For example, IBM uses IBM internal tools - SEGCHK to check for grammar errors, and CHKPII for bad characters, prior to translation. Commercial translation tools will have their own validation tools.

Browser Compatibility

Open up a published configuration in the supported browsers and compare each kind of page. A work product, a guideline, a checklist, a task and an activity with an activity diagram and work breakdown structure. Confirm that they look reasonably similar - that no odd fonts are appearing, that text is readable, links are working, and so on.

Then expand your screen to as large as possible (to allow maximum viewing of each page) and use the keypad to open each page in the treebrowser and confirm that nothing strange appears (missing graphics, corrupted HTML, and so on).

Publication options

Review the publication options, and confirm that they are correct.

Make sure the feedback for each configuration is directed to the right location, and works after publishing.

Synchronization

Confirm that processes that require synchronization have been synchronized after any and all method content was finalized.

Index, glossary, search

Confirm that the index, glossary, and search all work.

Supporting plug-ins

Confirm that the "is_supporting" flag has been used correctly on core plug-ins. Publish a configuration that only has a few practices, and look for additional elements that don't belong, such as unused roles, unused tasks, and unused work products.

Locking

Confirm that the released library has all plug-ins locked so that users that extend the library do no inadvertently modify them.

7.0 Rational Method Composer-only Tests

The following tests do not apply to EPF Composer.

Process Builders

Confirm that process builders work correctly.
- all practices are presented in the builders
- every practice has a brief description that is readable in the builder, and matches the brief description for the practice.

Confirm that the "all practices" configuration matches what is displayed by the IBM practices builder.  (The builder may do further decomposition to identify alternative practices).

Orphaned tags

Unused tags should be found and removed.