1.0 Introduction
Testing method assets is important because errors tend to build up over time. Some reasons for errors are:
-
parallel work on method assets that have dependencies
-
forgetting to make updates to all dependent plug-ins
-
inexperienced method authors
Before releasing a method asset, there are a number of types of tests that should be executed. These are
described in the following sections.
2.0 Tools
The following tools can be helpful for testing.
-
IBM(R) Rational(R) Method Composer - adds features useful for testing that aren't available in EPF Composer, such
as BIRT reporting and spell checking.
-
Funduc Search and replace - excellent tool for global search and replace
-
Parasoft(R) WebKing(R) - checks for accessibility, spelling
-
Adobe(R) Dreamweaver(R) - fully-featured HTML editor
-
Scooter Software Beyond Compare - tool for comparing files and directories
-
IBM Automated Content Testing - scripts that check for a variety of issues and inconsistencies. This is part
of the Rational Method Authoring Quality Assessor packaged service asset.
3.0 Orphans/Invalid Files/Dependencies
Invalid files
Search and delete files left by Windows or the version control system.
*.keep - IBM(R) Rational(R) ClearCase(R)
thumbs - Microsoft(R) Windows(R)
Also, many file types are difficult to translate, and so we try to standardize on jpg for graphics and keep all text in
xmi or xml formats.
IBM Automated Content Testing provides a script "Test for File Types" which lists non-standard file types.
All elements are categorized
Browse the "all practices" configuration for disciplines and domains and confirm that the "uncategorized" category
is empty.
Work product slots are an exception.
Orphan files
Make a copy of the library and delete all packages, custom categories, processes. Do not delete the plug-ins, as that
will delete all files including orphans. Any remaining xmi files must be orphans, and should be removed from
ClearCase. Caution β get the team to confirm before doing the delete β the problem may be that the file should be added
back in rather than be removed.
Orphan resource files
Try RMC's "Edit>Clean Resources" in a copy of the library, then use beyond compare to determine what changes were
made.
All tasks and work products have a responsible role
Browse the published web site looking at each role and work product. Can create BIRT reports to detect this.
IBM Automated Content Testing includes a report "Check for Primary Performer" that reports on tasks that have no
primary role and work products that have no responsible role.
All tasks have an input and output work product
Strictly speaking, a task might not have any input work products, but generally if a task is missing inputs and/or
outputs, it is usually an oversight. Browse the published web site looking at each task. Can create BIRT
reports to detect this.
IBM Automated Content Testing includes a report "Check for Input and Output Artifact" that reports on tasks that are
missing inputs and outputs.
Unreferenced core elements
Every element in "core" should be referenced by at least once practice in the library.
A BIRT report can be used for this. Unused elements should be moved out of the release.
External broken links
Choose the publishing option to check external hyperlinks.
Dependency Testing
Publish each practice on its own (can use the RMC Process Builder).
Confirm that there are no broken link errors (there may be errors if, for example, there is a link to a guideline, but
that guideline is not explicitly attached to any elements in the plug-in).
Create a configuration that has no practices, just Core. Confirm that there are no errors (that no core plug-ins
depend on practice plug-ins).
Confirm that there are no unnecessary dependencies. Publish a configuration with all practices, but no assign
plug-ins or definition plug-ins for those assignments.
4.0 Method composer error reports and logs
Publication errors
Fix or explain every error in the publication log for each of the available configurations.
Configuration errors
Fix or explain every error and warning in the error log tab for each of the available configurations.
Library problem view
Fix or explain every error in the library problem view.
- open every configuration to make sure errors are loaded
Library validation
Enable Preferences/Method/Authoring/Debug/Enable method library validation.
Run "Validate Method Library".
View the logs and correct any errors.
5.0 Regression testing
When upgrading to a new version of Method Composer, check that contributes, extends, replace work as expected.
Find an example in the library, confirm it works as expected.
Use Beyond Compare to list the differences in the published sites. Confirm that the changes were expected.
6.0 Standards
The tests below describe how to test and correct the most common violations of authoring standards.
File naming conventions
Use windows search and scan for mixed-case file names, spaces in filenames, or long filenames.
Bad HTML
Microsoft fonts: Search for MsoNormal, Font-family, and Microsoft to detect Microsoft word originated HTML. Use word
cleanup macro in Method Composer, Dreamweaver and manual means to clean it up.
Extra fonts on links (inserted by old version of Method Composer)
Search for font color="#0000ff and check if is part of a link. If lots, use search/replace script to do a global
replace.
Bad character testing
Certain non-ASCII characters cause problems with some browsers and translation tools. The most common problem
characters are curly quotes and bullets inserted when pasting from Microsoft(R) Word(R).
Use search and replace β search for [~-ΓΏ]. For any unexpected characters, replace with correct characters.
-
curly single and double quotes - replace with straight quotes typed from the keyboard (" and ')
-
bullets and dash/emdash - replace with hyphen
-
guillemets - replace with double angle brackets << >>>
Remember that the XMI files contain escaped HTML, so be careful with characters that are defined with an ampersand,
such as trademarks - in XMI you may need an additional ampersand.
Brief descriptions
Run a BIRT report to extract brief descriptions. Everything except checklists should have brief descriptions.
Project and directory names
When using workspaces, it is easy for plug-in name, plug-in folder name, and plug-in project names to become
inconsistent. Use search-replace to find all plug-in names and project names.
-
search for methodplugin*name in all plugin.xmi files
-
search for <NAME> tags in all .project files
Copy results into a text file (including filenames), remove non-name text, sort in Excel, and look for differences
between directories, project names, and plug-in names.
Version and copyright information
Every practice and the library as a whole should have a "release info" page that describes the latest release.
Confirm that these pages are included in the published configurations' views
-
In EPF Composer - you need to check the views by hand
-
In Rational Method Composer - you need to check that every "release_info" page is tagged
Confirm that every plug-in has the appropriate copyright (open source has EPF copyright, company owned content has
company copyright), and confirm it publishes correctly.
Confirm that licensing/legal information is attached and publishes.
Accessibility
Use Parasoft WebKing to check compliance with accessibility standards.
Spelling/Grammar/Readiness for translation
Spell check is provided by Rational Method Composer on a per element and per plug-in basis. You can also check an
entire library using WebKing.
IBM Automated Content Testing also has a script "Style Search" that checks for common style problems, such as words
that are difficult to translate.
Content that is intended to be translated has to go through additional checks in order to ensure that it will work with
translation memory tools.
For example, IBM uses IBM internal tools - SEGCHK to check for grammar errors, and CHKPII for bad characters, prior to
translation. Commercial translation tools will have their own validation tools.
Browser Compatibility
Open up a published configuration in the supported browsers and compare each kind of page. A work product, a guideline,
a checklist, a task and an activity with an activity diagram and work breakdown structure. Confirm that they look
reasonably similar - that no odd fonts are appearing, that text is readable, links are working, and so on.
Then expand your screen to as large as possible (to allow maximum viewing of each page) and use the keypad to open each
page in the treebrowser and confirm that nothing strange appears (missing graphics, corrupted HTML, and so on).
Publication options
Review the publication options, and confirm that they are correct.
Make sure the feedback for each configuration is directed to the right location, and works after publishing.
Synchronization
Confirm that processes that require synchronization have been synchronized after any and all method content was
finalized.
Index, glossary, search
Confirm that the index, glossary, and search all work.
Supporting plug-ins
Confirm that the "is_supporting" flag has been used correctly on core plug-ins. Publish a configuration that only
has a few practices, and look for additional elements that don't belong, such as unused roles, unused tasks, and
unused work products.
Locking
Confirm that the released library has all plug-ins locked so that users that extend the library do no inadvertently
modify them.
7.0 Rational Method Composer-only Tests
The following tests do not apply to EPF Composer.
Process Builders
Confirm that process builders work correctly.
- all practices are presented in the builders
- every practice has a brief description that is readable in the builder, and matches the brief description for the
practice.
Confirm that the "all practices" configuration matches what is displayed by the IBM practices builder. (The
builder may do further decomposition to identify alternative practices).
Orphaned tags
Unused tags should be found and removed. |