Task: Prepare For Vendor/Package Evaluation Workshop
This task focuses on preparing the Vendor and Package Evaluation Workshop.
Disciplines: Requirements
Purpose
  • To administer and facilitate all preparation for an evaluation workshop where a vendor will demonstrate their COTS package capabilities
  • To create a detailed workshop agenda and schedule for each package vendor to follow during the demonstration their COTS package's capabilities
  • To produce a vendor evaluation score sheet to be used by the evaluators to score the vendor's COTS package demonstration
Relationships
RolesPrimary Performer: Additional Performers:
InputsMandatory:
  • None
Optional:
    Outputs
      Steps
      Develop Evaluation Workshop Description

      The Vendor and Package Evaluation Workshop Description establishes the scope, priorities, and topic flow for the evaluation workshop criteria.

      Each topic within the agenda will have a listing of questions and issues to be covered. The list of detailed issues to be covered should be consistent with the time allotted to this subject area. The issues and questions will be presented on an Evaluation Score Sheet where the evaluation team members will be able to record their impressions as numeric scores and as comments. The vendors will be expected to cover each question or issue on the scoring sheet to the best of their ability.

      A "Confidentiality Statement" should be provided with the Vendor and Package Evaluation Workshop Description . This agreement protects the acquiring organization since the Vendor and Package Evaluation Workshop Description may include proprietary material.

      The demonstration's rating process should be described for the vendors in the Vendor and Package Evaluation Workshop Description. There should also be an explanation on how questions will be handled. Other assumptions and rules with regard to the demonstration should be clarified.

      A request for a "Reference List" may be included with the Vendor and Package Evaluation Workshop Description along with other vendor evaluation requirements. The Vendor and Package Evaluation Workshop Description is the artifact, which is provided to the vendor finalist.

      The Vendor and Package Evaluation Workshop Description is delivered to the COTS package vendors upon notification of participation in the candidate solution evaluation process. The Vendor and Package Evaluation Workshop Description should be delivered to the vendors on a timely basis to allow maximum preparation time and to enable each vendor to have the same notification of demonstration content.

      Create Evaluation Workshop Schedule

      A Vendor and Package Evaluation Workshop Description should include a detailed workshop agenda and schedule for each vendor to follow during the presentation of their COTS packages' capabilities. The dates and a detailed workshop agenda and schedule should be specified for each functional area being demonstrated by the vendors in an order defined by the Acquirer, to ensure that the critical use cases and critical nonfunctional requirements are covered. The timed agenda is important to pace the vendors during the workshop to ensure that all areas being scored are viewed.

      Produce Evaluation Score Sheet

      The score sheet is used by the evaluators to score the vendor's COTS package demonstration for each item defined in the Vendor and Package Evaluation Workshop Description. The score sheet facilitates a structured step-by-step flow through the vendor's demonstration of their COTS packages' capabilities.

      Include in the score sheet the evaluation workshop agenda and schedule as a reference for both the evaluator and the vendor. Its purpose is to remind both groups of the amount of time allotted for each demonstration area. Usually, there is less time than there is function to demonstrate. Consequently, the schedule is included so all are aware of the amount of time that the vendor has to demonstrate their COTS packages' capabilities in a functional area.

      List in the score sheet the evaluators that are to participate in a given area. It is important to track the participation of these evaluators so that the same evaluators will score all vendors. The same evaluators must participate to maintain an objective score. The participants are listed to provide an easy way to track the required evaluators.

      List in the score sheet time allotted for each functional area. To obtain objective data, each vendor must have the same time to demonstrate their product. The suggested time is listed so the vendor, with the facilitator's help, can keep the demonstration on schedule.

      Include a sequential numbering system for process area and functional area to help the vendor and evaluators identify scripted areas being shown. The vendor will use these numbers as a reference, so the evaluators will know what function is being demonstrated.

      Include item descriptions that identify each function that is being scored. These descriptions are contained in the Vendor and Package Evaluation Workshop Description and based on the critical use cases and critical nonfunctional requirements. The listed item descriptions will ensure that an evaluator's score reflects how the demonstrated COTS package supports these requirements.