IBM Rational TestManager Release Notes


Version 2003.06.13
Release Date: September 2004
Last Updated: 08/09/2004
Material ID: GI11-5907-03

© Copyright IBM Corporation. 2000, 2004. All Rights Reserved.

This documentation (the "Work") is protected under the copyright laws of the United States and/or other jurisdictions, as well as various international treaties. Any reproduction or distribution of the Work is expressly prohibited without the prior written consent of IBM Corporation.

Legal Notices Information

The complete listing of IBM Rational Legal Notices is automatically installed with IBM Rational Suite in <Install Path>\Rational\doc\legal_information.html. Optionally, the Legal Notices can be found on the "IBM Rational Solutions for Windows CD-ROM" at \doc\legal_information.html.


Scope

These release notes are accurate at the time this product was prepared for release. The most recent version of this file is available in the IBM Publications Center at http://www.ibm.com/shop/publications/order. Search for "GI11-5907-03" to find the latest version of these release notes in the IBM Publications Center. (To search for the IBM Publications Center, go to http://www.ibm.com and enter "IBM Publications Center" in the Search field.)

The testing products user guides that are translated into Japanese describe the Rational product release 2001A.04.10. You can find these documents on the IBM Rational Solutions for Windows Online Documentation CD-ROM in the "Extras" folder.

Product changes described in this document and contained within this service release apply if you are upgrading from:

If you are upgrading from any of these releases, see the IBM Rational Testing Products Service Release Notes for the defects fixed in this service release and the previous service release.

This release incorporates several enhancements to the IBM(R) Rational(R) TestManager product. These release notes provide information that is not available in the printed documentation or the Help for Rational TestManager.

Contents

Scope

Getting Started

Installation Information and System Requirements

Upgrading Test Datastores

New and Changed Features

New and Changed Features in v2003.06.13

Support for Business Objects Crystal Reports v10

New Extended Log Listing Report Layouts

Tracking of Test Case Progress Using States and Points

Support for VMware GSX Server 3.0.

Multiple Selection of Test Assets

New and Changed Features in v2003.06.12

Exporting and Reporting on Test Log Details

Using TestManager with Virtual Machine Management Software

General Enhancements in v2003.06.12

IBM Rational XDE Tester

New Built-in Test Input Type: Microsoft Excel

Test Impact Analysis

Enhanced Filtering of Test Input Sources

Defining Test Script Options

New Test Scripts View

Defining Test Tool Options

New Actions in the Test Script Console and Test Input Adapters

Test Asset Management and Reporting Enhancements in v2003.06.12

Importing and Exporting Test Assets

Changes to Test Case Reports

Performance Testing Enhancements in v2003.06.12

Enhanced High-Volume Testing

IBM Rational ManualTest Enhancements in v2003.06.12

Undo and Redo

Find and Replace

Step by Step Execution Mode

Exporting Test Assets to Text Files

Specifying the Expected Result

Specifying Result Details

Viewing the New Features in the Log

Executing Manual Scripts via the Web

Deleting Multiple Log Assets

Test Inputs View

Guidelines and Restrictions

Firewall Issues with Microsoft Windows XP Service Pack 2 (SP2)

Unsupported Platforms

Windows Me

Using Microsoft Excel Test Inputs

Submitting Defects in the Test Log

    Logging into a Different Datastore or Project

Data Code Pages: Character Set Compatibility with ClearQuest

Using the Suspicion Feature in TestManager

Ensuring the Performance of the Suspicion Feature

Updating Suspicion

Manually Updating Suspicion

Test Script Playback

VU Test Scripts

ManualTest Test Scripts

Visual Basic Test Scripts

Generated Visual Basic and Java Test Scripts

F5 (Refresh All) Does Not Work in Test Asset Workspace

Resource Monitoring

Report Generation

SAP Limitations

UCM on Windows NT 4.0 with ClearCase 5.0

Editing Reports with Long Path Names (Crystal Reports 8.5)

Performance Testing

ODBC Limitations

Testing CORBA Applications (for VisiBroker 3.3 only)

Issue With Rational Test Agents and NuTCRACKER

Problem with Exporting and Importing Test Case Designs

Problem with Using Double-Byte Japanese Characters

Issues with Extended Log Detail and Extended Log Summary Reports

TestManager and XDE Tester

Crystal Reports Issues

Expected Result Files for ManualTest in Windows XP

 

Corrections to Documentation

Wildcard References in Filter Operator Types

ManualTest Does Not Support Linux

TestManager Extensibility TIClearFilter Function

Contacting IBM Rational Customer Support

 

Getting Started

Installation Information and System Requirements

The IBM Rational Software Desktop Products Installation Guide, IBM Rational Software Server Products Installation Guide, and IBM Rational Suite Upgrade Guide contain complete installation and upgrade instructions and system requirements for Rational products for Windows. They are available electronically as PDF files on the IBM Rational Solutions for Windows Online Documentation CD-ROM, which also includes a version of the Acrobat Reader.

See the IBM Rational Testing Products Release Notes for important information about installing Rational testing products and creating test datastores.

Upgrading Test Datastores

When upgrading to TestManager v2003.06.13, note the following important information related to test datastores:

To upgrade a test datastore to v2003.06.13, perform the following steps:

  1. Make a backup copy of the datastore.
  2. Click Start > Programs > Rational Software > Rational Administrator to open the IBM Rational Administrator.
  3. In the project pane, select the Rational project that contains the test datastore to be upgraded. Right-click the project and click Upgrade Assets.
  4. Click OK to the message that appears reminding you to back up your data.
  5. Click View Log to examine the results of the operation and ensure that it completed successfully. Click Done.

New and Changed Features

New and Changed Features in v2003.06.13

The following list summarizes the new and changed features in v2003.06.13.

Each of these features is described below.

Support for Business Objects Crystal Reports v10

TestManager v2003.06.13 includes the Crystal Reports v10 runtime libraries (DLLs) required to view and run reports in TestManager clients. TestManager installs the files automatically.

However, to author or edit reports, you must purchase a copy of Crystal Reports Professional Edition or other Crystal Editions that contain Crystal Designer v10. For instructions about how to install and license Crystal Reports Professional Edition or other Crystal Editions, contact Business Objects at http://www.businessobjects.com.

Using TestManager with Crystal Reports v10

Test datastores that were created with previous versions of TestManager v2003 (v2003.06.00, v2003.06.01, or v2003.06.12) are compatible with TestManager v2003.06.13. However, to benefit from the new features provided in v2003.06.13, you must upgrade your test datastores to v2003.06.13.

Versions of TestManager 2003 prior to v2003.06.13 use Crystal Dictionary to run listing reports. Reports based on Crystal Dictionary are not supported in Crystal Reports v10, the newly supported Crystal version for v2003.06.13. For this reason, v2003.06.13 provides replacement listing reports that do not use dictionaries. These replacement listing reports were created with Crystal Reports v8.5 to be fully compatible with Crystal Reports v8.5 through v10.

You can continue to use Crystal Reports v8.5 to run old dictionary-based reports and to create and run new reports that do not rely on dictionaries.

New datastores that you create do not include dictionaries. When you upgrade a v2003.06.12 datastore that includes dictionaries, the datastore retains those dictionaries. If you previously modified a listing report in Crystal Reports v8.5 and you now want to edit or run that report in Crystal Reports v10, you must do the following tasks:

  1. Upgrade the report’s test datastore to v2003.06.13, or create a new datastore in v2003.06.13, and
  2. Re-create the report by copying and pasting its modified design elements to the corresponding replacement listing report. (For instructions, see the next section.)

Note: Before you upgrade a test datastore, you must upgrade all clients that use that datastore. If you are not ready to upgrade all clients to v2003.06.13, do not upgrade the test datastore.

Re-creating Reports with Crystal Reports v10

To re-create a listing report that you modified with Crystal Reports v8.5, perform the following steps:

  1. Click Start > Programs > Rational Software > Rational TestManager. In the Rational Test Login dialog box, type the password, select the Rational project, and click OK.
  2. In the test asset workspace, click the Analysis tab to display the reports. Expand the Listing Reports folder to display the listing reports. Find the replacement nondictionary-based listing report that corresponds to the dictionary-based report that you used to create the customized Crystal Reports v8.5 report. For example, if you created a report in v8.5 named Modified Build Detail Report based on the Build Detail Report, find the nondictionary-based Build Detail report. Right-click the nondictionary-based report and click Open.
  3. In the Listing Reports dialog box, click New. Type a name for the new report and click OK. Crystal Reports v10 opens the report for editing.
  4. Click Start > Programs > Crystal Reports to start Crystal Reports v8.5.
  5. In the Welcome to Crystal Reports dialog box, select Open an Existing Report and select the report from the list. The report opens in the Crystal Reports v8.5 editor.
  6. In the Crystal Reports v8.5 editor, select and copy the design elements that you modified.
  7. In the Crystal Reports v10 editor, paste the design elements into the new report.
  8. Click File > Save to save the re-created report. Click File > Close to end the editing session.
  9. In v2003.06.13, the structure of some listing reports has changed such that some assets have been renamed or moved to new tables. For example, the Build Details report now includes two tables, Build and Iteration. In previous versions of TestManager, the Iteration field identifies an iteration name. In v2003.06.13, the Iteration field in the Build table identifies a UID. The iteration name is provided by the Name field in the new Iteration table. To determine if this type of problem exists with your report, right-click the report in the TestManager test asset workspace and click Run. Examine the report for any unexpected field data. If a problem exists, continue with the next step. If no problems exist, you are done with this report and you can begin re-creating your remaining reports.
  10. Open the new report in Crystal Reports v10. Click View > Field Explorer to display the report’s tables and field definitions.
  11. In general, to resolve a problem with an asset, go to that asset’s table. For example, to resolve the Iteration field problem go to the Iteration table and use its Name field to replace your report’s Iteration field.
  12. Save the changes to the report and run it again to verify that you have solved the problem.
Uninstalling Crystal Reports v8.5 Runtime

By default, TestManager v2003.06.13 uses the Crystal Reports v8.5 runtime to run your listing reports. After you re-create all modified reports to be compatible with Crystal Reports v10, uninstall the Crystal Reports v8.5 runtime so that TestManager v2003.06.13 uses the Crystal Reports v10 runtime to run your reports.

New Extended Log Listing Report Layouts

The following new report layouts are now available in Extended Log Listing reports:

For more information about these report layouts, click here.

Tracking of Test Case Progress Using States and Points

TestManager now has the capability to track the execution progress of test cases using test case states and points. See the related white paper available in the IBM Publications Center at http://www.ibm.com/shop/publications/order. Search for "SY24-6181-00" to find the white paper.

Note: If you do not want to see state identifiers next to test case names in TestManager, click Tools > Options > View Options and clear the Show Test Case States check box.

Support for VMware GSX Server 3.0

TestManager now supports VMware GSX Server 3.0 for virtual machine management on Windows 2000 and Windows XP.

Multiple Selection of Test Assets

You can now select multiple test assets in the Asset Browser Planning View and Test Plan View.

Asset Browser Planning View:

Test Plan View:

New and Changed Features in v2003.06.12

Exporting and Reporting on Test Log Details

TestManager now provides the ability to export test log details and to report on test log details for functional tests. The ability to export and manage test logs details lets you further analyze test results. You can selectively filter data to identify trends and other valuable information.  For detailed information about this functionality, click here.

Note: Before you can access these new features, you must upgrade your test datastore. If you do not upgrade the datastore, options related to exporting and reporting on test log details are grayed out.

To export test log details:

  1. After running a test, click the Results tab in TestManager.
  2. Click a test log to export, or use CTRL-click to select multiple logs, or click a build to export all log detail for that build.
  3. Right-click and click Extract Log Info.

To overwrite log details previously exported:

  1. Select the log or logs to overwrite.
  2. Right-click and click Clear Log from Cache.

Creation and management of test log detail reports is much like the creation and management of other reports.

To create a report including extended test log details:

  1. Select the log or logs to use for the report.
  2. Export the test log details as described above.
  3. Select Reports > New > Extended Log Data.

To specify preferences for how log details are exported:

  1. Click Tools > Options.
  2. Click the Extended Log Detail Preferences tab.
    From this tab you can specify several choices for the management of log details:
    - The Always extract when exiting changed logs check box indicates that log data is extracted each time a log is closed. This is applied per user.
    - The options in the When overwriting a previously extracted log section indicate whether the user is asked about log extraction each time a log is closed, or whether the log is always or never extracted without prompting. These options also are applied on a per user basis.
    - The Log Details list lets you choose which of about 80 available log details you want to include when you export data, on a per datastore basis. You can choose one or all details, or any in between, to create truly customized data. If you do not specifically choose any log details for export, TestManager exports the core log details.
  3. When you are finished making changes, click OK. When you close the dialog box, TestManager exports log data for the current log according to options chosen or retained.

As with other reports, you can customize log detail reports through Crystal Reports. Consult the Crystal Reports documentation for information on customizing reports.

Using TestManager with Virtual Machine Management Software

TestManager now has the ability to interface with virtual machine management software, such as VMware GSX Server, to actively manage virtual machines hosting Rational Test Agents.

Note: TestManager currently supports VMware GSX Server v2.5.0 and v3.0 for virtual machine management on Windows 2000 and Windows XP. To purchase VMware GSX Server, contact VMware sales at sales@vmware.com or 1-877-4VMWARE.

A TestManager Computer asset may be designated as a virtual machine host by adding a special descriptor to its Computer Properties Description field. This descriptor associates virtual machine's name, server and other management parameters with the Computer.

During a test suite run, any included virtual machine hosts will have their execution states automatically managed by TestManager. If more virtual machine hosts than a virtual machine server can manage concurrently are included (for example, due to server resource limitations), then the virtual machine hosts will be automatically "multiplexed" between activated and deactivated states as necessary until the run is complete.

To take advantage of this new functionality, you must perform the following tasks:

  1. Install the VMware GSX Server Scripting API on the TestManager system.
  2. Set up Windows Auto-Logon for Test Agent Virtual Machines
  3. Configure the VMware Authorization Service for servers hosting Test Agent Virtual Machines
  4. Add a New VMHOST_INFO descriptor in Computer Properties Description field
Setting up Windows Auto-Logon for Windows-based Test Agent Virtual Machines

This feature does not support the use of the Rational Test Agent Service (rtpsvc.exe). The Rational Test Agent Daemon (rtprvd.exe) must be used instead and must be configured to start automatically when a virtual machine hosting a Rational Test Agent is started. To enable this, copy the daemon into the Startup folder of an account on the virtual machine's Windows guest operating system. Windows must then be configured to logon automatically using that account's credentials as described below.

To enable Windows Auto-Logon on virtual machine's Windows guest operating system, set registry keys to appropriate values:

  1. Open the Registry Editor:

a. From the Windows desktop, click Start > Run.
b. Type regedit and click OK.

  1. Within the Registry Editor, expand the tree on the left to display the keys for
    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon].

  2. For each of the following keys, right-click the key, click Modify, change the value, and click OK.
    Registry Key
    Value

    AutoAdminLogon

    1

    DefaultDomainName

    YourDomainName

    DefaultUserName

    YourUserName

    DefaultPassword

    YourPassword

  3. Click File > Exit to close the Registry Editor.

Note: This must be done for each VM hosting a Rational Test Agent.
Configuring the Authorization Service on VMware GSX Servers Hosting Test Agent Virtual Machines

To allow TestManager access to the VMware GSX Server remote functions needed by this feature, the VMware Authorization Service on the server must be configured with a set of authorizing user credentials. All virtual machines started by TestManager on that server are then run as that user.

To enable this, also known as VMware Trusted mode, you must:

Once this is done, TestManager will be allowed to remotely manage virtual machines on that server without any further configuration of the VMware GSX Server software.

Adding a New VMHOST_INFO Descriptor in Computer Properties Description Field

In order for TestManager to recognize that a Computer asset is virtual machine-hosted, a special textual descriptor must be added to the Description field of the Computer Properties dialog box of a Computer asset.

To edit the Computer Properties dialog box (which contains the Computer Properties Description field):

  1. From TestManager, click the Execution tab.
  2. Right-click any computer in the computer list, and click Properties.
  3. Add the appropriate descriptor, as described below. Then click OK.
    The descriptor takes the form:
    VMHOST_INFO= vmserver-name; vm-name; final-state; redo-action
    All fields are required.
     
    VMHOST_INFO= Required descriptor token. It must be spelled correctly (no space between VMHOST_INFO and =) or the remaining information will be ignored and the Computer will be assumed to be a non-virtual machine host.
    vmserver-name The name of the virtual machine's server. That is, the name of the physical machine that is running virtual machine management software and hosting the virtual machine. In the case of a VMware GSX Server-hosted virtual machine this would be the name of the VMware GSX Server host where the virtual machine is registered.
    vm-name The name of the virtual machine (also known as virtual host) that is hosting the test agent associated with the Computer. In the case of a VMware GSX Server-hosted virtual machine this would be the pathname to the virtual machine's configuration file that is registered with the VMware GSX Server host for that virtual machine.
    final-state Specifies the desired final state of the virtual machine after a suite run. This field can have one of four values:
    POWEREDON - The virtual machine will be left in the powered on (started) state.
    SUSPENDED - The virtual machine will be left in the suspended (deactivated) state.
    INITIAL - The virtual machine will be returned to the state it was in before the suite run.
    DEFAULT - The virtual machine will be set to the default final state (currently equivalent to INITIAL).
    redo-action Specifies how virtual disk redo logs associated with the VM should be handled when a decision is required by the virtual machine management software. This can happen when a VM is placed in a started or stopped stated and redo logs have been persisted. This field can have one of four values:
    COMMIT - Any pending redo log data is committed to the virtual disk images.
    DISCARD - Any pending redo log data is discarded. Note that once discarded it cannot be recovered.
    PRESERVE - Any pending redo log data is preserved, that is neither committed nor discarded.
    DEFAULT - The default redo action will be taken (currently equivalent to PRESERVE).

Limitations of Using TestManager with Virtual Machine Management Software

TestManager does not support the use of virtual machines in a powered-off initial or final state.  Transitioning a virtual machine from or to a powered-off state can result in modal blocking conditions that TestManager cannot respond to. This may cause hangs and other timeout-related failures.

As an alternative to the powered-off state, you should use suspended mode for initial and final virtual machine states. This allows virtual machines to free resources back to the host operating system when they are no longer needed without requiring powered-off transitions.

General Enhancements in v2003.06.12

IBM Rational XDE Tester

IBM Rational XDE Tester is an object-oriented automated testing tool that enables you to test Java and HTML applications running on the Windows NT, Windows 2000, Windows XP, and Linux platforms.

Note:  XDE Tester is the newest version of the product introduced as Rational RobotJ in Rational Suites Enterprise Studio, Rational Suites TestStudio, and Rational TeamTest version 2002.05.02. XDE Tester is integrated with TestManager. XDE Tester is a stand-alone point product, but it maintains the same integrations when you have the 2003.06.xx versions of both XDE Tester and TestManager installed. You purchase and install XDE Tester separately from TestManager or any other Rational testing products.

XDE Tester and TestManager

When you associate an XDE Tester datastore with a Rational project, you can:

  • Create and play back scripts, including XDE Tester scripts, from Rational TestManager. Use the TestManager Agent to play back scripts on a remote computer.
  • View scripts from the Test Scripts window of Rational TestManager.

You can use the Test Log window of TestManager to:

  • Open a test log to view a result.
  • Filter the data of a test log to view only the information you need.
  • Open the test script that generates a log event in the appropriate test script development tool. For example, if you create an XDE Tester script and run it from TestManager, when you click a log event, Rational XDE Tester opens and displays the script.
  • Analyze the results of a test log using the Verification Point Comparator. If you use Rational XDE Tester to record scripts, you can analyze the results in the Verification Point Comparator to determine why a verification point may have failed.
  • Submit a defect for a failed test case or for a failed log event. The test log automatically fills in build, configuration, and script information in the IBM Rational ClearQuest defect form.
  • Preview or print data displayed in the active test log in the Test Log window.

New Built-in Test Input Type: Microsoft Excel

You can now use one or more Excel spreadsheets as test input sources. This test input adapter can be useful in many ways. Many teams use Excel to manage information in their projects. For example, some teams may document requirements or capture project planning information in an Excel spreadsheet. In either case, the new Excel test input adapter allows teams to track the testing effort with respect to their information in an Excel spreadsheet. Specifically, they could associate test cases with tasks defined in rows of a spreadsheet so that a coverage report could be generated to show progress or completion of those tasks.

Test Impact Analysis

If a test input changes, TestManager now informs you of the impact of that change on your test plan. For example, if a requirement changes in some meaningful way, you need to know which test cases in your test plans are affected so that you can update them to reflect requirement changes. TestManager automatically identifies and marks the affected test cases as "suspect" for IBM Rational RequisitePro and Microsoft Excel test input sources.

Enhanced Filtering of Test Input Sources

The Test Input Adapter Extensibility feature of TestManager now supports the ability to save test input source filters and Test Case Coverage report filters. Test input sources and Test Case Coverage reports are unfiltered by default. When you set a filter for a test input source, the last filter that you set persists in the Test Inputs window and Test Input Select dialog box. When you set a filter for a test input source, TestManager saves the filter with the test input source. When you set a filter for a Test Case Coverage report, TestManager saves the filter with the report.

Defining Test Script Options

You can now define additional test script options controlling how test script execution adapters execute test scripts.

New Test Scripts View

You can now view all your test scripts sorted by type from the Test Scripts View. From this window you can perform test management operations on test scripts. You can also perform basic version control operations (such as check-in and check-out) on registered test scripts that support these operations.

Defining Test Tool Options

You can now set test tool options to control how a particular test tool plays back a test script. You can define a test tool option in the test script execution adapter that you have written for a particular test tool.

New Actions in the Test Script Console and Test Input Adapters

Test script console adapter and test input adapter developers can now add their own custom actions, or commands. You can run these custom actions or commands in TestManager.

Test Asset Management and Reporting Enhancements in v2003.06.12

Importing and Exporting Test Assets

You can move or copy existing test assets between and within Rational projects, use third party tools to create importable versions of test assets, or use third party tools to analyze exported test assets. Importing and exporting test assets are subject to the following restrictions:

  • Logs, log details, and test case results are not supported by the TestManager import functionality.
  • TestManager does not support importing and exporting queries.
  • Test Suite implementations are not imported or exported. Test Suite reference names can be imported or exported provided the reference name exists in the import destination test datastore.

Changes to Test Case Reports

You can now run test case reports that display test case suspicion status as presented by the test impact analysis capability. Using the test case suspicion feature you can track meaningful changes to test inputs so that you can update the associated test cases or their implementations.

Performance Testing Enhancements in v2003.06.12

Enhanced High-Volume Testing

You can customize your system settings for high-volume testing on a single Windows-based computer (over 2000 virtual testers for each Windows host). To do this, you must edit the registry to increase desktop heap memory (memory used for dynamic memory allocation). For more information, see Appendix A in the Rational TestManager User's Guide.

IBM Rational ManualTest Enhancements in v2003.06.12

Undo and Redo

Undo and Redo are now supported in the ManualTest Grid Editor, Text Editor, and Execution window. You can access the Undo and Redo options from the Edit menu. You can also press Ctrl+Z or Ctrl+Y to undo or redo tasks, respectively.

You cannot undo:

  • Adding a note to a step or verification point.
  • Adding a comment during execution.

Find and Replace

Find and Replace are now supported in the ManualTest Grid Editor and Text Editor.

You access the Find and Replace options from the Edit menu. You can also press Ctrl+F or Ctrl+H to find and replace text.

Step by Step Execution Mode

Step by Step execution mode allows you to run a manual test script in a small modeless dialog box outside of the ManualTest application window. This allows you to minimize the ManualTest application so that it does not interfere with the application under test. To run in Step by Step mode, click Tools > Options and click Run in Step by Step mode.

Exporting Test Assets to Text Files

You can export manual test scripts from the ManualTest application to text files.

Specifying the Expected Result

You can specify the expected result of a step or verification point in both the Text and Grid Editors. In the Grid Editor, Expected Result and Expected Result File are new columns in the grid. In the Text Editor, the expected result appears as an icon.

In the Grid Editor, use the Expected Result File column to associate a file with a step or verification point to describe the expected result. The Expected Result File column will contain an icon indicating the file and associated application. For example, you can reference a bitmap capture of a screen that illustrates an expected result.

Note: We recommend that you use Uniform Naming Convention (UNC) path names for expected result files.

Specifying Result Details

The Comments column in the Execution window has been renamed to Result Details. In addition to the column name change, the dialog box for entering the comment (Result Details) has been changed to allow you to specify the Result Details File. For example, you could attach screen shots of failed steps or verification points.

Note: The Note, Description, Expected Result, and Expected Result File columns are read-only during execution.

Viewing the New Features in the Log

The first 64 characters of the description of each step or verification point are now written to a log when a manual test is run.

Now that you can associate file references with manual script steps and verification points during development and execution of the tests, you can gain access to these files when viewing the results. The Expected Result file and Result Details file for each step and verification point can be accessed from the tree view within the Test Log Viewer.

Executing Manual Scripts via the Web

The Expected Result, Expected Result File, and Result Details fields appear during manual test script execution using the Web.

ManualTest Web supports the latest Netscape versions. However, in Netscape 7.0 and higher, Expected Result files display links as path names only.

Deleting Multiple Log Assets

You can select and delete multiple test logs at the same time from the Results view of the Test Asset Workspace. Assets covered by this functionality include Builds, Test Log Folders, and Test Logs.

Test Inputs View

The Test Inputs view in TestManager now mirrors the Rational RequisitePro folder structure. This change is a result of performance improvements. TestManager no longer uses the COM layer to communicate with RequisitePro. Instead, it communicates directly to the RequisitePro database. We recommend that you organize your RequisitePro requirements in a way that makes them easy to find in the Test Inputs view.

Guidelines and Restrictions

Firewall Issues with Microsoft Windows XP Service Pack 2 (SP2)

During the execution of a test suite from TestManager, there are several programs that should be allowed through the Microsoft firewall so that they can access the local computer from the network. For information about how to allow this access if you are running TestManager on Windows XP SP2, see the IBM Rational Suite Release Notes v2003.06.13.

Unsupported Platforms

TestManager does not support Citrix or Microsoft Windows Terminal Server.

Windows Me

When you open a manual test script in Windows Me, part of the text disappears, word wrap does not work, and the backspace and delete buttons do not work. To restore the missing text, click another step or verification point and the text appears correctly.

Using Microsoft Excel Test Inputs

In TestManager you can view test inputs that have been defined in Microsoft Excel spreadsheets. If, while viewing those test inputs, you or another user creates a new test input or modifies an existing test input in Excel, you must refresh the TestManager Test Inputs view to see those changes in TestManager.

If you are changing the Date Modified column in an Excel spreadsheet, and the TestManager datastore and spreadsheet are in different time zones, the Update Suspicion feature does not work properly.

When using an Excel spreadsheet as a test input source, be aware that if you insert or remove a row or column in the spreadsheet, you may need to update the Test Input Source configuration settings.

When an association is made between a test input and a test case, the value in the ID column is stored by TestManager in order to preserve the association. Any modifications to the spreadsheet that affect the specified columns (ID and Name), such as adding or removing rows or columns immediately around the specified columns or rows, may invalidate the spreadsheet as a source of test inputs and require that you update the associated test input source. Modifying the values in the specified column or rows can also result in a loss of connection between the test case and test input.

Submitting Defects in the Test Log

Logging into a Different Datastore or Project

If after submitting a defect in the Test Log Window, you want to log into a different datastore or project, or log in as a different user, TestManager may not display the login dialog box after restart. In this case you must either log off the operating system and log back in (if you are running on Windows NT or later), or bring up Task Manager and terminate the CQINTSRV11.EXE process. You can then log in to another datastore or project, or log in as a different user.

Data Code Pages: Character Set Compatibility with ClearQuest

You may see an error message about data code pages if your TestManager project is integrated with IBM Rational ClearQuest. The error message appears when you enter data into a tool and the characters you use are incompatible with the set of characters that a ClearQuest database accepts.

For example, assume that you are working in a TestManager project that is integrated with a ClearQuest database, and the ClearQuest database accepts characters in English only. If you use English to enter TestManager data, the error message does not appear. However, the error message does appear if you use another character set, such as Japanese or Chinese.

If you see this error message, consult with your ClearQuest administrator or systems administrator to resolve the character set incompatibility.

Using the Suspicion Feature in TestManager

Ensuring the Performance of the Suspicion Feature

With the suspicion feature, you can define TestManager to inform you when requirements or test inputs change in a way that impacts test cases. TestManager detects changes in test inputs based on timestamps from the systems on which they reside.

For suspicion to work correctly, you must perform the following actions:

  • Synchronize the time on the systems. If the timestamps on the systems are out of synchronization by several minutes or more, the suspicion feature does not work.

  • Set the time zone for each system so that the suspicion feature can work correctly even if the TestManager and test input systems are in different time zones.

  • Set the Last Modified field in a spreadsheet to the "date" data type.
  • Make sure that the date that you manually enter in the Last Modified field is greater than the last date that was previously entered in the same field.
  • Click View > Update Suspicion to see a suspicion.

Updating Suspicion

RequisitePro projects that use a Microsoft Access database may experience a delay between the entry of the data and the display of the modified data. This delay is caused by a defect in the Microsoft MDAC component. A similar delay is apparent from TestManager when RequisitePro test inputs are involved. The most noticeable case is when suspicion is updated and cleared frequently in the Test Inputs view.

Manually Updating Suspicion

You can manually update a suspect link between a test input and the associated test case by clicking View > Update Suspicion from the Test Inputs view or Test Plan window.

Test Script Playback

VU Test Scripts

Playback of VU test scripts is not supported for Windows Me or Windows 98, whether the system is a Local computer or an Agent computer.

To play back Oracle 7 with Oracle 7 libraries on HP-UX, you must set the following environment variable before invoking TestManager:

 RT_MASTER_USE_STATE_FOR_SQL

Set the variable in the Environment tab of the System Properties Control Panel.

ManualTest Test Scripts

When playing back manual test scripts from TestManager, be sure to exit Rational ManualTest to terminate the script. The alternative, terminating the user from the TestManager console, is not necessary and may lead to unexpected results.

Visual Basic Test Scripts

The Test Script Services (TSS) framework supports playback of test scripts either directly (by TestManager) or by an external proxy TSS server process.  However, Visual Basic test scripts support only direct TestManager playback.

Generated Visual Basic and Java Test Scripts

Visual Basic and Java test scripts that are automatically generated (by the Robot Session Recorder or by IBM Rational QualityArchitect) do not record script line numbers. As a result, the TestManager Script view feature (the Script check box under Tools > Options) does not work with these test scripts. If you manually add method calls to your Visual Basic and Java test scripts that pass test script line numbers to TestManager, the Script view works. The relevant methods are TSSMonitor.PositionSet (Visual Basic) and TSSMonitor.positionSet (Java).

F5 (Refresh All) Does Not Work in Test Asset Workspace

You can press the F5 key to refresh all in the Test Plan window and in the Test Input window. This key does not work in the Test Asset Workspace. You must right-click a folder in each Test Asset Workspace tab or click View > Refresh Workspace from the main menu to refresh all the Test Asset Workspace.

Resource Monitoring

Monitoring system resources during test runs is not supported for Windows XP Home, Windows Me, or Windows 98 for Local or Agent computers.

To monitor resources on an Agent computer that is running AIX, you must install the bos.acct package on that Agent if it is not already installed.

Report Generation

TestManager provides a "previous" button allowing you to change the test log selection after you have selected the virtual tester, script, and commandID filters to be used in generating a report. This causes a problem because the previous test log is not closed and conflicts with the new test log. If you need to change the test log selection after you have selected filters, start over rather than using the "previous" button.

SAP Limitations

The following list gives SAP testing limitations in TestManager:

  • UNIX Agents are not supported for the SAP protocol.
  • The personalizing option (the multicolored circle in the upper right of SAPGUI) is not supported.
  • Multiple SAP sessions with a single client are not supported.
  • Only the active screen is rendered, while SAPGUI sometimes renders inactive screens.
  • The control box in table controls is not supported.
  • Transaction IQ03 is not supported.
  • Automatic logon load balancing, as performed by SAPLOGON.EXE and the SAP Session Manager, is not supported.
  • The Manager's Desktop functionality under the "Human Resources" menu is not supported.
  • The highest supported SAP R/3 version is v4.5b.

UCM on Windows NT 4.0 with ClearCase 5.0

A possible problem has been identified on Windows NT 4.0 when ClearCase 5.0 is installed. After performing a Unified Change Management (UCM) datastore operation from the Rational Administrator (such as create a datastore, check-in all, add to or remove from source control), we suggest you shut down and restart Rational Administrator before attempting another such operation.

Editing Reports with Long Path Names (Crystal Reports v8.5)

Crystal Reports is used to generate or edit Listing reports. You can run a report regardless of the length of its pathname. However, if you attempt to edit a report whose pathname is longer than 80 characters, the message "Dictionary not found" appears. This is a Crystal Reports v8.5 restriction. If you create a new or customize an existing design layout, keep the root path to the test datastore as short as possible.

Performance Testing

ODBC Limitations

When playing back a SQL statement that generates multiple result sets, TestManager logs only the data for the first fetch. The other fetches show the number of rows returned, but do not log the data actually returned.

If a sqlfetch_cursor is run multiple times in succession with ALL_ROWS and EXPECT_ROWS 0, an error for the second sqlfetch_cursor is generated stating:

 ERROR -1: {error} STATE=S1010, CODE=0, MSG=[Microsoft][ODBC Driver 
Manager] Function sequence error

When logging data returned from sqlfetch_cursor, sometimes the data is truncated and the next SQL command appears on the same line as a row of data. This is a logging problem only; the test script is executing correctly.

Testing CORBA Applications (for VisiBroker 3.3 only)

To test CORBA applications with TestManager, you must have the following software installed on the local computer:

  • Windows XP, Windows 2000, or Windows NT Version 4.0 with at least Service Pack 4

  • VisiBroker (C++ or Java) Version 3.3.

CORBA Playback

Before playback, you must import into the project the IDL for each interface referenced in a test script.

TestManager provides special IIOP monitoring states in the state histogram monitor window.

TestManager does not support playback of ASCII hex (opaque) data between computers of differing byte ordering. Byte ordering specification is marked in the IIOP protocol. It is possible that data may be transferred with big-endian byte order, even on Windows NT systems. This case is more likely to occur if the application is Java. The result is that data specified opaquely in these cases may not play back correctly, even though the recording computer and the playback computer have the same byte order.

Supported and Unsupported CORBA Data Types for VU Test Scripts

TestManager supports playing back the following CORBA data types:

  • char
  • octet
  • short
  • unsigned short
  • long
  • unsigned long
  • float
  • double
  • boolean
  • enum
  • struct
  • union
  • array
  • sequence

The following data types are not supported, and result in methods being scripted opaquely if they occur as a parameter data type in a method:

  • any
  • long double
  • unsigned long long
  • long long
  • wchar
  • wstring
  • fixed
UNIX Agents

Only Solaris is supported for UNIX Agent playback of VU language CORBA/IIOP (VisiBroker 3.3 only) scripts recorded by IBM Rational Robot. The VisiBroker bind modus is not supported for playback. This limitation means that "VisiBroker" cannot be used to locate object references at play back. Therefore, another method, such as reading IORs (Inter-operable Object Reference) from a file, or specifying them explicitly in VU test scripts, must be used to locate objects when playing back test scripts from a Solaris Agent.

Issue With Rational Test Agents and NuTCRACKER

When using Rational Test Agent in a VMware integration environment (GSX Server/TestManager) you may see an error message stating that NuTCRACKER has experienced a fatal error and has not registered with the service. If this occurs, perform the following steps:

  1. Select OK in the error message box. Rational Test Agent exits.
  2. Manually start Rational Test Agent by clicking Start > Rational Software > Rational Test > Rational Test Agent.

This allows you to perform your normal functional testing in the integrated environment.

Note: Do not use the powered-off state of VMware integration. Instead, use the suspended mode only.

Problem with Exporting and Importing Test Case Designs

There is a known problem with importing and exporting test case designs. If there are carriage returns in the Note or Description boxes, the test case design does not import or export correctly.

To ensure that test case designs import and export correctly, when you create a test case design, do not enter carriage returns in the Note or Description boxes.

However, if you need to have multiple lines of information in these boxes, you can do the following:

  1. In TestManager, right-click a test case, and then click Design.
  2. in the Design Editor dialog box, in the Note or Description boxes, enter multiple lines of text, and then click OK.
  3. In TestManager, right-click the test case again and click Properties.
  4. Click the Implementation tab, and then click Import from Test Case Design.
  5. In the Manual Test Script Name box, type the name of a manual script and click OK.
  6. In Rational ManualTest, export the manual test to a text file.

Problem with Using Double-Byte Japanese Characters

Using an activity name with double-byte Japanese characters causes an error in the ClearCase check-in procedure. It is best to avoid using them.

Issues with Extended Log Detail and Extended Log Summary Reports

The following problems with Extended Log Detail reports and Extended Log Summary reports can be fixed by downloading new templates for these reports:

  • Warnings appear as failures in the extended log reports.
  • Test cases with Robot scripts associated with them do not appear in the Extended Log Summary report.
  • The Extended Log Detail report does not contain the additional information fields for steps in GUI scripts.
  • The Extended Log Detail report does not include test case names.

You can find the report templates on developerWorks Rational at the following link: 

http://www.ibm.com/developerworks/rational/library/4320.html

Place the templates in the TestDatastore\TMS_Logs folder. They will become immediately available in the drop-down list when you create a new Extended Log report.

TestManager and XDE Tester

You can no longer start XDE Tester from TestManager. The toolbar button and menu command to do this have been removed.

Crystal Reports Issues

Crystal Reports v10 does not support Windows 9x platforms. Therefore, the runtime libraries are not installed when TestManager is installed on Windows 9x.

If you uninstall TestManager v2002.05.00, the uninstall program does not remove the \windows\system32\crpe32.dll file. If you then install TestManager v2003.06.13 and attempt to run a listing report while the DLL file is present, TestManager displays an error message that you do not have a correct version of Crystal Reports installed. Remove or rename the DLL file to resolve this issue.

Expected Result Files for ManualTest in Windows XP

When running a manual test on a Windows XP system, you may need to change the program association of an expected result file so that it is opened with an .exe. ManualTest running on Windows XP cannot open an expected result file if the associated application is opened with a DLL.

To change the file type association:

  1. From Windows Explorer, right-click the expected result file.
  2. Click Open With > Choose Program.
  3. Select an appropriate program from the list and click OK.

Corrections to TestManager Documentation

Wildcard References in Filter Operator Types

The Rational TestManager Help for filter operator types incorrectly refers to the character asterisk (*) as the wildcard character for a string of characters and the question mark (?) as the wildcard character for a single character position. The correct characters are percent sign (%) for a string of characters and underscore (_) for a single character.

The Help topic "Filter Operator Types" should read as follows:

Like
Allows the use of wildcard characters in the Value field to include items that have any alphanumeric text character(s) in particular character positions. The wildcard characters are percent sign (%) for a string of characters and underscore (_) for a single character position. The Like operator works only with alphanumeric text data fields and is case-sensitive.

ManualTest Does Not Support Linux

Previous releases of Rational TestManager stated that manual test execution is supported for Linux agents. This is not the case. Rational TestManager does not support manual test execution on Linux agents.

TestManager Extensibility TIClearFilter Function

The following function is not documented in the current TestManager extensibility documentation.

TIClearFilter()

Clears filters for a test input source.

Syntax

 HRESULT TIClearFilter(const TCHAR SourceID[TI_MAX_PATH], TCHAR 
ErrorDescription[TI_MAX_ERROR])
Element
Description
 SourceID input. The handle identifying the connection to the test input source.
 ErrorDescription output. The message to be displayed to the TestManager user if there is an error.

Return Values

This function typically returns one of the following values:

  • TI_SUCCESS. The function completed successfully.
  • TI_ERROR. The function did not complete successfully.
  • TI_ERROR_INVALID_SOURCEID. The input source was identified incorrectly.

Comments

TestManager makes this call when a user chooses to remove a filter. Your adapter should clear any filters that have been set.

Example

 HRESULT TIClearFilter (const TCHAR SourceID[TI_MAX_PATH], TCHAR 
ErrorDescription[TI_MAX_ERROR])
{
 	AFX_MANAGE_STATE(AfxGetStaticModuleState());

	HRESULT rc = S_OK;
	CConnectionContext *pContext=0;
	// Figure out what instance of the adapter is in use. 
	m_ProjectConnections.Lookup(SourceID, (void *&)pContext);

	// Make sure we've got a pContext
	if (pContext)
	{	// Make sure the Instance of the adapter is connected and loaded 	
 		if (pContext->m_lpDispatchProject)
 		{
			//Clear your filters
			pContext->SetFilterSettings("");
		}
		else
			rc = TI_NO_PROJECT;
	}
	else
		rc = TI_NO_CONTEXT;

	// Apply the filter
	pContext->LoadFilterOptions();
	return rc;
}

Contacting IBM Rational Customer Support

The IBM software support Internet site provides you with self-help resources and electronic problem submission. The IBM Rational Software Support Home page can be found at http://www.ibm.com/software/rational/support/.

Voice Support is available to all current contract holders by dialing a telephone number in your country (where available). For specific country phone numbers, please go to http://www.ibm.com/planetwide/.