IBM Rational TestManager Release Notes
Version 2003.06.13
Release Date: September 2004
Last Updated: 08/09/2004
Material ID: GI11-5907-03
© Copyright IBM Corporation. 2000, 2004. All Rights Reserved.
This documentation (the "Work") is protected under the copyright laws of the United States and/or other jurisdictions, as well as various international treaties. Any reproduction or distribution of the Work is expressly prohibited without the prior written consent of IBM Corporation.
The complete listing of IBM Rational Legal Notices is automatically installed with IBM Rational Suite in <Install Path>\Rational\doc\legal_information.html. Optionally, the Legal Notices can be found on the "IBM Rational Solutions for Windows CD-ROM" at \doc\legal_information.html.
These release notes are accurate at the time this product was prepared for release. The most recent version of this file is available in the IBM Publications Center at http://www.ibm.com/shop/publications/order. Search for "GI11-5907-03" to find the latest version of these release notes in the IBM Publications Center. (To search for the IBM Publications Center, go to http://www.ibm.com and enter "IBM Publications Center" in the Search field.)
The testing products user guides that are translated into Japanese describe the Rational product release 2001A.04.10. You can find these documents on the IBM Rational Solutions for Windows Online Documentation CD-ROM in the "Extras" folder.
Product changes described in this document and contained within this service release apply if you are upgrading from:
IBM Rational Solutions for Windows v2003.06.00 (English version)
IBM Rational Solutions for Windows v2003.06.10 (Japanese version)
IBM Rational Solutions for Windows v2003.06.12 (English and Japanese service release)
v2003.06.00, 2003.06.10, and 2003.06.12 of other IBM Rational products.
If you are upgrading from any of these releases, see the IBM Rational Testing Products Service Release Notes for the defects fixed in this service release and the previous service release.
This release incorporates several enhancements to the IBM(R) Rational(R) TestManager product. These release notes provide information that is not available in the printed documentation or the Help for Rational TestManager.
Contents
Scope
Getting Started
Installation Information and System Requirements
New and Changed Features
New and Changed Features in v2003.06.13
Support for Business Objects Crystal Reports v10
New Extended Log Listing Report Layouts
Tracking of Test Case Progress Using States and Points
Support for VMware GSX Server 3.0.
Multiple Selection of Test Assets
New and Changed Features in v2003.06.12
Exporting and Reporting on Test Log Details
Using TestManager with Virtual Machine Management Software
General Enhancements in v2003.06.12
New Built-in Test Input Type: Microsoft Excel
Enhanced Filtering of Test Input Sources
New Actions in the Test Script Console and Test Input Adapters
Test Asset Management and Reporting Enhancements in v2003.06.12
Importing and Exporting Test Assets
Performance Testing Enhancements in v2003.06.12
IBM Rational ManualTest Enhancements in v2003.06.12
Exporting Test Assets to Text Files
Specifying the Expected Result
Viewing the New Features in the Log
Executing Manual Scripts via the Web
Guidelines and Restrictions
Firewall Issues with Microsoft Windows XP Service Pack 2 (SP2)
Using Microsoft Excel Test Inputs
Submitting Defects in the Test Log
Logging into a Different Datastore or Project
Data Code Pages: Character Set Compatibility with ClearQuest
Using the Suspicion Feature in TestManager
Ensuring the Performance of the Suspicion Feature
Generated Visual Basic and Java Test Scripts
F5 (Refresh All) Does Not Work in Test Asset Workspace
UCM on Windows NT 4.0 with ClearCase 5.0
Editing Reports with Long Path Names (Crystal Reports 8.5)
Testing CORBA Applications (for VisiBroker 3.3 only)
Issue With Rational Test Agents and NuTCRACKER
Problem with Exporting and Importing Test Case Designs
Problem with Using Double-Byte Japanese Characters
Issues with Extended Log Detail and Extended Log Summary Reports
Expected Result Files for ManualTest in Windows XP
Corrections to Documentation
Wildcard References in Filter Operator Types
ManualTest Does Not Support Linux
TestManager Extensibility TIClearFilter Function
Contacting IBM Rational Customer Support
Getting Started
Installation Information and System Requirements
The IBM Rational Software Desktop Products Installation Guide, IBM Rational Software Server Products Installation Guide, and IBM Rational Suite Upgrade Guide contain complete installation and upgrade instructions and system requirements for Rational products for Windows. They are available electronically as PDF files on the IBM Rational Solutions for Windows Online Documentation CD-ROM, which also includes a version of the Acrobat Reader.
See the IBM Rational Testing Products Release Notes for important information about installing Rational testing products and creating test datastores.
Upgrading Test Datastores
When upgrading to TestManager v2003.06.13, note the following important information related to test datastores:
- During the process of upgrading to this release, test datastores need to be upgraded as well, to take advantage of new reporting functionality, Crystal Reports v10 support, and new Test Case tracking capability. The upgraded clients, however, can connect to previous (non-upgraded) test datastore until such time as a test datastore can be upgraded manually.
Datastores that are created in this release can be accessed only in this and later releases. If the majority of your site is not upgrading to this release, do not create new datastores or upgrade existing datastores created in previous releases.
To upgrade a test datastore to v2003.06.13, perform the following steps:
- Make a backup copy of the datastore.
- Click Start > Programs > Rational Software > Rational Administrator to open the IBM Rational Administrator.
- In the project pane, select the Rational project that contains the test datastore to be upgraded. Right-click the project and click Upgrade Assets.
- Click OK to the message that appears reminding you to back up your data.
- Click View Log to examine the results of the operation and ensure that it completed successfully. Click Done.
New and Changed Features
New and Changed Features in v2003.06.13
The following list summarizes the new and changed features in v2003.06.13.
- Support for Business Objects Crystal Reports v10
- New Extended Log Listing report layouts
- Tracking of test case progress using states and points
- Support for VMware GSX Server 3.0
- Multiple selection of test assets for deleting, copying, pasting, exporting, and importing
Each of these features is described below.
Support for Business Objects Crystal Reports v10
TestManager v2003.06.13 includes the Crystal Reports v10 runtime libraries (DLLs) required to view and run reports in TestManager clients. TestManager installs the files automatically.
However, to author or edit reports, you must purchase a copy of Crystal Reports Professional Edition or other Crystal Editions that contain Crystal Designer v10. For instructions about how to install and license Crystal Reports Professional Edition or other Crystal Editions, contact Business Objects at
http://www.businessobjects.com.Using TestManager with Crystal Reports v10
Test datastores that were created with previous versions of TestManager v2003 (v2003.06.00, v2003.06.01, or v2003.06.12) are compatible with TestManager v2003.06.13. However, to benefit from the new features provided in v2003.06.13, you must upgrade your test datastores to v2003.06.13.
Versions of TestManager 2003 prior to v2003.06.13 use Crystal Dictionary to run listing reports. Reports based on Crystal Dictionary are not supported in Crystal Reports v10, the newly supported Crystal version for v2003.06.13. For this reason, v2003.06.13 provides replacement listing reports that do not use dictionaries. These replacement listing reports were created with Crystal Reports v8.5 to be fully compatible with Crystal Reports v8.5 through v10.
You can continue to use Crystal Reports v8.5 to run old dictionary-based reports and to create and run new reports that do not rely on dictionaries.
New datastores that you create do not include dictionaries. When you upgrade a v2003.06.12 datastore that includes dictionaries, the datastore retains those dictionaries. If you previously modified a listing report in Crystal Reports v8.5 and you now want to edit or run that report in Crystal Reports v10, you must do the following tasks:
- Upgrade the report’s test datastore to v2003.06.13, or create a new datastore in v2003.06.13, and
- Re-create the report by copying and pasting its modified design elements to the corresponding replacement listing report. (For instructions, see the next section.)
Note: Before you upgrade a test datastore, you must upgrade all clients that use that datastore. If you are not ready to upgrade all clients to v2003.06.13, do not upgrade the test datastore.
Re-creating Reports with Crystal Reports v10
To re-create a listing report that you modified with Crystal Reports v8.5, perform the following steps:
- Click Start > Programs > Rational Software > Rational TestManager. In the Rational Test Login dialog box, type the password, select the Rational project, and click OK.
- In the test asset workspace, click the Analysis tab to display the reports. Expand the Listing Reports folder to display the listing reports. Find the replacement nondictionary-based listing report that corresponds to the dictionary-based report that you used to create the customized Crystal Reports v8.5 report. For example, if you created a report in v8.5 named Modified Build Detail Report based on the Build Detail Report, find the nondictionary-based Build Detail report. Right-click the nondictionary-based report and click Open.
- In the Listing Reports dialog box, click New. Type a name for the new report and click OK. Crystal Reports v10 opens the report for editing.
- Click Start > Programs > Crystal Reports to start Crystal Reports v8.5.
- In the Welcome to Crystal Reports dialog box, select Open an Existing Report and select the report from the list. The report opens in the Crystal Reports v8.5 editor.
- In the Crystal Reports v8.5 editor, select and copy the design elements that you modified.
- In the Crystal Reports v10 editor, paste the design elements into the new report.
- Click File > Save to save the re-created report. Click File > Close to end the editing session.
- In v2003.06.13, the structure of some listing reports has changed such that some assets have been renamed or moved to new tables. For example, the Build Details report now includes two tables, Build and Iteration. In previous versions of TestManager, the Iteration field identifies an iteration name. In v2003.06.13, the Iteration field in the Build table identifies a UID. The iteration name is provided by the Name field in the new Iteration table. To determine if this type of problem exists with your report, right-click the report in the TestManager test asset workspace and click Run. Examine the report for any unexpected field data. If a problem exists, continue with the next step. If no problems exist, you are done with this report and you can begin re-creating your remaining reports.
- Open the new report in Crystal Reports v10. Click View > Field Explorer to display the report’s tables and field definitions.
- In general, to resolve a problem with an asset, go to that asset’s table. For example, to resolve the Iteration field problem go to the Iteration table and use its Name field to replace your report’s Iteration field.
- Save the changes to the report and run it again to verify that you have solved the problem.
Uninstalling Crystal Reports v8.5 Runtime
By default, TestManager v2003.06.13 uses the Crystal Reports v8.5 runtime to run your listing reports. After you re-create all modified reports to be compatible with Crystal Reports v10, uninstall the Crystal Reports v8.5 runtime so that TestManager v2003.06.13 uses the Crystal Reports v10 runtime to run your reports.
New Extended Log Listing Report Layouts
The following new report layouts are now available in Extended Log Listing reports:
- Configured Test Case Results Detail
- Configured Test Case Results Overview
- Configured Test Cases
- Manual Script Event Log
- Test Case Detailed Results Overview by Build
- Test Case Results by Suite
- Test Case Results Detail
- Test Case Results Overview
- Test Case Results Summary by Build
- Verification Point History
For more information about these report layouts, click here.
Tracking of Test Case Progress Using States and Points
TestManager now has the capability to track the execution progress of test cases using test case states and points. See the related white paper available in the IBM Publications Center at http://www.ibm.com/shop/publications/order. Search for "SY24-6181-00" to find the white paper.
Note: If you do not want to see state identifiers next to test case names in TestManager, click Tools > Options > View Options and clear the Show Test Case States check box.
Support for VMware GSX Server 3.0
TestManager now supports VMware GSX Server 3.0 for virtual machine management on Windows 2000 and Windows XP.
Multiple Selection of Test Assets
You can now select multiple test assets in the Asset Browser Planning View and Test Plan View.
Asset Browser Planning View:
If you select multiple test plans, you can delete, copy, paste, export, and import the multiple items simultaneously. From the pop-up menu, three commands are available: delete, copy, and export. There is only one parcel file for multiple exported items. When you import the parcel file, multiple items are imported.
If you select either multiple iterations or different asset types (such as a test plan and an iteration), you can only delete them.
Test Plan View:
If you select multiple assets of the same type (such as two test cases), you can delete, copy, paste, export, and import the multiple items simultaneously. From the pop-up menu, three commands are available: delete, copy, and export. There is only one parcel file for multiple exported items. When you import the parcel file, multiple items are imported.
If you select either multiple configured test cases or assets of different types, you can only delete them.
New and Changed Features in v2003.06.12
Exporting and Reporting on Test Log Details
TestManager now provides the ability to export test log details and to report on test log details for functional tests. The ability to export and manage test logs details lets you further analyze test results. You can selectively filter data to identify trends and other valuable information. For detailed information about this functionality, click here.
Note: Before you can access these new features, you must upgrade your test datastore. If you do not upgrade the datastore, options related to exporting and reporting on test log details are grayed out.To export test log details:
- After running a test, click the Results tab in TestManager.
- Click a test log to export, or use CTRL-click to select multiple logs, or click a build to export all log detail for that build.
- Right-click and click Extract Log Info.
To overwrite log details previously exported:
Creation and management of test log detail reports is much like the creation and management of other reports.
To create a report including extended test log details:
- Select the log or logs to use for the report.
- Export the test log details as described above.
- Select Reports > New > Extended Log Data.
To specify preferences for how log details are exported:
- Click Tools > Options.
- Click the Extended Log Detail Preferences tab.
- From this tab you can specify several choices for the management of log details:
- - The Always extract when exiting changed logs check box indicates that log data is extracted each time a log is closed. This is applied per user.
- - The options in the When overwriting a previously extracted log section indicate whether the user is asked about log extraction each time a log is closed, or whether the log is always or never extracted without prompting. These options also are applied on a per user basis.
- - The Log Details list lets you choose which of about 80 available log details you want to include when you export data, on a per datastore basis. You can choose one or all details, or any in between, to create truly customized data. If you do not specifically choose any log details for export, TestManager exports the core log details.
- When you are finished making changes, click OK. When you close the dialog box, TestManager exports log data for the current log according to options chosen or retained.
As with other reports, you can customize log detail reports through Crystal Reports. Consult the Crystal Reports documentation for information on customizing reports.
Using TestManager with Virtual Machine Management Software
TestManager now has the ability to interface with virtual machine management software, such as VMware GSX Server, to actively manage virtual machines hosting Rational Test Agents.
Note: TestManager currently supports VMware GSX Server v2.5.0 and v3.0 for virtual machine management on Windows 2000 and Windows XP. To purchase VMware GSX Server, contact VMware sales at sales@vmware.com or 1-877-4VMWARE.
A TestManager Computer asset may be designated as a virtual machine host by adding a special descriptor to its Computer Properties Description field. This descriptor associates virtual machine's name, server and other management parameters with the Computer.
During a test suite run, any included virtual machine hosts will have their execution states automatically managed by TestManager. If more virtual machine hosts than a virtual machine server can manage concurrently are included (for example, due to server resource limitations), then the virtual machine hosts will be automatically "multiplexed" between activated and deactivated states as necessary until the run is complete.
To take advantage of this new functionality, you must perform the following tasks:
- Install the VMware GSX Server Scripting API on the TestManager system.
- Set up Windows Auto-Logon for Test Agent Virtual Machines
- Configure the VMware Authorization Service for servers hosting Test Agent Virtual Machines
- Add a New VMHOST_INFO descriptor in Computer Properties Description field
Setting up Windows Auto-Logon for Windows-based Test Agent Virtual Machines
This feature does not support the use of the Rational Test Agent Service (rtpsvc.exe). The Rational Test Agent Daemon (rtprvd.exe) must be used instead and must be configured to start automatically when a virtual machine hosting a Rational Test Agent is started. To enable this, copy the daemon into the Startup folder of an account on the virtual machine's Windows guest operating system. Windows must then be configured to logon automatically using that account's credentials as described below.
To enable Windows Auto-Logon on virtual machine's Windows guest operating system, set registry keys to appropriate values:
a. From the Windows desktop, click Start > Run.
b. Type regedit and click OK.
Note: This must be done for each VM hosting a Rational Test Agent.
Within the Registry Editor, expand the tree on the left to display the keys for
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon].- For each of the following keys, right-click the key, click Modify, change the value, and click OK.
Registry Key
Value
Click File > Exit to close the Registry Editor.
Configuring the Authorization Service on VMware GSX Servers Hosting Test Agent Virtual Machines
To allow TestManager access to the VMware GSX Server remote functions needed by this feature, the VMware Authorization Service on the server must be configured with a set of authorizing user credentials. All virtual machines started by TestManager on that server are then run as that user.
To enable this, also known as VMware Trusted mode, you must:
- Run the program named authtrusted.exe in the VMware GSX Server product folder.
- Set the desired user credentials according to the program's directions.
Once this is done, TestManager will be allowed to remotely manage virtual machines on that server without any further configuration of the VMware GSX Server software.
Adding a New VMHOST_INFO Descriptor in Computer Properties Description Field
In order for TestManager to recognize that a Computer asset is virtual machine-hosted, a special textual descriptor must be added to the Description field of the Computer Properties dialog box of a Computer asset.
To edit the Computer Properties dialog box (which contains the Computer Properties Description field):
- From TestManager, click the Execution tab.
- Right-click any computer in the computer list, and click Properties.
- Add the appropriate descriptor, as described below. Then click OK.
- The descriptor takes the form:
- VMHOST_INFO= vmserver-name; vm-name; final-state; redo-action
- All fields are required.
VMHOST_INFO= Required descriptor token. It must be spelled correctly (no space between VMHOST_INFO and =) or the remaining information will be ignored and the Computer will be assumed to be a non-virtual machine host. vmserver-name The name of the virtual machine's server. That is, the name of the physical machine that is running virtual machine management software and hosting the virtual machine. In the case of a VMware GSX Server-hosted virtual machine this would be the name of the VMware GSX Server host where the virtual machine is registered. vm-name The name of the virtual machine (also known as virtual host) that is hosting the test agent associated with the Computer. In the case of a VMware GSX Server-hosted virtual machine this would be the pathname to the virtual machine's configuration file that is registered with the VMware GSX Server host for that virtual machine. final-state Specifies the desired final state of the virtual machine after a suite run. This field can have one of four values:
POWEREDON - The virtual machine will be left in the powered on (started) state.
SUSPENDED - The virtual machine will be left in the suspended (deactivated) state.
INITIAL - The virtual machine will be returned to the state it was in before the suite run.
DEFAULT - The virtual machine will be set to the default final state (currently equivalent to INITIAL).redo-action Specifies how virtual disk redo logs associated with the VM should be handled when a decision is required by the virtual machine management software. This can happen when a VM is placed in a started or stopped stated and redo logs have been persisted. This field can have one of four values:
COMMIT - Any pending redo log data is committed to the virtual disk images.
DISCARD - Any pending redo log data is discarded. Note that once discarded it cannot be recovered.
PRESERVE - Any pending redo log data is preserved, that is neither committed nor discarded.
DEFAULT - The default redo action will be taken (currently equivalent to PRESERVE).
Limitations of Using TestManager with Virtual Machine Management Software
TestManager does not support the use of virtual machines in a powered-off initial or final state. Transitioning a virtual machine from or to a powered-off state can result in modal blocking conditions that TestManager cannot respond to. This may cause hangs and other timeout-related failures.
As an alternative to the powered-off state, you should use suspended mode for initial and final virtual machine states. This allows virtual machines to free resources back to the host operating system when they are no longer needed without requiring powered-off transitions.
General Enhancements in v2003.06.12
IBM Rational XDE Tester
IBM Rational XDE Tester is an object-oriented automated testing tool that enables you to test Java and HTML applications running on the Windows NT, Windows 2000, Windows XP, and Linux platforms.
Note: XDE Tester is the newest version of the product introduced as Rational RobotJ in Rational Suites Enterprise Studio, Rational Suites TestStudio, and Rational TeamTest version 2002.05.02. XDE Tester is integrated with TestManager. XDE Tester is a stand-alone point product, but it maintains the same integrations when you have the 2003.06.xx versions of both XDE Tester and TestManager installed. You purchase and install XDE Tester separately from TestManager or any other Rational testing products.
XDE Tester and TestManager
When you associate an XDE Tester datastore with a Rational project, you can:
- Create and play back scripts, including XDE Tester scripts, from Rational TestManager. Use the TestManager Agent to play back scripts on a remote computer.
- View scripts from the Test Scripts window of Rational TestManager.
You can use the Test Log window of TestManager to:
- Open a test log to view a result.
- Filter the data of a test log to view only the information you need.
- Open the test script that generates a log event in the appropriate test script development tool. For example, if you create an XDE Tester script and run it from TestManager, when you click a log event, Rational XDE Tester opens and displays the script.
- Analyze the results of a test log using the Verification Point Comparator. If you use Rational XDE Tester to record scripts, you can analyze the results in the Verification Point Comparator to determine why a verification point may have failed.
- Submit a defect for a failed test case or for a failed log event. The test log automatically fills in build, configuration, and script information in the IBM Rational ClearQuest defect form.
- Preview or print data displayed in the active test log in the Test Log window.
New Built-in Test Input Type: Microsoft Excel
You can now use one or more Excel spreadsheets as test input sources. This test input adapter can be useful in many ways. Many teams use Excel to manage information in their projects. For example, some teams may document requirements or capture project planning information in an Excel spreadsheet. In either case, the new Excel test input adapter allows teams to track the testing effort with respect to their information in an Excel spreadsheet. Specifically, they could associate test cases with tasks defined in rows of a spreadsheet so that a coverage report could be generated to show progress or completion of those tasks.
Test Impact Analysis
If a test input changes, TestManager now informs you of the impact of that change on your test plan. For example, if a requirement changes in some meaningful way, you need to know which test cases in your test plans are affected so that you can update them to reflect requirement changes. TestManager automatically identifies and marks the affected test cases as "suspect" for IBM Rational RequisitePro and Microsoft Excel test input sources.
Enhanced Filtering of Test Input Sources
The Test Input Adapter Extensibility feature of TestManager now supports the ability to save test input source filters and Test Case Coverage report filters. Test input sources and Test Case Coverage reports are unfiltered by default. When you set a filter for a test input source, the last filter that you set persists in the Test Inputs window and Test Input Select dialog box. When you set a filter for a test input source, TestManager saves the filter with the test input source. When you set a filter for a Test Case Coverage report, TestManager saves the filter with the report.
Defining Test Script Options
You can now define additional test script options controlling how test script execution adapters execute test scripts.
New Test Scripts View
You can now view all your test scripts sorted by type from the Test Scripts View. From this window you can perform test management operations on test scripts. You can also perform basic version control operations (such as check-in and check-out) on registered test scripts that support these operations.
Defining Test Tool Options
You can now set test tool options to control how a particular test tool plays back a test script. You can define a test tool option in the test script execution adapter that you have written for a particular test tool.
New Actions in the Test Script Console and Test Input Adapters
Test script console adapter and test input adapter developers can now add their own custom actions, or commands. You can run these custom actions or commands in TestManager.
Test Asset Management and Reporting Enhancements in v2003.06.12
Importing and Exporting Test Assets
You can move or copy existing test assets between and within Rational projects, use third party tools to create importable versions of test assets, or use third party tools to analyze exported test assets. Importing and exporting test assets are subject to the following restrictions:
- Logs, log details, and test case results are not supported by the TestManager import functionality.
- TestManager does not support importing and exporting queries.
- Test Suite implementations are not imported or exported. Test Suite reference names can be imported or exported provided the reference name exists in the import destination test datastore.
Changes to Test Case Reports
You can now run test case reports that display test case suspicion status as presented by the test impact analysis capability. Using the test case suspicion feature you can track meaningful changes to test inputs so that you can update the associated test cases or their implementations.
Performance Testing Enhancements in v2003.06.12
Enhanced High-Volume Testing
You can customize your system settings for high-volume testing on a single Windows-based computer (over 2000 virtual testers for each Windows host). To do this, you must edit the registry to increase desktop heap memory (memory used for dynamic memory allocation). For more information, see Appendix A in the Rational TestManager User's Guide.
IBM Rational ManualTest Enhancements in v2003.06.12
Undo and Redo
Undo and Redo are now supported in the ManualTest Grid Editor, Text Editor, and Execution window. You can access the Undo and Redo options from the Edit menu. You can also press Ctrl+Z or Ctrl+Y to undo or redo tasks, respectively.
Find and Replace
Find and Replace are now supported in the ManualTest Grid Editor and Text Editor.
You access the Find and Replace options from the Edit menu. You can also press Ctrl+F or Ctrl+H to find and replace text.
Step by Step Execution Mode
Step by Step execution mode allows you to run a manual test script in a small modeless dialog box outside of the ManualTest application window. This allows you to minimize the ManualTest application so that it does not interfere with the application under test. To run in Step by Step mode, click Tools > Options and click Run in Step by Step mode.
Exporting Test Assets to Text Files
You can export manual test scripts from the ManualTest application to text files.
Specifying the Expected Result
You can specify the expected result of a step or verification point in both the Text and Grid Editors. In the Grid Editor, Expected Result and Expected Result File are new columns in the grid. In the Text Editor, the expected result appears as an icon.
In the Grid Editor, use the Expected Result File column to associate a file with a step or verification point to describe the expected result. The Expected Result File column will contain an icon indicating the file and associated application. For example, you can reference a bitmap capture of a screen that illustrates an expected result.
Note: We recommend that you use Uniform Naming Convention (UNC) path names for expected result files.
Specifying Result Details
The Comments column in the Execution window has been renamed to Result Details. In addition to the column name change, the dialog box for entering the comment (Result Details) has been changed to allow you to specify the Result Details File. For example, you could attach screen shots of failed steps or verification points.
Note: The Note, Description, Expected Result, and Expected Result File columns are read-only during execution.
Viewing the New Features in the Log
The first 64 characters of the description of each step or verification point are now written to a log when a manual test is run.
Now that you can associate file references with manual script steps and verification points during development and execution of the tests, you can gain access to these files when viewing the results. The Expected Result file and Result Details file for each step and verification point can be accessed from the tree view within the Test Log Viewer.
Executing Manual Scripts via the Web
The Expected Result, Expected Result File, and Result Details fields appear during manual test script execution using the Web.
ManualTest Web supports the latest Netscape versions. However, in Netscape 7.0 and higher, Expected Result files display links as path names only.
Deleting Multiple Log Assets
You can select and delete multiple test logs at the same time from the Results view of the Test Asset Workspace. Assets covered by this functionality include Builds, Test Log Folders, and Test Logs.
Test Inputs View
The Test Inputs view in TestManager now mirrors the Rational RequisitePro folder structure. This change is a result of performance improvements. TestManager no longer uses the COM layer to communicate with RequisitePro. Instead, it communicates directly to the RequisitePro database. We recommend that you organize your RequisitePro requirements in a way that makes them easy to find in the Test Inputs view.
Guidelines and Restrictions
Firewall Issues with Microsoft Windows XP Service Pack 2 (SP2)
During the execution of a test suite from TestManager, there are several programs that should be allowed through the Microsoft firewall so that they can access the local computer from the network. For information about how to allow this access if you are running TestManager on Windows XP SP2, see the IBM Rational Suite Release Notes v2003.06.13.
Unsupported Platforms
TestManager does not support Citrix or Microsoft Windows Terminal Server.
Windows Me
When you open a manual test script in Windows Me, part of the text disappears, word wrap does not work, and the backspace and delete buttons do not work. To restore the missing text, click another step or verification point and the text appears correctly.
Using Microsoft Excel Test Inputs
In TestManager you can view test inputs that have been defined in Microsoft Excel spreadsheets. If, while viewing those test inputs, you or another user creates a new test input or modifies an existing test input in Excel, you must refresh the TestManager Test Inputs view to see those changes in TestManager.
If you are changing the Date Modified column in an Excel spreadsheet, and the TestManager datastore and spreadsheet are in different time zones, the Update Suspicion feature does not work properly.
When using an Excel spreadsheet as a test input source, be aware that if you insert or remove a row or column in the spreadsheet, you may need to update the Test Input Source configuration settings.
When an association is made between a test input and a test case, the value in the ID column is stored by TestManager in order to preserve the association. Any modifications to the spreadsheet that affect the specified columns (ID and Name), such as adding or removing rows or columns immediately around the specified columns or rows, may invalidate the spreadsheet as a source of test inputs and require that you update the associated test input source. Modifying the values in the specified column or rows can also result in a loss of connection between the test case and test input.
Submitting Defects in the Test Log
You may see an error message about data code pages if your TestManager project is integrated with IBM Rational ClearQuest. The error message appears when you enter data into a tool and the characters you use are incompatible with the set of characters that a ClearQuest database accepts.
For example, assume that you are working in a TestManager project that is integrated with a ClearQuest database, and the ClearQuest database accepts characters in English only. If you use English to enter TestManager data, the error message does not appear. However, the error message does appear if you use another character set, such as Japanese or Chinese.
If you see this error message, consult with your ClearQuest administrator or systems administrator to resolve the character set incompatibility.
With the suspicion feature, you can define TestManager to inform you when requirements or test inputs change in a way that impacts test cases. TestManager detects changes in test inputs based on timestamps from the systems on which they reside.
For suspicion to work correctly, you must perform the following actions:
Synchronize the time on the systems. If the timestamps on the systems are out of synchronization by several minutes or more, the suspicion feature does not work.
Set the time zone for each system so that the suspicion feature can work correctly even if the TestManager and test input systems are in different time zones.
RequisitePro projects that use a Microsoft Access database may experience a delay between the entry of the data and the display of the modified data. This delay is caused by a defect in the Microsoft MDAC component. A similar delay is apparent from TestManager when RequisitePro test inputs are involved. The most noticeable case is when suspicion is updated and cleared frequently in the Test Inputs view.
You can manually update a suspect link between a test input and the associated test case by clicking View > Update Suspicion from the Test Inputs view or Test Plan window.
Playback of VU test scripts is not supported for Windows Me or Windows 98, whether the system is a Local computer or an Agent computer.
To play back Oracle 7 with Oracle 7 libraries on HP-UX, you must set the following environment variable before invoking TestManager:
RT_MASTER_USE_STATE_FOR_SQL
Set the variable in the Environment tab of the System Properties Control Panel.
When playing back manual test scripts from TestManager, be sure to exit Rational ManualTest to terminate the script. The alternative, terminating the user from the TestManager console, is not necessary and may lead to unexpected results.
The Test Script Services (TSS) framework supports playback of test scripts either directly (by TestManager) or by an external proxy TSS server process. However, Visual Basic test scripts support only direct TestManager playback.
Visual Basic and Java test scripts that are automatically generated (by the Robot Session Recorder or by IBM Rational QualityArchitect) do not record script line numbers. As a result, the TestManager Script view feature (the Script check box under Tools > Options) does not work with these test scripts. If you manually add method calls to your Visual Basic and Java test scripts that pass test script line numbers to TestManager, the Script view works. The relevant methods are TSSMonitor.PositionSet (Visual Basic) and TSSMonitor.positionSet (Java).
You can press the F5 key to refresh all in the Test Plan window and in the Test Input window. This key does not work in the Test Asset Workspace. You must right-click a folder in each Test Asset Workspace tab or click View > Refresh Workspace from the main menu to refresh all the Test Asset Workspace.
Monitoring system resources during test runs is not supported for Windows XP Home, Windows Me, or Windows 98 for Local or Agent computers.
To monitor resources on an Agent computer that is running AIX, you must install the bos.acct package on that Agent if it is not already installed.
TestManager provides a "previous" button allowing you to change the test log selection after you have selected the virtual tester, script, and commandID filters to be used in generating a report. This causes a problem because the previous test log is not closed and conflicts with the new test log. If you need to change the test log selection after you have selected filters, start over rather than using the "previous" button.
The following list gives SAP testing limitations in TestManager:
A possible problem has been identified on Windows NT 4.0 when ClearCase 5.0 is installed. After performing a Unified Change Management (UCM) datastore operation from the Rational Administrator (such as create a datastore, check-in all, add to or remove from source control), we suggest you shut down and restart Rational Administrator before attempting another such operation.
Crystal Reports is used to generate or edit Listing reports. You can run a report regardless of the length of its pathname. However, if you attempt to edit a report whose pathname is longer than 80 characters, the message "Dictionary not found" appears. This is a Crystal Reports v8.5 restriction. If you create a new or customize an existing design layout, keep the root path to the test datastore as short as possible.
When playing back a SQL statement that generates multiple result sets, TestManager logs only the data for the first fetch. The other fetches show the number of rows returned, but do not log the data actually returned.
If a sqlfetch_cursor is run multiple times in succession with ALL_ROWS and EXPECT_ROWS 0, an error for the second sqlfetch_cursor is generated stating:
ERROR -1: {error} STATE=S1010, CODE=0, MSG=[Microsoft][ODBC Driver Manager] Function sequence error
When logging data returned from sqlfetch_cursor, sometimes the data is truncated and the next SQL command appears on the same line as a row of data. This is a logging problem only; the test script is executing correctly.
To test CORBA applications with TestManager, you must have the following software installed on the local computer:
Windows XP, Windows 2000, or Windows NT Version 4.0 with at least Service Pack 4
Before playback, you must import into the project the IDL for each interface referenced in a test script.
TestManager provides special IIOP monitoring states in the state histogram monitor window.
TestManager does not support playback of ASCII hex (opaque) data between computers of differing byte ordering. Byte ordering specification is marked in the IIOP protocol. It is possible that data may be transferred with big-endian byte order, even on Windows NT systems. This case is more likely to occur if the application is Java. The result is that data specified opaquely in these cases may not play back correctly, even though the recording computer and the playback computer have the same byte order.
TestManager supports playing back the following CORBA data types:
The following data types are not supported, and result in methods being scripted opaquely if they occur as a parameter data type in a method:
Only Solaris is supported for UNIX Agent playback of VU language CORBA/IIOP (VisiBroker 3.3 only) scripts recorded by IBM Rational Robot. The VisiBroker bind modus is not supported for playback. This limitation means that "VisiBroker" cannot be used to locate object references at play back. Therefore, another method, such as reading IORs (Inter-operable Object Reference) from a file, or specifying them explicitly in VU test scripts, must be used to locate objects when playing back test scripts from a Solaris Agent.
When using Rational Test Agent in a VMware integration environment (GSX Server/TestManager) you may see an error message stating that NuTCRACKER has experienced a fatal error and has not registered with the service. If this occurs, perform the following steps:
This allows you to perform your normal functional testing in the integrated environment.
Note: Do not use the powered-off state of VMware integration. Instead, use the suspended mode only.
There is a known problem with importing and exporting test case designs. If there are carriage returns in the Note or Description boxes, the test case design does not import or export correctly.
To ensure that test case designs import and export correctly, when you create a test case design, do not enter carriage returns in the Note or Description boxes.
However, if you need to have multiple lines of information in these boxes, you can do the following:
Using an activity name with double-byte Japanese characters causes an error in the ClearCase check-in procedure. It is best to avoid using them.
The following problems with Extended Log Detail reports and Extended Log Summary reports can be fixed by downloading new templates for these reports:
You can find the report templates on developerWorks Rational at the following link:
http://www.ibm.com/developerworks/rational/library/4320.html
Place the templates in the TestDatastore\TMS_Logs folder. They will become immediately available in the drop-down list when you create a new Extended Log report.
You can no longer start XDE Tester from TestManager. The toolbar button and menu command to do this have been removed.
Crystal Reports v10 does not support Windows 9x platforms. Therefore, the runtime libraries are not installed when TestManager is installed on Windows 9x.
If you uninstall TestManager v2002.05.00, the uninstall program does not remove the \windows\system32\crpe32.dll file. If you then install TestManager v2003.06.13 and attempt to run a listing report while the DLL file is present, TestManager displays an error message that you do not have a correct version of Crystal Reports installed. Remove or rename the DLL file to resolve this issue.
When running a manual test on a Windows XP system, you may need to change the program association of an expected result file so that it is opened with an .exe. ManualTest running on Windows XP cannot open an expected result file if the associated application is opened with a DLL.
To change the file type association:
The Rational TestManager Help for filter operator types incorrectly refers to the character asterisk (*) as the wildcard character for a string of characters and the question mark (?) as the wildcard character for a single character position. The correct characters are percent sign (%) for a string of characters and underscore (_) for a single character.
The Help topic "Filter Operator Types" should read as follows:
Previous releases of Rational TestManager stated that manual test execution is supported for Linux agents. This is not the case. Rational TestManager does not support manual test execution on Linux agents.
The following function is not documented in the current TestManager extensibility documentation.
Clears filters for a test input source.
HRESULT TIClearFilter(const TCHAR SourceID[TI_MAX_PATH], TCHAR ErrorDescription[TI_MAX_ERROR])
Element |
Description |
---|---|
SourceID | input. The handle identifying the connection to the test input source. |
ErrorDescription | output. The message to be displayed to the TestManager user if there is an error. |
This function typically returns one of the following values:
TestManager makes this call when a user chooses to remove a filter. Your adapter should clear any filters that have been set.
HRESULT TIClearFilter (const TCHAR SourceID[TI_MAX_PATH], TCHAR ErrorDescription[TI_MAX_ERROR]) { AFX_MANAGE_STATE(AfxGetStaticModuleState()); HRESULT rc = S_OK; CConnectionContext *pContext=0; // Figure out what instance of the adapter is in use. m_ProjectConnections.Lookup(SourceID, (void *&)pContext); // Make sure we've got a pContext if (pContext) { // Make sure the Instance of the adapter is connected and loaded if (pContext->m_lpDispatchProject) { //Clear your filters pContext->SetFilterSettings(""); } else rc = TI_NO_PROJECT; } else rc = TI_NO_CONTEXT; // Apply the filter pContext->LoadFilterOptions(); return rc; }
The IBM software support Internet site provides you with self-help resources and electronic problem submission. The IBM Rational Software Support Home page can be found at http://www.ibm.com/software/rational/support/.
Voice Support is available to all current contract holders by dialing a telephone number in your country (where available). For specific country phone numbers, please go to http://www.ibm.com/planetwide/.