Automated Functional Testing
By Bhrigu Malhotra
First of all let me make you all aware that I’m a developer and what you are going to read further is a developer’s account, so it may sound to you like a layman tester. But what I’m going to share is something which has been very helpful to me to ‘functionally’ test the web application our team has been building and I thought this may help you as well. So even if you are a tester, try to think the way developer’s think for some time, I’m sure this would be useful to you too.
Before I go ahead, I think we all understand that Functional Testing is testing the functionality of our application which means you’d probably be going through a login using some credentials, clicking some links, verifying some outputs - in simplest terms. Now just give it a thought that if you can automate these i.e. invest some time for once and then while you are sipping a cup of coffee, someone is testing what you coded. Change your code 10 times and run your test 10 times to ensure nothing breaks, so easy. Not just this if you have continuous integration server setup which runs your tests, you’d come to know immediately if and when someone else broke your code. Believe me ones who have used it, feel it is fantastic.
There are a variety of tools available for this over internet each with pros/cons but I’m going to share some info here about one which I liked very much and have been using for a while now. I’d also share why I preferred it over others.
Ok, no longer prolonging the suspense the tool I’m talking about is called Canoo WebTest (webtest.canoo.com) which is a free Open Source tool for automated testing of web applications in a very effective way. The primary reason why I liked it is that it is FAST; very fast … most of the test cases that I’ve written are executed in less than a minute and that too over remote applications deployed on servers on the other side of globe. Second reason for liking it so much is the ease of writing; very quickly you can build up your tests. Though they have released a WebTest recorder plug-in which when installed in Mozilla Firefox can record the script for you, but believe me the more you start writing there test cases, the less reliant you will become on the recorder. Third thing is these tests do not use a browser so that completely eliminated browser-specific issues which some tools are plagued with.
Now some things about WebTest which some people don’t like is you actually do not see the tests being executed i.e. loading of a browser window, automatic clicking on buttons and errors on the screen (if any). But that I feel that it is so by design …. Doing all this takes time and then browser issues too creep in, so I’m happy like this way only. I agree they can improve on reporting of errors but again the objective of this tool is to test and just to test, if you want to report issues manual intervention is needed.
At last I’ll just say go give it a try, I’m sure once past initial hiccups, you’ll start liking it the way I am.
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
Tuesday, May 26, 2009
Script Maintenance
Script Maintenance
By Renu Bala
Script maintenance is a great challenge faced by automation testers. As the course of development proceeds, applications undergo changes, which cause automation test scripts to break. For example, Developers change underlying attributes (like object name) that the test script rely upon to identify the object. So testers need to rework on scripts.
Script maintenance at some level is a fact of automation but too much maintenance can be a drain on team’s time, resulting in decreasing value of automation. Automation testing tool that build scripts that are resistant to breaking are extremely valuable.
Testers must constantly update scripts to handle minor application changes but testers may make mistakes when they update scripts.
One solution to these changes is to update the object map so that all scripts can reference updated object map and identify objects.
Second solution to these changes is Script Assure Technology introduced by Rational Functional Tester. This technology makes test scripts more resilient to changes in applications. With script assure technology, RFT evaluates the closeness of objects that it finds and attempts to match something that is closest. If objects are close enough, test continues with no intervention from tester and will log a warning so that tester is made aware of a possible discrepancy. So more tests can execute with less intervention, even with significant changes being made to objects in application. In this way RFT automatically finds changed target objects in application without object remapping. Benefit of this technology is reduced maintenance and more reliable scripts. Additionally, this flexibility is completely under tester’s control. Tester can determine what level of closeness is sufficient enough for the project.
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
By Renu Bala
Script maintenance is a great challenge faced by automation testers. As the course of development proceeds, applications undergo changes, which cause automation test scripts to break. For example, Developers change underlying attributes (like object name) that the test script rely upon to identify the object. So testers need to rework on scripts.
Script maintenance at some level is a fact of automation but too much maintenance can be a drain on team’s time, resulting in decreasing value of automation. Automation testing tool that build scripts that are resistant to breaking are extremely valuable.
Testers must constantly update scripts to handle minor application changes but testers may make mistakes when they update scripts.
One solution to these changes is to update the object map so that all scripts can reference updated object map and identify objects.
Second solution to these changes is Script Assure Technology introduced by Rational Functional Tester. This technology makes test scripts more resilient to changes in applications. With script assure technology, RFT evaluates the closeness of objects that it finds and attempts to match something that is closest. If objects are close enough, test continues with no intervention from tester and will log a warning so that tester is made aware of a possible discrepancy. So more tests can execute with less intervention, even with significant changes being made to objects in application. In this way RFT automatically finds changed target objects in application without object remapping. Benefit of this technology is reduced maintenance and more reliable scripts. Additionally, this flexibility is completely under tester’s control. Tester can determine what level of closeness is sufficient enough for the project.
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
How to Access Command prompt from QTP
How to Access Command prompt from QTP
By Parul Wahi
We can even access Microsoft Windows command prompt also known as MS-DOS from QTP interface.cmd.exe is a 32-bit command prompt used in Windows NT, 2000 and XP and offers disk and file maintenance functions as well as network functions. It can be found under System32 folder.
To launch cmd.exe from QTP write :
SystemUtil.Run “cmd.exe”, “”, “c:\WINDOWS\system32”,”open”
Once DOS window launches, this window can be leant and added to object repository
To write on command prompt window use the following code:
Window( “object class:=ConsoleWindowClass”).Type “Ping crestechsoftware.com”
Window( “object class:=ConsoleWindowClass”).Type micReturn
Using WshShell object
The WshShell object gives your scripts the ability to work with Windows shell .Your scripts can use the WshShell object to perform a number of system administration tasks, including running programs, reading from and writing to the registry, and creating shortcuts.
Following are the list of few methods used with wcript object:
1)The Environment property helps in getting environment variables
2)The Exec method returns a WshScriptExec object, which provides status and error information about a script run with Exec along with access to the StdIn, StdOut, and StdErr channels. The Exec method allows the execution of command line applications only.
3)Pop Up Method Displays text in a pop-up message box.
4)RegRead Method helps in reading from registry
Example
The following example demonstrates the basics of the Exec method.
Dim WshShell, oExec
Set WshShell = CreateObject(”WScript.Shell”)
Set oExec = WshShell.Exec(”calc”)
Do While oExec.Status = 0
WScript.Sleep 100
Loop
WScript.Echo oExec.Status
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
By Parul Wahi
We can even access Microsoft Windows command prompt also known as MS-DOS from QTP interface.cmd.exe is a 32-bit command prompt used in Windows NT, 2000 and XP and offers disk and file maintenance functions as well as network functions. It can be found under System32 folder.
To launch cmd.exe from QTP write :
SystemUtil.Run “cmd.exe”, “”, “c:\WINDOWS\system32”,”open”
Once DOS window launches, this window can be leant and added to object repository
To write on command prompt window use the following code:
Window( “object class:=ConsoleWindowClass”).Type “Ping crestechsoftware.com”
Window( “object class:=ConsoleWindowClass”).Type micReturn
Using WshShell object
The WshShell object gives your scripts the ability to work with Windows shell .Your scripts can use the WshShell object to perform a number of system administration tasks, including running programs, reading from and writing to the registry, and creating shortcuts.
Following are the list of few methods used with wcript object:
1)The Environment property helps in getting environment variables
2)The Exec method returns a WshScriptExec object, which provides status and error information about a script run with Exec along with access to the StdIn, StdOut, and StdErr channels. The Exec method allows the execution of command line applications only.
3)Pop Up Method Displays text in a pop-up message box.
4)RegRead Method helps in reading from registry
Example
The following example demonstrates the basics of the Exec method.
Dim WshShell, oExec
Set WshShell = CreateObject(”WScript.Shell”)
Set oExec = WshShell.Exec(”calc”)
Do While oExec.Status = 0
WScript.Sleep 100
Loop
WScript.Echo oExec.Status
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
How to Execute QTP script from Command Prompt using Automation Object Model
How to Execute QTP script from Command Prompt using Automation Object Model
By Parul Wahi
In order to run QTP scripts on schedule, we can run them directly from command line.
In order to do so we can make use of QTP Automation Object Model which can help us in automating QuickTest operations. Using the objects, methods, and properties exposed by the QuickTest automation object model, we can write scripts that configure QuickTest options and run tests instead of performing these operations manually using the QuickTest interface.
In the following code we are trying to execute a QTP script called “Google_search” from command line. Let’s create a script in QTP and save it as Google_search.Write the following code in a vbs (VB Script) file as save it as demo.vbs:
Dim qtApp ‘ Declare the Application object variable
Dim qtTest ‘ Declare a Test object variable
Dim qtResultsOpt ‘Declare a Run Results Options object variable
Set qtApp = CreateObject(”QuickTest.Application”) ‘ Create the Application object
qtApp.Launch ‘ Start QuickTest
qtApp.Visible = True ‘ Make the QuickTest application visible
qtApp.Open “C:\Documents and Settings\pwahi\Desktop\Google search“, True ‘ Open the test in read-only mode
‘ set run settings for the test
Set qtTest = qtApp.Test
qtTest.Run’ Run the test
WScript.StdOut.Write “Status is:” & qtTest.LastRunResults.Status ‘ Check the results of the test run
qtTest.Close ‘ Close the test
Set qtTest = Nothing ‘ Release the Test object
Set qtApp = Nothing ‘ Release the Application object
Now in order to execute QTP script from command line we have to execute the above created vbs file from command prompt which will in turn launch QTP, open the script, run it and will display the result in command prompt only. Open command prompt and write following code:
C:\Documents and Settings\pwahi\Desktop>cscript //nologo demo.vbs
And press enter
This command will execute the demo.vbs file which in turn launch QTP, open the script Google search, execute it and will display the result in QTP result file. We can also see the results in command prompt as shown below.
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
By Parul Wahi
In order to run QTP scripts on schedule, we can run them directly from command line.
In order to do so we can make use of QTP Automation Object Model which can help us in automating QuickTest operations. Using the objects, methods, and properties exposed by the QuickTest automation object model, we can write scripts that configure QuickTest options and run tests instead of performing these operations manually using the QuickTest interface.
In the following code we are trying to execute a QTP script called “Google_search” from command line. Let’s create a script in QTP and save it as Google_search.Write the following code in a vbs (VB Script) file as save it as demo.vbs:
Dim qtApp ‘ Declare the Application object variable
Dim qtTest ‘ Declare a Test object variable
Dim qtResultsOpt ‘Declare a Run Results Options object variable
Set qtApp = CreateObject(”QuickTest.Application”) ‘ Create the Application object
qtApp.Launch ‘ Start QuickTest
qtApp.Visible = True ‘ Make the QuickTest application visible
qtApp.Open “C:\Documents and Settings\pwahi\Desktop\Google search“, True ‘ Open the test in read-only mode
‘ set run settings for the test
Set qtTest = qtApp.Test
qtTest.Run’ Run the test
WScript.StdOut.Write “Status is:” & qtTest.LastRunResults.Status ‘ Check the results of the test run
qtTest.Close ‘ Close the test
Set qtTest = Nothing ‘ Release the Test object
Set qtApp = Nothing ‘ Release the Application object
Now in order to execute QTP script from command line we have to execute the above created vbs file from command prompt which will in turn launch QTP, open the script, run it and will display the result in command prompt only. Open command prompt and write following code:
C:\Documents and Settings\pwahi\Desktop>cscript //nologo demo.vbs
And press enter
This command will execute the demo.vbs file which in turn launch QTP, open the script Google search, execute it and will display the result in QTP result file. We can also see the results in command prompt as shown below.
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
Script Services
Script Services
By Renu Bala
Script Support Functions enables you to insert code into Functional Tester Script to perform variety of tasks. These tasks include:
• Calling a script from a Functional Test Script
• Inserting Log Messages into Functional Test Script
• Using timers with Functional Test Script
• Setting delays and sleep states for Functional Test Script Playback
• Inserting comments into Functional Test Script
The Script Support Functions dialog box has following tabs:
1. Call Script for Calling a script from a Functional Test Script
2. Log Entry for Inserting Log Messages into Functional Test Script
3. Timer for Using timers with Functional Test Script
4. Sleep for Setting delays and sleep states for Functional Test Script Playback
5. Comment for Inserting comments into Functional Test Script
To use script support commands Click on Insert Script Support Commands button on Recording Monitor Toolbar.
Call Script
While recording or editing a functional test script, you can insert a call to a previously recorded script. This lets you avoid repeatedly recording similar actions on the AUT by taking advantage of scripts that already exist.
CallScript inserts following code in the script at the cursor location:
callScript (“scriptname”)
Where scriptname is the name you selected in the script name field.
Log Entry
You can insert a log message into a functional test script and indicate whether it is a message, a warning or an error. During playback, RFT inserts this information into the log.
LogEntry inserts the following code into the script based on the option you selected (message, warning, error):
logInfo(“Message”)
logWarning(“Message”)
logError(“Message”)
Where Message is the text you entered
Timer
You can insert timers with different names into the same script to measure time it takes to perform a variety of separate tasks. You have to explicitly stop the timer to calculate the time taken by the set of events.
The code generated is:
timerStart(“name”)
timerStop(“name”)
Where name is the variable name you want to use for the timer.
Sleep
You can insert sleep command into Functional test script to delay the script for specified time.
The code generated is:
Sleep(seconds)
Where seconds is the time entered in seconds fields.
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
By Renu Bala
Script Support Functions enables you to insert code into Functional Tester Script to perform variety of tasks. These tasks include:
• Calling a script from a Functional Test Script
• Inserting Log Messages into Functional Test Script
• Using timers with Functional Test Script
• Setting delays and sleep states for Functional Test Script Playback
• Inserting comments into Functional Test Script
The Script Support Functions dialog box has following tabs:
1. Call Script for Calling a script from a Functional Test Script
2. Log Entry for Inserting Log Messages into Functional Test Script
3. Timer for Using timers with Functional Test Script
4. Sleep for Setting delays and sleep states for Functional Test Script Playback
5. Comment for Inserting comments into Functional Test Script
To use script support commands Click on Insert Script Support Commands button on Recording Monitor Toolbar.
Call Script
While recording or editing a functional test script, you can insert a call to a previously recorded script. This lets you avoid repeatedly recording similar actions on the AUT by taking advantage of scripts that already exist.
CallScript inserts following code in the script at the cursor location:
callScript (“scriptname”)
Where scriptname is the name you selected in the script name field.
Log Entry
You can insert a log message into a functional test script and indicate whether it is a message, a warning or an error. During playback, RFT inserts this information into the log.
LogEntry inserts the following code into the script based on the option you selected (message, warning, error):
logInfo(“Message”)
logWarning(“Message”)
logError(“Message”)
Where Message is the text you entered
Timer
You can insert timers with different names into the same script to measure time it takes to perform a variety of separate tasks. You have to explicitly stop the timer to calculate the time taken by the set of events.
The code generated is:
timerStart(“name”)
timerStop(“name”)
Where name is the variable name you want to use for the timer.
Sleep
You can insert sleep command into Functional test script to delay the script for specified time.
The code generated is:
Sleep(seconds)
Where seconds is the time entered in seconds fields.
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
SOA Testing Simplified
SOA Testing Simplified (Series-III)
By Pallavi Sharma
So far, we have undertaken the difficult voyage of web services, SOA-Architecture and have come out of it empowered with the knowledge of what this new buzz SOA is all about. In this series of blog we will delve deeper into SOA and understand how do, we go about testing SOA. We will also see what all tools both open source and commercial are available in the market to help us achieve it. We will also take each tool one by one and understand how to best utilize it to test SOA.
Let’s first understand what all issues needs to be considered when we test any web service: [Ref 1]
• Web Services are intrinsically distributed and are platform and language agnostic.
• Web Services can be chained with dependencies on other 3rd party Web Services that can change without notice.
• Web Services ownership is shared across various stakeholders.
• Web Services client developers typically only have access to interfaces (WSDLs) and lack access to code.
• High level of reuse of web services.
As we know web services are based upon loosely coupled architecture. They are platform and language independent which increases the complexity of testing them. A web service usually calls another web service which might be a third party service. Oftenly 3rd party web service changes without prior information resulting in your web service failure. Also, if you are using a 3rd party web service all you have access to is a WSDL document of the web service and no code is available. A 3rd party web service can be used by a large number of users raising performance and security concerns which needs to be taken care of also.
Keeping all this in mind we need to also judge a test tool which takes the following issues into consideration:
• No User Interface to easily play a test
• High level of reuse of the services
• Testing at the service and end-to-end levels
• Interoperability, conformance, security penetration…
• Support of SOA standards and specific implementations WSDL, SOAP, UDDI, HTTP, WS-Security, …
A test tool to test any web service should support the various SOA standards like WSDL, SOAP, XML. Usually there is no UI available to test a web service. So the test tool should provide a user friendly interface to test the GUI less service. It should also assist in testing the interoperability, conformance issues which do arises in web service testing.
The commercial tools which are available in the market to test web services are:
1. IBM : Rational SOA Tester
2. HP: LR and QTP
3. Crosschecknet
4. Parasoft
The open source tools for SOA testing are as follows: [Ref 2]
1. Eviware SoapUI 1.6
2. PushToTest TestMaker
3. WebInject
In the coming series we will take each tool one by one and understand how do, we test the web service using it.
References:
1. http://www.crosschecknet.com/soa_testing_black_white_gray_box.php
2. http://www.infoworld.com/article/07/05/11/19TCwebservicetest_1.html
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
By Pallavi Sharma
So far, we have undertaken the difficult voyage of web services, SOA-Architecture and have come out of it empowered with the knowledge of what this new buzz SOA is all about. In this series of blog we will delve deeper into SOA and understand how do, we go about testing SOA. We will also see what all tools both open source and commercial are available in the market to help us achieve it. We will also take each tool one by one and understand how to best utilize it to test SOA.
Let’s first understand what all issues needs to be considered when we test any web service: [Ref 1]
• Web Services are intrinsically distributed and are platform and language agnostic.
• Web Services can be chained with dependencies on other 3rd party Web Services that can change without notice.
• Web Services ownership is shared across various stakeholders.
• Web Services client developers typically only have access to interfaces (WSDLs) and lack access to code.
• High level of reuse of web services.
As we know web services are based upon loosely coupled architecture. They are platform and language independent which increases the complexity of testing them. A web service usually calls another web service which might be a third party service. Oftenly 3rd party web service changes without prior information resulting in your web service failure. Also, if you are using a 3rd party web service all you have access to is a WSDL document of the web service and no code is available. A 3rd party web service can be used by a large number of users raising performance and security concerns which needs to be taken care of also.
Keeping all this in mind we need to also judge a test tool which takes the following issues into consideration:
• No User Interface to easily play a test
• High level of reuse of the services
• Testing at the service and end-to-end levels
• Interoperability, conformance, security penetration…
• Support of SOA standards and specific implementations WSDL, SOAP, UDDI, HTTP, WS-Security, …
A test tool to test any web service should support the various SOA standards like WSDL, SOAP, XML. Usually there is no UI available to test a web service. So the test tool should provide a user friendly interface to test the GUI less service. It should also assist in testing the interoperability, conformance issues which do arises in web service testing.
The commercial tools which are available in the market to test web services are:
1. IBM : Rational SOA Tester
2. HP: LR and QTP
3. Crosschecknet
4. Parasoft
The open source tools for SOA testing are as follows: [Ref 2]
1. Eviware SoapUI 1.6
2. PushToTest TestMaker
3. WebInject
In the coming series we will take each tool one by one and understand how do, we test the web service using it.
References:
1. http://www.crosschecknet.com/soa_testing_black_white_gray_box.php
2. http://www.infoworld.com/article/07/05/11/19TCwebservicetest_1.html
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
Performance Testing: Handling maximum database user connection
Performance Testing: Handling maximum database user connection
By Pankaj Goel
While working on performance testing assignments, many a times, I came across situation where following error was reported while applying load on the system
“The Maximum Connection Limit Has Been Reached”
On further analysis, I found that the error was being returned by the database (MS SQL Server in this case). Here is in this post, I have tried to detail what all to do when you get this error
So for MS SQL Server (2000 or 2005), if you have identified that the number of user connections is high, you can control the number of user connections by doing one of the following:
• Enable Microsoft SQL Server to automatically add user connections as needed. Do this through the Microsoft SQL Server SQL Server Properties window by setting the user connections configuration option to 0. If you specify a value other than 0, Microsoft SQL Server allocates memory for those connections at startup, which burns up portions of the memory pool.
• Specify a maximum number of user connections. Set this amount to the expected maximum number of concurrent users. Microsoft SQL Server rejects new login requests after reaching the maximum number of user connections when this value is set. After changing the maximum number of user connections, you must restart Microsoft SQL Server.
To re-configure the maximum number of user connection, In SQL Server 2000, you can use the sp_configure stored procedure. When you install SQL Server, the default value for user connections is 0 (32767 concurrent connections).
a) Start SQL server from command prompt
sqlservr.exe -c -f
b) Issue the following command
sp_configure ‘user connections’, 0
go
reconfigure with override
This code sets user connections back to the default setting.
c) Restart SQL server normally
For further details about the optimum SQL server settings, resources @ the following link contain some good information
http://support.microsoft.com/kb/319942/EN-US/
http://support.microsoft.com/kb/224587/
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
By Pankaj Goel
While working on performance testing assignments, many a times, I came across situation where following error was reported while applying load on the system
“The Maximum Connection Limit Has Been Reached”
On further analysis, I found that the error was being returned by the database (MS SQL Server in this case). Here is in this post, I have tried to detail what all to do when you get this error
So for MS SQL Server (2000 or 2005), if you have identified that the number of user connections is high, you can control the number of user connections by doing one of the following:
• Enable Microsoft SQL Server to automatically add user connections as needed. Do this through the Microsoft SQL Server SQL Server Properties window by setting the user connections configuration option to 0. If you specify a value other than 0, Microsoft SQL Server allocates memory for those connections at startup, which burns up portions of the memory pool.
• Specify a maximum number of user connections. Set this amount to the expected maximum number of concurrent users. Microsoft SQL Server rejects new login requests after reaching the maximum number of user connections when this value is set. After changing the maximum number of user connections, you must restart Microsoft SQL Server.
To re-configure the maximum number of user connection, In SQL Server 2000, you can use the sp_configure stored procedure. When you install SQL Server, the default value for user connections is 0 (32767 concurrent connections).
a) Start SQL server from command prompt
sqlservr.exe -c -f
b) Issue the following command
sp_configure ‘user connections’, 0
go
reconfigure with override
This code sets user connections back to the default setting.
c) Restart SQL server normally
For further details about the optimum SQL server settings, resources @ the following link contain some good information
http://support.microsoft.com/kb/319942/EN-US/
http://support.microsoft.com/kb/224587/
http://www.qacampus.com
http://www.crestech.in
http://www.crestechsoftware.com.au
Subscribe to:
Posts (Atom)