Posts

Outsourcing Your Software Testing: When Does it Make Sense

One of the biggest challenges facing any organization that produces software is testing it. Using in-house testing methods may lead to a number of issues. Insiders often have a particular feel for how a program works, and this can lead to them miss problems. Also, many organizations simply don’t have the experience dealing with testing as a specific trade. It’s not usual, therefore, for operations to outsource their software testing efforts. Here are a few key ideas to keep in mind as you face that choice.

Automation vs Manual Testing

Some tasks in testing are simply too hard for a human user to truly duplicate at a scale large enough to produce meaningful data. For example, companies trying to test the functionality of APIs will have a difficult time hitting servers with enough requests to simulate the load that millions of real-world users will produce in employing a fully deployed version of the software. This is a case where automation of testing efforts might prove to be highly advantageous. Similar cases often emerge when a company needs to test the scalability and endurance of software, and automation is also a common choice for testing how well spikes are handled.

Conversely, automated systems often struggle to fully reproduce usage behaviour. User interface issues are especially hard to detect by any means other than real-world use by actual humans. In instances where the user interaction with the software is more important, it may be wise to look toward a manual testing solution.

For companies using automated models, many are turning to Agile processes. This approach allows them to focus on providing support in shorter sprints. In environments where frequent releases are anticipated, an Agile model using functional and regression testing helps to see that nothing is broken. This also limits the leak of issues into production models, since the process has multiple redundancies built in.

In-House vs Outsourced

There is often a strong temptation to test software solely in-house. In the early phases of a project, this can produce cost savings. It also has the potential to speed projects up, but this swiftness may be paid for later if in-house testers aren’t able to identify potential problems early on. Unless an organization has the resources required to effectively build its own independent, in-house testing division, there are huge possible downsides to handling the task that way.

Outsourced solutions for testing are common. Not only do outsourcing firms bring greater independence to the process, they also are capable of a level of specialization that may actually speed projects up. For companies overhauling legacy products, outsourcing options are often very helpful, as they often have competencies in the use of older systems and programming languages.

On-Shore vs Off-Shore vs Hybrid

The question of outsourcing software testing also leads to questions about exactly how close to home the testing provider needs to be. On-shore firms tend to be much less cost-effective upfront, but they often carry with them some added advantages. Foremost, employees of on-shore firms are more likely to be native speakers of your company’s language. They also are more likely to work in a time zone close to yours, and that can make scheduling of conferences and consultations simpler.

Off-shore firms tend to bring one specific advantage to the table: price. Work that might cost hundreds of thousands of dollars to perform in the U.S. may cost less than $10,000 to do overseas. That said, many off-shore companies can deliver surprisingly impressive results for the price. If you’re comfortable handling conferencing in a flexible manner, off-shoring often can yield major savings.

This is another case where companies often pursue a hybrid approach. Working with a consulting firm to determine what tasks are better to do on- or off-shore may also be beneficial.

Determining Qualifications and Expertise

The most widely recognized governing body in the industry that grants qualifications is the International Software Testing Qualifications Board. The ISTQB offers two tier, certifying engineers who have beginner-level experience of less than 5 years and those who have more. In many instances, it may be simpler to look at technical qualifications, like programming and administrative certifications from Microsoft and Oracle, to establish competency.

It’s wise, however, to not overrate the value of qualifications when dealing with software testing. When possible, you should ask services providers to supply case studies and references.

Conclusion

Software testing, especially once an organization has expanded to the point that it expects a large user base, is important to perform. The question that each company faces is how to divvy up the workload. Some tasks are ideally suited to automation, while others are best handled manually. Likewise, it may make more sense to employ an outsourced firm or even an off-shore company to deal with these aspects of the job. With a little forethought, you can see that your software will be tested thoroughly and without breaking your project budget.

 

If you have questions about outsourcing or are interested in outsourcing your software testing activities, give us a call. We’re always happy to answer any questions.

Popular Java Testing Tools and Frameworks

There are a wide variety of testing tools and test frameworks for automating the testing of Java/J2EE applications and server components. Many are aimed at unit or functional testing, while others are utilized for specific types of Java components such as view, logic and validation components.

Unit Testing

JUnit is perhaps the best known testing framework from which other frameworks are derived. It is useful in test-driven development methodologies. JUnit links into an application as a JAR. It possesses several valuable features such as annotation and assertions. Due to its simplicity, unit tests are run quickly with instant results via a red/green progress bar.

Jasmine is a unit testing framework designed specifically for JavaScript. Testers familiar with unit test frameworks such as ScrewUnit, JSSpect and JSpeci quickly spin up on Jasmine as it is closely related. Jasmine is especially useful for behavior-driven development methodologies. It is most popular for testing AngularJS applications.

JWalk is another unit testing tool for Java that supports a specific test paradigm known as Lazy Systematic Unit Testing. Tests are run using the JWalkTester tool on any compiled Java class. It supports “lazy specifications,” dynamic and static code analysis, plus insertion of programmer hints.

Functional Testing

The test framework HTTPUnit is based on JUnit and used for both unit and functional testing. It works as a browser emulator for such functions as JavaScript validation, cookie management, page redirection and form submission. It also supports the simulation of GET and POST requests.

TestNG is another JUnit-derived framework used for unit, functional and integration testing. It is growing in popularity compared to JUnit due to a number of extra features such as annotations, parameters, embedding BeanShell, flexible test configurations and the ability to test if code is multithread safe.

JWebUnit is a meta-framework that wraps other frameworks such as HTMLUnit or Selenium for use in functional, integration and regression testing of Java applications. It provides a single, simple interface for test case writing. It is also useful for screen navigation testing.

Java Test Frameworks

In addition to the test frameworks above, there are other component-specific frameworks that are popular for test automation.

JQuery Testing

For JQuery testing, QUnit is perhaps the most popular tool in use due to its simplicity and easily understood screen display. It has no dependencies on jQuery but supports all the browsers that jQuery 1.x does including IE, Chrome, Firefox, Opera and Safari.

Java Server Page Testing

TagUnit is similar in use to JUnit. It also has test cases, test suites and tests that are written as assertions. However, tests are written as JSP pages instead of Java classes. Each assertion is a custom tag. A single test is a collection of tags within a page. For a given tag, all the tests associated with that tag make up a test case. Tag classes are called when the JSP is converted to a Servlet.

Java Virtual Machine Testing

The most innovative and extensible framework for JVM testing is Arquillian. It is used by developers for rapid creation of automated functional, integration and acceptance Java tests. Tests are run directly from the run-time by managing the entire container life cycle and bundling all relevant test cases plus class and resource dependencies. It integrates with other frameworks such as TestNG, Junit and HTMLUnit.

Java Web Application Testing

Selenium WebDriver is an official W3C specification that provides a method for direct web browser interaction in the same way that humans do via a hook supplied by popular browser developers. It can simulate all user web actions.

HTMLUnit is another open source, headless browser testing tool. It is mainly used for integration testing via JSPs that run inside its web container and are converted to Servlets. Even without the container, however, it can be used to test the View portion of the application.

Other Useful Java Testing Tools

XMLUnit is JUnit extension for validating XML structure and performing content comparisons between actual and expected XML. It also shows the result of transforming XML with XSLT and XPath expression evaluations relative to specific portions of XML.

Apache JMeter is a valuable tool for testing website performance by means of sending multiple requests and displaying behavior graphically based on statistical analysis. It works over a large number of protocols including HTTP, HTTPS, SOAP/REST, FTP, LDAP, SMTP, MongoDB and others.

Conclusion

Java developers have a large number of testing tools and frameworks available to them of which we have briefly discussed the most popular and useful instances. These tools are used to automate testing throughout the entire software lifecycle from development to deployment.

Software Quality Audit for a Web App

We recently did a project for a local organization that was about to launch a new web-based application. They had the application developed by another vendor and wanted to complete due diligence before deploying to production.

We reviewed the application, compared it to the requirements, conducted end-to-end quality assurance testing, and conducted a full code review before giving our client a complete report including defects and recommendations. They then shared this report with their development partner so they could complete the project and deploy with confidence.

Read the full Software Quality Audit case study.

Five Software Testing Tips

1. Prioritize your Fixes

Your software may have more bugs than you have time for at the moment, so it’s important to prioritize your fixes, ensuring that the most important issues and the issues that could be easiest to fix are tackled first.

It is absolutely necessary to have a plan of attack, and by dealing with smaller things that only take a few minutes to repair and ensuring that the most important and most widely reported issues are dealt with, you’ll be more efficient than fixing everything at once despite priorities. Keep your team in the loop – whatever process you are using for software testing should be transparent to help ensure everyone is on the same page and that priorities are clearly drawn to improve efficiency.

2. Respond to Users

You may have a list of problems or bugs that exist in production, but perhaps your users haven’t noticed or don’t care. If users are not asking for a fix, the issue can likely be put on the back burner unless it alters the functionality of the software. Listen to your users to prioritize the issues that matter most to them. After all, they’re the ones actually using your software.

typing-300x170 Five Software Testing Tips

3. Use the Right Tools

Software testing tools exist for a reason, and there’s no reason not to use them! Find the best tools for your own software and stop relying completely on your own testing abilities – the right tool could point you in the right direction or help you to discover issues you may not have known about or wouldn’t find otherwise. Most importantly, tools often provide metrics to prove what you may think you know, or can offer time saving techniques to allow you to get on with what’s important.

4. Share Your Results

You may want to keep your test results to yourself until you’re finished testing and have an exhaustive list of everything that needs to be fixed; however, if you share your issues as you find them, it is more likely that a fix will be found with more ease.

5. Learn & Improve Your Skills

 It is essential to improve not only your skills, but your testing processes and testing workflow, to ensure efficiency in software testing. Don’t keep testing the same way you’ve always done it – explore new tools, engage with your users, network with other testers, and solicit feedback from your employees and co-workers. It’s important to always be learning and improving your skills in order to up your software testing game and ensure that you strive for personal excellence, resulting in thoroughly tested software.

Running AutoIT Executable in Selenium WebDriver

Web applications do not always confine themselves to working entirely on the web. Sometimes they need to interact with the desktop to do things like upload files.

Automating these sorts of workflows are tricky in Selenium. Selenium is confined to automating browsers, so desktop windows are out of scope.

If you are looking to automate workflows that go from browser to desktop and back in Selenium, then a little AutoIT is in order.

AutoIt is a tool for desktop automation. Using AutoIT, you can write scripts for automating desktop-based application and convert them in to an executable file.

The first step is to make automate that desktop-based window in AutoIT and make an executable file. The exact substance of the file depends on your project, but it is a pretty straight-forward task in AutoIT.

The second step is to invoke that executable at run time. Use the following method to invoke the executable from Selenium.

import java.io.IOException;
 import java.lang.Runtime;
 //Method to execute an exe file
 public void executeScript(String scriptPath) throws IOException
 {
 //scriptPath is the path of the executable
 Runtime.getRuntime().exec(scriptPath);
 }

As soon as you invoke this method, all the recorded steps in the executable will be executed and the desktop based window will be automated.

Some disadvantages of using the AutoIT executable are following-

  1. These scripts are window resolution dependent, so it may fail sometime.
  2. In case of uploading a file, if the uploading is taking more time due to some server/internet error, the scripts may not be able to fulfill the purpose.

To solve the first problem, you can use object information of the controls and window so you won’t have to depend on screen resolution.

To solve the second problem, you can specify the timeout period using the winwaitactive function which will give you definite results regardless of server or Internet errors.

Using predefined or user-defined AutoIT functions will make your Selenium scripts more consistent.

The post Running AutoIT Executable in Selenium WebDriver appeared first on OptimusQA.

Keyword-Driven Alert Handling in Selenium

Alerts and other pop ups cause your well-crafted Selenium tests to grind to a stop unless you handle them properly. The reason for this is that control moves to the alert instead of continuing with the open browser.

In most cases, you just need just need to automate accepting or rejecting the alert buttons which is relatively straightforward.

If you are using a keyword-driven framework, you need to do the following to handle alerts in Selenium.

  1. Find the ID of the button which opens the alert window. You can do this easily with Firebug in Firefox.
  2. Pass this ID. We have an Excel-based framework that we use where we can specify keywords and objects. The ID of the button would be the object in our framework.
  3. Invoke acceptAlertButtonByID or dismissAlertButtonByID keywords by specifying the exact name of the keyword in Excel.
The following Java code contains the keywords for accepting or dismissing buttons.
//To click the button by which an alert is generated and then accept the alert using id
    public  String acceptAlertButtonByID(String object,String data){
           APP_LOGS.debug("Clicking on button and acccepting the alert");
           try{
               driver.findElement(By.id(object)).click();
               Alert alert = driver.switchTo().alert();
              alert.accept();
               }catch(Exception e){
                    return Constants.KEYWORD_FAIL+" Not able to accept alert";
              }
            return Constants.KEYWORD_PASS;
    }

//To click the button by which an alert is generated and then dismiss the alert using id
    public  String dismissAlertButtonByID(String object,String data){
           APP_LOGS.debug("Clicking on button and dismissing the alert");
           try{
               driver.findElement(By.id(object)).click();
               Alert alert = driver.switchTo().alert();
               alert.dismiss();
               }catch(Exception e){
                    return Constants.KEYWORD_FAIL+" Not able to dismiss alert";
              }
            return Constants.KEYWORD_PASS;
    }

The post Keyword-Driven Alert Handling in Selenium appeared first on OptimusQA.

Working with Multiple Windows in Selenium WebDriver

I normally have at least three or four windows open at a time. Some of my workflows rely on being able to smoothly switch between my strategically placed windows.

I recently had to solve a very interesting window-related testing problem.

We were testing a SaaS desktop application in Selenium and some of its workflows presumed that the user would have three or four application windows open at the same time.

So how do you automate switching between windows in Selenium?

Selenium WebDriver assigns an alphanumeric id to each window as soon as the WebDriver object is instantiated. This unique alphanumeric id is called window handle. Selenium uses this unique id to switch control among several windows. In simple terms, each unique window has a unique ID, so that Selenium can differentiate when it is switching controls from one window to the other. So Selenium has following methods to get window handles.

To get the window handle of the current window:

String  handle= driver.getWindowHandle();//Return a string of alphanumeric window handle

To get the window handle of all windows:

Set handles = driver.getWindowHandles();//Return a set of window handles

As soon as a link within the application is clicked, the user is presented with a new window. Selenium, however, does not automatically switch control to the new window. This has to be coded manually.

To tackle this issue, I created the following keyword.

Code for Switching to Second window:

public String firstWinHandle;//It store the handle of main window of application
//This method Click on a link/Button in the main window and then switch control to second window
public void clickAndSwitchToSecondWindow()
 {
 //Perform the click operation that opens new window

 driver.findElement(By.id(“id of the link/button which opens new window”)).click();
 //wait till two windows are not opened
 waitForNumberofWindowsToEqual(2);//this method is for wait

 Set handles = driver.getWindowHandles();
 firstWinHandle = driver.getWindowHandle(); handles.remove(firstWinHandle);
 String winHandle=handles.iterator().next();
 if (winHandle!=firstWinHandle){
 //To retrieve the handle of second window, extracting the handle which does not match to first window handle
 secondWinHandle=winHandle; //Storing handle of second window handle

//Switch control to new window
 driver.switchTo().window(secondWinHandle);
//Control is switched now
 }

In the above code you can see there is a method waitForNumberOfWindowsToEqual().

So we also need to write this method to execute above code. Let me explain the approach I took.

We don’t know how long it will take to open the new window after clicking a link that opens a new window, so we need to hold up code execution until the window is ready. We could use Thread.Sleep()  but we don’t know how long we should wait for the window which depends on things like Internet speed and processor speed.

So instead of a time-based solution, we count the number of windows. Once the number of open windows reaches the expected number, we can start testing again.

Here’s the code for wait:

	 public void waitForNumberOfWindowsToEqual(final int numberOfWindows)
	    {

		//Making a new expected condition
	        new WebDriverWait(driver,30){}.until(new ExpectedCondition()
	            		{
	            @Override public Boolean apply(WebDriver driver)
	                          {
	                return (driver.getWindowHandles().size() == numberOfWindows);
	                          }
	                    });

	    }

This is how I implemented automated testing across multiple windows in Selenium WebDriver.

Please share your interesting and valuable thoughts in the comments.

Happy Learning.

The post Working with Multiple Windows in Selenium WebDriver appeared first on OptimusQA.

Perils of Virtualization

Virtual image is an instance of the operating system, which shares system resource such as CPU time, memory and file system to simulate a computer machine. At the heart of the operating system lies a component called HAL (Hardware Abstraction Layer), which converts the CPU instructions to meaningful operating system calls, which are further used by the high level application layers. Virtualization is a software layer and runs along with the HAL and operating system to allow the creation of machine instances called guest systems. Guest systems that simulate a computer are capable of running any operating system such as Windows and Linux.

On a powerful machine multiple guest operating systems can run in parallel. This is similar to operating systems running several applications in multi tasking mode; the difference being that, in this case, several operating systems run using a shared CPU model in a multi threading mode.

We have written about using virtual machines and VMware specifically for QA before and we are generally big fans of them. However, there are some security and networking issues that you should be aware of is you are planning on using virtual machines in your IT environment.

Security Issues

  1. Hypervisor Security: The hypervisor is the layer between physical hardware and the operating system. It is bound to have vulnerabilities that can directly impact security. If the hypervisor is hacked, security of the entire host system is at risk and this can cause security issues for the entire organization
  2. Guest OS Security: Most of the time anti virus software is not installed on the guest OS and this can provide a entry point for a virus in to your IT environment. It is extremely difficult to detect the machine which caused the issue.
  3. Networking Security Issues: TCP IP communication between VM and host can’t be sniffed by intrusion detection systems if the communication is host only. So IDS systems fail to detect vulnerabilities and exploits happening within virtual machines.
  4. Host External Communication Security: In certain cases the firewall on the host is disabled in cases where the VMs require some port to be opened. This causes severe security threats for the host.

Networking Issues

  1. Very High Bandwidth Utilization: VMs are operating systems. Windows and Linux try to update their software from the Internet and this causes N * Update size bandwidth usage in the enterprise. In situations where the VMs are restored from original images, this causes updates to happen again causing huge network bandwidth utilization in the enterprise.
  2. High DHCP Address Usage: When VMs are used in bridge mode, VMs consume DHCP addresses from the enterprise router. Many times VMs are shutdown within short duration of bringing them up. But since the DHCP server keeps the lease for 8 hours, this causes high DHCP address consumption .

The post Perils of Virtualization appeared first on OptimusQA.

Test Scenarios for Credit Card Payment Through a POS Application

Point of sale applications need to handle a wide variety of transactions like cash, debit cards, credit cards, gift cards and loyalty cards. Credit cards play an important role in payment for anything purchased. Below are some of the most important scenarios which should be tested with any credit card payment solution when integrated with a POS application.

Prerequisites

Credit Card Configuration: Includes configuration of card length, card range and card type (VISA, AMEX, MASTER etc)

Merchant Configuration: A merchant needs to be configured who is authorized to accept card type payments

Credit Card Processor Configuration: Credit card processor needs to be configured to process credit card payment (e.g. Mercury Payment System or Lynk, etc.)

Test Scenarios

Capturing Card Details: Following areas should be tested while capturing card details.

  1. Credit card number : Test card numbers using the correct length and range and card numbers that are outside the correct length and range.
  2. Expiry date: Test valid expiry dates, invalid expiry dates and invalid date formats.
  3. CVV number: Test valid CVV numbers , mismatched CVV numbers and blank CVV numbers.
  4. AVS code: Entering AVS details for configured numeric or alphanumeric formats.
  5. Card reader to capture card details: Test swiping of cards from both sides and chips.
  6. Encryption: Verify that captured card numbers are properly encrypted and decrypted.

Authorization: Once card details are captured, they are sent to processor to be authorized. Following areas need to be tested during authorization.

  1. Authorized amount: Test that the correct amount is being authorized.
  2. Receipt printing: Test that merchant and customer copies of the receipts and any vouchers print properly.
  3. Receipt details: Check that the receipts are printing the proper date, time, card details, authorized amount etc…
  4. Response code: Test that the correct response codes are being returned for approved, declined, on hold and all other transactions.

Settlement: Once the payment is done following things should be tested:

  1. Reprinting receipt: Test that you can reprint the receipt for a closed transaction.
  2. Void credit card’s payment: Check that you can void a payment before posting it and that after posting a payment voiding is not allowed.
  3. Verifying report: All information regarding each credit card transaction should be reflected in reports. Any adjustments made in closed checks should be reflected in the report.

The post Test Scenarios for Credit Card Payment Through a POS Application appeared first on OptimusQA.

Testing a CRM Application

Customer Relationship Management (CRM) systems are at the core of most customer-oriented business. A buggy or improperly implemented feature in your CRM application can have direct impact on how you understand your customers and hurt your revenue. Conversely, a well-implemented CRM can help you gain customer insight and improve customer satisfaction and loyalty.

crm-business-processes Testing a CRM Application

CRMs also contain a lot of sensitive data about your company and your customers. Any CRM testing program should ensure that the data is accurate and secure during storage and retrieval.

Testing Cycles for a CRM Application

regression-test-cycle Testing a CRM Application

1. Data Quality and Conversion:

The first test cycle focuses on issues related to data quality and conversion. At every step of a migration or update you should be verifying that the CRM is working as expecte3d both with and without data.

Look in particular for the following issues and data quality scenarios:

  1. Duplication of data: No redundancy is allowed.
  2. Wrong fields populated: Transaction details for one card type should not appear under the transaction history of another card. Loyalty card transactions appearing as gift card transactions.
  3. Hidden data is indeed hidden: Only the appropriate data should be visible to various user roles.
  4. Data maps correctly: Grid issues where a selected row’s data goes out of alignment with the fields as you scroll down  are very common.
  5. New and updated data saves properly: Customer information and card information should save and update properly.
  6. Partial and full search work as desired: Users should be able to search by first name, last name, company, card and whatever else is deemed necessary.
  7. No data is missing: All of the required data should be available to the right user levels.
  8. Graphs represent data correctly: Critical data like store-specific and program-specific filters and sales percentages should be accurate.
  9. Data sorting: All of the fields that need to be sortable should works as desired.
  10. CRM installs correctly: No critical GUI issues should arise during installation even when the user misses data during installation.
  11. Updating data is smooth and doesn’t require you to move to another form: Information such as customer visit count should update automatically without visiting additional forms.
  12. Data saved in one field does not move to another field after saving it: Fields like address, which may need more than one field, should save to the correct fields.
  13. Check that data which is not supposed to be editable is indeed not editable: Data like date and time of transaction should not be editable.

2. Functionality:

The second cycle of testing focuses on testing the functional aspects of the system. Regression testing of data quality and conversion should also be done.

Here are a few typical functional areas and related scenarios of a CRM application that should be checked:

  1. Access level: User permissions are working as desired. In particular, non-admin users should not have access to any admin functions.
  2. Transaction upload: If the CRM integrates with a POS, then all of the customer’s transaction information should accurately update at the Point of Sale (POS) within a few seconds.
  3. Insufficient card balance: If the customer does not have sufficient balance in the card for complete order payment, then he should be able to pre-authorize for the amount equal to balance available on the account and select another payment for the remaining balance of the purchase.
  4. Connection lost: For an Enterprise-grade CRM application, when the connection between stores gets lost then cards should not work and the proper error message appears.
  5. Card information: A customer’s card number or card type should not be changeable after the transaction has been completed.
  6. Card amount: Negative balances never appear through direct or indirect methods.
  7. Partial search: Partial search for program numbers that can both act as loyalty cards and gift cards should return matches from both types of cards.
  8. Transaction type: Users should be able to change the transaction type before settling the transaction and transactions should settle properly after the change.
  9. Data mismatch: Customers with the same first and last name should not trigger data mismatches and other problems.
  10. Pre-authorization: When a transaction fails while a card is pre-authorized, the payment slip should not print incorrectly.
  11. Department-specific transactions: Any restrictions on which departments accept gift and loyalty cards are working as desired.
  12. Store-specific sales: Receipts are printing the correct store name and address.
  13. Tax: All tax-related scenarios should work fine while pre-authorizing and settling transactions including cancelling transactions.

3. Reporting and Integration:

The third testing cycle focuses on reporting and integration, passing data to or from external systems, with a regression testing done for data quality and functionality.

Testing CRM reports should cover the following areas and scenarios:

  1. Accuracy and reliability: The accuracy of  reliability of data and reports from the new CRM solution need to be checked to determine whether they match existing reports and how many, if any, reports must be revised.
  2. Exporting reports: Do these reports  export in all formats with the correct data.
  3. AND/OR filter: Check that AND/OR filters work properly even with lots of filters chained together.
  4. No input value: Check that fields with no input value are not getting past filters incorrectly.
  5. Labels match: Label names should be the same regardless of where they appear.
  6. Unnecessary dependencies: Generating a report at one store should not be dependent on any actions at other stores.
  7. Store filters: The correct data appears when store-specific filters are applied.
  8. Date/time: Reports should include the correct date and time of creation.
  9. Headings and build versions: Reports should show the correct headings and build versions.
  10. Report performance: Data should  populate the report in a timely manner.
  11. Date ranges: Filtering on date ranges should always be tested.

4. Regression and User Acceptance Testing:

All of the end-to-end business processes are validated by the customer through user acceptance testing. User acceptance testing is process-level testing. All of the testing that has been done to this point is module-level testing.

User acceptance testing should cover the following areas:

  1. Documentation: We refer to project documentation to write the test cases based on the requirements of the users of the application.If the original documentation is incorrect or the end-users have undocumented needs, then the documentation needs to be updated as per the actual customer’s requirement.
  2. Usability: In CRM applications, customization to things like the grid, forms, schema and templates affects the usability. Unless test confirms the implementation is acceptably easy to use, the application should not be released.
  3. Functional Correctness and Completeness: All features should be implemented correctly.
  4. Reliability: The CRM should meet or exceed service level uptime requirements and the end-users should not have to wait a long time for transactions to  upload and data should be available all the time.
  5. Confidentiality: Customer’s details (contacts, points, balance visits etc) should be kept confidential.
  6. Scalability: Test the application for a realistic number of stores connected to a head office.
  7. Stress: The following check points should be monitored for response time  with an increasing number of open (not settled) orders:
    1. Sending order to requisition (kitchen)
    2. Customer search (especially company search), and saving of the customer (on confirmation of customer info).
    3. Payment processing on order settlement.

    Begin stress testing by having 10 open checks and then settling them at the same time. Then increase the number to 20 open checks, then 50, 100, 300, and 600.

Finally, check that the CRM application is well integrated with the customer’s other applications or systems such as Outlook, Map Point and Squirrel Explorer.

5. Not-so-final regression test

Once all of the critical defects have been fixed run another round of regression tests on all of the above.

Any time requirements change (a common occurrence in CRM implementations), new features are added or defects are fixed, you should run these regression tests again.

regression-test-pyramid Testing a CRM Application

The post Testing a CRM Application appeared first on OptimusQA.

Events

Event-Driven Quality Assurance

We are excited to have Robert Reppel, the CTO at TwoTonic Labs, presenting to us on “Event-Driven Quality Assurance: How to test, prioritize backlogs and build better systems through business domain events.”

Robert Reppel wrote first commercial software applications more than 25 years ago. Since then, he has been involved in many more as developer, architect, entrepreneur, development manager and agile practitioner, for companies ranging from startups to large multinationals. His main interest is the evolution of software solutions over time and the cultures and organizational structures which shape them. He specializes in helping organizations to innovate and achieve faster turnaround for new product development while leveraging or replacing legacy assets.

Agenda:
5:30-6:00 pm – Registration & Refreshments
6:00-7:00 pm – Presentation from Robert
7:30-7:30 pm – Networking