Global software solutions provider adopts Polarion QA to help maintain quality assurance and facilitate cross-organizational collaboration

Strong commitment to quality assurance

NetSuite, Inc. (NetSuite) is a leading provider of cloud-based business management software. The corporation enables global companies to manage core business operations in a single system that includes enterprise resource planning (ERP), accounting, customer relationship management (CRM), e-commerce and professional services automation. NetSuite provides companies with a real-time, dashboard-based technology that facilitates an intuitive view of current, role-specific business information. The company has won more than 80 industry awards for its products, innovation, leadership and corporate culture.

Creating great software for its global customer base requires a strong commitment to quality assurance. NetSuite has always excelled in offering thoroughly tested products. To help stay ahead of the industry curve, the NetSuite quality assurance (QA) team wanted to find a better way to manage its existing spreadsheet-based testing.

“We would create great tests and afterwards they would just sit on a spreadsheet,” says Milan Vondra, director of quality assurance at NetSuite. “Version control was a problem as it was difficult to determine which test sets were current. We needed a quality assurance management system with a central repository to take advantage of previous work as we started new projects. And we also wanted a solution with an extensible application program interface so we could customize the testing process.”

The solution

NetSuite conducted an extensive search to determine the best test management system to meet its needs. “After evaluating approximately 30 solutions over several months, we did not find one that met our needs,” comments Vondra. “I discovered Polarion QA by Google searching, and it turned out to be the clear winner.”

The evaluation team was impressed by Polarion QA™ software from product lifecycle management (PLM) specialist Siemens Digital Industries Software for a number of reasons. The team liked the unified test and quality management software because it was intuitive, provided a complete solution and cross-project visibility functionality. The company was also impressed by the rich Polarion QA application programming interface (API) for customization purposes.

The Polarion QA implementation started with three NetSuite sites, including Brno, Czech Republic; Manilla, Philippines and San Mateo, California, United States. Early user enthusiasm paved the way as the company continued to increase its number of Polarion users. Currently, NetSuite has over 170 employees using Polarion QA, and the company is prepared to extend the rollout to new sites around the world as word about the software’s efficiency continues to spread internally.

Benefits

NetSuite cites a number of the beneficial features of Polarion QA, including providing a central repository for managing test cases and other resources, ease of use, complete visibility capabilities across projects, customization and flexible workflow for structured testing.

Central repository for easy access

NetSuite quality assurance managers and testers are delighted with the centralized repository of Polarion QA. Prior to deploying Polarion QA, spreadsheets were used to store test cases and results. Although the company had a central store for spreadsheets, it was difficult to search for test cases, and version control was a continuing problem. These issues worked against the idea that re-using data can create more efficiency.

“With Polarion QA, everything is in a centralized repository, which allows us to easily access and review test cases and eliminate version control issues while improving efficiency and boosting productivity,” says Robert Delos Reyes, manager of software test engineering at NetSuite.

The central repository helps NetSuite take advantage of previously completed work. When initiating a new project, the team can use filters and checklists to search the repository for other useful test cases.

“Polarion QA provides great templates for creating new test cases,” comments Vondra. “The central repository also makes it easy for our teams to see what test cases have already been created. When they find favorable cases in the repository, they can easily modify them to meet their current needs. The adapted test cases also become available for others to search and use.”

Vondra notes the central repository is also used for training purposes. “New team members can search for the most relevant tests and see exactly how the tests were developed,” says Vondra. “Polarion QA supports document view, so users can read text that provides additional test information, guidelines, background or feature sets. This is a great way to train new people.”

Ease of use supports early adoption

Polarion QA has proven to be easy to use across NetSuite operations. “With my first experience, I was impressed with its user interface,” comments Vondra. “Polarion QA is very intuitive and easy to use, which is important to a global operation like NetSuite.”

Delos Reyes had the same experience. “Within the first hour of our Brno orientation session, it was clear that the easy-to-use Polarion QA software was going to be a powerful test management system for us,” comments Delos Reyes. “You don’t have to be an expert with Polarion QA, as it’s very intuitive.”

Vondra feels that the real mark of success is that he can see people using Polarion QA as soon as it’s deployed to the team. “In the past, some tools were simply abandoned because they were difficult to use or didn’t provide value,” states Vondra. “Our people quickly embraced Polarion QA, and we are getting requests to expand our deployment to other NetSuite sites around the world, including London and Barcelona.”

Project visibility facilitates traceability

The Polarion QA central repository and customizable dashboards provide NetSuite managers with important visibility into the company’s testing activities and leads to project traceability across quality assurance teams.

“With Polarion QA, I can easily check on my teams to see what they are doing, what tests they have in process, how they are executing those tests and the results they are getting,” Vondra says. “This process provides extremely important visibility into individual features testing as well as full software product testing. It helps us make sure, for example, if a new feature is ready to ship.”

The company also values insights that can be gained by pulling up past tests to compare with current testing. “If something fails, we have previously-run testing data to help us identify new problems,” Delos Reyes says. “The central repository makes it easy to draw information from previous testing.”

Enhanced collaboration creates efficiency

Individual and multi-team collaboration have been greatly enhanced with Polarion QA. The teams have found Polarion QA makes it easier and more efficient to divide up tasks and seamlessly merge results.

“Polarion QA software allows us to split tasks among team members and monitor how everyone is doing,” says Delos Reyes. “This was difficult with our spreadsheet process, because first we needed to make sure everyone was working on the most recent version of the checklist. It was also more difficult to later merge results into a master spreadsheet. By using Polarion QA, we eliminate these problems.”

The Siemens Digital Industries Software solution also makes it easy to collaborate across teams and geographic locations, an important option for global companies. NetSuite products provide comprehensive solutions for CRM, ERP, e-commerce and professional services automation, and these solutions can be integrated with multiple infrastructure points within an organization.

NetSuite has built a reputation for quality by deploying teams that focus on specific elements of the product suite, making sure that all parts transparently function together.

“Some of our teams focus on infrastructure and data center synchronization, and we have other teams dedicated to third-party services integration,” comments Vondra. “Others are working at the product level, and within those groups we have teams focused on specific feature sets. As everything is interrelated, it requires information sharing among all teams. Polarion QA gives us the ability to comprehensively collaborate with all NetSuite development and test organizations. It feels like Polarion QA makes collaboration about 10 billion times easier. Sharing spreadsheets across multiple groups for something like synchronization was typically a nightmare. This is one reason we are so thankful for Polarion QA.”

Customization

NetSuite values the flexibility and extensibility built into Polarion QA, from the customizable interface and its software development kit (SDK) of open APIs, to its rich support of variants across products and test groups.

“I would classify Polarion QA customization capability as being one of its killer features,” Vondra says. “One of the first things we did was to create an integration between Polarion QA and automated tests using TeamCity, a build management and continuous integration server utilizing Java. With the flexibility of Polarion QA, we can connect with TeamCity and choose which tests, or fields within tests, any given team wants to import. As we build thousands of tests, this is an essential feature.”

NetSuite teams also like the ease with which they can modify Polarion QA templates and processes to match their specific needs.

“Other products we evaluated typically had a common configuration that was forced on everybody, which is not effective,” Vondra says. “When a system doesn’t fit the way people work, they tend not to use the solution. I believe its flexibility is another reason why Polarion QA has proven so popular with our users.”

Workflow flexibility

NetSuite uses Polarion QA Workflow Manager to organize and visualize projects and automatically create new work items with status changes.

“We wanted to create a common configuration, yet allow freedom for the test creators,” Vondra says. “We wanted to make the process as fluid as possible. By using workflows created using Siemens Digital Industries Software’s Polarion, we have achieved this goal as we can allow flexibility while retaining the ability to enforce mandatory elements.”

NetSuite quality teams appreciate the fact they can start a new project by writing test cases in sentence format knowing that the Siemens Digital Industries Software’s Polarion QA software will later prompt them for required implementation specifics.

“The test initiation process is facilitated by using Polarion QA Workflow Manager,” Vondra says. “The software’s flexibility and definable structure makes this a great feature for us.”