SAP IBP Success With UST Test Strategy

By Melanie Olson and David Wyllie

Introduction

Many SAP customers have difficulty planning test scenarios due to a lack of test data.

Standard SAP IBP deployments start with a two tier landscape of shared DEV (development) and Test (QA) tenants which are insufficient for complex scenarios with large data volumes. In this article we discuss deploying an additional Test tenant into an existing two-tier system landscape to improve testing with production data. The additional Test tenant is now offered as a standard option when a customer deploys a three-tier system landscape.

Most S/4HANA and IBP system landscapes provided by SAP are a two-tier system: DEV and Production. A recent trend in the cloud is moving from a two to a three-tier system landscape, including DEV, Test, and Production.
Test includes data more representative of a Production system by taking either a snapshot, or a subset of Production data.

When SAP APO/R3 implementations were common, you had at least two systems to test. This is typically both a DEV and Quality environment. You use the DEV system for Functional and Unit Testing while in the Quality system you perform Integration and end-to-end testing.

In the SAP IBP landscape, many clients have a single environment for testing. This works well for mid-sized companies with a single project. However, large organizations may have many projects with different timelines, and testing becomes complex. You may need to include a third tenant to perform detailed testing.

UST has developed an implementation experience which includes a third Test tenant in an SAP IBP landscape.
A two-tier landscape with one Test system, limits your ability to test bug fixes before moving to Production. This  can cause delays and project extensions where multiple projects overlap.

Planning data loads for multiple projects can be time consuming, leading to data overrides.

Also, performance testing can be limited until a complete data is fully loaded from production.

One solution to preventing data overrides and cross contamination of configuration is to split planning areas, performing testing in one and continuing configuration in the other. This works well if the changes made are limited to data integration or key figure configuration. However, any changes to the shared master data may affect both planning areas. A separate effort may be required for the eventual planning area merge and regression testing, which requires an extra level of diligence in project management to avoid issues. We recommend a three-tier system for complete separation of data between large scale projects.

Migrating To a Three-Tier System

During a recent project with one of UST’s large customers, we added a third tenant to the existing system landscape. This new IBP system, called IBP Pre-Prod, in the second graphic below, sits between the DEV and Production systems mirroring the current SAP IBP Production planning area. This allowed our team to fully load a Test tenant with production data to conduct full testing and training without affecting other project configuration and build activities.

Separate DEV and Test environments allows you to perform configuration and testing together without affecting each other. For example, many large companies have multiple projects at the same time, which means the projects can be in various stages of testing simultaneously. Managing the data can be difficult.

For one project, a solution architect may create mock-up data to perform Unit Testing while also performing End User Testing or Training for another project. The two-system landscape can be suitable if the data maintained in the DEV system is similar to production, and not simply a mock-up. However, many companies make the decision not to integrate production data into their DEV system for proprietary reasons. To properly test the configuration prior to going live in Production, sufficient data must be available to test all scenarios, which means having near production like data in the DEV environment.

How UST did it

One of the positive outcomes of including a third instance to the original two-tier SAP IBP system landscape was a reduction in bottlenecks due to MTP issues. With the addition of the Pre-Production environment, UST now provides the opportunity to rehearse cutover tasks prior to going live in Production. Thorough End-to-End Testing includes fully automated execution of processes prior to all IBP MTP’s reducing the effort in running manual data loads and tasks. Another positive outcome is gaining a stable environment with realistic production data used for  UAT (User Acceptance Testing.
Some key learnings discovered during the 3SL implementation started with how to map changes between SAP S4/HANA and BW along with SAP IBP and CI-DS (SAP Cloud Integration for Data Services). It quickly became apparent that changes in SAP S4/HANA and IBP should be split into a minimum of two releases:

* S4 Production and BW changes in first release

*IBP and CI-DS changes in subsequent releases

Some key considerations discovered:

*  New data sources going from IBP to S4 Production should be de-coupled from IBP processes in the first release. (Why??)
*  Changes should be imported into Pre-Production on a periodic basis after a notification is sent to project managers of affected IBP projects. (Why??)
*  IBP changes should be imported into Pre-Production at least three weeks before MTP. (Why??)
*  A Production Cutover Task List should be created and updated with adequate details to ensure smooth execution in Production after each import/cutover in the Pre-Production System. (Why??)

With regards to integration and the addition of the Pre-Production IBP tenant, there were no changes to the CI-DS integrations task sequencing as the list of tasks remained the same. The same CI-DS interface in the DEV sandbox was run against the new IBP instance. A new IBP connection for the Pre-Production testing environment in the same IBP datastore was required and the system configuration was created targeting the Pre-Production planning area. The IBP promotion path was now:

Existing IBP Development -> New IBP Pre-Prod -> Existing IBP Production

The CI-DS promotion path for CI-DS was now: New CI-DS Development -> Existing CI-DS sandbox (no changes allowed)  -> Existing CI-DS Production

With a new CI-DS instance along with a new Pre-Production environment, there were a lot of changes to the connections.
For example, the new CI-DS instance (with a new ORGID) was created by SAP only in the DEV sandbox. It points to the existing IBP DEV tenant and will become the CI-DS Pre-Production environment, which then targets the new IBP Pre-Production environment.

Four Steps to Integrate

1. In CI-DS, create new Connection and set the new IBP Pre-Production as the Target
2. Copy all of the tasks from Production into Sandbox
3. Change the Target to new connection
4. Change the original

How do you replicate data from Production to Test e.g., CI-DS points to both systems? Put some analysis that UST has done as there is some complexity.

Data Stores point to Pre-Prod (Target), to emulate a Production environment.

Rebuild job templates and scheduling need to point to Pre-Prod.

Test Preparation

During the Test Preparation phase, business processes are identified to be tested, which leads to the test case script creation. Test script creation is performed by consultants and reviewed by business users. This determines what test scenarios, both positive and negative, and test steps are to be included in the test case scripts and by which business users.
Test cases include determining the SAP role required, detailed SAP test script steps and test data to be used in the test cases. When setting up the Test environment, the data set to be used must be related to the client business to successfully complete the scripts. Typically, copying all Production data is not feasible, so determining how much data to use is required. The data does not have to come from the Production system but should be sufficient to mimic each situation or scenario. During Unit Testing, simply loading 1000 into the system may be enough to test that 1000 is the output (data in = data out). By the time the project gets to End User Testing, it may be required to have actual order quantities and dates to test the full range of functionality. Simply entering a single order for 1000 may not be enough, and therefore loading a sample dataset from Production is often needed. Since the tester is often an end user, it is very helpful if the tester recognized the data set used in testing. To determine how much data is needed, consultants should work with the business team and stakeholders to get their inputs on determining the right amount of data. The completed test case scenarios are initially tested by consultants before business users begin Test Execution.

Test Execution

After Test Preparation is complete, business users begin Test Execution to determine what needs defect resolution. Defect resolution recording occurs in either manual Excel spreadsheets or using test automation tools like SAP Cloud ALM, SAP Tricentis, HPQC, Jira, etc. Test automation tools are used to manage testing of active business processes, run automated tests live, and perform actions in a simulated user interface. Customers test their customized processes to validate configurations and confirm gaps are closed. Regression Tests occur in the Run Phase, after go-live. Customers perform automated testing of their business processes each quarter after upgrades are applied to the Test environment. Defect status and reporting are documented, and technical issues are resolved by consultants before business users retest defect steps in Functional Testing.

Test Evaluation

 After test execution is complete, test evaluation begins and documents the testing process and recording defect analysis to resolve any functional and technical issues.

You also perform change impact analysis to assess the impact of the newly deployed SAP functionality.

Types of Testing

1. Functional Testing

Ensures that the SAP implementation is meeting business requirements using test case scenarios and scripts performed by consultants and business users to ensure quality of functionality using the appropriate test data. Defect analysis is performed and recorded to fix configuration and process issues quickly removing uncertainty.

2. Unit Testing

Typically performed by developers, Unit Testing is performed in the Development tenant testing RICEFW items including Reports, Interfaces, Conversions, Enhancements, Workflows, and Forms. It tests the SAP functionality and its various components.

3. Integration Testing

Used to test components of SAP functionality and business processes using real-world data showing that interfaces and reports are configured appropriately, typically performed in the Test (Quality) tenant. Integration testing shows that the triggers work and data transfers are successful.

4. Regression Testing

When upgrading or setting up a new SAP system, Regression Testing is used to ensure that the new functionality does not interfere with the existing infrastructure and resolves any critical issues. Updated features should not affect what is introduced in the new release.

5. Performance Testing

System Performance Testing includes testing SAP response times, bottlenecks and, if expected, that user load volumes / batch jobs can be supported. This is typically executed using automation tools providing performance analysis.

6. User Acceptance Testing (UAT)

Used to ensure that the SAP system can be used by business users, testing that it meets business requirements, processes, and functionality. This is typically performed after Functional and Regression Testing to make end users feel comfortable with the new business environment leading them to take full ownership of the new features in the SAP system.

7. Cutover Testing

During an SAP implementation, Cutover Testing is typically performed after Functional Testing and review is complete but before project go-live. It facilitates the transport of updates made in Development and Testing environments to the Production environment achieving end user sign-off. Business processes User role assignments and application jobs are validated. Any production issues that occur after project cutover and go-live are resolved during Hypercare.

8. Security Testing

Security Testing ensures that role authorizations are distributed appropriately to end users depending on job role providing either read or write access confirming what they can do and see in the SAP system.