What we’ve learned so far is that the impact analysis is essential to test the right things at any given release time, from small patch to whole transformation scenarios. Each release comes with specific changes and to understand the exact impact on your business transactions it avoids wasting time testing untouched areas.
Your relief - test automation, but handle with care
Good, now your business key users understand WHAT to test (what is a big advantage compared to “test all”-approach). But is testing really business user’s business? Shouldn’t they better focus on their core tasks!? Testing YES, when it comes to acceptance tests and confirming the changes will meet their business needs. NO, when it comes to repetitive tests, the so-called regression tests (to make sure no existing functionality breaks). Regression tests are absolutely crucial but boring. So why not letting the machine do it for you. This is the essence of test automation. But be aware, it holds some traps for you.
• Test automation without proper risk assessment leads to higher costs to implement and maintain.
• Avoid automated UI tests when you can test the functionality from API-layer too. Broad testing of combinatorics from API combined with most common scenarios on UI is the recommended approach.
• Implementing test automation can be done in many wrong ways so that maintenance later becomes a burden. It is the same as writing code. You need the experts doing it; you need the right design! A crash course in test automation will not solve it. Beside WHAT to automate it is also about the HOW to do test automation.
Where to start test automation?
Your businesspeople have a pretty good view on what’s important to test. By interviewing them, you start assessing the risk from SAP application point of view and then drill down to transactions and functionalities within. Moreover, you must not forget the whole eco-system around and identify the most important interfaces and what applications interfere with your SAP system. Look at your end-to-end use cases and structure them. Risk, though, has to be assessed from these dimensions:
• Damage: what monetary or reputation loss you are faced with when the functionality breaks?
• Frequency: it might not be the monetary loss but using it a thousand times a day and failing is also no option.
• Test automation complexity: automation at all costs is not reasonable. In some cases, manual regression tests still can do it.
This structuring exercise is important to most effectively automate your test cases. The goal is to cover most of your risks by automated tests. This is the foundation to become agile.
Which specialists do you need?
First and foremost, testing requires curiosity. You need the right people being accurate and curious at the same time to uncover the unexpected. Key users will always try to make sure their own demand is well tested but once the test demand comes from SAP updates or other teams’ interfering changes, the interest to test becomes lower. Key users simply don’t have the time for comprehensive tests.
To bridge the gap between different teams and interconnecting SAP applications, to establish a sustainable test strategy and to kick-off or optimize test automation, you need experienced and dedicated QA specialists. It requires technical understanding as well as a decent understanding of the business you are in. The recommended setup is a combination of embedded testers and a test automation team. The embedded tester is the role close to business demand AND development, therefore understanding the risks from both ends. It builds the link to the test automation team. Within the test automation team the skillsets differ, leading to different roles: the methodology expert does the logical test case design and identifies required test data, the test automation expert overcomes the traps of automation and the execution expert focuses on maintainability of test cases, that execution of automated tests runs smoothly and the integration in your future Continuous Deployment pipeline is well managed. So, it’s not just about THE tester!
The technical solution is close
The good thing, technically the SAP world is well prepared for test automation and impact analysis! In 2020 SAP decided to freeze its development for SAP CBTA (Component-based Test Automation) and instead partner with Tricentis to bring their tool stack into the SAP Solution Extension world to leverage test automation capabilities.
Consisting of the following parts, it comes with:
• SAP Enterprise Continuous Testing (ECT)
• SAP Change Impact Analysis (CIA)
• SAP Enterprise Load Testing (ELT)
• Tricentis Test Automation for SAP (TTA)
SAP Enterprise Continuous Testing not only allows to take the necessary action for automating SAP tests, but also is the tool to automate all end-to-end scenarios within and beyond SAP. It is a model-based solution which means there are no special coding or scripting skills necessary. If you automate a SAP test or e.g., the CRM system that is interconnected with your SAP use case, the same mechanism and methodology is used to provide automated tests. With the support of 160+ technologies you can automate almost any end-to-end test scenario using the same platform and methodology.
TTA is the free version for test automation but with limitations compared to ECT. It is constrained to SAP technology and systems only but allows a quick start in test automation.
SAP Change Impact Analysis is the solution when it comes to understanding the impact of planned SAP changes. Two major advantages mentioned:
Firstly, SAP Change Impact Analysis allows to compare your current SAP ECC system with a S/4HANA environment and by doing so, addressing the questions mentioned in the article “Understanding the impact of SAP change”. Remember the asteroids arriving, it allows to destroy the big rocks coming on the path to S/4 early in the transformation process.
Secondly, it identifies those objects being impacted by the change and most relevant for test. Testing everything is no option, it is about testing the right (i.e. the most relevant) things AND taking the right decisions for test automation! This is achieved by the following:
• Understanding the SAP object structure and its dependencies (including custom code).
• Getting usage data from a production system.
• Combining both using an AI-based algorithm allowing to reduce the number of objects to test by 85%.
SAP Enterprise Load Testing prepares you for scaling your SAP system when load tests are required.
Conclusion
To summarize, no matter if you are using SAP CIA during the S/4HANA transformation phase, earlier than that in current SAP ECC environment or later after transformation, the value comes with each deployment. No matter if you deploy your custom code or SAP standard changes, you exactly understand WHAT are the transactions most-at-risk. Combining this information with the proper test automation strategy taking advantage of SAP ECT, the risk of deployment and preceding hypercare phase (the phase of fixing critical bugs revealing in production) tends to zero. And this comes at lowest test effort for business key users!
We hope, our explanation for understanding the impact of change and why test automation can support your roadmap to business agility, found some attraction.
Time for SAP Test Automation has come.