User Acceptance Test is often considered as one of the most intense project phases when the IT team passes the baton to Business in the project relay and holds their breath till Business users validate and verify the developed system and collect enough evidence to decide whether they are able to live with the end product. The greater the project scope and the longer the phase iterations are, the more tension there is in the air.
In that sense, the UAT test manager with well-established contact both to IT and Business stakeholders can significantly reduce this tension by ensuring tighter collaboration to make the UAT a success story. Let’s take a look at several most common UAT challenges where the closer attention is normally required.
Time & resources planning
Quite often UAT is done by Business testers in parallel to their daily business activities, which means in some cases that they are working up to 12 hours a day during UAT, or generally have very restricted capacity for testing. This is especially relevant when we are talking about local key users. Such setup can cause additional problems with the change acceptance, as well as quality of testing and should be avoided:
- the business testers’ availability and parallel project activities involving them must be considered in the project plan without too many overlaps (e.g., verification of production load files for data migration, key user trainings, UAT tests should rather not be executed simultaneously)
- for long End-2-End scenarios the test scheduling & tester assignment should be really thought through, e.g., the collaboration between different departments, the assigned testers having fitting Business roles and enabled to judge the test result, etc.
- the documentation of test results and required time for that activity must not be underestimated. If business users are not used to working with test tools the documentation of test protocols will take longer. Depending on the contract conditions and compliance requirements it might make sense to consider light-weighted documentation options.
- and of course, if there is a possibility to involve the dedicated business users earlier in the testing activities (prior to UAT), the better chances for smooth UAT are (e.g., some Walkthrough sessions with Business Users during Integration test phase, or joint review of the forms like invoices or delivery notes, etc.).
Key user trainings & Change Management
On the one hand, we have change management issues, when the business users are asked to “forget” the previous routines and learn new ones. On the other hand, the business users may be involved in testing for the first time and may be not familiar with any test methodologies or specific vocabularies. Altogether this might lead to the new requirements popping up very late or underestimated defects. To help handle this topic, two aspects shall be considered:
- set of trainings to be conducted prior to UAT:
- Key User trainings on the new system & process updates
- High-level introduction to test methodology & test procedures adapted for the project
- Test tool training - introduction of UAT supporter role from the project core team.
UAT Test scope
The concerns of the end users to be left 1:1 with the system after project end and to struggle alone with malfunctions not discovered during UAT is very understandable, but hardly can be mitigated with 100% test coverage of all E2E flows with all possible test data combinations in the acceptance test phase. Considering the complexity and size of SAP solutions, this is not achievable in reasonable timeframe. In the test methodology, the golden standard for such cases is the risk-based approach for test scope definition.
Besides risk-based approach, there are couple of points to be addressed during UAT preparation:
- include testing of “A day in a life” scenario into UAT scope – several real life E2E flows which represent the most critical areas
- test coverage of legal requirements must be regulatory-compliant
- if core team (IT) prepares the UAT test cases and not Business testers themselves, they should make sure that the tests are written in Business language (e.g., no checking of value changes in some internal system tables, etc) and Business testers are involved in the test case definition & review.
Collaboration & communication model
On the final note, let´s talk about communication, as it´s a powerful tool which may either compliment or harm any of the areas above, especially in case of remote work and distributed teams.
Firstly, the clear communication models shall be set up, including such aspects as:
- core UAT testing hours
- established Zoom/MS Teams rooms/ booked physical conference rooms to jointly test/discuss issues/ etc.
- available SME/bug fixers for relevant topics
- available UAT supporters with functional knowledge
Also, the main test procedures shall be integrated in the defined collaboration model, such as:
- setup of the optimal defect fixing procedures (not necessary every defect will be documented in the test tool)
- test execution (e.g., required level of documentation, ways to hand over the end-to-end flows to the subsequent departments, etc.)
- review process of the test results, identified issues, etc.
Conclusion:
Today we´ve taken a look together at the most common UAT challenges and brainstormed a couple of ways to untangle them which you can further tailor to your project needs. If you are interested in more details on how to set up a test approach for SAP project or looking for ways to enhance it, click on the button on the right and check out our whitepaper on Test Automation in SAP.
We at Sixsentix are the champions of quality assurance.