SAP test developmentThis is the final post in the series on SAP project shared success criteria. Doing this entire topic justice has been more of a challenge than I had originally anticipated. At some point, I will put it all together in a PDF e-book.

This series leaves you with a lot to digest but over the years these items generally make up the foundations for a very successful project that is able to transform your business. This series has been challenging but interesting and I am glad it is complete.

In a few more weeks, I will be coming back around to try to wrap up my series on SAP reimplementation for little more cost than a technical upgrade. That one turned out to be more challenging than I had originally expected. Until then, here is the final installment in this series on project success criteria:

No. SAP or ERP Critical Success Factor Company Integrator
5 Experienced SAP consultants   A
7 SAP implementation strategy z A
8 SAP project management A z
9 SAP tools, templates, and resources   A
10 SAP scope development z A
11 SAP scope management A z
12 Strong SAP project and business communication (inward and outward) A z
13 SAP change management A z
15 Sufficient SAP training (user and project team training) A A
16 SAP system vendor and customer trust   A
17 SAP system design decisions z A
18 Amount of custom ABAP or other SAP coding z A
19 Appropriate SAP software configuration (system settings) z A
20 SAP system change control process   A
21 SAP data analysis and conversion A z
22 SAP test planning A z
23 SAP test development z A

Legend

A = Primary Responsibility for the success factor
z = Shared but secondary responsibility for success

21. SAP Data Analysis and Conversion

One of the major activities of an SAP project is data analysis and conversion. Often, projects need too many new data requirements or changes after the system goes live. One way to avoid some data correction in a live production system is to ensure that you have sufficient integration and cutover testing. We will look at testing in a moment.

The primary responsibility for data conversion is generally on the customer. Customer primary responsibilities include the following:

  • Identifying legacy and other data sources
  • Cleaning or scrubbing the data
  • Ensuring the converted data meets business processing requirements
  • Defining sufficient test plans to validate data is properly converted
  • Providing sufficiently skilled employees to work the data
  • Making as many data corrections as possible in legacy systems before conversion
  • Ensuring sufficient hardware resources are available for meaningful data conversion tests

I have developed a short list of the common data conversion risks as adapted from the SAP ASAP 7.0 Data Migration and Risk Assessment Template. That template, provided by SAP, contains a sample issue and risk log, along with some suggestions on how to manage the following common data conversion risks:

SAP Data Conversion Project Management Risks

  • Improper estimates of data migration effort and activities
  • Client-side skills gaps on data migration
  • Deferring data corrections until after go-live
  • Knowledge of and access to data sources
  • Insufficient or incorrect change control processes
  • Insufficient participation from key functional project team members (client and consulting)
  • Improper management of dependencies among functional areas or modules

SAP Data Technical Conversion Risks

  • Lack of agreement on system of record, data definitions, standards, transformations, conversion methods, etc.
  • Data migration tool(s) not well defined or settled
  • Inconsistent hardware sizing and data volume
  • Required historical data conversion (rather than using legacy systems for historical purposes)
  • Multiple legacy sources of data for single master record loads in SAP
  • Data gaps – no corresponding data record for SAP conversion
  • Different data structures between legacy systems and SAP. For example, SAP may have structured hierarchies, and the source data may not.
  • Poor data quality – errors, duplicates, inconsistent use of fields

Preparing conversion programs, or more specifically, using SAP’s variety of conversion tools and resources is a vendor responsibility. The vendor, who has set up the system with the master data requirements, is also responsible for providing data load templates as well. Further, the vendor’s functional consultants must participate in the data conversion design and development activities because they are the ones that set up the system and know the master data requirements better than anyone. The only exception to this is if you decide to bring in consultants who do knowledge transfer to your implementation team, and your own internal team does all of the system setup.

For an overview of the various data transformation methods in an SAP project, please see the following post:

Planning For a Smooth SAP Go-Live: Part 2

SAP Blueprint Master Data Requirements

As a final note, a system integrator should be able to provide a first pass at basic master data requirements by the end of the Blueprint phase. In fact, your SAP Blueprint should contain a technical blueprint section with significant amounts of detail.

  • It should include each master data type (Material Master, Vendor, Customer, Routing, Sales or Production BOM, Work Center, Chart of Accounts, Profit Centers, Cost Centers, etc., etc., etc.)
  • Additionally, it should include each master data SUB type. Using just the Customer Master as an example, it should contain the requirements of sub-data types. For example, you may need Ship-To Customers, Sold-To Customers, Bill-To Customers, Payers, Agents, Freight Forwarders, Carriers, one-time customers, web shop or online customers, etc. This type of breakdown and segmentation should be done for every data type so that you know what data conversions you need. You must include this level of detail in a sufficient blueprint.
  • At the end of the blueprint, to ensure that you have all of the master data requirements covered as well, your system integrator SAP blueprint document should contain the specific details of every transaction data type. That transactional data requirement will be a key part of the data conversion. For example, using the Material Master and inventory movements that SAP provides, the blueprint document should contain the specific types of inventory movement transactions you perform. Simple goods receipts, goods issues to production, goods receipt from consignment stocks, issues to consignment stocks, scrapping, shipping/distribution goods issues, receipt into restricted/quality hold, move into unrestricted from quality hold, transfer from plant to plant, transfer from storage location to storage location, etc.

If your SAP system integrator Blueprint does not have this level of detail, then you cannot simply set up the system and perform the necessary master data conversion. Further, you may want to fire your system integrator and demand a refund of at least part of what they bill.

By having this level of detail at the end of SAP blueprinting (process details, master data details, interface requirements, forms, reports, etc.), you can immediately move into setting up the system. If you lack this level of detail, you may find yourself forever in design mode (revisiting normal blueprint requirements) and blowing both the budget and the timeline.

I suggest you add this summary of the type of detail you want by the end of the blueprint phase to your contract.

22. SAP Test Planning

You should determine who needs to perform what testing, when, and where. You will also want to use this as an opportunity to get both power users and non-power users involved in the testing. This becomes the early part of knowledge transfer, as well as user acceptance testing. Inevitably, the additional resources from outside of the project team will discover gaps or items that might need to be addressed.

The Test Planning responsibility is primarily on you as the SAP customer. Your primary goal here is to ensure that every process you put in scope is represented in some type of testing script. For example, in your order to cash process, you might want test scenarios that include credit processing with and without product returns, third-party drop shipping to the customer direct from the vendor, pro-forma invoice processing and then follow-up commercial invoice processing, interfaces to and from various external systems, online order entry processing with and without manual intervention, backorder processing and sourcing from other facilities, etc. At a high level, you will be responsible for covering all of the key processes that need to be tested.

SAP Test Strategy and Testing Approach

  • SAP Test Management
    • Test planning, scheduling, and logistics/coordination
      • Identify users to perform testing
      • Identify who will coordinate testing activities
      • Location and equipment logistics for testing
      • Method for communicating test schedule to participants
      • Test data – common master data sets to use for testing. This is different than the converted data testing.
  • Types of SAP Testing and Methods for Testing (whether manual or automated)
    • Unit testing (single transaction)
    • Business Process testing (unit test strings within the same functional module such as SD, MM, PP, FI, CO, etc.)
    • Integration testing (Business Process testing, including the integration points with other functional modules)
    • Interface testing
    • Data conversion testing (test the data load programs)
    • Converted data testing (test the converted data at performing various functions)
    • User Acceptance Testing
    • Regression Testing
    • Batch Job testing (test batch programs, timing, and sequence)
    • Security Testing
    • Performance/volume testing
    • Any positive and negative testing requirements for security or transactions
    • Ad hoc testing of unplanned variations and variants
  • Testing Toolset (various spreadsheets or automation tools for building and managing the testing process).
  • SAP Test Reporting and Analysis
    • Test Metrics
    • Defect Management

Change Control Process (for an overview and details of the SAP change control process see SAP System Integrator Shared Success Criteria Evaluation 4 which focuses critical success factor on #20 on this list of “System Change Control Process”).

As for a test script, I have included an example– one that is a modified version of the standard options SAP provides as part of the older ASAP methodology.

EXAMPLE SAP TEST SCRIPT with Ad Hoc or Variant Section information.

23. SAP Test Development

As for test development, this is clearly a vendor-primary responsibility. Unless you use SAP’s integrated Solution Manager resources for testing, you will need to rely heavily on your SAP partner in this area. As an SAP customer, “you don’t know what you don’t know.”

Unless you have adopted an approach where you require the implementation vendor to act as pure consultants, where your own project team does all of the system setup, you will not know the detailed testing requirements. As a result, a vendor must be able to walk you through the transaction strings and dependencies. They must be able to take the entire process-related solution they blueprinted from master data creation through cash processing and any interfaces or manual steps in between.

Because of time, budget, and limitless possibilities, a project can only have so much testing. These limitations create a requirement that only a “limited universe” of testing can be accomplished, and SAP system integrators will usually require someone to sign off on those limited tests.

A few things to beware of here:

1) When the tests have been “gamed” to support only a successful outcome and not to actually test the solution.

2) When a system integrator pushes back on testing variations that they may not have documented in the formal test scripts.

3) Any vendors who engage in hard push back against additional ad hoc, variant, or converted data testing. Unfortunately, I have seen a couple of the major SAP system integrators knowingly push trash on their customers.

For a thorough testing program, you should always ensure your tests include any of the custom development objects as well as manual processing steps. For example, a thorough test process must include testing of any outputs (forms), interfaces, enhancements, and reports. Data conversions should also be tested but may be done separately. However, one strong word of caution here: Once converted data is available, even if you do not do formal testing, be absolutely certain to do as much process testing as possible with the converted data. Testing with converted data will reveal a number of potential problems that can be corrected or resolved before you go live.

Related Posts: