Chapter 2: Why MDM Is Needed -- Master Data Silos Issues
How do organizations with multiple business applications manage their data without master data management (MDM)? This chapter explores the issues that arise when master data silos are maintained by different teams within an organization.
In this section, MDM architects and experts Andy Walker and Jagadeeshwaran Ganapathy discuss how using SAP NetWeaver MDM with CRM, SCM and even NetWeaver BI can help deal with master data silo issues. Learn why you need to combine SAP NetWeaver MDM programs with your organization's implementation projects so that data conversion is performed appropriately.
Effective Master Data Management with SAP NetWeaver MDM, Ch. 2
Table of contents:
Using SAP NetWeaver MDM with CRM, SCM and NetWeaver BI
Designing your SAP NetWeaver master data management strategy
The importance of master data management following mergers and acquisitions
Enterprise MDM: SAP MDM for reducing data silos and duplicate records
Why MDM is Needed: Master Data Silos Issues
This chapter considers how organizations with multiple business applications manage their master data without MDM. We'll analyze the business issues that are caused when these applications and master data "silos" are maintained by different organizational teams. We'll discuss two detailed case studies and also consider the implications of business mergers and acquisitions. Finally, we'll describe how the issues of master data silos are further compounded for large organizations.
Two case studies in this chapter illustrate the current customer and vendor maintenance processes in an organization and the related issues. In Case Study 1, we consider the issues faced by a fictional Company C1 with a relatively "simple" architecture. In Case Study 2, we move on to discuss the additional complexity caused when company C1 then merges with company C2 and acquires another set of business processes and applications.
Let's first introduce an analogy.
Young children often fight for their favorite toy, and they refuse to share with their siblings or best friends. As they grow up and become business leaders, they will have new toys – marketing divisions, customer services divisions, and financial accounting departments. They will each look to control their piece of the organization and will be the system owners of the Customer Relationship Management application, the Sales and Distribution application, and the Financial Accounting application.
Unfortunately, they still need to share because the management of customer records and order taking, invoicing, and payment processes span the entire organization. Sharing master data and integrating business processes across the applications is a critical success factor to enable them to "play happily" in a successful company.
There is a significant business investment in deploying an SAP CRM, SAP SCM, and SAP NetWeaver BI business application. Shared customer and vendor master data enabled by SAP NetWeaver MDM is a prerequisite to maximize the business value from these investments.
Let's now move on to Case Study 1, where we will describe the Company CO1, which unfortunately hasn't understood the importance of sharing or the value of maintaining its customer and vendor records.
2.1 Case Study 1 -- Master Data Silos
The case study highlights the issues caused by maintaining business partner master data in multiple systems across an organization. Let's imagine that you are a senior business leader for the fictional Company CO1. You are interested in understanding more about your company's profitability and your total expenditure with your leading customers and vendors.
- You'll consider who are your most profitable customers and how you can improve your revenue and operational efficiency when dealing with a customer. Similarly you will consider your spending with a vendor and whether you are leveraging the scale of your company, by understanding the global overview of dealings throughout the vendor's corporate structure.
We'll consider how the customer and vendor master data records are maintained in each of these business applications. Let's now discuss the Financial Accounting application FI1 and the processes for creating vendor records.
2.1.1 Financial Accounting Application (FI1) -- Vendors and Accounts Payable
In Company CO1, the system owner for the Financial Accounting (FI) application is the Chief Financial Officer (CFO), and the business users are the Accounts Payable (AP) and Accounts Receivable (AR) teams. In our example, Company CO1 has outsourced its financial accounting operational processes to a third party.
Let's consider the various ways in which the vendor records are created in the FI application. The AP team may need to create a new vendor as part of the process to manage a request for an invoice payment or a request to raise a purchase order. The AP team may receive a create vendor request form. Many of the FI1 vendor records were created in the SAP FI CO application as part of a data conversion exercise. Mergers and acquisitions also provide a source of new vendor records.
The following are the create vendor processes:
2. The procurement team raises a purchase order request.
3. A vendor request form is completed.
4. Vendor records are created as part of a data migration exercise.
5. A merger or acquisition creates a batch of new vendor records.
Each of these processes are discussed in the following subsections.
1. A Vendor Raises an Invoice That Requires Payment
You may have assumed that to pay an invoice, a purchase order would previously have been raised in the FI application. However, because the CO1 business processes and systems are not integrated, in this scenario, the first time the AP team is made aware of a purchase is when the vendor sends an invoice that needs to be paid. The invoice includes the vendor's payment name and address details, payment instructions, and payment date.
The AP team contacts the relevant person in your organization to authorize payment. This can take time and impact the Service Level Agreement, which has a measure based on managing the payment process in a defined number of days.
After authorization has been agreed upon, the AP user then searches the SAP database to see if he can find the vendor. If the record cannot be matched based on the name and address details provided on the invoice, then he creates a new vendor record in the FI1 application.
In these situations, creating the vendor record is an additional, unplanned step in the invoice payment process. In this scenario in Company CO1, limited data quality checks are carried out; if the invoice contains incomplete vendor information, these will nevertheless be entered into your FI1 application. The main business driver is to arrange the payment of the invoice to the correct bank account, with the quality of the vendor master data record of secondary importance.
The vendor data attributes entered relate to the accounting view of the vendor: the payment name and address, the bank details, and the payment terms.
2. The Procurement Team Raises a Purchase Order Request
Company CO1 has a rule that all invoices exceeding $10,000 must also have a corresponding purchase order raised in the FI application. Your procurement team currently raises a purchase order request by sending an email to the AP team using a standard template.
Once again, the AP user searches the SAP database to see if he can find the vendor. If the record cannot be matched using the name and address supplied on the purchase order, then a new vendor record will be created in the FI1 application with the base details provided on the purchase order. In this scenario, the creation of the vendor record is an additional step in the create purchase order process.
3. A Vendor Request Form Is Completed
In Company CO1, there is a vendor request form for the setup of new vendor records in the FI application. The AP team has created the template for the form, but it is rarely used. The form includes the data attributes required for the accounting view of the vendor, including payment names and addresses, bank details, and payment terms.
It's important for your MDM program to design a suitable customer and vendor maintenance form that captures the relevant details for the whole of your organization and not just a subset, such as for the AP and AR teams. A clear and comprehensive master data entry form is an excellent way to enforce data standards, and to mandate and validate relevant data attributes.
A good question to ask is how many customer and vendor master data forms currently exist across your organization. Is there one per application? What are the mandatory attributes? What data validation is carried out? Is the completion of the form mandatory?
Later, we'll discuss how implementing an SAP Interactive Forms by Adobe solution can improve your customer and vendor master data capture processes. The user interface is slick and flexible; you can also enforce mandatory attributes and allow look-ups with dropdown screens. The Adobe forms are integrated with the SAP NetWeaver Portal so that the details can be automatically uploaded without rekeying the details. This is excellent functionality that we'll discuss further in Part 2 of this book.
4. A Vendor Record Was Created During a Previous Data Migration
Company CO1 previously used a non-SAP FI application but migrated to the SAP FI CO solution two years ago. The deployment team's major focus was to ensure that all outstanding transactions and AP open items were migrated to the new FI1 application. The critical success factor was to match the opening and closing balances and to provide an audit trail to support this.
However, the previous non-SAP FI vendor master data records had been poorly maintained. The system had limited functionality, which meant that duplicate vendor records were created to handle situations such as companies with multiple payment terms or bank details. It was also difficult to search the previous application, so duplicate records were created. The non-SAP data model was limited, and when converted to the SAP model, this created yet further duplicate records. The legacy systems data attributes were a subset of the SAP attributes.
To meet the deployments schedule and critical success factor, the data conversion team decided to copy all vendor records, including duplicates, to the SAP FICO application. This also meant that records for inactive vendors that had not been used for several years were also created in the new system.
Because only partial vendor details were migrated during the data conversion to the FI1 application, this made these records difficult to retrieve as part of the ongoing search processes. As a result, further duplicate records have subsequently been created in the FI1 application.
Your SAP NetWeaver MDM program needs to be closely involved with each of your company's major implementation projects to influence their data conversion processes. The master data standards that you'll mandate on all new vendors and customers records also apply to the records that already exist in your business applications.
You should avoid copying incomplete, inaccurate, and duplicate vendor master data records from an old system to a new system. Make every effort to prevent GIGO (Garbage IN – Garbage OUT) with any future deployments of business applications.
5. A Merger or Acquisition Creates a Batch of New Vendor Records
A merger or acquisition is another way vendor records can be created in your FI application. This is a similar process to a data conversion process and involves loading a batch file of records into FI1. Again, a primary business driver is ensuring that all AP open items are migrated to your FI application.
If a vendor record already exists in your FI1 application, this will result in a duplicate record being created. If the company you are acquiring has a different data model with varying data attributes, this can cause data conversion issues. The implications of mergers and acquisitions are discussed later in this chapter, in Case Study 2.
Let's now move on to consider the processes by which customer records are created in your FI1 application.
2.1.2 Financial Accounting Application (FI1) -- Customers and Accounts Receivables
In Company CO1, your AR processes are also outsourced, and there are also multiple ways in which a customer record can be created:
2. An invoice is raised in the Sales and Distribution application SD1 for a new customer payment address, which automatically creates a new customer record in FI1.
3. A create customer request form is completed.
4. Customer records are created as part of a data migration exercise.
5. A merger or acquisition loads a batch of new customer records.
Each of these processes is discussed in the following subsections.
1. The Sales Team Raises a Manual Invoice for a New Customer
Unfortunately, your non-SAP Sales and Distribution application (SD1) has limited functionality and isn't suitable for maintaining the complex contracts and agreements that your sales representatives have recently negotiated with several of your customers. Their innovative new offer attracts a lot of interest with new customers and is managed outside the current systems in spreadsheets.
The customer services team sends the invoice to the AR team for manual entry into the FI1 application. This contains the customer's payment name and address details. The AR user searches the SAP FI1 application to see if he can find the customer. If the record cannot be matched based on the name and address details provided on the invoice, then a new customer record is created.
2. An Invoice Is Raised in the Sales and Distribution Application SD1 for a New Customer Payment Address, which Automatically Creates a New Customer Record in FI1
In Company CO1, established customer interfaces exist between the SD1 and FI1 applications. The SD1 application maintains the majority of your customer details, including the legal entity details (or the equivalent to the SAP sold-to record) and the delivery, invoice, and payment address details. The SD1 legal entity and the payer details are interfaced to the FI1 application and stored as sold-to and payer records. A minimal attribute set is interfaced to the FI1 application, including the SD1 system key. The SD1 application currently contains duplicate records that are transferred into FI1.
3. A Create Customer Request Form Is Completed
A create customer request form provides the details to set up a new customer in the FI1 application. The AR team created the template, but as with the vendor request form, this is rarely used. The form includes the data attributes required for the accounting view of the customer, including the payment names and addresses, the bank details, and the payment terms.
4. A Customer Record Is Created During a Data Migration Exercise
As with the vendor records, at the time of the FI1 go-live, the data conversion team copied all of the customer records, including duplicate records, from the legacy system to the FI1 application. These records included customers who had been inactive for several years and were of poor quality, with missing address details.
Because incomplete customer name and address details were migrated to the FI1 application, these original converted customer records are difficult to retrieve as part of the ongoing search processes. This has been a cause of further duplicate customer records over time.
5. A Merger or Acquisition Creates a Batch of New Customer Records
A merger or acquisition is another way for customer records to be created in your FI1 application. Again, the primary driver at the time of the merger is to ensure that all AR open items are migrated to the new system so as a result all customers are transferred.
If the customer record already exists in your FI1 application, this results in a duplicate record being created. Also, if the company you are acquiring has a different customer data model with varying data attributes, this causes issues.
Let's now switch applications and consider how customer master records are created in your CRM application.
2.1.3 Customer Relationship Management Application (CRM1)
Your CRM1 business application provides the functionality to cover many customer interaction activities, including marketing, sales, and customer retention. CRM1 supports communication with customers via a number of channels, including the Internet, contact centers and mobile clients. Sales representatives can access data and functions contained in CRM1 via offline applications, which are then synchronized.
The advantages of the CRM1 application include providing a global overview of your customer data and understanding who your customers are and their contribution to sales. This overview helps you identify the potential strengths (and weaknesses) in your customer relationships and helps you better understand the people and accounts you are dealing with. You can also reach out to customers with better contact information details.
Your CRM1 application stores a lot of detailed information regarding your customers, including contact names, account details, channels of trade details, and complaint management information.
In Company CO1, the CRM1 application owner is the marketing director and the business users are the marketing team, the sales representatives, and the customer services division. Unfortunately, in Company CO1, however, data standards are not enforced when entering new customers in the CRM1 application. The sales representatives in particular have insufficient time to complete the administrative tasks.
Many of the customer record details were originally migrated into CRM1 from the Sales and Distribution application (SD1). At that time, all records were uploaded, including the inactive records and the duplicates. The SD1 and CRM1 data models are quite different, and only a small number of data attributes were initially transferred.
Two years ago, the marketing team purchased a list of prospects for a new market segment that was then being targeted. These were uploaded as a batch file into the CRM1 application, but unfortunately, the business idea did not materialize. The prospect list is now out of date, and these prospects are unlikely to be converted into new actual customers. However, there are no archiving procedures implemented with the CRM1 application, so these records continue to appear when users attempt to search and retrieve details. This data quality issue has caused negative feedback, and several more duplicate records have subsequently been created.
There are no operational interfaces from the CRM1 application to either the SD1 or the FI1 applications. Customer orders are placed in the SD1 application because of the requirements of the stock availability functionality. Unfortunately, the CRM1 application does not show the current financial transaction details with customers, which reduces the value of the functionality.
In Case Study 1, we describe a situation where Company CO1 isn't getting good value from its CRM program. The root causes are that the CRM program hasn't appreciated the importance of sharing its customer data and processes across the CO1 organization and also the difficulties of integrating "best of breed" applications. Dividing processes such as customer management, order taking, and invoicing across multiple, disparate business applications introduces risk and needs careful process and data design consideration. For the CRM program to be successful, it requires consistent, shared customer data with the SD1 and the FI1 applications. SAP NetWeaver MDM is the tool to provide this and to "glue" these applications together.
We won't be so bold as to suggest that by implementing MDM with CRM you will get a successful CRM program. However, MDM is an important building block for CRM that will help to overcome some of the obstacles by enabling customer data to be shared across the organization.
The process of converting a prospect record to a customer record is an important one for SAP NetWeaver MDM. At this point, duplicate records can be created, or incomplete data may be entered. If these prospects to customer processes are driven from the CRM1 application for Company CO1, then SAP NetWeaver MDM is required to be tightly integrated with CRM1.
2.1.4 Sales and Distribution Application (SD1)
The Sales and Distribution (SD1) application provides functionality to manage the customer sales, delivery, and billing tasks in Company CO1. Key elements include customer quotation processing, contract management, sales order processing, delivery processing, and billing and sales information details. The application owner is the customer services division manager, and the customer services division team members are the application users.
In SAP, the SD component allows users to manage sales and distribution activities. The business processes include scenarios for sales, shipping, billing, sales support, and sales information.
However, the SAP solution is not used in Company CO1, which chose instead to implement a non-SAP package solution several years ago. In the subsequent years, the company has tailored the solution to meet its specific needs. The original package provider no longer supports the application.
The SD1 application provides a lot of the customer management processing for Company CO1. Customers and contracts are created, orders are placed, and invoices are produced and sent both to customers and to the FI1 application for payment. However, in an increasing number of cases, invoices are now created in spreadsheets and sent directly to customers. Manual invoicing is now required because the types of contracts currently being negotiated are too complex for the SD1 contracts application. The manual invoices are processed in the FI1 application.
The customer records in the SD1 application are not in a healthy state. The original data migration process did make some attempt to cleanse the data, but the records have been poorly maintained in the intervening years. Archiving processes have not been implemented, and customer records are updated haphazardly, without formal data standards. The business users have moved their attention to the maintenance of the spreadsheets to manage the complex contracts, and there is little confidence in the quality of the customer records in SD1.
SD1 is now a "tired" application that has seen little business or IT investment in recent years. The operational support team has been reduced to a minimal level to save costs, and all of the team members who tailored the original solution have now moved elsewhere. Most of Company CO1's recent IT spend has been on the CRM1 and BI1 programs.
When Company CO1 decides to proceed with its MDM program, the SD1 application is a classical case where the consolidated MDM model can practically be applied.
SD1 now has a limited lifetime, and several business leaders are advocating that the application should now be replaced by SAP ERP 6.0 software. By following the consolidated model, the D&B D-U-N-S Number can be tagged to each of the SD1 customer records, and the key mapping can be stored in SAP NetWeaver MDM. Periodically, SD1 customer records will be extracted for comparison and to ensure consistency with MDM.
This provides the benefits of matching the SD1 customer records to the CRM1, FI1, and BI1 records without initially needing to change the SD1 processes to integrate with SAP NetWeaver MDM. This will be the quickest way to implement SAP NetWeaver MDM. The centralized model can then be designed as part of the SD1 replacement project.
We will now consider Company CO1's Supply Chain Management (SCM) application.
2.1.5 Supply Chain Management Application (SCM1)
The Supply Chain Management application (SCM1) provides the functionality to perform sourcing, procurement, and logistics management activities. It covers the movements and storage of materials, inventory, and finished goods. In global companies, sourcing may be managed on a global basis. The vendor master record details include the bidder details, and the sourcing and procurement contacts. The Head of Procurement is the SCM1 application owner, and the business users are the Supply Chain Management team members.
The quality of the vendor data in Company CO1 has been significantly improved in the past six months. The Head of Procurement recently joined the CO1 organization, and he has introduced some new ideas regarding the procurement strategies. As part of this initiative, many vendors have recently been blocked. However, the supply chain processes were severely impacted when a key supplier was suddenly made bankrupt. The Head of Procurement has seen the business value of SAP NetWeaver MDM for vendor management and is the initial MDM champion in the CO1 organization.
In many respects, establishing vendor master data management processes in SAP NetWeaver MDM is easier than for customers. Your procurement processes tend to have tighter controls with a defined set of rules that must be followed before company funds can be spent. Customer sales can be more entrepreneurial with marketing teams and sales representatives developing nonstandard innovative offers to generate new revenue.
Vendor data models also tend to be more straightforward as you will typically transact closely with the company's legal entity structures. Customer data models can be more complex with multiple ship-to and bill-to addresses to maintain.
For these reasons, vendor management is a good master data object to start your SAP NetWeaver MDM journey with in an initial pilot implementation.
Let's now consider how master data silos impact SAP NetWeaver Business Intelligence (BI1).
2.1.6 Business Intelligence (BI1)
The Business Intelligence Warehouse (BI1) provides the data and tools for analyzing, monitoring, and measuring your organization's key performance indicators. It gathers information from several applications (CRM1, FI1, SCM1, and SD1 in Case Study 1). Vendor and customer details are key data elements and business partner hierarchies also provide important business information, particularly for spend analytics and credit limit reporting. Typically, both vendor and customer names and addresses details are stored along with the company hierarchies. The BI1 application owner can vary, and in Company CO1, is jointly managed by the head of procurement and the customer services division manager. An operational performance team manages the BI Extract, Transform, and Load (ETL) processes to populate the data warehouse and to produce the standard management reports.
The Business Intelligence Warehouse has been Company CO1's biggest recent IT investment. The business case was based on the benefits of aggregating the data in one place to enable a better understanding of the total company spend and profitability. The BI1 solution also provides the functionality to drill down into hierarchical data to explore areas of business interest.
The BI1 application is required to provide reliable information easily to the people who need it, when they need it. It should facilitate speedier, informed decision making so that you can find the information you need quickly and with certainty.
Accurate and accessible information is needed to support management processes, which include setting customer and vendor plans and targets, monitoring operations, analyzing outcomes, and reporting the results to your key stakeholders.
Unfortunately, the poor quality of the underlying customer and vendor master data records in Company CO1 is a major barrier to the success of the Business Intelligence Warehouse. As we've described, inconsistent master data exists across the organization, with duplicate records, partial records, and out-of date records to analyze and no single source of truth. There is also redundant data, and the data models differ across the business applications.
The BI1 ETL processes attempt to develop some rules and logic to improve on this situation, but in the end, your application programmers cannot create system-matching rules when the core master data maintenance rules are not in place. Attempting to match company records across systems by using matching algorithms and phonetic matching techniques cannot overcome duplicate, incomplete and out-of-date records in the source systems.
Unfortunately, the BI1 reports now provide limited value. It's not possible to accurately identify some of your customers and vendors, and in the absence of the D&B Corporate Linkage, you can't aggregate spend and sales up to the Global Ultimate level. The BI1 application is unfortunately a victim of the Garbage in -- Garbage Out problem that we described earlier. The Business Information warehouse reports the CO1 Company had hoped would add so much value are hardly used. The misleading information provided by partial spend and sales aggregation reports are in many ways worse than no information because invalid business decisions based on the incomplete and inaccurate BI1 data can prove to be costly.
Not surprisingly, MDM programs are sometimes led by a Business Intelligence program initiative. Accurate customer and vendor records are an essential prerequisite for a successful BI program to deliver valuable business information.
However, if you are considering combining your SAP NetWeaver MDM and BI programs, then you should take care to fully consider your objectives. Ideally, your SAP NetWeaver MDM processes will be integral to the creation of the vendor and customer records in your consuming systems. These need to be established as frontend processes, to avoid the creation of duplicate records and to authenticate your business partners ahead of any transactional processing.
However, BI reporting is a backend process; that is, it captures the transactional data from the various business applications and provides analysis tools to assess the data. It's important not to confuse the SAP NetWeaver MDM and BI objectives in these BI-led initiatives because the frontend and backend processes are very different.
To summarize Case Study 1, vendor master data is maintained in two applications: SCM1 and FI1 (AP data). Customer master data is maintained in three applications: CRM1, SD1, and FI1 (AR data). Finally, both vendor and customer data are maintained and analyzed in BI1. The system, the master data attributes, and the organizational users are shown in Table 2.1.
|Application||Master Data Attributes||CO1 Application||CO1 Organization||Internal/External|
|SCM1 -- Supply Chain Management Application||Vendor purchasing names and addresses; supply chain relationships details||Non-SAP||Supply chain management team||Internally maintained by employees|
|FI1 -- Financial accounting application||Vendor purchasing names and addresses, payment terms, and bank details||SAP FI CO component (CO1 instance)||Financials AP team||Outsourced to an external third-party service provider|
|CRM1 -- Customer Relationship Management application||Customer contact names and addresses, marketing and relationships details||Non-SAP||Marketing and sales representatives and customer services division||Internally maintained by employees|
|SD1 -- Sales and Distribution application||Customer names and addresses, contract details, invoicing details||Non-SAP||Customer services division||Internally maintained by employees|
|FI1 -- Financial Accounting application||Customer payment names and addresses; Financial Accounting details, e.g., payment terms and bank details||SAP FI CO component (CO1 instance)||Financials AR team||Outsourced to an external third-party service provider|
|BI1 -- Business Intelligence||Business partner mapping tables; company hierarchy structures||Non-SAP||Operational performance team||Internally maintained by employees|