Analyze - Identify Data Structure Issues
Armed with the results of profiling, the project moves into the analysis phase. Applaud's analysis tools include four styles of report generators and other advanced analysis features. We use the tools to quickly identify data quality problems such as redundant data, orphans, failures in data relationship rules, invalid integrity constraints, parent/child issues, etc.
The analysis tools also make is easy to "drill-down" into the data content problems found in profiling to further understand the issues. For example, suppose that profiling identifies unexpected values in a code field. Applaud's analysis tools can quickly identify the exact conditions where the unexpected values exist, and can easily provide detailed reports broken-down by each related business unit for subsequent analysis and resolution.
Depending on the size of the project, the profiling and analysis phase process typically takes from a few days to a few weeks (although it could be substantially more on huge projects).
The Data Quality Conference
We then hold a "data quality conference". Our professionals meet with the experts for the legacy system (typically the end-client) and experts for the new system (typically the consulting or software firm). We present the results of the profiling and analysis to the team.
At this point a common reaction from the end-client is: "Wow, we had no idea our data was in such bad condition!" When this occurs the next steps include a series of cleansing projects. In some cases the cleansing effort is as simple as providing the end-client with Applaud's reports and providing instructions for the correction. In other cases we utilize Applaud's suite of cleansing tools to cleanse the data. Among other capabilities, Applaud's cleansing tools include robust name/address cleansing and CASS certification.
Defining Data Mapping Specifications
Next, specifications for the data mapping must be defined. The results of profiling and analysis are used as the basis for the data mapping specifications. In other words, the mapping rules are based on facts learned during the profiling and analysis phase, instead of out-of-date documentation and incorrect assumptions.
Our professional consultants can document the mapping specifications. This is accomplished through further meetings with the experts for the old and new systems. Alternatively, we can leave the project while the consulting or software firm defines the mapping rules. We add much value in this phase but in some cases the consulting firm or software firm prefers the mapping specifications to be their product. We can adapt to any method for this effort.
Delivering the Solution
When the data mapping specifications are complete, we begin development of the transformation and load components. The development can be on-site if desired. Since Applaud's data repository enables us to easily simulate the target tables, the majority of the development can be accomplished in our offices in Chicago, thus reducing travel expenses.
We are typically on-site for all major test runs and for the actual "go live" process. However some clients prefer us to run the processes remotely, further reducing travel expenses.