CreditXpert has been growing as a company and expanding on their technology solutions for each of their clients. There were many reporting related challenges that were addressed as part of the initial assessment.

• The client’s usage of their products results in very large logs that CreditXpert needs to parse and map/transform every month. While their scripts handle the log parsing, the data volume is heavy and hard to analyze

• The log files could be archived but there was no central location to review the data after the files were parsed and saved. Going back to analyze a historical month required re-processing

• The product data captured in the operational logs was used for client billing and the duration of parsing/saving would often be delayed or not follow any traditional schedule

• Product data could be more useful to other areas of the company, like marketing, but the typical reporting mechanisms only allow for narrow views

• Comparing company data to 3rd party or market data could only be accomplished at a very high level

• Financial data such as the GL was not able to be frequently used and analyzed

• Standard operating procedures regarding the log files were not well documented and circulated. Knowledge was siloed to specific team members in most functional areas

Archetype and CreditXpert scoped out a modern cloud data solution that addressed the internal technology requirements, business requirements and budget/timeline. Together we embarked on a partnership to create new code, new processes, and new reports. The project addressed the following areas to improve the culture of data at CreditXpert:

• The first initiative focused on data requirements and governance:

  • The team used Jira boards in an agile manner to organize tasks and drive productivity

  • Documentation was a major focus where all inventories, metrics, dimensions, and new operating procedures were kept in an easy-to-maintain Confluence wiki site

  • The broader project team met with executive leadership weekly to provide updates and discuss issues and decisions

• Second was data staging and automation within the data warehouse

  • Fivetran was used to ingest flat files and other manual loads

  • Pre-processing using Amazon lambda functions and S3 buckets were automated and tested

  • Log file loading and parsing was completely re-written to minimize unnecessary code and provide efficiencies wherever possible

  • Frequency of loading data was set to as often as possible (now set to daily when prior loads were monthly/weekly)

  • Increased exception reporting and automated data checks cuts down on bad or errant data

  • Snowflakes staging, ODS and data warehouse layers updated daily with clean, single version of truth data

• Third was the reporting initiative which used Looker as the technology

  • Report catalog converted to Looker and tested with CreditXpert resources

  • Multiple functional groups trained on new reporting tool by Archetype and a train the trainer approach

  • Executive team trained on benefits and use cases for BI reporting solution

• Managed services were set up to help with enhancements and stabilization while the newly hired BI Lead was initiated to the role

  • There was an enhancement added to connect Quickbooks directly to Snowflake via Fivetran’s QuickBooks online connector once the Quickbooks project went live (right after the original project)

There were many successes garnered from the initial implementation and on-going improvements leading to high degrees of optimization. The initiatives above have allowed CreditXpert to realize the following benefits:

• A single platform for BI reporting encompassing functional areas: operations, finance, sales and marketing, management and leadership alike

• The same type of monthly and quarterly reports that took hours thru SAS processing now take minutes in the new platform

• Dependencies on 3rd party vendors for financial reporting have been eliminated and now take minutes to produce

• New data sources implemented after the initial implementation can sync into new environment via Fivetran and Snowflake

• A modern cloud data infrastructure built to last years which requires very little IT infrastructure support

• A secure data warehouse platform

• Well documented requirements, design and knowledge transfer documentation that is easy to access, read, and edit for all employees and contractors. Low on-boarding time required

• Development framework is well documented and followed to promote code changes and keep versioning in tact. Code testing within DBT Cloud and promotion requirements ensure a far less buggy or errant data structure

• Business needs have been constantly changing but the agile approach, governance mentality and modern tools allow for reporting to catch up quickly