CreditXpert’s products help individuals get better insights to their credit history and understand how actions can improve their score. CreditXpert offers cutting edge products which lead to better credit and the ultimate goal of homeownership.
Aimed at identifying gaps in the current vs. desired long-term state of the business.
Recommended technology solutions to better integrate disparate data sources that would also provide efficiencies in their data wrangling processes
Appropriate data warehouse tools to efficiently ingest, transform and report their required subject areas.
In an agile way using Jira and documentation was added to a Confluence wiki which was easy to navigate and included a data governance initiative as well.
CreditXpert has been growing as a company and expanding on their technology solutions for each of their clients. There were many reporting related challenges that were addressed as part of the initial assessment.
The client’s usage of their products results in very large logs that CreditXpert needs to parse and map/transform every month. While their scripts handle the log parsing, the data volume is heavy and hard to analyze
The log files could be archived but there was no central location to review the data after the files were parsed and saved. Going back to analyze a historical month required re-processing
The product data captured in the operational logs was used for client billing and the duration of parsing/saving would often be delayed or not follow any traditional schedule
Product data could be more useful to other areas of the company, like marketing, but the typical reporting mechanisms only allow for narrow views
Comparing company data to 3rd party or market data could only be accomplished at a very high level
Financial data such as the GL was not able to be frequently used and analyzed
Standard operating procedures regarding the log files were not well documented and circulated. Knowledge was siloed to specific team members in most functional areas
Archetype and CreditXpert scoped out a modern cloud data solution that addressed the internal technology requirements, business requirements and budget/timeline. Together we embarked on a partnership to create new code, new processes, and new reports. The project addressed the following areas to improve the culture of data at CreditXpert:
The first initiative focused on data requirements and governance:
The team used Jira boards in an agile manner to organize tasks and drive productivity
Documentation was a major focus where all inventories, metrics, dimensions, and new operating procedures were kept in an easy-to-maintain Confluence wiki site
The broader project team met with executive leadership weekly to provide updates and discuss issues and decisions
Second was data staging and automation within the data warehouse
Fivetran was used to ingest flat files and other manual loads
Pre-processing using Amazon lambda functions and S3 buckets were automated and tested
Log file loading and parsing was completely re-written to minimize unnecessary code and provide efficiencies wherever possible
Frequency of loading data was set to as often as possible (now set to daily when prior loads were monthly/weekly)
Increased exception reporting and automated data checks cuts down on bad or errant data
Snowflakes staging, ODS and data warehouse layers updated daily with clean, single version of truth data
Third was the reporting initiative which used Looker as the technology
Report catalog converted to Looker and tested with CreditXpert resources
Multiple functional groups trained on new reporting tool by Archetype and a train the trainer approach
Executive team trained on benefits and use cases for BI reporting solution
Managed services were set up to help with enhancements and stabilization while the newly hired BI Lead was initiated to the role
There was an enhancement added to connect Quickbooks directly to Snowflake via Fivetran’s QuickBooks online connector once the Quickbooks project went live (right after the original project)
There were many successes garnered from the initial implementation and on-going improvements leading to high degrees of optimization. The initiatives above have allowed CreditXpert to realize the following benefits:
A single platform for BI reporting encompassing functional areas: operations, finance, sales and marketing, management and leadership alike
The same type of monthly and quarterly reports that took hours thru SAS processing now take minutes in the new platform
Dependencies on 3rd party vendors for financial reporting have been eliminated and now take minutes to produce
New data sources implemented after the initial implementation can sync into new environment via Fivetran and Snowflake
A modern cloud data infrastructure built to last years which requires very little IT infrastructure support
A secure data warehouse platform
Well documented requirements, design and knowledge transfer documentation that is easy to access, read, and edit for all employees and contractors. Low on-boarding time required
Development framework is well documented and followed to promote code changes and keep versioning in tact. Code testing within DBT Cloud and promotion requirements ensure a far less buggy or errant data structure
Business needs have been constantly changing but the agile approach, governance mentality and modern tools allow for reporting to catch up quickly