ASA has been evolving as a company over the past few years with a renewed focus on how the data they receive can help them in the competitive higher education landscape. ASA reached out to Archetype for an initial assessment with the following gaps.

• The data that ASA gathers through partnerships, school districts, government statistics, and career services all live in isolated source files and data silos

• Since ASA has sensitive information, school and student data needs special consideration and the shared drive previously in use, while secure, would not pass for best practices

• ASA needed a single platform for the data and load processes to sync the data on a regular basis

• ASA lacked repeatable reporting capabilities; efforts were typically spent gathering data from disparate sources and creating one-time use Excel tables

• Documentation on standard operating procedures was hard to find or needed to be created

Archetype worked alongside ASA on three components of the implementation project after identifying and assessing the gaps mentioned in the problem statement above.

• Archetype dedicated weeks of discussions on the topic of Data Governance and implemented Azure DevOps as the development framework tool to document (in the wiki):

  • Policies and Standard Operating Procedures

  • Documentation of the data sources, reports, and data dictionary

  • Solution implementation documentation

  • Code repository and development framework documentation (in the repo)

  • Training and knowledge transfer sessions documentation

• Data Staging initiative

  • Initialized Snowflake as the data platform and Fivetran as the ELT and scheduling tool

  • Initiated single-load files from over 40 source files into a secure blob storage layer and then loaded to the data warehouse staging layer

  • Additional sources were loaded via out of the box or custom connectors into the stage layer

• Data Automation

  • Sources requiring automation were set up to run on schedules (from every few minutes to once per week)

  • Data movements from the Stage layer were automated through the data warehouse reporting layer(s)

  • Transformations occur within the warehouse layers with built-in quality checks and alerts in case anything needs attention

  • Dev/Test environments are easily created to test new code and promotional paths include DevOps checks between coding teams

Archetype and ASA enjoyed a positive team experience throughout the project which completed on time and on budget. Upon completion, ASA garnered the following results:

• A scalable and sustainable first-class technology platform which adhered to a Zero Footprint Data Management goal

• Secure staging, ODS and reporting layers

• Automated transformations based on the clients required business and reporting logic that is documented and easy to maintain and update

• Reporting data marts that are always up to date including a separate “Scratch” reporting layer for analysts looking for a more custom and secure place to work on data science initiatives

• Less time compiling data sets for one-off reports and more time allocated to analysis which leads to greater thought leadership in the form of original industry content

• A stronger and more data-driven culture which promotes clarity and longevity of goals

• Ability to measure program and partnership improvements and outcomes with new KPIs and metrics

• A DevOps framework that allows their data engineers to maintain and build upon the ELT and data warehouse capabilities which will ensure future growth

• A governance framework that is documented and will continue to show promise as the one-stop-shop for all the client policies and procedures now and going forward