Implementing Generate's Greenfield Data and AI Platform

Highlights

• Successfully deployed the greenfields data and analytics platform on AWS using Snowflake’s Data and AI Cloud, Omnata for data ingestion, dbt for data transformation and orchestration of core business data models, and Tableau for self-service reporting and analytics.
• Converted manual reporting processes for the executive leadership team to a fully automated solution, reducing delivery time from several days to immediate availability.
• A threefold increase in Operations team's productivity by decreasing data and analytics reporting efforts.
• A twofold increase in data quality and accuracy.
• Implemented rigorous testing, validation, and data quality measures to ensure platform integrity.
• Developed a data and analytics roadmap to deliver new use cases, including newly unlocked Machine Learning and AI initiatives that were previously not possible.
• Provided strategic guidance in data strategy, including team training, capability recommendations, new processes, and technology suggestions.
• Established a robust partnership between Generate and Data Domain, resulting in Generate entrusting Data Domain to help lead and work in close collaboration on new data and analytics projects.

Introduction

Generate Investment Management Limited (Generate) is an award-winning (Consumer NZ People's Choice Award for KiwiSaver 2022 – 2025), New Zealand-owned KiwiSaver scheme and wealth manager, renowned for its strong long-term performance and industry recognition for community and environmental impact investments.

To support its growth and maturity strategies, Generate identified the need to consolidate and enable a scalable data and analytics capability. A robust and extensible data and AI platform was required to facilitate Generate's evolution into a data-driven organisation, thereby supporting both current and future business initiatives and priorities.

Following a competitive Request for Proposal (RFP) process, Generate engaged Data Domain to deliver on the following primary objectives:

• Establish Modern Data and Analytics Capabilities: Understand and consolidate business requirements, establish data, analytics, and AI capabilities to achieve Generate's medium to long-term vision and strategy.
• Assessment and Planning: Assess and consolidate current and future data infrastructure, develop a technical implementation roadmap with timelines and milestones, leveraging identified problems and opportunities.
• Data Platform Design: Design a scalable, secure, flexible data and AI platform architecture with appropriate technologies for ingestion, storage, processing, and analytics.
• Data Integration: Implement robust data integration processes to aggregate internal and external sources, ensuring data quality and consistency.
• Analytics and Reporting: Select and procure analytics tool, enable predictive modelling, machine learning, real-time analytics, and develop customisable dashboards and reports.
• Security and Compliance: Implement robust security measures for the data and AI platform and analytics environment, ensure compliance with Privacy Act 2020, FMCA, Generate's policies, and provide audit documentation and support.
• Training and Knowledge Transfer: Provide training sessions for internal team on data and AI platform usage and offer ongoing support and knowledge transfer.

The Problem

• The existing reports for the leadership and executive management team were complex and time-consuming to manually process, requiring significant man-hours.
• Some business initiatives were ambitious and achievable only after data consolidation and orchestration.
• The business aimed to unlock AI and Machine Learning capabilities and utilise their data for driving business decisions.
• Generate had limited in-house expertise in data and analytics, requiring assistance with their strategy, implementation roadmap, and capability uplift.
• Despite significant investment in new technologies for data creation and capture, the ROI was not fully realised due to the inability to utilise all the data to improve business processes, decisions, and opportunities.

The Solution

Data Domain took the following initiatives to deliver a comprehensive solution which addressed Generate's data and analytics goals.

Assessment and Recommendation

The initial advisory phase involved key business and IT stakeholders to understand critical data needs, pain points, future ambitions, and priorities. The outputs of this phase allowed Data Domain to:

• Identify several Use Cases that span the entire business to support the strategies anticipated to be delivered by the new Generate Data and AI Platform.
• Create a delivery roadmap and designed a cloud-native architecture centred on: ‍
  
  • Snowflake on AWS: Scalable data storage, compute, transformation and advanced analytics capabilities.
  • Omnata: Data ingestion.‍
  • dbt: Data transformation and orchestration of core business data models. ‍
  • Tableau: Self-service reporting and analytics.
• The design incorporated data governance by default, ensuring lineage, metadata, and access controls were embedded from day one.

Implementation

The implementation was phased with key checkpoints to ensure continuous alignment of scope, budgets, and delivery timelines. The phases had the following objectives:

• Build of the modern data and AI platform using industry best practices.
• Implement robust security and governance controls for the data and AI platform.
• Implement the MVP data product solution, covering end-to-end data processing from sources, data integration, data modelling, and front-end presentation in the form of dynamic dashboards and AI insight features.

Agile Delivery

The delivery followed a hybrid Agile approach, based on Kanban methodology to ensure continuous delivery of high-value features. Each iteration involved close collaboration with key business stakeholders to ensure outputs aligned with expectations and added immediate value.

Change Management and Knowledge Transfer

Executive sponsors played a key role in reinforcing the cultural shift towards data-led decision-making, supported by a dedicated change management plan which included:

• Platform usage training sessions for analysts and operational staff.
• Data quality and business process upliftment within business teams.
• Development of a data glossary and user-friendly documentation for the platform and use cases implemented.

Ongoing Support

Data Domain transitioned to a support model that included platform monitoring, knowledge transfer, and on-going advisory and implementation services, allowing Generate’s internal team to steadily build capability and confidence.

The Outcomes

The greenfields data and AI platform and MVP data product solution allowed Generate to improve its data and analytics capabilities. The new centralised Cloud Data and AI Platform offers a scalable, secure, and flexible solution that aligns with Generate’s business objectives. Data Domain delivered the following key results for Generate:

• Delivery of key reporting for the executive leadership team was reduced from several days to a fully automated solution.
• Generate is now in a position to activate their machine learning and AI ambitions using their stored data.
• Enabled self-service capability for data access and analytics which resulted in increasing the Operations team's productivity by threefold.
• Strategic guidance in data strategy with capability uplift for people, process, and technology, including hiring recommendations, establishing new processes, and technology suggestions.
• Formulated a data and analytics roadmap to deliver new use cases.
• Successful deployment of the greenfields data and analytics platform has forged a robust partnership between Generate and Data Domain.
• Generate has entrusted Data Domain with all data-related initiatives, collaborating closely to lead and deliver new data and analytics projects.
• Uplifted Generate's data and analytics capability by providing training and addressing knowledge gaps.
• Implemented rigorous testing, validation, and data quality measures to ensure the integrity of the platform which resulted in a twofold increase in data quality and accuracy.

Read More

Find out more about dbt or read about our work on dbt's page by clicking here.

We are a Snowflake Select Tier Partner. Read more here.