Simple because it matters.
Maintained and developed for Munich Re by ERGO Technology & Services S.A
Data Pipeline Factory
The Data Pipeline Factory is an Azure cloud-based solution built to provide the right data at the right time for analytical solutions.
Mission
The Data Pipeline Factory is a managed service for analytical projects that addresses the challenges of data integration:
- Required data sourcesare stored in disparate internal and external environments and come in different formats
- Data change constantly
- Associated data is isolated, and the same entity comes with different identifiers
- Information is hidden in raw data without context
- Unstructured data contain a wealth of information that cannot be used
- Cyberattacks threaten data in motion and at rest
- Regulatory requirements need to be met (e.g., GDPR, HIPAA, etc.).
Key Features
What Data Pipeline Factory is based on
Key Features
Capabilities of the Data Pipeline Factory
- Dynamic data integration
(e.g., internal data, client data, third party data, web data, etc.) - Trusted data products
(e.g., data typing, data matching, data mapping, data enrichment, metadata management, etc.) - Smart solutions for structuring data
(e.g., natural language processing, table recognition, auto-mapping to target format, etc.) - Standardization of data
(e.g., mapping to uniform target format, sharing of algorithms, creating data pools, etc.) - Secure & compliant platform
(e.g., certified technical architecture, monitoring and exception handling)
Key Features
The Data Pipeline Factory principles
Unlike traditional infrastructure and operations services with highly standardized processes and strict responsibilities, the Data Pipeline Factory follows principles from agile development:
Example
Primary insurance data from customers
The continued demand for new pipelines and the increasing number of active pipelines requires changing the way of working to a more mature operating model.
Integration of Motor Insurance ta a Standard Data Model
Weekly Update
- Receive Custumer Data over SFTP
- Typically CSV Format
- Policy & Loss Data
Transform to Standard Insurance Data Model
- Mapping of Customer Data into Industry Standard Data Model
- Technical & Business Mapping
Generate Analytical Base Tables
- Generate model Specific Analytical Base Tables
Scoring
- Call Machine Learning Models over API
- Monitor Model Performance
- Retrain, if needed
Final Delivery
- Deliver Final Results into SQL Database
- Dashboards & API for Customers
Technologies
Technologies we use
