Snowflake Infrastructure Engineer
What you will be doing
Project Overview: Data Reconciliation & Reporting Pipeline
Objective: Build and operationalize a robust data pipeline in Snowflake to support regulatory and compliance reporting for the Acquire business line.
Environment:
- Data sources include internal financial transactions, Mastercard/Visa network merchant reports, and processor data on merchant activity.
- Data lands as raw files (CSV, XLSX) in AWS S3 and will be ingested into Snowflake via Snowpipe.
- Data will be standardized, reconciled, and exposed through views used by PowerBI reporting teams.
Project Pipeline & Technology Stack
Data Flow:
- Automated ingestion of files from AWS S3 to Snowflake (using Snowpipe).
- Data transformation with dbt (data build tool): raw → staged → conformed layers.
- Reconciliation logic to validate, match, and audit transactions across all systems.
- Creation of final reporting views in Snowflake for downstream PowerBI reporting (report development itself is out of scope).
Core Technologies:
- Snowflake
- dbt (Data Build Tool)
- Terraform (for IaC: managing Snowflake objects, AWS S3/IAM, etc.)
- Amazon S3
Objective: Build and operationalize a robust data pipeline in Snowflake to support regulatory and compliance reporting for the Acquire business line.
Environment:
- Data sources include internal financial transactions, Mastercard/Visa network merchant reports, and processor data on merchant activity.
- Data lands as raw files (CSV, XLSX) in AWS S3 and will be ingested into Snowflake via Snowpipe.
- Data will be standardized, reconciled, and exposed through views used by PowerBI reporting teams.
Project Pipeline & Technology Stack
Data Flow:
- Automated ingestion of files from AWS S3 to Snowflake (using Snowpipe).
- Data transformation with dbt (data build tool): raw → staged → conformed layers.
- Reconciliation logic to validate, match, and audit transactions across all systems.
- Creation of final reporting views in Snowflake for downstream PowerBI reporting (report development itself is out of scope).
Core Technologies:
- Snowflake
- dbt (Data Build Tool)
- Terraform (for IaC: managing Snowflake objects, AWS S3/IAM, etc.)
- Amazon S3