|
Описание: |
We are seeking a Senior Data Engineer for the cutting-edge FinTech project from the US.
The Hiring Process
* Recruitment interview — up to 30 minutes;
* Technical interview — 1 hour;
* PM call — 30 min.
Requirements:
* 5+ years of commercial experience as a Data Engineer, with hands-on experience building and maintaining ETL pipelines
* Strong expertise in PostgreSQL, including writing, optimizing, and troubleshooting complex SQL queries in production environments
* Commercial experience with Apache Airflow, including developing, maintaining, and monitoring DAGs
* Hands-on experience with AWS data services such as RDS (PostgreSQL/Aurora), S3, EC2, CloudWatch, and ideally QuickSight
* Ability to communicate directly with clients, proactively ask questions to understand business requirements, and translate them into technical solutions
* Proven ability to work independently with minimal Business Analyst support, taking ownership of data engineering tasks from analysis to delivery
* Strong documentation skills, with experience documenting ETL pipelines, SQL scripts, workflows, and technical processes to support knowledge sharing and operational continuity
* Upper-Intermediate or higher English level
Responsibilities:
* ETL Development & Data Pipeline Management
* Design, develop, and maintain ETL pipelines for tenant data onboarding and validation
* Build and manage the data mapping tool/ETL platform (a core product stream)
* Automate data import and transformation processes to support the goal of onboarding ~4 new tenants per quarter
* Database Performance & Script Optimization
* Analyze and optimize long-running PostgreSQL queries that cause production performance degradation
* Rewrite and improve SQL scripts used by the client team (tenant-specific data processing scripts)
* Set up CI/CD pipelines for SQL scripts
* Address IO spikes, database locks, and replication issues
* Apache Airflow Operations
* Configure, maintain, and develop DAGs in Apache Airflow for workflow orchestration
* Coordinate Airflow deployment and validation with the client team
* Monitor and troubleshoot Airflow pipelines in production
* AWS Data Infrastructure
* Work with AWS services: RDS (PostgreSQL/Aurora), QuickSight, EC2, S3, CloudWatch
* Support QuickSight optimization for reporting
* Contribute to AWS cost optimization related to data services
* Collaboration & Data Quality
* Collaborate with the client team on data processes and tenant scripts
* Ensure data integrity during tenant onboarding, payment history imports, and data cleanup
* Handle deduplication and data validation challenges in the multi-tenant environment
Відгукнутись на вакансію |
