Over Zhe
Data (Platform) Engineer with 6+ years of experience building and operating production-grade data platforms on GCP. Specialized in batch and streaming ingestion, data platform reliability, and platform self-service for analytics and ML use cases. Co-architected a streaming-first market data platform in the European energy trading domain (~20 external sources, ~200 pipelines, ~90% legacy migration completed). Currently transitioning from permanent roles to freelance consulting, open to contract opportunities in the Netherlands
Chinees
Tweetalig / moedertaal
Engels
Tweetalig / moedertaal
Kan op locatie werken
Amsterdam (tot 50km)
Werkervaring
- Dexter EnergyData Platform Engineer / Data EngineerENERGIEseptember 2023 - maart 2026 (2 jaren en 6 maanden)Amsterdam, NetherlandsWorked on the Market Data team responsible for ingesting and operating energy market data used by trading and forecasting systems.
- Co-led and built the Market Data Platform (MDP) from scratch (Medallion + Kappa-style), a streaming-first ingestion platform replacing legacy ingestion with ~90% migration completed.
- Designed a connector-based ingestion framework (polling and push) with YAML data contracts as the single source of truth for dataset configuration and schema validation.
- Implemented 7–9 production connectors (EPEX MATS, EEX, Volue, APG, GME, IPTO, DAMAS, etc.) with custom rate-limit handling and schema validation.
- Delivered critical integrations such as EPEX MATS conformance and DAMAS PDF parsing, enabling support for key European energy markets.
- Built core platform mechanisms including sanitization, idempotent ingestion, and Redis-backed distributed deduplication to ensure consistent backfills and reliable streaming ingestion.
- Owned production reliability of market data pipelines, triaging data gaps, schema drift, and outages, and improving observability through Grafana, Prometheus, and Sentry dashboards and alerting.
- Maintained dbt consolidation layer feeding DWH, analytics, and model pipelines.
- FedExData Platform Engineernovember 2020 - september 2023 (2 jaren en 10 maanden)Hoofddorp, Netherlands
- Built and operated a GCP-based analytical data platform supporting multiple logistics and operational domains.
- Designed scalable ELT pipelines ingesting multi-domain enterprise datasets (logistics, orders, customer, company) and SaaS platforms (Salesforce, Google Ads) into a GCP-based data platform.
- Delivered self-service PySpark/SQL templates enabling 50+ analysts to build and schedule jobs independently, significantly increasing platform self-service capabilities.
- Implemented CI/CD pipelines for automated testing and deployment of data pipelines, improving reliability and release consistency.
- Enabled lakehouse data exploration through Superset, allowing analysts and product teams to query platform datasets directly.
- Operated platform infrastructure on GKE including orchestration, compute runtime, and monitoring of production data workloads.
- FedExData Engineermei 2020 - november 2020 (6 maanden)Hoofddorp, Netherlands
- Built data pipelines and microservices for the O2P migration (TNT -> FedEx).
- Developed orchestration tooling for hourly migration across users.
- Built an MVP with scheduled PySpark SQL jobs + microservices for automated campaign engagement (Salesforce/Adobe Target).
Aanbevelingen
Wees de eerste die Zhe aanbeveelt
Help deze freelancer om te schitteren door te vertellen hoe het is om met hem of haar te werken.
Deze freelancerprofielen matchen ook met zoekopdracht.
Agatha Frydrych
Backend Java Software Engineer
4.7
(3)
2
Baptiste Duhen
Fullstack developer
4.6
(4)
5
Amed Hamou
Senior Lead Developer
4
(2)
7
Audrey Champion
Web developer
4.3
(3)
4
Opleidingen
- MSc Biomedical EngineeringUniversity of Groningen2016MSc
- BSc Biomedical EngineeringSouth China University of Technology2014BSc Biomedical Engineering