Summary
Backend Engineer, Data Engineer, and Machine Learning Engineer background. Contributor to open source projects like Hudi, Airflow, and Kyuubi.
- Big Data Stack: Spark, Flink, Kafka, Airflow, Presto, Apache Hudi, Apache Kyuubi, DBT/SQLMesh, etc.
- Cloud Platforms: AWS, GCP, and Alicloud.
- DevOps: GitLab CI/CD, GitHub Actions, Docker, Kubernetes, Terraform/Terragrunt, etc.
- MLOps: Ray, MLflow, JupyterHub, etc.
- Programming Languages: Python, Java, Bash shell, Scala, Go, etc.
Work Experience
GoTo Financial, LENDING DATA TEAM
Data Engineer Manager Jan. 2023 - Present
- Manage a streamlined, high-performing team of 10+ Data Engineers working across 5 main divisions: Data Ingestion, Stream Processing, Data Warehousing, Business Intelligence, and Machine Learning Engineering; built an open and agile team culture while promoting team members' growth.
- Lead cross-team collaboration, supporting multiple product lines and data requirements (synchronization, analytics, reporting) for over 10 business teams.
- Plan and execute data platform migration from AWS to GCP and from GCP to Alicloud; design equivalent data architecture on new cloud platforms while ensuring timely and high-quality migration completion.
- Identify potential system bottlenecks and implement optimization strategies to improve performance, reduce costs, and enhance efficiency; built internal data portal and self-service data capabilities to improve team productivity.
BYBIT SINGAPORE, DATA TEAM
Principal Data Engineer Sep. 2020 - Dec. 2022 (2 years and 4 months)
As a founding member of the data team, helped design and build the company's data platform and machine learning platform from scratch, continuously evolving the tech stack to improve performance and stability.