- Implemented Auto purge in Data Lakes using Scala, GCS as per new CCPA rules
- Developed the CICD process for the team to ensure the builds and merges are thoroughly monitored and migrated scalable databases capable of ETL processes using SQL and Spark
- Developed automation scripts in Python to scrape certified metric queries, ensuring up to date information
- Migrated 600 Automic jobs to Airflow while ensuring data quality and sanity
- Removed duplicate records from datasets using Snowflake