Senior Software engineer, Data San Francisco, CA
About:
Our Data team plays a critical role in this mission by writing code that processes customer-provided data to power both the ordering application and the recommendations within, allowing us to optimize our partner's ordering and business processes.
What you will do
- Implement fast, reliable, scalable data pipelines in Python (and Pandas and Spark) and SQL to process billions of historical data points collected from tens of thousands of retail stores across the US
- Build out, scale up, and deploy our compute platform (running on Azure cloud infrastructure using Kubernetes) and data platform (powered by a combination of traditional databases, data warehouses, and data lakes) that power our recommendation engine and ordering system
- Work with our customers to integrate, triage issues, and find novel workarounds to extract the signal that we need from the data provided by our customers
- Monitor, analyze, and understand the data flowing through our system by adding the necessary visualizations, dashboards, and health checks
- Collaborate with an interdisciplinary team of experts in machine learning, data scientists, design, software engineering, and business operations to ensure that the resulting data is as expected
What skills and experience do you need?
- 4 years of professional experience building, deploying and maintaining production software and systems at scale using modern development practices and programming languages (preferably Python)
- 2 years of professional experience writing data pipelines that clean up, process, and provide access to data using high-level programming languages, such as Python or Scala, and SQL
- Strong understanding and experience with various data stores (databases, data warehouses, key/value stores, etc.)Experience with Apache Spark or other Big Data frameworks; working with cloud infrastructure; data visualization tools; and applied statistics preferred
- Dedication to code quality, testing, design processes, automation, and operational excellence. Excellent written and verbal communication, collaboration, and social skills