Sorry, this listing is no longer accepting applications. Don’t worry, we have more awesome opportunities and internships for you.

Data Engineer - Data Warehouse Team

Shipt

Data Engineer - Data Warehouse Team

San Francisco, CA
Full Time
Paid
  • Responsibilities

    JOB DESCRIPTION

    Shipt is a membership-based marketplace that helps people get the things they need. Our friendly shoppers handpick fresh groceries and household essentials and deliver them to members in as soon as one hour.

    We are currently looking for a Data Analyst to join the Data Warehouse team. The Data Warehouse team at Shipt is core to the organizational goal of moving to multiple, independent micro-services and increasing our feature deployment velocity.

    The Data Warehouse team is responsible for building a managed data lake and from it an enterprise data warehouse.  The data lake will serve as a raw, unprocessed store of business events and entity CRUD activity. The intent behind the enterprise data warehouse is to create a store of cleansed, pre-related data from which a business user or analyst should rapidly be able to create actionable information.  On the other hand, the data lake exists for two purposes:

    • a raw data store for utilization by data science team members and other data experts

    • a source for processing and augmenting the enterprise data warehouse, over time.

    As a member of the Data Warehouse team, you will be developing, maintaining and supporting:

    • data pipelines to move data from the enterprise service bus messaging to our data lake and ultimately data warehouse analytical stores
    • test the data pipeline code to ensure quality builds
    • collect and monitor the metrics necessary to quantify system performance and forecast future capacity needs.

    WHAT YOU'LL GAIN

    You'll join a team of talented individuals who will provide you with hands-on mentorship on topics ranging from design to operational monitoring.  Furthermore, you will have the freedom to solve interesting, web-scale problems with the appropriate technology.

    YOUR RESPONSIBILITIES

    • Develop Data Pipeline - working within the Data Warehouse team and with other members of the Engineering organizations to build services that subscribe and collect messages from our next generation services for entity CRUD and business activity. describe, document intended use and finally surface data as actionable information.
    • Ideate and Collaborate on Solutions- be a thought leader within the Tech organization to build new and improved data tools and services that can scale with the company
    • Invest in the Process - execute and continuously improve our development process

    REQUIREMENTS

    • 4+ years in Data Engineering and/or Engineering
    • Experience working in a web-scale data environment (for example, millions to billions of messages per day)  
    • Experience working in an environment with a bias towards action
    • Strong development skills especially with tools such as Tableau, Looker, ChartIO and their relations.
    • An understanding of technologies and design patterns in fields such as: micro services, streaming / queuing systems, SQL and key-value stores, and high-performance solutions (vectorization, task and data parallelism)
    • Expertise with Presto, Snowflake and Redshift (or their relations) is a major plus
    • Expertise with AWS tech and deployments is a major plus (we currently use Jenkins, Docker, AWS Lambda, and AWS Batch)
    • A Bachelor's Degree in CS, Information Systems, a related field or equivalent work experience

    We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

    _Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records. _