Data Engineer (Azure Databricks)- Greenfield Project

duvari group

Data Engineer (Azure Databricks)- Greenfield Project

Maryland Heights, MO
Paid
  • Responsibilities

    Senior Data Engineer

    We are seeking a hands-on Senior Data Engineer to build a greenfield, AI-ready enterprise data platform on Azure. Working alongside Data Architects, you will spend 90%+ of your time developing scalable pipelines in Azure Databricks and Microsoft Fabric to power enterprise reporting, analytics, and GenAI/RAG initiatives.

    What You'll Do

    • Build and optimize batch and real-time pipelines in Azure Databricks (90%+ focus) using SQL, Python, PySpark, and Delta Lake, utilizing Microsoft Fabric for ingestion.

    • Implement Lakehouse and medallion architectures, migrating legacy Data Warehouse workloads and integrating ERP/CRM/HR sources (Dynamics 365 F&O/CE, SAP, legacy ERPs, Workday, Entra).

    • Develop dimensional and Lakehouse data models with built-in data quality, lineage, and observability.

    • Support data governance and MDM via Databricks Unity Catalog, Microsoft Purview, and enterprise catalog/MDM platforms.

    • Lead engineering standards, code reviews, CI/CD practices, and mentor onsite/offshore engineers (individual contributor role).

    What We're Looking For

    • Experience: Minimum 5 years in cloud data engineering (7+ years optimal) and a Bachelor's degree or equivalent experience.

    • Core Stack: Deep expertise in Azure Databricks (Lakeflow, Spark, Delta) plus hands-on Microsoft Fabric, ADF, Azure Synapse, ADLS, Azure SQL, and Azure DevOps.

    • Code & Modeling: Advanced SQL, expert Python, Spark/Pandas, ETL/ELT, and dimensional/Lakehouse modeling.

    • AI & Modern Data: Experience building AI/ML, GenAI, and RAG-ready datasets, including familiarity with MCP servers, data guidance, and AI-assisted data engineering.

    • Delivery: Strong debugging skills and ability to drive priorities in Agile/DevOps teams.

    Nice to Have

    • Manufacturing industry experience in complex data environments.

    • Direct integration experience with Dynamics 365 (F&O/CE) or similar ERP/CRM platforms.

    • Familiarity with enterprise data catalogs and MDM platforms.

    Additional Details

    • Location: St. Louis, MO preferred (hybrid, 3 days/week onsite); remote/flexible hybrid considered.

    • Employment Type: Contract-to-Hire (strong intent to convert; direct hire considered for exceptional candidates).

    • Team & Process: 4-engineer greenfield team (alongside catalog/MDM engineers). Interview process: Lead Solutions Architect screen followed by hiring manager interview.

     

  • Compensation
    $175,000 per year