Data Platform technical lead

United Global Technologies

Data Platform technical lead

Houston, TX
Full Time
Paid
  • Responsibilities

    As a Data Tech Lead, you will provide end-to-end leadership across architecture, engineering, governance, and capability development.

    You will:

    • Deliver sustainable and innovative solutions to unlock value from data.
    • Lead the architecture, Design and implementation of on-premises/cloud-based application development and Data solutions.
    • Engage cross-functional teams to support project stakeholders in delivering data scope and operational excellence.
    • Drive governance, metadata management, Data Modelling, cost optimization, high-volume performance tuning and agility in delivery.
    • Engage effectively with senior leadership, resolve conflicts proactively, and represent the data organization with confidence and clarity.

    Key Responsibilities

    Leadership & Strategy

    • Drive the architecture, design, and delivery of Data ecosystems for futuristic data platform.
    • Define data product roadmaps and drive development of data solution using approved tech stack and related technologies.
    • Align data engineering initiatives with long-term business and technology strategies.
    • Provide technical leadership to ensure adherence to best practices, standards, and quality expectations.

    Technical Oversight

    • Lead data modelling and architecture using tools such as SAP PowerDesigner and WizDesign.
    • Develop and optimize intelligent, AI-augmented data pipelines across ADF, ADLS, Databricks, DLT, and Structured Streaming to enable high-performance data delivery.
    • OLTP schema design, normalization, indexing strategy, Performance tuning for transactional systems
    • Transaction management, concurrency, deadlocks
    • PostgreSQL, MySQL, SQL Server expertise (depending on Shell standards)
    • Optimize Databricks workloads with a strong focus on performance and cost efficiency.
    • Implement robust security, governance, lineage, and metadata controls using Unity Catalog.
    • Troubleshoot pipeline issues, ensure high data quality, and enforce SLA compliance.

    Governance & Quality Assurance

    • Establish data governance frameworks covering security, privacy, quality, and lifecycle management.
    • Monitor pipeline performance and lead root cause analysis for data and process issues.
    • Drive CI/CD and touchless change management frameworks to enable seamless deployments.

    4\. Stakeholder Management

    • Collaborate closely with PMs, BAs, Data Scientists, Architects, and business stakeholders.
    • Translate complex technical concepts into business terms for executive audiences.
    • Resolve conflicts and ensure alignment across strategic priorities.
    • Stakeholder management & executive communication

    Required Skills & Experience

    Technical Expertise

    • 12–17 years of experience in data engineering, data architecture, or analytics platforms.
    • Deep expertise in:
      • Azure Data Engineering (ADF, ADLS, Databricks, , DLT), AZURE Function, SQL, REST/FAST API, LogicApps
      • Databricks Delta Live Tables & Structured Streaming
      • Unity Catalog governance and lineage
    • Strong hands-on skills in SQL, Python, Spark/Scala.
    • Strong experience in data modeling, data architecture, and metadata management.
    • CI/CD for application delivery (Azure DevOps/GitHub Actions)
    • Containerization (Docker), Kubernetes basics
    • API gateway, reverse proxies, load balancing
    • Solution Data Architecture, Data Modeling tools like SAP PowerDesigner, BiZ Design

    AI & Future-Ready Skills

    • Working knowledge of AI/ML, GenAI, Agentic AI, and applying AI for data engineering automation.
    • Ability to envision and build future-ready, scalable data ecosystems.

    Soft Skills / Leadership Competencies

    • Proven ability to lead mid-sized to large technical teams.
    • Strong decision-making capabilities backed by data.
    • Exceptional communication and executive presentation skills.
    • Conflict management and stakeholder alignment experience.
    • High accountability, ownership, and ability to drive independent outcomes.
    • Adaptability and continuous learning mindset.

    Nice to Have

    • Experience with Kafka, Hadoop, Spark Streaming.
    • Experience with workflow tools like Airflow, Azkaban, Luigi.
    • Knowledge of PowerBI/QlikSense visualizations.
    • Certifications in Azure Data Engineer, Databricks, AI/ML, or Data Science.
    • Experience with GitHub/Azure DevOps, CI/CD, Agile/Kanban methodologies.
    • Understanding of Freight Domain like Voyage Management, Chartering, Freight Pricing, Demurrage, Laytime, Port calls, Vessel scheduling
    • Integration with RightAngle, Endur, Veson IMOS or internal Shell Freight systems
    • Knowledge of T&S business processes