Big Data Engineer

S

Select Minds LLC

Big Data Engineer

Lansing, MI
Full Time
Paid
  • Responsibilities

    Benefits:

    ONSITE

    Competitive salary

    Opportunity for advancement

    Big Data Engineer Location: Chicago, IL (Day 1 Onsite) Job Type: Contract / Full Time Experience: 8+ yrs Duration: Long Term

    We are seeking an experienced Big Data Engineer to design, develop, and optimize enterprise-scale big data solutions. The ideal candidate will have strong hands-on expertise in the Hadoop ecosystem, Spark, Scala, Hive, Apache NiFi, and Kafka, along with experience building scalable data pipelines and supporting modern CI/CD practices. Candidates should have worked directly with clients, participated in requirement gathering sessions, and delivered end-to-end Big Data implementations.

    Key Responsibilities

    • Design, develop, and optimize large-scale data processing pipelines using Spark, Scala, and Hive.
    • Build and maintain data ingestion workflows using Apache NiFi.
    • Develop and support real-time data streaming solutions using Apache Kafka.
    • Monitor application logs and system health using Kibana, Elasticsearch, and Logstash.
    • Implement and support CI/CD pipelines using Git, Jenkins, and Nexus.
    • Ensure data quality, integrity, and governance across Big Data platforms.
    • Participate in architecture discussions and recommend scalable design solutions.
    • Troubleshoot and resolve complex production issues across the Big Data stack.
    • Collaborate with cross-functional teams in an Agile environment.
    • Mentor junior engineers and contribute to technical best practices.

    Required Qualifications

    • 8+ years of experience in Big Data technologies.
    • Strong hands-on experience with Spark, Scala, Hive (HQL), and the Hadoop ecosystem.
    • Experience implementing Apache NiFi and Apache Kafka.
    • Proficiency in Shell Scripting or Python.
    • Experience with PostgreSQL.
    • Familiarity with scheduling tools such as Oozie and Control-M.
    • Working knowledge of Git, Jenkins, Nexus, Maven, Ansible, and SonarQube.
    • Strong understanding of CI/CD practices.
    • Experience in client-facing roles, including requirement gathering workshops.
    • Minimum of two end-to-end Big Data implementation projects.
    • Ability to work independently and contribute effectively within Agile teams.