Job Description

Job Description

** Note: All candidates must be able to work as a W2 or 1099 employee for any employer in the US.**

** (The role is not eligible for those requiring sponsorships now or potentially in the future.)**

Title: Big Data Design Engineer

Type of role: Contract (12mo+ assignment)

Location: Fully remote, no travel, candidate is expected to work CST hours

Compensation: $65/hr – 85/hr (Based on relevant experience.)

Job Description: The Big Data Design Engineer is responsible for architecture design, implementation of Big Data platform, Extract/Transform/Load (ETL), and analytic applications.

Primary Responsibilities

• Oversees implementation and ongoing administration of Hadoop infrastructure and systems. Manages Big Data components/frameworks such as Hadoop, Spark, Storm, HBase, Hadoop Distributed File System (HDFS), Pig, Hive, Sqoop, Flume, Ozie, Avro, etc.
• Analyzes latest Big Data analytic technologies and innovative applications in both business intelligence analysis and new offerings.
• Aligns with the systems engineering team to propose and deploy new hardware and software environments required for Hadoop and expand existing environments.
• Handles cluster maintenance and creation/removal of nodes using tools like Ganglia, Nagios, Cloudera Manager Enterprise.
• Handles performance tuning of Hadoop clusters and Hadoop MapReduce routines Screens. Hadoop cluster job performances and capacity planning.
• Monitors Hadoop cluster connectivity and security.
• Manages and reviews Hadoop log files.
• Handles HDFS and file system management, maintenance, and monitoring.
• Partners with infrastructure, network, database, application, and business intelligence teams to guarantee high data quality and availability.
• Collaborates with application teams to install operating system and Hadoop updates, patches,

and version upgrades when required Acts of point of contact for vendor escalation.

This position is exempt from timekeeping requirements under the Fair Labor Standards Act and is not

eligible for overtime pay.

Requirements

Bachelor's degree in a related field

Seven (7) years of experience in architecture and implementation of large and highly complex projects

Must have Cloudera Admin skills (ex. understanding error msgs., configuring and set-up of clusters etc)

Must have experience maintaining/navigating Cloudera in Hadoop

Must have good communication skills and experience working collaboratively within a highly technical

environment.

Experience with YARN

Knowledge of HIVE and Impala

Skills and Competencies

Experience with Airflow, Argo, Luigi, or similar orchestration tool Experience with DevOps principals and CI/CD Experience with Docker and Kubernetes Experience with No-SQL databases such as HBase, Cassandra, or MongoDB Experience with streaming technologies such as Kafka, Flink, or Spark Streaming Experience working with Hadoop ecosystem building Data Assets at an enterprise scale Strong communication skills through written and oral presentations