The Data Engineer will manage and manipulate data and data flows for both existing and new systems. Additionally, they will provide support in the areas of data extraction, transformation, and load (ETL), data mapping, analytics, operations, databases, and maintenance of data and associated systems. As a member of the team, candidate will work in a multi-tasking, quick-paced, dynamic, process-improvement environment that requires experience with the principles of large-scale (terabytes) database environments, large-scale file manipulation, data modeling, data mapping, data testing, data quality, and documentation preparation.

RESPONSIBILITIES

• Provide support in the areas of data extraction, transformation, and load (ETL), data mapping, analytics, operations, database administration, and maintenance of data and associated systems
• Develop and manage complex data flows or makes significant enhancements to existing pipelines
• Troubleshoot complex problems and provide customer support for the ETL process
• Advise hardware engineers on machine characteristics that affect software systems, such as storage capacity, processing speed, and input/output requirements
• Conduct investigations and tests of considerable complexity
• Provide ongoing maintenance, support, and enhancements in existing systems and platforms
• Collaborate cross-functionally with software engineers, data scientists, analysts, project managers, and other engineering groups
• Research emerging cloud-native technologies to determine impact on application execution
• Communicate clearly and effectively with teammates, customers, and external partners
• Prepare written and verbal communications on analyses, findings, and project progress
• Write and update technical documentation such as user manuals, system documentation, training materials, processes, and procedures
• Provide recommendations for continuous improvement

REQUIREMENTS

• Five (5) + years of experience in working with Big Data technology and using methods to ingest, process, clean, and analyze big data
• Bachelor's degree in Computer Science, Information Technology, or other related discipline, or equivalent combination of education, technical certifications, training, and work/military experience
• Experience working with large and complex data sets as well as experience analyzing large volumes of data
• Experience working with relational databases such as Postgres, MySQL, and Microsoft SQL Server
• Experience with data ingest tools (e.g., Sqoop, Kafka, and Spark Streaming)
• Experience in system management expertise with monitoring, disaster recovery, backup, automated testing, automated schema migration, and continuous deployment
• Experience in Agile software methodologies
• Security+ Required
• Full Scope Poly Required

PREFERRED

• Five (5) + years of implementation experience in Hadoop technologies and have worked in multiple Hadoop distributions like AWS and Cloudera
• Demonstrated experience with large data stores such as data lakes, data warehouses, and VLDB sharded RDBMS databases
• Intensive skill in scripting using Bash, Python, and Shell
• Demonstrated experience in creating indexes and working with indexes
• Demonstrated experience in Kibana and Elasticsearch
• Demonstrated experience in ETL, data integration, and migration
• Experience with different file formats like ORC, Parquet, AVRO, JSON
• Experience in data cleansing scripts like Spark and MapReduce
• Orchestrated multiple Hadoop application jobs using Oozie or Airflow
• Experience in working with various IDE's such as Eclipse, VS Code, and PyCharm
• Exposure in infrastructure as Code languages like Chef, Puppet, Ansible, Cloudformation

All qualified applicants will receive consideration for employment without regard to race, color, sex, sexual orientation, gender identity, religion, national origin, disability, veteran status, age, marital status, pregnancy, genetic information, or other legally protected status

Required Skills Required Experience

• High School diploma/GED preferred required. • Valid Drivers License preferred. • Warehousing and material handling experience preferred. • Must be able to obtain a certification to operate all required motorized material handling and rigging equipment. • Clear oral and written communication skills, particularly legible handwriting required. • Ability to show adequate manual dexterity, finger dexterity, arm-hand steadiness, and multi-limb coordination required. • Ability to read and comprehend written instructions as applied to work situations required. • Basic measurement skills, including taking, converting, and/or using measurements of length, width, height, and weight to perform basic math computations correctly required. • Basic arithmetic skills to calculate and convert decimals and fractions required. • Ability to multi-task, prioritize, and manage time effectively required. • Ability to make sound decisions based on a combination of factors including knowledge, experience, and judgment required. • Ability to develop and maintain good working relationships with supervisors and co-workers required.