Job Description

** Please note:** Although our position is remote / virtual / work-from-home, you MUST reside, and be authorized to work, in the United States without sponsorship. Sorry, only US citizens and US permanent residents - no C2C or H1 transfers.

DATA ENGINEER (DATABRICKS, PYTHON, SPARK)

This is a full-time role in our product organization for a highly experienced Data Engineer in Big Data systems design with skills in data architecture, especially Spark and Delta/Data Lake technology.

Individuals in this role will assist in the design, development, enhancement, and maintenance of complex data pipelines products that manage business critical operations, and large-scale analytics pipelines. Qualified applicants will have a demonstrated capability to learn new concepts quickly, have a data engineering background, and/or have robust software engineering expertise.

Responsibilities

• Scope and execute together with team leadership. Work with the team to understand platform capabilities and how to best improve and expand those capabilities.
• Strong independence and autonomy.
• Design, development, enhancement, and maintenance of complex data pipeline products which manage business-critical operations and large-scale analytics applications.
• Experience leading mid- and senior-level data engineers.
• Support analytics, data science and/or engineering teams and understand their unique needs and challenges.
• Instill excellence into the processes, methodologies, standards, and technology choices embraced by the team.
• Embrace new concepts quickly to keep up with fast-moving data engineering technology.
• Dedicate time to continuous learning to keep the team appraised of the latest developments in the space.
• Commitment to developing technical maturity across the company.

Qualifications

• 5+ years of Data Engineering experience including 2+ years designing and building Databricks data pipelines is REQUIRED ; Azure cloud preferred , will consider AWS, GCP or other cloud platform experience in lieu of
• Experience with conceptual, logical and/or physical database designs is HIGHLY DESIRED
• 2+ years of hands-on Python/Pyspark/SparkSQL experience is REQUIRED; experience with Scala is a plus
• 2+ years of experience with Big Data pipelines or DAG Tools (Airflow, Dbt, Data Factory, or similar) is REQUIRED
• 2+ years of Spark experience (especially Databricks Spark and Delta Lake) is REQUIRED
• 2+ years of hands-on experience implementing Big Data solutions in a cloud ecosystem, including Data/Delta Lakes, is REQUIRED
• 2+ years of experience with source control (git) on the command line is REQUIRED
• 2+ years of SQL experience, specifically to write complex, highly optimized queries across large volumes of data is HIGHLY DESIRED
• Strong data modeling / data profiling capabilities with Kimball/star schema methodology is HIGHLY DESIRED
• Professional experience with Kafka or other live streaming technology is HIGHLY DESIRED
• Professional experience with database deployment pipelines (i.e., dacpac’s or similar technology) is HIGHLY DESIRED
• Professional experience with one or more unit testing or data quality frameworks is HIGHLY DESIRED

\ #LI-CA1

\ #REMOTE

\ #DATABRICKS

\ #SPARK

\ #DATALAKEHOUSE

\ #BIGDATA

Additional Information

** We are an equal opportunity employer. All applicants will be considered for employment without attention to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.**

Additional Information:

All your information will be kept confidential according to EEO guidelines.

Base Salary Pay Range*: USD $97.5K – USD $175K

*The current applicable Base Salary Pay Range for this role is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills relevant to the role, internal equity, alignment with market data, or other law.

Other Compensation / Benefit Overview:

In addition to Base Salary, the successful candidate may be eligible to participate in the following plans / programs, upon satisfying all hiring requirements:

· Bonus Plan

· Medical, Dental and Vision Coverage

· Life Insurance and Disability Programs

· Retirement Savings with Company Match

· Paid Time Off

· Flexible Work Arrangements including Remote Work

Beware of scams

Our recruiting team may communicate with candidates via our @hitachisolutions.com domain email address and/or via our SmartRecruiters (Applicant Tracking System) notification@smartrecruiters.com domain email address regarding your application and interview requests.

All offers will originate from our @hitachisolutions.com domain email address. If you receive an offer or information from someone purporting to be an employee of Hitachi Solutions from any other domain, it may not be legitimate.