Minimum Requirements

1. Elasticsearch
2. MySQL
3. Java software development skills

Position Description:

Shine Management's client is seeking a Software Engineer with Big Data skills who will create a data processing pipeline that will consume data from a number of sources that is expected to grow over time, and clean, transform, and enrich and deposit it into Elasticsearch and MySQL repositories. As a Software Engineer, you must be able to also analyze the performance of these repositories and implement optimizations to improve data insertion and retrieval performance. Additionally, you will need to extract metadata from and optimize performance for federated searches, all while staying in conformance with the data model designed by the project’s architect.

Your responsibilities will include the following:

is seeking a Software Engineer with Big Data skills who will create a data processing pipeline that will consume data from a number of sources that is expected to grow over time, and clean, transform, and enrich and deposit it into Elasticsearch and MySQL repositories. As a Software Engineer, you must be able to also analyze the performance of these repositories and implement optimizations to improve data insertion and retrieval performance. Additionally, you will need to extract metadata from and optimize performance for federated searches, all while staying in conformance with the data model designed by the project’s architect.

Your responsibilities will include the following:

• Collecting massive sets of structured and unstructured data from contrasting and disparate data sources
• Understanding the content and format of the various data sources
• Verifying relevant data sources for the customer’s business needs
• Interpreting data to discover solutions
• Validating data to ensure completeness, uniformity, and accuracy

Coordination with other data engineers on the team to ensure data is being ingested properly

• Collecting massive sets of structured and unstructured data from contrasting and disparate data sources
• Understanding the content and format of the various data sources
• Verifying relevant data sources for the customer’s business needs
• Interpreting data to discover solutions
• Validating data to ensure completeness, uniformity, and accuracy
• Coordination with other data engineers on the team to ensure data is being ingested properly

Required Skills:

• Bachelor’s degree in computer science, engineering, or related field
• Experience with Elasticsearch
• Experience with MySQL or equivalent RDBMS
• Ability to monitor pipeline performance and apply optimizations, where and when necessary
• Experience with the following programming languages:
  • Structured Query Language (SQL)
  • Java

• Ability to multi-task and prioritize
• Great time management skills
• Excellent verbal and written communication skills
• Ability to work well in a team environment
• Understanding of the Agile framework and working within Scrum

Education

Candidate must have one of the following:

• 12 Years of job related experience and High School/GED diploma
• 10 Years of job related experience and Associates degree
• 8 Years of job related experience and Bachelor’s degree
• 6 Years of job related experience and Master’s degree
• 4 Years of job related experience and Doctorate

Desired Skills

Familiarity with financial terms and business processes