Sorry, this listing is no longer accepting applications. Don’t worry, we have more awesome opportunities and internships for you.

Big Data Engineer

MitSix

Big Data Engineer

Sacramento, CA
Full Time
Paid
  • Responsibilities

    Job Description

     

     

    JOB DESCRIPTION SUMMARY

    OVERVIEW

    MASTERCARD IS THE GLOBAL TECHNOLOGY COMPANY BEHIND THE WORLD'S FASTEST PAYMENTS PROCESSING NETWORK. WE ARE A VEHICLE FOR COMMERCE, A CONNECTION TO FINANCIAL SYSTEMS FOR THE PREVIOUSLY EXCLUDED, A TECHNOLOGY INNOVATION LAB, AND THE HOME OF PRICELESS®. WE ENSURE EVERY EMPLOYEE HAS THE OPPORTUNITY TO BE A PART OF SOMETHING BIGGER AND TO CHANGE LIVES. WE BELIEVE AS OUR COMPANY GROWS, SO SHOULD YOU. WE BELIEVE IN CONNECTING EVERYONE TO ENDLESS, PRICELESS POSSIBILITIES.

     

    DATA ENGINEERING AND PLATFORM IS FOCUSED ON ENABLING INSIGHTS INTO MASTERCARD NETWORK AND HELP BUILD DATA-DRIVEN PRODUCTS BY CURATING AND PREPARING DATA IN A SECURE AND RELIABLE MANNER. MOVING TO A “UNIFIED AND FAULT-TOLERANT ARCHITECTURE FOR DATA INGESTION AND PROCESSING” IS CRITICAL TO ACHIEVING THIS MISSION.

     

    AS A SENIOR SITE RELIABILITY / DEVOPS ENGINEER IN DATA ENGINEERING AND PLATFORM, WITH YOUR STRONG TECHNICAL CHOPS AND TO HELP OUR PLATFORM AND SERVICES ESTABLISH A TRUE SRE CAPABILITY. YOU WILL BE PARTNERING WITH THE DATA ENGINEERING TEAM, SO THE ABILITY TO INFLUENCE AND PROVIDE OPERATIONAL GUIDANCE IS KEY. INITIALLY, THE SRES FOCUS WILL BE CONTRIBUTING TO THE DEVELOPMENT OF OPERATIONAL TOOLS AND PRACTICES THAT HELP MAINTAIN SERVICE AVAILABILITY ACROSS HOSTED AND CLOUD-BASED INFRASTRUCTURE. YOU MUST HAVE AN UNDERSTANDING OF THE FULL STACK AND HOW SYSTEMS ARE BUILT AS WELL AS A GRASP OF OPERATIONAL BEST PRACTICES.

     

    ROLE

    • PARTNER ON THE DESIGN THE NEXT IMPLEMENTATION OF MASTERCARD SECURE, GLOBAL DATA AND INSIGHT ARCHITECTURE, BUILDING NEW STREAM PROCESSING CAPABILITIES AND OPERATIONALIZING “UNIFIED DATA ACQUISITION AND PROCESSING (UDAP) PLATFORM”

    • IDENTIFY AND RESOLVE PERFORMANCE BOTTLENECKS EITHER PROACTIVELY

    • WORK WITH THE CUSTOMER SUPPORT GROUP AS NEEDED TO RESOLVE PERFORMANCE ISSUES IN THE FIELD

    • EXPLORE AUTOMATION OPPORTUNITY AND DEVELOP TOOLS TO AUTOMATE SOME OF THE DAY TO DAY OPERATIONS TASKS

    • PROVIDE PERFORMANCE METRICS AND MAINTAIN DASHBOARDS TO REFLECT PRODUCTION SYSTEMS HEALTH

    • CONCEPTUALIZE AND IMPLEMENT PROACTIVE MONITORING WHERE POSSIBLE TO CATCH ISSUES EARLY

    • EXPERIMENT WITH NEW TOOLS TO STREAMLINE THE DEVELOPMENT, TESTING, DEPLOYMENT, AND RUNNING OF OUR DATA PIPELINES.

    • WORK WITH CROSS FUNCTIONAL AGILE TEAMS TO DRIVE PROJECTS THROUGH FULL DEVELOPMENT CYCLE.

    • HELP THE TEAM IMPROVE WITH THE USAGE OF DATA ENGINEERING BEST PRACTICES.

    • COLLABORATE WITH OTHER DATA ENGINEERING TEAMS TO IMPROVE THE DATA ENGINEERING ECOSYSTEM AND TALENT WITHIN MASTERCARD.

    • CREATIVELY SOLVE PROBLEMS WHEN FACING CONSTRAINTS, WHETHER IT IS THE NUMBER OF DEVELOPERS, QUALITY OR QUANTITY OF DATA, COMPUTE POWER, STORAGE CAPACITY OR JUST TIME.

    • MAINTAIN AWARENESS OF RELEVANT TECHNICAL AND PRODUCT TRENDS THROUGH SELF-LEARNING/STUDY, TRAINING CLASSES AND JOB SHADOWING.

     ALL ABOUT YOU

    • AT LEAST BACHELOR'S DEGREE IN COMPUTER SCIENCE, COMPUTER ENGINEERING OR TECHNOLOGY RELATED FIELD OR EQUIVALENT WORK EXPERIENCE

    • INTERMEDIATE EXPERIENCE IN DATA WAREHOUSE RELATED PROJECTS IN PRODUCT OR SERVICE BASED ORGANIZATION

    • FOUNDATIONAL EXPERIENCE AS A SITE RELIABILITY ENGINEERING OR DEVOPS ENGINEER

    • FOUNDATIONAL EXPERIENCE OVERALL WITH EXPERIENCE AS A SOFTWARE ENGINEER OR SOFTWARE ARCHITECT

    • EXPERIENCE SOLVING FOR SCALABILITY, PERFORMANCE AND STABILITY

    • EXPERT KNOWLEDGE OF LINUX OPERATING SYSTEMS AND ENVIRONMENT AND SCRIPTING (SHELL AND PYTHON PREFERRED)

    • A DEEP EXPERTISE IN YOUR FIELD OF SOFTWARE ENGINEERING

    • EXPERT AT TROUBLESHOOTING COMPLEX SYSTEM AND APPLICATION STACKS

    • OPERATIONAL EXPERIENCE IN BIG DATA STACKS ( HADOOP ECOSYSTEM, SPARK IS A PLUS)

    • OPERATIONAL EXPERIENCE IN REAL-TIME ,STREAMING AND DATA PIPELINES RELEVANT FRAMEWORKS ( KAFKA AND NIFI IS A PLUS)

    • OPERATIONAL EXPERIENCE TROUBLESHOOTING NETWORK/SERVER COMMUNICATION

    • EXPERIENCE WITH PERFORMANCE TUNING OF DATABASE SCHEMAS, DATABASES, SQL, ETL JOBS, AND RELATED SCRIPTS

    • EXPERTISE IN ENTERPRISE METRICS/MONITORING WITH FRAMEWORKS SUCH AS SPLUNK, DRUID, GRAFANA

    • EXPERIENCE WITH CLOUD COMPUTING SERVICES, PARTICULARLY DEPLOYING AND RUNNING SERVICES IN AZURE OR GCP

    • A BELIEF IN DATA DRIVEN ANALYSIS AND PROBLEM SOLVING AND A PROVEN TRACK RECORD IN APPLYING THESE PRINCIPLES

    • AN ORGANIZED APPROACH THE PLANNING AND EXECUTION OF MAJOR PROJECTS

    ADDITIONAL NOTES TO VENDOR

    CALL NOTES 15460 SENIOR SOFTWARE ENGINEER • NON SPONSORED CANDIDATES ONLY USC OR GC HOLDERS • 2 ROLES OPEN • SAN FRANCISCO, CA • 12 MONTH CONTRACT WHAT ARE YOUR TOP 3 REQUIRED TECHNICAL SKILLS? • AUTOMATION DEV OPS ENGINEERING [HADOOP, SPARK] • OPERATIONAL EXPERIENCE IN REAL-TIME, STREAMING AND DATA PIPELINES RELEVANT FRAMEWORKS [NIFI OR AIRFLOW] • JAVA/ SCALA AND PYTHON PROGRAMMING • PCF OR CLOUD EXPERIENCE IN GENERAL. • DATA WAREHOUSE, DATA PLATFORM TEAM, DATA ENABLEMENT PROGRAM, O & T • CLOUD DATE INTEGRATION • FOCUS IS MORE ON AZURE • HANDS ON CLOUD ENGINEERING • NEED NIFI OR AIRFLOW EXPERIENCE • NOT JUST A DEVELOPER ROLE, LOOKING FOR BIG DATA DEVELOPMENT • HADOOP ADMINISTRATION • AGILE • OPERATION SUPPORT • SPARK, IMPALA • DATA WAREHOUSE RELATED PROJECTS IN PRODUCT OR SERVICE BASED ORGANIZATION • OPERATIONAL EXPERIENCE IN BIG DATA STACKS ( HADOOP ECOSYSTEM, SPARK IS A PLUS) • OPERATIONAL EXPERIENCE TROUBLESHOOTING NETWORK/SERVER COMMUNICATION • EXPERIENCE WITH PERFORMANCE TUNING OF DATABASE SCHEMAS, DATABASES, SQL, ETL JOBS, AND RELATED SCRIPTS

     

     

    Company Description

    About[[company]]. founded in 2015 with a single mission: to assist our clients with technology decisions. Maximizing the value of existing IT investment, de-risking the adoption of new technology, and reducing the interval to achieve positive results. We compare technology to an untapped vein of gold. It does nothing for you until you mine it.