Job Description
Work Location: Remote
Duration: 4+ Months
W2 Only Position
ETL Developer :
Architecture
GLOBALSCAPE – SFTP TO RETRIEVE FILES
Windows Archive VM server or S3 – Store files
AWS Lambda to provide the following 5 services:
- Delete files on Globalscape
- Error handling
- Email notification
- SNOW ticket logging
- Decrypting files in S3
• Step Functions or Managed Workflow Apache Airflow (MWAA) – Service to manage ETL/ELT workflow orchestration
- Snowflake – Database
- Tableau – Reporting tool
Data Flow:
- Globalscape picks up 5 files and stores them on the VM server or S3
- Files are staged to allow Snowflake to ingest the data
- File contents are copied into the Snowflake database. I’ll assume there is a staging database in Snowflake
- Perform record reconciliation checks with source data file record counts or checksum.
- Perform record reconciliation for Pazien between Snowflake Staging database and Snowflake Trusted Zone (conformed schema) to make sure the record counts match.
- Record verification is perform. Determine if the record an insert, update, or error.
- Transformations are performed on the data. Right now we only have one requirement to convert to USD amounts
- Insert or update the record in the main database in Snowflake
- Create semantic views for Tableau access to the Snowflake data
Basic Qualifications:
- ETL Developer tasks – USING AWS LAMBDA / SQL
• Stage data, copy data from files, load data and transform data
• Log data steps
• Error checking and error handling the ability to create Tableau reports
- SQL - WILL BE WRITING QUERIES FOR SNOWFLAKE DB
- Database skills - experience with data warehousing
- Familiarity with star schemas
- Understanding of data modeling experience to give recommendations
- TABLEAU SKILLS - HOW TO BUILD REPORTS- USING SUPPORTING STRUCTURES AND WRITING VIEWS AGAINST DATABASE
Required Education:
· BS or Equivalent
Company Description
TenTek has been in business since 1989, and is recognized as a leading staffing provider of tech professionals to a growing client base.