overview
- Technologies: Abinitio, Python, SQL, Linux, Bash Scripting
- Responsibilities
- Involved in Analysis, Design, Process interfaces design and documentation
- Developed various data loading strategies and performed various transformations for analysing datasets
- Developed and maintained complex Ab Initio graphs for extracting, transforming, and loading data from various source systems into data warehouses
- Designed and implemented data integration solutions to support business intelligence and reporting requirements, ensuring data quality, accuracy, and consistency
- Conducted data analysis and profiling to identify patterns, trends, and anomalies in large datasets, providing actionable insights to stakeholders
- Collaborated with business analysts, data architects, and stakeholders to gather requirements, define data integration strategies, and deliver solutions that meet business objectives
- Worked on performance tuning and optimization of Ab Initio graphs to improve processing speed, resource utilization, and scalability
- PROJECT - 2
- December 2022
- Project Name: Secure Cloud Data Storage Infrastructure
- Technologies: AWS S3, VPC, AWS IAM
- Designed and implemented a secure data lake using S3 buckets within a private VPC environment
- Configured VPC subnets and security groups to restrict access to data stored in S3
- Created and assigned IAM roles and policies to control user and application access to specific data resources within the S3 buckets
- PROJECT - 3
- June 2023
- Project Name: Big Data ETL Pipeline with Glue and S3
- Technologies: AWS Glue, S3, EC2, Apache Spark
- Selected a public movie review dataset stored in S3
- Launched and configured an EC2 instance with Apache Spark for large-scale data analysis
- Utilized AWS Glue to extract and potentially pre-process the movie review data from S3 for efficient analysis on the EC2 instance
- Performed data exploration, sentiment analysis, or built a basic machine learning model to analyze the reviews using Spark or Python libraries on the EC2 instance
- PROJECT - 4
- Project Name: Interactive Stock Market Trends
- Technologies: AWS E2, S3, Flask, PowerBi
- Developed a web application using Flask (or Django) to retrieve stock market data from S3
- Configured an EC2 instance to host the web application
- Implemented functionalities to access and retrieve stock data from S3 buckets using the AWS SDK or other tools
- Integrated data visualization tool PowerBi to display the retrieved stock data in an interactive format within the web application