Shri Dharmasthala Manjunatheshwara College of Engineering and Technology
Work Experience
B
Brillio
Senior Data Engineer
November 2021 - June 2023
company
Brillio
title
Senior Data Engineer
overview
- Automated Data Cleansing, Transformation, and Reporting (ETL) for University Fundraising and Enrolment
- Automated a manual process involving file uploads from servers and issue resolution, reducing time spent from 7 hours
- Developed a dynamic rule engine in PySpark Databricks to accommodate machine learning models with diverse rules
- Led an end-to-end automation process for campaign data, including cleansing, issue analysis, and resolution using Azure
- Data Factory
- Implemented dynamic parameterization of pipelines, datasets, and linked services, optimizing workflows and generating
- Migrated 98% of legacy systems to Azure, with some restricted scripts managed through Windows Schedulers on
- Achieved a 95% reduction in manual effort for data analysts through automation, significantly increasing efficiency
- Trained and supervised four junior employees, guiding them from knowledge transfer to independently handling project
- Organized engaging virtual team-building activities during COVID-19 to boost morale and maintain team connection in a remote work environment
M
Mindtree
Senior Engineer
June 2017 - October 2021
Skills
Agile MethodologyApache HadoopApache SparkAutomationAzure Active DirectoryAzure Data FactoryAzure Machine LearningBrand ManagementCloud ComputingContinuous IntegrationCost OptimisationCOVID-19 TestingDashboardsDatabasesDatabricksData CleansingData IntegrationData ScienceDecision Making SkillsDecision TreesDevOpsDockerExtract Transform Load (ETL)FundraisingGitHard Work and DedicationInformation EngineeringKnowledge of CampaignsKnowledge of EngineeringKnowledge of FinanceKnowledge of StatisticsLegacy SystemsLogistic RegressionMachine LearningMatplotlibMicrosoft AzureMicrosoft Certified ProfessionalMicrosoft ExcelMicrosoft WindowsMoney InvestmentsMySQLNetwork ServerNLTK (NLP Analysis)NumPyOrdinary Least Squares (Regression Analysis)PandasParametrizationPower BIPresentationsProblem SolvingPysparkPython (Programming Language)Random ForestRecommender SystemsRecruitmentResearch and DevelopmentRule EngineScikit LearnSQL AzureSQL DatabasesStreamlineSupport Vector MachineTeam BuildingTeam WorkingVirtual MachinesVirtual TeamsWorkflows