Senior data engineer
Accenture - India
Total years of experience :7 years, 11 Months
Strong experience in Data analysis with experience in handling big data related components.
A Microsoft Certified Data Engineer Associate with fair understanding of Azure cloud environment.
Developed data ingestion frameworks using Scala and Spark.
Experience in building data factory pipelines from Azure Data Factory.
Implemented Notebooks in both Scala and Python using Azure Databricks.
Created and used storage accounts in Azure of type blob and Data Lake storage.
Performed data manipulations using SparkSQL and Data frames and applied analytical functions.
Implemented D-streams with RDD?s and performed streaming data based on time intervals.
Good understanding of Azure HDInsight.
Implemented real-time big data projects using Cloudera CDH 2.6
Developed streaming applications using Spark and Kafka with real time data.
Strong experience with Hive and handled huge datasets.
Developed Workflows using Apache Oozie and scheduled jobs that run on the set frequency.
Lead Data Engineer