Data Engineer
Google Cloud India Pvt Ltd.
Total years of experience :7 years, 9 Months
Completed Professional Data Engineer certification which helped me to learn in detail about Google
•Cloud technologies like Big Query, Cloud SQL, Data Flow and Cloud Pub/Sub.
•Worked on Flipkart project where I migrated their Hive database to Big Query using Data Proc.
Migration
•Worked on migrating multiple SSIS packages (stored procedures) from SQL Server to Azure Databricks (more than 300 tables with PB of data)
•Created more than 30 pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform, and sync data (multiple sources and destination).
•Wrote notebooks in SQL, Pyspark and Python to perform data cleaning, ingestion, processing and publishing.
•Created a generic test framework for data comparison between multiple systems.
•Implemented partition pruning, query optimization for performance improvement.
•Created logging, alerting and monitoring framework for data pipelines.
•Created Build and Release for deploying pipelines, triggers, linked services and notebooks using Azure DevOps and PowerShell.
Worked as a core developer in writing Web APIs using microservices architecture to provide data to various applications in the bank.
•Created Web API controllers with complex routing and exposed their endpoints for CRUD operations by other applications.
•Developed data access and business layer using Entity Framework (Code-First) and followed best practices like Repository Pattern and Dependency Injection.
•Added Splunk for doing log analysis and generating real-time dashboards depicting logging pattern and API statistics. It helped to track the performance of each Web API.
•Migrated code base and maintained automated CI/CD pipelines using Jenkins, Bitbucket and Ansible.
Worked as a core developer for a metrics project to design Single Page Application displaying various dashboards (charts, grids, pop-ups) to help the business in gaining better insights of available data.
•Implemented filtering, sorting and paging on client-side using Telerik Kendo in AngularJS.
•Created various database objects (Tables, Views, and SPs) and wrote services in C#.
•Designed front-end by using HTML5, CSS3 and AngularJs.
84
URL removed due to policy violation. Please contact support for further information.