Dinesh Nalla

As an Azure Data Engineer, I design, develop, and maintain scalable data pipelines using Azure Data Factory, Azure Databricks, PySpark, and Azure Data Lake Storage Gen2. I build end-to-end ETL/ELT solutions to ingest data from multiple sources, including SQL databases, APIs, FTP/SFTP, and cloud storage, ensuring reliable and efficient data movement.

I implement Delta Lake and Medallion Architecture (Bronze, Silver, and Gold layers) to create high-quality, analytics-ready datasets. I optimize PySpark jobs by applying partitioning, caching, broadcast joins, and performance tuning techniques to process large volumes of data efficiently.

I develop metadata-driven frameworks to automate data ingestion and transformation processes, implement data quality validation and reconciliation checks, and monitor production pipelines using Azure Monitor and Azure Data Factory alerts. I also integrate Azure Key Vault for secure credential management and use Azure DevOps with Git for version control and CI/CD deployments.

I work closely with business analysts, solution architects, QA teams, and stakeholders to understand business requirements, deliver data solutions, troubleshoot production issues, and ensure successful project delivery using Agile methodologies.

As an Azure Data Engineer, I design, develop, and maintain scalable data pipelines using Azure Data Factory, Azure Databricks, PySpark, and Azure Data Lake Storage Gen2. I build end-to-end ETL/ELT solutions to ingest data from multiple sources, including SQL databases, APIs, FTP/SFTP, and cloud storage, ensuring reliable and efficient data movement.

I implement Delta Lake and Medallion Architecture (Bronze, Silver, and Gold layers) to create high-quality, analytics-ready datasets. I optimize PySpark jobs by applying partitioning, caching, broadcast joins, and performance tuning techniques to process large volumes of data efficiently.

I develop metadata-driven frameworks to automate data ingestion and transformation processes, implement data quality validation and reconciliation checks, and monitor production pipelines using Azure Monitor and Azure Data Factory alerts. I also integrate Azure Key Vault for secure credential management and use Azure DevOps with Git for version control and CI/CD deployments.

I work closely with business analysts, solution architects, QA teams, and stakeholders to understand business requirements, deliver data solutions, troubleshoot production issues, and ensure successful project delivery using Agile methodologies.