Pawan Harne

• Led a 3-member data engineering team within PwC's Analytics COE, driving the architecture of robust end-to-end data infrastructure for financial services clients.
• Ensured scalability, high availability, and cost-efficiency in all data solutions delivered.
• Designed and implemented production-grade data pipelines on Azure Databricks and Synapse, enhancing ingestion, transformation, and serving layers.
• Utilized Medallion architecture to significantly reduce data latency by standardizing processing SLAs.
• Owned the solution design across Azure Data Lake and Azure Data Factory (ADF), ensuring seamless integration and data flow.
• Fostered collaboration and innovation within the team, enhancing overall project delivery and client satisfaction.

• Transitioned from Associate Data Engineer to Data Engineer Expert, leveraging extensive experience in housing finance and investment products.

• Developed and optimized ETL pipelines using PySpark, Databricks, and AWS Glue, enhancing data ingestion and transformation processes.

• Achieved a 30% improvement in pipeline performance through strategic Spark optimization, partition tuning, and query redesign.

• Built comprehensive data models that support 360-degree customer investment views, effectively managing data for over 800K customers.

• Utilized technologies such as AWS Glue, ADF, Power BI, and MySQL to drive data-driven decision-making and insights.

• Collaborated with cross-functional teams to ensure data integrity and accessibility for stakeholders.

• Transitioned from Senior Analyst to Data Engineer Expert, leveraging extensive experience in capital markets and investment banking.

• Engineered robust ETL workflows for trade lifecycle data, enhancing execution, clearing, and settlement processes for Barclays and Nomura.

• Processed over 30M records daily, ensuring high reliability and accuracy in data management.

• Developed performance-optimized PySpark pipelines, achieving a 40% reduction in job runtime through advanced Spark optimization techniques.

• Utilized AWS Glue, Databricks, and Apache Kafka to streamline data integration and processing, driving efficiency in data engineering practices.

• Collaborated with cross-functional teams to deliver scalable data solutions that support critical business decisions.

• Leveraged strong SQL skills to design and optimize complex database queries, enhancing data retrieval efficiency by 30%.

• Developed and maintained ETL processes to ensure seamless data integration from multiple sources, improving data accuracy and consistency.

• Collaborated with cross-functional teams to gather requirements and deliver data solutions that meet business needs.

• Implemented data warehousing solutions, enabling advanced analytics and reporting capabilities for stakeholders.

• Utilized big data technologies such as Hadoop and Spark to process large datasets, driving insights and decision-making.

• Ensured data quality and integrity through rigorous testing and validation processes, resulting in a 25% reduction in data-related issues.

• Continuously explored new tools and technologies to enhance data engineering practices and drive innovation.