Anuj Upadhyay

• ABSA (South Africas largest bank) — Personally built a generative AI document intelligence pipeline
using Azure OpenAI and Azure Document Intelligence to automate KYC compliance document extraction
and classification, reducing manual review time significantly
• ABSA — Developed and maintained RAG-based Q&A workflows over internal policy and regulatory
documents using Azure AI Search (vector store), LangChain-style orchestration, and prompt engineering
on GPT-4o — Designed and implemented an end-to-end data ingestion and transformation pipeline on
Azure Data Factory and Databricks, processing structured and unstructured banking datasets for credit risk
analytics — Built MLflow-tracked model experimentation workflows on Databricks for customer
segmentation; managed model registry, versioning, and batch scoring pipelines
• Built a supplementary data extraction workflow on AWS using S3, Lambda, and AWS Glue to pull and
stage third-party data feeds prior to ingestion into the primary Azure-based pipeline
• Developed reusable Python libraries for data extraction, embedding generation, and semantic search —
packaged and deployed via Azure DevOps CI/CD pipelines
• Implemented agent-based AI workflows for multi-step reasoning tasks across structured datasets, applying
Azure OpenAI function calling and tool-use patterns
• Maintained data quality checks, pipeline monitoring via Azure Monitor and CloudWatch, and documented
solution architecture for handover and production support

• SingPost — Personally designed and implemented a cloud-native data pipeline on Azure Data Lake Gen2
and Azure Synapse Analytics for logistics KPI reporting, consolidating data from multiple operational source
systems
• SingPost — Built PySpark transformation jobs for large-scale structured data processing; optimised query
performance and reduced reporting latency through partition pruning and caching strategies
• Delivered a parallel AWS-based analytics workload using Amazon Redshift Serverless and AWS Glue
Studio for a client with existing AWS infrastructure, enabling cross-cloud reporting consolidation
• Deployed containerised model scoring services on Amazon EKS, integrating with an existing AWS
CodePipeline CI/CD setup for automated testing and release
• Conducted statistical analysis and KPI trend modelling across multi-year datasets to surface actionable
insights for business stakeholders; delivered findings through Power BI dashboards
• Set up Azure DevOps CI/CD pipelines for automated testing and deployment of data pipeline code,
ensuring consistent environments across dev, staging, and production
• Integrated Azure RBAC, Key Vault, and Log Analytics alongside AWS IAM policies for data platform
security and observability across the project lifecycle

• DBS Bank — Individually built a hybrid data lakehouse on Azure Databricks and Azure Data Factory,
migrating legacy SQL-based ETL workloads to Delta Lake; implemented Unity Catalog for data governance
and access control
• DBS Bank — Developed ML pipeline foundations using Databricks Feature Store and MLflow 2.0 for a
customer churn prediction model; managed experiment tracking, model registry, and REST API serving via
Azure API Management
• SATS (Singapore Airlines ground handling) — Designed and implemented event-driven data pipelines
using Azure Event Hubs and Azure Stream Analytics for real-time flight operations analytics; built
downstream aggregation jobs in PySpark
• SATS — Extended the pipeline to an AWS leg using Amazon S3 and AWS Glue for archival and cross
system reconciliation, integrating with the clients existing AWS data lake
• Heineken APAC — Personally re-platformed monolithic SQL ETLs to distributed Spark pipelines on Azure
Databricks; optimised job performance using Photon runtime and autoscaling cluster configurations,
reducing processing cost by 35%
• YCH Group (logistics) — Built end-to-end data ingestion and transformation pipelines using Azure Data
Factory and Azure Data Lake Gen2, integrating supply chain data sources for operational reporting; set up
AWS S3-backed cold storage tier for historical data archival
• Packaged reusable data pipeline components as internal Python libraries, deployed via Docker containers
and Azure DevOps CI/CD — reducing new project setup time by 40%
• Developed a paid MVP solution published on the Microsoft Azure Marketplace, demonstrating full
ownership from architecture through to production deployment

• Bajaj FinServ — Personally built data ingestion pipelines on Azure Data Factory v2 pulling from SQL
Server and flat file sources into Azure Data Lake Gen2; developed PySpark transformation jobs for loan
analytics and risk scoring datasets
• ICICI Bank — Implemented CI/CD pipelines using Azure DevOps for automated deployment of data
pipeline code and model scoring services; containerised workloads with Docker and Kubernetes (AKS)
• ICICI Bank — Built supplementary AWS Lambda functions for event-driven data preprocessing, storing
outputs in S3 before ingestion into the primary analytics pipeline
• Dr. — Built ETL workflows using Azure Data Factory and Python to consolidate
clinical and supply chain data; automated data quality validation scripts that flagged anomalies before
downstream consumption
• Reliance General Insurance — Re-platformed legacy stored procedure-based ETLs to Azure Databricks
Spark jobs; reduced batch processing time from hours to minutes on claims and policy datasets; used AWS
S3 as intermediate staging layer for raw data feeds from third-party providers