Deepesh Agrawal

• Built an audio analytics pipeline for Presight Synergy to identify who spoke, when, and what was
said by combining pyannote.audio speaker diarization with Whisper large-v3 speech-to-text, while
evaluating Microsoft VibeVoice for long-form, multi-speaker conversational audio workflows.
• Worked on video analysis and signature detection for Synergy use cases using Qwen-family
multimodal / vision-language models, applying VLM prompting, frame-level analysis, structured
detection outputs, and agentic AI workflow patterns.
• Used Ray Serve, NVIDIA Triton Inference Server, and vLLM for scalable model serving across LLM,
VLM, speech, and multimodal workloads, supporting MCP-ready / agentic AI integration for
exposing model inference and analytics workflows through conversational interfaces.
📌 Technologies: Python, Java, PyTorch, pyannote.audio, Whisper, Microsoft VibeVoice, Qwen-family
VLMs, Ray Serve, NVIDIA Triton Inference Server, vLLM, LangChain, LangGraph, MCP, Vector DBs

• Cross-lingual RAG-based QA System:
• Designed and deployed a multilingual RAG system for knowledge retrieval across languages.
• Integrated semantic search, Generative AI (Gen AI), and LLM fine-tuning, improving response accuracy by 30%.
• Deployed a real-time API using Flask, optimized for ARM-based architecture.
• End-to-End MLOps with CI/CD:
• Developed a fully automated CI/CD pipeline for model training, deployment, and monitoring.
• Implemented quantization techniques, reducing inference latency by 40%.
• Optimization of AI Models for ARM CPUs:
• Optimized oneDNN primitives/operators for deep learning workloads on aarch64 CPUs, improving inference
efficiency.
Technologies: Python, C++, PyTorch, LangChain, Haystack, LlamaIndex, Generative AI, Jenkins, Git, MLflow,
Kubeflow, Grafana, Prometheus, Apache Airflow, AWS, Azure, llama.cpp.

• Contributed to pretraining-related work and led the post-training, alignment, evaluation, and enterprise
deployment of Granite-4.0-H-Small from Granite-4.0-H-Small-Base, using supervised fine-tuning
(SFT), reinforcement-learning-based alignment (RLA), and model merging to improve instruction
following and production readiness for large-scale agent systems.
• Re-architected production LLM-based agent platform for large-file workloads, reducing memory
usage by ~3× and eliminating stability issues under load, enabling reliable enterprise-scale operations.
• Converted blocking data operations into asynchronous execution pipelines, improving platform
scalability and supporting higher concurrent agent interactions.
• Designed context-aware, multi-step agent workflows using graph-based orchestration and vector
retrieval, enhancing planning depth and response quality.
• Developed actionable agents via MCP-based tool integration and external APIs (e.g., GitHub),
enabling reliable real-world system interaction.
• Improved LLM output robustness and alignment using advanced prompting techniques, including
Chain-of-Thought (CoT) and Tree-of-Thought (ToT) reasoning, along with structured reasoning
evaluation and cross-team model optimization, boosting reliability and production readiness.
• Enabled dynamic, user-specific registry and CRUD operations for agents, OpenAPI tools, and MCP
servers within the Agentic AI Framework (ARIA), supporting agent-to-agent communication and
enhancing platform flexibility, customization, and real-time adaptability.
📌 Technologies: Python, watsonx.ai, OpenAI GPT-4o, LangChain, LangGraph, MCP,
Kubernetes/OpenShift, Kafka, Redis, Vector DBs (Milvus/Chroma), Grafana, Terraform, pytest.

• Natural Text-to-SQL Query Generation (Capital Group):
• Fine-tuned T5-LM-Large-text2sql-spider, converting natural language queries into SQL.
• Achieved 90%+ query accuracy, leveraging LLMs, NLP, and tokenization techniques.
• Excel Report Table Extraction (Swiss Re):
• Automated table extraction from unstructured spreadsheets using machine learning & OCR.
• Reduced manual processing time by 70%, enhancing data engineering pipelines.
Technologies: Python, PyTorch, Hugging Face Transformers, Flask, Docker, MySQL, Generative AI (GenAI), LLMs,
OpenPyXL, Pandas, scikit-learn, NLP, BitBucket.

• Integrated ZenDNN (AMDs deep learning library) with PyTorch & ONNX Runtime.
• Conducted extensive benchmarking of ZenDNN vs. ONNX Runtime, achieving a 25% speedup in inference on AMD
EPYC CPUs
Technologies: Python, PyTorch, ONNX Runtime, ZenDNN, Docker, PyTest.

• Integrated ZenDNN (AMDs deep learning library) with PyTorch & ONNX Runtime.
• Conducted extensive benchmarking of ZenDNN vs. ONNX Runtime, achieving a 25% speedup in
inference on AMD EPYC CPUs
📌 Technologies: Python, PyTorch, ONNX Runtime, ZenDNN, Docker, PyTest.

• Optimized Daimlers deep learning code in collaboration with NVIDIA engineers.
• Deployed YOLO-based object detection models on Jetson Nano for real-time video processing.
Technologies: Python, CUDA, PyTorch, TensorRT, Jetson Nano, Nsight System, OpenCV.

• Diabetic Retinopathy Detection: Trained InceptionResNetV2 to classify retinal images.
• Video Analysis for Social Media Content: Built ML models for content moderation in social media
videos

Hand Gesture-Controlled Drone: Developed a gesture-controlled drone using an SSD-based deep learning model,
deployed on Raspberry Pi 3. Drone Object Tracking & Navigation: Built an object tracking & navigation system using Mean Shift algorithm and
GPS data for drone movement.
Technologies: Python, TensorFlow API, SSD, CNN, Raspberry Pi 3, OpenCV, GPS, Mean-Shift Algorithm

• Hand Gesture-Controlled Drone: Developed a gesture-controlled drone using an SSD-based deep
learning model, deployed on Raspberry Pi 3.
• Drone Object Tracking & Navigation: Built an object tracking & navigation system using Mean Shift
algorithm and GPS data for drone movement. Technologies: API, Pi Mean-Shift
Algorithm