Application Operations
Saudi Payments
Total years of experience :4 years, 4 Months
Running and Monitoring of ETL Batch, ETL Testing and Troubleshooting. Improved ETL performance using optimization techniques and automated ETL through SAP Data Services. I managed access and storage of EDW and Landing Server. Other services include managing Database Accesses and Storage, Archiving Data, Database Administration Activities.
Technologies Deployed: EDW (Teradata), SAP Data Services
I was responsible for daily Hadoop Administration, monitoring and troubleshooting of Hadoop Operations jobs. I did Offloading and Compression of data from Teradata to Hadoop, developed Automation of Partitions Validation and Purging of Out-of-Retention data. Other services include Upgradation of Hadoop Cluster (Cloudera), Clusters (Prod/Lab) maintenance as well as addition and removal of nodes using tools like Cloudera Manager, Performance tuning of Hadoop Clusters (Prod/Lab) and Databases. Also handled internal as well as external escalations and performed Root Cause Analysis of Operations jobs issues.
Technologies Deployed: EDW (Teradata), Hadoop (Cloudera), Spark, Informatica, MSTR, Celebrus, Remedy, NiFi, Service Level Agreements (SLA), Linux, BTEQ, SQLA, Crontab, ViewPoint
I developed Data pipelines from different sources (Oracle) to Hadoop (Cloudera) using Spark, Implement CDC and SCD-1. Ingested data from files to Hadoop using Shell Scripting. I did automated Data pipelines using Crontab and implemented logging and failure/success alerts. Also developed Recon Scripts using SQL and Shell Scripting. Migrating Data pipelines from Spark to NiFi and performed Data Validation and Data Verification.
Technologies Deployed: Hadoop (Cloudera), NiFi, Streamsets, Spark
Running and Monitoring of ETL Batch, ETL Testing and Troubleshooting. Improved ETL performance using optimization techniques and automated ETL.I did implementation of logging, checks (duplicate or null) and failure/success alerts in SSIS and also scheduling of reports on SSRS. Involvement in Hadoop Migration, performance tuning of Hadoop Clusters and SQL Server Databases. I performed Cluster maintenance as well as addition and removal of nodes using tools like Cloudera Manager. I managed access and storage on Data Lake and Database. I designed maintenance procedures and put them into operations. Other services include: managing Database and Data Lake security/integrity and backup procedures, writing reports, documentation and operating manuals, implement disaster recovery plans, archiving Data, developed Recon scripts and retention scripts using SQL and Shell Scripting.
Technologies Deployed: Hadoop (Cloudera), Spark, MS SQL Server, SSIS, SSRS, SSAS, Qlik Replicate
I have performed ETL Migration of Revenue Access Control (RACs) from Teradata to Oracle and automation of the RACs. Also did ETL Operations of the RACs and developed retention scripts using Shell Scripting.
Technologies Deployed: Teradata, Oracle