Senior Data Engineer
Daraz (Part of Alibaba Group)
مجموع سنوات الخبرة :7 years, 0 أشهر
Daraz is an Alibaba owned ecommerce company with presence in 5 countries (Pakistan, Bangladesh, Srilanka, Nepal and Myanmar).
As member of the regional data engineering team, I play an impactful role in supervising data engineering operations related to strategy & planning, customer experience, and payments. My primary objective is to create reliable and efficient ETL processes while seamlessly integrating data from various systems and APIs. Moreover, I ensure the Data Warehouse is maintained according to industry best practices, serving as a crucial resource for over 750 analysts within Daraz. In addition to my technical responsibilities, I frequently collaborate with business stakeholders to understand their specific requirements for dashboards and essential tables utilized by data science and data analyst teams.
Technology stack: Hive, Python, Alicloud, Max Compute (Data), mongodb
As a DataOps Engineer, I worked within the Strategic Markets Insights (SMI) team to streamline tasks using Python automation. This involved creating data-extraction APIs and converting VBA code into Python for data modeling. I also developed data crawlers that utilized public sources like WHO and OECD, supporting analysts in their daily work. Additionally, I created a Python Dash web app and desktop applications for different teams, including a sales-focused app providing comprehensive visibility of Statistas KPIs for improved sales pitches.
Technology stack: Python (3.6), Major lib: Peewee (ORM), Pandas, mysql, Bitbucket (Version Control), Jenkins (for Automation), AWS (Lamdba functions)
In my role, I focused on key tasks such as developing the Insights Service Platform, enabling Excel table generation, maintaining backend end points, creating a data export service for API-based data exchange, and enhancing web applications. These efforts aimed to improve data accessibility, reporting, and overall user experience, ensuring efficient energy and building management for customers.
Technology stack: C#, .NET Core, RabbitMQ, InfluxDB (for real-time), MS SQL Server, Gitlab (Version Control)
Daraz is currently Pakistans largest and fastest-growing e-commerce startup and was backed by Rocket Internet. The Alibaba Group bought all of Darazs business ventures in May 2018. I was a member of the Central Business Intelligence team, which was based in Pakistan and worked in the data analytics and engineering fields.
Tasks before Alibaba acquisition: 1. Improved the analytical architecture on Qlikview and data warehouse on MS SQL Server. 2. Automated reports which were required on recurring basis. (Java, VBS). Tasks after Alibaba acquisition: 1. Understood the new tools in Alibaba ecosystem (Databases, Visualization, Data Integration). 2. Worked with the teams of Alibaba and Lazada (another e-commerce in Singapore) to create ETL architecture for Daraz. We worked extensively in Singapore for 4 months to migrate legacy data from MS SQL Server to Max Compute (Alibaba Cloud Database). 3. After migration, developed and maintained ETL processes for all countries of Daraz. 4. Shifted all reports from old architecture to Alibabas analytical platform.
Learning Toolbox: MS SQL Server, SSIS, Python, HiveQL, Max Compute (Data IDE), Airflow
Qordata is a data analytics and integration firm that focuses on the pharmaceutical, biotechnology, and medical device industries. I started as a part-time BI Developer at the company and later (after graduation) joined the Product Development (PD) team as a full-time employee.
The primary focus of the PD team was on analytical applications (SaaS) for the US and EU regions. My primary responsibilities included creating and debugging dashboards on TIBCO Spotfire in all AWS environments, including Production, Trial, and Demo. I was also in charge of Spotfire Server administration and related issues. I also used to collaborate closely with the Data Engineering team to create the necessary table/views on MS SQL Server for use on Dashboards.
Further responsibilities were as follows:
1. Understanding of multiple Healthcare data-sets in US (CMS, Medicare, FDA etc.) and EU (EFPIA and Non EFPIA) region. 2. Understanding TIBCO Spotfire architecture (7.0 and 7.5+) and administering Spotfire Server and Automation Services on Production Servers.
Learning Toolbox: SQL Server, SSIS, TIBCO Spot-fire, Python, Mongo DB, Scrum and Kanban
Specialization in AI Grade: 1.5/1 [1 being the highest]
