Senior Data Engineer
Astek Middle East
مجموع سنوات الخبرة :15 years, 9 أشهر
o Contributing to the architecture, design, and development of Big Data Solutions, including planning a roadmap and building the infrastructure.
o Working with project teams to implement Data Lakehouse and advanced analytics solutions by developing data pipelines (CI/CD, real-time streaming, batch processing). We are also developing a self-service BI platform using Power BI in the cloud.
o Implementation of Big Data ecosystem (HDFS & Iceberg, NIFI, Hive, Spark, Python, Kudu, Kafka and Flink) with Cloudera CDP Architecture.
o Building enterprise data engineering projects using DBT, Prefect, Docker and GitLab.
o Creating a master data management and data quality strategy across all systems as well as creating an enterprise data catalog.
o Leading and mentoring a team of data engineers as a senior member.
o Cross-trains other team members on technologies being developed, while also continuously learning new technologies from them.
o Coordinated with different ministries and government agencies for high level design, performing quality checks on deliverables, ensuring on time deliveries.
o Prepared Technical Design Documents based on the Functional Specifications provided by Business Users.
o A Used Erwin for Logical and Physical database modeling of the warehouse, responsible for database schemas creation based on the logical models.
o Defined technical requirements for ETL initial, historical and incremental load processes.
o Develop ETL jobs for extracting, verifying, integrating and loading data to the National Data Warehouse at MOI.
o Preformed Unit / Integration Testing for all the components and validated Results.
o Created and monitored sessions using workflow manager and workflow monitor.
o Developed Web services, open data portal and Dashboards to share data with others.
o Data warehouse Capacity and performance management.
IGAPE (Integration with government Agencies and Public DB Expansion)
The project aims to collect data from various ministers and government agencies, integrate, applying business rules, verify with MOI data and load to the national data warehouse. Make the data available to the security sectors and other government agencies by developing web service and national business intelligence and open data warehouse for the kingdom.
Technology Used: APS SQL Server DWH, MS SSIS, MS Visual C#, QlikView and Power BI
NLO (National Labor Observatory)
The project aims to collect data from various government agencies, applying business rules, verify with MOI data, automate loading and mask it then load to ELM zone.
Technology Used: Informatica, Oracle Exadata DWH
o Analyzed the business, collected and documented system requirements.
o Evaluated the existing DWH and BI settings, provided strategy and architecture solutions.
o Coordinated with team explaining high level design, performing quality checks on deliverables, ensuring on time deliveries.
o Estimated effort and time of project implementation.
o Performed data source analysis and Provided analysis documents on specific modules
o Designed and enhanced the Conceptual and logical data model that represent business information and define business rules.
o Working with users and coordinating with application developers to understand and capture data/reporting requirements
o Responsible for Documentation / Training for both technical non-technical end users.
o Cognos installation and administration.
o Developed various business models to allow users to create their own ad hoc reports.
o Provided end-user training to educate business users on technical aspects of various business units.
Jeddah Municipality BI Solution (Desktop& Mobile) and Open Data.
The purpose of the project is to help the municipality decision makers by analyzing the business data across the sub- municipalities, identify the needs, and many other factors as well as reporting (web and mobile) of the operational activities. Also create a BI platform for website users that shows different types of information and format that allow users to get easier, quicker and effective access to the organization data as BI self-service.
Technology Used: IBM Cognos BI, Framework Manager, Cube Designer, Cognos Studios
Jeddah Municipality BSC Solution
The project mission is to offers a proven approach to communicating business strategy throughout the municipality and measuring & managing performance against that strategy. It allows executives and business managers to instantly visualize (web and mobile) how the business is performing against key performance indicators (KPI’s).
Technology Used: Cognos Scorecards, Oracle DWH
o Worked in various projects and managed onsite and offshore teams.
o Developed multiple enterprise DW & BI solution.
o Involved in Designing the specifications, Design Documents, Data modeling and design of data warehouse.
o Implemented BI dashboards producing different summary results based on user view and role membership.
o Analyzed, designed, and developed ETL processes.
o Developed Test Matrices, Plans and Test Cases
Jeddah Municipality Data Warehouse
The project mission is to take advantage and utilize of a huge, historical amount data stored at municipalities databases by building an integrated platform that contains and consolidate data from around 25 apps, such as Correspondence tracking system, Financial system and incidents.
Technology Used: Oracle DWH, Oracle Warehouse Builder ETL
Makeyoon BSC solution
Design and implement Cooperate strategy map scorecards including objectives, KPIs, and initiatives level.
Technology Used: QlikView, SSIS ETL
o Developed data mart to support data management and BI needs for marketing team.
o Developed Ad-hoc and standard Reports/Dashboards with different Analytics Views.
o Developed, maintained, and managed all analytical and operational mission-critical business reports.
o Created stored procedures, scripts, and packages for applying the business rules.
o Involved in Business Acceptance Testing and User Acceptance Testing.
o Worked as an Enterprise Data warehouse team member to compile all data models into common model.
o Involved in analysis of end user requirements and business rules based on given documentation.
o Participated in the creation of the conceptual Enterprise data model (EDM).
o Mapped business requirements with the data requirements
o Implemented ETL process and applied business rules.
o Developed data mart to support data management and BI needs for marketing team.
o Created stored procedures, scripts, and packages for applying the business rules.
o Produced the data history and versioning design
o Involved in writing test cases, assisted Testing team.
Commercial International Bank (CIB) EDW Solution
This solution provides the basis for quality analysis of available data by deriving accurate information from data. The solution is business oriented, designed to support different business needs from regulatory and daily/weekly/decade/monthly operational and management reporting.
Technology Used: Teradata Database, MS BI (SSAS&SSRS), Data Stage.
Egyptian Taxation Authority (ETA) Teradata Taxation Solutions - Tax Compliance Solution
Develop overall solution for the ETA components, this solution covers having overall solution diagram, ETL architecture and lifecycle, BI architecture, staging and data integration.
Technology Used: Teradata Database and Utilities, MS BI (SSAS&SSRS)
Teaching the following courses:
- C/C++ Programming Language (Basic and Advanced)
- XML and Java Script (Basic and Advanced)
- Java Programming Language
Data Warehousing and Business Intelligence Diploma, Information Technology Institute (ITI), Egypt, 2009
B.Sc. Computer Science, Cairo Higher Institute, Egypt, 2007
