Data Architect
Capgemini
Total years of experience :14 years, 0 Months
Data modeler and Architect for several clients like Unilever, Swed bank and Telia.
Data warehousing professional :
DATA MODELING
Requirement gathering - Interviews, Questioners, etc - Output - Business Rules, KPIs list
Source system analysis -
There should be a solution Architect for internal DWH who will drive all the standards and etc - one man defining and approving all.
Data architect should work with him and also they both should be in touch with someone known as ENTRIPRISE DATA ARCHITECT ( if any here in Etisalat)
Also the data modeler will be with this data architect to maintain the meta data and the below conceptual, physical and logical models
Top-down - Data model is derived from complete understanding of the business
Bottom-up - Data model is derived by reviewing specifications and business documents only required
Deliver Conceptual, Logical and physical Data model
In conceptual - highlevel tables and relations
In logical - tables, column names, relations, primay key, forign keys etc
Physical - tables, scripts(DDL) data types, cardinalities, indexes etc
Design Documents
Solution Architecture
Data Model
ETL Framework
Report Framework
Operation Manual
Back up and Recovery Strategy
ETL DESIGN
• DEFINE ETL CODING STANDARDS AND CHECKLIST
• DEFINE ETL FOLDER STRUCTURE
• ETL VERSION CONTROL
• ETL PROCESS FLOW
• DEFINE SOURCE FORMATS
• IDENTIFY PRE-SESSION AND POST-SESSION UNIX SCRIPTS
• IDENTIFY SCHEDULE FREQUENCY
• LOADING STRATEGY
• ERROR HANDLING STRATEGIES
• EXCEPTION HANDLING STRATEGIES
• DEFINE FAILOVER STRATEGIES
• POST PROCESSING HOUSEKEEPING
• DATA RETENTION
Datastage Job Folders
Unix Data Folders
Unix Sub Directories
Sample ETL Process Flow
Exception Handling Process
E2E ETL Process Flow
Source to Staging RDBMS data movement
ONE OF THE JD for DATA MODELER from GOOGLE “
• Requirements analysis: In conjunction with a solution architect or independently, work with the business to collect, document and review functional and non-functional requirements for the application
• Architect: In conjunction with a solution architect or independently, decompose the application into various components and make appropriate technological choices in the implementation and integration of the components also giving due consideration to data availability, recovery, security, performance and governance requirements.
• Analysis and design: Define and document the logical data model and create the physical data model. Define and create semantic data models for reporting or analysis. Identify the various contributing source systems and source data elements; help define & build the extraction functions to obtain the source data supporting change data capture. Define and document the various data transformations that have to occur during this process. Incorporate maintainability, security and performance requirements into the design and specifications. Review design and specifications with technical team.
• Build and test: Create physical data models and create semantic data model. Support the development of extraction functions and transformation logic. Document all design and specifications. Document metadata information. Develop early prototypes for the application as necessary. Support the development of complex logic or functions as necessary. Remain accountable for the quality of all data modeler deliverables with appropriate testing.
• Consult and manage change: The data modeler remains actively engaged during the build of the application, addressing any design or information gaps, supplementing any development or testing needs, addressing functional and non-functional issues, making enhancements to deliverables to accommodate changes and providing technical guidance as necessary.
Always interested in Database , RDBMS and scored highest during college