Data Engineer - IT
The Software Works Inc
مجموع سنوات الخبرة :12 years, 7 أشهر
The Software Works Inc is emerging as next-gen consulting, services and products company focused on Retail, Supply Chain & Financial Services Modernization. Tsworks (country) is an owned subsidiary of The Software Works Inc
Responsibilities:
Key contributor to the “FinWorks” platform for tsworks
Working for multiple retail and FinTech clients for tsworks, building data platforms and applications
Designing and implementing conceptual, relational, and dimensional data models and meta models in Snowflake (OLAP) and Postgres (OLTP).
Building real-time data integration framework using Python, WebSocket connections, Webhook APIs, Azure Kubernetes, Azure functions, Azure Blob Storage, Azure Postgres, Kafka and AMQP protocol on Azure EventHub for “FinWorks” platform and for clients of tsworks
Developing and maintaining APIs on Azure using Azure API Management, Azure Functions, Python, Fast APIs, and containers on AKS for multiple clients. Used Continuous Integration and Continuous Delivery pipelines using multiple tools
Developing re-usable patterns and corresponding and pipelines for each pattern for data ingestion from flat, relational and JSON data sources into Azure Data Lake Store and then to Snowflake using Azure Data Factory.
Implementing Lambda Architecture for data management combining real-time data streaming and batch pipelines
Deploying Azure Infrastructure using Azure Resource Manager (ARM) templates, Azure CLI and Terraform.
Understand project scope, timelines, objectives, and direction of different projects and lead the execution by coordinating with business and technology teams and delivering on-time by supporting the offshore development team
Develop and maintain technical documentation for all implemented solutions.
Responsibilities:
Work with internal and external stakeholders to understand and document business needs and requirements. Translate business needs into detailed specifications, integrate and process large datasets from various systems. Develop Data transmission process to stage FIS provided Customer Demographic & Transaction files.
Develop process automation solutions for a variety of business challenges including Tokenzatation on PIC data. Created a atert process that communicates to TCB, FIS & VGS if the file transfer was successful or failed. Developed an ETL process using SSIS to handle Card data and implemented data retention policy.
Data mapping requirements to satisfy Actimize requirement, Test and validate solutions and related data.
Designed SSIS Packages to extract, transfer, load (ETL) existing data into SQL Server from different environments for the SSAS cubes.
Expert in creating Star schema cubes using SSAS, extensively worked on OLAP cubes using SSAS.
Created and Managed OLAP Cubes using SSAS Tabular model.
Accepted responsibility to lead the new data quality initiative using Collibra Data Quality Tool to manage Data Quality across TCB under one umbrella.
Experience using various sources (Azure, Snowflake, SQL, AWS) to build a repository of Data Quality Rule using Collibra definitions.
Implemented Data Quality rules using DQ dimensions including Consistency, Timeliness, Uniqueness, Completeness and Validity.
Build a Data Quality model to leverage high-quality, trusted data to drive strategic decisions across enterprises. Develop and maintain technical documentation for all implemented solutions.
Expertise in critical thinking skills and complex problem-solving abilities
Experience in managing projects and processes effectively and delivering according to scheduled deadlines
Ability to positively and proactively manage multiple priorities under tight deadlines and thrive in a fast-paced, dynamic team environment.
Involved in requirement gathering from business users and help Business Analysts and to generate Data mapping documents for new Automation ETL Process.
Developed, deployed and monitored SSIS Packages for new automated ETL Processes for the on-going manual ETL Processes.
Responsible for the design, development and implementation of dynamic and customized SSIS packages for ETL development following company standards and conventions.
Developed Stored Procedures, Functions, and SQL scripts for performing automation tasks.
Expertise in SSIS Project Model, using Project Parameters, and deployment process in Production.
Proficient in secluding a SQL Server Agent job to execute a SSIS package and operating SSISDB and debugging Packages using reports.
Excellent in Troubleshoot and debug Packages found the root cause and fixed the issue.
Created Logs to record the success and failure of the package executions.
Configured database mail and created a job alert using SSMS to send notification using email.
Implemented error handling and utilized event handlers for automated notifications using SSIS.
Created various environments in SSIS package, deployed the Ispac files directly to the Integration Service Catalog in SSISDB.
Accomplish various transformations like data conversions, lookups, adding derived columns, Merge Join, Multicast in SSIS for the checking and processing Data in the Applications.
Developed and maintained report models to be used as data source for ad-hoc reports.
Hands on experience in generating on-demand and scheduled reports for business analysis or management decisions using SSRS.
Scheduled Daily, Weekly, Monthly reports for various categories and regions based on business needs using Reporting services.
Performed unit testing, regression testing, and End-to-End testing accordingly.
Supported 24/7 production On - Call on rotation basis.
Interstate Battery System of America, Inc., Interstate Batteries, is a privately owned company that markets automotive batteries. Interstate Batteries operates a distributor network that supplies batteries and power solutions to over 200, 000 dealers.
Responsibilities:
Analyzed business requirements and Created High level (Server/Application) Migration Plan Document.
Worked in a large group with responsibility to recognize and reinforce strong performance of the team.
Worked in Business analyzing team with responsibility to recognize and reinforce strong performance of the team.
Identify, coordinate, and work with Production, Development, and Q.A teams to solve complex issues.
Collaborated with ETL, BI and DBA teams to analyze and provide solutions to data issues and other challenges while implementing the OLAP model.
Migrated database from SQL 2008 and 2012 to 2014 including T-SQL Jobs, SSIS Packages, and SSRS Reports. Troubleshoot, debug and modified Stored Procedures, UDFs, Views and other database objects to fully functional in 2014 environment.
Track and Sync the database objects between different environments, by generating /comparing Scripts using third party tool Red Gate.
Upgraded SSIS Package from Package deployment model in SSIS 2008 to new Project deployment model in SSIS 2012 and 2014.
Manipulate SSIS Control components like Execute SQL Task, Data Flow Task, Execute Package Task, File System Task, Script Task, Containers, and perform various Transformations in Data Flow Task.
Created shared data sources in Report Manager to make data source connections more manageable.
Created data-driven subscriptions that roll out reports to a large recipient list.
AIM Specialty Health is a national leader in specialty benefits management. The project is working in a way that it collects information from patients, according to their problem it matches the corresponding provider with respective medication service. It collects the feedback from customers about the service, analyzes the wellness of service so that the report is used to recommend the service for new customers.
Responsibilities:
Identify, coordinate, and work with Production, Release, and Q.A teams to solve complex issues.
Attended Business meetings to assemble functional requirements and also served as the primary point of contact for most of the Enterprise Business.
Wrote Complex SQL Queries using Joins, Functions, CTE, With and Group by Clause, to generate Reports based on the business requirement
Wrote Stored Procedure for Front end System and SQL queries to solve production problems.
Created SSIS Packages for migration of data to MS SQL Server database from other databases and sources like XML, MS Excel, and Flat Files.
Performance tuning of existing database objects including stored procedures and indexes.
Designed SQL Server Integration Services (SSIS) packages to extract data from different sources to SQL Server Data Warehouse using various control flow tasks.
Scheduled Daily, Weekly, Monthly reports for various categories and regions based on business needs using Reporting services.
Developed and maintained report models to be used as data source for ad-hoc reports.
Tools: Microsoft SQL Server Enterprise edition, SSRS, SSIS, SQL Server Profiler, MS Office, MS Access, MS Windows Server 2012, and FoxPro.
STARS: Samsung test automation and requirements is a database system between Samsung devices and carrier’s requirements.
Responsibilities:
Created new database objects like Procedures, Functions, Packages, Triggers, Indexes and Views using T-SQL in SQL Server 2008 r2.
Worked on Client tools like SQL Server Management studio to administer SQL Server. Used SQL Server System Tables to retrieve metadata.
Write Complex SQL Queries to generate Reports based on the business requirements.
Deployed created reports and assigned credentials.
Created SSIS Packages to import and export data from different carriers like AT&T, Verizon, and Sprint. Scheduled reports for Daily, Weekly, Monthly reports for various categories and regions based on business needs using Reporting services (SSRS)
Occasionally responsible for the design, development and implementation of dynamic and customized SSIS packages for ETL (extract, transform, and load) development following company standards and conventions.
لقد تم حذف الرابط بسبب انتهاكه لسياسة الموقع. يرجى التواصل مع قسم الدعم لمزيد من المعلومات.