Data Engineer - IT
Mercury
Total years of experience :3 years, 1 Months
Responsibilities:
- building ETL pipelines;
- search for data sources within the company;
- creation of data structures in accordance with business requirements;
- access control to datamarts stored in DWH;
- deployment and support datamarts in GCP;
- data visualization in Power BI;
- use of Gitlab and CI/CD.
Achievements:
- implemented Dagster to orchestrate the work of ETL instead of the Windows Scheduler, which made it possible to increase the reliability of updating pipelines;
- implemented a Gitlab to collaborate with data analysts on the pipeline code, which allowed the team to always be sure that corrections are made to the latest version of the code;
- implemented CI/CD based on Gitlab to check the code before the next merge to the main branch, which allowed to reduce the number of errors in the code;
- allocated a separate server with the possibility of parallel work of several users. This made it possible to increase the reliability of the Dagster and Data Analysts were able to work in Power BI from their MacBooks by connecting to the server.
Responsibilities:
- deployment and support of DWH in GCP;
- building ETL pipelines;
- search for data sources within the company;
- creation of data marts or data slices;
- data visualization in Power BI.
Achievements:
- Implemented the transition to the use of Google Cloud Platform for data storage, which significantly increased fault tolerance and data update speed;
- Provided automatic updating of all reports in Power BI according to the schedule, which freed up additional time for analysts to work with requests from management;
- Created a new report in Power BI with sales data, which is used by the whole country to make management decisions;
- Compiled a knowledge base in Conflunce on the content of all current reports in Power BI, which made it possible to simplify communication with their users and systematize information on the sources involved;
- Implemented a sprint task scheduling system in Jira, which made it possible to create a single backlog visible to the entire team, as well as to evenly distribute the workload among other analysts and increase the planning horizon.
Responsibilities:
- draw up technical specifications for data marts and slices, as well as communicate with the DWH team within the framework of tasks;
- writing SQL queries;
- compiling dashboards in MS Power BI;
- participate in the development of a unified system of metrics and indicators for the entire company;
- update of terminology;
- participation in research related to the search for business growth points in general;
- analysis of the financial performance of the division, building an economic model, analysis of deviations;
- preparation of presentation materials for shareholders.
