Data Engineering Intern - IT
Stellantis
Total years of experience :1 years, 4 Months
- Developed an end-to-end data application inside Foundry, from data collection to dashboard creation
- Monitored and analyzed all Data Governance Rules and Policies inside Foundry Palantir.
- Used PySpark and Slate Application with PostgreSQL to build the interactive monitoring application.
- Defined the rules and policies for all the platform datasets, projects, ontologies, and data sources.
- Implemented a centralized CI/CD process for all ELT projects, using TeamCity, Kotlin and GitHub.
- Tested new technologies, proposed new solutions to enhance some of the projects.
- Utilized agile methodologies, prioritized tasks effectively and optimized project workflows.
- Analyzed & predicted data from building equipment to optimize maintenance and reduce costs.
- Collaborated with the project manager to define objectives and create KPIs in Power BI.
- Pre-processed Data and developed machine learning algorithms.
- Analyzed, made predictions, and tried to improve the decision-making
- Built Deep Learning Pipeline system from scratch to facilitate the work of data scientists .
- Studied the Software architecture and use cases.
- Created an end-to-end process for the pipeline, starting from uploading, then job management and
execution, training, and concluding with tracking and monitoring results and logs.
- Used Django web application, MinIO Object storage, Nomad HashiCorp, Docker and MLflow.
- Hands on experience with Python
- Tested CI/CD tools to automate training on PR using GitHub Actions.
- Distributed training using Horovod on multiple servers & provided redundancy using Docker Swarm.
- Each task follows TDD process & was version controlled using git