Business Data Analyst
Credence Integrated Marketing Pvt Ltd
مجموع سنوات الخبرة :3 years, 5 أشهر
Credence Integrated Marketing Private Limited logo
Data EngineerData Engineer
Credence Integrated Marketing Private Limited · Full-timeCredence Integrated Marketing Private Limited · Full-time
Jan 2022 - Present · 1 yr 10 mosJan 2022 - Present · 1 yr 10 mos
Mumbai, Maharashtra, India · On-siteMumbai, Maharashtra, India · On-site
helped me get this jobLinkedIn helped me get this job
Extracted huge volumes of data from disparate data sources like Rss feeds, Rest Apis, S3 bucket, Azure Blob using Python libraries.Built ETL pipeline using Apache Airflow with various transformations and enrichment steps along with destination Data store as a highly scalable Data lake in Apache Jena.Created Jobs in Scala to store batch data and implement Spark for faster processing of data (40% faster).Implemented Data Validation pipelines to identify Data anomalies and data quality issues.Created multiple reports for transformed data to help data driven decision making. Created Dashboards for stakeholders to track KPIs.Leveraged Docker, kubernetes, helm charts, AWS ECR, EKS for deployments.
Worked on Fintech Data finding keen insights by Exploratory Data Analysis to showcase patterns and trends in data. Used Power Bi to transform and generate Reports . Presented Monthly detailed reports to members of management
for efficient organizational decision making. Created, modified and optimized existing SQL queries with joins, subqueries, CTE's, and complex formulae to derive critical insights
from. Engaged with Data Science team for EDA and Feature Selection for Machine Learning model to optimize the responses.
Designed and built logical, physical Data Models to support business objectives.Automated Data Ingestion and Manipulation related tasks of raw data.Followed coding standards, guidelines, code reviews and release cycles.
Studied bachelors of computer application.