Are you passionate about data engineering and ready to take on the challenges of working with cutting-edge technologies? At DreamzTech Solutions, we are seeking a skilled Data Engineer to join our dynamic team in Kolkata (Hybrid/Remote). With a focus on advanced data solutions, machine learning, and cloud technologies, this role offers the perfect opportunity to work on impactful projects for high-profile clients. If you have 3 to 5 years of experience and a strong technical background in Python, PySpark, AWS, and Azure, we invite you to bring your expertise to DreamzTech, where innovation thrives and career growth is a priority.
Position: Data Engineer
Location: Kolkata (Hybrid/Remote)
Company: Dreamztech Solutions
Experience Required: 3 to 5 Years
Employment Type: Full-time
Technical Skills:
Python Programming Language Pyspark PostgreSQL Amazon Web Services(AWS)
Integration Azure databricks Azure DevOps MLOps Elastic Search Machine
Learning Natural Language Processing Selenium Beautiful Soup Scrapper API
Ubuntu NLTK Git Data Engineering Data Mining MS Excel
Experience as Data Scientist:
● Proficient in Python and PySpark for data manipulation, transformation, and model
development.
● Utilized scikit-learn (sklearn) and Apache MLlib libraries to build and evaluate
various machine learning models, including neural net and time series models.
● Leveraged Azure Data Bricks for scalable data processing and designing, testing,
and deploying machine learning models.
● Implemented Azure DevOps for continuous integration and continuous deployment
(CI/CD) of ML models.
● Integrated MLOps practices to enhance the lifecycle management of machine
learning models.
● Created automated pipelines for model evaluation, monitoring data drift, and
detecting model drift, ensuring sustained model accuracy and relevance.
● Implemented monitoring tools to continuously track and respond to changes in
data and model performance, ensuring robust and reliable ML solutions.
Project Inventory Management:
● Ideated and developed a forecast model using traditional machine learning and
time series techniques to predict a 4-month lead period based on 3 years of weekly
sales history.
● Trained multiple models in loops for territory and option-level cuts, training around
1600 models monthly based on data refreshes from the source.
● Used a feature store to dynamically retrieve and store data features, ensuring
seamless integration of new features into the model training process.
● Integrated MLFlow to track model performance, register and log model metrics and
parameters monthly. The best model prediction is dynamically chosen and used as
the final output.
● Used MLOps methodology to create an end-to-end pipeline from data ingestion to
monitoring data and models. Utilized Evidently AI to detect data drift.
● Collaborated with the platform team to productionize the ML pipeline.
● Worked with the data engineering team to manage data and meet data
requirements. ● Collaborated with Business Relationship Managers (BRMs) to
incorporate business input in retail.
Why Join Us?
Be part of a forward-thinking company that encourages innovation and
professional growth.
Lead a team of talented developers working on cutting-edge technologies.
Work on exciting projects that make a real impact for high-profile clients.
Competitive salary and benefits, with ample opportunity for career
advancement.