Summary:

We are seeking skilled AI/ML Engineer to develop, implement and manage AI/ML workflows for various industries. The ideal candidates will have a minimum of 5+ Years of experience in Designing, Implementing, and leading data engineering & AI/ML projects. In this role, you will be responsible for the ideation, pipeline creation, deployment, testing of data ingestion, transform, load from various sources while working with cross-functional teams. Working experience in Machine Learning (ML), GenAI models will be added advantage. Following are the Key requirements:

• Establish technical designs to meet business and technical requirements
• Optimize solutions (ETL / data pipelines) to balance performance, functionality, cost, and other operational requirements
• Design, develop, and optimize data workflows and notebooks using Databricks to ingest, transform, and load data from various sources into the data lake
• Build and maintain scalable and efficient data processing workflows using Spark (PySpark or Spark SQL) by following coding standards and best practices
• Collaborate with technical and business stakeholders to understand data requirements and translate them into technical solutions
• Develop data models and schemas to support reporting and analytics needs
• Ensure data quality, integrity, and security by implementing appropriate checks and controls
• Monitor and optimize data processing performance, identifying, and resolving bottlenecks
• Understanding of DevOps principles and practices

Qualifications:

·      Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or related field

• 5+ years of experience in designing, implementing, and maintaining data solutions on Databricks
• Experience with at least one of the cloud platforms – Azure, AWS or GCP
• Experience with Azure Data Factory, Azure Databricks, Delta Lake along with Python & SQL
• Experience with ETL (Extract, Transform, Load) processes
• Experience with prompt engineering
• Experience in Generative AI, Large Language Models, AI, ML, Deep Learning, and related technologies
• Experience with distributed computing and scalable systems
• Experience with DevOps and CI/CD
• Excellent problem-solving skills and the ability to diagnose and resolve complex technical issues

Strong communication skills, both written and verbal, to effectively convey technical information to non-technical stakeholders