We are looking for a skilled Data Engineer with a background supporting LLM applications to join our team. You will work closely with data scientists and be responsible for developing and implementing large language model (LLM)-based applications. This includes working with both proprietary and open-source models and leveraging frameworks such as LangChain to ensure seamless integration and deployment.

Responsibilities

• Develop and implement applications that interact with LLM models.
• Build RAG-based applications.
• Work with vector databases for LLM-based applications.
• Integrate models with existing systems and APIs.
• Develop and maintain production-quality data pipelines and ETL processes.
• Preprocess and manage data for training and deployment.
• Collaborate with cross-functional teams to define, design, and deploy new features.
• Write clean, maintainable, and efficient code.
• Document development processes, code, and APIs.

Requirements

• Proven experience in developing and deploying machine learning APIs.
• Experience in building scalable applications capable of handling large volumes of data.
• Strong knowledge of API integration (RESTful, GraphQL).
• Experience with data preprocessing, SQL, and NoSQL databases, as well as vector stores (e.g., Postgres, MySQL, Solr, Elasticsearch, OpenSearch).
• Familiarity with deployment tools (Docker, Kubernetes).
• Experience with DevOps tools such as Jenkins, Terraform, or Cloud Formation templates is a plus.
• Strong problem-solving and communication skills.
• Experience with distributed computing technologies such as Spark, Hadoop, or EMR is preferred.
• Ability to work effectively in an agile team environment.

Preferred Qualifications

• Degree in Computer Science, Data Science, or a related field.
• Certifications in machine learning, data science, or cloud computing.