Overview

We are looking for a Python Data Engineer with expertise in real-time data monitoring, extraction, transformation, and visualization. The ideal candidate will have experience working with Oracle SQL databases, multithreading, and AI/ML techniques and should be proficient in deploying Python applications on IIS servers. The role involves developing a system to monitor live files and folders, extract data, transform it using various techniques, and display insights on a Plotly Dash-based dashboard.

Responsibilities

• • Backend & Frontend Development: Build end-to-end solutions using Python for both backend and frontend functionalities.
  • Data Extraction & Transformation: Implement data cleaning, regex, formatting, and data handling to process extracted information.
  • Database Management: Insert and update records in an Oracle SQL database, ensuring data integrity and efficiency.
  • Live File & Folder Monitoring: Develop Python scripts using Watchdog to monitor logs, detect new files/folders, and extract data in real time. Fetch live data from the database using multithreading for smooth real-time updates.
  • Data Visualization: Develop an interactive dashboard using Plotly Dash or react for real-time data representation.
  • Data Analytics & Pattern Finding: Perform exploratory data analysis (EDA) to identify trends, anomalies, and key insights.
  • Cloud & AI/ML Integration: Leverage AI/ML techniques for data processing.
  • Deployment & Maintenance: Deploy applications on an IIS server/Cloud and ensure system scalability and security.

Qualifications

BE/BTECH degree in Computer Science, EE, or related field.

Essential skills

• Strong Python programming skills
• Experience with Watchdog for real-time monitoring.
• Expertise in Oracle SQL (data insertion, updates, query optimization).
• Knowledge of AI/ML techniques and their practical applications.
• Hands-on experience with Plotly Dash/React/Angular any UI framework for dashboard development.
• Familiarity with IIS deployment and troubleshooting.
• Good understanding of data cleaning, ETL pipelines, and real-time data streaming.
• Strong debugging and problem-solving skills.
• Prior experience working on real-time monitoring systems.
•  

Experience

Year of Experience:  5 - 6 years