A Python-based project to migrate MySQL data to Elasticsearch and expose it via a FastAPI-powered REST API with filtering capabilities.

This project enables seamless migration of employee data from a MySQL database to Elasticsearch and provides a FastAPI-based interface to access and query that data. The system is designed for performance, simplicity, and flexibility. It supports filtering on employee attributes like name, email, and salary, making it ideal for building search and analytics dashboards. The migration script (mysql-elastic.py) uses SQLAlchemy and Pandas for data processing, and FastAPI handles the API layer. Optional Docker support is included for setting up MySQL, Elasticsearch, and Logstash environments.

• Launch the FastAPI server: uvicorn api:app --reload

• Access API at: http://localhost:8000/employees

Query Parameters:

• name: Filter by first name (prefix match)

• email: Filter by email (exact match)

• salary: Filter by salary (exact match)

Examples:

• /employees?name=John

• /employees?email=john@example.com

• /employees?salary=50000

MySQL to Elasticsearch Data Migration & API

This project demonstrates data migration from a MySQL table to Elasticsearch and exposes a FastAPI endpoint with filtering.

Features

• MySQL to Elasticsearch data transfer
• FastAPI endpoint to query data
• Filters by name, email, and salary
• Docker setup (optional)

Requirements

• Python 3.8+
• Running MySQL instance with employees table
• Elasticsearch 7.x/8.x

Setup

git clone https://openforge.gov.in/plugins/git/smartblocks/elk-mysql-api.git
cd elk-mysql-api
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt

Configure DB connection and run: python mysql-elastic.py
Run API - uvicorn api:app --reload