Projects on Data Trenches

Real-time ML Model Serving

leandrojlfernandes@gmail.com (Leandro Fernandes) — Sun, 04 Jan 2026 00:00:00 +0000

The Challenge

Deploying machine learning models to serve real-time inference requests for client-facing applications with strict latency requirements.

The Solution

Built and deployed low-latency inference services using modern microservices architecture:

FastAPI-based REST endpoints
Docker containerization for consistency
Load balancing and auto-scaling
Health monitoring and logging

Technologies Used

FastAPI
Docker
Machine Learning Deployment
API Development

Impact

Real-time model inference capabilities
Low-latency responses for client applications
Scalable architecture handling varying load
Easy model updates and rollbacks
Production-grade reliability

This project showcased the ability to bridge the gap between ML models and production applications, ensuring models could be consumed by real users with minimal latency.

NLP Analytics Engine

leandrojlfernandes@gmail.com (Leandro Fernandes) — Thu, 18 Sep 2025 00:00:00 +0000

The Challenge

Building a production-grade NLP analytics engine capable of processing semantic data from 25,000 daily targets while maintaining high availability and delivering actionable insights to enterprise clients.

The Solution

Designed and implemented an end-to-end pipeline from model training to deployment, including:

Data ingestion and preprocessing pipeline
Model training infrastructure
Inference serving layer
Monitoring and alerting system

Technologies Used

Python
Machine Learning/NLP libraries
Distributed processing
Containerization (Docker)
API development (FastAPI)

Impact

$700k recurring revenue generated from the analytics solution
Processes semantic data from 25,000+ daily targets
Production-grade reliability and performance
Real-time analytics delivery to clients

This project demonstrated the full lifecycle of deploying ML models in production, from data pipeline to client-facing application. The atual output of this project can’t be shared publicly given it was trained with confidential data.

PySpark Infrastructure Optimization

leandrojlfernandes@gmail.com (Leandro Fernandes) — Sat, 17 Feb 2024 00:00:00 +0000

The Challenge

Handling massive-scale data processing while maintaining reasonable query latency and managing compute resource costs in a distributed environment.

The Solution

Architected distributed processing jobs using PySpark with multiple optimization strategies:

Algorithmic improvements to reduce computational complexity
Storage optimization using Trino and Hive
Query execution plan optimization
Resource allocation tuning
Data partitioning strategies

Technologies Used

PySpark
Apache Hadoop
Trino
Hive
Distributed Systems

Impact

25% reduction in query latency
25% decrease in resource consumption
Improved processing efficiency for massive datasets
Significant cost savings on compute resources

This optimization effort required deep understanding of distributed systems, Spark internals, and data storage patterns to achieve measurable performance gains.

nAttrMon Open Source Contribution

leandrojlfernandes@gmail.com (Leandro Fernandes) — Tue, 07 May 2019 00:00:00 +0000

The Challenge

Improve system observability across distributed clusters by developing custom plugins for the nAttrMon monitoring tool.

The Solution

Contributed code and developed custom monitors to detect and visualize real-time system bottlenecks:

Real-time system monitoring
Custom monitoring architecture
Alerting and notification systems
Performance metrics collection

Technologies Used

Java
Open Source Development
System Monitoring
Distributed Systems

Impact

Enhanced observability across distributed clusters
Reduced mean-time-to-resolution (MTTR) for outages
Better system performance insights
Community contribution to open source project
Improved system stability for carrier networks

This open source work demonstrated the ability to understand and contribute to complex systems while providing practical value to the community.