Changelog

All notable changes to the Luminara project will be documented in this file.

v0.8.2 (2024-03-01)

Added

• Support for Python 3.12 asyncio improvements.
• New PrometheusMetricsSink for exporting pipeline metrics to Prometheus.
• Added pipeline.settings.shutdown_timeout configuration option.

Fixed

• Fixed a memory leak in the RedisSource when connection drops occurred.
• Corrected type hints for Transform.process method signature.

v0.8.0 (2024-01-15)

Changed

• BREAKING: Renamed Worker class to Stage for clarity.
• Default buffer size increased from 100 to 1000 items.

Added

• Introduced YAML configuration support (luminara.load_config()).
• New CLI tool luminara run for executing YAML pipelines.

v0.7.5 (2023-11-20)

Added

• Integration with AWS Kinesis (source and sink).
• Implemented Schema validation using Pydantic V2.

Fixed

• Race condition in graceful shutdown logic causing some messages to be lost.

v0.7.0 (2023-09-05)

Added

• Added support for backpressure management. Sinks can now signal sources to slow down.
• New BatchTransform base class for vectorized processing.

v0.6.1 (2023-07-12)

Fixed

• Fixed JSON serialization issue with datetime objects in standard sink.

v0.6.0 (2023-06-30)

Added

• Initial public release of the plugin system.
• Documentation site launch.

Changed

• Refactored internal event loop handling to be more robust.

v0.5.0 (2023-04-10)

Added

• Basic Kafka support (Source and Sink).
• Added Filter and Map standard transforms.

v0.1.0 (2023-01-01)

• Initial proof of concept release.
• Basic in-memory queue support.