damast.core.tracking

Trackers that observe what each step of a damast.core.dataprocessing.DataProcessingPipeline does to the data.

A pipeline is given a list of PipelineElementTracker instances and calls each one before and after every step. What a tracker returns at the end of a step is merged into the pipeline’s damast.core.dataprocessing.DataProcessingPipeline.processing_stats(), so a new kind of measurement is a new class here rather than a change to the pipeline.

Not to be confused with damast.integrations.tracking.ExperimentTracker, which reports a finished run to an experiment tracker such as MLflow.

Classes

KeyTracker(keys[, max_removed_keys])	How many distinct entities each step consumed and produced.
PipelineElementTracker()	Observes a single damast.core.transformations.PipelineElement as a pipeline runs it.
RowCountTracker()	How many rows each step consumed and produced.
TimingTracker()	How long each step took.