damast.core.dataprocessing

Module containing decorators and classes to model data processing pipelines

Module Attributes

DECORATED_DESCRIPTION	Attribute description for describe()
DECORATED_ARTIFACT_SPECS	Attribute description for artifacts()
DECORATED_INPUT_SPECS	Attribute description for input()
DECORATED_OUTPUT_SPECS	Attribute description for output()
DAMAST_PIPELINE_SUFFIX	Suffix of DataProcessingPipeline files created in DataProcessingPipeline.save() and used by DataProcessingPipeline.load()
VAEX_STATE_SUFFIX	Suffix of DataProcessingPipeline files created in DataProcessingPipeline.save_state() and used by DataProcessingPipeline.load_state()

Functions

artifacts(requirements)	Specify the output for the decorated function.
describe(description)	Specify the description for the transformation for the decorated function.
input(requirements)	Specify the input for the decorated function.
output(requirements)	Specify the output for the decorated function.

Classes

DataProcessingPipeline(*, name[, base_dir, ...])	A data-processing pipeline for a sequence of transformers
PipelineElement()	Allow to get the reference to a parent pipeline