How to perform experiments with damast

How to perform experiments with damast#

One of the main motivation of this library is to facilitate the development and evaluation of Machine-Learning models. Hence, ‘damast’ offers a mini-framework and API to simplify the development of machine learning models. This requires a ‘hopefully’ minimal set of constraints - as what is envisioned by the authors of this library - so that researchers and ML-starters have lower entry barrier into running machine learning.

That being said, we give an example here on a minimal experiment.

!pip install damast

Requirement already satisfied: damast in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (0.1.2)
Requirement already satisfied: astropy in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (6.1.7)
Requirement already satisfied: cloudpickle in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (3.1.1)
Requirement already satisfied: keras>=3.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (3.9.0)
Requirement already satisfied: matplotlib in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (3.10.1)
Requirement already satisfied: numba in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (0.61.0)
Requirement already satisfied: numpy in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (2.1.3)
Requirement already satisfied: polars in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (1.24.0)
Requirement already satisfied: psutil in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (7.0.0)
Requirement already satisfied: pyais in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (2.9.0)
Requirement already satisfied: pyarrow in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (19.0.1)
Requirement already satisfied: pydantic in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (2.10.6)
Requirement already satisfied: scikit-learn in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (1.6.1)
Requirement already satisfied: tables in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (3.10.1)
Requirement already satisfied: jax[cpu] in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (0.5.2)
Requirement already satisfied: tf-nightly in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (2.20.0.dev20250310)
Requirement already satisfied: torch in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast) (2.6.0)
Requirement already satisfied: absl-py in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from keras>=3.0->damast) (2.1.0)
Requirement already satisfied: rich in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from keras>=3.0->damast) (13.9.4)
Requirement already satisfied: namex in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from keras>=3.0->damast) (0.0.8)
Requirement already satisfied: h5py in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from keras>=3.0->damast) (3.13.0)
Requirement already satisfied: optree in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from keras>=3.0->damast) (0.14.1)
Requirement already satisfied: ml-dtypes in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from keras>=3.0->damast) (0.5.1)
Requirement already satisfied: packaging in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from keras>=3.0->damast) (24.2)
Requirement already satisfied: pyerfa>=2.0.1.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from astropy->damast) (2.0.1.5)
Requirement already satisfied: astropy-iers-data>=0.2024.10.28.0.34.7 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from astropy->damast) (0.2025.3.10.0.29.26)
Requirement already satisfied: PyYAML>=3.13 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from astropy->damast) (6.0.2)
Requirement already satisfied: jaxlib<=0.5.2,>=0.5.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from jax[cpu]->damast) (0.5.1)
Requirement already satisfied: opt_einsum in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from jax[cpu]->damast) (3.4.0)
Requirement already satisfied: scipy>=1.11.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from jax[cpu]->damast) (1.15.2)

Requirement already satisfied: contourpy>=1.0.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast) (1.3.1)
Requirement already satisfied: cycler>=0.10 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast) (4.56.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast) (1.4.8)
Requirement already satisfied: pillow>=8 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast) (11.1.0)
Requirement already satisfied: pyparsing>=2.3.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast) (3.2.1)
Requirement already satisfied: python-dateutil>=2.7 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast) (2.9.0.post0)
Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from numba->damast) (0.44.0)
Requirement already satisfied: bitarray in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pyais->damast) (3.1.1)
Requirement already satisfied: attrs in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pyais->damast) (25.1.0)
Requirement already satisfied: annotated-types>=0.6.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pydantic->damast) (0.7.0)
Requirement already satisfied: pydantic-core==2.27.2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pydantic->damast) (2.27.2)
Requirement already satisfied: typing-extensions>=4.12.2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pydantic->damast) (4.12.2)

Requirement already satisfied: joblib>=1.2.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from scikit-learn->damast) (1.4.2)
Requirement already satisfied: threadpoolctl>=3.1.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from scikit-learn->damast) (3.5.0)
Requirement already satisfied: numexpr>=2.6.2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tables->damast) (2.10.2)
Requirement already satisfied: py-cpuinfo in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tables->damast) (9.0.0)
Requirement already satisfied: blosc2>=2.3.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tables->damast) (2.7.1)
Requirement already satisfied: astunparse>=1.6.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (1.6.3)
Requirement already satisfied: flatbuffers>=24.3.25 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (25.2.10)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (0.6.0)
Requirement already satisfied: google-pasta>=0.1.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (0.2.0)
Requirement already satisfied: libclang>=13.0.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (18.1.1)
Requirement already satisfied: protobuf<6.0.0dev,>=4.21.6 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (5.29.3)
Requirement already satisfied: requests<3,>=2.21.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (2.32.3)
Requirement already satisfied: setuptools in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (75.8.0)
Requirement already satisfied: six>=1.12.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (1.17.0)
Requirement already satisfied: termcolor>=1.1.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (2.5.0)
Requirement already satisfied: wrapt>=1.11.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (1.17.2)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (1.71.0)
Requirement already satisfied: tb-nightly~=2.19.0.a in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (2.19.0a20250218)
Requirement already satisfied: keras-nightly>=3.6.0.dev in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tf-nightly->damast) (3.9.0.dev2025031103)
Requirement already satisfied: filelock in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (3.17.0)
Requirement already satisfied: networkx in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (3.4.2)
Requirement already satisfied: jinja2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (3.1.6)
Requirement already satisfied: fsspec in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (2025.3.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (12.4.127)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (12.4.127)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (12.4.127)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (12.4.5.8)
Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (11.2.1.3)
Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (10.3.5.147)
Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (11.6.1.9)
Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (12.3.1.170)
Requirement already satisfied: nvidia-cusparselt-cu12==0.6.2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (0.6.2)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (2.21.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (12.4.127)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (12.4.127)
Requirement already satisfied: triton==3.2.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (3.2.0)
Requirement already satisfied: sympy==1.13.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast) (1.13.1)

Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from sympy==1.13.1->torch->damast) (1.3.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from astunparse>=1.6.0->tf-nightly->damast) (0.45.1)
Requirement already satisfied: ndindex>=1.4 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from blosc2>=2.3.0->tables->damast) (1.9.2)
Requirement already satisfied: msgpack in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from blosc2>=2.3.0->tables->damast) (1.1.0)

Requirement already satisfied: charset-normalizer<4,>=2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from requests<3,>=2.21.0->tf-nightly->damast) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from requests<3,>=2.21.0->tf-nightly->damast) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from requests<3,>=2.21.0->tf-nightly->damast) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from requests<3,>=2.21.0->tf-nightly->damast) (2025.1.31)

Requirement already satisfied: markdown>=2.6.8 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tb-nightly~=2.19.0.a->tf-nightly->damast) (3.7)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tb-nightly~=2.19.0.a->tf-nightly->damast) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tb-nightly~=2.19.0.a->tf-nightly->damast) (3.1.3)

Requirement already satisfied: MarkupSafe>=2.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from jinja2->torch->damast) (3.0.2)

Requirement already satisfied: markdown-it-py>=2.2.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from rich->keras>=3.0->damast) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from rich->keras>=3.0->damast) (2.19.1)
Requirement already satisfied: mdurl~=0.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.0->damast) (0.1.2)

# The list of modules used for this example
from collections import OrderedDict
from typing import List, Optional
from pathlib import Path

# For performance reasons the underlying data handling library is 'polars'
import polars

# The current development has focused on a keras+tensorflow based Machine Learning setup
import tensorflow as tf
import keras

import damast
from damast.core.transformations import CycleTransformer
# You can define custom units to annotate data, but otherwise astropy units will be used
from damast.core.units import units
# Data ranges can be defined as list, or marked with a lower-bound (min), upper-bound (max)
from damast.core.datarange import MinMax, CyclicMinMax
# The AnnotatedDataFrame combines a data specification and actual 'numeric' data
from damast.core.dataframe import AnnotatedDataFrame
# An AnnotatedDataFrame contains MetaData to describe the data
from damast.core.metadata import MetaData

# Data processing is centered around a DataProcessingPipeline which consists of multiple PipelineElement being run
# in sequence
from damast.core.dataprocessing import DataProcessingPipeline, PipelineElement


# To allow the machine learning process to be simplified, we offer a 'BaseModel' that should be inherited from
from damast.ml.models.base import BaseModel

# The experiment setup
from damast.ml.experiments import Experiment, LearningTask, ForecastTask, ModelInstanceDescription, TrainingParameters

# Allow to generate data for this particular example that uses data from the maritime domain
from damast.domains.maritime.ais.data_generator import AISTestData, AISTestDataSpec

2025-03-11 14:43:01.109476: E external/local_xla/xla/stream_executor/cuda/cuda_platform.cc:51] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: UNKNOWN ERROR (303)

To illustrate a full experiment, we require a data processing pipeline to be set up. This pipeline will extract all those features, that are necessary to train the Machine Learning model(s). The pipeline will run transformations on the data, as provided here by a LatLonTransformer.

class LatLonTransformer(PipelineElement):
    """
    The LatLonTransformer will consume a lat(itude) and a lon(gitude) column and perform
    cyclic normalization. It will add four columns to a dataframe, namely lat_x, lat_y, lon_x, lon_y.
    """
    @damast.core.describe("Lat/Lon cyclic transformation")
    @damast.core.input({
        "lat": {"unit": units.deg},
        "lon": {"unit": units.deg}
    })
    @damast.core.output({
        "lat_x": {"value_range": MinMax(-1.0, 1.0)},
        "lat_y": {"value_range": MinMax(-1.0, 1.0)},
        "lon_x": {"value_range": MinMax(-1.0, 1.0)},
        "lon_y": {"value_range": MinMax(-1.0, 1.0)}
    })
    def transform(self, df: AnnotatedDataFrame) -> AnnotatedDataFrame:        
        lat_cyclic_transformer = CycleTransformer(features=["lat"], n=180.0)
        lon_cyclic_transformer = CycleTransformer(features=["lon"], n=360.0)

        _df = lat_cyclic_transformer.fit_transform(df=df)
        _df = lon_cyclic_transformer.fit_transform(df=_df)
        df._dataframe = _df
        return df

The selected example model here, will require the above listed features as input - and provide a likewise-shaped output (for illustration purposes).

class Baseline(BaseModel):
    """
    This is a placeholder ML model that illustrates the minimal
    requirements.
    """
    input_specs = OrderedDict({
        "lat_x": {"length": 1},
        "lat_y": {"length": 1},
        "lon_x": {"length": 1},
        "lon_y": {"length": 1}
    })

    output_specs = OrderedDict({
        "lat_x": {"length": 1},
        "lat_y": {"length": 1},
        "lon_x": {"length": 1},
        "lon_y": {"length": 1}
    })

    def __init__(self,
                 name: str,
                 features: List[str],
                 timeline_length: int,
                 output_dir: Path,
                 targets: Optional[List[str]] = None):
        self.timeline_length = timeline_length

        super().__init__(name=name,
                         output_dir=output_dir,
                         features=features,
                         targets=targets)

    def _init_model(self):
        features_width = len(self.features)
        targets_width = len(self.targets)

        self.model = tf.keras.models.Sequential([
            keras.layers.Flatten(input_shape=[self.timeline_length, features_width]),
            keras.layers.Dense(targets_width)
        ])


class BaselineA(Baseline):
    """Placeholder Model to illustrate the use of multiple models"""
    pass


class BaselineB(Baseline):
    """Placeholder Model to illustrate the use of multiple models"""
    pass

This example operates with synthetic, i.e. automatically generated data which is specific to the maritime domain. You will see a previous of the first 10 columns when running the following cell.

import tempfile
import shutil

tmp_path = Path(tempfile.gettempdir()) / "test-output-ais_preparation"
if tmp_path.exists():
    shutil.rmtree(tmp_path)
tmp_path.mkdir(parents=True)

pipeline = DataProcessingPipeline(name="ais_preparation",
                                  base_dir=tmp_path) \
    .add("cyclic", LatLonTransformer())
features = ["lat_x", "lat_y", "lon_x", "lon_y"]

data = AISTestData(1000)
adf = AnnotatedDataFrame(dataframe=data.dataframe,
                         metadata=MetaData.from_dict(data=AISTestDataSpec.copy()))
dataset_filename = tmp_path / "test.hdf5"
adf.save(filename=dataset_filename)

adf.head(10)

/home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages/damast/core/polars_dataframe.py:256: PerformanceWarning: 
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->block2_values] [items->Index(['date_time_utc', 'source'], dtype='object')]

  pandas_df.to_hdf(path, key=DAMAST_HDF5_ROOT)
naive plan: (run LazyFrame.explain(optimized=True) to see the optimized plan)

SLICE[offset: 0, len: 10]

DF ["mmsi", "lon", "lat", "date_time_utc", ...]; PROJECT */11 COLUMNS

A central idea to the experiment framework lies in providing a means for a consistent input and output to perform experiments. Hence, define a LearningTask (here: ForecastTask) that collects the learning parameters that define this task.

forecast_task = ForecastTask(
    label="forecast-ais-short-sequence",
    pipeline=pipeline, features=features,
    models=[ModelInstanceDescription(BaselineA, {}),
            ModelInstanceDescription(BaselineB, {}),
            ],
    group_column="mmsi",
    sequence_length=5,
    forecast_length=1,
    training_parameters=TrainingParameters(epochs=1,
                                           validation_steps=1)
)

The actual experimentation takes a single LearningTask as input and it will output

experiment = Experiment(learning_task=forecast_task,
                        input_data=dataset_filename,
                        output_directory=tmp_path)
report = experiment.run()
    
with open(report, "r") as f:
    print(f.read())        

INFO:damast:Recommended steps_per_epoch=np.float64(52508.0)

INFO:damast:Recommended steps_per_epoch=np.float64(52508.0)

You must install graphviz (see instructions at https://graphviz.gitlab.io/download/) for `plot_model` to work.

/home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages/keras/src/layers/reshaping/flatten.py:37: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.
  super().__init__(**kwargs)

Model: "sequential"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ flatten (Flatten)               │ (None, 20)             │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense (Dense)                   │ (None, 4)              │            84 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 84 (336.00 B)

 Trainable params: 84 (336.00 B)

 Non-trainable params: 0 (0.00 B)

 1/10 ━━━━━━━━━━━━━━━━━━━━ 4s 503ms/step - loss: 3.4215e-05 - mse: 3.4215e-05


 2/10 ━━━━━━━━━━━━━━━━━━━━ 1s 233ms/step - loss: 3.0563e-05 - mse: 3.0563e-05


 3/10 ━━━━━━━━━━━━━━━━━━━━ 1s 240ms/step - loss: 2.8373e-05 - mse: 2.8373e-05


 4/10 ━━━━━━━━━━━━━━━━━━━━ 1s 242ms/step - loss: 2.7004e-05 - mse: 2.7004e-05


 5/10 ━━━━━━━━━━━━━━━━━━━━ 1s 243ms/step - loss: 2.5822e-05 - mse: 2.5822e-05


 6/10 ━━━━━━━━━━━━━━━━━━━━ 0s 244ms/step - loss: 2.4792e-05 - mse: 2.4792e-05


 7/10 ━━━━━━━━━━━━━━━━━━━━ 0s 244ms/step - loss: 2.3818e-05 - mse: 2.3818e-05


 8/10 ━━━━━━━━━━━━━━━━━━━━ 0s 246ms/step - loss: 2.2944e-05 - mse: 2.2944e-05


 9/10 ━━━━━━━━━━━━━━━━━━━━ 0s 247ms/step - loss: 2.2387e-05 - mse: 2.2387e-05


10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 247ms/step - loss: 2.1980e-05 - mse: 2.1980e-05


10/10 ━━━━━━━━━━━━━━━━━━━━ 3s 270ms/step - loss: 2.1647e-05 - mse: 2.1647e-05 - val_loss: 2.3732e-05 - val_mse: 2.3732e-05

/home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages/keras/src/backend/tensorflow/core.py:158: DeprecationWarning: __array__ implementation doesn't accept a copy keyword, so passing copy=False failed. __array__ must implement 'dtype' and 'copy' keyword arguments.
  return np.array(x)
WARNING:absl:You are saving your model as an HDF5 file via `model.save()` or `keras.saving.save_model(model)`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')` or `keras.saving.save_model(model, 'my_model.keras')`. 

INFO:damast:Recommended steps_per_epoch=np.float64(52508.0)

INFO:damast:Recommended steps_per_epoch=np.float64(52508.0)

You must install graphviz (see instructions at https://graphviz.gitlab.io/download/) for `plot_model` to work.

/home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages/keras/src/layers/reshaping/flatten.py:37: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.
  super().__init__(**kwargs)

Model: "sequential_1"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ flatten_1 (Flatten)             │ (None, 20)             │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 4)              │            84 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 84 (336.00 B)

 Trainable params: 84 (336.00 B)

 Non-trainable params: 0 (0.00 B)

 1/10 ━━━━━━━━━━━━━━━━━━━━ 4s 456ms/step - loss: 3.9101e-05 - mse: 3.9101e-05


 2/10 ━━━━━━━━━━━━━━━━━━━━ 1s 238ms/step - loss: 3.5758e-05 - mse: 3.5758e-05


 3/10 ━━━━━━━━━━━━━━━━━━━━ 1s 245ms/step - loss: 3.5500e-05 - mse: 3.5500e-05


 4/10 ━━━━━━━━━━━━━━━━━━━━ 1s 250ms/step - loss: 3.4331e-05 - mse: 3.4331e-05


 5/10 ━━━━━━━━━━━━━━━━━━━━ 1s 249ms/step - loss: 3.2847e-05 - mse: 3.2847e-05


 6/10 ━━━━━━━━━━━━━━━━━━━━ 0s 249ms/step - loss: 3.2257e-05 - mse: 3.2257e-05


 7/10 ━━━━━━━━━━━━━━━━━━━━ 0s 251ms/step - loss: 3.1759e-05 - mse: 3.1759e-05


 8/10 ━━━━━━━━━━━━━━━━━━━━ 0s 252ms/step - loss: 3.1418e-05 - mse: 3.1418e-05


 9/10 ━━━━━━━━━━━━━━━━━━━━ 0s 252ms/step - loss: 3.1029e-05 - mse: 3.1029e-05


10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 253ms/step - loss: 3.0635e-05 - mse: 3.0635e-05


10/10 ━━━━━━━━━━━━━━━━━━━━ 3s 272ms/step - loss: 3.0313e-05 - mse: 3.0313e-05 - val_loss: 3.3552e-05 - val_mse: 3.3552e-05

/home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages/keras/src/backend/tensorflow/core.py:158: DeprecationWarning: __array__ implementation doesn't accept a copy keyword, so passing copy=False failed. __array__ must implement 'dtype' and 'copy' keyword arguments.
  return np.array(x)
WARNING:absl:You are saving your model as an HDF5 file via `model.save()` or `keras.saving.save_model(model)`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')` or `keras.saving.save_model(model, 'my_model.keras')`. 

INFO:damast:Recommended steps_per_epoch=np.float64(52508.0)

batch_size: 2
evaluation:
  BaselineA-forecast-ais-short-sequence:
    loss: 2.810329715430271e-05
    mse: 2.810329715430271e-05
  BaselineB-forecast-ais-short-sequence:
    loss: 3.273458787589334e-05
    mse: 3.2734591513872147e-05
evaluation_steps: 1
input_data: /tmp/test-output-ais_preparation/test.hdf5
label: damast-ml-experiment
learning_task:
  class_name: ForecastTask
  features: &id001
  - lat_x
  - lat_y
  - lon_x
  - lon_y
  forecast_length: 1
  group_column: mmsi
  label: forecast-ais-short-sequence
  models:
  - class_name: BaselineA
    module_name: __main__
    parameters: {}
  - class_name: BaselineB
    module_name: __main__
    parameters: {}
  module_name: damast.ml.experiments
  pipeline:
    base_dir: /tmp/test-output-ais_preparation
    name: ais_preparation
    steps:
    - - cyclic
      - class_name: LatLonTransformer
        module_name: __main__
        name_mappings: {}
  sequence_length: 5
  targets: *id001
  training_parameters:
    epochs: 1
    learning_rate: 0.1
    loss_function: mse
    steps_per_epoch: 10
    validation_steps: 1
output_directory: /tmp/test-output-ais_preparation
split_data_ratios:
- 1.6
- 0.2
- 0.2
timestamp: 20250311-144310

The outputs of an experiment are collected inside a dedicated (timestamped) folder. This folder will also contain a subfolder for each of the parametrized models that defines a LearningTask.

last_experiments = sorted([str(f) for f in Path(experiment.output_directory).glob(pattern="*") if f.is_dir()])
print("Last experiment in: ", last_experiments[-1])

experiment_folder = sorted([str(f) for f in Path(last_experiments[-1]).glob(pattern="*")])
file_listing = '\n'.join(experiment_folder)
print("Contents:\n")
print(file_listing)

Last experiment in:  /tmp/test-output-ais_preparation/20250311-144303-damast-ml-experiment
Contents:

/tmp/test-output-ais_preparation/20250311-144303-damast-ml-experiment/.damast_experiment
/tmp/test-output-ais_preparation/20250311-144303-damast-ml-experiment/BaselineA-forecast-ais-short-sequence
/tmp/test-output-ais_preparation/20250311-144303-damast-ml-experiment/BaselineB-forecast-ais-short-sequence
/tmp/test-output-ais_preparation/20250311-144303-damast-ml-experiment/experiment-report.yaml

Once the training is running it can be monitored using tensorboard:

    tensorboard --logdir=<experiments-directory>