How to perform experiments with damast#
One of the main motivation of this library is to facilitate the development and evaluation of Machine-Learning models. Hence, ‘damast’ offers a mini-framework and API to simplify the development of machine learning models. This requires a ‘hopefully’ minimal set of constraints - as what is envisioned by the authors of this library - so that researchers and ML-starters have lower entry barrier into running machine learning.
That being said, we give an example here on a minimal experiment.
!pip install 'damast[ml]'
Requirement already satisfied: damast[ml] in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (0.2.3)
Requirement already satisfied: astropy in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (6.1.7)
Requirement already satisfied: cloudpickle in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (3.1.2)
Requirement already satisfied: iso8601 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (2.1.0)
Requirement already satisfied: matplotlib in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (3.10.8)
Requirement already satisfied: networkx in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (3.4.2)
Requirement already satisfied: numba in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (0.63.1)
Requirement already satisfied: numpy>=2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (2.2.6)
Requirement already satisfied: polars>=1.20 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (1.36.1)
Requirement already satisfied: psutil in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (7.2.1)
Requirement already satisfied: pyais in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (2.14.0)
Requirement already satisfied: pyarrow in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (22.0.0)
Requirement already satisfied: pydantic>=2.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (2.12.5)
Requirement already satisfied: ratarmount>=1.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (1.2.1)
Requirement already satisfied: scikit-learn in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (1.7.2)
Requirement already satisfied: tables in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (3.10.1)
Requirement already satisfied: tqdm in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from damast[ml]) (4.67.1)
Collecting keras>=3.0 (from damast[ml])
Downloading keras-3.12.0-py3-none-any.whl.metadata (5.9 kB)
Collecting torch (from damast[ml])
Downloading torch-2.9.1-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (30 kB)
Collecting jax[cpu] (from damast[ml])
Downloading jax-0.6.2-py3-none-any.whl.metadata (13 kB)
Collecting tensorflow (from damast[ml])
Downloading tensorflow-2.20.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.5 kB)
Collecting absl-py (from keras>=3.0->damast[ml])
Downloading absl_py-2.3.1-py3-none-any.whl.metadata (3.3 kB)
Collecting rich (from keras>=3.0->damast[ml])
Downloading rich-14.2.0-py3-none-any.whl.metadata (18 kB)
Collecting namex (from keras>=3.0->damast[ml])
Downloading namex-0.1.0-py3-none-any.whl.metadata (322 bytes)
Collecting h5py (from keras>=3.0->damast[ml])
Downloading h5py-3.15.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (3.0 kB)
Collecting optree (from keras>=3.0->damast[ml])
Downloading optree-0.18.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (34 kB)
Collecting ml-dtypes (from keras>=3.0->damast[ml])
Downloading ml_dtypes-0.5.4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (8.9 kB)
Requirement already satisfied: packaging in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from keras>=3.0->damast[ml]) (25.0)
Requirement already satisfied: polars-runtime-32==1.36.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from polars>=1.20->damast[ml]) (1.36.1)
Requirement already satisfied: annotated-types>=0.6.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pydantic>=2.0->damast[ml]) (0.7.0)
Requirement already satisfied: pydantic-core==2.41.5 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pydantic>=2.0->damast[ml]) (2.41.5)
Requirement already satisfied: typing-extensions>=4.14.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pydantic>=2.0->damast[ml]) (4.15.0)
Requirement already satisfied: typing-inspection>=0.4.2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pydantic>=2.0->damast[ml]) (0.4.2)
Requirement already satisfied: ratarmountcore~=0.10.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (0.10.2)
Requirement already satisfied: mfusepy~=3.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmount>=1.1->damast[ml]) (3.1.0)
Requirement already satisfied: fast_zip_decryption in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (3.0.0)
Requirement already satisfied: rapidgzip~=0.15.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (0.15.2)
Requirement already satisfied: python-xz~=0.4.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (0.4.0)
Requirement already satisfied: rarfile~=4.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (4.2)
Requirement already satisfied: indexed_gzip~=1.7 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (1.10.3)
Requirement already satisfied: indexed_zstd<2.0,>=1.2.2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (1.6.1)
Requirement already satisfied: libarchive-c<6.0,~=5.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (5.3)
Requirement already satisfied: py7zr~=1.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (1.1.0)
Requirement already satisfied: texttable in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from py7zr~=1.0->ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (1.7.0)
Requirement already satisfied: pycryptodomex>=3.20.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from py7zr~=1.0->ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (3.23.0)
Requirement already satisfied: brotli>=1.2.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from py7zr~=1.0->ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (1.2.0)
Requirement already satisfied: backports.zstd>=1.0.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from py7zr~=1.0->ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (1.3.0)
Requirement already satisfied: pyppmd>=1.3.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from py7zr~=1.0->ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (1.3.1)
Requirement already satisfied: pybcj>=1.0.6 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from py7zr~=1.0->ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (1.0.7)
Requirement already satisfied: multivolumefile>=0.2.3 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from py7zr~=1.0->ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (0.2.3)
Requirement already satisfied: inflate64>=1.0.4 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from py7zr~=1.0->ratarmountcore[7z,bzip2,fat,gzip,rar,xz,zip,zstd]~=0.10.0->ratarmount>=1.1->damast[ml]) (1.0.4)
Requirement already satisfied: pyerfa>=2.0.1.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from astropy->damast[ml]) (2.0.1.5)
Requirement already satisfied: astropy-iers-data>=0.2024.10.28.0.34.7 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from astropy->damast[ml]) (0.2026.1.5.0.43.43)
Requirement already satisfied: PyYAML>=3.13 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from astropy->damast[ml]) (6.0.3)
Collecting jaxlib<=0.6.2,>=0.6.2 (from jax[cpu]; extra == "ml"->damast[ml])
Downloading jaxlib-0.6.2-cp310-cp310-manylinux2014_x86_64.whl.metadata (1.3 kB)
Collecting opt_einsum (from jax[cpu]; extra == "ml"->damast[ml])
Downloading opt_einsum-3.4.0-py3-none-any.whl.metadata (6.3 kB)
Requirement already satisfied: scipy>=1.12 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from jax[cpu]; extra == "ml"->damast[ml]) (1.15.3)
Requirement already satisfied: contourpy>=1.0.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast[ml]) (1.3.2)
Requirement already satisfied: cycler>=0.10 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast[ml]) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast[ml]) (4.61.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast[ml]) (1.4.9)
Requirement already satisfied: pillow>=8 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast[ml]) (12.1.0)
Requirement already satisfied: pyparsing>=3 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast[ml]) (3.3.1)
Requirement already satisfied: python-dateutil>=2.7 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from matplotlib->damast[ml]) (2.9.0.post0)
Requirement already satisfied: six>=1.5 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib->damast[ml]) (1.17.0)
Requirement already satisfied: llvmlite<0.47,>=0.46.0dev0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from numba->damast[ml]) (0.46.0)
Requirement already satisfied: bitarray in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pyais->damast[ml]) (3.8.0)
Requirement already satisfied: attrs in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from pyais->damast[ml]) (25.4.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from rich->keras>=3.0->damast[ml]) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from rich->keras>=3.0->damast[ml]) (2.19.2)
Requirement already satisfied: mdurl~=0.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.0->damast[ml]) (0.1.2)
Requirement already satisfied: joblib>=1.2.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from scikit-learn->damast[ml]) (1.5.3)
Requirement already satisfied: threadpoolctl>=3.1.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from scikit-learn->damast[ml]) (3.6.0)
Requirement already satisfied: numexpr>=2.6.2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tables->damast[ml]) (2.14.1)
Requirement already satisfied: py-cpuinfo in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tables->damast[ml]) (9.0.0)
Requirement already satisfied: blosc2>=2.3.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tables->damast[ml]) (3.12.2)
Requirement already satisfied: ndindex in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from blosc2>=2.3.0->tables->damast[ml]) (1.10.1)
Requirement already satisfied: msgpack in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from blosc2>=2.3.0->tables->damast[ml]) (1.1.2)
Requirement already satisfied: platformdirs in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from blosc2>=2.3.0->tables->damast[ml]) (4.5.1)
Requirement already satisfied: requests in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from blosc2>=2.3.0->tables->damast[ml]) (2.32.5)
Requirement already satisfied: charset_normalizer<4,>=2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from requests->blosc2>=2.3.0->tables->damast[ml]) (3.4.4)
Requirement already satisfied: idna<4,>=2.5 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from requests->blosc2>=2.3.0->tables->damast[ml]) (3.11)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from requests->blosc2>=2.3.0->tables->damast[ml]) (2.6.2)
Requirement already satisfied: certifi>=2017.4.17 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from requests->blosc2>=2.3.0->tables->damast[ml]) (2026.1.4)
Collecting astunparse>=1.6.0 (from tensorflow->damast[ml])
Downloading astunparse-1.6.3-py2.py3-none-any.whl.metadata (4.4 kB)
Collecting flatbuffers>=24.3.25 (from tensorflow->damast[ml])
Downloading flatbuffers-25.12.19-py2.py3-none-any.whl.metadata (1.0 kB)
Collecting gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 (from tensorflow->damast[ml])
Downloading gast-0.7.0-py3-none-any.whl.metadata (1.5 kB)
Collecting google_pasta>=0.1.1 (from tensorflow->damast[ml])
Downloading google_pasta-0.2.0-py3-none-any.whl.metadata (814 bytes)
Collecting libclang>=13.0.0 (from tensorflow->damast[ml])
Downloading libclang-18.1.1-py2.py3-none-manylinux2010_x86_64.whl.metadata (5.2 kB)
Collecting protobuf>=5.28.0 (from tensorflow->damast[ml])
Downloading protobuf-6.33.2-cp39-abi3-manylinux2014_x86_64.whl.metadata (593 bytes)
Requirement already satisfied: setuptools in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tensorflow->damast[ml]) (80.9.0)
Collecting termcolor>=1.1.0 (from tensorflow->damast[ml])
Downloading termcolor-3.3.0-py3-none-any.whl.metadata (6.5 kB)
Requirement already satisfied: wrapt>=1.11.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from tensorflow->damast[ml]) (1.17.3)
Collecting grpcio<2.0,>=1.24.3 (from tensorflow->damast[ml])
Downloading grpcio-1.76.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (3.7 kB)
Collecting tensorboard~=2.20.0 (from tensorflow->damast[ml])
Downloading tensorboard-2.20.0-py3-none-any.whl.metadata (1.8 kB)
Collecting markdown>=2.6.8 (from tensorboard~=2.20.0->tensorflow->damast[ml])
Downloading markdown-3.10-py3-none-any.whl.metadata (5.1 kB)
Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard~=2.20.0->tensorflow->damast[ml])
Downloading tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl.metadata (1.1 kB)
Collecting werkzeug>=1.0.1 (from tensorboard~=2.20.0->tensorflow->damast[ml])
Downloading werkzeug-3.1.4-py3-none-any.whl.metadata (4.0 kB)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from astunparse>=1.6.0->tensorflow->damast[ml]) (0.45.1)
Requirement already satisfied: markupsafe>=2.1.1 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from werkzeug>=1.0.1->tensorboard~=2.20.0->tensorflow->damast[ml]) (3.0.3)
Requirement already satisfied: filelock in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast[ml]) (3.20.2)
Collecting sympy>=1.13.3 (from torch->damast[ml])
Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
Requirement already satisfied: jinja2 in /home/runner/work/damast/damast/.tox/build_docs/lib/python3.10/site-packages (from torch->damast[ml]) (3.1.6)
Collecting fsspec>=0.8.5 (from torch->damast[ml])
Downloading fsspec-2025.12.0-py3-none-any.whl.metadata (10 kB)
Collecting nvidia-cuda-nvrtc-cu12==12.8.93 (from torch->damast[ml])
Downloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cuda-runtime-cu12==12.8.90 (from torch->damast[ml])
Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cuda-cupti-cu12==12.8.90 (from torch->damast[ml])
Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cudnn-cu12==9.10.2.21 (from torch->damast[ml])
Downloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-cublas-cu12==12.8.4.1 (from torch->damast[ml])
Downloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cufft-cu12==11.3.3.83 (from torch->damast[ml])
Downloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-curand-cu12==10.3.9.90 (from torch->damast[ml])
Downloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cusolver-cu12==11.7.3.90 (from torch->damast[ml])
Downloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-cusparse-cu12==12.5.8.93 (from torch->damast[ml])
Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-cusparselt-cu12==0.7.1 (from torch->damast[ml])
Downloading nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (7.0 kB)
Collecting nvidia-nccl-cu12==2.27.5 (from torch->damast[ml])
Downloading nvidia_nccl_cu12-2.27.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.0 kB)
Collecting nvidia-nvshmem-cu12==3.3.20 (from torch->damast[ml])
Downloading nvidia_nvshmem_cu12-3.3.20-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.1 kB)
Collecting nvidia-nvtx-cu12==12.8.90 (from torch->damast[ml])
Downloading nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-nvjitlink-cu12==12.8.93 (from torch->damast[ml])
Downloading nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
Collecting nvidia-cufile-cu12==1.13.1.3 (from torch->damast[ml])
Downloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
Collecting triton==3.5.1 (from torch->damast[ml])
Downloading triton-3.5.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.7 kB)
Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch->damast[ml])
Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Downloading keras-3.12.0-py3-none-any.whl (1.5 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/1.5 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 13.6 MB/s 0:00:00
?25hDownloading absl_py-2.3.1-py3-none-any.whl (135 kB)
Downloading h5py-3.15.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (4.7 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/4.7 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 37.7 MB/s 0:00:00
?25h
Downloading jax-0.6.2-py3-none-any.whl (2.7 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/2.7 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.7/2.7 MB 63.2 MB/s 0:00:00
?25h
Downloading jaxlib-0.6.2-cp310-cp310-manylinux2014_x86_64.whl (89.9 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/89.9 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━ 47.2/89.9 MB 236.2 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 89.9/89.9 MB 279.0 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.9/89.9 MB 211.6 MB/s 0:00:00
?25hDownloading ml_dtypes-0.5.4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (5.0 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/5.0 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.0/5.0 MB 233.5 MB/s 0:00:00
?25hDownloading namex-0.1.0-py3-none-any.whl (5.9 kB)
Downloading opt_einsum-3.4.0-py3-none-any.whl (71 kB)
Downloading optree-0.18.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (386 kB)
Downloading rich-14.2.0-py3-none-any.whl (243 kB)
Downloading tensorflow-2.20.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (620.4 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/620.4 MB ? eta -:--:--
━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.7/620.4 MB 318.4 MB/s eta 0:00:02
━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.8/620.4 MB 325.5 MB/s eta 0:00:02
━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.3/620.4 MB 324.6 MB/s eta 0:00:02
━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━ 261.6/620.4 MB 324.0 MB/s eta 0:00:02
━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━ 326.9/620.4 MB 325.3 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━ 396.4/620.4 MB 327.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━ 463.5/620.4 MB 329.3 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━ 534.5/620.4 MB 337.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺ 604.8/620.4 MB 343.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 620.2/620.4 MB 343.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 620.4/620.4 MB 86.0 MB/s 0:00:04
?25h
Downloading grpcio-1.76.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (6.6 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/6.6 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 232.8 MB/s 0:00:00
?25hDownloading tensorboard-2.20.0-py3-none-any.whl (5.5 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/5.5 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.5/5.5 MB 234.0 MB/s 0:00:00
?25hDownloading tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl (6.6 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/6.6 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 246.3 MB/s 0:00:00
?25hDownloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Downloading flatbuffers-25.12.19-py2.py3-none-any.whl (26 kB)
Downloading gast-0.7.0-py3-none-any.whl (22 kB)
Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
Downloading libclang-18.1.1-py2.py3-none-manylinux2010_x86_64.whl (24.5 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/24.5 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.5/24.5 MB 288.0 MB/s 0:00:00
?25hDownloading markdown-3.10-py3-none-any.whl (107 kB)
Downloading protobuf-6.33.2-cp39-abi3-manylinux2014_x86_64.whl (323 kB)
Downloading termcolor-3.3.0-py3-none-any.whl (7.7 kB)
Downloading werkzeug-3.1.4-py3-none-any.whl (224 kB)
Downloading torch-2.9.1-cp310-cp310-manylinux_2_28_x86_64.whl (899.8 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/899.8 MB ? eta -:--:--
━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.4/899.8 MB 338.2 MB/s eta 0:00:03
━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 142.6/899.8 MB 355.8 MB/s eta 0:00:03
━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.0/899.8 MB 385.0 MB/s eta 0:00:02
━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━ 322.4/899.8 MB 419.0 MB/s eta 0:00:02
━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━ 413.1/899.8 MB 446.2 MB/s eta 0:00:02
━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━ 500.7/899.8 MB 442.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━ 589.8/899.8 MB 440.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━ 679.5/899.8 MB 439.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━ 769.4/899.8 MB 442.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━ 860.9/899.8 MB 445.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 899.7/899.8 MB 443.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 899.8/899.8 MB 49.2 MB/s 0:00:06
?25hDownloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl (594.3 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/594.3 MB ? eta -:--:--
━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.0/594.3 MB 327.0 MB/s eta 0:00:02
━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 153.9/594.3 MB 383.8 MB/s eta 0:00:02
━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━ 246.2/594.3 MB 408.3 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━ 336.1/594.3 MB 453.6 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━ 427.0/594.3 MB 451.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━ 517.7/594.3 MB 449.3 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 594.3/594.3 MB 450.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 594.3/594.3 MB 64.2 MB/s 0:00:04
?25h
Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (10.2 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/10.2 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.2/10.2 MB 221.9 MB/s 0:00:00
?25hDownloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (88.0 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/88.0 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━ 75.5/88.0 MB 378.6 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.0/88.0 MB 262.7 MB/s 0:00:00
?25h
Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (954 kB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/954.8 kB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 954.8/954.8 kB 136.4 MB/s 0:00:00
?25hDownloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl (706.8 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/706.8 MB ? eta -:--:--
━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 76.0/706.8 MB 381.2 MB/s eta 0:00:02
━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 153.4/706.8 MB 382.3 MB/s eta 0:00:02
━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━ 228.6/706.8 MB 379.0 MB/s eta 0:00:02
━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━ 304.9/706.8 MB 377.3 MB/s eta 0:00:02
━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━ 382.2/706.8 MB 379.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━ 460.3/706.8 MB 380.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━ 530.1/706.8 MB 371.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━ 601.4/706.8 MB 365.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━ 666.6/706.8 MB 348.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 706.7/706.8 MB 345.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 706.8/706.8 MB 59.3 MB/s 0:00:05
?25hDownloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (193.1 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/193.1 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━ 88.9/193.1 MB 445.0 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━ 178.3/193.1 MB 443.3 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 192.9/193.1 MB 441.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 193.1/193.1 MB 244.8 MB/s 0:00:00
?25hDownloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.2 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/1.2 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 156.8 MB/s 0:00:00
?25hDownloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl (63.6 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/63.6 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 63.4/63.6 MB 435.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.6/63.6 MB 287.3 MB/s 0:00:00
?25hDownloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl (267.5 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/267.5 MB ? eta -:--:--
━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.2/267.5 MB 431.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━ 169.1/267.5 MB 421.2 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━ 256.4/267.5 MB 424.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 267.4/267.5 MB 419.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 267.4/267.5 MB 419.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 267.4/267.5 MB 419.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 267.5/267.5 MB 211.2 MB/s 0:00:01
?25h
Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (288.2 MB)
?25l ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/288.2 MB ? eta -:--:--
━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.4/288.2 MB 448.2 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━ 178.5/288.2 MB 440.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━ 253.2/288.2 MB 416.6 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 288.1/288.2 MB 404.6 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 288.1/288.2 MB 404.6 MB/s eta 0:00:01^C
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 288.1/288.2 MB 404.6 MB/s eta 0:00:01
# The list of modules used for this example
from collections import OrderedDict
from typing import List, Optional
from pathlib import Path
# For performance reasons the underlying data handling library is 'polars'
# The current development has focused on a keras+tensorflow based Machine Learning setup
import tensorflow as tf
import keras
from damast.data_handling.transformers.cycle_transformer import CycleTransformer
# You can define custom units to annotate data, but otherwise astropy units will be used
# Data ranges can be defined as list, or marked with a lower-bound (min), upper-bound (max)
# The AnnotatedDataFrame combines a data specification and actual 'numeric' data
from damast.core.dataframe import AnnotatedDataFrame
# An AnnotatedDataFrame contains MetaData to describe the data
from damast.core.metadata import MetaData
# Data processing is centered around a DataProcessingPipeline which consists of multiple PipelineElement being run
# in sequence
from damast.core.dataprocessing import DataProcessingPipeline
# To allow the machine learning process to be simplified, we offer a 'BaseModel' that should be inherited from
from damast.ml.models.base import BaseModel
# The experiment setup
from damast.ml.experiments import Experiment, ForecastTask, ModelInstanceDescription, TrainingParameters
# Allow to generate data for this particular example that uses data from the maritime domain
from damast.domains.maritime.ais.data_generator import AISTestData, AISTestDataSpec
To illustrate a full experiment, we require a data processing pipeline to be set up. This pipeline will extract all those features, that are necessary to train the Machine Learning model(s). The pipeline will run transformations on the data, as provided here by a LatLonTransformer.
The selected example model here, will require the above listed features as input - and provide a likewise-shaped output (for illustration purposes).
class Baseline(BaseModel):
"""
This is a placeholder ML model that illustrates the minimal
requirements.
"""
input_specs = OrderedDict({
"lat_x": {"length": 1},
"lat_y": {"length": 1},
"lon_x": {"length": 1},
"lon_y": {"length": 1}
})
output_specs = OrderedDict({
"lat_x": {"length": 1},
"lat_y": {"length": 1},
"lon_x": {"length": 1},
"lon_y": {"length": 1}
})
def __init__(self,
name: str,
features: List[str],
timeline_length: int,
output_dir: Path,
targets: Optional[List[str]] = None):
self.timeline_length = timeline_length
super().__init__(name=name,
output_dir=output_dir,
features=features,
targets=targets)
def _init_model(self):
features_width = len(self.features)
targets_width = len(self.targets)
self.model = tf.keras.models.Sequential([
keras.layers.Flatten(input_shape=[self.timeline_length, features_width]),
keras.layers.Dense(targets_width)
])
class BaselineA(Baseline):
"""Placeholder Model to illustrate the use of multiple models"""
pass
class BaselineB(Baseline):
"""Placeholder Model to illustrate the use of multiple models"""
pass
This example operates with synthetic, i.e. automatically generated data which is specific to the maritime domain. You will see a previous of the first 10 columns when running the following cell.
import tempfile
import shutil
tmp_path = Path(tempfile.gettempdir()) / "test-output-ais_preparation"
if tmp_path.exists():
shutil.rmtree(tmp_path)
tmp_path.mkdir(parents=True)
pipeline = DataProcessingPipeline(name="ais_preparation",
base_dir=tmp_path) \
.add("lat_cycle_transform", CycleTransformer(n=180), name_mappings={"x": "lat"})\
.add("lon_cycle_transform", CycleTransformer(n=90), name_mappings={"x": "lon"})
features = ["lat_x", "lat_y", "lon_x", "lon_y"]
data = AISTestData(1000)
adf = AnnotatedDataFrame(dataframe=data.dataframe,
metadata=MetaData.from_dict(data=AISTestDataSpec.copy()))
dataset_filename = tmp_path / "test.hdf5"
adf.save(filename=dataset_filename)
adf.head(10)
A central idea to the experiment framework lies in providing a means for a consistent input and output to perform experiments. Hence, define a LearningTask (here: ForecastTask) that collects the learning parameters that define this task.
forecast_task = ForecastTask(
label="forecast-ais-short-sequence",
pipeline=pipeline, features=features,
models=[ModelInstanceDescription(BaselineA, {}),
ModelInstanceDescription(BaselineB, {}),
],
group_column="mmsi",
sequence_length=5,
forecast_length=1,
training_parameters=TrainingParameters(epochs=1,
validation_steps=1)
)
The actual experimentation takes a single LearningTask as input and it will output
experiment = Experiment(learning_task=forecast_task,
input_data=dataset_filename,
output_directory=tmp_path)
report = experiment.run()
with open(report, "r") as f:
print(f.read())
The outputs of an experiment are collected inside a dedicated (timestamped) folder. This folder will also contain a subfolder for each of the parametrized models that defines a LearningTask.
last_experiments = sorted([str(f) for f in Path(experiment.output_directory).glob(pattern="*") if f.is_dir()])
print("Last experiment in: ", last_experiments[-1])
experiment_folder = sorted([str(f) for f in Path(last_experiments[-1]).glob(pattern="*")])
file_listing = '\n'.join(experiment_folder)
print("Contents:\n")
print(file_listing)
Once the training is running it can be monitored using tensorboard:
tensorboard --logdir=<experiments-directory>