Click CLI

CLI Usage Guide

Primary interface for pipeline execution, validation, profiling, and infrastructure management.

Click-based CLI Local + GCS runs Airflow controls
dbt Observability Architecture & Ops Data Contract Data Dictionary Metrics Snapshot

Quick Start

Recommended CLI

Use the Click-based ecomlake command. Legacy entrypoints emit warnings.

ecomlake bronze profile --date-range YYYY-MM-DD..YYYY-MM-DD

Help Menu

Explore available groups, commands, and flags.

ecomlake --help
ecomlake pipeline --help
ecomlake airflow --help

Core CLI Commands

Command Purpose Key Output
ecomlake bronze profile Profile Bronze Parquet samples Quality report, schema map JSON, data contract updates
ecomlake bucket report Generate bucket size report Markdown report with table-level storage metrics
ecomlake sample pull Pull sample partitions from GCS Local Bronze samples for profiling
ecomlake airflow bootstrap Initialize Airflow environment Airflow directories and Docker containers
ecomlake pipeline dev-gcs Run dev pipeline (GCS native) Full Bronze to Enriched execution
ecomlake pipeline sim-prod-gcs Run prod simulation (GCS native) Production simulation against GCS
ecomlake silver run Run Base Silver (dbt-duckdb) Base Silver tables + manifests

Command Groups

Environment Control (Airflow)

ecomlake airflow up
ecomlake airflow down
ecomlake airflow restart
ecomlake airflow logs
ecomlake airflow shell

Pipeline Execution (Airflow)

ecomlake pipeline run-sample --date YYYY-MM-DD
ecomlake pipeline run-sample-strict --date YYYY-MM-DD
ecomlake pipeline run-sample-bq --date YYYY-MM-DD
ecomlake pipeline backfill-easy --start YYYY-MM-DD --end YYYY-MM-DD

Local Development (No Docker)

ecomlake local dims --date YYYY-MM-DD
ecomlake local silver --date YYYY-MM-DD
ecomlake local enriched --date YYYY-MM-DD

GCS Pipeline Execution (Native)

ecomlake pipeline dev-gcs YYYY-MM-DD
ecomlake pipeline sim-prod-gcs YYYY-MM-DD
ecomlake pipeline dev-docker YYYY-MM-DD

Development & Testing

ecomlake dev test
ecomlake dev lint
ecomlake dev format
ecomlake dev type-check

dbt Utilities

ecomlake dbt deps
ecomlake dbt build
ecomlake dbt test

Notes