dg CLI
dg check
Commands for checking the integrity of your Dagster code.
dg check [OPTIONS] COMMAND [ARGS]...
defs
Loads and validates your Dagster definitions using a Dagster instance.
If run inside a deployment directory, this command will launch all code locations in the deployment. If launched inside a code location directory, it will launch only that code location.
When running, this command sets the environment variable DAGSTER_IS_DEFS_VALIDATION_CLI=1. This environment variable can be used to control the behavior of your code in validation mode.
This command returns an exit code 1 when errors are found, otherwise an exit code 0.
dg check defs [OPTIONS]
Options:
- --log-level <log_level>
Set the log level for dagster services.
Default:
'warning'
Options: critical | error | warning | info | debug
- --log-format <log_format>
Format of the logs for dagster services
Default:
'colored'
Options: colored | json | rich
- -v, --verbose
Show verbose error messages, including system frames in stack traces.
- --check-yaml, --no-check-yaml
Whether to schema-check defs.yaml files for the project before loading and checking all definitions.
- --target-path <target_path>
Specify a directory to use to load the context for this command. This will typically be a folder with a dg.toml or pyproject.toml file in it.
- --verbose
Enable verbose output for debugging.
yaml
Check defs.yaml files against their schemas, showing validation errors.
dg check yaml [OPTIONS] [PATHS]...
Options:
- --watch
Watch for changes to the component files and re-validate them.
- --validate-requirements, --no-validate-requirements
Validate environment variables in requirements for all components in the given module.
- --verbose
Enable verbose output for debugging.
- --target-path <target_path>
Specify a directory to use to load the context for this command. This will typically be a folder with a dg.toml or pyproject.toml file in it.
Arguments:
- PATHS
Optional argument(s)
dg dev
Start a local instance of Dagster.
If run inside a workspace directory, this command will launch all projects in the workspace. If launched inside a project directory, it will launch only that project.
dg dev [OPTIONS]
Options:
- --code-server-log-level <code_server_log_level>
Set the log level for code servers spun up by dagster services.
Default:
'warning'
Options: critical | error | warning | info | debug
- --log-level <log_level>
Set the log level for dagster services.
Default:
'info'
Options: critical | error | warning | info | debug
- --log-format <log_format>
Format of the logs for dagster services
Default:
'colored'
Options: colored | json | rich
- -p, --port <port>
Port to use for the Dagster webserver.
- -h, --host <host>
Host to use for the Dagster webserver.
- --live-data-poll-rate <live_data_poll_rate>
Rate at which the dagster UI polls for updated asset data (in milliseconds)
Default:
2000
- --check-yaml, --no-check-yaml
Whether to schema-check defs.yaml files for the project before starting the dev server.
- --target-path <target_path>
Specify a directory to use to load the context for this command. This will typically be a folder with a dg.toml or pyproject.toml file in it.
- --verbose
Enable verbose output for debugging.
- --autoload-defs-module-name <autoload_defs_module_name>
A module to import and recursively search through for definitions.
- -m, --module-name <module_name>
Specify module or modules (flag can be used multiple times) where dagster definitions reside as top-level symbols/variables and load each module as a code location in the current python environment.
- -f, --python-file <python_file>
Specify python file or files (flag can be used multiple times) where dagster definitions reside as top-level symbols/variables and load each file as a code location in the current python environment.
- -d, --working-directory <working_directory>
Specify working directory to use when loading the repository or job
- -w, --workspace <workspace>
Path to workspace file. Argument can be provided multiple times.
- --empty-workspace
Allow an empty workspace
Environment variables:
- DAGSTER_autoload_defs_module_name
Provide a default for
--autoload-defs-module-name
- DAGSTER_MODULE_NAME
Provide a default for
--module-name
- DAGSTER_PYTHON_FILE
Provide a default for
--python-file
- DAGSTER_WORKING_DIRECTORY
Provide a default for
--working-directory
dg launch
Launch a Dagster run.
dg launch [OPTIONS]
Options:
- --assets <assets>
Comma-separated Asset selection to target
- --job <job>
Job to target
- --partition <partition>
Asset partition to target
- --partition-range <partition_range>
Asset partition range to target i.e. <start>…<end>
- --config-json <config_json>
JSON string of config to use for the launched run.
- -c, --config <config>
Specify one or more run config files. These can also be file patterns. If more than one run config file is captured then those files are merged. Files listed first take precedence.
- --target-path <target_path>
Specify a directory to use to load the context for this command. This will typically be a folder with a dg.toml or pyproject.toml file in it.
- --verbose
Enable verbose output for debugging.
- -a, --attribute <attribute>
Attribute that is either a 1) repository or job or 2) a function that returns a repository or job
- --package-name <package_name>
Specify Python package where repository or job function lives
- --autoload-defs-module-name <autoload_defs_module_name>
A module to import and recursively search through for definitions.
- -m, --module-name <module_name>
Specify module where dagster definitions reside as top-level symbols/variables and load the module as a code location in the current python environment.
- -f, --python-file <python_file>
Specify python file where dagster definitions reside as top-level symbols/variables and load the file as a code location in the current python environment.
- -d, --working-directory <working_directory>
Specify working directory to use when loading the repository or job
Environment variables:
- DAGSTER_ATTRIBUTE
Provide a default for
--attribute
- DAGSTER_PACKAGE_NAME
Provide a default for
--package-name
- DAGSTER_autoload_defs_module_name
Provide a default for
--autoload-defs-module-name
- DAGSTER_MODULE_NAME
Provide a default for
--module-name
- DAGSTER_PYTHON_FILE
Provide a default for
--python-file
- DAGSTER_WORKING_DIRECTORY
Provide a default for
--working-directory
dg list
Commands for listing Dagster entities.
dg list [OPTIONS] COMMAND [ARGS]...
components
List all available Dagster component types in the current Python environment.
dg list components [OPTIONS]
Options:
- -p, --package <package>
Filter by package name.
- --json
Output as JSON instead of a table.
- --target-path <target_path>
Specify a directory to use to load the context for this command. This will typically be a folder with a dg.toml or pyproject.toml file in it.
- --verbose
Enable verbose output for debugging.
defs
List registered Dagster definitions in the current project environment.
dg list defs [OPTIONS]
Options:
- --json
Output as JSON instead of a table.
- -p, --path <path>
Path to the definitions to list.
- -a, --assets <assets>
Asset selection to list.
- -c, --columns <columns>
Columns to display. Either a comma-separated list of column names, or multiple invocations of the flag. Available columns: key, group, deps, kinds, description, tags, metadata, cron
- --verbose
Enable verbose output for debugging.
- --target-path <target_path>
Specify a directory to use to load the context for this command. This will typically be a folder with a dg.toml or pyproject.toml file in it.
envs
List environment variables from the .env file of the current project.
dg list envs [OPTIONS]
Options:
- --target-path <target_path>
Specify a directory to use to load the context for this command. This will typically be a folder with a dg.toml or pyproject.toml file in it.
- --verbose
Enable verbose output for debugging.
projects
List projects in the current workspace.
dg list projects [OPTIONS]
Options:
- --verbose
Enable verbose output for debugging.
- --target-path <target_path>
Specify a directory to use to load the context for this command. This will typically be a folder with a dg.toml or pyproject.toml file in it.
registry-modules
List dg plugins and their corresponding objects in the current Python environment.
dg list registry-modules [OPTIONS]
Options:
- --json
Output as JSON instead of a table.
- --target-path <target_path>
Specify a directory to use to load the context for this command. This will typically be a folder with a dg.toml or pyproject.toml file in it.
- --verbose
Enable verbose output for debugging.
dg scaffold
Commands for scaffolding Dagster entities.
dg scaffold [OPTIONS] COMMAND [ARGS]...
Commands:
- build-artifacts
Scaffolds a Dockerfile to build the given Dagster project or workspace.
- component
Scaffold of a custom Dagster component type.
This command must be run inside a Dagster project directory. The component type scaffold
will be placed in submodule <project_name>.lib.<name>.
- defs
Commands for scaffolding Dagster code.
- github-actions
Scaffold a GitHub Actions workflow for a Dagster project.
This command will create a GitHub Actions workflow in the .github/workflows directory.
dg scaffold example
Note: Before scaffolding definitions with dg
, you must create a project with the create-dagster CLI and activate its virtual environment.
You can use the dg scaffold defs
command to scaffold a new asset underneath the defs
folder. In this example, we scaffold an asset named my_asset.py
and write it to the defs/assets
directory:
dg scaffold defs dagster.asset assets/my_asset.py
Creating a component at /.../my-project/src/my_project/defs/assets/my_asset.py.
Once the asset has been scaffolded, we can see that a new file has been added to defs/assets
, and view its contents:
tree
.
├── pyproject.toml
├── src
│ └── my_project
│ ├── __init__.py
│ └── defs
│ ├── __init__.py
│ └── assets
│ └── my_asset.py
├── tests
│ └── __init__.py
└── uv.lock
cat src/my_project/defs/assets/my_asset.py
import dagster as dg
@dg.asset
def my_asset(context: dg.AssetExecutionContext) -> dg.MaterializeResult: ...
Note: You can run dg scaffold defs
from within any directory in your project and the resulting files will always be created in the <project-name>/src/<project_name>/defs/
folder.
In the above example, the scaffolded asset contains a basic commented-out definition. You can replace this definition with working code:
import dagster as dg
@dg.asset(group_name="my_group")
def my_asset(context: dg.AssetExecutionContext) -> None:
"""Asset that greets you."""
context.log.info("hi!")
To confirm that the new asset now appears in the list of definitions, run dg list defs:
dg list defs
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Section ┃ Definitions ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Assets │ ┏━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓ │
│ │ ┃ Key ┃ Group ┃ Deps ┃ Kinds ┃ Description ┃ │
│ │ ┡━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩ │
│ │ │ my_asset │ my_group │ │ │ Asset that greets you. │ │
│ │ └──────────┴──────────┴──────┴───────┴────────────────────────┘ │
└─────────┴─────────────────────────────────────────────────────────────────┘