Managing multiple projects with dg
This feature is considered in a preview stage and is under active development. It can change significantly, or be removed completely. It is not considered ready for production use.
If you're just getting started, we recommend scaffolding a single project instead of a workspace with multiple projects.
If you need to collaborate with multiple teams, or work with conflicting dependencies that require isolation from each other, you can scaffold a workspace directory that contains multiple projects, each with their own separate Python environment.
A workspace directory contains a root pyproject.toml with workspace-level settings, and a projects directory with one or more projects.
A workspace does not define a Python environment by default. Instead, Python environments are defined per project.
Scaffold a new workspace and first project
To scaffold a new workspace with an initial project called project-1, run dg init with the --workspace-name option:
dg init --workspace-name dagster-workspace
Scaffolded files for Dagster workspace at /.../dagster-workspace.
Enter the name of your Dagster project: project-1
Creating a Dagster project at /.../dagster-workspace/projects/project-1.
Scaffolded files for Dagster project at /.../dagster-workspace/projects/project-1.
...
This will create a new directory called dagster-workspace with a projects subdirectory that contains project-1. It will also set up a new uv-managed Python environment for this project.
Review workspace structure
The new workspace has the following structure:
cd dagster-workspace && tree
.
├── libraries
├── projects
│ └── project-1
│ ├── project_1
│ │ ├── __init__.py
│ │ ├── definitions.py
│ │ ├── defs
│ │ │ └── __init__.py
│ │ └── lib
│ │ └── __init__.py
│ ├── project_1_tests
│ │ └── __init__.py
│ ├── pyproject.toml
│ └── uv.lock
└── pyproject.toml
...
The pyproject.toml file for the workspace folder contains an is_workspace setting that marks this directory as a workspace:
[tool.dg]
directory_type = "workspace"
[tool.dg.workspace]
[tool.dg.workspace.scaffold_project_options]
use_editable_dagster = true
[[tool.dg.workspace.projects]]
path = "projects/project-1"
project-1 also contains a virtual environment directory called .venv that is not shown above. This environment is managed by uv and its contents are specified in the uv.lock file.
The project-1 directory contains a pyproject.toml file that defines
it as a Dagster project:
...
[tool.dg]
directory_type = "project"
[tool.dg.project]
root_module = "project_1"
...
Add a second project to the workspace
As noted above, environments are scoped per project. dg commands will only use the environment of project-1 when you are inside the project-1 directory.
Let's create another project:
dg scaffold project projects/project-2
Creating a Dagster project at /.../dagster-workspace/projects/project-2.
Scaffolded files for Dagster project at /.../dagster-workspace/projects/project-2.
...
Now we have two projects. We can list them with:
dg list project
projects/project-1
projects/project-2
Load workspace with dg
Finally, let's load up our two projects with dg dev. dg dev will automatically recognize the projects in your workspace and launch them in their respective environments. Let's run dg dev back in the workspace root directory and load up the Dagster UI in a browser:
cd ../.. && dg dev
