Using jobs in Dagster projects

info

dg and Dagster Components are under active development. You may encounter feature gaps, and the APIs may change. To report issues or give feedback, please join the #dg-components channel in the Dagster Community Slack.

Prerequisites

Before following this guide, you will need to create a project with the create-dagster CLI.

Assets frequently use jobs that are instantiated elsewhere in the project.

For example, if you have created a new Dagster project with dg called my_project, you can define the jobs at src/my_project/defs/jobs.py:

Job binding can happen at any level of the defs hierarchy. If you moved asset_one in this example to a subdirectory, you could leave the existing jobs.py file at src/defs/jobs.py:

src
└── my_project
    └── defs
        ├── assets
        │   └── asset_one.py # contains def asset_one():
        └── jobs.py # contains dg.job

Scaffolding jobs

To create a job dictionary like the above, you can run the following:

dg scaffold defs dagster.job path/to/jobs.py

which will create

src/<project_name>/defs/jobs.py

import dagster as dg


@dg.job
def jobs():
    pass

and you can fill out the job dictionary as needed.