Solving NYT Connections with DSPy
In this example, you'll build an AI system that solves NYT Connections puzzles that:
- Loads and process Connections puzzle data
- Uses DSPy for structured reasoning
- Optimize the solver using MIPROv2 automatic optimization
- Evaluate puzzle-solving performance with custom metrics
- Deploy and monitor the AI system in production
Prerequisites
To follow the steps in this guide, you'll need:
- Basic Python knowledge
- Python 3.9+ installed on your system. Refer to the Installation guide for information.
- Understanding of language models and AI pipelines
- Basic knowledge of the NYT Connections puzzle format
Step 1: Set up your Dagster environment
First, set up a new Dagster project.
-
Clone the Dagster repo and navigate to the project:
cd examples/docs_projects/project_dspy
-
Install the required dependencies with
uv
:uv sync
-
Activate the virtual environment:
- MacOS
- Windows
source .venv/bin/activate
.venv\Scripts\activate
-
Ensure the following environments have been populated in your .env file. Start by copying the template:
cp .env.example .env
Then populate the fields.
Step 2: Launch the Dagster webserver
To make sure Dagster and its dependencies were installed correctly, navigate to the project root directory and start the Dagster webserver:
dg dev
Navigate to http://127.0.0.1:3000 to view the Dagster UI.
Next steps
- Continue this example with puzzle data ingestion