Programmatic DAG representation

General
1

Hi,

We use the dag: true feature heavily. I’m interested in profiling our build steps, but in a way that incorporates knowledge of the step dependencies. For example, if step C depends on step B and step A, the step dependency information can tell us that step C is on the critical path.

I’m aware of the adding “/dag” to the build URLs, but that gets a visual HTML response, whereas I’d like a more immediately consumable representation. I tried looking through the JSON representation of a build, but I didn’t find anything that expressed which jobs any particular job was dependent on.

Does this information already exist?

If not, could we do something like adding “/dag.json” to the URL, or some REST endpoint to get something like:

{
  "nodes": ["A", "B", "C"],
  "edges": [
    ["A", "C"],
    ["B", "C"]
  ]
}

(this is just an example schema for purposes of illustration, real schema can be something else)

Thanks!

Andrew

2

Alternatively, add a "dependencies" field to each step in the build JSON, which lists the step keys of the jobs they depend on.

3

Hey @Andy!

Thanks for your post. I did a little digging myself and can see that, while we do give information on step_name in the API, we don’t actually deliver any information on the dependencies, as you’ve discovered.

I have raised this as a feature request with the team and will keep you up to date with any additional information I get back around this.

Cheers,

Ben

4

Hey @Andy!

Just following up on this one; we’ve captured it for future work but won’t be able to visit step improvements until the new year. It’s definitely something we’re keen on doing though and agree that there would be a lot of value in bubbling this information up to the surface.

Cheers,

Ben

5

@benmc reviving this old thread! We’d really love a way to analyze the critical path of our builds and getting this step/dependencies information is critical.

6

Hey @samsternatretool :wave:

Welcome to the community!

I wonder if the new Canvas feature would fit your use case - at least from a visual perspective?