`artifact_paths` should accept directories

Features
#1

I put in a PR here that already supports this with my rationale: Directory artifact paths by jsoo1 · Pull Request #1571 · buildkite/agent · GitHub
(Sorry I did not think to put in a feature request here first)

To summarize my thoughts from the github issue and PR, there are two big user experience reasons the artifact_paths field should accept directories as well as glob patterns.

  1. Principle of least surprise. If artifact_paths contains some directory, it is very surprising when the contents of the directory are not uploaded, nor tried.
  2. Using $BUILDKITE_ARTIFACT_PATHS in scripts with only glob paths makes some uses impossible. In particular, it is because you can’t unambiguously un-glob a string in bash. For instance, make prefix=$BUILDKITE_ARTIFACT_PATHS will simply never work unless directories can be in artifact_paths.
#2

Hey @john.soo thanks for syncing these up. We take feature requests from all sorts of places!

I’ll ping the team to make sure they’ve seen that PR and can continue it from there :+1: