How to conditionally fail a build in Pull Requests with TFS/VSTS

By terje

When you have a build that is used for CI also covering pull requests (PR), you often want to enable more checks before you let this go into the master (or any target) branch. It can be extra tests you want to run, or, you might want to block the PR if you have warnings, e.g. from tests.

Setting a single test task to fail could be done, but that only works well if you have a single step, with multiple you can have build stops for one and every step, so it also reduces the error reporting granularity.  And, it is nice to be able to see what is an absolutely blocker, red, and what is quality issues, yellow.

You can achieve this by adding a standard command line task that fails given these conditions.

In the build shown below, there are seven tasks that may give rise to warnings (green checkmarks).  They are checked using the red circled command line task below them.

  1. Give it a good name like: "If above is partially succeeded, and we have a PR , then fail"
  2. Use the command "echo"
  3. The arguments are "1>&2".  This mean we redirect the standard output to the error output
  4. Set to fail on Standard Error.
  5. Set to custom conditions
  6. Set to: and(eq(variables['Agent.JobStatus'], 'SucceededWithIssues'), eq(variables['Build.Reason'], 'PullRequest'))

The color of the build will then be red, not yellow, but that is how the Azure Pipelines work, so - at least it will block your build and then your Pull Requests too.

YAML

If you need a yml snippet for this, the above was converted by Edward Bordin - see this issue comment, into the following:

- script: 'echo 1>&2'
  failOnStderr: true
  displayName: 'If above is partially succeeded, then fail'
  condition: eq(variables['Agent.JobStatus'], 'SucceededWithIssues')