docs: clarify --trace flag behavior with --output flag

[thevilledev]

Improve documentation to make it clear that when both --trace and --output flags are specified, the output format takes priority over tracing. This helps users understand that they cannot combine tracing with custom output formats like JSON or table.

Relates to #1031

[jalseth]

I am concerned that this will break users who accidentally have --trace enabled with an output other than Standard. I think the --output flag is more important to prioritize.

Is it possible to enable tracing without changing the output style?

[thevilledev]

At this time tracing is only implemented in the standard output format. Since choosing the output format is a single pass it's one or the other. If we prioritise output over trace then the only option would be to ignore it, which is what conftest is already doing. Maybe give a warning, although if the warning is written to stdout it will break piping use cases like conftest test <args> | jq .

A bit of a larger refactoring would be to separate trace output handling from the output format and write it to a separate stream. Quite common approach for debug/trace outputs. Something like this:

• Default behavior (no -o flag):
  • Standard output format → stdout
  • Trace info (if enabled) → stderr
• Custom output (with -o flag):
  • Chosen format (table/json/etc) → stdout
  • Trace info in standard format (if enabled) → stderr

So with conftest test --trace -o json you would see both trace and the json output but on different streams, enabling piping as before and likely less of a breaking change for many.

[jalseth]

That makes sense, thanks. For now, I'd like to leave it as-is with preference for the output flag, and we can add to the description of the trace flag to clarify that it only has an effect for the standard output.

[jalseth]

🚀 Thanks!