encoding/json: show nested fields path if DisallowUnknownFields leads to an error

[nabokihms]

Hello, Go community!

Problem

I tried to utilize the DisallowUnknownFields option for json.Decoder and faced a problem. It is not possible get the path of the problematic field from the error.

Example:

The structure

type Credentials struct {
	BasicAuth struct {
		Username string `json:"username"`
		Password string `json:"password"`
	} `json:"basic_auth"`
	BearerToken string `json:"bearer_token"`
}

and the following config

{
  "basic_auth": {
    "unknown_token": "test"
  },
  "unknown_token": "test"
}

on decoding will throw an error json: unknown field "unknown_token", yet it is not clear exactly which field caused it.

https://go.dev/play/p/zzLCOBn3AI8

If the config is a little, it is fine to find the field, but it is not suitable to validate cumbersome configs.

Proposal

Decoder already has all the info. The only thing left is to return it to a user.

The following block

go/src/encoding/json/decode.go

Lines 747 to 749 in 06b6759

	} else if d.disallowUnknownFields {
	d.saveError(fmt.Errorf("json: unknown field %q", key))
	}

should be rewritten like this

} else if d.disallowUnknownFields {
        d.errorContext.FieldStack = append(d.errorContext.FieldStack, string(key))
        d.saveError(fmt.Errorf("json: unknown field %q", strings.Join(d.errorContext.FieldStack, ".")))
}

or (to avoid Join)

} else if d.disallowUnknownFields {
    errMsg := fmt.Sprintf("json: unknown field %q", key)
    if len(d.errorContext.FieldStack) > 0 {
        errMsg += fmt.Sprintf(" in context %q", d.errorContext.FieldStack)
    }
    d.saveError(fmt.Errorf(errMsg))
}

[ianlancetaylor]

If I understand correctly, this just changes an error message. That doesn't need to be a proposal, so taking it out of the proposal process.

CC @dsnet @mvdan

[dsnet]

This is tangentially related to #43126, where the existing mechanism for recording paths is inconsistent about which type system (Go or JSON) it is sources from.

I have a slight preference for fixing #43126 first before opting more errors into having path context.

[nabokihms]

@dsnet thanks for guiding me there. If it is possible, I'd to fix this error myself after #43126 become merged.

[justinfx]

Any thoughts on this one? I have hit the same issue, where a failure from DisallowUnknownFields is very ambiguous and hard to know where there is an unknown field in a nested json object.

[nabokihms]

I still didn’t solve the issue, and I believe it can just be implemented straight forward without any further refactoring.

I can invest time in implementing it, this is only a matter whether the feature will be accepted or not.

[machine424]

I opened #71638

[gopherbot]

Change https://go.dev/cl/648055 mentions this issue: encoding/json: add context during decoding when disallowUnknownFields is set