*How do you know, if you don't check? FTR, it's not just about sending garbage, it's also about requests accidentally being truncated or just generally garbled.*
json.NewDecoder(r.Body).Decode(&payload) will return an error if the request is garbled or truncated. The other nice thing about this code is that it composes nicely with the context. net/http provides the body as a Reader. And json NewDecoder takes a reader. The two fit together nicely, without any need for adapters. The API looks like it should be able to decode a large payload without allocating a large buffer to hold the bytes. Sadly this is not the way encoding/json is implemented. On Tuesday, 29 December 2020 at 16:31:58 UTC [email protected] wrote: > On Tue, Dec 29, 2020 at 5:17 PM Amnon <[email protected]> wrote: > >> I always use `json.NewDecoder(r.Body).Decode(&payload)` >> > > You shouldn't. > > The code is more succinct than reading the entire body into a buffer, and >> then unmarshalling it. And there is only one error to check. >> > > There is only one error to check, because you are ignoring one. Namely > this one: > > If I was super concerned about people sending trailing gibberish to my >> server, I could call `dec.Buffered()` to see >> > > if there was anything left after the json object. I generally have not >> seen people sending garbage after their requests. >> > > How do you know, if you don't check? FTR, it's not just about sending > garbage, it's also about requests accidentally being truncated or just > generally garbled. > > FWIW, if your concern is that the "read body and unmarshal" is two error > checks, one solution would be to add a helper that does it for you - which > you can then re-use. If that doesn't seem worth it because you would only > use it once - well, then I'd question if the one extra error check is > really that much of an issue. > > IMO, `Unmarshal` is simply the correct function to use and if I don't have > a good reason to make my software behave in strange ways in the presence of > bugs, I prefer not to do that. But YMMV, of course. > > > >> And it is not clear what the correct action in these case is. I generally >> will ignore it. >> >> On occasions that I need to consume very large json arrays in my backend, >> without consuming much memory, >> I sometimes do something like >> >> https://play.golang.org/p/Isw_3p7mR5- >> >> On Tuesday, 29 December 2020 at 10:51:23 UTC [email protected] >> wrote: >> >>> There is an important semantic difference between the two, which means >>> you almost never want to use a `Decoder`: `Unmarshal` is for parsing a >>> single json document, whereas a `Decoder` is for parsing a stream of >>> concatenated documents, like so: https://play.golang.org/p/4uiNyJlNIKh. >>> In particular, this means that using a `Decoder` silently drops trailing >>> data and might not detect erronous json: >>> https://play.golang.org/p/cuOAUnKCuEk >>> So, unless you specifically know that you have a stream of concatenated >>> json documents `Decoder` is not actually doing what you want. >>> >>> On Tue, Dec 29, 2020 at 11:37 AM Amit Saha <[email protected]> wrote: >>> >>>> On Tue, Dec 29, 2020 at 11:35 AM burak serdar <[email protected]> >>>> wrote: >>>> > >>>> > On Mon, Dec 28, 2020 at 5:22 PM Amit Saha <[email protected]> wrote: >>>> > > >>>> > > Hi all, let's say I am a single large JSON object that I want to >>>> > > process in my HTTP server backend. >>>> > > >>>> > > I am trying to get my head around if there is any performance >>>> > > advantage - memory or CPU to use the json.NewDecoder()/Decode() >>>> > > mechanism versus the Unmarshal() function? >>>> > >>>> > Unmarshal uses the decoder for unmarshaling. Unless you are planning >>>> > to process the JSON object piece by piece using a decoder, the two are >>>> > identical in terms of performance. >>>> >>>> Thank you for confirming. >>>> >>>> >>>> > >>>> > > >>>> > > Thanks, >>>> > > Amit >>>> > > >>>> > > -- >>>> > > You received this message because you are subscribed to the Google >>>> Groups "golang-nuts" group. >>>> > > To unsubscribe from this group and stop receiving emails from it, >>>> send an email to [email protected]. >>>> > > To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/golang-nuts/CANODV3%3DU2KRRkvAAEfYqRtCVtYnh2dmGreqePF8QXLo1PriSPw%40mail.gmail.com >>>> . >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "golang-nuts" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/golang-nuts/CANODV3%3DpXGcHzhSmaLr3f911i2CCYb0_j8CGH7zKAZXd_xTD-A%40mail.gmail.com >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/e9a6f76d-17bb-40da-84b3-2b1b8d0f3b35n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/golang-nuts/e9a6f76d-17bb-40da-84b3-2b1b8d0f3b35n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9af4c4a2-0267-4ed7-9bf0-7cc7afd4029en%40googlegroups.com.
