Good question and useful discussion!
What is Go community guidance on the _value_ of unit testing the `if err i=
nil { return err }` idiom?
To make the question a little more precise, let's consider the code snippet
in the first email in this thread. Let's assume that I already have
coverage for Foo() function happy path. Does it make sense to increase the
code complexity (adding mocks) in order to achieve a higher test coverage
(covering 'return err' line)? Would that additional coverage be useful
given that 'return err' has no complexity and Go has the compiler/linter?
Full disclosure: I'm biased to avoid unit testing those idioms by default.
However, I'm very curious what's the community guidance, any
documents/links I can read, any reference codebases?
Thank you all!
On Tuesday, December 8, 2020 at 4:39:05 AM UTC-6 [email protected]
wrote:
> Hi,
>
> On Tue, Dec 8, 2020 at 1:19 AM 'Charles Hathaway' via golang-nuts <
> [email protected]> wrote:
>
>> Hi all,
>>
>> I'm looking for a good study/quantitative measure of how well-written Go
>> code looks compared to other languages, such as Java, when it comes to test
>> coverage. In particular, how handling errors may reduce the percentage of
>> code covered by tests in Go relative to other languages.
>>
>> For example, in this code snippet:
>>
>> func Foo() error {
>> // do some stuff that actually adds value
>> if err := somelib.Bar(); err != nil {
>> // triggering the error case in Bar is hard, i.e. requires simulating
>> network troubles
>> // or causing a file write to fail, but we don't do anything with
>> result besides
>> // return it. Testing it by adding an interface or wrapper isn't
>> worth the effort
>> // and the only impact is really reported test coverage.
>> return err
>> }
>> // do more stuff
>> return nil
>> }
>>
>> In Java, you would just add 'throws SomeException' to your method
>> declaration. The effect is that we have one line in the Go code which is
>> not easily covered by a test, whereas Java does not report that untested
>> case because the return path is not visible in the code.
>>
>> The result is that otherwise equivalent code, we will report different
>> code coverage values, with Go being slightly lower. I'm just looking for
>> something written on that topic that can give us a notion of how much of a
>> difference we might expect.
>>
>
> I don't think there is as much of a difference as you think.
>
> You seem to be considering the `throws SomeException` to not impact
> coverage - but that's not true. It's code you add for error handling and
> that code is not hit, unless your test actually triggers that exception -
> just as the code you add for error handling in Go isn't hit. So if you
> don't count `throws SomeException` as code to be covered in java, you also
> shouldn't count `if err i= nil { return err }` as code to be covered in Go.
> So the semantic difference really comes down to a single `throws
> SomeException` line being able to cover *multiple* branches with the same
> exception type. It's a difference, but it should be small in practice.
>
> But really, I think what this comes down to is that line-coverage - or,
> what's actually measured and then projected down to lines,
> "instruction-coverage" - just isn't a super meaningful measure in this
> context. More interesting would be branch- or path-coverage - and that
> would be exactly the same in both cases. Every point where a
> `SomeException` *could* be thrown would branch off a separate path, just as
> every `if err != nil` in your Go code. And in both languages they are
> covered iff you write a test-case that triggers that error condition.
>
> So… I'm sorry that I can't really provide a quantitative, meaningful
> answer to your question. I don't know what relative difference there would
> be in line-coverage for Go vs. Java in a case like that. But your question
> sounds as if you would like to use line-coverage as a metric (maybe even in
> CI *shudder*) to determine whether you tested enough. And the point I'm
> trying to make is that I think that goal is fallacious :) If you need a
> coverage-metric, use branch- or path-coverage, which won't have that
> difference. But really, coverage reports are IMO most useful if inspected
> manually, to choose where to invest further tests. As a metric, it just is
> too unreliable.
>
> https://en.wikipedia.org/wiki/Goodhart%27s_law
>
>
>>
>> Thanks,
>> Charles
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/6b48ed73-1963-482e-aff0-b91f3aa6a2aen%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/golang-nuts/6b48ed73-1963-482e-aff0-b91f3aa6a2aen%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/1acd3558-93ca-494c-b639-af6f694a1fcfn%40googlegroups.com.
