On 2020-07-16 12:53, Pirate Praveen wrote: >> Generally speaking, I think it's a mistake to apply the question of >> "preferred form for modification" to unit test payloads. Unit tests are >> purely about functionality. The original source to a payload is an >> arbitrary choice (possibly even randomly generated), and could be >> replaced with any other appropriate arbitrary choice at no detriment to >> the software or the user. >> > > I think this needs to be clearly documented in policy. I don't think > this interpretation is generally accepted. I have seen many cases where > tests are disabled for this reason.
Perhaps I spoke too generally. For example, I can see, as one of probably many counter-examples, the case where the input is not completely arbitrary (eg: input is a captured stream). But to take the other extreme, using completely arbitrary data, as an example: say my code implements a ROT13 function and I create a test for it using a blob of random data as well as the expected output. That random data was generated somehow, eg: using Python's random module, and could therefore be regenerated given the correct program and seed. However, I did not include the code to generate that data. Would we really reasonably expect anyone to act upon that random blob in any way?
