Re: Testing edge cases and undocumented behavior

2011-08-30 Thread Steven Edwards
On Fri, Aug 26, 2011 at 6:00 AM, Michael Stefaniuc wrote: > It is a good advice. But that doesn't means one has to submit tests that > don't make sense. It seems like a simple comment describing the undocumented behavior would help. I wrote some tests years ago (the details are not important beca

Re: Testing edge cases and undocumented behavior

2011-08-30 Thread Scott Ritchie
On 08/30/2011 04:05 AM, joerg-cyril.hoe...@t-systems.com wrote: > For instance, Scott Ritchie's example is not unusual: > + * If lpszStr is Null, returns how long a formatted string would be. > This is a very common pattern. Probably MSDN forgot to mention it. > Expect apps to use that when they

Testing edge cases and undocumented behavior

2011-08-30 Thread Joerg-Cyril . Hoehle
Hi, Michael Stefaniuc wrote: >What you do want to test is: >- the de jure API (as documented), and >- the de facto API (as used by applications You plead for operational profile testing. I say that the API "as used by applications" is unknown to me because the 10 apps out there don't send me

Re: Testing edge cases and undocumented behavior

2011-08-26 Thread Michael Stefaniuc
On 08/26/2011 12:45 PM, joerg-cyril.hoe...@t-systems.com wrote: > Scott Ritchie wrote: >> There was a bit of a philosophical discussion on #winehackers about the >> merits of creating tests for functions that might be testing undefined >> or unimportant behavior. Windows behaves one way, we behave

Re: Testing edge cases and undocumented behavior

2011-08-26 Thread Francois Gouget
On Fri, 26 Aug 2011, Scott Ritchie wrote: [...] > >- APIs that take an 'LPSTR output_buffer, DWORD *buffer_size' pair of > > parameters. If they allow the programmer to pass 'NULL, &size' > > where size=0 as parameters to determine the required buffer size, > > then you can exp

Re: Testing edge cases and undocumented behavior

2011-08-26 Thread Scott Ritchie
On 08/26/2011 12:52 AM, Francois Gouget wrote: > On Thu, 25 Aug 2011, Vincent Povirk wrote: > [...] >> * Is a Windows application likely to need this? > > I'd add a couple of factors that pertain to this: > * Is the behavior documented by the MSDN? If yes then applications are >more likely t

Re: Testing edge cases and undocumented behavior

2011-08-26 Thread Reece Dunn
On 26 August 2011 11:53, wrote: > Vincent Povirk wrote: >>A test that passes on Windows and fails on Wine is not sufficient to >>motivate a change to Wine. > > I wished Wine had "wine_dont" beside "todo_wine".  Exact same > behaviour, it justs reads differently. > > 2 examples: > > mmdevapi has a

Testing edge cases and undocumented behavior

2011-08-26 Thread Joerg-Cyril . Hoehle
Vincent Povirk wrote: >A test that passes on Windows and fails on Wine is not sufficient to >motivate a change to Wine. I wished Wine had "wine_dont" beside "todo_wine". Exact same behaviour, it justs reads differently. 2 examples: mmdevapi has a bug where after Initialize(SHARED, unsupported_f

Testing edge cases and undocumented behavior

2011-08-26 Thread Joerg-Cyril . Hoehle
Scott Ritchie wrote: >There was a bit of a philosophical discussion on #winehackers about the >merits of creating tests for functions that might be testing undefined >or unimportant behavior. Windows behaves one way, we behave another, >the tests measure this delta, but it's unknown if this will a

Re: Testing edge cases and undocumented behavior

2011-08-26 Thread Francois Gouget
On Thu, 25 Aug 2011, Vincent Povirk wrote: [...] > * Is a Windows application likely to need this? I'd add a couple of factors that pertain to this: * Is the behavior documented by the MSDN? If yes then applications are more likely to rely on it. * Does the behavior correspond to a known us

Re: Testing edge cases and undocumented behavior

2011-08-25 Thread Vincent Povirk
A test that passes on Windows and fails on Wine is not sufficient to motivate a change to Wine. I think we normally consider the following factors: * Is the change correct? If not, we should consider leaving Wine alone, but I can imagine there being other compelling reasons, in rare circumstances,

Testing edge cases and undocumented behavior

2011-08-25 Thread Scott Ritchie
There was a bit of a philosophical discussion on #winehackers about the merits of creating tests for functions that might be testing undefined or unimportant behavior. Windows behaves one way, we behave another, the tests measure this delta, but it's unknown if this will actually improve a real wo