bug#20987: Replacing "15 Support for test suites" in the Automake manual

2015-07-05 Thread Arthur Schwarz
The existing Automake description is terse enough to be unreadable. Significant issues are unaddressed or addressed in a manner which is more confusing than enlightening. The replacement manual covers each test regime fully (except for Dejagnu) and provides a introduction which gives a general basi

bug#20714: tap-driver.sh outputs incorrect .trs tag

2015-06-02 Thread Arthur Schwarz
Eric. You know, I really appreciate your effort. I do. Now to the substance of it. 1. Now that I know the bandwidth limitations and constraints I will be much more careful in transmitting files. I did not know before. I know now. I will take care to determine what compression format to use prior

bug#20715: tap-driver.sh 'missing test plan' message contradicts the TAP Standard

2015-06-02 Thread Arthur Schwarz
On 2015-06-02 09:40 -0700, Arthur Schwarz wrote: > [Nick Bowler wrote] > The plan line has to be mandatory for this feature to work as > intended (i.e., for the TAP consumer to determine whether a > producer has run to completion or not). An optional plan would >

bug#20714: tap-driver.sh outputs incorrect .trs tag

2015-06-02 Thread Arthur Schwarz
ry. art -Original Message- From: Eric Blake [mailto:ebl...@redhat.com] Sent: Tuesday, June 02, 2015 10:17 AM To: Arthur Schwarz; 'Nick Bowler' Cc: 20...@debbugs.gnu.org Subject: Re: bug#20714: tap-driver.sh outputs incorrect .trs tag On 06/02/2015 10:24 AM, Arthur Schwarz wrote: >

bug#20715: tap-driver.sh 'missing test plan' message contradicts the TAP Standard

2015-06-02 Thread Arthur Schwarz
The plan line has to be mandatory for this feature to work as intended (i.e., for the TAP consumer to determine whether a producer has run to completion or not). An optional plan would be useless. Sorry to be such a pest. Your statement that "... determine whether a producer has

bug#20714: tap-driver.sh outputs incorrect .trs tag

2015-06-02 Thread Arthur Schwarz
e with this judicious comment in hand. I think I would have just answered the question and then pointed out the source, but then, that's me. art -Original Message- From: Nick Bowler [mailto:nbow...@elliptictech.com] Sent: Tuesday, June 02, 2015 7:47 AM To: Arthur Schwarz Cc: 20..

bug#20716: TAP Protocol skip is not recognized.

2015-06-01 Thread Arthur Schwarz
http://testanything.org/ DIRECTIVES Directives are special notes that follow a # on the test line. Only two are currently defined: TODO and SKIP. Note that these two keywords are not case-sensitive. The tap-driver.sh does accept 'ok # skip' but requires 'ok # SKIP'. The TAP standard says that sk

bug#20715: tap-driver.sh 'missing test plan' message contradicts the TAP Standard

2015-06-01 Thread Arthur Schwarz
15.4.1 Introduction to TAP http://testanything.org/ Explicitly specifies that a test plan is optional. "The plan The plan tells how many tests will be run, or how many tests have run. It's a check that the test file hasn't stopped prematurely. It must appear once, whether at the beginning or

bug#20714: tap-driver.sh outputs incorrect .trs tag

2015-06-01 Thread Arthur Schwarz
The .trs tag :global-test-result: should be :test-global-result:. 15.3.3.2 Log files generation and test results recording :test-global-result: This is used to declare the "global result" of the script. Currently, the value of this field is needed only to be reported (more or less verbatim) i

bug#20693: SKIP not counted in TAP

2015-05-29 Thread Arthur Schwarz
-Original Message-In a way, because... > skip > ok > PASS: test2.tap 1 > not ok > FAIL: test2.tap 2 > 1..2 ... it is your program's output that does not follow the specification. There are two basic styles of skipping in TAP. You can skip a single test result: ok 1 # skip reason o

bug#20693: SKIP not counted in TAP

2015-05-29 Thread Arthur Schwarz
g file. o o o st = getline curline = nextline nextline = $0 $0 = curline while(st >= 0) if (st < 0) # I/O error. fatal("I/O error while reading from input stream") } On 2015-05-29 08:15 -0700, Arthur Schwarz wrote: > skip

bug#20693: SKIP not counted in TAP

2015-05-29 Thread Arthur Schwarz
I think this is an error. skip causes the following issues: 1: It is not reported. 2: It is not summarized. 3: It must be subtracted from the plan (tests run), and 4: The .log file has TAP generated output and the test data intermixed. I think this is against the TAP definition? ===

bug#20673: YAML not included in tap-driver.sh

2015-05-26 Thread Arthur Schwarz
15.4.3 Incompatibilities with other TAP parsers and drivers YAML should be included in the list of incompatibilities. TAP version < 13: http://testanything.org/tap-specification.html Java tap4j provides objects to create TestSets, TestResults, BailOuts and, using a TapProducer, dump it into a j

bug#18274: [ENHANCEMENT] Additional Tables

2014-08-15 Thread Arthur Schwarz
It would be useful for there to be a set of tables to allow ready identification of macros and variables and .. required for different functional areas (especially for newbies like me). For example: What stuff is optional/required/relevant for Placing support files into standard places (RE