Dear Steve,
Thank you for the feedback.
Also I would like to thank other people whose e-mails
also give an glimpse of the depth of the problem.
On 2014/10/03 4:49, Steve Fink wrote:
On 10/02/2014 11:59 AM, ISHIKAWA,chiaki wrote:
Hi,
I was looking at a large number of JavaScript (strict) warnings, and
errors from the recording of |make mozmill| test run of locall-built
DEBUG BUILD of TB last several days after a refresh of C-C source tree.
The number of such has increased very much both
- due to the seemingly stricter checks done by JS infrastructure, and
- new carelessly written code,
I think.
Did you compare the count from a while ago to the count today? I was
kind of assuming that we've always had lots of these.
I just did. It is true that warnings and errors come and go.
Only the hard ones (or ignored ones) remain constantly.
That said,
compared with the local mochitest-plain done on June 18,
the different types of the JS warnings in the log jumped
from 45 to 67, and the total count of such warnings jumped
from 82 to 594.
In the case of TB, it was worse: from |make mozmill| test suite runs.
June 18: 11 types, 137 count. <=== well manageable.
Sept 28: 89 types, 5699 counts. <=== Ha!
(Gasp. OK, a single type of
getter only warning accounted for about 3100 lines. But still there are
other 2200 lines of warnings.)
Now, the particular # mentioned above was culled by the
lines picked up by the following command ($1 is the log file
recorded by "script" command from which the test was invoked.)
--- begin quote
echo
echo " ========================================"
echo " JavaScript strict warning"
echo " jquery.js and jquery-ui.js are ignored. "
echo " ========================================"
echo
#
# 24.4.0/comm-esr had jquery-1.5.1.min.js
#
grep ^JavaScript $1 | sort | uniq -c | sort -r -n | egrep -v
"(jquery.js|jquery-ui.js|jquery-1.5.1.min.js)"
--- end quote ---
This means that, for the moment, I am IGNORING the C++ side
bugs/warnings to limit the scope of
discussion. And there are so many of them, I have no idea what to make
of them actually.
And, of course, outright ERROR are also excluded mostly.
They seem to have jumped, too, but I have not bothered to check now.
To me, the whole situation looks to be a serious regression in terms of
quality assurance and testing.]
BTW, C-C source tree had more such warnings than June 18 figures, say,
around the beginning of 2012.
But thanks to the efforts of aceman and many others
and my effort to alert the situation in
Bug 826732 - JavaScript strict warning seen during "make mozmill" run of
TB (DEBUG BUILD)
the warnings diminished to the June 18th figures, and only very hard
ones remain there.
But suddenly the number has exploded out of hands after this summer,
so to speak.
Regarding the warnings/errors reported in DEBUG build test runs,
after hearing the opinions voiced,
I have no idea how to tackle this situation from the perspective
of a TB user who occasionally experiences real-world bugs and
reports them, and in some cases tries to create a patch and
is overwhelmed by the bad noise/signal ratio in the log when the patch
is applied and tested locally.
I completely share the sentiment that "I couldn't believe that such
serious-sounding errors could have existed before my patch, but
eventually discovered that they did." mentioned by Steve.
At least, a concerted efforts over 24 months or so had diminished
the numbers of conspicuous warnings to manageable (easy to monitor
and pay attention occasionally) in TB JS area until June this year.
- I am for a strict check and I am of the opinion that we should
remove such warnings/errors. They often cover real bugs underneath.
Once I see the log, it is difficult to trust mozilla software
completely :-)
- How do we assign precious man-power to fixing these issues,
which seems to be delegated to a second-class citizen in the
mozilla dev community, is a problem, I think.
- Raising the awareness of very poor situation is a must, I think.
-- That it hurts the quality, and a quick detection of real-world bugs
is now difficult because of noise ought to be understood by many
who are contributing new code.
-- The bad signal-to-noise ratio does not help someone creating a new
patch since it may mask the newly introduced issues.
- Automatic detection and assignment of bugzilla mentioned
by Mike de Boer may be a worthwhile idea to pursue.
I think a discussion about this is in order.
(After all, we get a regular intermittent (?)/"orange bugzilla today
for permanent(?) orange failures. They get reported to respective
bugzilla entries, I think.)
Setting "assignment" aspect aside for a moment,
at least, automatic detection of such errors and warnings
and see whether it has already been
filed, and if not, create the list of not-yet-filed
such warnings/errors in a web page or something [in a manner
so that a filing of the bug and possible assignment from the listing
is easy would be useful.]
(I believe FF debug build of nightly? is tested regularly.
So the log data is there. I am not sure if TB debug build is tested
regularly any more in this manner.)
BTW, bug 826732 to keep track of the errors and warnings
may have lingered too long. The comments and patches have gotten very
long. Maybe we should close the bug and re-start
at 6 months interval so that the comments and patches do not extend too
long. I think I can carry over the remaining known bugs to a new
bugzilla entry, in the form of a new list and transcribe the "depends
on" entries to a new one. This now seems to be a good idea.
Yeah, it seems nobody looks at it. Except when some random developer
happens to break a test, and tries to figure out what warnings/errors
are real and what are expected noise. It's pretty awful.
This is the sentiment I have had.
Then again, we can't get people to fix intermittent test failures, and
instead disable tests frequently. So finding such a sheriff is going to
be... "challenging". The understanding of many of these warnings tends
to be specific to very localized parts of the codebase, so a single
sheriff would need to pester a lot of people.
Or not -- actually, a lot of these are probably easily solvable. If
someone were willing to triage, they could probably at least cut down
the number by quite a bit if they were willing to ignore the harder ones.
I think this is exactly what happens for the last 12-24 months for the
problematic TB code until June this year.
So a few dedicated people can do wonders, I suspect.
These references to undefined whatever seem to have appeared suddenly
thanks to
the stricter checking of JS infrastructure.
I wonder if that is a side effect of the 'let' TDZ (temporal dead zone)
changes.
I have begun to suspect this, but
there are definitely other issues, too, like the assignment to a
property that has only a getter. And a few others.
TIA
PS: Compile-time warnings have a bugzilla entry and
the number seems to be steadily decreasing although I see
GCC is getting better to detect possible problems and so
the absolute number may not have decreased much
(still the number is impressive. Thanks!)
but still the efforts are there and reassuring.
Bug 187528 - (buildwarning) [META] Fix compiler 'Build Warnings'
PPS: BTW, it seems that |make mozmill| infrastructure seems to be more
picky about errors/warnings and where FF tests may not pick up
suspicious log lines, |make mozmill| seems to pick them.
(I think it has something to do with the regular expression used in
these tests.)
_______________________________________________
dev-platform mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-platform
