https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71291
Maxim Ostapenko <m.ostapenko at samsung dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |m.ostapenko at samsung dot com
--- Comment #9 from Maxim Ostapenko <m.ostapenko at samsung dot com> ---
Hm, looking to corresponding source code (dist/include/mozilla/gfx/Matrix.h):
705 template<class F>
706 size_t TransformAndClipRect(const RectTyped<SourceUnits, F>& aRect,
707 const RectTyped<TargetUnits, F>& aClip,
708 PointTyped<TargetUnits, F>* aVerts) const
709 {
710 // Initialize a double-buffered array of points in homogenous space
711 // with the input rectangle, aRect.
712 Point4DTyped<UnknownUnits, F> points[2]kTransformAndClipRectMaxVerts];
713 Point4DTyped<UnknownUnits, F>* dstPoint = points[0];
............
727 // Iterate through each clipping plane and clip the polygon.
728 // In each pass, we double buffer, alternating between points[0] and
729 // points[1].
730 for (int plane=0; plane < 4; plane++) {
731 planeNormals[plane].Normalize();
732
733 Point4DTyped<UnknownUnits, F>* srcPoint = points[plane & 1];
734 Point4DTyped<UnknownUnits, F>* srcPointEnd = dstPoint;
735 dstPoint = points[~plane & 1];
736
737 Point4DTyped<UnknownUnits, F>* prevPoint = srcPointEnd - 1;
738 F prevDot = planeNormals[plane].DotProduct(*prevPoint);
............
I suspect this scenario to happen:
1) On iteration 2 (i == 1) dstPoint becomes points[0] at line 735.
2) Later on iteration 1 dstPoint doesn't change for some reason.
3) On iteration 3 (i == 2) srcPointEnd becomes srcPointEnd = dstPoint (==
point[0]) at line 734.
4) Later on iteration 3 prevPoint = srcPointEnd - 1 (point[-1]) at line 737.
5) At line 738 we use *prevPoint (points[-1]) that leads to ASan report (valid,
because points[-1] overflows).
Could you check this? If this is what happens, than ASan is innocent and
something else went wrong here.
