https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100927
--- Comment #3 from Michael Crusoe <michael.crusoe at gmail dot com> --- Good question, lets check the reference. Summary: it is specified behavior that _mm_cvttpd_epi32 returns Integer Indefinite (80000000H) for NaN inputs. All references below are from the December 2022 edition (Order Number: 325462-078US) of "Intel® 64 and IA-32 Architectures Software Developer’s Manual Combined Volumes: 1, 2A, 2B, 2C, 2D, 3A, 3B, 3C, 3D, and 4" from https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html The formal signature of the _mm_cvttpd_epi32 intrinsic is in Table C-1 "Simple Intrinsics" on page 2987, reminding us that the mnemonic is CVTTPD2DQ. The formal definition of CVTTPD2DQ is given in section 5.6.1.6 "Intel® SSE2 Conversion Instructions" on page 133 > Convert with truncation packed double precision floating-point values to > packed double- word integers. On page 106 we learn more about what truncation means in the definition of CVTTPD2DQ > 4.8.4.2 Truncation with Intel® SSE, SSE2, and AVX Conversion Instructions > The following Intel SSE/SSE2 instructions automatically truncate the results > of > conversions from floating-point values to integers when the result it > inexact: CVTTPD2DQ, > CVTTPS2DQ, CVTTPD2PI, CVTTPS2PI, CVTTSD2SI, and CVTTSS2SI. Here, truncation > means the > round toward zero mode described in Table 4-8. There are also several Intel > AVX2 and > AVX-512 instructions which use truncation (VCVTT*) Table 4.8 from section 4.8.4 states > Rounding Mode: Round toward zero (Truncate) > Description: Rounded result is closest to but no greater in absolute value > than the infinitely precise result. Section 11.4.1.6 ("SSE2 Conversion Instructions") states that > The CVTTPD2DQ (convert with truncation packed double precision floating-point > values to > packed doubleword integers) instruction is similar to the CVTPD2DQ > instruction except > that truncation is used to round a source value to an integer value. Table 11-1. "Masked Responses of SSE/SSE2/SSE3 Instructions to Invalid Arithmetic Operations" states that > Condition: Conversion to integer when the value in the source register is a > NaN, ∞, or > exceeds the representable range for CVTPS2PI, CVTTPS2PI, CVTSS2SI, CVTTSS2SI, > CVTPD2PI, > CVTSD2SI, CVTPD2DQ, CVTTPD2PI, CVTTSD2SI, CVTTPD2DQ, CVTPS2DQ, or CVTTPS2DQ > Masked Response: Return the integer Indefinite More explicitly stated is in section D.4.2.2 "Results of Operations with NaN Operands or a NaN Result for SSE/SSE2/SSE3 Numeric Instructions" where Table D-8 (page 455) ("CVTPS2PI, CVTSS2SI, CVTTPS2PI, CVTTSS2SI, CVTPD2PI, CVTSD2SI, CVTTPD2PI, CVTTSD2SI, CVTPS2DQ, CVTTPS2DQ, CVTPD2DQ, CVTTPD2DQ") states that the masked result from any type of NaN (SNaN or QNaN) will be the Integer Indefinite (80000000H in for 32-bit values).
