After some time of setting this up and building I can reproduce it and
got a better backtrace which seems well inside flint (up to frame 6):
Program received signal SIGSEGV, Segmentation fault.
0x00000040014cb29c in flint_mpz_set_ui (r=0x400c8e3dd0, r=0x400c8e3dd0,
u=9223372036854775837)
at ./gmpcompat.h:73
73 ./gmpcompat.h: No such file or directory.
(gdb) bt
#0 0x00000040014cb29c in flint_mpz_set_ui (r=0x400c8e3dd0,
r=0x400c8e3dd0, u=9223372036854775837)
at ./gmpcompat.h:73
#1 fmpz_set_ui (val=9223372036854775837, f=0x4000b9e7a0) at ./fmpz.h:214
#2 _fmpz_poly_xgcd_modular (len2=3, poly2=0x400c8e3d50, len1=3,
poly1=0x4003fe51c0, t=0x400c8ee520,
s=0x400c8e3c90, r=0x400c8e3cb8) at xgcd_modular.c:129
#3 _fmpz_poly_xgcd_modular (r=0x400c8e3cb8, s=0x400c8e3c90,
t=0x400c8ee520, poly1=0x4003fe51c0,
len1=3, poly2=0x400c8e3d50, len2=3) at xgcd_modular.c:34
#4 0x00000040014ec9f8 in _fmpz_poly_xgcd (len2=3, poly2=<optimized
out>, len1=3,
poly1=0x4003fe51c0, t=0x400c8ee520, s=0x400c8e3c90, r=0x400c8e3cb8)
at ./fmpz_poly.h:632
#5 _fmpq_poly_xgcd (G=0x400c8e3cf0, denG=0x400c8e3cb8, S=0x400c8e3c90,
denS=0x400c8e3c58,
T=0x400c8ee520, denT=0x400c8ee4e8, A=<optimized out>,
denA=0x400c8e3808, lenA=3, B=0x400c8e3d50,
denB=0x400c8e3d18, lenB=3) at xgcd.c:106
#6 0x00000040014ecf00 in fmpq_poly_xgcd (G=0x400c8e3cb0,
S=0x400c8e3c50, T=0x400c8ee4e0,
A=0x400c8e3800, B=0x400c8e3d10) at xgcd.c:234
#7 0x000000400137027c in pm::FlintPolynomial::xgcd (p2=..., p1=...,
t=..., s=..., g=...)
at ./include/core/polymake/FlintPolynomial.h:689
#8 pm::FlintPolynomial::xgcd (p2=..., p1=..., t=..., s=..., g=...)
at ./include/core/polymake/FlintPolynomial.h:685
...
I have tried to create a flint-only testcase for this which is attached
and gives on debian amd64:
lorenz@debian-unstable-amd64:~/pm40$ gcc flint-mips.cc -lflint
lorenz@debian-unstable-amd64:~/pm40$ ./a.out
(1) * (x^2 + 3*x + 1) + (-1) * (x^2 + 3*x) = 1
But in a mips64el qemu chroot:
$ gcc flint-mips.c -lflint
$ ./a.out
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault
The backtrace for this is slightly different:
Program received signal SIGSEGV, Segmentation fault.
0x00000040008b922c in fmpz_get_mpz (x=0x4000811fd8, f=<optimized out>)
at ./gmpcompat.h:66
66 ./gmpcompat.h: No such file or directory.
(gdb) bt
#0 0x00000040008b922c in fmpz_get_mpz (x=0x4000811fd8, f=<optimized out>)
at ./gmpcompat.h:66
#1 0x0000004000908f8c in _fmpq_poly_get_str_pretty (poly=0x40015c86f0,
den=0x4000812110, len=3, var=0x4000001170 "x") at get_str_pretty.c:138
#2 0x0000004000909844 in fmpq_poly_get_str_pretty (poly=<optimized out>,
var=<optimized out>) at get_str_pretty.c:215
#3 0x0000004000000e10 in main (argc=1, argv=0x4000812318) at
flint-mips.c:20
> It looks like it's flint related?
Yes, I think this is a bug in flint and for now I suggest disabling the
flint-interface of polymake with the configure option --without-flint on
mips64el. This has basically no functionality loss as polymake will use
its own generic polynomial arithmetic instead (it will be somewhat slower).
Currently rebuilding polymake to check the testsuite again but this will
take some time and I will report back tomorrow.
#include <stdlib.h>
#include <stdio.h>
#include "flint/fmpq_poly.h"
int main(int argc, char* argv[])
{
char *strp1, *strp2, *strg, *strs, *strt;
fmpq_poly_t p1,p2;
fmpq_poly_t g, s, t;
fmpq_poly_init(p1);
fmpq_poly_init(p2);
fmpq_poly_init(g);
fmpq_poly_init(s);
fmpq_poly_init(t);
fmpq_poly_set_str(p1, "3 1 3 1");
fmpq_poly_set_str(p2, "3 0 3 1");
fmpq_poly_xgcd(g,s,t,p1,p2);
strp1 = fmpq_poly_get_str_pretty(p1, "x");
strp2 = fmpq_poly_get_str_pretty(p2, "x");
strg = fmpq_poly_get_str_pretty(g, "x");
strs = fmpq_poly_get_str_pretty(s, "x");
strt = fmpq_poly_get_str_pretty(t, "x");
flint_printf("(%s) * (%s) + (%s) * (%s) = %s\n", strs, strp1, strt, strp2, strg);
flint_free(strp1);
flint_free(strp2);
flint_free(strg);
flint_free(strs);
flint_free(strt);
fmpq_poly_clear(p1);
fmpq_poly_clear(p2);
fmpq_poly_clear(g);
fmpq_poly_clear(s);
fmpq_poly_clear(t);
return EXIT_SUCCESS;
}
