PPA with the fix is available at: https://launchpad.net/~dragan-s/+archive/ubuntu/lp1616213
** Description changed: + [Impact] + + * During "service multipath-tools stop" multipath daemon + is trying to cleanup and shut down several concurrent + threads. At times depending on a race condition between + two threads, one thread might free resources that are still + used by another thread. + + This is causing the multipathd to dump crash core on + stop events. + + * Fix should be backported to trusty to avoid more support + issues being filed. + + * This change delays freeing resources that another thread is + still using. + + [Test Case] + + * install multipath-tools, create a basic multipath.conf with + devices under management. Run: "service multipath-tools start" + run I/O on devices and keep the system CPU busy, then run + "service multipath-tools stop". + + [Regression Potential] + + * There should be no regression potential with this change, + this problem happens on the exit path and we are only delaying + a free call. + + [Original Description] + On ubuntu trusty 14.04.4 in multipath-tools version 0.4.9-3ubuntu7.14 there is bug in multipathd on shutdown. The code will access pathvec pointer which is a valid address: Reading symbols from /sbin/multipathd...Reading symbols from /usr/lib/debug//sbin/multipathd...done. done. [New LWP 41631] [New LWP 41584] [New LWP 41633] [New LWP 41632] [New LWP 41582] [New LWP 41583] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/sbin/multipathd'. Program terminated with signal SIGSEGV, Segmentation fault. - #0 0x00000000004075db in checkerloop (ap=0x1b81040) at main.c:1150 + #0 0x00000000004075db in checkerloop (ap=0x1b81040) at main.c:1150 - 1150 vector_foreach_slot (vecs->pathvec, pp, i) { + 1150 vector_foreach_slot (vecs->pathvec, pp, i) { (gdb) list - 1145 pthread_cleanup_push(cleanup_lock, &vecs->lock); - 1146 lock(vecs->lock); - 1147 condlog(4, "tick"); - 1148 - 1149 if (vecs->pathvec) { - 1150 vector_foreach_slot (vecs->pathvec, pp, i) { - 1151 check_path(vecs, pp); - 1152 } - 1153 } - 1154 if (vecs->mpvec) { + 1145 pthread_cleanup_push(cleanup_lock, &vecs->lock); + 1146 lock(vecs->lock); + 1147 condlog(4, "tick"); + 1148 + 1149 if (vecs->pathvec) { + 1150 vector_foreach_slot (vecs->pathvec, pp, i) { + 1151 check_path(vecs, pp); + 1152 } + 1153 } + 1154 if (vecs->mpvec) { Pathvec is a valid pointer: (gdb) p vecs->pathvec $1 = (vector) 0x1b81280 But the contents of the structure are just garbage: (gdb) p *vecs->pathvec $2 = {allocated = 1651076143, slot = 0x756e696c2d34365f} (gdb) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1616213 Title: Core dump on multipathd shutdown - trusty 14.04.4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1616213/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs