On Sat, 23 Apr 2011 03:09:06 you wrote:
> Package: lrzip
> Version: 0.552+20110217+gitcd8b086-1
> Severity: normal
>
> [NOTE: This still seems to be a problem with the most recent upstream
> version of 0.603 - try with "lrzip -lvv" - hence Cc:ing to upstream
> Con Kolivas.]
>
> Hi,
>
> While trying to compress a big (172 GiB) file (with lrzip -nvv), I ran
> into this error:
>
> ------------------------------------------------------------
> Starting thread 7 to compress 19381057 bytes from stream 1
> Starting thread 0 to compress 19381057 bytes from stream 1
> pthread_createResource temporarily unavailable
> Fatal error - exiting
> ------------------------------------------------------------
>
> It turns out lrzip does not pthread_join() the threads it creates,
> which in turn eventually exhausts the threads. You can verify this by
> modifying it to print some debug information:
>
> ------------------------------------------------------------
Thanks for that Sami.
I guess this answers this comment in the manpages:
"It is unspecified whether a thread that has exited but remains unjoined counts
against {PTHREAD_THREADS_MAX}."
Detaching the threads on the compression side should be enough to fix this
issue. I've committed a fix to my git repo and the patch is attached below.
Regards,
Con
--
-ck
diff --git a/stream.c b/stream.c
index 6f840e7..c11da00 100644
--- a/stream.c
+++ b/stream.c
@@ -135,13 +135,19 @@ static void cond_broadcast(pthread_cond_t *cond)
fatal("pthread_cond_broadcast failed");
}
-void create_pthread(pthread_t * thread, pthread_attr_t * attr,
+void create_pthread(pthread_t *thread, pthread_attr_t * attr,
void * (*start_routine)(void *), void *arg)
{
- if (pthread_create(thread, attr, start_routine, arg))
+ if (unlikely(pthread_create(thread, attr, start_routine, arg)))
fatal("pthread_create");
}
+void detach_pthread(pthread_t *thread)
+{
+ if (unlikely(pthread_detach(*thread)))
+ fatal("pthread_detach");
+}
+
void join_pthread(pthread_t th, void **thread_return)
{
if (pthread_join(th, thread_return))
@@ -1435,6 +1441,7 @@ static void clear_buffer(rzip_control *control, struct stream_info *sinfo, int s
s->i = i;
s->control = control;
create_pthread(&threads[i], NULL, compthread, s);
+ detach_pthread(&threads[i]);
if (newbuf) {
/* The stream buffer has been given to the thread, allocate a
