For overview see:
http://ffmpeg.org/pipermail/ffmpeg-devel/2018-August/233665.html
Attached is patch 3/3 which was at "main.patch" of that message.
After the previous speedups, configure spent 20-60% of its runtime
at check_deps(). It's particularly slow with bash. After some local
optimizations - mainly avoid pushvar/popvar and abort early in one
notable case (empty deps), it's now x4-x25 faster.
After this patch, pushvar() and popvar() are not used but
I was hesitant to remove them (nice to have). If you think it should
be removed, the patch "opt2-remove-pushvar.patch" removes them.
On Saturday, August 25, 2018 7:55 PM, Timo Rothenpieler
<[email protected]> wrote:
Please use git send-email to send your patches, or at least send each
patch, created by git format-patch, as individual attachment. Your files
seem to contain multiple patches one after another, which makes them
very hard to follow.
But nice work! Let's hope this does not cause any regressions.
_______________________________________________
ffmpeg-devel mailing list
[email protected]
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
From a068913d2c4c460ef0a256cb3676bd1227f59667 Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <[email protected]>
Date: Sun, 5 Aug 2018 06:16:44 +0300
Subject: [PATCH] configure: check_deps(): x4 - x25 speedup
check_deps() recursively enables/disables components, and its loop is
iterated nearly 6000 times. It's particularly slow in bash - currently
consuming more than 50% of configure runtime, and about 20% with other
shells.
This commit applies few local optimizations, most effective first:
- Use $1 $2 ... instead of pushvar/popvar, and same at enable_deep*
- Abort early in one notable case - empty deps, to avoid costly no-op.
- Smaller changes which do add up:
- Handle ${cfg}_checking locally instead of via enable[d]/disable
- ${cfg}_checking: test done before inprogress - x2 faster in 50%+
- one eval instead of several at the empty-deps early abort path.
- The "actual work" part is unmodified - just its surroundings.
Biggest speedups (relative and absolute) are observed with bash.
---
configure | 86 +++++++++++++++++++++++++++++--------------------------
1 file changed, 46 insertions(+), 40 deletions(-)
diff --git a/configure b/configure
index b74c44e7..e9cb7703 100755
--- a/configure
+++ b/configure
@@ -676,13 +676,10 @@ disable_sanitized(){
do_enable_deep(){
for var; do
enabled $var && continue
- eval sel="\$${var}_select"
- eval sgs="\$${var}_suggest"
- pushvar var sgs
- enable_deep $sel
- popvar sgs
- enable_deep_weak $sgs
- popvar var
+ set -- $var
+ eval enable_deep \$${var}_select
+ var=$1
+ eval enable_deep_weak \$${var}_suggest
done
}
@@ -694,9 +691,9 @@ enable_deep(){
enable_deep_weak(){
for var; do
disabled $var && continue
- pushvar var
+ set -- $var
do_enable_deep $var
- popvar var
+ var=$1
enable_weak $var
done
}
@@ -756,40 +753,49 @@ is_in(){
return 1
}
+# The cfg loop is very hot (several thousands iterations), and in bash also
+# potentialy quite slow. Try to abort the iterations early, preferably without
+# calling functions. 70%+ of the time cfg is already done or without deps.
check_deps(){
for cfg; do
- enabled ${cfg}_checking && die "Circular dependency for $cfg."
- disabled ${cfg}_checking && continue
- enable ${cfg}_checking
-
- eval dep_all="\$${cfg}_deps"
- eval dep_any="\$${cfg}_deps_any"
- eval dep_con="\$${cfg}_conflict"
- eval dep_sel="\$${cfg}_select"
- eval dep_sgs="\$${cfg}_suggest"
- eval dep_ifa="\$${cfg}_if"
- eval dep_ifn="\$${cfg}_if_any"
-
- pushvar cfg dep_all dep_any dep_con dep_sel dep_sgs dep_ifa dep_ifn
- check_deps $dep_all $dep_any $dep_con $dep_sel $dep_sgs $dep_ifa $dep_ifn
- popvar cfg dep_all dep_any dep_con dep_sel dep_sgs dep_ifa dep_ifn
-
- [ -n "$dep_ifa" ] && { enabled_all $dep_ifa && enable_weak $cfg; }
- [ -n "$dep_ifn" ] && { enabled_any $dep_ifn && enable_weak $cfg; }
- enabled_all $dep_all || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but not all dependencies are satisfied: $dep_all"; }
- enabled_any $dep_any || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but not any dependency is satisfied: $dep_any"; }
- disabled_all $dep_con || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but some conflicting dependencies are unsatisfied: $dep_con"; }
- disabled_any $dep_sel && { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but some selected dependency is unsatisfied: $dep_sel"; }
-
- enabled $cfg && enable_deep_weak $dep_sel $dep_sgs
-
- for dep in $dep_all $dep_any $dep_sel $dep_sgs; do
- # filter out library deps, these do not belong in extralibs
- is_in $dep $LIBRARY_LIST && continue
- enabled $dep && eval append ${cfg}_extralibs ${dep}_extralibs
- done
+ eval [ x\$${cfg}_checking = xdone ] && continue
+ eval [ x\$${cfg}_checking = xinprogress ] && die "Circular dependency for $cfg."
+
+ eval "
+ dep_all=\$${cfg}_deps
+ dep_any=\$${cfg}_deps_any
+ dep_con=\$${cfg}_conflict
+ dep_sel=\$${cfg}_select
+ dep_sgs=\$${cfg}_suggest
+ dep_ifa=\$${cfg}_if
+ dep_ifn=\$${cfg}_if_any
+ "
+
+ # most of the time here $cfg has no deps - avoid costly no-op work
+ if [ "$dep_all$dep_any$dep_con$dep_sel$dep_sgs$dep_ifa$dep_ifn" ]; then
+ eval ${cfg}_checking=inprogress
+
+ set -- $cfg "$dep_all" "$dep_any" "$dep_con" "$dep_sel" "$dep_sgs" "$dep_ifa" "$dep_ifn"
+ check_deps $dep_all $dep_any $dep_con $dep_sel $dep_sgs $dep_ifa $dep_ifn
+ cfg=$1; dep_all=$2; dep_any=$3; dep_con=$4; dep_sel=$5 dep_sgs=$6; dep_ifa=$7; dep_ifn=$8
+
+ [ -n "$dep_ifa" ] && { enabled_all $dep_ifa && enable_weak $cfg; }
+ [ -n "$dep_ifn" ] && { enabled_any $dep_ifn && enable_weak $cfg; }
+ enabled_all $dep_all || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but not all dependencies are satisfied: $dep_all"; }
+ enabled_any $dep_any || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but not any dependency is satisfied: $dep_any"; }
+ disabled_all $dep_con || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but some conflicting dependencies are unsatisfied: $dep_con"; }
+ disabled_any $dep_sel && { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but some selected dependency is unsatisfied: $dep_sel"; }
+
+ enabled $cfg && enable_deep_weak $dep_sel $dep_sgs
+
+ for dep in $dep_all $dep_any $dep_sel $dep_sgs; do
+ # filter out library deps, these do not belong in extralibs
+ is_in $dep $LIBRARY_LIST && continue
+ enabled $dep && eval append ${cfg}_extralibs ${dep}_extralibs
+ done
+ fi
- disable ${cfg}_checking
+ eval ${cfg}_checking=done
done
}
--
2.17.1
From 77f897c8ed4eec9119d758037b0311629f549a5b Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <[email protected]>
Date: Wed, 1 Aug 2018 09:10:12 +0300
Subject: [PATCH] configure: remove unused pushvar()/popvar()
---
configure | 19 -------------------
1 file changed, 19 deletions(-)
diff --git a/configure b/configure
index e9cb7703..d6c1d032 100755
--- a/configure
+++ b/configure
@@ -619,25 +619,6 @@ get_sanitized(){
eval echo \$$(sanitize_var_name "$1")
}
-pushvar(){
- for pvar in $*; do
- eval level=\${${pvar}_level:=0}
- eval ${pvar}_${level}="\$$pvar"
- eval ${pvar}_level=$(($level+1))
- done
-}
-
-popvar(){
- for pvar in $*; do
- eval level=\${${pvar}_level:-0}
- test $level = 0 && continue
- eval level=$(($level-1))
- eval $pvar="\${${pvar}_${level}}"
- eval ${pvar}_level=$level
- eval unset ${pvar}_${level}
- done
-}
-
request(){
for var in $*; do
eval ${var}_requested=yes
--
2.17.1
_______________________________________________
ffmpeg-devel mailing list
[email protected]
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
