Ways to speed up Pick
You can use UnitStep
to construct your selector array:
SeedRandom[1]
a = RandomReal[1, 10^7];
b = RandomReal[1, 10^7];
c = RandomReal[1, 10^7];
sel = MapThread[(Abs[#1] <= 0.1) ∧ (Abs[#2] > 0.9) &, {b, c}]; //
AbsoluteTiming // First
15.332532
d = Pick[a, sel]; // AbsoluteTiming // First
0.605384
sel2 = UnitStep[.1 - Abs[b]](1- UnitStep[.9 - Abs[c]]); // AbsoluteTiming // First
0.630973
d2 = Pick[a, sel2, 1]; // AbsoluteTiming // First
0.015618
d == d2
True
If sel
is already created, you can convert it a packed array. Additional processing time to do this is more than compensated for by speed gains in Pick
:
sel3 = Developer`ToPackedArray[With[{True = 1, False = 0}, Evaluate[sel]]]; //
AbsoluteTiming // First
0.37475
d3 = Pick[a, sel3, 1]; // AbsoluteTiming // First
0.031245
d == d3
True
See this answer by Mr.Wizard for the origin of the With[{True = 1, False = 0}, ...]
trick.
Update: Slightly faster way to construct the selector array (thanks: Fred Simons):
With[{sel3 = UnitStep[.1 - Abs[b]] - UnitStep[0.9 - Abs[c]]},
Pick[a, sel3, 1]]
@kglr already showed how to do this with vectorization, which offers much better performance.
However, the syntax with all those UnitSteps
is not very friendly. My BoolEval package offers much better syntax for the very same operation.
Demo:
a = RandomReal[1, 10^7];
b = RandomReal[1, 10^7];
c = RandomReal[1, 10^7];
sel = MapThread[(Abs[#1] <= 0.1) ∧ (Abs[#2] > 0.9) &, {b, c}]; // AbsoluteTiming
(* {10.2691, Null} *)
d = Pick[a, sel]; // AbsoluteTiming
(* {0.460785, Null} *)
<< BoolEval`
e = BoolPick[a, Abs[b] <= 0.1 && Abs[c] > 0.9]; // AbsoluteTiming
(* {0.186719, Null} *)
e == d
(* True *)
We went from ~11 seconds to only 0.18 s, while shortening the code to a very readable one-liner. I hope this already convinced you that you really need BoolEval ;-)
I should also point out that @kglr's solution actually uses Abs[c] >= 0.9
and not Abs[c] > 0.9
as you requested. To achieve a true >
, we could use 1 - UnitStep[0.9 - Abs[c]]
, which would make the code even less readable. BoolEval takes care of this for you.
One more thing to note:
Pick
is much faster if the second argument is a packed array and the third argument is a simple number (not the default True
).