Building sparse array from packed arrays
In version 10.1 under Windows I find that a subtle syntax difference is critical to this issue.
The form SparseArray[{a -> b}]
causes unpacking while SparseArray[a -> b]
does not. Further this affects the output of SparseArray
itself, not only its memory consumption during creation.
On["Packing"]
a = {#, # + 1}\[Transpose] & @ Range @ 1*^6;
b = RandomReal[1, 1*^6];
c = SparseArray[a -> b];
Developer`PackedArrayQ /@ {a,b}
MaxMemoryUsed[]
ByteCount[c]
{True, True} 85279216 24000792
In a fresh session the exact same code as above adding only the brackets around a -> b
On["Packing"]
a = {#, # + 1}\[Transpose] & @ Range @ 1*^6;
b = RandomReal[1, 1*^6];
c = SparseArray[{a -> b}]; (* <-------- CRITICAL SYNTAX DIFFERENCE *)
Developer`PackedArrayQ /@ {a,b}
MaxMemoryUsed[]
ByteCount[c]
Developer`FromPackedArray::punpack: Unpacking array with dimensions {1000000,2} in call to Rule. >>
Developer`FromPackedArray::punpack: Unpacking array with dimensions {1000000} in call to Rule. >>
{True, True} 286566696 40000728
The two objects are functionally equivalent despite the size difference:
c1 = SparseArray[a -> b];
c2 = SparseArray[{a -> b}];
ByteCount /@ {c1, c2}
c1 === c2
{24000792, 40000728} True
I surmise that specific optimizations exist in SparseArray
for exactly your application, however the syntax is brittle and the optimization is not triggered for {a -> b}
.