When should I choose Vector in Scala?
Well, a List
can be incredibly fast if the algorithm can be implemented solely with ::
, head
and tail
. I had an object lesson of that very recently, when I beat Java's split
by generating a List
instead of an Array
, and couldn't beat that with anything else.
However, List
has a fundamental problem: it doesn't work with parallel algorithms. I cannot split a List
into multiple segments, or concatenate it back, in an efficient manner.
There are other kinds of collections that can handle parallelism much better -- and Vector
is one of them. Vector
also has great locality -- which List
doesn't -- which can be a real plus for some algorithms.
So, all things considered, Vector
is the best choice unless you have specific considerations that make one of the other collections preferable -- for example, you might choose Stream
if you want lazy evaluation and caching (Iterator
is faster but doesn't cache), or List
if the algorithm is naturally implemented with the operations I mentioned.
By the way, it is preferable to use Seq
or IndexedSeq
unless you want a specific piece of API (such as List
's ::
), or even GenSeq
or GenIndexedSeq
if your algorithm can be run in parallel.
As a general rule, default to using Vector
. It’s faster than List
for almost everything and more memory-efficient for larger-than-trivial sized sequences. See this documentation of the relative performance of Vector compared to the other collections. There are some downsides to going with Vector
. Specifically:
- Updates at the head are slower than
List
(though not by as much as you might think)
Another downside before Scala 2.10 was that pattern matching support was better for List
, but this was rectified in 2.10 with generalized +:
and :+
extractors.
There is also a more abstract, algebraic way of approaching this question: what sort of sequence do you conceptually have? Also, what are you conceptually doing with it? If I see a function that returns an Option[A]
, I know that function has some holes in its domain (and is thus partial). We can apply this same logic to collections.
If I have a sequence of type List[A]
, I am effectively asserting two things. First, my algorithm (and data) is entirely stack-structured. Second, I am asserting that the only things I’m going to do with this collection are full, O(n) traversals. These two really go hand-in-hand. Conversely, if I have something of type Vector[A]
, the only thing I am asserting is that my data has a well defined order and a finite length. Thus, the assertions are weaker with Vector
, and this leads to its greater flexibility.
Some of the statements here are confusing or even wrong, especially the idea that immutable.Vector in Scala is anything like an ArrayList. List and Vector are both immutable, persistent (i.e. "cheap to get a modified copy") data structures. There is no reasonable default choice as their might be for mutable data structures, but it rather depends on what your algorithm is doing. List is a singly linked list, while Vector is a base-32 integer trie, i.e. it is a kind of search tree with nodes of degree 32. Using this structure, Vector can provide most common operations reasonably fast, i.e. in O(log_32(n)). That works for prepend, append, update, random access, decomposition in head/tail. Iteration in sequential order is linear. List on the other hand just provides linear iteration and constant time prepend, decomposition in head/tail. Everything else takes in general linear time.
This might look like as if Vector was a good replacement for List in almost all cases, but prepend, decomposition and iteration are often the crucial operations on sequences in a functional program, and the constants of these operations are (much) higher for vector due to its more complicated structure. I made a few measurements, so iteration is about twice as fast for list, prepend is about 100 times faster on lists, decomposition in head/tail is about 10 times faster on lists and generation from a traversable is about 2 times faster for vectors. (This is probably, because Vector can allocate arrays of 32 elements at once when you build it up using a builder instead of prepending or appending elements one by one). Of course all operations that take linear time on lists but effectively constant time on vectors (as random access or append) will be prohibitively slow on large lists.
So which data structure should we use? Basically, there are four common cases:
- We only need to transform sequences by operations like map, filter, fold etc: basically it does not matter, we should program our algorithm generically and might even benefit from accepting parallel sequences. For sequential operations List is probably a bit faster. But you should benchmark it if you have to optimize.
- We need a lot of random access and different updates, so we should use vector, list will be prohibitively slow.
- We operate on lists in a classical functional way, building them by prepending and iterating by recursive decomposition: use list, vector will be slower by a factor 10-100 or more.
- We have an performance critical algorithm that is basically imperative and does a lot of random access on a list, something like in place quick-sort: use an imperative data structure, e.g. ArrayBuffer, locally and copy your data from and to it.