Fastest way to generate delimited string from 1d numpy array

Convert the numpy array into a list first. The map operation seems to run faster on a list than on a numpy array.

e.g.

import numpy as np
x = np.random.randn(100000).tolist()
for i in range(100):
    ",".join(map(str, x))

In timing tests I found a consistent 15% speedup for this example

I'll leave others to explain why this might be faster as I have no idea!


A little late, but this is faster for me:

#generate an array with strings
x_arrstr = np.char.mod('%f', x)
#combine to a string
x_str = ",".join(x_arrstr)

Speed up is on my machine about 1.5x


Very good writeup on the performance of various string concatenation techniques in Python: http://www.skymind.com/~ocrow/python_string/

I'm a little surprised that some of the latter approaches perform as well as they do, but looks like you can certainly find something there that will work better for you than what you're doing there.

Fastest method mentioned on the site

Method 6: List comprehensions

def method6():
  return ''.join([`num` for num in xrange(loop_count)])

This method is the shortest. I'll spoil the surprise and tell you it's also the fastest. It's extremely compact, and also pretty understandable. Create a list of numbers using a list comprehension and then join them all together. Couldn't be simpler than that. This is really just an abbreviated version of Method 4, and it consumes pretty much the same amount of memory. It's faster though because we don't have to call the list.append() function each time round the loop.

Tags:

Python

Numpy