How to remove \n from a list element?
It sounds like you want something like the Perl chomp() function.
That's trivial to do in Python:
def chomp(s):
return s[:-1] if s.endswith('\n') else s
... assuming you're using Python 2.6 or later. Otherwise just use the slightly more verbose:
def chomp(s):
if s.endwith('\n'):
return s[:-1]
else:
return s
If you want to remove all new lines from the end of a string (in the odd case where one might have multiple trailing newlines for some reason):
def chomps(s):
return s.rstrip('\n')
Obviously you should never see such a string returned by any normal Python file object's readline() nor readlines() methods.
I've seen people blindly remove the last characters (using s[:-1] slicing) from the results of file readline() and similar functions. This is a bad idea because it can lead to an error on the last line of the file (in the case where a file ends with anything other than a newline).
At first you might be lulled into a false sense of security when blindly stripping final characters off lines you've read. If you use a normal text editor to create your test suite files you'll have a newline silently added to the end of the last line by most of them. To create a valid test file use code something like:
f = open('sometest.txt', 'w')
f.write('some text')
f.close()
... and then if you re-open that file and use the readline() or readlines() file methods on it you'll find that the text is read without the trailing newline.
This failure to account for text files ending in non-newline characters has plagued many UNIX utilities and scripting languages for many years. It's a stupid corner base bug that creeps into code just often enough to be a pest but not often enough for people to learn from it. We could argue that "text" files without the ultimate newline are "corrupt" or non-standard; and that may be valid for some programming specifications.
However, it's all too easy to ignore corner cases in our coding and have that ignorance bite people who are depending on your code later. As my wife says: when it comes to programming ... practice safe hex!
If you want to remove \n from the last element only, use this:
t[-1] = t[-1].strip()
If you want to remove \n from all the elements, use this:
t = map(lambda s: s.strip(), t)
You might also consider removing \n before splitting the line:
line = line.strip()
# split line...
From Python3 onwards
map no longer returns a list but a mapObject, thus the answer will look something like
>>> map(lambda x:x.strip(),l)
<map object at 0x7f00b1839fd0>
You can read more about it on What’s New In Python 3.0.
map()andfilter()return iterators. If you really need alist, a quick fix is e.g.list(map(...))
So now what are the ways of getting trough this?
Case 1 - The list call over map with a lambda
map returns an iterator. list is a function that can convert an iterator to a list. Hence you will need to wrap a list call around map. So the answer now becomes,
>>> l = ['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3\n']
>>> list(map(lambda x:x.strip(),l))
['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3']
Very good, we get the output. Now we check the amount of time it takes for this piece of code to execute.
$ python3 -m timeit "l = ['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3\n'];list(map(lambda x:x.strip(),l))"
100000 loops, best of 3: 2.22 usec per loop
2.22 microseconds. That is not so bad. But are there more efficient ways?
Case 2 - The list call over map withOUT a lambda
lambda is frowned upon by many in the Python community (including Guido). Apart from that it will greatly reduce the speed of the program. Hence we need to avoid that as much as possible. The toplevel function str.strip. Comes to our aid here.
The map can be re-written without using lambda using str.strip as
>>> list(map(str.strip,l))
['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3']
And now for the times.
$ python3 -m timeit "l = ['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3\n'];list(map(str.strip,l))"
1000000 loops, best of 3: 1.38 usec per loop
Fantastic. You can see the efficiency differences between the two ways. It is nearly 60% faster. Thus the approach without using a lambda is a better choice here.
Case 3 - Following Guidelines, The Regular way
Another important point from What’s New In Python 3.0 is that it advices us to avoid map where possible.
Particularly tricky is
map()invoked for the side effects of the function; the correct transformation is to use a regularforloop (since creating a list would just be wasteful).
So we can solve this problem without a map by using a regular for loop.
The trivial way of solving (the brute-force) would be:-
>>> l = ['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3\n']
>>> final_list = []
>>> for i in l:
... final_list.append(i.strip())
...
>>> final_list
['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3']
The timing setup
def f():
l = ['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3\n']
final_list = []
for i in l:
final_list.append(i.strip())
import timeit
print(min(timeit.repeat("f()","from __main__ import f")))
And the result.
1.5322505849981098
As you can see the brute-force is a bit slower here. But it is definitely more readable to a common programmer than a map clause.
Case 4 - List Comprehensions
A list comprehension here is also possible and is the same as in Python2.
>>> [i.strip() for i in l]
['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3']
Now for the timings:
$ python3 -m timeit "l = ['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3\n'];[i.strip() for i in l]"
1000000 loops, best of 3: 1.28 usec per loop
As you can see the list-comprehension is more effective than map (even that without a lambda). Hence the thumb rule in Python3 is to use a list comprehension instead of map
Case 5 - In-Place mechanisms and Space Efficiency (T-M-T)
A final way is to make the changes in-place within the list itself. This will save a lot of memory space. This can be done using enumerate.
>>> l = ['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3\n']
>>> for i,s in enumerate(l):
... l[i] = s.strip()
...
>>> l
['Name1', '7.3', '6.9', '6.6', '6.6', '6.1', '6.4', '7.3']
The timing result would be 1.4806894720022683. But however this way is space effective.
Conclusion
A comparitive list of timings (Both Python 3.4.3 and Python 3.5.0)
----------------------------------------------------
|Case| method | Py3.4 |Place| Py3.5 |Place|
|----|-----------------|-------|-----|-------|-----|
| 1 | map with lambda | 2.22u | 5 | 2.85u | 5 |
| 2 | map w/o lambda | 1.38u | 2 | 2.00u | 2 |
| 3 | brute-force | 1.53u | 4 | 2.22u | 4 |
| 4 | list comp | 1.28u | 1 | 1.25u | 1 |
| 5 | in-place | 1.48u | 3 | 2.14u | 3 |
----------------------------------------------------
Finally note that the list-comprehension is the best way and the map using lambda is the worst. But again --- ONLY IN PYTHON3