'UCS-2' codec can't encode characters in position 1050-1050
Your data contains characters outside of the Basic Multilingual Plane. Emoji's for example, are outside the BMP, and the window system used by IDLE, Tk, cannot handle such characters.
You could use a translation table to map everything outside of the BMP to the replacement character:
import sys
non_bmp_map = dict.fromkeys(range(0x10000, sys.maxunicode + 1), 0xfffd)
print(x.translate(non_bmp_map))
The non_bmp_map
maps all codepoints outside the BMP (any codepoint higher than 0xFFFF, all the way up to the highest Unicode codepoint your Python version can handle) to U+FFFD REPLACEMENT CHARACTER:
>>> print('This works outside IDLE! \U0001F44D')
This works outside IDLE!
>>> print('This works in IDLE too! \U0001F44D'.translate(non_bmp_map))
This works in IDLE too! �
None of these worked for me but the following does. This assumes that public_tweets was pulled from tweepy api.search
for tweet in public_tweets:
print (tweet.text)
u=tweet.text
u=u.encode('unicode-escape').decode('utf-8')
this unicode issue has been seen in python 3.6 and older versions, to resolve it just upgrade python as python 3.8 and use your code.This error will not come.