how to tokenize a string in python code example

Example 1: how to split a string by character in python

def split(word): 
    return [char for char in word]  
      
# Driver code 
word = 'geeks'
print(split(word)) 

#Output ['g', 'e', 'e', 'k', 's']

Example 2: separate a string in python

string = "A B C D"
string2 = "E-F-G-H"

# the split() function will return a list
stringlist = string.split()
# if you give no arguments, it will separate by whitespaces by default
# ["A", "B", "C", "D"]

stringlist2 = string2.split("-", 3)
# you can specify the maximum amount of elements the split() function will output
# ["E", "F", "G"]

Example 3: split string python

file='/home/folder/subfolder/my_file.txt'
file_name=file.split('/')[-1].split('.')[0]