PySerial non-blocking read loop

I would warn against using blocking IO in a thread. Remember Python has a GIL and at one time only one thread can execute. Now please note that pyserial module is a wrapper over an OS implementation of accessing the serial port. That means it calls code external to the Python. If that code blocks, then the interpreter also get blocked and nothing will execute in the Python program, even the main thread.

This can even happen when using non-blocking IO or timeout based polling if the underlying device driver does not implement timeout well.

A more robust approach is to use multiprocessing module with a queue. Run serial read code in a separate process. This will make sure main and other threads don't block and the program can exit in clean way.

Put it in a separate thread, for example:

import threading
import serial

connected = False
port = 'COM4'
baud = 9600

serial_port = serial.Serial(port, baud, timeout=0)

def handle_data(data):
    print(data)

def read_from_port(ser):
    while not connected:
        #serin = ser.read()
        connected = True

        while True:
           print("test")
           reading = ser.readline().decode()
           handle_data(reading)

thread = threading.Thread(target=read_from_port, args=(serial_port,))
thread.start()

http://docs.python.org/3/library/threading

Using a separate thread is totally unnecessary. Just do this for your infinite while loop instead (Tested in Python 3.2.3). I use this technique in my eRCaGuy_PyTerm serial terminal program here (search the code for inWaiting() or in_waiting).

import serial
import time # Optional (required if using time.sleep() below)

ser = serial.Serial(port='COM4', baudrate=9600)

while (True):
    # Check if incoming bytes are waiting to be read from the serial input 
    # buffer.
    # NB: for PySerial v3.0 or later, use property `in_waiting` instead of
    # function `inWaiting()` below!
    if (ser.inWaiting() > 0):
        # read the bytes and convert from binary array to ASCII
        data_str = ser.read(ser.inWaiting()).decode('ascii') 
        # print the incoming string without putting a new-line
        # ('\n') automatically after every print()
        print(data_str, end='') 

    # Put the rest of your code you want here
    
    # Optional, but recommended: sleep 10 ms (0.01 sec) once per loop to let 
    # other threads on your PC run during this time. 
    time.sleep(0.01) 

This way you only read and print if something is there. You said, "Ideally I should be able to read serial data only when it's available." This is exactly what the code above does. If nothing is available to read, it skips on to the rest of your code in the while loop. Totally non-blocking.

(This answer originally posted & debugged here: Python 3 non-blocking read with pySerial (Cannot get pySerial's "in_waiting" property to work))

pySerial documentation: http://pyserial.readthedocs.io/en/latest/pyserial_api.html

UPDATE:

• 27 Dec. 2018: added comment about in_waiting vs inWaiting(). Thanks to @FurkanTürkal for pointing that out in the comments below. See documentation here: https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.in_waiting.
• 27 Oct. 2018: Add sleep to let other threads run.
• Documentation: https://docs.python.org/3/library/time.html#time.sleep
• Thanks to @RufusV2 for bringing this point up in the comments.

Note on multi-threading:

Even though reading serial data, as shown above, does not require using multiple threads, reading keyboard input in a non-blocking manner does. Therefore, to accomplish non-blocking keyboard input reading, I've written this answer: How to read keyboard-input?.

References:

1. Official pySerial serial.Serial() class API - https://pyserial.readthedocs.io/en/latest/pyserial_api.html