cursor sometimes changes from a vertical line to an underscore in Rstudio

Right click and select Reload.

press ins button or fn+ins to solve this problem underscore cursor means you are on overwrite mode

Press the Insert button on your keyboard.

This is a very common operation in different software and platforms. The underscore means that you are in Overwrite mode, meaning that anything you type will overwrite the text at the cursor position.

The "normal" cursor indicates that you are in insert mode and all text you type will force the rest of the line to move further.

Switching between these modes happens by pressing the Insert key on your keyboard (usually there are two of them, one somewhere above of the arrow keys and another on the Num Pad).