Is removing the Ethernet cable from the router (when I'm not using it) a good security measure?

If there is not an internet connection to your device then a hacker is not going to be able to communicate with that device. (Edit: As some have pointed out...this is assuming an attacker is attempting over the internet from a remote location)

With that said, eventually you will have to connect to the internet again if you want to use the internet and if you were to eventually obtain malware on your computer such as a keylogger. That keylogger is going to rely on the internet to send its data back to the hacker. If the keylogger is written properly, when you disconnect, it will just wait for you to connect to the internet again to send its data back to the attacker.

In my opinion, I think disconnecting from your internet will prove to be more of a hassle than a protection. Instead, focus on the security of your device and your actions on the internet. Being a smart internet user can provide a great deal of security to your device.

Elaboration (EDIT):

I do agree that this method with decrease the time of opportunity for an attacker but the reason I chose to put emphasis on endpoint security and user education is because if you imagine an enterprise environment, they have devices and services that rely on an internet connection 24/7. So an enterprise can't rely on disconnecting from the internet as a viable security measure. Instead they focus on securing the devices on the network and the network itself. So I believe this will achieve 2 things: 1) greater security. 2) better user experience(always have internet access on demand) and I believe you can apply these strategies to your personal network as well.

This would reduce your risk by minimizing the time the attack surface is vulnerable to attack so yes technically it is a helpful security control. It falls into the category of Layer 1 access control in the OSI model.

This said you are also losing data created by attackers at night which could be useful for trending attackers activity. It might still be possible to collect this while physically unplugging the network but if you aren't collecting this type of data currently you aren't losing anything by doing this.

More importantly, it sounds like you don't have enough security controls in place to trust what's going on with your network so you still need to address that at some point too.

As a basic security control, it does lend itself to a reduction in attacks being able to access your systems especially ones that are tested against large blocks of IP's or the Internet as a whole which may be run during the time your systems are disconnected. So yes there is some benefit and if this benefit outweighs any downside it sounds like it would be a good security measure in your situation. Keep in mind that this will have almost no impact on a persistent attacker it mainly helps with reducing the number of large-scale attacks, large-scale reconnaissance efforts, or when things like worms occur on the Internet.

Note: Your router will still be vulnerable to attack in the scenario you mention. I'm assuming this is because some other equipment needs access but if not you might think about disconnecting the WAN cable or even turning off the router itself. This may not be an option but I just thought I'd mention this given your scenario.

Something else you may want to consider as an alternative option is scripting your firewall rules, or system(s), to disable their interfaces on a set schedule or via a simple script you run to turn things on and off.

Given how many attacks are aimed at routers, the number of security flaws / backdoors in consumer grade routers, and the fact your router is an always-on gateway to the big bad internet, I'd say powering the router down is easier and a more effective measure than disconnecting your PC from it. You're moving the (debatable) security benefit one step further up the chain.

You could even put your router on a timer-socket or one of those remote-control mains sockets for laziness.

I won't speculate on how much real-world benefit this might give as it's subject to so many variables and your own personal paranoia level.