Does hanging up on a UK landline call not terminate the connection?

This is because of Called Subscriber Held (CSH). This is not specific to telephony in the UK but rather a line state applicable on PSTNs caused by the person who made the call not hanging up.

The person from which the call originates must hang up for the call to disconnect as it is the person from which the call originates that is paying for the bill. The person receiving the call may hang up, but this will not disconnect the call unless the originating caller also hangs up. Usually once a CSH condition is detected, a timer will start which will clear the call after a specified period of time (for example 3 minutes).

Back in ye olden days when people had phone handsets connected to base stations with a wire(!) this feature meant they could put the phone down and move to another room without it dropping the call. :)

Using the recall (R) button on most phones will put any current call into an on hold state and give you a dial tone.

To add to the original answers (and consolidate some comments):

Analog exchanges (certainly (*1) Strowger exchanges in the UK, probably others elsewhere) did not permit the called party to clear the line (hang up). My understanding of the original reason for this is that the calling party was paying the bill, and there was no effective signalling (pre digital exchanges) up the line (from called party to calling party) to terminate the billing. Remember on long distance calls these calls were patched through manually. This could be used (and relied on) by those receiving calls, e.g. to hang up and pick up the phone elsewhere. I believe this to be the case universally in original analog exchanges (i.e. in every country). This feature is called Called Subscriber Held (CSH).

When digital exchanges came in in the UK, some people had come to rely on being able to hang up and pick up the phone elsewhere, and BT (well, the GPO as it was then) maintained this feature. All modern exchanges have a configuration knob or two ("Called Party Clear" and "Called Party Clear Timeout") which determines whether, and after how long, the called party can clear the call. BT have in recent times set this knob to 3 minutes. This knob has been a feature of System X, System Y (aka Eriksson AXE10) and 21CN phone exchanges. BT use a much shorter timeout on POTS lines configured for analog PABXs. The called party can clear automatically on ISDN2 and ISDN30, and also on mobile and VoIP.

This is described in full in BT SIN 351 ("Technical Characteristics Of The Single Analogue Line Interface") under section 7.1.2:

7.1.2. [Call Clearing] By The Called Terminal

When a call is ended by the called terminal, the BT network interface will detect an off-line condition (see section 3.1 Off-line d.c. Condition) and initiate a time-out process lasting between two seconds and three minutes. After the time-out period has expired, network initiated clearing (see section 7.2 Network Initiated Clearing) is provided to the calling terminal.

Calls that are made to certain services (e.g. Number translation services and Premium rate services) are subject to first party clearing. In these circumstances, when the called terminal ends the call there is no time-out process and the calling terminal is provided with network initiated clearing (see section 7.2 Network Initiated Clearing) immediately.

In other countries, the same knobs are available. The removal of this 'feature' (or introduction of such a safeguard) would have been up to the operator. Eire has a similar telephone network to the UK, as do various current or former British dependencies (another answerer says this is/was the case in Canada), so I would guess their incumbent operators may share the same configuration. However, various cable operators in the UK do not share this configuration. Historically, I believe this configuration has been used in the US (as this patent) would suggest, and was present on Strowger exchanges (see here (PDF) and here).

In March 2014 BT announced it was drastically reducing the time for Called Party Clear. You can find the announcement here (and the text quoted above is post this announcement):

Here are some extracts:

There are potential problems and the risk of fraud when the called party replaces their handset to end a call but the calling party does not. Currently, in this situation, the network will wait between 2 and 3 minutes before initiating call clearing. During this time, the calling party is still connected to the called party. If the called party picks up their handset within the timeout period, they will still be connected to the calling party. Such a feature has always been available on analogue lines to allow the called party to hang up and subsequently re-answer the call for instance when moving from one extension to another. However, this feature has of late been exploited by fraudsters who hold the line open.

...

It is planned to roll out the proposed changes using a phased approach across the BT network, starting with the AXE10 exchanges which equate to around one third of the local exchanges currently in service or approximately 6 million exchange lines. It is intended to commence the rollout early April to change the AXE10 configuration for call clearing to 10 seconds with a target completion date of 10 April 2014. Further information regarding the timeline for implementing the same changes to System X and UXD5 exchanges will be made available in due course.

Note AXE10 is for this purpose a synonym to System Y. Also note AXE10 exchanges are deployed throughout Europe; the setting of this feature is (as previously indicated) a matter for the service provider.

So, the answers to the questions are:

How does this work?

The caller waits for the called party to hang up, plays a dummy dial tone that disappears when DTMF is received, hopes the called party does not hang up for more than the relevant timeout, and after a few digits have been dialled plays a ringing tone. I imagine Asterisk is eminently suitable for this task.

Why can't the telephone network fix this?

They can, and in the case of BT are fixing it. Other operators may not need to fix it as they may not offer this 'back compatible' feature.

Does the scammer require specialized equipment or does this work from any landline phone?

You do not need specialised equipment. However, you do need to dial someone who is on a landline configured for it, and hope they hang up for less than the relevant delay. May people may hang up for less than 10 seconds (which is the new standard it would appear).

Note this technique is not only much loved by fraudsters, but (when the delay was longer) was often used by journalists who, having contacted someone for a 'scoop', would leave the phone off the hook to prevent their competitors phoning the same person whilst they raced around to do an interview.

*1 = it has been suggested that even Strowger exchanges supported this, evidence of which I would be interested in. My research (see here) suggests that in the original Strowger configuration, it merely lit an alarm lamp. Even if this is correct, this is not to detract from the fact that at least some earlier exchanges did not disconnect for a considerable time after the called party hang up.

It used to be a thing, at least in Canada, that the caller could hold on to the line and the callee would be locked into that circuit with the caller. One could prank a callee by staying on the line, waiting for the callee to attempt another call, and carry on a new conversation with the callee pretending to be who the callee intended to speak with.

This was fixed a very long time ago when the telco lines and equipment changed to digital communications.