Why are Edison screws allowed to be used?

The Edison lamp base design dates predates all twentieth-century safety regulations (because it predates the twentieth century altogether). Having light bulbs screw in and out is not great in environments where vibration is a problem, and replacing a bulb which has just burned out while in use may require use of a cloth to hold it, but it is generally advantageous to have a lamp which can be inserted or removed by handling the bulb rather than having to mess with the holder. Making a metal thread assembly that can mate reliably with a metal socket is easier than trying to mold threads into glass, and having one contact in the center of the socket is easier than trying to have two concentric contacts.

While I doubt the Edison base would be approved by any safety agency if it were being introduced as an entirely new product, it has been effectively "grandfathered in" because it has been used for a long time, people are familiar with it. A "safer" design that people aren't familiar with might lead to more accidents than the century-old design which, for all its imperfections, is well understood.

Here's your opportunity. The market's looking for that right now so build a better mousetrap.

The USDOE and California CEC want to murder the Edison base to finally stop people from using incandescent bulbs, and enable fixture designs that don't have to worry so much about dissipating heat. They mandated GU24 in 2008, which solves some of your concerns. Take a look at how that's going 8 years later. LOL.

There are several flaws in the GU24 that you should address in your new design.

• Ease of installing "blind" when you just can't see the socket or it's deep in a recess.
• Equipment Grounding Conductor.
• 3-way lamp support.
• Or since dinosaurs called and want their dual-filament bulbs back... how about a standard for a signal pin and protocol to command the bulb to "dim". In track lighting, the signal line could be bussed to each outlet and controlled by a single dimmer.
• Multi-voltage, either standardize that all bulbs must be multi-voltage, or have different keying for 120V, 220-240V and 277V.

Good luck!

Nobody answered why were they designed that way?

I recall it was for practicality and cost of manufacturing.

I heard about it indirectly, on a TV show about Tesla. He was contracted to provide lights as Edison's competitor, but could not use Edison's patented sealing/base method and had to use his own which lacked the advantages.

Trying to find more about it, I think the point works like this: you seal the glass capsule and shape it like a bottle top easily enough. The fitting needs to slip over that, and be simple and cheap to make. The contacts along the outside surface of the simplest cap shape is the simplest. Naturally the contacts will be co-axial, and you need at minimum 2 metal areas separated by insulation. Making one of those parts be the thread as well saves components and joints. It also becomes easy to connect the emerging wires to the cap without any alignment.

In short, it's the simplest possible fitting to manufacture.

From the link above, the author quotes Alan Makkos,

The familiar screw-in base was a whole other can of worms. Edison’s first bulbs slipped into their sockets without a way to secure them, until he was struck by a utilitarian design while having lunch in his workshop. “Edison saw a can of kerosene on a shelf, and said, ‘Oh, the lid for that kerosene would make a dandy screw base for a light bulb.’ So they got the can down, cut the lid off with a band saw, and made a light-bulb socket out of it,” says Jenkins. “By the time they were in production in 1885, they had reduced the size of the base quite a bit, and it looked a lot more like modern bulbs.”