What tools were used to analyze & challenge the records for Dragster?

They're describing tool-assisted speedrunning, aka TASing. A TAS uses an emulator, allowing a player to make frame-perfect button inputs, simulating absolutely perfect play, and allowing any errors to be rewound and redone quickly.

In the case of Dragster, we already know that perfect, down to the individual frame level (shifting and accelerating at the exact best possible moment), results in a time of 5.57.

The 5.51 time submitted by the player in question has no known way to reproduce, has never been able to be reproduced by anyone even with a TAS. Omnigamer has provided a spreadsheet indicating exhaustively what different combinations of input results in which times.

The "tools" they mean is something that can be used to analyze the game. There are probably a few ways to do this, but one video here discusses using a RAM analyzer for the Atari 2600. What it does is when the CPU writes to the RAM, the RAM analyzer will detect this, and copy the information into a different memory array that they can look at further. They then use a microcontroller that examines the RAM, and based off the information in the memory, it can make decisions on how to play the game.

A second video shows just this. By tweaking their settings for the "simulation" if you will, they can run the game using specific timing for when to change gear in the dragster, control acceleration, etc. The reason this world record is so controversial is that even when using the most ideal computer simulated run of the game, it was still not able to tie or beat the world record run by Todd Rogers.