How to compare mp3, flac audio data in a file, ignoring header data (ID3 tag) etc.?

Ah, the eternal plight. I myself struggled with this very question for so long and tried so many duplicate-file-finding apps that I eventually gave up and decided to write one myself. And then I found AllDup.

AllDup made me indefinitely back-burner my own project because it is a fast DFF that has the ability to compare MP3 and JPEG files, ignoring their ID3 tags and Exif data respectively. Even better, Michael Thummerer is very responsive to feedback and is quick to fix bugs and implement suggestions (you can suggest ignoring FLAC headers). To top it all off, AllDup is free.

Here's a way to do it at the shell. You need avconv, which in Debian/Ubuntu is in libav-tools.

$ avconv -i INPUT_FILE -c:a copy -f crc - 2>/dev/null | grep CRC

You'll get a line like this:

CRC=0xabfdfe10

This will compare every frame of audio data and generate a CRC for it. So a command like this can compare multiple files:

ls *.mp3 | while read line; do echo -n "$line: "; avconv -i "$line" -f crc - 2>/dev/null | grep CRC; done

Foobar2000 with the Binary Comparator plugin will do this.