use basename in find -exec?

Try:

find -iname "*.txt" -exec sh -c 'for f do basename -- "$f" .txt;done' sh {} +

Your first command failed, because $(...) run in subshell, which treat {} as literal. so basename {} .txt return {}, your find became:

find . -iname "*.txt" -exec echo {} \;

which print file name matched.

Instead, there is another way by using bash pipelines and xargs command:

    find . -iname '*.txt' | xargs -L1 -I{} basename "{}"

In the xargs command above:

The -L1 argument means executing the output of the find command line by line.

The -I{} argument is intended to wrap the output of the find command with double quotes to avoid bash word splitting (when filenames contain whitespace).

You can still use $() for command substitution if you use sh -c and single quotes:

find . -iname "*.txt" -exec sh -c 'echo "$(basename {} .txt)"' \;

The single quotes prevent the main shell from executing the sub-command inside $() so it can instead be executed by sh -c after the find command has replaced {} with the file name. See this answer on Stack Overflow for a more thorough explanation.

Note that you can also add double quotes inside the $() to handle spaces in file names:

find . -iname "*.txt" -exec sh -c 'echo "$(basename "{}" .txt)"' \;