How do I set a default choice in jenkins pipeline?

You can't specify a default value in the option. According to the documentation for the choice input, the first option will be the default.

The potential choices, one per line. The value on the first line will be the default.

You can see this in the documentation source, and also how it is invoked in the source code.

return new StringParameterValue(
  getName(), 
  defaultValue == null ? choices.get(0) : defaultValue, getDescription()
);

As stated by mkobit it doesn't seem possible with the defaultValue parameter, instead I reordered the list of choices based on the previous pick

defaultChoices = ["foo", "bar", "baz"]
choices = createChoicesWithPreviousChoice(defaultChoices, "${params.CHOICE}")

properties([
    parameters([
        choice(name: "CHOICE", choices: choices.join("\n"))
    ])   
])


node {
    stage('stuff') {
        sh("echo ${params.CHOICE}")
    }
}

List createChoicesWithPreviousChoice(List defaultChoices, String previousChoice) {
    if (previousChoice == null) {
       return defaultChoices
    }
    choices = defaultChoices.minus(previousChoice)
    choices.add(0, previousChoice)
    return choices
}