Why does the logical OR operator in switch cases behave strangely?

Because it wasn't intended to be used this way.

|| returns its first operand if it's truthy, else its second operand.

3 || 4 returns 3 because 3 is truthy, therefore case will check only 3:

console.log(3 || 4); //3, because it is truthy
console.log(0 || 1); //1, because 0 is falsy

To make your code work, use separate cases, that fall through:

function testMyNumber(number) {
  switch (number) {
    case 6:
      return number+" is 6";
    case 3:
    case 4:
      return number+" is 3 or 4";

    case 9:
    case 10:
      return number+" is 9 or 10";
    default:
      return number+" is not in 3,4,6,9,10";
  }
};
console.log(testMyNumber(6));
console.log(testMyNumber(3));
console.log(testMyNumber(4));
console.log(testMyNumber(9));
console.log(testMyNumber(10));

Why 4 returns default case?

Because (3 || 4) is equal to 3, so case (3 || 4): is the same as writing case 3:.

You are using incorrect syntax. Write it like this:

case 3:
case 4:
    return number + " is 3 or 4";