wp_nav_menu change sub-menu class name?

There is no option for this, but you can extend the 'walker' object that WordPress uses to create the menu HTML. Only one method needs to be overridden:

class My_Walker_Nav_Menu extends Walker_Nav_Menu {
  function start_lvl(&$output, $depth) {
    $indent = str_repeat("\t", $depth);
    $output .= "\n$indent<ul class=\"my-sub-menu\">\n";
  }
}

Then you just pass an instance of your walker as an argument to wp_nav_menu like so:

'walker' => new My_Walker_Nav_Menu()

This is an old question and I'm not sure if the solution I'm going to mention was available by the time you asked, but I think it's worth mentioning. You can achieve what you want by adding a filter to nav_menu_submenu_css_class. See the example below - you can replace my-new-submenu-class by the class(es) you want:

function my_nav_menu_submenu_css_class( $classes ) {
    $classes[] = 'my-new-submenu-class';
    return $classes;
}
add_filter( 'nav_menu_submenu_css_class', 'my_nav_menu_submenu_css_class' );