On a CSS hover event, can I change another div's styling?

Yes, you can do that, but only if #b is after #a in the HTML.

If #b comes immediately after #a: http://jsfiddle.net/u7tYE/

#a:hover + #b {
    background: #ccc
}

<div id="a">Div A</div>
<div id="b">Div B</div>

That's using the adjacent sibling combinator (+).

If there are other elements between #a and #b, you can use this: http://jsfiddle.net/u7tYE/1/

#a:hover ~ #b {
    background: #ccc
}

<div id="a">Div A</div>
<div>random other elements</div>
<div>random other elements</div>
<div>random other elements</div>
<div id="b">Div B</div>

That's using the general sibling combinator (~).

Both + and ~ work in all modern browsers and IE7+

If #b is a descendant of #a, you can simply use #a:hover #b.

ALTERNATIVE: You can use pure CSS to do this by positioning the second element before the first. The first div is first in markup, but positioned to the right or below the second. It will work as if it were a previous sibling.

This can not be done purely with css. This is a behaviour, which affects the styling of the page.

With jquery you can quickly implement the behavior from your question:

$(function() {
  $('#a').hover(function() {
    $('#b').css('background-color', 'yellow');
  }, function() {
    // on mouseout, reset the background colour
    $('#b').css('background-color', '');
  });
});

The following example is based on jQuery but it can be achieved using any JS tool kit or even plain old JS

$(document).ready(function(){
     $("#a").mouseover(function(){
         $("#b").css("background-color", "red");
     });
});