Easiest way to eliminate NULLs in SELECT DISTINCT?

Try this:

select distinct * from test
where b is not null or a in (
  select a from test
  group by a
  having max(b) is null)

You can get the fiddle here.

Note if you can only have one non-null value in b, this can be simplified to:

select a, max(b) from test
group by a

;WITH CTE
    AS
    (
    SELECT DISTINCT * FROM #test
    )
    SELECT a,b
    FROM CTE        
    ORDER BY CASE WHEN b IS NULL THEN 9999 ELSE b END ; 

Try this:

create table test(
x char(1),
y char(1)
);

insert into test(x,y) values
('a',null),
('a','b'),
('b', null),
('b', null)

Query:

with has_all_y_null as
(
    select x
    from test
    group by x
    having sum(case when y is null then 1 end) = count(x)
)
select distinct x,y from test
where 

    (
        -- if a column has a value in some records but not in others,
        x not in (select x from has_all_y_null) 

        -- I want to throw out the row with NULL
        and y is not null 
    )
    or 
    -- However, if a column has a NULL value for all records, 
    -- I want to preserve that NULL
    (x in (select x from has_all_y_null))

order by x,y

Output:

 X    Y
 A    B
 B    NULL

Live test: http://sqlfiddle.com/#!3/259d6/16

EDIT

Seeing Mosty's answer, I simplified my code:

with has_all_y_null as
(
    select x
    from test
    group by x

    -- having sum(case when y is null then 1 end) = count(x) 
    -- should have thought of this instead of the code above. Mosty's logic is good:
    having max(y) is null
)
select distinct x,y from test
where 
    y is not null
    or 
    (x in (select x from has_all_y_null))
order by x,y

I just prefer CTE approach, it has a more self-documenting logic :-)

You can also put documentation on non-CTE approach, if you are conscious of doing so:

select distinct * from test
where b is not null or a in 
  ( -- has all b null
  select a from test
  group by a
  having max(b) is null)