What do you get when you complete bonus objectives (with the blue plus on it)?

Completing bonus objectives contributes to the Compl33tionist challenge, which in turn gives you badass ranks.

There are five levels to this challenge, similar to most other challenges:

    Level       Badass ranks gained
---------------------------------------
    Level 1     10  
    Level 2     25
    Level 3     45  
    Level 4     70  
    Level 5     100 

I can say with certainty that completing bonus objectives adds to the cash reward, and was true for each quest i tested.

If you look at the quest menu before completing a bonus objective, take note of the stated cash reward, then go and complete one (or more) bonus objectives. Now return to the quest menu and the new, increased dollar reward amount should have changed.

As far as I've been able to tell, this (along with the “completionist” challenge, and sometimes a bit more dialogue) appears to be the only benefit of completing the bonus objectives. I have never seen any direct experience point increase, or any kind of correlation to the stats of quest reward items. The only other possible benefit may be an increase in drop rates for quest related enemies, but thats just total speculation, and I have not noticed any significant change (and with BL2's drop rates, testing this would be very difficult, and would require literal thousands of test runs).

The 'Dwarven Allies' mission in the 'Tiny Tina's Assault on Dragon Keep' DLC seems to be different from the other cases. Completing the bonus objective (Punch ancient dwarven puzzle) will make you miss a loot reward and access to a locked dice loot chest.