Does this design smell bad? Can I pull a pin simultaneously down and up?
Yes, it smells. Use a transistor. This is the normal way to invert a signal. The OE input is a sourcing input. These are designed to be used with a switch which is either open or switched to common. It's not meant to have Vcc connected directly to the input! You have a switch (your header), which would be perfect, but you want the behaviour inverted - so invert it with a transistor. That's why it's called TTL. The resistor to ground also increases the noise sensitivity of the circuit needlessly.
Here when the header is open the base of the transistor is high and OE is pulled low. When the header shorts the transistor turns off and OE goes high via the internal pullup. I've shown 10k to the transistor base here, which is a bit greedy for power - a lot of values would work here, though. The higher you go the less power it draws, but the more sensitive to noise you become.
If you can spare the current and want a more noise resistant circuit you can also tie OE to Vcc with a parallel pullup (Rp) to the internal 270k. Absent a compelling reason not to, giving the input a lower pullup resistor here is probably a good idea.
Well, you can't really remove that 270K pull-up, so that just means you have to use a significantly smaller (stronger) pull-down. Also, as an on-chip resistor, the precise value of that pull-up is not going to be very well controlled and could vary by quite a bit. I would recommend going even smaller on the pull-down, perhaps 10k or even 4.7k or 1k.
Your 40.2k pulldown is probably fine.
The datasheet, 'DC Electrical Characteristics' table on page 2 gives you all the info you need for this.
The VIL line tells you that 0.8V is the maximum value which the OE pin will recognize as being 'low'.
The R line tells you that the OE pin has a 270k Pullup.
You know you have a 3.3V supply, so with this info it's possible to calculate the maximum value pulldown resistor you can use and still have the input recognized as pulled low - and that value is 86.4k.
So since your 40.2k is less than half that you're well into the 'safe' range (you should expect about 0.43V).
The only other thing you might want to consider is putting a cap on that pin (since you're connecting it to a header and that might pick up some noise). I'd probably put a 100n there.