Why is work done defined the way it is? What's the intuition here?

Assuming a constant force, the time factor doesn't really matter when it comes to Energy.

The integral of $F$ over time gives you the momenta of the bodies, which need not be the same after whatever you've done.

The momenta would be equal if you applied the force over the same time interval, and the energies are same when the force is applied over the same space interval.

Also, remember that the total work done ($\int F.dl$) is the change in kinetic energy of the body the work is done on!

(The total biochemical energy expended is certainly much more in the case of the Elephant, as you're applying the same force over a much longer time. The energy gained by it will be the same as the energy gained by the pen, though.)

Intuitively, it means you imparted 100 J of energy to the object to bring it to that position.

Your confusion arises from two sources (as far as I can tell):

1. You are conflating Work with Power
2. The definition of Work you are using.

Work is always independent of time. It by definition, describes the difference in energy of a system between two states. Take your elephant and pen for example. We'll take the mass of the elephant, $m_e$, to be $1000 kg$ and the mass of the pen, $m_p$, to be $1 kg$. Now, as you stated, you exerted $1 N$ of force of each object for $100 m$. Finally, we'll assume their initial velocity was $0 m/s$. Using kinematics, we may conclude the following:

1. You applied a constant acceleration, $a = \frac{F}{m}$ of to each object. The Elephant accelerated to $\frac{1N}{1000kg} = 10^{-3} m/s^2 $ and the pen accelerated to $\frac{1N}{1kg} = 1 m/s^2 $.
2. At $100m$, each object will have accelerated to $v_f = \sqrt{v_i ^2 + 2 ad} = \sqrt{2ad}$. The Elephant accelerated to $\sqrt{2 \cdot 10^{-3} m/s^2 \cdot 10^2 m} = 0.447 m/s$. The pen accelerated to $\sqrt{2 \cdot 1 m/s^2 \cdot 10^2 m} = 14.1 m/s$.
3. It took $t = \frac{v_f}{a}$ seconds to bring each object to 100m. For the Elephant it took $\frac{0.447 m/s}{10^{-3} m/s^2} = 447s $. For the Pen it took $\frac{14.1 m/s}{1 m/s^2} = 14.1 s $.

Notice: Because the elephant was 100x more massive than the pen, it took significantly more time to push it 100m, and with a constant force it was traveling at one-tenth the velocity as the pen.

Now if we calculate the calculate the difference in kinetic energy to move the pen and elephant 100m: $$T_e = \frac{1}{2} m_e v_{ef}^2 - \frac{1}{2} m_e v_{ei}^2 = 0.5 \cdot 1000kg \cdot (0.447 m/s^2)^2 - 0.5 \cdot 1000kg \cdot (0 m/s^2)^2 = 100 J$$ $$T_p = \frac{1}{2} m_e v_{ef}^2 - \frac{1}{2} m_e v_{ei}^2 = 0.5 \cdot 1 kg \cdot (14.1 m/s^2)^2 - 0.5 \cdot 1 kg \cdot (0 m/s^2)^2 = 100 J$$

You'll derive that the kinetic energy is identical. According to the Work-Energy theorem, the difference in kinetic energy of the system is equal to the total work done on the system.

Still confused? Take a look at the power used to move the objects:

$$P = \frac{W}{t} = \frac{Fd}{t} = Fv $$

You'll find that the power used to move the elephant was 0.224 Watts and the power used to move the pen was 7 Watts. You still expended the same amount of energy for either scenario; but, you expended energy at a faster rate pushing the pen, than pushing the elephant. This may seam counter-intuitive; but, remember, you exerted 1 N of force on a friction-less surface. On surfaces with friction, we must exert enough force to overcome static friction before the object will move.

EDIT: corrected erroneous calculations.