Why would the height increase with a smaller font size?

Since there are already two answers that explain well why the height is increased, to quickly fix this problem you simply need to remove the units in line-height.

.lorem, .ipsum, .dolor, .sit {
    line-height:3; border:1px solid green;
}

According to MDN

The line-height CSS property sets the height of a line box. It's commonly used to set the distance between lines of text. On block-level elements, it specifies the minimum height of line boxes within the element. On non-replaced inline elements, it specifies the height that is used to calculate line box height.

Values
normal Depends on the user agent. Desktop browsers (including Firefox) use a default value of roughly 1.2, depending on the element's font-family.

number (unitless) The used value is this unitless number multiplied by the element's own font size. The computed value is the same as the specified number. In most cases, this is the preferred way to set line-height and avoid unexpected results due to inheritance.

length The specified length is used in the calculation of the line box height. Values given in em units may produce unexpected results.

percentage Relative to the font size of the element itself. The computed value is this percentage multiplied by the element's computed font size. Percentage values may produce unexpected results.

So basically your question is one of the cases of unexpected results due to inheritance.

.container {
    display:inline-block;
}
.lorem, .ipsum, .dolor, .sit {
    line-height:3; border:1px solid green;
}
.ipsum:before {
    content:'world!';
}
.sit:before {
    font-size:.6rem;
    content:'world!';
}

<div class="container">
    <div class="lorem">Hello</div>
</div>
<div class="container">
    <div class="ipsum"></div>
</div>
<hr style="clear:both"/>
<div class="container">
    <div class="dolor">Hello</div>
</div>
<div class="container">
    <div class="sit"></div>
</div>

Edit: This question has had quite a number of new eyeballs lately, so here's an update to make it more useful.

Alohci's solution is correct, but it may not be absolutely clear for the more graphically-inclined.

So allow me to clarify the solution a bit, with pictures.

First, the line-height is inherited as its calculated size, so although it's specified in em units, children will inherit value in pixels. For example, with a font size of 20px and a line height of 3em, the line height will be 60 pixels, even for descendants with different font sizes (unless they specify their own line heights).

Now let's assume a font with a 1/4 descender. That is, if you have a 20px font, the descender is 5 pixels and the ascender 15 pixels. The remaining line-height (in this case, 40 pixels) is then divided equally above and below the baseline, like this.

For the block with the smaller font (0.6em or 12 pixels), the remaining amount of line-height is 60-12 or 48 pixels, which also gets divided equally: 24 above and 24 below the baseline.

Then if we combine the two fonts on the same baseline, you will see that the line heights are not divided in the same way, so the total height of the containing block increases, even though both line heights are 60 pixels.

Hope this explains things!

The height of the .lorem, .ipsum, .dolor, and .sit boxes is each the height of the single line box that they contain.

The height of each line box is the maximum of the height above the baseline + the maximum height below the baseline of the strut of the line and the text in the line. since the strut and the text are aligned on the baseline.

For clarity, heights below in em, refer to the font size of the overall container (i.e. the body element)

In .ipsum, (where the font size is 1em) the height above the baseline is 1em (the upper half-leading) + 13/16em (the ascender, approx) for both the strut and the text, and the height below the baseline is 1em (the half-leading) + 3/16em (the descender, approx) + 1em (the lower half-leading) making a total of 3em.

In .sit (where the font size is 0.6em) the height above the baseline is the maximum of [1em (the upper half-leading) + 13/16em (the ascender, approx) for the strut] and [1.2em (the upper half-leading) + 0.6 x 13/16em (the ascender, approx) for the text], and the height below the baseline is the maximum of [1em (the lower half-leading) + 3/16em (the descender, approx) for the strut] and [1.2em (the lower half-leading) + 0.6 x 3/16em (the descender, approx) for the text].

Evaluating that and converting to decimal gives 1.8125em above the baseline and 1.3125em below the baseline making a total of 3.125em, which is larger that the 3em of .ipsum.