What does thermal conductivity actually measure?

Thermal conductivity measures the speed at which heat energy travels through material.

That's different to the speed at which changes in temperature travel through material, which is driven by a combination of thermal conductivity and thermal mass.

So, to use your example, concrete has a high thermal conductivity: it will lose heat energy quite quickly, so a hot thing inside a concrete box can cool down quite quickly. However, concrete has high thermal mass: it takes a lot of energy to raise its temperature by 1 Kelvin. So even with heat going into it quickly, its temperature will rise slowly.

That's why concrete and earth walls are used in some passive solar designs: not necessarily for their insulation properties, but for their properties as a heat buffer: they can absorb a lot of heat for relatively low changes in their own temperature, and radiate it back out again. That gives you a wall surface with a fairly steady radiant temperature, which feels a lot more comfortable than a surface with a highly variable radiant temperature; and it gives you a huge buffer that allows you to store solar energy in the day and release it at night, thus giving you cooling during the day when you need it, and heating during the night when you need it.

It represent the speed actually. it is defined as:

"The amount of energy that is transferred from A to B where $AB=1meter$ and difference between the temperature of point A and B is 1 kelvin, in each second."

for example the the thermal conductivity of wood is about 0.4 . it means if you have a wood with a length of 1 meter and and $\Delta(\theta)=1 degree$ (of both end of the wood) then 0.4 joules will be transferred in each second, from one end to another.

One of the best ways to explain concepts like this are to use labelled diagrams, such as

with a very nice explanation from the CBFT blog page, with a nice definition:

When the temperature of one surface of a solid material is higher than another, heat will move through the material. Depending on the characteristics of the material, this conductive heat transfer may be slow or it may occur quickly. The rate of heat transfer is defined by the coefficient of thermal conductivity.

Essentially, thermal conductivity is how fast will heat from its source pass through the material - if the material is thicker, then it will take more time to conduct through.

It's reciprocal is thermal resistance

(not a 'caveman' question at all!)