At what distance (from the player) do iron golems spawn in villages?
This problem is caused by chunk unloading. Once you are far, the chunk with the spawner in it is unloaded from memory thus unable to provide you with iron golems or their drops. Try to shift your farm to spawn chunk and check if it starts working, as spawn chunks always remain loaded in memory.
This means, for instance, that redstone machines and iron golem farms will only work if a player is nearby. This can make effective farming difficult when the player spends a lot of time exploring or in another dimension. Since spawn chunks do not get unloaded, they continue to process events, even when no players are nearby.
From Minecraft Wikia