(Java Minecraft 1.14.2) How to recursively (or otherwise) determine if a closed 2D arbitrarily sized rectangle of blocks has been placed?

This was a nice programming challenge. I had fun, learned some things and discovered some Minecraft bugs. Thanks to vdvman1 in the Eigencraft Discord chat for commands help, mainly with edge cases of facing and anchored, for the idea not to use entities at all for raytracing and for the recursion tail optimisation tip.

Here is the complete data pack: https://drive.google.com/file/d/1aw_KfHyEQwtCiWCP4R3H6TYVczmLT1-s

The file structure:

rectangle
└pack.mcmeta
└data
 └rectangle
  ├advancements
  │└place_iron_bar.json
  └functions
   ├init.mcfunction
   ├raycast.mcfunction
   ├search_origin.mcfunction
   ├x_first.mcfunction
   ├z_second.mcfunction
   ├z_first.mcfunction
   └x_second.mcfunction

pack.mcmeta is just the minimum required: {"pack":{"pack_format":5,"description":""}}
You can adjust it to display whatever you want, the format is explained here (archive).

place_iron_bar.json is an advancement that is triggered by placing an iron bar, which calls the init function (which resets the advancement):

{
 "criteria":{
  "place_iron_bar":{
   "trigger":"minecraft:placed_block",
   "conditions":{
    "block":"minecraft:iron_bars"
   }
  }
 },
 "rewards":{
  "function":"rectangle:init"
 }
}

init.mcfunction resets the advancement and then start the recursive raycast function with the correct alignment to your eyes:

#reset so that this doesn't only trigger once
advancement revoke @s only rectangle:place_iron_bar
#double anchor as a workaround for MC-124140
execute anchored eyes positioned ^ ^ ^ anchored feet run function rectangle:raycast

raycast.mcfunction moves the execution position forwards by 0.01 blocks until it hits iron bars, then starts search_origin. If you look very, very closely at the edge of a block when placing the iron bars, the raytracing might miss it, but that's unlikely. You could also intentionally make it miss, for example by standing right at a wall with a torch on it and placing the last iron bar behind you that way. But if you do that… well, then it's your own fault, I guess. It would be possible to just perfectly track every block around you and monitor every single change, but that would permanently cause an immense about of lag for almost no gain.
If the raytracing fails, it will keep going for 327 blocks by default, determined by the maxCommandChainLength gamerule.

execute if block ~ ~ ~ iron_bars run function rectangle:search_origin
execute unless block ~ ~ ~ iron_bars positioned ^ ^ ^.01 run function rectangle:raycast

search_origin.mcfunction is another recursive function (recursion is just the easiest way to make loops in Minecraft), this one goes into the negative X direction as long as it finds iron bars there and into negative Z direction as long as it finds iron bars there. If you have an arrangement like this…

…then it will go to the end of this chain. But since the rectangle search afterwards will fail anyway in this case, that doesn't matter much. The lag it causes it also negligible, I'm actually unable to see any spike in the FPS or TPS graph when placing an iron bar.
Once the point of origin is found, the execution branches off into two functions (which are actually executed strictly after each other, this becomes important later), one goes into the positive X direction first and then the positive Z direction, the other into positive Z direction first and then the positive X direction. There are also some validations for the start of the rectangle, otherwise for example a 1×1 arrangement of iron bars would be considered a rectangle.
In this version of the data pack there is actually still a bug which causes it to not find a rectangle of size 2×3, 2×4, 2×5, etc. 2×2 rectangles are recognised, but nothing that is longer in one direction. Fixing this bug would be complicated, but when I thought about it more, I actually liked this behaviour, because in a 2×3 arrangement, the middle two iron bars actually connect, making it not look like a single rectangle. Example:

#This function traverses a series of iron bars in negative X and Z direction to find the negative corner of a rectangle. If the shape is not a rectangle, it will prefer going in negative X direction over the negative Z direction and just end whereever it can't find another iron bar.
execute unless block ~-1 ~ ~ iron_bars unless block ~ ~ ~-1 iron_bars positioned ~1 ~ ~ if block ~ ~ ~ iron_bars run function rectangle:x_first
execute unless block ~-1 ~ ~ iron_bars unless block ~ ~ ~-1 iron_bars positioned ~ ~ ~1 if block ~ ~ ~ iron_bars run function rectangle:z_first
execute unless block ~ ~ ~1 iron_bars run kill @e[type=armor_stand,tag=z_end]
execute positioned ~-1 ~ ~ if block ~ ~ ~ iron_bars run function rectangle:search_origin
execute unless block ~-1 ~ ~ iron_bars positioned ~ ~ ~-1 if block ~ ~ ~ iron_bars run function rectangle:search_origin

x_first.mcfunction goes into positive X direction as long as it finds iron bars, then starts z_second, if there are iron bars in the positive Z direction. It also checks along the way if there are any iron bars to the side, which invalidate the rectangle. In that case it just stops executing, which will lead to no result at the end.

execute unless block ~1 ~ ~ iron_bars unless block ~ ~ ~-1 iron_bars positioned ~ ~ ~1 if block ~ ~ ~ iron_bars run function rectangle:z_second
execute unless block ~ ~ ~1 iron_bars unless block ~ ~ ~-1 iron_bars positioned ~1 ~ ~ if block ~ ~ ~ iron_bars run function rectangle:x_first

z_second.mcfunction goes into positive Z direction as long as there are iron bars and checks for any on the side that would make the rectangle invalid, then summons a marker armour stand at the end. This is necessary to check if both paths arrive at the same ending location.

Only after x_first and z_second are done, z_first.mcfunction is started. It does the same as x_first, but with X and Z swapped. It also kills the marker armour stand if it encounters something that invalidates the rectangle.

execute unless block ~ ~ ~1 iron_bars unless block ~-1 ~ ~ iron_bars positioned ~1 ~ ~ if block ~ ~ ~ iron_bars run function rectangle:x_second
execute if block ~-1 ~ ~ iron_bars run kill @e[type=armor_stand,tag=z_end]
execute if block ~1 ~ ~ iron_bars if block ~ ~ ~1 iron_bars run kill @e[type=armor_stand,tag=z_end]
execute unless block ~1 ~ ~ iron_bars unless block ~-1 ~ ~ iron_bars positioned ~ ~ ~1 if block ~ ~ ~ iron_bars run function rectangle:z_first

x_second.mcfunction does the same as z_second, but with X and Z swapped and it also kills the marker armour stand if it finds anything that invalidates the rectangle. If everything goes through without problems, it checks if its ending location is the same as the one of z_second, meaning that it arrived at the exact location of the marker armour stand. If it doesn't, it means for example that the positive X/Z corner of the rectangle is missing.

execute unless block ~1 ~ ~ iron_bars unless block ~ ~ ~1 iron_bars if entity @e[type=armor_stand,tag=z_end,distance=0] run say Rectangle found!
execute unless block ~1 ~ ~ iron_bars run kill @e[type=armor_stand,tag=z_end]
execute if block ~ ~ ~1 iron_bars run kill @e[type=armor_stand,tag=z_end]
execute if block ~ ~ ~-1 iron_bars run kill @e[type=armor_stand,tag=z_end]
execute unless block ~ ~ ~1 iron_bars unless block ~ ~ ~-1 iron_bars positioned ~1 ~ ~ if block ~ ~ ~ iron_bars run function rectangle:x_second

You can of course replace say Rectangle found! with anything that should be done if the rectangle is found.
If you need any of the positions, I recommend setting some scoreboard or whatever when the rectangle is found and then checking for it in the different functions after the final function call. Examples:

• If you need the positive corner, just do something instead of say Rectangle found!.
• If you need to do something on the negative Z edge of the rectangle, create some markers in search_origin with the same condition as the x_first call and in x_first anywhere without condition, then use them at the end of init if the rectangle was successfully validated, otherwise kill them there.
• For other cases you might have to invert some of the checks, for example to do it for all but the last execution of a loop.

This datapack should be completely multiplayer compatible, not even two players standing in the exact same spot and placing an iron bar at the exact same time should cause any issue, since all the functions only start for one player once they're done for another player. There are also no reference like @p that could cause any issues, the executioner is always handed over from function to function.
I've also tried many different arrangement of iron bars, like two rectangles that share a corner or an edge, extra bits or missing bits in every possible location, different last placed iron bars, etc. It should hopefully be foolproof.
If the rectangle goes outside of loaded chunks, it will probably just fail as if the iron bars simply weren't there.