Cleaning blocks pattern from urban LIDAR (blocks uplifted from streets)

If you are open to using alternative software to solve your problem, then I can suggest the Remove Off-Terrain Objects tool of the cross-platform open-source GIS Whitebox Geospatial Analysis Tools (of which I am lead developer). I realize that you said in your question that you could not convert your data to LAS format, but the tool takes a raster, not LAS file, as the input. I think you were perhaps mixing this tool up with the Bare-Earth DEM tool, which does take a LAS input. You can import your raster DEM into Whitebox as a GeoTIFF file or an ArcGIS floating point binary raster (.flt) or any number of other common raster formats.

Here is another example of it's ability to remove buildings from a raster DEM:

enter image description here

enter image description here

Importantly, the algorithm is not a filter and therefore will not impact the elevations of grid cells outside of 'off-terrain objects' (buildings). This is critical if you want to use your bare-Earth DEM to calculate the wetness index. (That said, I'd question the usefulness of estimating the wetness index of an urban or dense suburban location. Furthermore, there is no stream network through the dense urban landscape you show in your image...I'm certain most of the streams have been culverted.)

EDIT

Actually, looking at the sample dataset that you uploaded, I don't know if your DEM is suited to use with the Remove Off-terrain Objects tool. I had thought that the image that you uploaded was simply suffering from poor symbology, but I see now that it is in fact terraced (i.e. there are large flat areas steps within the DEM). Take a look at the following profile from your DEM:

enter image description here

The tool relies on accurate measures of slope (which I don't think are possible with your DEM...this should be flagged up if you want to calculate TWI as well) and the preservation of sharp slope boundaries between the ground surface and the buildings. But in your case the DEM is severely smoothed and these sharp edges aren't as apparent (your houses also appear to be about 2-3 m in height, which is a bit odd). Are you able to get your hands on the original unprocessed LiDAR DEM, or better yet, the point cloud data? For your application, I would seriously consider re-interpolating the DEM.


I would have to say that without the original LAS point cloud, you will only be incorporating more inaccuracies into the data through raster manipulation. The DEM provided looks relatively clean for a heavily urbanized 1m resolution DEM. The "uplifted squares" are a result of triangulations across data voids, where the buildings are not included in the final ground class. You may also want to consider that water will not flow across a bare earth DEM in the real world as it does in your model. The buildings will present boundaries to actual hydrographic processes.