Simplify GeoServer WFS output
WFS/GeoServer does not really provide any way to do generalization on the fly based on scale, but there is a special extension that does what you are looking for by doing some pregeneralization.
http://docs.geoserver.org/stable/en/user/data/featurepregen.html
Using the OL cluster strategy, you can simplify the representation (for users) but the amount of data transfered will not be lower, because the clusterization is done on the client side.
See also, the documentation page.
(I don't know about available options on the server side)
Some possible solutions:
- Do you really need the data in a vector format? If the large complex polygons are just for reference then they'll be fine as WMS / images. If you can simplify them then you probably don't need the geometry anyway.
- You can pass in a generalisation parameter to a map layer through a custom paramter, and use this to simplify geometry. This thread is about MapServer, but I assume something similar can be done with GeoServer - http://osgeo-org.1803224.n2.nabble.com/WFS-Server-Control-output-by-scale-td2279383.html
- Also referenced in the above thread, you can store the datasets presimplified for different scales and hide/show them in your map as needed
- Make sure you have gZip compression turned on at your webserver as this will shrink GML/XML files to at least half the size