Is there some well-known algorithm which turns user's drawings into smoothed shapes?

The Douglas-Peucker algorithm is used in geography (to simplify a GPS track for instance) I guess it could be used here as well.

Based on your description I guess you're looking for a vectorization algorithm. Here are some pointers that might help you:

• https://en.wikipedia.org/wiki/Image_tracing
• http://outliner.codeplex.com/ - open source vectorizer of the edges in the raster pictures.
• http://code.google.com/p/shapelogic/wiki/vectorization - describes different vectorization algorithm implementations
• http://cardhouse.com/computer/vector.htm

There are a lot of resources on vectorization algorithms, I'm sure you'll be able to find something that fits your needs. I don't know how complex these algorithms are to implement them, though,