Analog isolation vs Digital Isolation
Any reason not to just use a miniature PCB mount mains transformer at the input of the thing and measure the secondary? Instant isolation, no need for power on the non iso side of the barrier, readily available with any safety certification you like and cheap....
Now if you are trying to measure detailed harmonics or fast transient events this may not cut it, but for many applications a miniature PCB mount mains transformer is entirely appropriate, a modest load resistor on the secondary will help with performance in most cases.
As to the digital isolation, depending on the speed you need, a jellybean 6N137 will do 30Kbaud or so, and there are some very small micros these days to encode the ADC output into something async that will pass the opto...
On the analogue side, do not discount things like the IL300, old school but still very useful.
And yea, chip vendors tend to publish white papers favouring their solutions, in much the same was as the marketing department write the front page of the datasheet....
Given the ridiculously low prices of MCUs with decent on-board ADCs, the simplest way to do it would be to do digital isolation between MCUs. Simply treat a small single-chip microcontroller as a "custom" ADC and digital modulator in one. That way you have full flexibility in choosing the format of the isolated digital signal to match your requirements.
E.g. if you only need to measure the RMS value, you can run the ADC at maximum speed and calculate the average between zero crossings, and send only that value to the isolated side on a slow-speed link, using a cheap optocoupler driven directly from the MCU. The MCU and the optocoupler have very modest power consumption requirements, thus you can power them directly from the mains line you're monitoring using a series capacitor, resistor and a cheap shunt voltage regulator, perhaps just a Zener diode would do. The ballpark price of the high-voltage side could be just a bit over $10 in qty 100 or thereabouts.
As a general rule of thumb, AC mains voltage should always be isolated by a transformer or some type of isolation device. There should also be some way to limit both current (fuse) and voltage (TVS or MOV) on each line.
Depending on your plans for this application and where you intend to sell it, you will likely have to go through some type of UL or CE certification. Take a look at IEC 61010. That document covers many of the basic tests required.
When making decisions about required isolation, you have to consider the environment(lab bench, outside, can it get wet?), the enclosure, clearance distances, and what happens when things go wrong at a minimum. Say a resistor or diode in the design fails short, does your design start a fire? If something did catch fire, could the fire spread out of the box? Could someone be shocked by touching the box if a wire came loose? These are things you have to think about before trying to measure mains voltage. They also determine the amount of isolation required.
To answer the original question which was about cost. I've tried both approaches. Both work but I usually prefer digital isolation. In older products, isolation was usually done in analog space. The front end was typically a differential op amp with several large resistors in series with the inputs with protection diodes followed by an ISO124 for galvanic isolation. An ISO124 is not a cheap part ($19.85) but it is tried and true. It does have limitations such as bandwidth and offset voltage that have to considered. It is also bipolar. Many MCU AtoD's cannot handle any voltage below 0V. Since you are measuring an AC signal, you may have to use a seperate AtoD.
The really cheap isolation op amps are usually intended for measuring current across a shunt and have a small input range. Although you can divide down the input signal using precision resistors, signal to noise ratio will be impacted. Also gain may vary from channel to channel. This may or may not be an issue for your application. It depends on the accuracy you are trying to achieve. The small input range has usually been an issue for me so I haven't tried this approach although it could work.
I've also tried digital isolation. Digital isolators are plentiful and cheap. I've personally used digital isolators for CAN bus, I2C, and SPI. Typical front end is usually some type of filter circuit followed by an opamp, and then AtoD. The AtoD is isolated from the MCU using the digital isolator. Since the AtoD is upstream of the isolator, the quantization noise of the isolator does not have to be considered. This produces a more accurate measurement with a wider bandwidth.
One final point. Safety always trumps cost and should be your primary concern.