Approximating a $\sigma$-algebra by a generating algebra

Proof: Let $$\mathcal S:=\left\{A\in \mathcal{B}\mid \forall\varepsilon>0,\exists A'\in\mathcal A,\mu(A\Delta A')\leq \varepsilon\right\}.$$ We have to prove that $\cal S$ is a $\sigma$-algebra, as it contains by definition $\cal A$.

• $X\in\cal S$ since $X\in\cal A$.
• If $A\in\cal S$ and $\varepsilon>0$, let $A'\in\cal A$ such that $\mu(A\Delta A')\leq \varepsilon$. Then $\mu(A^c\Delta A'^c)=\mu(A\Delta A')\leq \varepsilon$ and $A'^c\in\cal A$.

• First, we show that $\cal A$ is stable by finite unions. By induction, it is enough to do it for two elements. Let $A_1,A_2\in\cal S$ and $\varepsilon>0$. We can find $A'_1,A'_2\in\cal A$ such that $\mu(A_j\Delta A'_j)\leq \varepsilon/2$. As $$(A_1\cup A_2)\Delta (A'_1\cup A'_2)\subset (A_1\Delta A'_1)\cup (A_2\Delta A'_2),$$ and $A'_1\cup A'_2\in\cal A$, $A_1\cup A_2\in \cal A$.

  Now, let $\{A_k\}\subset\cal S$ pairwise disjoint and $\varepsilon>0$. For each $k$, let $A'_k\in\cal A$ such that $\mu(A_k\Delta A'_k)\leq \varepsilon 2^{-k}$.
  Let $N$ be such that $\mu\left(\bigcup_{j\geq N+1}A_j\right)\leq \varepsilon/2$ (such a choice is possible since $\bigcup_{j\geqslant 1}A_j$ has a finite measure and $\mu\left(\bigcup_{j\geq n+1}A_j\right)\leq \sum_{j\geq n+1}\mu\left(A_j\right)$ and this can be made arbitrarily small). Let $A':=\bigcup_{j=1}^NA'_j\in\cal A$. As $$\left(\bigcup_{k\geq 1}A_k\right)\Delta A'\subset \bigcup_{j=1}^N(A_j\Delta A'_j)\cup\bigcup_{k\geq N+1}A_k,$$ and we conclude by sub-additivity.

I have found in Probability And Examples, by Rick Durret, second edition, 1996, the following question as an exercise: (Page 452, Appendix: Measure theory, Exercise 3.1):

Let $\mathcal{A}$ be an algebra, $\mu$ a measure on $\sigma(\mathcal{A})$ and $B \in \sigma(\mathcal{A})$ with $\mu(B) < \infty$ . Then for any $\epsilon > 0$ there is an $A \in \mathcal{A}$ with $\mu(A\Delta B) < \epsilon$.

The interesting fact is that there is no any assumption here that the measure is finite or even $\sigma$ finite! The ONLY assumption is that the measure of $B$ is finite. Another interesting fact is that this question was omitted in the 4-th edition of the book. (I didn't find it...)

So is it a mistake?

If the measure is not finite, i.e. $\mu(\Omega) = \infty$, then the collection of all well approximated sets may not be a $\sigma$ algebra! The regular proof fails when one has to show that it is closed under countable unions. It is still an algebra, and it is also closed under countable unions which have finite measure. But it may (?) not be closed under general countable unions...

However, if the restriction of the measure $\mu|_{\mathcal{A}}$ is $\sigma$ finite on $\mathcal{A}$ then this restriction has a unique extension to the $\sigma(\mathcal{A})$, i.e. $\mu$ defined on $\sigma(\mathcal{A})$ must coincide with $(\mu|_{\mathcal{A}})^*$ (the outer measure obtained by $\mu|_{\mathcal{A}}$)

Hence: $(\mu|_{\mathcal{A}})^*(B) < \infty$ which is equivalent to: $$\mu(B) = (\mu|_{\mathcal{A}})^*(B) = \inf{\left\{\sum_{i=1}^{\infty}{\mu(A_i)} : B \subseteq \bigcup_{i = 1}^{\infty}{A_i}, A_i \in \mathcal{A}\right\}} < \infty$$

So I can pick up some countable covering of $B$ (with disjoint sets from $\mathcal{A}$) for which: $$\mu(B) \le \sum_{i=1}^{\infty}{\mu(A_i)} \le \mu(B) + \epsilon/2 $$ and then drop the tail of the series and get some finite subcovering which will approximate $B$.

So under the additional assumption that the restriction $\mu|_\mathcal{A}$ is $\sigma$ finite, this result is true.

But is it true in general? Does anybody have a counter example? Or a proof?