Prove that $\vert E \vert=0$ if $\frac{x+y}{2}\notin E$ for $x,y \in E$
Let $x\in E.$ Define $A_h = E\cap (x-h,x)$ and let $B_h$ be the reflection of $A_h$ across $x.$ Note $m(B_h) = m(A_h).$ Note also that $B_h\cap E = \emptyset,$ otherwise $x$ is the average of points in E. Therefore, for all $h>0,$
$$\tag 1\frac{1}{2h}m(E\cap (x-h,x+h)) \le \frac{1}{2h}[m(A_h ) + h -m(B_h)]$$ $$ = \frac{1}{2h}[m(A_h ) + h -m(A_h)] = \frac{1}{2}.$$
Now for a.e. $x\in E,$ the Lebesgue differentiation theorem shows the limit on the left side of $(1)$ equals $1.$ We've just shown that for every $x\in E$ the limit on the left side of $(1)$ is no larger than $1/2,$ if it exists. It follows that $m(E) = 0.$
Let $I(x)$ be the indicator function of $E$. By Lebesgue's density theorem, there exists a segment $[a,b]$ such that $\mu(E\cap [a,b]) \geq \frac{9}{10} |b-a|$. Denote $L= b-a$. Then $$\int_a^b \int_a^b I\left(\frac{x+y}{2}\right) dx dy \leq \int_a^b \int_a^b dx dy - \int_{[a,b]\cap E} \int_{[a,b]\cap E} 1 - I\left(\frac{x+y}{2}\right) dx dy = \left(1- \frac{9}{10}\times \frac{9}{10}\right) L^2 = \frac{19}{100}L^2.$$ On the other hand, letting $s=(x+y)/2$ and $t=(x-y)/2$, we get $$\int_a^b \int_a^b I\left(\frac{x+y}{2}\right)\, dx dy \geq 2 \int_{(3a+b)/4}^{(3b+a)/4} \int_{(a-b)/4}^{(b-a)/4} I(s)\, ds dt $$ In the $xy$-plane, the integral on the right is over a subset of the square $[a,b]^2$; namely it is over the square with the corners in the middle points of the sides of the square $[a,b]^2$. We have, $$ 2 \int_{(3a+b)/4}^{(3b+a)/4} \int_{(a-b)/4}^{(b-a)/4} I(s)\, ds dt \geq (b-a) \int_{(3a+b)/4}^{(3b+a)/4} I(t) dt \geq (b-a)\left(\frac{b-a}{2} - \frac{1}{10} (b-a)\right) =\frac{2}{5}\,L^2.$$ We get a contradiction.
First we prove the following claim.
Let $a<b$ be distinct points contained in $E$. Consider the interval $I = [a,b]$ and let $A = E \cap I$. Then $\lambda(A) \leq \lambda(I)/2$.
Proof:
Fix $x = a$ and let $y$ vary across $I$. Set $f(y) = (x+y)/2 = (a+y)/2$. If $y \in A$ then $f(y) \not\in A$. Thus we have identified a set $f(A) \subset [a,(a+b)/2]$ which is a scaled copy of $A$ (half the length of $A$) and which contains no points of $A$.
Now fix $y = b$ and let $x$ vary across $I$. Set $g(x) = (x+y)/2 = (x+b)/2$. If $x \in A$ then $g(x) \not \in A$. Thus we have identified a set $g(A) \subset [(a+b)/2,b]$ which is half the length of $A$ and which contains no points of $A$.
Now, the sets $A$, $f(A)$, and $g(A)$ are pairwise disjoint (except that $f(A)$ and $g(A)$ both contain $(a+b)/2$), and all are subsets of $[a,b]$. Therefore $$b-a \geq \lambda(A) + \lambda(f(A)) + \lambda(g(A)) = \lambda(A) + \lambda(A)/2 + \lambda(A)/2 = 2\lambda(A)$$ so $$\lambda(E \cap I) = \lambda(A) \leq (b-a)/2 = \lambda(I)/2$$
The claim implies a more general result.
If $J$ is any finite-length interval, then $\lambda(E \cap J) \leq \lambda(J)/2$.
Proof: Clearly if $E\cap J$ is empty or contains only one point, then $\lambda(E \cap J) = 0$ so the inequality holds. Otherwise, let $a = \inf(E \cap J)$ and $b = \sup(E \cap J)$, and let $a_n, b_n$ be sequences in $E \cap J$ such that $a_n \downarrow a$ and $b_n \uparrow b$, respectively. Then by the claim, $\lambda(E \cap [a_n, b_n]) \leq (b_n - a_n)/2$ for each $n$. Taking limits, we have $$\begin{aligned} \lambda(E\cap J) &= \lambda(E \cap [a,b]) \\ &= \lim_{n \to \infty}\lambda(E \cap [a_n, b_n]) \\ &\leq \lim_{n \to \infty} (b_n - a_n)/2 \\ &= (b - a)/2 \\ &\leq \lambda(J)/2 \\ \end{aligned}$$
Now let $J$ be any finite-length interval, and $A = E \cap J$. Suppose that $\lambda(A) > 0$. Let $\epsilon = \lambda(A)/2$ and suppose that $O$ is an open set containing $A$, with $\lambda(O) < \lambda(A) + \epsilon = 3\lambda(A)/2$. Such a set exists because $A = E \cap J$ is an intersection of measurable sets, hence measurable, and $A$ has finite measure because it is a subset of $J$.
As $O$ is an open subset of $\mathbb R$, we may express it as a countable union of disjoint open intervals, $O = \cup_{n=1}^{\infty}U_n$, and each $U_n$ has finite length since $O$ has finite measure.
For each $n$, we have two possibilities. If $A \cap U_n = \emptyset$ or $A \cap U_n$ is a singleton, then clearly $\lambda(A \cap U_n) = 0$. Otherwise, $A \cap U_n$ contains at least two distinct points of $A$, (hence of $E$), so $\lambda(A \cap U_n) = \lambda(E \cap (J \cap U_n)) \leq \lambda(J \cap U_n)/2 \leq \lambda(U_n)/2$.
Then $$\begin{aligned} \lambda(A) &= \lambda(A \cap O) \\ &= \lambda\left(A \cap \left(\bigcup_{n=1}^{\infty}U_n\right)\right) \\ &= \sum_{n=1}^{\infty}\lambda(A \cap U_n) \\ &\leq \sum_{n=1}^{\infty}\lambda(U_n)/2 \\ &= \frac{1}{2}\sum_{n=1}^{\infty}\lambda(U_n) \\ &= \frac{1}{2}\lambda(O) \\ &< \frac{3}{4}\lambda(A) \end{aligned}$$ This contradiction resulted from the assumption that $\lambda(A) > 0$, so in fact we must have $\lambda(A) = 0$. This shows that the intersection of $E$ with any finite-length interval has measure zero. Therefore $$E = E \cap \mathbb R = E \cap \left(\bigcup_{n=-\infty}^{\infty} [n,n+1]\right) = \bigcup_{n=-\infty}^{\infty}(E \cap [n, n+1])$$ also has zero measure, by countable additivity.