About the category of von neumann algebras
This is not an answer to the original question, but rather a summary and conclusion of the discussion that Dmitri and I had in the comments to his accepted answer. At the outset, I should make two caveats: (1) I am not a set theorist. (2) In addition to the conversations with Dmitri, my answer is heavily influenced by conversations with (indeed, based on what I learned from) Alex Chirvasitu and Will Johnson. Probably most of this story is well-known, and from the 1930s (related results are due to Ulam, for example), but I finally learned it. Throughout I assume the axiom of choice.
The question that came up in the comments above begins with the following (already remarkable!) fact. Let $\text{Man}$ denote the category of smooth manifolds. There is a natural functor $\operatorname{Spec}(\mathcal C^\infty(-)) : \text{Man} \to \text{AffSch}$ from this category to the category of affine schemes (over $\mathbb Z$). Provided your manifolds are Hausdorff, this functor is faithful. The remarkable fact, which is also an amusing exercise, is that if you consider only second-countable manifolds, then this functor is also full. Indeed, you can recover everything there is to know about the smooth geometry of a manifold from its ring of $\mathcal C^\infty$ functions, as a ring. In this sense "manifolds are a part of algebraic geometry".
More generally, let $M$ denote a smooth manifold. Then there is a continuous map $M \to \operatorname{Spec}(\mathcal C^\infty(M))$, functorial in $M$, which sends a point $m\in M$ to the maximal ideal of functions that vanish at $m$. Another remarkable fact is that the topology of $M$ is precisely the (induced) Zariski topology on the image of this map. The topological space $\operatorname{MaxSpec}(\mathcal C^\infty(M))$ is canonically identified with the Stone–Cech compactification $\beta M$ of the topological space $M$. All of this is true without any second-countability conditions.
Aside 1: It is important to use real-valued smooth functions, and not complex-valued smooth functions. For the statement above, you must make heavy use of the fact that $\mathbb R$ has no nontrivial automorphisms as a ring. If you work with schemes over $\mathbb C$, then you can use complex-valued maps, but if you want schemes over $\mathbb Z$, then you need to make sure that the point doesn't have automorphisms.
Aside 2: The slogan "manifolds are part of algebraic geometry" is very misleading. Given a smooth manifold $M$, it has a structure sheaf $\mathcal C^\infty_M$ that assigns to an open $U \subseteq M$ the ring $\mathcal C^\infty(U)$. This sheaf is not a quasicoherent sheaf of rings. Put another way, even though $M \hookrightarrow \operatorname{Spec}(\mathcal C^\infty(M))$ is an isomorphism onto its image as topological spaces, it is not an isomorphism of ringed spaces.
The question then arises whether you can drop second-countability (weaken it to paracompactness) from the definition of smooth manifold and still preserve these conditions. The answer surprised me a bit. As you can read in the comments above, Dmitri claimed that the functor $\operatorname{Spec}(\mathcal C^\infty(-))$ was full on the category of paracompact manifolds. I claimed that it failed to be full as soon as the manifold was not "second-continuum" (has a basis for the topology consisting of $2^{\aleph_0} = |\mathbb R|$ opens). Both are false. Dmitri's claim is closer to correct, although the actual story is close to the proof I tried to give (not surprising, since I'm the one saying it — probably if Dmitri were writing this "answer", he would phrase it to sound closer to his approach). In short, fullness fails for sufficiently large manifolds, but those manifolds must be very large indeed. I will explain.
It suffices to think about 0-dimensional manifolds, i.e. sets. Let $S$ be a set. Then $\mathcal C^\infty(S) = \mathbb R^S$ is the ring of all functions $S \to \mathbb R$. The question is to understand the set of ring homomorphisms $\mathbb R^S \to \mathbb R$. (I.e. the set of maps $\operatorname{Spec}(\mathcal C^\infty(\text{pt})) \to \operatorname{Spec}(\mathcal C^\infty(S))$, compared to the set of maps $\text{pt} \to S$). An easier exercise than the above (again using that $\mathbb R$ has no automorphisms) is that any such ring homomorphism must be an $\mathbb R$-algebra homomorphism. So we can ask about, for any field $\mathbb k$, the set of $\mathbb k$-algebra homomorphisms $\mathbb k^S \to \mathbb k$. Each point $s \in S$ gives such a homomorphism (by projection = evaluation). Are there any others?
Let $\phi : \mathbb k^S \to \mathbb k$ be a $\mathbb k$-algebra homomorphism. For $T \subseteq S$, let $\mathbf 1_T \in \mathbb k^S$ be the characteristic function of $T$. Then $(\mathbf 1_T)^2 = \mathbf 1_T$, and so $\phi(\mathbf 1_T)\in \mathbb k$ is either $0$ or $1$. Say $T$ is large w.r.t. $\phi$ if $\phi(\mathbf 1_T) = 1$, and small w.r.t. $\phi$ if $\phi(\mathbf 1_T) = 0$. Exercise: With these rules, $\phi$ defines an ultrafilter on $T$.
Indeed, you can replace the target $\mathbb k$ with any field, and again determine an ultrafilter. On the other hand, the ultraproduct of any field is again a field. So this shows that $\operatorname{MaxSpec}(\mathbb k^S) = \beta S$. Since $\mathbb k$ is a $\mathbb k$-algebra in only one way, $\hom_{\mathbb k\text{-alg}}(\mathbb k^S,\mathbb k)$ consists precisely of those ultrafilters on $\operatorname{MaxSpec}(\mathbb k^S)$ with residue field $\mathbb k$.
Denote the cardinality $|\mathbb k| = \kappa$. For any cardinal $\lambda$, say that an ultrafilter is $\lambda$-additive if the union of $\lambda$ many small sets is small. So a usual ultrafilter is finitely-additive. I claim that $\hom_{\mathbb k\text{-alg}}(\mathbb k^S,\mathbb k)$ consists precisely of the $\kappa$-additive ultrafilters on $S$. On the one hand, suppose you are given a $\kappa$-additive ultrafilter. Then for each $f\in \mathbb k^S$, (precisely) one of its level-sets must be large (as $S = \bigcup_{k \in \mathbb k} f^{-1}(k)$ would otherwise be a $\kappa$-sized union of small sets), and set $\phi(f)$ to be the value of the large level-set. Conversely, suppose given a homomorphism $\phi : \mathbb k^S \to \mathbb k$, and build an ultrafilter as above. If we are given a $\kappa$-sized collection of sets with large union, we want to show it has a large set. It suffices to suppose that the union is everything (else throw in for good measure the small complement of the union), and that the sets are pairwise disjoint. Then we index the sets by $k \in \mathbb k$, and consider them as the level sets of a function $f$. But this function is mapped to some $\phi(f) \in \mathbb k$, and $f - \phi(f)$ has non-invertible image in the ultraproduct, so $f(s) - \phi(f)$ must be non-invertible for a large number of $s$, i.e. the $\phi(f)$-level set must be large.
In summary: if $S$ is a set thought of as a discrete (but not second-countable) manifold, then $S = \hom_{\text{man}}(\text{pt},S) \hookrightarrow \hom_{\text{ring}}(\mathcal C^\infty(S),\mathcal C^\infty(\text{pt})) = \hom_{\mathbb R\text{-alg}}(\mathbb R^S,\mathbb R)$ is an isomorphism iff $S$ does not admit a non-principal $2^{\aleph_0}$-additive ultrafilter.
A potential "large-cardinal" axiom posits such sets to exist. Indeed, a measurable cardinal is an uncountable cardinal $\mu$ that admits a non-principal ultrafilter which is $\lambda$-additive for every $\lambda < \mu$. ZFC cannot prove that such cardinals exist (as I will explain in a moment), but set theorists do consider them. Existence of measurable cardinals is stronger than existence of Grothendieck universes (aka strongly inaccessible cardinals).
Now I will explain why having even a non-principal $\aleph_0$-additive ultrafilter (let alone a $2^{\aleph_0}$-additive ultrafilter) requires $S$ to be very large. Specifically, I will show that $S$ must be "inaccessible from $\aleph_0$" in the sense that it is larger than any set formed by starting with $\aleph_0$, and repeatedly applying union and power-set constructions, doing the number of times of any set already built. In particular, $S$ is at least as large as the smallest Grothendieck universe. (Later I will show that it is at least as large as the smallest measurable cardinal.) I begin by the observation that in a non-principal ultrafilter, all singletons are small. So if $S$ has a non-principal $\lambda$-additive ultrafilter, then $|S| > \lambda$.
Now I will show that if an ultrafilter is $\lambda$-additive, then it is also $2^\lambda$-additive. Suppose given such an ultrafilter on $S$, and a collection $\lbrace S_L\rbrace$, indexed by $L \in 2^\lambda$, of subsets of $S$, such that $\bigcup_{L\in 2^\lambda} S_L$ is large. I would like to show that some $S_M$ is large. As above, it suffices to assume that $\bigcup_{L\in 2^\lambda} S_L = S$ and that the $S_L$s are pairwise disjoint.
By the sufficing assumptions, I can package together the collection $\lbrace S_L\rbrace$ into a function $f: S \to 2^\lambda$, by setting $f(s)$ to be the set $L \subseteq \lambda$ with $s \in S_L$. By the hom-tensor adjunction, I can then think of $f$ as a function $S \times \lambda \to 2 = \lbrace Y,N\rbrace$, where $Y$ and $N$ are the words "yea" and "nay".
Now the elements $s\in S$ will have a round of voting. The candidates for office are the elements $\ell \in \lambda$. Each $\ell$, in turn, stands up, and all $s\in S$ vote by the value of $f(s,\ell)$. I.e. $s$ votes in favor of all members of the $L \subseteq \lambda$ for which $s \in S_L$, and against all the other elements of $\lambda$. Because I have an ultrafilter, each of these votes gives an answer. If a majority (i.e. a large set for the ultrafilter) vote in favor of $\ell$, then it goes and stands on the "YEA" side of the room. Otherwise, it stands on the "NAY" side. Let $\Upsilon \subseteq \lambda$ denote the set of $\ell$ that are voted into office.
Now we play some formal games. Consider the set $S_\Upsilon$. Then $s\in S_\Upsilon$ iff $f(s,-) = $ is the characteristic function for $\Upsilon$, iff $\forall \ell \in \Upsilon$, $f(s,\ell) = Y$, and $\forall \ell \not\in \Upsilon$, $f(s,\ell) = N$. I.e. $s \in S_\Upsilon$ iff $s$ always votes with the majority. But we can say that another way: we're saying that for every $\ell$, $s$ voted with the majority for the $\ell$th round of balloting. Put another way, $$ S_\Upsilon = \bigcap_{\ell \in \lambda} \begin{cases} \lbrace s \text{ s.t. } s \text{ voted for } \ell\rbrace , \text{ if this set is large} \\\ \lbrace s \text{ s.t. } s \text{ voted against } \ell\rbrace , \text{ if this set is large} \end{cases} $$ But this is then a $\lambda$-sized intersection of large sets, all of which are large. If the ultrafilter is $\lambda$-additive, then $S_\Upsilon$ is large, completing the proof.
Finally, suppose that you are $\mu_a$-additive for some collection of cardinals indexed by $a\in \alpha$, and also $\alpha$-additive. Suppose given a $\bigcup_{a\in \alpha}\mu_a$-indexed collection $S_m$ of small subsets of $S$. Then you can form their union as $$ \bigcup_{ m \in \bigcup \mu_a} S_m = \bigcup_a\left( \bigcup_{m \in \mu_a} S_m\right) $$ and so it is also small. The inaccessibility follows.
In summary, I have shown that if $S$ admits a non-principal $\lambda$-additive ultrafilter, then $S$ is at least as big as the smallest universe containing $\lambda$. In particular, if $S$ is a manifold witnessing the failure of $\operatorname{Spec}(\mathcal C^\infty(-))$ to be full, then $S$ is larger than the smallest universe.
To complete the story, let me come back to the comment about measurable cardinals, by mentioning that the smallest $S$ admitting a $\lambda$-additive non-principal ultrafilter must in fact be measurable. Indeed, suppose that $S$ is not measurable. Then there is a cardinal $\mu < S$ and a $\mu$-indexed family of small sets $S_m$ whose union is $S$. Then I can "push-forward" my ultrafilter from $S$ along this family to $\mu$, by declaring that $M \subseteq \mu$ is large iff $\bigcup_{m\in \mu}S_m$ is large in $S$. Easy exercise: this is a non-principal $\lambda$-additive ultrafilter on $\mu$, violating the minimality of $S$.
To end with some concluding words, unless you work in logic it is reasonable to declare that the category $\text{Set}$ of "sets" consists only of those sets which are "accessible from $\aleph_0$" in the sense above. This is to say: declare that you are working only in the smallest universe. If you insist on having the axiom of Grothendieck universes, probably you can still disallow measurable cardinals with impunity. So any "manifold" that any working mathematician cares about will be "affine" in the sense of being part of the category that embeds fully into affine schemes. Logicians may have other interests. As to the question that spurred this whole discussion, the strong intuition is that with these "accessible" sets, you have nothing to worry about when dropping separability.
The standard reference for such matters is Guichardet's paper Sur la catégorie des algèbres de von Neumann. Bulletin des Sciences mathématiques 90 (1966), 41–64. PDF file: http://math.berkeley.edu/~pavlov/scans/guichardet.pdf
I don't think separability (or the more general property of σ-finiteness) is important. Requiring it is more of a tradition than a real necessity, similar to requiring smooth manifolds to be second countable instead of just being paracompact
I agree with Dmitri Pavlov that separability is not so important in the modern theory of von Neumann algebras, and this answers the second question. However, an example answering the first question has not yet been given, so here is one.
We take $\mathbf{vNA}$ to be the category of von Neumann algebras and normal *-homomorphisms, and the category $\mathbf{C}^*\mathbf{Alg}$ to be the category of unital C$^*$-algebras and *-homomorphisms. I will be using the fact that the forgetful functor $U : \mathbf{vNA} \rightarrow \mathbf{C}^*\mathbf{Alg}$ has a left adjoint $\mbox{-}^{**}$, the enveloping von Neumann algebra (the double dual of a C$^*$-algebra).
First, observe that $C(2^\omega)$ is the colimit of the diagram $$ C(2) \rightarrow C(2^2) \rightarrow C(2^3) \rightarrow \cdots $$ in $\mathbf{C}^*\mathbf{Alg}$. (The easiest way I know to prove this is to define the colimiting map on the indicator functions of clopen subsets of $2^\omega$, and then use Stone-Weierstrass to extend this to all of $C(2^\omega)$.)
Each of the C*-algebras $C(2^n)$ is finite-dimensional, and therefore isomorphic to its own enveloping von Neumann algebra. Therefore the diagram above is also a diagram in $\mathbf{vNA}$, consisting of separable von Neumann algebras. Because left adjoints preserve colimits, we have that $C(2^\omega)^{**}$ is the colimit of this diagram in $\mathbf{vNA}$.
The Dirac delta functions $\{\delta_x\}_{x \in 2^\omega}$ form an uncountable norm-discrete subset of $C(2^\omega)^*$. Therefore $C(2^\omega)^*$ is not norm-separable, so $C(2^\omega)^{**}$ does not have a separable predual, i.e. it is not a separable von Neumann algebra. But as we saw above, it is an inductive limit of finite-dimensional (and therefore separable) von Neumann algebras.