Concerns on Switching to Plain LuaTeX or ConTeXt Mark IV
I expect LuaTeX to be able to implement plain TeX,
that has thing backwards really, luatex implements a more or less Tex compatible engine, it does not implement plain tex, it is compatible enough that plain.tex may be processed by luatex instead of tex.
there are some places where there are documented differences in behaviour (notably around hyphenation) but mostly if you stick to classic plain tex input it will work in the same way as pdftex.
The latest version of ConTeXt is Mark IV 0.61. Since the version number is less then 1, is it stable?
I wouldn't read too much into version numbers. It is true that more or less no software is as frozen as tex itself, things change, but not in unmanagable ways.
The danger of LaTeX is that packages often conflict, and old packages may disappear or be deprecated. This has made me tired and disappointed. Is the same true for ConTeXt?
I don't think that's a latex issue really. Any system with lots of extension packages has the problem of different extensions. LaTex has many many more users than Context. You could stick to the core latex release you don't have to load every contributed package on ctan,
If I write journal articles, is it unlikely that publishers accept ConTeXt? What about plain LuaTeX? If I submit ConTeXt or plain LuaTeX, will my article be rejected merely because of this?
as far as I know essentially no journals will accept plain or context sources.
Or, to enhance compatibility, is it best that I wrap all functions I use, so that I may implement in both LaTeX and ConTeXt?
While that's possible in theory in some simple cases the two systems are sufficiently different that this is going to make things vastly more complicated and in practice limit you to a subset of features that are easily available in both systems. I really wouldn't recommend this.
It's different at low level programming level, expl3, pgf etc work just above the level of tex primitives and work in latex and context, but for any higher level document structure things diverge rather more.
I expect LuaTeX to be able to implement plain TeX, while natively allowing UTF-8 characters and arbitrary font. Is it so?
It is possible to natively use utf-8 characters with both xetex and luatex irrespective of the macro package. Both LaTeX and ConTeXt (with xetex or luatex engines) handle this seamlessly. It is also possible to use utf8 characters with pdftex engine with some jugulary. Both LaTeX and ConTeXt support this but it can bite you in unexpected ways.
The latest version of ConTeXt is Mark IV 0.61. Since the version number is less then 1, is it stable?
The version 0.61 is of the context wrapper script. The macro package does not use semantic versions, rather the releases are numbered according to the release date.
Will syntax change greatly? Will packages persist for years to come—that is, will what there are present now, still be released by TeX Live ten years later?
ConTeXt syntax has remained more or less the same for the 15 years that I have used it. There are have been a few changes (particularly with the way fonts were handled) when ConTeXt moved from pdftex as default to luatex as default (some 10 years ago). That said, it also depends on what you mean by stability. ConTeXt (in fact even LaTeX), is not as stable as Knuth TeX. But the LaTeX team and package writers make much more effort in ensuring backward compatibility than the ConTeXt developers and package writers. Part of the reason for this is that it is relatively simply to have multiple context installations in parallel and use a frozen version for tasks which require stability and use the latest version for other tasks.
Are ConTeXt packages managed by an organized team and well-tested, so that they are less likely to conflict?
No. The difference is that there is a lot more functionality implemented in ConTeXt core (which is managed by a single person, Hans Hagen, and has no conflicts) so there is a less need for using packages. But, as far as I know (and I have released 4-5 context packages), user-contributed ConTeXt packages are not managed any different from LaTeX packages.
If I write journal articles, is it unlikely that publishers accept ConTeXt? What about plain LuaTeX? If I submit ConTeXt or plain LuaTeX, will my article be rejected merely because of this?
For almost all journals, articles are accepted based on their scientific merit; not how they are prepared. After an article has been accepted, authors are asked to submit the article source (either tex or Word files). I know of only one journal that accepts source files written in ConTeXt (a Dutch math journal). Most publishers who accept TeX source files only accept LaTeX.
When you think about it, it makes sense. For a journal, all articles must be typeset with a uniform style. There is no advantage of maintaining multiple style files for different macro packages. That said, these days many journals actually convert your article to some XML format (so that an online HTML version can also be generated). So, there are some journal that will accept a PDF version and retype the article (of course, this may delay the publication or may incur additional cost).
Or, to enhance compatibility, is it best that I wrap all functions I use, so that I may implement in both LaTeX and ConTeXt?
Note that many publishers explicitly prohibit (or strongly discourage) user defined macros. For tables, it is relatively easy to write a program that generates appropriately formatted output for LaTeX and ConTeXt; but this becomes difficult for general document markup (unless you restrict yourself to a limited subset of macros).
I shall summarize my latest impression as an answer to make it more visible. This is just some random thoughts, and I ask the reader's tolerance. Also these are just general thoughts, and I hope are not off-topic according to site rules.
Using LuaTeX purely
I consider the possibility very exciting, because, now that luatex can handle UTF-8 natively, I may implement my own functions whenever possible. This makes my source easier to maintain, and customizable in the future. Since LuaTeX released version 1.0, I am looking forward to new possibility people haven't known of. (See their website, though seemingly not updated often)
Using ConTeXt MkIV
ConTeXt is said not to have changed much in terms of syntax. At least, I find people recommend ConTeXt, an Excursion even as of late in forums, but that's almost twenty years ago. Indeed, relevant manuals found on the internet are often written more than ten years ago. But given the fact that their website are still updated, I suppose they are actively maintained (They meet every year, I suppose). I guess I will give it a try.
Fear of package-clashing
It seems to me reasonable that the fact that LaTeX packages exist in great number made them more likely to clash. If we stick to "core packages", they may be better tested through age, and there must have been threads that points out what conflicts with what (or are there official guides to avoid conflicts?).
But how do we know what package is a core package? There is an instruction on how to print all available packages. But when I find someone using some package, I just Google, and cut and paste someone's code, and TeX Live will download that package for me. There is no way to tell what is TeX Live core, and what is user submitted and manually installed later.
Writing for publication
By now I am a student and have not published. But if I will have to publish something, there will be heavy editing in any rate. Many journals in fact provide their own templates (see for example: LaTeX templates), so modification is inevitable. On the other hand, if I managed to use LuaTeX or ConTeXt well, the effort of manual converting code into pdflatex-compilable one, is negligible. After all, they have similar syntax for essential needs. (Unless some day I write a book, but that probably justifies a more sophisticated tool like ConTeXt, and I should stop daydreaming.)
I may well prepare two templates, one for my personal use, adjusted for my taste, to be compiled with LuaTeX or ConTeXt; and another a "vanilla" LaTeX, to be compiled with pdflatex, as simplified as possible. I probably will write a little parser to simplify the job of expanding \newcommand's back to their respective definitions.
I guess it boils down to what makes myself happy. I often study material I writing, looking on the screen, while calculating, and I am meticulous on details (but not everyone is). If a better tool (LuaTeX, ConTeXt, whatever) gives me pleasure and make me working more confortably, than the benefits gained from using it is probably greater than the time lost in converting it back to pdfLaTeX, especially as I said, such effort is often inevitable after all.
Such thought is sort of solace of my initial frustration with LuaTeX recently. Complete new to this, I am sure I will have something to share with you readers, but only after I have got some time to make enough solving efforts.