When a theorem is the first thing in a list, prevent extra space
Two working answers has already been given to this questions; nonetheless, both answers fail to provide a detailed explanation of what is actually going on, and their authors plainly admit that their understanding of why those answers work is not perfect. Indeed, it is difficult—even for me—to tell whether or not the aforementiond solutions are guaranteed to work under any reasonable circumstance.
I decided to write this answer after getting a headache in the effort of sorting this out, essentially because it was clearly impossible to make it fit even in more than one comment. I hope that I’ll be able to shed some light on the matter, but please note that my understanding is not yet perfect too.
“Theorem-like” environments defined by means of the default facilities provided by the LaTeX kernel interact with lists in a perfectly correct way, as the following example shows:
% My standard header for TeX.SX answers:
\documentclass[a4paper]{article} % To avoid confusion, let us explicitly
% declare the paper format.
\usepackage[T1]{fontenc} % Not always necessary, but recommended.
% End of standard header. What follows pertains to the problem at hand.
% \usepackage{amsthm}
\newenvironment*{changemargins}[2]{%
\list{}{%
\setlength\leftmargin {#1}%
\setlength\rightmargin{#2}%
}\item\relax
}{\endlist}
\newtheorem{thrm}{Theorem}
\begin{document}
``Theorem-like'' environments defined by means of the default facilities
provided by the \LaTeX\ kernel interact correctly with lists.
A theorem at the outer level:
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
Now inside a list: first making up an item by itself\ldots
\begin{enumerate}
\item
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\item
\ldots and then after some text.
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\item
A third item follows.
\end{enumerate}
In the same way, their behavior when used by themselves inside a
\texttt{changemargins} environment is absolutely normal:
\begin{changemargins}{2cm}{2cm}
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\end{changemargins}
Another example, in which the ``theorem-like'' environment follows some text:
\begin{changemargins}{2cm}{2cm}
Some text, what suffices to get to the right margin\ldots\space
Well, perhaps just a little more.
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\end{changemargins}
Uncommenting
\begin{verbatim}
\usepackage{amsthm}
\end{verbatim}
will break all the above examples, so it looks like the
\textsf{amsthm} package is the real culprit!
\end{document}
If you uncomment the \usepackage{amsthm} declaration, you will see that the “culprit” is actually this package. Indeed, the machinery of the various switches like \if@newlist, \if@inlabel, and so on, relies on the assumption that list-type environments are nested according to the following pattern:
\begin{list}{...}{...}
\item
\begin{list}{...}{...}
\item
Text
In other words, every list-type environment (including “trivlist-type” ones!) should directly enclose an \item command, or something equivalent, before any other actual contents, lest a ”missing \item” error be returned. But we can see (file amsthm.sty, lines 186–199) that the amsthm package, after having issued (line 123) a \trivlist, does not directly use an \item command, as the corresponding facility provided by the LaTeX kernel does, but rather “something equivalent”, which, though, does not honor the \if@noparitem switch.
The following example patches the relevant macro in the amsthm package, to make “theorem-like” environments behave as expected inside lists too. I prefer not to use the \patchcmd facility from the etoolbox package, both because the patch is a bit complex and for reasons of readability; bold guys are of course free to do differently.
% My standard header for TeX.SX answers:
\documentclass[a4paper]{article} % To avoid confusion, let us explicitly
% declare the paper format.
\usepackage[T1]{fontenc} % Not always necessary, but recommended.
% End of standard header. What follows pertains to the problem at hand.
\usepackage{amsthm}
\makeatletter
\def\deferred@thm@head#1{%
\if@noparlist
\@donoparitem
\else
\if@inlabel \indent \par \fi % eject a section head if one is pending
\if@nobreak
\adjust@parskip@nobreak
\else
\addpenalty\@beginparpenalty
\addvspace\@topsep
\addvspace{-\parskip}%
\fi
\global\@inlabeltrue
\fi
\everypar\dth@everypar
\global\sbox\@labels{\unhbox\@labels\normalfont#1}%
\ignorespaces
}
\makeatother
\newenvironment*{changemargins}[2]{%
\list{}{%
\setlength\leftmargin {#1}%
\setlength\rightmargin{#2}%
}\item\relax
}{\endlist}
\newtheorem{thrm}{Theorem}
\begin{document}
``Theorem-like'' environments defined by means of the default facilities
provided by the \LaTeX\ kernel interact correctly with lists.
A theorem at the outer level:
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
Now inside a list: first making up an item by itself\ldots
\begin{enumerate}
\item
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\item
\ldots and then after some text.
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\item
A third item follows.
\end{enumerate}
In the same way, their behavior when used by themselves inside a
\texttt{changemargins} environment is absolutely normal:
\begin{changemargins}{2cm}{2cm}
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\end{changemargins}
Another example, in which the ``theorem-like'' environment follows some text:
\begin{changemargins}{2cm}{2cm}
Some text, what suffices to get to the right margin\ldots\space
Well, perhaps just a little more.
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\end{changemargins}
\end{document}
Please note that, in order to keep the example simple, I’ve not considered the xparse syntax: that is an “orthogonal” feature.
This patch has the following advantages over the other ones:
I know precisely why it works
;-);it works with all types of list environments.
Nevertheless, this does not mean that I am completely sure that it doesn’t break anything; indeed, the comment on line 187 of amsthm.sty suggests that it could. For this reason, I also propose another solution that keeps the patch to the amsthm macro local to the changemargins environment:
% My standard header for TeX.SX answers:
\documentclass[a4paper]{article} % To avoid confusion, let us explicitly
% declare the paper format.
\usepackage[T1]{fontenc} % Not always necessary, but recommended.
% End of standard header. What follows pertains to the problem at hand.
\usepackage{amsthm}
\makeatletter
\newcommand*\PatchAmsThmMacros{%
\def\deferred@thm@head##1{%
\if@noparlist
\@donoparitem
\else
\if@inlabel \indent \par \fi % eject a section head if one is pending
\if@nobreak
\adjust@parskip@nobreak
\else
\addpenalty\@beginparpenalty
\addvspace\@topsep
\addvspace{-\parskip}%
\fi
\global\@inlabeltrue
\fi
\everypar\dth@everypar
\global\sbox\@labels{\unhbox\@labels\normalfont##1}%
\ignorespaces
}%
}
\makeatother
\newenvironment*{changemargins}[2]{%
\list{}{%
\setlength\leftmargin {#1}%
\setlength\rightmargin{#2}%
}%
\PatchAmsThmMacros
\item\relax
}{\endlist}
\newtheorem{thrm}{Theorem}
\begin{document}
``Theorem-like'' environments defined by means of the default facilities
provided by the \LaTeX\ kernel interact correctly with lists.
A theorem at the outer level:
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
Now inside a list: first making up an item by itself\ldots
\begin{enumerate}
\item
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\item
\ldots and then after some text.
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\item
A third item follows.
\end{enumerate}
In the same way, their behavior when used by themselves inside a
\texttt{changemargins} environment is absolutely normal:
\begin{changemargins}{2cm}{2cm}
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\end{changemargins}
Another example, in which the ``theorem-like'' environment follows some text:
\begin{changemargins}{2cm}{2cm}
Some text, what suffices to get to the right margin\ldots\space
Well, perhaps just a little more.
\begin{thrm}
The ducks in dutch quack quaveringly on the docks.
\end{thrm}
\end{changemargins}
\end{document}
As you can see, this makes the patch work with the changemargins environment, but not with enumerate.
EDIT (rewrite and simplification of code)
I delved back into the definition of theorem-like environments and I think that I found what causes the problem when trying to use \list to do this. Namely, it is lines 4559-4566 from latex.ltx which form part of the definition of \trivlist:
\if@inlabel
\@noparitemtrue
\@noparlisttrue
\else
\if@newlist \@noitemerr \fi
\@noparlistfalse
\@topsep \@topsepadd
\fi
What is happening is that \list sets \@inlabeltrue and, in turn, this leads to extra space being added at the top of any environment that uses \trivlist, of which there are many. To stop this happening we just need to disable this. Once this is done the MWE in the question produces:
Here is the new code, which is quite a bit simpler than my first attempt:
\documentclass[12pt]{article}
\usepackage{amsthm}
\usepackage[margin=16mm,showframe]{geometry}% to show the margins
\usepackage{xparse}
\makeatletter
\NewDocumentEnvironment{changemargin}{O{3cm}D<>{3cm}}%
{\list{}{\leftmargin=#1\rightmargin=#2}\item%
\bgroup\@inlabelfalse\@newlistfalse}% change flags only in a group
{\egroup\endlist}
\makeatother
\newtheorem*{theorem}{Theorem}
\begin{document}
The rain in Spain falls mainly on the plain.
The rain in Spain falls mainly on the plain.
\begin{theorem}
This theorem has the correct spacing, because it's outside the changemargin environment.
\end{theorem}
The rain in Spain falls mainly on the plain. The rain in Spain falls mainly on the plain.
\begin{changemargin}[2cm]<2cm>
\begin{theorem}
This theorem has the right spacing and it is the first thing in the change margin environment.
\end{theorem}
\end{changemargin}
The rain in Spain falls mainly on the plain. The rain in Spain falls mainly on the plain.
\begin{changemargin}[2cm]<2cm>
The rain in Spain falls mainly on the plain.
\begin{theorem}
This theorem has the correct spacing, because it is not the first thing in the changemargin environment.
\end{theorem}
\end{changemargin}
The rain in Spain falls mainly on the plain. The rain in Spain falls mainly on the plain.
\end{document}
The \@newlistfalse is necessary because without latex gives an error, saying that there is a missing \item. As far as I can see there are no additional side-effects of adding the lines \@inlabelfalse\@newlistfalse as they will simply stop extra space being added to any environment that uses \trivlist. Of course, I may have missed something!
The only other comment worth making is that I have changed the OP's \changemargin macro into an enviroment by using \NewDocumentEnvironment from xparse. The environment has two optional arguments, which can be used independently, for setting the margins in the changemargin environment. For example,
\begin{changemargin}[2cm]<4cm>
Here is the correct spacing.
\end{changemargin}
sets the left-hand margin to 2cm and the right-hand margin to 4cm. Both margins default to 3mm.
Note: I don't know what I'm doing.
Caveat emptor ...
I added the additions just to check I'd not broken the spacing inadvertently. The code change is from latex.ltx. Also, I corrected the use of #1 in place of #2.
\documentclass[12pt]{article}
\usepackage{amsthm}
\usepackage{xparse}
\makeatletter
\NewDocumentEnvironment{changemargin} {O{3cm}D<>{3cm}}
{%
\list{}{\leftmargin=#1\rightmargin=#2}\item
\if@inlabel\global\@inlabelfalse\fi
\@newlistfalse
}{%
\endlist
}
\makeatother
\newtheorem*{theorem}{Theorem}
\begin{document}
\begin{changemargin}[0.9cm]<0.5cm>
The rain in Spain falls mainly on the plain.
\begin{theorem}
The rain in Spain falls mainly on the plain.
\end{theorem}
\end{changemargin}
The rain in Spain falls mainly on the plain.
\begin{theorem}
The rain in Spain falls mainly on the plain.
\end{theorem}
\end{document}
