Why does \def not work as TikZ parameters?

The key-values contained in \sty aren't expanded this way, this causes an error.

You can use \draw \expandafter[\sty]... however

Using a tikzstyle is better, however.

\nopagenumbers
\input tikz.tex

\def\center#1\endcenter{\centerline{#1}}

\def\sty{step=.5cm,gray,very thin,dashed}
This is my first plain \TeX\ file. Let me try some commands below:
\vskip 1cm
\center
\tikzpicture[scale=3]
\draw \expandafter[\sty] (-1.4,-1.4) grid (1.4,1.4);
\draw [->](-1.5,0) -- (1.5,0);
\draw [->](0,-1.5) -- (0,1.5);
\draw (0,0) circle [radius=1cm];
\path[fill,draw][fill=green!20!white, draw=green!50!black] (0,0) -- (3mm,0mm) arc [start angle=0, end angle=30, radius=3mm] -- cycle;
\endtikzpicture
\endcenter

\bye

Although it is possible to use a macro as shown in Christian Hupfer's answer, this is not really the best approach when using TikZ, whichever macro format you choose for your document. Instead, it would be better to use TikZ's key-value syntax to create a sty style with \tikzset{}. Note, however, that \tikzstyle is deprecated and ought not be used in new code.

\tikzset{sty/.style={step=.5cm,gray,very thin}}

can be used in the usual way

\tikzset{sty/.style={step=.5cm,gray,very thin}}

\nopagenumbers
\input tikz.tex

\def\center#1\endcenter{\centerline{#1}}

\tikzset{sty/.style={step=.5cm,gray,very thin}}
This is my first plain \TeX\ file. Let me try some commands below:
\vskip 1cm
\center
\tikzpicture[scale=3]
\draw [sty] (-1.4,-1.4) grid (1.4,1.4);
\draw [->](-1.5,0) -- (1.5,0);
\draw [->](0,-1.5) -- (0,1.5);
\draw (0,0) circle [radius=1cm];
\path[fill,draw][fill=green!20!white, draw=green!50!black] (0,0) -- (3mm,0mm) arc [start angle=0, end angle=30, radius=3mm] -- cycle;
\endtikzpicture
\endcenter

\bye

The macro hides the syntax characters. The comma as separator for the key value pairs and the equals sign need to be visible by the key value parser.

But it is possible to define styles via \tikzstyle (seems to be deprecated, the documentation uses it, but does not explain it) or via the .style handler, e.g.:

\nopagenumbers
\input tikz.tex

\def\center#1\endcenter{\centerline{#1}}

\tikzset{
  mygridstyle/.style={
    step=.5cm,
    gray,
    very thin,
  },
}

This is my first plain \TeX\ file. Let me try some commands below:
\vskip 1cm
\center
\tikzpicture[scale=3]
\draw [mygridstyle] (-1.4,-1.4) grid (1.4,1.4);
\draw [->](-1.5,0) -- (1.5,0);
\draw [->](0,-1.5) -- (0,1.5);
\draw (0,0) circle [radius=1cm];
\path[fill=green!20!white, draw=green!50!black] (0,0) --
(3mm,0mm) arc [start angle=0, end angle=30, radius=3mm] -- cycle;
\endtikzpicture
\endcenter

\bye