Symbol for non-squares \not\square looks awful

You can avoid any guesswork, by letting TeX do the computations.

\documentclass{article}
\usepackage{amsmath,amssymb}

\makeatletter
\DeclareRobustCommand{\notsquare}{\mathord{\mathpalette\generic@not\square}}
\newcommand{\generic@not}[2]{%
  \sbox\z@{$\m@th#1/$}%
  \sbox\tw@{$\m@th#1#2$}%
  \sbox\z@{\raisebox{\dimexpr(\ht\tw@-\dp\tw@-\ht\z@+\dp\z@)/2\relax}{$\m@th#1/$}}%
  \vphantom{\usebox{\z@}}%
  \ooalign{\hidewidth\usebox{\z@}\hidewidth\cr$\m@th#1#2$\cr}%
}
\makeatother

\begin{document}

$\square\notsquare_{\square\notsquare}$

\end{document}

I used \mathord because such is \square.

Since \not is made to apply to math relations, a better syntax would be \mathord{\not\mathrel{\square}}. However, while that fixes the horizontal placement, vertical is still not ideal.

So I propose a "BEST" stackengine approach.

However, the OP notes the desire to avoid extra packages so I edited to also provide a native Alternative.

\documentclass{article}
\usepackage{amssymb,stackengine}
\begin{document}
\[
\textrm{Better: }\quad\mathord{\not\mathrel{\square}}
\]
\[
\textrm{Best: }\quad\ensurestackMath{\stackinset{c}{}{c}{}{/}{\square}}
\]
\[
\textrm{Alternative: }\quad\raisebox{1pt}{$\not\mathrel{\raisebox{-1pt}{$\square$}}$}
\]
\end{document}

You can define your own symbol as an overlay of \square with / using \ooalign:

\documentclass{article}

\usepackage{amssymb}

\newcommand{\notsquare}{
  \mathrel{
    \ooalign{
      $\square$\cr
      \hidewidth\raise.225ex\hbox{$/$}\hidewidth
    }
  }
}

\begin{document}

\[
  f \mathrel{\square} g \notsquare h
\]

\end{document}

Here is a short course on \ooalign: \subseteq + \circ as a single symbol ("open subset")