Remove = sign before number when using siunitx round-minimum

You can compare the value with 0.001 before \num does its job:

\documentclass{article}
\usepackage{siunitx}

\ExplSyntaxOn
\NewDocumentCommand{\pequals}{m}
 {
  \fp_compare:nTF { #1 < 0.001 }
   {
    \ensuremath{ p < \num{0.001} }
   }
   {
    \ensuremath{ p = \num[round-mode=places, round-precision=3]{#1} }
   }
 }
\ExplSyntaxOff

\begin{document}

\pequals{0.0123}  

\pequals{0.0003}

\end{document}

A fancier version, where you can add options to pequals: min states the threshold under which a number is simply printed with < n; other options are passed to sisetup, so they should be meaningful for \num.

\documentclass{article}
\usepackage{siunitx}

\ExplSyntaxOn
\NewDocumentCommand{\pequals}{O{}m}
 {
  \group_begin:
  \keys_set:nn { pyrrhic/pequals }
   {
    min=0.001,
    round-mode=places,
    round-precision=3,
    #1
   }
  \fp_compare:nTF { #2 < \l_pyrrhic_pequals_min_fp }
   {
    \ensuremath{ p < \num{\fp_use:N \l_pyrrhic_pequals_min_fp} }
   }
   {
    \ensuremath{ p = \num{#2} }
   }
  \group_end:
 }
\keys_define:nn { pyrrhic/pequals }
 {
  min .fp_set:N = \l_pyrrhic_pequals_min_fp,
  unknown .code:n = \use:x { \sisetup { \l_keys_key_tl = \exp_not:n { #1 } } },
 }
\ExplSyntaxOff

\begin{document}

\pequals{0.0123}

\pequals{0.0003}

\pequals[min=0.0001,round-precision=4]{0.0003}

\pequals[min=0.0001,round-precision=4]{0.00003}

\end{document}

Note that with round-minimum=0.001, there is no space between < and 0.001. In addition, \num (which is used inside \pequals) outputs <0.001 only if its argument is actually less than 0.0005, not 0.001. (This is because rounding has been enabled via the option round-mode=places; check the output of \pequals for 0.00051 and 0.00049 to verify this claim.) This outcome may well run counter to common expectations that (IMNSHO) that the <0.001 string should be generated whenever the argument of \num is less than 0.001, not whenever it's less than 0.0005. For sure, this is what you appear to expect, as you wrote that one of your objectives was

with anything below 0.001 given as <0.001.

In order to get the spacing around the < symbol to be the same around the = symbol, and in order to ensure that the cutoff <0.001 means what readers are entitled to think it means, it's best to write the routine from scratch. The following example shows how this may be done by using LuaLaTeX. The left-hand column shows the output of the new macro, called \ptest, and the right-hand column shows the output of \pequals.

% !TEX TS-program = lualatex
\documentclass{article}
\usepackage{amsmath,luacode}
\begin{luacode}

function ptest ( i )
   if i<0.001 then
      tex.sprint ( "\\ensuremath{p<0.001}" )
   else
      tex.sprint ( "\\ensuremath{p="..string.format("%.3f",i).."}" )
  end
end

\end{luacode}
\newcommand\ptest[1]{\directlua{ptest(#1)}}

%% The OP's code for '\pvalue' and '\pequals':
\usepackage{siunitx}
\newcommand{\pvalue}[1]{\num[round-mode=places, 
         round-precision=3, round-minimum=0.001]{#1}}
\newcommand{\pequals}[1]{\ensuremath{p=\pvalue{#1}}}

\begin{document}
\ptest{0.3456},  \pequals{0.3456}

\ptest{0.0123},  \pequals{0.0123}

\ptest{0.00101}, \pequals{0.00101}

\ptest{0.00098}, \pequals{0.00098}

\ptest{0.00049}, \pequals{0.00049}
\end{document}