How do I write a macro having comma-separated and variable number of arguments?

The comments what color we have is only for demonstration here.

\documentclass{article}
\usepackage[T1]{fontenc}

\makeatletter  
\def\setmycolour#1{\expandafter\setmycolour@i#1,,,,\@nil}
\def\setmycolour@i#1,#2,#3,#4,#5\@nil{% 
  \ifx$#2$ we have gray => #1 \else
    \ifx$#3$ we have a wrong color setting \else
      \ifx $#4$ we have a rgb setting => #1,#2,#3\else
                we have a cmyk setting =>#1,#2,#3,#4
      \fi
    \fi
  \fi 
}  
\makeatother

\begin{document}

\setmycolour{0.5}\par
\setmycolour{0.5,0.6}\par
\setmycolour{0.5,0.6,0.7}\par
\setmycolour{0.5,0.6,0.7,0.8}\par

\end{document}

\documentclass{minimal}

\makeatletter

\def \setmycolour #1{
    \newcount \n
    \n = 0
    \setmycolour@ #1,\stopmarker ,
    \ifnum \n = 1
        This is gray hue (#1).
    \else \ifnum \n = 3
        This is rgb colour (#1).
    \else \ifnum \n = 4
        This is cmyk colour (#1).
    \else
        \message{Wrong number of values.}
    \fi\fi\fi
}

\def \stopmarker{EOV}

\def \setmycolour@ #1,{
    \edef \colorvalue{#1}
    \ifx \colorvalue \stopmarker
        \let \next = \relax
    \else
        \advance \n by 1
        \let \next = \setmycolour@
    \fi
    \next
}
\makeatother

\begin{document}

    \setmycolour{0.5}\par
    \setmycolour{0.5,0.6}\par
    \setmycolour{0.5,0.6,0.7}\par
    \setmycolour{0.5,0.6,0.7,0.8}\par

\end{document}

A LaTeX3 solution:

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\NewDocumentCommand{\setmycolour}{ m }
  {
   \prg_case_int:nnn { \clist_length:n { #1 } }
     {
      {1}{ \dogray{#1} }
      {3}{ \dorgb{#1} }
      {4}{ \docmyk{#1} }
     }
     {OOPS}
  }
\ExplSyntaxOff

\def\dogray#1{This is gray hue #1.}
\def\dorgb#1{This is rgb colour #1.}
\def\docmyk#1{This is cmyk colour #1.}

\begin{document}
\setmycolour{0.85}\\
\setmycolour{1,0,0}\\
\setmycolour{1,0,0,0}

\end{document}

Customizations of the commands performed in the admissible cases are, of course, possible.

Important change

Due to the changes made to expl3 in Summer 2012, the functions

\prg_case_int:nnn
\clist_length:n

should be changed into

\int_case:nnn
\clist_count:n

with the same syntax.