What is the use of percent signs (%) at the end of lines? (Why is my macro creating extra space?)

The short answer is what others have said, % starts a comment that goes to the end of the line. The normal effect is that it doesn't insert the space (or a \par) from the newline.

The longer answer is that as TeX parses its input, it reads the input file line by line. It strips off tailing whitespace (including any carriage return and newline) and then appends a character corresponding to the number in the \endlinechar register (provided it isn't -1 or greater than 255). By default, the value is 13 which corresponds to an ASCII carriage return. (TeX denotes this by ^^M.) You can change this register to be any value.

Unless the category codes have been changed (e.g., by the \obeylines macro), ^^M has category code 5, or end of line. When TeX reaches an end of line character, it discards the rest of the line and does one of three things: nothing, insert a space token, or insert a \par token, depending on what state TeX was in (S, M, or N, respectively).

So what does this have to do with %? Well, since the comment character causes TeX to ignore the rest of the input line, the end of line character that got appended gets ignored too.

This can frequently be important when playing around with category codes of ^^M (again, using \obeylines or similar).

The long answer is contained in Chapter 8 of The TeXbook.

One final use that no one has mentioned is that it is sometimes necessary for the line to end with a space character and not a end of line character. One example is that a backslash followed by a space is different than a backslash followed by a newline:

\show\ 
\show\ %

In the first line, there's a space following the \, but it'll get stripped off as described so what you get instead \^^M as you can see by what TeX prints out.

> \^^M=macro:
->\ .

That is, \^^M is a macro that expands to a control space: \ . In the second case, the comment prevents the space from being stripped off and the end of line char is ignored. TeX prints out the following.

> \ =\ .

That is, \  is a TeX primitive (see control space in either The TeXbook or TeX by Topic).

The usual reason for % is suppressing spaces in macro definitions. Consider the macros \nopercents and \percents below.

\documentclass{minimal}
\newcommand*\bracket[1]{[#1]}%
\newcommand*\nopercents[1]{
    \bracket{#1}
}
\newcommand*\percents[1]{%
    \bracket{#1}%
}
\begin{document}
X\nopercents{blah}X

X\percents{blah}X
\end{document}

Superficially, they appear to do the same thing: pass their input to \bracket. The difference is that the newlines becomes space tokens in \nopercents but are ignored in \percents due to the %. So X\nopercents{blah}X expands to X [blah] X whereas X\percents{blah}X expands to X[blah]X.

Addendum regarding spaces at the beginning of a line.

A % only swallows whatever follows it on a line. It does not have any effect on white spaces that begin the next line. Under most circumstances, spaces at the beginning of a line are ignored by TeX itself. There are a couple of exceptions:

• When the line is otherwise entirely blank, it is interpreted as a paragraph break.

• When \obeyspaces is in effect, every space is carried into the output; this is true within verbatim mode and can also be requested explicitly.

If \obeyspaces is in effect while a command or environment is being defined, if the definition occupies more than one line, any spaces at the beginning of a continuation line will be preserved in the definition. Indentation is often used in defining commands or environments to make the code easier to understand (usually a good thing), but while \obeyspaces is in effect, this has an unwelcome result and should be avoided. \obeyspaces should normally be used only within a delimited scope ({ } or \begingroup ... \endgroup) to avoid unwanted results.

You already got lots of answers. You can also just experiment yourself:

\documentclass{article}
\begin{document}
Hello%
world!
\end{document}

Try compiling this with and without the %. Then you see yourself that the % makes the space produced by the newline disappear. (Note that you'll still get the space if you write Hello % with a space before the % – try it out!) All the details are given in TH's great answer.

A percent sign, %, allows to end a line without generating a space character -- very useful when writing macros.