Does diversity in undergraduate research topics look bad on my graduate school application?

Trying out several projects in different areas does not necessarily say that you do not know what interests you. You have to show them that you are interested in several areas and that you wanted to get an insight in order to obtain a wider range of possibilities.

I think that there are few people who immediately find the right area that fully interests them. It is important that you present it that way. Working on different projects brings more extensive experience in any case. If you have then found the right area, you can deal with it more intensely.

When applying to grad school in ecology, I had research experience in stream restoration, dynamical modeling of HIV infection, and a NASA internship where I worked on bioregenerative life support. I had also done a research project in cultural geography (cosmopolitan thought in Cold War America) and completed two internships in which I had done a number of different projects. While I only got into one program (out of the three I applied to), it was one of the top programs in my field and came with the best fellowship the university could award. Now, I had good GRE scores but a so-so GPA (3.2), so I'm sure the research projects (and the variety of letters of recommendation that came with them) really helped my application. The lab I got accepted into was highly integrative, so my broad background made me a good fit.

Also, as the other commenter said, nobody really expect an undergrad to immediately know what they wanted to specialize in. I'd be concerned if they did!