Can an atom orbit the Sun?
Cute idea! Thanks for posting this question. I've enjoyed thinking about it.
Geometrical absorption
Suppose we start by assuming that we're talking about a particle that just absorbs all light that impinges on its cross-section. The sun's gravitational force on a particle is proportional to its mass, and therefore to the cube $a^3$ of its linear dimension $a$. Radiation pressure is proportional to the cross-sectional area, and therefore to $a^2$. Since the exponents are different, it follows that for small enough objects, the net force will be repulsive, and there can be no closed orbits.
For objects that are just a little above that size cut-off, we could have Keplerian orbits, but they would not obey Kepler's law of periods with the same constant of proportionality as for objects such as planets that are large enough to make radiation pressure negligible.
Without having to do a numerical estimate, we can tell that atoms are below the cut-off size for closed orbits, since solar sails exist, and a solar sail is considerably thicker than one monolayer of atoms.
All of this holds regardless of the distance $r$ from the sun, because both radiation pressure and gravitational forces go like $1/r^2$. This is also why the orbits are still Keplerian: the interaction with the sun acts like gravity, but just with a different gravitational constant.
A stable, electrically neutral particle such as a neutrino or a dark matter particle can orbit, because it doesn't interact with electromagnetic radiation. In fact, I think dark matter is basically known to exist only because it's gravitationally bound to bodies such as galaxies.
Wave model
But as pointed out by Rob Jeffries in a comment, this is not right at all for objects that are small compared to the wavelength of the light. In the limit $a \ll \lambda$, we have Rayleigh scattering, with a cross-section $\sigma \sim a^6/\lambda^4$. Let
$$R=\frac{F_\text{rad}}{F_\text{grav}}$$
be the ratio of the radiation force to the gravitational force. If we don't worry about factors or order unity, then it doesn't matter if we're talking about absorption, reflection, or scattering. Pretend it's absorption, and let $a$ be the radius of a spherical particle. We then have
$$R=\frac{3}{16\pi^2 Gc}\cdot\frac{L}{M}\cdot\frac{1}{\rho a^3}\cdot\sigma,$$
where $\rho$ is the density of the particle, $L$ is the luminosity of the sun, and $M$ is the mass of the sun.
For a particle with $a\sim 300\ \text{nm}$, the geometrical absorption approximation $\sigma\sim \pi a^2$ is pretty good, and the result is that $R$ is of order unity.
For a particle with $a\sim 50\ \text{nm}$, the Rayleigh scattering approximation is valid, and we have $\sigma\sim a^6/\lambda^4$. The result is $R\sim 10^{-4}$.
So it seems that the result is somewhat inconclusive. For a star with the $L/M$ of our sun, there is a pretty broad range of sizes for particles, with $a\sim\lambda$, such that there is fairly even competition between radiation pressure and gravity.
Ionization
Leftroundabout's answer pointed out the importance of ionization, and he estimated that effect for high-energy electrons. Actually I think UV is more important. For a 25 eV photon, which is at the threshold for ionization of helium, the cross-section is about $7\times 10^{-18}\ \text{cm}^2$. Suppose that $\sim10^{-2}$ of the sun's radiation is above this energy. For an atom at a distance of 1 AU from the sun, the result is that ionization occurs at a rate of $\sim10^{-3}\ \text{s}^{-1}$.
This suggests that there is no way an atom is going to complete a full orbit around the sun without being ionized. If we assume that our atoms(/ions) are all independent of one another, then an atom will basically spiral around the sun's magnetic field lines. One thing I don't know from this analysis is whether it's really valid to assume the atoms are independent. We could also imagine that there are parcels of gas orbiting the sun, and these are electrically neutral in bulk.
Summary
This analysis seems inconclusive for particles of baryonic matter with sizes less than about 300 nm. It seems like we need more work to understand this -- or someone could find where the subject has been treated in more detail in the astrophysics literature.
For stars off the main sequence, I think we can make some definite conclusions. Giant and supergiant stars, which have $L/M$ much higher than that of the sun, will efficiently sweep out all particles from $a\sim\lambda$ up to some upper size limit. For white dwarfs and such, with very small $L/M$, radiation pressure will never be significant.
Here is a paper (Mann et al., "Dust in the interplanetary medium," Plasma Phys. Control. Fusion 52 (2010) 124012) on dust in the interplanetary medium. It describes things like the trajectories of charged dust particles.
No, and radiation pressure is not the only reason.
The interplanetary space is not empty; apart from the optical photons it is in particular also flooded with the charged particles of the solar wind. This includes in particular a significant popolation of electrons with energies in (amongst others) the range of $100\:\mathrm{eV}$, where they can quite efficiently ionize helium atoms (with cross-section $\sigma\approx3\cdot10^{-16}\:\mathrm{cm}^2$). That energy range corresponds to a velocity-cube of ca. $v=6000\:\mathrm{\frac{km}{s}}$, i.e. $v^3 = 2\times10^{20}\:\mathrm{\frac{m^3}{s^3}}$.
The velocity density of such electrons at 1 AU is around $10^{-27}\:\mathrm{\frac{s^3}{cm^6}} = 10^{-15}\:\mathrm{\frac{s^3}{m^6}}$, i.e. a density of $\rho \approx2\times10^{5}\:\mathrm{m^{-3}}$ electrons with relevant energy. Moving at $v$, those electrons impinge on the atom's cross-section at a rate of $$ \nu = v\cdot\rho\cdot\sigma \approx 40\cdot 10^{6+5-20}\:\mathrm{s^{-1}} = 4\cdot10^{-8}\:\mathrm{s^{-1}} \approx \frac{1}{0.8\:\mathrm a}. $$ So, the atom is likely to get ionized before surrounding the sun once, and once it's ionized, its path is dominated by electrodynamic rather than gravitational forces. It certainly wouldn't keep a stable orbit.
I actually expected the collision frequency to be significantly higher than those $4\cdot10^{-8}\:\mathrm{s^{-1}}$; quite possibly I made an error in the calculation. If the rate is correct, then UV is actually the dominant cause of ionisation as Ben Crowell points out. It's plausible enough, seeing as the cross-section for those photons is actually not much lower than the cross-section for electron.