Derivation of self-inductance of a long wire
The original question talked about a discrepancy between the result obtained by calculating the flux directly and using the definition $L= \Phi/I$. The confusion arises because of a concept known as "flux linkage". When you calculate the flux enclosed by the region of unit length between r and r+dr, you calculated an expression for flux which you integrated to get the total flux. However, the entire flux calculated by you is not "linked" to this area since the current enclosed by the contour of this radius is a fraction $\pi r^2/R^2$ of the total current. Thus the linked flux is $d\Phi= \dfrac{\mu_0 I r dr}{2\pi R^2} \dfrac{\pi r^2}{\pi R^2}$. If you integrate this expression, you would get the correct result. $$\Phi = \dfrac{\mu_0 I}{2\pi R^4}\int_0^R r^3 dr= \mu_0 I/8\pi$$ "Flux linkage' is not a very easy concept but consider what happens when you have N turns of the wire through which the same flux passes. In order to calculate emf using Faraday's law, you will need N times the flux to get the correct emf.
I know this post is old and has been answered I thought I would post the exact derivation to help out anybody in the future.
In order to calculate the the internal inductance of a wire we have to equate the equation for the energy of the magnetic field to the energy from the inductor/inductance.
Energy of the $B$ field: $\frac{1}{2\mu}\int B^2dV$, where $B$ is integrated over all space.
Energy of an inductor: $\frac{1}{2}LI^2$
To solve for the Magnetic field B we use an Amperian loop - $\oint \vec{B}\cdot d\vec{l}=\mu I_{enc}$
First we make an assumption that the current is uniformly distributed throughout the wire (which is a reason this inductance is usually neglected, at higher frequencies the current is not uniform but is carried on the surface of the wire which creates a more real resistance in the wire and less self inductance).
With a uniform current $I_{enc} = I\frac{\pi r^2}{\pi R^2}$ for r$ < R$ where $r$ is a variable distance inside the wire, $R$ is the radius of the wire, and I is the total current running through the wire. This equation is a simple ratio of the variable area to the total area of the wire multiplied by the total current inside the wire to find the current for any variable amount of the wire.
Now plug this back into Ampere's equation $$\oint \vec{B}\cdot d\vec{l}= μI\frac{\pi r^2}{\pi R^2} = μI\frac{ r^2}{ R^2}$$ Now because B is independent of the closed loop itself for any r distance the B can be pulled out of the integral. $$B\oint dl=μI\frac{ r^2}{ R^2}$$ The closed loop integral along dl can be calculated out, but for simplification we will realize that the loop is simply the circumference at any variable r distance. $$B\cdot2\pi r=μI\frac{ r^2}{ R^2}$$ Solving for $$B = B(r) = \frac{\mu Ir}{2\pi R^2}$$ $$B^2 = \frac{\mu^2 I^2 r^2}{4\pi^2R^4} \quad\text{for}\quad r < R$$ Meaning this is only the B field inside of the wire
Now we can equate the two energy equations: $$\frac{1}{2\mu}\int B^2 dV = \frac{1}{2} LI^2$$ Plugging in for $B^2$ that we solved previously we get: $$\frac{1}{\mu}\int\frac{\mu^2 I^2r^2}{4\pi^2R^4}d = LI^2$$Plugging in the volume integral in polar form: $$\iiint\frac{\mu r^2}{4\pi^2R^4}r\mathrm{d}r\mathrm{d}ϕ\mathrm{d}z = L$$
At this point is where some people "screw up" and get an infinite inductance. Because the magnetic field outside of the wire is not coupled to anything and is not bounded it does not contain any real energy (thus why we solved for $B$ inside of the wire only), now our integrations go from $0$ to $R$ in the $r$ direction, $0$ to $2\pi$ in the $\phi$ direction, and $0$ to $l$ in the $z$ direction where $l$ is the length of the wire. This integrates over the entire wire.
$$L = \frac{\mu}{4\pi^2R^4} \int\limits_0^l\mathrm{d}z \int\limits_0^{2\pi}\mathrm{d}\phi \int\limits_0^R r^3\mathrm{d}r$$
$$\therefore L = \frac{\mu}{4π^2R^4}\left(l2\pi\frac{1}{4}R^4\right)$$
$$\therefore L = \frac{\mu l}{8\pi}$$
To match the original questioner's format realize that $\mu=\mu_r\mu_0$
Note that in this equation the $R$'s have cancelled out, which means the inductance is independent of the radius of the wire itself. Hope this helps anyone out there!
Edit 1: I think I just understood you question: you are actually trying to calculate some sort if “internal” inductance, i.e. the contribution to the inductance of only the field inside the conductor.
When calculating the flux, you have to choose a closed path over which you would want the electromotive force, and then integrate the magnetic flux over the surface limited by this path. Normally the path would be the whole electrical circuit, but since you are only interested in the contribution of the internal field, you chose the return path along the edge of the wire, which is fine. Now you have to choose the forward path.
The forward path should be along the lines of current. The problem is that, different lines of current give different fluxes. Then you can calculate the flux as a function of where, in the conductor's cross-section, you take the forward path. But since you are using the low-frequency approximation (no skin effect, then uniform current density), you can just average the forward-path dependence over the whole cross-section. Then you get the missing factor two.
A somewhat different argument is given in this old bulletin of the Bureau of Standards: the author instead weights individual flux lines as per the fraction of the conductor they enclose. This gives the same factor two.
Edit 2: As requested, a few clarifications.
By “integrate the magnetic flux” I really mean “calculate the magnetic flux”. I used “integrate” because the calculation involves an integral: $$ \phi = \int_A \mathbf{B}\cdot\mathbf{n}\; \mathrm{d} A $$ where $\mathbf{n}$ is the unit normal to the surface. It's not exactly the same as “integrate the magnetic field” because of the dot product with $\mathbf{n}$.
I talked about “forward path” and “return path” because, if it's not an antenna (as the low-frequency approximation suggests), a wire is usually part of a transmission line which consists of at least two conductors. Assume for example that you use a pair of wires to connect a source to a load, like in the figure below (I hope everyone can see Box Drawing characters):
╔════════╗ ╔════════╗
║ ╟→→→→→→→→→→→→→→→→→╢ ║
║ source ║ (flux here) ║ load ║
║ ╟←←←←←←←←←←←←←←←←←╢ ║
╚════════╝ ╚════════╝
where the arrows represent the electric current. I assume the wire you are interested in is the top one, which I called “forward path”. The bottom wire, which I called “return path”, brings the current back to the source. Taken together, these two wires form a loop and the current will make some magnetic flux through the loop. Then, if you try to change the current, some electromotive force will appear because of this flux, and you will be able to model this as the effect of an inductor along the transmission line, as below:
╔════════╗ ╔════════╗
║ ╟────(inductor)───╢ ║
║ source ║ ║ load ║
║ ╟─────────────────╢ ║
╚════════╝ ╚════════╝
This is the self inductance of the transmission line, and is what I first thought you where trying to calculate.
The self inductance of a bare wire is somewhat ill-defined. Well, it is defined, but with some assumptions about the surface over which to integrate the flux, and it scales as $l\log\frac{l}{r}$, which makes it's value per unit length diverge logarithmically when considering an arbitrarily long wire, as pointed out by Zassounotsukushi and mmc. Once you add the second wire, the surface over which you have to integrate the flux is clearly defined, and the inductance of the line scales like $l\log\frac{d}{r}$, where $d$ is the distance between the wires. No more logarithmic divergence with respect to $l$. On the other hand, it depends logarithmically on the distance between the wires, therefore you cannot just assume that the return path is just far enough to be ignored. BTW, the return path is not necessarily a wire, it could be, e.g., a ground plane.
For the particular calculation you are doing (only the contribution of the field inside the conductor), you use a very narrow loop where the return path is replaced by a line along the edge of the conductor, in order to enclose only the internal field.
Original answer below, which is somewhat bogus, as I thought you where after the total self-inductance (including external field) per unit length of an infinite wire. The comments of Georg refer to this original version.
You cannot assign an inductance to a long wire alone: you have to consider the whole circuit. The current carried by the wire has to come back in some way, and you need to know how far from your wire is the way back.
Assume for a moment that the wire is actually the inner conductor of a coaxial cable. You can easily calculate the linear inductance of the cable as a function of the inner an outer conductor radii. Now make the outer radius go to infinity and you have a diverging self-inductance! This means that in practice you can never assume that the way back is “far enough” to ignore it.