Does the sea level increase if an iceberg melts?
The Archimedes principle says that a floating body will displace an amount of fluid that is equal to its weight.
Since the iceberg floats, it weighs the same as the water it displaces. If it had the same salt concentration as the ocean, then once thawed, it would occupy exactly the same volume as it displaced and the sea level wouldn't change.
But most icebergs are made of nonsalty water, with a density a bit lower than sea water. So once melted, that same mass will occupy more volume (same mass, less density equals more volume), and the sea level will increase… very very slightly.
The one you're not contemplating: that the sea level rises because of melting of ice that's currently over land. As noted in Rodrigo's answer, when sea ice melts there is no change in the water level, and if the ice is made from fresh water then there will be a small change due to the mismatch in densities. However, that ignores the fact that there is a huge quantity of ice in ice caps and glaciers over land, and if that ice melts then it will contribute to a rise in sea levels.
It is true that when pure ice melts in pure water, the water level does not rise. However, the oceans are salty and this makes a big difference. When pure water melts into the salty ocean, water level rises. Your latter reasoning is the correct reasoning.
This answer discusses the physics and the mathematical details behind what we observe and then applies the same to the reality.
Mathematical Details:
Let the density of ice and water be $\rho_{\text{ice}}$ and $\rho_{\text{water}}$ respectively.
The density of ice is lesser than that of water and hence ice floats on water. Part of the ice will be submerged and the rest will stay above the surface.
The submerged part of the ice is responsible for the buoyant force provided by water.
Let the total volume of ice be $V_{\text{tot}}$ and the volume of the ice submerged under water be $V_{\text{sub}}$.
According to Archimedes principle, the buoyant force provided by water is given by:
$$F_{\text{buoy}} = V_{\text{sub}} \rho_{\text{water}} g$$
The weight of the ice is given by: $$F_{\text{weight}} = m_{\text{ice}}g = \rho_{\text{ice}} V_{\text{tot}} g $$
As the ice floats on water, the buoyant force must balance the weight of the ice.
$$F_{\text{buoy}} = F_{\text{weight}}$$
$$V_{\text{sub}} \rho_{\text{water}} g = \rho_{\text{ice}} V_{\text{tot}} g$$
$$V_{\text{sub}} = V_{\text{tot}}\frac{\rho_{\text{ice}}}{\rho_{\text{water}}} \tag{1}$$
After the ice melts, the mass of ice would've turned into liquid water. The density has changed but the mass hasn't.
Let $V_{\text{new}}$ be the volume occupied by the ice mass in its water form.
$$m_{\text{ice}} = V_{\text{tot}} \rho_{\text{ice}} = V_{\text{new}} \rho_{\text{water}}$$
$$V_{\text{new}} = V_{\text{tot}}\frac{\rho_{\text{ice}}}{\rho_{\text{water}}} \tag{2}$$
Inferences:
Compare equation $(1)$ with equation $(2)$. You'll notice that $V_{\text{new}}$ is exactly equal to $V_{\text{sub}}$.
This result can be interpreted as: the volume occupied by the submerged portion is equal to the total volume occupied by the ice in its water form.
To put in a nutshell, the volume under the sea surface hasn't changed due to the melting of ice.
Therefore, melting of ice does not affect the sea level.
What if the ice and water both had dissolved salts?
This in no way alters the equations $(1)$ and $(2)$. Therefore, the previous inference remains valid.
What if the ice is made up of pure water and the sea is salty?
The equations $(1)$ and $(2)$ would change to the equations given below:
$$V_{\text{sub}} = V_{\text{tot-pure-ice}}\frac{\rho_{\text{ice}}}{\rho_{\text{water-salty}}} \tag{3}$$
$$V_{\text{new}} = V_{\text{tot-pure-ice}}\frac{\rho_{\text{ice}}}{\rho_{\text{water-pure}}} \tag{4}$$
As salt water is more dense than pure water, $\rho_{\text{water-pure}}$ is smaller than $\rho_{\text{water-salty}}$.
Using the previous statement and analyzing the equations, it can be inferred that sea level rises when a pure block of ice melts in a salty sea.
What if the ice is made up of salt water and the sea is pure water?
$$V_{\text{sub}} = V_{\text{tot-salty-ice}}\frac{\rho_{\text{ice-salty}}}{\rho_{\text{water-pure}}} \tag{5}$$
$$V_{\text{new}} = V_{\text{tot-salty-ice}}\frac{\rho_{\text{ice-salty}}}{\rho_{\text{water-salty}}} \tag{6}$$
As salt water is more dense than pure water, $\rho_{\text{water-pure}}$ is smaller than $\rho_{\text{water-salty}}$.
Using the previous statement and analyzing the equations, it can be inferred that sea level goes down when a salty block of ice melts in a sea made up of pure water.
Conclusions:
If an ice made up of pure water melts in an ocean of pure water, the sea level does not change.
If an ice made up of pure water melts in an ocean of salty water, the sea level rises.
If an ice made up of salty water melts in an ocean of pure water, the sea level goes down.
If an ice made up of salty water melts in an ocean of salty water, the sea level does not change.
Melting of ice & its relationship with sea level
Out of the 4 cases mentioned in the conclusion of the mathematical details section, only the following are likely scenarios on earth:
Pure ice melts into the salty ocean
Salty ice melts into the salty ocean
The remaining two cases are quite unlikely because we don't really have pure water (a.k.a fresh water) oceans and seas.
The glacial ice and ice shelves (these are on land) are made up of snow which is frozen pure water. Due to warming climate (climate change!), large chunks of these fresh water reserves break off and floats into the sea. These are called icebergs (these are floating in the sea). Therefore, when these kind of icebergs melt, they cause the sea level to rise.
When salt water begins to freeze, the ice formed contains the dissolved salts. When this salty ice melts into the salty ocean, the sea level does not change.
If you put those two cases together, we can conclude that the sea level can either rise or remain same due to the melting of ice.
Significance of melting ice
Sea-level rise is governed by processes that alter the volume of water in the global ocean—primarily thermal expansion of sea water and transfers of water from terrestrial reservoirs, such as land ice and groundwater, to the ocean. The Intergovernmental Panel on Climate Change (IPCC) Fourth Assessment Report found that thermal expansion accounted for about one-quarter of the observed sea-level rise for 1961–2003, melting of land ice accounted for less than half, and changes in land water storage accounted for less than 10 percent (Bindoff et al., 2007). For the last 10 years of that period (1993–2003), the IPCC estimated that thermal expansion and land ice melt each contributed about half to the total sea-level rise. The improved agreement between estimates of the individual contributions and the total sea-level rise for the later time period was attributed to the availability of satellite altimetry data and other global ocean data sets and to better knowledge of the processes causing sea-level rise. Subsequent work has corrected instrument biases, reducing estimates of the thermal expansion contribution to sea-level rise, and recorded increased rates of land ice loss. In the most recent estimate, for 1993–2008, the contribution from land ice increased to 68 percent, the contribution from thermal expansion decreased to 35 percent, and land water storage contributed -3 percent (sea-level fall; Church et al., 2011).
Source: https://www.nap.edu/read/13389/chapter/5
Melting of ice is the largest contributor towards sea level rise.
Additional references for the previous statement:
Causes of Sea Level Rise: What the Science Tells Us (2013) - www.ucsusa.org
Is sea level rising? - oceanservice.noaa.gov
Sea level rise - Contributions - Wikipedia