Why isn't momentum conserved in this pulley problem?

The pulley (and the attachment to the ceiling) are part of the system here. Because of this, you cannot simply use conservation of momentum on the three given masses.

If the final velocity were $v$, then the total energy of the system would have increased since both the pan and counterweight would be moving and the other mass would not have slowed.

You can't use conservation of momentum equations on only a portion of a system. If you imagine a ball bouncing on the floor, you can't say the momentum of the ball is conserved before and after the bounce. You have to consider the change in momentum of the floor as well.

In your problem, the change in momentum of the ceiling will be small, but relevant.

$$\Delta p_{m1} + \Delta p_{m2} + \Delta p_{pan} + \Delta p_{ceiling} = 0$$

As you do not know the change in this final component, you can't use conservation to solve for the remaining momentum of the other three masses.

Let's change the situation to make this more explicit. Instead of a counterweight, consider two pans and two weights.

Let's imagine the pulley and string to be massless, so the two pans and two weights have a total mass of $4m$. If the balls have a velocity $v$, then the total momentum of the pulley inside the room is $1mv + 1mv = 2mv$.

But by symmetry, we can see that the pulley isn't going to turn. If we imagine the pans at rest after the collision, we find the momentum is now $0mv$.

If the pulley's connection to the ceiling/room/earth is not part of the system, then we say that forces from that connection were external and changed the total momentum. We cannot use conservation of momentum due to external forces.

If the ceiling/room/earth are part of the system, then after the collision, they have gained $2mv$ downward momentum, so the total system does not change. If we picture the box as nearly massless and in a spaceship instead of earth, the entire box would be moving downward at $v/2$ after the balls hit the pans. (assuming completely inelastic collision). The more massive the box, the slower it moves to retain the velocity. Consider it attached to a building/earth, and the momentum still changes, but the velocity change is no longer measurable.

Your calculation of final momentum after the collision has a sign error in it. The pulley serves to change the direction of the motion. This means that a mass moving upward on the left side of the pulley is given a mathematical sign of "+" for the associated velocity. As the string goes over the pulley, the direction of the motion changes such that a downward velocity on the right side of the pulley is given a "+" mathematical sign. Thus, $mV + mV + mV = 3mV$. Since $V = \frac {v}{3}$, $3mV = mv$, and momentum is conserved.

According to me, the momentum of the mass on the other side shouldn't be negative because the system considered is a connected system, i.e, the momentum transfer to the mass hanging on the other side of the pulley is momentum transferred through the pulley's string. I agree with @BowlOFRed about the contribution of the momentum transferred to the ceiling, however the situation asks us to consider an ideal pulley (no rotational friction at all), which implies that a majority of the momentum transferred across the pulley's string affects the mass on the other end and the contributions towards the ceiling are negligible so we shouldn't care (EDIT: Refer to comments). So when we write down the equations, $$\Delta p_{i}=\Delta p_{f} $$ this should imply: $$mv\approx mV+mV+mV $$ which gives us the required result. Of course, since the pulley is ideal, (theoretically) the error of measurement would be really little.