Dynamic graphics are less responsive inside a Manipulate

Putting the Dynamic wrapper of the point graphics outside the list seems to make a difference.

data = Thread[RandomChoice[Range[200], 350] -> RandomChoice[Range[200], 350]];

vrf = Module[{mo}, 
    PopupWindow[Dynamic[mo = CurrentValue["MouseOver"];
      Dynamic@{PointSize[If[mo, 0.03, 0.01]], Point[#1]}, 
      TrackedSymbols -> {}], Dynamic[mo = CurrentValue["MouseOver"];
      Panel["Popup Window Text", Background -> Dynamic[If[mo, Pink, White]]]]]] &;

I notice an improvement in both

gp = GraphPlot[data, VertexRenderingFunction -> vrf]

and especially in

Manipulate[gp, {x, 0, 10}]

The Manipulate is nearly as fast as the plain dynamic graphics.

---

I noticed that

Manipulate[gp]

does not behave as badly as the Manipulate with a slider, although perhaps slightly worse than just gp.

A wild guess is that Manipulate inserts an EventHandler in front of the output that causes extra checking. For instance, it might be that a "MouseOver" event is checked at all points instead of only the relevant one. I tried CurrentValue[foo++; "MouseOver"], which showed that it is updated only at one point, but that doesn't mean the front end doesn't do extra checking behind the scenes. [Update: This seems less likely after further investigation. See below.]

The wild guess does not really explain why the solution above works, though. It seems like familiarity with the internals workings of Manipulate will be needed to explain what the problem is.

---

Updated Edit: A better guess.

It seems that wrapping the visible object with Dynamic is to be preferred to wrapping Dynamic around code inside the object. For instance, as in the above or better yet as in @Rojo's comment,

Dynamic[PointSize[If...]]

is better than

PointSize[Dynamic[If...]]

Why might that be?

I stress might, because I don't know. The system is designed to dynamically update objects that appear in a visible cell in a notebook. Dependencies have to be parsed, and at a superficial glance, it seems to me it might be easier if the visible object is the dynamic object and not some internal part of it that in itself is not visible. For instance it would be better if Dynamic wrapped the PointSize directive and not the If statement inside PointSize. In any case, all the points are being updated when Dynamic is inside PointSize, as observed by @Rojo. Further there are four updates per mouse-over for each point (estimated), which is quite a lot.

Now the order of PointSize@Dynamic@... seems confusing. It seems to say PointSize, being outside, is not dynamic, only what is inside it. So what does the system do when the inside changes? The original code has

Dynamic[mo = CurrentValue["MouseOver"];
 {PointSize[Dynamic[If[mo, 0.03, 0.01]]], Point[#1]}]

Notice that PointSize is itself inside another Dynamic. One ought to be able to remove the inside Dynamic.

An improvement

When the inside Dynamics are removed, you run into another problem already pointed out implicitly by @MikeHoneychurch that each of the two linked objects, the point and the popup, overwrites mo. So you need separate state variables. This led to the following:

vrf = Module[{mo1 = False, mo2 = False}, 
    PopupWindow[Dynamic[mo1 = CurrentValue["MouseOver"];
      {PointSize[If[mo1 || mo2, 0.03, 0.01]], Point[#1]}], 
     Dynamic[mo2 = CurrentValue["MouseOver"];
      Panel["Popup Window Text", Background -> If[mo1 || mo2, Pink, White]]]]] &;

But further investigation reveals that there is still some inefficiency. There are two updates per mouse-over on a Point. A change CurrentValue["MouseOver"] causes an update to mo1; since PointSize depends on mo1, another update is incurred. This led me to the following changes.

A better improvement

Here is a more efficient implementation:

vrf = Module[{mo1 = False, mo2 = False},
    PopupWindow[
      Dynamic@{PointSize[If[(mo1 = CurrentValue["MouseOver"]) || mo2, 0.03, 0.01]], Point[#1]},
      Panel["Popup Window Text",
        Background -> Dynamic@If[mo1 || (mo2 = CurrentValue["MouseOver"]), Pink, White]]]] &;

It has one update per mouse-over. Putting the assignment inside If makes the expression for the point not depend on the value of mo1. Ditto for the expression for the Panel.

Why does Manipulate run so slowly? Hmm…

I'm beginning to agree with @MikeHoneychurch that Manipulate is a strange beast. I attempted to measure the CPU load by running my mouse quickly back and forth over closely connected points. My computer is a 2.5GHz 2-core Mac, and the monitor reports loads as percentages that can add up to 200% (100% per core). It's not an exact measurement, obviously, but it seems to reveal something about the behavior.

Using the original code, without Manipulate, the load is 80%-90%, about %50+ by the front end and 30%+ by the kernel, and the response time is ok but noticeably slower than any of the improvements. With Manipulate the load is about 100%+ when the graph is inside Manipulate, 40%+ by the front end, 60% by the kernel. Compared to the performance without Manipulate cited above, the decrease in the front end load is reflected in the slow performance. The substantial increase in the kernel load, about double, reflects a substantial amount of additional computation. However, the number of updates isn't different than the original dynamic graphics, four for each point (all points updated every mouse-over). By comparison, with any of the improvements I can get the front end up above 60% and the kernel remains below 2%, whether or not inside Manipulate, and the response is quite rapid.

I have not been able to think of a reason for the increased demand on the kernel inside.

As per comment thread the thing that produces a more responsive Manipulate is to use Dynamic[Point[If[...]]] rather than Point[Dynamic[If[...]]]. Using two mouse over variables enables you to mouse over the popup and change the size of the point.

(*create some data*)
data = Thread[
   RandomChoice[Range[200], 350] -> RandomChoice[Range[200], 350]];

(*define the vertex rendering function*)

vrf = DynamicModule[{mo1 = False, mo2 = False}, 
    PopupWindow[
     Dynamic[mo1 = 
       CurrentValue["MouseOver"]; {Dynamic@
        PointSize[If[mo1 || mo2, 0.03, 0.01]], Point[#1]}], 
     Dynamic[mo2 = CurrentValue["MouseOver"]; 
      Panel["Popup Window Text", 
       Background -> Dynamic@If[mo2, Pink, White]]]]] &;

(*graph plot*)gp = GraphPlot[data, VertexRenderingFunction -> vrf]

and the Manipulate:

Manipulate[gp, {x, 0, 10}, TrackedSymbols :> {x}]