Base layer object showing up on layer when it isn't supposed to

Not sure if this is a bug but it's annoying. I attached an example. This is the set up:

4 objects on base layer.

When you hover over 3 of the objects, the state of the 4th object changes to an associated state. 

When you click on any of the 3 objects, a corresponding layer is shown.

Once the corresponding layer is shown and you move the mouse, the Object on the base layer that has been changing states suddenly appears. 

All layers are set to hide base layer objects. 

The only way to stop this is to have Restore previous state when user hovers out unchecked. This however leads to the state of the 4th object staying in the previous hover state which is not the original plan. 

It's frustrating that once on the layer, the mouse movement is still affecting the base layer and over riding the hide all objects setting. 

If anyone knows a way to have the Restore previous state checked and not have the object show up on the layers I'd love to know how you did it.