Flow between slides is ok on first try but then starts to "jump"

I want to be able to scroll up and down a long "flow chart" and click into a section to take me to another slide.  When I return to the main slide I want to be back in the same section that i left. this works fine the first time I click in a section ( pink/ blue/ green)  but when I come back to the main slide, click nowhere else and click on the exact same same pink/blue/ green , when I return again it "shifts"  Any ideas ?

 

What i finally want to have is a scrollable  flowchart that users can click into to find out more info / go to instructional slides.

 

I have attached a video of the issue , I'm not sure how to "share" my project , any help on that would be appreciated, many thanks