Actually, you didn't move the timer to the slides. You only moved the timer's display. The timer (the two layers) is still on the master, and starts when the first slide starts.
There are two ways around that in the attached project.
The first one is the Original Scene. It will work well if you want to use the timer some other place. I created a new layout named Blank, and renamed the original blank to Timer. I set slides one and three to use the Blank layout, and the second slide (with the timer display) to use the Timer layout. If you want to reuse the timer some other place, just set that slide layout to Timer, and place a display on it.
The other method is on scene Alternate, and may be more straight forward. I recreated the two timer layers and their triggers on slide two.
In all cases, I set Seconds to 0 when the slide starts, so the timer starts over if the slide is revisited. I subtracted 1 from it only if it is greater than 0, so it doesn't go to negative numbers.