Previous button go back to previous slide's original layer, rather than last layer

Apologies if this is elsewhere on the board. I searched but couldn't find anything that answered my question.

I have a slide with four layers on it. The user clicks on the base layer, which reveals the next layer, and so on. When the last layer is closed, it automatically advances the user to the next slide. However, on slide 2 if the user clicks on the previous button, it takes them back to slide 1, but the 4th layer is displayed, rather than the base layer. Is there a way to make the previous button work so that when someone clicks on it, the user is taken back to the slide as it appeared originally, rather than with the layers displayed? Thanks.