Back button when the previous layer is unknown

Hello all,

I have a slide with fifteen layers (!) that can be selected in any order.  The way it is laid out is a large circle in the center surrounded by eight smaller ones.  The smaller ones can be selected in any order, however all must be selected prior to moving to the next slide.  Some of the smaller circles have links to other layers in them.  

Is there a way to create a 'back" button for a slide with this many layers so it takes the user to the previous layer (i.e., one layer back or two plus layers back) viewed?  I created a simple example with a variable that captures the layer viewed.  I then added a trigger to a button that would view the layer the variable states.  Unfortunately, it doesn't seem to work.  When you click on a layer (numbered circles),  it takes you to the layer assigned.  You have to then select the little house button to return to the main layer.  From the main layer, the back arrow will show the previous layer.  However, you have to be at the main layer for the back arrow to work.  As this solution won't work for my original project.

Any suggestions would be much appreciated.

Thanks,

Matt