Prevent user from selecting a layer while audio is playing

I have a scene with one slide and 3 layers in this slide. Each layer (say layer 1, 2 and 3) will be called by users clicking on 1 of 3 hotspot (say hotspot 1, 2 and 3) on the base layer (layer 0). Each layer have a own audio file (audio 0, 1, 2 and 3).

When the user clicks on the hotspot 1, before the audio 0 on layer 0 is fully played, he will hear both audios, the one from the base layer and the one from the layer connected to the hotspot he clicked, audio 1.

If I want to prevent that, I'll have to activate the hotspots when the speaker is done on the base layer. No problem.

But: Layer 1 has his own audio file (audio 1). When the user clicks on hotspot 2 before audio 1 is done, that will start layer 2 and audio 2, so audio 1 and audio 2 will play at the same time. As the hotspots are on the base layer, I cannot figure out how I can delay/deactivate them for a while on the layers.

Important: Users should decide which hotspot (1, 2 or 3) they want to click first. So no way on getting hotspot 2 on layer 1, hotspot 3 on layer 3, etc.

Any solution?