Time shown on layer compared to base laye

I'm a little confused by something.  On my base layer, I have a video playing that will play for the entire duration of the slide.  I then have other layers that I want to appear and disappear at certain times and when a given layer is visible, it has its own actions.

I noticed that each layer has a timeline that is different than the base layer.  As such, I have to play around quite a bit in order to get the timing correct. So for example, let's say I have an animation that I want to occur 50 seconds into the video on the base layer and that my animation later becomes visible at 40 seconds into the base layer.  I have to set the animation for 10 seconds on the layer, knowing that at that point, it's actually been 50 seconds of total play time.

Perhaps I'm taking the wrong approach here but it seems as though if I only have 2 layers (base and one other) I see the full timeline, otherwise, I only see the timeline for that specific layer.

Thoughts?

Thanks in advance