Volume question

I'm putting finishing touches on a presentation in Storyline 3. Normally the "optimize audio volume" option is great, but it's giving me an issue right now.

In one part, I have the audio track voiceover going on, but then I have some music coming on in the background for a small time. The optimizing audio is making them both the same volume, but I want the music to be background music. Is there a way to do this, or do I have to manually adjust the audio for everything in order to get it the way I'm looking to?

Thanks