Hi there Thomas!
Have you inserted the video as an MP4 file? There isn't a full screen button as a part of the Storyline video controls or within the Storyline player itself. If you want the embedded video to play full-screen within the Storyline player, you'll want to adjust the Player Properties to 'Resize browser to fill screen' and 'Scale player to fill browser window'.
Another option is to insert the video as a Web Object (for example from YouTube or Vimeo). That would allow you to us the normal video controls, which often include full screen viewing. If you insert the video as a Web Object and you don't see the full screen button, it could be because the web browser is restricting fullscreen mode for embedded iframe content.