I'm glad to say I found a workaround for this problem.

First I run AmigaAmp, and keep it running in the background without playing music.
Then I can watch youtube with OWB (or use mplayer to watch a video) without any audio problems when pausing and resuming the video.

I don't know why I have to do this, but it works so I'm happy.