> OS4 seems to be using more compositing (until recently perhaps),
> to provide some 3D functionality.

That's true with "recently" = "some years ago", if the third-party developments led by A-Eon are considered. Else (only Hyperion-led developments considered) it's true even without modifier.

> it just seems to me that our MorphOS team members working on
> video card drivers and graphics support in general are doing things
> differently, which might make updating OS4emu more difficult.

I think it could be possible to simply not implement Warp3D Nova compatibility so that only OS4 software not depending on it (and neither on things building on top of it) would run. This would be similar to how Warp3D v5 was never supported by OS4Emu.