Hi Dave
Its been a while, but I am still struggling with the same issue.
See attached the pictures of the renders, you can see that the Panorama is still to heavily lit by ambient light, while all settings are the same.
I also attached the VWX file for you to see.
Hope you can find a misser on my end.
cheers
Thijs
20211011 VWX Test.vwx