This function does not return nil on failure (VW2019 SP5.3), instead it seems to be returning the original object.
Worse, I have not been able to find any documentation as to which object types are supported by the function.
So far I have found that passing it a kLine2D returns the original object, thus the following snippet destroys the original drawing:
MCObjectHandle tempPoly = gSDK->ConvertToPolygonN(h, (Boolean) true, 512);
Fortunately this causes our plugin to crash, so our users have not lost data (yet)