When a source or layer is removed transfer ownership back to the caller so it can (optionally) take it. Preserve the behavior that removing a CustomLayer triggers deinitialization. Deinitialize all custom layers when a style is destroyed in case those layers are not explicitly removed.