When a source or layer is removed transfer ownership back to the
caller so it can (optionally) take it. Preserve the behavior that
removing a CustomLayer triggers deinitialization. Deinitialize all
custom layers when a style is destroyed in case those layers are not
explicitly removed.