skops Changelog

v0.17

  • Loading a skops file now checks that each part of the file holds the kind of content its loader expects, for instance that the keyword arguments of a functools.partial are stored as a dict. A file that does not is refused with an error while it is read, before anything in it is audited or constructed, instead of failing with an unrelated error, or being accepted, during construction. #547 by Adrin Jalali.

  • Fix a regression since v0.12.0 where saving an object whose __reduce__ raises failed at dump time. __reduce__ is called on every object to detect a plain constructor call, but Cython extension types with a __cinit__ and no __reduce__ raise instead of returning one. pandas’ BlockValuesRefs is such a type and every pandas Index except RangeIndex holds one, so objects containing such an index, or a Series or DataFrame using one, could not be saved. Such objects are now saved through __getstate__/__dict__ again, as before v0.12.0. #550 by Adrin Jalali.

  • Objects which contain a reference to themselves can now be saved and loaded, and subclasses of the built-in containers are persisted the way pickle does it. #549 and #554 by Adrin Jalali.

    • Circular references: objects which refer to themselves, directly or through their attributes, dicts, lists, or sets, including subclasses of these, can now be saved and loaded. They used to fail with a RecursionError; this affected for instance the discrete distributions of scipy.stats and fitted sklearn.cluster.Birch models, which is no longer listed as unsupported. Such a reference through any other type, e.g. a tuple, raises an UnsupportedTypeException when saving.

    • Container subclasses: subclasses of dict, list and set are now saved and loaded the way pickle does it for dicts and lists. The instance is created with __new__ and filled in place instead of through its constructor, so a subclass whose constructor requires arguments can now be loaded, and its instance attributes, which used to be dropped, are saved and restored. Subclasses of set and of collections.defaultdict are treated the same way, although pickle builds them through their constructor, so that items which refer back to the container can be loaded. Files written with an earlier protocol load as before.

    • Loading: bound methods inherited from a class defined in another module can now be loaded, they used to be rejected as corrupted. A file in which nodes of different types share an __id__ is now rejected instead of silently loading one of them in place of the other.

  • Restore the skops command line entry point. It was declared in setup.py and lost when the packaging moved to pyproject.toml in v0.11.0, so skops convert and skops update had not been available from the command line since. python -m skops now also runs the CLI, and running it without a subcommand prints a usage error instead of a traceback. #548 by Adrin Jalali.

  • Fix saving of operator.methodcaller objects with keyword arguments: the method name and the keyword arguments were dropped, so the saved object could not be loaded. The file format now stores the keyword arguments and the persistence protocol is bumped to 3; files written with an earlier protocol load as before. #551 by Adrin Jalali.

  • Fix loading of collections.defaultdict objects whose keys are not strings, which failed with a TypeError, and of objects whose class defines __slots__, which failed with an AttributeError on Python 3.11 and later. Subclasses of defaultdict are now loaded as their own type, with their instance attributes; they used to be loaded as a plain defaultdict. #554 by Adrin Jalali.

v0.16

  • Fix loading of time-zone-aware datetime.datetime and datetime.time objects, and of zoneinfo.ZoneInfo and datetime.timezone instances. Their __reduce__ output was not recognized as a constructor call, so they were saved without their state and could not be loaded even when trusted. ZoneInfo objects created with ZoneInfo.from_file are not backed by a key and now raise an error when saved instead of producing a file that cannot be loaded. #546 by Adrin Jalali.

  • Importing skops.io is faster: resolving the module of objects without __module__, such as scipy ufuncs, no longer raises and catches an exception for every loaded module that lacks the attribute. #543 by Adrin Jalali.

  • The trusted argument of skops.io.load(), skops.io.loads(), skops.io.visualize() and skops.card.Card now officially accepts types in addition to their fully qualified names, and skops.io.load() accepts a binary file object in addition to a path. #541 by Adrin Jalali.

v0.15

  • The protocol number stored in a .skops file is now validated before any of its content is inspected: it must be an integer between 0 and the protocol of the installed skops version. Files saved with a newer protocol used to be silently loaded with the current loaders and now raise an error asking to update skops. The protocol selects which loaders audit and construct the file, so the developer notes in skops.io._protocol now also require that audit fixes to a loader are mirrored to all of its older versions. #537 by Adrin Jalali.

  • sklearn.tree._tree.Tree and sklearn.ensemble._hist_gradient_boosting.predictor.TreePredictor are no longer trusted by default: skops validates that a loaded object is of a trusted type, but not the contents of its raw state, and these types store unchecked node indices that scikit-learn can dereference out of bounds during .predict(), crashing the process. The raised UntrustedTypesFoundException now explains the risk for these types. #535 by Adrin Jalali.

  • Fix a security issue where loading a numpy.random.Generator would resolve and call a numpy.random attribute whose name was taken from the file, even though get_untrusted_types reported nothing untrusted for that file. The bit generator type is now surfaced to the audit, and only genuine bit generators are instantiated when loading. #536 by Adrin Jalali.

  • Fix a security issue where skops update (skops.cli) would load a file while blindly trusting every type declared in it, allowing a malicious .skops file to execute arbitrary code. skops update now refuses to load types that are not trusted by default; use the new --trusted option to explicitly allow types you have reviewed. #533 by Adrin Jalali.

  • Support persisting scipy sparse arrays (e.g. csr_array) through the same efficient npz format used for sparse matrices, and handle the scipy.special ufunc wrappers introduced in scipy 2.0. This adds compatibility with scipy 2.0 and scikit-learn 1.9. #534 by Adrin Jalali.

  • Fix importing skops.io raising ModuleNotFoundError when an already imported package exposes lazily loaded attributes whose optional dependencies are missing, for example transformers without torchvision. Trusted type discovery now skips such attributes. #528 by Adrin Jalali.

v0.14

  • Trust internal scikit-learn types needed by GradientBoosting and HistGradientBoosting models, so they no longer surface as untrusted types. #513 by cakedev0 and Adrin Jalali.

v0.13

  • Card now requires a new parameter, allow_pickle, to call get_model with models that are not .skops files. This change is to mitigate security risks associated with pickles. #485 by Io_no.

v0.12

  • huggingface_hub dependency is now optional. #462 by Adrin Jalali.

  • Objects’ __reduce__ is used when the output of it is of the form (type, (constructor_args,) where type is the same as the type(obj). #467 by Adrin Jalali.

  • MethodNode and OperatorNode have a hardened audit now, removing certain security vulnerabilities. #482 by Adrin Jalali.

v0.11

v0.10

v0.9

v0.8

  • Adds the abillity to set the Section.folded property when using Card.add(). #361 by Thomas Lazarus.

  • Add the CLI command to update Skops files to the latest Skops persistence format. (cli._update.main()). #333 by Edoardo Abati

  • Fix a bug that prevented persisting np.mean when using numpy>=1.25.0. #373 by Adrin Jalali.

v0.7

v0.6

v0.5

  • Add CLI entrypoint support (cli.entrypoint.main_cli()) and a command line function to convert Pickle files to Skops files (cli._convert.main()). #249 by Erin Aho

  • Support more array-like data types for tabular data and list-like data types for text data. #179 by Francesco Cariaggi.

  • Add an option use_intelex to skops.hub_utils.init() which, when enabled, will result in the Hugging Face inference API running with Intel’s scikit-learn intelex library, which can accelerate inference times. #267 by Benjamin Bossan.

  • Model cards that have been written into a markdown file can now be parsed back into a skops.card.Card object and edited further by using the skops.card.parse_modelcard() function. #257 by Benjamin Bossan.

v0.4

  • io.dump() and io.load() now work with file like objects, which means you can use them with the with open(...) as f: dump(obj, f) pattern, like you’d do with pickle. #234 by Benjamin Bossan.

  • All scikit-learn estimators are trusted by default. #237 by Edoardo Abati.

  • Add model_format argument to skops.hub_utils.init() to be stored in config.json so that we know how to load a model from the repository. #242 by Merve Noyan.

  • Persistence now supports bytes and bytearrays, added tests to verify that LightGBM, XGBoost, and CatBoost work now. #244 by Benjamin Bossan.

  • card.Card now allows to add content to existing sections, using a / to separate the subsections. E.g. use card.add(**{"Existing section/New section": "content"}) to add “content” a new subsection called “New section” to an existing section called “Existing section”. #203 by Benjamin Bossan.

v0.3

  • Utility function to add arbitrary files to be uploaded to the hub by using hub_utils.add_files(). #123 by Benjamin Bossan.

  • Add private as an optional argument to skops.hub_utils.push() to optionally set the visibility status of a repo when pushing to the hub. #130 by Adrin Jalali.

  • First release of the skops secure persistence feature (#128) by Adrin Jalali and Benjamin Bossan. Visit Secure persistence with skops for more information. This feature is not production ready yet but we’re happy to receive feedback from users.

  • Fix a bug that resulted in markdown tables being rendered incorrectly if entries contained line breaks. #156 by Benjamin Bossan.

  • Raise an error instead of warning the user if a given model file is empty. #214 by Adrin Jalali.

  • Use huggingface_hub v0.10.1 for model cards, drop modelcards dependency. #162 by Benjamin Bossan.

  • Add source links to API documentation. #172 by Ayyuce Demirbas.

  • Add support to load model if given Path/str to model argument in skops.card . #205 by Prajjwal Mishra.

v0.2

  • Tables, e.g. cross-validation results, can now be added to model cards using the Card.add_table() method. #90 by Benjamin Bossan.

  • Add method Card.render() which returns the model card as a string. #94 by Benjamin Bossan.

  • Make skops.hub_utils.init() atomic. Now it doesn’t leave a trace on the filesystem if it fails for some reason. #60 by Adrin Jalali

  • When adding figures or tables, it’s now possible to set folded=True to render the content inside a details tag. #108 by Benjamin Bossan.

  • Add skops.hub_utils.get_model_output to get the model’s output using The Hugging Face Hub’s inference API, and return an array with the outputs. #105 by Adrin Jalali.

v0.1

This is the first release of the library. It include two main modules:

  • skops.hub_utils: tools to create a model repository to be stored on Hugging Face Hub, mainly through skops.hub_utils.init() and skops.hub_utils.push.

  • skops.card: tools to create a model card explaining what the model does and how it should be used. The model card can then be stored as the README.md file on the Hugging Face Hub, with pre-populated metadata to help Hub understand the model.

Contributors

Adrin Jalali, Merve Noyan, Benjamin Bossan, Ayyuce Demirbas, Prajjwal Mishra, Francesco Cariaggi, Erin Aho, Thomas Lazarus