cloudpathlib Changelog¶
v0.23.0 (2025-10-07)¶
- Added support for Python 3.14 (Issue #529, PR #530)
- Changed
CloudPath.copyto have the first parameter namedtargetinstead ofdestinationand added newfollow_symlinksandpreserve_metadatakwargs. Breaking change for users that relied on the first parameter being nameddestinationinstead oftarget. - Added
CloudPath.copy_intoto copy a file or directory into another file or directory. - Added
CloudPath.moveto move a file or directory to another location. - Added
CloudPath.move_intoto move a file or directory into another file or directory. - Added
CloudPathInfoandCloudPath.infoto get information about a file or directory. - Added additional no-op kwargs to
mkdir,touch,glob,rglob,statto match pathlib.
v0.22.0 (2025-08-29)¶
- Fixed issue with GS credentials, using default auth enables a wider set of authentication methods in GS (Issue #390, PR #514, thanks @ljyanesm)
- Added support for http(s) urls with
HttpClient,HttpPath,HttpsClient, andHttpsPath. (Issue #455, PR #468) - Added experimental support for patching the builtins
open,os,os.path, andglobto work withCloudPathobjects. It is off by default; see the new "Compatibility" section in the docs for more information. (Issue #128, PR #322) - Added support for
CloudPath(*parts)to create aCloudPathobject from a list of parts (to matchpathlib.Path). This is a potentially breaking change for users that relied on the second arg being theclientinstead of making it an explicit kwarg. (PR #322)
v0.21.1 (2025-05-14)¶
- Fixed
rmtreefail on Azure with nohnsand more than 256 blobs to drop (Issue #509, PR #508, thanks @alikefia)
v0.21.0 (2025-03-03)¶
- Removed support for deprecated env var that had a typo (
CLOUPATHLIB_FILE_CACHE_MODE; you should useCLOUDPATHLIB_FILE_CACHE_MODE). - Added support for
timeoutandretrykwargs forGSClient. (Issue #484, PR #485, thanks @Mchristos) - Fixed
CloudPath(...) / otherto correctly attempt to fall back onother's__rtruediv__implementation, in order to support classes that explicitly support the/with aCloudPathinstance. Previously, this would always raise aTypeErrorifotherwere not astrorPurePosixPath. (PR #479) - Added
md5property toGSPath, updated LocalGSPath to includemd5property, updated mock_gs.MockBlob to includemd5_hashproperty. (Issue #482, PR #483) - Fixed an uncaught exception on Azure Gen2 storage accounts with HNS enabled when used with
DefaultAzureCredential. (Issue #486, PR #487) - Removed support for Python 3.8, which was EOL in October 2024. (PR #502)
v0.20.0 (2024-10-18)¶
- Added support for custom schemes in CloudPath and Client subclases. (Issue #466, PR #467)
- Fixed
ResourceNotFoundErroron Azure gen2 storage accounts with HNS enabled and issue that some Azure credentials do not haveaccount_name. (Issue #470, Issue #476, PR #478) - Added support for Python 3.13 (Issue #472, PR #474):
.full_matchadded.from_uriaddedfollow_symlinkskwarg added tois_fileadded as no-opfollow_symlinkskwarg added tois_diradded as no-opnewlinekwarg added toread_textrecurse_symlinkskwarg added toglobadded as no-oppatternparameter forglobcan be PathLikerecurse_symlinkskwarg added torglobadded as no-oppatternparameter forrglobcan be PathLike.parserproperty added
v0.19.0 (2024-08-29)¶
- Fixed an error that occurred when loading and dumping
CloudPathobjects using pickle multiple times. (Issue #450, PR #454, thanks to @kujenga) - Fixed typo in
FileCacheModewhere values were being filled by environment variableCLOUPATHLIB_FILE_CACHE_MODEinstead ofCLOUDPATHLIB_FILE_CACHE_MODE. (PR #424, thanks to @mynameisfiber) - Fixed
CloudPathcleanup viaCloudPath.__del__whenClientencounters an exception during initialization and does not create afile_cache_modeattribute. (Issue #372, thanks to @bryanwweber) - Removed support for Python 3.7 and pinned minimal
boto3version to Python 3.8+ versions. (PR #407) - Changed
GSClientto use the nativeexists()method from the Google Cloud Storage SDK. (PR #420, thanks to @bachya) - Changed default clients to be lazily instantiated (Issue #428, PR #432)
- Fixed
download_toto check for the existence of the cloud file (Issue #430, PR #433) - Added env vars
CLOUDPATHLIB_FORCE_OVERWRITE_FROM_CLOUDandCLOUDPATHLIB_FORCE_OVERWRITE_TO_CLOUD. (Issue #393, PR #437) - Fixed
globforcloudpathlib.local.LocalPathand subclass implementations to match behavior of cloud versions for parity in testing. (Issue #415, PR #436) - Changed how
cloudpathlib.local.LocalClientand subclass implementations track the default local storage directory (used to simulate the cloud) used when no local storage directory is explicitly provided. (PR #436, PR #462)- Changed
LocalClientso that client instances using the default storage access the default local storage directory through theget_default_storage_dirrather than having an explicit reference to the path set at instantiation. This means that callingget_default_storage_dirwill reset the local storage for all clients using the default local storage, whether the client has already been instantiated or is instantiated after resetting. This fixes unintuitive behavior wherereset_local_storagedid not reset local storage when using the default client. (Issue #414) - Added a new
local_storage_dirproperty toLocalClient. This will return the current local storage directory used by that client instance. by reference through the `get_default_ rather than with an explicit.
- Changed
- Refined the return type annotations for
CloudPath.open()to match the behavior ofpathlib.Path.open(). The method now returns specific types (TextIOWrapper,FileIO,BufferedRandom,BufferedWriter,BufferedReader,BinaryIO,IO[Any]) based on the providedmode,buffering, andencodingarguments. (Issue #465, PR #464) - Added Azure Data Lake Storage Gen2 support (Issue #161, PR #450), thanks to @M0dEx for PR #447 and PR #449
v0.18.1 (2024-02-26)¶
- Fixed import error due to incompatible
google-cloud-storageby not usingtransfer_managerif it is not available. (Issue #408, PR #410)
Includes all changes from v0.18.0.
Note: This is the last planned Python 3.7 compatible release version.
0.18.0 (2024-02-25) (Yanked)¶
- Implement sliced downloads in GSClient. (Issue #387, PR #389)
- Implement
as_urlwith presigned parameter for all backends. (Issue #235, PR #236) - Stream to and from Azure Blob Storage. (PR #403)
- Implement
file:URI scheme support forAnyPath. (Issue #401, PR #404)
Note: This version was yanked due to incompatibility with google-cloud-storage <2.7.0 that causes an import error.
0.17.0 (2023-12-21)¶
- Fix
S3Clientcleanup viaClient.__del__whenS3Clientencounters an exception during initialization. (Issue #372, PR #373, thanks to @bryanwweber) - Skip mtime checks during upload when force_overwrite_to_cloud is set to improve upload performance. (Issue #379, PR #380, thanks to @Gilthans)
v0.16.0 (2023-10-09)¶
- Add "CloudPath" as return type on
__init__for mypy issues. (Issue #179, PR #342) - Add
with_stemto all path types when python version supports it (>=3.9). (Issue #287, PR #290, thanks to @Gilthans) - Add
newlineparameter to thewrite_textmethod to align topathlibfunctionality as of Python 3.10. PR #362, thanks to @pricemg. - Add support for Python 3.12 (PR #364)
- Add
CLOUDPATHLIB_LOCAL_CACHE_DIRenv var for setting local_cache_dir default for clients (Issue #352, PR #357) - Add
CONTRIBUTING.mdinstructions for contributors (Issue #213, PR #367)
v0.15.1 (2023-07-12)¶
- Compatibility with pydantic >= 2.0.0. (PR #349)
v0.15.0 (2023-06-16)¶
- Changed return type for
CloudPathMeta.__call__to fix problems with pyright/pylance (PR #330) - Make
CloudPath.is_valid_cloudpatha TypeGuard so that type checkers can know the subclass ifis_valid_cloudpathis called (PR #337) - Added
follow_symlinkstostatfor 3.11.4 compatibility (see bpo 39906) - Add
follow_symlinkstois_dirimplementation for CPythonglobcompatibility (see CPython PR #104512)
v0.14.0 (2023-05-13)¶
- Changed to pyproject.toml-based build.
- Changed type hints from custom type variable
DerivedCloudPathtotyping.Self(PEP 673). This adds a dependency on the typing-extensions backport package from Python versions lower than 3.11. - Fixed a runtime key error when an S3 object does not have the
Content-Typemetadata set. (Issue #331, PR #332)
v0.13.0 (2023-02-15)¶
- Implement
file_cache_modes to give users finer-grained control over when and how the cache is cleared. (Issue #10, PR #314) - Speed up listing directories for Google Cloud Storage. (PR #318)
- Add compatibility for Python 3.11 (PR #317)
v0.12.1 (2023-01-04)¶
- Fix glob logic for buckets; add regression test; add error on globbing all buckets (Issue #311, PR #312)
v0.12.0 (2022-12-30)¶
- API Change:
S3Clientsupports anextra_argskwarg now to pass extra args down toboto3functions; this enables Requester Pays bucket access and bucket encryption. (Issues #254, #180; PR #307) - Speed up glob! (Issue #274, PR #304)
- Ability to list buckets/containers a user has access to. (Issue #48, PR #307)
- Remove overly specific status check and assert in production code on remove. (Issue #212, PR #307)
- Update docs, including accessing public buckets. (Issue #271, PR #307)
v0.11.0 (2022-12-18)¶
- API change: Add
ignoreparameter toCloudPath.copytreein order to matchshutilAPI. (Issue #145, PR #272) - Use the V2 version for listing objects
list_objects_v2inS3Client. (Issue #155, PR #302) - Add ability to use
.existsto check for a raw bucket/container (no additional path components). (Issue #291, PR #302) - Prevent data loss when renaming by skipping files that would be renamed to the same thing. (Issue #277, PR #278)
- Speed up common
glob/rglobpatterns. (Issue #274, PR #276)
v0.10.0 (2022-08-18)¶
- API change: Make
staton base class method instead of property to followpathlib(Issue #234, PR #250) - Fixed "S3Path.exists() returns True on partial matches." (Issue #208, PR #244)
- Make
AnyPathsubclass ofAnyPath(Issue #246, PR #251) - Skip docstrings if not present to avoid failing under
-00(Issue #238, PR #249) - Add
py.typedfile so mypy runs (Issue #243, PR #248)
v0.9.0 (2022-06-03)¶
- Added
absolutetoCloudPath(does nothing asCloudPathis always absolute) (PR #230) - Added
resolvetoCloudPath(does nothing asCloudPathis resolved in advance) (Issue #151, PR #230) - Added
relative_totoCloudPathwhich returns aPurePosixPath(Issue #149, PR #230) - Added
is_relative_totoCloudPath(Issue #149, PR #230) - Added
is_absolutetoCloudPath(always true asCloudPathis always absolute) (PR #230) - Accept and delegate
read_textparameters to cached file (PR #230) - Added
exist_okparameter totouch(PR #230) - Added
missing_okparameter tounlink, which defaults to True. This diverges from pathlib to maintain backward compatibility (PR #230) - Fixed missing root object entries in documentation's Intersphinx inventory (Issue #211, PR #237)
v0.8.0 (2022-05-19)¶
- Fixed pickling of
CloudPathobjects not working. (Issue #223, PR #224) - Added functionality to [push the MIME (media) type to the content type property on cloud providers by default. (Issue #222, PR #226)
v0.7.1 (2022-04-06)¶
- Fixed inadvertent inclusion of tests module in package. (Issue #173, PR #219)
v0.7.0 (2022-02-16)¶
- Fixed
globandrglobfunctions by using pathlib's globbing logic rather than fnmatch. (Issue #154) - Fixed
iterdirto not include self. (Issue #15) - Fixed error when calling
suffixandsuffixeson a cloud path with no suffix. (Issue #120) - Changed
parentsreturn type from list to tuple, to better match pathlib's tuple-like_PathParentsreturn type. - Remove support for Python 3.6. Issue #186
v0.6.5 (2022-01-25)¶
- Fixed error when "directories" created on AWS S3 were reported as files. (Issue #148, PR #190)
- Fixed bug where GCE machines can instantiate default client, but we don't attempt it. (Issue #191
- Support
AWS_ENDPOINT_URLenvironment variable to set theendpoint_urlforS3Client. (PR #193)
v0.6.4 (2021-12-29)¶
- Fixed error where
BlobPropertiestype hint causes import error if Azure dependencies not installed.
v0.6.3 (2021-12-29)¶
- Fixed error when using
rmtreeon nested directories for Google Cloud Storage and Azure Blob Storage. (Issue #184, PR #185) - Fixed broken builds due mypy errors in azure dependency (PR #177)
- Fixed dev tools for building and serving documentation locally (PR #178)
v0.6.2 (2021-09-20)¶
- Fixed error when importing
cloudpathlibfor missingbotocoredependency when not installed with S3 dependencies. (PR #168)
v0.6.1 (2021-09-17)¶
- Fixed absolute documentation URLs to point to the new versioned documentation pages.
- Fixed bug where
no_sign_requestcouldn't be used to download files since our code required list permissions to the bucket to do so. (Issue #169, PR #168).
v0.6.0 (2021-09-07)¶
- Added
no_sign_requestparameter toS3Clientinstantiation for anonymous requests for public resources on S3. See documentation for more details. (#164)
v0.5.0 (2021-08-31)¶
- Added
boto3_transfer_configparameter toS3Clientinstantiation, which allows passing aboto3.s3.transfer.TransferConfigobject and is useful for controlling multipart and thread use in uploads and downloads. See documentation for more details. (#150)
v0.4.1 (2021-05-29)¶
- Added support for custom S3-compatible object stores. This functionality is available via the
endpoint_urlkeyword argument when instantiating anS3Clientinstance. See documentation for more details. (#138 thanks to @YevheniiSemendiak) - Added
CloudPath.upload_fromwhich uploads the passed path to this CloudPath (issuse #58) - Added support for common file transfer functions based on
shutil. Issue #108. PR #142. CloudPath.copycopy a file from one location to another. Can be cloud -> local or cloud -> cloud. Ifclientis not the same, the file transits through the local machine.CloudPath.copytreereucrsively copy a directory from one location to another. Can be cloud -> local or cloud -> cloud. UsesCloudPath.copyso ifclientis not the same, the file transits through the local machine.
v0.4.0 (2021-03-13)¶
- Added rich comparison operator support to cloud paths, which means you can now use them with
sorted. (#129) - Added polymorphic class
AnyPathwhich creates a cloud path orpathlib.Pathinstance appropriately for an input filepath. See new documentation for details and example usage. (#130) - Added integration with Pydantic. See new documentation for details and example usage. (#130)
- Exceptions: (#131)
- Changed all custom
cloudpathlibexceptions to be located in newcloudpathlib.exceptionsmodule. - Changed all custom
cloudpathlibexceptions to subclass from new baseCloudPathException. This allows for easy catching of any custom exception fromcloudpathlib. - Changed all custom exceptions names to end with
Erroras recommended by PEP 8. - Changed various functions to throw new
CloudPathFileExistsError,CloudPathIsADirectoryErrororCloudPathNotADirectoryErrorexceptions instead of a genericValueError. - Removed exception exports from the root
cloudpathlibpackage namespace. Import fromcloudpathlib.exceptionsinstead if needed.
- Changed all custom
- Fixed
download_tomethod to handle case when source is a file and destination is a directory. (#121 thanks to @genziano) - Fixed bug where
hash(...)of a cloud path was not consistent with the equality operator. (#129) - Fixed
AzureBlobClientinstantiation to throw new errorMissingCredentialsErrorwhen no credentials are provided, instead ofAttributeError.LocalAzureBlobClienthas also been changed to accordingly error under those conditions. (#131) - Fixed
GSClientto instantiate as anonymous with public access only when instantiated with no credentials, instead of erroring. (#131)
v0.3.0 (2021-01-29)¶
- Added a new module
cloudpathlib.localwith utilities for mocking cloud paths in tests. The module has "Local" substitute classes that use the local filesystem in place of cloud storage. See the new documentation article "Testing code that uses cloudpathlib" to learn more about how to use them. (#107)
v0.2.1 (2021-01-25)¶
- Fixed bug where a
NameErrorwas raised if the Google Cloud Storage dependencies were not installed (even if using a different storage provider).
v0.2.0 (2021-01-23)¶
- Added support for Google Cloud Storage. Instantiate with URIs prefixed by
gs://or explicitly using theGSPathclass. (#113 thanks to @wolfgangwazzlestrauss) - Changed backend logic to reduce number of network calls to cloud. This should result in faster cloud path operations, especially when dealing with many small files. (#110, #111)
v0.1.2 (2020-11-14)¶
- Fixed
CloudPathinstantiation so that reinstantiating with an existingCloudPathinstance will reuse the same client, if a new client is not explicitly passed. This addresses the edge case of non-idempotency when reinstantiating aCloudPathinstance with a non-default client. (#104)
v0.1.1 (2020-10-15)¶
- Fixed a character-encoding bug when building from source on Windows. (#98)
v0.1.0 (2020-10-06)¶
- Initial release of cloudpathlib with support for Amazon S3 and Azure Blob Storage! 🎉