asepharyana
db4f277336
- Introduced corex-storage crate with support for local disk, S3-compatible, and Google Cloud Storage backends. - Implemented StorageDriver trait for various storage backends. - Added LocalFileStorage for local disk operations. - Added S3Storage for S3-compatible object storage with multipart upload support. - Added GcsStorage for Google Cloud Storage operations. - Included error handling for storage operations. - Added tests for each storage backend to ensure functionality. - Created README.md for documentation and usage examples. - Added Apache and MIT licenses for open-source compliance.
corex-storage
A unified storage & file system abstraction so file handling isn't locked to one physical location.
- Local disk (default) — plain files under a root directory.
- S3-compatible object storage (
s3) — Amazon S3, and any S3-compatible endpoint (MinIO included) via a custom endpoint URL. - Google Cloud Storage (
gcs).
All operations are stream-based, so uploading/downloading a huge object never requires holding it entirely in memory.
Usage
use corex_storage::{LocalFileStorage, StorageDriver, bytes_stream, read_to_vec};
let storage = LocalFileStorage::new("/var/data");
storage.put("uploads/report.csv", bytes_stream(data)).await?;
let bytes = read_to_vec(storage.get("uploads/report.csv").await?).await?;
Swap LocalFileStorage::new(path) for S3Storage::connect("bucket").await? or
GcsStorage::connect("bucket").await? to move to a distributed backend
without touching call sites.