Dataset storage
datashard
AvailableA table format for datasets and blobs that doesnae need a database.
Storage for datasets, recordins an muckle files that keeps its history. Records are eikit on an snapshots taen, an a dataset can be read as it steid at onie earlier moment. It replaces lowse CSV an JSON dumps, haund-managed folders an databases set up anerly tae haud files.
Wha it is for. Python projects that keep recordings, datasets or analytical data and need versioned, readable storage.
- Status
- Available
- Pairt o
- Open soorce
- Licence
- Open soorce, Apache 2.0
- Tags
- PythonSnapshotsTime traivelOpen formatDisk or object storage
Whit it dis
-
Append-anely, wi snapshots
Acause records are eikit on an snapshotted, ye can read a dataset as it steid at a pynt in time.
-
Disk or S3
The table format warks on local disk an in object storage, an flittin atween the twa is a configuration chynge.
-
Parquet unnerneath
Data is stored as Parquet, that onie tuil that reads Parquet can open.
For documentation, a pilot or an integration, contact us.