Dataset storage
datashard
AvailableA table format fur datasets an blobs that disnae need a database.
Storage for datasets, recordins an muckle files that keeps its history. Records are eekit on an snapshots taen, an a dataset can be read as it stude at onie earlier moment. It replaces loose CSV an JSON dumps, haund-managed folders an databases pit thegither juist tae haud files.
Wha it is for. Python projects that haud ontae recordings, datasets or analytical data an need versioned, readable storage.
- Status
- Available
- Pairt o
- Open soorce
- Licence
- Open soorce, Apache 2.0
- Tags
- PythonSnapshotsTime travelOpen formatDisk or object storage
Whit it daes
-
Append-only, wi snapshots
Acause records are eekit on an snapshotted, ye can read a dataset as it stude at a point in time.
-
Disk or S3
The table format warks on local disk an in object storage, an shiftin atween the twa is a configuration chynge.
-
Parquet unnerneath
Data is stored as Parquet, that onie tool readin Parquet can open.
For documentation, a pilot or an integration, contact us.