Dataset storage
datashard
AvailableA table format for datasets and blobs that doesnae need a database.
Stor for datasets, recordins an muckle files that keeps its history: records are appendit, snapshots are taen, an a dataset can be read as it stude at ony earlier moment. It taks the place o loose CSV an JSON dumps, haund-managed folders, an databases stood up juist tae haud files.
Wha it is for. Python projects that keep recordings, datasets or analytical data and need versioned, readable storage.
- Status
- Available
- Pairt o
- Open soorce
- Licence
- Open soorce, Apache 2.0
- Tags
- PythonSnapshotsTime traivelOpen formatDisk or object storage
Whit it dis
-
Append-anely, wi snapshots
Records are appendit an snapshottit, sae a dataset can be read as it stuid at a pynt in time.
-
Disk or S3
The same table format works on local disk or in object storage, so moving atween them is a configuration chynge.
-
Parquet unnerneath
Data is stowed as Parquet, sae ony tool that reads Parquet can read it.
For documentation, a pilot or an integration, contact us.