Download Dataset
Download an existing dataset using the Knowledgator TypeScript / JavaScript SDK.
The platform provides a secure HTTP 307 redirect to object storage, which the SDK resolves automatically. The SDK supports two primary modes:
- Direct Download (
kg.datasets.download): Save directly to a local file or parse into an in-memory object. - Streaming Download (
kg.datasets.streamDownload): Stream chunks with an async generator (AsyncGenerator<Uint8Array>), ideal for memory efficiency and large datasets.
Direct Download (kg.datasets.download)
1. Save Directly to Disk (filePath)
Streams the file from storage directly into the local destination file, automatically creating parent directories if needed:
import { createClient } from '@knowledgator/typescript-sdk';
const kg = createClient();
const savedPath = await kg.datasets.download('6babd48a-2256-4466-b16b-9fd45ba59688', {
filePath: './data/dataset.json'
});
console.log('Dataset saved to:', savedPath);
2. Parse Directly into Memory (returnData: true)
Directly parses the dataset content into a JavaScript array/object:
import { createClient } from '@knowledgator/typescript-sdk';
const kg = createClient();
const data = await kg.datasets.download('6babd48a-2256-4466-b16b-9fd45ba59688', {
returnData: true
});
console.log('Loaded samples:', Array.isArray(data) ? data.length : typeof data);
3. Retrieve Raw Bytes (Uint8Array)
When neither filePath nor returnData is passed, the method returns the entire dataset as a Uint8Array:
import { createClient } from '@knowledgator/typescript-sdk';
const kg = createClient();
const rawBytes = await kg.datasets.download('6babd48a-2256-4466-b16b-9fd45ba59688');
console.log('Downloaded bytes:', (rawBytes as Uint8Array).length);
Streaming Download (kg.datasets.streamDownload)
For large datasets, streamDownload returns an AsyncGenerator<Uint8Array>, allowing you to stream data chunk by chunk without loading the entire payload into RAM:
import { createClient } from '@knowledgator/typescript-sdk';
import { createWriteStream } from 'fs';
const kg = createClient();
const fileStream = createWriteStream('./large_dataset.json');
for await (const chunk of kg.datasets.streamDownload('6babd48a-2256-4466-b16b-9fd45ba59688', { chunkSize: 65536 })) {
fileStream.write(chunk);
}
fileStream.end();
console.log('Streaming download completed.');
Method Parameters
kg.datasets.download(datasetId, options)
| Parameter | Type | Required | Description |
|---|---|---|---|
datasetId | string | Yes | Target dataset UUID. |
options.filePath | string | No | Local destination file path. Streams content directly to disk. |
options.returnData | boolean | No | If true, parses JSON content and returns a JavaScript object/array. Defaults to false. |
kg.datasets.streamDownload(datasetId, options)
| Parameter | Type | Required | Description |
|---|---|---|---|
datasetId | string | Yes | Target dataset UUID. |
options.chunkSize | number | No | Chunk size in bytes (defaults to 65536 / 64 KB). |