Skip to main content

Upload Dataset File

Upload a dataset file (JSONL, CSV, Parquet, JSON) to an existing dataset using the Knowledgator Python SDK.

The SDK streams the file directly using multipart form data, supporting both local filesystem paths (file_path) and in-memory streams/bytes (file).


Example Usage​

Streaming directly from disk avoids loading large datasets into RAM:

from knowledgator import Knowledgator

kg = Knowledgator(api_key="YOUR_API_KEY")

response = kg.datasets.upload(
dataset_id="6babd48a-2256-4466-b16b-9fd45ba59688",
file_path="./annotations.jsonl"
)
print("Upload status:", response.status)

2. Upload from In-Memory Binary Stream / Bytes​

from knowledgator import Knowledgator

kg = Knowledgator(api_key="YOUR_API_KEY")

with open("annotations.jsonl", "rb") as f:
response = kg.datasets.upload(
dataset_id="6babd48a-2256-4466-b16b-9fd45ba59688",
file=f
)
print("Upload status:", response.status)

3. Asynchronous Upload (upload_async)​

import asyncio
from knowledgator import Knowledgator

async def main():
kg = Knowledgator(api_key="YOUR_API_KEY")
response = await kg.datasets.upload_async(
dataset_id="6babd48a-2256-4466-b16b-9fd45ba59688",
file_path="./annotations.jsonl"
)
print("Async upload status:", response.status)

asyncio.run(main())

Uploading During Dataset Creation​

You can also create a new dataset and upload its data in a single call:

dataset = kg.datasets.create(
name="ecommerce_ner_dataset",
type="ner",
format="jsonl",
short_description="Annotated NER product dataset",
file_path="./annotations.jsonl"
)
print("Created dataset ID:", dataset.id)

Method Parameters​

kg.datasets.upload(...) / kg.datasets.upload_async(...)​

ParameterTypeRequiredDescription
dataset_idstrYesTarget dataset UUID (2–64 alphanumeric characters with hyphens).
file_pathstr | PathNoPath to local file. Automatically streams binary data line-by-line.
filebytes | BinaryIONoRaw bytes or open binary file handle.
timeout_msintNoRequest timeout in milliseconds (defaults to client configuration).

ℹ️ Note: Either file_path or file must be provided.