Skip to main content

Open Models

The open_model_inference tool allows LLMs and autonomous agents to execute predictions directly on standalone, full Open Models (such as custom-trained GLiClass, GLiNER, or GLiFormer checkpoints) hosted on the Knowledgator platform.


Tool Overview​

  • Tool Name: open_model_inference
  • Underlying Endpoints:
    • POST /inference/{model_id}/ (standard prediction)
    • POST /inference/{model_id}/{method}/ (sub-method execution, e.g. relations)
  • SDK Methods: kg.model(model_id).predict(...) / kg.model(model_id).call(method, ...)
LoRA Adapters vs. Open Models

Use open_model_inference when you are querying a standalone base model or full fine-tuned model checkpoint identified by its model_id. For lightweight LoRA adapters attached to default platform inference endpoints, use comprehend_inference, extract_inference, or instruct_inference with the adapter_id argument instead.


Parameters​

ParameterTypeRequiredDescription
model_idStringYesThe Open Model identifier (e.g. m-d9be7ede307943439a30c6f814026566).
dataObject / DictionaryYesInference input payload conforming to the model's expected schema (e.g. {"text": "...", "labels": [...]}).
methodStringNoOptional sub-method name for multi-head endpoints (e.g., relations). When omitted, invokes /inference/{model_id}/.
timeout_msIntegerNoRequest timeout in milliseconds (default: 60000 = 60s).

Example Usage​

1. Direct Model Prediction​

Natural Language Prompt to Agent​

"Query open model m-ner-bio-99a to extract disease mentions from: 'Patient presented with acute myocardial infarction.'"

Tool Call Arguments​

{
"name": "open_model_inference",
"arguments": {
"model_id": "m-ner-bio-99a",
"data": {
"text": "Patient presented with acute myocardial infarction.",
"labels": ["disease", "symptom"]
}
}
}

Tool Response​

{
"outputs": [
{
"span": "acute myocardial infarction",
"label": "disease",
"start": 23,
"end": 50,
"score": 0.978
}
]
}

2. Sub-Method Invocation (e.g., relations)​

Tool Call Arguments​

{
"name": "open_model_inference",
"arguments": {
"model_id": "m-relation-corp-12x",
"method": "relations",
"data": {
"text": "Sundar Pichai leads Google in Mountain View.",
"entities": ["person", "company", "location"],
"relations": ["leads", "headquartered_in"]
}
}
}