I’m using DEVONthink 4.3.2 with the OpenAI (Compatible) provider and an Academic Cloud API endpoint. The model is Mistral Medium 3.5 128B, for which Academic Cloud provides a 256K context window.
However, DEVONthink only allows me to set the Context Window to a maximum of 131,072 tokens.
I initially wondered whether this was a general DEVONthink limitation, but that doesn’t seem to be the case. For example, when using Gemini 3.1 Flash Lite, DEVONthink correctly reports a context window of 1,048,576 tokens.
I also checked DEVONthink’s preferences. For the Mistral model, the context window is stored as:
ChatContextWindow-mistral-medium-3.5-128b-OpenAI (Compatible) = 131072
I tried setting this preference manually to 262144 while DEVONthink was closed. The value was accepted by defaults, but after launching DEVONthink it was reset to 131072.
This makes it look as though the OpenAI (Compatible) provider specifically limits manually configured context windows to 128K, rather than this being a limitation of DEVONthink’s AI subsystem in general.
Would it be possible to increase or remove this 131,072-token limit for OpenAI-compatible providers? There are now several OpenAI-compatible APIs exposing models with context windows larger than 128K, so allowing values such as 256K, 512K, or 1M would be useful.
Thanks!
Development would have to respond more authoritatively but it may be a default size reported by the provider or a default size used if the context window isn’t reported.
Thanks for the fast reply. I also checked Academic Cloud’s /v1/models response. It reports the model and its capabilities, but does not expose the context-window size, so DEVONthink cannot infer the 256K value from model discovery.
For example:
{
“id”: “mistral-medium-3.5-128b”,
“name”: “Mistral Medium 3.5 128B”,
“demand”: 1,
“owned_by”: “chat-ai”,
“input”: [
“text”
],
“created”: 1787131851,
“object”: “model”,
“status”: “ready”,
“output”: [
“text”
]
}
The provider documentation confirms that the model has a 256k context window. This would make the manually configurable Context Window field important for OpenAI-compatible providers, but that field currently appears to be capped at 131,072 tokens.
Thanks!
DEVONthink isn’t reading the documentation. It’s getting information from the API response. And since the response it’s reporting a context window size for the model(s), it’s quite likely the current value is just a safe cap.
Yes, that’s exactly my point. I don’t expect DEVONthink to read the provider’s documentation or otherwise know the context size of an arbitrary model.
Academic Cloud’s /v1/models response does not report a context-window size, so DEVONthink has no way to infer the 256K value from that response.
As far as I can tell, context length is also not part of the model metadata defined by the official OpenAI /v1/models API. Some OpenAI-compatible implementations may expose additional metadata (e.g., Ollama), but a service omitting the context length (e.g., LM Studio) is therefore not unusual.
This is precisely why the manually configurable Context Window setting for OpenAI-compatible providers is useful. My question is simply whether its current 131,072-token upper limit could be increased or removed, so that models with larger context windows can be configured correctly.
Thanks again!
The next release will support context windows up to 1m tokens.
1 Like