Problem with 4.1.3 - cannot get LM Studio to work

DEVONthink To Go
1

I am having a similar problem to one posted earlier. I tried to follow the solution in that chat, but cannot get it to work. The frustration here is that I can access my LM Studio server from the desktop version of DEVONthink 4 (running on a different Mac than that running LM Studio), but I cannot set up the iPad version of DEVONthink 4 to do the same. I think the problem is in how DTG sends the chat command to LM Studio, where DT does it properly.

On the desktop version, I go to settings, and in the AI settings I select LM Studio as the Provider, the app sets the API URL as http://localhost:1234/v1/chat/completions, and then I select the model I want to use. This then allows me to chat with the model hosted with LM Studio. That is to say, the AI chat function runs as expected - I can ask questions about a selected document, which is then run through LM Studio running on the other Mac on my network.

In DTG, I go to settings, select LM Studio under “Chat”, then under “Element Labs URL” it loads http://localhost:1234/v1/chat/completions. However, it does not then allow me to select a model. Nor is the chat feature enabled in DTG. The only way I can get a model to show up is to change the URL setting to http://192.168.1.248:1234/v1/models. When I do this, I am able to chose a model, and the chat feature is enabled, but I get the following response in the chat window: “Unexpected endpoint or method. (POST /v1/models)"

It seems to me that something is broken in how DTG 4 loads models and sends the chat request, or the correct settings are not obvious in the settings feature. Any help would be appreciated. I am testing these simultaneously, and it works on my Mac but not on my iPad. At all times, LM Studio has the model loaded that I want to use, and I do not change any of the Local Server settings in LM Studio.

2

Of course. Localhost is … well, the local machine. You do not run your AI on your DTTG device, do you? So use the correct hostname or IP address – as you did. DTTG behaves as you tell it to.

3

@chrillek is correct here: localhost is the current device, which is logically your iPad when you’re on it.

And I will add, you need to ensure LM Studio is exposed to your network. That is in the Server Settings button on its Developer view.

4

Sorry, ha, I see that I misstated what I was doing with my Mac. I had hooked my “remote” Mac into LM studio on that Mac that was itself linked to my Mac running the local LM Studio server over the local network. Hence why localhost was working.

However, I did get the AI Chat feature using to work on DTG on my iPad, but I had to change some of the settings on the server, and in DTG. On the Mac running LM Studio server, I had to enable the CORS function in the Server Settings. On DTG, I selected the Chat as LM Studio, then pointed the URL to the internal IP address for the Mac, with “1234/v1/chat/completions” appended to it, namely “http ://192.168.1.248/v1/chat/completions.” This then populated the model with the one loaded on the server.