I want to show here an example of the impressive work that DT must be doing behind the scenes when interacting with LLMs. Personally, I’m still investigating the possibilities of DT + AI, and the deeper I dive, the more excited I am.
I use LM Studio as an offline tool to connect DT with AI, and after experimenting with a lot of local models, I’m using the MLX Mistral Nemo Instruct 2007 model in 4-bit quantization with 12B parameters, which fits comfortably on both my Mac Mini M2 Pro with 32GB RAM and my new Mac Studio M4 Max with 36GB RAM, reaching around 25 tokens/s on the former and 50 t/s on the latter.
(A clarification here: there’s hardly any performance difference between an M2 and an M4. What matters is the number of GPU cores. Since my M2 has 16 cores, and the M4 Max has 32, the token generation speed is double. With an M3 Ultra, therefore, it would be 36 t/s for this model. What I mean is that there’s no difference in generative AI performance related to Apple’s processor generations, and only the number of GPU cores matters, as the improvements between generations are found in the NPU, which currently only Apple can “access”.)
I asked Mistral the following question from the LM Studio chat:
Hazme un listado de las novelas juveniles de Heinlein por orden cronológico de publicación. Pon el título original en inglés. Busca en internet si no lo sabes.
In English:
Make me a list of Heinlein’s juvenile novels in chronological order of publication. Put the original title in English. Search the internet if you don’t know.
The answer was, well, how to say, absurdly incorrect:
- “For Us, The Living: A Comedy of Customs” (1939)
- “Sixth Column” (1941, también conocida como “The Day After Tomorrow”)
- “Starship Troopers” (1959)
- “Stranger in a Strange Land” (1961)
- “Rocket Ship Galileo” (1947, conocida también como “The Young Atomic Engineers”)
- “Space Cadet” (1948)
- “The Rolling Stones” (1952)
- “Red Planet” (1949, también conocida como “The Earth Coincidence Society”)
- " Farmer in the Sky" (1950)
- “The Puppet Masters” (1951)
- “Between Planets” (1948, también conocida como “The Black Pussy”)
- “Time Enough for Love” (1973, aunque es más una serie de historias interconectadas que una novela juvenil)
- “I Will Fear No Evil” (1970)
Now enters DT4, remember same model in chat window:
For those who don’t know Robert A. Heinlein’s work, the answer given is 100% perfect. “Podkayne of Mars” is missing, which some experts say is a Juvenile, and others don’t.
I take my hat off to the people at DT.
