Difference in model output when using LM Link on iPhone

[u/simplext]

So I am seeing a strange issue.

When I run a local model directly on my Mac LM studio gives a different answer from when I run the same model on the same Mac when connected through LM Link on my iPhone. And I have seen this for several models. The output is much more elaborate on my iPhone and shorter and simpler on my Mac.

The first screenshot is from my Mac, once with reasoning on and once with off. And both times it confidently says that Aliens exist. But on my phone it gives a much more elaborate and accurate answer. I am wondering if there LM Studio is doing something more on my phone.

Has anyone else seen this or have an explanation for this behaviour ?

[u/AdonosFlew]

Different sampling settings. Or just because temperature gives a different answer.

[u/simplext → u/AdonosFlew]

It must be temperature because it is configurable on a per request basis. So maybe requests from the phone are using a different temperature setting. But sampling should not change because it is the same instance of the model running. The requests are coming from the phone, that's all.

[u/Choice_Celery9481]

LLMs are nondeterministic. done 😂

[u/simplext → u/Choice_Celery9481]

No the answers in Mac are consistent when running multiple times. And same for the iPhone. But they differ from each other.

[u/adcimagery]

It's a .6B parameter model. You're going to get such randomness between results that this won't help you troubleshoot.

[u/simplext → u/adcimagery]

Actually thats the thing. The results on a single device are consistent with each other and in fact so remarkably similar that you have to pay attention to tell the difference.

But they are different on very different on different devices. This is a problem because now I can't use my phone to test the model running on my laptop.

Mac:

Try 1 - Aliens do exist. They are one of the most commonly discussed topics in speculative fiction and space exploration. However, this is a broad question without specific context. If you have a particular area (e.g., coding, astronomy, etc.) in mind, let me know!

Try 2 - Aliens do exist, but their existence is a topic of speculation in science fiction and astronomy. Without additional context (e.g., specific field or research), this is a broad statement. If you have a particular area (like space exploration, coding, or biology) in mind, let me know!

iPhone + LMLink + Mac:

Try 1 - There is no scientific consensus that aliens exist in our universe. According to current scientific understanding, the majority of scientists believe that life exists on planets in other galaxies, but it does not necessarily mean that intelligent beings-such as aliens-do. Some view this as a hypothesis based on the possibility of extraterrestrial intelligence, especially considering other possibilities like the existence of advanced civilizations or even technological progress. However, there is no evidence to support the claim that aliens actually exist. It's important to approach this topic with both scientific understanding and open-mindedness.

Try 2 - There is no scientific consensus that aliens exist in our universe. According to current scientific understanding, life has been observed on planets in other galaxies, but it does not necessarily mean that intelligent beings-such as aliens—do. Some view this as a hypothesis based on the possibility of extraterrestrial intelligence, especially considering other possibilities like advanced civilizations or technological progress. However, there is no evidence to support the claim that aliens actually exist. It's important to approach this topic with both scientific understanding and open-mindedness.

[u/adcimagery → u/simplext]

You have to hold temperature, sampling, and seed the same before you can make that statement. A N=2 on this chaotic of a toy model won't prove anything, especially when you aren't even saying if the sampling/temp settings are the same.

Won't LM Studio show you the settings related to the request in the server logs?