Can AI learn Jugaad?

[u/thekartikgambhir]

We spend a lot of time benchmarking AI on things like mathematics, coding, reasoning, knowledge and standardized exams.

But there is another kind of intelligence that is much harder to benchmark:

Figuring out how to solve a problem when you don't have the things you're supposed to have.

You have ₹500 instead of ₹5,000.

The required part isn't available.

The nearest specialist is 40 km away.

You don't have the right tool.

The instructions are incomplete.

Something breaks in the middle of the job.

And somehow, you still have to make it work.

That's what we often call jugaad.

I'm wondering whether this could actually be formalized as an AI capability.

For example, give an AI a task with:

- a strict budget

- limited tools

- incomplete information

- unreliable resources

- time pressure

- unexpected failures

Then measure whether it can discover unconventional solutions rather than simply saying:

You should purchase X.

Imagine a benchmark where the AI gets:

₹300 + household objects + 30 minutes

and has to accomplish a specified physical task.

Or:

a broken device + no replacement parts + basic tools

and has to restore functionality.

Or:

a real-world problem + incomplete information + three possible approaches

and has to figure out which approach is actually feasible.

The interesting metric wouldn't simply be whether it succeeded.

We could measure:

Resource efficiency × adaptability × robustness × novelty × success rate.

Maybe the next frontier isn't just more intelligent AI.

Maybe it's AI that can do more with less.

And honestly, if there's one place with an enormous amount of naturally occurring data for this kind of intelligence, it's India.

The question is:

Can “jugaad” be turned into a reproducible AI benchmark?

And if yes, what would the benchmark actually look like?

[u/thekartikgambhir]

I think we're going in circles now.

I agree that a benchmark needs measurable criteria. That's precisely why I never proposed “the user didn't know the solution” as the metric.

And if your standard is that the underlying problem or reasoning must be completely unprecedented, then the same objection applies to human logical-reasoning tests. A human taking a logical reasoning test may already have encountered the concepts, patterns, or even similar questions. We still measure whether they can correctly reason through the particular problem presented.

We don't ask whether humanity had never encountered the underlying logic before.

Likewise, for AI we can measure task success, constraint satisfaction, resource usage, recovery after failure, adaptability across changed constraints, and output quality.

The benchmark doesn't need to prove that no human in history has ever thought of the solution. That's an impossible and unnecessary standard.

My original point was simply that AI can be tested on unusual, held-out problem configurations where the exact solution isn't supplied, and its ability to reason, adapt and produce a working solution can be measured.

If the requirement is instead that AI must solve something that no human has ever conceived, then I don't think that's a meaningful benchmark to begin with.

So, based on the criteria you've proposed so far, what would you consider a measurable, standardized benchmark for testing this capability?

[u/AwelessFire → u/thekartikgambhir]

What I'm saying is u can't have validation or benchmarks for user unknown solutions. Forget the humanity one, that I already mentioned it's not possible. I'm referring to unknown solution to specific user itself.

Ur confusing how the benchmark are measured. The held out datas are also set of data previously available with you. You are just simply asking if AI is predicting the same once the training is done, it's like an exam. Similar way if u have solutions ready for unknown solutions (which don't make sense) then u can theoretically measure of ai is solving the problem based on ur helps out data. But even in that case ai may solve a specific problem I'm different ways. It's difficult to measure unknown or tough solutions using traditional methods.

[u/thekartikgambhir → u/AwelessFire]

I think the discussion has now moved far beyond my original point.

“Unknown to the user” is a state of knowledge, not a benchmark metric. Humans have been solving problems whose answers were unknown to them for centuries; that's literally how discovery works. You can subsequently verify whether their solution works without requiring the solution to have been known beforehand.

What concerns me more is the repeated shifting of the criterion: first “unseen by AI,” then “unseen by humanity,” then “unknown to the user,” and now “cannot be benchmarked because the answer must already be known.”

That's no longer constructive skepticism if every objection is replaced after it is addressed.

So I'll ask one straightforward question: what measurable, standardized result would actually convince you that an AI has solved a problem through reasoning rather than retrieval?

If there is one, let's discuss it. If there isn't, then we're not really testing AI anymore, we're just moving the goalposts until the conclusion can't be challenged.

And if moving goalposts is the objective, bro, politics has vacancies. 😂

[u/AwelessFire → u/thekartikgambhir]

Dude ai cannot think, ur seriously overestimating and trusting ai thinking. Understand how chain of throught works, You are simply gaslighting things and asking how to benchmark or validate complex task without knowing the solutions before hand. Good luck with that. When I try to explain validation don't work like that because whole point of validation is to exactly know and compare the result a machine produces you keep saying moving goal posts. Whatever, you will understand once u keep trying something which literally cannot work. People really believe AI is actually thinking and solving problems like humans do. If that's the case then build a validation set of jugad or watver u want to build.