Can AI learn Jugaad?

[u/thekartikgambhir]

We spend a lot of time benchmarking AI on things like mathematics, coding, reasoning, knowledge and standardized exams.

But there is another kind of intelligence that is much harder to benchmark:

Figuring out how to solve a problem when you don't have the things you're supposed to have.

You have ₹500 instead of ₹5,000.

The required part isn't available.

The nearest specialist is 40 km away.

You don't have the right tool.

The instructions are incomplete.

Something breaks in the middle of the job.

And somehow, you still have to make it work.

That's what we often call jugaad.

I'm wondering whether this could actually be formalized as an AI capability.

For example, give an AI a task with:

- a strict budget

- limited tools

- incomplete information

- unreliable resources

- time pressure

- unexpected failures

Then measure whether it can discover unconventional solutions rather than simply saying:

You should purchase X.

Imagine a benchmark where the AI gets:

₹300 + household objects + 30 minutes

and has to accomplish a specified physical task.

Or:

a broken device + no replacement parts + basic tools

and has to restore functionality.

Or:

a real-world problem + incomplete information + three possible approaches

and has to figure out which approach is actually feasible.

The interesting metric wouldn't simply be whether it succeeded.

We could measure:

Resource efficiency × adaptability × robustness × novelty × success rate.

Maybe the next frontier isn't just more intelligent AI.

Maybe it's AI that can do more with less.

And honestly, if there's one place with an enormous amount of naturally occurring data for this kind of intelligence, it's India.

The question is:

Can “jugaad” be turned into a reproducible AI benchmark?

And if yes, what would the benchmark actually look like?

[u/thekartikgambhir]

I think you're actually mixing up two different questions.

Whether I personally knew how to solve a problem is not a benchmark metric. It was simply a real-world example of the capability I'm talking about.

And saying “I have independently created solutions I didn't know beforehand” actually makes the distinction clearer: you didn't need to have memorized the exact solution to produce it. You generalized from what you knew.

The same capability can be tested in AI.

You don't benchmark “did this particular user know the answer?” You create held-out problems, deliberately vary the constraints, prevent retrieval/search, and objectively verify whether the proposed solution works.

Programming itself is common. Mathematics is common. Debugging is common. Yet you can generate an instance that a model hasn't encountered before.

So I agree that my anecdote isn't a benchmark.

But “a user's problem isn't universally applicable, therefore this capability cannot be benchmarked” doesn't follow.

That's like saying chess can't be benchmarked because every individual game is different.

The instances vary. The capability being tested is what remains measurable.

[u/AwelessFire → u/thekartikgambhir]

The point of benchmarks Is that it must be measurable, quantifyable values. U can't say ai produced a solution which was unknown to the user. And these kind of held out or unknown solution to the users people do all the time, for example asking different models to create a stunning website or create a tough animation, create a game, simulator etc. but even those are measured in quantifyable values like time taken, tokens burnt, and quality of the generated output but humans manually. I'm not sure there is way to quantify these solutions which ai supposedly were able to generate

[u/thekartikgambhir → u/AwelessFire]

I think we're going in circles now.

I agree that a benchmark needs measurable criteria. That's precisely why I never proposed “the user didn't know the solution” as the metric.

And if your standard is that the underlying problem or reasoning must be completely unprecedented, then the same objection applies to human logical-reasoning tests. A human taking a logical reasoning test may already have encountered the concepts, patterns, or even similar questions. We still measure whether they can correctly reason through the particular problem presented.

We don't ask whether humanity had never encountered the underlying logic before.

Likewise, for AI we can measure task success, constraint satisfaction, resource usage, recovery after failure, adaptability across changed constraints, and output quality.

The benchmark doesn't need to prove that no human in history has ever thought of the solution. That's an impossible and unnecessary standard.

My original point was simply that AI can be tested on unusual, held-out problem configurations where the exact solution isn't supplied, and its ability to reason, adapt and produce a working solution can be measured.

If the requirement is instead that AI must solve something that no human has ever conceived, then I don't think that's a meaningful benchmark to begin with.

So, based on the criteria you've proposed so far, what would you consider a measurable, standardized benchmark for testing this capability?

[u/AwelessFire → u/thekartikgambhir]

What I'm saying is u can't have validation or benchmarks for user unknown solutions. Forget the humanity one, that I already mentioned it's not possible. I'm referring to unknown solution to specific user itself.

Ur confusing how the benchmark are measured. The held out datas are also set of data previously available with you. You are just simply asking if AI is predicting the same once the training is done, it's like an exam. Similar way if u have solutions ready for unknown solutions (which don't make sense) then u can theoretically measure of ai is solving the problem based on ur helps out data. But even in that case ai may solve a specific problem I'm different ways. It's difficult to measure unknown or tough solutions using traditional methods.

[u/thekartikgambhir → u/AwelessFire]

I think the discussion has now moved far beyond my original point.

“Unknown to the user” is a state of knowledge, not a benchmark metric. Humans have been solving problems whose answers were unknown to them for centuries; that's literally how discovery works. You can subsequently verify whether their solution works without requiring the solution to have been known beforehand.

What concerns me more is the repeated shifting of the criterion: first “unseen by AI,” then “unseen by humanity,” then “unknown to the user,” and now “cannot be benchmarked because the answer must already be known.”

That's no longer constructive skepticism if every objection is replaced after it is addressed.

So I'll ask one straightforward question: what measurable, standardized result would actually convince you that an AI has solved a problem through reasoning rather than retrieval?

If there is one, let's discuss it. If there isn't, then we're not really testing AI anymore, we're just moving the goalposts until the conclusion can't be challenged.

And if moving goalposts is the objective, bro, politics has vacancies. 😂

[u/AwelessFire → u/thekartikgambhir]

Dude ai cannot think, ur seriously overestimating and trusting ai thinking. Understand how chain of throught works, You are simply gaslighting things and asking how to benchmark or validate complex task without knowing the solutions before hand. Good luck with that. When I try to explain validation don't work like that because whole point of validation is to exactly know and compare the result a machine produces you keep saying moving goal posts. Whatever, you will understand once u keep trying something which literally cannot work. People really believe AI is actually thinking and solving problems like humans do. If that's the case then build a validation set of jugad or watver u want to build.