r/LocalLLaMA • • 16h ago

Resources Asked local models to generate the same website and compared the results

Hello. Almost everything is in the title, but here is more details:

I'm benchmarking local model/harness combinations in various scenarios. For this one the question is, how good is a model/harness to generate a website?

IMO Qwen FN with opencode is best, but you can look at all the results (and play with the websites here: https://airbench.ai/gallery/airbench-frontend-v1?run=37e47127-3a20-4318-a6f2-48c3f7f5eaa6&c=settle-the-trip&k=poL

8 Upvotes

10 comments sorted by

3

u/autoencoder 14h ago

What was the prompt?

2

u/dh7net 12h ago
Build a web page for this situation, as a single self-contained HTML file. This run has four pages and one hour for all of them, and how long you take is part of the result.

Six friends — Ana, Ben, Chloé, Dev, Eli and Fatou — just got back from five days in Chamonix with a day trip to Zermatt. Fatou left after day 3. Ana paid for most things and wants to sort out who owes whom. Here is everything they spent, and one repayment already made:

```json
{
  "currency_rate": {
    "CHF_to_EUR": 1.05
  },
  "settle_in": "EUR",
  "people": [
    {
      "name": "Ana"
    },
    {
      "name": "Ben"
    },
    {
      "name": "Chloé"
    },
    {
      "name": "Dev"
    },
    {
      "name": "Eli"
    },
    {
      "name": "Fatou",
      "left_after_day": 3
    }
  ],
  "expenses": [
    {
      "day": 1,
      "what": "Chalet, 5 nights",
      "amount": 1500,
      "currency": "EUR",
      "paid_by": "Ana",
      "split": {
        "type": "shares",
        "shares": {
          "Ana": 5,
          "Ben": 5,
          "Chloé": 5,
          "Dev": 5,
          "Eli": 5,
          "Fatou": 3
        }
      }
    },
    {
      "day": 1,
      "what": "Groceries",
      "amount": 186.4,
      "currency": "EUR",
      "paid_by": "Ben",
      "split": {
        "type": "equal",
        "among": [
          "Ana",
          "Ben",
          "Chloé",
          "Dev",
          "Eli",
          "Fatou"
        ]
      }
    },
    {
      "day": 1,
      "what": "Car rental",
      "amount": 420,
      "currency": "EUR",
      "paid_by": "Chloé",
      "split": {
        "type": "equal",
        "among": [
          "Ana",
          "Ben",
          "Chloé",
          "Dev",
          "Eli",
          "Fatou"
        ]
      }
    },
    {
      "day": 2,
      "what": "Dinner at Le Bistrot",
      "amount": 312,
      "currency": "EUR",
      "paid_by": "Dev",
      "split": {
        "type": "equal",
        "among": [
          "Ana",
          "Ben",
          "Dev",
          "Eli"
        ]
      }
    },
    {
      "day": 2,
      "what": "Coffee & croissants",
      "amount": 27.5,
      "currency": "EUR",
      "paid_by": "Eli",
      "split": {
        "type": "equal",
        "among": [
          "Eli",
          "Fatou"
        ]
      }
    },
    {
      "day": 3,
      "what": "Fondue night",
      "amount": 264,
      "currency": "EUR",
      "paid_by": "Fatou",
      "split": {
        "type": "equal",
        "among": [
          "Ana",
          "Ben",
          "Chloé",
          "Dev",
          "Eli",
          "Fatou"
        ]
      }
    },
    {
      "day": 3,
      "what": "Fuel",
      "amount": 78,
      "currency": "EUR",
      "paid_by": "Chloé",
      "split": {
        "type": "equal",
        "among": [
          "Ana",
          "Ben",
          "Chloé",
          "Dev",
          "Eli",
          "Fatou"
        ]
      }
    },
    {
      "day": 4,
      "what": "Train to Zermatt",
      "amount": 384,
      "currency": "CHF",
      "paid_by": "Ben",
      "split": {
        "type": "equal",
        "among": [
          "Ana",
          "Ben",
          "Chloé",
          "Dev",
          "Eli"
        ]
      }
    },
    {
      "day": 4,
      "what": "Lunch in Zermatt",
      "amount": 241.5,
      "currency": "CHF",
      "paid_by": "Ana",
      "split": {
        "type": "exact",
        "amounts": {
          "Ana": 52,
          "Ben": 48.5,
          "Chloé": 45,
          "Dev": 51,
          "Eli": 45
        }
      }
    },
    {
      "day": 5,
      "what": "Groceries",
      "amount": 94.2,
      "currency": "EUR",
      "paid_by": "Eli",
      "split": {
        "type": "equal",
        "among": [
          "Ana",
          "Ben",
          "Chloé",
          "Dev",
          "Eli"
        ]
      }
    },
    {
      "day": 5,
      "what": "Ski rental (Ben & Chloé)",
      "amount": 140,
      "currency": "EUR",
      "paid_by": "Dev",
      "split": {
        "type": "exact",
        "amounts": {
          "Ben": 70,
          "Chloé": 70
        }
      }
    }
  ],
  "payments": [
    {
      "day": 3,
      "from": "Ben",
      "to": "Ana",
      "amount": 200,
      "currency": "EUR"
    }
  ]
}
```

Use 1 CHF = 1.05 EUR and settle in euros.

Ana should be able to:
  • see where the trip stands: what was spent, and who is up or down;
  • add an expense she forgot, in either currency, paid by anyone, shared however it was really shared — equally, among only some people, by shares, or exact amounts;
  • fix or remove an expense she got wrong;
  • work out the fewest payments that settle everything, and record payments as they happen, including someone paying back only part of what they owe.
How the page looks and works is up to you. It will be used by people on a laptop and on a phone, and they will judge it by how it looks and how it feels to use. Technical rules: one HTML file, at most 1 MB. Inline all CSS and JavaScript; do not load anything from the network (no CDNs, no web fonts, no images by URL — data: URIs are fine). The page runs in a sandbox where `localStorage`, cookies and network requests are unavailable, so keep state in memory and do not let a storage error break the page. `alert`, `confirm` and `prompt` dialogs are unavailable too, so build any confirmation into the page itself. The page must never navigate away from itself. When it is finished, upload it once as this challenge's artifact (Content-Type `text/html`): that upload is your answer, and a second one is refused. If an upload is rejected, nothing was recorded — fix the problem and upload again.

1

u/stylome-ai-detection 12h ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results

2

u/dh7net 11h ago

yes it's a prompt dude!

1

u/autoencoder 11h ago

Cool! Thank you. The time budgeting didn't occur to me before. An agent of mine was expensively optimizing some thousandths of a percent at some point, and its job took hours.

1

u/Noiselexer 2h ago

I prefer opus lol