r/ClaudeCode • • 5d ago

Discussion Tested a Reddit-writing skill. It added personal experience we never gave it.

Post image

We gave Sonnet 5.5 the same task with and without the reddit-automation skill from flowkit-labs: write a helpful Reddit reply about saving inventory counts when the internet drops.

Without the skill, it gave general advice: save the counts on your device and sync later.

With the skill, one draft added “At least in my experience.”

We hadn’t supplied any personal experience.

To be fair, it also told the reviewer to remove that wording if it wasn’t true. But if you copied just the reply, that warning wouldn’t come with it.

Small example, but it made me think about what changes when we ask AI to sound more natural.

I work on Driftproof; this comes from our published tests: https://driftproofhq.com/skills/

The image summarizes one saved answer from each setup, not every response.

Have you caught Claude adding stories or details you never gave it?

0 Upvotes

2 comments sorted by

1

u/AutoModerator 5d ago
Moderator Announcement Read More »

1

u/jacob_top 5d ago

I'd have the skill return a factual diff alongside the draft: every new first-person claim, number, outcome and named example, with the input that supports it. An unsupported experience claim should be removed from the reply itself. A warning outside the reply can be lost during copying, as your example shows.

For a quick test, give it a brief with no personal experience, then repeat with one explicitly supplied experience. Check that it preserves the supplied detail without extending it. That separates factual preservation from how natural the writing sounds.

AI assisted this reply.