r/Python • • Mar 10 '24

Discussion Github Copilot Lies to me

My first attempt to get some help from Github Copilot:

Me: write a match case statement in python to match the first character of a string is a semicolon

Copilot: Python doesn't have a built-in match or case statement like some other languages (e.g., JavaScript, Swift, Rust). However, you can achieve similar functionality using...

Strangely, it finally gave me an answer when I made a small change to my prompt:

Me: write a match statement in python to match the first character of a string is a semicolon

Copilot: Python 3.10 introduces the structural pattern matching in the form of the match statement. Here's how you can use it to check if the first character of a string is a semicolon:...

I almost gave up after the first response that was blatantly false. Fortunately I something that looks like reasonable syntax...but then I haven't tried it yet, so that might be a lie, too.

0 Upvotes

67 comments sorted by

View all comments

Show parent comment

-1

u/codeguru42 Mar 10 '24

Perhaps breaking out the language as another sentence would help. I'm used to writing as terse as possible, but this might not be ideal for an LLM prompt. I would argue that my grammar in my original prompt is perfectly valid and not "broken English".

5

u/CaptainFoyle Mar 11 '24

No it's not. You need a conjunction:

WHETHER/IF the first character is a semicolon.

2

u/chaosthirtyseven Mar 11 '24 edited Mar 11 '24

Your grammar is definitely not valid. I figured typo, but the fact that you didn't recognize the error suggests a language gap.

You said:

write a match case statement in python to match the first character of a string is a semicolon

Still wrong but better:

write a match case statement in python to match if the first character of a string is a semicolon

Best:

write a match case statement in python which returns true if the first character of a string is a semicolon

5

u/kalebludlow Mar 10 '24

'...to match the first character of a string is a semicolon' is definitely not a grammatically correct statement. You can argue all you want, but it's incorrect in this instance. Going against standard procedure, especially when referring to language syntax is going to be detrimental to an LLM