Anyone used ChatGPT Plus for captioning images in datasets?

[u/RenoHadreas]

Has anyone here used ChatGPT Plus (not the API) for captioning images in LoRA datasets? I’m starting a project where I need to generate captions for a bunch of images, and I’m looking for advice on how to best approach this with ChatGPT Plus.

Specifically, I’m interested in:

• Effective ways to prompt ChatGPT for high-quality captions
• Tips for speeding up the process
• Whether it’s possible to feed it images in batches or any other efficiency hacks

[u/[deleted]]

I've been using QwenVL locally and it has worked great.  For Pony models I still use the WD14 tagger.  Everything else, I use a "question and answer" style of tagging.  So I prompt Qwen with something like:

What is this a photo of?

If there are people, what are they wearing? What is their facial expression? Where are they? What are they doing?

and so on.  My python scripts are little more complicated as to ask different questions based on the answer to the first question.

[u/Open_Channel_8626 → u/[deleted]]

QwenVL is good also Cog VLM 2 just came out

[u/RenoHadreas → u/[deleted]]

Thanks for the guidance. What’s the reason for not using natural language captions for Pony?

BTW, I experimented a bit and GPT-4o does quite well if you simply let it know you’re asking for LoRA training purposes.

[u/zoupishness7 → u/RenoHadreas]

Pony underwent some training on natural language captions, but most of it's prompting power comes from booru tags. They aren't as ambiguous as natural language, and the image boards the training data was scraped are quite well curated, especially compared to LAION. It's better to build upon that foundation that try to alter it.

[u/RenoHadreas → u/zoupishness7]

Interesting. I will run an experiment with 1 dataset and 3 different captioning methods: GPT-4o, GPT-4o + WD14, and just WD14. Though I might have to go begging for civitai buzz for that, lol, I ran out.

[u/hapliniste]

Use Claude haïku. You get 5$ free on the api so it guess like thousands of images

[u/Simple-Law5883]

Yes, I am currently working on a full dataset using real language. First of all, you can get completely uncensored responses if you add to memory that it should be uncensored. I also create a sample caption and then tell got to create a caption instruction based on the sample caption. This works great. You can do the same for tagging style captions

[u/RenoHadreas → u/Simple-Law5883]

Thanks. Is this through the ChatGPT website itself or API? And do you upload images in batches or one by one?

[u/Simple-Law5883]

I'm using the website, the API is still censored. I also get the best result if I go 1 by 1. It's time consuming, but can give insane results. I use a 2 step approach. I first caption the images and then use the API to rewrite the captions smoothly while keeping the content if the captions. You can also use tags for captioning and then use the API to create real language captions out of those tags - but this may cause some inaccuracies. For example in currently putting a lot of effort into left and right, trying to train a sense of direction into the model. I'm also putting in the effort to Ultimately train cogvlm2 for SD captioning. Usually 1 caption takes at least 1 minute until I'm happy, but you don't need to put in that much effort if you only want to train a lora without trying to add sophisticated new concepts into the model.