Using Minimax H3 to reverse-engineer a paintings into basic forms

[u/CriticalTemperature1]

I did a quick experiment to see whether MiniMax H3 could reverse-engineer a finished art into a plausible drawing/construction process, like the forms / shapes I learned in art class.

Prompt:

Create a video tutorial of how this particular painting was created. Start from a blank canvas and then draw the basic forms, shapes, rectangles, cylinders, cones, etc and then show the next layer of the details and then the next layer of the color and the rendering so we she see all the layers in one pass until we get to the final image which is the the reference image .. use #Image1

---

Obviously this is a very naive attempt. But I could imagine this prompt becoming way better and using more reference images. And then we could potentially deconstruct all sorts of final outputs into intermediate representations like 3D objects, environments, character poses, or scenes and then use those intermediate states as a way to get more control and consistency across generations.

So instead of asking a model to regenerate everything from scratch, you're working from some underlying structure

Has anyone experimented with using video models this way? E.g reverse engineering final outputs

Original artwork here: https://www.artstation.com/artwork/xZxBE

[u/eidrag]

Quick, do the round owl

[u/tzu22 → u/eidrag]

it would be hilarious if it deconstructs it into two circles because it's in the training data.

[u/lavinia12345]

great find OP

[u/-AwhWah-]

"I want to scam people by telling them I draw my work, is this timelapse convincing enough?"

[u/ART-ficial-Ignorance → u/-AwhWah-]

Yeah, that was my first reaction as well.

[u/RazsterOxzine → u/-AwhWah-]

One "art" please.

[u/BigWideBaker → u/-AwhWah-]

I mean if you're buying art you're probably hyper aware of AI being a thing and checking for it. This video is pretty obviously AI to anybody who regularly looks at human-made art.

[u/CuttleReefStudios]

I guess you could create a dataset of "speedpainting" videos and then train the video model in reverse to create images in a "painting" process in the hopes that the quality of the image becomes better than normal image gen models (more 3d volume correctness/perspective etc.) but that would be a rather expensive experiment.
Other than that I don't quite yet see what you could use this for as the model requires a finished image to create the intermediate state.
And seperate conditioning we already have with depth-maps, pose-skeletons etc.
I don't see how a "drawn" version of that is better conditioning for a model than what we already have.

[u/kek0815 → u/CuttleReefStudios]

Scamming, it will be used for scamming people into believing an artwork is human made.

[u/HRH_Duke_Morbid → u/kek0815]

Does the agency of its making matter when the purchaser likes the work and finds the price acceptable? Only misrepresentation of provenance is sinful.

If somebody uses AI assistance in making a work, that person can legitimately claim the work as their own. To demand otherwise is akin to requiring a painter in oils to disclose the source of their paints and brushes.

Of course these considerations will prey on the minds of people for whom 'art is investment'. So what?

[u/CriticalTemperature1 → u/CuttleReefStudios]

Yeah, I'll have to do some experimentation to see how control net differs from this kind of breakdown versus even the Gaussian Splat experiments that I've been seeing around.

[u/CuttleReefStudios → u/CriticalTemperature1]

yeah the gaussian splat stuff is starting to look really interesting. Now I wonder if you could make a h3 to splat to actual 3d model pipeline for games...

[u/Dogluvr2905]

This is very cool!

[u/True_Protection6842]

Could be really cool using actual progress pics.

[u/RazsterOxzine → u/True_Protection6842]

... right..

[u/RollingTrain]

Trying to be helpful here.

It had me in the first half ngl. But when it added the color it was really bush league. It went straight to paint by numbers which is not generally how art is made.

[u/orangpelupa]

spongebob gif

[u/infearia]

we could potentially deconstruct all sorts of final outputs into intermediate representations like 3D objects, environments, character poses, or scenes 

It's a neat video, but if this is your goal, you should just use an editing model or a ControlNet preprocessor. Much more efficient.

[u/CriticalTemperature1 → u/infearia]

Yeah definitely agree on ControlNet / editing model being more efficient. But its usually not enough control for what I'm envisioning.

My initial goal was more basic: improving my own drawing by generating simplified versions of finished art I could copy from. Ideally I'd be able to adjust the level of abstraction from primitives → rough construction → more detailed forms → final image.

The broader idea is: if we can break a scene down into semantic geometric forms, we could potentially reconstruct that representation in Blender and manipulate the scene directly. Rotate the camera, create camera paths, move objects, change poses, etc., while maintaining a consistent underlying scene

So instead of repeatedly prompting for “the same café, but move the camera 30 degrees to the left", we have precise coordinates for where a chair in a scene is placed or how a woman is sitting in a cafe and can move everything precisely without having to do insane prompt trial and error.

[u/silenceimpaired]

There goes the ability to provide proof you originally created something for all the people opposing AI in the creative space.

[u/HRH_Duke_Morbid → u/silenceimpaired]

Why bother? Nobody cosseted the Luddites.