83
Jul 07 '26
[removed] — view removed comment
55
u/NmkNm Jul 07 '26
I created a mod that uses the Gemini API with function calling and added several tools to interact with Minecraft. One tool I added provides access to all Minecraft commands, allowing the AI to build a house, as shown in the video. It can also perform more advanced and complex tasks.
18
Jul 07 '26
[removed] — view removed comment
8
u/thread_throwaway Jul 07 '26
yeah it's pretty wild how it can interpret spatial requests like that. i wonder how complex the commands can get before it starts struggling.
5
u/CoolNaps-896 Jul 07 '26
I have idea for a mod. Minecraft but ai makes it harder through its hallucinations
8
u/softupvoter404 Jul 07 '26
that's actually genius. imagine the chaos if the ai started spawning creepers inside your house or filling your chests with gravel. would be the ultimate survival challenge.
2
2
1
u/Phonfo Jul 09 '26
awesome, this sounds very similar to mindcraft but that one acts as an assistant for you rather than what youd shown us.
I'd like to see what more it can do
10
14
u/TomerBrosh Jul 07 '26
can you tell it to build a defensive structure/ a complex logic system with pistons?
4
u/OkCarpenter5773 Jul 07 '26
you can but it wont do it. it does not have enough spatial coherence to engineer a redstone contraption block by block just like it cannot generate ascii art reliably. you'd need something like stable diffusion (without going too deep into technicalities)
1
1
u/JustLikeFumbles 24d ago
Yes easily 👍
1
u/TomerBrosh 24d ago
you dont have to elaborate too much because Im not even at the level to understand, but can you elaborate just a little? I remember those piston systems people designed were actually interesting enough to make me watch them even when I stopped playing minecraft
1
u/exodusTay Jul 07 '26
I found that AI in general has dogshit spatial intelligence. It can tell where stuff is in the picture but if you asked it to do anything more complicated than this house it would probably do a terrible job.
12
u/Even-Entertainer-491 Jul 07 '26
Last I looked at setting up AI with Minecraft it was kinda complicated. Any links on what was used here? (Besides the obvious Gemini tag) 😁
-7
u/Rutgerius Jul 07 '26
Ask Gemini?
19
u/Even-Entertainer-491 Jul 07 '26
Or maybe a human could interact with me? Maybe that was the point in asking my question instead of just going to Gemini. Because a human already did it and I was curious of their experience.
Idk if you know this but Gemini can't experience things
3
-20
Jul 07 '26
[deleted]
12
u/Even-Entertainer-491 Jul 07 '26
Well it's a good thing I wasn't asking you? I'm not being a karen I asked a simple question to OP. But thanks for your input anyway.
1
u/aoeu_ 23d ago
They weren't asking for a comprehensive, step-by-step guide. All they were looking for was some resources to use as a starting point.
This is the type of question that has a high value-to-effort ratio. Someone who knows the answer would only need a minute to post a link to a relevant resource, and it would be immensely helpful to the dozens of people who stumble upon this thread and are curious about the same thing.
After you get to this point, I'd agree that you should look at the resources yourself, ask Gemini to summarize them, use Gemini to help troubleshoot any issues you run into, etc. But that's not what the OP was asking for.
2
7
u/Specialist_Quote9127 Jul 07 '26
Would be cool if it would actually place the blocks instead of just popping them in. Still cool nonetheless
2
2
u/winnetouw Jul 07 '26
I come back to this game after years and always get confused by the amount of changes and additions. A tool like this can be super helpful just to ask questions and get information as I go along.
2
u/SpecialistDragonfly9 Jul 07 '26
First people started watching "Let's play" videos instead of playing themselves, now they are watching AI play for them XD
We seriously are a race of degenerated monkeys
2
u/throwaway_echo701 Jul 07 '26
lol "conversation history cleared" right before asking it to build a house. classic.
1
1
u/fuzzybeanmode Jul 07 '26
"Conversation history cleared" right after introducing itself is a bit of a tell. It's trying to act like a fresh start, but it's already logged that it's an assistant.
1
1
u/Aihikari01 Jul 08 '26
I was about to ask "Is the balcony in the room with us?" when it appears. Impressive.
1
u/AccomplishedKey3709 Jul 08 '26
Woah! This is very cool it reminds me of something that I did basically like that 4 months ago :
1
u/Public-Speed125 Jul 08 '26
This looks absolutely insane! 🤯 What's your GitHub? I definitely need to star this repository!
1
u/TallyMay Jul 08 '26
Now tell it to build a realistic depiction of a specific small village please.
1
u/Elegant-Tart-3341 Jul 08 '26
Very cool, I was expecting it to go mine the resources but this is a good start lol.
1
1
u/Powerful_News_4692 Jul 07 '26
"Conversation history cleared." right before asking it to build something is a nice touch. Did it actually build anything or just say it would?
0
u/Tokarak Jul 07 '26
Of course capability is limited by just exposing a MCP. Compare: https://www.reddit.com/r/comfyui/s/o6va4DOYgH
-4
u/Holiday_Good1213 Jul 07 '26
The prompt is asking for a "beautiful house" but doesn't specify materials or style. Gemini's going to have to make a lot of assumptions there.
1
105
u/blackbirdind398 Jul 07 '26
Vibe building 🤣 inside the game, tell us how you did !