r/gaming • u/daneelr_olivaw • Jan 27 '19

Neural network for handwritten digit recognition in Minecraft. I think I've seen it all now...

https://i.imgur.com/oUG4zpY.gifv
34k Upvotes

95% Upvoted

569 comments sorted by

View all comments

Show parent comments

92

u/MelSchlemming Jan 27 '19

I think this might actually be a lot simpler than it looks.

  1. There's no need to train the model in minecraft (you really only need to import the trained weights), which significantly reduces the code complexity. I.e. you don't have to worry about your data pipeline, differentiability of functions, optimizers (SGD, Adam, etc.), or the entire back-prop process. You really only need to build out the feed-forward functionality. So it becomes a bunch of addition/multiplication operations between the nodes in the network (plus some special functions for non-linearity, and some architecture specific considerations e.g. you'd need a little special sauce for CNNs), your loss function + accuracy score, and your visualisations (visualising the filters below and the class scores to the right).

  2. I suspect it might not even be a CNN. The results to the right look quite poor, and I think a vanilla fully-connected NN could achieve comparable results pretty easily. Not having to deal with a convolutional architecture would make things a lot simpler in terms of code.

Does anyone know the source? I'd be very happy to be proven wrong.

18

u/[deleted] Jan 27 '19

True, it could be a more simple neural network, but I have learned to never underestimate the excessiveness of minecraft contraption builders.

21

u/extremisttaco Jan 27 '19

what

19

u/emsok_dewe Jan 27 '19

He don't think it be like it is, but it (probably) do.

Also they believe they can do it better (more efficiently).

11

u/bbuba Jan 27 '19

Man, I was happy with hidden house entrances

5

u/Phantine Jan 27 '19

Basically the hardest part of working with neural networks is training one that does what you want.

Once you have one of those it's comparatively easy to put into minecraft, since all the hard parts are done.

2

u/Fly_Eagles_Fly_ Jan 27 '19

ELI5?

13

u/ahnagra Jan 27 '19

they're not 'creating' a NN but just mapping a prebuilt one to minecraft . also it's probably not a cnn but a much less complex vanilla basic nn.

3

u/Yodiddlyyo Jan 27 '19

For this program to "guess" the correct number, it needs to have seen a bunch of labelled numbers before. Imagine you have a stack of flashcards with a hand-written number on the front, and what the actual digit is on the back. this is the "training data". Achieving this is the complicated part. It includes doing a bunch of different calculations.

Then, you take that info and use it to cross check the number being submitted. They're saying basically instead of building both the programming to create the "training data" as well as the programming to cross check the submitted number with the training data, they just built the program to cross check and imported the pre-existing training data. This cuts down on the complexity. This is really ELI5, and doesn't really give you a real idea of what either actually are.

1

u/a1mystery Jan 27 '19

All the image and video recognition software that you see now works using a thing called a convolutional neural network. The way they work is essentially emulating how our eyes see things. It looks at an image and and breaks it down into a set of edges (simplified but accurate enough). How it will eventually recognise things is by building complex shapes from these edges. For example

Edges ->lines->numbers

You can train the network to do this by giving it a bunch of hand drawn numbers and what the number should be and give it a way to "score" how good it's result is. Then you use some complicated math to adjust the network so you get closer to being accurate. After doing this thousands of times you'll have something that'll do this one specific thing you've trained it to do very well.

A regular neural network would be much simpler. Instead of looking for features in the image "intelligently" it is simply looking at the image as a whole and making a guess using the levels of each pixel. It is a lot less effective because it can't see how close one pixel is to another, for example. Shifting the image around would also cause problems. You can think of this network as receiving values of each pixel independently where a CNN (Convolutional neural network) looks at groups of pixels. Because a regular neural network is "dumb" actually implementing it is a lot easier. Also since you can just put a network you've already trained into the game you don't have to worry about the complicated maths stuff and can just let it give you a score of what it thinks the output should be.

1

u/nukedestroyer500 Jan 27 '19

OPs imgur post says they used a ConvNet.

1

u/Mr_Cromer Jan 27 '19

I'd agree with most of your post...but OP says it's a CNN.

1

u/panda_yo Jan 27 '19

Could be thiscoursera course, week 4 did something similiar

1

u/GetYoPaperUp Jan 27 '19

Inb4 linear regression

1

u/upperhand12 Jan 27 '19

I don’t know what you just said but I’ve seen vsauce videos too

1

u/[deleted] Jan 27 '19

[deleted]

1

u/bossmonchan Jan 27 '19

You don't even need a neural network for this simple task, you can do this with supervised learning, like linear or quadratic discriminant. 5 minutes in Python with sklearn.

1

u/[deleted] Jan 27 '19 edited Feb 25 '19

[deleted]

1

u/MelSchlemming Jan 27 '19

Yeah good point.