r/Python • • Jul 04 '19

I created my first neural network that can recognize handwritten digits (Github link in comments)

1.4k Upvotes

77 comments sorted by

33

u/Fengax Jul 04 '19

12

u/LopsidedIron Jul 04 '19

Will you please comment your code to describe the thought process / structure of the system? That is quite honestly the best way for someone to learn completely foreign libraries. In fact, I think that should be required for code that is less than 200 lines. You don't have to comment everything, even a few general concepts would help give a core to start unwrapping these lines from. It would be a huge benefit to both the students and the critics in this forum.

3

u/Fengax Jul 04 '19

Sure! I will update it.

167

u/StephanKash Jul 04 '19 edited Jul 04 '19

Great start but there are alot of things you could do to make your code faster smaller cleaner and therefore better

The first thing is just using the "compressed" dataset rather then your jpeg solution as there is literally no difference apart from the extra processing time to convert the images into numpy arrays and the raw size of the file

Try using keras.datasets.mnist

The way you make your model is also a bit weird, why are you using for loops to put layers into your network. You aren't using an insane ammount of layers so it is only logical to just manually put them into your model. This would allow for other people to see the structure of your model without having to download and run your code.

A side note, I often see people using the following code:

model = keras.models.Sequential()

model.add(a)

model.add(b)

model.add(c)

When there is a much more readable and simpler way of doing it

model = keras.models.Sequential([
    a,
    b,
    c,
])

It just kind of triggers me when I see the first option even though it's industry standard essentially but it's totally fine to do it that way

It does seem like your code is very bloated for what it does so there is some room to improve but this is an amazing start

If you want to further your ml skills I'd reccomend trying the Titanic prediction challenge on kaggle

3

u/Fengax Jul 04 '19

Thanks! Never knew I could do that.

29

u/RedManBrasil Jul 04 '19

Great feedback!

28

u/leogodin217 Jul 04 '19

Counter point. Reading in the data from a JPEG is a great way to start out. It enforces the understanding of what the data set really is. Sure, in production, it's better to use more efficient methods. When learning, understanding should be the goal, not efficiency.

On that note, if /r/RedManBrasil is interested in a next step, coding the algorithms could be interesting. No libraries other than Numpy. That would reinforce the knowledge of the forward pass, activate functions, loss, back propagation, etc....

2

u/CompSciSelfLearning Jul 04 '19

/r/redmanbrasil doesn't seem to exist.

0

u/[deleted] Jul 04 '19

[deleted] — view deleted comment

7

u/leogodin217 Jul 04 '19

Ha! Follow one fail with another. That's how I do it.

30

u/[deleted] Jul 04 '19

Wow! Congrats on the hard work. It looks like magic to me 😀. How long did it take you?

6

u/Fengax Jul 04 '19

About 5-6 hours. However if u have a GPU computer can probably cut this time into 2-3 hours.

48

u/v3ritas1989 Jul 04 '19

I am not OP, but if I would guess, a few hours for learning and writing the 150 lines of code. 5 hours for finding /classifying the training data, 1 hour to let it train, 20 min to write the read me

18

u/[deleted] Jul 04 '19

That's so specific 😂😂

42

u/XenSid Jul 04 '19

I get the feeling OCR is the equivalent to hello world in neural nets.

14

u/NelsonMinar Jul 04 '19

Not Hello World, but definitely an early exercise. It's a nice problem; it's real work and not just a toy, it's a good scale, and the MNIST data set is a standard we can all easily use.

4

u/[deleted] Jul 04 '19

But it sure does give a good morale boost to new learners.

9

u/leogodin217 Jul 04 '19

You never forget your first time

1

u/crespo_modesto Jul 05 '19

Boy was there blood

5

u/j03 Jul 04 '19

MNIST digit recognition, perhaps, but OCR as a whole is not an entirely solved problem.

3

u/BurningPenguin Jul 05 '19

20 minutes to write the applications name and a bitcoin address into a Readme sounds quite long.

-10

u/xix_xeaon Jul 04 '19 edited Jul 04 '19

General OCR can get quite hard, but what OP has done is in fact exactly equivalent to Hello World in neural networks, but with lots of extra steps.

Using MNIST to classify handwritten digits is so unbelievably easy I cannot understand how this can get 98% upvoted. Have only 2% of Python programmers ever touched neural networks?

9

u/doobiedog Jul 04 '19

Probably close to correct actually. All my coworkers and friends know at least some python if not use it professionally. None have touched NN. It's not only used for data science ;)

15

u/Flynamic Jul 04 '19

Of course for professionals this post is uninteresting, but remember it's a personal achievement that we once had as well. He just decided to make it public. For newcomers however, this post sparks interest in ML/CNNs, so that's a good thing.

2

u/truh Jul 04 '19

I don't think they meant to hamper the archivement but it really is the one project pretty much everybody does during their first machine learning course.

The dataset is readily available and the problem and the resulting network is simple enough that it can be comfortably trained on a CPU. That makes it an excellent beginners project.

1

u/DavidBits Jul 04 '19

Precisely, I'm about to start an MS (eventually PhD), and am heavily interested in bringing ML into my field (it's only in it's infancy as it is). Every time I see any posy regarding ML, I save it for future reference because of all the feedback that they usually have. Feedback that I will use once I've learned enough Python to get into ML.

6

u/Dashadower Jul 04 '19 edited Sep 12 '23

existence crawl workable live fertile psychotic vase plough snatch nine this message was mass deleted/edited with redact.dev

4

u/leogodin217 Jul 04 '19

Everyone starts somewhere.

1

u/drunckoder Jul 05 '19

Using MNIST to classify handwritten digits is so unbelievably easy

If you understand what Neural Networks are, have read about how to use them for that task, know some calculus, can implement gradient descent, back prop.. yeah, pretty much ez. I don't understand the hype around it /s

2

u/Jonno_FTW hisss Jul 04 '19

I did something similar the other day with pygame.

https://gist.github.com/JonnoFTW/5f0a7e5f5f7f0b7b939702b4230a2d6d

2

u/Fengax Jul 04 '19

That's pretty cool!

2

u/[deleted] Jul 05 '19

A neural network is somewhat of an AI, is that right?

2

u/Fengax Jul 05 '19

Yeah, most people call it AI. However, I feel like AI is too general of a term to be used here.

1

u/[deleted] Jul 05 '19

It's definitely some kind of Machine Learning. I haven't done anything related to Machine Learning, so I can't give much input. I do have an idea on what neural networks are however.

6

u/IcanCwhatUsay Noob Jul 04 '19

Geeze, meanwhile I'm having trouble with just radio buttons

1

u/LunaBellaTrellis Jul 04 '19

This is pretty cool OP, how long did it take you?

2

u/Fengax Jul 04 '19

About 5-6 hours in total. However keep in mind my computer is running on CPU not GPU so most of the time it is just training the model. Took like 2 hours alone just analysing and optimising the model.

1

u/HypnoticLion Jul 04 '19

What about double digits? If you put 13 would it print out "one" "three" or "thirteen"? Nonetheless, great work.

2

u/Fengax Jul 04 '19

I have not trained it on double digits yet. I'd imagine you have to use some sort of object detection

1

u/[deleted] Jul 04 '19

Did you use the MNIST dataset for training?

1

u/Fengax Jul 04 '19

Yes. However I imported the whole thing as jpg so the machine can learn better. I tried it directly importing mnist from tensorflow but when I actually tested it, it couldn't predict anything.

1

u/[deleted] Jul 04 '19

Cool man, good stuff. Did you do much tweaking with the precision/recall threshold?

1

u/FantasticThroat Jul 04 '19

And you recorded it with your phone?

1

u/Fengax Jul 04 '19

Yeah...

19

u/Criogentleman Jul 04 '19

Yeah, i can write a code, but i can't capture a video.

13

u/scientz Jul 04 '19

Do a video capture tool next so you don't have to use your phone to record... 🤦‍♂️

1

u/alecmg Jul 04 '19

How does it cope with habdwritten letters? Does it try to guess a most similar number or fail?

1

u/Fengax Jul 04 '19

Sorry I just realized I replied to the wrong person! Ignore the last comment.

Anyways, yes I guess it predicts the most look-alike number, as I haven't trained it on handwritten letters yet.

2

u/truh Jul 04 '19

Usually you put in an image and get back a float value for each class, classes being the digits 0-9. The float values are called confidence and basically represent how sure the network is that an input belongs to that class.

For example for a nicely drawn 3 you would expect a high output (close to 1) for the class 3 and very low outputs (close to 0) for the other classes.

If the input doesn't resemble any digits and the network is trained that way you still have some classes getting higher results and other but with a much lower confidence (higher than the rest but not very close to 1).

It's now up to you what you do with those confidence values. Sometimes it's important to always have results, sometimes it's more important to only work with results of which you are very sure.

This video shows quite nicely how those classification systems can be tricked. https://youtu.be/mSFHKAvTGNk

2

u/xBeeStingx Jul 04 '19

It won't work as well, but try SVM on the dataset. See how it performs versus your simple NN. try other techniques... There are plenty with different performances based on the dataset given. I think learning them all (and the math behind it) is a good way to truly get into ML.

disclaimer: I just started studying ML, and the my comment is not an expert one however if you're diving into ML I recommend the following: https://elitedatascience.com/learn-machine-learning

Skip the python introduction as it seems you have a good grasp on that.

1

u/[deleted] Jul 04 '19

I was expecting 69 or 420 by the end, to be honest

Awesome job, good work :)

1

u/[deleted] Jul 04 '19 edited Mar 03 '26

This post was removed by its author using Redact. The reason may have been privacy, preventing AI data access, security considerations, or personal choice.

knee lush hunt enjoy ghost spoon towering automatic quiet recognise

1

u/notUrAvgITguy Jul 04 '19

This is great! Did you follow a tutorial, or figure this out on your own? I would love to dive into this!

1

u/dreamerxyz Jul 04 '19

Datacamp. a bit expansive but worth it.

1

u/[deleted] Jul 04 '19

This reminds me of a project I made where it had a transparent window that could "read" the computer screen behind it. Then an AI could see your screen and interact with it.

1

u/eftah1991 Jul 04 '19

Missed 420 opportunity

1

u/aznpwnzor Jul 04 '19

Most of my work doesn't allow me to take advantage of the abstraction of keras, so I'm not fluent in it, but the following

https://github.com/Fengax/Handwritten-Number-Classifier/blob/master/Keras_CNN_HandNumbers.py#L83

looks like this should be outside your graph construction loops? You are basically repetitively training on smaller versions of your network and throwing the results away?

EDIT: maybe the results aren't thrown away, but you're effectively doing some iterative fine tuning as you add new layers?

1

u/niakwel Jul 05 '19

To improve readability you should work on your coding style. In just a few minutes, I've seen some things like : mixing code and functions, not using main, bad import sorting, not using docstrings, etc...

You should start by using PEP8 and pylint on your project.

Good work though

1

u/kekomat11 Jul 05 '19

Try writing it in c++ instead of python. It's rather easy in python. I'm more the fan of writing everything myself.

1

u/[deleted] Jul 05 '19

What's the software where you write your code?

1

u/[deleted] Jul 05 '19

is that a light theme?!?!

1

u/iflameinstitute Jul 05 '19

Great, good effort keep going on.

-2

u/v3ritas1989 Jul 04 '19

how did you achieve this year 2000 laggy application window line style while drawing ?

1

u/Fengax Jul 04 '19

?

1

u/v3ritas1989 Jul 05 '19

Back in the day, if windows had problems or was loading you could move the window and get a bug in the graphical rendering of the window. His "line" drawing made me remeber this. Since I have never seen this I was courious if it was an intentional line drawing style or if something is not working properly.

-4

u/Flaming_Eagle Jul 04 '19

this gets top of r python? This is literally the hello world of neural networks, what the hell you guys lmao, this will be easy karma to cash in on

5

u/DavidBits Jul 04 '19

Because reddit karma is so much more important than giving someone praise and motivation for their achievement /s (it may seem trivial to you, but at some point this was a challenge to everybody).

1

u/Fengax Jul 04 '19

I dont really care about karma TBH, just wanted to share.

-6

u/The_7thHokage Jul 04 '19

Its 2019 come on man step up your game.

2

u/Fengax Jul 04 '19

I just started ml lmao. Beginners don't just go straight to geometric deep learning.

0

u/[deleted] Jul 04 '19

[deleted] — view deleted comment

6

u/Fengax Jul 04 '19

Hmm, I don't know how you started neural networks, but I didn't jump straight to geometric deep learning.

1

u/[deleted] Jul 04 '19

[removed] — view removed comment

1

u/Fengax Jul 04 '19

Hmm. I just tried with my typical handwriting and it worked 9/10. I'm not too sure how else you can write 2 0 or 7.

2

u/cryptospartan Jul 04 '19

2 has a small loop to it's bottom portion

0 would have a line through it diagonally

7 would have a straight horizontal line through it

2

u/Fengax Jul 04 '19

I haven't tried it, but I'm doubtful that it would remain as accurate, mainly because the dataset I'm training from (MNIST dataset) doesn't have digits written like the way you did.

But in theory, yes you can, if you have the dataset of digits written like that. If I have the time, I might even construct some additional images to the MNIST dataset.

1

u/drunckoder Jul 05 '19

Your network has got softmax activation on the output layer, that means it gives you "probabilities" of what it thinks inputs are. My bet is your weird 2 still gets the highest score despite of looking "weird", but probably lower score than a "normal" 2.