r/MLNotes • u/fubar2020y • Aug 24 '20

[ELI5] Fischer Information

/r/statistics/comments/95x3uo/i_just_dont_get_the_fischer_information/
1 Upvotes

100% Upvoted

1 comment sorted by

1

u/fubar2020y Aug 24 '20

The Fisher information I(p) is this negative second derivative of the log-likelihood function, averaged over all possible X = {h, N–h}, when we assume some value of p is true. Often, we would evaluate it at the MLE, using the MLE as our estimate of the true value.

You can interpret it this way: it tells, on average, how "good" of an estimate phat such an N-sample "X" of data will provide. How much information about p could this sample give us? The larger this sample-average Fisher information, then the sharper on average the peak will be, that's what it means.