Subject: var and std Newsgroups: gmane.comp.python.numeric.general Date: Tuesday 28th November 2006 12:42:05 UTC (over 10 years ago) Hello, I was wondering if someone could explain the rationale for changing .var() and .std() in release 1.0b1 from normalizing by n-1 (unbiased estimate from sample) to normalizing by n (population)? I have found the note that this change happened in the Release Notes http://www.scipy.org/ReleaseNotes/NumPy_1.0 and the change itself in Changeset 2560 http://projects.scipy.org/scipy/numpy/changeset/2560 as well as a related documentation change in Ticket 388 http://projects.scipy.org/scipy/numpy/ticket/388 but I have not been able to find a description of why the change was made, despite searching the website, the Trac and the mailing list. I am aware of the argument that, if the difference between n and n-1 matters to you, then you are "up to no good". On the other hand, this change breaks a lot of my unit tests. It also seems to violate the principle of least surprise: every other numerical environment that I have used divides by n-1 by default. Examples include MATLAB: http://www.mathworks.com/access/helpdesk/help/techdoc/ref/index.html?/access/helpdesk/help/techdoc/ref/std.html& http://www.mathworks.com/access/helpdesk/help/techdoc/ref/index.html?/access/helpdesk/help/techdoc/ref/var.html& Octave: http://www.gnu.org/software/octave/doc/interpreter/Basic-Statistical-Functions.html and R: http://finzi.psych.upenn.edu/R/library/stats/html/cor.html http://finzi.psych.upenn.edu/R/library/stats/html/sd.html It also seems to present an inconsistent interface: cov() still normalizes by n-1 instead of n. It also has a 'bias' parameter that allows normalizing by n, which is similar to the compromises provided in the other numerical packages listed above. As an aside, cov() also does not seem to be provided as a method, only as a function. In light of all that, I am sure there must have been a good reason for the change, and I am very curious what it was. Thanks for any insight you can offer. Regards, Aarre -- Aarre Laakso http://www.laakshmi.com/aarre/ |
|||