Posts: 4,587
Threads: 105
Joined: Jul 2005
Hi experts,
So far, I thought I know some statistical data analysis, but here's something strange leaving me confused:
Fitting a model curve to measured data points, i.e. data with errors > 0 , reduced chi square (RC2) is one method to assess the goodness of fit. Browsing some textbooks and the net, I find just onesided tests for RC2. I.e. it is said to be a good fit model if RC2 is less than or approximately equal to 1, and a bad fit model if RC2 >> 1. Why isn't this a twosided test?
Reasoning: The model curve hitting all data points right in the middle is very improbable for physical data. So it should be a twosided test for RC2, excluding very low values of RC2, too. OTOH, fitting is done minimizing the sum of squared distances from all data points to the curve, so zero would be optimum.
You see me confused. What's wrong? Help is appreciated.
Walter
Posts: 2,247
Threads: 200
Joined: Jun 2005
Hi Walter,
Most books use Rsquare and the F statistics to measure goodness of fit. One popular tool is found in the Excel Data Analysis addin. It generates an ANOVA table that shows the F statistics and its significance. Also, the tool calculates and shows the Rsquare and its adjusted value. Finally the output also shows the standard error, studentt, and significance of studentt for each regression coefficient (and intercept).
I rely on the regression ANOVA table and other statistics displayed by the Excel Data Analysis addin more than just the Chisquare.
If you want to examine the relevance of yoru curve fit I highly recommend Excel's tool and the richness of the output.
Namir
Posts: 3,229
Threads: 42
Joined: Jul 2006
Quote: Why isn't this a twosided test?
You're squaring the differences so the closer to zero you get the better the fit. Thus no need for the other side. Or am I missing something in the question?
 Pauli
Posts: 776
Threads: 25
Joined: Jun 2007
Quote: so zero would be optimum.
Not really! Reduced chisquare includes the estimated size of your errors, so if you have estimated your errors correctly (sometimes a task in itself for realworld data), the fit value should be near 1. If it is much below one, either your error estimates are not good (too pessimistic), or you are fitting to too many variables or the wrong equation(s) and fooling yourself that you know what is going on!
Posts: 4,587
Threads: 105
Joined: Jul 2005
Hi Pauli,
Quote:
Or am I missing something in the question?
Maybe. Please check my "reasoning" above and Dave's answer below. IMHO a contradiction remains.
Walter
Edited: 30 July 2009, 3:03 a.m.
Posts: 850
Threads: 10
Joined: Mar 2009
The Reduced Chi Square test is the Chi Square devided by the degrees of freedom. So it seems to me to be a "normalised" Chi Square, hence 1 being the optimum value. This makes it easier to present results without the reader having to worry about D.O.F., number of samples etc.
The Chi Square test itself can be either a one sided (upper or lower) test, or a two sided test. However, only the upper onesided test lends itself to the Reduced Chi Square.
Bart
Posts: 4,587
Threads: 105
Joined: Jul 2005
Hi Bart,
I agree with everything you wrote but the last sentence. Nothing prevents me from dividing a lower Chi Square by its DF.
Walter
Posts: 3,229
Threads: 42
Joined: Jul 2006
Missed that bit for some reason  probably my own stupidity and haste :(
I've never heard of a test for a toogood fit of the data. That has always been a cause for great rejoicing not concern. However, if you are interested in that why not do another test at the 99.9999% (or whatever) level and reject if that accepts the fit?
 Pauli
Posts: 4,587
Threads: 105
Joined: Jul 2005
Hi Namir,
Thanks for your friendly advise. I know and use this tool already. It is just that almost every author I see demonstrating simple Chi Square tests uses single sided tests to measure goodness of fit, while the same authors have no problems with double sided ttests, for example. That's what puzzles me. Maybe a bit academic, but I just want to understand it.
Walter
Posts: 4,587
Threads: 105
Joined: Jul 2005
Hi Dave,
What you state matches what I remember from the old days, too. But it doesn't seem to be very popular, does it? So maybe I miss an important reason for single sided testing here, being so simple nobody thinks its worth explaining?
Posts: 850
Threads: 10
Joined: Mar 2009
Hi Walter
The problem at the lower tail of the CS is that except for very large DOF, CS/DOF < 1, even if it is a bad fit.
Bart
Posts: 4,587
Threads: 105
Joined: Jul 2005
Hi Bart,
CS/DOF will *always* stay <1 for the lower tail, as well as it will stay >1 for the upper tail d8) Both values will approach 1 for large DOF.
Walter
Posts: 850
Threads: 10
Joined: Mar 2009
Hi Walter,
Sorry, I seem to have the wrong end of the "tail", RC2 is more suited for a lower tail GOF check, if <=1 is to be the acceptance criteria. However, as CS/DOF=1 is around the 60th percentile, the RC2 seems a very loose check for goodness of fit. As the 5th percentile is a more acceptable criteria, then RC2 < 0.78 for 100 DOF or <0.5 for 20 DOF.
Let me add that I've used the Chi2 and looked into the RC2 when I saw your question this morning.
Posts: 4,587
Threads: 105
Joined: Jul 2005
Just to illustrate what we are talking about: This is a diagram of Reduced Chi Square over Degrees Of Freedom. The lines are for different confidence levels and number of sides, and "einseitig" means single sided, while "zweiseitig" means (surprise!) double sided. Solid lines are for single sided, blue lines are for 99%, so broken lines are for double sided, and red lines are for 95%.
