What is the form of statistical model that is being used?
What are the parameters to be estimated?
What are the assumptions that underlie the statistical model?
Were the assumptions met?
How did you test those assumptions?
Describe the statistical techniques that you used in detail. Demonstrate that
you understand how each statistic is actually used (and how it is computed)
and whether it accomplishes what you want it to accomplish.
Your interpretations must demonstrate not only that you understand the statistical
component of the analysis but also how the results can be stated in practical
terms for a non-specialist to understand and appreciate. For example, can
you explain the results to your mother?
In this class, your data analyses may become fairly lengthy (anywhere from
five to twenty pages). It would be very useful if you learned how to use the
Equation Editor on whatever word processor you normally use.
Use your spell-checker and be very careful about the details of equations.
GENERAL STATISTICS REFERENCES:
Anderson, T.W. An Introduction to Multivariate Data Analysis. Wiley.
*Atkinson, A. Plots, Transformations, and Regression. Clarendon
Press.
*Barnett & Lewis. Outliers in Statistical Data. Wiley.
Belsley, Kuh & Welsch. Regression Diagnostics. Wiley.
Bock, R.D. Multivariate Statistical Methods in Behavioral Research.
McGraw-Hill.
Campbell, D.T. & Kenny, D.A. (1999). A Primer on Regression Artifacts.
Guilford.
Chatterjee & Price. Regression Analysis by Example. Wiley.
Cohen & Cohen. Applied Multiple Regression/Correlation Analysis for
the Behavioral Sciences. LEA.
Cooley,W.W. & Lohnes, P.R. Multivariate Data Analysis. Wiley.
Cook & Weisberg. Residuals and Influence in Regression. Chapman
& Hall.
Cook, R.D. (1998). Regression Graphics: Ideas for Studying Regressions
through Graphics. Wiley.
Daniel & Wood. Fitting Equations to Data. Wiley.
Draper & Smith. Applied Regression Analysis. Wiley.
Dunteman, G.H. Introduction to Multivariate Analysis. Sage.
Edwards. Multiple Regression and the Analysis of Variance and Covariance
Freeman.
Frees, E.W. (1996). Data Analysis Using Regression Models. Prentice
Hall.
Fox, J. (1997). Applied Regression Analysis, Linear Models, and Related
Methods. Sage.
Freund,R.J. & Wilson, W.J. (1998). Regression Analysis: Statistical
Modeling of a Response Variable. Academic Press.
**Green, P. Mathematical Tools for Applied Multivariate Analysis. Academic
Press.
Grimm, L.G. & Yarnold, P.R. Reading & Understanding Multivariate
Statistics. APA,.
**Hammer, A. Elementary Matrix Algebra for Psychologists and Social Scientists.
Pergamon Press.
**Hanushek & Jackson. Statistical Methods for Social Scientists.
Academic Press.
Hawkins. Identification of Outliers. Chapman & Hall.
Hosmer, D.W. & Lemeshow, S. (1989). Applied Logistic Regression.
Wiley.
**Huitema, B.E. The Analysis of Covariance and Alternatives. Wiley.
Long, J.S. (1997). Regression Models for Categorical and Limited Dependent
Variables.
Sage: Advanced Quantitative Techniques #7.
Neter & Wasserman. Applied Linear Statistical Models. Irwin.
Tabachnick, B. & Fidell, L. Using Multivariate Statistics. Harper
Collins.
Weisberg. Applied Linear Regression. Wiley.
*Wickens, T.D. The Geometry of Multivariate Statistics. LEA.
Wonnacott & Wonnacott. Regression. Wiley.
Yates, A. Multivariate Exploratory Data Analysis. SUNY Press.
Younger. A Handbook for Linear Regression. Duxbury.
Specific References:
Chatterjee, S. and Yilmaz, M. (1992). A Review of Regression Diagnostics for
Behavioral Research. Applied Psychological Measurement, 3, 209-227.
Greenland, S., Schlesserman, J.J. and Criqui, M.H. (1986). The Fallacy of
Employing Standardized Regression Coefficients and Correlations as measures
of Effect. Journal of Epidemiology, 2, 203-208.
Schafer, J.L. & Graham, J.W. (2002). Missing data: Our view of the state
of the art. Psychological Methods,7, 147-177.
Thomson, B. Editorial Comment: Misuse of Ancova and related "Statistical
Control" Procedures. Educational and Psychological Measurement.
Wilkinson, L and Task Force on Statistical Inference (1999). Statistical Methods
in Psychology Journals: Guidelines and Explanations. American Psychologist,
Vol 54, 8, 594-604
Ludlow, L.H. (2002). Residuals: Trash or treasure? Popular Measurement, 4,
1-7. http://www.rasch.org/pm4.pdf
Additional Resources:
Any SPSS Regression manual (look for them in used book stores)
MS Word Equation Editor manual
Ludlow & Bell: "Dirty data can be cleaned"-unpublished manuscript
(in my files)
ED/PY 667
General Linear Models
Prof. Larry H. Ludlow
Boston College
Assignment 1
(80 points)
Using the Course Evaluation data, regress the Excellent ratings upon the Course
number variable. Your analysis should begin with:
(a) the purpose for conducting a regression analysis in general, and for these
variables in particular (include an explanation of what COURSE number serves
as a proxy for),
(b) an explanation of what you think "General Linear Model" refers
to,
(c) an explanation of what you think "ordinary least squares" refers
to,
(d) explain what "BLUE" as a estimation term refers to, and
(e) provide a complete description of the data set so that someone not familiar
with the data would know what the variables are,
(You do not need to address the assumptions of an OLS analysis at this point.)
1. Obtain the bivariate plot of the raw data with the regression line and
the 2 SE upper and lower confidence bounds (click "Edit" in the
Chart carousel, select the Chart menu, then select Options, then click the
"Fit Line-Total" box, then click the "Regression Prediction
Lines-Individual" box). Discuss the information in the plot (e.g., distribution
of the points, unusual data points, linearity) and include the plot in your
write-up. Identify the specific courses that seem "unusual".
2. Given that there are at least three ways (what are the three we talked
about?) to formulate a statistical hypothesis for a simple regression analysis,
pick one and explain the scientific hypothesis to be tested and write its
statistical form (both null and alternative). Interpret the result of having
tested the hypothesis. Explain in statistical and practical terms what the
intercept and slope mean--specifically addressing what it means in terms of
the change in Excellent ratings as a function of change in Course number using
the results you obtained.
3. Reproduce (place the correct values into the appropriate equations) the
observed F ratio, the regression coefficient t statistic, and the 95% CI for
the regression coefficient parameter estimate. Interpret each of them. What
is the relationship between the F and t?
4. Show (again through equations) why the R SQUARE and ADJUSTED R SQUARE values
differ and then explain what they represent (including what they are estimates
of) and why their difference is important. Why is the R SQUARE an "inflated"
estimate? What do these statistics tell you about your solution?
5. What would you suggest as the next model to test (i.e., which variable(s)
as predictors) and why?
Submit your write-up, not your complete computer output. You should include
in the text all tables and plots that you think are relevant to the interpretation
of these data. You should try to learn the Equation Editor feature of your
word processor. You should also try to learn to use the Chart Editor in SPSS
in creative ways to reveal patterns (e.g. the "magnifying glass")
and interesting features in the data (e.g. using "reference lines"),
and to create relevant titles, axis labels, and axis minimum and maximum values.
Please double space your text, spell-check your text, double-check any calculator
results against the SPSS output, and proof-read the details in your statistical
material.
NOTE: This assignment is due Sept 30.
ED667
General Linear Models
Prof. Larry H. Ludlow
Boston College
Assignment 2
(100 points)
Begin your write-up with a brief description of the data set and the general
regression results from Assignment #1. Now, using the Course Evaluation data,
you are to regress the Excellent ratings upon the RGATTEND variable. What
do you think is the purpose (and hypothesis) for this particular regression
problem? In addition, explain the general purpose of a diagnostic (or "fit")
analysis, particularly with respect to the OLS regression assumptions.
1. Obtain the basic scatterplot and regression solution and interpret them.
Note that there are missing data for this variable-what did you do about them?
Comment on the distribution of the RGATTEND values.
2. Questions about the residuals:
· For the standardized (ZRESID) and studentized residuals (SRESID)--show
how they are computed and explain why the latter values are always larger
than the former. Use an actual pair of residual values and show how the 2
values differ.
· Plot the SRESID's against (unstandardized). Create the SRESID histogram
with the normal curve and create the normal probability plot. Explain whether
or not the distribution of the SRESID's appear to meet the OLS assumptions
of normality and homoscedasticity. Explain whether or not the SRESID's suggest
any unusual values, or "outliers", or patterns. If so, which courses
are they and what was unusual about their ratings?
3. Questions about influential observations:
· Define "influence" and "leverage".
· Explain what the Cook's D, and the SDFBETA statistics are used for
and show how they are computed. What guidelines does Pedhazur or Fox use for
interpreting Cook's D and SDFBETA? What does a positive and negative value
for SDFBETA for the regression coefficient mean(use an example from your results)?
· Plot Cook's D, and SDFBETA (b-not the intercept) against the RGATTEND
variable. Discuss whether or not the individual plots suggest the presence
of influential points. Use the "magnifying glass" in the Chart Editor
Toolbar to identify unusual cases.
· Plot against the "SEQNUM" of each course: the predicted
excellence rating, studentized residual, Cook's D, and SDFBETA and explain
whether or not there is evidence of a pattern, and if so, what might it be
due to?
4. Now, at this point, try to give an explanation for what you think might
have contributed to these unusual residuals and influential points--what is
it about these courses that might have caused them to stand out? Explain whether
or not any of these cases should be removed from the model.
5. Questions about auto-correlation:
· What is a lag-1 auto-correlation?
· Why are we testing for the presence of one (recall your above plot
of studentized residuals across SEQNUM?
· What are the negative consequence to an OLS solution if there is
a statistically significant auto-correlation?
· Show how the Durbin-Watson statistic is computed and explain whether
it suggests a serial correlation (what is the D-B critical value?).
6. What is the next model you would propose now and explain why you propose
it.
Submit your write-up including in the text all tables and plots that you think
are relevant to the interpretation of these data.
NOTE:
This is a three week assignment. It is due Oct 21st . Do not wait until the
last weekend to start this.
· Although the steps in this assignment are sequentially numbered that
does not mean that the analysis itself must necessarily follow a sequential
process. You may find that you have to flip back and forth to provide a complete
and thorough interpretation of the analysis. I do, however, want your write-up
to follow the sequence as presented here.
· Try to write the assignment in a research article style: number the
equations, the tables, and the graphs; include tables and graphs in the body
of the text immediately following their discussion in the text. Don't include
tables and graphs that aren't explained. Provide explanations and interpretations
of results--don't simply say "there are 2 influential points" and
leave it at that.
· Give your tables and figures titles-pay attention to the labels on
the axes-think about adjusting your axis boundaries-think about using "reference
lines".
· Use an Equation Editor and pay attention to detail. Define the symbols
in your equations. Re-read your text for typos, clarity, and complete sentences.
· "Try-out" analyses and graphs are encouraged-meaning you
can do more than what is required for this assignment. Just make sure you
explain to me what you are doing since I am reading many papers and am looking
for common material in each.
-Finally, "lifting" material from published sources is discouraged.
Consulting and re-wording/re-expressing referenced work is encouraged. Detailed,
technical material can usually be stated in your own personal "voice".
My Professor Folder has a folder called "Handouts" with articles
that may be useful.
ED/PY 667
General Linear Models
Prof. Ludlow
Assignment 3
(100 points)
In this assignment you are to continue with your efforts to understand the
course evaluation data. In particular, you are to continue to use the "percent
excellent ratings" as your outcome variable and now you are free to choose
any two variables in the data set as your predictor variables. I suggest that
these variables should be "new" in the sense that they not be "COURSE"
or "RGATTEND"-in this way you will discover other interesting variables
rather than sticking with the same already known ones.
1. Briefly explain your overall findings from assignments 1 and 2-this is
designed to show the progressive connection between the assignments.
2. Now explain your choice for the two new predictors:
· Why did you pick them?
· Was your choice more for predictive or confirmatory purposes?
· What did you think the basic result would be using each separately
as a predictor?
3. Generate the basic scatterplot (including the regression and confidence
intervals) for each variable as a predictor "Excell". Provide a
basic interpretation of what the plot is telling you. Don't include your statistical
results-just focus on the plot. Essentially you are simply telling a technical
story in a non-technical fashion.
Now, in order to conduct the rest of the statistical part of the assignment
in some sensible way, I will refer to one of your predictors as "X1"
and the other as "X2". You should use their actual names in the
analyses and your interpretation (as above in #3) but since everyone may have
a different set of predictors, I will refer to them as X1 and X2 in the following
steps.
4. Obtain the regression of :
a) Excell on X1, and Excell on X2 (this is two separate simple regressions);
(now supplement the graphical story you gave above with the statistical result
of each separate regression solution--interpret the b's, t's, p-values, and
R2. Explain the difference in what an unstandardized and standardized regression
coefficient is-when does one or the other tend to be used? Which of your variables
is the stronger predictor-and why? ),
b) Excell on X1, X2; and
c) Excell on X2, X1.
(4b and 4c are multiple regression models. You are to use separate Enter commands
for each variable. (FYI--take a look at the solution when you only use one
Enter command and SPSS puts both variables in simultaneously-do you see what
it did?)
5. (a). Briefly write and state the different hypotheses being tested in 4a
to 4c.
(b). Summarize the regression results (using at least the coefficients, F,
p, and R2) in a single table.
(c). Give a brief but complete interpretation of what happened to the overall
solution and the individual coefficients as you go from model 4a to 4c. For
example, you will see changes in the b's, their p-values, and the overall
-state the changes you see, and provide
an explanation for the changes.
6. Reproduce the F CHANGE for the 4b solution and explain the relation between
F CHANGE and tX2 in 4b. Show the statistical relationship
between F CHANGE and tX1.
7. Explain and show through Venn diagrams how the proportion of variance accounted
for by X1 in 4b and 4c changes. That is, what is the percent of unique variance
accounted for by each predictor in these two models and what percent of variance
do the predictors share in common?
8. Do the partial regression plots-what do they represent, how are they used,
and how do you interpret the slopes in the graphs?
· Explain the difference in a zero-order correlations and a 1st order
correlation.
9. Provide a brief summary of your understanding of the results from the three
assignments at this point. That is, what has this analysis added to your understanding
of the ratings? What general model would you suggest next?
Due in three weeks-Nov. 11th . Submit all relevant output in a nicely
put together product. Use all of the stylistic, formatting, editing, and graphing
principles introduced in the previous assignments.
ED/PY 667
General Linear Models
Prof. Ludlow
Assignment 4
(60 points)
1.Create a graphical example with a verbal explanation that illustrates the
relationship between:
a) the distance between 2 points in a 2-D space,
b) the angle between those two vectors, and
c) the correlation between those 2 vectors.
2. Represent graphically (a hand drawing well done is sufficient) and verbally
explain what it means to minimize the sum of squared errors (recall your study
of the calculus) for a 2 predictor regression solution. (Idea: simulate a
set of data containing the 3-D coordinates where the b1 and b2 values range
from -1 to +1, the sum of squared errors ranges from 2 to 10, and the minimum
occurs at b1= .5, b2= -.5. Now graph the solution.)
3. Briefly explain the function that 1st and 2nd derivatives perform in the
estimation of statistical model parameters. Graphically represent a first
and second derivative solution for the normal distribution..
4. Create two 2x2 matrices where the two sets of columns represent the coordinates
of points on X and Y axes. One matrix must be singular and the other matrix
must be non-singular.
a). Compute the determinants and represent graphically what the determinants
mean.
b). How is the area of the parallelogram generated from a non-singular matrix
related to
that matrix's determinant?
c). Explain in relatively simple terms the concept of an eigenvalue and eigenvector
as
represented in your non-singular matrix representation.
Due in two weeks (Nov 25th ).
ED/PY667
General Linear Models
Prof. Larry H. Ludlow
Final (150 points)
Given the course evaluation data you have been analyzing this semester, your
problem is to formulate and test the model that is your choice as the "best"
for understanding the evaluation ratings. "Best" is defined by the
purpose you choose for building the model. This exercise is primarily concerned
with your thinking about a model, explaining the model, and justifying the
steps you took to arrive at a final solution. Everyone will likely have a
different final model. I particularly want to know how you arrived at your
final choice.
I would like you to think of this final as a cumulative project. This means
you may cut-and-paste from your previous assignments. For example, the description
of the data set, the OLS assumptions, and general diagnostic analyses should
be included. But, I'm not interested in your repeating and including everything
you have already done.
In general, you may take either a predictive/exploratory approach or a theoretical/confirmatory
approach. Furthermore, you may want to think of the data set as consisting
of three general sets of variables: instructor variables, student variables,
and institutional variables. Whichever way you choose and however you think
of the data set, explain the rationale for the model you propose. This explanation
should include
1. the variables you selected (why did they interest you),
2. their order of entry (if it is a confirmatory model, then why did you put
them in a specific order),
3. the statistical variable selection procedure you used (if it is a predictive
model, then why did you chose either forward, backward, or stepwise),
· you may enter variables as blocks and test their block effects, not
necessarily their individual effects (this would hold for both predictive
and confirmatory models),
4. your reasons for using and ultimately discarding different variables and
approaches, and
5. your final conclusion about these data That is, for this set of data what
variables seem to have influence about the student ratings?
You must include the following:
Create a three-level indicator variable (e.g. marital status, level of degree,
type of course, semester taught, day of the week taught, etc).
· This variable may be dummy, effect, or orthogonal coded--the choice
is up to you, as is the way the codes are assigned. You will have to explain
why you chose the approach you did.
· Explain the results and the coefficients:
· What group comparisons were formed?
· What is the overall effect?
· What do the a and b's mean?
I recommend that you do this piece of the assignment last-after you have built
your "best" model. This final step would then be similar in concept
to an ANCOVA>
You have the following options available to you as you think about your model:
1. you may use the "excell" ratings or any other combination of
other variables to arrive at a "rating" variable as your outcome
(e.g. "excell" + "vgood", or you may look at "poor"),
2. you may use any of the predictors,
3. you may use any interactions between predictors (e.g. TIME * SIZE),
4. you may use any transformations (e.g. log, square, square root), and
5. you may use any regression diagnostics and plots to aid your choice for
your "best" model. You do want to address OLS residual assumptions
and comment on the presence of influential points.
6. You do want to address the issue of missing data-recall that 460 and 468
have missing values for two summer sessions. What are you going to do about
that fact?
Whatever choices you make you do need to explain them to me. When you re-read
your paper, ask yourself "Is he going to ask why I did this?" If
the answer is "yes", then make sure you have an answer.
I think it would be useful to write up your project following an APA article
style: purpose/introduction, method (sample, instrument, procedure), results
(assumptions tested, analytic procedures and their rationale, statistical
results), discussion (interpretation of the statistical results, practical
benefit of the results--"so what and who cares"). In terms of format,
you might think of including a table of contents containing the various sections
addressed by your paper.
This project may be conducted as a joint/group effort but individual write-ups
are required. Your class presentation should be planned for no more than 10
minutes a person--practice your timing. The presentations should be short
and they should emphasize the relevant point (s) you most want to make.
NOTE: the class presentation is required the last night of class. If
you can turn in your write-up then, fine. If you can't get it to me by the
end of that week, then an incomplete is an acceptable alternative.
DUE: Dec 16th.