-------------------------------------------------------------------------------
name: <unnamed>
log: Q:\C-modelling\runmlwin\website\logfiles\2024-10-11\18\15_Diagnost
> ics_for_Multilevel_Models.smcl
log type: smcl
opened on: 11 Oct 2024, 16:44:30
. ****************************************************************************
. * MLwiN User Manual
. *
. * 15 Diagnostics for Multilevel Models 225
. *
. * Rasbash, J., Steele, F., Browne, W. J. and Goldstein, H. (2012).
. * A User’s Guide to MLwiN, v2.26. Centre for Multilevel Modelling,
. * University of Bristol.
. ****************************************************************************
. * Stata do-file to replicate all analyses using runmlwin
. *
. * George Leckie and Chris Charlton,
. * Centre for Multilevel Modelling, 2012
. * https://www.bristol.ac.uk/cmm/software/runmlwin/
. ****************************************************************************
.
. * 15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . .225
.
. use "https://www.bristol.ac.uk/cmm/media/runmlwin/diag1.dta", clear
.
. runmlwin n_ilea cons n_vrq, ///
> level2(school: cons n_vrq, ///
> residuals(u, standardised leverage influence deletion) ///
> ) ///
> level1(pupil: cons, residuals(e)) nopause
MLwiN 3.13 multilevel model Number of obs = 907
Normal response model (hierarchical)
Estimation algorithm: IGLS
-----------------------------------------------------------
| No. of Observations per Group
Level Variable | Groups Minimum Average Maximum
----------------+------------------------------------------
school | 18 21 50.4 79
-----------------------------------------------------------
Run time (seconds) = 2.98
Number of iterations = 4
Log likelihood = -847.14086
Deviance = 1694.2817
------------------------------------------------------------------------------
n_ilea | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
cons | .0067881 .0308227 0.22 0.826 -.0536233 .0671996
n_vrq | .7254297 .036323 19.97 0.000 .654238 .7966214
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: school |
var(cons) | .0090049 .0056377 -.0020447 .0200545
cov(cons,n_vrq) | .0099072 .005238 -.0003591 .0201734
var(n_vrq) | .015591 .0078941 .0001188 .0310631
-----------------------------+------------------------------------------------
Level 1: pupil |
var(cons) | .3673971 .017567 .3329664 .4018278
------------------------------------------------------------------------------
.
. predict xb
.
. generate yhat = xb + u0 + u1*n_vrq
.
. drop xb
.
. twoway (line yhat n_vrq if school~=17, connect(a)) ///
> (line yhat n_vrq if school==17, lwidth(*3)), ///
> legend(off)
.
. rename yhat yhatold
.
. twoway (scatter n_ilea n_vrq if school~=17) ///
> (scatter n_ilea n_vrq if school==17, msize(*2)), ///
> legend(off)
.
. egen pickone = tag(school)
.
. egen u0rank = rank(u0) if pickone==1
(889 missing values generated)
.
. egen u1rank = rank(u1) if pickone==1
(889 missing values generated)
.
. serrbar u0 u0se u0rank if pickone==1, scale(1.4) yline(0) ///
> addplot(scatter u0 u0rank if pickone==1 & school==17, ///
> mcolor(maroon) msize(3)) legend(off)
.
. serrbar u1 u1se u1rank if pickone==1, scale(1.4) yline(0) ///
> addplot(scatter u1 u1rank if pickone==1 & school==17, ///
> mcolor(maroon) msize(3)) legend(off)
.
. twoway (scatter u1 u0 if pickone==1 & school~=17) ///
> (scatter u1 u0 if pickone==1 & school==17, msize(3)), legend(off)
.
.
.
. * 15.2 Diagnostics plotting: Deletion residuals, influence and leverage . 231
.
. histogram u0 if pickone==1, width(0.01)
(bin=26, start=-.09472238, width=.01)
.
. histogram u0std if pickone==1, width(0.1)
(bin=35, start=-1.3235798, width=.1)
.
. histogram u0lev if pickone==1, width(0.01)
(bin=21, start=.15773312, width=.01)
.
. histogram u0inf if pickone==1, width(0.025)
(bin=21, start=.01923586, width=.025)
.
. histogram u0del if pickone==1, width(0.1)
(bin=38, start=-1.3558191, width=.1)
.
. twoway (scatter u0lev u0std if pickone==1 & school~=17) ///
> (scatter u0lev u0std if pickone==1 & school==17, msize(3)), legend(of
> f)
.
. histogram u1 if pickone==1, width(0.01)
(bin=37, start=-.14800606, width=.01)
.
. histogram u1std if pickone==1, width(0.1)
(bin=37, start=-1.4235827, width=.1)
.
. histogram u1lev if pickone==1, width(0.01)
(bin=20, start=.1226208, width=.01)
.
. histogram u1inf if pickone==1, width(0.025)
(bin=27, start=.02138345, width=.025)
.
. histogram u1del if pickone==1, width(0.1)
(bin=41, start=-1.4715746, width=.1)
.
. twoway (scatter u1lev u1std if pickone==1 & school~=17) ///
> (scatter u1lev u1std if pickone==1 & school==17, msize(3)), legend(of
> f)
.
. egen e0rank = rank(e0)
.
. summarize e0
Variable | Obs Mean Std. dev. Min Max
-------------+---------------------------------------------------------
e0 | 907 -1.51e-16 .6008925 -1.671944 1.746477
.
. generate e0std = (e0 - r(mean))/r(sd)
.
. generate e0uniform = e0rank/(r(N) + 1)
.
. generate e0nscore = invnorm(e0uniform)
.
. twoway (scatter e0std e0nscore if school~=17) ///
> (scatter e0std e0nscore if school==17, msize(3)), ///
> yline(0) xline(0) ylabel(-4(1)4) xlabel(-4(1)4) aspectratio(1)
.
. bysort school (pupil): generate pupilnumber = _n
.
. sort e0
.
. list school pupil pupilnumber if _n==1
+---------------------------+
| school pupil pupiln~r |
|---------------------------|
1. | 17 886 22 |
+---------------------------+
.
. generate s17p22 = (school==17 & pupilnumber==22)
.
. drop u0* u1* e0*
.
. sort school pupil
.
. runmlwin n_ilea cons n_vrq s17p22, ///
> level2(school: cons n_vrq, residuals(u)) ///
> level1(pupil: cons) initsprevious nopause
Model fitted using initial values specified as parameter estimates from previou
> s model
MLwiN 3.13 multilevel model Number of obs = 907
Normal response model (hierarchical)
Estimation algorithm: IGLS
-----------------------------------------------------------
| No. of Observations per Group
Level Variable | Groups Minimum Average Maximum
----------------+------------------------------------------
school | 18 21 50.4 79
-----------------------------------------------------------
Run time (seconds) = 2.94
Number of iterations = 4
Log likelihood = -843.10189
Deviance = 1686.2038
------------------------------------------------------------------------------
n_ilea | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
cons | .007785 .0313655 0.25 0.804 -.0536901 .0692602
n_vrq | .7281126 .0375109 19.41 0.000 .6545927 .8016326
s17p22 | -1.765948 .6179108 -2.86 0.004 -2.977031 -.5548652
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: school |
var(cons) | .0096563 .0058358 -.0017817 .0210942
cov(cons,n_vrq) | .0108369 .0055546 -.0000499 .0217236
var(n_vrq) | .017216 .0084226 .000708 .033724
-----------------------------+------------------------------------------------
Level 1: pupil |
var(cons) | .3635034 .0173815 .3294363 .3975705
------------------------------------------------------------------------------
.
. egen u0rank = rank(u0) if pickone==1
(889 missing values generated)
.
. egen u1rank = rank(u1) if pickone==1
(889 missing values generated)
.
. serrbar u0 u0se u0rank if pickone==1, scale(1.4) yline(0) ///
> addplot(scatter u0 u0rank if pickone==1 & school==17, ///
> mcolor(maroon) msize(3)) ///
> legend(off)
.
. serrbar u1 u1se u1rank if pickone==1, scale(1.4) yline(0) ///
> addplot(scatter u1 u1rank if pickone==1 & school==17, ///
> mcolor(maroon) msize(3)) ///
> legend(off)
.
. generate s17 = (school==17)
.
. generate s17Xn_vrq = s17*n_vrq
.
. drop u0* u1*
.
. runmlwin n_ilea cons n_vrq s17p22 s17 s17Xn_vrq, ///
> level2(school: cons n_vrq) ///
> level1(pupil: cons) initsprevious nopause
Model fitted using initial values specified as parameter estimates from previou
> s model
MLwiN 3.13 multilevel model Number of obs = 907
Normal response model (hierarchical)
Estimation algorithm: IGLS
-----------------------------------------------------------
| No. of Observations per Group
Level Variable | Groups Minimum Average Maximum
----------------+------------------------------------------
school | 18 21 50.4 79
-----------------------------------------------------------
Run time (seconds) = 2.81
Number of iterations = 5
Log likelihood = -837.75697
Deviance = 1675.5139
------------------------------------------------------------------------------
n_ilea | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
cons | -.0068628 .0281243 -0.24 0.807 -.0619854 .0482599
n_vrq | .7084521 .0333986 21.21 0.000 .642992 .7739122
s17p22 | -1.760465 .6218455 -2.83 0.005 -2.97926 -.5416704
s17 | 1.252792 .4791477 2.61 0.009 .3136802 2.191904
s17Xn_vrq | -.2770559 .3002206 -0.92 0.356 -.8654775 .3113657
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: school |
var(cons) | .00584 .0045151 -.0030094 .0146893
cov(cons,n_vrq) | .005982 .0039723 -.0018036 .0137677
var(n_vrq) | .0109481 .0062602 -.0013216 .0232178
-----------------------------+------------------------------------------------
Level 1: pupil |
var(cons) | .361849 .0173012 .3279392 .3957587
------------------------------------------------------------------------------
.
. runmlwin n_ilea cons n_vrq s17p22 s17, ///
> level2(school: cons n_vrq, residuals(u)) ///
> level1(pupil: cons) initsprevious nopause
Model fitted using initial values specified as parameter estimates from previou
> s model
MLwiN 3.13 multilevel model Number of obs = 907
Normal response model (hierarchical)
Estimation algorithm: IGLS
-----------------------------------------------------------
| No. of Observations per Group
Level Variable | Groups Minimum Average Maximum
----------------+------------------------------------------
school | 18 21 50.4 79
-----------------------------------------------------------
Run time (seconds) = 2.83
Number of iterations = 5
Log likelihood = -838.18229
Deviance = 1676.3646
------------------------------------------------------------------------------
n_ilea | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
cons | -.0080495 .02813 -0.29 0.775 -.0631834 .0470843
n_vrq | .7050538 .0333325 21.15 0.000 .6397232 .7703843
s17p22 | -1.835283 .6167756 -2.98 0.003 -3.044141 -.6264248
s17 | .883581 .2633688 3.35 0.001 .3673877 1.399774
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: school |
var(cons) | .0058641 .0045261 -.0030068 .0147351
cov(cons,n_vrq) | .0060503 .0039975 -.0017846 .0138851
var(n_vrq) | .0111038 .0063118 -.0012672 .0234747
-----------------------------+------------------------------------------------
Level 1: pupil |
var(cons) | .3621366 .0173152 .3281994 .3960738
------------------------------------------------------------------------------
.
. predict xb
.
. gen yhat = xb + u0 + u1*n_vrq
.
. twoway (line yhatold n_vrq if school~=17, connect(a)) ///
> (line yhatold n_vrq if school==17, lwidth(*3)), ///
> legend(off)
.
. twoway (scatter yhat n_vrq if school~=17) ///
> (scatter yhat n_vrq if school==17, msize(*2)), ///
> legend(off)
.
.
.
. * 15.3 A general approach to data exploration . . . . . . . . . . . . . .240
.
. * Chapter learning outcomes . . . . . . . . . . . . . . . . . . . . . . .240
.
.
.
. ****************************************************************************
. exit
end of do-file