This is an old revision of the document!
Researchers trying to summarize the constantly growing body of research in the social, health, and natural sciences are increasingly using a technique called meta-analysis. Meta-analysis encompasses an entire array of statistical methods for aggregating and comparing the results from related studies in a systematic manner. For example, meta-analysis is frequently used to determine whether a particular treatment or intervention is actually effective overall and whether the effectiveness of the treatment or intervention depends on certain study and/or subject characteristics (so-called moderator variables). The focus of this course will be on current methods and techniques for analyzing meta-analytic data.
We will start out with a short overview of the entire meta-analytic process (consisting of seven steps: problem formulation, literature search, information gathering, quality evaluation, analysis, interpretation of findings, and presentation of results). Next, we will examine how the results from a study can be summarized with various effect size or outcome measures. We will then delve into equal-, fixed-, and random/mixed-effects models for combining the observed outcomes and for examining whether the outcomes depend on one or more moderator variables. The use of so-called meta-regression models will be emphasized in this context. Model diagnostics and methods for sensitivity analyses will be covered as well.
A major problem that may distort the results of a meta-analysis is publication bias (the fact that the published literature may not be representative of all the research that has been conducted on a particular topic). Therefore, current methods for detecting and dealing with publication bias will be discussed next. Finally, time permitting (and depending on the interests of the participants), we will examine missing data issues, sequential/cumulative methods in the context of meta-analysis, meta-analytic techniques using individual subject data, multilevel and multivariate models, methods for dealing with dependent/correlated outcomes, and Bayesian approaches to meta-analysis.
The course consists of a mixture of lectures, hands-on tutorials, and computer exercises to cover not only the theoretical background, but also provide practical experience with analyzing real meta-analytic datasets. Emphasis throughout the course is on the application of the various methods and the interpretation of the results obtained (supplementary references can be provided to those interested in the mathematical/statistical details).
Note: Despite the level of detail, this schedule is tentative. The starting and ending times of the course are definite, but everything in between is subject to change. Also, breaks are not explicitly indicated in the schedule below, but are planned in throughout the days.
|09:15-10:45||Lecture: Introduction to systematic reviews and meta-analysis|
|10:45-12:00||Lecture: Outcome measures for meta-analysis|
|14:00-15:30||Lecture: The meta-analytic equal- and random-effects models|
|15:30-16:30||Lecture: Meta-analysis with R|
|17:00-18:00||Exercise 2 (part a)|
|09:00-10:30||Lecture: Conditional vs. unconditional inferences (equal/fixed/random-effects models)|
|10:30-12:00||Lecture: Moderator analysis (meta-regression and subgrouping)|
|13:00-14:00||Exercise 2 (part b)|
|14:00-15:00||Lecture: Quantifying and examining heterogeneity|
|15:00-16:00||Exercise 2 (part c)|
|16:00-17:00||Lecture: Model diagnostics (residuals, outliers, influential studies)|
|17:00-18:00||Exercise 2 (part d)|
|09:00-10:30||Lecture: Publication bias|
|11:30-12:00||Lecture: Refined tests and CIs for random/mixed-effects models|
|13:00-14:30||Lecture: Multilevel, multivariate, and network meta-analysis|
|14:30-16:00||Lecture: A mixed bag of other topics and final Q&A session|
|16:00-16:30||Some literature suggestions|
In general, all efforts will be made to make the course as self-contained as possible. However, there are a couple things one can do to prepare for the course.
First of all, although all aspects of the entire meta-analytic process will be discussed, emphasis will be placed on the analysis and interpretation step of a meta-analysis. Therefore, a general familiarity with how meta-analyses are conducted will help to provide a better understanding of the course contents (reading a few meta-analyses from one's field of interest would be sufficient to obtain a general impression).
Second, some basic knowledge of statistical methods at the undergraduate level (e.g., regression, analysis of variance, hypothesis testing) will also be beneficial. Also, matrix algebra notation will occasionally be used to present certain equations. However, if you are not familiar with matrix algebra, then this is not a problem. We will use software anyway to carry out the computations and the most important aspect for the applied researcher is the interpretation of the results (which will be covered in much detail).
Finally, the primary software package to be used during the course for the analyses will be R (see below for more details on R). Although everything that you need to know to do the computer exercises will be explained in the course, it would be useful to familiarize yourself with R a little bit ahead of time (here are some notes on preparing to use R). You could also consider following the Intro to R Course first.
We will conduct a number of practical exercises throughout the course that require a computer. Therefore, please bring a laptop to the course. Moreover, you should have the current version of R installed on the laptop (R is a software environment for statistical computing and graphics). More information about R can be found at the R Project Website. You can download R from the Comprehensive R Archive Network (CRAN) (if you use Windows, choose "Download R for Windows", then "base", and then download the setup program; versions for MacOS and Linux are also available).
It may also be useful to install an integrated development environment (IDE) for R. A popular choice these days is RStudio, which is available for Windows, MacOS, and Linux.
Once R is installed, please install the metafor package (an add-on for conducting meta-analyses with R). More details about the package can be found at the metafor package website. If you have an internet connection, you should be able to install the package by starting R and then typing
install.packages("metafor") (when prompted to choose a mirror, select the default option or a location that is nearby).
You can also pair up with another course participant for the computer exercises, so technically not everybody needs to bring a laptop. However, if at all possible, please bring your own laptop so we don't end up with the unfortunate situation that nobody brings one!
The course fee is 600 Euros and includes lunch on all three days, refreshments (e.g., coffee, tea, water) during the breaks, but not dinner or accommodations.
When the course is sponsored by the Interuniversity Graduate School of Psychometrics and Sociometrics (IOPS), then the course is free for IOPS PhD students.
Department of Psychiatry and Neuropsychology
P.O. Box 616 (VIJV1)
6200 MD Maastricht
Tel: +31 (43) 388-3511
E-mail: jolanda.koch (at) maastrichtuniversity.nl
The course will be held at Hotel van der Valk, Maastricht, the Netherlands.
6227 AL Maastricht
Google Maps Link
If you arrive by car, just use the Google Maps Link to obtain directions depending on your starting point. There is free parking at the hotel.
If you arrive by train, you have a couple options. If you leave the train at the main station in Maastricht, then you can take bus number 350 in the direction Aachen (the bus leaves every 15 minutes the last time I checked; you can also check here for departure times). Take the bus to the Akersteenweg stop (this takes about 10 minutes). From there, it's about a 5 minute walk to the hotel (walk back a bit on the Akersteenweg in the direction from where the bus came, then turn left into the 1 Juliweg, then right into the Nijverheidsweg).
Alternatively, if you leave the train at Randwyck train station, it takes about 10 minutes by foot to get to the hotel (just cross the bridge, follow the Joseph Bechlaan, then turn left into the Demertstraat).
|Min/Max Number of Participants||10/40|
|Certificate for Participation||Yes|
|Number of European Credit Points||1|