Toolbox
On this page
Resources, software, tutorials, coding projects, and tools developed by us or by other research groups.
π¦ denotes resources developed by members of our lab
Data analysis & scientific computing guides ππ»π
Data sci & statistics
-
R for Data Science
Free online book on how to do data science with R (largely in tidyverse). Learn how to get your data into useful structures, visualize it, and analyze it. -
Statistical inference via data science
Online tutorial on data science in R, larging using tidyverse. -
Tidyverse skills for data science
Another online tutorial about the tidyverse ecosystem of R packages, with practical examples. -
Statistical methods for behavioral and social sciences
Free online materials for a statistics course designed by Tobias Gerstenberg. Last updated 2024/08/16. Best used in combination with course notes. -
Design of experiments: Advanced statistics
Course materials for Elizabeth Page-Gould’s graduate class on advanced statistics. -
General linear models
Course materials and video lectures for Patrick S. Forscher’s class on general linear models. -
Data science: Theories, models, algorithms, and analytics
Class notes developed by Sanjiv Ranjan Das for his course on machine learning with R. -
Understanding mixed-effects models though data simulation
Tutorial on simulating data for mixed-effects models in R, by Lisa Debruiine and Dale Barr. Includes extended examples with cross-classified models and binomial data structures. See associated paper. -
An introduction to data analysis
By Michael Franke. Last updated 2023/02. -
10 common statistical mistakes to watch out for when writing or
reviewing a manuscript -
The Good Research Code handbook
For grad students, postdocs, & PIs who do a lot of programming as part of their research. -
R best practices
Suggested list of best practices for coding in R and organizing projects. -
R code for meta-analyses
Introduction to meta-analyses with focus on statistical methods. Last updated 2023/10. -
Learn R 4 free
More R resources and tutorials.
Data visualization
-
Data visualization with ggplot2 π¦
Paper on data visualization (philosophy and tutorial using ggplot2 in R), with commented code for a wide variety of common data viz situations (means, proportions, relationships). Co-authored with Eric Hehman. -
Modern data visualization with R
Free online guide by Robert Kabacoff. -
Data visualization with R
Tutorial by Martin Schweinberger, aimed at beginners and intermediate users of R. -
Data visualization course
A full data visualization course by Andrew Heiss. -
The R graph gallery
A collection of charts made using R. -
Choosing what type of data visualization to use
Helpful data viz cheatsheet found in a twitter thread. -
From Data to Viz
Super clean and useful resource for deciding what type of visualization to use with your data, and the R code to implement it.
Machine learning
-
Open-source video lectures from the Summer Institutes in
Computational Social Science
Boot camp materials. -
Google Colab tutorials | Python
Coding tutorials for Python, data science, and machine learning, with a few hands-on examples in Google Colab. -
TensorFlow tutorials | Python
Coding tutorials for ML and deep learning methods in Google Colab. -
HuggingFace course | Python
Another Coding tutorial for ML and deep learning methods. -
Lightweight machine learning classics with R
Online textbook on machine learning with R. -
CalTech 2022 data science and AI for neuroscience summer school
Videos and slides. -
Neuromatch computational neuroscience course
Good introduction to theoretical modeling and data-driven analyses. The substantive focus is less relevant for our lab but the tutorials are helpful.
Tools for collecting data & running experiments π¦π’οΈ
-
ExperienceSampler app extension for jsPsych π¦
Open-source smartphone app framework for building your own experience-sampling study app, integrated into jsPsych (7.3). App development code and documentation builds on the ExperienceSampler framework developed by Sabrina Thai & Liz Page-Gould. Built with Cordova, an open-source library that allows JavaScript to access native device functions on smartphones. Last updated 2023/09. -
jsPsych
Open-source Javascript library for running experiments on the internet in a web browser, with numerous contributors. Currently managed by Josh de Leeuw, Becky Gilbert, and BjΓΆrn Luchterhandt. -
Experiment Maker
Drag-and-drop, βWYSIWYGβ browser-based editor for behavioural experiments, created by Tamara Schembri and Peter Budziszewski. No coding required, free templates. -
PsyToolkit
Free and easy-to-use toolkit for generating experiments, programming, and running psychological studies. Part experiment generator, part teaching aid. -
How to maintain data quality when you can’t see your participants
-
SoPHIE Labs platform
For online experiments, behavioral research, and games. -
Chatplat
Platform that allows you to design, administer, record, and analyze real chats between participants. -
Overview of social media research tools
Last updated 2019/06.
Theory construction & metascience π‘
-
Theory before the test: How to build high-versimilitude
explanatory theories in psychological science
By Iris van Rooij and Giosuè Baggio. -
Beyond Newton: Why assumptions of universality are critical
to cognitive science, and how to finally move past them.
By Ivan Kroupin, Helen Davis, and Joseph Henrich. -
A contextualist theory of knowledge: Its implications for
innovation and reform in psychological research
By William J McGuire.
Some background reading on the various replication/theory/generalizability/validation crises in our field:
-
Theory in social psychology: Seeing the forest and the trees
By Yaacov Trope. See the article by Tal Yarkoni below for a kind of counterargument. -
Choosing prediction over explanation in psychology:
Lessons from machine learning
By Tal Yarkoni and Jacob Westfall arguing for a shift away from parsimonious explanations and toward prediction. -
Integrating explanation and prediction in computational social science
By computational social scientists and metascientists arguing for the field’s approach to resolving the trade-off between parsimony/explanation and complexity/prediction in social science. -
Dynamical minimalism: Why less is more in psychology
By Andrzej Nowak, on a non-computational approach to reconciling the trade-off between parsimony and complexity in psychological theories. -
Replicability, robustness, and reproducibility in psychological science
Metascientists’ reflection on the replication crisis. -
The generalizability crisis
By Tal Yarkoni. -
The validation crisis in psychology
By Ulrich Schimmack. So many crises! -
Addressing the theory crisis in psychology
By Klaus Oberauer and Stephan Lewandowsky.
Some approaches to theory-building from psychological scientists:
-
Making a theory useful: Lessons handed down
By Tory Higgins. -
Dynamical minimalism: Why less is more in psychology
By Andrzej Nowak, on one approach to reconciling the trade-off between parsimony and complexity in psychological theories. -
A perspectivist approach to theory construction
By William J McGuire, on a ‘perspectivist’ approach which “assumes that all hypotheses and theories are true–as all are false–depending on the perspective from which they’re viewed.” -
Collaboration: The social context of theory development
By John M Levine and Richard L Moreland. -
Mind the gap: In praise of informal sources of formal theory
By Susan Fiske. -
Creative hypothesis generating in psychology: Some useful heuristics
By William J McGuire. -
The replication crisis in psychology: An overview for theoretical
and philosophical psychology
By Bradford Wiggins and Cody Christopherson.
Software & packages π οΈ
-
Bootstrapping confidence intervals around ICCs π¦
R script & tutorial to boostrap 95% confidence intervals around intraclass correlation coefficients (ICCs) and variance components in linear mixed-effects models (based on lme4). The 95% CIs of ICCs can then be examined for overlap to determine if they differ. Tested on two-level and cross-classified multilevel models (but should accommodate any number of clusters and their interactions). Last updated 2020/02. -
Estimate ICCs in linear mixed-effect models π¦
Convenience function to estimate intraclass correlation coefficients from lmer models (lme4) in R. Accommodates any number of clusters and their interactions. Formulas for calculating ICCs based on Raudenbush & Bryk (2002). Last updated 2021/10. -
Sampling from data to assess when averages are stable π¦
R script & tutorial to assess how many observations are needed for an average to become stable. Co-authored with Gabe Nespoli, Eric Hehman, Eugene Ofosu.
Other reading lists & book recommendations π
-
Gantman lab library
Topics include moral cognition, moral values in institutional settings, and gender and morality. -
Computing & modeling reading list
Topics include network analysis and dimensionality reduction.