Stat 9917
Topics in Selective Inference and Predictive Inference
Description
This PhD course covers topics in selective inference and distribution-free predictive inference. The first half focuses on selective inference, introducing problems and tools in multiple testing, post-selection inference, and adaptive/sequential inference. The second half centers on distribution-free predictive inference, with an emphasis on conformal inference. Throughout, we highlight recent advances, practical considerations, and open problems in both areas.
Syllabus
- Multiple testing: global null, FWER, FDR, local FDR
- Post-selection inference, sequential testing
- Testing with e-values
- Empirical Bayes, compound decision-making
- Permutation test, conformal inference
- Distribution-free predictive inference
Grading
Grade will be based on a course project. Each student is expected to complete a research project that is related in some way to the themes and topics of the course. Projects may be theoretical or empirical (ideally a mix of both), and students are encouraged to build on or connect to their thesis research. The instructor will meet with students individually to discuss project ideas and provide feedback on progress throughout the semester. The grading components are:
- 10% - Proposal (due Feb 16, 2026): Submit a short report (no more than 1 page) stating the research problems that you plan to work on. Describe why they are important or interesting and provide some appropriate references.
- 10% - Progress report (due Mar 23, 2026): Submit a progress report (no more than 3 pages) summarizing what you have completed so far, any preliminary results, and your updated plan for the remainder of the project.
- 40% - Final presentation (in class): Give an in-class presentation on your project, outlining the problem, approach, main findings, and takeaways. Final timing may be adjusted depending on enrollment.
- 40% - Final report (due May 4, 2026): Submit a final project report summarizing your goals, methodology, main results, and conclusions. Discuss limitations and possible next steps. Include complete references and, when relevant, provide a link to reproducible code and data
We use canvas for submission.
Reading
The following references are related to the topics covered in the class.
- Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction. B. Efron, IMS Monographs.
- Hypothesis Testing with E-values. A. Ramdas and R. Wang, Foundations and Trends in Statistics.
- Theoretical foundations of conformal prediction. Anastasios N. Angelopoulos, Rina Foygel Barber, and Stephen Bates.
Resources
The international seminar on selective inference (ISSI) is a weekly online seminar on selective inference and related topics. Every week, an invited speaker will give a 45-min talk, followed by a 15-min discussion.
Schedule
| Date | Topic | Reading |
|---|---|---|
| Wed Jan 14 | 1. Preliminaries, p-values and e-values, global null testing | Efron Ch. 3.1, Ramdas & Wang Ch. 1-2 |
| Mon Jan 19 | MLK day (no class) | |
| Wed Jan 21 | 2. Power analysis of Bonferroni and the combination test | Owen (2009), Donoho & Jin (2004) |
| Mon Jan 26 | cancelled due to SNO | |
| Wed Jan 28 | 3. Simes test, combining evidence (I) | Ramdas & Wang Ch. 12 |
| Mon Feb 2 | 4. Combining evidence (II) | Ramdas & Wang Ch.8, Ch. 12 |
| Wed Feb 4 | 5. Family-wise error rate, simultaneous inference | Efron Ch. 3.2, Tukey (1991) |
| Mon Feb 9 | 6. The closure principle, graphical testing procedures | Efron Ch. 3.3, Goeman & Solari (2011), Angelopoulos et al. (2025), Hartog & Lei (2025) |
| Wed Feb 11 | 7. False discovery rate, Benjamini-Hochberg | Efron Ch. 4.1 & 4.2 |
| Mon Feb 16 | 8. Empirical Bayes perspective of BH, PRDS, e-BH | Efron Ch. 4.3 & 4.4 |
| Wed Feb 18 | 9. Competition-based testing | Barber & Candes (2015), Candes et al (2018) |
| Mon Feb 23 | 10. local FDR, compound decision-making | Efron Ch. 5, Soloff, Xiang & Fithian (2024) |
| Wed Feb 25 | 11. Testing with structures (I) | Ignatiadis & Huber (2016), Lei & Fithian (2018) |
| Mon Mar 2 | 12. Testing with structures (II) | |
| Wed Mar 4 | 13. Conditional inference | |
| Mon Mar 9 | Spring break (no class) | |
| Wed Mar 11 | Spring break (no class) | |
| Mon Mar 16 | 14. Post-selection inference, FCR | |
| Wed Mar 18 | 15. Sequential testing | |
| Mon Mar 23 | 16. Basics of conformal prediction, score functions | |
| Wed Mar 25 | 17. Full conformal, Jackknife+, CV+ | |
| Mon Mar 30 | 18. Conformal prediction under distribution shift | |
| Wed Apr 1 | 19. Distribution-free conditional coverage | |
| Mon Apr 6 | 20. Distribution-free risk control | |
| Wed Apr 8 | 21. Online conformal inference | |
| Mon Apr 13 | 22. Multiple testing with conformal p-values | |
| Wed Apr 15 | 23. TBD | |
| Mon Apr 20 | Final presentation | |
| Wed Apr 22 | Final presentation | |
| Mon Apr 27 | Final presentation | |
| Wed Apr 29 | Final presentation |