Stat 9917

Topics in Selective Inference and Predictive Inference


Description

This PhD course covers topics in selective inference and distribution-free predictive inference. The first half focuses on selective inference, introducing problems and tools in multiple testing, post-selection inference, and adaptive/sequential inference. The second half centers on distribution-free predictive inference, with an emphasis on conformal inference. Throughout, we highlight recent advances, practical considerations, and open problems in both areas.

Syllabus
  • Multiple testing: global null, FWER, FDR, local FDR
  • Post-selection inference, sequential testing
  • Testing with e-values
  • Empirical Bayes, compound decision-making
  • Permutation test, conformal inference
  • Distribution-free predictive inference
Grading

Grade will be based on a course project. Each student is expected to complete a research project that is related in some way to the themes and topics of the course. Projects may be theoretical or empirical (ideally a mix of both), and students are encouraged to build on or connect to their thesis research. The instructor will meet with students individually to discuss project ideas and provide feedback on progress throughout the semester. The grading components are:

  • 10% - Proposal (due Feb 16, 2026): Submit a short report (no more than 1 page) stating the research problems that you plan to work on. Describe why they are important or interesting and provide some appropriate references.
  • 10% - Progress report (due Mar 23, 2026): Submit a progress report (no more than 3 pages) summarizing what you have completed so far, any preliminary results, and your updated plan for the remainder of the project.
  • 40% - Final presentation (in class): Give an in-class presentation on your project, outlining the problem, approach, main findings, and takeaways. Final timing may be adjusted depending on enrollment.
  • 40% - Final report (due May 4, 2026): Submit a final project report summarizing your goals, methodology, main results, and conclusions. Discuss limitations and possible next steps. Include complete references and, when relevant, provide a link to reproducible code and data

We use canvas for submission.

Reading

The following references are related to the topics covered in the class.

Resources

The international seminar on selective inference (ISSI) is a weekly online seminar on selective inference and related topics. Every week, an invited speaker will give a 45-min talk, followed by a 15-min discussion.

Schedule
Date Topic Reading
Wed Jan 14 1. Preliminaries, p-values and e-values, global null testing Efron Ch. 3.1, Ramdas & Wang Ch. 1-2
Mon Jan 19 MLK day (no class)  
Wed Jan 21 2. Power analysis of Bonferroni and the combination test Owen (2009), Donoho & Jin (2004)
Mon Jan 26 cancelled due to SNO  
Wed Jan 28 3. Simes test, combining evidence (I) Ramdas & Wang Ch. 12
Mon Feb 2 4. Combining evidence (II) Ramdas & Wang Ch.8, Ch. 12
Wed Feb 4 5. Family-wise error rate, simultaneous inference Efron Ch. 3.2, Tukey (1991)
Mon Feb 9 6. The closure principle, graphical testing procedures Efron Ch. 3.3, Goeman & Solari (2011), Angelopoulos et al. (2025), Hartog & Lei (2025)
Wed Feb 11 7. False discovery rate, Benjamini-Hochberg Efron Ch. 4.1 & 4.2
Mon Feb 16 8. Empirical Bayes perspective of BH, PRDS, e-BH Efron Ch. 4.3 & 4.4
Wed Feb 18 9. Competition-based testing Barber & Candes (2015), Candes et al (2018)
Mon Feb 23 10. local FDR, compound decision-making Efron Ch. 5, Soloff, Xiang & Fithian (2024)
Wed Feb 25 11. Testing with structures (I) Ignatiadis & Huber (2016), Lei & Fithian (2018)
Mon Mar 2 12. Testing with structures (II)  
Wed Mar 4 13. Conditional inference  
Mon Mar 9 Spring break (no class)  
Wed Mar 11 Spring break (no class)  
Mon Mar 16 14. Post-selection inference, FCR  
Wed Mar 18 15. Sequential testing  
Mon Mar 23 16. Basics of conformal prediction, score functions  
Wed Mar 25 17. Full conformal, Jackknife+, CV+  
Mon Mar 30 18. Conformal prediction under distribution shift  
Wed Apr 1 19. Distribution-free conditional coverage  
Mon Apr 6 20. Distribution-free risk control  
Wed Apr 8 21. Online conformal inference  
Mon Apr 13 22. Multiple testing with conformal p-values  
Wed Apr 15 23. TBD  
Mon Apr 20 Final presentation  
Wed Apr 22 Final presentation  
Mon Apr 27 Final presentation  
Wed Apr 29 Final presentation