Dustin Tran

Cambridge, MA

dustintran.com

Age: 22

Grad student at Harvard. Research interests in machine learning (Monte Carlo methods, variational inference, Bayesian statistics).

Jan
4
comment Running tests over samples from cross-validation on multiple datasets. To average or not to average?
Is this a homework question? As I've said, they're both bad. If you're looking for the better of two evils, you can use the same reasoning applied above to try to deduce this.
Jan
2
comment Running tests over samples from cross-validation on multiple datasets. To average or not to average?
In order to see if the model is statistically significant in general, you test against a null hypothesis, i.e., "random guessing". This can be reformulated as a null model, and thus what you now do is model selection comparing how different the two models are. If you want to do an averaging using CV as your comparison, then what I described above is the best way to do so. You form a $t$-test on one statistic, the mean of the $k$ errors. This is slightly problematic as I mentioned above, but not as much as the other proposed $t$-tests.
Dec
31
awarded Teacher
Dec
30
answered Can I include covariate as random effect in glmer?
Dec
30
answered Running tests over samples from cross-validation on multiple datasets. To average or not to average?
Dec
30
awarded Autobiographer
Dec
30
answered random forest modelling with high dimensional data
Dec
16
awarded Caucus
Dec
4
awarded Student
Dec
4
asked iTerm2 not pasting newlines for long chunks of code
Nov
28
accepted autochdir and NERDTree
Nov
28
asked autochdir and NERDTree
Nov
3
awarded Popular Question
Sep
27
awarded Autobiographer
Jul
25
awarded Autobiographer
Jul
25
awarded Yearling
Jul
25
awarded Autobiographer
Jul
25
awarded Autobiographer
Jul
25
awarded Yearling
Jul
25
awarded Autobiographer
1 2 3 4 5