As much as one can be said to enjoy thinking through the details of how to evaluate AI (and it actually can be kinda fun!), I enjoyed this in-depth guide from Hamel Husain & Shreya Shankar.

All year I’ve been focusing pretty intently on how to tease out the details of what makes image creation & editing models “good” (e.g. spelling, human realism, prompt alignment, detail preservation, and more). This talk pops up a level, focusing more on holistic analysis of end-to-end experiences. If you’re doing that kind of work, or even if you just want to better understand the kind of thing that’s super interesting to hiring managers now, I think you’ll find watching this to be time well spent.