{"id":19721,"date":"2022-08-07T23:00:44","date_gmt":"2022-08-08T06:00:44","guid":{"rendered":"http:\/\/jnack.com\/blog\/?p=19721"},"modified":"2022-08-07T23:00:48","modified_gmt":"2022-08-08T06:00:48","slug":"explainer-large-language-models-from-scratch","status":"publish","type":"post","link":"https:\/\/jnack.com\/blog\/2022\/08\/07\/explainer-large-language-models-from-scratch\/","title":{"rendered":"Explainer: &#8220;Large Language Models from scratch&#8221;"},"content":{"rendered":"\n<p>I wish I&#8217;d gotten to work more with <a href=\"https:\/\/www.cs.washington.edu\/people\/faculty\/seitz\">Steve Seitz<\/a> at Google, as I&#8217;ve long admired his wide-ranging work (from Photosynth to <a href=\"http:\/\/jnack.com\/blog\/2014\/10\/20\/a-semi-forgotten-gem-in-picasa-face-movies\/\">Face Movies<\/a> to the company&#8217;s new 3D video collaboration tech). Here he provides a pretty accessible overview of how large language models (e.g. those behind DALL\u2022E &amp; similar systems) actually work:<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Large Language Models from scratch\" width=\"604\" height=\"340\" src=\"https:\/\/www.youtube.com\/embed\/lnA9DMvHtfI?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>I wish I&#8217;d gotten to work more with Steve Seitz at Google, as I&#8217;ve long admired his wide-ranging work (from Photosynth to Face Movies to the company&#8217;s new 3D video collaboration tech). Here he provides a pretty accessible overview of how large language models (e.g. those behind DALL\u2022E &amp; similar systems) actually work:<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[66],"tags":[],"_links":{"self":[{"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts\/19721"}],"collection":[{"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/comments?post=19721"}],"version-history":[{"count":2,"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts\/19721\/revisions"}],"predecessor-version":[{"id":19763,"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts\/19721\/revisions\/19763"}],"wp:attachment":[{"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/media?parent=19721"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/categories?post=19721"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/tags?post=19721"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}