JournalClub.io

Excerpt from our upcoming course: The Foundations of Artificial Intelligence:⁠ - - - - -⁠ An eigenvalue is a way of summarizing the hidden structure of a matrix. For this problem, think of the similarity matrix as describing the shape of the answer cloud in semantic space. If all the answers are close variants of each other, that cloud is narrow. It stretches mainly along one axis. If the answers are genuinely varied, the cloud spreads across multiple axes. Eigenvalues tell you how much of the structure is concentrated along each of those underlying axes.⁠ ⁠ - A large eigenvalue means one dominant semantic pattern explains a lot of the relationships among the answers.⁠ ⁠ - Several medium-sized eigenvalues mean the set is not reducible to one theme, it has multiple distinct axes of variation.⁠ - - - - -⁠ Full course coming soon: JournalClub.io

Excerpt from our upcoming course: The Foundations of Artificial Intelligence:⁠ - - - - -⁠ This brittleness contributed to what is known as the first “AI winter”. Very early on, the field had made bold promises: about a bright shiny future that was just around the corner. But by the mid 1970s those promises had largely failed to materialize, and many of the systems of the day couldn’t scale beyond carefully controlled demonstrations. On a practical level: they simply required more compute than was available at the time, more hand-coded knowledge than researchers could practically supply, more robustness than the underlying design was capable of, and more funding than was available for these types of projects at the time. ⁠ ⁠ The 80s brought on another period of optimism, as expert systems started to see commercial success in a few domains. But it turns out that the designs of the day were simply too expensive to build, too difficult to update, too narrow in scope, and too vulnerable to failure when conditions changed. So by the end of the decade another 2nd “AI Winter” had begun.⁠ - - - - -⁠ Full course coming soon: JournalClub.io

Excerpt from our upcoming course: The Foundations of Artificial Intelligence:⁠ - - - - -⁠ And interestingly enough, their rise was due largely to external forces. It wasn’t that everyone suddenly realized that learned systems were better, it’s that the playing field changed in a way that lent itself towards their adoption. In the 90s and 2000s data became much more abundant. Computers, storage, and networking became cheaper and more powerful. And statistical methods matured. Then web platforms emerged, generating enormous datasets, and hardware and virtualization advancements made large-scale training more practical. The result was a gradual shift away from trying to encode intelligence entirely by hand and toward systems that could extract useful behavior from examples.⁠ ⁠ This does not mean that symbolic reasoning fully disappeared. My dinosaur analogy is a bit too strong there, to be fair. Nor does it mean that machine learning replaced every form of programmed logic: most code written today is still procedural afterall. It only means that the center of gravity shifted, and that it has continued to shift further and further in that same direction ever since.⁠ ⁠ And just now, over 70 years after Turing’s “Imitation Game” (what we now call the Turing Test) was proposed, are we just starting to be able to build machines that can actually pass that test. Machines that can converse and can perhaps even fool a human being into thinking that they too are human.⁠ - - - - -⁠ Full course coming soon: JournalClub.io