data.bythebay.io: Scott Waterman, Automatic Links for a Web of Text
Recording: data.bythebay.io: Scott Waterman, Automatic Links for a Web of Text
great I don't really need to introduce myself you can find me on Twitter at TS Waterman if you need to talk to me and everybody's in the schedule too so great off we go so I had planned this talk out as a really technical talk I wanted to show you a bunch of stuff linking up browsers and web archives and search engines and all this kind of stuff and I kind of got through that plan and I thought well what the hell is the point because it doesn't make a lot of sense some of it anyway unless you put it in a bigger context and as I started to explore the context I thought well this is a much more rich area to talk about so I'm going to be talking more about the the fundamental architecture of what text is and what the web is and how those fit together so since all the other stuff today or a lot of it's about data engineering and and and science and NLP and that sort of stuff and you kind of need a grounding in what the matter you're working is when you're doing science consider this your daily dose of philosophy and history of science so if it gets a little weird just remember that so I'm going to talk about how we got to now with text because its quest because text is a funny thing and where are we actually at now and what is now span and then do a little blue sky or not so blue skies some near-term projections on where where we're going with how we interact with text and and what text is in the world so that's that this is the question I'm trying to answer today what really is text and you know text is not written speech if I gave my whole talk like that you'd kill me right you know it's a very different thing entirely so let me say that a different way if I write it out this way it looks different it's got all the properties of text that you that you recognize so either I'm going to spend the next half an hour explaining the background of all this or i'm just going to bore you to tears so text really is the original technology it's it's literally technology it comes from the same the same root as art and craftsmanship it's it's an artifact it's something that we as humans created for task for purpose so so let's consider it a technology what is it what it what does it actually do it's a user interface it's a way to preserve and communicate and translate speech and by speech we mean human communication which is composed of words and probably ultimately represents thoughts if you if you go that route like I said it's an artifact we had to invent it it's it's changed incredibly over the span of use and I'm going to talk maybe a little too much about that span and how it's changed sort of give you the you know the the versions of it as we go but like any technology it's got huge problems although we use it everyday we swim in it it's completely non intuitive it's the one technology that we spend literally years teaching kids how to use how do you read how do you write how do you construct a thought how do you how do you compose an essay how do you get through a newspaper how do you how do you know which which place to go on the web which place not to go on the web if you've got kids that's a big issue and you know over time it's got you know huge like any technology huge interoperability problems you have different scripts you have different languages you have different writing styles you have different media they're all buggy they all suck and we use them all the time so as a artifact what are some of the useful properties of text it's visual you know you look at it you don't have to hear it you can actually read silently I think it took people a few thousand years to figure that out and there's some there's a lot of good evidence for that but we'll talk about that it's nonlinear in the way that speech is linear one word has to come after another because you can't speak them all at once all the words on a page or in a document are there all at once a temporarily they're just there you can pick it up you can move it around you can mix it you can add stuff to it you can annotate it it's totally a credible and rearranged abul and movable and you can do all kinds of things so like I was saying I want to talk about how we got here how did this stuff what's the early or the early stage developments a text and then metatext is an interesting category which I want to spend some time on as we write about and describe and label and tagged text the medium we use the technology we use is also text and so that itself becomes part of the thing that we're looking at so if we're looking at processing text and trying to understand it and trying to use computers to understand it or at least manipulate it knowing that all of that is this is actually part of the same subject matter is really important I'm going to blow through the the modern history of computers in hypertext and land on the web and search engines and computability and talk a little bit about where we're going to go and like I said I'm not going to talk a lot about search in IR and LP and sentiment analysis and word vectors and all the things you've been hearing about this morning and we'll hear about this afternoon because that's your day job right we do that all the time so I want to take this opportunity to explore how all that stuff fits together so we can see what it means so one of the great advantages of text if you're looking at it from a historical perspective is that its written history you know the joke sort of write themselves here and so any any artifact that's that's left around is written you can kind of know what it means you can kind of piece it together except for all of those scripts and languages and artifacts that you can't because nobody's figured them out yet right so if you look at Minoan linear a nobody can read it and so there's a big chunk of history that's just missing so its history its code it's basically got its own so source control mechanism because the versions are written down you can look at you know what happens you know fifty years later what happens a century later and that's what we'll talk about so the written word is by itself a totally remarkable invention its representation of speech which is this activity you do we don't really have good you know other representations of things we do in the same way you know how do you write down go into the grocery store and and you know buying lunch except as a representation of the words you would use to communicate them so I it's just a profoundly weird thing in and of itself let's go over the history a little bit so about 5,000 years ago you start seeing texts show up all over the place and surprising how how uniformly all over the old world it shows up and all of the early scripts are the same form there are basically loga forms logographs there's a specific one symbol / word and that persists for a couple thousand years it takes people a really long time of using this and writing it and rewriting it and copying it and putting it on statues and seeing it every day to boil it down and start to simplify it as a UI and so over those couple thousand years you start to see phonetic symbols get introduced people actually are just representing instead of the the the meaning of the thing you say they just represent the sound and that's good enough because if you can go from the sound back to the word you can then say the word to yourself and know what it meant and keep going but but that simple insight takes a couple thousand years in the development of human history and so somewhere around 1000 BC you see the first instance of what we'd really call a modern phonetic alphabet which is early Greek in which there's this really simplified set of symbols just a couple dozen symbols so all of this complex you know hieroglyphics and cuneiform and all this stuff is boiled down to a couple dozen symbols all written with straight lines that are basically so simple a child could remember them and draw them which is exactly the point a child could remember them and draw them we get the Latin alphabet showing up out of a form of Greek about 500 years later in India East Asia you start to see Brahmi script which is one of the precursors to devin gari which is another phonetic script the mean hold out here is is China and Central Asia while all around it all sorts of cultures are inventing phonetic scripts the Chinese have stuck with this complex loga form for 5,000 years which says something about the centralized political control in China and how it's never ever going to go away but that's another issue so besides just the form of the alphabet how you write it down and how its structured is a huge part of what text is if you look at any page or any any display or anything with text it's not just a stream of characters that you have to break up and figure out except it is for 2,000 years exactly what it is so this is this is the earliest story that we actually have evidence for it's about 3200 BC it's the tale of Gilgamesh and the flood and it's literally just a sequence of characters there's a little bit of white space here and there but there are no sentence breaks or no word breaks their know there are there are sort of chapter breaks and that you fit as much as you can on to one of these tablets which are really small by the way it's about the size of a paperback and so that's 5,000 years ago 2,000 years later so this is the constitution of athens written out by aristotle in 1 / 3 or 400 BC in what we consider classic Greek there are no word breaks there are no headings there are no page numbers there are no nothing there's no visible structure and it's designed to be read linearly as if you were speaking it if you read this out it's kind of conversational it's like addicted it's like if somebody had transcribed this talk it's just sort of its structured internally by the rhetoric but it's not structured physically and visually and so as a UI it's missing something is missing a lot of things that we take for granted now and so I want to go through the history of the next while and talk about we're all that comes in so with Latin even as opposed to Greek we start seeing word breaks what they call inter pumps in Latin and little dots between the words shortly after you see this thing in Greek called a paragraph goes alongside the graph it's just a little mark that says you know here's a new thought here's a new sentence and that'll be critical as we go so somewhere around 200 latin loses its mind and they decide now we don't need word separators anymore and it mostly stays that way for the next four or five hundred years as kind of just as Latin as the as the Roman Empire is peaking whether it was sort of a political elite thing to try to keep people from reading or not it's kind of unclear a few hundred years later Latin decides how we did we really did need word breaks and so spaces appear and they also get casing they get a minuscule forms small letters people are writing in cursive it's much faster to write it's much easier to transcribe it it takes up less space also so there's valuable stuff that you're writing on you don't have to use as much of it it takes another 500,000 years to for people to put paragraph markers in Latin and the Western European languages and then you know finally right around the time of the printing press people are finally starting just just starting to do things like indent paragraphs and get a spatial form for what's going on so here are some examples just just for fun this is the story of Tristan and is old from about 1300 it's written in French there are there are word spaces there are minimal dots for sentence ends there are no capitals except for the sort of these chapter heading capitals they tell you this begins a major section of the story there's no other structures yes it's just it's just a story as if you know you just narrated it at a sermon or a campfire or wherever you're telling your story a few hundred years later you start seeing marks these are paragraph marks to divide sentences in a manuscript from Germany the letter forms have changed they start to look more readable more modern this is what they call Carolinian minuscule or Carolingian and you know but at the same time as this as things are still being written out by hand it looks like that you get this and this if you don't know is people who would it's the Gutenberg Bible it's it's in a really uniform type because it's actually a uniform type they were all cast and lead from the same mold so it looks good but the spacing is horrible the German black letters is gives you sort of know shape form to read on the words and there's there's just as little semantic layout as there's been for the previous you know 5,000 years there's just there's just nothing so month if you're interested in all this stuff you don't have to go you know to the Louvre or the British Museum anymore to see these things because they've digitized huge collections of them and you can find almost all of them on the web so if you go to the British Library you can see manuscripts carefully digitized from you know 3,000 BC up to present day it's really amazing a little more on form and layout this some of the big stuff so for the first few thousand years if it wasn't literally written in stone it was on papyrus and there's some odd balls like slats of wood and some people were using parchment and that sort of stuff but papyrus was the was the word of the day and it was all rolled up so you had it starts it starts as a cylinder you unroll it you get to see about this much you you know you just you keep unrolling it because you don't have the space to roll out you know three metres of this stuff you can sort of see one page at a time and it's linear just like the text there's no weight is like put in a bookmark and go you know remember something interesting toward the end because there's there's just no physical format for it around 200 the Codex comes out bound book of leaves the kind of thing that we instantly recognized as a book and it begins to gain a lot of popularity and in the space of just a few hundred years wipes out the scroll in all manuscripts manufacture and one of the reasons for this is it's not necessarily because it was better the Codex was introduced time and again over the preceding five hundred years ago before this and hadn't caught on and the reason it didn't catch on this time seems to be that it had a small set of really committed fervent users you know real evangelists for the cause and literally because these were the Christians and they decided that they were going to write the Christian Bible as a codex rather than a scroll and it begins and and it begins to catch on we start to see things like chapter headings like barely appearing in in the late medieval period we didn't really see any in the examples I showed suddenly we get the printing press in 1440 things start to change mostly because of the volume of production in the next 50 years after this there will be more books printed and produced than there were papyrus scrolls written out in the previous two thousand years finally a thousand years after the codex somebody puts page numbers in the damn book so that you can talk about it and say you know oh it's that set piece beginning on page 73 and this wasn't really important before because every copy of a book you know if you write out the works of Aristotle each time you write it out it's going to have a different pagination and so my copy is going to be different from your copy is going to be different from everybody else's copy so there's no uniformity to the you to the two text as a UI in this sense until until we start getting automation into the process and and that'll be a theme as soon as we automate stuff it changes dramatically and that obviously is true of the last hundred years we start to see things like running page head so if you open a book in the middle you kind of know what section you're in because it says rather than just being you know page 73 it actually says you know you know section 3 on solids or whatever it says in Aristotle sort organizational meta content doesn't really show up until well into the Enlightenment you start getting tables and contents back the book index reference six sections really show up in the 20th century it's a really modern invention right so we all write scientific papers we're reading all these technical papers all the time there's footnotes bibliography citations that wasn't there a hundred years ago nobody had them we'll show some examples and again some of this stuff isn't new to this era it just starts to catch on so a table of contents Pliny's history in in 70 ish ad had a actual table of contents labeling out what each book was going to be about the fact that each book was a separate scroll and they were all in a box and they're about a hundred of them maybe that contributed to it but they all actually had tags like literal tags on string hanging off the end of the roll telling you what the thing was so it just it took a while for the technology to catch on all these things that we think of is really useful as sort of interaction design and you I aren't nearly as obvious as they are after the fact and that's well we'll talk about that more so so here's an example of how crappy some stuff is this is this is a first edition of Newton's Principia this is actually Newton's Principia this is his copy of the print capilla they've got at the British Library with his annotations and marks and things like that because he made a bunch of Corrections that went into the second edition when he fixed up the calculus 30 years later it does have page numbers it does have some kind of loose section marking it's got no table of contents it's got no index it's got no running heads you open in the middle you don't know where you are you sort of you have to read the thing linearly to understand or stick in a bunch of bookmarks and it's in goddamn Latin so english-speaking science beginning of the Royal Society everybody's actually doing their work in English but the official the official text of the time is latin which means it's really limited 50 years later it's finally translated to english there's a volume out that has running heads and this sort of thing it takes more than a century after it came out for there to be a copy that had it back of the book index right and it's still the same English translation that was that was made 75 years before that so this book that's kind of the the centerpiece of the Scientific Revolution in Europe still didn't get any attention from a usability perspective so as we as we work with data and computers and things know that it's really hard however at the same time other people are starting to do stuff that actually looks like a modern book you know this is sort of a history of London it's God page numbers it's got running heads it's got margin comments it's got liberal use of different typefaces to try to indicate different kinds of contents they actually have quotations which are kind of interesting because they're not bracketed they're just there's sort of a running marginalia against the the quoted section to make you know that that's somehow special sort of thing you might use you know color for or set it off and block print or something these days okay so all these things layout them and metadata and all this stuff around the actual content words there the API to the content in the text and so if we start to think about them as an API from mine put on our engineer programming hats you can begin to understand how everything fits together and how you can make better use of text today with the tools we have I've got 14 but good enough so so we'll talk about all these things library catalogs start to come out as the collection of books gets bigger at the beginning they're literally a bibliography somebody wrote down the books so you'd have a library with 500 books you'd actually make a list so you know what's where and which ones you meant your friends that kind of thing and you might only have 500 books if you were bloody rich as size of collections come up actually cataloging things becomes important French Revolution they seize all this property they suddenly have a national library what's in here right the stuffs obviously valuable but you need to catalog it same is true of library Congress here in the states the british library and in the 1900s you certainly get sort of much more rigorous and meaningful classifications so the same thing that Linnaeus did with you know the animal taxonomy in in the Enlightenment we now have to do with books about everything because there's suddenly enough books that it makes a difference most of you are old enough to have used one of these but in the state university library it's huge you know they're thousands of drawers my author by subject by by title and you have to paw through them and find what you want you know maybe we could monetize this put a little ad on every card because you're because you're there all the time but so so that's the that's the indexing system to see what books you have but when you're writing about them and and cross-referencing them in the content that bit of API again took a long time so for about three thousand years references were exactly that their references as you're talking long you say oh yeah in Aristotle he said blah blah blah so these are two things from the philosophical transactions of the royal society right around 1700 the first one mentions you know treatise by Christian Huygens and the next one talks about Bernoulli but it's exactly they talk about them right so I you know I used the theorem from mr. Bernoulli that he wrote in 1694 great there's no actual citation until we actually you know so this 150 years later made 1800s you suddenly start seeing things that look a little bit like modern citations in scientific papers this is after the discovery of electricity and and and oxygen and all kinds of stuff now we start to see this stuff and again these are all from the Proceedings of the Royal Society which you can actually now just go read from you know your your living room from your laptop so it's been a long time talking about the last few thousand years because a lot happened and I'm gonna spend a little bit of time talking about the last hundred years because even more has happened but you're probably familiar with more of this so I put punch cards at the top because it's the first time that text is now represented as something that's totally non-obvious before you wrote down text on a piece of paper or a wall or a stone carving and it was there as text it was a code you had to know the code you have to read it but you could now it's in little holes on pieces of paper and boxes and you can't read it and so so this era assumes is ever growing complex set a machinery that you need that you absolutely need to understand text you know most of text i consume these days is on one of these things it's on a computer you know it to read an Aristotle scroll from 2,500 years ago I don't need any technology except maybe an air-conditioned room and and and the permission of the librarians but to read you know the to read the Proceedings of a technical conference I need a thousand dollars worth of equipment I need internet infrastructure I need all this stuff so hopefully there's something on this on this list that maybe aren't familiar with there's been a lot that happened in the last hundred years until you know 20 30 years ago when we when we suddenly got the web which really has changed everything and it is the biggest change since what since since the printing press it's more than that it's at least the biggest change since the codex and it might be the biggest change since the alphabet it provides a lot of things random access very persistent data it's mixable in a way that actual printed written text never was and you know the it's it's universal it's really all connected and in some ways you can think of it and I'll show that you should think of it as one thing it's not like it's a paper here and a page there and a and a tweet there it's really all one thing and you're looking at a page of it and this why you know maybe we call these web things pages and the link in in HTML the href is really the thing that does is it abstracts this horribly complex project you know process of reading a citation figuring out what it is going to find it whether it's on your shelf or on your desk near at hand or whether you have to go to the library or whether you have to like travel to another city to get it it's a long process and now it all becomes one thing so changes this horrible process of going and looking stuff up and looping around through the catalog and looking the stacks and getting your book to something that's may be as simple as flipping a page and we call them web pages for a reason I think so I think this is where we are now and i'm just going to posit that the text is this giant collection of stuff we don't have a bunch of different documents and different things we've got a web of stuff the same way you had a library of stuff 100 years ago and so it used to be that the substrate that we had was dumb it didn't do anything for you now it's totally active not only locally on your device or in your pocket you know everybody's got one of these it's got more compute power than the world had 50 years ago but it's it's it's smart so i'm actually going to run a couple minutes over so tough but the web isn't all linked together there's a lot of there's a lot of links missing at the early days 20 years ago if you wrote out a web page which you did in a text editor you put in the links yourself you tried to link to authoritative content you tried to make it interesting now there's too much stuff it's always changing you don't put that link in because you know it's not going to be there in six months and your page is going to look screwed up you leave it to the search engine you leave it to the CMS system you leave it to some automatic process so we already have automatic processes tied intimately into the stuff we're writing but at least for some of these things it pushes the effort off to the reader so instead of just following that link you know and clicking you read a page you know you go to Google you get some results you loop around through the results you have to read stuff figure out what it meant finally you go someplace so the search engine is taking this giant collection of data and munging it all up and turning it into something that is actually usable and it adds this layer of global intelligence onto the text and and well I guess one of the things I'm trying to claim today is that global intelligence is now part of the text the same way that that page numbers became part of the text in a thousand years ago to search engine the CMS that whatever is part of the text so it's smart enough to to actually tell you you know what's related and what's going on it can figure this out with you're really dumb 2.7 word query I used to work at Bing the kinds of queries you'd see in the log are amazingly short and crappy and you'd still have to return something useful and it does and it does that by watching what everybody is doing so not only is this is the compute system now part of the text the actual writing creation reading using operating with it is really part of the system and if you think of it all as a system it can change your perspective I'm gonna skip so one of the problems you get since nobody's putting in links for you or theirs or maybe they're local you know one newspaper has a bunch of internal links because you've related stuff to read your facebook page has only stuff from your friends you don't really know where you are you don't know what the rest of the stuff is you've kind of lost you know you've lost your peripheral vision of what's going on in fact often you don't even know what's behind the link on the page or especially on the phone right you get a get a web page on the phone there's a little link it's hard to see the URL whatever Apple's starting to help by giving you a preview for that stuff so you can kind of figure out what's going on what's what's happening but these layers are pretty dumb so the thing I wanted to close with is some investigation and some commentary on where this is all going because these layers really are adding intelligence human intelligence an AI and machine learning and all this stuff that we all do in our day jobs are becoming part of the system and if you want the stuff that you're doing to be a successful part of the system you should understand the whole kind of ecology of it and figure out where you where you fit in so you know these are literally off the cuff you know things you might want the text to do for you things that the browser and plus some global compute resource like Google or now you can almost build this yourself for the top you know 5 million web pages it doesn't it really doesn't cost a lot on Amazon I've done it you know that the page itself can tell you what other people think about it you can see other people's comments on this content you can find out what your own community thinks about this particular page you can sort of like the page can be smart and instantly find all that stuff thats related to it that's out there in the world you can look through your own history to find you know that that thing that I lost last week you know where was it right you can do the same the same search engine search and try to find it or the system that you're interacting with can just remember that it can it can sort of build automatic encyclopedic references for the things that it's reading because it can read it it's it's sort of dumb right now it can't make sense of it the way you can but there's enough NLP out there that you can you can sort of automatically glue all this stuff together and the point is that it's not none of this stuff is crazy blue sky stuff because it's already here you just you've seen bits and pieces of it and this and that and you know just like the fact that page numbers took a couple hundred years to really come into play or the alphabet took a thousand years to kind of develop out of this other stuff it's coming we've hit time I had some some things and and products that are out there that demonstrate some of this stuff I have happy to talk about more of them with people after the talk and I'm happy to this slides will be online with all this so I want to say thank you for waiting for your lunch until I finish and let's have a good day here you