SF Scala: Eugene Burmako & Alex Suhan, Swift as syntactic sugar for MLIR

Recording: SF Scala: Eugene Burmako & Alex Suhan, Swift as syntactic sugar for MLIR

[Music] again whoo okay yeah now it's going to better well thanks a lot of extra for the introduction and my introductory speech is completely unnecessary now which is great we can jump straight into the content my name is Eugene and as you might have guessed I work at Google on the school project actually I joined the company just half a year ago in in February and before that I was doing a lot of Scala for about seven the half years first starting at EPFL this is the University where Scala got invented and then moving on to Twitter arguably Twitter has the biggest scale code base in the world so if you if you did score before you may have heard of some of the projects that I've mentioned here reasonable Scala compiler that's a new compiler is developed at Twitter some a former colleague Quinn is here to represent that project is champion of it then there is color matters color macros - all sorts of interesting stuff that are related to compilers and I love Swift for tensorflow because it's about it provides lots of opportunities to his compilers for the greater good which is awesome and now alex is gonna say a few words about himself okay hello I'm Alex and I work with Eugene on the suport tensorflow project and before that I worked on many compiler projects using accelerators actually also a sequel analytics engine using GPUs so plenty of stuff that my github handled if your curse and all right I'll take the mic again so it's not just about the two of us at this project the system earlier integration it would not have happened without lots of our colleagues and you know other folks outside Google there was a lots of iteration on the design and which added with the guys at Apple had the Swift team also talked to fast fast AIA folks for instance Jeremy Howard so that's a share contributions from a lot of people and thank you all guys and the hyperlink here and we'll share the slides later and it goes to our design document to the credit section there's a lot of people mentioned there all right so our title which I guess well forgotten because it was there on the screen for for so long is Swift as syntactic sugar formula we are I guess it's pretty much clear what Swift is it's the language that he used to write iOS apps of course but it turns out after the day I think you know that it's capable of much much more but what's the syntactic sugar stuff and what Somali are that's what we'll figure out in the next few minutes so mo ER is something that's also developed at Google it's called multi level intermediate representation but it actually is D abbreviated in multiple ways including machine learning intermediate representation and it's a lot of things that's it's a very unique project that encompasses a lot of things and here are the three cool things that I can personally recount about it so first of all it's a it's not just an AI are an intermediate representation but it's it's a way to build the earth to build modular radars and it is designed to be shared between multiple compilers we will see examples shortly additionally it's also an infrastructure and infrastructure that can be used by compilers to transform code or to do great error messages lots of other interesting things and finally one of the main design goes over my yard is to accommodate the explosion of hardware that we've seen recently and I'm talking not just about the usual suspects CPUs and GPUs but also about various kinds of accelerators now to elaborate on those points so first of all why do we need a common compiler er so we have a VM right and that should be enough for everyone or you know so people thought like 20 years ago so actually if we recall the experience of multiple compilers that use all V Mir we see that many of them they need something in between all of the M and s T's of the underlying language so in case of Swift that SIL in case of rust that's another representation julia has something like that and you know this facepalm emoji that's inserted by Chris that that means that he wishes that there was an intermediate they are for clang as well and you know it turns out that for every language it's it's useful to have something that reflects some of its high-level features and not just jumps immediately into all of em and the current situation is can define right because you know all the M is clearly the industry standard now but what's what's suboptimal there is that improvements in individual yars for you know individual languages they don't neatly propagate to each other so let's say in SIL they pay a lot of attention to error messages and so their intermediate representation it keeps track of source positions everywhere even for things like you know literals but you know how even if the Swift team spent a lot of effort making this happen it does not benefit other similar ers so I think we as an industry could use some reuse in that sense and it would be great if there was you know an overarching infrastructure that would allow to build the top-notch compiler ARS and that's one of the goals of Amalia another thing speaking of compiler infrastructure so I understand this may have sounded abstract when I put it that way but so let's let's get into a concrete example from the tensorflow ecosystem this is something that you know our or get Google is working on so in the task focus system there's a you know pretty solid intermediate representation called the TF graph and you can do all the things with TF graphs including transformations grappler this is one module that deals with that but then when you have to to compile TF graphs down to concrete Hardware this is where the fun happens right so all those arrows that go to the right they kinda show different compilation pathways so the top we have excel a this is an abbreviation for accelerated linear algebra so that's that's a compiler for for TF graphs and they shallow that's the high level representation of Excel a and so they have their own mini Erica system and so it can go into LVM it can go to GPU IR TP uses something called tensor processing units this is custom hardware that we're doing at Google and so on so forth so that's that's one island another island at the very bottom we have TF light tensorflow light this is as you could say a subset of TF that runs can run on mobile devices and you know it can interface with an API that's that's the API for Android Android and you know others and you know the common theme here is that each of those islands they have to solve the same similar problems not the same but similar let's say how do you take it TF graph and then transform it to whatever representation that you have on your Island and if there is an error in transformation for instance some unsupported features how do you report that in a sane way so I think it would be good if you did not just be no core dump but you know provided good error messages so that's that's the same thing that we observed on the previous slide how do we build compilers with all you know state-of-the-art techniques without having to reinvent them again and again and the final point why I think Emily R is interesting is hardware support so you know we all know about CPUs we all know about GPUs but actually there's more and more demand and as a result more and more different accelerators that also need to be programmed and the picture that is shown here is a picture of something called a TPU pod this is a bunch of GPU accelerators something that that we do at Google connected together into a big thing that's very parallel and has super impressive computational capability so you know there's lots of computational power there how do how do you make use of it because you know the the architecture of those GPUs it's very different from what CPUs can do and you know we have several versions of GPUs already here one two three there are other accelerators that other companies developed so how do we weekly spin up new compilers and support new hardware and molière provides some answer to that to give an example here we're going to get acquainted with the homework Fisher male are called dialects so what's a dialect dialect is like a mini IR and thanks to Emily are you can combine those dialects together in a single program so here on the Left we see examples of operations from TF dialect this TF dot EDF cone 2d so that's you know operations available in tensor flow then we have Excel a this very cute logo it has its own operations let's say broadcast all to all and you know what's the boss of stuff that these are not just few operations mesh and finally I mentioned tensor flow right and here it is so with the male our it becomes possible to define those operation in a modular fashion and then just putting them together in the same source program and moreover when I mentioned infrastructure I mentioned transformations so it it's easy to express transformations that rewrite those operations to each other to the same dialect or even to different dialects so let's take an example here we start with TF n sum which is a high level machine learning operation which can do the sugars into a combination of ads mat moles and reshapes and so this is this is one of the lowerings that we can define so if you see Einstein just rewrite it right so that you know lower level lower levels of our compiler don't have to deal with it at all so that they they only have to deal with more elementary operations and then afterwards you know depending on on the back end that you have let's say you want to compile for GPUs and for that you need to use Excel a and so Excel Abe has this H low low intermediate representation so then you write a lowering from from those ops to a Shalom add mall and then Excel a takes care of everything else so that's this one compilation pathway let's say you want to compile exactly the same thing for mobile devices so what do you do you you transform that to TIA flight mattemore let's G FL thing and finally let's say you're you know a prominent start-up you develop your own custom hardware and your chips they can do islands really really well and it's one of the main abstractions you literally have this you know instruction in your assembler then you don't care about any of that and you provide the lower indirectly from TF to your chip and this is so awesome here we've been talking about multiple universes that have been existent in kind in a kind of isolated fashion so far right but thanks to them we are were able to express transformations and lowerings in a modular fashion and what's cool is that we don't have to we don't have to specify these multiple times like you know TF goes to a cello where this the sugaring to simpler operations and then lower into to excel a and so on so forth for other devices you can say okay so this high level operation gets lowered to its elementary components the thing in the cloud and then those components get lowered further so it's like a graph you know where you go from a high level code to progressively lower and lower level components and I think this is pretty powerful a great way to write compilers so that's what a person who like a bottom only R and since as I alluded it's planned to target lots of devices CPUs GPUs accelerators so let's just let's just find a way to ride this simile are and you know be first to make use of it because it promises to be a really good thing so this is what this talk is about now let's see some code that was a lot of philosophical discussions pretty pictures but I love code so let's just see how it goes so let's write some of male art this is a function this is real male our centers in something called standard dialect which which is shown by the lack of prefixes here so anyway now we're going to write the matrix multiplication kernel in a bit of an unusual way and we'll discuss why exactly the later so here we have we see a signature of this kernel and it involves two inputs are 0 and R 1 and an output arc 2 and so all of those inputs and outputs their matrices right there type is a bit unusual it says question mark x question mark meaning that dimensions are unknown but you might as well specify the dimensions in advance if you'd like so and then the dialects that care about static shapes static sizes of matrices can apply some optimizations based on that but not in our case let's not go there right now so anyway we have matrices of arbitrary dimensions and you know the last thing after X's f/32 which is basically float now let's see how the code looks like now so Emily are just just like many hours of it uses SSA something called the static single assignment it's a special presentation for code that makes it easy to do a certain class of optimizations and you know we won't go into details there there are books written about that stuff but let's just focus on the feel of it right and so here we see that all operations they look roughly like something equals some operations with possibly some arguments let's say you know percent 0 equals deem of some arguments are 0 and 0 so what happens here present something is like a local variable and the special thing about static single assignment is that those variables are immutable so basically they are assigned only once and if you need to mutate a variable you just introduce another local variable you know and use the old value so that comes in pretty handy because then you don't have this nasty miscibility and many optimizations become easy so that's just a convenient way to write programs convenient way for compiler infrastructure normal air cos types this is something that was grayed out in the previous slides but here I wanna kinda you know put it prominently so all operations are statically typed and we can see results of those operations so you know the knowledge range produces a range of you produces a bunch of stuff and you know this is also very convenient for compiler transformations now speaking of dialects would see an example how they show up in real Emily our code so here on this slide we have something called fine dialect so this I find out for this is you know from from the prefix I find out something you can tell what what the dialect is and so what's it all about it's about something called polyhedral optimizations there's tons of maths about that tons of papers and basically the gist of it is that if you write your for loops maybe nested for loops in a certain way with a certain constraints and then use the indices of those loops like IJ and so and so forth and a certain way then you can rehash your code you can optimize your code in in a very powerful fashion so let's say you write a matrix multiplication kernel which is you know a bunch of nested for loops and then thanks to polyhedral optimizations you can actually take into account super fine-grained details of your hardware let's say sizes of caches like l1 l2 l3 for your CPUs and both some good stuff that's very involved maths and you know it's it's pretty hard to implement from scratch which is why it's great that you know we can have a dialogue that takes care of that for us now there's another dialect which is kind of hidden here linear algebra dialect it provides some primitives that simplify operations with matrices basically and with tensors in general so here this little review what it does it slices the the second and the third parameter of the of the method and it transforms matrix multiplication into a series of matrix by that vector multiplications that's not the most straightforward way to multiply matrices the standard one would be just you know three nested loops right and then tell them in a certain way but you know this will be useful for further examples okay so here it is this was a quick introduction to Molly R and let's see how we can write it in a productive fashion so you know you might have guessed that being on the Swift first answer for team we write stuff and Swift and you know that was a natural that was a natural candidate because you know to express this simple thing that we've seen on the previous slide one for loop and a cola function we had to spend several you know several screens of text so maybe that can be done simpler and I think Swift is very well suited to this task and so here's what I think so in one of the you know one of our design documents of our project you can see this saying that Swift is just a syntactic sugar fall over LLVM this is basically a running joke that we have and what this means is that despite the fact that Swift has a lot of high-level features they actually they are designed to be lowered down to low level stuff so basically tall yam which is pretty low level in itself and in that sense what we'll be doing map in Swift onto Amala yar it's very much in the spirit of the language so here's how one could rewrite the example that we've just seen in Swift so I won't be going into syntactic details here because you know Swift looks like many curly brace using languages you know one detail is that types go after colons but you know that that's also pretty common let's say Scala and you know unlike in the wall of text that the notion of the previous slides we can actually figure out what the code was doing so as I said we create multiple views the views of the entire first parameter you know then we slice the second parameter and the third one the output and finally represent matrix multiplication as a sequence of matrix by vector multiplications so it's all cool you know we don't need to write all those types that we've seen on one of the slides because Swift also has type inference and yeah this is the way how I personally would love to program on the wire so let's make this happen right so let's go from here to here now how do we do that one way to approach this would be to build the male artist translation rules directly into the Swift compiler so you know switch compiler already compiled stuff down to all of the end so please just compile it to Emily are why not that would be kinda easy to implement but on the other hand that would not be flexible enough but I was talking about the explosion of various accelerator devices I was meaning it there's lots of them and you know people experiment with them constantly and in general and more research is a pretty protective area of development so long story short if we just hard code these translation rules what's adequate for today into the Swift compiler then tomorrow we have some new paper people come up with a new way to develop you know machine learning models and we have to change the Swift compiler and again and again and again doesn't be annoying so a common common solution to that is to allow programmers like library authors will say hardware vendors to write these transformations themselves and that's what this talk is gonna be about metaprogramming so here's the quote from Wikipedia which explains this big word meta programming it sounds fancy but essentially this is about programs which manipulate other programs so I don't know if if you write a simple code generator where you put together strings and then spit out a program in your favorite language with was a boilerplate you know in Java in Scala and Swifton in whatever in bash you've written a meta program so there's nothing too fancy about it but you know meta programming has has a big history you know perhaps 6070 years and there's also approaches to doing that stuff in the design document that I mentioned before so this with the syntactic sugar for male are we posted that to our mailing list a few weeks ago we go through a few ways how we could approach this so how do we met a program and while we are using Swift and so as I mention there's three manipulation it's a good way to do meta programming but it works and there's other stuff I think it would take too much time to talk about all of those approaches today so I'll be just talking about quality calls and let's just jump straight into it so quasi calls they our current prototype of quasi codes in Swift anyway they're based on two abstractions quoting an unquote and so we'll start with the first one first of all we introduce a new language feature called the pound quote and this is a new syntactic form so this is not the function this is a special syntax which is kind of signified by this pound in front of quote it can take any expression any Swift expression including lambda expressions like here and by virtue of that it can include basically whatever for loops local declarations local functions even local classes like lots of stuff and what happens after you do pound quote the compiler creates a representation of this code and saves it until runtime so see we define the variable called matmo which holds the body of this quote and if we pretty print it at runtime so this is the cab repo like syntax that told me employing the slides then we will see the representation of the code as I was referring to and it's not just you know it's not like we took the source code of the program and just copy pasted it here because see here we don't have any type annotations except for parameters and here we have a bunch of type annotations so we know that VA is mu VB is a view and so and so forth so clearly it was doing something so let's just let's just inspect the structure of of this map mul thing so there is the scandi method called dot structure which pretty prints you know nested ast is and here we can see what it consists of right so I said that this we've been causing a closure so here we have a struct called closure you know closure has a bunch of parameters a body so and so forth so this is all hidden behind those ellipses but you know the the first part the first parameter we can see here so no parameter nil it's it's the something that perhaps I will not talk about right now but you know name it contains the name of the parameter plus you know some fun stuff so first of all you know we have something called symbols every name if a symbol and you know basically every definition and every reference it has a unique ID and by comparing those those unique IDs you can tell whether something refers to something else so Swift has overloading and as a result we really need that mechanism to take apart things like integer addition falls in points addition tensor addition so that's that's a fundamental thing that people do in compilers and so we implemented that now representation as well also all expressions they have types so here it's kind of trivial we specified this matrix thing the matrix type explicitly in source code but literally every expression if I were to write 1 + 2 + 3 we would know in excruciating detail that 1 is an integer and 1 plus 2 is also an integer and so on so forth again this may seem obvious for that contrived example but when you write compiler transformations you really need this information like what's the type of this thing and again this is a now this is something that that's useful in many compiler related tasks so that's part of our data model now that we've seen quote it's a pretty simple functionality right so you can just say pound quote you can provide any expression and the compiler will give it to you and so you as a library author will then be able to do whatever you want with this expression translate to him well they are pretty printed translate to query to a database like link even c-sharp does sky's the limit so that's well this is just of metaprogramming you can do really whatever with your program but you know let's let's talk about unquote this is going to be less obvious but I hope I'll be able to explain it properly so I mentioned before that matmo is calling into another function Matt back right matrix by a vector multiplication we've kind of glossed over that aspect but if you think how we're going to transform this this quoted code into a male are eventually we would need to know the body of math back it's not just enough to say okay this is some function called blah we really need that body in order to emit executable code and so here's how it will look like in in our representation in our current representation so as I mention before there's this name thing and it has unique IDs so that's what we see format back it's kind of useful to know that it's a function from you to you to you blah right but where's the body so it's it's not here and that's a problem that's I make it sound easy but this is actually a fundamental design choice and meta programming systems and how do you how do these things compose because it's easy to quote code just the stuff that you see but how do you do this in a modular fashion so this is something where a link you based approaches have some problems so what do we do about this and you know as I mentioned before we have 60 or 70 years of research put into meta programming and you know that there is an answer called unquote and so there is another syntactic form that we introduced called bound unquote and it takes something that has been quoted before so you've seen at the top of the slide actually introduced another definition this let Matt vaca cross quote and then after this thing has been quoted we just put it back into the into other quotes thanks to this operator and now when we inspect what's going on in the structure of the result we see that instead of just name we have a special ast node called unquote and apart from just the name it also has the body of the thing that we've been causing so that's that's what we need right yeah so the question was does this mean that we need to unquote everything or otherwise we're doomed well in the current implementation yes and there are ways to go to work around this but we will not have time to go into that so in this quote-unquote scheme in the pure calculus then yes but on the other hand hopefully it's not such a big deal because let's say you add two ends you actually don't need to know how addition of twins is implemented right it's it's a primitive in your hardware most likely so you just compare the ideas like this stuff right you make sure that this is an addition of integers and you just omit your assembly instruction so yes this is a problem yes there's a solution but it's not a huge deal if if you go about this very short all right so now we're done with quote-unquote and I'm done with my part of presentation I'm gonna hand mic I'm not sure I'm using this right but go vote irregardless so I'm going to present our current state of Lauren to him and I are it's very early work so there are some rough edges and Eugene just described the dialects of ml I are the standard direct is a bit it's a bit interesting in the sense that it's an eclectic collection of operators we are talking about split into dialect maybe in certain subjects but that's the current state of things it contains elementwise operations it contains cost operations it contains control flow so it's it's a big dialect but there are some good things about it it offers the granular memory access which we need to translate certain structures and shift and it offers the things I've mentioned the elementary operations the control which are necessary to lower arbitrary shift code so it doesn't offer the sophisticated loop optimizations based on polyhedral techniques but at least it's easy to map low-level kernels to it and since we are just starting we picked this target and it also has backends for CPUs and GPUs so you can actually go all the way from high-level swift code to running on on a GPU with the current infrastructure we built so we pick this because it's always nice to have something which works and iterate upon it so the translation input is the quasi quasi code structure described by Eugene earlier and the current focus is on GPUs and CPUs following CUDA like model of execution so if you're not familiar with it CUDA means that you run the same computation on a bunch of Hardware threads and the result is the combination of all that work so this is a I hope I hope you can see the text it's a kernel body for implementing the average pool operation same machine learning the average pool means you take a bunch of pixels in a window which moves over a base and you take the average so you can see here that it's it's fairly idea Matic shift code we move with with the stride which is the thread count so that we can achieve the parallelism on GPUs and yeah it's you can see the two outer loops which sorry which I trait over the the base the out index Y and doubt index X and then there is two inner loops for input index Y in beginning the Y and read the Y and the one over X which actually I trades over the the window we over we overlap and within that window we compute or some by accumulating into the output and then we just average the values we divide the sum by the size of the window so this looks like a fairly natural shift code and there are some quirks in how the Swift types map to to MLR types so to actually present those multi-dimensional arrays we've introduced our own type called tensor view and if we actually need to accumulate into it or do other sorts of mutations it's called mutable tensor view and also windows size windows right are are obviously integers and let's see how this goes to MLR types so this is shift the those are shift types and in ml I are you can see that the the tensor views both the mutable and immutable tensor view they represented at nested memories so memories in ml are those multi-dimensional memory buffers which can represent answers and we also have to do this thing which people using GPUs are probably familiar with to actually wrap the integers the window size and the windows right in memory as well because kernels run in a different address space so everything is basically it cannot be an immediate you cannot have a parameter which is an integer because that integer has to be allocated on the device transferred and then you can actually launch the car so if you're curious a why is this why are those two integers memories of what single development this is the answer it's it's how the hardware works basically when you're not in the same address space and the way you call it it's it's very similar to an actual function so I've abbreviated the definition of the colonel here and you can see that we just call average pool to do over our input X with a size a window size of two and the stride of two and the results are written to do output so the fact that we have an output parameters is also meant to map as as simply as possible to to the GPU model or accelerator model enjoy where accelerators don't return an immediate they actually deposit the results in some output buffer provided by the user so at some point we are probably going to sugar this to make it more convenient but right now that's how it is and in terms of runtime support translating to ml I are is not sufficient to actually run on the device devices also need ways to to actually read from memory and and just output the results so right now we treat all those tensor views are shown as buildings so those are magical types known to the emitter and the meter knows that those that's how these types behaves behave they are multi-dimensional arrays so basically we will introduce those types to just quiz squeeze it past the pipe chatter and we actually implement this runtime we are going to implement it for for each device and the indexing functionality those views implement it's going its reimplemented in the in the emitter so we actually meet code whenever you access an element in the array we emit some ml our code the plan is of course to grow this library support as needed by by the kernels we are going to discover by adding more use cases and the challenge here is that we are going to find the clean way to hide the standard library from the low-level currents you don't want your low-level curves to be able to use parts of the seafront I'm you'd have no hope of implementing on a tip you for example so the accelerators pose this challenge that they are not complete machines in some sense each accelerator has to has certain demands so we have to find a clean way to to specify the the set of features for each accelerator and lower with respect to that in terms of future plans there is a lot of exciting stuff ahead of us I think the most obvious is that we want to expand the language support it's nice to be able to write some food a style Colonels low-level cars would you actually want to use the full language we don't want to write C Swift so the goal here is to be able to write motor swift and translate that to to CUDA code efficient CUDA code and efficient TPU code and so on so forth and tied into that is also we want this seamless and holistic interaction between the low-level parts of your code and the high-level parts so you could have some high-level layers you write with a very high level Swift and you could have some kernel you write in them in a more low-level fashion using a bunch of loops and we want to be able to ID not move loop in variants across multiple layers of abstraction so having a single target from the high level a high level language MLR with multiple dialects will allow for that and a third direction which is particularly exciting with something like Swift which is a compiled language is ahead of time compilation and I can think of many use cases for that it's great if you really care about deterministic behavior so you want the performance to look a certain way all the time without warm-up issues without any sort of weirdness you're not aware of when you're deploying the code and also probably mobile devices could from that we know that so it is fairly beginner in the mobile space so it would be interesting to to really see what was there in by by using relative time compilation so with this I think I think we are done and I suppose we could take questions yes yeah so it's not a final presentation but yes you have to lower to the actual native code for the device obviously but there is also it's not just representation you could represent pretty much anything with buildings you just say okay I have this built in which allocates memory with this layout and does whatever magic to it so sure it could add that in in the dialect for let's say that you can introduce a dialect for a specific device but then you actually have to implement it so the or internal IR is half or less than half of the work I think a lot is going a lot of work is going to be implementing the road to supporting run time for it it's it's the same for any language job implement the compiler in half the time and the other half is adding writing support but I think you alright yeah so to repeat the question winners asking about multiple levels because in Swift right so I guess Alex wanted to come as well I don't know where your slide is but we actually had here you can see that we have we have linear we have three dialects in this example we have define dialect we have the lean out lean our dialect and we also have a bit of standard dialect things like return so here we already have three dialects yes so a fan is the most low level well naturally a fine is a fairly low level but it also offers polyhedral tricks so the lineage is the more macro dialect so it has operations big operations in it so you can expect to to find things like old in our view view is a big operation if if you need to do some reshapes so it's more macro whereas the standard dialect and the four dialect are more more grammar so that the multi-level part here it's the dialect and so the role of swift here is that we can use the same language to drive all of them supposedly this is so you know early days of the project and additionally Alex was talking about transformations animal er and here we get a slide it said you know here are multiple dialects different levels of abstraction and you can can lower between one another in contrast for one to another and currently this is this transformation is going to be written in C++ in other ways and wouldn't it be cool to write those transformations also in a high-level language which is Swift and wouldn't it be cool to have a compiler to chain that uses the same language for everything because currently if we think about existing two chains let's take tensorflow there's a Python Python front-end there's there's the runtime written in C++ there is a compiler there's also other stuff that we've seen a bit earlier here and so these are kind of different universes and themselves usually written in different languages so what if we could write all of that or you know kinda embed all of that consumed over time into one high-level language I think that would be really awesome so one of the themes another saying that we have in our project is this central theme of infinite HEC ability so what if we could give users you guys basic the power to modify every component in your tool chain it's not like you need it every day but when you really need it what do you do right so let's say you want to add a new operation to answer for it just go ahead and you know write up in C++ you register it in a bunch of places and no not everyone knows it was pause for instance yeah and it's not just that not everyone knows C++ it's also the fact that if you write your extension in the same language as the rest of the system you don't have this barrier between oh the part which is actually a compiler and optimizes and this other thing which I marshal some data over to and you know I lose some sub performer so this this actually will make it possible to cross barriers between user extensions and whatever is in the compiler already then transparently optimized across that at least that's I think that's the aspiration yeah so we just leave the ultimate plan of the project yeah of course but I think it's a it's a pretty exciting thing that you know a good way to start this to start small which is why we're showing this you know weird things how about where I told that stuff and Swift and you know but this is all part of a big plan yeah so the question was obvious support for a new target so do you have to support whatever Daleks were there or what do you do there are multiple ways again so see here we have multiple approaches right you can lower high level operations and the low level ones and reuse that thing like a cello and the TF light thou did but also if you know better you can just rewrite it yourself so you have multiple options that's the thing and I guess don't quote me on that because you know I'm not on the MOA our team and kinda just one of the users right and I guess the idea is that you really should be able to pick and choose you just put together your own dialect like scene says mine she does something right if that's use you well and you define translation rules you have lots of yeah there are plenty of opportunities to use here so let's say you've implemented do so on your hardware let's say you have some matrix multiplication some reshape you don't need to implement I'm speaking from experience implementing guys on is not fun so if you can actually just take an implementation which already exists and maybe doesn't have for the peak performance so maybe you can look at certain patterns later later but at least you have a baseline you you actually have it in on place and you can revisit disease yeah that would be the idea you lower as much as you need it you use the passes the mandatory optimizations you actually need and it speeds up the process considerably yeah so instead of writing an entire compiler from scratch right you can reuse most of the stuff like Alex said let's say lowering for ransom or you know whatever other countries of operations that you may find useful yeah you see this a bit with with Excel as well where you just with Excel a you implement the back end for your target but the high level transformations are actually already existing in Excel a so if you implement a back end for its la you have tensorflow working on your device so that's a lot a lot of the newer startup chip startups do this thing they implement a back end for for Excel a or for ML ir and they to get something for free so the approach is working yeah and so I might yes but you're not going to make everything you rely on it you're not going to make our unrelated hardware rely on your Dalek so okay you've added a dialect with some operations for your hardware so it only that would be fragile if you don't really test tested well so I assume that if someone wants to contribute some standard transformations they're going to be reviewed carefully and people are going to notice really quickly if some loop invariant code motion is broken because everything will break really quickly so yes it's it's just a matter of a fraction of what people build error is actually going to go back to the standard parts of MLA arc and then you're going to have their extensions they're their transformations and if those are brittle I don't think it's making anything worse trying to make the entire system end-to-end by yourself it's going to be more brittle than shearing at least one part with the with a bigger group of people yeah I mean this argument works for okay why why shouldn't every one silo their code and build their own thing and not step on each other's toes I think there is the downside that you're going to re-implement I'm some 11 times and that's that's not sustainable yeah yeah you can push this to an extent that will see exactly where the world line is drawn because sometimes you have this situations where okay should this should this really be in the in the disk dialects for GPUs or would it see it better in the in the standard data because it's more generally useful with a few changes and you know this is going to be as always how well it's done depends on the taste and depends on the people for review who make make the decisions and type I trust them I think oh I think people involved are pretty capable of managing distro all right any other questions go all right so to repeat the question what's the ideal timeline for merging the stuff that we have a transsexual including quasi codes into master of this compiler I presume right this remains to be seen something that we talked about with Alexi before the Meetup so Swift the community has a robust a swift evolution process and it's community driven it has its shepherds this revolution we need the course well team and you know I think it would be really good to go through this process you know for this project so we are part of the surf community and we respect the existing tradition so how this will work understand this is kind of a weasel answer right but it's not intended to be one once certain parts of tensorflow mature we will be submitting Swift evolution proposals and funny that you asked just a few days ago we submitted an automatic differentiation proposal and so afterwards this thing is evaluated like any other proposal we get feedback from the core team from the community and if people think that's a good idea that's going to get merged otherwise well back to the drawing board all right yeah so to repeat the question this is there are certain parallels between the ad proposal that's been submitted recently and this work basically both if you just squint they're both about metaprogramming about transforming Swit programs into something else so why do this differently why not come with a some sort of a mega framework like compiler plugins versus compiler yeah that's a great question I agree an immediate answer would be so quad cyclists they provide a the functionality that has much smaller scope than was necessary for Indy I cannot this this is probably not not the place to go into details of how it is implemented but it really enjoys the integration with the Swift compiler so with quasi holes will just provide symbols and types and here we go you cannot even modify anything but ad it needs to generate code and it needs to integrate into both the type checker and the parts of the compiler that work with still the intermediate representation of the SOS compiler so it's more overarching I can perhaps speak from my scholar experience so in Scala we tried to build this framework that would allow for a lot of power this is called scala effect and then we moved on to scala meta and so if there's one thing that they learn from that you know it's really hard to do this in a robust manner so perhaps there are some of you in the room for some Scala macros and you guys can can tell what so so I think to sum it up detail they our answer is that sometimes it's really useful to have different systems for different things and if you write one mega system especially for meta programming there is a risk that the dope in too powerful meaning you know basically interactable or hard hard to control alright another question so I didn't get the second part of the question I think MLR follows a similar process of proposal so there are open meetings for the MLR interest group you should be able to attend any of those and submit proposals so it's there are many stakeholders and I don't get the second part of the question so if you could repeat that I don't think if I'd call it 1.0 but the things mostly work you can actually generate working code for things I've shown to you like the average project we can actually generate code which runs so we have tests which actually run it I think it's it's not polished on all architectures CPUs are and GPM GP are probably the furthest along and the standard dialect of oval so that's probably not that far from a 1.0 release but yeah I'm not very sure where isn't version in guillotine ml are kids I guess that would be a good question for the actual Emily our team yeah so as Alex said they hold open design meetings which is basically a Google hangout anyone can join and this stuff happens every Thursday 10:00 a.m. Pacific time and this week it's cancelled I think or not don't don't take my word for it there's yeah there's a mailing list for Emily are and they post their agenda so perhaps Thursday 10:00 a.m. you'd be able to ask question and this is how it goes there is a bunch of people who join and they write their questions on the chat and then people from their show our team go ahead and answer them there's also another part that you asked about the balance between on was open source and proprietary parts so clearly was say TP use their partly closed right so there's an API to program them but how they are built and you know how the drivers look like this is proprietary and we we cannot tell the world about that however you know as we mentioned before and that's a good picture right Emily R is designed to be modular so it's possible to have an open-source core and then whatever hardware producers can do whatever and they're open source or not it doesn't matter and you know ultimately when you want to lower your code to something that can execute on your accelerator we just plug in those modules and it works and if you decide not to publish those modules well it's only you could be able to do that yeah and in general what I've noticed is that people are biased towards putting as many transformation as possible in the higher layers of a system so it's the it's if something is not in the open source parties because it probably wouldn't make sense for any other piece of hardware than the one a certain deep group of people have targeted so it's it's as open as it actually makes sense I'd say speaking of openness by the way it's it's a good day for this question because just today our company decided to donate them all er to the OEM Foundation and so this is a this is a very cool day for us and our CEO he should be tweeted about the project which is completely amazing let's let's see how things go okay alright I'll repeat the question do we have plans to implement macros for Swift yeah and perhaps go back to the thing about powerful metaprogramming systems and unless that's absolutely necessary perhaps now and concretely for this project I think we can get away with quasi goals if there are other use cases that arise and that are pertinent perhaps we can I mean I already did that once so doing that the second time is feasible all right another question thanks women isn't right okay yeah to recap the question to what extent are we willing to go into the debt level depths to provide static validation of tensor programs right so actually there was some type of stuff already in the slides but it was a really well mask just unmasking so this here right so here we see this this average cool 2d right it's a quote but on the other hand you can call it as a function and it statically checks the arguments because this is this quote it says that you know how does a mutable tensor few of you and you know input is a tensor of you if you pass like integers there it will not compile so Swift can do some type of stuff thanks to protocols and I think this is this is pretty good more advanced things like validating shapes of tensors that's that's more advanced yeah I think for something like that you have to be really mindful of for example dynamic page size so let's say you have a batch matrix multiplication and sure the batch can be dynamic but the inner dimension of the first matrix has to be equal to the outer dimension of the second Matrix so you basically need to introduce some type constraints if you want to go before further and say oh okay those sizes are actually constant and this those other two dimensions have to match so it's it's it's a scary part I'm not saying we are never going to go that path but yeah it's very tough yeah let's put it you say there are proposal at MLR level and we have to be careful what kind of box is will open because some of them might contain yes all about things as I said it's it's actually possible to replace those question marks with funny stuff you can't replace them with constants as of now so that that's something we can do if you want to type statically everything then yeah that's something you probably could ID one of the higher layers but then people complain all the time that oh now I'm going to have to recompile my program with a different batch size and this is silly and yeah ok I'm not I'm not much of a front-end of the language person so I think you Jim is probably the right person to say whether this is positively scary or I'll say the boring thing that I was saying about meta programming it would be good to think about usability aspects first like what kind of error messages are we gonna show the users so Scala in Scala it was very not very easy but feasible to do all that but error messages they were kind of optimal so it was possible to say ok here's a matrix multiplication function you can only multiply matrices whose dimensions yeah how do you do there are messages also that's one thing another thing now if you require matrices to specify their dimensions statically or as type parameters if you don't know about them how do you write functions where we don't care about dimensions well you cannot rescue have to specify them everywhere which is annoying in its own right so there are open questions and I think it would be good to find answers to them before we do that all right another question the mrs. cow well scoff if they're similar in many regards actually which is which is very refreshing I think they're both modern and clean languages that are a joy to work with they they have different aspects so so if this is really good at compiling down to native code which is amazing you can just take your code and compile to whatever and this project a system oh you are right it explodes this this possibility whereas in Scala that'll be a tougher sell right skull is pretty tied to the JVM so how do you know JVM bytecode or translated so there are open questions on the other hand as I mentioned Scala has more powerful type level features which is sometimes good sometimes bad so but but I think the both languages are very forward-looking and they contribute to the entire programming languages community very much in that sense both are great all right the question is what's the difference between a JVM and native languages that just you know jump with me when I made the switch let's see startup times I think that's absolutely amazing it's just compiler program and it it runs and you can compile it again and run again while the genome is still warming up besides birth we are trying really hard in the seat compiler to close to solve that problem and make it a bit more competitive and also you know to the question of whether I miss Scala I did not miss nail-gun for sure yeah that's that's a painful experience and you know in that sense if we reflect on the startup time I think it's really impressive what Dennis Chabot and his contributors didn't scholar hazel so it's it's so amazing that you can actually have that stuff that I described in Scala that's that's a technological marvel would say yeah I think the difference is really more about a language with a lot of runtime and garbage collection and the language which at least in theory has not so much wanted I mean you're you're not going to find anywhere close as much of a rat I mean I probably any other language than the JVM based languages so implementing a garbage collector on GPU or in tpo I think that's that's too crazy even for some of you probably psychological so that that has to count for something I did this was actually a research topic in 2010 hopefully people have given up on it but sorry garbage collection on the GPS accelerated collection as well there are some papers and there are some reversal JavaScript on GPUs but we are going to find it also and a great to mention of disavow disavow yeah I don't know those have been tried in the past the job giselle that I think back then in hard yeah I think there was there was some sort of a hardware virtualization I don't know I haven't studied it I'll take the Sun a malaria is supposed to be more expressive than a la lluvia Maya so whatever you can express in LLVM you should be able to express with the standard dialect of a Millar at least that's my understanding and I don't think we have a good reason to arbitrary limit what it can represent so the answer is that we probably wouldn't be possible [Applause]