Scale By The Bay 2018: Alon Muchnick, From Scala to ByteCode...

Scale By The Bay 2018: Alon Muchnick, From Scala to ByteCode...

Recording: Scale By The Bay 2018: Alon Muchnick, From Scala to ByteCode...

so hi guys what's up and my name is Allen I'm a team leader for weeks comm it's my second time here it's kalibata Bay and I'm really excited and today I'm going to talk to you about byte code well actually the byte code scholar generates it's gonna be a low-level talk so if you don't understand or can like follow all the details the slides are available and you can ask me later on but just try and get a hold of the principal you're gonna see here and try to have fun okay so sorry bunny so we all use case glasses objects later definition pass functions as values and we do other medical stuff that Scala allows us but those things do not exist on the JVM so how can be they be supported by the Jaypee I'm sorry those things are not supported by Java but how they're implemented on top of the JDM so our agenda for today is we're gonna see a bit one byte code is and what's the structure of a class file and later on we're gonna see how the different scale features I mentioned earlier implemented on top of the JVM so what is bytecode and is there such a thing as scallop bytecode well actually there is no such a thing as Scala bytecode the Scala compiler compiles of scale files the class files the same way the Java compiler combines Java files class files at the end is all about class files in the JVM Java uses Java C to combated source files and sky uses Scala see the JVM is agnostic to the language that created the bytecode so what is actually bytecode it is a set of low-level instructions to be executed by the JVM there are about 200 of those instructions each one between one to two bytes in size I add a a load a a star of just a few of those examples in order to compute the instructions we saw perform bytecode employs an assembly-line register and stick there are two sticks we have the lock of stacks used to hold variables and the operand stack used to hold values of fields functions and binary operations such as + - need to buy things like that operating to parameters so the tools we are going to use during this session are Scala C and Java pink the Java B command allows us to disassemble class file and look at the structure and signatures and the Scala C command allows us to see the different phases of code goes through in the compilation phase by the Scala compiler so if you ever felt dissatisfied or angry at the Scala compiler we shouldn't be because it does a lot of work for us combine your code it actually does 25 things for us if we would run Scala C - actual phases will get the list of 25 steps each one has a step name and ID and a small description of what it does those are all the steps our god is being is going to in the compilation Bailon let's quickly go for just a few of them that we agree that that I'm going to be relevant later on number one is the parser spouses are souls into a STIs number four is the type herb it adds type to our code number six super ancestors had super excesses in trade and nested classes number twelve specialized number sixteen lazy balls and so on and so on so on until you get to find the number twenty four which is the JVM where the bytecode is actually being generated so let's see it's more example of two of those phases if we have a small symbol sorry if we have a small simple class plus four with some vowels and a small method called we move and we're gonna try and see the faces is going to up to me compile it so after the first step after the process step we can see a few differences in the compiled in the code that is going through the compilation so first of all you can notice that the class is now inside a package empty as we did not declare package in the original file you can also see that in each that acts like a pursue the constructor has been added the plus operation is signed okay as we Excel expanded to the metal clasp and the for comprehension we saw before for inside of the limo has been transformed and expanded to a met call so it's after phase number one if we would look at the same code after phase number four the diaper we're gonna see that types on our signed notice all the ins and the string bolded that we've been added for example our val is type int and some of the other valves and death in the class we can also see the synthetic methods added to access the value of I J and K then I left in the FK those are just two examples of all the faces of God is being going through until the class file is being created so let's have a look of what is inside a class file and what is the structure of a class let's say we have another class class regular polygon takes one parameter in the constructor number of sides it has one method get perimeter which does a single calculation multiplying the side length by designed this num sites if you would run Java P minus P minus V on the class files will get this output the minus P and minus P flags are used to produce the more verbose output and the - feedback is used to also say all the private methods and fields of the class so it would go over this huge file this is not even the entire phallus is just the beginning we will see that every class file has the same structures on the JVM and has a few sections in it so the first one is the system info which is the metadata about the class itself when it was changed and how many bytes it has and so on another thing is the version it's not very interesting it's just the class version and the format an interesting really interesting reporting part is something called and scariest well as hell is called the constant ball so what is the constant call and what does it contain first of all a bit of background the development of the deferment cycle of C++ applications includes the compilation and linkage stages the development cycle for Java skips and explicit link and stay because linkage happens at runtime so when a method our field is referred to is referred to the JVM searches the actual address of the method of field on the memory by using the runtime constant pool it is a table of all references needed by the linker and it contains entries of different types let's try and explain one of these records okay so let's say how class file has a constant integer let's say about x equals 365 and we would run Java P and we get the output on their left which is X 0 3 0 0 0 160 this is the back on if you want Java people will get more of a human readable form and you'll see this line found 14 equals integer 3 6 5 like trying to explain the structure so the first byte of each entry is a numeric tag indicating the type of the entry they're making bytes provide information about the value of the entry the number of bytes and rules for the interpretation depends on the bite indicating the first byte so in our case the first fight is x 0 3 identifying the entry type in the JVM which is a constant integer this informs the linker that the next four bytes contains the value of the integer so you will guessed by now that the next four bytes 0 0 0 1 16 I'm actually 365 in hexadecimal so there's an example oh how would constant in we represented in the constant poll let's see another example the same function only now with a string so the string will have to pass in tension in the concept of two entries in the constant form the first entry will be type constant shrink and the second entry of type constant utf-8 the entry of type constant utf-8 will contain the actual UTF representation of the string volume the entry type of constant string contains a reference to the constant utf-8 entry you can see the reference just by power 25 type string and there's a reference to the line above with the utf-8 which is the actual value so those are just two simple types int and string that we solve the more complex types we'll have the more and more complex representation in the constant pool but the structure will remain the same it will be the definition of a type the value and a reference to a sub type so if you would continue looking at the file from before this is the next structure of the class file we're gonna see there is a few more a few more sections the class file also contains a field table that contains information about each field defined in the class these are references to the constant pool entries that describe the fields name and type as well as access control flags and other relevant data you can see the control Flags private final and on top of it on the top and another thing that we have in the constant pool is another table a method table represents immense naval presence in the classifier however in addition to name and timing for me from each non-abstract method it also contains the actual bytecode two instructions to be executed by the JDM as well as the data structures used by the method stack frame so this is the actual code that is going to be run by the JVM when we gonna run this file this is the actual instruction sets let's zoom in on the get parameter function that we saw before okay so let's try and see what's going on here we started the camera and the function before does a simple thing it just multiplies the number of sides by the perimeter okay so each instructions start with a one byte code with a one byte opcode identifying the Jaden instructions followed by zero among structures operate to be operated on depending on the format of the specification of the specific instructions these are typically either constant values or references into the constant pool for other parameters Java P helpfully transferred the byte code into you know into a human readable form displaying the offset or position of the first byte of the instruction within the code and also the human name of the function you can see it on the remained and the value of the operand if it exists northern operand that are displayed with pound sign such as pound 23 are once again reference to entries in the console book as we saw before so let's go even further just one line and we can see the actual multiplication is sign ninth time this number of sides and those this is the instruction sets that actually involves the code let's see a demo of what's going on so in this crazy animation there's a few things and go inscribe the instruction center that are being called and what does each one do so the load one pushes the object reference from scott van into the local variable this a sorry denote one pushes the object reference from slot one of the local variable onto the operand stack in this case it's the argument slight length l o0 pushes the object reference and slot zero to the local value to the table over seconds well involve virtual 31 execute the instrumental number of sites invoke virtual pops the top operand of the spec to identify from white classic must call the method once the method returns which results the result is pushed back onto the stack item d2 pops the integer value of the stacks convert into a floating-point format and pushes it back onto the stack the stack now contains the floating-point result on top followed by the value sign make the argument that was passed to the method and finally do move pops these two values from a stack performs floating point in duplication and pushes the result of a confidence and so hopefully the house proud of this dock is over and now we can do some more fun stuff now that we understand the basic flow of what is happening inside of the JVM and the compilation phase of our code and how it is actually being run in how the bytecode is actually being round we can go back and see the cool features of Scala T so forth and check how they're actually implemented on so let's stop in the beginning if we have two classes one in Scala one in Java both doing the same thing and you would run Java p on them we'll see if your differences the static glass is public by default implementing the Scala object in addition to extending the Java object the scallop constructor is created and is public by default the Java main method must be declared static another example is do not take if simple classes power and check out the valve in the constructor so we can now say that from each field in the Scala class currently only the name field a field and it's ghetto method are generated the field is private and final by the method is public if we would replace the valve from before we can borrow they're gonna see small difference the fields of final modifier is job since now it's a farm and you can play with it and the setter method is added for us automatically if any vow of our defined inside the class body then the corresponding private field and accessor methods are created as well and initialized properly upon instance creation so objects we all have objects but how how they implemented on so if we are the class in small people feet with one parameter of a long deal and we try to compile the code what would we actually created we're gonna see that from config Scala to class file we created config and configured all class one of these two what inside is 2 so the config lass is just a decorator for the syntactic config dog class that embeds the sink shonali all he does is basically call on line three in perpetual 18 degrees in the next line the config dollar class actually concedes the logic of the method and has the following the syntactic model dollar sorry the syntactic variable model diamond to each other object access the singular functionality on the top it's a public reference to the single denote the static initializer also known as the class initializer a getter method for the static field home do in this case is just one method if a singleton has more fields more methods will be added as a phone if it's a very if it's a bounce mode setting records will be added as well and it also has a private method config dollar used to initialize model dollar model done modeled on and set its filter different balance so you will think such smooth code produces small guard on the JVM but these two lines actually produces this sometime thing okay next one trace so we have the trade which to method is similar and is not similar it's supposedly very simple once again if we examine what's its what are the class pal that are being generated we'll see two classes similarity and similarity dollar class similarity class is a regular Java interface the interface declaring both methods we saw before in top public option boolean but is similar and the same for is not similar similarity class providing the default implementations when a class implements these traits and call the method is not similar the Scala compiler generates the bytecode instrumentation option involved interface you can see it in my - in the world interface which caused the e similar method which was to call the static method provided by the common e-class we can see the call to a similar reason mine too and by calling reference found 13 and bound 13 is once again you can imagine is a reference to the constant pool where the a similar method is defined we can also say the jump condition at 9 7 n 10 which are implementing the not logic operation so if we have several traits and you want to implement them all what would happen on the JM so let's say we have a small a calculator train with addition division multiplication subtraction operation and you would like to create a class that extends them all then what happened well actually the compiler would create two files from each trace just as we saw before this is the addition operation to add which were looking at and you can see in the same pattern again the addition operation with the public interface in the up in the in the top and actually in addition to our class which holds the logic and this is the same class once it has implemented all the methods and you can see the delegation each time you would like to call one of the traits we are implementing there's a delegation using the static class that is being created down below so let's talk about something interesting if you have a class person and one mental ill and you would like to reference that method is in three ways in a vowel in the VAR a in a death how are each of these reference and constructed and when does and get executed on writing them in the class time and see what's going on so that's out of code but it's quite simple let's first take a higher look at what's going on on the screen so on the top as usual once again we have the constant all with the end fields definitions and we have n m1 m2 on the left and one and two and anyone on the left and we have an m3 and the constructor on the right I'm three is it the constructor so in the constant pool we see the reference to the method M is stored at index 30 right index 30 method ref person M if the constructor code on the right over here we see this this method this method is invoked twice during initialization with the instruction invoke virtual and appearing first at byte offset 11 and then offset 19 the first invocation is followed by the instruction put 22 which assigns the result of this man to the field and one preferred by index panel 20 doing the constant 4 so this is just a delegation we are calling em and assigning it inside of em 1 the second implication is followed by the same pattern this time assigning the value of the value to the field m2 at index power 24 in the Carson pole in other words assigning a method to a variable defining with a vowel or a farm only signs the result of the method to the virus it does not go it every time from the beginning we can also say that m1 and m2 God gathers in order to access the values of this file although all these parameters in case of m2 we can also see a setter method and two yoni is created which behaves just like any other setter and overriding the value in the field and 2 is about meaning will get a center automatically however you think the keyword death gives us a different result rather than fetching the field to return the method m3 also includes the instruction invoke virtual 30 that is each time this method is called it calls em and return and it turns the result of the method so this is quite a simple example but it actually shows us what we already knew already knew this behavior but it's super cool because you can actually see on the bytecode level of how this is being done to make this even interesting the grass is what happens if we add a lazy to the game how our lazy being implemented on top of Scala and what happens in the background so lazy as you think the valuable field ladies on field m4 which is the lazy definition today's assignment of the of the method and we know that m4 is not calculated until it is needed so the special private method M for raising compute is produced to counter the laser value and the field bitmap 0 to track state why do you attract it States well the method M for checks if the field value is 0 indicating that m4 has not been initialized yet in which case M foe is a compute is invoked populating m4 and returning its value the private method also sets the value of bitmap 0 to 1 so then so that the next time M forms cold it will skip invoking the initialization method and instead simply return the value of M form the bytecode Scott produces you is designed to be both thread safe and effective to be thread say the LASIK compute method uses the monitor enter monitor exit pair of instructions basically we are locking the monitor the method remains effectively since the performance overhead of this synchronization only cross on the first read of the laser values another interesting question is how many lazy bodies can one class help so we already know that we need at least one beat to indicate the state of the laser value so if there are no more than 30 to lazy values a single integer field can be and drag them by any bitmask to the operation but if there are more than 30 to lazy values another big mess will be added and so on and so on its arm so the last thing we are going to talk about before lunch is some things okay if you have an object with some basic functions that work on primitives contact and strings some some ISA couple some lumps and so on what's going on with them so if you would look at the concat function again and see what goes on under the hood we can see few things it's a 5 plus extending abstract function to shakes the permanent string string string we're getting two strings and we're becoming a string notice the plus operation on the string is being placed on the string buildin so if you are still using explicitly replacing classes with string buildings you can drop it and the compiler will just do it from him this is a more interesting example what happens when you look at some in function so some in stage two integers and it turns another integer and we would expect we would expect it to extend something abstract function integer integer integer by x6x and besides that it does something different for some reason it exchange something called abstract function two and C III SP well what the heck is that well turns out it's something called specialized implementation generated by a scallop on file on this explain and shortly the NC high speed at the name end the faction returning 1/8 ceiling to the SP and indicating it's a special type specialized type so while specialized type specialized I are officially allows you to generate separate version of generic classes for primitive types the supporting boxing in most cases really improving the performance the only specialized functions in the scholar a function stupid 14-0 the rest of them functions three two functions 22 are not specialized if you dinner sent by now the number of function 0 to indicates how many parameters are taking while what is why we saw this different NCI is P well there is a correlation between the Java type and the scalability in Java primitive and the type character so if we have a function from an int and a double to a boolean it will be specialized to an abstract function to MCS NID just go and check the type int double Gooden and you get the name another thing that if we actually try and become file to of discuss a library we're gonna get huge huge huge list of all the permutations available we can see all the different names Zen DVD and tries to improve performance by advanced generating all implementation you can do with reduce so partially the black functions we only 310 provide two functions less progress and they expect it created positive like function based placeholders how does it actually work so if we have three functions compact the low and a low expand the function takes two parameters and custom we can create a power supply function by just pressing one string into place occurs and the same time Bigelow expanded how does the psychopaths police region well cell phone generates a new class from each actually greatly seen him this is help me help function the first money to solve you this is the above function which passes one string in two place holders notice the object it's an object it's two down in the end and also the name and this is the hello expecting with problem x3 so function is values inside [Music] so actually to my methods one company 100 are we going to false type checking in case moves gene in something really good with ritalin and this is the main hello class we can see offset 20 running the execution of the reverse function is triggered by a call with slight object method just when you invoke the interface and call the biometric fault and from the print perspective this function is just same as they not without its stored in Class B output and again our output is created the only difference is the base function one so specific instructions we also know will be close to assault itself basically just write small objects for a small class files run south sea or choppy get the output and suddenly she was going on and obviously sound of sobbing he may not show you by I really encourage you so yourself it's not a public and transcendent and those are great his lecture which we also Bella online I heard you two amazone and thank you [Applause]