SF Scala: David Yu, Protostuff -- Serialization with a focus on efficiency and flexibility
okay I'm going to talk about the history why I started this so initially um I think Google Google's protocol buffers started in 2008 so at that time also my uh the company I was working for we were working on GWT and Jetty the web server so so basically most of most of uh uh why it exist is because of our of the problems we had so um okay so so when um before we had to handwrite our server side classes and our client side classes and we have to sync them together so after um the project was over I decided that um we need a better tool you know um uh why don't we just use a single code base from a Proto file and then generate the server side and client side classes so that's what I did from the the year after uh I started this project and then I was using it was not really a library yet it was just a code generator for protocol buffers so the goal was because the uh the client was GWT so it's the browser so we uh I had to because the the the default format of protocol buffers is the binary one right so with the generated class I I generate uh the the tool also generated another another utility class to glue to glue the Json parts so the the problem with this approach before is that I also wanted uh another format you know for readability so in order to still use Proto protocol buffers I had to generate another set of static helpers in order to uh to have another format so what I did is i instead I um I instead I just made the implementation and interface so before uh many of us developer uh some developers also filed an issue in uh the Google issue on protoo and they they they were um they requested that they coded input stream class and coded output stream class that's the that's the io classes basically uh to make it non-final uh basically that means that um if it's not final then users can override the implementation and can Implement their own format like Json or XML or yo so but the Google develop ERS uh rejected that idea primar primarily um one they didn't want any incom incompatibilities in the future with their API so when they want to add another method they don't want to break uh the the implementers so it's a it's another overhead for them and secondly for performance reasons um the final modifier on the input and output classes are are uh ads uh op optimizations well the jvm has extra optim optimizations for the final modifier okay so also I wanted uh I implemented uh the library also not only for serialization but also for indexing uh I was using the library as a serializer and also an indexer so I was using it with my uh with a native database so uh what happens is uh during serialization there are hooks uh in in your custom output that you can you can index certain Fields uh depending on your um definition or your configuration so also uh along along the way as I tried to improve the library um I also wanted uh because data in the database uh address is usually if you want to serve it over the client uh on Json format you have to um get the binary data and then you parse the and you construct your object and then using that object you write the uh you write uh Json to the client so for for the library I added another um feature that allows me to read the data binary data and just transfer it on the wire without constructing any objects or any messages so uh so that's the main goal is uh avoid wasting resources to construct messages from binary data only to send them to the network for another format so this is the this is the um implementation it's a pipe so the primary use case is for transcoding so uh it transfers the data from the input to the output while changing the format to what the client expects so apart from that you can also use it um yeah this one Implement a custom output and you can use it as a hook so you can index fields at the time of serialization or you can filter Fields uh during during serialization so this is how you would use the pipe so assuming you got this from the data store your uh binary data and then you construct this uh the link buffer is you can there's a static method that you can say link buffer that allocate default which basically says you allocate 5002 uh bytes as a buffer and then your output stream your your response to the client's request so this is how you construct the pipe um So Pro I util a new pipe so the data is assume we this is a protocol buffers data the binary data and then with this uh if you write it to Json then you just transfer it Json I till that right to out then pipe get pipe spe this one food that get pile schema uh that is generated from the from the Proto definition so yeah protop protop data written as Json without instantiating a message so also another use case that um that I use protos personally is uh I use a level DB database uh it's basically uh key value data store sorted so uh traditionally for jni Java native access uh uh J Java native interface uh usually you just glue uh for a single method in a a native a C++ method you implement uh Java counterpart uh one to one call so what I did was um Implement a a micro protocol where you uh a single request would uh so basically for a single call the the you have to create a C++ method that streams the results so instead of calling if if you're visiting uh uh key value uh by uh by row instead of uh doing that jni per uh per call you can call it once and maybe after a thousand calls depending on the buffer you you stream it back uh to Java and then you process that you reindex you you add indexes and then after the after reading the full buffer again until the stream ends you process it so it's very efficient uh in terms of if you want to avoid the overhead of jni java native axess so extras um over the years people have um requested that they use Proto Stu uh with uh Java simple objects like uh plain Java objects existing objects and also they have also requested that it would handle um circular references so by along I I kept delaying it and until there's so many users requesting that form uh feature then I implemented protos soft graph with um usually it's used with prosoft runtime so you can use this um if you have a big cache and it's and the and the objects are circular then you can use this uh as a binary format so it's very efficient it's it's uh on the benchmarks third party benchmarks is uh I've seen uh with the um with the comparatively to other sterilizers that handle circular references um the protos graph is the fastest so that's largely in part of the efficiency of the protobuf format so nowadays um who's familiar with um flat buffers flat buffers yeah so the advantages of flat buffers is that you don't have to to to read a message you don't have to parse it sequentially you uh basically you can read any parts of the data and I I've been using flat buers also I've been trying to integrate it with protop so uh as I've used thought buers I I realized that I can also do this with protuff if you if you stick to the fixed fixed uh Fields like the bull field the fixed types like the bull fix 32 fix 64 float and double so how uh for important fields that I frequently access that I don't need to deserialize the message uh I use this technique so for for a sample message you have a string and then you have a and which is variable data and this is uh fixed and this is also fixed so with with my generator with my custom generator uh I generate this um V Active is uh vo basically means value offset and then that's for the fields so update T is the is uh 64bit and um 8 bit uh active the bullion is eight bits so one and 10 so the reasoning why one is we count backwards and then and then the offset you start at the offset so to to get the to get that data um directly and this is how you do it so you have a static method get bull so this is the value of set the vo the generated the generated uh Fields so in order to get the bullan field without distalizing the whole message uh you do this so so it's all relative from the back of the message so the same applies to the other to the other field types you just have uh for example the other one was 64bit this one so what you have to do is starting this Offset you parse it uh the whole 64bit and turn it into a long field is okay so for the production users that directly informed me on jiub um one is the jet brains upsource um they're using they're mostly using the Proto protoc compiler for their um front end code which is basically I think they're using GWT and and I guess one of the reasons is I was also using GWT and I I put some sample code on the on the go uh Google code project and it had already a base uh the data transfer objects for GWT was already there so what they did they enhanced it for their use case and also play Tech is an online gaming bet platform that uh they they're using basically the compiler and the serialization library so they're using the pro protos graph format for their I think this for their cash and also for their data transfer objects from their client side and server side okay that's it thank you guys uh if you have any questions question so uh I think I heard it somewhere that Proto Stu is more efficient than Proto c generat yeah right you show The Benchmark and so the reason was that at some point uh there is a better algorithm and I was a bit surprised that there is something non linear in prot C and is like can you talk a bit more about it okay so basically with Proto off um what they do is if uh before seral serializing a message they uh they they have they want to determine the size the exact size of the message so in order to do that you have to Traverse example if there's a root message and then there's a embedded message field so what they do is they uh parse the uh Traverse the whole object and then for the fields the inner message field they also Traverse that and then count append append the size of the child message into the root message and then determine the size after determining the size they then do the serialization so it's basically you Traverse the object graph twice in order to serialize with Proto stuff because uh mostly what I did is the the the computation of the size and the serialization is all done in one step so in order to do that um I had to add an extra field uh where you when you serialize you also update a size so for every field that you serialize you update the size so but the problem with that one also is that you have to have a buffer uh that is mostly close to the size of your message an estimate so if if if the buffer you allocated earlier is smaller then that's what uh that's what the link buffer is for so there is uh it uh the link buffer is basically um a single link list so a reference uh it has a field that points to the next one so so when you serial uh when you serialize that into a link buffer if there is a over uh the the the size is bigger then the next message will point to the next buffer so mostly most of the time if you if you allocate your buffers that are large enough there isn't much overhead so that's when you gain the advantage and even if there isn't uh there is uh overage uh you just it's just a copy so it's basically a trade-off so you create a dynamic object later on if if the buffer doesn't fit so you allocate memory extra memory but you exchange that in terms for Speed yeah you do the serialization and computation once okay questions so why should we should everybody use this instead of propy um my yeah this one actually uh there is for me there's nothing wrong with protocol buffers firstly it was designed specifically for that use case buffering so um I think where Google uses it is and when they transfer their network over the network so they want they want to uh for uh they have filters uh in inside their RPC server so they they want to basically filter parts of the message and also with the pro Proto format um the advantage is if you have the size uh if it's um if the embedded message has a size inside then you can easily skip it with the native protop format um you have to for the inner messages you have to really parse it so that's the disadvantage so for their use case they want to skip parts of the message and go directly to the important parts so it works perfectly for their use case so yeah so there's nothing wrong with uh using protocol buffers and I uh I would advise you use Proto stuff the native format not the Proto format for if you want to um serialize uh for mostly for serialization not really uh you don't want to um skip parts of the message or yeah you want to read the message at home could you show The Benchmark uh The Benchmark is actually online I can show you links okay while that's loading okay still loading oh bad format okay so this is the graph so it's actually the fastest if you yeah this is the native format so cry you and one Comm plus make it oh okay command plus six oh not the Mac you a plus okay okay so anyone familiar with cryo yeah so it's actually a really good Library also and uh I think in version one of PR it was a lot slower than protos then he figured out he was using basically um bite buffers so in his API he had bite buffers and all along I knew that if you use bite buffers directly it is a lot slower than serializing with a native bite aray object so in his version two he changed the his library and turned it into a in bite uh bite array directly so that's why it's very competitive now so yeah is this something new that uh it's I think we can see the last update of the it's been here for a while I think come on minus I think mostly this is also because I don't do any marketing for this uh I think this is mostly how people find out about Proto stuff ah I see November 25 that was the and then there's another one with a let me see so this is the only plain data no cyclic references so for this is the breakdown for serialization protoo is the fastest and theer realization still the same I protuff it's not here so size this is the one good thing about the others it's uh more compact this just the creation time it's not it's not really that important and then let's see I want to see where the one with Proto buff here so this is the one with protuff in it the pro format so mostly if you look at the Gap it's yeah the overhead of serializing twice shows a lot in this Benchmark you have to Traverse twice so with uh some of you are probably wondering why with a Proto format it's still very fast with my implementation so I I still uh I actually do the same I count it twice but there's a catch um so this this format is very efficient for small messages so for messages uh I mean small nested messages the you know the the fields uh the message fields that are small in size so if it's 127 the size is 127 and smaller then the overhead is not there's no overhead basically because it fits in a single bite so what I do because for the messages it's Dynamic right so for a nested message uh it could have a very big string field so what what proo that the official protu does is in order to be sure of the size it computes the size of the nested message before it allocates the whole buffer for my uh my strategy is I assume it that the SM message is small so if it fits the small message then there's no overhead it fits one bite the delimeter the size DM so if not what I do is um I reuse the buffer but I alloc uh basically if the buffer is this one and then I use part the half part and then instead of using a contigous array of buffers uh I skip I skip this one bite and then proceed so it's basically the your your the the message in the buffer is no longer linear because you're skipping in the middle one bite so in order to send it to the war efficiently or if you don't want the overhead of one bite you have to copy it into another message and remove the one bite so that's just that's the overhead and also also the wrapper for the so is for the buffer to create this View you have to create uh another link buffer just to link between the two and skip the middle so there's the overhead of one bite and then the overhead of creating the wrappers which is the link buffer just to connect so for small nested messages the phob buff format my implementation is very good yeah that's the tradeoff now is this only is that no the B AR bite buffers like n bite buffers yeah does that only apply to things like Nest messages or can this does this also have is there also overhead with linear buffers like in my my use cases in this case matri matri yeah everything everything with the bite buffers is slow compared to using bite array directly what about okay what about what about net because I know they do some they do some stuff with unsafe and all this other yeah yeah so NTI uh I I've heard lately they have they also have a native B unsafe yeah they use Direct that's what I use right now yeah so with the unsafe you have a little bit of speed Advantage versus the bite buffer yeah you're correct so if you use the unsafe only for certain Fields like the float double fix uh long and int but if you have string Fields then it's a problem because you have the to double caping so yeah that yeah that makes sense that's the a lot of over the wi that's about it yeah yeah so for numerical Fields the the unsafe is very fast I guess I'll do one more question though so what about what about communication with j and I so I also do a lot with plus and Java like inter a like what what have you found as what have you found as the fastest there like still think it's going to be unsafe because you can directly like you know cast a pointer and then just get a direct reference yeah there's yeah there's actually a hack I found um there was this there's this Chinese developer who's developing against engine X you're familiar with engine X the the it's a web server see so he's integrating it with the jvm so what I saw in this code is usually if you uh interface with jni and you use uh primitive AR critical I think yep so you have to release it after you use it right so but there is a way around that depending on the garbage collector that you use if you use a linear garbage collector basically a single threaded one so you can skip doing that and there's a in there's an un unsafe Library where you can access directly the real offset of the bite array okay so you have a bite array and then depending on the jfm the the offset is pretty constant so if this is the address of the bite array plus six usually from my test is always six so when you pass that to the uh to the native interface to uh to see so you don't have to do primitive AR critical anymore you just get that offset and then and then once you grab the bite array Offset you just uh add it and then you have the real array so I what is yeah so this is the name of the project I I actually have this implemented also on my project but it's not yet open source so uh clour yeah engine exclosure yeah this is the guy guy who who found the hack and I've been using the same since and I've been benchmarking it and it's very sign significant because when you do a release primitive array critical yeah there's some sort of locking or but there's a cleanup overhead so if you skip that there's and especially if you have a single threaded uh process for data processing is very fast because no more locking so B essentially when you're Crossing from java to C there's no locks the only overhead left is the is the um um jvm trying to convert the object to uh yeah yeah they serialize the Java object to Native one yeah yeah I I still have to find that um actually I yeah I didn't see I think I can find it here it's hard upet let's try search b I think I remember this nope I mean I'll find it it's not yeah it's okay I just need I I just want to know yeah unsafe it's here oops not a very good Ma you sir okay get that so I think this is the one yeah I saw that yeah I saw the address in the offset yeah this is the offset this is the static offset so the r is I think a b array yeah the the offset of the B yeah the yeah the B bite array offset the one with the relative offset and this is the real you append this uh the static offset to that and on the C side so you get a pointer right essentially A J byte array it's basically a a wrapper oh no it's a pointer so so you add the pointer pointer arithmetic with the offset and then that's the real address and you can use it okay so to wrap up basically uh with um Proto stuff it's it's good for if you use it as a serialization library mainly and if you're if you're also if you don't have because with schema uh with protuff uh schema evolution is built in right so if you want to remove a sample in the future you have a message field and then you want to remove that field you delete that field the problem uh with protuff is very efficient because the size is embedded in the nested message so you can directly skip it just uh just add the offset to the size and then you can proceed to the next message with Proto stuff the N native Proto Stu format you have to scan the entire contents of the message in order to skip it so that's basically it if you don't have if you're not deprecating a lot of nested message Fields then you won't feel any overhead so it's a tradeoff thanks