Scale By The Bay 2018: Kuassi Mensah, Reactive Java Programming: a new Asynchronous Database...
you okay good morning thank you for coming so I'm glad to be here I was here last year I'm glad to be back so this time I'm presenting a DBA a synchronous database access API this presentation has been put together by myself and our JDBC architected a-class server the class cannot be here today so I'm presenting the Oracle usual disclaimer I'll go through a quick intro and then walk you through the API using code you know it's gonna be a lot of code so be prepared to see code and the code will explain the API okay so you will understand and then we will wrap up okay so a DBA this is a new Java standard proposal so we are proposing this to become a Java standard because we believe that when the adoption goes first and you know if you have it as a Chavez Turner people will adopt it okay so this is developer the Java JDBC expert group we won't be hope it's gonna make into a new future release okay so why are we doing that well because I think has better throughput okay but it's not just async as we will see later we are also addressing the reactive stream approach okay so these are the goals no user thread blocks so the user Fred something and returns immediately you don't block and then the implementation or you can say the driver will take care of completing the operation asynchronously you know so the user Fred will return and you can go do something else and some other thread will take care of completing the operation this is an alternate alternative API it's not a replacement for JDBC okay we target high throughput and reactive application entirely built on the Java SE library there is no third-party library everything you will see here are based on Java standard mechanism either in Java SC 8 or Java SE 9 okay well we can support third party reactive stream database access API for example okay so why it's not that okay later there was a had a slide on r2 dbcp anybody has heard about this reactive whatever database access by pivotal it's r2 DBC so r2 DBC can work on top of a DBA anyway so design choice is no reference to Java dot Seco Java dot C called askgaryvee see so rigorous use of types what does it means it means if you look at JDBC JDBC statement that execute takes a string okay now you look at prepare statement in JDBC which is in here which is an extension of statement you cannot have prepare statement dot execute with a string it will throw an exception so it's not a rigorous use of ties that's what we mean build a pattern I think everybody familiar with that fluent API everybody's familiar with that objects are immutable after initialization nor reuse no changing one way to do something is good enough we don't try to come up with different ways to do to something no sequel processing by the driver we will see one slide where you can see that different database vendors have different markers parameter etc so the driver will not deal with those thing we just pushed the secret statement as is from the user code a go avoid hot callback hell I mean people familiar with JavaScript have heard about this they have workaround but this a peer has been designed to avoid this phenomenon okay so what about streams reactive streams well so a DBA uses Java util concurrent flow where appropriate it's not a hundred percent full freight reactive stream API you know if you're looking for that you need to look at our 2d BC which is on the next slide so what is our 2d BC it is a reactive relational database connectivity this is something proposed by pivotal ability it's a no get up you can see the links there and get her so these adhere to the reactive manifestos non-blocking bad fresher okay dependency on reactive library so remember what I said a DBA has no dependency on third party library whereas this has a dependency on reactor library which is a different reactive library from the one you would get from link built for example it--but so that's a problem anyway how did DBC out to DBC has a wrapper on a DBA so they can use a DBA under the cover okay and we can work with them to make sure that they can use a DBA if they want to okay now I'm gonna go through code to explain what is the API and this is the most interesting part of the presentation okay this is a very trivial insert so we have an insert method it has an argument in argument it has a session and I and item is sort of a POJO so session is sort of connection where it's not a real database connection it's just a session there is no circuit establishment to the database it's a session for now okay so we have a session then we have a row count operation in a DBA there is no statement prepared statement colorable statement like JDBC know there are row count operation if the operation returns a count or they are raw operation if the operation returns a rose you know series of rows so you will see row operation row count operation a row publish' operation etcetera cetera so these are based on the return of the operation so here in this example we have an insert statement insert into a table with three columns three different attributes and those attributes we will use them to build the item which is the POJO so we will set the item with the ID with the name of the answer and then we submit so that's all the user friend has to do prepare the operation submit and return a DBA is heavily based on completion stage okay this is a very large interface if the implementation is a completable future so it's a very big and I cannot cover it here if you want more details you can look at this but this is a standard Java mechanism and to do event based mechanism an event based programming it suppose lambda expression fluent programming composition of tasks etc so this is a very good tool for a DBA execution model everything in a DBA is an operation an operation is everything which deals with the database so it could be a sequel or some other operation that attached to the database that's an operation ok results and handling submission completion stages each receipt or so user thread creates and submits operations that's that's it you submit the operational you return ok the implementation now execute those operations asynchronously so you disconnect you know the submission from the execution another example get data source ok get the results it is already provided by the API you know so what you need to do is get data source which is a function we're gonna get a new factory in this case the factory will get the data source for the Oracle database well you can imagine same thing for Postgres MySQL etc ok so we get the builder we get the URL we set the value for the URL for the username for the password and then we build the data source but you don't have to use it you know we act we have convenient method that you can use for example get session this is the implementation of get session take a bit of the build and then attach so attach translates into and I'm showing you the code of a DBA this is the code of the implementation so it translates into submit an operation group I will explain later what is an operation group then attach operation and submit attach operation attaches the session to the datasource so when you get a session that's what happens under the cover and you return ok another example IDs for answer with datasource as a parameter and it will return a list of integer which is the result ok so we have a try we've resource here try with resource brach so we get a session and then on the session we invoked a row operation so row operation is an operation will returns rows okay so the row operation will execute the Select statement select ID name answer from a table well answer equal a value so we will set the value for answer which is the bind variable and you can see that every time we specify the type the type of the the parameter okay so how do you process results you collect them right so here we can I use the collect method this is a convenience method provided by the implementation but under the cover it translates into the collector you know the usual Java collector so here we give him a supplier and then we give in the accumulator and an accumulation function so what's happening here is gonna build a lid we gonna build a list with an integer which is at the column ID okay so you stuff the the IDS into the list and so we prepare everything we submit when the execution when the implementation execute the operation that's what's gonna happen under the cover is synchronously it's gonna execute the seeker statement and stuff the result into the list and return a list of integers so one thing I need to mention here is the type witness we use this a lot this is important but if you're using a framework the framework will generate the code for you it we don't think that people should be coding a DBA API directly I mean skilled people will do it but we think if framework should be doing that this is the close operation so a closed operation is never skip and a closed operation as you can see it's it calls closed operation and then submit so it never blocks nothing blocks you know the whole API is built in a way that no operation blood nothing everything returns immediately to the user code sequel so all sekolah vendor-specific we do not try to come up with an abstract seeker like in JDBC and we don't deal with parameter markers we don't deal we have all those code all sort of things because they add overhead to the implementation and increased complexity okay so for example at the bottom of this slide you can see the different parameter markers used by different database vendors you know like db2 use column MySQL question mark date Oracle database same as db2 posterize etcetera cetera so we don't want to deal with those things at the API level at the implementation level so we just push down the sequel as receive from the user code of course if you are using a specific database the implementation of a DBA will deal with that specific parameter marker for example if you are using a DBA with Oracle we cannot deal with the Oracle parameter markers we're not going to try to deal with all sort of parameter Marcus okay so this is a little bit more complex query operation so here we have a public method named items for answer which will return a completion stage which is a list of item so I don't suppose you okay so we have a data source as a parameter and we have an answer as integer okay so try with resource Asian okay and we're gonna return a list of item and we're gonna press we're gonna perform a row operation the row operation will process the Select ID name answer from tab etc you know as the same example we use previously and we see we set the value for the parameter target and then we have a collector and we're gonna collect so this is the convenience method which uses the Java collectors and we are mapping so this is a mapping function which means we're gonna apply for each row we're gonna build a new item by getting the ID the name and the answer from the row from the row we recoup divide different values and then we build the item okay once we build the item we stop this into a list we invoke another collector to produce a list of items so that's what will be returned so the user code specify all those the submit and then get a completion stage okay so submit returns two things cancel and get completion stage so this is how we can get a completion stage when the operation is available everybody's here familiar with completion stage computable future right it's a software box which will carry the result when the execution has happened and then it will trigger the execution of other completion stage which are attached to it transaction this is more complicated okay so transaction we have a transaction and any data source try we've resource block Carrie's Asian if we cannot get accession week if gary session does not complete successfully we're gonna return an error okay assume everything goes well so here is a problem remember this API you submit and you return the execution happens later in the future so how would you make the determination of the outcome of a transaction you don't know what's going to happen to a transaction and by the way in distance we will have to operation we will have a select operation and we will have an update operation in the next slide okay so it is a transaction made of two operation but we don't know what's gonna be the end result and we cannot anticipate now you cannot code to it because you don't know what's gonna happen well during the execution so we have to make a provision for a mechanism that we're calling transaction completion so that's what the trance is trance here is a transaction compression mechanism sorry okay so this is a transaction completion and it's a flag and we gonna set the flag depending on the outcome of the different execution of different operation that made up the transaction so here so the first thing we have compression stage a compression session integral ID promise is a completion stage it's an integer by this a compression stage which means its value will be set in the future okay so it's a we got a session we do a row operation which is the Select statement select ain't no in name from employee we're in an equal a parameter for update select for update okay we send a value for the bind variable and then we cannot collect so in this collector we invoke collector so collecting and then it's gonna do the mapping okay so we do the map create a list and then get the first value of the list so we cannot just get the first integer of the raw sequence okay so that's what all this collects stuff does okay so on error set the flag to rollback only if this operation is not successful we cannot set the flag the transaction completion we cannot set the flag to rollback okay and then we submit we get a completion stage and we move on to the next operation which here is a rowcount operation rock-out operation because update returns account okay so we have an update we set the value for the the the bind variables we set the value 50 for the first one and the second one the value of the second bind variable is ID promise which is a completion stage which means the value will be set when the previous operation completes okay I think that that's easy to understand so if the count you know we apply gate count if the count is different from one then there is a problem so we say wrong number of rows otherwise we just return the count if there is an error during the execution of these we set the transaction to rollback the completion to rollback and then we submit so that's what the user Fred does and submit here this is the place where we cashed the error of both execution of both operations okay so if any of those operation was not successful we're gonna read we gonna set rollback or may commit maybe rollback you know depending on the value of trance it's gonna be either commit or rollback this is how you can only work with transaction multiple operations happening in the future you have no control you cannot go back and change the outcome but you know you cannot send a con me or rollback once you submit the operation you have no control no access okay operation group operation group set of operation Wow we only have five minutes okay I need to rush okay it surprised a group of operation you know it's a handle as a completion stage so you have members and members can be submitted in see app can be processed in sequence and in parallel can be conditional and we have an error handling as well okay I don't have time to go through the operation but please look at the slide and you can understand the different thing but it's the same approach you submit you get a completion stage etc cetera okay okay this is where we we support flow we support reactive trips you know we do support concurrent flow for those not familiar this is how it works so you can implement the subscriber interface which means you will implement unsubscribe on next an on error and uncomplete okay so you request a number of rows and the the the a DBA will give you the number of rows so the idea is we give you what you can observe what you can process so this is a usually known technique known you know bad fresh non blocking back pressure okay so we will use this to support what we're calling a row publish operation so here you implement the interface I'm going quickly because we don't have time anyway we're using this flow API to provide you a reactive stream support so for processing rows same thing for controlling the submission rate if you're submitting session you know you submit you operation you need to make sure the implementation can sustain the rate of submission you have are you t submitting things you need to control that so we give you something we calling request hook so you need to implement the subscribe interface and look at here request hook this is how you tell the publisher that you can sustain or you cannot so it's gonna slow down last controlling session creation rate same mechanism you have to implement the SUBSCRIBE interface and etc cetera okay I need to wrap up because we short of time we have the API is here so you can go and look at the API it's on this slide I have published my slide i sent i'm the slide so you can download the slides you can have an implementation of the api over JDBC and play with so your code will compile and will run but you will not have the asynchronous behavior because this is over JDBC this is a DBA over JDBC it's a functional implementation so that you can play with the code but we really want everybody here to get on the discussion forum and see what people are saying about this API and contribute because it's a community thing if you don't contribute you know you can say later you know it was not okay I have two minutes I can take one question anybody we have two minutes yes oh I sent to the the event they will post yeah I sent I'm despise everything here is on this slide no more questions no question okay I hope you like it thank you for coming and enjoy the rest of the event [Applause]