sfspark.org: Alexy Khrabrov interviews Dmitriy Setrakyan (GridGain)
Recording: sfspark.org: Alexy Khrabrov interviews Dmitriy Setrakyan (GridGain)
[Music] hello everybody I'm Alexi kov uh the organizer of SF spark here on location at splice machine and the main talk tonight will be uh about uh aach ignite and G gain and we have with us uhan who is a chief prot officer uh of gri gain also PMC of Papik Knight and he's also co-founder of G gain great it's great to have you with us thank you thank you great tell us a little bit uh how did you come up with this idea of gri gain it's uh it's pretty interesting technology what it is you know how it came to be where it is now well uh Nikita Nikita my co-founder like the guy confounded with and I we had the idea we always uh were part of distributed computing and we always were working in all those projects that required big scale and lots lots of fance and scalability so essentially when we just started I mean the market was not as populated and saturated if you will as it it is today at that time we had Globus we had Sun engine Sun GD engine and probably a couple more and um we just uh and it was hard it was hard to use grid Computing actually to spin off a project on lobus probably would take you maybe several days M so Nikita and I we had this idea of grid Computing Made Simple and that's when we started grid gain as an open source project and the whole idea was that you you should be able to start Computing on a clustered environment in a full hor fashion within 5 minutes what year was that I'm not going to give you it's probably 7even eight years ago okay so that's before spark yeah but uh yeah that's before spark and we started out in a garage kind of mode and uh we're working uh after after working hours and then at certain point we land at a Consulting project which we used for sponsoring a team in Russia that helped us develop grid game so that's how grid game came out to to be it started out as a computer grid mhm and uh it actually uh became pretty popular from the GetGo and as we progress we figured that jazz computer is not going to be enough so we added the data site MH and then we had a few changes in the company and then we donated our code to Apache ignite and here where Co so so you take GRE Computing and so you mentioned sub systems I think this is not kind of the traditional distributed systems kind of in the uh area right so can you talk a little bit about grid Computing area as kind of preced in uh Hadoop and what what is specific what are the customers of this uh you know what is mostly used well U from the uh generally speaking I mean Hadoop and grid gain are designed to solve different types of problems Hadoop initially uh was created to take uh it's a batch oriented system so there's it's very batchy and uh essentially if you have a problem that takes an hour maybe a couple hours and you use map Ru you may be able to complete it in maybe 30 30 minutes or so so it actually provides good acceleration for those batch oriented offline systems that need to Crunch on a lot of data our Focus was to take a job that maybe takes a few seconds and execute it in milliseconds MH so it's a real time from the GetGo we designed it as a real time very responsive system M uh that uh uh was meant to perform and was meant to scale so that's where would be the difference so and all the use cases that I'm seeing today in Apache ignite there is a lot of use cases that are that are transactional and that require those hundreds of thousands of uh transactions per second MH not to say that we're not doing the analytical part but there's a lot of a lot of that idea I'm seeing today when I'm seeing you our users R transactions under huge loads so I I I I think I've read uh agre used on Wall Street and by for trading uh so this is not typical application of kind of traditional open source we see around directly right so how you know how does that work you know how these customers you know use it if you can talk a little bit about it so yeah if you mention Wall Street then let's talk about Wall Street a little bit so what is important in financials right I mean Financial uh if you think about financial companies they move money yes uh they don't like mistakes they don't like down times uh so for them uh to move to a new system would be a huge decision and they and that's why they make moves very rarely and and that's why I mean they a lot of times when you walk into a large Wall Street firm you'll see a traditional two three tier architecture where there's one database maybe with some replication that gets uh loaded but it's transactional and it performs under high Lo and generally the main goal the main um goal there to achieve when they start migrating to an in memory architecture is performance and scale so at certain point buying more Hardware stops working for them and uh from skillability standpoint and probably from uh economics uh from um simply it just gets too pricey and uh the there comes two features in aide that actually differentiated from uh the rest of the products on the market and one is full asset compliance we're not eventually consistent we're fully consistent so you can take an oral transaction or my transaction and uh run it an ignite or G gain and not only we will execute it uh in with full asset guarantees uh regardless of whatever failures happen on the cluster we will not we will actually make your database part of the transaction as well so if you still want to write to database uh then database transaction will become part of transaction and another side is SQL so because all these companies are so database Centric uh they the main language they talk used to talk to their data is SQL so our support for SQL with strong indexing so we care about performance wi index a lot makes it a lucrative solution stre interesting this actually very interesting because uh SP machine itself is actually doing rgbm which works on app and LP so uh I just talk about M as been the C so this is very I think like what I realized when we're hosting you know uh g game up at splice machine actually we fin a lot of synergies so I think you know what would be interesting tonight uh is kind of uh I think we'll kind of these topics will be raised right and maybe kind of questions kind of can come up which will not come up by itself but basically have two different companies probably addressing two different markets but kind of talking about you know SQL asset right and you know Essen going after Oracle from different angles from different from different performance angles different use case angles yes yeah yeah yeah yeah this exciting so tell me how so I think Apachi U ignite started a year ago so not or two years ago kind of putting to a test with this dates not too not too long I think uh year over a year so over year ago so uh there are two parts right there's incubation and then there is graduation yes so if you start from when we donated uh the code to Apache it was 2 years ago yep and and we graduated within 9 months so we graduated pretty fast and uh the reason for that was the very rapidly growing Community oh so it's uh we're I I mean if you put spark outside I think we're one of the fastest uh growing projects in Apache and uh uh the community is very responsive the community is very willing to uh engage and uh the stuff we're working on is very interesting I me it's all a lot of distributed stuff distributed locks transactions distributed data consistency so all these problems uh create like this unique environment where you can get one of experience that you wouldn't get anywhere else and that attracts a lot of commuters and a lot of contributors to the project so we've been growing ever since cool yeah that was actually my question and how how it affected you guys but how did it affect the company right so basically you know you have dual roles uh does it mean you spend more time in the open source or does it kind of like do you have a lot of synergies so it kind of feeds the business how do you kind of connect this to the two two things well I mean one first and foremost I mean it's very important that as a grid gain as a the all the features we provide on top of a Patrick Knight so we first of all the difference between grid G and Patrick Knight is g game provides Enterprise certain Enterprise level features like data center application production upgrade or rolling upgrades uh security so things that you would need in serious Enterprise deployment but you probably could be get by in simpler open source type of project deployments MH and the uh the main uh goal that we we're trying to pursue here is that you should not be worried when you write a code with when you code to a pach you should not be wor which Edition whether it's a patch ignite or GD G Enterprise Edition use so GD gain actually does not introduce any new API most of the uh uh most of the functionality that grid game adds is a configuration okay so you want security it's a configuration you want uh rolling production upgrades it's a flack you enable on a configuration you want uh data center application again it's configuration if you want to get some metrics specific to Data Center application then yeah there's some API there but generally if you use just plain vanilla patch that you can continue using it in G cool cool cool so this is this is really exciting uh we're looking forward to your technical talk and thanks you know thanks very much for coming all right thank you good to be here [Music]