SBTB 2023: PANEL - Streaming.

SBTB 2023: PANEL - Streaming.

Recording: SBTB 2023: PANEL - Streaming.

[Music] really hope that you enjoyed the first day of scale by the way in person back after [Applause] years I'm here uh with you I'm o if you don't know one of Co organizers Alexi is the founder and I'm co-organizer also I do uh devil and Community for B and this panel is going to be about this treatment technology and we uh experts in the their opinions on the technology where it's going what to expect and more so honorable ladies and Gentlemen please come to the [Applause] stage okay so um I want to warm up a little bit with the short introductions so let's just be boring and go from left to right is this on yeah um so I know Oli well because I uh am the founder of bite wax and so I work with Oli on a regular basis um yeah I live uh little south and west of the Bay Area in Santa Cruz um I I talked about bwax in my talk but it's a stateful stream processor so I'm here to talk about streaming data because it's something I think about on a regular basis and uh I don't know what I else I was supposed to say about that's me I love it I'm Tim Bergland um I work for a company called star tree we make a fully managed cloud service based on Apache Pino which is a database that doesn't make a lot of sense outside of a world of streaming and what else about me I live in Mountain View with my wife and my stepdaughter three kids three grandchildren um I have hobbies we can talk about that later hello hey everyone my name is Jerry I currently work at a company called Data bricks hope you know many people have heard of that um I currently work on structure streaming where spark streaming at data bricks and that is what my team is responsible for is developing maintaining and supporting structur streaming as far as streaming both to internal customers but also also to external customers as well and I've been kind of working on streaming kind of my whole entire career if people have been around in this space worked in a lot of an open source Apache storm back in the day you know aache Heron and then moved over to Pub sub some on aache pulsar and you know now SP Park I live in Santa Clara been in the Bay area for a while love to play tennis so my name is Mary greski I'm a developer Advocate at data Stacks so if you are familiar with Cassandra then that's the company that we are the commercial arm I guess of Cassandra um but actually I joined um data STX last year um first I was with the streaming team so I worked on aache Pulsar there um and then uh about four months ago company has kind of this manded everybody is going to work on geni so anyway so that's what I'm working on right now and in fact my talk here I just did today was about boosting llms with event streaming and uh rag pattern so that's my talk there and uh I'm also active with the Java Community I'm also a Java champion and also president of the Chicago Java users group so very honored to be here thank you hi I'm not holding caral who could not make it here today I'm wamper I also live in Chicago and uh was involved in streaming a lot in my days at lightbend which is the scholar company many of you know uh currently I'm at IBM research working on their Watson X generative AI platform and their AI strategy for open source uh so combined this people have uh years decades of expertise in streaming and I want to key of the discussion was the question why is streaming happening is it a fat or a response to real endurable Trend like if the latter what are the trends and uh Zender you have to start this GNA go around and around um so the reason I got into um stream processing was because of machine learning uh so the reason we built bite wax was to help solve a problem in real-time machine learning and that problem was that it was very difficult to do the same transformations to the data that you did to train the model which happened in an offline fashion to then do those online if you wanted to do your inference in an online fashion and so if I was to speak about some trends that would lead to us suggesting that maybe it is a durable thing um I would say that uh my experience would suggest that as there's an inrease in machine learning then there will be an increasing um uh stream processing uh to solve some of that problem so that's I'll say that's where I come from so uh I'll I'll leave it on to the next I don't know how we're g to we're going to continue to do the little no we don't have to actually if someone wants to step in uh Jerry sure um so give a little context about my journey and streaming right um I started my um journey and working on stream processing grad school this was many years ago right my adviser told me Jerry go do this thing that was I didn't know anything about but he said it was the next hot thing right and that was kind of the story for a long time is oh streaming is the next hot thing but I think you know now we've arrived at streaming is the thing right like people are starting to really look into it and we see that at data bricks um how much like usage we've have on SP on structure streaming so I think one of the reasons why is people one is I think education right people Now understand what streaming is and the value it brings that there is an opportunity cost for waiting too long right people for a longest time in big big data right they're like doing batch processing once a day once whatever like longer duration of time that was that was good right that was moving from a situation where that was not even possible but now as we've you know de technologies have developed as people understand technology better they say wow if we can process this data quicker we can make decisions quick and we can take advantage of situations quicker and generate more value so I think nowadays people realize that a lot better than before so that's why there's like you know definitely a rise in the popularity of streaming I I like the question because there's no question that streaming is happening the way you ask the question o um but sometimes these things we experience these things in software and they don't endure right they really are just a thing that everybody rushes to blockchain I don't want to you know it see it seems like this is the wrong crowd for scholar jokes but you you could do that but you know there's we we rush to these things because we are preoccupied with trying to discover what's next and get some advantage and how to build things and you know could it be that that is streaming now I don't think so that would be moderately hypocritical of me if I I thought that but I I like the question because just the fact that we're all interested and there's conferences and you you go to the Expo floor and everybody has to say streaming on their thing you know that doesn't mean anything uh so what I look for are things outside of us um you know why is it that we don't allow four to six weeks for delivery so that's a thing that TV commercials used to say um that that's not you don't submit a request and get something back a long time ago what's changing in the world that would make right now so important because there isn't a question that anybody here who builds things getting an answer right now is what you have to do um it can't be tomorrow anymore and why is that uh so I this it always feels like a Cheesy answer but honestly I think smartphones are are a driver of that that they're not a new thing anymore you know you've had one in your pocket for a good 10 15 years um and that that that transition that I don't know that we really adequately noticed from I go to a computer to use the internet to it's just there and it Wiggles when the state of the world happens it it it gets my attention and more and more of my commercial and social life is mediated through that device and it's always on it's always there well you know those of us who are building the systems that that drive that activity uh now is when things need to happen we don't get to wait so I I I think streaming is real and I think streaming is a response to that could I add on to that but I do want to give the other panelists yeah so I think completely agree with what Tim said here like obviously the you know um the the rise of iot right like phones and whatnot but I think um to add on to that um I think especially working in an open source and working at some companies that adopted open source like Yahoo you know and whatnot back in the day a Big Driver of streaming Tech technology and also Twitter right um is originally like in the early 2010s was actually ads right a lot of a lot of a lot of these internet companies why are they developing or leveraging streaming Technologies was to drive real time ad campaigns and real-time analytics on those ads so it was very important for them to develop these Technologies and that gave like a boom essentially especially in developing streaming Technologies especially streaming Technologies in open source so that's all I wanted to add as well I'll add one more advantage to streaming and that is that you you basically amortized the processing load over a smaller chunks like a typical Hado scenario especially on premise was you have this cluster sitting around all day and then suddenly you spin up a big job you know a couple every few hours to process data you don't need to do that if you're processing it as as it arrives and I I think for me I would think more from consumer or users perspective too we like there are also cases in which in today's world right there's fraud detection system for example you do need like techniques that comes from event streaming streaming to help it and you can't wait you know those things you know somebody something suspicious you want to take action right away so that kind of Demands and you know kind of it it becomes like a mandatory thing we need to actually have applications that have to be able to respond quickly and there's no time loss and I think that's you know it's no longer a fat we do need streaming Technologies how I'm looking at it so okay then uh I want to zoom out a little bit on this because uh many of you mentioned insights that insights that we can get from the streaming data uh but we are all technologists and uh one of the way to look at the streaming data is also the connectivity between different uh pieces of your uh design architecture so my question is more like um okay so we got the inside part but is it the better way to connect your microservices or whatever pieces you have yeah I think um that's a good question and actually that's the hard part a lot of cases it's just too difficult to hook things up and keep them reliable so I think there's plenty of room for innovation in the you know the people who are really working hard on the Pino and Kafka and all of that so I do hope that it becomes so easy that people stop you know questioning whether they should do it it just works that's really what we need to get to Fair Point like we're early days it it there isn't there's a there's a pipe and you can put things in it you'll get things out of it there's it's all feels very bespoke it's like you're going to have to you know cut the threads on the pipe and and all those things it's the the pieces are not welldeveloped um but it seems like that I mean my my answer to your question would be yes because and and again it takes time to validate these things but we've been micros servicing for 13 years you know and it it it seemed like it was a terrible bloody exercise at first that was just failure and and and then there was kind of a middle period of like all right we can do this synchronously and you know kind of build a bunch of scaffolding around synchronous services and the the new asynchronous event-driven thing it seems like it's working right like we need another five years to to to tell uh but it seems like it's actually and that that by the way that's the other driver that I think of I think of mobile and I think of the way we choose to build applications now which is services and events are a good way to connect them um I think it's working like let's find out but seems good so I think there's a couple of like hard problems in this area like sure one is always building connectors right but that's always been like the case for a while is you develop some sort of new framework build connectors it could be you know microservices or whatnot but I think one of the things I've seen more recently is when you have basically you know chains of microservices right that work for whatever purpose right microservice a if Fe to microservice feeds to microservice you know whatever right what if there's like a failure in between how do you recover you know from that situation how do you recover consistently from that situation that's one of the harder problems is when you have chains of microservices it almost resembl stream processing to some degree right how do you effectively and consistently recover you know from failures from you know between and still deliver a correct result at the end right or deliver correct results to the end user and number two is a lot of it is also observability is though there's a number of Frameworks out there I think to help observability your micro services but how well how widely is that adopted is how do you like actually monitor the performance and the health of all your microservices that's surprising to me in a lot of even large organizations that's almost still very rudimentary to some extent it feels like we are uh just barely at the point in the development of this stuff where it might be a little more good than harm because like you said those those things are just not obvious I mean there are answers I was listening to a podcast episode about temporal this morning which is a a thing that answers the the first question and there's like there's all kinds of observability things but they're they're not it's not obvious and and we don't have the one baked kind of way to do it um and I I think of this analogy of uh the development of medicine you know there was some time I I think because I'm a historian and people come to me for my opinion no they don't um but I think um somewhere in the middle of the 19th century or towards the end of the 19th century like the practice of medicine came to be probably more good than harm like there was some inflection point you were like it was better than even money to go see a doctor before that you know um and we're we're we're maybe at that same point with some of these things where um you know sagas there's just no real consensus way to do that are there ways to get it done yes but gee that's a thing we just don't know yet we haven't been doing it for very long and and maybe we're on the cusp of that this is actually slightly more good than not and which sounds terribly pessimistic I mean I'm completely optim this is as I like to say there are people who are right now somewhere learning how to eat food because they're little babies uh who will be using the paradigms that we're talking about professionally as software developers in 2025 years I we're on to something real here but it's it's early but there is certainly delay between um medicine start working and people believe in it uh yes yes so let unpack a little bit on the challenges that we meet with the streaming and um um first one already mentioned is that the um streaming seems to be disrupting the application stack so um if before it was clear use the web framework use database everything is clear now it's like uh C of different microservices connected by cavka and then you use some sort of streaming which streaming we don't have an answer we have gazillion of different Frameworks now we have to choose and um your company does this why nobody knows everyone forgot uh or it's just simply because you have people who are expert in uh some Frameworks you just keep using it and going but at the same time so many things are happening and I think is um streaming is one of the turbulences where like a lot of things happening at the same time um so my question would be do you see that it'll be stabilized somehow in a few years say five or fewer so oh okay one no I meant to say that I feel right this whole field it's like we going around in circle I mean not just in streaming if you look into all of the things in Computing it seems like you know we go from single thread to multi- thread or it's back to oh it's actually not too good and it goes back and forth I feel that maybe to everything there's like a balance point and maybe we're trying to reach that and what it is maybe we're in the Journey of discovering it right now so the same thing maybe for streaming too we're talking more about it it should be like this and should be like that use Kafka use Pula use this and that and then now there are people actually using there's another net right they are very simple net uh for also doing messaging too so that's kind of completely kind of like a barebone kind of like thing so I feel like well are we going back to you know kind of something simpler to use I I don't know it this is actually a very interesting thing and I do like how you raise the question up and applies to everything too so yeah Tim's the historian so he can probably tell us that human nature to repeat itself um I I think it's really interesting to think about um what you were just talking about with respect to microservices as well and and the way we grab on fads and Trends as a an industry specifically if we're to think about this conference and large language models and generative Ai and I talk about this effect of like the boardroom trickle down and so like someone in the boardroom is like we got to do something about generative AI and then it slowly trickles down to whoever has to implement it and so maybe the reason we're at microservice is like oh hey uh some Fang company said that like this Google or something like that I don't know said that this is a really good pattern we should all follow it and then the boardroom trickle down effect happens and then we all have microsurfaces irrespective of if we want it or not and at some point that can happen with streaming too and I think like other people have sort of sort of alluded to this but I so what all what I'm trying to say is like I don't necessarily think think that we make the best decisions for ourselves all the time and sometimes there are um things that we can't totally influence and so it's really hard to like strong man or Steel Man or whatever be strong against the the forces that you know essentially feed your family or whatever it might be but I don't know I just wanted to add that in when we talk about like uh are we making the right decisions is this a fat and all that and there's sometimes it's so complex and there are so many things that we don't control on these sort of things so so I'd like to offer maybe a slightly contrarian opinion contrarian is a strong word but we we have to remember that microservices are a solution to a problem they're usually about scale out or you know team boundaries and all this kind of stuff but it's kind of bad that we actually can't use modeli let's be honest it would make saying so much simpler we could just do function calls um so I like to think about that every now and then to make sure I don't like just you know drink the Kool-Aid too much and really understand why I'm using stuff and one one of the revelations for me so I worked for any scale for a while the company behind Ray and what a lot of people don't realize is when you actually use Ray behind the scenes it's scaling out your work as tasks over a cluster so you can kind of go back to just having like one entry point per microservice or service instead of like five because you have to scale out and just let Ray do a lot of it so I'm hoping that things like that will actually kind of force us to rethink well maybe if I can scale out completely my One mic M service over a cluster I don't have to do streaming between instances of that microservice for whatever reason and that makes makes things simpler and and simplicity is like the most important thing we can do in this business to you know keep our sanity so I'm not trying to trash anything of course that we do because they're all great Solutions but it's really good sometimes to step back and think how can I make this simple and what tools do I have at my disposal for that I don't think it's that contrarian actually we don't all need microservices I hope there's nobody up here who thinks you should never build a monolith I mean I I think you should default to that and we just get it beaten out of us I said it was a strong word yeah okay so what Dean said about scalin actually uh bring us to the another challenge so streaming in real time can be very um can eat a lot of resources it's really not a cheap thing so you have to think twice if you need it at at all in your opinion how companies are solving this problem right now well I I I don't necessarily think that to be honest I think it's always going to be a trade-off for companies whether what what they're going to trade off cost versus latency right or throughput versus latency it's always a trade-off between that Spectrum right is how many dedicated resources you're willing to commit to run this specific thing thus how low of a lency you can get it right for example in systems like spark right it is easier to trade off uh cost and latency right because imagine you have a streaming query you have many stages you can say I only want to run one stage at a time and my cluster is always provisioned to run one stage at a time right versus I will run all of the stages at one time and have a larger cluster dedicated for running it right if you run all the stages at once you may be able to get lower latency but you have to commit more resources versus if you have a smaller cluster and I can have only lower concurrency I have to run fewer things at a time you would incur larger latency so it really it's at the end of the day like coming back to the requirements of what you want for streaming right streaming does not necessarily mean you have to process something in a millisecond right it could be you just want to your your SLA is 10 minutes an hour it's not really defined here it's really defined by the use case of how low Lancy or how you know High Lany your requirement is and based off of that you know what are the cost implications so I think like there are definitely ways for users to mitigate costs and a lot of users especially using structure streaming at data bricks ly is actually not many for a majority of them ly is not actually like a very high concern a lot of it is cost especially on the ingestion path is how little money can I spend doing this oh come on can't be the only one could could could there be selection bias in that in not seeing customers who are concerned about latency sure yeah um they're all using Flink instead or is that you meant I they might they might if if that's the thing they're optimizing for they might because I and I the I think you made a strong point about uh we only have trade-offs to make I me we're fundamentally we're Engineers that's what we do and pretty much anybody only gets to make tradeoffs you don't really get to solve most problems you just push the piles around and um that you made that point strongly but latency not mattering to me sort of brings it out of the world of streaming and I we don't have a robust definition of what streaming means but informally I always I always try to Define it as um I have a thing and I'm going to do something with it right now I'm not going to put it somewhere and come back to it later um to do something with now if it's Kafka you know I'm putting it somewhere and I can still go back and do something with it later but you know my pipeline is I have an event I do something rather than you know I put it in a bucket somewhere and I go back and run a process on it later that that would be batch so there is no implied SLA in just saying streaming but I feel like definitionally there is the implication that I mean to get to this right away oh yeah I definitely agree that if it's framing latency is definitely going to be a dimension it's just a matter of how high or low is that right right we sure you can say selection bias oh Flink oh maybe they do have the best lency but that's also relative right like that's relative in the world of maybe you know soft real time or in in the realm of in the milliseconds I've also worked in finance where we're literally running the stuff trading on fpj and we're dealing with nobody's using any of this stuff because we're all dinosaurs there sure right um like that's the thing everything is kind of relative right but there I think there are there are more and more use cases that would require like you know Lan seas and the milliseconds and we see that data bricks and we do we are actively improving lency and structure streaming to basically handle those use cases but the thing is even not at data bricks at my you know previous life at you know working at a startup working at Yahoo the real is like many people especially coming from like Hadoop and they're like oh if you can give us minute latencies like that is already good enough that is like we are happy with that so that's why like it's hard sometimes to justify engineering work right it's like oh we have to build this thing to deliver that latency but then people ask well where are the use cases and right I so I think that's super interesting question which uh sometimes gets asked of me because I'm the guy who's like no 15 milliseconds is too long you know um that's an exaggeration but um when do we need real time uh introducing that term into the conversation and it's probably not for business analysts looking at dashboards in most cases it just doesn't a minute is great I mean yesterday is too long that that's that's from the 990s but man 10 minutes ago is was pretty much okay for for most Executives business anals making decisions you don't need those things to be faster so I was I took a question after my talk or I think it was after my talk yeah if you if you can make an analytics database a thousand times faster it is it is not valuable for that dashboard to refresh 10 times a second or 50 times a second and you also don't need to make a thousand times more of them you don't have this pent up demand for dashboards that you're suddenly going to satisfy now that they're a thousand times cheaper makes no sense but even if you can I would ask like is it worth the cost because if it's not generating the value of refreshing it more often then what is the point right exactly so if if the question is uh what does a business analyst need or what do what does that dashboard user need they have it that's a really Wells solved problem and and we're we're in the territory there where we were 10 years ago building monoliths where there were still choices I had to pick my language I had to pick which relational database I wanted to use I had to pick my web framework you know I felt like I was defining myself with all these choices but really filling in a form um lots of different ways to do a dashboard these days but really we know what those choices are and they all work and it's good that it's a solved problem that you know Innovation continues I'm not I'm not this is all bread and butter stuff that's really important but the case for real time does not consist in dashboards like if if that's all you do then you just don't need to go that fast we're kind of we're kind of at the speed we need to go for those so yeah so if I may say so too it almost seems like latency really depends on your particular kind of use case and when Tim kind of mentioned too about places actually some shops don't even use any Frameworks and they develop their own and because I actually work at the Chicago Mercantile Exchange before and the board options exchange they actually or at the time they did not actually use any framework when I said oh what do you use it we write our own and see and then it's much faster then if I use another Library cuz I just don't want any other layers in there I just want to write right there and go into the metal and it does the work for me right there just like the trading stuff you the trading yeah they they actually do their own too and they don't want to use any other libraries at all so so so yeah so I think it all depends but to me though like as I'm thinking more and more you know we tend to kind of talk more about serving the business but how about you know in my mind I'm thinking you know the world has so many problems can we solve some of these social problems with streaming I think that might be actually an interesting thing to think of maybe trafficking right if we can kind of find out you know how can we track you know somebody who's a I don't know pedop pedophile or something things like that right if we can find ways I think that would actually be even a a better thing for us to do for Humanity to do to solve the humanity problems yeah so anyway just a thought that I have too I think that gets to an interesting point it's still too hard for people who just want to solve a problem but don't have the technical background so what I hope we get to is templates recipes queries to chat GPT I don't know but things that actually help us set up not only just like balance that latency and throughput you know knobs and cost knobs but also all the air handling that's required what's what are the semantics that should be invoked in Failure there these are all really the hard problems that ultimately prove the most difficult I'd like to see them solved in a standardized way in some sense no you asked I want to see if it's your next question no no there's a that I was it's just do you have do you have another okay do you have more questions uh yes I do I'm going to let you ask them if it doesn't come up I'll I'll I'll well just uh I wanted to add a um comment to um I did a little research preparing for a talk about the latency how important or not important it is and uh unfortunately people are very impatient and uh even um a second of a difference um in many cases would make a huge difference in conversions so many businesses would say that yeah we still need that haven't said that God can I just cuz that's what I wanted to talk because you you you just moved who the consumer of the computation was because I was I was saying real time doesn't matter and I was hoping somebody's going to ask me well then why do you talk about it Tim but nobody did so but for for the the dashboard consumer it doesn't freaking matter it's not not actually valuable to spend extra money on that um all you were just talking about a user of an application a person on a a smartphone or web browser um and now we're in the game is different yeah because you're doing real time analytics yes and I am notoriously impatient when it comes to stuff like that it's not a good thing I'm not bragging about that I should grow in that area um but and and I'm also you know we're all very synchronous in our use of the software that we interact with like you touch something and you wait for it to change and you're not going to wait for long and so uh I think real time matters when you begin to interact with humans who now have this instant expectation going back to when smartphones entered the world um we just expect to inter to interact with responsive software on devices that are always connected we can get answers all the time and and more and more of our life is lived through those devices my Tik Tock recommendations now please yes yeah I want to know how long that hamburger is going to take uh to get to me it's funny I'm kind of hungry right now so you know anyway um okay so we talked a little bit about different problems and um at some point this uh thought struck me and now I forgot why I thought that but actually I think there is a little correlation maybe not little between how established your company is and how big is your legacy code base and how fast you're going to pick up on the uh streaming as a technology do you agree as panelist and what challenges do you see there is that mind set or actual like technology learning curve or something wait can I unpack that question is the question if you're a established company with a legacy code base is it that you will not adopt streaming Technologies is that or did I misunderstand the question well let's put it this way it's okay I don't know I I if that is the question I I'm not I don't I think it's very nu my question was that I think that it is more challenging for legacy big Legacy systems and established businesses to do so uh than like something that appeared just a few years ago for example and I want to um see what's like what ways to improve panelists see you included but I I think in that in the way that the question is framed is that the incentives are not aligned the same way for both those compan companes when you're an incumbent your incentives are aligned to drisk so that your stock price doesn't go down or you don't churn the most recent customer if you're a challenger company or not an incumbent you're trying to leverage whatever capabilities you can IE technology or IE real time to have an advantage over over that incumbent I mean I don't know taxi cab companies versus Uber or whatever so I think that maybe it could boil down to incentives instead of just the code base I don't know that would be my answer well I guess we do have some tools at our disposal even Legacy companies with a lot of spaghetti Cod often figure out ways to slice it and you know introduce abstraction so they might spin up a team of you know the young hot shots to do the streaming stuff that they have to have to survive like for customer engagement or or or C customer happiness but then they'll often figure out ways to shove that data into the main frame or whatever database and Legacy batch system they have so I think a lot of companies will figure out how to do it and they'll leverage some of the things that we you know talked about and know how to do which is create abstraction boundaries so I think like at a legacy company right which you know I've worked in finance and there's plenty of Legacy things sprinkled all around in a finance company it's always hard to adopt new Technologies right um that's I think regardless of if it's a finance company or not like there's always some risk there right that this thing will not will fail or you know it would not do as well or corre incorrect results compared to whatever you have right there's always a risk of going to do the new thing but I think a lot of time what people realize is is I think it's too important things one is maintaining the existing system is just too costly the people that know this thing has long left no one knows exactly what the code base is what things do and it's just running and no one really knows you know how to maintain it right if something goes down if something fails well like tough luck we're kind of like all screwed here right so that's one thing is maintenance and basically maintenance in the future I think that's a big factor I think leadership and a lot of companies take into account is what going in the future like well we have this legac thing going on how feasible is it for us to maintain it right um number two is once they kind of I think see the potential benefits in using adopting some of the new the the the new Frameworks right I think one of them could be maintenance but there could be other things that these incentives right just like he said like will outweigh the cost of these risks right that looking into the future this is the future Direction um everyone is going and we should definitely invest in it and not be like left behind here right that the incentives I can basically make decisions faster or do whatever and I can get you know these very quantifiable benefits right once like customers realize that and I think education is definitely a piece here like once C once users realize that it's a lot easier decision to like say okay now let's like you know move over to this new technology 100% um when I was a confluent uh sales and marketing called this application modernization which meant there's a monolith and we're going to rebuild it as event driven micros servic basically is just what have their their language for that so yes people do that um other side of the coin is um Mary and I never overlapped at data Stacks but I was I was there a few years before uh the guy who was the CEO at the time Billy Bosworth was a young developer during the the dawning of the relational database and client server era in the late 80s and early 90s um you know it's I was there Gandalf I was there 3,000 years ago um Le he was I was doing different things but um he talked about kind of the wild-eyed Fanatics advocating for that new way of Building Systems and they definitely were I was in the late 80s I was a nerdy teenager writing reading bite magazine and reading about this client server thing like it was this massive social transformation the kind of breathless tones that people wrote about it with I didn't understand at the time but those Advocates were pushing for this new way of building things and they didn't just set fire to the main frames it wasn't a revolution where you know you go to the barricades and overthrow the whatever um the main frame's there it's fine it's probably still running the code it was in 1987 uh you know when all that debate was happening sometimes you build a new thing with the new thing so yes 100% you refactor Legacy systems application modernization is a thing and a lower risk path for adopting a new technology is here's this new functionality this new system new business that we as an incumbent player are entering and so let's do that with streaming and realtime analytics and blah blah blah um so two two different ways of making it happen and if you're an incumbent player they don't they don't always just cocoon and die you know they they they build things small R&D projects that never see the light of day no yeah right right yeah I was just thinking about the entire architecture patterns uh for solving problems like Lambda and you know how to bring that okay so um I want to move a little bit into the future and um I want you to share some new exciting promising or nerdy streaming technology that you're following and while following I'll go I'll go first here um so we talked about cost tradeoffs we talked about latency um and we've talked about new technologies and we've talking about we've spoken about um incumbents and Etc and some of us here have worked on confluent and Kafka so a new um startup and a new way way to think about uh streaming that I think is really interesting is from warp stream and they use um object storage and you have to pay a penalty for latency but I think what we've all spoken about a little bit here is there's this trade off between ease of use and what people actually expect and who's actually expecting things and do we all need the latency of the bank Etc and so you pay this penalty of latency but you can have a much lower cost and a much more simple architecture you're not wor just you know with discs you have a very like decoupled architecture so warp stream is my plug um outside of bite wax which is another great technology yall should check out but so actually I think for me one of the interesting things right now is we train these generative AI models in a batch way and we can't train them often because it's so expensive so you were talking today Xander about and in fact there were several talks about like the retrieval augment a generation pattern where you keep like the recent data in some sort of database that you can look up and it's associated with your query I'm going to be very interested to see how that evolves because it's it's very much a kind of a hacky solution now it's kind of slamming together streaming in a sense you what's the latest Twitter feed or whatever versus this Legacy model that's growing older by the day and I think figuring out that balance and trying to keep models up to date and not have to use rag as much I think will be an interesting trend well I was going to say you mentioned about pre-training I think I think that would be really breakthrough is if we can actually do the pre-training right of llms if we can do it like cheaper and faster and but I don't know how right because right now it's huge amounts of data it just has to go through that kind of painful process of pre-train and it takes so long takes so many gpus still it would take days or weeks and so I think you know and I was trying to ask that question but nobody talks about it too I think it's because the cost is just super you know monstrous cost to do that but I kind of have often thought if there's a good way of doing pre-training really fast like you said it's a badge processing too and and it just you just have to go through that but it's I don't know right now I can't think of what way um I don't know if anybody has been thinking about it can can it be done processor wise something to make it faster can it still be done faster or maybe some of you have kind of thought about it so I think one of the benefits of stream processing right over like traditional batch processing is that stream processing typically it comes with some sort of management of state right that you're able to incrementalization right you're able to process one part of the data at a time and you're going to be able to basically memorize kind of the progress you've made and basically continue right right and what that really gives you is allows you to only process the difference right the change the changes right so that's a I'm not an llm expert but at least in terms of like what I think is a very interesting thing um that I believe in is um materialized views and using stream processing to power materialized views it's not necessarily A New Concept per se but I think it's you know a very powerful concept that can allow users to that allows users to use stream processing in a more userfriendly fashion people don't have to go and think about oh you know event time watermarks lay data doesn't really matter I have a materialized view I want to basically refresh it this is my refresh interval I want to commit this amount of dollars go figure it out right um and basically using stream processing to incrementalization reduce the cost for your materialized views and that's something at data breakes were working on as well which is I think very powerful and very like easy for users to reason about because like materialized views that's something like people understand right not necessarily like if you know anything about databases SQL like materialized views I get that right I get that I can cor query my materialized view I can get the result right and I can use stream processing to incrementalization see as um you know exciting things is not necessarily something that's broadly like oh the hot thing but like one is you know using basically familiar quering Concepts and somehow applying a streaming you know spin to it and also just making it simple right making it user friendly a lot of running streaming is one it's I don't understand the concepts I don't know how to run it you know in production right so at the end of the day if you can just say say oh I have a materialized view I want to update it in real time and all the autoscaling running you know cost equation is all solved for you that becomes very compelling to the end user to like say oh it's super easy for me to adopt this kind of thing did wait did you say I I have this this is my budget I want you to like solve this materialized query using my budget is that what you said yeah I want to be able to say what is the freshness of my materialized view right that's super interesting actually it would be really interesting to see the the data like how much are people willing to spend for various different use cases or various different materialized views to get what they want is like fundamentally half of the data problem is like hey I want this data in this shape to be correct and be in this place at that time and like how much are you willing to pay to get it there so it'd be really interesting to see the data I don't know if is that the thing that you're doing at data breaks be um yeah so it's related there's another there's another product called Delta live tables that basically allows you to do that as well I mean at the end of the day look like when you talk to any of our customers at like a at a senior leadership level the bottom line is how much is this going to cost us like none of the other things are details for engineers to figure out but like is what is the dollar amount here like any exciting technology for UT okay um it's a tough one I'm I'm not I'm not very good at the future I mean there's there's the stuff I work with I I'm excited about that but the the you know what's the what's the far out I was going to say AI as a joke but then Dean and Mary actually had thoughtful things to say about AI so you know that didn't work out um yeah to you know to some degree the the stuff I work with with penino that is a very forward looking thing because you know the practice of user facing analytics is not yet EXA exactly ubiquitous um and that's mostly what I focus on okay so um I want to ask my last question for today and then we gonna um move to the audience if they have questions um so the main theme of this conference is uh the code and data in the age of AI and we'll look at it holistically uh we want to learn uh how to build AI systems how to leverage a AI system on your like on everyday basis if you can use co-pilot or something that your work please tell us uh and uh here at the streaming panel I wanted to ask in your opinion does streaming matter in the age of AI and if so where does it fit and how it helps I'll go first I guess um we can go in that the order or near the order we've been going in um so earlier I I spoke a little bit about uh why I worked on bwax and uh online machine learning and um I use this term like uh operational transforms with respect to um real-time data processing uh to avoid talking about uh analytical Transformations uh which would be more suitable for Pino or something like that um and the reason I talk about those is in the age of AI we can do so operational Transformations are a part of automation so when can you embed logic in the application so it can have machines talking to each other in the age of AI you could imagine that that increases potentially and I think that stream processing is a really important part of that because you have to understand the context of what is happening in many different streams of data essentially so that you can make a appropriate um decision in real time and so all of this is to extrapolate that more and more automation because of AI will lead to more and more uh will necessitate more and more um stream processing and and streaming uh platforms so that we can understand what's happening in real time and have things react to it appropriately like we're humans right now we're taking in so many inputs and we're making decisions in real time and so if we were to take that and make it into a machine making those decisions you know we have tons of ability to take in input and they have to be able to take in similar types of inputs in streams and perform some understanding of what's going on so they can react to it so I would I would say yes tldr yes are we going in order um I yeah I don't want to disappoint anybody but my answer is also yes so I'm going to attempt to summarize what's going on in Ai and and uh I could be absolutely wrong so so disagree with me audience whatever but like in the last 10 or 15 years we've had this steady march to being really good at classifiers um deep deep neural networks have just made really good classifiers and then in the last year all of a sudden generative AI is a thing uh that's you know the kind of chat GPT in the public discourse is about a year old now uh and and that's a game Cher for all of us I mean I don't in my role write code all the time but when I do I'm kind of in chat GPT asking questions you know uh it's it's great and I'm learning how to use mid journey and just have fun with that and these are these are real things um that I think are much more limited than we might like to talk about all the time but impactful tools um but the current obsession is driven by generative AI um and and that doesn't mean anything for how my services talk to each other or for how I respond to events in real time and and get results to users of an applic that's just that's we're that's Plumbing you're still we're still plumbers and that work continues unabated maybe you've got co-pilot and that paint in the butt API you don't have to go look for an example that's not even there in the docs it'll just give you something or chat GPT will game Cher okay not to be discounted but the actual conduct of our jobs if you're you know a typical say Enterprise software developer is the same which means streaming still matters matters for all the reasons the streaming mattered 5 minutes before everybody knew what chat GPT was so I think this is an interesting question from my perspective machine learning AI right sure there's newer innovations that can help you know stream processing and vice versa but the fate of those two has always been intertwined from the beginning like especially when I look at back in the day when I worked at you know Yahoo when it was still its own company right um those days right um when all the founders were still there right um so it's always been Big Data what one of the big drivers of the development of Big Data platforms was machine learning was because there was so much data that we want to do some sort of machine learning on there we developed these distributed systems to process that right and a subset of that with stream processing or distributed stream processing so when I look at the use cases especially back in the day like a lot of back in the day we were using Apache storm right um a lot of the use cases was online machine learning and if anyone knows like actually Yahoo was an early kind of Pioneer in online uh machine learning there was a open source project called on Apache Samoa back in the day it was a whole Library built on top of storm and later ported to other like streaming platforms to do online machine learning so like it was always the fate of these like platforms was always like interwined to some degree like it's been that for I think you know a while so like I I think it's obviously still going to you know continue the reality is for any of these machine learning you know um algorithms to work it requires a lot of data and the infrastructure right to process that data so those things will always be hand inand yeah so so I look at it it's true because we are actually living in a world we depend on the data so the data is really the thing and and then we have different ways of getting the data travel here and there and going you know kind of morphing it whatever so I think streaming definitely will be there and I also look at it like Tim said it's like Plumbing we're just plumbers how do we get it moved from one place to the point A to point B that is always what we're trying to solve if you kind of think of boiling down to some barebone thing we're trying to move data from one point to the other is we're constantly doing that and so streaming is the technique so I think it will always be there it will and it's definitely true it has um kind of you know kind of before you know when we're doing things synchronous and now streaming enables it us to do things more in an a synchronous fashion so I think it will continue to stay it just won't like go away and maybe we have new ways of handling it but we're still dealing with the data like you said like J Jeremy right talk about the yeah the data we're still dealing with it and streaming is there restful thing is there but I think the key is to try to figure out what we're trying to solve and then apply the right techniques use the right tools for the right job I think that's what we want to try to do understand our problem clearly and then apply the right jobs the right techniques and tools to it is what I'm thinking so so I think we've all seen these uh like futuristic police shows where some guy starts talking to an interface and he drills into this data like you know cameras looking over a whole city and suddenly narrowed it down and he's actually interacting with this thing that has this superhuman knowledge of what's going on and then maybe runs a query by talking to the system like or tell me a little bit more about that guy that you just identified in the camera so I think it's absolutely correct that streaming is going to have a massive impact here for all the reasons that we talked about in in talks today about the rag pattern everybody wants the latest data it's a pain that open AI is stuck in 2021 or whatever so I think it's going to drive a lot of evolution about how we hybridize these kind of very different Technologies because people will want to be able to do that stuff we just invented like the best query language we've ever had we can actually you know somebody who doesn't know anything about SQL can actually query a database now I mean might get the right answer depending on how much it hallucinates but you know it shows what's possible I think all right thank you that was really interesting discussion and I want to continue with the uh questions from the uh audience so during this entire discussion the thing that kept coming to my mind is whether or not you know they are very small number of use cases for streaming and the reason for this is because for me it feels like Overkill most of the time unless you're doing real time lots of data like people like Twitter because otherwise it's just event driven architectures you got a bunch of events uh worst case scenario is that you do quanze events of time and you deal with events as they come so serverless type of architecture trying to do streaming is mixing you know is is over complicated the issue over complicating issue what do you think of that or do you see event driven architecture and streaming style architectures as being completely different well I definitely think there's the like event Ren architectures is here to stay and that's what we want you don't want to wait you don't want one microservice to wait five minutes for a reply but just on the case of data if you think about it even batch processing is actually like a finite stream because whether it's spark reading a file you know one row at a time or uh even it's pretty much anything we use so there has been a lot of discussion about really they're kind of two sides of the same coin in a sense and whether you find a streaming architecture for even your batch analytics is the right model kind of depends on your sophistication and the tools you have or conversely you run a batch job every minute and that's sort of streaming but it's but you've got your latency budget met so I really do think you have to decide to the points that have been made what is my latency budget what is my cost budget and what's the easiest way to get there I think whe whether you call it streaming or not and complexity of the software so it might be we talked about monoliths early earlier uh it might be that that's the right architecture and that all of this event- driven stuff is actually Overkill so there's a scale a very legitimate scale at which build a dang monolith and and and if you don't need these things don't do them I mean if like you said a function call is literally Hardware resources in the processor to make that work well you know instead of going over the network why would you go over the network if you didn't really have to so if you can build something in a monolith it runs it scales you can deploy it you can test it you can fit it into your head yes every day there are times you can't but I'm I I would say that that's the right thing to do I would say like really there's no difference I mean are I I think the distinction here is what you're saying is are you using a stream engine right are you using one of like the platforms to do stream processing versus I just wrote a jar and it processes event by event right in reality those are just implementation details like this is just streaming right you're processing live data you know as it comes in so it really there's to me there's no you know there's no kind of Distinction there it's just a matter of what you're using underneath so you don't need a stre datab yeah for it depends on your it depends on your depends on your use case right if you're just say moving data from point A to point B what is the point right like if that's the streamy use case or you're doing some sort of you know trivial operation right arguably sure if you if that's all you're doing right maybe it's just worthwhile to just develop your own custom application to read data in and pipe data out right so it's really depend on kind of what you what the application is what potential applications you're going to develop in the future right what's if this is just a standalone use case or do you think oh in the future I'll build you know a hundred of these use cases and I don't want to write a custom thing every single time I want to have one single framework I can declare all of these Transformations on and just run it right like these are kind of things you have to think about but at the end of the day I think that's just you know streaming whether you develop your own thing or you develop or use another you know platform and us usually know when your bacon is being saved by a tool and if your bacon is not being saved by it then it's not get new bacon keep what's that get new bacon get new well or that you don't need that tool you know if if it doesn't appear as a useful thing probably isn't um and then to people that it does appear to be useful it is just depends what you're doing um yeah right here so uh if you of you guys talked about the trade-off between uh cost and latency uh the um authors of apachi beam said the trade-off is actually three dimensions is cost latency and correctness because in a world where events May you know arrive late or out of order or never arrive uh right we don't know whether we're materializing correct results and so you know there's there's streaming as kind of a execution Paradigm right but there's also uh at the application Level right if we're dealing with streams of event in the real world that streaming whether we process it in in batch or using a stream engine um so I'm just curious to you know hear a perspective on the panel on that yeah we we talk about like uh I mean the question before was like why do I have to take on all this complexity or like is is it should I even do it it's too complex and I love how we force people to ask the question of like how correct do you need to be like oh sure you can do stream processing but how correct do you really need to be like how long do you want to wait until you're correct it's like really fundamentally kind of weird thing to think about and that we haven't crossed the chasm so to speak I don't think with a lot of streaming Technologies because we make people a have to ask and answer a lot of those questions which you just fundamentally don't want to have to deal with you're like why would I try and think about how correct I need to be when I can I can wait five minutes and process the batch or whatever so and use batch tools that are easy easier and so I think coming back to the very first question and maybe I'm not even answering your question I'm just ranting now we we haven't really crossed the chasm because it's still so freaking complex and hard because there are hard problems we're trying to deal with like was mentioned in this panel and we expose those to the users like everywhere they're like oh you want it to be eventually consistent that what you know or oh sorry your whole your whole entire system isn't item potent so now you're going to have to mess around with your guarantees and it's like there are so many hard questions you still have to answer that you shouldn't have to answer in your Day-Day life so why the hell should you answer them so I mean I don't know if that answers your question but I think the whole thing I'm trying to say is this just too damn hard still like why do you have to have that third dimension you're got to answer the question to it should just just be cost and complexity someone else deal with the rest hey I paid for you to solve this problem I didn't pay for you to tell me look if you really need your answer to be right you got to wait for a minute and a half today so and and I I was thinking uh exactly about the opposite like if you wait for a minute then your answer might not be the correct one anymore so it's still yeah and uh I don't know the stock went down and now you're about to lose your money CU you clicked too late so um yeah cor I have a question um so um going by the crossing the chasm thing no so um if you want to reach wider audience um so what are the uh so I want to hear more about the user experience uh with respect to streaming and the future of user experience so today is is little bit more Dev developer focused experience right so if you want to like really simplify this for uh High LEL uh thinkers like uh who are thinking about use cases rather than the technology in itself um and want to leverage streaming so what is a progress being made towards the user experience and how do we think we should evolve so that people don't have to think there is streaming underneath right and yet be able to get real time information uh from the systems so yeah I I want to hear more about the advancements in the user experience yeah so I think I talked a little bit about this is using basically existing Concepts right people maybe know from SQL that may that are just powered by streaming right one of the things I think that you know I mentioned is materialized views right people understand that um traditionally materialized views in a database you recompute across a whole data set right um that you know once data comes in you re you recompute basically the new data with all the old data and that's very costly traditionally right but with stream processing you can incrementalization right of course there's Al although you know um things that you know we are also you know exploring at data bricks like oh can we use like llms just go from natural language to um actually you know some sort of query right by themselves that could also obviously help right um so I I do think it is important um but those are kind of some of the things I'm I'm looking at and as well as not necessarily I think the defining what you want to query but also the operational side of things is traditionally that's always been very hard to run a streaming query you know in a production setting but if you were you know given basically some sort of mechanism to Fire and forget that it will basically scale to whatever load and it would be within kind of your slas that would make it very like you know usable to the end user that they don't have to worry about the nitty gray details of operations impacting user experien is like precisely the thing that I'm trying to help people trying to help developers who are building these systems do more easily so uh examples LinkedIn that's that's where Pino was born you can't load a LinkedIn page without lots of things being impacted by decisions that Pino is making uh meal delivery globally for whatever reason it's this vertical that P's just just I don't know why but like all of the meal delivery companies in in on the planet um getting getting food to you quickly for too much money uh with you not having you go anywhere that's that's all realtime analytics powered stuff um Finance in ways that they don't like talking about uh stripe which counts as Finance but sort of like Mom and Pop Finance you know uh they've got actual uh well dashboards I said dashboards didn't matter they have dashboards where it does matter uh that that they do need to be real time uh various retailers who also don't own always uniformly like being talked about by name uh with uh behavior on their website being directly impacted by user-facing analytics so uh ever growing list of use cases and the whole point of the stuff I work on is to provide experiences to people who are using computer programs and make those experiences richer more engaging stickier you know more valuable whatever however the business is measuring that but but um we're generally you know utility maximizing actors and so we those features actually have to do things that we like in order for us to use them and that's the goal all right everyone um first I want to uh thank our panelists and I want uh you to give them a round of applause please and thank you everyone uh for today and uh we have a happy hour in that room um thanks to our sponsors uh this event is happening today tomorrow so uh give them some love and attention thank [Music] you