Is Generative BI failing? | Boris Renski, AI By the Bay 2025
Recording: Is Generative BI failing? | Boris Renski, AI By the Bay 2025
Thank you everybody. Um, so starting with a bit of a repeat of an introduction. So I am not a very well-known entity in the AI space. Um my background is primarily in u open-source infrastructure and um I'm mostly a startup guy. About 15 years ago when cloud was cool I started a company called Morantis um helped co-ound it and uh scale it. Then maybe 10 years later when cloud was no longer cool but uh 5G was really cool I built a company um focused on uh open-source private 5G and that was fairly successful quick exit only two and a half years um after founding that company got uh acquired by Helium mobile which is a um telecom carrier basically. And since now all the rage is about AI, I started dabbling a little bit in open-source AI and um AI infrastructure space. Um so the talk today is about uh GenBI
So what's GenBI? Genbi is a cool term that stands for generative business intelligence. In plain speak that means offering an ability to people uh to basically talk to data. So the world of BI is all kind of charts and tables. Obviously, now that we're in a world of uh, you know, chat GPT and chat bots, you just want to be able to ask any specific quantitative question and get an instant answer. Sounds like a very cool value proposition. And from a business standpoint, I was always puzzled um as to why there is not really a lot of successes that we hear about when it comes to uh generative BI. In my mind, from the standpoint of value ad, it is not less than coding co-pilots. Um the value is tremendous and there is probably over a dozen coding co-pilot companies uh that are you know well over a billion dollar valuation but um if you look at the the number of generative BI companies um there isn't almost any that are worth a billion or more
Uh and this is not for the lack of trying. there's a lot of very credible open-source efforts as well as small funded startups that are actually trying to do stuff in the space and I was kind of you know puzzled by this and I think that that's maybe an opportunity. So I started by actually you know trying things hands-on because the way that I started my two previous companies is I just basically went in and started doing the thing that they've ended up basically you know becoming a company around. So for Morantis we had a um basically like an internet service provider who was looking to build a cloud and we went in and we just tried to build a cloud for them for freedom. I literally like stood up a 5G cell tower on the roof of my house and tried to build it and see how it works. And for this particular case, um I figured that what we can do is uh we can take uh the company that acquired my previous startup um Helium mobile and see if we can implement um a generative BI system internally and see what happens. These types of systems are particularly useful for companies that deal with a lot of data and um like a mobile broadband company, you know, deals with a lot of subscribers, a lot of customers. Um a lot of its success is based on its ability to understand what tends, you know, what what allows people to subscribe, you know, what the churn rate is, etc
So, it's it's it's a very data- hungry business. And the way we went about implementing is okay let's just you know take all the data that we had and uh attach effectively a chatbot with some rag engine to it and the general cycle would work as follows. So you you know you go in there you ask a question you get some answer and then you repeat. So very much similar to what you do with just the typical chatbot like uh chat GPT. Um the way that you implement most Genbi systems is you know roughly the same across the board. You basically take the user prompt um you do some rag with the user prompt and do some prompt augmentation. You you know take a user prompt you in addition to the user prompt effectively explain what the database structure looks like what you know kind of data is in different columns supplement it with maybe some business term uh definition so that the system knows what for example a subscriber is and then you send that to an LLM to generate an SQL you get that SQL you interpret it and then you spit out the answer and that's kind of a general pattern for it so We implemented it and uh kind of did a small push internally and a few people used it maybe a couple of times and then like nothing happened. So it was like I would say a disaster
Um so our first impression was like okay well it's probably because we are not um you know giving people the correct answers so something's wrong with our text to SQL engine. But then we spend some cycles and we talked to some users of the system actually and looked at the data a little bit deeper and what we've uncovered I think is u potentially um an answer not just why we weren't successful but also why a lot of the GenBI systems today broadly are not as successful as they could be. So if you go back to the example um between like BI copilots and coding copilots um it's important to understand that the audience and the usage patterns and the expectations of these two groups are vastly different. So the coding copilots um you know you're dealing with engineers BI co-pilots you're dealing with a business user and the engineer goes in into effectively a chatbot experience like cloud code for instance um with the expectation that they're building stuff and they're going to be iterating on the outcome. No engineer goes in and loads cloud COD and uh you know puts in a single prompt and expects that the application is immediately built and can be launched with like one shot prompt. When it comes to a business user, the business user they expect, you know, oneshot magic almost every single time. So if I want to go in and find out how many subscribers I had in California um yesterday, I want the exact answer immediately. And the problem with that is uh often times um the person that is generating the prompt they don't even fully understand themselves what what is it that they want to ask
Um so the second part is that you know you can make the coding co-pilots better by making actually the AI agent better make the agentic flow better make it write code better etc. When it comes to the BI co-pilots, the problem of generating SQL is not that hard. Like if you take literally cloud code and just point it to a database, it's able to, you know, pull in the structure of a database and you can talk to the database using cloud code. The problem is disambiguating the intent of the user because the business user that is actually using a BI copilot, they're not an engineer and oftentimes they go in, they don't even know what to ask. In fact, if you present them with a blank screen, they don't even understand like what the system is capable of, what the limitations of the data are. And finally, the pattern for the engineer is that, you know, you again you you iterate, you build something, you know, you come back, you build something else. So, you're constantly iterating. For a business user, like if I go into a BI system and I ask, um, you know, how many subscribers did we have yesterday? I don't want to be coming back every single day and asking that same question
I expect I'm used to working with charts. I'm used to working with graphs. So if I've built basically an answer for myself, I want to have a place where I can come back to and I will immediately see this without having to explain to the system what is it that I want. So um we did a take two on it and with this all of this in mind we basically altered the flow dramatically which also dramatically improved the uh uptake and then the usage on the system. So we said okay we're not going to start with an empty chat box. We're going to start with actually a table or a chart so that a person exactly comes to a chart that is showing something and then you can edit this chart and you can basically decompose it into like if it's a chart it'll show you the tabular data behind the chart and the explanation exactly of how this data been derived and uh we call this thing like a semantic editor of a chart. So you can actually talk to the semantic editor you can highlight columns, rows etc. Um and you can through iteration arrive to an outcome that is actually the answer
And then finally you can actually you know whatever is the final answer you can save it and have a place to come back. And what this does is first of all it completely resets the expectations of a user. So you're no longer presenting them with an empty chat box chat box. They they know what to ask because they start with a chart. um the iteration through the semantic editor actually allows them to kind of like maintain the context and they understand the numbers as they change. So it's not like they ask some question, oh the question is wrong. Okay, I'm not coming back to it. They can see exactly how this question was derived
And if something is wrong about the data, they can see what exactly was wrong. Maybe some extra column, maybe too many columns, something like this. Um, and then they can save it and they come back to it. So um, final words. Um, so we've been iterating on this. We're still not quite, you know, anywhere finished. Our objective is to kind of understand what is the best way to uh sort of build a system that is easy for an organization to adopt and we know that uh you know um companies especially big companies with established patterns they don't like implementing new tools. So um the thing that we're working on now is um we are basically replacing the existing sort of a charting and tabular engine that we have in semantic grid with the ability to integrate with the uh existing BI system
So the one company in particular working with is using superset very heavily and that's open source and pretty easy to integrate with and you know the chatbot/interface that we've built for um iterating on actually modifying the charts. Uh we've had some luck experimenting actually integrating into um like third-party web chat clients using uh a pretty cool project called MCPUX which I suggest most of you guys should check it out. Um, which again will allow us to deliver that same experience of actually starting with a chart, iterating on a chart or a table and then saving it to some place uh without necessarily introducing a whole bunch of like new tools or interfaces into the organization just piggybacking off of uh what's already being used internally hopefully. So we just recently opened the code up. So we are you know iterating in the open on this. So if anybody's interested in playing with it or has any questions about it feel free to ask me or ping me. Um and that's it. So if we time for questions maybe questions if not I'm done
>> One minute for question if anybody has Yeah. No. Okay. Well if you can think of any more questions. I think bars you still here so you can go over to the interview room or the hallway to ask him questions too. Um in the meantime we will be getting ready for the next speaker. The last one of the day will be Francis. Yeah
>> Okay. >> Thank you.