data.bythebay.io: Kanu Gulati, Investing in Data Analytics
Recording: data.bythebay.io: Kanu Gulati, Investing in Data Analytics
hi can y'all hear me okay awesome hey everyone I'm gonna Gulati and I'm going to speak on the opportunities in the zettabyte era I work out of fun called the set up venture partners and we invest in enterprise software that learns from data and we call it intelligence Enterprise Fund industry analysts have predicted that the annual IP global traffic will exceed one Sara byte in 2006 2016 this is what is called the zettabyte era where in addition to the amount of data and the characteristics of data lots of other things the business models how you approach problems where to focus on a lot of other components changes and that's what I'm here to talk about and I will be focusing a lot on the industrial Internet of Things or the IOT we think that intelligence is the fourth era of computing to preface that the first era was the hardware that was in the 1980s which was the production of semiconductors that enabled us to quickly process data the second era was the development of software to process real-world applications the third was a distribution of computing to significantly reduce costs and allowed the development of new types of software the fourth era is the intelligence which combines law large volumes of data with clouded distributed computing and with intelligent learning solutions and this is what we call the intelligence era just as all workflows became software in the 1990s we are convinced that all software is now quickly becoming analytical this is the alpha end of my talk I will begin with some of the drivers of this new era talk about the implications of this new era on business models roles and product development but focus the bulk of my talk on the opportunities which arise in industrial analytics in addition to the growth in data there are several other drivers to the sport era of computing the recent results in learning and natural language processing solutions and the rise in number of mobile or handheld solutions are often cited about and you must have all read about it I really want a big attention exclusively to three other drivers one the rapid growth in the number of public datasets in the last few years the number of public API czar have been launched in the current administration has been more than 193 thousand similarly the rise of cloud computing more than 50 percent of all enterprise apps today are now hosted on a cloud and third the number of connecting devices it's now rapidly approaching 50 billion in the next few years all of these drivers in addition with learning solutions algorithms etc are playing a huge part in the advent of this intelligent era this in turn has implications on business models and roles in the enterprise one of the biggest implications of intelligence is that analytics companies build proprietary datasets which when applied along with the core business model built defensibility that organically grows with more usage and more data just as google's defense ability is achieved by intelligently collecting and applying query logs is today one of the hardest to challenge another implication of the fourth era of computing is that every role on the board today requires data and intelligence driven tools and solutions to carry out their activities some of these roles already have tools that integrate intelligence but these are open opportunities for all the entrepreneurs in the room looking for where to invest their time a pretty broad sense of open opportunities are also in data science development and collaboration for example data scientists today still struggle with deploying the models to business cases because Lyra science tools today don't integrate well with current software similarly data scientists for them to collaborate with each other is still a challenge because the joint versioning of code and data is still not a solved problem solutions that require large volumes of data to Train HAP's struggle with the cold start or the bootstrapping problem and delivery of data in real time requires new distribution models which are all open opportunities however in this talk I want to focus on the open opportunities which we have realized looking at hundreds of companies in this space and talking to a lot of customers which we think are still on address the industrial Internet of Things or industrial IOT connects the physical world of sensors devices and machines in our industries to the Internet it enables unique objects to collect and analyze a variety of data the in size generated from this data using analytics are singular and valuable and can fundamentally transform businesses and technologies industrial analytics are predicted to grow at a 30% kegger over the next decade and there are several reasons some of them have already covered in generally in reducing costs of hardware reducing form factors improvement in big data tools an infrastructure and a variety of other reasons are causing a rapid growth of Internet of Things this this framework is our way of trying to understand where are the new opportunities and where are the current companies being built in this space the industrials analytics framework in our view is on the top layer is the data or the knowledge generation layer and the bottom layer is the analytics application layer this is a very abstracted view the leftmost column is the streaming analytics which is data collection from a variety of sensors and devices this could be reader such as temperature humidity pressure or electrical measurements and this is gathered and stored and streams to data management systems which here is labeled as analytical database these are designed to ingest millions of events per second these systems allow data aggregation cleaning storage and real-time comparison of multiple streams and with historical values and models analytical database in conjunction with monitoring systems and visual data exploration systems which are labeled as visual analytics on the chart enable discovery and learning from this data which can be useful in updating historical models or in identifying anomalies when a specific anomaly or a condition appears it can result in an automatic action such as a trigger or a digital adjustment or an alert which is designed by the reporting system today's Suman today human supervision is quite indispensable in all of the aspects here and the challenge is in automating almost every component here so that we can work with the machine generated data and at that rate one of the areas of gap that we've identified is data warehouses today are simply not designed to handle high velocity of streaming data most of this new data is unstructured and with no standard data or versioning systems both historical and real-time data is required in order to understand the scenarios and take appropriate actions to delve further into this problem there are multiple reasons why this is a huge problem today you're integrating data from multiple sources this data could be available in data houses in public databases in global web-based systems and you have to integrate legacy data with real-time event or time series data these sources often contain redundant data in different representations and there are no clear winner in techniques which can provide filtering querying schema translation aggregation or migration today another reason why this problem is very complicated is because news newer schemas are more and increasingly getting hybrid so traditionally the schema on the Left used to be employed to aggregate data from a variety of sources the data warehouse approach which would offer a tightly coupled architecture because these data is already physically reconciled in a single variable architecture however more recently newer schematics have been employed which favor loose coupling between hybrid arrow sets and provide a unified interface to access real-time data over a mediated schema the sources in this mediated schema are hybrid they are not the same kind of databases they could be both open source or commercial solutions and to further complicate this problem as mentioned previously both the characteristics of data and applications are evolving so increasingly applications require analysis done on this more complicated more volumous data in real-time analysis of streaming data is to be required at hundreds of gigabytes per minute and results honestly can't wait for batch processing and instead of processing serially imagine the power of processing data and queries in parallel or using predictive or probabilistic methods and so these are the areas where I think our big open opportunities for entrepreneurs and investors to invest in coming back to the same IOT analytics framework another area of open problems are solutions that allow decentralized decision-making in industrial workflows this is because analytics in general IOT Nellore analytics require any tends to wave flow of information in order to coordinate and manage a huge conglomerate of producers and consumers of data and everyone in between in order to not overload the system decisions should be made as close as possible to the source which creates the need for decentralized decision-making capability actions need to be performed quickly autonomously and without human intervention or error solutions could be in the form of intelligent edge computing or mobile computing in industrial workflows this is some of the sample problems and possible intelligent solutions which you know I hear from other companies from from potential customers begin with responding in real-time so being able to run algorithms on streaming source data prioritizing among incidents so incorporating information among previous forces of action to determine what's worth investigating next modeling user behavior an intelligent solution there could be two model behavior and compute risk course at the individual user level and then adapt these models as behavior revolves towards mobile and decentralized and more collaborative solutions now define another problem is finding vulnerabilities before being exposed solutions here could be using neural networks to act like human responders and therefore exploring all applications and another problem is managing in multiple entry or endpoints and a percent potential intelligent solution here could be to set up neural networks to explore all endpoints these are very high level problems and solutions which I'm listing but this is what I hear from customer interviews in industrial Internet of Things another area of application which is also ties back to this decentralized decision-making is in trying to improve the efficiency of analyzing correlations so oftentimes false or redundant alerts reduce the efficiency of data driven processes there are limited commercial solutions that identify the source of an anomaly what is required for this are basically a fast anomaly detection quick regression and correlation analysis and causality inference to identify the source of that problem and once you have identified the source of the problem corrective actions can be taken and the analytics loops can be sped up but again today there are no good clear winners in solutions which enable analyzing correlations really quickly to quickly cover some of the risks of investing your time and resources in analytics one of the risks are focusing on solutions with either of these things like with unclear proprietary data acquisition strategy lack of advanced analytics and statistics experience particularly in learning solutions insufficient compliance and governance of data or reliance on a previous generation of computer infrastructure to solve today's problems all of these are huge risks when I look at investing in opportunities and IOT another big area of risk is an over focus on specific technology trends in general it's always hard to pick industry and market trends than just broad generation-defining technology trends so we like to invest in more market trends and I'd you know implore you to focus on where the markets are shifting and spending your time and energy and resources on that spaces to summarize then you know today you know it is the new era of computing the intelligence era which combines huge volumes of data with software with learning solutions and distributed computing there are major implications for this era on the enterprise for business readers rolls product development and deployment and specifically opportunities in industrial IOT which I've listed here are about two areas one systems are aggregate real time data streams and legacy disk data and two scalable solutions that allow decentralized decision-making in industrial workflows thank you we have a few minutes for any questions Hey so to repeat your question you're saying that today the valuation of IOT solutions is unfairly high and you think it's going to fall down is that your question so the way you would value a company on a solution in IOT would be the impact it's having on an organization and its compute and its ecosystem and the vendors and its customers so we you know a lot of the community and I definitely believe that in specially in a nutshell IOT if you're just beginning to scratch the surface and what data you are collecting and even looking to analyze I believe we can see that a report last year it said today not even 1% of all of the IOT data that's being collected is being analyzed and in some ways you're speaking to my you know initial thought that most of the value is in analytics and in using insights from the data to make your solutions better and so improving both the top line and reducing the costs so why my in fact understanding is the valuations today I don't know if you're pointing to many particular valuations and we can talk about those offline but in general I think IOT is still a very in itself of open opportunity people have just been trying to understand that how do I even collect all that data and analyzing that data hasn't even started in the true sense and applying those to have a quicker feedback is this again a big bite open opportunity the insights from the data combined with you know learning solutions combined with the distributed computing infrastructure we have today yes that that value will only improve thank you sure another question on the back there right now see you're absolutely right like a lot of the bigger companies today if you see some of the biggest investments in IOT are by companies from corporate pcs from companies such as GE and Tyco and Siemens and that's because they are the first people who are facing those challenges and they're trying to identify where I can find solutions and so you know I'm out here to come and tell people that these are still unsolved problems they can start with trying to solve a tiny solution but the area effectively should be to generate insights to continue having you know building your defense ability and that's why I talked about the defensible models it is Google when compared to informatica and how do you apply those insights back but you have to start somewhere start one of these challenges such as this data aggregation and cleaning an IOT is a huge challenge you can start with that and say for a particular vertical within that solve that problem and grow your defense ability by collecting data and become the best solution to providing data clearing and aggregation for IOT or providing context for IOT and then go from there I well the reason I working in venture capital is because I think startups out there cans are just more efficient faster can try out new products models and solutions and big companies are just too sluggish and slower so that's where the advantage of being in a start-up is so yeah I'm definitely bullish on startups with all these problems alright one last question I think in fact I see increasingly these demands of real-time processing and real-time is a very fluid term people use real time sometimes for a day's data and sometimes in milliseconds data so in your real time where I talk about it like milliseconds is required first say when you mention retail but also in general say the energy and the transportation industry like trucks today have so many sensors and to collect that data in real time to provide the right guidance to the trucking the truckers information the routing etcetera has to be done in real time similarly dynamic pricing what is needed by ubers and you know all of the share economy like on-demand economy just changes drastically if you can really do real-time pricing as opposed to batch processing for historical data so increasingly there's so many areas that is moving towards real-time and it's still a struggle but that's where the opportunities are so happy to talk more about it offline and you know areas which I see a lot of companies or areas which I still see are open thank you