scale.bythebay.io: Marius Eriksen Interview
Recording: scale.bythebay.io: Marius Eriksen Interview
[Music] my name is Myers Eriksson and I work for Grail we detects cancers early when they can be cured so I would say primarily there's a huge data analytics challenges where we're trying to make sense of vast quantities of data and there's a sort of infrastructure components through that as well so that we want to make sure that we're able to sort of fully utilize and harness cloud computing to to do these sorts of analysis of large scale data sets [Music] so there's a couple different dimensions so one is sort of pure data the sequencers that we have I'll put a lot of data for each sample it can be upwards the terabyte of data and even just moving that data around can be challenging and storing it can be quite costly so some of the some of those challenges how acquired us to do reasonably sophisticated you know engineering in terms of data storage and data movement and so on the other side of that coin is it's again computation to be able to analyze these large quantities of data in an efficient way and and one that minimizes costs we make use of cloud computing providers and we make heavy use of things like the Amazon spot market and try to be you know pretty clever about how we schedule our work to be done in order to minimize our costs [Music] I would say a lot of the same so we're involved in some very large clinical trials and we're getting more and more data and they're accelerating pates and so this going to be you know basically a level up of the same set of challenges that's one thing the other thing is that as our analyses are getting more sophisticated they also becoming more computationally intensive and so we need to solve those problems as well the the thing about by informatics is that a lot of the standard tooling around bioinformatics has not been built with a sort of efficient architecture in mind so for example there's very little attention being paid to the way data flow between different tools the sort of i/o paths within the tools and so on and so forth and there's a lot of efficiencies to be gained potentially by sort of revisiting that an architectural level [Music] so we actually so we use I guess two functional languages one is when is our the language for statistics which is say a functional language and that's used quite heavily in most of our sort of model building and statistics the other one is our own proprietary language called reflow which is what I give a talk about today and it's now open source and that is a language that allows us to express these workflow computations at a high level in a way that gives an underlying runtime a lot of freedom to efficiently execute those computations I think my favorite thing about scale by the bay is the diversity of people here there's people from everything everything from people who do a lot of online transaction processing to running large websites to people who do a lot of large-scale data analytics and sort of people from a wide variety of industries it's always nice to get the person [Music]