scala.bythebay.io: Chris Fregly Interview
Recording: scala.bythebay.io: Chris Fregly Interview
yeah my name is Chris fregley pipeline IO is the name of the company I'm a research scientist they're building out an m/l platform yeah everything from the training pipeline portion to the actual serving or the final like last mile prediction yeah I picked up Scala I joined well the first the first Scylla open-source project I picked up with SPARC and I contributed to that I needed a way to get Kinesis data into SPARC and there was already a there was a Kafka connector for SPARC so I had to go build one for caduceus and so I was learning SPARC I was learning Scala and yeah I'm not like necessarily the most fluent but pretty yes I still have to go out to Stack Overflow to get the best practices and keep myself in check so yeah so I mean at pipeline i/o were we're basically building the ideal stack but stacks I've seen in the past which is what rightly basically led us to build this company is you know sort of this right like disjointed just a lot of different pieces like elasticsearch kind of off running on its own with its own scheduler and write like cluster Orchestrator just you know sort of piecemeal together and cron jobs that you can't track down or if they're broken you have no insight into them so from the beginning with pipeline i/o we have yeah basically all the metrics right like everywhere throughout all the different pieces they're all stitched together nicely with rather airflow and let's see yeah just having different options for the notebooks we have Jupiter notebook we have zeppelin notebook yeah that way we don't rule anyone out people that that want to do scala can use Upland people want to use Python can use Jupiter things like that so yeah yeah there I missed the flink talk by Jamie Greer so actually I have to go back to watch the videos for that but I was looking forward to that one I think that was yesterday who else is coming up I have to look at the yeah like the schedule again but yeah some of the fling stuff and then there's some talks by some of the Twitter guys I think there's a talk on Heron which is built in Scala and it's kind of the next generation storm starting to see a lot of resurgence with storm because of spark streaming not really keeping up with the sort of like development cycle with spark streaming has been slowing down quite a bit or it just isn't I guess taking off like it it should have been after a couple years so seeing more and more people interested in storm so Heron which is the next generation storm looks pretty promising it's an open source project out of Twitter