Back To Schedule
Thursday, November 16 • 3:20pm - 3:40pm
Serverless Type Safe Data Pipelines With OpenWhisk

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Operationalizing data pipelines is difficult at scale. It often requires standing up large compute and memory resources around the clock in order to do small computations on demand. This architecture makes it difficult to isolate pain points in the pipeline, understand resource usage and orchestrate events in the pipeline. Functions As A Service (FaaS) has helped many in the Internet of Things and web applications space to operationalize usage patterns and overcome this problem. In this talk, I discuss how I used OpenWhisk to operationalize data pipelines, increase orchestration and decrease costs. I talk about the benefits of using Scala and it's functional paradigms and type safety for communication across functions, building event driven data pipelines and deploying it with open source technologies. Half of the talk will be background information about OpenWhisk and the other half will be deploying a machine learning model using Spark and OpenWhisk.

avatar for Jowanza Joseph

Jowanza Joseph

Data Architect, One Click Retail
Jowanza Joseph is a senior software engineer at One Click Retail, a Business Intelligence company in Salt Lake City. Jowanza's work is focused on distributed data and streaming architectures.

Thursday November 16, 2017 3:20pm - 3:40pm PST