Loading…
Reactive [clear filter]
Thursday, November 16
 

9:50am PST

Return of the Transaction King
There has been a long-held belief based on the CAP Theorem that it is impossible for a database to have both strong consistency along with global scaling and high availability. This has led to a whole category of databases that don’t have one of the most fundamental features of a database and that is transactions. A whole new generation of databases are rising that bring back transactions and prove that we can have our cake and eat it to! This talk will focus on one example of this, FaunaDB which provides ACID consistency at a global scale. We will look at Scala code examples of using the FaunaDB functional query language to write transactions in a flexible and type safe style.

Speakers
avatar for Ryan Knight

Ryan Knight

Principal Software Architect / CEO, Grand Cloud
Ryan Knight is Principal Solution Architect at Grand Cloud. He is a passionate technologist with extensive experience in large scale distributed systems and data pipelines. He first started Java Consulting at the Sun Java Center and has since worked at a wide variety of companies... Read More →


Thursday November 16, 2017 9:50am - 10:30am PST
Reactive

10:40am PST

Spark Datasets: why they aren't great and what could be done about it
The Dataset API enabled Spark to implement advanced optimizations that were not feasible with RDD, but it also introduced a less intuitive and type-safe user-facing API. This talk will explore the reasons for the Dataframe/Dataset compromises and present a potential solution that would enable Spark to provide a fully type-safe and more intuitive Scala API through quotations without losing the ability to apply advanced optimizations.

Speakers
avatar for Flavio Brasil

Flavio Brasil

Software Engineer, Twitter


Thursday November 16, 2017 10:40am - 11:00am PST
Reactive

11:10am PST

IoT, Wavelets and Machine Learning in the Smart Home
The smart home, driven by AI is a quickly developing sector. Security in the home is a hot sub-sector of the smart home.

This talk will focus on a project that uses vibration sensors, wavelet transformations, and machine learning techniques to perform identification and intruder detection in the home.

This talk will focus on:
  • the sensor considerations and the HW stack, 
  • upgrading the project from a python to a spark stack, and 
  • an examination of the data from raw input through the wavelet transformations. 

A live demo will be given, showing live predictions of this system.

Speakers
avatar for Keita Broadwater, PhD, MBA

Keita Broadwater, PhD, MBA

Founder, Oxygen AI


Thursday November 16, 2017 11:10am - 11:30am PST
Reactive

11:40am PST

Bringing Deep Learning into the Big Data Analytics: BigDL on Apache Spark
Deep Learning is rapidly spreading as one of the most successful and widely applicable set of techniques across a range of applications (vision, language, speech, computational biology, robotics, etc.), leading to some significant commercial success. While the cutting-edge researches emerging in a breathtaking speed, from industry we often see the gaps from papers to prototypes, and even larger gaps between prototypes and production. In this talk, we present several key deep learning applications we built successfully with BigDL, an arising distributed deep learning framework on Apache Spark. For perception applications, we will introduce the object detection pipeline with SSD and how it’s used in production scenarios. For industry applications, we’ll present the neural recommendations system and fraud detection system we built with BigDL and Spark ML pipeline.

Speakers
avatar for Yuhao Yang

Yuhao Yang

Yuhao Yang is a senior software engineer in Intel BigDL team, focusing on BigDL development and deep learning applications. His area of focus is distributed deep learning/machine learning and has accumulated rich solution experiences, including fraud detection, recommendation, speech... Read More →


Thursday November 16, 2017 11:40am - 12:00pm PST
Reactive

12:00pm PST

Reasonable Scala Compiler
Reasonable Scala compiler is an experimental Scala compiler focused on compilation speed. We believe that it possible to achieve dramatic compilation speedups (5-10x) for typical Scala codebases, and we are currently well on track to realizing this vision. Join us at Scala by the Bay to see this with your own eyes. 

Speakers
avatar for Eugene Burmako

Eugene Burmako

Language tools lead, Twitter
Language tools lead at Twitter, member of the Scala language committee, founder of Rsc, Scalameta and Scala Macros.


Thursday November 16, 2017 12:00pm - 12:20pm PST
Reactive

1:10pm PST

Tame Your Data with Akka Streams
Stream processing is a hot topic today and it’s easy to get lost among all the possibilities. In this live coding session we will explore the Reactive Streams approach used by the Akka Streams project. On an almost real-life example we’re going to walk through both the basics and some more advanced usages of the library.

Speakers
avatar for Jacek Kunicki

Jacek Kunicki

Passionate Software Engineer, SoftwareMill
I'm a passionate software engineer living in the JVM land - mainly, but not limited to. I also tend to play with electronics and hardware. When sharing my knowlegde, I always keep in mind that a working example is worth a thousand words.


Thursday November 16, 2017 1:10pm - 1:50pm PST
Reactive

2:10pm PST

Introduction To Deep Learning

The speaker will provide a primer on Deep Learning.  The following topics will be covered:

1) What is deep learning?

2) What is deep learning capable of and what are the limits of deep learning in terms of technological advancement?

3) How is deep learning related to machine learning and artificial intelligence?

4) How did deep learning originate and progress to it's present state?

5)  How does deep learning work?

6) How can deep learning lead to the automation of intellectual and non-intellectual tasks and processes?

7) What are some barriers to entry and how can these barriers to entry be overcome?


Speakers
avatar for Alexander Tsyplikhin

Alexander Tsyplikhin

Chief Operating Monster, Data Monsters
Alexander Tsyplikhin, PhD, has been working in the area of data science for fifteen years, building consumer and enterprise products. Alexander is currently the Chief Operating Monster of Data Monsters, a machine intelligence and data engineering consulting and implementation company... Read More →


Thursday November 16, 2017 2:10pm - 2:50pm PST
Reactive

3:00pm PST

Multi Runtime Serving Pipelines
tl;dr: ML Functions as a Service: Envoy proxy powered Machine Learning Lambdas. 
Once a machine learning model has been trained, it can be used to generate predictions for online requests. Real world use cases require different machine learning frameworks and ad-hoc transformers to be chained into a single prediction pipeline.
Imagine NLP use case when we have custom feature extractors followed by weak random forest classifier and fastText model that prepares input vectors for the final neural network built with TensorFlow. All these stages have to be executed in a real time multi-framefork cross-runtime serving meta pipelines.
There is a solution to export the every single model into PMML/PFA and then import it into a separated scoring engine. It has multiple challenges, such as:
- Code duplication. Two different implementations and exporting functions of the same model in different languages needs to be maintained. This leads to inconsistencies, bugs and a lot of excess work.
- Limited extensibility. The prediction pipeline is not only a machine learning model. It has pre-processing steps, ad-hoc logic and very specific dependencies that could not be encoded in XML. Myriad of ML frameworks such as fastText, deeplearnign4j could not fit into a single scoring engine a priori.
- Inconsistency. Different implementations for streaming, large scale and scoring use cases cause different outputs in different contexts for the same inputs.
- Extra moving parts. Exporting/importing pipelines create additional complexity and points of failure. It should be eliminated rather than automated.

In this demo based talk with live coding we’ll propose an alternative solution and build a real life use case chaining different machine learning frameworks and runtimes that generate real time predictions.

Speakers
avatar for Stepan Pushkarev

Stepan Pushkarev

CTO, Hydrosphere.io
hydrosphere.io CTOAutomation of AI/ML Operations: deployment, serving, monitoring, subsampling, retraining.


Thursday November 16, 2017 3:00pm - 3:20pm PST
Reactive

3:20pm PST

Serverless Type Safe Data Pipelines With OpenWhisk
Operationalizing data pipelines is difficult at scale. It often requires standing up large compute and memory resources around the clock in order to do small computations on demand. This architecture makes it difficult to isolate pain points in the pipeline, understand resource usage and orchestrate events in the pipeline. Functions As A Service (FaaS) has helped many in the Internet of Things and web applications space to operationalize usage patterns and overcome this problem. In this talk, I discuss how I used OpenWhisk to operationalize data pipelines, increase orchestration and decrease costs. I talk about the benefits of using Scala and it's functional paradigms and type safety for communication across functions, building event driven data pipelines and deploying it with open source technologies. Half of the talk will be background information about OpenWhisk and the other half will be deploying a machine learning model using Spark and OpenWhisk.

Speakers
avatar for Jowanza Joseph

Jowanza Joseph

Data Architect, One Click Retail
Jowanza Joseph is a senior software engineer at One Click Retail, a Business Intelligence company in Salt Lake City. Jowanza's work is focused on distributed data and streaming architectures.


Thursday November 16, 2017 3:20pm - 3:40pm PST
Reactive

4:00pm PST

Scaling to Billions of Real-time Recommendations Daily
Twitter is a very real-time product, where the people from across the world are talking about what is happening. Recommendation systems (delivered via push notifications for example), often need to trade off relevance against real-timeness. In this talk, we will discuss a state of the art system built in-house at Twitter for generating highly relevant candidates, with low latency and at scale.

Speakers
avatar for Ajeet Grewal

Ajeet Grewal

Sr. Manager, Twitter


Thursday November 16, 2017 4:00pm - 4:40pm PST
Reactive

6:30pm PST

Fireside Chat with Ben Hindman and Ian Downes: Mesos, Then and Now
In this fireside chat, Ben Hindman, the creator of Apache Mesos and the original lead at Twitter, will discuss Mesos history, present, and future with Ian Downes, the current Mesos team lead.

Speakers
avatar for Ian Downes

Ian Downes

Director of Engineering, Twitter
Head of Mesos team, Twitter
avatar for Ben Hindman

Ben Hindman

Mesosphere Founder - Apache Mesos Co-Creator, Mesosphere
Ben is one of the creators of Apache Mesos, a platform for building and running resource-efficient distributed systems at scale. Ben started working on Mesos as a PhD student at Berkeley before he brought it to Twitter where it runs on thousands of machines. An academic at heart... Read More →
avatar for Alexy Khrabrov

Alexy Khrabrov

Program Chair, Reactive Summit


Thursday November 16, 2017 6:30pm - 7:00pm PST
Reactive
 
Friday, November 17
 

9:50am PST

Streamlio: Towards a Unified End-to-end Real Time Solution

Handling real-time data has become a critical capability for data-driven organizations. However, today’s reality is a

disconnected patchwork of incomplete technologies that make it a struggle to deliver real-time solutions because of

frustrating complexity, inefficiency, and incompleteness. In this talk, we address these challenges with an unified solution for real-time data. An end-to-end real time system needs

• Messaging: receive and distribute streaming data with support for publish-subscribe and queuing scenarios with built-in durability, scalability, and performance using the Apache Pulsar (incubating) messaging solution.

• Processing: process data transformations and analytics with the Heron real-time processing engine, built for performance and scalability.

• Storage: leverage Apache BookKeeper streaming log storage to ensure durability, resiliency, and performance for streaming data.

In our talk, we will provide an overview of the underlying three systems and how they are used the core for a unified end-to-end real time solution.

 

 


Speakers
avatar for Sijie Guo

Sijie Guo

Co-Founder, Streamlio
Sijie Guo is the cofounder of Streamlio, a company focused on building next generation real time processing engines. Streamlio mainly focus on 3 open source projects, which include Apache BookKeeper, Apache Pulsar, and Heron. Previously, he was the tech lead for messaging group... Read More →
avatar for Sanjeev Kulkarni

Sanjeev Kulkarni

CTO, Streamlio
Sanjeev Kulkarni is the co-founder of Streamlio that focuses on building next generation real time processing engines. Before Streamlio, he was the technical lead for real-time analytics at Twitter where he co-created Twitter Heron. Before that, we was at Locomatix where he handled... Read More →


Friday November 17, 2017 9:50am - 10:30am PST
Reactive

10:40am PST

Serverless Tensorflow 1.0.0 on AWS. Wait what?
This year tensorflow 1.0.0 was finally released. I succeeded to fit it into AWS Lambda and built recognition of images trained on imagenet. Why? Given the price tag lambda one run (recognition of one picture) will cost $0.00005. That is, for a dollar you can recognize 20,000 images. It is much cheaper than almost any alternatives, though completely scalable (1000 functions can be run in parallel), and can be easily integrated into cloud infrastructure. If you have a trained model in your pocket my talk will help you to fit it into AWS Lambda.

Update: I've ported Tensorflow 1.4 on AWS Lambda so I will include content about it and Keras.

Link to github: https://github.com/ryfeus/lambda-packs/tree/master/Tensorflow

Speakers
avatar for Rustem Feyzkhanov

Rustem Feyzkhanov

Staff Machine Learning Engineer, Instrumental
Rustem Feyzkhanov is a staff machine learning engineer at Instrumental, where he works on analytical models for the manufacturing industry, and AWS Machine Learning Hero. Rustem is passionate about serverless infrastructure (and AI deployments on it) and is the author of the course... Read More →



Friday November 17, 2017 10:40am - 11:00am PST
Reactive

11:10am PST

Fearless Multitenancy with Istio
Ever wished you could let your users run their own event processing code on your logging platform? Or allowed ecommerce customers the ability to run their own analytics backend? Container orchestration and service meshes offer seamless ways to run untrusted code in multitenant platforms. This talk will offer a brief overview of what service meshes can offer, along with a live demonstration of sandboxing untrusted services according to policy at both the container and network level. Leave this talk with new found confidence in your ability to grant more interesting and powerful opportunities on your own platform without compromising security.

Speakers
avatar for Ben Edwards

Ben Edwards

Tech Lead, starbucks


Friday November 17, 2017 11:10am - 11:30am PST
Reactive

11:40am PST

Immutable AWS Deployments with Packer and Jenkins
In this session I will talk about Immutable Deployments - which have become almost essential in the world of Microservices. As the frequency of deployments across multiple services increases with increasing granularity, it is critical to have repeatable, predictable and immutable deployments serving our customers. In practice, this is achieved via several DevOps tools. We will use Hashicorp Packer (packer.io) and Jenkins to build a simple, immutable AWS deployment of a hello-world microservice. Familiarity with AWS is recommended but not required for this talk.

Speakers
avatar for Manish Pandit

Manish Pandit

Director of Engineering, Marqeta
Manish is Director of Platform Engineering at Marqeta, a FinTech startup, where he is responsible for the Hybrid Cloud Architecture and Delivery of the Payments Platform. Prior to Marqeta, he has worked at Capital One, Netflix, IGN, E*Trade, and Accenture in various engineering leadership... Read More →



Friday November 17, 2017 11:40am - 12:20pm PST
Reactive

1:10pm PST

Real World Serverless
Serverless is a hot topic in the software architecture world and also one of the points of contention. Serverless let us run our code without provisioning or managing servers. We don't have to think about servers at all. Things like elasticity or resilience might not longer be our problem anymore. On the other hand, we have to embrace a little bit different approach how to design our applications. We also have to give up a lot of control we might want and the most importantly we have to use technology which just might not be ready. In this talk, I’d like to discuss if it is worth to use serverless in our applications, what are the advantages and disadvantages of this approach. Secondly, I'd like to describe various use cases we were considering serverless and what was the result. And finally, I’d like to talk about how Scala fits into this. This talk should be interesting for everyone who is considering using serverless or just heard this word somewhere and would like to learn more. The talk is a little bit more focused on AWS but the understanding of the concepts I’m going to talk about should be beneficial even if you prefer a different service provider.

Speakers
avatar for Petr Zapletal

Petr Zapletal

Lead Consultant, Cake Solutions
Petr is a Software Engineer who specialises in the design and implementation of highly scaleable, reactive and resilient distributed systems. He is a functional programming and open source enthusiast and has expertise in the area of big data and machine classification techniques. Petr... Read More →


Friday November 17, 2017 1:10pm - 1:30pm PST
Reactive

1:40pm PST

Intelligent system optimizations
This talk introduces engineering and machine learning techniques for achieving optimized performance, resilience, availability, cost or other attributes of a large scale distributed system. Firstly, the presentation introduces the topic by discussing the complexities of large scale production system properties as well as important optimization techniques used in state of the art distributed systems, microservice architectures, databases and stream and fast data processing frameworks such as Akka Streams, Spark or Flink. As the next step, a framework using machine learning and artificial intelligence approaches - specifically supervised, reinforcement and meta deep learning - is introduced as a tool for optimization, continuous evolution, learning, and improvement of the specific system and in the specific runtime environment and conditions. The discussion includes details of the problems, data, techniques, code in Keras or TensorFlow, examples and experience with some other modern machine learning and data processing tools. The attendees will gain an understanding of optimization approaches, including some novel machine learning techniques, and ultimately will be able to apply some of the techniques to optimize their own system, whether general distributed systems written in Scala or any other language, systems based on Akka or any of the aforementioned technologies.

Speakers
avatar for Martin Zapletal

Martin Zapletal

CTO, Cake Solutions Inc.
Martin Zapletal is heading up the technical team of Cake Solutions Inc. in New York. Martin specialises in the design and implementation of reactive, scalable, resilient distributed systems, machine learning and working with large amounts of data. He also has background in functional... Read More →


Friday November 17, 2017 1:40pm - 2:00pm PST
Reactive

2:10pm PST

A framework for rapid reporting API development
We’ll briefly talk about the reasons behind our decision to develop a framework to rapidly develop APIs for reporting. We’ll talk about the features for such a framework and how we make use of it. We’ll dive into an example application and talk through configuration, definitions, and how you can also easily embed this in an existing API. We’ll also talk through some of the design decisions and our ability to do resource isolation at the API layer to adhere to SLAs across multiple use cases. We'll also talk about how we analyze the usage of these APIs in order to do continuous optimizations and identify bugs.

Speakers
avatar for Hiral Patel

Hiral Patel

Director of Engineering, Oath
Hiral's been working with Scala for the past 7 years and Big Data for the past 13 years. He's built data platform's, data intensive applications, and real-time analytics frameworks. Hiral is currently a Senior Principal Architect/Director of Engineering at Yahoo/Oath (a Verizon c... Read More →


Friday November 17, 2017 2:10pm - 2:30pm PST
Reactive

2:30pm PST

Druid Lookups for High Cardinality Dimensions
Druid is a high-performance, column-oriented, distributed data store. Lookups are a concept in Druid where dimension values are (optionally) replaced with new values. The common use case of query-time lookups is to replace one dimension value (e.g. an ID) with another value (e.g. a human-readable Name). This is similar to a star-schema join. Druid has limited sup- port for joins through query-time lookups. Very small lookups (count of keys on the order of a few dozen to a few hundred) can be passed at query time as a "map" lookup as per dimension specs. For large lookups, Druid has an extension called Namespaced lookups. Namespaced lookups are appropriate for lookups that cannot be passed at query time due to their size, or are not desired to be passed at query time because the data is to reside in and be handled by the Druid servers. But Druid’s namespaced lookups has following limitations, • It is not suitable for high cardinality dimensions • It is not scalable for large data in the order of hundreds of millions of rows • Namespaced lookup support is limited to one key column with a corresponding value column • Real time updates to the lookup data is not possible These limitations encouraged us to develop a highly scalable, multi-column, configurable Druid lookup framework that supports real time updates on lookup data. Framework uses embeddable persistent key-value data store, kafka for messaging and HDFS for deep storage. 

Speakers
avatar for Pavan

Pavan

Principal Software Development Engineer, Oath


Friday November 17, 2017 2:30pm - 2:50pm PST
Reactive

3:00pm PST

Build a Modern, End-to-End, Fully Open Source, Big Data Scala Reference Application
Developing an end-to-end data pipeline can be difficult, especially when it comes to integrating multiple Big Data services. Current end-to-end reference applications are extremely outdated or not Scala-focused. We will walk through a modern real-time streaming application which serves as a reference framework for Scala developers wanting to develop a big data pipeline. We review code, best practices and considerations involved when integrating different components into a complete data platform (think Kafka, Spark, Storm, NiFi, Avro serialization, HBase, etc.). From IoT data collection from the edge, to flow management, real-time stream processing and analytics, through to machine learning and prediction, this reference application aims to help developers seed their own open source solutions – fast. Developers looking for reference code or documentation for integrating platform-agnostic tools will walk away with an actively supported resource. This talk hopes to empower developers to leverage open source efforts, and not be discouraged by the sometimes overwhelming undertaking that is trying out Big Data projects. We may not be able to live code or go over developing an *entire* pipeline, but the things we don't cover will be up in a repo, 100% working and documented, for people to check out. For Scala developers that may not necessarily have much experience with things Hadoop, I can also make available some kick-start tutorials and documentation that I have written while at Hortonworks.

Speakers
avatar for Edgar Orendain

Edgar Orendain

Software Engineer, Hortonworks


Friday November 17, 2017 3:00pm - 3:40pm PST
Reactive

4:00pm PST

Building a high-performance Future
Asynchronous programming is a popular choice for high-scale systems, but it often comes with a hidden performance cost. This talk will present a study of the performance characteristics of the asynchronous programming solutions on the JVM and the benefits that their abstractions provide to the user. Based on this study, the talk will present detailed report of the techniques and tools used to build the new the trane.io library, a high-performance Future implementation, and how they were recently applied to optimize Twitter's Future. It will include an overview of high-level and low-level profiling tools, JIT optimizations, and micro-benchmarking.

Speakers
avatar for Flavio Brasil

Flavio Brasil

Software Engineer, Twitter


Friday November 17, 2017 4:00pm - 4:40pm PST
Reactive
 
Saturday, November 18
 

9:50am PST

Nelson: Rigorous Deployment for a Functional World

Functional programming finds its roots in mathematics - the pursuit of purity and completeness. We functional programmers look to formalize system behaviors in an algebraic and total manner. Despite this, when it comes time to deploy ones beautiful monadic ivory towers to production, most organizations cast caution to the wind and use a myriad of bash scripts and sticky tape to get the job done. In this talk, the speaker will introduce you to Nelson, an open-source project from Verizon that looks to provide rigor to your large distributed system, whilst offering best-in-class security, runtime traffic shifting and a fully immutable approach to application lifecycle. Nelson itself is entirely composed of free algebras and coproducts, and the speaker will show not only how this has enabled development, but also how it provided a frame with which to reason about solutions to fundamental operational problems.

 


Speakers
avatar for Tim Perrett

Tim Perrett

Head of Infrastructure Engineering, Verizon
Avid functional programmer, experienced distributed systems engineer and published author. Primarily interested in schedulers, datacenter design, low-latency data access and the application of functional paradigms in large enterprise applications.


Saturday November 18, 2017 9:50am - 10:30am PST
Reactive

10:40am PST

Building systems on top of Kubernetes
With the rise of microservices and IaaS, interest in container orchestration systems - like Kubernetes, Mesos, and Docker Swarm - has increased significantly. While many engineers will use these systems as a target to deploy their service to, they also expose an API that can be integrated in an application, allowing it to interact with it. In this talk, we will create a simple reverse proxy in Scala that can act as a Kubernetes Ingress Controller, similar to Traefik and linkerd. We are going to use some features from Kubernetes such as its event streams, ingresses, and services. This will provide a good opportunity to expose how an engineer can use the Kubernetes API to build systems needing to be both dynamic and responsive to what is happening on it.

Speakers
avatar for Alexandre Bergeron

Alexandre Bergeron

Software Engineer


Saturday November 18, 2017 10:40am - 11:00am PST
Reactive

11:10am PST

Just enough DevOps for data scientists.
In this talk, I've distilled the best practices we've learned while building the Spark & microservice architecture for Salesforce Einstein.  You'll come away from this talk with a peek into the secret life of SREs.

Speakers
avatar for Anya Bida

Anya Bida

Senior Member of Technical Staff, Salesforce
Anya loves her position as Senior Member of Technical Staff (SRE) at Salesforce. She's also a co-organizer of the SF Big Analytics meetup group, and is always looking for ways to make platforms more scalable / cost efficient / secure. Before Salesforce, Anya enjoyed contributing... Read More →


Saturday November 18, 2017 11:10am - 11:30am PST
Reactive

11:40am PST

Building model testing infrastructure and scaling AI predictions with AWS Lambda
AI model training and validation is just a small part of the story. After the model is built, the real work begins. To service millions of customers seamlessly, every application must scale. In this talk I will showcase how customers can build multi arm bandit strategy to test their models at scale and also scale their prediction pipeline using AWS Lambda.

Speakers
avatar for Sunil Mallya

Sunil Mallya

Deep Learning, Amazon Web Services


Saturday November 18, 2017 11:40am - 12:20pm PST
Reactive

1:10pm PST

How to Pick Your Next Streaming Architecture
Streaming is the major trend in the evolution of Big Data technologies and it is becoming a necessary part of systems designed to handle high volume of events. In this talk, we begin with characteristics of Stream Processing, and then discuss the dominant architectures for Streaming systems and continue into examples of technologies and application solutions for streaming applications. We compare the important technologies, such as Akka Streams, Storm, Kafka Streaming, Spark, Flink, Beam. We explore Quality Attributes of such systems as a mechanism for making architectural decisions and then discuss an approach for choosing a streaming systems. Outline: · Stream Processing: What is it? · Architectures for Stream Processing · Akka, Streaming, and Backpressure · Apache Storm: The Dedicated Stream system · Kafka Streaming: Taking Streaming Mainstream · Apache Spark Streaming: A versatile in-memory batch/streaming system · Apache Flink: Novel integration of batch and streaming · Apache Beam: The common API layer · Architectural Scenarios for Streaming · Choosing your Streaming System (Without Remorse!)

Speakers
avatar for Vladimir Bacvanski

Vladimir Bacvanski

Principal Architect, Strategic Architecture, PayPal
Dr. Vladimir Bacvanski's interest is in better and more productive ways to develop highly scalable and reliable software systems. Before joining PayPal, he was the CTO and founder of SciSpike, a company doing custom development and consulting. His recent projects include Big Data... Read More →


Saturday November 18, 2017 1:10pm - 1:30pm PST
Reactive

1:40pm PST

Managing Nation-Wide Traffic Cameras and Sensors
Belgium is deploying a large network of traffic cameras and sensors. This nation-wide infrastructure aims at implementing smart city initiatives, better controlling air pollution, responding faster to emergency situations, and increasing security on the roads by better enforcing traffic law. This presentation describes the big data architecture put in place to collect and process in near-real time data generated by thousand of cameras and sensors and how this architecture is implemented using Apache Kafka, and the Play! and Akka Scala frameworks. It discusses the main challenges met during the project and the approaches selected to overcome them and ensure the reliability, scalability, and robustness of the overall infrastructure.

Speakers
avatar for David Massart

David Massart

Solution Architect, D.E.Solution



Saturday November 18, 2017 1:40pm - 2:00pm PST
Reactive

2:10pm PST

Live coding server-less, single codebase iOS, Android and Web Apps using ScalaJS, React-Native and GraphQL
Scala-JS just hit 1.0.0 and React-native is maturing well. Combining the advantages of both yields the promise of end-to-end FP to generate 3 different apps from a single code base. Add to the mix GraphQL in a server-less configuration and we have a real shot at building a meaningful app in 40 minutes. Come early.

Speakers
avatar for Irfan Ahmad

Irfan Ahmad

Founder, Magnition
Irfan Ahmad is a full-stack founder of two Silicon Valley startups. He loves new code smell and started developing with Scala back before it was cool. Prior to this, Irfan was at VMware hacking on hypervisors and at Transmeta working on the Crusoe software microprocessor. Irfan... Read More →


Saturday November 18, 2017 2:10pm - 2:50pm PST
Reactive

3:00pm PST

Declare, verify and execute microservices-based process flows with Baker
At ING we have a microservices-based architecture. It certainly sounds cool, but how can we enable developers create innovative customer experiences even faster? How can they compose APIs in a functional fashion, reason about their system complexity and change functionality comfortably without breaking it? I'll talk about an open-source Scala library we developed to declare, verify and execute microservices-based orchestration flows. I'll demonstrate how: - to write a recipe for such a flow with an internal DSL (domain specific language); - visualize the recipe as a graph to communicate the steps of the process with business and technical stakeholders; - and run the resulting flow within a RESTful API in an event-driven architectural model; - talk about near future plans to develop an external DSL and host recipes in a serverless architecture;

Speakers
avatar for Nikola Kasev

Nikola Kasev

Software Engineer, ING Bank


Saturday November 18, 2017 3:00pm - 3:40pm PST
Reactive

4:00pm PST

Implementing a Lazy Functional Language with Combinatory Logic
Topic Change: Slick Schema Code Generation

In this session, we discuss the use of code generation to remove the pain of writing out Slick database schema when working with or evolving an existing database schema. We'll demo a plugin that, with only a few lines of configuration, enables:
  • generation of tagged id types for primary and foreign keys
  • restricting schema generation to specific tables
  • separation of row and table specifications into distinct compilation units
  • customizable mappings between application and database types

This work arises from real-world experience working with Slick and legacy database schemas. The feelings of safety of having a database schema described in Scala are sometimes dwarfed by the pain of actually having to write and maintain the Scala mappings. This becomes obvious when building a new project with an existing database.

The code generation component is bundled with Slick. However, the documentation and default implementation show only the most trivial use-case and require a more complecated build. With every new project we have updated and expanded on the use-cases the plugin covers. Now, we would like to share what we have built and learned.


Speakers
avatar for Stewart Stewart

Stewart Stewart

Software Consultant, Inner Product LLC
Stewart Stewart is a software developer at Driver, a San Francisco based startup that analyzes tumors and connects cancer patients with personalized medicine. He also helps organize events at SF Scala.


Saturday November 18, 2017 4:00pm - 4:40pm PST
Reactive
 
Filter sessions
Apply filters to sessions.