DevConf.cz 2021

In this talk you will learn how to build an end-to-end analytics application using (almost) only jupyter notebooks as the basic unit of development.

When developing AI driven applications, there is often a friction point in the development life cycle when porting your code from a Data Scientistโ€s experimental notebook into the production ready code a Software Engineer expects. What if we could find a solution to this issue, and let the notebooks themselves be run in a DAG workflow, connected to each other, allowing them to share and exchange data and avoid this porting step all together, making it simpler for data scientists and software engineers to collaborate and quickly iterate on their AI driven application?

We will walk you through a case study where we did just that. Using the Open Data Hub toolkit, specifically: Jupyterhub, Ceph, Hive, Hue, Superset and Argo on Openshift, to build a recurring email list analytics and dashboard application. Highlighting some pitfalls we made along the way, how we could improve in the future with Elyra and how this process is general enough to be applied to many AI driven application development use cases.

Deep learning and machine learning more broadly depend on large quantities of data to develop accurate predictive models. In areas such as medical research, sharing data among institutions can lead to even greater value. However, data often includes personally identifiable information that we may not want to (or even be legally allowed to) share with others. Traditional anonymization techniques only help to a degree.

In this talk, Red Hat's Gordon Haff will share with you the active activity taking place in academia, open source communities, and elsewhere into techniques such as differential privacy. The goal of this research and ongoing work is to help individuals and organizations work collaboratively while preserving the anonymity of individual data points.

Exploiting the business value of data science doesnโ€t end with training a Machine Learning model โ€” in fact that is just the beginning. Data scientists want to maximize model performance while application developers want a deployment that builds repeatably and behaves predictably. Model serving smooths the transition from data science to applications in production. This talk will explain what model serving is, who should care, and show participants how to use the open source model-serving project, Seldon Core, to serve models on Kubernetes.

This session is told from a data scientist's point of view and documents building a model serving pipeline as a whole. This includes pain points of model serving such as clunky pipelines, but also celebrates the parts that work well, such as scalability within Kubernetes.

The audience will learn: the basics of model serving, why this is a relevant issue, how model serving offers relief for the data scientist/software engineer handoff, and know how to deploy their machine learning model with Seldon Core.

Responsible AI: Building ethical practices in your software development lifecycle
Advancements in AI are different than other technologies because of the pace of innovation, and its proximity to human intelligence - impacting us at a personal and societal level. The industry today is optimistic about the incredible potential for AI and other advanced technologies to empower people, widely benefiting current and future generations, thereby working for the common good.

However, nearly 9 out of 10 organizations across countries have encountered ethical issues resulting from the use of AI. As Microsoft, we recognize that these same technologies also raise important challenges that we need to address clearly, thoughtfully, and affirmatively. In this talk, I will present Microsoft's approach towards Responsible AI, talking about the six principles to develop technology responsibly.
This will be a focussed session where I will start with a demo of how, if not used properly, AI can lead to biases. Post that we will go into Microsoft's principles and how they lead to an efficient system - showcasing how the same demo will look after incorporating the Responsible AI Principles.

OpenShift being a fast-growing open source application platform, makes deploying and managing machine learning models very easy and convenient. Since the development of machine learning models is an iterative procedure, many data scientists prefer to store their model source code like their Jupyter notebooks in Git so that they can perform frequent updates to their models. These models are then deployed via OpenShift in a production environment. To obtain the most optimized model, it is necessary for the models to be continuously retrained and deployed. How can we efficiently manage this periodic retraining and deployment of these machine learning models?

In this talk you will learn how to leverage DevOps for managing machine learning models on OpenShift. With the help of CI/CD tools like Tekton pipelines, we can now extend version control for data science applications as well. You will walk away from this talk knowing how to:
1. Train a machine learning model on an example use case
2. Maintain ML model code in Git
3. Setup CI/CD pipeline for your ML application

Abstract

Although building ML models on small/ toy data-set is easy, most production-grade problems involve massive datasets which current ML practices don't scale to. In this talk, we cover how you can drastically increase the amount of data that your models can learn from using distributed data/ml pipes.

It can be difficult to figure out how to work with large data-sets (which do not fit in your RAM), even if you're already comfortable with ML libraries/ APIs within python. Many questions immediately come up: Which library should I use, and why? What's the difference between a 'map-reduce' and a 'task-graph'? What's a partial fit function, and what format does it expect the data in? Is it okay for my training data to have more features than observations? What's the appropriate machine learning model to use? And so on...

In this talk, we'll answer all those questions, and more!

We'll start by walking through the current distributed analytics (out-of-core learning) landscape in order to understand the pain-points and some solutions to this problem.

Here is a sketch of a system designed to achieve this goal (of building scalable ML models):

1. a way to stream instances
2. a way to extract features from instances
3. an incremental algorithm

Then we'll read a large dataset into Dask, Tensorflow (tf.data) & sklearn streaming, and immediately apply what we've learned about in last section. We'll move on to the model building process, including a discussion of which model is most appropriate for the task. We'll evaluate our model a few different ways, and then examine the model for greater insight into how the data is influencing its predictions. Finally, we'll practice this entire workflow on a new dataset, and end with a discussion of which parts of the process are worth tuning for improved performance.

Detailed Outline

1. Intro to out-of-core learning
2. Representing large datasets as instances
3. Transforming data (in batches) - live code [3-5]
4. Feature Engineering & Scaling
5. Building and evaluating a model (on entire datasets)
6. Practicing this workflow on another dataset
7. Benchmark other libraries/ for OOC learning
8. Questions and Answers

Key takeaway

By the end of the talk participants would know how to build petabyte scale ML models, beyond the shackles of conventional python libraries.

Participants would have a benchmarks and best case practices for building such ML models at scale.

Live audio transcription and other similar applications require stateful processing to support both multi-user sessions and dynamic scale-out. We can persist audio state with a Kafka kappa architecture, but that state must also be preserved across the OpenShift cluster boundary to user web clients. Fortunately, OpenShift's sticky sessions allow stateful sessions to be implemented without complicated custom configurations.

In this talk, Gage will explain how to convert your single user constrained application to support stateful sessions with any number of users. Using the power of OpenShift and Open Data Hub's data monitoring and streaming tools, a stateful architecture can be developed and managed easily. We will showcase a real-time audio transcription use case, including a Kafka streaming architecture, in a practical data science application.