oscar

README

OSCAR - Open Source Serverless Computing for Data-Processing Applications

Introduction

OSCAR is an open-source platform to support the Functions as a Service (FaaS) computing model for file-processing applications. It can be automatically deployed on multi-Clouds in order to create highly-parallel event-driven file-processing serverless applications that execute on customized runtime environments provided by Docker containers than run on an elastic Kubernetes cluster.

Deploy   |   Documentation  

Overview

• About OSCAR
• Components
• Licensing
• Acknowledgements

Why OSCAR?

FaaS platforms are typically oriented to the execution of short-lived functions, coded in a certain programming language, in response to events. Scientific application can greatly benefit from this event-driven computing paradigm in order to trigger on demand the execution of a resource-intensive application that requires processing a certain file that was just uploaded to a storage service. This requires additional support for the execution of generic applications in existing open-source FaaS frameworks.

To this aim, OSCAR supports the High Throughput Computing Programming Model initially introduced by the SCAR framework, to create highly-parallel event-driven file-processing serverless applications that execute on customized runtime environments provided by Docker containers run on AWS Lambda.

With OSCAR, users upload files to a data storage back-end and this automatically triggers the execution of parallel invocations to a service responsible for processing each file. Output files are delivered into a data storage back-end for the convenience of the user. The user only specifies the Docker image and the script to be executed, inside a container created out of that image, in order to process a file that will be automatically made available to the container. The deployment of the computing infrastructure and its scalability is abstracted away from the user.

Components

OSCAR runs on an elastic Kubernetes cluster that is deployed using:

• EC3, an open-source tool to deploy compute clusters that can horizontally scale in terms of number of nodes with multiple plugins.
• IM, an open-source virtual infrastructure provisioning tool for multi-Clouds.
• CLUES, an elasticity manager that horizontally scales in and out the number of nodes of the Kubernetes cluster according to the workload.

The following components are deployed inside the Kubernetes cluster in order to support the OSCAR platform:

• MinIO, a high performance distributed object storage server that provides an API compatible with S3.
• OpenFaaS, a FaaS platform that allows creating functions executed via HTTP requests.
• OSCAR, the main application, responsible for the management of the services and the integration of the different components to support event-driven serverless computing for file processing. It includes a web-based GUI aimed at end users to facilitate interaction with OSCAR.

As external storage providers, the following services can be used:

• External MinIO servers, which may be in clusters other than the platform.
• Amazon S3, the Amazon's object storage service that offers industry-leading scalability, data availability, security, and performance in the public Cloud.
• Onedata, the global data access solution for science used in the EGI Federated Cloud.

OSCAR has also been integrated with the EC3 Portal available in the EGI Applications on Demand service to deploy the platform on the EGI Federated Cloud resources.

Further information is available in the documentation.

Licensing

OSCAR is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Acknowledgements

This development is partially funded by the EGI Strategic and Innovation Fund.

Please acknowledge the use of OSCAR by citing the following scientific publication (preprint available):

A. Pérez, S. Risco, D. M. Naranjo, M. Caballer, and G. Moltó, “Serverless Computing for Event-Driven Data Processing Applications,” in 2019 IEEE International Conference on Cloud Computing (CLOUD 2019), 2019.