README
¶
healthcare-federated-access-services
Purpose
The Global Alliance for Genomics and Health ("GA4GH") has launched an open standard for requesting and granting access to genomic datasets, known as the "GA4GH Passport". This allows different identity providers and data hosts to interact with each other, independent of their hosting platform and identity provider. For example, the owner of a genomic dataset hosted on Google Cloud (e.g. a national genomics institute) can grant access to a researcher with an organizational identity (e.g. an academic or corporate email address) via a GA4GH passport.
The GA4GH Passport specification is a technology to eliminate barriers between users and data, even in complex multi-cloud and hybrid-cloud environments, while still adhering to data consents and strict sharing policies between the parties involved.
Data Access Manager
This repository contains the Data Access Manager ("DAM"), which performs the role of a GA4GH Passport Clearinghouse.
The problem
Sensitive data is often organized in controlled-access datasets where only qualified individuals or organizations should have access. Data controllers must identify these data accessors ahead of time, and then configure their datasets to permit access. This manual and error-prone process slows down collaboration and can make some use-cases impossible.
The solution
GA4GH Passports are a standard way to securely communicate information between data controllers and data accessors. The Data Access Manager (DAM) enables data controllers to seamlessly leverage GA4GH passports to make their data accessible, but also secure.
DAM enables the translation of abstract qualifications (e.g. I am a physician, I am an academic researcher, etc) into platform-specific access management configurations (e.g. I can access this file, I can run this operation). Once an administrator configures DAM with policies describing how qualifications should translate into data access (e.g. academic researchers should have access to files A and B, but not C), verification of those qualifications and the resulting reconfiguration of underlying permissions will occur automatically as data access requests are received.
DAM evaluates identities against policies in real-time, which means data controllers do not need to have a relationship with data accessors – in fact, data controllers and accessors do not need to know one another exist prior to a transaction. DAM provides the option for data accessors to be billed directly for expenses associated with their requests, rather than those costs being incurred by the data controller. DAM is designed to work as a component within a broader data hosting platform, and also as a standalone service.
Identity Concentrator
This repository contains the Identity Concentrator ("IC"), which performs the role of a GA4GH Passport Broker
The problem
In order to access controlled-access datasets, data accessors must prove to data controllers that they have the qualifications required by the data controller. This is done by submitting an application to the data controller who manually reviews the information provided. If acceptable, the data controller adds the data accessor to a whitelist or other static access control mechanism. The data accessor must then use the specific identity (e.g. a Google Cloud credential) for which the access was granted. The data accessor must repeat this process for each dataset that they wish to work with. This results in data accessors accumulating many disparate identities, each specific to a different data controller.
The solution
The IC is an open-source service that securely combines identity qualifications (e.g. I am an academic researcher, I am a physician, I have taken ethics training XYZ, etc) collected from disparate sources into a single identity that can be used to access controlled-access datasets. Without the IC, data accessors must obtain and manage identities that are specific to a given data controller (e.g. a data controller hosting data on Google Cloud may have required data accessors to obtain Google Cloud credentials rather than using their existing corporate or academic credential).
Because data accessors often require access to data siloed across many locations, data accessors must shift between identities to obtain the data that they need. This makes running complex workflows that depend on data from diverse sources challenging and unreliable. With IC, data accessors (and the workbench platforms that they use) are able to combine relevant identities before executing a given workflow. This enables the workflow to leverage all data that the data accessor is permitted to access, regardless of how fragmented their identity qualifications may be. IC is designed to work as a component within a broader platform, but can also be deployed as a standalone service.
Some datasets will have visa requirements that can be collected from multiple sources, but need to be presented on one passport. The IC can combine lists of visas pertaining to one user from various visa sources.
For more information, visit:
- GA4GH Overview of Passports and GA4GH Researcher Identity Introduction.
- GA4GH Passport v1.0 full specification.
- GA4GH AAI OpenID Connect Profile v1.0 specification.
- GA4GH
Contributing to the repository
For information on how to contribute to the repository, see How to Contribute.
Notice
This is not an officially supported Google product.
How to Deploy
For information on how to deploy Federated Access, see How To Deploy Federated Access.
The deploy.bash script is designed to get a test environment up and running quickly and make it easy to develop services that use them in a non-sensitive environment.
When planning the next phase where these services need to be prepared for a production environment with live, sensitive data, the productionization documentation can be helpful.
More technical information about components and how to manage them:
Troubleshooting
See the how-to guide.
Configuration
For configuration examples, see deploy/config/dam-template and deploy/config/ic-template.
For more information, see IcConfig and DamConfig.
Test Personas
Test Personas are a means to create mock test users that are defined to hold a set of visas. The DAM can use test personas to verify that access privileges behave as expected for users with such visas. Each test persona reports an "access list" that describes the resources and roles their visas provide access to.
A playground environment includes a Test Persona Broker ("Persona Broker") that allows users to impersonate Test Personas. If the DAM and IC are both set up to trust a Persona Broker, then end to end tests and training can be conducted.
Note: Production deployments of DAM and IC should never be configured to trust Persona Brokers. However, production DAMs can still use Test Personas to verify access without allowing users to impersonate them.
APIs
For information about API endpoints available in Federated Access components, please refer to API documentation.
Bugs, feature requests and general feedback
Please consult the open issues, or file a new issue. Your feedback is appreciated!
Directories
¶
| Path | Synopsis |
|---|---|
|
apis
|
|
|
hydraapi
Package hydraapi contains models generate from https://raw.githubusercontent.com/ory/hydra/master/docs/api.swagger.json by github.com/go-swagger/go-swagger.
|
Package hydraapi contains models generate from https://raw.githubusercontent.com/ory/hydra/master/docs/api.swagger.json by github.com/go-swagger/go-swagger. |
|
gcp
|
|
|
dam
This package provides a single-host reverse proxy that rewrites bearer tokens in Authorization headers to be Google Cloud Platform access tokens.
|
This package provides a single-host reverse proxy that rewrites bearer tokens in Authorization headers to be Google Cloud Platform access tokens. |
|
dam_import
Binary dam_reset to reset the storage of a DAM
|
Binary dam_reset to reset the storage of a DAM |
|
damdemo
Binary damdemo is a demo of DAM.
|
Binary damdemo is a demo of DAM. |
|
ic
This package provides a single-host reverse proxy that rewrites bearer tokens in Authorization headers to be Google Cloud Platform access tokens.
|
This package provides a single-host reverse proxy that rewrites bearer tokens in Authorization headers to be Google Cloud Platform access tokens. |
|
ic_import
Binary ic_reset to reset the storage of an IC
|
Binary ic_reset to reset the storage of an IC |
|
icdemo
Binary icdemo is a demo of IC.
|
Binary icdemo is a demo of IC. |
|
personas
This package provides a persona broker service for offering a playground environment where users can log in and manage the system using personas.
|
This package provides a persona broker service for offering a playground environment where users can log in and manage the system using personas. |
|
lib
|
|
|
adapter
Package adapter allows the DAM to take actions.
|
Package adapter allows the DAM to take actions. |
|
auditlog
Package auditlog contains logging structs.
|
Package auditlog contains logging structs. |
|
auditlogsapi
Package auditlogsapi provides implementations of tokens API defined in /proto/auditlogs/
|
Package auditlogsapi provides implementations of tokens API defined in /proto/auditlogs/ |
|
auditlogsapi/itest
Binary itest is an integration test for the API with the Stackdriver.
|
Binary itest is an integration test for the API with the Stackdriver. |
|
auth
Package auth contains authorization check wrapper for handlers.
|
Package auth contains authorization check wrapper for handlers. |
|
cli
Package cli adds support for command line interfaces or micro-services to establish an access and/or refresh token via user participation.
|
Package cli adds support for command line interfaces or micro-services to establish an access and/or refresh token via user participation. |
|
clouds
Package clouds provides interfaces for accessing cloud APIs
|
Package clouds provides interfaces for accessing cloud APIs |
|
consentsapi
Package consentsapi contains a service manages user's remembered consent
|
Package consentsapi contains a service manages user's remembered consent |
|
dam
Package dam contains data access management service.
|
Package dam contains data access management service. |
|
dsstore
Package dsstore is a Datastore-based storage for DAM/IC.
|
Package dsstore is a Datastore-based storage for DAM/IC. |
|
dsstore/itest
Binary itest runs some code against Datastore.
|
Binary itest runs some code against Datastore. |
|
errutil
Package errutil contains helpers for error.
|
Package errutil contains helpers for error. |
|
faketokensapi
Package faketokensapi includes a mock server of token apis.
|
Package faketokensapi includes a mock server of token apis. |
|
ga4gh
Package ga4gh provides primitives for dealing with identities as described by the Global Alliance for Genomics and Healthcare's Data Use and Researcher Identity workstream.
|
Package ga4gh provides primitives for dealing with identities as described by the Global Alliance for Genomics and Healthcare's Data Use and Researcher Identity workstream. |
|
ga4gh/example
example is an example of how to use ga4gh package.
|
example is an example of how to use ga4gh package. |
|
globalflags
Package globalflags contains global flags of binary, eg.
|
Package globalflags contains global flags of binary, eg. |
|
grpcutil
Package grpcutil provides utilities to work with gRPC.
|
Package grpcutil provides utilities to work with gRPC. |
|
handlerfactory
Package handlerfactory allows creating HTTP handlers for services.
|
Package handlerfactory allows creating HTTP handlers for services. |
|
httputils
Package httputils contains utilities for handling HTTP requests.
|
Package httputils contains utilities for handling HTTP requests. |
|
hydra
Package hydra contains helpers for using hydra
|
Package hydra contains helpers for using hydra |
|
hydraproxy
Package hydraproxy contains a hydra proxy service to proxy request to hydra if needed.
|
Package hydraproxy contains a hydra proxy service to proxy request to hydra if needed. |
|
ic
Package ic is identity concentrator for GA4GH Passports.
|
Package ic is identity concentrator for GA4GH Passports. |
|
jsonutil
Package jsonutil contains helpers for working with JSON.
|
Package jsonutil contains helpers for working with JSON. |
|
kms
Package kms offers interfaces for providing encryption services and signing services.
|
Package kms offers interfaces for providing encryption services and signing services. |
|
kms/fakeencryption
Package fakeencryption is using for testing
|
Package fakeencryption is using for testing |
|
kms/gcpcrypt
Package gcpcrypt contains a client of GCP Cloud KMS symmetric encryption.
|
Package gcpcrypt contains a client of GCP Cloud KMS symmetric encryption. |
|
kms/gcpcrypt/manual_test
Binary gcpcrypt contains a symmetric encryption test run on real CloudKMS.
|
Binary gcpcrypt contains a symmetric encryption test run on real CloudKMS. |
|
kms/gcpsign
Package gcpsign contains a client of GCP Cloud KMS RSA256 asymmetric signning.
|
Package gcpsign contains a client of GCP Cloud KMS RSA256 asymmetric signning. |
|
kms/gcpsign/manual_test
Binary gcpsign contains a signning test run on real CloudKMS.
|
Binary gcpsign contains a signning test run on real CloudKMS. |
|
kms/localsign
Package localsign contains a jwt signer use jose/jwt.
|
Package localsign contains a jwt signer use jose/jwt. |
|
oathclients
Package oathclients contains clients endpoints and helpers related to client credentials.
|
Package oathclients contains clients endpoints and helpers related to client credentials. |
|
osenv
Package osenv provides utilities to read flag-like enviroment variables.
|
Package osenv provides utilities to read flag-like enviroment variables. |
|
permissions
Package permissions contains codes share between IC and DAM.
|
Package permissions contains codes share between IC and DAM. |
|
persona
Package persona provides a persona broker for use by clients.
|
Package persona provides a persona broker for use by clients. |
|
process
Package process is for background processes and listed at the ../processes endpoint.
|
Package process is for background processes and listed at the ../processes endpoint. |
|
processgc
Package processgc provices an Account Manager Garbage Collection.
|
Package processgc provices an Account Manager Garbage Collection. |
|
retry
Package retry includes config for backoff
|
Package retry includes config for backoff |
|
saw
Package saw abstracts interacting with certain aspects of Google Cloud Platform, such as creating service account keys and access tokens.
|
Package saw abstracts interacting with certain aspects of Google Cloud Platform, such as creating service account keys and access tokens. |
|
scim
Package scim implements a SCIM-like interface for group and user management.
|
Package scim implements a SCIM-like interface for group and user management. |
|
server
Package server provides a http server with request timeout and grateful shutdown.
|
Package server provides a http server with request timeout and grateful shutdown. |
|
serviceinfo
Package serviceinfo stores service runtime info.
|
Package serviceinfo stores service runtime info. |
|
srcutil
Package srcutil provides utilities for working with files under go module.
|
Package srcutil provides utilities for working with files under go module. |
|
storage
Package storage provides storage for IC and DAM.
|
Package storage provides storage for IC and DAM. |
|
strutil
Package strutil provides utility functions for working with strings.
|
Package strutil provides utility functions for working with strings. |
|
test
Package test contains test utility code shared between IC and DAM.
|
Package test contains test utility code shared between IC and DAM. |
|
test/debugutil
Package debugutil provides utilities for debugging.
|
Package debugutil provides utilities for debugging. |
|
test/fakegrpc
Package fakegrpc provides a fake gRPC client/server for testing purpose.
|
Package fakegrpc provides a fake gRPC client/server for testing purpose. |
|
test/fakehttp
Package fakehttp provides a fake HTTP server for tests that have dependencies using HTTP clients.
|
Package fakehttp provides a fake HTTP server for tests that have dependencies using HTTP clients. |
|
test/fakehydra
Package fakehydra contains fake hydra server for testing
|
Package fakehydra contains fake hydra server for testing |
|
test/fakeiam
Package fakeiam provides a fake implementation for IAM services: IAM Admin IAM Credendtials
|
Package fakeiam provides a fake implementation for IAM services: IAM Admin IAM Credendtials |
|
test/fakeissuer
Package fakeissuer provides a minimal fake OIDC issuer for testing purpose.
|
Package fakeissuer provides a minimal fake OIDC issuer for testing purpose. |
|
test/fakeoidcissuer
Package fakeoidcissuer contains a fake OIDC issuer which can use in go-oidc provider.
|
Package fakeoidcissuer contains a fake OIDC issuer which can use in go-oidc provider. |
|
test/fakesdl
Package fakesdl provides a fake for Stackdriver Logging.
|
Package fakesdl provides a fake for Stackdriver Logging. |
|
test/fakestore
Package fakestore provides a fake in-mem storage.
|
Package fakestore provides a fake in-mem storage. |
|
test/httptestclient
Package httptestclient contains a http client request to the given http handler.
|
Package httptestclient contains a http client request to the given http handler. |
|
test/muxtest
Package muxtest contains test helpers for testing endpoints client credentials requirement.
|
Package muxtest contains test helpers for testing endpoints client credentials requirement. |
|
test/testhttp
Package testhttp contains helpers for test http request.
|
Package testhttp contains helpers for test http request. |
|
testkeys
Package testkeys provides private/public RSA keys for testing.
|
Package testkeys provides private/public RSA keys for testing. |
|
timeutil
Package timeutil provides utilities for working with time related objects.
|
Package timeutil provides utilities for working with time related objects. |
|
tokensapi
Package tokensapi provides implementations of tokens API defined in /proto/tokens/
|
Package tokensapi provides implementations of tokens API defined in /proto/tokens/ |
|
translator
Package translator provides implementations of the ga4gh.Translator interface for translating between different identity providers and GA4GH identities.
|
Package translator provides implementations of the ga4gh.Translator interface for translating between different identity providers and GA4GH identities. |
|
validator
Package validator contains implementations of the ga4gh.Validator interface.
|
Package validator contains implementations of the ga4gh.Validator interface. |
|
verifier
Package verifier provides a token verifier.
|
Package verifier provides a token verifier. |
|
proto
|
|
|
auditlogs/v0
Package v1 provides protocol buffer for AuditLogs API.
|
Package v1 provides protocol buffer for AuditLogs API. |
|
common/v1
Package com defines messages shared between IC and DAM server.
|
Package com defines messages shared between IC and DAM server. |
|
consents/v1
Package v1 provides protocol buffer versions of Remembered Consents API for listing and revoking Remembered consents.
|
Package v1 provides protocol buffer versions of Remembered Consents API for listing and revoking Remembered consents. |
|
dam/v1
Package dam provides protocol buffer versions of the DAM API, allowing end points to receive requests and returns responses using these messages.
|
Package dam provides protocol buffer versions of the DAM API, allowing end points to receive requests and returns responses using these messages. |
|
ic/v1
Package v1 ic provides protocol buffer versions of the DAM API, allowing end points to receive requests and returns responses using these messages.
|
Package v1 ic provides protocol buffer versions of the DAM API, allowing end points to receive requests and returns responses using these messages. |
|
process/v1
Package process provides protocol buffers for background process state.
|
Package process provides protocol buffers for background process state. |
|
scim/v2
Package scim provides protocol buffer versions of the SCIM 2.0 User objects and related operations, allowing endpoints to receive requests and return responses using these messages.
|
Package scim provides protocol buffer versions of the SCIM 2.0 User objects and related operations, allowing endpoints to receive requests and return responses using these messages. |
|
store/consents
Package consents provides Remembered Consents PB for storage
|
Package consents provides Remembered Consents PB for storage |
|
store/tokens
Package tokens provides object in storage for tokens.
|
Package tokens provides object in storage for tokens. |
|
tokens/v1
Package v1 tokens provides protocol buffer versions of tokens API.
|
Package v1 tokens provides protocol buffer versions of tokens API. |