AIStore TensorFlow Integration
Experimental project to provide TensorFlow-native AIS dataset (AisDataset
) and
associated data loaders. The objective is two-fold:
- allow Python developers and researchers to run existing TF-based models with almost no modifications
- utilize AIStore on the backend using the code that looks as follows:
conversions = [Rename(img="jpeg;png"), Decode("img"), Rotate("img"), Resize("img", (224, 244))]
selections = [Select("img"), Select("cls")]
ais = AisDataset(lpr-imagenet, http://ais-gateway-url, conversions, selections)
train_dataset = ais.load("train-{0..9999}.tar", num-workers=64)
This repository provides for deploying custom ETL containers on AIStore, with subsequent user-defined
extraction, transformation, and loading in parallel, on the fly and/or offline, local to the user data.
Please also see the main AIStore repository,
AIStore documentation, and
AIStore and ETL videos.