README
¶
Go实现的模型推理和API部署框架
深度学习模型在部署时通常以云方式部署，通过API对外提供推理服务。这个框架提供了部署API时的基本架构组件，实现了几个目标：
- API处理模块与模型推理模块解耦，降低高并发造成的网络和计算阻塞风险
- API处理模块与模型推理模块可进行分布式部署，均可实现横向扩展
- 使用Go语言实现，提高执行效率，简化部署和运维
- 业务逻辑使用callback方式调用，隐藏通用逻辑，开发时只需关注业务逻辑
其他功能：
- 服务端配置使用Yaml，分布式部署时可进行针对性配置
- API验签支持SHA256和SM2算法
- 模型示例
分布式部署架构
开发文档
Star History
Directories
¶
| Path | Synopsis |
|---|---|
|
Package to provide command-line settings
|
Package to provide command-line settings |
|
Package provides some helping funcs, suchs as redis-related and settings parsing
|
Package provides some helping funcs, suchs as redis-related and settings parsing |
|
Package to run a Http server to provide API service the http server is implemented by valyala/fasthttp
|
Package to run a Http server to provide API service the http server is implemented by valyala/fasthttp |
|
Package to run dispatcher server of model inference service the inference service is implemented by goroutine, created when request coming.
|
Package to run dispatcher server of model inference service the inference service is implemented by goroutine, created when request coming. |
|
Package to provide base Model interface definition
|
Package to provide base Model interface definition |
Click to show internal directories.
Click to hide internal directories.