data_integration

package module
v0.0.0-...-1b7ffe2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 17, 2021 License: MIT Imports: 1 Imported by: 0

README

Data Integration API

Fetch data from a Legacy API and serve a CRUD

Instructions

  • Install Docker and docker-compose;
  • In your terminal, run docker-compose up;
  • Wait until all containers are ready;

You will know that the API will be ready to receive requests when you see something like this:

data-integration-container | 2021/03/17 22:15:28 Connected!
data-integration-container | [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.
data-integration-container | 
data-integration-container | [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
data-integration-container |  - using env:	export GIN_MODE=release
data-integration-container |  - using code:	gin.SetMode(gin.ReleaseMode)
data-integration-container | 
data-integration-container | [GIN-debug] GET    /swagger/*any             --> github.com/swaggo/gin-swagger.CustomWrapHandler.func1 (3 handlers)
data-integration-container | [GIN-debug] POST   /api/v1/auth/login        --> github.com/klasrak/data-integration/api/handlers.AuthHandler.Login-fm (3 handlers)
data-integration-container | [GIN-debug] POST   /api/v1/auth/logout       --> github.com/klasrak/data-integration/api/handlers.AuthHandler.Logout-fm (3 handlers)
data-integration-container | [GIN-debug] POST   /api/v1/auth/refresh-token --> github.com/klasrak/data-integration/api/handlers.AuthHandler.Refresh-fm (3 handlers)
data-integration-container | [GIN-debug] GET    /api/v1/users/get         --> github.com/klasrak/data-integration/api/handlers.UserHandler.Find-fm (4 handlers)
data-integration-container | [GIN-debug] GET    /api/v1/users/get/:email  --> github.com/klasrak/data-integration/api/handlers.UserHandler.FindByEmail-fm (4 handlers)
data-integration-container | [GIN-debug] GET    /api/v1/negativations/fetch --> github.com/klasrak/data-integration/api/handlers.Handler.Fetch-fm (4 handlers)
data-integration-container | [GIN-debug] GET    /api/v1/negativations/get --> github.com/klasrak/data-integration/api/handlers.Handler.GetAll-fm (4 handlers)
data-integration-container | [GIN-debug] GET    /api/v1/negativations/get/:customerDocument --> github.com/klasrak/data-integration/api/handlers.Handler.Get-fm (4 handlers)
data-integration-container | [GIN-debug] GET    /api/v1/negativations/get-id/:id --> github.com/klasrak/data-integration/api/handlers.Handler.GetByID-fm (4 handlers)
data-integration-container | [GIN-debug] POST   /api/v1/negativations/create --> github.com/klasrak/data-integration/api/handlers.Handler.Create-fm (4 handlers)
data-integration-container | [GIN-debug] PUT    /api/v1/negativations/update/:id --> github.com/klasrak/data-integration/api/handlers.Handler.Update-fm (4 handlers)
data-integration-container | [GIN-debug] DELETE /api/v1/negativations/delete/:id --> github.com/klasrak/data-integration/api/handlers.Handler.Delete-fm (4 handlers)
data-integration-container | [GIN-debug] Listening and serving HTTP on :8080

Docs

With the API running, you can access http://localhost:8080/swagger/index.html to see the documentation.

Swagger

If you use the Insomnia REST Client, you can import the schema from here.

Getting started

All routes, with the exception of login, require the user to be authenticated, so first log in with the pre-registered admin user.

Email: admin@mail.com
Password: abcd1234

$ curl --location --request POST 'http://localhost:8080/api/v1/auth/login' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "email": "admin@mail.com",
    "password": "abcd1234"
  }'

Once logged in, the answer will be an access token valid for 1 hour and a refresh token.

{
  "accessToken": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NVVUlEIjoiNjdjYjU3MjEtYTE5MC00NzVkLWEwNjgtMDEyMzhkMWE4OWI2IiwiZW1haWwiOiJhZG1pbkBtYWlsLmNvbSIsImV4cCI6MTYxNjAyNDM3OSwidXNlcklEIjoiNjA1MjZiOWMwZTEzNzIzMjEyMzEwYjlkIn0.8cL6Lk20l8Iw8CNsIHNirKleO8jUMr4JAHx7O9pa90s",
  "refreshToken": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImFkbWluQG1haWwuY29tIiwiZXhwIjoxNjE2NjI1NTc5LCJyZWZyZXNoVVVJRCI6IjY3Y2I1NzIxLWExOTAtNDc1ZC1hMDY4LTAxMjM4ZDFhODliNisrNjA1MjZiOWMwZTEzNzIzMjEyMzEwYjlkIiwidXNlcklEIjoiNjA1MjZiOWMwZTEzNzIzMjEyMzEwYjlkIn0.esm9rdSdQ1Q-b50VCmSekLdeGPHmEMggaVKsHp4EyPM"
}

To integrate the data from the Legacy API, you must call the endpoint /negativations/fetch. Remember to send the Authorization header with the Bearer Token

$ curl --location --request GET 'http://localhost:8080/api/v1/negativations/fetch' \
  --header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NVVUlEIjoiMGNjOTc3NGUtNGYwZi00ODFmLThlZGItYTgzM2E0MWVlOWI3IiwiZW1haWwiOiJhZG1pbkBtYWlsLmNvbSIsImV4cCI6MTYxNjAyNjEzOCwidXNlcklEIjoiNjA1MjZiOWMwZTEzNzIzMjEyMzEwYjlkIn0.plv7yZSNgmt5ofvnAyuBo3j8BMgjlABWAYc6AwxmxcY'

The legacy API will be called, and the data returned will be saved to a MongoDB instance only the first time you call.

[
    {
        "companyDocument": "59291534000167",
        "companyName": "ABC S.A.",
        "customerDocument": "51537476467",
        "value": 1235.23,
        "contract": "bc063153-fb9e-4334-9a6c-0d069a42065b",
        "debtDate": "2015-11-13T20:32:51-03:00",
        "inclusionDate": "2020-11-13T20:32:51-03:00"
    },
    {
        "companyDocument": "77723018000146",
        "companyName": "123 S.A.",
        "customerDocument": "51537476467",
        "value": 400,
        "contract": "5f206825-3cfe-412f-8302-cc1b24a179b0",
        "debtDate": "2015-10-12T20:32:51-03:00",
        "inclusionDate": "2020-10-12T20:32:51-03:00"
    },
    {
        "companyDocument": "04843574000182",
        "companyName": "DBZ S.A.",
        "customerDocument": "26658236674",
        "value": 59.99,
        "contract": "3132f136-3889-4efb-bf92-e1efbb3fe15e",
        "debtDate": "2015-09-11T20:32:51-03:00",
        "inclusionDate": "2020-09-11T20:32:51-03:00"
    },
    {
        "companyDocument": "23993551000107",
        "companyName": "XPTO S.A.",
        "customerDocument": "62824334010",
        "value": 230.5,
        "contract": "8b441dbb-3bb4-4fc9-9b46-bdaad00a7a98",
        "debtDate": "2015-08-10T20:32:51-03:00",
        "inclusionDate": "2020-08-10T20:32:51-03:00"
    },
    {
        "companyDocument": "70170935000100",
        "companyName": "ASD S.A.",
        "customerDocument": "25124543043",
        "value": 10340.67,
        "contract": "d6628a0e-d4dd-4f14-8591-2ddc7f1bbeff",
        "debtDate": "2015-07-09T20:32:51-03:00",
        "inclusionDate": "2020-07-09T20:32:51-03:00"
    }
]

If you call the endpoint again and no new data is returned to be inserted, an error will be returned.

{
    "statusCode": 400,
    "message": "legacy API data already fetched and there is no new data to save"
}

From here, you can perform any CRUD operation. Take a look at the documentation for more information.

Test

There are mocks for the repository layer, and tests implemented on the internal and handler layer.

To run the tests, simply run the command go test ./... at the root of the project.

How it works

Folder structure:

.
├── api
│   ├── handlers
│   │   ├── auth-handler.go
│   │   ├── handler.go
│   │   ├── negativation-handler.go
│   │   ├── negativation-handler_test.go
│   │   └── users-handler.go
│   ├── helpers
│   │   ├── dto.go
│   │   └── error.go
│   ├── middlewares
│   │   └── token.go
│   └── routes
│       ├── auth-routes.go
│       ├── negativation-routes.go
│       └── users-routes.go
├── api-legacy
│   ├── Dockerfile
│   ├── index.js
│   ├── negativacoes.json
│   ├── package.json
│   └── package-lock.json
├── auth
│   └── auth.go
├── auth.go
├── cmd
│   ├── data-integration
│   │   └── main.go
│   └── server
│       ├── routes.go
│       └── server.go
├── config
│   ├── env.go
│   └── Insomnia_2021-03-17
├── docker-compose.yml
├── Dockerfile
├── docs
│   ├── docs.go
│   ├── swagger.json
│   └── swagger.yaml
├── go.mod
├── go.sum
├── internal
│   ├── converter
│   │   └── converter.go
│   └── cryptography
│       ├── cryptography.go
│       └── cryptography_test.go
├── jwt
│   └── jwt.go
├── LICENSE
├── mocks
│   ├── auth-repository.go
│   ├── http.go
│   ├── negativation-repository.go
│   └── user-repository.go
├── mongo
│   └── mongo.go
├── negativation.go
├── README.md
├── redis
│   └── redis.go
├── repositories
│   ├── auth-repository.go
│   ├── negativation-repository.go
│   └── user-repository.go
├── user.go
└── utils
    └── utils.go
Legacy API

The endpoint to consume the legacy API is made available through a json-server, in a small Node project inside api-legacy folder.

const jsonServer = require('json-server')
const server = jsonServer.create()
const router = jsonServer.router('negativacoes.json')
const middlewares = jsonServer.defaults()

server.use(middlewares)
// CORS
server.set('access-control-allow-origin', '*');
server.set('access-control-allow-methods', '*');
server.set('access-control-allow-headers', '*');
// -----
server.use(router)

server.listen(3333, () => {
    console.log('Server listening...')
})
Docker

All services, APIs, including MongoDB and Redis, are orchestrated using docker-compose.

TODO list

  • WRITE MORE TESTS!!!!!
  • Add a cache layer
  • Encryption before saving data. The package is ready, just integrate with the negativation repository.
  • Logout and revoke active tokens
  • Endpoint to refresh access token with refreshToken

Purpose of the project

This project was (and will continue to be) developed for the purpose of studies. Any feedback or contribution, positive or not, will be very welcome and will be taken into consideration.

License

This project follows MIT License.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AccessDetails 

type AccessDetails struct {
	TokenUUID string
	UserID    string
	Email     string
}

AccessDetails represents AccessDetails domain type

type Negativation 

type Negativation struct {
	ID               string    `json:"id,omitempty" bson:"_id,omitempty"`
	CompanyDocument  string    `json:"companyDocument,omitempty" bson:"companyDocument,omitempty"`
	CompanyName      string    `json:"companyName,omitempty" bson:"companyName,omitempty"`
	CustomerDocument string    `json:"customerDocument,omitempty" bson:"customerDocument,omitempty"`
	Value            float64   `json:"value,omitempty" bson:"value,omitempty"`
	Contract         string    `json:"contract" bson:"contract"`
	DebtDate         time.Time `json:"debtDate,omitempty" bson:"debtDate,omitempty"`
	InclusionDate    time.Time `json:"inclusionDate,omitempty" bson:"inclusionDate,omitempty"`
}

Negativation represents Navigation domain type

type TokenDetails 

type TokenDetails struct {
	AccessToken  string
	RefreshToken string
	TokenUUID    string
	RefreshUUID  string
	AtExpires    int64
	RtExpires    int64
}

TokenDetails represents TokenDetails domain type

type User 

type User struct {
	ID       string `json:"id,omitempty" bson:"_id,omitempty"`
	Email    string `json:"email" bson:"email"`
	Password string `json:"password" bson:"password"`
}

User represents User domain type

func (*User) InvalidPassword 

func (u *User) InvalidPassword(password string) bool

InvalidPassword returns true if the given password does not match the hash.

Source Files

View all Source files

Directories

Path Synopsis
api
handlers
helpers
middlewares
routes
cmd
data-integration
server
internal
converter
cryptography

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.