csvutil

README

csvutil

대용량 데이터를 csv파일 형태로 다운로드하는 라이브러리

Requirement

---

• 라이브러리
  • gin-gonic : https://github.com/gin-gonic/gin
  • gorm : https://github.com/go-gorm/gorm
• 데이터베이스
  • postgresql

Installation

---

go get github.com/innodep-tms/csvutil

Usage

---

일반 사용 예시 (gin-gonic handler)

• csv 태그를 이용하여 테이블 라벨을 지정할 수 있다.

package handler

import (
    "fmt"

    "github.com/gin-gonic/gin"
	"gorm.io/gorm"
    "github.com/innodep-tms/csvutil"
)

func HandlerFunc(c *gin.Context) {
    db, _ := gorm.Open()

    err := csvutil.TransferCSVFileChunked[data](c, db, "query", "filename", 1000)

    if err != nil {
        fmt.Println(err)
    }
}

type data struct {
    Name string `gorm:"column:name" csv:"이름"
    Value int `gorm:"column:value" csv:"값"
}

Documentation

Index

• func TransferCSVFileChunked[T any](ginContext *gin.Context, dbconn *gorm.DB, query, filename string, ...) error
• func TransferChunk[T any](chunker *Chunker[T], cursor *Cursor[T]) error
• type Chunker
  • func NewChunker[T any](ginContext *gin.Context) *Chunker[T]
  • func (chunker *Chunker[T]) ResetWriter()
  • func (chunker *Chunker[T]) SetHeader(filename string)
  • func (chunker *Chunker[T]) WriteChunk(data []T) error
  • func (chunker *Chunker[T]) WriteCsvLabel()
• type Cursor
  • func NewCursor[T any](dbconn *gorm.DB, query string, fetchSize int) (*Cursor[T], error)
  • func (cursor *Cursor[T]) Close() error
  • func (cursor *Cursor[T]) FetchCursor() ([]T, error)

Functions

func TransferCSVFileChunked

func TransferCSVFileChunked[T any](
	ginContext *gin.Context, dbconn *gorm.DB,
	query, filename string,
	chunkSize int,
) error

TransferCSVFileChunked is a function that responses a large csv file . The function works as follows: 1. Initializes the chunker and cursor. 2. Writes the header and the label of the CSV file. 3. Fetches the data from the cursor and writes the data in chunks. 4. Resets the writer. Need to gin.Context and gorm.DB to use this function. The fetchSize is the number of rows to be fetched at a time.

func TransferChunk

func TransferChunk[T any](chunker *Chunker[T], cursor *Cursor[T]) error

TransferChunk is a function that works as follows: 1. Fetches the data from the cursor. 2. Writes the data in chunks. 3. Resets the writer. Need to Chunker and Cursor to use this function.

Types

type Chunker

type Chunker[T any] struct {
	GinCtx *gin.Context
	Writer *csv.Writer
}

func NewChunker

func NewChunker[T any](ginContext *gin.Context) *Chunker[T]

func (*Chunker[T]) ResetWriter

func (chunker *Chunker[T]) ResetWriter()

ResetWriter resets the csv writer.

func (*Chunker[T]) SetHeader

func (chunker *Chunker[T]) SetHeader(filename string)

SetHeader sets the response header for the CSV file. The filename is the name of the file to be downloaded. The default encoding is UTF-8. Transfer-Encoding is set to chunked.

func (*Chunker[T]) WriteChunk

func (chunker *Chunker[T]) WriteChunk(data []T) error

WriteChunk writes the data in chunks.

func (*Chunker[T]) WriteCsvLabel

func (chunker *Chunker[T]) WriteCsvLabel()

WriteCsvLabel writes the Label of the CSV data.

type Cursor

type Cursor[T any] struct {
	DBconn    *gorm.DB
	Query     string
	FetchSize int
}

func NewCursor

func NewCursor[T any](dbconn *gorm.DB, query string, fetchSize int) (*Cursor[T], error)

func (*Cursor[T]) Close

func (cursor *Cursor[T]) Close() error

func (*Cursor[T]) FetchCursor

func (cursor *Cursor[T]) FetchCursor() ([]T, error)