ligma

README

Ligma Go Module

Index

• Ligma Go Module
  • Overview
  • Install
  • Usage
    • Documentation
    • Examples
  • See also
  • Contributing to the project
    • Via forking
    • Request developer access
  • Authors

Overview

A go module of a tool, that used to be on my website and called ligma scraper. It connects to https://www.sigmaaldrich.com allowing you to search for a chemical and pull safety- and other relevant information off their site, in order to make labelling chemicals in the hobby lab less tedious.

Install

go get gitlab.com/ligmascraper/ligma

Usage

Documentation

https://pkg.go.dev/gitlab.com/ligmascraper/ligma

Examples

See https://gitlab.com/ligmascraper/ligmascraper-cli/ for an use case, or just in ´examples´

See also

https://pubchem.ncbi.nlm.nih.gov/ghs/

Contributing to the project

Via forking

• create an issue and discuss (optional, but encouraged)
• fork the project
• Make changes
• submit merge request

Request developer access

• Request access
• create an issue and discuss (optional, but encouraged)
• Create a new branch
• Make changes
• Submit merge request

Authors

Conceived and built by Holger M Mürk

Documentation

Index

• Variables
• func DecodeHWords(hcodes []string) string
• func DecodePWords(pcodes []string) string
• func FetchPageReader(url string) (io.ReadCloser, error)
• func FetchSearchResultJSON(query string) ([]byte, error)
• func ParseHCodes(input string) []string
• func ParsePCodes(input string) []string
• type ChemData
  • func ProcessRawHTMLBody(r io.ReadCloser) (output ChemData, err error)
  • func Scrape(url string) (output ChemData, err error)
  • func (a ChemData) PrintConsolidatedHP()
  • func (a ChemData) PrintHStatements()
  • func (a ChemData) PrintName()
  • func (a ChemData) PrintPStatements()
  • func (a ChemData) PrintParams()
  • func (a ChemData) PrintPictograms()
• type Chemical
• type Product
• type SearchRequestBody
• type SearchResult
  • func ProcessSearchResultJSON(json []byte) (output SearchResult)
  • func Search(query string) (SearchResult, error)
• type SearchVariables

Variables

var HazardStatements = map[string]string{}/* 104 elements not displayed */

var Pictograms = map[string]string{
	"GHS01": "GHS01 - Explosive",
	"GHS02": "GHS02 - Flammable",
	"GHS03": "GHS03 - Oxidizer",
	"GHS04": "GHS04 - Pressured Gas",
	"GHS05": "GHS05 - Corrosive",
	"GHS06": "GHS06 - Acute Toxicity",
	"GHS07": "GHS07 - Exclamation Mark",
	"GHS08": "GHS08 - Health Hazard",
	"GHS09": "GHS09 - Environment",
}

Pictograms text descriptions of the standard ghs pictograms

var PrecautionaryStatements = map[string]string{}/* 136 elements not displayed */

var SearchBody = SearchRequestBody{
	OperationName: "ProductSearch",
	Variables: SearchVariables{
		SearchTerm:     "",
		Page:           1,
		Group:          "substance",
		SelectedFacets: nil,
		Sort:           "relevance",
		Type:           "PRODUCT",
	},
	Query: "query ProductSearch($searchTerm: String, $page: Int!, $sort: Sort, $group: ProductSearchGroup, $selectedFacets: [FacetInput!], $type: ProductSearchType, $catalogType: CatalogType, $orgId: String, $region: String, $facetSet: [String], $filter: String) {\r\n  getProductSearchResults(input: {searchTerm: $searchTerm, pagination: {page: $page}, sort: $sort, group: $group, facets: $selectedFacets, type: $type, catalogType: $catalogType, orgId: $orgId, region: $region, facetSet: $facetSet, filter: $filter}) {\r\n    ...ProductSearchFields\r\n    __typename\r\n  }\r\n}\r\n\r\nfragment ProductSearchFields on ProductSearchResults {\r\n  metadata {\r\n    itemCount\r\n    setsCount\r\n    page\r\n    perPage\r\n    numPages\r\n    redirect\r\n    __typename\r\n  }\r\n  items {\r\n    ... on Substance {\r\n      ...SubstanceFields\r\n      __typename\r\n    }\r\n    ... on Product {\r\n      ...SubstanceProductFields\r\n      __typename\r\n    }\r\n    __typename\r\n  }\r\n  facets {\r\n    key\r\n    numToDisplay\r\n    isHidden\r\n    isCollapsed\r\n    multiSelect\r\n    prefix\r\n    options {\r\n      value\r\n      count\r\n      __typename\r\n    }\r\n    __typename\r\n  }\r\n  didYouMeanTerms {\r\n    term\r\n    count\r\n    __typename\r\n  }\r\n  __typename\r\n}\r\n\r\nfragment SubstanceFields on Substance {\r\n  _id\r\n  id\r\n  name\r\n  synonyms\r\n  empiricalFormula\r\n  linearFormula\r\n  molecularWeight\r\n  aliases {\r\n    key\r\n    label\r\n    value\r\n    __typename\r\n  }\r\n  images {\r\n    sequence\r\n    altText\r\n    smallUrl\r\n    mediumUrl\r\n    largeUrl\r\n    brandKey\r\n    productKey\r\n    label\r\n    videoUrl\r\n    __typename\r\n  }\r\n  casNumber\r\n  products {\r\n    ...SubstanceProductFields\r\n    __typename\r\n  }\r\n  __typename\r\n}\r\n\r\nfragment SubstanceProductFields on Product {\r\n  name\r\n  displaySellerName\r\n  productNumber\r\n  productKey\r\n  isSial\r\n  isMarketplace\r\n  marketplaceSellerId\r\n  marketplaceOfferId\r\n  cardCategory\r\n  cardAttribute {\r\n    citationCount\r\n    application\r\n    __typename\r\n  }\r\n  substance {\r\n    id\r\n    __typename\r\n  }\r\n  casNumber\r\n  attributes {\r\n    key\r\n    label\r\n    values\r\n    __typename\r\n  }\r\n  speciesReactivity\r\n  brand {\r\n    key\r\n    erpKey\r\n    name\r\n    color\r\n    __typename\r\n  }\r\n  images {\r\n    altText\r\n    smallUrl\r\n    mediumUrl\r\n    largeUrl\r\n    __typename\r\n  }}",
}

var SearchRequestHeaders = map[string]string{
	"User-Agent":      "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:103.0) Gecko/20100101 Firefox/103.0",
	"Host":            " www.sigmaaldrich.com",
	"Accept":          " */*",
	"Accept-Language": "en-US,en;q=0.5",
	"Accept-Encoding": "none",
	"Content-Type":    "application/json",
	"Referer":         "https://www.sigmaaldrich.com/EE/en/search/",
	"x-gql-country":   "EE",
	"x-gql-language":  "en",
}

SearchRequestHeaders the minimal headers the api seemed to work with

Functions

func DecodeHWords

func DecodeHWords(hcodes []string) string

DecodeHWords Creates a consolidated full text of all the GHS words

func DecodePWords

func DecodePWords(pcodes []string) string

DecodePWords Creates a consolidated full text of all the GHS words

func FetchPageReader

func FetchPageReader(url string) (io.ReadCloser, error)

FetchPageReader fetches the whole page

func FetchSearchResultJSON

func FetchSearchResultJSON(query string) ([]byte, error)

FetchSearchResultJSON fetches the appropriate results json from Sigma API

func ParseHCodes

func ParseHCodes(input string) []string

ParseHCodes splits the string gotten from sigma into usable array

func ParsePCodes

func ParsePCodes(input string) []string

ParsePCodes splits the string gotten from sigma into usable array

Types

type ChemData

type ChemData struct {
	Name        string   `json:"name"`
	CAS         string   `json:"CAS"`
	MW          string   `json:"MW"`
	Mp          string   `json:"mp"`
	Bp          string   `json:"bp"`
	Signalword  string   `json:"signalword"`
	Pictograms  []string `json:"pictograms"`
	Hcodes      []string `json:"hcodes"`
	Hstatements string   `json:"hstatements"`
	Pcodes      []string `json:"pcodes"`
	Pstatements string   `json:"pstatements"`
	SMILES      string   `json:"SMILES"`
}

ChemData the structure of returned data

func ProcessRawHTMLBody

func ProcessRawHTMLBody(r io.ReadCloser) (output ChemData, err error)

ProcessRawHTMLBody process a fetched reader to extract data

func Scrape

func Scrape(url string) (output ChemData, err error)

Scrape pulls relevant data off the url

func (ChemData) PrintConsolidatedHP

func (a ChemData) PrintConsolidatedHP()

PrintConsolidatedHP prints out the

func (ChemData) PrintHStatements

func (a ChemData) PrintHStatements()

PrintHStatements prints out the consolidated gazard statements

func (ChemData) PrintName

func (a ChemData) PrintName()

PrintName prints the name of the chemical to stdout

func (ChemData) PrintPStatements

func (a ChemData) PrintPStatements()

PrintPStatements prints out the consolidated precautionary statements

func (ChemData) PrintParams

func (a ChemData) PrintParams()

PrintParams prints out the fetched parameters

func (ChemData) PrintPictograms

func (a ChemData) PrintPictograms()

PrintPictograms prints out the ghs pictogram descriptions

type Chemical

type Chemical struct {
	Name     string    `json:"name"`
	Products []Product `json:"products"`
}

Chemical a substance from results

type Product

type Product struct {
	Name        string `json:"name"`
	Url         string `json:"url"`
	ProductKey  string `json:"productKey"`
	Description string `json:"description"`
}

Product a product page results, use this url to scrape

type SearchRequestBody

type SearchRequestBody struct {
	OperationName string          `json:"operationName"`
	Variables     SearchVariables `json:"variables"`
	Query         string          `json:"query"`
}

type SearchResult

type SearchResult struct {
	SearchQuery string     `json:"searchQuery"`
	SetsCount   int64      `json:"setsCount"`
	Chemicals   []Chemical `json:"chemicals"`
}

SearchResult of a single search

func ProcessSearchResultJSON

func ProcessSearchResultJSON(json []byte) (output SearchResult)

ProcessSearchResultJSON processes sigma api response json

func Search

func Search(query string) (SearchResult, error)

Search perform search for a query string

type SearchVariables

type SearchVariables struct {
	SearchTerm     string `json:"searchTerm"`
	Page           int    `json:"page"`
	Group          string `json:"group"`
	SelectedFacets []any  `json:"selectedFacets"`
	Sort           string `json:"sort"`
	Type           string `json:"type"`
}