[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
This repository has been archived by the owner on Oct 12, 2020. It is now read-only.

Example connector written in Go for postgres using the gRPC data-loading API in Qlik Associative Engine.

License

Notifications You must be signed in to change notification settings

qlik-oss/core-grpc-postgres-connector

Repository files navigation

Example Postgres GRPC Connector

As of 1 July 2020, Qlik Core is no longer available to new customers. No further maintenance will be done in this repository.

The Example Postgres GRPC Connector shows how to load data into Qlik Associative Engine from Postgres using a dockerized connector built in Golang. It streams the data asynchronously using go channels though the following components before sending it onto Qlik Associative Engine.

  • postgres_reader - reads the data from the database into reasonably sized SQL data chunks.
  • async_translator - takes the SQL data chunks and translates them into GRPC data chunks.
  • async_stream_writer - takes the GRPC data chunks and writes them onto the GRPC stream.

The reason for the division is to be able to utilize multiple CPU cores to process the different stages simultaneously.

Example

The /example directory defines a simple stack of services using docker-compose:

  • Qlik Associative Engine
  • Postgres GRPC Connector
  • Postgres Database
  • Node Test Runner (only used for automated testing)

The script in example/reload-runner is used to instruct Qlik Associative Engine to load the example data (originally defined in example/postgres-image/airports.csv) using the connector.

Steps to run the example

Run in a *nix environment (or Git Bash if on Windows), note that you must accept the Qlik Core EULA by setting the ACCEPT_EULA environment variable:

$ cd example
$ ACCEPT_EULA=yes docker-compose up -d --build
$ cd reload-runner
$ npm install
$ npm start

About

Example connector written in Go for postgres using the gRPC data-loading API in Qlik Associative Engine.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published