Adding Custom Country Map to Apache Superset
In this post I demonstrate the steps followed to add a custom country map to superset repository and rebuild the app.
In order to plot coordinates of a dataset over a Country Map graph in Superset, unfortunately the country map of my interest (South Africa) is not present in the available countries compiled with the current version of Superset 2.1.0
Therefore I had to follow some instructions provided in the official documentation to get map of South Africa in GeoJSON format, and compile Superset application from source.
I’ll demonstrate the steps I followed to achieve this for anyone who might need to use a country map which is currently not included in the official superset package.
Prerequisites
following tools are needed for this exercise:
- Git
- Javascript package management (yarn or npm)
Step 1 — Getting the map
The first step is finding or producing a GeoJSON format of the map of the country of interest. I had to check few online sources to finally find what was needed.
If the map is in other standard formats such as shapefile or TopoJSON , it is still possible to use an offline or online tools such as this website to convert it into GeoJSON format.
You can also try the provided tool in Superset Repository to generate the map file in this link But I decided to rather search and download the map from an external source.
One important aspect is to meet superset requirement of including ISO 3166–2 Codes of provinces or regions within the country map. I had to manually update the map file to include the related codes for each province in the file in a few places, in the properties section of each province and add ISO value-key pair to it:
{ "type": "Feature",
"properties": { "id": "8", "ISO": "ZA-WC*", "NAME_1": "Western Cape",
"CODE": "WC", "PROVINCE": "Western Cape"...}
}Step 2 — Forking the Apache Superset repository
In order to avoid redoing this operation every time I want to upgrade when there is a new version of Superset, I first forked the superset repository into our own organisation repos. This way when there is a new version all I need to do is merge my version with the upstream changes and redeploy the app.
Step 3 — Cloning the forked repository
After forking the official repository, clone the repository from the forked page into the workstation. I prefer to download from the latest release branch (v2.1.0 at the time of writing this post) rather than master branch and specifying the --depth 1 parameter to create shallow clone and avoid downloading the entire history of commits which I don’t need.
After cloning checkout the clone repo to a new local branch with similar branch name (ex v2.1.0)
# <organisation> should be replated in the link
$ git clone --depth 1 --branch 2.1.0 https://github.com/<oranisation>/superset.git
$ git checkout -b v2.1.0Step 4 — Configure nvm
If you look the files and directories in the downloaded repository, there is a directory called superset-frontend which is what we are interested in as it contains the front-end assets.
In order to be able to build the assets after including the new map assets of interest we need a Javascript package manager such as NodeJS NPM or Yarn.
For building with node, If you have nvm tool installed, all you need to do is execute the following command to install the required node version by the source specified in the .nvmrc file (ex v16.9.1) inside the superset-frontend directory:
# install nvm from version specified in .nvmrc file in the following directory
$ cd superset-frontend
$ nvm install
$ nvm useIf you already have the required node version installed you can just execute nvm use to switch to the required version.
After successfully executing the above commands if you execute nvm current or node -v it should point to the required node version specified in .nvmrc file
$ nvm current
v16.9.1Step 5 — Build and install dependencies
After performing the above configurations, we need to build using npm ci command. In the below code snippet I have increased the fetch retry timeout due to failures I got with timeout error, and also set PUPPETEER_SKIP_DOWNLOAD environment variable as specified in instructions of referenced pages.
# clean and install dependencies
$ npm config set fetch-retry-maxtimeout 240000
$ export PUPPETEER_SKIP_DOWNLOAD='true'
$ npm ciIf the build was successful you should be able to launch and view the chart plugins in your browser by building and running the storybook plugin package as follows:
$ npm run plugins:storybookStep 6 — Add the map asset
In order to add the new map two changes are requried:
#1 Add the new map geo-json file to the target countries directory under ./plugins/legacy-plugin-chart-country-map/src/countries/
#2 open and edit countries.ts file located inside plugins/legacy-plugin-chart-country-map/src/ sub-directory, and add the name of the country in the following to locations.
In the first section of the file add the import statement to with the path and name of the geo-json file of your map saved in the previous step. In the following example I added a line to import the south_africa map in the second line:
import slovenia from './countries/slovenia.geojson';
import south_africa from './countries/south_africa.geojson';
import sweden from './countries/sweden.geojson';You also need to add the country name in the countries list object:
export const countries = {
...
slovenia,
south_africa,
spain,
...
} Step 7 — Build Assets
Once the required code changes are completed we can build the front-end assets with npm:
$ npm run buildIf the build completes successfully without errors, the compiled assets will be generated in superset/static/assets in the root directory.
Step 8 — Build Supeset App
as the final step we need to build the superset python app which will include the changes we made to the front-end assets.
Navigate to the parent directory (out of superset-frontend) and run the following command. This command will install the required python packages from setup.py file included.
$ pip install . Ensure to have setup a python virtual environment beforehand. You can refer to the guidelines specified here.
Once build is finished the superset executable will be generated in the virtual environment’s bin directory. Next you would need to install the dependency packages as usual.
Deployment to Production
For working with a local superset app the above steps would be sufficient. However for deployment to a production environment the changes can be committed and pushed back to Github. From the production environment the repository can to be cloned and the node and superset app build process (from step #4) repeated.
Troubleshooting
In case your builds and dependencies installation fails at any stage or you need to start over, in order to start clean, remove the generated node_modules directory before rebuilding:
$ npm remove node_modules 
