Python Geo Data Visualization With Kepler.gl
Get a Python web service up and running in minutes
If you have ever worked with any sort of geo data, you know it’s not super useful to just stare at it in a CSV, table or JSON format. You need to put it on a map in order to review it and spot any patterns. This might not be the end all be all way to gather insights, you’ll still need to do a proper analysis or/and run your ML models on it but it’s an absolute must.
What is Kepler.gl?
Kepler.gl is Uber’s open source project (GitHub link here) for creating high-performing and data-agnostic web applications. It is built on top of Mapbox and can handle large data sets, provides a lot of flexibility in terms of configuring what and how you want to visualize.
It uses layers as basic building blocks to allow you to create interactive maps — you can add, remove, show and hide layers, and change the way they look. This is a super basic example using a public flight data set from data.world:
As you can see you can layer out grids, points, arches/lines, polygons and even do a 3D map. There are ways for you to do very granular filtering (to the extent your dataset allows):
The dataset I am showing here only has 37,042 rows but I have tried it out with millions and it handles it pretty well. The geo calculations are GPU-accelerated for better performance.
Here is also a great post in Uber’s engineering blog with lots of examples of how data can be represented and more details on Kepler itself.
Ways to use Kepler.gl?
- upload a dataset on their demo page — if you want to check out what this is or if you need a one-off quick way of visualizing a moderate amount of non-sensitive data, this is the way to go.
- as a react component — if you want to embed this functionality in your existing project, this will give you a lot of control (over state, data flow, look, etc). Just do:
npm install kepler.gl- as a python module — if you are a non-front-end expert and you need to get things done quickly and without too many fancy UI components besides the maps, this is the one for you. To start off, run:
pip install keplerglThis last one is what I am gonna quickly go over to demonstrate how impressively quickly you can get an impressive MVP app up and running and present your data to impatient stakeholders or clients.
Creating a web app with Kepler’s Python module
I’m gonna make some quick assumptions — that you know a bit about python, pandas, and fast API (don’t need to be an expert to follow).
1) let’s import what we need:
import uvicorn
import pandas as pd
from keplergl import KeplerGl
from fastapi import FastAPI
from fastapi.responses import HTMLResponse2) load your data:
# load a csv file
df = pd.read_csv('my_data.csv')
# or a json file
df = pd.read_json('my_data.json')Now, this can and will probably be more than one file so just need to do that in a loop and combine the dataframes resulting from each file like this:
all_data = pf.concat([all_data, df])3) initialize Kepler.gl map:
kepler = KeplerGl(config=my_map_config)The config here is what you’ll probably need to play around a bit while you get it right for whatever your use case is.
To do that, at least for the purely representational side of it, the best way is to use a notebook to quickly load up your data and customize as needed, then export the config and apply it to your project. You can also either manually or programmatically adjust the parameters that matter from a more functional perspective.
An example config will be something like this:
config = {
"version": "v1",
"config": {
"visState": {
"filters": [
{
"dataId": ["my_data"],
"id": "11111",
"name": ["some_col_name"],
"type": "multiSelect",
"value": [],
"enlarged": False,
"plotType": "histogram",
"animationWindow": "free",
"yAxis": None,
"speed": 1,
}
],
"layers": [
{
"id": "22222",
"type": "point",
"config": {
"dataId": "my_data",
"label": "my_data",
"color": [30, 150, 190],
"highlightColor": [252, 242, 26, 255],
"isVisible": True,
"visConfig": {
"radius": 5,
"fixedRadius": False,
"opacity": 0.8,
"outline": False,
"thickness": 2,
"strokeColor": None,
...
},
"hidden": False
}
}
],
"interactionConfig": {
"tooltip": {
"fieldsToShow": {
"my_data": [
{"name": "col_1", "format": None},
{"name": "col_2", "format": None}
]
},
"compareMode": False,
"compareType": "absolute",
"enabled": True,
},
"brush": {"size": 0.5, "enabled": False},
"geocoder": {"enabled": False},
"coordinate": {"enabled": False},
},
"layerBlending": "normal",
"splitMaps": [],
"animationConfig": {"currentTime": None, "speed": 1},
},
"mapState": {
"bearing": 0,
"dragRotate": False,
"latitude": 40.710394,
"longitude": -74.000288,
"pitch": 0,
"zoom": 12.41,
"isSplit": False,
},
"mapStyle": {
"styleType": "dark",
"topLayerGroups": {},
"visibleLayerGroups": {
"label": True,
"road": True,
"border": False,
"building": True,
"water": True,
"land": True,
"3d building": False,
},
"threeDBuildingColor": [
9.665468314072013,
17.18305478057247,
31.1442867897876,
],
"mapStyles": {},
},
},
}Basically, there are 3 main sections — visState, mapState and mapStyle.
MapState is used to define where on the world map you’ll start off every time you reload the page, the starting/default location.
MapStyles set the general theme of the KeplerGl application. It has few preset options available like dark, muted night, light, muted light, and satellite.
VizState is the more interesting one. It defines the layers (whether it’s point, hexagon, line etc ) and what dataset each layer to use — note that I put the dataId in bold, that is how you bind your data with the config, will show an example in the next section. You can also define filters that you can apply to one or more sets and base on one or more fields in that dataset. Additionally, here’s where you can define what you want to show in your tooltip (or other interactive option) and how.
4) pass your data to Kepler
# the name here should match the dataId in the config
kepler.add_data(data=all_data, name="my_data")5) turn the map into HTML
kepler_html = kepler._repr_html_()6) start up your app and return the generated HTML
app = FastAPI()@app.get("/")
async def index():
return HTMLResponse(content=kepler_html, status_code=200)if __name__ == "__main__":
uvicorn.run("web_app:app", host="X", port="X")
7) have fun with it
.. and thanks for reading!
