Datashader Dashboard#
This notebook contains the code for an interactive dashboard for making Datashader plots from any dataset that has latitude and longitude (geographic) values. Apart from Datashader itself, the code relies on other Python packages from the HoloViz project that are each designed to make it simple to:
lay out plots and widgets into an app or dashboard, in a notebook or for serving separately (Panel)
build interactive web-based plots without writing JavaScript (Bokeh)
build interactive Bokeh-based plots backed by Datashader, from concise declarations (HoloViews and hvPlot)
express dependencies between parameters and code to build reactive interfaces declaratively (Param)
describe the information needed to load and plot a dataset, in a text file (Intake)
import os, colorcet, param as pm, holoviews as hv, panel as pn, datashader as ds
import intake
import xyzservices.providers as xyz
from holoviews.element import tiles as hvts
from holoviews.operation.datashader import rasterize, shade, spread
from collections import OrderedDict as odict
hv.extension('bokeh', logo=False)
/home/runner/work/examples/examples/datashader_dashboard/envs/default/lib/python3.8/site-packages/dask/dataframe/utils.py:367: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
_numeric_index_types = (pd.Int64Index, pd.Float64Index, pd.UInt64Index)
/home/runner/work/examples/examples/datashader_dashboard/envs/default/lib/python3.8/site-packages/dask/dataframe/utils.py:367: FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
_numeric_index_types = (pd.Int64Index, pd.Float64Index, pd.UInt64Index)
/home/runner/work/examples/examples/datashader_dashboard/envs/default/lib/python3.8/site-packages/dask/dataframe/utils.py:367: FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
_numeric_index_types = (pd.Int64Index, pd.Float64Index, pd.UInt64Index)
You can run the dashboard here in the notebook with various datasets by editing the dataset below to specify some dataset defined in dashboard.yml. You can also launch a separate, standalone server process in a new browser tab with a command like:
DS_DATASET=nyc_taxi panel serve --show dashboard.ipynb
(Where nyc_taxi can be replaced with any of the available datasets (nyc_taxi, nyc_taxi_50k (tiny version), census, osm-1b, or any dataset whose description you add to catalog.yml). To launch multiple dashboards at once, you’ll need to add -p 5001 (etc.) to select a unique port number for the web page to use for communicating with the Bokeh server. Otherwise, be sure to kill the server process before launching another instance.
dataset = os.getenv("DS_DATASET", "nyc_taxi")
catalog = intake.open_catalog('catalog.yml')
source = getattr(catalog, dataset)
The Intake source object lets us treat data in many different formats the same in the rest of the code here. We can now build a class that captures some parameters that the user can vary along with how those parameters relate to the code needed to update the displayed plot of that data source:
plots = odict([(source.metadata['plots'][p].get('label',p),p) for p in source.plots])
fields = odict([(v.get('label',k),k) for k,v in source.metadata['fields'].items()])
aggfns = odict([(f.capitalize(),getattr(ds,f)) for f in ['count','sum','min','max','mean','var','std']])
norms = odict(Histogram_Equalization='eq_hist', Linear='linear', Log='log', Cube_root='cbrt')
cmaps = odict([(n,colorcet.palette[n]) for n in ['fire', 'bgy', 'bgyw', 'bmy', 'gray', 'kbc']])
maps = ['EsriImagery', 'EsriUSATopo', 'EsriTerrain', 'EsriStreet', 'OSM']
bases = odict([(name, getattr(hvts, name)().relabel(name)) for name in maps])
gopts = hv.opts.Tiles(responsive=True, xaxis=None, yaxis=None, bgcolor='black', show_grid=False)
class Explorer(pm.Parameterized):
plot = pm.Selector(plots)
field = pm.Selector(fields)
agg_fn = pm.Selector(aggfns)
normalization = pm.Selector(norms)
cmap = pm.Selector(cmaps)
spreading = pm.Integer(0, bounds=(0, 5))
basemap = pm.Selector(bases)
data_opacity = pm.Magnitude(1.00)
map_opacity = pm.Magnitude(0.75)
show_labels = pm.Boolean(True)
@pm.depends('plot')
def elem(self):
return getattr(source.plot, self.plot)()
@pm.depends('field', 'agg_fn')
def aggregator(self):
field = None if self.field == "counts" else self.field
return self.agg_fn(field)
@pm.depends('map_opacity', 'basemap')
def tiles(self):
return self.basemap.opts(gopts).opts(alpha=self.map_opacity)
@pm.depends('show_labels')
def labels(self):
return hv.Tiles(xyz.CartoDB.PositronOnlyLabels()).opts(level='annotation', alpha=1 if self.show_labels else 0)
def viewable(self,**kwargs):
rasterized = rasterize(hv.DynamicMap(self.elem), aggregator=self.aggregator, width=800, height=400)
shaded = shade(rasterized, cmap=self.param.cmap, normalization=self.param.normalization)
spreaded = spread(shaded, px=self.param.spreading, how="add")
dataplot = spreaded.apply.opts(alpha=self.param.data_opacity, show_legend=False)
return hv.DynamicMap(self.tiles) * dataplot * hv.DynamicMap(self.labels)
explorer = Explorer(name="")
If we call the .viewable method on the explorer object we just created, we’ll get a plot that displays itself in a notebook cell. Moreover, because of how we declared the dependencies between each bit of code and each parameters, the corresponding part of that plot will update whenever one of the parameters is changed on it. (Try putting explorer.viewable() in one cell, then set some parameter like explorer.spreading=4 in another cell.) But since what we want is the user to be able to manipulate the values using widgets, let’s go ahead and create a dashboard out of this object by laying out a logo, widgets for all the parameters, and the viewable object:
logo = "https://raw.githubusercontent.com/pyviz/datashader/main/doc/_static/logo_horizontal_s.png"
panel = pn.Row(pn.Column(logo, pn.Param(explorer.param, expand_button=False)), explorer.viewable())
panel.servable("Datashader Dashboard")