Iex trading

View a running version of this notebook. | Download this project.

IEX, the Investors Exchange, is a transparent stock exchange that discourages high-frequency trading and makes historical trading data publicly available. The data is offered in the form of daily pcap files where each single packet corresponds to a stock trade.

Even with this specialized pcap file format, these daily records can exceed a gigabyte in size on a given day. In this notebook, we will develop a dashboard that will allow us to explore every single trade that happened in a day, including the associated metadata. To visualize all this data at once both rapidly and interactively, we will use datashader via the HoloViews API.

Loading the data

The IEX stock data is saved in two formats of pcap file called TOPS and DEEP. These formats are complex enough to make it non trivial to parse the trades with standard packet loading tools. For this reason, the trades for Monday 21st of October 2019 are supplied as a CSV file that has been generated from the original pcap file using the IEXTools library.

In [1]:
import datetime
import pandas as pd
df = pd.read_csv('./data/IEX_2019-10-21.csv')
print('Dataframe loaded containing %d events' % len(df))
Dataframe loaded containing 1222412 events

We can now look at the head of this DataFrame to see its structure:

In [2]:
symbol size price timestamp
0 ZVZZT 50 10.015 1571659221573444414
1 RSX 300 23.210 1571659313752906463
2 BABA 100 171.400 1571659356868902969
3 BABA 3 171.400 1571659357585239782
4 KMT 83 25.000 1571659403813905391

Each row above corresponds to a stock trade where price indicates the stock price, the size indicates the size of the trade and the symbol specifies which stock was traded. Every trade also has a timestamp specified in nanoseconds.

Note that multiple trades can occur on the same timestamp.

Visualizing trade with Spikes

We can now load HoloViews with the Bokeh plotting extension to start visualizing some of this data:

In [3]:
import holoviews as hv
from bokeh.models import HoverTool
from holoviews.operation.datashader import spikes_aggregate
hv.config.image_rtol = 10e-3 # Fixes datetime issue at high zoom level