Sign in for more options. Each user starts with 1,000 free credits.
KzO9K
Predict Metrics by Zipcode¶
Here is an dummy tool template that you can use to prototype your tool. This tool template assumes that each row of your training dataset corresponds to a zipcode.
Note that this tool uses a dummy model. Please modify the inputs, outputs and model to fit your chosen hypothesis and training dataset.
Thanks to the following groups for making this work possible:
- CrossCompute for providing mentors and tutorials
- Tech Incubator at CUNY Queens College for guiding the student teams
- NYC Open Data for publishing datasets
- NYC Mayor's Office of Data Analytics for organizing NYC Open Data Week
{zipcode_table : Zipcodes ? Specify the zipcodes for which you would like to predict metrics}
In [1]:
# CrossCompute
zipcode_table_path = 'zipcode.csv'
target_folder = '/tmp'
In [ ]:
Load Arguments¶
In [2]:
import pandas as pd
zipcode_table = pd.read_csv(zipcode_table_path)
zipcode_table[:3]
Out[2]:
Normalize Variables¶
Here we want to normalize tree count. We can do that by dividing tree count by zipcode area in square meters.
In [3]:
url = 'https://data.cityofnewyork.us/download/i8iw-xf4u/application%2Fzip'
In [4]:
# Use default projection to get zipcode area in square feet
# import geotable
# nyc_zipcode_table = geotable.load(url)
# print(nyc_zipcode_table.iloc[0]['AREA'])
# print(nyc_zipcode_table.iloc[0]['geometry_object'].area)
In [5]:
# Get UTM projection
import geotable
utm_proj4 = geotable.load_utm_proj4(url)
utm_proj4
Out[5]:
In [11]:
# Get zipcode area in square meters
nyc_zipcode_table = geotable.load(url, target_proj4=utm_proj4)
nyc_zipcode_table.iloc[0]['geometry_object'].area
Out[11]:
In [12]:
nyc_zipcode_table['Area in Square km'] = nyc_zipcode_table[
'geometry_object'].apply(lambda g: g.area/1000000)
nyc_zipcode_table[['ZIPCODE', 'Area in Square km']][:5]
Out[12]:
In [13]:
# Extract relevant columns
nyc_zipcode_table = nyc_zipcode_table[['ZIPCODE', 'Area in Square km']].copy()
In [14]:
zipcode_table[:3]
Out[14]:
In [15]:
# Merge tables
zipcode_table['zipcode'] = zipcode_table['zipcode'].astype(str)
dataset_table = pd.merge(zipcode_table, nyc_zipcode_table, left_on='zipcode', right_on='ZIPCODE')
dataset_table[:5]
Out[15]:
In [16]:
dataset_table[:3]
Out[16]:
In [17]:
# Add normalized column
dataset_table['Tree Count Per Square km'] = dataset_table[
'Tree Count'] / dataset_table['Area in Square km']
dataset_table[:5]
Out[17]:
Run Model to Estimate Target Variable¶
In [18]:
# Load model
from pickle import load
model = load(open('model.pkl', 'rb')) # !!! Replace dummy model with your model
model
Out[18]:
In [19]:
dataset_table[:3]
Out[19]:
In [22]:
# Run model
X = dataset_table[['Tree Count Per Square km']].values
y = model.predict(X)
y
Out[22]:
In [23]:
# Add column
dataset_table['Predicted Air Pollution'] = y
dataset_table
Out[23]:
Render Table¶
In [28]:
# Select columns
output_table = dataset_table[[
'zipcode',
'Tree Count Per Square km',
'Predicted Air Pollution',
]].copy()
In [29]:
# Save file to target folder to include it in the result download
target_path = target_folder + '/a.csv'
output_table.to_csv(target_path, index=False)
print(f'a_table_path = {target_path}') # Print table_path to render table
Render Map¶
In [30]:
output_geotable = output_table.copy() # Prevent SettingwithCopyWarning
output_geotable
Out[30]:
In [31]:
# Define wkt_by_zipcode
import geotable
url = 'https://data.cityofnewyork.us/download/i8iw-xf4u/application%2Fzip'
# Specify target_proj4 to convert from NYC spatial reference to longitude and latitude
nyc_zipcode_table = geotable.load(url, target_proj4=geotable.LONGITUDE_LATITUDE_PROJ4)
In [32]:
wkt_by_zipcode = {}
for index, row in nyc_zipcode_table.iterrows():
zipcode = row['ZIPCODE']
geometry = row['geometry_object']
wkt_by_zipcode[zipcode] = geometry.wkt
In [35]:
# Geocode zipcode polygons
output_geotable['WKT'] = output_geotable['zipcode'].apply(
lambda zipcode: wkt_by_zipcode.get(str(int(zipcode))))
output_geotable = output_geotable.dropna(subset=['WKT'])
output_geotable[:3]
Out[35]:
In [36]:
nyc_zipcode_table[nyc_zipcode_table['ZIPCODE'] == '10019'].iloc[0]['geometry_object'].wkt
Out[36]:
In [44]:
# Set color for each geometry using a gradient
output_geotable['FillReds'] = output_geotable['Predicted Air Pollution']
#tput_geotable[:3]
In [ ]:
In [46]:
# Save file to target folder to include it in the result download
target_path = target_folder + '/b.csv'
output_geotable.to_csv(target_path, index=False)
print(f'b_geotable_path = {target_path}') # Print geotable_path to render map
Render Plot¶
In [48]:
%matplotlib inline
axes = output_table[[
'Predicted Air Pollution',
]].plot(kind='bar')
In [49]:
# Save file to target folder to include it in the result download
target_path = target_folder + '/c.png'
figure = axes.get_figure()
figure.savefig(target_path)
print(f'c_image_path = {target_path}')
Predicted Metrics by Zipcode¶
YOUR INTERPRETATION OF THE RESULTS
{a_table : Tree Count vs Air Pollution ? YOUR TABLE DESCRIPTION}
{b_geotable : Tree and Pollution overlap ? YOUR MAP DESCRIPTION}
{c_image : Predicted Air Pollution ? YOUR PLOT DESCRIPTION}