Visualizing Station Delays on the TTC

By: Alexander Shatrov

Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018.

Intro:

The topic of this geovisualization project is the TTC. More specifically, the Toronto subway system and its many, many, MANY delays. As someone who frequently has to suffer through them, I decided to turn this misfortune into something productive and informative, as well as something that would give a person not from Toronto an accurate image of what using the TTC on a daily basis is like. A time-series map showing every single delay the TTC went through over a specified time period.  The software chosen for this task was Carto, due to its reputation as being good at creating time-series maps.

Obtaining the data:

First, an excel file of TTC subway delays was obtained from Toronto Open Data, where it is organised by month, with this project specifically using August 2018 data. Unfortunately, this data did not include XY coordinates or specific addresses, which made geocoding it difficult. Next, a shapefile of subway lines and stations was obtained from a website called the “Unofficial TTC Geospatial Data”. Unfortunately, this data was incomplete as it had last been updated in 2012 and therefore did not include the recent 2017 expansion to the Yonge-University-Spadina line. A partial shapefile of it was obtained from DMTI, but it was not complete. To get around this, the csv file of the stations shapefile was opened up, the new stations added, the latitude-longitude coordinates for all of the stations manually entered in, and the csv file then geocoded in ArcGIS using its “Display XY Data” function to make sure the points were correctly geocoded. Once the XY data was confirmed to be working, the delay excel file was saved as a csv file, and had the station data joined with it. Now, it had a list of both the delays and XY coordinates to go with those delays. Unfortunately, not all of the delays were usable, as about a quarter of them had not been logged with a specific station name but rather the overall line on which the delay happened. These delays were discarded as there was no way to know where exactly on the line they happened. Once this was done, a time-stamp column was created using the day and timeinday columns in the csv file.

Finally, the CSV file was uploaded to Carto, where its locations were geocoded using Carto’s geocode tool, seen below.

It should be noted that the csv file was uploaded instead of the already geocoded shapefile because exporting the shapefile would cause an issue with the timestamp, specifically it would delete the hours and minutes from the time stamp, leaving only the month and day. No solution to this was found so the csv file was used instead. The subway lines were then added as well, although the part of the recent extension that was still missing had to be manually drawn. Technically speaking the delays were already arranged in chronological order, but creating a time series map just based on the order made it difficult to determine what day of the month or time of day the delay occurred at. This is where the timestamp column came in. While Carto at first did not recognize the created timestamp, due to it being saved as a string, another column was created and the string timestamp data used to create the actual timestamp.

Creating the map:

Now, the data was fully ready to be turned into a time-series map. Carto has greatly simplified the process of map creation since their early days. Simply clicking on the layer that needs to be mapped provides a collection of tabs such as data and analysis. In order to create the map, the style tab was clicked on, and the animation aggregation method was selected.

The color of the points was chosen based on value, with the value being set to the code column, which indicates what the reason for each delay was. The actual column used was the timestamp column, and options like duration (how long the animation runs for, in this case the maximum time limit of 60 seconds) and trails (how long each event remains on the map, in this case set to just 2 to keep the animation fast-paced). In order to properly separate the animation into specific days, the time-series widget was added in the widget tab, located next to to the layer tab.

In the widget, the timestamp column was selected as the data source, the correct time zone was set, and the day bucket was chosen. Everything else was left as default.

The buckets option is there to select what time unit will be used for your time series. In theory, it is supposed to range from minutes to decades, but at the time of this project being completed, for some reason the smallest time unit available is day. This was part of the reason why the timestamp column is useful, as without it the limitations of the bucket in the time-series widget would have resulted in the map being nothing more then a giant pulse of every delay that happened that day once a day. With the time-stamp column, the animation feature in the style tab was able to create a chronological animation of all of the delays which, when paired with the widget was able to say what day a delay occurred, although the lack of an hour bucket meant that figuring out which part of the day a delay occurred requires a degree of guesswork based on where the indicator is, as seen below

Finally, a legend needed to be created so that a viewer can see what each color is supposed to mean. Since the different colors of the points are based on the incident code, this was put into a custom legend, which was created in the legend tab found in the same toolbar as style. Unfortunately this proved impossible as the TTC has close to 200 different codes for various situations, so the legend only included the top 10 most common types and an “other” category encompassing all others.

And that is all it took to create an interesting and informative time-series map. As you can see, there was no coding involved. A few years ago, doing this map would have likely required a degree of coding, but Carto has been making an effort to make its software easy to learn and easy to use. The result of the actions described here can be seen below.

https://alexandershatrov.carto.com/builder/8574ffc2-9751-49ad-bd98-e2ab5c8396bb/embed

Visual Story of GHG Emissions in Canada

By Sharon Seilman, Ryerson University
Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018

Background

Topic: 

An evaluation of annual Greenhouse Gas (GHG) Emissions changes in Canada and an in-depth analysis of which provinces/ territories contribute to most of the GHG emissions within National and Regional geographies, as well as by economic sectors.

  • The timeline for this analysis was from 1990-2015
  • Main data sources: Government of Canada Greenhouse Gas Emissions Inventory and Statistics Canada
Why? 

Greenhouse gas emissions are compounds in the atmosphere that absorbs infrared radiation, thus trapping and holding heat in the atmosphere. By increasing the heat in the atmosphere, greenhouse gases are responsible for the greenhouse effect, which ultimately leads to global climate change. GHG emissions are monitored in three elements -its abundance in the atmosphere, how long it stays in the atmosphere and its global warming potential.

Audience: 

Government organizations, Environmental NGOs, Members of the public

Technology

An informative website with the use of Webflow was created, to visually show the story of the annual emissions changes in Canada, understand the spread of it and the expected trajectory. Webflow is a software as a service (SaaS) application that allows designers/users to build receptive websites without significant coding requirements. While the designer is creating the page in the front end, Webflow automatically generates HTML, CSS and JavaScript on the back end. Figure 1 below shows the user interaction interface of Webflow in the editing process. All of the content that is to be used in the website would be created externally, prior to integrating it into the website.

Figure 1: Webflow Editing Interface
The website: 

The website it self was designed in a user friendly manner that enables users to follow the story quite easily. As seen in figure 2, the information it self starts at a high level and gradually narrows down (national level, national trajectory, regional level and economic sector breakdown), thus guiding the audience towards the final findings and discussions. The maps and graphs used in the website were created from raw data with the use of various software that would be further elaborated in the next section.

Figure 2: Website created with the use of Webflow
Check out Canada’s GHG emissions story HERE!

Method

Below are the steps that were undertaken for the creation of this website. Figure 3 shows a break down of these steps, which is further elaborated below.

Figure 3:  Project Process
  1. Understanding the Topic:
    • Prior to beginning the process of creating a website, it is essential to evaluate and understand the topic overall to undertake the best approach to visualizing the data and content.
    • Evaluate the audience that the website would be geared towards and visualize the most suitable process to represent the chosen topic.
    • For this particular topic of understanding GHG emissions in Canada, Webflow was chosen because it allows the audience to interact with the website in a manner that is similar to a story; providing them with the content in a visually appealing and user friendly manner.
  2. Data Collection:
    • For the undertaking of this analysis, the main data source used was the Greenhouse Gas Inventory from the Government of Canada (Environment and Climate Change). The inventory provided raw values that could be mapped and analyzed in various geographies and sectors. Figure 4 shows an example of what the data looks like at a national scale, prior to being extracted. Similarly, data is also provided at a regional scale and by economic sector.

      Figure 4: Raw GHG Values Table from the Inventory
    • The second source for this visualization was the geographic boundaries. The geographic boundaries shapefiles for Canada at both a national scale and regional scale was obtained from Statistics Canada. Additionally, the rivers (lines) shapefile from Statistics Canada too was used to include water bodies in the maps that were created.
      • When downloading the files from Statistics Canada, the ArcGIS (.shp) format was chosen.
  3. Analysis:
    • Prior to undertaking any of the analysis, the data from the inventory report needed to be extracted to excel. For the purpose of this analysis, national, regional and economic sector data were extracted from the report to excel sheets
      • National -from 1990 to 2015, annually,
      • Regional -by province/territory from 1990 to 2015, annually
      • Economic Sector -by sector from 1990 to 2015, annually
    • Graphs:
      • Trend -after extracting the national level data from the inventory, a line graph was created in excel with an added trendline. This graph shows the total emissions in Canada from 1990 to 2015 and the expected trajectory of emissions for the upcoming five years. In this particular graph, it is evident that the emissions show an increasing trajectory. Check out the trend graph here!
      • Economic Sector -similar to the trend graph, the economic sector annual data was extracted from the inventory to excel. With the use of the available data, a stacked bar graph was created from 1990 to 2015. This graph shows the breakdown of emissions by sector in Canada as well as the variation/fluctuations of emissions in the sectors. It helps understand which sectors contribute the most and which years these sectors may have seen a significant increase or decrease. With the use of this graph, further analysis could be undertaken to understand what changes may have occurred in certain years to create such a variation. Check out the economic sector graph here!
    •  Maps:
      • National map -the national map animation was created with the use of ArcMap and an online GIF maker. After the data was extracted to excel, it was saved as a .csv files and uploaded to ArcMap. With the use of ArcMap, sixteen individual maps were made to visualize the varied emissions from 1990 to 2015. The provincial and territorial shapefile was dissolved using the ArcMap dissolve feature (from the Arc Tool box) to obtain a boundary file at a national scale (that was aligned with the regional boundary for the next map). Then, the uploaded table was joined to the boundary file (with the use of the Table join feature). Both the dissolved national boundary shapefile and the river shapefile were used for this process, with the data that was initially exported from the inventory for national emissions. Each map was then exported a .jpeg image and uploaded to the GIF maker, to create the animation that is shown in the website. With the use of this visualization, the viewer can see the variation of emissions throughout the years in Canada. Check out the national animation map here!
      •  Regional map -similar to the national one, the regional map animation was created in same process. However, for the regional emissions, data was only available for three years (1990, 2005 and 2015). The extracted data .csv file was uploaded and table joined to the provinces and territories shapefile (undissolved), to create three choropleth maps. The three maps were them exported as .jpeg images and uploaded to the GIF maker to create the regional animation. By understanding this animation, the viewer can distinctly see which regions in Canada have increase, decreased or remained the same with its emissions. Check out the regional animation map here!
  4. Final output/maps:
    • The graphs and maps that were discussed above were exported as images and GIFs to integrate in the website. By evaluating the varied visualizations, various conclusions and outputs were drawn in order to understand the current status of Canada as a nation, with regards to its GHG emissions. Additional research was done in order to assess the targets and policies that are currently in place about GHG emissions reductions.
  5. Design and Context:
    • Once the final output and maps were created, and the content was drafted, Webflow enables the user to easily upload external content via the upload media tool. The content was then organized with the graphs and maps that show a sequential evaluation of the content.
    • For the purpose of this website, an introductory statement introduces the content discussed and Canada’s place in the realm of Global emissions. Then the emissions are first evaluated at a national scale with the visual animation, then the national trend, regional animation and finally, the economic sector breakdown. Each of the sections have its associated content and description that provides an explanation of what is shown by the visual.
    • The Learn More and Data Source buttons in the website include direct links to Government of Canada website about Canada’s emissions and the GHG inventory itself.
    • The concluding statement provides the viewer with an overall understanding of Canada’s status in GHG emissions from 1990 to 2015.
    • All of the font formatting and organizing of the content was done within the Webflow interface with the end user in mind.
  6. Webflow:
    • The particular format that was chosen in for this website because of story telling element of it. Giving the viewer the option to scrolls through the page and read the contents of it, works similarly as story because this website was created for informative purposes.

Lessons Learned: 

  • While the this website provides informative information, it could be further advanced through the integration of an interactive map, with the use of additional coding. This however would require creating the website outside of the Webflow interface.
  • Also, the analysis could be further advanced with the additional of municipal emissions values and policies (which was not available in the inventory it self)

Overall, the use of Webflow for the creation of this website, provides users with the flexibility to integrate various components and visualizations. The user friendly interface enables uses with minimal coding knowledge to create a website that could be used for various purposes.

Thank you for reading. Hope you enjoyed this post!

Traffic.me: Mapping live traffic with ArcGIS Runtime SDK and HERE Technologies using Android App Developer

by Nicholas Pulsone
Geovis Class Project @RyersonGeo, SA8905, Fall 2018

Topic & Background

Driving through congested parts of Toronto is a tedious and troubling problem that many people would like to avoid. The goal was to create a mobile application using android app developer that can use traffic data as a live input to map traffic patterns across North America. Many companies such as HERE Technologies record traffic information that updates regularly and can be used to map and observe traffic patterns across the entire world. Using android app developer, it is easy to add software developer packages such as ArcGIS Runtime SDK to develop new tools that can be used on a day-to-day basis.

Data

The first problem when creating an app for a purpose or goal is where to find the data. As previously mentioned, HERE technologies is a company owned by NOKIA and currently has its headquarters in Amsterdam. HERE technologies records live weather, routing and traffic information using a combination of both geolocation and intelligence algorithms. Geolocation services that HERE tracks include:

  • Devices with location or GPS tracking
  • Tables or other devices with WIFI and signal strength
  • Phones while measuring varying strength of reception via cell tower signals

HERE technologies contains a global database of over 93 million cellular towers and over 2.3 billion Wi-Fi hotspots which record and store data. The data needed to be able to map varying levels of traffic or traffic density as well as potential collisions or other disruptions affecting driving conditions. The data would need to be able to be displayed visually on a mapping platform and accessible by android app developer software.

Methods

There are multiple ways a live traffic application can be created using data from HERE technologies:

  1. Creating a live traffic app using HERE API and map creator
  2. Creating a live traffic app using HERE data in ArcGIS Runtime SDK (requires ArcGIS developer license)

The methods in this blog will be describing how to create an application using the data from HERE technologies with ArcGIS Runtime SDK.

The first step is to download the needed requirements. First, is to download the newest version of android app developer studio. Currently, the newest version of Android App developer studio is 3.2.1 and available online for Windows, Mac and Linux. Once android app developer is downloaded, the next step is download the second part of the software that will be used in this creation, which is the ArcGIS Runtime SDK for Android 4.0.

The second step is to set the back-end of the application. After specifying the operating system the application will work on, and inputting the name of the application, the first thing to set up is include the ESRI bintray for ArcGIS.

As ESRI’s repository is not open source, the url must be specified to manually add the url for the ESRI bintray. Then the app dependencies need to updated to include the ArcGIS Runtime SDK.

Once the Gradle scripts were synced, the next step was to add a map view for the app. By default, we can remove the text view element and manually create a map view with the following syntax:

After adding the map view for the data, the next step was to specify a basemap then access the data:

The above syntax is a sample of how a basemap and starting location can be specified upon opening. The final step was to be able to access the data. HERE technologies has collaborated with ESRI to develop a world traffic service that can be accessed from mobile and desktop services using the url:

http://traffic.arcgis.com/arcgis/rest/services/World/Traffic/MapServer

Additionally, ESRI and HERE technologies have also created a layer available on the ArcGIS developer portal to users with an ArcGIS developer license. Once the layer is accessible, it is important to open and save the layer in ArcGIS online and enable sharing and public access permissions. As layers used in ArcGIS require a login to be viewed, the next step is to setup a proxy to bypass the login error that would prevent the data from being used, even if permissions are set to public.

To setup a proxy using the ArcGIS developer server proxy, the application must be authenticated and registered in the ArcGIS developer platform. Once registered, the user has access to many services such as a proxy service which will be used along with the traffic layer.

To enable this proxy under services, we must specify what type of proxy service and request limit the proxy will use. Once the requirements are specified, the service outputs a URL which contains a proxy service from ArcGIS.

To use the proxy, simply add the link as a layer from web in ArcGIS online, and the proxy should be active.

Figure 1: Adding Services to ArcGIS Online Map

The final step was to add in the url for the ArcGIS online webmap which contains the traffic data, into android app developer.

Once the url was added into the Android App developer; just click Sync & Run and the app will appear on your device similar to the picture below!

Figure 2: Example of Traffic App

Limitations

A limitation that was experienced while coding the application was ease of use. Without using a legend or slider, it is very hard to distinguish which areas of Toronto are being affected by what kind of problem. The symbology can be changed, however integrating a legend as a button feature in android app developer was more useful and ultimately was included in the final iteration of the app shown in Figure 3.

Figure 3: Final Iteration of Traffic.me

 

Using LEGO to create a physical 3D elevation model of Ontario

by: Adam Anthony | Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018

Using LEGO blocks to visualize the landscape elevation throughout the province of Ontario was an the objective of this project and the steps I took to execute this project will be outlined below.

I first sourced the elevation data from Scholar’s GeoPortal and used the north and south PDEM files for Ontario as the foundation for the elevation model. Using ArcGIS I added the north and south PDEM layers and merged the two files using Mosaic To New Raster tool. This produced a merged PDEM.

Next, the merged PDEM needed to be resampled, to increase the pixels size so that it would align with the size of a 1×1 LEGO block. Using the Resample tool, I resampled the pixel size from 30x30m to 30,000×30,000m resolution. This resolution was influenced by a number of factors:

  1. maintaining the integrity of the elevation levels (699m was the highest peak at 30x30m, but it reduced to 596m when resampled to 30kx30k)
  2. scale of physical model as it relates to size and cost of the LEGO blocks

Below is the resampled layer to 30k resolution and clipped to a raster tiff of Ontario (also at same resolution)

In the Properties dialiogue box I converted the Stretched symbology to Classificled symbology which would allow me to isolate specific elevation interval classes. I seleccted seven classes based on the following criteria:

  1. Wanted to isolate the high and low values
  2. Using intervals of 75m depicted the more visually appealling variation in elevation and did so most effectively. It allowed for a <75m and a >450m class
  3. No more than seven classes because of LEGO colour options and available stock
  4. Equal interval of 75m increments

Colour selection at this stage was preliminary and a divergent scheme from green to dark burgundy seemed to be most aesthetically pleasing.

To isolate each elevation layer to determine the number of pixels (i.e. LEGO blocks) each layer requires the raster layer had to be converted to a vector layer.

Using Raster Calculator and the Int Tool, I converted the current raster from a float to an integer raster layer which is needed to be done to convert raster to polygon. This converted each cell value of the raster to an integer.

This new raster file was then converted to a polygon layer using the Raster to Polygon tool, creating this output.

Activating the raster layer from a previous step, I was able to then manually select each pixel for each respective layer to determine the number of pixels (ie LEGO pieces) that comprised the layer.

Each pixel was selected using the Selection tool and then onces all pixels for the appropriate layer were selected, the Create Layer from Selected Features was used to create an individual layer for each elevation level.

This process was repeated 7 times, producing 7 layers of elevation. Each layer’s Attribute Table was then used to identify the total number of pixels present in the layer and then was used to determine the number of LEGO pieces needed for that layer, where 1 pixel = 1 single-block LEGO piece.

These individual layers will also be used during the build, as a guideline for the distribution and placement of each LEGO piece.

Each colour class is an individual layer. Colours are still preliminary and the number of LEGO pieces per layer is as follows:

  • <75m: 1089 pcs
  • 75-150m: 987 pcs
  • 150-225m: 809 pcs
  • 225-300m: 657 pcs
  • 300-375m: 455 pcs
  • 375-450m: 221 pcs
  • >450m: 51 pcs

Using BrickLink, I was able to purchase 1×1 LEGO bricks for each layer. Factors that influenced the colour selection for each layer are as follows:

  • Quantity of colour available
  • Price of individual bricks
  • Location of supplier (North American)

The resulting colour scheme selected is a divergent scheme, as follows:

  • <75m: dark green
  • 75-150m: medium grey
  • 150-225m: light green
  • 225-300m: tan
  • 300-375m: light lavender
  • 375-450m: medium lavender
  • >450m: dark purple

Here is the final product.

Here is a time lapse video of the LEGO build:

https://www.youtube.com/watch?v=RP6PxkPlK1w&feature=youtu.be

#AddressingTheSurface Translucent Maps inspired by GIS and Open Data

by Edgar Baculi #themapmaker
Geovisualization Project @RyersonGeo, SA8905, Fall 2017

#AddressingTheSurface was a collaborative geovisualization project with recent OCAD University graduate, Graphic Designer and Fine Artist Jay Ginsherman, with ideas and direction from Ryerson University, Master of Spatial Analysis candidate Edgar Baculi. This project was inspired by the previous work of Ginsherman entitled ‘Liquid Shadows’ using translucent images or maps as well as a lighting device nicknamed the ‘Lightbox’. This piece along with Ginsherman’s previous and on-going work can be found here http://jginsherman.format.com/. While attending OCAD University’s 102nd GradEx, Baculi encountered the work of Ginsherman and the GIS like experience of the attendees. From this the idea of using open data and actual GIS to produce a piece had begun.

After consulting with Ginsherman a piece based on the lived experience of Baculi, open data and GIS was established. Having previous research work in open data, Baculi was familiar with exploring and downloading open data. The Toronto Open Data Catalogue provided all the data relevant to the project. The key focus of the data collection were datasets related to Toronto Community Housing and services of interest for these residents and other locations.

The following datasets were downloaded and manipulated from the catalogue:
1. Toronto Community Housing Corporation Residences (with high, mid and low rise buildings selected and divided into three map layers)
2. The boundary of the city of Toronto (dissolved former municipality shape file)
3. City of Toronto Neighbourhoods
4. Street file
5. Fire Stations
6. Police Stations
7. Park Land
8. TTC Subway lines
9. Three heat/ kernel density maps on services of interest for TCHC residents (based on Rent Bank Centres, Community Cooling Centres and Shelters.

A key aspect of this project was the use of subtractive colours (Magenta, Yellow and Cyan) for the heat maps to show interesting overlap, resulting in new colours. The overlap of colours were designed intentionally to be open to interpretation to the map readers.

Using ArcGIS the previously mentioned datasets were adjusted by Baculi with ideal symbology before being sent to Ginsherman. The discussions between Baculi and Ginsherman involved understanding how GIS works and cartographic ideals for the look of the maps, with great design to appeal to the audience. Baculi wanted to create a hands on GIS experience, with a legend that built itself up and remained legible to the map reader. Ginsherman incorporated these ideals into the final look under Baculi’s direction.

Once Baculi completed the GIS portion of the layers, they were sent off to Ginsherman to improve design, layout and to print. Ginsherman used PDF’s of the layers in adobe illustrator, and ensured map alignment by limiting the work to the same illustrator file and giving each map its own layer. Printing involved using a laser printer, specifically at the OCAD University Digital Print Centre. Previous draft layers were also created to test the colour combinations and the best level of transparency for the maps.

A key component of the piece was the Lightbox from Ginsherman’s previous work which was designed and built by Ginsherman and his father. The Lightbox is made of wood, acrylic glass, and LED lights which were screwed together. The Toronto boundary layer was the only layer not printed on a translucent sheet, but on the glass. The boundary along with the north arrow acted as guides to align the layering of the maps. The LED lights improved the clarity of the layering maps as well as directed attention to the piece.

The end result was presented on Ryerson’s 2017 GIS Day and consisted of a Lightbox with the Toronto boundary printed on top and a total of 12 translucent maps. A variety of combinations were possible for interaction and discussion for the attendees. Please see the YouTube video below!