Desperate Journeys

By Ibrahim T. Ghanem

Geovis Project Assignment @RyersonGeo, SA8905, Fall 2019

Background:

Over the past 20 years, Asylum Seekers have invented many travel routes between Africa, Europe and Middle East in order be able to reach a country of Asylum. Many governmental and non-governmental provided information about those irregular travel routes used by Asylum Seekers. In this context, this geovisualization project aims at compiling and presenting two dimensions of this topic: (1) a comprehensive animated spider map presenting some of the travel routes between the above mentioned three geographic areas; (2) develop a dashboard that connects those routes to other statistics about refugees in a user-friendly interface. In that sense, the best software to fit the project is Tableau.

Data and Technology

Creation of Spider maps at Tableau is perfect for connecting hubs to surrounding point as it allows paths between many origins and destinations. Besides, it can comprehend multiple layers. Below is a description of the major steps for the creation of the animated map and dashboard.

Also, Dashboards are now very useful in combining different themes of data (i.e. pie-charts, graphs, and maps), and accordingly, they are used extensively in non-profit world to present data about a certain cause. The Geovisualiztion Project applied geocoding approach to come up with the animated map and the dashboard.

The Data used to create the project included the following:

-Origins and Destinations of Refugees

-Number of Refugees hosted by each country

-Count of Refugees arriving by Sea (2010-2015)

-Demographics of Refugees arriving by Sea – 2015

Below is a brief description of the steps followed to create the project

Step 1: Data Sources:

The data was collected from the below sources.

United Nations High Commissioner for Refugees, Human Rights Watch, Vox, InfoMigrants, The Geographical Association of UK, RefWorld, Broder Free Association for Human Rights, and Frontex Europa.

However, most of the data are not geocoded. Accordingly, Google Sheets was used in Geocoding 21 routes, and thereafter each Route was given a distinguishing ID and a short description of the route.

Step 2: Utilizing the Main Dataset:

Data is imported from an excel sheet. In order to compute a route, Tableau requires data about origins,and destination with latitude and longitude. In that aspect, the data contains different categories:

A-Route I.D. It is a unique path I.D. for each route of the 21 routes;

B-Order of Points: It is the order of stations travelled by refugees from their country of origin to country of Asylum;

C-Year: the year in which the route was invented;

D-Latitude/Longitude: it is the coordinates of the each station;

F-Country: It is the country hosting Refugees;

E- Population: Number of refugees hosted in each country.

Step 3: Building the Map View:

The map view was built by putting longitude in columns, latitude in rows, Route I.D. at details, and selecting the mark type as line. In order to enhance the layout, Oder of Points was added to Marks’ Path, and changing it to dimensions instead of SUM.  Finally, to bring stations of travel, another layer was added to by putting another longitude to columns, and changing it to Dual Axis. To create filtration by Route, and timeline by year, route was added Filter while year was added to page.

Step 4: Identifying Routes:

To differentiate routes from each other by distinct colours, the route column was added to colours, and the default setting was changed to Tableau 20. And Layer format wash changed to dark to have a contrast between the colours of the routes and the background.

Step 5: Editing the Map:

After finishing up with the map formation. A video was captured by QuickStart and edited by iMovie to be cropped and merged.

Step 6: Creating the Choropleth map and Symbology:

In another sheet, a set of excel data (obtained from UNHCR) was uploaded to create a Choropoleth map that would display number of refugees hosted by each country by year 2018. Count of refugees was added to columns while Country was added to rows. The Marks’ colour ramp of orange-gold, with 4 classes was added to indicate whether or not the country is hosting a significant number of refugees. Hovering over each country would display the name of the country and number of refugees it hosts.

Step 7: Statistical Graphs:

A pie-chart and a graph were added to display some other statistics related to count of Refugees arriving by Sea from Africa to Europe, and the demographics of those refugees arriving by sea. Demographics was added to label to display them on the charts.

Step 8: Creation of the Dashboard:

All four sheets were added in the dashboard section through dragging them into the layer view. To comprehend that amount of data explanation, size was selected as legal landscape. Title was given to the Dashboard as Desperate Journeys.

Limitations

A- Tableau does not allow the map creator to change the projection of the maps; thus, presentation of maps is limited. Below is a picture showing the final format of the dashboard:

B-Tableau has an online server that can host dashboard; nevertheless, it cannot publish animated maps. Thus, the animated maps is uploaded here a video. The below link can lead the viewer to the dashboard:

https://prod-useast-a.online.tableau.com/t/desperatejourneysgeovis/views/DesperateJourneys_IbrahimGhanem_Geoviz/DesperateJourneys/ibrahim.ghanem@ryerson.ca/23c4337a-dd99-4a1b-af2e-c9f683eab62a?:display_count=n&:showVizHome=n&:origin=viz_share_link

C-Due to unavailability of geocoded data, geocoding the routes of refugees’ migration consumed time to fine out the exact routes taken be refugees. These locations were based on the reports and maps released by the sources mentioned at the very beginning of the post.

A Shot in the Dark: Analyzing Mass Shootings in the United States, 2014-2019

By: Miranda Ramnarayan

Geovis Project Assignment @RyersonGeo, SA8905, Fall 2019

The data gathered for this project was downloaded from the Gun Violence Archive (https://www.gunviolencearchive.org/), which is a non-for Profit Corporation. The other dataset is the political affiliation per state, gathered by scrapping this information from (https://www.usa.gov/election-results). Since both of these datasets contain a “State Name” column, an inner join will be conducted to allow the two datasets to “talk” to each other.

The first step is importing your excel files, and setting up that inner join.

There are four main components this dashboard is made of: States with Mass Shootings, States with Highest Death Count, Total Individuals Injured from Mass Shootings and a scattergram displaying the amount of individuals injured and killed. All of these components were created in Tableau Worksheets and then combined on a Dashboard upon completion. The following are steps on how to re-create each Worksheet. 

1. States with Mass Shootings

In order to create a map in Tableau, very basic geographic information is needed. In this case, drag and drop the “State” attribute under the “Dimensions” column into the empty frame. This will be the result:

In order to change the symbology from dots to polygons, select “Map” under the Marks section.

To assign the states with their correct political affiliation, simply drag and drop the associated year you want into the “Colour” box under Marks.

This map is displaying the states that have had mass shootings within them, from 2014 to 2019. In order to automatic this, simply drag and drop the “Incident Date” attribute under Pages. The custom date page has been selected as “Month / Year” since the data set is so large.

This map is now complete and when you press the play button displayed in the right side of this window, the map will change as it only displays states that have mass shootings within them for that month and year.

2. States with Highest Death Count

This is an automated chart that shows the Democratic and Republican state that has the highest amount of individuals killed from mass shootings, as the map with mass shootings above it runs through its time series. Dragging and dropping “State” into the Text box under Marks will display all the states within the data set. Dragging and dropping the desired year into Colour under Marks will assign each state with its political party.

 In order for this worksheet to display the state with the highest kill count, the following calculations have to be made once you drag and drop the “# Killed” from Measures into Marks.

To link this count to each state, filter “State” to only display the one that has the maximum count for those killed.

This will automatically place “State” under Filters.

Drag and drop “Incident Date” into Pages and set the filter to Month / Year, matching the format from section 1.

Format your title and font size. The result will look like:

3. Total Individuals Injured from Mass Shootings

In terms of behind the scenes editing, this graph is the easiest to replicate.

Making sure that “State Name” is above “2016” in this frame is very important, since this is telling Tableau to display each state individually in the bar graph, per year.

4. Scattergram

This graph displays the amount of individuals killed and injured per month / year. This graph is linked to section 1 and section 2, since the “Incident Date” under Pages is set to the same format. Dragging and dropping “SUM (#Killed)” into Rows and SUM (#Injured) into Columns will set the structure for the graph.

In order for the dot to display the sum of individuals killed and injured, drag and drop “# Killed” into Filter and the following prompt will appear. Select “Sum” and repeat this process for “# Injured”.

Drag and drop “Incident Date” and format the date to match Section 1 and 2. This will be your output.

Dashboard Assembly

This is where Tableau allows you to be as customizable as you want. Launching a new Dashboard frame will allow you to drag and drop your worksheets into the frame. Borders, images and text boxes can be added at this point. From here, you can re-arrange/resize and adjust your inserted workbooks to make sure formatting is to your desire.  

Right clicking on the map on the dashboard and selecting “Highlight” will enable an interactive feature on the dashboard. In this case, users will be able to select a state of interest, and it will highlight that state across all workbooks on your dashboard. This will also highlight the selected state on the map, “muting” other states and only displaying that state when it fits the requirements based on the calculations set up prior.

Since all the Pages were all set to “Month/Year”, once you press “play” on the States with Mass Shootings map, the rest of the dashboard will adjust to display the filtered information.

It should be noted that Tableau does not allow the user to change the projection of any maps produced, resulting in a lack of projection customization. The final dashboard looks like this:

GeoVis: Mapdeck Package in R

Gregory Huang
Geovisualization Project, @RyersonGeo, Fall 2019

Introduction

This project is a demonstration of the abilities of the mapdeck package in R, including its shiny interactive app compatibility.

Mapdeck is an R package created by David Cooley. Essentially, it integrates some of mapbox’s functionality into the R environment. Mapbox is a popular web-based mapping service that is community-driven and provides some great geovisualization functionalities. Strava’s global heat map is one example.

I am interested in looking at flight routes across global hubs and see if there are destination overlaps for these routes. Since the arc layer provided by mapdeck has impressive visualization capabilities of the flight routes, I’ve chosen to use mapdeck to visualize some flight route data around the world.

Example of a map generated by mapdeck: arcs, text, lines, and scatterplots are all available. Perspective changes can be done by pressing down Ctrl and clicking. The base maps are customizable with a massive selection of both mapbox and user-generated maps. This map is one of the results from longest_flights.R, which uses the “decimal” basemap.
The Map has some level of built-in interactivity: Here is an example of using a “tooltip” where if a user hovers over an arc, the arc highlights and shows information about that particular route. Note that mapdeck doesn’t want to draw flight routes across the Pacific – so if accuracy is key, do keep this in mind.

Software Requirements

To replicate this project, you’ll need your own mapbox access token. It is free as long as you have a valid email address. Since the code is written in R, you’ll also need R and R Studio downloaded on your machine to run the code.

Tl;dr…

Here’s the Shiny App

The code I created and the data I used can also be found on my GitHub repository, Geovis. To run them on your personal machine, simply download the folder and follow the instructions on the README document at the bottom of the repository page.

Screenshot of the shiny app: The slide bar will tell the map which flights to show, based on the longitudes of the destinations. All flights depart out of YYZ/KEF/AMS/FRA/DXB.

Details: Code to Generate a Map

The code I’ve written contained 2 major parts, both utilizing flight route data. The first part is done with longest_flights.R, demonstrating the capabilities of the mapdeck package using data I curated for the longest flights in the world. The second part is done with yyz_fra.R and shinyApp.R to demonstrate the shiny app compatibility and show how the package handles larger datasets (hint – very well). The shinyApp uses flight route data from 5 airports: Toronto, Iceland-Keflavik, Amsterdam, Frankfurt, and Dubai, pulled from openflights.org.

For the flight route data for the 5 airports, in particular, the data needed cleaning to make the data frame useable to mapdeck. This involved removing empty rows, selecting only the relevant data, and merging the tables.

Code snippet for cleaning the data. After the for loop completes, the flight route data downloaded from openflights.org becomes available to be used for mapdeck.

Once the data were cleaned, I began using the mapdeck functions to map out the routes. The basic parts of the mapdeck() function are to first declare your key, give it a style, and assign it a pitch if needed. There are many more parameters you can customize, but I just changed the style and pitch. Once the mapdeck map is created, use the “pipe” notion (%>%) to add any sort of layers to your map. For example, add_arc() to add the arcs seen in this post. Of course, there are many parameters that you can set, but the most important are the first three: Where your data come from, and where the origin/destination x-y coordinates are.

An example creating an arc on a map. In addition to the previously mentioned parameters, tooltip generates the little chat boxes when you hover over a layer entry, and layer_id is important when there are multiple layers on the same map.

Additional details on creating all different types of layers, including heatmaps, can be found on the documentation page HERE.

Details: Code to make a “Shiny” app

On top of the regular interactive functionalities of mapdeck, incorporating a mapdeck map into shiny can add more layers of interactivity to the map. In this particular instance, I added a slider bar in Shiny where the user can indicate the longitudes of the destinations they want to see. For example, I can filter to see just the flights going to East Asia by using that slider bar. Additional functions of shiny include using drop-menus to select specific map layers, and checkboxes as well.

The shiny code can roughly be broken down into three parts: ui, server, and shinyApp(ui, server). The ui handles the user interface and receives data from the server, while the server decides what map to produce by the input given by the user in ui. shinyApp(ui,server) combines the two to generate a shiny app.

Mapdeck integrates into the shiny app environment by mapdeckOutput() in ui to specify the map to display, and by renderMapdeck() and mapdeck_update() in server to generate the map (rendeerMapdeck) and appropriate layers to display (mapdeck_update).

Below is the code used to run the shiny app demonstrated in this blog post. Note the ui and server portions of the code bode. To run the shiny app after that, simply run shinyApp(ui,server) to generate the app.

Creating the UI
Snippet of the Server creation section. Note that the code listens to what the UI says with reactive() and observeEvent().

This concludes my geovis blog post. If you have any questions, please feel free to email me at gregory.huang@ryerson.ca.

Here is the link to my GitHub repository again: https://github.com/greghuang8/Geovis

Visualizing Station Delays on the TTC

By: Alexander Shatrov

Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018.

Intro:

The topic of this geovisualization project is the TTC. More specifically, the Toronto subway system and its many, many, MANY delays. As someone who frequently has to suffer through them, I decided to turn this misfortune into something productive and informative, as well as something that would give a person not from Toronto an accurate image of what using the TTC on a daily basis is like. A time-series map showing every single delay the TTC went through over a specified time period.  The software chosen for this task was Carto, due to its reputation as being good at creating time-series maps.

Obtaining the data:

First, an excel file of TTC subway delays was obtained from Toronto Open Data, where it is organised by month, with this project specifically using August 2018 data. Unfortunately, this data did not include XY coordinates or specific addresses, which made geocoding it difficult. Next, a shapefile of subway lines and stations was obtained from a website called the “Unofficial TTC Geospatial Data”. Unfortunately, this data was incomplete as it had last been updated in 2012 and therefore did not include the recent 2017 expansion to the Yonge-University-Spadina line. A partial shapefile of it was obtained from DMTI, but it was not complete. To get around this, the csv file of the stations shapefile was opened up, the new stations added, the latitude-longitude coordinates for all of the stations manually entered in, and the csv file then geocoded in ArcGIS using its “Display XY Data” function to make sure the points were correctly geocoded. Once the XY data was confirmed to be working, the delay excel file was saved as a csv file, and had the station data joined with it. Now, it had a list of both the delays and XY coordinates to go with those delays. Unfortunately, not all of the delays were usable, as about a quarter of them had not been logged with a specific station name but rather the overall line on which the delay happened. These delays were discarded as there was no way to know where exactly on the line they happened. Once this was done, a time-stamp column was created using the day and timeinday columns in the csv file.

Finally, the CSV file was uploaded to Carto, where its locations were geocoded using Carto’s geocode tool, seen below.

It should be noted that the csv file was uploaded instead of the already geocoded shapefile because exporting the shapefile would cause an issue with the timestamp, specifically it would delete the hours and minutes from the time stamp, leaving only the month and day. No solution to this was found so the csv file was used instead. The subway lines were then added as well, although the part of the recent extension that was still missing had to be manually drawn. Technically speaking the delays were already arranged in chronological order, but creating a time series map just based on the order made it difficult to determine what day of the month or time of day the delay occurred at. This is where the timestamp column came in. While Carto at first did not recognize the created timestamp, due to it being saved as a string, another column was created and the string timestamp data used to create the actual timestamp.

Creating the map:

Now, the data was fully ready to be turned into a time-series map. Carto has greatly simplified the process of map creation since their early days. Simply clicking on the layer that needs to be mapped provides a collection of tabs such as data and analysis. In order to create the map, the style tab was clicked on, and the animation aggregation method was selected.

The color of the points was chosen based on value, with the value being set to the code column, which indicates what the reason for each delay was. The actual column used was the timestamp column, and options like duration (how long the animation runs for, in this case the maximum time limit of 60 seconds) and trails (how long each event remains on the map, in this case set to just 2 to keep the animation fast-paced). In order to properly separate the animation into specific days, the time-series widget was added in the widget tab, located next to to the layer tab.

In the widget, the timestamp column was selected as the data source, the correct time zone was set, and the day bucket was chosen. Everything else was left as default.

The buckets option is there to select what time unit will be used for your time series. In theory, it is supposed to range from minutes to decades, but at the time of this project being completed, for some reason the smallest time unit available is day. This was part of the reason why the timestamp column is useful, as without it the limitations of the bucket in the time-series widget would have resulted in the map being nothing more then a giant pulse of every delay that happened that day once a day. With the time-stamp column, the animation feature in the style tab was able to create a chronological animation of all of the delays which, when paired with the widget was able to say what day a delay occurred, although the lack of an hour bucket meant that figuring out which part of the day a delay occurred requires a degree of guesswork based on where the indicator is, as seen below

Finally, a legend needed to be created so that a viewer can see what each color is supposed to mean. Since the different colors of the points are based on the incident code, this was put into a custom legend, which was created in the legend tab found in the same toolbar as style. Unfortunately this proved impossible as the TTC has close to 200 different codes for various situations, so the legend only included the top 10 most common types and an “other” category encompassing all others.

And that is all it took to create an interesting and informative time-series map. As you can see, there was no coding involved. A few years ago, doing this map would have likely required a degree of coding, but Carto has been making an effort to make its software easy to learn and easy to use. The result of the actions described here can be seen below.

https://alexandershatrov.carto.com/builder/8574ffc2-9751-49ad-bd98-e2ab5c8396bb/embed

Global Impacts of Earthquakes and Natural Disasters

Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018

Author: Katrina Mavrou

Date: Tuesday November 13th 2018

Topic: Global Impacts of Earthquakes and Natural Disasters

Link: Natural Disasters: Earthquake

Purpose:

To educate an audience with a wide variety of educational backgrounds on natural disaster impacts and side effects through visualization of spatial data using Esri’s Story Map Applications. As well, to gain a personal understanding  and experience using ArcGIS online and available tools.

Data Source: USGS Earthquake Database, 2018; Global International Displacement Database, 2018.

Software: ArcGIS Pro, ArcGIS Online

Outline:

The topic of this project is Natural Disaster impacts, in specific Earthquakes, and analyzes the global displacement and infrastructure loss. The platform used is ArcGIS Online, using hosted feature layers, web maps and mapping applications, such as a variety of Esri’s story map applications. The data is provided from USGS Earthquake database and Global International Displacement Database. The purpose of this exercise is to utilize multiple Esri story map functions, and other online plat forms into one map.

The first step was to clean up the datasets. The CSV files were imported into Microsoft SQL Server Management Studio to preform spatial queries and extract data needed from each dataset for the displacement dataset data was extracted by year and imported into a new table. The subset data was imported into ArcGIS Pro, where is was joined to the global shapefile and exported as new layers. Each year created a new layer so that the animation tool could be used on the time enabled field to create a time lapse of displacement. Each new layer was exported as a shapefile zipped folder so that it could be imported into ArcGIS Online as a hosted feature layer.

The first section, monthly earthquake report, of the story map is created by using a live link to publish data. To do this a web map is created using the live updating CSV from USGS. The data is collected and published every 5 minutes, therefore each time the story map is opened or refreshed the map will look different from the previous time. The data is displayed using the magnitude and depth of the earthquake (Image 1). This process was repeated for the second section, the weekly earthquake report. Another feature that was used for this map is pop-ups. Each instance displayed on the map when clicked will introduce a pop-up window that gives the user more information on that specific earth quake.

Live data of Monthly Earthquakes

Image 1: Live data of Monthly Earthquakes

The next section introduces the user to fault lines, specifically San Andreas Fault located in California, USA (Image 2). Using KML and KMZ files LiDAR layers of the fault lines are displayed on a satellite image map. Data of the most tragic earthquakes are also displayed on the map. This is historic data of earthquakes with magnitude greater than 5. By clicking on each earthquake location a pop-up is enabled and gives historical facts about each instance.

San Andreas Fault and Earthquakes

Image 2: San Andreas Fault and Earthquakes

The following section of the story map regards displacement caused by earthquakes. The main screen includes a heat map of displaced people from 1980-2018. The side panel includes a story map slide show. This was created from a simple web map with time enabled layers. The presentation function was then used to create a time lapse of global displacement for each year from 2010 to 2017 (Image 3). The presentation was then embedded into the side panel of the displacement section of the story map.

Presentation of Timelapse

Image 3: Presentation timelapse

The slide displacement VS population includes a swipe and slide story map embedded into the main story map. This is a global map that swipes two layers global displacement due to natural disasters for 2017 and world population for 2017 (Image 4). The swipe and slide map allows the user to easily compare the two layers side-by-side.

Swipe and Slide Map

Image 4: Swipe and Slide Map

The last section uses a story map shortlist to display images of earthquake impacts. This uses geocoded images to place images on a map (Image 5). The main panel holds with map with pointers indicating the coordinates of the geocoded images. The side panel displays the images. When an image is clicked on it pops up with the map and information regarding the image and earthquake can be found.

Geocoded image and reference map

Image 5: Geocoded image and reference map

The purpose of this technology is to easily display spatial data for individuals who are unfamiliar with utilizing other Esri products. Story maps are an easy and interactive way for users will all backgrounds of knowledge to interact with spatial maps and learn new information on a topic.

Limitations and Summary:

The final project is a story map journal with easy navigation and educational purpose. In the future I would like to incorporate even more of Esri’s online functions and applications, to expand my understanding. As well, a limitation to the project is the amount of space allotted to each student at Ryerson with ArcGIS online. Some processing functions were unavailable or used too much space, therefore these procedures had to be processed using ArcPro and then import the final product into ArcGIS online. Overall, the project reached its purpose of providing contextual information on spatial impacts of earthquakes.

Invasive Species in Ontario: An Animated-Interactive Map Using CARTO

By Samantha Perry
Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018

My goal was to create an animated time-series map using CARTO to visualize the spread of invasive species across Ontario. In Ontario there are dozens of invasive species posing a threat to the health of our lakes, rivers, and forests. These intruding species can spread quickly due to the absence of natural predators, often damaging native species and ecosystems, and resulting in negative effects on the economy and human health. Mapping the spread of these invasive species is beneficial for showing the extent of the affected areas which can potentially be used for research and remediation purposes, as well as awareness for the ongoing issue. For this project, five of the most problematic or wide-spread invasive species were included in an animated-interactive map to show their spatial and temporal distribution.

The final animated-interactive map can be found at: https://perrys14.carto.com/builder/7785166c-d0cf-41ac-8441-602f224b1ae8/embed

Data

  1. The first dataset used was collected from the Ontario Ministry of Natural Resources and Forestry and contained information on invasive species observed in the province from 1982 to 2012. The data was provided as a shapefile, with polygons representing the affected areas.
  2. The second dataset was downloaded from the Early Detection & Distribution Mapping System (EDDMapS) Ontario website. The dataset included information about invasive species identified between 2010 and 2018. I obtained this dataset to supplement the Ontario Ministry dataset in order to provide a more up-to-date distribution of the species.

Software
CARTO is a location-intelligence based website that offers easy to use mapping and analysis software, allowing you to create visually appealing maps and discover key insights from location data. Using CARTO, I was able to create an animated-interactive map displaying the invasive species data. CARTO’s Time-Series Widget can be used to display large numbers of points over time. This feature requires a map layer containing point geometries with a timestamp (date), which is included in the data collected for the invasive species.

CARTO also offers an interactive feature to their maps, allowing users control some aspects of how they want to view the data. The Time-Series Widget includes animation controls such as play, stop, and pause to view a selected range of time. In addition, a Layer Selector can be added to the map so the user is able to select which layer(s) they wish to view.

Limitations
In order to create the map, I created a free student account with CARTO. Limitations associated with a free student account include a limit on the amount of data that can be stored, as well as a maximum of 8 layers per map. This limits the amount of invasive species that can be mapped.

Additionally, only one Time-Series Widget can be included per map, meaning that I could not include a time-series animation for each species individually, as I originally intended to. Instead, I had to create one time-series animation layer that included all five of the species. Because this layer included thousands of points, the map looks dark and cluttered when zoomed out to the full extent of the province (Figure 1). However, when zoomed in to specific areas of the province, the points do not overlap as much and the overall animation looks cleaner.

Another limitation to consider is that not all the species’ ranges start at the same time. As can be seen in Figure 1 below, the time slider on the map shows that there is a large increase in species observations around 2004. While it is possible that this could simply be due to an increase in observations around that time, it is likely because some of the species’ ranges begin at that time.

Figure 1. Layer showing all five invasive species’ ranges.

Tutorial

Step 1: Downloading and reviewing the data
The Ontario Ministry of Natural Resources and Forestry data was downloaded as a polygon shapefile using Scholars GeoPortal, while the EDDMapS Ontario dataset was downloaded as a CSV file from their website.

Step 2: Selection of species to map
Since the datasets included dozens of different invasive species in the datasets, it was necessary to select a smaller number of species to map. Determining which species to include involved some brief research on the topic, identifying which species are most prevalent and problematic in the province. The five species selected were the Eurasian Water-Milfoil, Purple Loosestrife, Round Goby, Spiny Water Flea, and Zebra Mussel.

Step 3: Preparing the data for upload to CARTO
Since the time-series animation in CARTO is only available for point data, I had to convert the Ontario Ministry polygon data to points. To do this I used ArcMap’s “Feature to Point” tool which created a new point layer from the polygon centroids. I then used the “Add XY Coordinates” tool to get the latitude and longitude of each point. Finally, I used the “Table to Excel” conversion tool to export the layer’s attribute table as an excel file. This provided me with a table with all invasive species point data collected by the Ontario Ministry that could be uploaded to CARTO.

Next, I created a table that included the information for the five selected species from both sources. I selected only the necessary columns to include in the new table, including; Species Name, Observation Date, Year, Latitude, Longitude, and Observation Source. This combined table was then saved as an excel file to be uploaded to CARTO.

Finally, I created 5 additional tables for each of the species separately. These were later used to create map layers that show each species’ individual distribution.

Step 4: Uploading the datasets to CARTO
After creating a free student account with CARTO, I uploaded the six datasets as excel files. Once uploaded, I had to change the “Observation Date” column from a “string” to “date” data type for each dataset. A “date” data type is required for the time-series animation to run.

Step 5: Geocoding datasets
Each dataset added to the map as a layer had to be geocoded. Using the latitude and longitude columns previously added to the Excel file, I geocoded each of the five species’ layers.

Step 6: Create time-series widget to display temporal distribution of all species
After creating a blank map, I added the Excel file that included all the invasive species data as a layer. I then added a Time-Series Widget to allow for the temporal animation. I then selected Observation Date as the column to be displayed, meaning that the point data will be organized by observation date. I chose to organize the buckets, or groupings, for the corresponding time-slider by year.

Since “cumulative” was not an option for the Time-Series layer, I had to use CARTCSS to edit the code for the aggregation style. Changing the style from “linear” to “cumulative” allowed the points to remain on the screen for the duration of the animation, letting the user see the entire species’ range in the province. The updated CSS code can be seen in the screenshots below.

Step 7: Creating five additional layers for each species’ range
Since I could only add one Time-Series Widget per map, and the layer with the animation looks cluttered at some extents, I decided to create five additional layers that show each of the species’ individual observation data and range.

Step 8: Customizing layer styles
After adding all of the layers, a colour scheme was selected where each of the species’ was represented by a different colour to clearly differentiate between them. Colours that are generally associated with the species were selected. For example, the colour purple was selected to represent Purple Loosestrife, which is a purple flowering plant. The “multiply” style option was selected, meaning that areas with more or overlapping occurrences of invasive species are a darker shade of the selected colour.

A layer selector was included in the legend so that users can turn layers on or off. This allows them to clearly see one species’ distribution at a time.

Step 9: Publish map
Once all of the layers were configured correctly, the map was published so it could be seen by the public.

Mapping Toronto Green Space in Android

By Jacob Lovie | GeoVis Project Assignment                @RyersonGeo | SA8905 | Fall 2018

Introduction

With today’s technology becoming more and more mobile, and the ability to access everything you need on your mobile device, it is more important than even to ensure that GIS is evolving to meet these trends. My GeoVisualization project focused on designing an android application to allow users to explore Toronto’s green space and green initiatives, making layers such as parks and bike stations accessible in the palm of your hand. However, it is not just having access to this that is important. What’s important when working with these technologies is that a user can explore the map and retrieve the information seamlessly and efficiently.

Data and Hosting Feature Services

All the data for the project was retrieved from the city of Toronto’s open data portal. From there, all the data was uploaded to ArcGIS Online and set up as hosted feature services. A base map was also designed using ArcGIS for Developers and hosted. The application was able to target these hosted feature layer and use them in the map, making the size of the app small. The symbology and setup of the hosted feature layers was also done in ArcGIS online, so the app didn’t have to make any changes or set symbology when it wasn’t necessary.

Methods

The developer environment that I worked in to design my app was Android Studio, the baseline for designing any android apps. The programming language used in Android Studio is Java. Within Android Studio, the functionality of ArcGIS Runtime Software Developer Kit (SDK) for Android can be brought in, bringing in all the libraries and functions associated with ArcGIS Runtime SDK for Android. With this I was able to use ArcGIS functionality in android, designing maps, accessing hosted feature services, and perform geoprocessing.

Understanding how ArcGIS SDK for Android worked with in Android Studio was an important key in designing my app. When creating a map, I first had to create a Map object. An object is a variable that is of a certain datatype. If you were talking about having a text object as a variable that could be called, it would be of datatype string, and the word itself would be an object that is callable and referential. The Map object is what is displayed in an activity window (more on this later), which is what the user visualizes when using the app. The map can be set to view a certain area, which was Toronto in my app. A user can pan around the map like they would on any interactive map without having additional coding in Android Studio (it is natural to the Map datatype). The Map also has associated Layer objects that have their own set of parameters.

While designing my app, any time I would want something done in my design, such as creating a map object or adding a layer to a map object, I created a function that wold performs an action. This reduces repetition in the code when I attempted to do something complex multiple times. I designed 3 functions. The first was to create a Map, the second was to add a Layer that could be activated and deactivated in the Map through a switch that would be displayed in the main Activity Window. The final function added a layer that could be queried and would extract information from that layer.

When designing an android app, there are many fine details that are not necessarily considered when using an app on your phone. Simple things like having a window or text appear, opening a second window, or displaying information were things I very much appreciated after designing the app. Within my app, I wanted to display a second activity window to display information on neighbourhoods in Toronto when the user touched them. Within Android Studio this required creating a second activity window, and transferring the information obtained in the map to the second activity. This was done through my displayInformation function. I was then able to create a second activity and display this information using a custom list display to show the attribute data of a selected neighbourhood.

     >>>>>>>>>>>     

Setting up the display in Android Studio is relatively simple. There is an interface that allows you to anchor different objects to parts of the screen. This allows the app to run smoothly across all devices, not based on the size and ratio of the device. The switches in my Main Activity window were anchored to the top left, and to each other. My Map is in the background, but appears as white in this activity window.

The Application

Once all the coding and testing was completed, running the app was simple. I was able to bundle my code and send it to my personal phone, a Galaxy S9. The functions called the hosted service layers and displayed them in the map (Wifi or internet connection was required). I was also able to click on neighbourhoods and it would open my second activity that displayed the attribute information of that neighbourhood. If you want a more in-depth look at my code, it is available at https://github.com/jclovie/GeoVis-Ryerson/.

Creating an animated map with CartoDB Torque and QGIS

Author: Melissa Dennison, November 18, 2015
Course Assignment, SA8905, Fall 2015 (Rinner)

I chose the history of Native Title claims in Australia as the topic for this map.   “Native Title” is a general term for the legal framework and process for indigenous land claims in Australia. Following the landmark 1992 Mabo v. Queensland (No. 2) case, which was the first legal recognition of the interest of indigenous Australians in their traditional lands and waters, the Australian Parliament passed the Native Title Act in 1993. Claims under the Act began in 1994. This video provides an excellent quick introduction to the topic:

https://www.youtube.com/watch?v=1RBJhQ4hE_8

My goal was to create an animated map in CartoDB showing Native Title activity across Australia, both active claims and determined outcomes, from Mabo to the present.

My data source was the National Native Title Tribunal (NNTT) open data portal. I faced the following constraints in achieving my goal:

  • CartoDB’s animation (Torque) function is available on points only, on one layer only
  • The NNTT spatial data is for polygons, and comes in 4 datasets
  • My CartoDB account only provided 100MB storage
  • Native Title is an extremely complex topic while an animated map is simplistic

Step 1:            Downloading and reviewing the data

On September 22, 2015, I downloaded the current Native Title claims and outcomes data, and the geospatial data model, from:

http://www.nntt.gov.au/assistance/Geospatial/Pages/Spatial-aata.aspx

The data came in four shapefiles, two for current claims and two for determined outcomes.

Current claims files Determined outcomes files
Schedule of Native Title Determination Applications

Registered Native Title Determination Applications

Determinations of Native Title

Determinations of Native Title ‐ Native Title Outcomes

The Schedule of Native Title Determination Applications lists claims that have been filed but have not yet passed the registration test (the test involves meeting certain legal requirements in order for the claim to proceed). The Registered Native Title Determination Applications file lists those claims that have passed the test. Working with the files I found some overlap between them, in that they referred to many of the same tribunal cases and land areas. (I expect this has to do with a time lag between when various files are updated on the Tribunal’s main database, and then extracted and uploaded to its open data portal. By downloading them all on the same day I inevitably got some outdated information.) I chose to work only with the Schedule for this project, as it contained the most Tribunal cases.

The determined outcomes files also overlapped, but contained different information. The Determinations of Native Title file indicates areas of land where Native Title was found to exist, to have been extinguished and/or a combination of each. The Determinations of Native Title Native Title Outcomes file shows areas where Native Title exists, exclusively or non-exclusively, or where Native Title does not exist or has been extinguished. Because there is no unique identifier in the dataset for each land area, only for claims and determinations (which can refer to multiple parcels of land), it was not possible to combine the files in such a way as to capture all the determination information for individual parcels of land. I chose to use both files in order to retain this information and just live with the overlap.

Note that Native Title claims that are withdrawn, discontinued, rejected, struck out or pre‐combined are transferred to a historical dataset that is not publicly available, and are therefore not shown on this map.

Step 2:                        Merging files in QGIS

The three shapefiles were too large, over 130MB, for my CartoDB account, so I loaded them into QGIS and used the MMQGIS plugin to merge the files. I named the new shapefile NTmerge1.

What to click in QGIS to merge files: MMQGIS>Combine>Merge Layers

Step 3:                        Preparing the shapefile for uploading to CartoDB

NTmerge1.shp was 116.8MB, still too big for my 100MB limit on CartoDB, so I had to reduce its size. Fortunately converting the polygons to points, which I had to do anyway because Torque only works on point data, was the solution. I named the new file NTpoints1. Its size was 428KB, well below my CartoDB limit.

What to click in QGIS to convert polygons to points: Vector>Geometry Tools>Polygon Centroids

Step 4:                        Uploading the data to CartoDB and preparing it for Torque animation

Uploading the data on CartoDB went smoothly, literally just clicking “New Dataset” and then selecting the file. Preparing the data for animation was a bit more involved. I should note that I actually used the Torque Cat (Category) function, which is a refinement of Torque in that it enables depiction of categories within a variable. However, it is only possible to run Torque Cat on one time column and one category column. As a merged file, NTpoints1.shp contained multiple date columns (such as “date lodged” for active claims and “determination date” for determined outcomes) and null values for active claims in the “determination outcome” column. Using simple SQL statements (seen in the table below) in CartoDB’s SQL workspace, I created a single date column and a single outcome category column.

To create and fill new date column To create and fill new category column
ALTER TABLE ntpoints1

ADD COLUMN dateall date;

UPDATE ntpoints1

SET dateall=datelodged

WHERE datelodged IS NOT NULL;

UPDATE ntpoints1

SET dateall=detdate

WHERE detdate IS NOT NULL;

ALTER TABLE ntpoints1

ADD COLUMN current CHAR(255);

UPDATE ntpoints1

SET current=’active claim’

WHERE datelodged IS NOT NULL;

UPDATE ntpoints1

SET current=detoutcome

WHERE detoutcome IS NOT NULL;

 

Step 5:                        Activating Torque Cat and finishing the map

In the CartoDB “Map layer wizard”, I selected Torque Cat, my newly-created columns as the time and category columns, and colours for each category. I set the duration of the animation to 60 seconds, and left the defaults for everything else.

In the CartoCSS editor (seen in the image below), I changed the default specification “linear” to “cumulative” so that the dots would remain on the map for a cumulative effect.

Screen Shot 2015-11-17 at 6.13.17 PM

Finally, I added a title and some text for further information.

The end result can be seen at:

https://melissadennison.cartodb.com/viz/0673f7fe-866d-11e5-92d7-0e3ff518bd15/public_map

 

TTC Subway Stations and LRT Expansion Animated 1954-2021

An animated look at TTC’s subways and LRT expansion by when they first opened. Includes 2017’s Finch West subway expansion and 2021’s Eglinton LRT expansion.

By Khakan Zulfiquar – Geovis Course Assignment, SA8905, Fall 2015 (Rinner)

As a course assignment, we were required to develop a professional-quality geographic visualization product that uses novel mapping technology to present a topic of our interest. I chose to create an animated-interactive map using CartoDB to visualize the construction of Toronto Transit Commission (TTC) stations from years 1954 to 2021. The interactive map can be found at https://zzzkhakan.cartodb.com/.

khakan_2

Project Idea

This idea was inspired by Simon Rogers who animated the London’s Rail System. It was interesting to see an animated map slowly draw together a footprint of an entire infrastructure system. A number of (non-interactive) animations of Toronto’s subway system development were collected by Spacing magazine in 2007 and can be viewed at http://spacing.ca/toronto/2007/09/21/ttc-subway-growth-animation-contest/.

A feature within CartoDB called “torque” was used to create the envisioned map. Torque is ideal for mapping large number of points over time. Torque has been famously used in media for mapping tweets as pings.

Execution

As a beginner to CartoDB, I had to go through tutorials and online courses to get familiar with the interface. As I became comfortable with CartoDB and its features, I recalled an example I had seen in the CartoDB gallery. It was Simon Roger’s London Rail System map. I knew exactly the kind of data I would need to make a similar map for TTC stations. There was an instant halt as the data was not readily available. Using Wikipedia, ttc.ca, and OpenStreetMap I was able to compile the data I required. The data was uploaded into CartoDB and the following map was created.

khakan_1

Tutorial / How-to-Use

For the heading numbers above, please find the associated instructions below.

  1. Title and Subtitle – Speaks for itself.
  2. Toronto Subway/ LRT Map [full resolution]- A Map of Toronto’s Subway and future LRT produced by the TTC.  This map is the most common visual representation of TTC’s subways and LRT.  The map’s color scheme was mimicked to help viewers, especially those familiar with TTC, make the transition to the animated map smoothly.
  3. Timeline – The timeline is in a continuous-loop.  You can press pause to stop the animation and resume to start the animation again.  You can also control the speed of the animation by sliding the play-bar back-and-forth.
  4. Hover Window – As you hover over the stations, a window will pop up automatically with the name of the station.  No clicks required.  The names will only appear if the “ttc_station” layer is switched on (more on this in step 7).
  5. Info Window – If you would like further information on a certain station, simply click on the station and you will be presented with the station’s name, line #, grade (above, at, or underground), platform type, and etc. The info window will only appear if the “ttc_station” layer is switched on (more on this in step 7).
  6. Legend – as the name implies…
  7. Layer Switch – a tool to turn on or off the layers being used in this map.  The map was created with the intent to be both animated and interactive.  The animated bit is the stations being plotted and the interactive part was for the user to find further information about the station. However, the animated bit is both intrusive and resource-heavy.  Because of this, an option is being included to turn layers on-or-off as required. Be sure to try out the combinations.
  8. MAIN SHOW – the main map area has a beautiful CartoDB Dark Matter basemap with all of the TTC stations plotted. Feel free to zoom in and out.

Enjoy viewing and exploring.

Story Swipe Map – 2011 / 2015 Election Results

Geovis Course Assignment, SA8905, Fall 2015 (Rinner)
Author: Austin Pagotto
Link to Web app: http://arcg.is/1Yf8Yqn
(Note: project may have trouble loading using Chrome – try Internet Explorer)

Project Idea:

The idea of my project was to comprehensively map the past two Canadian federal election results. When looking for visualization methods to compare this data I came across the Swipe feature on the ArcGIS Online story maps. Along with all the interaction features of any ArcGIS online web map, this feature lets the user swipe left and right to reveal either different layers or in my case different maps. As you can see in the screenshot below the right side of the map is showing the provincial winners of the 2015 election while the left side of the map is showing the provincial winners of the 2011 election. The middle line in the middle can be swiped back and forth to show how the provincial winners differed in each election.

Pic1

Project Execution:

The biggest problem in executing my project was that the default ArcGIS online projection is web Mercator, which greatly distorts Canada. I was able to find documentation from Natural Resources Canada explaining how Lambert Conformal Conic basemaps can be uploaded to an ArcGIS online map and replace the default basemaps.

Another problem with my visualization of the project was that when zoomed to a national scale level, a lot of the individual polling divisions became impossible to see. This creates an issue because each polling division is designed to have a somewhat equal population count in them. So the small ones aren’t less important or less meaningful than the big ones. To solve this, when zoomed out, I changed the symbology to show the party that had won the most seats in each province, so it would show the provincial winner as seen in the previous screenshot. When zoomed in however the individual polling divisions become visible, showing the official name at increased zoom levels. The years of each election were added to the labels to help remind the user what map was on what side.
pic2

The methodology I used to create this project was to create two different online maps, one for each election year. Then I created the swipe web app which would allow both of these maps to be loaded and swipeable between the two. It was important here to make sure that all the settings for each map were the exact same (colors, transparency and attribute names).

The data that is shown on my maps were all downloaded from ArcGIS online to Arcmap Desktop and then zipped and reuploaded back to my project.  It was important to change my data’s projection to Lambert Conformal Conic before uploading it so that it wouldn’t have to be reprojected again using ArcGIS online.

This project demonstrated how web mapping applications can make visualizing and comparing data much easier than creating two standalone maps.

Data Sources: Projection/Basemap information from Natural Resources Canada
Election Data from ESRI Canada (downloaded from ArcGIS Online)

Link to Web app: http://arcg.is/1Yf8Yqn