Creating an animated map with CartoDB Torque and QGIS

Author: Melissa Dennison, November 18, 2015
Course Assignment, SA8905, Fall 2015 (Rinner)

I chose the history of Native Title claims in Australia as the topic for this map.   “Native Title” is a general term for the legal framework and process for indigenous land claims in Australia. Following the landmark 1992 Mabo v. Queensland (No. 2) case, which was the first legal recognition of the interest of indigenous Australians in their traditional lands and waters, the Australian Parliament passed the Native Title Act in 1993. Claims under the Act began in 1994. This video provides an excellent quick introduction to the topic:

https://www.youtube.com/watch?v=1RBJhQ4hE_8

My goal was to create an animated map in CartoDB showing Native Title activity across Australia, both active claims and determined outcomes, from Mabo to the present.

My data source was the National Native Title Tribunal (NNTT) open data portal. I faced the following constraints in achieving my goal:

  • CartoDB’s animation (Torque) function is available on points only, on one layer only
  • The NNTT spatial data is for polygons, and comes in 4 datasets
  • My CartoDB account only provided 100MB storage
  • Native Title is an extremely complex topic while an animated map is simplistic

Step 1:            Downloading and reviewing the data

On September 22, 2015, I downloaded the current Native Title claims and outcomes data, and the geospatial data model, from:

http://www.nntt.gov.au/assistance/Geospatial/Pages/Spatial-aata.aspx

The data came in four shapefiles, two for current claims and two for determined outcomes.

Current claims files Determined outcomes files
Schedule of Native Title Determination Applications

Registered Native Title Determination Applications

Determinations of Native Title

Determinations of Native Title ‐ Native Title Outcomes

The Schedule of Native Title Determination Applications lists claims that have been filed but have not yet passed the registration test (the test involves meeting certain legal requirements in order for the claim to proceed). The Registered Native Title Determination Applications file lists those claims that have passed the test. Working with the files I found some overlap between them, in that they referred to many of the same tribunal cases and land areas. (I expect this has to do with a time lag between when various files are updated on the Tribunal’s main database, and then extracted and uploaded to its open data portal. By downloading them all on the same day I inevitably got some outdated information.) I chose to work only with the Schedule for this project, as it contained the most Tribunal cases.

The determined outcomes files also overlapped, but contained different information. The Determinations of Native Title file indicates areas of land where Native Title was found to exist, to have been extinguished and/or a combination of each. The Determinations of Native Title Native Title Outcomes file shows areas where Native Title exists, exclusively or non-exclusively, or where Native Title does not exist or has been extinguished. Because there is no unique identifier in the dataset for each land area, only for claims and determinations (which can refer to multiple parcels of land), it was not possible to combine the files in such a way as to capture all the determination information for individual parcels of land. I chose to use both files in order to retain this information and just live with the overlap.

Note that Native Title claims that are withdrawn, discontinued, rejected, struck out or pre‐combined are transferred to a historical dataset that is not publicly available, and are therefore not shown on this map.

Step 2:                        Merging files in QGIS

The three shapefiles were too large, over 130MB, for my CartoDB account, so I loaded them into QGIS and used the MMQGIS plugin to merge the files. I named the new shapefile NTmerge1.

What to click in QGIS to merge files: MMQGIS>Combine>Merge Layers

Step 3:                        Preparing the shapefile for uploading to CartoDB

NTmerge1.shp was 116.8MB, still too big for my 100MB limit on CartoDB, so I had to reduce its size. Fortunately converting the polygons to points, which I had to do anyway because Torque only works on point data, was the solution. I named the new file NTpoints1. Its size was 428KB, well below my CartoDB limit.

What to click in QGIS to convert polygons to points: Vector>Geometry Tools>Polygon Centroids

Step 4:                        Uploading the data to CartoDB and preparing it for Torque animation

Uploading the data on CartoDB went smoothly, literally just clicking “New Dataset” and then selecting the file. Preparing the data for animation was a bit more involved. I should note that I actually used the Torque Cat (Category) function, which is a refinement of Torque in that it enables depiction of categories within a variable. However, it is only possible to run Torque Cat on one time column and one category column. As a merged file, NTpoints1.shp contained multiple date columns (such as “date lodged” for active claims and “determination date” for determined outcomes) and null values for active claims in the “determination outcome” column. Using simple SQL statements (seen in the table below) in CartoDB’s SQL workspace, I created a single date column and a single outcome category column.

To create and fill new date column To create and fill new category column
ALTER TABLE ntpoints1

ADD COLUMN dateall date;

UPDATE ntpoints1

SET dateall=datelodged

WHERE datelodged IS NOT NULL;

UPDATE ntpoints1

SET dateall=detdate

WHERE detdate IS NOT NULL;

ALTER TABLE ntpoints1

ADD COLUMN current CHAR(255);

UPDATE ntpoints1

SET current=’active claim’

WHERE datelodged IS NOT NULL;

UPDATE ntpoints1

SET current=detoutcome

WHERE detoutcome IS NOT NULL;

 

Step 5:                        Activating Torque Cat and finishing the map

In the CartoDB “Map layer wizard”, I selected Torque Cat, my newly-created columns as the time and category columns, and colours for each category. I set the duration of the animation to 60 seconds, and left the defaults for everything else.

In the CartoCSS editor (seen in the image below), I changed the default specification “linear” to “cumulative” so that the dots would remain on the map for a cumulative effect.

Screen Shot 2015-11-17 at 6.13.17 PM

Finally, I added a title and some text for further information.

The end result can be seen at:

https://melissadennison.cartodb.com/viz/0673f7fe-866d-11e5-92d7-0e3ff518bd15/public_map

 

Displaying Brooklyn’s Urban Layers by Mapping Over 200 Years of Buildings

Renad Kerdasi
Geovis Course Assignment
SA8905, Fall 2015 (Rinner)

Growth in Brooklyn
Located at the far western end of Long Island, Brooklyn is the most populous of New York City’s five boroughs. The borough began to expand between the 1830s and 1860s in downtown Brooklyn. The borough continued to expand outwards as a result of a massive European immigration, the completion of the Brooklyn Bridge connecting to Manhattan, and the expansion of industry. By mid 1900s, most of Brooklyn was already built up as population increased rapidly.

Data
The data in the time series map are from PLUTO, which is a NYC open data site created by NYC Department of City Planning and released in 2015. The data contain information about each building located in the boroughs, including the year the construction of the building was completed (in numeric 4 digits format) and the building footprints. The building years range from 1800 to 2015, there are some missing dates in the dataset as well as some inaccuracy in the recorded dates. The data are available in Shapefile and Windows Comma Separated format, found on NYC Planning website: http://www.nyc.gov/html/dcp/html/bytes/dwn_pluto_mappluto.shtml

The Making of the Time Series
To present the structural episodes of Brooklyn’s built environment, QGIS 2.10 was utilized with the Time Manager plugin. QGIS is an open source GIS application that provides data visualization, editing, and analysis through functions and plugins (https://www.qgis.org/en/site/about/). The Time Manager plugin animates vector features based on a time attribute (https://plugins.qgis.org/plugins/timemanager/). This tool was effective in presenting a time series of Brooklyn’s building construction dates.

To create the time series, the PLUTO SHP was downloaded and prepared by removing any unnecessary fields. The columns of interest are: FID, Shape, and YearBuilt. Because we are interested in the time column, the formatting must fit with QGIS Time Manager. QGIS Time Manager requires timestamps to be in YYYY-MM-DD format whereas the building dates in the PLUTO SHP are in a four-digit format. Therefore, the date in the dataset must be modified to fit the Time Manager format before it can be brought into QGIS.

Table 1_BrooklynData

In QGIS, Time Manager plugin must be installed first. The SHP can then be added into QGIS as well as other Shapefiles needed: roads, highways, state boundaries, etc. Note: to use Time Manager, the data must be in SHP format.

Layer_BrooklynData

Once the data are added, the polygons (i.e. buildings) are styled based on age. This will be effective in distinguishing the oldest buildings from the newest. In QGIS, there are a large number of options available to apply different types of symbology to the data. The layer is styled based on the attribute Year Built, since the time series will show urban layers using building dates. Also, Graduated is chosen because features in the layer should not be styled the same way. The other data file, such as roads, highways, and state boundaries, are styled as well.

Once all the data are added and styled, it can be oriented and applied to the Time Manager plugin. To truly see the urban layers, the map is zoomed on the upper portion of Brooklyn. In Time Manager settings, the layer with building dates is added and the Start Time is the Year Built field, which includes the timestamp data. To get features to be configured permanently on the map, in the End Time option “No End Time” is selected. For animation options, each time frame will be shown for 100 milliseconds, and timestamp (i.e. built year) will be displayed on the map.

Layer_BrooklynData

In the Time Manager dock, the time frame is changed to years since the animation will be showing the year the construction of the building was completed. The size of the time frame will be 5 years. With these settings, each frame will display 5 years of data every 100 millisecond. Playing the video will display the animation inside QGIS, and one can see the time scrolling from 1800-2015 in the dock.

Dock_BrooklynData

Time Manager also enables you to export the animation to an image series using the “Export Video” button. Actual video export is not implemented in Time Manager. To play the animation outside of QGIS, various software applications can be used on the resulting image series to create a video file.

In addition, QGIS only allows users to insert a legend and title in the Composer Manager window. Currently, it is not possible to get the legend rendered in the main map window. One approach to generate a video with a legend is to create a dummy legend and add the image containing the legend into the PNGs that Time Manager produces. A dummy legend and a title for Brooklyn’s urban layers was created outside of QGIS, and added to each PNG.

Finally, to create a time-lapse and compile the images together, Microsoft Movie Maker was utilized. Other software applications can be used, including mancoder and avidemux.

Results

Link: https://youtu.be/52TnYAVxN3s

Working with WMS-T layers and Time Manager in QGIS

By: Lauren Blumberger
Geovis Course Assignment, SA8905, Fall 2015 (Dr. Rinner)

Are you ever troubled by the amount of large files that you need to download and store on your computer?  If so, a Web Map Service (WMS) may be useful as it delivers georeferenced map images over the internet that are generated by a map server. All you need is an internet connection to be able to access maps from a remote server and load them into your GIS. A request is made, and the response is one or more map images (returned as PNG, JPEG etc.). While WMS services are commonly used for retrieving base maps, they also provide a quick and easy way to work with data that has already been symbolized and styled by the author.  Many WMS servers also have time support, WMS-T, which enables users to define a temporal subset for the rendering of layers that are properly configured with a time dimension.

In this post, I explore how to work with WMS-T retrieved maps in QGIS and how to animate them using Anita Graser’s Time Manager plugin (http://anitagraser.com/projects/time-manager/ and https://plugins.qgis.org/plugins/timemanager/). Anita Graser’s Blog includes a guest post by Karolina Alexiou on using Time Manager for WMS-T layers. I will describe the process, as well as discuss the limitations.

The first step is to find the service layers that you want to work with. This is not always the easiest task as you must use the GetCapabilities URL to access the maps, which in some cases doesn’t reveal the image until the connection is made from QGIS to the server. Thus, there might be some trial and error in terms of retrieving the maps you are looking for.

The map I use for this demonstration is made available by the Fire Information Resource Management System (FIRMS), who offer fire-based map images. The WMS-T allows access to FIRMS Active Fire Hotspots for a specific day. The GetCapabilities URL is provided on the website:

https://firms.modaps.eosdis.nasa.gov/wms-t/?SERVICE=WMS&VERSION=1.1.1&REQUEST=GETCAPABILITIES

Navigate to Add WMS/WMTS Layer in QGIS and click on new. Add the above URL and name the connection FIRMS. Make sure to enable the Ignore GetMap/GetTile URI reported in capabilities, as this will redirect you to the WMS instead of the WMS-T. When you click connect, multiple layers will appear. Load the MODIS_Hotspots layer to the canvas.*2

Now that you have connected to the server you can overlay this image with other layers accessed through servers or available locally. Download any country boundary shapefile and load it to the canvas. Now you can change the style of this new layer and adjust the transparency to make sure that you can see the MODIS image underneath.

new

The next step involves the TimeManager plugin, which provides some support for stepping through this spatial temporal data.** Once you have installed the plugin, click on settings and add raster layer. The WMS-T standard uses many different time formats and the plugin requires you to know this format for inputting the start and end times. Therefore, you must examine the XML document with the web service description and find the section that defines the format of the time dimension. For this example, add the MODIS layer and input start time as 2005-11-16 and end time as 2015-11-16. Set the time step to months and press play. Now you will see that TimeManager renders each month interval by querying the FIRMS WMS-T.window

Since WMS-T is a live service that will automatically update on the map, there are significant delays when working with these layers. Querying the web service and waiting for response takes time, and thus, the interactive mode using TimeManager does not result in a visually pleasing animation. TimeManager includes an option to export all the frames, which you can then move into another software to create a smooth animation.

The resulting animation shows the MODIS Active Fire Hotspots globally for the past ten years. The visualization works best if you maximize the video to full screen view.

Overall, working with WMS-T servers can be tricky. As there are many different time formats that can be used, it is quite a complex task to support them all in a GIS. Also, while you want to be connected to the server to be able to access live, current data, this connection slows down the rendering process and creates choppy animations. This being said, exporting the frames to create an animation provides a nice result and a fast way to visualize current data. As FIRMS constantly provides access to active fire data with the MODIS satellite, through reconnecting to the WMS-T and adjusting the time I can update the animation to 3 hours ago.

*The MODIS satellite detects both flaming and smoldering fires ~1000m2

**Note that FIRMS WMS-T only supports one date per request and date ranges are not allowed. To make Time Manager send the correct request, the source code file wmstlayer.py must be adjusted. If Time Manager is installed, the file can be found in the user folder on your computer. Replace lines 59 to 63 with:

self.layer.dataProvider().setDataSourceUri(self.IGNORE_PREFIX + \
self.originalUri + self.addUrlMark() + “TIME={}” \
.format(
time_util.datetime_to_str(startTime, self.timeFormat)))

This ensures that Time Manager only sends a point in time. A time range will return a black image for this WMS-T.

Special thanks to Dr. Eric Vaz and Anita Graser for their help.

Building Visualizations Using D3

By Jessica Whitehead, Geovis Course Assignment, SA8905, Fall 2015 (Rinner)


“D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.” 

d3js.org


OVERVIEW: D3 has a steep learning curve and for someone who has only dabbled in html and css, it did not come easy. Fortunately there is a growing community of D3 and javascript geniuses online, allowing relatively inexperienced programmers, like myself, to piece together chunks of code and create something (surprisingly) visually-pleasing and informative.


THE DATA

I decided to stick with a simple dataset. The World Food Programme (WFP) maintains an online database of international food aid deliveries that can generate reports and tables based on the variables chosen by the user. For this project I looked at food aid deliveries in the year 2012 and produced 3 separate tables that would translate well into data visualizations.

  1. The first table was an overview of international food aid deliveries. It provided the total amount delivered for the the year, as well as breaking it down by food aid types and delivery modes.
  2. The second table summarized all recipient countries, the metric tons received, and for which food aid type. I reduced the dataset by focusing on the top 15 recipient countries.
  3. The third table summarized all donor countries, the metric tons donated, and to which countries it was delivered. I reduced the dataset by focusing on the top 15 donor countries and amalgamating the recipient countries into regions using categories specified on the WFP website.*

*As this project had a geospatial focus, I did not include donors that were not countries, such as the UN or private organizations.


THE CREATIVE PROCESS

So, back to D3. Looking at my data tables, I started exploring the wide array of examples available on the D3.org and GitHub websites. It was important that I found codes that could incorporate my data, either within the code or through .csv file.

Dancing BubblesEventually I found the “dancing bubbles” example and thought this would be an interesting way to show the different types of food aid and delivery modes. I was able to download the code off GitHub, however, it included hundreds of additional files that were unnecessary. As a D3 novice, I did not realize most of the files were irrelevant for a long time.
Source code: http://vallandingham.me/bubble_charts_in_d3.html

To show dashboardthe top 15 recipients I liked the idea of an interactive dashboard. This would allow the user to hover over a country and see a breakdown by food category. For example, North Korea received 372,555 mt of food in 2012 and all deliveries were for emergency purposes.
Source code: http://bl.ocks.org/NPashaP/96447623ef4d342ee09b

mapitTo show the top 15 donors I wanted to create a choropleth map that would produce a donut graph of receiving regions when hovering over each donor country. This proved to be very difficult, so for this project I ended up creating  the map and donut infographic separately.
Source code: http://bl.ocks.org/mbostock/3888852


THE CODING

interactivedatacoverA very useful resource I found was a book called Interactive Data Visualization for the Web written by Scott Murray and is available online for free at Interactive Data Visualization Text

It provides you with all the information required to get started with D3, explaining everything from web fundamentals to D3 setup and creating a web server. Most browsers have issues when interpreting D3 on your local computer, so it is necessary to set up a local web server to view your output.

A very brief look at how D3 works

Essentially you have to download the D3 library and store in a “js” javascript folder in the home directory of your website. Screen Shot 2015-11-18 at 4.34.46 AM Here you will also have folders for css (the style code that makes your visualizations look cool), data (the csv and GeoJson files that give meaning to your visualizations and shape your maps), and index html file (home page of your website).Screen Shot 2015-11-18 at 4.43.24 AM

You start off with a simple index html file and reference the D3 library in the head of your html. Your actual script will go in the body so that it can append items into the Document Object Model (DOM). The DOM is essentially the structure of your html document. D3 uses javascript to then bind your csv and Json files to SVG elements in the DOM. These SVG objects can range from simple rectangles forming bar charts to extremely complicated interactive visualizations.


TOP 3 THINGS ABOUT D3

  1. Web-based – so people can easily access and view the visualizations you spend so long creating.
  2. You are not limited by pre-built charts – endless room for creativity.
  3. D3 programmers seem to be pretty generous when uploading their codes to the web. There is wide variety of open source codes available online to manipulate.

END PRODUCT

If you would like to see the final product please see below
http://www.jwdataviz.ca/sa8905-geovis/


Developing a 3-Dimensional Model of the Everest Region in Nepal Using Blender

By Matthew Abraham for SA8905 Geovis course project (Dr. Rinner)

This Geovisualization project developed a working 3D model of the Sagarmatha region using graphics and animation software, Blender, culminating in a fly-through of the region outlining all mountains above 8,000m. The purpose of this blog is to detail the steps needed to develop a 3D model of any desired region around the world, using Blender, and is therefore not limited to this one example.

Blender is a free open-source graphics and animation program that has many uses beyond what was explored in this project. Many of their open film projects can be seen on their website at https://www.blender.org/features/projects/. Since this program has incredible diversity in its applications and can create photorealistic imagery, I chose it to produce a 3D mapped mountain environment of the Everest region in Nepal, combining both graphic design and geospatial data. It should be noted that this entire project was done in Blender Cycles (a version of Blender).Technology Blog Pics

This process from geography to 3D model included four critical steps and involved two core programs. The steps were:

  1. Collect geospatial data and identify the size of the region analyzed;
  2. Process this geospatial data within a geographic information system (GIS) – Quantum GIS;
  3. Convert the map to a 3D model using Blender – an open-source graphics animation program; and
  4. Process and develop a fly-through of the desired mountains.

The first step involved extracting digital elevation model (DEM) data from the US Geological Survey website, using http://earthexplorer.usgs.gov/ to define the region of interest. Using a simple search for Mount Everest on Earth Explorer, the region was pulled up. Once the region was located, multiple points were used to help define the regions of interest for data acquisition. Once the region was selected, ASTER DEM data was pulled for all four Lat/Long regions identified by Earth Explorer.

Technology Blog pic 2

After downloading the DEM data, it was uploaded into QGIS to merge and crop the four DEM layers to develop the zone of interest. Raster –> Miscellaneous –> Merge was used in order to combine the underlying four DEM layers into one crop-able sheet. Next, a Raster Clipper tool was used to select the desired region from the merged DEM layer as shown below. This clipped section was saved as a TIFF file to be imported into Blender.

Technology Blog pic 3

Once the desired DEM region was converted into a TIFF file, the work could begin in Blender. Upon opening up an empty project in Blender, the user is given an empty canvas with a cube in the center of the 3D matrix as seen here:

Technology Blod pic 3

The first step involved deleting the cube by pressing X and then clicking “delete”. Next, it was necessary to bring in a blank plane to display the geospatial data. This was done by using the shortcut, Shift-A, and then selecting under Mesh –> Plane. This produced a blank plane in the centre of the grid, where the cube was located.

Tech Blog pic 5

The next step was to subdivide the plan into multiple grids. This was done in the Edit Mode by hitting tab and then scrolling down in the Transform sidebar to Subdivide. Subdivide breaks the plane down into as many smaller planes as desired, however the more subdivisions there are, the more information and challenge it is for your computer to handle the detail. For the purpose of this assignment and the limitations of my computer, 500 subdivisions were made to the plane creating over 250,000 squares and 370,000 vertices.

Once the plane was subdivided, the plane was scaled up to a more appropriate size. In order to make it easier to scale up, units were given to the plane by going to the Scene tab and changing the units to “Metric”. In the Transform sidebar, the plane was scaled up to 500 by 500 meters. Although this is not the actual scale of the DEM region we are looking at, it provide enough size and detail to appropriately map the desired region.

After the plane as set up and given an appropriate scale, it was necessary to import the geospatial data onto this plane. This was done by going to the Modifier tab and then selecting a “Displace” modifier from the pull-down menu. Click “New Texture”, and then under Type select “Image or Movie”. Under image, select the TIFF file saved earlier.

pic8Tech Blog 7

The plane at this point will not show the features, mainly due to the strength of the displacement. This can be adjusted by going back to the Modifier tab and changing the strength of the displace modifier. The strength can be adjusted until the desired look is achieved. It was also necessary to adjust the Z-axis location to be half of the displaced value in order to account for the displacement effects. The following was the result:

Tech Blog pic 6

The next step was adding texture to this terrain to give it some realistic definition and colouring. There are multiple methods for texturing in Blender. One step explored in this project was the use of a colour ramp based on vertical height. This was all done in the Node Editor and involved multiple nodes. This first node was a Texture Coordinate, which tells Blender what object the colours will apply to. In this case “Generated” was selected, as it would automatically generate the colours on the desired object. Next a Separate XYZ node was used to separate the desired Z-axis to create a vertical layering of the selected colours. After separating the Z-axis, a Mapping node was added to help further identify the locations of the colours on the object, specifying the Z-axis under Texture. Next a Gradient Texture was used alongside a ColorRamp node to develop the desired colour ranges for the 3D plane. Colours were chosen based on personal examination of the mapped region, going from a dark green and brown for low-lying forests to white for snow-capped peaks. This is all part of a Diffuse BSDF, which is a tool that creates the material for the desired plane. The resulting rendered image shows how the colour ramp looks on the 3D plane.

Pic9

Pic10

The second last thing to add prior to creating the fly-through was the sky texture, which was done by simply going to the World tab and under Surface and selecting “Background” –> “Sky Texture”. The intensity of the Sun and location can be altered using the provided Vector and Colour mapping options. For this project, Ambient Occlusion was turned on as it added more realism to the lighting of the 3D plane.

Lastly, text was added to the map to identify the four mountains above 8,000 meters. This was done through Shift-A, and selecting “Text”. The scale of the text was adjusted using the same method as done for the plane. Next, the text was given 3-dimensions through the Depth option, and the appropriate text was written in Edit Mode. This was then rotated and moved to a location on the 3D model as shown here:

Pic 11

Once the map was ready, I added camera animations to fly-through the mountains. This was done by first creating a path for the camera to rest on. Once again, this was accessed by pressing Shift-A, going to Curve, and selecting “Path” at the bottom. The desired path can then be scaled and shaped by moving the XYZ vector points of the line.

Pic 12

Once this path was adjusted to the appropriate location, I added a Follow Path object constraint to the camera under the Constraints tab. After adding the constraint, change Forward direction to “–Z”, and Up to “Y”. In addition to fixing the camera’s orientation, I defined the position on the path by pressing “I” on Offset at “0.000”. Next, I went to Video Sequence view and moved the current frame to the desired end frame (in this case 1600) and then changed the Offset value to “1.000” and once again pressed “I”. This sets the end of the path of the camera at the end frame, 1600.

Next it was necessary to add an Empty object that the camera can track throughout the animation. Once again, Shift-A was used to select “Empty” and then “Plain Axis”. Another object constraint needed to be added to the camera, this time a Track To constraint, which used the same Forward as “–Z” and Up as “Y”. The camera should now be on the path created and pointed at the plain axis. This plain axis could be positioned at any desired region on the map.

Pic 13

For this project I put this plain axis near the peak of the first mountain and moved it throughout the animation. For each movement, the plain axis was selected at a desired frame by pressing “R” and selecting “Location”. The object was then moved to the second mountain and the frame was adjusted to approximately 200 frames later. Once again, the plain axis was selected by pressing “R” and selecting “Location”. This identified that the camera needed to move to the new location of the plain axis, giving it 200 frames to do so. This process was done twice more to capture the remaining mountains.

At this point it was possible to animate the camera fly-through. In order to do so, a few minor things needed to be done in the Render Tab, including defining the desired resolution (1080p in this example), setting the Start and End Frame to 1 and 1600 respectively, making the Frame Rate 24 frames per second, and choosing an output type. The project was first rendered as pictures or .png files because if the render process crashed, you would be able to continue rendering from the individual picture frame it crashed at. In addition, under Sampling, the samples were increased to help reduce any picture noise caused by light scattering. “Render Animation” was then clicked after all the settings were finalized. The rendering process varies in length and depends on the number of samples, detail of the images, and number of frames. For this project, the render took around 4 hours.

Pic 14Pic 15

 

 

 

 

 

After the rendering process was complete it produced 1600 individual pictures, which were loaded into Blender’s Video Sequence Editor by locating the folder the render output was saved in and selecting all the image frames. Once uploaded into the Video Sequence Editor, the output type was changed from a .png image to an h.264 file, which is a video output. “Lossless Output” was also selected, and is found under Encoding. Lossless Output ensures that there is no compression in the photos between the original frame and the new video output. This produced the video file of the entire project.

Pic 16         This example demonstrated how to create a 3D model of the Sagarmatha region in Nepal and create a detailed fly-through of the region using the graphics and animation program Blender. This same process can be applied to anywhere in the world with DEM data.

Pic 17

Thanks for Reading!

 

3D Hexbin Map Displaying Places of Worship in Toronto

Produced by: Anne Christian
Geovis Course Assignment, SA8905, Fall 2015 (Rinner)

Toronto is often seen as the city of many cultures, and with different cultures often come different beliefs. I wanted to explore the places of worship in Toronto and determine what areas have the highest concentrations versus the lowest concentrations. As I explored the different ways to display this information in a way that is effective and also unique, I discovered the use of hexbin maps and 3D maps. While doing some exploratory analysis, I discovered that while hexbin maps have been created before and 3D maps have been printed before, I was unable to find someone who has printed a 3D hexbin prism map, so I decided to take on this endeavor.

Hexbin maps are a great alternative technique for working with large data sets, especially point data. Hexagonal binning uses a hexagon shape grid, and allows one to divide up space in a map into equal units and display the information (in this case the places of worship) that falls within each unit (in this case hexagon grids). The tools used to create this project include QGIS, ArcGIS, and ArcScene, although it could probably be completed entirely within QGIS and other open-source software.

Below are the specific steps I followed to create the 3D hexbin map:

  1. Obtained the places of worship point data (2006) from the City of Toronto’s Open Data Catalogue.
  2. Opened QGIS, and added the MMQGIS plugin.
  3. Inputted the places of worship point data into QGIS.
  4. Used the “Create Grid Lines Layer” tool (Figure 1) and selected the hexagon shape, which created a new shapefile layer of a hexagon grid.

    Figure 1: Create Grid Lines Layer Tool
  5. Used the “Points in Polygon” tool (Figure 2) which counts the points (in this case the places of worship) that fall within each hexagon grid. I chose the hexagon grid as the input polygon layer and the places of worship as the input point layer. The number of places of worship within each hexagon grid was counted and added as a field in the new shapefile.

    Figure 2: Points in Polygon Tool
  6. Inputted the created shapefile with the count field into ArcGIS.
  7. Obtained the census tract shapefile from the Statistics Canada website (https://www12.statcan.gc.ca/census-recensement/2011/geo/bound-limit/bound-limit-2011-eng.cfm) and clipped out the city of Toronto.
  8. Used the clip tool to include only the hexagons that are within the Toronto boundary.
  9. Classified the data into 5 classes using the quantile classification method, and attributed one value for each class so that there are only 5 heights in the final model. For example, the first class had values 0-3 in it, and the value I attributed to this class was 1.5. I did this for all of the classes.
  10. The hexagons for the legend were created using the editor toolbar, whereby each of the 5 hexagons were digitized and given a height value that matched with the map prism height.
  11. Inputted the shapefile with the new classified field values into ArcScene, and extruded the classified values and divided the value by 280 because this height works well and can be printed in a timely manner.
  12. Both the legend and hexagonal map shapefile were converted into wrl format in Arcscene. The wrl file was opened in Windows 10 3D Builder and converted into STL format.
  13. This file was then brought to the Digital Media Experience (DME) lab at Ryerson, and the Printrbot Simple was used to print the model using the Cura program. The model was rescaled where appropriate. My map took approximately 3 hours to print, but the time can vary depending on the spatial detail of what is being printed. The legend took approximately 45 minutes. Below is a short video of how the Printrbot created my legend. A similar process was used to created the map.

The final map and legend (displayed in the image below) provide a helpful and creative way to display data. The taller prisms indicate areas with the most places of worship, and the shorter prisms indicate the areas in Toronto with the least places of worship. This hexagonal prism map allows for effective numerical comparisons between different parts of Toronto.

IMG_5392

Use of a Laser Cutter to Create a 3D Bathymetric Chart

Mallory Carpenter,  SA8905 Geovisualization Assignment, Fall 2015

Bathymetric, or depth data collected about oceans and other water bodies are typically displayed in one of two ways –  as a bathymetric chart, or as a depth raster.  New technologies such as 3D printers and laser cutters allow for the better communication of depth data. Laser cutters in particular allow for “etching,” which can simultaneously communicate topographic data.  This allows the viewer to better situate themselves in the landscape.  Examples of this can be seen here and here.

A fjord is a coastal feature formed by glaciers.  Typically, they contain steep vertical sidewalls, and deep basins separated by shallow sills (ridges of bedrock which rise to depths of less than 50 m).  Mapping Nachvak Fjord in 3D, located in the Torngat Mountains in Labrador, will help to better illustrate the unique bathymetric features.

The basic process is this:

  • Collection and processing of bathymetric data into useable raster format.
  • Importation of the raster data into GIS software.
  • The creation and export of contour data as vector files to secondary graphics.
  • The division of contours into separate layers, and the addition of any graphics for “etching.”
  • Different colours in the vector file are used to differentiate between etching and cutting.

The screenshots below show the bathymetric data collected between 2003 and 2009 by the Canadian Hydrographic Service and ArcticNet. The data are available for free for download from the Ocean Mapping Group website. The spatial resolution of the data is 5×5 m with a vertical accuracy of 1 m. The data ranges in depth from 211 m to 1 m.  Contours were created at 20 m intervals, smoothed and exported as vector files.
The data used for etching the topographic map on the top layer are a product called CanVec, which is downloadable for free from Geogratis. The contour interval was reduced to 200 m to improve visibility. Extraneous shapefiles such as points were removed.

Nachvak1

The data were manipulated in iDraw (a Mac-based vector graphics program) to smooth out overlapping lines and crop to an appropriate area as shown in the following screenshot.

Nachvak2

The laser printer has a 2 x 4 foot printing bed.  In order to save materials and cutting time, layers need to be nested in the bed space, colour coded for cutting and etching, and exported as either a PDF or SVG.  Each contour makes up a layer – with a solid rectangle for the base, and the topographic information etched into the top layer.  The following screenshot shows two cutting surfaces, each with 5 map layers.

Nachvak5

nachvak4

The laser cutting was done at the Danforth Tool Library (http://torontotoollibrary.com), out of 1/4 inch Birch Plywood.  They were cleaned (the cutting produces soot), stained, and glued together with carpenter glue.

Nachvak5

Initial plans included the use of etching to detail habitat and substrate information.  Time and finanical constraints limited the amount of etching work that could be done.  Additionally, if the project were repeated it could be worth either using thinner materials, or increasing the contour interval.  The slope on the side walls is so steep, and the fiord so narrow that the fine details are hard to see in the final version.

Nachvak7

 

Creating a Vintage Bathymetric Map with QGIS

Creator: Jenny Mason, Ryerson University Graduate Spatial Analysis Student, for Geovis course project, SA8905 (Dr. Rinner)

My geovisualization, in a sense, goes back in time. Despite many data and geo-visualizations recently showing how technology has changed the way we view spatial data, I wanted to create a geovisualization that tried to replicate the roots of mapping technology. I chose to create a vintage bathymetric survey map using QGIS.

Based on the numbers of tutorials and blog posts available online, replicating the characteristics and design of a vintage map using modern GIS mapping technology is popular for GIS enthusiasts. Using a combination of these geographer’s ideas, as well as my own, I chose to replicate the timeless style of these maps.

The final map product: 

Vintage Bathymetric Map Using QGIS

Technology: QGIS offers rendering options that blend your layers (map data, elements, text and desired stain/finish) in the Print Composer. For this look, the multiply function was used.

Data source: The Great Lakes bathymetry contours data were provided by Esri Canada for educational purposes only. They come in Shapefile format and it looks as if they were vectorized from the National Oceanic and Atmospheric Administration’s (NOAA) bathymetry grid, which is available at https://www.ngdc.noaa.gov/mgg/greatlakes/superior.html.

Approach: To achieve a vintage map look, open a new print composer in QGIS and import a photo along with your map layers. For my design, I used bathymetric data to replicate a historic hydrographic map/survey using the contours of Lake Superior.

Anita Graser’s Blog Post on  Vintage Map Design Using QGIS suggests using Lost and Taken’s Gallery to find aged paper designs at high resolutions. The options for paper on this site are perfect for recreating a vintage style map.

Once your map and image of desired paper/texture are added as layers in your print composer, go to the rendering options for the map in the panel on the right of your screen. Change the option to multiply. You will now see in the window that the map elements have been blended with the paper texture.

With this function, the creator is able to recreate a vintage map of any desired spatial data.

The appropriate choice of text, labels, and other map elements are essential to replicate a vintage map. I found it helpful to reference existing historic maps online and try to replicate their design within QGIS. I imported a North Arrow, but all other elements are available in QGIS.

Animating Toronto Parking Enforcement with heatmap.js

by Justin Pierre – Geovis course project for SA8905, Fall 2015 (Dr. Rinner)

Heatmap.js is a project developed by Patrick Wied to create heatmaps online using JSON data and javascript. It’s lightweight, free to use and comes with tons of great customization options.

For my geovisualization project for SA8905 I created an animated heat map of parking tickets issued in Toronto during the 24 hour period of May 1st 2014. Parking ticket data is supplied on the Toronto Open Data Portal.

Thursday May 1st, 2014 was one of the busiest days of the year for parking tickets. There were 9,559 issued in 24 hours. 6am was the safest time with only 25 tickets issued and 9am was the busiest with 1,451.

To create the heatmap I  geocoded the Toronto parking ticket data using the city of Toronto street data with address ranges. About 10% of the records had to be manually geocoded to intersections, which was a time consuming process! Once I had the locations, it was simple to create a JSON object for each hour in excel, like this:

var h=[ {
 max: 100000,
 data: [
{lat: 43.667229, lng: -79.382666, count: 1},
{lat: 43.728744, lng: -79.30461, count: 1},
{lat: 43.778933, lng: -79.418283, count: 1},
{lat: 43.647378, lng: -79.418484, count: 1},

etc…

h is an array where each element is a JSON object containing the lats and lngs of each traffic ticket. The count is required for the heatmapping function and is always 1, unless you’re this driver:

Using heatmap.js is super straightforward. Initialize your web map in leaflet or openlayers (I used leaflet), configure some simple parameters:

var cfg = {
 "radius": .008,           //set for interpolation radius
 "maxOpacity": .8,         //set to .8 to show the basedata
 "scaleRadius": true,      //recalibrate radius for zoom
 "useLocalExtrema": true,  //reset data maximum based on view
 latField: 'lat',          //where is latitude referenced 
 lngField: 'lng',          //where is longitude referenced
 valueField: 'count'       //where is the numerical field
 };

Attach that to your heatmap object and point it at your datasource like so:

heatmapLayer = new HeatmapOverlay(cfg);
map.addLayer(heatmapLayer);
i=0;
heatmapLayer.setData(h[i]);

Remember that h[] is the array where the ticket data is stored and so h[0] is the first hour of data, midnight to 1am. This will create a static heatmap like this:

Screenshot

Now comes the part where we cycle through the hours of data with a setInterval() function:

setInterval(function(){ 
 i+=1;
 if (i>23) i=0;
 $( ".heatmap-canvas" ).fadeOut( "slow", function() 
   {heatmapLayer.setData(h[i]);
   heatmapLayer._draw();
   $( "#hour").html(i);
 });
 $( ".heatmap-canvas" ).fadeIn( "slow", function() {
 });
}, 2000);

Every 2,000 milliseconds (2 seconds) the page will fade out the heatmap layer, switch the data for the next hour and fade it back in. If the cycle has reached the end of the day it resets. The $( “#hour”).html(i) bit refers to changing the hour printed on the webpage itself.

You can check out the finished project at http://justinpierre.ca/tools/heatmap/ and be sure to let me know what you think at https://twitter.com/jpierre001.

T.Orientation: Colouring the Grids of Toronto

By Boris Gusev, Geovis Course Assignment, SA8905, Fall 2015 (Rinner)

 

The way in which we settle the land around us can paint a rich picture of how our cities have developed over years.  By the turn of the 19th century, urban planners generally agreed that grid-like patterns were the optimal solution and held the most promise for the future of transit. Physical planning led to the development of automotive cities like Los Angeles, Chicago and Detroit. Toronto’s history of growth can also be traced through its sprawling grid of roads.

In this visualization, a MapZen extract of OpenStreetMap road network was used to represent the compass-heading-based orientation of  Toronto roads. Streets that are orthogonal, meaning that they intersect at a right angle, are assigned the same colours. At a 90 degree angle, the streets are coloured with the darkest shades of orange or blue, decreasing in intensity as the intersection angle becomes more obtuse.

Follow the link to take a look at: Toronto Streets by Orientation

Vis_overview

More exciting details and a DIY guide under the cut. Kudos to Stephen Von Worley at Data Pointed for the inspiration and Mathieu Rajerison at Data & GIS Tips for the script and a great how-to.

Continue reading T.Orientation: Colouring the Grids of Toronto