Urban Development of San Francisco

By Hannah Burdett

SA8905 Geovisualization Project, Ryerson University

The Development of San Francisco

San Francisco is located in the center of Northern California. It started as a base for the gold rush of 1849, the city quickly became one of the most populated cities in the United States. Shortly thereafter, San Francisco was devastated by the 1906 earthquake. Development peaked in the 1900’s as San Francisco rebuilt areas demolished by the earthquake and fires to compensate the growing population. During the 1930’s the San Francisco-Oakland Bay Bridge and the Golden Gate Bridge were opened. Additionally, during World War II, San Francisco was a major mainland supply point and port of embarkation for the war in the Pacific. Both factors led to another peak in construction. After World War II, many American military personnel who had fallen in love with the city while leaving for or returning from the Pacific settled in the city. This led to promoting the development of the Sunset District, Visitacion Valley, and the total build-out of San Francisco. Starting in the latter half of the 1960’s, San Francisco became most recognized for the hippie movement. Currently, San Francisco has become known for finance and technology industries. There is a high demand for housing, driven by its close proximity to Silicon Valley, and a low supply of available housing has led to the city being one of America’s most expensive places to live.

Data

The data used for the time series animation was imported from data.gov. Data.gov is a repository for the US Governments open source data. The imported data included a Land use Shapefile for San Francisco. The shapefile included information such as land use, shape area, street address, street number, etc. The land use shapefile also included the year the building was built. The building years range from 1848 to 2016 displaying 153 years of urbanization. The buildings were represented as polygons throughout San Francisco. Additionally, a grey scale base map from ArcGIS Pro was displayed to create a more cohesive map design.

 

 

Time Series Animation

To develop the reconstruction of San Francisco throughout the years, both QGIS and ArcGIS Pro were utilized. Both platforms were used so to provide a comparison between time series animation tools from an open source application and a non-open source application.

QGIS is an open source geographic information systems application that provides data visualization, editing, and analysis through functions and plugins. To create the time series animation the Time Manager plugin was utilized. The Time Manager plugin animates vector features based on a time attribute. For this study the time attribute was the years built.

ArcGIS Pro is the latest professional desktop GIS from Esri. ArcGIS Pro enables users to view, explore, analyze, edit and share maps and data. Unlike QGIS, no additional plugins are required to create the animated time series.

QGIS Methodology

To generate the time series in QGIS, the land use shapefile was downloaded and opened in QGIS. The attribute table from the land use shapefile was then exported and opened in Excel so that the yrbuilt column could be reformatted to meet QGIS Time Manager requirements. The yrbuilt column had the data presented as YYYY format for building dates. QGIS Time Manager requires timestamps to be in YYYY-MM-DD. To correct the format, -01-01 was added to the end of each building year. The modified values were then saved into a new column called yrbuilt1. The Excel sheet was then imported into QGIS and joined to the land use shapefile.

In QGIS, each of the buildings was presented as polygons. The shapefile symbology was changed from single symbology to quantified symbology. In other words, the symbology for each of the polygons was broken down to seven classes defined by years. Each class was then distinguished by color, so that one may differentiate the oldest building from the newest buildings. Furthermore, a grey scale basemap was added to create a more cohesive map.

Furthermore, in the Time Manager settings, “Add Layer” was selected. The land use shapefile was chosen as the Layer of interest. The start time was set to the yrbuilt1 attribute, whereas the end time was set to “No end time – accumulate features”. This allows newer buildings to be added without older buildings being removed from the map. For the animation, each time frame will be shown for 100 milliseconds. The Time Manager plugin was then turned on so that the time series may run.

 

In order to export the time series animation, Time Manager offers an “Export Video” option. However, this exports the animation as an image series, not as an actual video. To correct this, the image series was uploaded to Mapbox where additional Mapbox styles were used to render the map. It was then exported as a Gif from Mapbox.

ArcGIS Pro Methodology

In ArcGIS Pro, the land use shapefile was imported. The symbology for each of the polygons was then broken down to seven classes defined by years. The same colours utilized in QGIS were applied to the classes in ArcGIS Pro to differentiate between the building years. Within the layer’s properties, the Layers Time was selected as “each feature has a single time field”. Furthermore, the start and end times were set to the newest and oldest building years. The number of steps were assigned a value of sixteen. In View, the animation was added, and the Time Slider Steps were imported. The time frames were set to match the QGIS animation so that both time series animations would run at the same speed. The time series animation was then exported as a Gif.

Final Animated Map

Finally, to create a cohesive animated map the exported Gif’s were complied together in PowerPoint. Additional map features, such as a legend, were designed within PowerPoint. A bar graph was added along the bottom of the map to show years of peak building construction. The final time series map was then exported as a .mp4 and upload to YouTube.

Displaying Brooklyn’s Urban Layers by Mapping Over 200 Years of Buildings

Renad Kerdasi
Geovis Course Assignment
SA8905, Fall 2015 (Rinner)

Growth in Brooklyn
Located at the far western end of Long Island, Brooklyn is the most populous of New York City’s five boroughs. The borough began to expand between the 1830s and 1860s in downtown Brooklyn. The borough continued to expand outwards as a result of a massive European immigration, the completion of the Brooklyn Bridge connecting to Manhattan, and the expansion of industry. By mid 1900s, most of Brooklyn was already built up as population increased rapidly.

Data
The data in the time series map are from PLUTO, which is a NYC open data site created by NYC Department of City Planning and released in 2015. The data contain information about each building located in the boroughs, including the year the construction of the building was completed (in numeric 4 digits format) and the building footprints. The building years range from 1800 to 2015, there are some missing dates in the dataset as well as some inaccuracy in the recorded dates. The data are available in Shapefile and Windows Comma Separated format, found on NYC Planning website: http://www.nyc.gov/html/dcp/html/bytes/dwn_pluto_mappluto.shtml

The Making of the Time Series
To present the structural episodes of Brooklyn’s built environment, QGIS 2.10 was utilized with the Time Manager plugin. QGIS is an open source GIS application that provides data visualization, editing, and analysis through functions and plugins (https://www.qgis.org/en/site/about/). The Time Manager plugin animates vector features based on a time attribute (https://plugins.qgis.org/plugins/timemanager/). This tool was effective in presenting a time series of Brooklyn’s building construction dates.

To create the time series, the PLUTO SHP was downloaded and prepared by removing any unnecessary fields. The columns of interest are: FID, Shape, and YearBuilt. Because we are interested in the time column, the formatting must fit with QGIS Time Manager. QGIS Time Manager requires timestamps to be in YYYY-MM-DD format whereas the building dates in the PLUTO SHP are in a four-digit format. Therefore, the date in the dataset must be modified to fit the Time Manager format before it can be brought into QGIS.

Table 1_BrooklynData

In QGIS, Time Manager plugin must be installed first. The SHP can then be added into QGIS as well as other Shapefiles needed: roads, highways, state boundaries, etc. Note: to use Time Manager, the data must be in SHP format.

Layer_BrooklynData

Once the data are added, the polygons (i.e. buildings) are styled based on age. This will be effective in distinguishing the oldest buildings from the newest. In QGIS, there are a large number of options available to apply different types of symbology to the data. The layer is styled based on the attribute Year Built, since the time series will show urban layers using building dates. Also, Graduated is chosen because features in the layer should not be styled the same way. The other data file, such as roads, highways, and state boundaries, are styled as well.

Once all the data are added and styled, it can be oriented and applied to the Time Manager plugin. To truly see the urban layers, the map is zoomed on the upper portion of Brooklyn. In Time Manager settings, the layer with building dates is added and the Start Time is the Year Built field, which includes the timestamp data. To get features to be configured permanently on the map, in the End Time option “No End Time” is selected. For animation options, each time frame will be shown for 100 milliseconds, and timestamp (i.e. built year) will be displayed on the map.

Layer_BrooklynData

In the Time Manager dock, the time frame is changed to years since the animation will be showing the year the construction of the building was completed. The size of the time frame will be 5 years. With these settings, each frame will display 5 years of data every 100 millisecond. Playing the video will display the animation inside QGIS, and one can see the time scrolling from 1800-2015 in the dock.

Dock_BrooklynData

Time Manager also enables you to export the animation to an image series using the “Export Video” button. Actual video export is not implemented in Time Manager. To play the animation outside of QGIS, various software applications can be used on the resulting image series to create a video file.

In addition, QGIS only allows users to insert a legend and title in the Composer Manager window. Currently, it is not possible to get the legend rendered in the main map window. One approach to generate a video with a legend is to create a dummy legend and add the image containing the legend into the PNGs that Time Manager produces. A dummy legend and a title for Brooklyn’s urban layers was created outside of QGIS, and added to each PNG.

Finally, to create a time-lapse and compile the images together, Microsoft Movie Maker was utilized. Other software applications can be used, including mancoder and avidemux.

Results

Link: https://youtu.be/52TnYAVxN3s

Working with WMS-T layers and Time Manager in QGIS

By: Lauren Blumberger
Geovis Course Assignment, SA8905, Fall 2015 (Dr. Rinner)

Are you ever troubled by the amount of large files that you need to download and store on your computer?  If so, a Web Map Service (WMS) may be useful as it delivers georeferenced map images over the internet that are generated by a map server. All you need is an internet connection to be able to access maps from a remote server and load them into your GIS. A request is made, and the response is one or more map images (returned as PNG, JPEG etc.). While WMS services are commonly used for retrieving base maps, they also provide a quick and easy way to work with data that has already been symbolized and styled by the author.  Many WMS servers also have time support, WMS-T, which enables users to define a temporal subset for the rendering of layers that are properly configured with a time dimension.

In this post, I explore how to work with WMS-T retrieved maps in QGIS and how to animate them using Anita Graser’s Time Manager plugin (http://anitagraser.com/projects/time-manager/ and https://plugins.qgis.org/plugins/timemanager/). Anita Graser’s Blog includes a guest post by Karolina Alexiou on using Time Manager for WMS-T layers. I will describe the process, as well as discuss the limitations.

The first step is to find the service layers that you want to work with. This is not always the easiest task as you must use the GetCapabilities URL to access the maps, which in some cases doesn’t reveal the image until the connection is made from QGIS to the server. Thus, there might be some trial and error in terms of retrieving the maps you are looking for.

The map I use for this demonstration is made available by the Fire Information Resource Management System (FIRMS), who offer fire-based map images. The WMS-T allows access to FIRMS Active Fire Hotspots for a specific day. The GetCapabilities URL is provided on the website:

https://firms.modaps.eosdis.nasa.gov/wms-t/?SERVICE=WMS&VERSION=1.1.1&REQUEST=GETCAPABILITIES

Navigate to Add WMS/WMTS Layer in QGIS and click on new. Add the above URL and name the connection FIRMS. Make sure to enable the Ignore GetMap/GetTile URI reported in capabilities, as this will redirect you to the WMS instead of the WMS-T. When you click connect, multiple layers will appear. Load the MODIS_Hotspots layer to the canvas.*2

Now that you have connected to the server you can overlay this image with other layers accessed through servers or available locally. Download any country boundary shapefile and load it to the canvas. Now you can change the style of this new layer and adjust the transparency to make sure that you can see the MODIS image underneath.

new

The next step involves the TimeManager plugin, which provides some support for stepping through this spatial temporal data.** Once you have installed the plugin, click on settings and add raster layer. The WMS-T standard uses many different time formats and the plugin requires you to know this format for inputting the start and end times. Therefore, you must examine the XML document with the web service description and find the section that defines the format of the time dimension. For this example, add the MODIS layer and input start time as 2005-11-16 and end time as 2015-11-16. Set the time step to months and press play. Now you will see that TimeManager renders each month interval by querying the FIRMS WMS-T.window

Since WMS-T is a live service that will automatically update on the map, there are significant delays when working with these layers. Querying the web service and waiting for response takes time, and thus, the interactive mode using TimeManager does not result in a visually pleasing animation. TimeManager includes an option to export all the frames, which you can then move into another software to create a smooth animation.

The resulting animation shows the MODIS Active Fire Hotspots globally for the past ten years. The visualization works best if you maximize the video to full screen view.

Overall, working with WMS-T servers can be tricky. As there are many different time formats that can be used, it is quite a complex task to support them all in a GIS. Also, while you want to be connected to the server to be able to access live, current data, this connection slows down the rendering process and creates choppy animations. This being said, exporting the frames to create an animation provides a nice result and a fast way to visualize current data. As FIRMS constantly provides access to active fire data with the MODIS satellite, through reconnecting to the WMS-T and adjusting the time I can update the animation to 3 hours ago.

*The MODIS satellite detects both flaming and smoldering fires ~1000m2

**Note that FIRMS WMS-T only supports one date per request and date ranges are not allowed. To make Time Manager send the correct request, the source code file wmstlayer.py must be adjusted. If Time Manager is installed, the file can be found in the user folder on your computer. Replace lines 59 to 63 with:

self.layer.dataProvider().setDataSourceUri(self.IGNORE_PREFIX + \
self.originalUri + self.addUrlMark() + “TIME={}” \
.format(
time_util.datetime_to_str(startTime, self.timeFormat)))

This ensures that Time Manager only sends a point in time. A time range will return a black image for this WMS-T.

Special thanks to Dr. Eric Vaz and Anita Graser for their help.