Module 1.2 - Data Quality: Standards

In this lab assignment 1.2 Data Quality – Standards, we measured the quality of two road networks: the streets of Albuquerque, derived from a shapefile provided by the city of Albuquerque, and the Street Map USA data sourced from the TeleAtlas product. The objective was to evaluate the positional accuracy using metrics established by the National Standards for Spatial Data Accuracy (NSSDA). 

The application of the NSSDA standard involves seven distinct steps. 

The first step is to determine whether the assessment apply to horizontal accuracy, vertical accuracy, or both. For this lab, our focus is solely on determining the horizontal accuracy.

The second step requires selecting a study area and identifying a set of test points within that area. These test points correspond to street intersections, ensuring that we select good intersections and include a minimum of 20 street intersection points. The distribution of these points must be such that, when the study area is divided into four quadrants, each quadrant contains 20% of the test points. I selected 24 test points, ensuring that each quadrant included 6 test points. Furthermore, the distance between these points should exceed one-tenth of the length of the study area’s diagonal. Given that the diagonal length of my study area was 6650 feet, I ensured that my test points were spaced at least 665 feet apart. The test points were chosen from both Albuquerque Street data and Street Map USA data correspond to the same street intersection. Both layers of test points are populated with identical point IDs that match to the same intersection. 

Screenshot of study area and sampling locations

The third step involves selecting a higher accuracy dataset for comparison with the data collected from the test area. Initially, we created a new layer designated as Reference, utilizing high-resolution aerial photography to digitize all points that correspond to the true locations of the test points. It was imperative to populate the ID field with the same test point number across all three datasets: Albuquerque Street, Street Map USA, and the Reference true point layer.

To determine the coordinate values for each test point for both the test data sets and the reference data set, I ran Calculate Geometry Attributes tool from Data Management on all three layers to add two new fields for the X and Y coordinates for the test points. At this point, we have three datasets of the same street intersections and have a common field “Point ID”.

Our next step is to assess the accuracy of both the Albuquerque Street layer and the Street Map USA layer. To achieve this, I employed the join tool in ArcGIS Pro to first join the Albuquerque Street layer with the Reference layer based on the common field, Points ID. Subsequently, I applied the join tool to join the Street Map USA test points layer with the Reference layer. Finally, I exported both layers into an Excel spreadsheet.

Using NSSDA accuracy statistic worksheet to calculate the positional accuracy statistic. 

For this step I added five new fields to the excel table they are:

DIFF IN X = the difference between the X coordinate from test point and reference true point

SQUARED DIFF IN X (1) = This is the square root of the x difference 

DIFF IN Y = the difference between the Y coordinate from test point and reference true point

SQUARED DIFF IN Y (2) = This is the square root of the Y difference 

(1) + (2) = SQUARED DIFF IN X + SQUARED DIFF IN Y (2)

From field (1) + (2), three key values can be calculated: The sum, the average and the root mean square error (RMSE).

The sum is the total of the squared differences between the coordinate values of the test data set and those of the Reference data set. The average is obtained by dividing the sum by the number of test points. The root mean square error (RMSE) is the square root of the average. 

The NSSDA statistic is then calculated by multiplying the RMSE by a factor that signifies the standard error of the mean at a 95 percent confidence level which is 1.7308 for the horizontal accuracy. Finally, NSSDA accuracy statement is prepared in a standardized report form.

NSSDA accuracy statement for Albuquerque Street Data:

Horizontal Positional Accuracy: Tested 14.32 feet horizontal accuracy at 95% confidence level

NSSDA accuracy statement for Street Map USA Data:

Horizontal Positional Accuracy: Tested 189.94 feet horizontal accuracy at 95% confidence level

Module 1.1 - Calculating Metrics for Spatial Data Quality

Part A: Determining accuracy and precision

Precision indicates the degree to which the collected points are in proximity to one another; it is considered high when the points are closely grouped. Conversely, accuracy assesses how near the points are to the actual reference location. Average accuracy is deemed high when the mean of the points is situated close to the true location. 

In this laboratory exercise, we initially computed the estimated horizontal precision for a set of points gathered using a handheld GPS device. By applying the 68th percentile, we determined that the horizontal precision was 4.47 meters. This suggests that the points are clustered and are closed to each other. 

Subsequently, we assessed the horizontal distance from the average waypoint to the reference point, which represents the true location of the mapped point, in order to determine the horizontal accuracy, which was found to be 3.24 meters. This suggest that the points are near the true location. This minimal discrepancy indicates a high level of accuracy. 

 

Part B: Root-mean-square error (RMSE) and cumulative distribution function (CDF)

Error metrics for the GPS positions:

Minimum = 0.14

Maximum = 6.95

Mean = 2.67

Median = 2.45

RMSE = 3.06

68th Percentile = 3.18

90th Percentile = 4.67

95th Percentile = 5.69

In the first section of part B of the lab, we used metrics to calculate the RMSE (Root Mean Square Error), and also calculated the Mean, Average, minimum, maximum, 68th percentile, 90th percentile and 95th percentile values for error_xy. The error_xy is the distance error from a given point to the benchmark point. In the second section of part B, we created a cumulative distribution function (CDF) graph showing the complete distribution instead of the selected metrics. To accomplish that we used the error_xy data, sorting by smallest to largest values and adding a new column which represent the percentage from 0 to 100. We created a CDF graph showing the entire probability distribution of the error_xy. We can say that the CDF is a distributional description of data, while metrics are summary statistics derived from data.

The cumulative distribution function (CDF) provides significantly greater insight than metrics. The slope of the CDF indicates variations in density. In this analysis, the curve rises steeply until approximately 68%, where the majority of points are situated near the true location, while the flatter sections signify lower density with fewer points present. Additionally, the CDF offers a visual representation of cumulative distribution, enabling the comparison of multiple distributions. In summary, the CDF enables a more detailed interpretation than merely depending on a mean or median obtained from metrics.

Module 5 - Part 2 - Corridor Analysis

In part two of the analysis lab we are asked to create a corridor that model the potential movement of black bears between two protected areas.

The extent environment should first be modified to align with the elevation or land cover layer.

Running Euclidean tool on the road layer and reclassifying the raster layer based on the distance from roads. The further the distance from the road the higher the suitability value. Distance from the roads greater than 500 meter is given the value of 10 while values lower than 100 meter is classified as value 1, and values between 100-500 m are given the value of 5. 

Reclassified Elevation layer based on suitability value. Elevation between 1200-3000m is considered the most suitable, it is given the number 10.

Land Cover are reclassified based on suitability values, landcover I, 2, and 4 are classified as value 10  

Runing the Weighted Overlay tool to integrate the three criteria, assigning weights of 60% to land cover, 20% to elevation, and 20% to distance from roads, thereby generating the suitability model surface. 

To develop a cost surface, I inverted the suitability model surface layer utilizing the reclassify tool. In this context, a higher suitability value corresponds to a lower cost, while a lower suitability value indicates a higher cost. 

Following this, I executed the Cost Distance tool for the Coronado 1 shapefile and repeated the process for the Coronado 2 shapefile. 

I applied the Corridor tool using the two cost distance surfaces. The corridor raster represents cells that hold values indicating the total travel cost between the source and destination points in both directions.

Finally, I assigned an appropriate symbology to the newly created layer.

Module 5 - Part 1 - Rating Locations in Raster

In this laboratory assignment, we are tasked with identifying the most appropriate location for a developer to establish a new project. Our analysis will encompass five distinct layers, each based on the provided suitability ratings: Land Cover, Soils, Slope, Rivers, and Roads. 

To begin, we will utilize the Reclassify tool to categorize the Land Cover into the specified suitability ratings.

Next, we will convert the vector soil layer into a raster format using the polygon to raster method, followed by reclassifying this new raster layer according to the designated suitability ratings.

We will then employ the slope tool to generate a slope raster from the Digital Elevation Model (DEM) and subsequently reclassify this new layer based on the suitability ratings.

Additionally, we will apply the Euclidean tool to create a raster representing the distance from the river, which will also be reclassified according to the provided suitability ratings.

Lastly, we will generate a distance raster from the Roads vector layer using the Euclidean tool and reclassify this new raster based on the specified suitability ratings.

After completing these steps, we will utilize the Weighted Overlay tool, assigning a scale of 1-5 and incorporating all raster layers, with each layer contributing 20% to the overall analysis.

The outcome will be a new map illustrating areas ranked from the most suitable to the least suitable for development.

Alternative Scenario

In the alternative scenario, we applied the Weighted Overlay tool again using the same layers but adjusted the percentage contributions as follows:

▪ Land Cover – 20%

▪ Soils – 20%

▪ Slope – 40%

▪ Distance to Streams – 10%

▪ Distance to Roads – 10%

This adjustment resulted in a new map depicting different suitability areas.

Module 4 - Damage Assessment

In this lab assignment, we are tasked with evaluating the damage caused by hurricanes within a designated study area. We began by creating a new map and establishing a new feature class specifically for structural damage. This new layer incorporates an attribute domain to distinguish between various types of structures and their corresponding damage. I employed a raster image taken prior to the hurricane to assess different properties, placing points on each structure and filling in attributes with the correct building type. Additionally, using raster image taken after the event helped in identifying the structural damage. 

In the last step, we established a new polyline feature class layer intended for digitizing the coastline. This new layer will facilitate the analysis of structural damage based on five categories within distances of 100, 200, and 300 feet from the coastline. The categories include: no Damage, Affected, Minor Damage, Major Damage, and destroyed. There are several methods to evaluate the number of houses according to damage categories and their distance from the coastline. One method I used involved the Multiple Ring Buffer tool to create non-overlapping buffers at 100, 200, and 300 meters from the coastline. Afterward, I employed the Overlay tool alongside the building layer and each buffer to generate three separate layers, each illustrating the houses affected by the hurricane based on the distance specified by each buffer. Finally, I symbolized each layer according to the level of structural damage and activated the Show Count feature to ascertain the number of houses classified by structural damage within each buffer.

Damage Assessment Map

Table showing number of houses damaged based on structure damage category

Module 3 - Coastal Flooding

Hurricane Sandy made landfall in October 2012 in Mantoloking, New Jersey, resulting in extensive destruction. This week’s module involved downloading two LAZ files from the NOAA website, corresponding to the conditions before and after Hurricane Sandy. 

I utilized the Spatial ETL tool to convert both files into LAS format. The analysis of the two Lidar datasets, collected prior to and after the hurricane, reveals notable alterations. A color gradient is utilized, with low values represented in red and high values in blue. The dataset acquired post-hurricane predominantly displays red, indicating erosion, and a high value of 38.48, a decrease from 39.49 in the pre-hurricane dataset, signifying damage and changes in elevation.

In the following step, I applied the LAS dataset to TIN tool, followed by the TIN to Raster tool to create DEMs. The final step was to utilize the Raster Calculator to subtract the pre-Sandy raster from the post-Sandy raster dataset. The outcome illustrates the changes observed between the pre and post-event images. 

 

A screenshot of the change area that has significant changes due to the hurricane. The red areas indicate where erosion has occurred, where as the blue area signify regions where sand and debris have accumulated resulting from storm surge and flooding. 

Module 2 - Forestry and Lidar

In this week's module, we explored .laz and .las lidar data files, and we generated Digital Elevation Models (DEMs) and Digital Surface Models (DSMs) from lidar data. Additionally, we computed forest height and assessed biomass density.

Initially, I retrieved the data from the Virginia Lidar application and employed the convert LAS tool to transform the downloaded .laz file into a .las file.

Subsequently, I calculated the forest height. To begin, I utilized the Point File Information tool, which produces statistical data regarding the .las layer, including the number of points, as well as the maximum, minimum, and average point spacing.

Following this, I created a DEM layer from the lidar ground point layer by using the LAS Dataset to Raster tool. Furthermore, I generated a DSM layer for the non-ground points. Furthermore, the Minus tool is used to subtract the ground height from the non-ground height in order to determine the tree height.

In the final step to calculate biomass density, I utilized a combination of tools, specifically: the LAS to MultiPoint tool, setting Class code 2 for bare earth and 1 for vegetation; the Point to Raster tool to convert the two new layers into raster format; the IS Null tool to create binary files; the Con tool to assign a value of 0 to true raster and to pull false values from the original raster; the Plus tool to merge both layers from the previous step; and the Float tool to convert the Plus result from integer to float. Finally, we applied the Divide tool to compute the density.