Module 6 - Working with Geometries

In the final module of this course, Module 6, we will explore the topic of geometries. This includes learning how to interpret the properties of geometry objects derived from existing features, as well as the process of creating new geometry objects based on coordinate information.

Features are objects that are depicted on a map or within a dataset. A point represents a single location like a fire hydrant or a tree; a polyline represents a linear feature such as a road, river, and a polygon feature that represents an area or region, like a lake, park, or administrative boundary. Each feature in a data set is typically represented by a row in an attribute table.

A point feature class in GIS represents a single location consists of a single vertex defined by x,y coordinates. On the other hand, polyline and polygons features consist of multiple vertices and are constructed using two or more points objects. These vertices define the shape of a polyline or polygon feature.

To effectively work with geometry objects, it is essential to establish a cursor on the geometry field. Tokens such as SHAPE@ provide access to the complete geometry object; however, in the context of a large dataset, this approach may lead to slower performance. For scenarios where only specific properties are required, alternative tokens can be utilized. For instance, SHAPE@XY returns a tuple of x and y coordinates that represent the centroid of the feature, while SHAPE@LENGTH provides the length of the feature.

ArcPy provides a variety of classes designed for working with geometry objects, including the generic arcpy.Geometry class, which is utilized to create geometry objects. In addition to this, ArcPy includes four specific geometry classes: MultiPoint, PointGeometry, Polygon, and Polyline. Furthermore, ArcPy incorporates two additional classes that facilitate the construction of geometry: Array and Point. The relevant classes include arcpy.Array() and arcpy.Point().

Certain features are composed of multiple parts but are represented as a single feature within the attribute table; these are referred to as multipart features. To determine whether a feature is classified as a single part or multipart, the isMultipart statement is utilized. Additionally, the partCount function will provide the total number of geometry parts associated with a given feature. Furthermore, polygons that contain holes present a challenge. These shapes typically feature one exterior ring, which is defined as a clockwise ring, and one or more interior rings that are defined as a counterclockwise ring.

Lastly, we explored the process of writing geometries, which encompasses the creation of new features. New features can be developed utilizing the InsertCursor class from the arcpy.da module. This process requires creating a geometry object, and then saving the result as a feature using insertRow() method.

cursor = arcpy.da.InsertCursor()

The lab assignment for Module 6 requires the development of a Python script that generates a new text file. This script should populate the file with the Object Identifier (OID) of each feature or row, the vertex ID, the X and Y coordinates of the vertices, as well as the name of the river feature derived from a feature class containing polylines representing rivers in Maui, Hawaii.

Initially, I imported all necessary modules and classes, enabled the overwriteOutput setting, configured the workspace, and defined the feature class variable. Subsequently, I created a rivers.txt file in write mode ("w") and established a search cursor for the rivers.shp file. This cursor was designed to access the OID, SHAPE, and NAME fields.

Furthermore, I implemented a for loop to iterate through each row or feature within the cursor/ShapeFile, while also creating a variable to serve as a vertex ID number. A second for loop was introduced to iterate through each point or vertex in the row, utilizing the .getPart() method to extract the x and y coordinates of the vertices, incrementing the vertex ID number with each iteration.

The output.write() method was employed to append a line to the .txt file, detailing the Feature/ROW OID, Vertex ID, X coordinate, Y coordinate, and the name of the river feature. A print statement was also included to display the output.write result. Finally, I ensured that the .txt file was properly closed and deleting row and cursor variables outside of all loops.

The execution of the script for Module 6 generated a formatted text file corresponding to each river feature within the Maui rivers feature class.

 

Module 6 script flowchart

Module 5 - Exploring & Manipulating Data

This module provides an in-depth examination of methods for exploring spatial data, including checking for datasets, describing, and listing data in a workspace. Additionally, it addresses the utilization of lists, tuples, and dictionaries, as well as the application of for loops in lists. For loops are fundamental tools in Python for working with lists and other iterable objects. They allow you to execute a block of code repeatedly, once for each item in the list.

In the second segment of the module, which focuses on data manipulation, we are introduced to the use of cursors for data access, the integration of SQL within Python, and the management of table and field names. The module places a strong emphasis on the use of cursors to traverse table rows, highlighting various types for executing search operations, adding new records, and updating existing ones. Furthermore, it covers the essential topic of validating text and field names to ensure data integrity.

In this lab, we will develop a script composed of three code blocks. The initial block is designed to establish a new file geodatabase (fGDB) and transfer existing shapefiles from the Data folder into the newly created geodatabase. The second block involves creating a Search Cursor to extract and display the city NAME, FEATURE, and POP_2000 fields for cities identified as 'County Seats'. Lastly, we will initialize an empty dictionary and fill it with the names (as keys) and populations (as values) of all County Seat cities and utilizing a for loop to print the dictionary.

After importing the necessary modules and classes, and setting overwriteOutput to True, we created an empty geodatabase in the Module5/Results folder using the arcpy.CreateFileGDB_management statement. Next, we defined the fclist variable to enumerate all feature classes in the current workspace, followed by creating a for loop to transfer all features from the Data folder to the new geodatabase (fGDB).

When transferring shapefiles to a file geodatabase (fGDB), it is essential to modify the names appropriately. The default naming convention for shapefiles includes the file extension “.shp,” which must be removed. it necessary to use the Describe() functions basename property of shapefiles when copied via Python script from a folder to a file geodatabase. The script utilizes the basename property of the feature class instead of the default name for the feature class that is stored within the fGDB.

The subsequent section of the script involves establishing a Search Cursor for the cities layer to retrieve the Name, Population (POP_2000), and Feature of all cities designated as 'County Seat' in the Feature field. To achieve this objective, we must first configure the workspace to the new fGDB workspace. Once the variables are set and the Delimited field is established, we created a Search Cursor. We then utilize a for loop within the cursor to display all the cities' NAME, POP_2000, and FEATURE categorized as 'County Seat.'

The last block of code in the script is to create an empty dictionary and populate it with names (keys) and populations (values) of each 'County Seat' city, and print the populated dictionary using a for loop. This final step of the lab assignment presented significant challenges for me. I successfully created the empty dictionary and implemented the for loop utilizing the format <dictionary variable>.update({<key>:<value>}). However, upon execution, I did not encounter any error messages, yet the output was not as anticipated. I reviewed the materials from Module 5, including exercises and lectures, and conducted further research online, but I was unable to identify the error. I discovered that the print statement had incorrect indentation; after correcting the indentation and rerunning the script, the script ran successfully and the output met my expectations.

Here is a screen shot showing the output of the Module 5 Python script       

              

Module 5 Python script flowchart

Module 4 - Geoprocessing

This week’s module covered geoprocessing with Python using the ArcPy package. ArcPy is a collection of modules as well as functions and classes that enhance Python's capabilities. To start working with ArcPy we first need to import the package by using the line of code import arcpy. Once arcpy is imported all geoprocessing tools found in ArcGIS Pro can be run.

The subsequent step involves configuring the workspace to the default location designated for the input and output files that will be utilized in our tasks. This can be accomplished by employing the arcpy.env.workspace function. The syntax to be followed is arcpy.<class>.<property>, where 'env' is classified as a class, and 'workspace' serves as a property of this class.

Module 4 exercise focused on geoprocessing with ArcGIS Pro, we utilized the Clip tool from the analysis toolbox to clip the soils shapefile with the basin shapefile. Subsequently, we performed a batch process using the Clip tool to clip multiple shapefiles, including flood zones, lakes, rivers, and roads. Additionally, this exercise provided us with the opportunity to learn how to configure geoprocessing options and to explore models and the ModelBuilder feature in ArcGIS Pro.

In the lab assignment, we undertook two primary tasks. The first task involved the creation of a model designed to identify all soils within the basin shapefile feature that are currently or potentially suitable for agricultural purposes. The second task required the development of a script to execute three geoprocessing functions utilizing ArcPy.

To initiate the first task, we ensured that the option to "Allow geoprocessing tools to overwrite existing datasets" was enabled. This setting can be configured via the options menu under geoprocessing. We then created a new project and incorporated the shapefiles for soils and basin into a new map. Subsequently, we established a new toolbox and a new model named "SoilErase." Locating the Clip tool within the geoprocessing analysis toolbox and adding it to the model. Double-clicking on the Clip tool, to accessed the tool dialog box, where we designated "soils" as the input features and "basin" as the clip features. Upon executing the model, a new shapefile, "Soils_Clip," was generated and saved in the default directory, and added to the project.

Clip tool

The next phase involved selecting all soils classified as "Not prime farmland" for removal from the shapefile. To achieve this, we incorporated the select tool from the geoprocessing toolbox. After double-clicking the select tool, we designated "Soils_Clip" as the input feature. To eliminate soils classified as "Not prime farmland," we added the expression [FARMLNDCL] <> 'Not prime farmland' to select all entries that do not fall under this classification within the [FARMLNDCL] field of the attribute table. Running the model the new shape file “Soil_Clip_Select is being created and added to the project.

 

Selection tool

Created Clip_Select Model 

Screenshot of the final result

The second task required the development of a script to execute three geoprocessing functions utilizing ArcPy. There are various approaches to complete this task effectively. Initially, I duplicated the existing shapefile from the Data folder to the Result folder. This practice guarantees that the original layer remains untouched and serves as a backup. Subsequently, I incorporated the XY coordinates into the new shapefile. Following this, I proceeded to create a buffer and, ultimately, performed a dissolve operation. Although the Buffer and Dissolve processes can be executed in a single statement, I chose to implement them as two distinct statements. This decision allows for the generation of two separate layers for the buffer: one containing all individual buffers and the other representing the dissolved buffer.

1. First, I Imported arcpy, and from arcpy import env to set the workspace to the Data folder.

2. In order to enable the script to overwrite outputs, it is essential to check the option “Allow

geoprocessing tools to overwrite existing datasets” within the Geoprocessing settings in ArcGIS Pro

options. Additionally, to ensure that outputs can be overwritten during geoprocessing operations in Python IDLE, it is necessary to include the command “arcpy.env.overwriteOutput = True.”

3. Using arcpy.Copy_management to copy the shape file from the Data folder to the Results folder.

4. Using env.workspace to change and reset the workspace to the Results folder where the new shapefile is been saved.

5. Using arcpy.AddXY_management to add XY coordinates to the new copied shapefile.

6. Using the arcpy.Buffer_analysis and consequently arcpy.Dissolve_management to create a 1000 meter around the hospital features and then dissolve the created buffers into a single feature. The dissolve buffer step can be combined with the Buffer statement by using ALL in the buffer statement.

Using ALL for the dissolve_option instead of NONE will dissolve all buffers together into a single feature, removing any overlap.

arcpy.analysis.Buffer(in_features, out_feature_class, buffer_distance_or_field, {line_side}, {line_end_type}, {dissolve_option}, {dissolve_field}, {method})

7. Adding a print statement at the beginning of each task block and GetMassages() function at the end. This modification will ensure that when the script is executed, it displays an explanation of the processes being carried out, along with the respective start and end times.

Running the code from Notebook ArcGIS Pro 

Running the code in Python IDLE 

Flowchart for Module 4 lab

 

 

 

 

 

Module 3 - Debugging and Error Handling

This week's module focuses on debugging procedures and provides a comprehensive review of the most prevalent errors encountered in Python programming. We have explored syntax errors and exceptions, established a shortcut to the Integrated Development and Learning Environment (IDLE), debugged a script within IDLE, and learned how to handle exceptions using the try-except statement. 

Additionally, we have been introduced to the built-in syntax checking feature in Python IDLE, which allows for a syntax verification without executing the code through the interpreter. A pop-up dialog box will appear to notify users of any syntax errors detected. It is important to note that while the error check indicates the location of the syntax error, it does not specify the exact nature of the error, such as whether a colon is missing.

Syntax errors arise when the code does not adhere to the established rules of the Python programming language. Common causes of these errors include misspelled keywords, incorrect punctuation, omitted colons, and errors in indentation. Conversely, run-time errors, also known as Exception errors, occur when the interpreter encounters a situation that it cannot process. If these exceptions are not managed appropriately, the script will terminate unexpectedly. 

The laboratory exercise focused on debugging and error handling and was divided into three parts. 

Part 1 involved a script containing two errors. After executing the script, I was able to identify and rectify the errors, resulting in a successful execution of the script. The intended outcome of the script was to print the names of all the fields in the parks shapefile.

Module 3 – Part 1 results 

The second part of the lab assignment focused on identifying and rectifying additional errors and exceptions. Prior to executing the script, we ensured that the four shapefiles were incorporated into the project file named TravisCountyAustinTX.apx. The expected result upon successful execution of the script is the display of the names of all layers within the project.

Module 3 – Part 2 results

Part three of the lab assignment was about using the try-except statement to make a code run even with errors. The try-except statement in Python is used for handling exceptions errors. This functionality enables us to evaluate a block of code for potential errors and manage those errors effectively, thereby preventing the program from encountering a crash. The try-except statement traps the exception and provide additional error handling. 

Process of adding the try-except statement to the script:

1. Running the code, the first time resulted in receiving an error message line 13 

project = arcpy.mp.ArcGISProject() TypeError: ArcGISProject.__init__() missing 1 required positional argument: 'aprx_path' 

This error indicates that when we are trying to create an instance of the ArcGISProject class, we are not providing the necessary aprx_path argument.

2. To verify arguments and to catch any exception I added try-except statement. Prior to proceeding with this task, I verified that the file exists and confirmed that both the name and location are accurate

Try: placed before the first argument where the error exists and except Exception at the end of part A. Assigning a variable e and a print command

except Exception as e:

        print (e)

3. Executed the program once more, resulting in an error message for part A while part B executed successfully, displaying the name, data source, and spatial reference for each layer.

Module 3 - Part 3 results

Flowchart Part 3: Making a code run even with errors using try-except statement

Logic errors in Python, or semantic errors, arise when code executes without crashing but yields unintended or incorrect results due to a misalignment between the program's logic and the programmer's intended outcome. Unlike syntax errors, which halt execution, or runtime errors, which occur during operation, logic errors can be more difficult to detect.

Debugging is the process of identifying and removing errors from a program. Python offers several ways to debug code, which include:

1. Carefully reviewing the content of error messages.

2. Adding print messages to the script: Adding print messages after each geoprocessing tool or other important steps to confirm they were run successfully.

3. Selectively commenting out code: Involve removing certain lines to see if this eliminates the error. Double number signs (##) can be used to manually comment out lines of code.

4. Using Python debugger: A debugger is a tool that allows us to step through the code line by line, to place breakpoints in the code to examine the conditions at that point, and to follow how certain variables change throughout the code.

        a. Python built-in debugger called pdb

        b. IDLE debugger.