The purpose of this knowledge base document is to outline how to utilize a python library to query data from an ArcGIS REST service.
ArcGIS Server queries are typically limited to 1000 features at a time. This can pose a problem if one is looking to say, query the parcel data from a large county. This method automatically queries a defined REST service in batches of 1000 features, concatenates the batches, and returns all off the features in a
.geojson file. That
.geojson file is then converted directly into an Esri Shapefile through the use of the ogr2gui application.
Note - There is an excellent alternative to this method which is to use data pillager (linked below). However, it is dependent upon arcpy and may struggle with large datasets. The method described below is platform agnostic and only limited by the amount of RAM (and time) you have.
The intended audience for this document is a GIS Analyst with basic programming skills.
This document should be used by any GIS Analyst who needs to query a large amount of features from an ArcGIS REST Service.
I:\GIS\Development\Python\Scriptson MosaicGIS. You should see a heavily commented version of the script similar to the below.
import json import arcgis from arcgis import ArcGIS service = ArcGIS("http://staging.storymaps.esri.com/arcgis/rest/services/Gettysburg/GettysburgTroops8/MapServer") ##Defines the specific REST service that we want to target #descriptor = service.get_descriptor_for_layer(1) ##Returns the JSON descriptor for the layer. ##This tells you things like what fields are in the layer, what sort of geometry it contains, etc. #field_list = service.enumerate_layer_fields(1) ##Returns a list of the field names in the layer. query = service.get(35, count_only=False) ##Retrieves a single, defined layer from a web service. ##If count_only=True, we return a simple count of the number of features in the layer you're querying. json_query = json.dumps(query) ##Creates a new variable, 'json_query', and defines it as a JSON object. f = open("union_move.geojson", "w") f.write(json_query) f.close() ##Validates our geoJSON and either writes it to a file if successful or returns an error message if unsuccessful. #print query
This script carefully outlines what each field does and what inputs are necessary to make it work.
Alter the inputs to point to the REST service you are looking to query, query the correct layer, and return the correct file name.
Open the ogr2gui application which should look like so.
Select the .geojson file generated by the query in the Source field. Then choose where you would like to save the resulting file within the Target field. Ensure that the Source format is set to
geoJSON and the ‘Target’ format is set to ESRI Shapefile.
Use 7zip, to compress the resulting files so that it may be imported into ArcMap, QGIS, ArcGIS Online, or other.
Quality Control Checklist
Import the resulting Shapefile into ArcMap/QGIS and ensure that you are able to view the attribute table. If an error is received ensure that there are no column names that include special characters. If a column name does include special characters it can be altered or removed entirely within editing mode.
GIS Dictionary: http://support.esri.com/en/knowledgebase/Gisdictionary/browse