Section 2.2 Acquiring relevant data
We will be using three different sources of data in this chapter. First, we will acquire our crime data, which is what we used in the previous chapter, so this should be familiar. Then we will meet the new format for boundary data, geoJSON. And finally, we will look at Open Street Map for data on our points of interest.
Subsection 2.2.1 Reading in crime data
The point-level crime data for Greater Manchester can be found in the file
2019-06-greater-manchester-street.csv we used in Chapter 1. We can import it with the read_csv() function from the readr package.
crimes <- read_csv("data/2019-06-greater-manchester-street.csv")
If you replicate the exercises for getting to know the dataset from the previous chapter, you might notice that in this case the column names are slightly different. For example, Latitude and Longitude are spelled with uppercase "L". You should always familiarise yourself with your dataset to make sure you are using the relevant column names. You can see just the column names using the
names() function like so:
names(crimes)
## [1] "Crime ID" "Month" ## [3] "Reported by" "Falls within" ## [5] "Longitude" "Latitude" ## [7] "Location" "LSOA code" ## [9] "LSOA name" "Crime type" ## [11] "Last outcome category" "Context"
This is because last time, we cleaned these names using the
clean_names() function from the janitor package. Let’s do this again, as these variable names are not ideal, with their capital letters and spacing...so messy!
# clean up the variable names
crimes <- crimes %>% clean_names()
#print them again to see
names(crimes)
## [1] "crime_id" "month" ## [3] "reported_by" "falls_within" ## [5] "longitude" "latitude" ## [7] "location" "lsoa_code" ## [9] "lsoa_name" "crime_type" ## [11] "last_outcome_category" "context"
Much better! Now let’s get some boundary data for Manchester.
Subsection 2.2.2 Meet a new format: geoJSON
GeoJSON is an open standard format designed for representing simple geographical features, along with their non-spatial attributes. It is based on JSON, the JavaScript Object Notation. It is a format for encoding a variety of geographic data structures and is the most common format for geographical representation in the web. Unlike ESRI shapefiles, with GeoJSON data everything is stored in a single file.
Geometries are shapes. All simple geometries in GeoJSON consist of a type and a collection of coordinates. The features include points (therefore addresses and locations), line strings (therefore streets, highways and boundaries), polygons (countries, provinces, tracts of land), and multi-part collections of these types. GeoJSON features need not represent entities of the physical world only; mobile routing and navigation apps, for example, might describe their service coverage using GeoJSON. To tinker with GeoJSON and see how it relates to geographical features, try geojson.io, a tool that shows code and visual representation in two panes.
Let’s read in a geoJSON spatial file, again from the web. This particular geoJSON represents the wards of Greater Manchester.
manchester_ward <- st_read("data/wards.geojson")
Let’s now select only the city centre ward, using the
filter() function from the dplyr package. Notice how we’re using the values in the data (or attribute) table; therefore, this is an attribute operation.
city_centre <- manchester_ward %>%
filter(wd16nm == "City Centre")
Let’s see how this looks. In Chapter 1 we learned about plotting maps using the
ggplot() function from the ggplot2 package. Here, let’s look at a different way, which can be useful for quick mapping — usually used when just checking in with our data to make sure it looks like what we were expecting.
With the
st_geometry() function in the sf package, we can extract only the geometry of our object, stripping away the attributes. If we include this inside the plot() function, we get a quick, minimalist plot of the geometry of our object.
#extract geometry only
city_centre_geometry <- st_geometry(city_centre)
#plot geometry object
plot(city_centre_geometry)
Now we could use this geometry to make sure that our points are in fact only licensed premises in the city centre. This will be your first spatial operation. Excited? Let’s do this!
Subsection 2.2.3 Open Street Map and points of interest
To map licenced premises, we will be accessing data from Open Street Map, a database of geospatial information built by a community of mappers, enthusiasts and members of the public, who contribute and maintain data about all sorts of environmental features, such as roads, green spaces, restaurants and railway stations. You can see all about open street map on their online mapping platform. One feature of Open Street Map, unlike Google Map, is that the underlying data is openly available for download for free. In R, we can take advantage of a package written specifically for querying Open Street Map’s API, called
osmdata. For more detail on how to use this, see [113], and Appendix C: sourcing geographical data for crime analysis of this book.
If we load the package
osmdata we can use its functions to query the Open Street Map API. To find out more about the capabilities of this package, see [114]. While this is outside the scope of our chapter here, you may want to explore osmdata more, as it is an international database, and has lots of data that may come in handy for research and analysis.
Here we focus specifically on Manchester. To retrieve data for a specific area, we must create a bounding box. You can think of the bounding box as a box drawn around the area that we are interested in (in this case, Manchester, UK) which tells the Open Street Map API that we want everything inside the box, but nothing outside the box.
So, how can we name a bounding box specification to define the study region? One way to do this is through a search term. Here, we want to select Greater Manchester, so we can use the search term "greater manchester united kingdom" within the
getbb() function (stands for get bounding box). Using this function, we can also specify what format we want the data to be in. In this case, we want a spatial object, specifically an sf polygon object (from the sf package), which we name bb_sf.
bb_sf <- getbb(place_name = "greater manchester united kingdom",
format_out = "sf_polygon")
We can see what this bounding box looks like by plotting it (notice this time we combine the
plot() and st_geometry() functions to save needing to create a new object):
plot(st_geometry(bb_sf))
We can see the bounding box takes the form of Greater Manchester. We can now use this to query data from the Open Street Map API using the
opq() function. The function name is short for ’Overpass query’, which is how users can query the Open Street Map API using search criteria.
Besides specifying what area we want to query with our bounding box object(s) in the
opq() function, we must also define the feature which we want returned. Features in Open Street Map are defined through ’keys’ and ’values’. Keys are used to describe a broad category of features (e.g., highway, amenity), and values are more specific descriptions (e.g., cycleway, bar). These are tags which contributors to Open Street Map have defined. A useful way to explore these is by using the comprehensive Open Street Map Wiki page on map features (see also Appendix C: sourcing geographical data for crime analysis of this book).
We can select what features we want using the
add_osm_feature() function, specifying our key as ’amenity’ and our value as ’bar’. We also want to specify what sort of object (what class) to get our data into, and as we are still working with spatial data, we stick to the sf format, for which the function is osmdata_sf(). Here, we specify our bounding box as the bb_sf object we created above. If you use the bounding box obtained through getbb() one can subsequently trim down the outputs from add_osm_feature() using trim_osmdata(). For instance, we could add trim_osmdata(bb_poly = bb_sf) to our initial query.
bb_gm <- getbb(place_name = "greater manchester united kingdom",
format_out = "data.frame")
osm_bar_sf <- opq(bbox = bb_gm) %>% # select bounding box
add_osm_feature(key = 'amenity', value = 'bar') %>% # select features
osmdata_sf() # specify class
The resulting object
osm_bar_sf contains lots of information. We can view the contents of the object by simply executing the object name into the Console.
osm_bar_sf
This confirms details like the bounding box coordinates. It also provides information on the features collected from the query. As one might expect, most information relating to bar locations has been recorded using points (i.e. two-dimensional vertices, coordinates). We also have around fifty polygons. For now, let’s extract the point information.
osm_bar_sf <- osm_bar_sf$osm_points
We now have an
sf object with all the bars in our study region mapped by Open Street Map volunteers, along with ~90 variables of auxiliary data, such as characteristics of the bar (e.g., brewery, or cocktail) as well as address and contact information, amongst many others. Of course, it is up to the volunteers whether they collect all these data, and in many cases, they have not added information (you may see lots of missing values if you look at the data). Given the work relies on volunteers, there are unavoidably some accuracy issues (including how up-to-date the information may be). Nevertheless, when the details are recorded, they provide rich insight and local knowledge that we may otherwise be unable to obtain.
One column we should consider is the
name which tells us the name of the bar. There are missing values here as well, and for this example, we will choose to exclude those lines where there is no name included, as we would like at least a little bit of context around our bars. To do this, perform another attribute operation using the filter() function:
osm_bar_sf <- osm_bar_sf %>% filter(!is.na(name))
We are still left with 283 bars in our data set.
