Package: scrapers.police_coders_groups¶
Submodules:¶
group_count¶
Module containing functions required to count the number of groups per area.
-
backend.scrapers.police_coders_groups.group_count.count_groups(input_path, boundary_info, areaID_name)¶ Creates and returns a pandas df containing the number of community support groups within each geographical boundary area.
- Parameters
input_path (str) – File path to cleaned community group data
boundary_info (dict) – Dictionary containing {‘polygons’: list of boundary polygon objects, ‘areaID_list’ : list of local area IDs}
areaID_name (str) – Column header name containing the IDs of local areas in the cleaned community group data csv file
- Returns
df containing the number of community support groups within each geographical boundary area
- Return type
pd.DataFrame
-
backend.scrapers.police_coders_groups.group_count.get_boundaries_LA(fileNm_areas)¶ Generates a dictionary of LA boundary polygons and areas IDs.
- Parameters
fileNm_areas (str) – File path to geojson file containing LA level boundaries
- Returns
Format: {‘polygons’: list of LA level boundary polygon objects, ‘areaID_list’ : list of local area IDs}
- Return type
dict
-
backend.scrapers.police_coders_groups.group_count.get_boundaries_LSOA(fileNm_areas)¶ Generates a dictionary of LSOA boundary polygons and areas IDs.
- Parameters
fileNm_areas (str) – File path to geojson file containing LSOA level boundaries
- Returns
Format: {‘polygons’: list of LSOA boundary polygon objects, ‘areaID_list’ : list of local area IDs}
- Return type
dict
-
backend.scrapers.police_coders_groups.group_count.locate_group(polygons, group_coords)¶ Locate the area the group is contained within.
- Parameters
polygons (list) – List of polygon boundary objects
group_coords (list) – Coordinates to search the polygons for
- Returns
The local area ID within which the given group is located
- Return type
str
localise¶
Functions required to locate groups in Wales.
-
backend.scrapers.police_coders_groups.localise.filter_welsh_groups(input_path, polygon)¶ Return a csv list of groups which are located in Wales, as well as the last row which is used as a header.
- Parameters
input_path (str) – File path to raw data location
polygon (shapely.Polygon) – Polygon object of Welsh country boundary
-
backend.scrapers.police_coders_groups.localise.get_welsh_boundary(fileNm_Wales)¶ Given the .geojson file of the Welsh boundary, returns a polygon shape of the Welsh boundary.
-
backend.scrapers.police_coders_groups.localise.write_data_to_CSV(output, row, out_path)¶ Writes given data to .csv
- Parameters
output (str) – Data on each group
row (str) – Row to be used as header
out_path (str) – File path to write cleaned data
police_scraper¶
Main worker module for collecing the Police Rewired data from Google Sheets.
-
backend.scrapers.police_coders_groups.police_scraper.police_coders_scrape(filename, root_path)¶ Downloads police coders community support group list and saves as CSV.
- Parameters
filename (str) – File path to raw data output location
root_path (str) – File path to folder containing this script
run_scraper¶
This module executes the police coders scraper.
Notes
Intended purpose of this script: 1. Get updated data using googleScrape () 2. Save scraping data archived by date (csv) 3. Overwrite most recent data (csv) 4. Produce groupCount layer as a count of groups per area
-
backend.scrapers.police_coders_groups.run_scraper.run_police_coders_scraper(LIVE_RAW_DATA_FOLDER, LIVE_DATA_FOLDER, GEO_DATA_FOLDER)¶ Executes the tasks required to run the Police Coders data collector.
- Parameters
LIVE_RAW_DATA_FOLDER (str) – Path to the live raw data folder.
LIVE_DATA_FOLDER (str) – Path to the live cleaned data folder.
GEO_DATA_FOLDER (str) – Path to the geoboundaries data folder.