Elections and housing

When we made our General Election 2024 website , we processed demographic data for each constituency. One of the topics in that data was housing tenure. For each constituency there is a percentage of homes owned outright, with a mortgage or loan, private rented, social rent, and other. Here we are going to explore the data from the results of the election and the housing tenure breakdown for each constituency.

First, lets import some modules to help us and set up paths so that they're system independent.

from pathlib import Path
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
plt.rcParams["font.sans-serif"] = ["Arial"]
ROOT = Path('../')
ROOT.resolve()

For election results, we'll use the Commons Library detailed results by constituency CSV. We'll download this at put it in the data/misc folder. It's in pretty good shape. We don't need all the columns so we will specify which ones we do need when reading the CSV. ONS ID is the code for the constituency. First party is the winning party. The rest should be self-explanatory.

ge_results = pd.read_csv(ROOT / 'data/misc/HoC-GE2024-results-by-constituency.csv', 
                         usecols=['ONS ID', 'First party', 'Electorate', 'Valid votes', 'Invalid votes'])

geo_codes = pd.read_csv(ROOT / 'metadata/lookups/Westminster_Parliamentary_Constituencies_(Future)_Names_and_Codes_in_the_United_Kingdom_v2.csv',
                        usecols=['PCON24CD', 'PCON24NM'])

Breakdown of housing tenure by new (May 2024) parliamentary constituencies is a bit more involved. Commons library has new constituency data dashboards, but the one on housing tenure hasn't yet been updated. So, we'll use the XLSX file on the data for new constituencies page. Breakdown on housing tenure is only available for England, Wales and Northern Ireland. The XLSX file is 1.2MB, which is very large. I'll put it in the raw directory and not check it into GitHub. If you wanted to run this code, you'd need a copy of the XLSX file.

We need some code to help clean the data and put it all together. I'm going to write these as functions to save time, as there are multiple sheets in the xlsx file to read.

get_variable is a utility function that filters the data set by selecting the variable topic_name from the column col_name.

get_data is collection of steps to clean and reshape the data in the XLSX file. Given that we'll need to do this across 2 different sheet, and there are other potential variables we may want to investigate in the future, it's worth generalising this code somewhat. get_data takes 3 arguments: sheet_name: the sheet in the XLSX file to read, topic_name: passed through to get_variable, and drop_cols: columns we won't need at the end.

def get_variable(data, col_name, topic_name):
    data = data.loc[data[col_name] == topic_name]
    return data

def get_data(sheet_name, topic_name, drop_cols, value_type=float, header=2, is_percent=True):
    # Set file path
    path = 'raw/Demographic-data-for-new-parliamentary-constituencies-May-2024.xlsx'

    # Read the file and cut the data by topic_name
    data = data = pd.read_excel(ROOT / path, sheet_name=sheet_name, header=header).pipe(get_variable, 'Topic', topic_name)

    # Fix a naming error
    data.replace('Ynys Mon', 'Ynys Môn', inplace=True)

    # Drop unused columns
    data.drop(columns=drop_cols, inplace=True)
    
    # Rename columns to merge with constituency codes
    data.rename(columns={'New constituency name': 'PCON24NM', 'Constituency value': topic_name}, inplace=True)

    # Convert the data types to integer
    data[topic_name] = data[topic_name].astype(value_type)

    # Get a list of unique Variable names, and how many there are, for the given topic.
    v_names = list(data['Variable'].unique())
    v_num = len(v_names)

    # If there are multiple variables for a given topic, we want to pivot the data so that each of of these has its own column.
    if v_num > 1:
            print(f'More than one variable type: {v_names}. Pivoting the data\n')
            data = data.pivot(index='PCON24NM', columns='Variable', values=topic_name)
    
    # If there is only one variable, we can drop the variable column as we don't need it.
    if v_num == 1:
        print(f'Only one variable type: {v_names}. Dropping the variable column\n')
        data.drop(columns='Variable', inplace=True)

    # Now we can merge the geography codes and data, and set the index to the geography code.
    data = data.merge(geo_codes, on='PCON24NM', how='inner').set_index('PCON24CD', drop=True)

    # If the data is a pct, make it out of 100 and round to 1dp.
    if is_percent:
        data[v_names] = data[v_names].mul(100).round(1)

    # Drop PCON24NM column as we don't need it.
    data.drop(columns='PCON24NM', inplace=True)

    return data

ew_total_households = get_data('EW_data', 'Households', ['Topic', 'England & Wales value'], is_percent=False).drop(columns='index')
ni_total_households = get_data('NI_data', 'Households', ['Topic', 'Northern Ireland value'], is_percent=False).drop(columns='index')
# Concatenate (join) the two dataframes.
total_households = pd.concat([ew_total_households, ni_total_households])
uk_tenure = pd.concat([ew_housing_tenure, ni_housing_tenure]).merge(total_households, on=['PCON24CD', 'PCON24NM'])

# While we're here write this file to the data directory
uk_tenure.to_csv(ROOT / 'data/tenure/tenure_type.csv')
uk_tenure

Next let's calculate some mean averages for each housing tenure type. We would need to work out the actual number of households for each tenure type per constituency, add them up for the whole of the UK and divide by the total number of households. However, since the number of households in each constituency are approximately equal, we can go ahead and just use the averages of the percentages for each constituency. In this case our "sample size" for each constituency is roughly the same, so it works out. We can ONLY do this because the number of households are approximately equal, and below we've showed that the two methods produce identical results to 0 decimal places.

def avg(data, column):
    number_households_by_tenure = (data[column] / 100) * data['Households']
    total_households_by_tenure = sum(number_households_by_tenure)
    total_households = sum(data['Households'])
    average_of_tenure_type = 100 * total_households_by_tenure / total_households
    return print(f"{column}: {round(average_of_tenure_type)}")

print('Using the full method with sample size:\n')
avg(uk_tenure, 'Owned outright')
avg(uk_tenure, 'Owned with a mortgage or loan')
avg(uk_tenure, 'Private rented')
avg(uk_tenure, 'Social rented')
avg(uk_tenure, 'Other tenure')

uk_tenure_means = uk_tenure.mean(numeric_only=True).round()
print('\n Using the mean of the percentages:\n', uk_tenure_means)

We want to merge the uk_tenure and ge_results data using the geography code to match the rows. In order to do this, we need to ensure the columns containing that data have the same name.

ge_results.rename(columns={'ONS ID':'PCON24CD'}, inplace=True)
d = ge_results.merge(uk_tenure, on='PCON24CD')
d

Now, let's define a dictionary for some colours of each party. We'll use the official hex codes.

cmap = {
    "Con":"#0485A8", 
    "Lab":"#DC4343",
    "RUK": "#09d2e0",
    "LD": "#EAA544",
    "DUP": "#BF3759",
    "SF":  "#2C604D",
    "SDLP": "#53BC5B",
    "Ind": "#440d54",
    "Green":"#5EBD4C",
    "PC": "#4FBA7C",
    "Spk": "#909090",
    "APNI": "#EAA544",
    "TUV": "#6dcad2",
    "SNP":  "#EBDB1C",
    "UUP": "#3b75a8"
}

We're ready to make the visualisation. We want to plot the percentage of homes by each housing tenure type for each constituency, coloured by the winning party. We can do this using something called a stripplot.

def make_stripplot(data, x, y, hue, xlabel, ylabel, palette=cmap):
    ax = sns.stripplot(data=data, x=x, y=y, hue=hue, palette=palette)
    ax.set(xlabel=xlabel, ylabel=ylabel)
    avg = uk_tenure_means[x]
    ax.vlines(avg, ymin=0, ymax=13, linestyles='--', label='UK average') # This plots a vertical line at the average
    return ax

Results

For all constituencies (excluding Scotland), the avereage percentage of homes owned outright was 33%. Labour seats have the greatest range, winning in constituencies where fewer than 10% of homes are owned outright, and some, conversely, with nearly 50% owned outright. Overall, 37% of their seats were above the average for percentage of homes owned outright.

Conservative and Lib Dem constituencies follow similar trends to eachother. The majority of constituencies are in areas where the percentage of homes owned outright is above the national average. 91% of conservative seats are above the national average for percentage of homes owned outright, compared to 79% for Lib Dems.

ax = make_stripplot(d, x='Owned outright', y='First party', hue='First party', xlabel='Owned outright (%)', ylabel='Winning party')
plt.savefig(ROOT / "src/assets/images/owned_outright.png")
plt.show()

Notes:

1. Only parties with 4 or more seats are included.
2. Home tenure type breakdown is not available for Scotland so no Scottish constituencies are shown here.
3. Data points are jittered depending on the number of constituencies each party won. This is to make the data clearer to see when there are lots of points close together.

Let's calculate the percentage of seats won by the three largest parties that had a higher than average number of people for each type of tenure. First, I'll define a function to calculate that, then call it below for different tenure types.

def constituencies_won_higher_than_average_tenure_type(data, tenure_type):
    for party in ['Lab', 'Con', 'LD']:
        party_seats_above_average = len(data.loc[(data['First party']==party) & (data[tenure_type] > uk_tenure_means[tenure_type])])
        party_seats = len(data.loc[data['First party']==party])
        frac = party_seats_above_average / party_seats
        p = round(frac * 100)
        print(f'For {party}, {p}% of their seats have higher than UK average percentage of homes with the tenure type: {tenure_type}')
    return 

constituencies_won_higher_than_average_tenure_type(d, 'Owned outright')

For Lab, 37% of their seats have higher than UK average percentage of homes with the tenure type: Owned outright. For Con, 91% of their seats have higher than UK average percentage of homes with the tenure type: Owned outright. For LD, 79% of their seats have higher than UK average percentage of homes with the tenure type: Owned outright.

ax = make_stripplot(d, x='Social rented', y='First party', hue='First party', xlabel='Social rented (%)', ylabel='Winning party')
plt.savefig(ROOT / "src/assets/images/social_rented.png")
plt.show()
constituencies_won_higher_than_average_tenure_type(d, 'Social rented')

For Lab, 56% of their seats have higher than UK average percentage of homes with the tenure type: Social rented. For Con, 5% of their seats have higher than UK average percentage of homes with the tenure type: Social rented. For LD, 2% of their seats have higher than UK average percentage of homes with the tenure type: Social rented.

ax = make_stripplot(d, x='Private rented', y='First party', hue='First party', xlabel='Private rented (%)', ylabel='Winning party')
plt.savefig(ROOT / "src/assets/images/private_rented.png")
plt.show()
constituencies_won_higher_than_average_tenure_type(d, 'Private rented')

For Lab, 46% of their seats have higher than UK average percentage of homes with the tenure type: Private rented. For Con, 9% of their seats have higher than UK average percentage of homes with the tenure type: Private rented. For LD, 20% of their seats have higher than UK average percentage of homes with the tenure type: Private rented.

ax = make_stripplot(d, x='Owned with a mortgage or loan', y='First party', hue='First party', xlabel='Owned with a mortgage or loan (%)', ylabel='Winning party')
plt.savefig(ROOT / "src/assets/images/mortgage_or_loan.png")
plt.show()
constituencies_won_higher_than_average_tenure_type(d, 'Owned with a mortgage or loan')

For Lab, 47% of their seats have higher than UK average percentage of homes with the tenure type: Owned with a mortgage or loan. For Con, 73% of their seats have higher than UK average percentage of homes with the tenure type: Owned with a mortgage or loan. For LD, 61% of their seats have higher than UK average percentage of homes with the tenure type: Owned with a mortgage or loan.