{"id":1136,"date":"2024-04-04T07:00:01","date_gmt":"2024-04-04T12:00:01","guid":{"rendered":"https:\/\/wollen.org\/blog\/?p=1136"},"modified":"2024-10-18T02:58:52","modified_gmt":"2024-10-18T07:58:52","slug":"violent-crime-in-the-united-states","status":"publish","type":"post","link":"https:\/\/wollen.org\/blog\/2024\/04\/violent-crime-in-the-united-states\/","title":{"rendered":"Violent crime in the United States"},"content":{"rendered":"

The FBI Uniform Crime Reporting Program<\/a> (UCR) collects crime-related data from thousands of law enforcement agencies across the country. The FBI then compiles the data and publishes quarterly and annual reports. It isn’t a perfect snapshot of crime as participation among agencies is voluntary, but it is the best picture we have.<\/p>\n

For this post I’ll be digging into the most recent annual release, October 2023, which covers data through the end of 2022. You can download the spreadsheet I’m using by clicking over to the FBI’s Crime Data Explorer<\/a> and finding the section shown below. I’ll also link the file directly at the bottom of this post.<\/p>\n

\"\"<\/a><\/p>\n

\n

1. Prepare the data.<\/h4>\n

You might notice the spreadsheet isn’t as clean as the CSV files I usually work with here. The federal government doesn’t have the softest touch. But in the spirit of imperfect data I want to leave the Excel file as it is and go through the process of cleaning it for presentation. Sure, it would be easier to copy-and-paste the target data into a clean file and manually remove any weird stuff, like footnotes or trailing white space. But data is rarely in a perfect form so let’s practice meeting it where it is.<\/p>\n

After importing pandas and Matplotlib, set an option that will allow us to see the entire DataFrame’s width on screen. Because the spreadsheet has so many columns\u2014and several with very long names\u2014pandas will try to collapse the DataFrame whenever we print it to screen. It will replace the majority of the data with ellipses. Changing this setting will avoid truncation and make it easier to visualize our work.<\/p>\n

import pandas as pd\r\nimport matplotlib.pyplot as plt\r\nfrom matplotlib.offsetbox import OffsetImage, AnnotationBbox\r\n\r\npd.set_option(\"display.expand_frame_repr\", False)<\/pre>\n
Read the spreadsheet into a DataFrame and take a look.<\/p>\n
df = pd.read_excel(\"Table_1_Crime_in_the_United_States_by_Volume_and_Rate_per_100000_Inhabitants_2003-2022.xlsx\")\r\n\r\nprint(df.head())<\/pre>\n
The output is shown below. In this case I’m using ellipses because it would be too much to display in a blog post. The important thing is to see what a mess the DataFrame is.<\/p>\n
                                             Table 1                    Unnamed: 2  [...]\r\n0                         Crime in the United States          NaN              NaN  [...]\r\n1  by Volume and Rate per 100,000 Inhabitants, 20...          NaN              NaN  [...]\r\n2                                               Year  Population1  Violent\\ncrime2  [...]\r\n3                                               2003    290809777          1459416  [...]\r\n4                                               2004    293655404          1428745  [...]<\/pre>\n
The first problem is that the column labels (Year<\/em>, Population1<\/em>, et al.) are located in the third row.<\/p>\n
To solve it we can grab that specific row using iloc<\/code> and parse the strings into something more readable, then tell the DataFrame that those are its new column labels.<\/p>\n
column_labels = [str(item).replace(\"\\n\", \" \").replace(\"  \", \" \").strip() for item in df.iloc[2]]\r\ndf.columns = column_labels<\/pre>\n
The second problem is the extraneous rows at top and bottom of the spreadsheet. We’re only interested in rows that correspond to yearly data from 2003 to 2022.<\/p>\n
We can simply redefine df<\/code> to be a subset of its rows.<\/p>\n
df = df[3:23]<\/pre>\n
The last change we’ll need to make is in the Year<\/em> column. There is a footnote in the 2021 row that changes its value. The tail of the column looks like this:<\/p>\n
     Year\r\n18   2018\r\n19   2019\r\n20   2020\r\n21  20215\r\n22   2022<\/pre>\n
But this is an easy fix. Rather than applying a function we can just redefine the row to be the numbers from 2003 to 2022.<\/p>\n
df.loc[:, 'Year'] = range(2003, 2023)<\/pre>\n
At this point we could begin plotting but I have an idea to make the data a little more informative.<\/p>\n
I want to create two plots in the output image. On top will be the most-often cited statistic, overall violent crime rate. And on bottom it will show percent change in each subcategory (murder, assault, robbery) over the past 20 years. In absolute terms, robbery rate is much higher than murder rate, but we can peg both at 100% and see how they change relative to each other.<\/p>\n
Create new columns to hold this data. Divide each cell by the topmost row in its column (2003) and multiply by 100 to express the value as a percentage.<\/p>\n
df.loc[:, 'murder_rate_change'] = df['Murder and nonnegligent manslaughter rate'] \/ df['Murder and nonnegligent manslaughter rate'].iloc[0] * 100\r\ndf.loc[:, 'assault_rate_change'] = df['Aggravated assault rate'] \/ df['Aggravated assault rate'].iloc[0] * 100\r\ndf.loc[:, 'robbery_rate_change'] = df['Robbery rate'] \/ df['Robbery rate'].iloc[0] * 100<\/pre>\n
\n
2. Plot the data.<\/h4>\n
This plot uses a custom Matplotlib style that I’ll link at the bottom of this post.<\/p>\n
Specify the shape of the subplot array in plt.subplots()<\/code>\u20142 rows, 1 column. In other words we’ll have two plots stacked in a vertical orientation. When I create multiple axes on the same figure I like to name them axs<\/code> rather than the usual ax<\/code>. Each can be addressed as axs[0]<\/code> or axs[1]<\/code>.<\/p>\n
plt.style.use(\"wollen_dark.mplstyle\")\r\nfig, axs = plt.subplots(2, 1, figsize=(14, 11))<\/pre>\n
The violent crime rate plot is below. The Matplotlib code is fairly straightforward so I won’t bog down the post with too much commentary.<\/p>\n
I’m overlaying an FBI seal onto the plot with low alpha<\/code> to give it a watermark effect. The trickiest part is to correctly set box_alignment<\/code>. (0, 0) means the lower-left corner of the image will be placed at the specified location. (0.5, 0.5) would center the image at that point.<\/p>\n
axs[0].plot(df['Year'], df['Violent crime rate'], color=\"#B0E441\", marker=\"o\", markersize=6)\r\n\r\nx_ticks = df['Year'].tolist()\r\naxs[0].set_xticks(x_ticks)\r\nx_range = x_ticks[-1] - x_ticks[0]\r\nx_left, x_right = x_ticks[0] - x_range * 0.02, x_ticks[-1] + x_range * 0.02\r\naxs[0].set_xlim(x_left, x_right)\r\n\r\ny_ticks = range(360, 540, 20)\r\naxs[0].set_yticks(y_ticks)\r\ny_range = y_ticks[-1] - y_ticks[0]\r\ny_bottom, y_top = y_ticks[0] - y_range * 0.01, y_ticks[-1] + y_range * 0.01\r\naxs[0].set_ylim(y_bottom, y_top)\r\naxs[0].set_ylabel(\"Rate per 100,000\")\r\n\r\naxs[0].set_title(\"United States Violent Crime Rate  \u2022  2003\u20132022\")\r\n\r\naxs[0].text(x_right - x_range * 0.005, y_ticks[-1] - y_range * 0.01,\r\n            \"Data:  FBI Crime in the Nation, October 2023.\",\r\n            size=10, ha=\"right\", va=\"top\")\r\n\r\nab = AnnotationBbox(OffsetImage(plt.imread(\"fbi_seal.png\"), zoom=0.2, alpha=0.05),\r\n                    (x_ticks[0], y_ticks[0]), box_alignment=(0, 0), frameon=False)\r\naxs[0].add_artist(ab)<\/pre>\n
Next is the bottom plot, axs[1]<\/code>, which will display change in violent crime subcategories over the past 20 years.<\/p>\n
This code is a little simpler because it doesn’t include an image or citation text. We can also reuse previous definitions for the x-axis.<\/p>\n
Remember to include a legend so readers can identity each crime subcategory. The data series trend downward so lower-left is a nice spot for a legend.<\/p>\n
axs[1].plot(df['Year'], df['murder_rate_change'], marker=\"o\", markersize=6, label=\"Murder and Nonnegligent Manslaughter\")\r\naxs[1].plot(df['Year'], df['assault_rate_change'], marker=\"o\", markersize=6, label=\"Aggravated Assault\")\r\naxs[1].plot(df['Year'], df['robbery_rate_change'], marker=\"o\", markersize=6, label=\"Robbery\")\r\n\r\naxs[1].set_xticks(x_ticks)\r\naxs[1].set_xlim(x_left, x_right)\r\n\r\ny_ticks = range(0, 140, 20)\r\naxs[1].set_yticks(y_ticks)\r\naxs[1].set_yticklabels([f\"{n}%\" for n in y_ticks])\r\ny_range = y_ticks[-1] - y_ticks[0]\r\ny_bottom, y_top = y_ticks[0] - y_range * 0.01, y_ticks[-1] + y_range * 0.01\r\naxs[1].set_ylim(y_bottom, y_top)\r\n\r\naxs[1].set_title(\"Subcategories  \u2022  Change Since 2003\")\r\n\r\naxs[1].legend(loc=\"lower left\")<\/pre>\n
Finally, save the figure. I bump up dpi<\/code> from its default 100 to aid readability.<\/p>\n
plt.savefig(\"fbi_violent_crime.png\", dpi=150)<\/pre>\n
\n
3. The output.<\/h4>\n
\"\"<\/a><\/p>\n
Overall the country is much safer than it was 20 years ago. We saw an uptick in violent crime during the COVID-19 pandemic, especially within the murder category, but we’ve nearly returned to the lowest levels on record.<\/p>\n
So far, quarterly data has pointed to another large decline in 2023<\/a>. Experts say it could be the largest recorded year-over-year drop in the murder rate. We’ll learn more when the FBI released full 2023 data this October.<\/p>\n
\n
Download the data.<\/strong><\/a><\/p>\n
Full code:<\/strong><\/p>\n
import pandas as pd\r\nimport matplotlib.pyplot as plt\r\nfrom matplotlib.offsetbox import OffsetImage, AnnotationBbox\r\n\r\n\r\npd.set_option(\"display.expand_frame_repr\", False)\r\n\r\ndf = pd.read_excel(\"Table_1_Crime_in_the_United_States_by_Volume_and_Rate_per_100000_Inhabitants_2003-2022.xlsx\")\r\n\r\ncolumn_labels = [str(item).replace(\"\\n\", \" \").replace(\"  \", \" \").strip() for item in df.iloc[2]]\r\ndf.columns = column_labels\r\n\r\ndf = df[3:23]\r\n\r\ndf.loc[:, 'Year'] = range(2003, 2023)\r\n\r\ndf.loc[:, 'murder_rate_change'] = df['Murder and nonnegligent manslaughter rate'] \/ df['Murder and nonnegligent manslaughter rate'].iloc[0] * 100\r\ndf.loc[:, 'assault_rate_change'] = df['Aggravated assault rate'] \/ df['Aggravated assault rate'].iloc[0] * 100\r\ndf.loc[:, 'robbery_rate_change'] = df['Robbery rate'] \/ df['Robbery rate'].iloc[0] * 100\r\n\r\nprint(df.head(50))\r\n\r\nplt.style.use(\"wollen_dark.mplstyle\")\r\nfig, axs = plt.subplots(2, 1, figsize=(14, 11))\r\n\r\naxs[0].plot(df['Year'], df['Violent crime rate'], color=\"#B0E441\", marker=\"o\", markersize=6)\r\n\r\nx_ticks = df['Year'].tolist()\r\naxs[0].set_xticks(x_ticks)\r\nx_range = x_ticks[-1] - x_ticks[0]\r\nx_left, x_right = x_ticks[0] - x_range * 0.02, x_ticks[-1] + x_range * 0.02\r\naxs[0].set_xlim(x_left, x_right)\r\n\r\ny_ticks = range(360, 540, 20)\r\naxs[0].set_yticks(y_ticks)\r\ny_range = y_ticks[-1] - y_ticks[0]\r\ny_bottom, y_top = y_ticks[0] - y_range * 0.01, y_ticks[-1] + y_range * 0.01\r\naxs[0].set_ylim(y_bottom, y_top)\r\naxs[0].set_ylabel(\"Rate per 100,000\")\r\n\r\naxs[0].set_title(\"United States Violent Crime Rate  \u2022  2003\u20132022\")\r\n\r\naxs[0].text(x_right - x_range * 0.005, y_ticks[-1] - y_range * 0.01,\r\n            \"Data:  FBI Crime in the Nation, October 2023.\",\r\n            size=10, ha=\"right\", va=\"top\")\r\n\r\nab = AnnotationBbox(OffsetImage(plt.imread(\"fbi_seal.png\"), zoom=0.2, alpha=0.05),\r\n                    (x_ticks[0], y_ticks[0]), box_alignment=(0, 0), frameon=False)\r\naxs[0].add_artist(ab)\r\n\r\naxs[1].plot(df['Year'], df['murder_rate_change'], marker=\"o\", markersize=6, label=\"Murder and Nonnegligent Manslaughter\")\r\naxs[1].plot(df['Year'], df['assault_rate_change'], marker=\"o\", markersize=6, label=\"Aggravated Assault\")\r\naxs[1].plot(df['Year'], df['robbery_rate_change'], marker=\"o\", markersize=6, label=\"Robbery\")\r\n\r\naxs[1].set_xticks(x_ticks)\r\naxs[1].set_xlim(x_left, x_right)\r\n\r\ny_ticks = range(0, 140, 20)\r\naxs[1].set_yticks(y_ticks)\r\naxs[1].set_yticklabels([f\"{n}%\" for n in y_ticks])\r\ny_range = y_ticks[-1] - y_ticks[0]\r\ny_bottom, y_top = y_ticks[0] - y_range * 0.01, y_ticks[-1] + y_range * 0.01\r\naxs[1].set_ylim(y_bottom, y_top)\r\n\r\naxs[1].set_title(\"Subcategories  \u2022  Change Since 2003\")\r\n\r\naxs[1].legend(loc=\"lower left\")\r\n\r\nplt.savefig(\"fbi_violent_crime.png\", dpi=150)<\/pre>\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
The FBI Uniform Crime Reporting Program (UCR) collects crime-related data from thousands of law enforcement agencies across the country. The FBI then compiles the data and publishes quarterly and annual reports. It isn’t a perfect snapshot of crime as participation<\/p>\n","protected":false},"author":1,"featured_media":1158,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[238],"tags":[136,245,250,241,253,240,248,242,24,126,244,251,137,30,249,187,25,254,246,63,116,255,247,252,243],"class_list":["post-1136","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-government","tag-annotationbbox","tag-assault","tag-cops","tag-crime","tag-excel","tag-fbi","tag-federal","tag-government","tag-matplotlib","tag-mplstyle","tag-murder","tag-offsetbox","tag-offsetimage","tag-pandas","tag-police","tag-politics","tag-python","tag-read_excel","tag-robbery","tag-statistics","tag-stats","tag-subplots","tag-ucr","tag-united-states","tag-violent-crime"],"_links":{"self":[{"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/posts\/1136","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/comments?post=1136"}],"version-history":[{"count":23,"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/posts\/1136\/revisions"}],"predecessor-version":[{"id":1817,"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/posts\/1136\/revisions\/1817"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/media\/1158"}],"wp:attachment":[{"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/media?parent=1136"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/categories?post=1136"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wollen.org\/blog\/wp-json\/wp\/v2\/tags?post=1136"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}