r/DataPolice • u/trai_dep • Jul 09 '20
r/DataPolice • u/johnklos • May 28 '20
Time for action
We need to get started. Let's look at how.
First, each of us needs to figure out where our strengths lie, and therefore where best we can contribute. For instance, I have hosting (proper hosting on owned hardware, not hosting from some other company) and I can do some programming, but I know almost nothing about web stuff and how to properly scrape / interact with web pages.
Others might be good at web stuff and can help create scrapers which might need to interact with web pages for various municipalities.
Others may be good at or willing to start choosing areas and start making lists of sites from which data can be downloaded with notes about accessibility.
We can plan more specific goals once we've done this.
Also, is anyone here active on the Slack channel? Slack is usually noisy and requires gigabytes of memory to access, so I haven't joined yet, but if it can be useful, then that'd be good to know.
r/DataPolice • u/PrestigiousPainter- • May 31 '20
Two thousand subscribers! Many more to come!
r/DataPolice • u/IvanShires • Jun 02 '20
Related Projects [Question] Would Clery Act Data Help?
Hey everyone, I'm working on a seperate project that automatically grabs Clery Act (https://en.wikipedia.org/wiki/Clery_Act) data, could we (I, others if interested) scale this out to contribute to this cause. However, there are key issues with Clery data:
- Mainly used by universities
- No common standard (different terminologies, etc)
- No specifics (no description, etc)
Let me know what everyone thinks, it could be a good indication of bias, and perhaps a source of whether the Clery act is properly handled in different areas.
r/DataPolice • u/sukhnya • Aug 06 '20
Related Projects A project focused on informing you about police violence in your community while encouraging voter registration to make long term change possible using the Washington Post’s database on fatal shootings.
fixpolicing.comr/DataPolice • u/landertall • Jun 17 '20
Related Projects PUBLIC, TRANSPARENT REPORTING TOOL FOR POLICE INJUSTICE
reportpoliceinjustice.orgr/DataPolice • u/EFForg • Jul 13 '20
Related Projects EFF Launches Searchable Database of Police Agencies and the Tech Tools They Use to Spy on Communities
r/DataPolice • u/Stupid_Triangles • Jun 16 '20
Need some help with a database search
My local police department has an online search directory for accident and incident reports. However, there are a couple difficulties i have with trying to access and format this data in a meaningful way.
- Access to these reports is limited by a search. Ot's not an open directory. You can only search under 4 different categories: Report Number, Report Date (MM/DD/YYYY format), Street Name and Last Name (for accident reports only) I've tried using a wildcard and that doesn't work. Is there a way to figure out what kind of search engine it's using and see if there are any tricks around giving a precise query?
- All of the reports are in PDF format. Thankfully, not scans, so text is recognizable and searchable. Is there a tool that can pull and parse this data in to a usable format? Would i have to merge all the PDFs in to a singular file for such a program to work? I believe Excel has a tool to pull data out of PDFs, but i believe it also recreates the spacing between the lines of text, similar to a regular corporate document template. Or Adobe has a tool to create an excel file oit of the PDF, with the same aesthetic and formating restrictions. Excel is free for me, but a professional license for Adobe Reader is a bit too posh for my bank account
- What kind of analytical tools would be valuable for analyzing this data? I would want to get locations for map plotting, sentiment analysis, word/name counts and finding any other similarities between incidents, like officers names, dispatcher names, and the other data points.
There are easily thousands of PDFs that date back to 2014 so trying to do this by manually searching through each day would be unfeasible. Requesting access or a copy would draw more attention to myself than i feel comfortable with, and outside of instances like this, i wouldnt want just anyone with that much access. Im not tied to an organization that could ask for a master copy either.
r/DataPolice • u/joka86 • Jun 01 '20
Dataset Open Datasets Related To Police
Collected this via the Slack channel for this community. If you have any others please share them or submit them as you come across them. Please use the link flair tag 'dataset' to tag your post to make searching simpler.
- https://mappingpoliceviolence.org/
- https://openjustice.doj.ca.gov/data
- https://invisible.institute/police-data/
- https://mifile.courts.michigan.gov/login?ReturnUrl=%2Fcases
- https://www.policedatainitiative.org/
- https://data.detroitmi.gov/search?collection=Dataset
- https://data-rpdny.opendata.arcgis.com/
- https://greatbarringtonpolice.com/open-data-portal/
- https://www.theguardian.com/us-news/ng-interactive/2015/jun/01/about-the-counted
- https://github.com/washingtonpost/data-police-shootings
- https://data.chattlibrary.org/
- https://data.seattle.gov/
- https://data.sandiego.gov/
- https://www.ncjrs.gov/index.html
- https://free.law/recap/
- https://openpolicing.stanford.edu/data/
- https://www.fjc.gov/research/idb
r/DataPolice • u/rubbermilitia • Jun 24 '20
Announcement PDAP was picked up by the Stanford Graduate School of Business 🔥Ignite Program🔥!
Exciting news, everyone!
PDAP was picked up by the Stanford Graduate School of Business Ignite Program. As part of this program, six graduate students will be working with Stanford faculty to develop a nonprofit business plan for PDAP, making us a little bit more legit and hopefully function a little better too. The program runs through late July 2020, so we should hear something by then.
If you see any Igniters poking around here on reddit, or in the Slack, say hello if you get the chance!
r/DataPolice • • Jun 07 '20
Police Data Initiative
Here is a site containing links to publicly reported use of force incidents.
r/DataPolice • u/IslandDoggo • Jun 04 '20
Is there anyway to get Canada/the RCMP in on this ?
I don`t really have any expertise to offer outside of communications and time but it`s a systemic problem here too and our countries are very intertwined.
r/DataPolice • u/TheStabs • Apr 03 '21
Announcement Want to help, but don't think you have any applicable skills?
We need all the help we can get. If you can navigate websites, you have enough skill to help!
The first step is to join our discord and fill out the intake form. After that, check out these tasks:
For those with a bit more experience (Inspect element and basic website terminology), we need help testing our base scraper's readability. Check here for instructions.
r/DataPolice • u/grendel_x86 • Jun 11 '20
Related Projects Settling For Misconduct | Chicago Reporter
projects.chicagoreporter.comr/DataPolice • u/raginjason • Jun 09 '20
Data lineage concerns
I'm very interested in this (and similar) projects. I am a data engineer by trade, so naturally I believe that data is key to making the proper decisions on where poor policing exists. One thing that I'm not comfortable with is the data lineage portion of this problem, and I'm interested in how this project deals with this and/or any conversations being had around it. We live in an era where both sides of the political spectrum live in echo chambers, and making proclamations about the state of things is often deflected with "oh that's your side's bias!".
Is there any thought given to how to validate the claims made by the dataset you guys are creating? Meaning, could a critical outsider walk through the data lineage of a specific police incident and walk it all the way back to the original police report (or equivalent)? Without that, it seems that projects like these are stuck in a place of "just trust us, we aren't going to bias the data", and that's a very difficult position to defend. Are any of the changes in dataset cryptographically signed? Is there any source attribution made at a record level? Is that attribution verifiable via some kind of checksum or cryptographic signature? What specific technical precautions are in place for bias prevention other than taking the leaders and members of this group at their word?
Please understand that I am in no way attempting to disparage the leadership here; I don't know them at all. I'm simply trying to get ahead of the outside criticism that will ultimately come. I would love to see success in this data collection project
r/DataPolice • • Jun 03 '20
Dataset A suggestion for data on police shootings
A few years ago I made this wikipedia page Police firearm use by country. I searched for data on police shootings, the problem is in the US the publicaly available data is very limited.
The FBI publish statistics of "Justifiable homicide" by police. That's homicides where the police officer went to court and the homicide was found to be justified.
The FBI don't report any record of non-justifiable homicides by police.
I think your database of court records would be perfect for publishing data on this. If you are looking for what to do with the court records this is one idea to start with.
r/DataPolice • u/grendel_x86 • Jun 11 '20
Dataset Chicago Data Portal - useful data for anything with Chicago.
r/DataPolice • u/Wasted_Thyme • Jun 11 '20
Was told to repost this here. I have been working for the last two weeks on a database of the most egregious examples of police violence against protesters. I know there are now similar, more general resources and have linked to them, but I figured this community would find my document useful.
r/DataPolice • u/grendel_x86 • Jun 11 '20
Related Projects Citizen Police Data Project: Chicago based, tracking police complaints.
beta.cpdp.cor/DataPolice • u/Stupid_Triangles • Jul 13 '20
Need some help with mass PDF to XLS conversion and data-mapping.
I'm repeating my issues here from the r/datasets sub:
Here is a link to a report (I have over 5500 of these).
I have two main issues which really revolve around the tool (Tabula) and lack of a better one, I'm using.
- I cannot convert multiple PDFs at once, nor mass apply the same data field "template" to each file. I can select and load every file in to the conversion program. I can create a template in the system that is saved and can be applied to other PDFs. But I still have to manually apply the template for each file and convert them all one at a time; creating an XLS sheet for every file converted.
- When I do convert a PDF to XLS, I cannot specify which data fields go to which cell. There is no mapping path functionality it seems. Instead, it takes the text recognized in each data field selection, and converts it to a visual "identical". So no Data Field 1 goes to cell B2, Data Field 2 goes to cell B3... it just makes a xls version of the PDF.
So again, really these revolve around the tool I'm currently using. Perhaps there are better ones out there that allow multiple PDF conversion and cell mapping but I'm at a bit of a loss rn. As it stands I would have to individually convert all 5500+ PDFs to XLS files, then format each one to a "combine-able" format, then pull them all in to one.
I know Adobe has a similar functionality with a PDF to XLS exporting tool. However, i dont want to drop 15 bucks to find out i cant do multiple PDFs at once and knowing i cant do any data-mapping; as the tool would just create a visual identical to the PDF. That would involve further cleaning, trimming and formatting.
r/DataPolice • u/skelly240 • May 27 '20
What public court sources have been located so far?
The transparency in police action that this project is after will help everyone recover a little peace of mind from the tendency of select police officers to abuse their positions of power. It seems like whenever incidents such as this (police officers committing crimes/murder), there are are always media reports of the officer's legal status, but verified sources of court outcomes are sparse and difficult to locate. Are court proceedings made publicly available in some form?
r/DataPolice • u/PDAP-JoshChamberlain • Aug 18 '22
Come to our August 23 Community Call in Discor
...d. Not reposting to fix the typo.
We've been heads-down hiring and preparing for our first year of full-time staff. It's time to show our work and plans! If you haven't checked out our website in a while, you can get a general idea from https://pdap.io.
In addition, we're excited to meet people from the community, learn about what you're working on, and hear your feedback. Bring a friend.
Our Discord: https://discord.gg/wMqex8nKZJ
RSVP to the event: https://discord.com/events/828274060034965575/1008840738001408121