data.bythebay.io: Jason Lally - Gathering around the data table
Recording: data.bythebay.io: Jason Lally - Gathering around the data table
thank you and thank you everyone for uh coming to my talk today um a couple of things uh I I frame this mostly around the notion of democracy in data um I'm logged in there at on the schedule as pipelines I can speak to pipelines I'm happy to discuss technical aspects of our program that's not what I'm going to talk about entirely in my presentation but I can answer questions to that to that effect as well if you're interested in that kind of stuff so um let's talk about gathering around the data table and I want to start with the question of how might we use data to improve our democracy um and I those of you who may have a a user centered design background May notice that I'm using the how mighte because the real the reality is we can use data in lots of different ways to improve our democracy I'm only going to focus on one little thread I'm not going to talk about um big data and dark money and politics I'm not going to talk about getting out the vote um and those sort of traditional uh uses of data to to sort of improve our democracy there's a lot of folks talking about that and talk about another aspect of this um which is really about H how we can all play a role in in having more conversations in in our in our democracy and so before I start again I'm Jason L um I am the open data program manager uh for um our open data program this is data SF uh I work with our City's Chief data officer um to operationalize our open data program and what that means is getting data pipelines out to the public so you can use it access it um and do interesting things with it um our mission is to empower use of the city's data and that is actually really important we actually take that very seriously um because uh our entire program is built around use of data as opposed to Simply the active publishing uh the publishing is important pipelines are important but really we're motivated by what people are going to do with it to improve our community um so democracy is important like I said we should all vote but democracy doesn't end at The Ballot Box um uh it it keeps going in San Francisco it happens here it happens in a lot of other places this is City Hall uh for those of you who haven't had the chance uh to go I suggest going I have the honor um to actually work in this building and um it's a very active Civic space protests weddings um it's a beautiful building uh and um there are constant acts of democracy happening here sometimes invisible sometimes visible um but within that within City Hall and throughout the city there are hundreds of boards and commissions or I'm sorry over a hundred not hundreds but over a hundred boards and commissions um there's the Board of Supervisors the legislative branch you've got the mayor's office and then this 50 or so departments I could show you all that stuff so what's one to do with all of those choices all of that activity right it's hard to make sense of it and I'm not going to talk about today making apps to connect to those things although someone needs to help us with that um there's lots of ways to plug into the City and it's sometimes very hard to know where to plug in what I'm going to talk about is a little bit more conceptual so bear with me um and that that's really about uh what I think is one of the most important tools that any data practitioner uh engineer any practitioner frankly can bring to democracy and discourse um does anyone take want to take a guess at what tool that is what's the most important tool telephone database database Twitter Twitter okay I'm gonna it's empathy um and so you may you may think I I know I that was a setup I'm sorry so uh it's this is actually it's lowc cost it's free it's something that we can all practice and you may be wondering what does empathy have anything to do with data and data pipelines and all this other stuff um data's rational right um but really especially when you talk about bringing data to a convers ation or bringing context to to an important issue you really have to understand where other folks are coming from you have to understand where where people are starting from because they may not be starting from where you're you're starting from right you may have particular skill or knowledge that someone else doesn't have and when we're talk about engaging in a conversation and bringing data to that really starting from a place of empathy is one of the most important tools that you you can bring to bear on that and what does empathy look like um there's three things that I I boil this down to three things to keep in mind um in terms of practicing empathy because um there's some people who are good you know just natural empaths um people who are drawn to sort of user centered design human- centered design those types of of of practices but it really is something that you can you can hone and you can really uh do you can act on it and there's two three things that I want to point out you can adopt a beginner's mind um when you're a approaching a big data or not even a big data problem but any data problem um going in uh trying to drop your preconceived notions um really adopting curiosity and and questioning the data is really important and I I point this out particularly in this context because I see a lot in the sort of in the sort of common discourse people come in with Solutions right and they say this data shows that my solution's right and there's not a lot of questions or what if or and the tone is very different so I think adopting a beginner's mind is one way to to practice empathy being TR transparent and this is really about methodology right so you're doing data work you're you're you're making you know you're making an assumpt a set of assumptions be transparent about what those assumptions are um and invite participation uh this is about using language and tone and not being too p Jal istic or or patronizing you know really putting uh your work out there um that invites participation I'll show you uh a couple examples that actually demonstrate this so this is a um blog post done by uh a gentleman Lance Martin um PhD down in I believe Stanford we didn't ask him to do this he just took some data off the portal and he uh he wanted to look at San Francisco's drug GE geography he grabbed all the data from SF open data this is from his blog post if you read the whole blog post there's a lot of sort of he's asking questions he's like a you know there's a really great tone that struck in the blog post and at the bottom but there's a lot more to look with this data I invite folks to pull the notebook and explore for themselves this is transparency that's what this looks like in the data world right all his methods are there this is in a py IPython notebook um it's not you don't have to use this tool to be transparent but it's it's one thing one way to act transparently um particularly when it relates to data um I uh had the opportunity to go to code for San Francisco which is a hack night uh every Wednesday night um and we participate in collaborative design around key Civic Civic challenges a lot of them uh involve data one of the groups there um that participates regularly is the data science working group they're digging into problems around 311 and and uh locations of fires and predicting fires and some really interesting data questions that are based on the data that's on the open data portal we work uh closely with them and there's the housing data Hub this was a product that we built with the with the Brigade and with departments to really try to describe all of the different programs that are happening around housing in San Francisco not to try to tell you what what was what we should do but just to say this is what we are doing right so again empathy just trying to surface the important data that people were asking questions about so that they could make more informed decisions and start to really look at the data in a different way um and this is a blog post that we wrote that describes that you're you you should check it out this describes our our sort of approach to to building that with the Brigade um and coming up uh national day of Civic hacking this happens every year um this year uh you the tagline is don't call it a hackathon um and that's very intentional uh it's really this year is really about user- centered design it's really about building empathy for users um government and uh citizens alike um I suggest you check it out uh come if you can um this will be a great way to again put empathy into practice so to um how much time do I have Okay so I'm going to really quickly before I finish up and to show you a couple things so I mentioned the open data portal um this is this is where data ends up uh you you can go to data. sfgov.org um it's very easy to find you can Google it um uh you can explore any number of data sets through categories through search you can if you know the department you want to get data from you can go there I'm not going to give you a full rundown of how to use the portal but I just want to show you I want to point you in in the right direction here so here's PD uh incidents crime data really popular data set um and under the export tab uh not download unfortunately but export um you can go to the API um and so there is actually on every data set where there's a table you can actually access a it is not a private data set um it's a public data set I don't know why I'm getting that but um uh so generally it shouldn't do that and if you ever get something like that let me I'll you can send a help request we'll make sure that you you should shouldn't be private there but basically um there's API documentation on every data set uh that will tell you how to use the data how to access it um and another thing that we're working on right now um oh and I should mention it you can access it in Jon uh c as a CSV as well as geojson if there's geographic information in there um one last thing that we're working on that is I'm going to give you a little preview of and this is not um we're not we this isn't anywhere that you can look at yet um but it will be an open source project we have a Sprint to do yet um before we really kind of feel like we can it more formally or not really formally but just put it out on GitHub for everyone to access um we're working on we've heard from our departments uh that they've they are they're not able to really visualize the data and so we're leveraging those apis that I just showed you to actually build a comprehensive uh chart analysis tool that will allow um folks to um I this is very much in preview so it's not it's very very unpolished right now but uh so here's eviction notices eviction notice data I chose file date and here it is over time um you know and I can quickly go and say you know I wouldn't actually want to do this this is a who uh supervisor District you know bu supervisor District but the main point here is what we're doing is over the next couple weeks we're going to be doing some user testing with our departments really understand what kinds of simple charts they want to do and then we're going to get that out into the world um in in our uh open source repository uh and GitHub um and let me with that couple of things you can do uh that are just very straightforward explore data on the open data portal I just showed you how to do that come to a hack night um code for sanfrancisco.org there this is a link I'll make sure this gets out to folks uh join us at National Day Civic hacking I showed you where you can get that um follow us data SF on GitHub um so uh again that's a link so you can get to us github.com SF and then keep an eye on our blog for more opportunities so datas sf.org blog um we'll be talking more and more about ways that you can engage with us um so please join us at the data table uh we want to create with you um so that we can actually improve uh our Democratic discourse with that take questions um could you give talk a little bit about privacy in this context and um and the kind of issues that you've experienced as trying this and how um the city has work to I love that question because I actually think I have an answer to it um well I don't have an answer uh we have but we have an approach so um my team I and I I should have mentioned this so we have the chief data officer who's my boss uh Joy bonaguro um I am one of an of three other people and uh Erica finle is our Shar SF cor um program manager and that's about internal data sharing she's also focusing on creating a comprehensive um sort of privacy framework for open data so the the short answer is this is actually um this is this question is coming up a lot more and people have not yet are just beginning to Grapple with this and we are are we air on the side of being right now being very cautious until we have a very um robust privacy framework an approach we're going to actually publish uh that approach um through our blog um we're going to make that public document we're working with Partners at Harvard and and other places to to build that out and and because that is actually there's a deficit right now around thinking about the interplay between openness and privacy um particularly there was a question in the earlier talk about um as you get more and more data out there there are more and more ways you can de identify things right and so what is the ri what is the risk there and so we're going to be probably taking a there there's no such thing as um uh a 100% um uh zero risk rather I should say but we're we're going to be probably taking a risk-based approach um to to that that very question so it's not there yet but it it's coming questions other question you want to know what stack we're on or anything like that thank you very much thank you