data.bythebay.io: Katherine Ahern, UX Techniques Supporting Varying Levels of Aggregation
Recording: data.bythebay.io: Katherine Ahern, UX Techniques Supporting Varying Levels of Aggregation
thank you so much it's been uh such a pleasure and a privilege to hear all the other speakers and it's been very validating because I've been in this room and I think I would just estimate about 70% of the speakers have name checked Don Norman design of things and I'm going to do the same I get so excited when um uh people from a design background um bring up that book I actually I'm a a data engineer I I don't have a design background but um I've gotten very interested in uh user experience and then the results that you get at various levels of aggregation over large data sets my company clear story data um is a cloud-based uh largescale analytics platform and uh my team does the exploratory data selection and visualization it's on a a D3 react stack so we pay a lot of attention to user experience of data selection um and I had levels of aggregation just on my mind you know because when you change the granularity of a a chart um it looks really different and helping users understand why they get results um at levels of aggregation and helping users understand that you actually often have to aggregate if you want to visualize large data sets you can't put all of your four billion rows onto a a screen um even a big one uh and you're certainly not going to get a chart um this is just a slide uh one of my uh this is just an example from a statistics book um um about how the same data set can yield very different results based on your level of aggregation so one example is uh you could have a a politician or a policy analyst say things are our economy is in trouble because in average income has gone down in significantly more states in the United States than it's gone up and uh between 20 I'm totally making this up but between 20133 and 2014 um 48 of our 50 states had an average income that declined and we need to take aggressive measures to turn things around and vote for someone else um and the exact same data set can say actually average income went up between 2013 and 2014 and you can have the same data set and come to exactly opposite conclusions and you can have visualizations that will support opposite conclusions for policy analysts and the reason for this is that um our most populous states May or our urban areas may have had incomes that went up so you may have more individuals whose income increased but more States like some of the less populous states maybe because of distribution of industry or uh whatever um both can be true you can have a majority of States whose income went down I used to do data analysis um for a company that did emergency room analytics and we'd have all these we talked about like time to provider a lot like how long are you going to have to wait if you get to an emergency room how much Staffing do you want to do so people don't wait too long um and uh at the hospital level there was tremendous variation in in average weight times like we had some hospitals where you were going to wait 40 minutes to see someone in the emergency room you had hospitals where you would see someone in two minutes um and so there was the question of the population of patients what is the average time to provider um because if you look at it at the patient level it was actually pretty high it was like 30 minutes um but if you average Hospital weight times because a proportion of the hospitals had almost instantaneous time to provide her you would get something like 8 minutes um and these were very different conclusions to present to the CEO of the emergency room management company you know there were a thousand hospitals and you're saying oh you know it's eight minutes time to provider we're doing great or be like 30 minutes you know that's the that's kind of long if you're in the emergency room and it was all levels of aggregation it was are you doing the the hospitals that had a long time to provider tended to have the highest throughput you you would have one hospital if they see a 100,000 patients a year versus a hospital that sees 600 patients a year you just have very different results for those averages um and so aggregation is super important um and levels of aggregation are are very important and you have to think about that for what you want to communicate because doing um an average of the hospitals if you're doing Hospital emergency room management could there were people who argued that that was reasonable that an 8 minute wait time by hospital is actually a good metric like that doesn't undervalue some of the quicker hospitals uh this is where I I just um I just had I don Norman on my mind um how many of you are designers great awesome uh data Engineers great uh data scientists okay well that I think that sums up to just about 100% um so uh this is the famous door where you shouldn't give users things that are misleading uh so for example the handle indicates maybe you could pull on the door but this is a door that only opens by pushing um uh and then this is my you know my favorite Norman example cuz uh every bathroom you go to like figuring out how to get the hot water and the cold water and like which way to twist it um which handle is hot which one is cold um and sometimes you have Clues like you'll have hot or cold you'll have red or blue uh and for this you're really trying to do one thing like you want to get kind of a warm temperature from the sink and then you think about this and you think about a data analytics workflow where you're combining data sets from different sources you don't know what to join on you don't know how anything's going to pass the smell test because you don't even really have industry knowledge like uh choosing what features to offer the user how to present them what buttons what to do underneath those buttons to give plausible results um I I one thing I love about this conference is uh this morning the keynote speaker I think he's the CEO of analytic um was talking about how this industry is actually kind of young and that uh the idea that you're getting big data onto a tiny screen and that that's the ultimate goal and developing well he didn't really talk about developing workflows to support that uh I am talking about developing workflows to support that um because I mean Don Norman talks a lot about how uh if you misuse something it's probably bad design and you shouldn't blame yourself and you shouldn't feel stupid it's probably the designer's fault um and as designers of data products or Engineers of data products like um we have to be really thoughtful about what we're offering the users because they're going to get results which can vary a lot in terms of sort of um you know uh proceeding from or having some kind of causal relationship to the data um having some kind of persuasive effect to the audience um because you know getting results is not hard often you can write a query and you'll probably get something uh if you write it wrong and you don't get anything that may be true too it may be that there's nothing to get but um but getting the right thing and then offering users a way to do that um at Clear story we talk a lot about sort of what should we give users buttons for we still have a very Tech like a lot of our operators are text-based and you have to write a kind of a custom DSL language you have to use the spark operators via our interface um and and I'm comfortable with that I personally I even think of that as its own user interface I mean um it isn't ux but uh it's how I do things uh one thing that I think a lot of products do but that I think Bears repeating is um from the availability of data types we only show you visualizations visualization types that you could plausibly use so this is just an example data set um and it's just to measure in a category so you can do a bar chart you can do a box plot you can do tree map you can do pie chart unless you have any like designers and or um people who hate pie charts which you probably will or a table visualization um but if you have location data then you could look at a map visualization you know and and giving the users things that are useful based on the data in this case it's the data types which is kind of a a good one to start with um really makes a difference in usability because you're not offering them visualizations that wouldn't be supported by the data you're not giving them a handle to pull on a door that only opens outward so we aren't the only ones doing this but I thought it was super clever uh so when you're thinking about granularity um like in the states down individuals up uh one metaphor that's kind of in the world that I like a lot is vertical drilling some kinds of data have um have granularity just in the world we understand that cities are contained inside counties are contained inside States and that and they're a subset like the superet entirely contains the the more granular data um and so we can zoom we can offer users the experience of zooming in and out and it makes sense to us um geoanalytics great example because like you know we've seen the pictures from spaceships that show the entire Earth and then we're coming down and then things get more granular uh timeline is also a good way um because uh second is entirely inside a minute is entirely inside an hour um is entirely inside a day year month week week gets funky so if any of you have to work with time data um you know not every month oh I see some nods not every month maps to four weeks you're got to get tricky with it or um remove it from your user affordances um sor sorry just use two parallel time is day to week to year day to month to year um day to week yes absolutely yeah and and how do you offer how do you inform a user about user has a selection which level to drill down or up to which which hierarchy to use exactly yeah and and we do that um we do that in inest and and you can use like moment libraries and you can have metadata that says like Easter season moves every year and this year it's April something to April something um which I'm going to totally talk about later in this presentation I'm going to talk about Windows I wasn't going to talk about metadata but now I'm going to um uh we going to like the week to-on granularity or or um Seasons that don't fall on the same date every year like uh if you're analyzing um we have some uh clients who are uh consumer goods clients and so they have seasons they we know when Christmas is going to be and we know when Halloween is going to be and we know what we sell for our Christmas promotions and our Halloween promotions and we can do these timeline visualizations and compare year-over-year and and see what's going on compare them to our marketing efforts see what's effective um but can't really do that with Easter because it moves like and so comparing like your well actually this is where you can do it with metadata where you can just like tell the computer like this year it's these days and this year it's these days and so look in between these days on this year um and that's how I've sort of done it in the past is you just do a join on a a calendar metadata um or you can use um operators like lag TS interval or what we are going to come to which is window operators where you define a a date range that sort of moves around um but first we're going to leave the vertical drilling metaphor does is everyone pretty kind of understand how like granularity and drilling make sense like because you're zooming in and out just like you'd go deeper um so I love the idea of curation and just because it's it it makes so much intuitive sense when you're asking a user to do something to sort of make an experience for another user in this case so like um if you have a a data scientist using a tool and they are going to tell the business analyst like their next steps in um consuming a data visualization um so unlike like a geom map where you can zoom in and out and you have an a predefined um granularity like a location based granularity or area based um curated drilling is where you could like give a user a navigation path like they can choose categories and they have to tell the system like what are connected semantically meaningful categories uh I sort of think of the example of um like in stores if you have jackets you have data about jacket sales um and and granularity would be like if you had subgroups totally contained in that like you have down jackets and you have Bolero jackets and you have uh trench coats um but curated would be if a user had to tell you like here's a dashboard and here I want to see jacket sales here I want to see pants here I want to see you shirts um they're just different types and a human can tell you sort of what people are going to care about and um tell the system so we have some visualizations where uh human being selects the dimension and then you can click through so things like tree map which show proportion um you you can have like a little stack of categories and then zoom in and out by category and manipulate that with filters um the data Engineers would uh know how to do that uh window operators which I mentioned before um uh this one is moving average uh have you all seen these in stock charts where you have like your 52 week high and your 52 we low and then um it really Smooths out like day or month or in this case year granularity um uh I don't actually remember what this picture was it might be like a fiveyear average and then you have yearly average bumping around um and the reason why this is relevant is because of the aggregation levels um it's like five years in this case and then you see how it it Smooths out and control for spikes and valleys um so sometimes that's just really helpful in your visualization um I just I did some script examples I think if you're really interested in implementation there are much better educational resources online um but I was I was just thinking about uh aggregating over a whole data set and and presenting that aggregate with the categorical breakdown so uh this is just some data from Health uh hospital compare. goov um it's all the quality measures for reporting so like aspiron arrival um I filtered this data I live in Oakland so I'm always interested in Alam County I'm like where do you want to avoid if you don't want to get a catheter Associated infection um uh and you can see from this chart uh the line is the average uh over all Alama County hospitals um and then the bars are um are the hospital reported scores do you want a low number or do you want a high you want a low number for this one you want a high number for other ones like as like percentage of aspirin on arrival for heart attack patients but catheter Associated infections lower the better so don't go to Eden Medical Center if you need a catheter um so when I signed up to do this talk I was like and I'm going to show people how to use D3 CSV to just like iterate through your arrays and like do all this just straight from your CSV but then even the the hospital compare. goov data that's like 65,000 rows which is not a lot um at a conference like this but still is non-trivial when you're just kind of iterating through so um my actual advice there would be uh filter it or um do it in SQL uh these are just some examples of um the SQL over operator um how many of you do SQL programming we we just heard okay cool um so you might even know more about this um but uh just in my investigation over how to get an aggregate row level next to an individual value so that we can get a chart like this um uh this had some sort of good examples so um if you just don't specify a a partition um and you use over it will aggregate over the enti ire data set so for this example select someon cost over uh everything um but uh you can use other um operators that are aggregation operators like rank um if you just want rank uh if I wanted to create a chart like my little um Hospital compare. goov chart um I would use average over everything which means you just don't specify a partition so you don't specify a window so like when I was talking about window functions another kind of window is category it's like the level of aggregation you can you can have um average salary uh over Partition by Department like if you want to sort of look at people's salaries relative to their Department salaries and you want to aggregate over their Department and that way your engineers don't screw up the values that was a joke CU Engineers are expensive these days um which is good news for us it's great news for us um so so that was a little discussion of the window operators um the SQL window operators and some aggregation techniques um I would love to hear a little bit about what uh brings you guys here today um whether it's data visualization specifically like uh how to lie with Statistics how to how to get big um there's a book about that is that um how to get large data sets onto a small visualization how to best communicate um one of yeah one of the interesting things that you brought up I really like is the choices of choosing visualization types depending on the kind of data you have and you're just limiting those choices to only what would be possible for that and you do that that's great because I uh use Tableau and Tableau kind of gray out exactly but they're still in the interface right yeah so I'm sure in your like the whole data product or even when you're designing a dashboard there are so many such choices you would have to make where you could give a lot of options configuration options for a dashbo even when you're making a dashboard like you have you could add 15 Sliders in there which is great but it also adds a lot of um like it makes it more overwhelming for the users yeah yeah um it's funny I think that some of our designers were very disciplined about um uh choosing sort of where the presentation happens and where the more um power user options are available but even so like even in our sort of exploratory area where you do have configuration options where you have all measures from all data sets you know you're ingesting any you're doing your data modeling um like here I'll show you uh I think it's still very minimal as far as interfaces go uh so like this is just example data sets um with the many viz types um but you need um you know you can't have a scatter plot if you only have one measure so I thought that was very clever too yeah I wish I could take credit for it um I have a a predic are quite brilliant um I was also sort of congratulating our product because earlier we heard about the um affordances for power users and how you um don't want to be too busy but you want to uh and I was thinking specifically of like the long running query example where you want to say like who hold the phone turbo like that's going to take some resources but you still want to allow users to do that if they if they're like no I really want the results of that query um because this product um is really uh for large data sets that's the the business use case and our server Engineers can handle it and so uh having a UI that coaches a user without bothering a user um is really really helpful um we it's funny we recently implemented or we're like in the next couple Sprints we're going to be putting in a library called clippy so uh because I sort of Love clippy and I want clippy for data analysis like it looks like you're trying to analyze data um one thing that our users have a ton of trouble with is this idea that all measures have to be aggregated we have um so uh you know here in our standard interface we have all these buttons that like Max you know Min standard deviation like um standard deviation I would sort of argue with including it because it is less helpful at um categorical um levels because you're not if you look at this and you see that you have it's it's just standard deviation by year it's not standard deviation over the whole data set and there are other metrics that are kind of useless if you are including categories um simply because you need as much like the the less data the less helpful it is um so on top of that it's not robust uh it's not robust to outli so if you're showing exactly exactly really giving very little information and that's why it is um less valuable at the best misleading at the worst um you need as much uh similar with things like P value you just you want it over the whole data set you don't want to slice it down um because it it won't be helpful um but aggregating in order to put things on a chart uh is necessary and sort of coaching people that they do need some aggregation operator if we're going to put it on an axis for you um so when you're trying to accommodate power users who want to know the P value and you're trying to accommodate people who just want to see a Time series of like sales by year um and they don't want to have to ask it for it uh it's like the classic ux problem like how do you accommodate like how do you make it simple enough for beginners and powerful enough for experts so um effort sorry colleting the interface and commen effort right exactly um so we have uh you know operator reference that's searchable I really enjoyed the the kind of um like using the idea that the um the predictive the power SQL interface used where you sort of crowdsource like what data do other people think is interesting because if you have you know 100 columns in the data um what uh how do you know what is Meaningful and what's just noise uh what's sparse data because we're a big data analytics company we get a lot of um sort of automatically detected data which has a lot of meaningless columns like just lots and lots uh so crowdsourcing that idea of like and then um bubbling up you know almost like a stack Overflow model or a Google search results model even so columns also a lot of times you would have the same table replicated in 15 different by because uh in reality we want a single source of truth but well it doesn't happen like that so you have 14 other tables which is not used ever but you can't tell by the name of the table so the US should and trying to figure out what is the most current data set when you might have a user who has I mean like uh for us um you know we have data connectors and then there's like an ingest process but a user may have ingested old data more recently than the more current data um just depending on who it is and what they have available so detecting like a data set versioning um and then making that Dynamic when some users want a lot of control like some users they're like I have this this is our our storyboard interface which is kind of this top level um like organizational idea where you can really uh see your data stories um and it's it it's a dashboard tool you've probably seen similar um similar interfaces uh and then um uh how Dynamic should those be because a a business analyst it was like the the uh it might be true I can't tell my boss um and some people want this Old Excel metaphor they want to be like look I have this chart and it shows that my department is doing great and it has uh seven um categories on the x-axis and I like the way it looks and I want to put um you know and I I want to drag and drop images on it I want chart junk uh and they don't want it automatically updated they don't want like March sales to make the line Go a different direction they want this kind of like presentational control I get frustrated because um because of the the principles of big data include velocity and you just have to accommodate Dynamic data um but I have to be sensitive to people who have um sort of presentational interests because they're presting this to an audience and they're making a recommendation or they're making a point and they can get very insecure um and then sometimes um like lower quality data or they might question the data and then their work um they can feel a real loss of control and that uh the tool failed them in some kind of significant foundational way it puts them in Jeopardy um so definitely like whether your tool is a presentational tool where you want users to have control or whether it is um a reporting tool and where you really want it just Dynamic from the data source or if you have like an iot or any kind of um like sensor monitoring data you you don't want any control over the presentation like you don't want to be like suppressing or um I really enjoyed the talk about thresholds like setting thresholds is another thing that can like change over time you know like you have your dashboard and uh red was meant something different last year like now your numbers have gone up um and what was green last year might be red this year like if you're lucky if you're crushing it that the thresholds change if yellow are you at the bottom of the yellow or top exactly right right um and it's funny like when we uh oh excuse me when we look at um the opportunities that you give analysts to control the pallet um you know even something like I don't think this is overwhelming but uh some people already think that this is way too sort of airplane cockpit um for an exploratory analytic Tool uh and other people think like boy you don't really give people a lot of opportunities to control and present their data I'm trying to get round my head around a couple things one is it seems very generic with the story so I'm I'm a ux person I think about these cases are users and context so it's hard to you know sort of generically think about these kinds of things so I'm trying to understand that part of it more and that's sometimes a real challenge when you're building a generic tool when you're not building for a specific data use case is that sort of what you're think or are you thinking about the storyboard story metaphor well what I mean is that you're that you're able to that the tool this tool is relatively easily adjusts or adapts to the users or the needs cuz it's probably going to be if there's a class of users that's using it they're probably to have things like cyber security but times absolutely so how much does machine figure out or this tool figure out what's actually helpful or not or that's time um we actually don't do a lot of sort of that that crowd sourcing data recommendation um if if that's what you're talking about or are you talking about um one thing that we've put a lot of thought into is permissions and like data set permissions row level permissions um what we call storyboard permissions which are consumable at that level but not um you can't modify the system to know kind of what to help the user know what to see kind yeah well one of the strengths of our product is really in uh and I think that this is considered the market differentiator is the backend intelligence that does data blending from multiple sources so um comb finding data sets and then useful visualizations without the SQL modeling tool where you don't have to like where we detect joinable attributes for example um and then join them automatically to create a data set underneath um for selection and interaction UMES am I answering your question I don't know if I understood the question I'm basically just saying from the presentation layer purely that there may be some intelligence to figure out what presentational layer shows in a particular context particular users you know you can probably make some guesses about optimizing or at least starting off with a particular set of pres you know parameter s presentation based on where it's being used or the sort of nature of the user um that's actually a great idea like a sort of rules-based what is this user probably interested in yeah um from uh from this analysis it'll be a a bar chart every time I mean I love bar charts so one one aspect to consider there is if you are deciding to make a change uh so let's say going back to the beginning of the talk you're talking about different presentations at different Agate levels oh yeah like if I'm looking at the DAT at monthly agregate level and the system decides best visualization for that is part AB down to daily level the system decides best visualization for that is a light chart now to user that's not necessarily expect Behavior yes and as we know users are okay with some surprises in the data users are not okay generally with surprises inor yeah I wouldn't recommend changing visualization type at Drill level for that exact reason um the some drill levels yeah and the way we handle drilling is uh um filters and then redrawing the visualization uh at the next uh at the incremental lower level of granularity um unless it's a curated uh sort of I mean more about the sort of level levels of resolution or things like that right and I I think of granularity and resolution as as the same yeah yeah um oh so you're thinking of different business users as being interested at different levels yeah so this is a I had a conversation with a buddy of mine about like what's your favorite level of aggregation and I thought and my first thought was like I like over the whole data set I like one value from everything like oh like and I thought that that was kind of like the the super user the God view like oh the data and then aggregated it to one point and then I thought oh wait a minute like that totally messes you up because then you don't know about outliers you don't know you know like all kinds of things that you'll miss if you don't have granularity um if you don't have that resolution and I was like no I take it back like no aggregation like that's the best because then you know everything about everything like except then you kind of can't really as a human come to any conclusions so uh so it's probably somewhere in the middle depending on your data set we've tried doing aort of VI thing right A Time series of BS which I personally love but a lot of business users have difficulty understanding it so it's almost like every time we have to use that chart for something I need to sit down with business users and have a conversation about what it means right and you need to make sure you have enough data to have a real distribution you need to make sure that it isn't already aggregated by years somewhere because then you'll have a box plot that looks exactly like nothing so sorry to interrupt here actually one nice way to do it times is those spark line charts now spark lines yeah often times they like these tiny line charts indicating Trends but you could also do it for the whole distribution right like a histogram like a tiny histogram under that one number that everyone wants to see exactly yeah tiny histogram at least tells you look there is some out sorry I missed your talk because sometimes I See Spark lines so abused they're they're wonderful in the right context but they're sometimes they be separated out and they don't have much there's not much to see so you have bu of spark lines but you can't make any meaningful Compu visually but I I loved using and this can be true of aggregation too is like use it as sort of a click-through tool you know and then you can have your um more informative or your more helpful visualization on Mouse over like you know like coaching the users after you've gotten their attention and I got the T symbol so thank you everyone and thank you data by the bay been such a a pleasure and a privilege so thank [Applause] you what