data.bythebay.io: Michael Terry - Data From Click To Settlement: A Full-Funnel UX Approach
Thanks a lot, Michael. How's everybody doing? Good. All right. Oh, let's start at the beginning. Okay. Uh, so as Mike mentioned, I'm the technical co-founder of Lofty. We connect people searching the web with lawyers in their local region. Uh, usually I only get about a minute to give the pitch
So, people often walk away thinking we're some kind of SEO company or some kind of ad agency or who knows what. But, uh, fortunately, Alexi, you know, and and company, they gave me all of 20 minutes. So I hope to get into another level of detail to really kind of describe some of the technical challenges that we have and how they are uh sort of interwoven with the user experience with the the people problems. And so I'll kind of be bouncing back and forth between non-technical and technical uh and describing a lot of these challenges and how we've overcome them. Uh so yep. Um so so the problem that that we addressed um when we got started about 3 years ago was was this was happening already right I don't need to you know preach to the choir in the echo chamber uh about how the digital disruption has already happened you know using data and mobile you know data systems and mobile technology to you know acquire customers in mass uh create massively scalable systems you know the drill um however um when I partnered with my co-founder uh Todd, a litigator, you know, I started talking about a lot of these things and and it was it was really really foreign to him. He, you know, saw the need for a lot of this to come to the the legal industry. Uh and, you know, here we are 3 years later
Um and we've just introduced the, you know, the law track here at Data by the Bay. So, I think that's a it's a pretty big uh milestone for us. Um the uh I thought that uh you know I sort of credit IBM for kind of summarizing all this recently and uh decided to cite them here but it it really states succinctly really where where we kind of stand today and and it's not that it's coming it's it's actually already here. Uh but so so but what about the legal industry right? um uh it's highly you know diversified into different uh niches of of practice areas and and uh each state regulates what you can and cannot do independently. And so um this presents a challenge uh both culturally within the within the law and uh with um regulations and and you know when we think about digital systems and scaling um well you know 50 states is it's not a it's not a hard problem but um when you deal with both people problems and uh the the data and the integration problems uh it you know can it can create quite a challenge and um I think we've made plenty of progress since uh we got started. Uh we we're primarily focused on client acquisition and scaling businesses and things but um there's a lot of a lot of work done by you know folks in this room and and folks at uh Stanford Codeex which we uh we have friends over there uh that that have made significant progress in in legal but um you know the work work still cut out. Um, so, so my talk, uh, here I'm going to be, uh, kind of trying to generalize a little bit because I know not everybody here is in law. So, um, I kind of sort of pose the question of, of of what about local businesses, right? So, like law is a a certain intermediate scale for for a lot of law firms
Um, you know, small, medium, large. It's not exactly things that you can throw big data at. Um, and I think some of the the parallels exist for, you know, the the hundreds of thousands of of and millions of, you know, um, local businesses that are starting to really um, bring in analytics and use data at at this kind of a scale, right? So, um, you know, it's good that, you know, sort of the term big data has fallen a little bit out of favor um, because it doesn't quite capture a lot of uh, what these types of businesses are looking at. Um but you will see that Google's now starting to sort of introduce this this sort of consumer analytics uh popular times type thing. Uh and and it's unclear to me whether businesses like this um have yet to start using this but but the opportunity is there. They didn't actually have to do anything to get some of these analytics which they might be able to apply to say their staffing problems or um you know operations and that sort of thing. Uh this was sort of a little bit of a throwback to my to my previous startup where uh my uh co-founder Cory and I would go uh to bars and restaurants in Manhattan and we'd install essentially a data acquisition system streaming video. Um and but they had very very basic needs of you know when do we staff our bar and when do we do this do that and weren't quite ready for it
Um, fast forward here, I think the, you know, the the the barrier to entry has certainly dropped for, uh, small teams, smaller budgets, um, less sophistication, you know, not necessarily hiring a whole bunch of data scientists to to really be able to optimize your your operation. Uh, okay. So, three parts of my talk, the click, the inquiry, and the intake case. So, um, we built a digital system that focuses on, uh, really tracking and, uh, crafting a user experience from the clicks through litigation all the way into settlement. Uh, there's really nobody in our industry that that I've I've run into that um, is interested in really um, sort of creating that and tracking that from start to finish. There are a lot of people that have made u you know progress on different phases of the user experience but at Lofty we kind of take this uh holistic approach to from start to finish um and uh I guess mostly I'll be talking about some of the dynamics around the clicks and uh what we do to uh scale up uh client acquisition at the start of the of the experience. Um the benefit there is that we start to see uh cross-domain optimizations. So uh things that are happening offline and data that we're collecting from offline activity uh to really optimize for our upstream activity and uh I think it's pretty interesting
There's a sort of another slant and that's um uh you know uh with the regulations and everything in in law there there are very clear guidelines of what you can do. Uh what you can put on a landing page for example is is is is pretty strictly um sort of regulated. uh what you can say on the phone, how you represent yourself, everything. So if you actually are the same uh database, same house looking at you know from the start all the way to the finish that gives you opportunity to do right by the uh by the client. These clients are coming to us in time of need and we have a you know we sort of have we have an obligation to them to uh represent the information very accurately and and very you know truthfully and u when you do what the industry normally does and send it out to you know an ad agency to do your or SEO company or a you know a digital media agency they there's there's a certain amount of the the ethical obligation that's that's lost in translation. So we hope to bring that um you know similar to what Eric was talking about that kind of user empathy uh at the at the personal level from from start to finish um certainly is is an ideal and it's hard to achieve but um okay so let's dive dive into the data uh quick little introduction um a lot of folks actually don't really know the difference between paid search and and organic stuff and so we get into conversations about that I'll just suffice to say that you know it's ads or organic placement And um some of the terminology that that we're focused in on optimizing um when you do a Google search like for example this person you know I need a lawyer really really bad um you know you get like four ads up top and uh many of you are probably familiar that Google has now really upped that on the desktop side of things to where you know this is my whole screen right I'm I'm getting all ads before I even start to scroll from a user um experience and analytics thing this is pretty big deal um and getting getting exposure to this this many ads. Um and it's and it's a high stakes game because these uh these keywords are as as most people know like legal keywords are extremely expensive and so um we'll get into kind of the the challenges using data scientist data science that we've overcome there. Uh this is what it looks like on mobile
Usually you do a mobile search and you get all ads um you know above the fold here. All right. So uh first thing first challenges we overcame looking at just the clicks. How do we get uh client acquisition at scale? Um most of the features in paid search kind of follow this bode curve here like uh so what you're looking at here on the on the x is how much we invest into ads and on the y you're seeing in the red cloud how many clicks we get. And this you know by machine learning is is pretty easy to model right it's it's not linear but it starts to tail off here. So we use uh you know um used some linear regression there to to to look at things in mass and um that did pretty well but it didn't actually do um as well as as we would have liked. So then you sort of take it one level further um and do some additional modeling. Now it's important to note that I talked about the cost here earlier of getting these clicks on this scatter plot um at at a good value and volume
And so I've defined here uh a high value region where you can actually bid pretty low for most keywords and get some level of activity. Um but often times you're not getting the volume that you're interested in. So uh there's a high value region, high volume region. So this is a challenge in both natural language processing to come up with your set of keywords to uh create these these models. Uh but it's a mostly a bid optimization problem from our perspective. um the fun stuff of targeting these ads into geographic regions by device, by time of day. Um so I think that um you know the big part is uh getting that efficiency and driving those clicks is actually a emphasis on a few of these issues. Optimizing landing pages, developing negative keyword sets, bunch of additional um benefits that you can do there
But this actually isn't that interesting, right? Like you get a efficiency boost. This is kind of what everybody's doing. The really interesting part is when we start to look at modeling of what happens next, which is the call. People are calling us uh different times of day with different needs. And so then things start to get a little bit interesting. We start to model the call activity against the the clicks, right? So not all clicks are equal. And uh what we've been able to observe is some pretty interesting trends with uh converting clicks into actual uh legal cases and people signing up for uh for the services. So um lo and behold, there's actually uh what you're looking at here is a 7 days uh time span and uh on the y- axis you're looking at like at the conversion from click to actual inquiries
We say calls, but there's there's there's a few other ways that people can inquire online through chats, through form submissions, that sort of thing. But we actually see like a stark contrast between certain times a day and others where, you know, we get four times the likelihood of turning a click into a case uh and these times and uh very low likelihood down here. Well, uh, and then interestingly enough, uh, you see this downward trend where people start really winding down for the day and then so I think we sort of coined the term the after dinner spike that we see uh, later in the day and it actually is pronounced most prominently on Friday. There's a huge after-dinner spike where we get a whole bunch of phone calls. And so we put that back into the ad optimization to get to get our benefits. So um as you know not all clicks are equal and we're also looking at uh geographic optimizations. Um but the key point here is that we have to link the activity. So somebody calls us we have to link that back to the original click
Uh so we developed a a system to actually attribute one for one these clicks to calls by delivering unique phone numbers. Thanks to you know the API integrations with Twilio phone numbers um spinning up new phone numbers and configuring them is is something that you can do programmatically. So we actually serve out each session a unique phone number um and rotate through some expiration but uh realistically uh we were able to do that uh a few years ago. Um but then um Google actually introduced that as a feature. You can actually put it as an HTML tag and the and they'll put on your landing pages a unique phone number. But but we had it first so that was cool. Um, okay. I have a a quick demo of um the landing page experience
Uh, we built out a system that um allows you to uh replay visitors uh the other full screen. So, you're going to be looking at a replay of my replay or of our team's replay system, but I'm I'm essentially Let me see if I can configure this. Okay, so we we put it on higher speed, but it's essentially uh we built out this landing page and this system that allows you to see the the actual user experience and see how they're engaging with the page. Um, and the goal here is to um really build that that that true empathy. Actually watching the person engage with your content uh allows you to to find different ways of optimization um beyond what you can do with like say an optimized lead or uh some other landing page type stuff. Um so yeah, so this person went down, they kind of read some of our of our Spanish landing page content. We knew what they searched for, which was uh Buffetta Avagados. Uh and this actually turned into a case that one of our law firm partners in New York um acquired this as a client, this person as a client
We can go back and use this scale of analytics to try to repeat this, make this repeatable and a little bit more precise. All right. So, the point with this slide here is is um right right now in the industry um there's still still a lot of account managers on on on staff right now for for people buying legal keywords and stuff. And um the models that we've been building, you know, we've been looking at 10 distinct bid levels across seven days a week, targeting these different days of week. And that creates a tremendous number of combinations for the number of bids that we can put out there. And that really just can't be done by individuals. So we use machine learning. Um we actually uh used uh um and means clustering to do the different day segments and find like times of day and and cluster them into three different segments
um and then uh targeting those with enough that that created enough sample size for us to target specific bid optimizations for those segments. Uh we move beyond the linear regression into uh random forest model thanks to uh some some interns and and the staff at the University of San Francisco. They have a tremendous data analytics program and a masters that they're just growing year over year. So, I'd highly recommend you guys checking checking them out and some some of the students that they're producing are pretty top caliber. I know a lot of folks here are are hiring for data scientists like we are. Um, but yeah, so this some of the work of some of our students to uh to really find the best optimization of those ads. Um, the first iteration kind of looked like this. So, uh, these day segments um are actually not contiguous
So, Sunday 0, Sunday 1, Sunday 2 are not in sequence. they're actually binned um portions of the day that are like each other u with respect to clicks, impressions, and cost. Um and so we ran it uh you know in our cloud platform um we sent it up to sent it up to you know the um the AWS system that we have and came up with a range of bids for each of these days. Um during the search uh we took the median and put it into production. Uh we ran it again uh with a couple more iterations of the search and we actually narrowed these bands and found a a pretty highly optimal uh bid schedule to to run throughout the day throughout the week. We ran this last week uh and it actually turned out really well. Um so we're pretty excited to continue down that path of of finding these bid optimizations. Uh the future of what we're doing now is to then go even deeper into the funnel and now we're getting uh information and data and analytics from the cases that we generated two years ago
Um hope to use that to optimize upstream and really look at the the entire funnel. Um things like litigation steps and outcomes and settlements really um that's the true offline stuff that's happening way later. Um correlating that upstream. Um and in summary I guess uh you know big thing that we did I think is is by putting the lawyers the data scientists developers marketers everybody in the same room with not not a ton of them right so you have a big organizations there's a lot of information that doesn't get passed through you know we put a you know a manageable set of folks together to to solve this problem of creating a repeatable system for uh law law firms to acquire new cases uh and we're communicating the language as much as possible. It's really hard from across those domains to even uh understand among our team what we're all we're all saying to each other. But um that's really uh kind of big big picture. We kind of think the biggest uh take away from from what we've done. And then our our case generation system because it goes from click all the way through to settlement
Uh we've been able to span uh the offline online domain and create optimizations across the user experience across what's typically known as a marketing funnel or a salesunnel. Um and then just some takeaways. Um digital disruption is al also happening um not just with the big companies that we all read about but you know the barrier to entry is actually much lower for small law firms and small local businesses and lots of other organizations, government different organizations are able to take advantage of that and that has implications for um for for a lot of different uh areas. Um for modelers definitely find problems that have systematically been neglected. Um we talked about the echo chamber. Everybody's many many times people get stuck working on the same problems and and really trying to get that um those uh incremental improvements but realistically there's a lot of problems that haven't even been touched. Um and these problems have large uh industries behind them. U for lawyers focus on partnering with real disruptors people who can are looking at data not just the popular jingles on the on the radio
um and you know people with money versus people who actually have expertise and the ability to uh innovate what you're doing uh for for the next uh the next wave. Um small mediumsiz businesses uh many of them tell us that they don't know what to do. They like data but they don't know what to do with it. So just store it now uh store as much of it as possible and try to get talking to people like us about what they can do. Uh and then finally for P PBC folks, pay-per-click folks like uh that are industry uh focus on that high value high value region so that you don't have to actually bid up these crazy expensive keywords like $500 for these legal keywords that you can actually um get benefits without going breaking the bank. So that that's that's it. Thanks. Thank you, Mike
Uh so our uh next talk is going to be at uh uh uh 10:40. So please stick around and uh ask Mike uh questions. Uh and uh Mike, thank you very much. Thanks. Uh I have to say that the idea of binning together nonadjacent time intervals that are like each other. Thank you. You just solved the problem of trying to figure out what so That thing alone was Okay. Well, thanks
Um, yeah, follow us on Twitter. Uh, we'd like to keep the conversation going offline. Um, you know, come up, talk to us. Uh, we're we're local. So, yeah. Thanks. Hey, thank you so much. Appreciate that.