Ignoring Data Bias is Detrimental to Your Business Strategy


Originally recorded on the Pricing Matters Podcast by Digitory Legal. Digitory Legal is now BigHand Impact Analytics. Learn more here.

Aurelia Spivey, Marketing Consultant with BigHand, chats with Adam Minielly, formally the pricing manager at Osler, Hoskin, and Harcourt. Adam is currently the National Manager, Pricing and Practice Management at McMillan LLP. Adam plays a key role in overseeing the firm’s pricing activities and operations, driving firm strategy related to the development, analysis, and implementation of creative, workable solutions in response to client pricing demands.

Prior to Osler, Adam worked in roles in competitive intelligence and pricing at Dentons LLP, Blakes, Cassels and Graydon, and Norton Rose Fulbright.

Adam holds an undergraduate degree in political science and a Masters Degree in public policy. His graduate work examined how think tanks influence public policy. He has applied his social science training to his work in pricing to develop data-driven decision making. He recently presented Framing, Anchoring, and Data Biases - What Influences Pricing Decisions? at the Buying Legal Conference.

Top three takeaways from this episode

  • All data has a story to tell. Just because data is dirty doesn’t mean you can’t do anything with it, you need to dissect the data to understand what it’s telling you.

  • Tackling data bias. Set up systems and processes to account for common data biases including sample, exclusion and observer bias.

  • Effective Scoping. Focus on the actual hours to completion of phases and tasks rather than anchoring on a fee amount.

I think firms are moving towards benchmarking and that's something that we're able to do when we get called into a pricing meeting for an RFP or a pitch, that we're able to in a few clicks set up a variety of search criteria and be able to come to a meeting prepared with something for the partner to consider.

Podcast Transcript:

Adam Minielly 
But the danger of not taking a chance to remove the data bias or reduce it can have some detrimental effects and strategies if decisions are being made. But you really haven't dissected the data and looked at what the story is actually telling.

Aurelia Spivey 
Welcome to Pricing Matters, a podcast by Digitory Legal. Digitory is a data analytics and cost management platform and service, bringing data-driven pricing and cost prediction to law. My name is Aurelia Spivey, and I will be your host as we speak to leaders who are making an impact in legal pricing, discuss market trends, and find out from them why pricing matters. Morning and welcome to the Pricing Matters Podcast. This morning our guest is Adam Minielly. He is the pricing manager at Osler, Hoskin, and Harcourt. Good morning, Adam. Welcome.

Adam Minielly 
Good morning. How are you?

Aurelia Spivey 
I'm good, thanks. Thanks for joining us.  

Adam Minielly 
It's my pleasure. 

Aurelia Spivey 
Excellent. So it's always really helpful for our listeners to hear a little about your background. So I'd love for you to tell us a little bit about your pricing journey.

Adam Minielly 
Yes. It's always interesting when those cocktail conversations or party conversations, where you get asked what you do and you mentioned your pricing manager at a law firm. So they kind of assume that there's an MBA or CA or a law degree in there. But I actually have a master's in political science and started in law on a happy accident. I was interviewing for other roles within Toronto, where Osler's headquarters are located, and there happened to be a role in Competitive Intelligence and Research at a firm, and landed there. I spent about four years at that firm, working in competitive intelligence, and going through a global merger. That was in '09, sort of post-recession, during the recession and that's really when you started to see the demand for pricing within the legal industry. That pressure from clients to come up with, at the time, I believe it was ABAs (alternative billing arrangements), and now we're at the AFA (alternative fee arrangements) age. Then from there, I transitioned into a pricing-specific role at a large national firm in Canada, kind of building from the ground up. They saw the demand in the market for pricing, but they were well aware that they need some help with their data to get there. So I sort of helped set the groundwork for building the foundation, from cleaning up the data and setting up the systems. Working with the enterprise team there to help build platforms and analytics reporting, to help us get down to the science of the data in there and help provide some greater certainty and pricing and launching the use of EMS coding, at that firm as well on the litigation side. Canada was a little bit slower to the uptake on the mandatory UTMs tracking, but now it's in Canada, as is the US, it's kind of universal and pretty much expected on all matters to a certain degree in some form. Then I've kind of moved through different roles with other global firms, working in pricing on different sides, working in the profitability piece, the leverage, and I've been at Osler a little over a year now, leading the pricing team here.

Aurelia Spivey 
So a very deep background across a number of firms. So I'm going to dig a little bit later into the giant data science background. But I first connected with you at the Buying Legal Counsel Conference in America in June 2019, and Katherine and I really enjoyed your presentation on Framing Anchoring, and Data Biases - What Influences Pricing Decisions. So, I'd love it if you could give us your key messages from that session for our listeners to hear.

Adam Minielly 
Yeah, absolutely. I enjoyed your session as well. Yeah, so for sure. So just to give some background, I guess, in my graduate work, I looked at how framing anchoring is used by think tanks to influence policy decisions and those theoretical concepts. As I've sort of progressed through my career and legal, I saw how data is used in law firms, and I mean, I guess, across all industries, but also understanding the biases that come in with that data. How we collect it, how we look at it, the answers we wanted to give us, and how we can create processes and use a variety of data science techniques or analysis to sort of help reduce that bias and actually get us to actionable insights. Because a lot of organizations are across all industries when looking at information and data. But the danger of not taking a chance to remove the data bias or reduce it can have some detrimental effects and strategies if decisions are being made. But you really haven't dissected the data and looked at what the story is actually telling.

Aurelia Spivey 
So I'd love to dig in a little bit more, if you can, in terms of what side of a sort of biases you see most frequently in your pricing role at law firms.

Adam Minielly 
Right, yeah. So I think the ones that we see the most are kind of the sample bias, the exclusion bias, the measurement bias, and the observer bias. In terms of an exclusion on the law firm side, when we start cleaning data and getting rid of that concept of dirty data. Are we excluding things for good reasons or are we just deciding that that's something that shouldn't be included? And from the observer bias, it's just, you set up an experiment or hypothesis, and there's an answer that you'd like to get to. But the tendency is to expect to see what we want to see, but understanding that the objective, when running an analysis of data, is to get the actual truth of what's behind the data. And from the measurement bias standpoint, it's how we set up the models, sort of a systematic value distortion. When we look at an issue or device used to measure, that could be the scale that's used, by hours and rates, but that could be something else that should be brought in from a law firm analysis standpoint. And then, again coming back, exclusion sample bias, has a bit of an overlap, but make sure you're bringing in a strong sample into your analysis program. Whether that's the samples big enough, too small, has unnecessary pieces that aren't really going to give you the right story on one specific, that is relative to the scope of the data project.

Aurelia Spivey 
So I would love to know how some conversations have gone in law firms with all of this bias because I can, in my head, imagine you're pricing out a particular type of matter and everyone thinks it's going in one direction and one of these biases coming in. How does the data go down in law firms, when you present it and sort of start tackling those biases?

Adam Minielly 
Well, I think that it has been certainly a learning experience when I've tried to start with that precedent gathering. To understand the lawyer may give you some precedents, and the parliament gave you some precedents, but those might be the sample bias right there. They're wanting to give you their big Rainmaker-type matters, and they're big important matters, but they may not have the samples relevant to this particular matter. So it's kind of weeding those out and looking at how those actually attribute to how they parallel with each matter. So the conversations just need to be sort of direct and really get down to what is actually being asked in terms of the pricing and making sure that we are pulling the right data, but also that we have the right, our teams, outside of the face to face of the lawyers, have our own systems in place to help eliminate that and account for that and set up a few different, be able to prevent a few different scenarios to the partners. I think that's one thing that I found has been effective is saying, okay, you know, we look at this particular way of looking at it, but if we tweak this variable in that variable, you know, we can get a little bit closer to what's actually being asked for one particular pricing instance.

Aurelia Spivey 
Continuing on the data discussion, I'd like to know from you, how you think that your particular data science background influences your approach to pricing and practice management at the firms that you've been working at.

Adam Minielly 
Well, yeah, I think and I just want to kind of preference that data science, I don't consider myself in the ranks of those who have PhDs in math and computer science, but in terms of my sort of general experience with data science. So as I mentioned, in my graduate work I looked, at my sample size within that was 20 years of newspapers and analyzing context, and really getting down to what and how think tanks influence public policy across multi variables. So I think taking that experience, and applying, whether it's correlations or using statistics programs, like SPSS or other tools that are out there in the market, it's really coming back down to, as I've mentioned already several times, is getting down to what the data is actually really saying. But then understanding what the data is trying to say, even if we get to a certain number but you're really dissecting that. Conversations that I have in terms of building teams. One of my points, one of my closing slides, if you remember at the Buying Legal Counsel Conference, was that firms will hire more data scientists or social scientists. And that I might be a little biased, but I think it's important to have that sort of side and the way that we social scientists look at things and from a quantitative and a qualitative piece, creating, we're trained to create a hypothesis and look at theories. Having that social scientist background but then on our team, we have those who have MBAs and CAs, different types of backgrounds, and bringing that kind of holistic approach to how we look at the data and analyze it. So, for example, one of my analysts has an MBA, and he may look at that one pace, or I may look at it one way, and bringing those two different pieces together, helps us really sort of provide a good picture of what is happening with our data and how we price our matters.

Aurelia Spivey 
That's a great point, I think the collaboration piece is just growing bigger and bigger at law firms, and being able to have different minds looking at it and sharing that with the firm and moving things forward, I think is very helpful. That's what's taking us, it's going to be very helpful in the next decade. 

Adam Minielly 
Yes, exactly. 

Aurelia Spivey 
So how are you leveraging your data to increase precision around your pricing? With Osler and the other firms that you've worked with?

Adam Minielly 
Yeah, so I think it's getting that buy-in that the data is going to be the piece that drives our decisions. We hear about big data-driven decision-making and all those types of statements and I think that's really where we are. It isn't just isolated to whether it's retail or marketing or other industries, law firms have a lot of data and at times, we can get bogged down in the fact that, oh, it's just dirty data, and we can't do anything with it. So the one piece that I tried to emphasize it just because it's dirty, doesn't mean we can't do something with it, it still has a story to tell. And as I go through and clean data, I always look to see what's at various stages of quote-unquote cleaning, and what story each of that data told. But then it's also one area that I'm moving more towards is leveraging the use of logarithms and I don't think that, by no means am I, I don't think that logarithms are the answer and end all. Because even talking earlier about biases is also biases can happen when you're setting up your logarithm, but helping set up those automated processes in the background, kind of training the models and checking again, for biases and that type of thing. That has helped reduce the time the team spends going through Excel sheets and sort of pivot tables. If you can have that logarithm running, because one of the ones that I'm using now is one called the nearest neighbor, which essentially clusters various groupings of types of matters, and just setting up the right models to sort of move those clusters around, and see how close our benchmarks based on hours and fees are to the matters that we're actually running, whether it's at the phase and task level, then saying, Okay, let's bring in lawyer one and lawyer two and see how close those lawyers operate in isolation or with other people, or different clients or industries. So it's really kind of just bringing in a very methodical approach, but also understanding that there's going to be iterations to this type of analysis. It isn't going to be an overnight solving of any problems or any challenges for the better term. But just leveraging the technology and the processes that are out there.

Aurelia Spivey
I think, what would be interesting is, we've talked a little bit about this, but if we can expand a little, maybe with some examples on how you're now leveraging this data when you are out there scoping and responding to RFPs, or other requests from clients in terms of fee arrangements.

Adam Minielly 
Right, so I think firms are moving towards benchmarking and that's something that we're able to do when we get called into a pricing meeting for an RFP or a pitch, that we're able to in a few clicks set up a variety of search criteria and be able to come to a meeting prepared with something for the partner to consider. Whether it is outside of just a specific one matter, but what the actual aggregate of matters within that criteria or silo for that RFP and have, in what I try to move away, I try to focus less on the fee, and I'm more concerned about bringing what hours to completion by phase or matter to the partner. Because my concern, when I bring fees to the partner is that they get fixated or anchored on that number and that may not be what is appropriate. For example, if we said, you know, phase one, or the case assessment, we're gonna say, is $25,000, but that may not be the number but if we were to go, reverse engineer, it and say, okay, well, actually, based on the scope, the hours, it's 35. I've been in those experiences where the partner starts to get nervous, well, we're going to be pricing over if we thought it was 25,000, that's $10,000 over. So that's why I like to focus on looking at the hours by associate and partner and the leverage, and then getting to the number. Because of that, there have been comments about the death of the billable hour, but the lawyers still work. They're still putting in the time and we still have to get to that number. So I think, yeah, just focusing on the hours and presenting them with something that they can work with and say, okay, this task is it realistic to complete this task in eight hours? If not, how can we? What do we need to change plus or minus?

Aurelia Spivey 
That's fantastic. So I love that piece around, getting fixated on the number, but what you're actually talking about is how many hours? What does this unit cost? What does this task cost for us to complete? Rather than talking about the number at the front, I think that's a great approach. So I'd love to know, from you, what are the characteristics of the most successful pricing arrangements that you've been able to make with clients and work with your firm to achieve?

Adam Minielly 
Right? Yeah, because I think it's really, it's building that relationship of trust with the clients and that getting away from trying to sort of game each other on the pricing and saying, okay, how can we find, reduce each other's risk and develop a mutually beneficial pricing arrangement on this? Understanding what the client's needs are, and what they're actually trying to achieve in this RFP, whether it's litigation or corporate M&A, or whatever the matter might be. It's really understanding what they're hoping the end product to be and providing that to them, and where possible, providing them with mutually beneficial options. Where we can do a fixed fee if this or we can do a cap fee if that, and we'd be happy to discuss. I think one of the key pieces is that we are happy to revisit this after Q1 of 2020, are we actually in the position we are? Have we set up ourselves with the pricing arrangement that was actually, that we decided, to start at this matter? Is it still work? Because I think just signing the agreement and saying, okay, this is what it is, and we start to see that maybe this fixed fee really wasn't the right decision but having that because then there are times it can be resentment on both sides. But coming back and visiting and saying, okay, are we both in a good place on this, yes. The transparency, we're tracking in the right direction, we're in a good place, and both continue to agree, this is the right fee arrangement. So yeah, it just comes down to that communication and trust with each other.

Aurelia Spivey 
I think those are two wonderful overarching principles and I think it's very helpful to have that trust and relationship where you can come back. That's the collaboration that everyone's looking for in this market because everyone's looking for innovation and change but it comes with that level of trust and collaboration. So that's a great thing to think about. So the concept of value seems to be something we've been talking about on the show, and I'd love to know, from you, if you've got any views on how firms can better define and demonstrate value to their clients.

Adam Minielly 
Yeah, so I think, I mean I'm sure we're familiar with this, the Warren Buffett quote of prices, what you pay in value is what you get. So the client will come and want to pay the price, but they want to know that, that price is coming back to that mutually beneficial club relationship and that they're getting what they actually need out of it. So it's if your value can range from if it could be $100,000 matter, and the outcome was exactly what the client wants. Or you could pay $80,000 for the same matter but the outcome wasn't what the client wants. The client didn't get the value, they may have paid less, but they didn't actually get what they wanted. And you see that across a variety, outside of the law, whether it's going for a meal, or buying clothes or buying a TV or that piece. So it's really sort of getting, it's what you get it again, it sounds cliche, but you get what you pay for, in a lot of cases. So it's just creating that communication, providing what the client needs.

Aurelia Spivey 
I say this on the show a lot, switch gears said, I'd love to count the number of times I've said that. I'd like to talk a little bit about pricing teams and the pricing skill set. What makes teams successful? So, at the moment we've heard on the show, we have teams are varying in size, but the narrative average team is around two to three people, which is a lot of work for people in such an important role in firms. So, I'd love for you to tell us a little bit about your team at Osler and any advice that you have for teams prioritizing this increasing demand that we're seeing for their time across law firms.

Adam Minielly 
Yeah, so we have, so I have four people on my team. Two are focused on sort of the administration side, whether it's run through reconciliation or general client renewals, that I think everyone's going through right now, at least in Canada. Then we have a team of analysts as well that, helping with the day-to-day pricing support that I might need on various RFP, but also training them on my experience in the industry, and sort of what I've seen happen in the industry over the past 10 years in terms of pricing and bringing that together. But coming back to my earlier point of just having a diverse team with different skill sets, I think is really important because each person brings their own unique view, how they're trained, and how they like to look at various pieces related to pricing. We're a firm of about 450 lawyers across Canada and we were able to support that with a collaboration of our finance team and our billing team as well. So, I think in terms of size we're very happy with for working, and having that collaboration across departments, I think is also important.

Aurelia Spivey 
Fantastic. So, I'd love to hear a little bit about it, you've talked a little about the logarithms. But I'd love to hear more about any role that technology and process management play in making your team successful.

Adam Minielly 
Yeah, so one of our sort of, we're looking at process improvement is our go-to is how. If there's a process that needs to be improved, how can we do it with process people and technology? Now, those three tenants of, okay, what if we're looking at a challenge or happenings a bottleneck somewhere or there's something happening needs to be improved? What's the best tool from that people process and toolbox? And saying, okay, can we get to 100% fixing this, probably not, but if we can 80/ 20 improve that, at some point that's going to significantly help overall efficiency throughout the firm, you're going downstream. It's finding those technological fits. It could be just as simple as building the right Excel model, that's just, helps drive a little bit, easier to find various things, or plug and play, and to be able to just put in a part and rate and quickly calculate a blended rate for a partner, or if we did this, what's various scenario analysis. There's a lot of great technology out there to use. I think that's what's been so interesting or being in the legal industry during this time, is how much new technology, legal tech is starting to come out, and seeing how it's evolving. I don't think if you'd asked any lawyer 20 years ago, that legal tech would be a word that people will be talking about, and companies would be looking to be a part of it. But I think that's sort of is a testament to how agile and how flexible the legal industry has been to be able to compete with industries and be able to properly support our clients.

Aurelia Spivey 
I think that's true and we're definitely evolving, and we're excited to be doing what we do in that in that space as well. So here's a very open question. Adam, what is on your pricing wish list?

Adam Minielly 
It's a good question. I don't know if I answer have a wish list. I think it's just continuing cooperation with clients. I think there are still some growing pains when it comes to the legal procurement side of how RFPs are presented, that they may not necessarily directly translate into legal. So it's just finding that balance within that side of what is in the RFP, that clarity of what the pricing is actually being asked for. I think that's sometimes that our team spend is sort of what is, if we can go back, we can certainly go back to clients with questions, but also trying to sort of be self-serving and take the initiative to sort of figure out what the client's needs are if they don't quite know what they are. But I think it's just, not to repeat myself too much, but just that collaboration between clients and firms on the pricing side. I guess just that communication. 

Aurelia Spivey 
A question I like to end on, we're on the pricing matters podcast. So, Adam, why does pricing matter to you?

Adam Minielly 
I think it's, we are exposed to pricing decisions every day, for the most part. I'm interested in its pricing psychology of it. I mean, looking at how various things, various ways of presenting things, influence our buying decisions. Whether it's, what's referred to as a left digit effective, you could have something priced for $40, but you put it out at $39.99. It's only one penny from $40, but even that effect, has a significant influence on buying decisions. So I'm just curious, I'm always just curious, as a, I guess, that social scientist of understanding how people work and operate in the world. It affects our decisions for buying TVs to clothes, to houses, all that stuff, and to kind of be able to do that in my job, it's interesting because I get to get excited about that. Working that and every day. 

Aurelia Spivey 
This has been a fantastic conversation. Thank you so much for being on the show.

Adam Minielly 
No, my pleasure. I appreciate you reaching out and bringing me on.

Aurelia Spivey 
Thank you for listening to Pricing Matters, a podcast Digitory Legal. To find out more about our guests please visit our podcast page. If you have any feedback or guests that you think we should feature, please reach out to me. Thank you for listening, see you next time.

About BigHand Impact Analytics

BigHand Impact Analytics combines strategic advice and change management expertise with AI-enabled data analytics to transform legal billing data into DEI success. The solution allows firms to identify opportunities for career advancing work, supports DEI initiatives, and focuses on areas where time recording needs to be improved - ultimately creating a smoother billing and collection process, with better data insights. 

BigHand Impact Analytics