April 13, 2023
Should Law Firms Play ‘Moneyball’?
Laura Leopard of Leopard Solutions recently suggested law firms should play Moneyball. In a post titled, “Law Firm Hiring, Go Where the Money (Value) Is,” Ms. Leopard suggests firms should focus on hiring entry-level attorneys and laterals based on their value. Much like the Oakland A’s did for many years when hiring talent in free agency, as detailed in the famous book “Moneyball,” by Michael Lewis. The perpetually “poor” (low revenue) Oakland A’s looked for high quality, undervalued players who could generate the same wins as overvalued players playing for “rich” teams like the Boston Red Sox or New York Yankees.
Ms. Leopard discusses entry-level and lateral attorneys’ failure to make a return on investment (ROI) to benefit their firms. She notes that many firms invest as much as $350,000 on the development of entry-level attorneys, which is nearly equal to the cost of recruitment of laterals. “They may not have the same recruitment costs as laterals, but they come with onboarding and training costs that can quickly add up. To achieve the return on the firm’s investment, that entry hire (just as that lateral associate) must be at the firm for three years,” states Ms. Leopard.
One of the challenges noted in the post is the focus many large firms have on recruiting from the Top Ten Law Schools. The competition for these graduates bids up their starting compensation, making the upfront expense even higher. In addition, these entry hires are highly sought after by subsequent firms, which means they are unlikely to stick around long enough to make back the expense.
“Interestingly, our data show that hires from the Top Ten schools underperform in ROI, year over year. A single percentage point can mean millions of dollars lost in unsuccessful recruitment. Note that the Top Ten School group has the lowest success rate of any of the law school tier groups year after year. What drives the low ROI of this group? One word: competition. The group of Top Ten associates is aggressively recruited year after year and that lowers their opportunity to achieve their ROI for their initial employers,” writes Ms. Leopard.
What’s the solution? One option noted by Ms. Leopard is to expand recruiting to the next tier of law schools where successfully meeting ROI targets are consistently high. “Their success rates were more evenly distributed... The success of these hires could be based on many factors, which individual firms hired them, their locations across the country, and more. But it is undeniable that the competition for this group was slightly lower than for the associates from the Top Ten group,” notes Ms. Leopard. In other words, look for “value” in the associates in the Top Twenty Schools who can make your firm just as good, but at a lower cost and possibly higher loyalty.
Another suggestion is to dig through the firm’s data on associates to understand their success and failure. “Firms should know what hiring patterns breed success and which ones they should reconsider. It takes data for that. Firms should have good data on past hiring, both laterals and entry hires, and know why that hire didn’t work out. Did they lose them due to the competitiveness of the market or other reasons? Firms should also try to benchmark their success scores with their competitors to understand if they are truly underperforming… If law firm leaders realized that flawed hiring impacted their wallets, they might get some results! Moneyball can be won, but only if you follow the data and make decisive changes where needed,” concluded Ms. Leopard.
Authors Note: If you have not read Moneyball and you are a law firm Managing Partner, COO or CFO (or want to become one) you should. The inside baseball details aside, the book dissects how a small group of leaders turned a “poor” organization into a competitive franchise by using data and financial savvy to properly allocate their limited resources. The Oakland Athletics had a payroll half that of many of its competitors but managed to consistently field a competitive team and make it to the divisional playoff round several times using data to identify “value” players (not unlike what Warren Buffet does with companies, and just as Ms. Leopard suggests law firms do with associates in her post).
Generative AI and Law
The consensus seems to be that ChatGPT, or generative AI, will not replace lawyers, but will be used to reduce workloads on legal tasks, such as first drafts of contracts or briefs. Is this true? In an article titled, “AI or Not To AI: Observations from Legalweek NY 2023,” for Law.com, Steven Salkin explores the issue.
First of all, generative AI has to be trained on the “right” data. If it’s not, it will produce wrong answers, and who is liable for those wrong answers? “There’s also a liability issue, as Level Legal’s Director of Client Solutions, Daniel Bonner, pointed out. Suppose you draft a brief based on generative AI output and submit it. Whether it was fact- and case-checked or not, it contains false information. ‘Who is liable for a ChatGPT-generated memo?’ Bonner asked.”
To train generative AI you need to build a Large Language Model (LLM) for the AI to learn. “Building an LLM just for legal... I immediately thought of two giants in the industry that just so happen to have an extensive database of decisions, pleadings and motions, ready-made for AI to comb: LexisNexis and Thomson Reuters’ Westlaw... LexisNexis Legal & Professional EVP and CTO, Jeff Riehl, not[es] that Lexis already uses a specialty version of Google’s BERT that does the job 'just as well — if not better — than GPT in some cases.' The applications for legal professionals, Riehl said, are just what one would think: improved, more interactive searching; cutting hours of low-value tasks from legal staff and associates; and generating early drafts of legal briefs and contracts (subject to review, of course)," notes Salkin.
Interestingly, Salkin suggests another use for generative AI, asking it to write legal briefs for the opposition. This would give perhaps an additional perspective and be done much faster. “For example, say you’re representing a client accused of copyright infringement. You can ask ChatGPT (or whatever AI platform you prefer) to come up with the main arguments for why your client did not infringe on said copyright. You can make sure you have thought of all the issues that show in the results. Maybe you missed an issue or two. Then, you can ask AI to generate a list of reasons why your client did infringe. You can then use the two lists to form the story of your case. Yes, you go through that exercise anyway, but both of those steps can be completed in minutes with AI, whereas it would take a paralegal or first-year associate hours and hours to compile,” suggests Salkin.
So, will generative AI replace all those associates, paralegals, and other legal support staff? Not likely according to Salkin: “Just like many other new technologies, AI will not replace people. It will make what they do more efficient and save time with the everyday mundane repetitive tasks that we complain about having to spend hours doing and leave that time to do your real job — and that applies to lawyers as much as support staff… If there are jobs displaced by AI technology, there will be others created. Data scientists, for example, will be needed.”
Generative AI is still at the beginning of its development, not the end. There is much to learn about the technology and much for the technology to learn. “A long time ago, one of my IT friends advised me not to adopt the first version of any software. Let it get out there, let it breathe and let the bugs get discovered and fixed first. Software needs to evolve. As new features are added, new bugs are found... So is AI the best thing since sliced bread? Maybe. Just not now,” concludes Salkin.
Below is a sample of more expert comments on generative AI from the Legalweek event:
“Generative AI is a very promising technology with significant potential in the legal industry,” said Ariyah Mandel, VP of Discovery Services, Gulfstream Legal Group. “But as with any new technology, I think it is important to approach its use with caution and to carefully consider the potential benefits and risks before implementing it in legal workflows.”
“The danger of immediate deployment is a lack of understanding,” said Rick Kaminski, Senior Director of eDiscovery Operations at Veristar. “Deploy it too quickly at your own peril; it needs exhaustive testing.”
“You have to be skeptical,” Seeber said. “Would you be comfortable with AI advising clients? If life, liberty, treasure are at stake, would you trust machine over people?”
Performance Appraisal
Measuring employees’ performance is an essential tool to evaluate success and encourage improvement. One tool employers use is an appraisal form, which they may refer to during an employee evaluation or performance review. However, many small firms have informal appraisal processes, or do not engage in performance appraisals at all. Yet formal performance appraisals are necessary for employee development and provide managers with a useful tool for determining bonuses, pay raises, promotions, etc. If you have not instituted a formal performance appraisal, read on to learn what you should include in your employee appraisal forms and how to implement them at your firm.
A well-made performance appraisal form helps both the manager and the employee understand what the employee is doing successfully and how they can improve. The best appraisal forms include relevant appraisal factors that break down the evaluation by different skills and qualities. Appraisal forms also help to guide managers through an employee’s performance review by highlighting specific items to discuss. Finally, the appraisal form allows the employee and manager to work together to create goals for the employee.
You should customize the form to fit your firm’s goals and the metrics you want to track for your employees. Here’s what to include when creating an appraisal form:
A great performance appraisal form should help you thoroughly evaluate your employees. It’s a useful tool to keep track of their efforts and achievements. It can also help you compensate your best performers accordingly and incentivize others to perform better in the future.
Laura Leopard of Leopard Solutions recently suggested law firms should play Moneyball. In a post titled, “Law Firm Hiring, Go Where the Money (Value) Is,” Ms. Leopard suggests firms should focus on hiring entry-level attorneys and laterals based on their value. Much like the Oakland A’s did for many years when hiring talent in free agency, as detailed in the famous book “Moneyball,” by Michael Lewis. The perpetually “poor” (low revenue) Oakland A’s looked for high quality, undervalued players who could generate the same wins as overvalued players playing for “rich” teams like the Boston Red Sox or New York Yankees.
Ms. Leopard discusses entry-level and lateral attorneys’ failure to make a return on investment (ROI) to benefit their firms. She notes that many firms invest as much as $350,000 on the development of entry-level attorneys, which is nearly equal to the cost of recruitment of laterals. “They may not have the same recruitment costs as laterals, but they come with onboarding and training costs that can quickly add up. To achieve the return on the firm’s investment, that entry hire (just as that lateral associate) must be at the firm for three years,” states Ms. Leopard.
One of the challenges noted in the post is the focus many large firms have on recruiting from the Top Ten Law Schools. The competition for these graduates bids up their starting compensation, making the upfront expense even higher. In addition, these entry hires are highly sought after by subsequent firms, which means they are unlikely to stick around long enough to make back the expense.
“Interestingly, our data show that hires from the Top Ten schools underperform in ROI, year over year. A single percentage point can mean millions of dollars lost in unsuccessful recruitment. Note that the Top Ten School group has the lowest success rate of any of the law school tier groups year after year. What drives the low ROI of this group? One word: competition. The group of Top Ten associates is aggressively recruited year after year and that lowers their opportunity to achieve their ROI for their initial employers,” writes Ms. Leopard.
What’s the solution? One option noted by Ms. Leopard is to expand recruiting to the next tier of law schools where successfully meeting ROI targets are consistently high. “Their success rates were more evenly distributed... The success of these hires could be based on many factors, which individual firms hired them, their locations across the country, and more. But it is undeniable that the competition for this group was slightly lower than for the associates from the Top Ten group,” notes Ms. Leopard. In other words, look for “value” in the associates in the Top Twenty Schools who can make your firm just as good, but at a lower cost and possibly higher loyalty.
Another suggestion is to dig through the firm’s data on associates to understand their success and failure. “Firms should know what hiring patterns breed success and which ones they should reconsider. It takes data for that. Firms should have good data on past hiring, both laterals and entry hires, and know why that hire didn’t work out. Did they lose them due to the competitiveness of the market or other reasons? Firms should also try to benchmark their success scores with their competitors to understand if they are truly underperforming… If law firm leaders realized that flawed hiring impacted their wallets, they might get some results! Moneyball can be won, but only if you follow the data and make decisive changes where needed,” concluded Ms. Leopard.
Authors Note: If you have not read Moneyball and you are a law firm Managing Partner, COO or CFO (or want to become one) you should. The inside baseball details aside, the book dissects how a small group of leaders turned a “poor” organization into a competitive franchise by using data and financial savvy to properly allocate their limited resources. The Oakland Athletics had a payroll half that of many of its competitors but managed to consistently field a competitive team and make it to the divisional playoff round several times using data to identify “value” players (not unlike what Warren Buffet does with companies, and just as Ms. Leopard suggests law firms do with associates in her post).
Generative AI and Law
The consensus seems to be that ChatGPT, or generative AI, will not replace lawyers, but will be used to reduce workloads on legal tasks, such as first drafts of contracts or briefs. Is this true? In an article titled, “AI or Not To AI: Observations from Legalweek NY 2023,” for Law.com, Steven Salkin explores the issue.
First of all, generative AI has to be trained on the “right” data. If it’s not, it will produce wrong answers, and who is liable for those wrong answers? “There’s also a liability issue, as Level Legal’s Director of Client Solutions, Daniel Bonner, pointed out. Suppose you draft a brief based on generative AI output and submit it. Whether it was fact- and case-checked or not, it contains false information. ‘Who is liable for a ChatGPT-generated memo?’ Bonner asked.”
To train generative AI you need to build a Large Language Model (LLM) for the AI to learn. “Building an LLM just for legal... I immediately thought of two giants in the industry that just so happen to have an extensive database of decisions, pleadings and motions, ready-made for AI to comb: LexisNexis and Thomson Reuters’ Westlaw... LexisNexis Legal & Professional EVP and CTO, Jeff Riehl, not[es] that Lexis already uses a specialty version of Google’s BERT that does the job 'just as well — if not better — than GPT in some cases.' The applications for legal professionals, Riehl said, are just what one would think: improved, more interactive searching; cutting hours of low-value tasks from legal staff and associates; and generating early drafts of legal briefs and contracts (subject to review, of course)," notes Salkin.
Interestingly, Salkin suggests another use for generative AI, asking it to write legal briefs for the opposition. This would give perhaps an additional perspective and be done much faster. “For example, say you’re representing a client accused of copyright infringement. You can ask ChatGPT (or whatever AI platform you prefer) to come up with the main arguments for why your client did not infringe on said copyright. You can make sure you have thought of all the issues that show in the results. Maybe you missed an issue or two. Then, you can ask AI to generate a list of reasons why your client did infringe. You can then use the two lists to form the story of your case. Yes, you go through that exercise anyway, but both of those steps can be completed in minutes with AI, whereas it would take a paralegal or first-year associate hours and hours to compile,” suggests Salkin.
So, will generative AI replace all those associates, paralegals, and other legal support staff? Not likely according to Salkin: “Just like many other new technologies, AI will not replace people. It will make what they do more efficient and save time with the everyday mundane repetitive tasks that we complain about having to spend hours doing and leave that time to do your real job — and that applies to lawyers as much as support staff… If there are jobs displaced by AI technology, there will be others created. Data scientists, for example, will be needed.”
Generative AI is still at the beginning of its development, not the end. There is much to learn about the technology and much for the technology to learn. “A long time ago, one of my IT friends advised me not to adopt the first version of any software. Let it get out there, let it breathe and let the bugs get discovered and fixed first. Software needs to evolve. As new features are added, new bugs are found... So is AI the best thing since sliced bread? Maybe. Just not now,” concludes Salkin.
Below is a sample of more expert comments on generative AI from the Legalweek event:
“Generative AI is a very promising technology with significant potential in the legal industry,” said Ariyah Mandel, VP of Discovery Services, Gulfstream Legal Group. “But as with any new technology, I think it is important to approach its use with caution and to carefully consider the potential benefits and risks before implementing it in legal workflows.”
“The danger of immediate deployment is a lack of understanding,” said Rick Kaminski, Senior Director of eDiscovery Operations at Veristar. “Deploy it too quickly at your own peril; it needs exhaustive testing.”
“You have to be skeptical,” Seeber said. “Would you be comfortable with AI advising clients? If life, liberty, treasure are at stake, would you trust machine over people?”
Performance Appraisal
Measuring employees’ performance is an essential tool to evaluate success and encourage improvement. One tool employers use is an appraisal form, which they may refer to during an employee evaluation or performance review. However, many small firms have informal appraisal processes, or do not engage in performance appraisals at all. Yet formal performance appraisals are necessary for employee development and provide managers with a useful tool for determining bonuses, pay raises, promotions, etc. If you have not instituted a formal performance appraisal, read on to learn what you should include in your employee appraisal forms and how to implement them at your firm.
A well-made performance appraisal form helps both the manager and the employee understand what the employee is doing successfully and how they can improve. The best appraisal forms include relevant appraisal factors that break down the evaluation by different skills and qualities. Appraisal forms also help to guide managers through an employee’s performance review by highlighting specific items to discuss. Finally, the appraisal form allows the employee and manager to work together to create goals for the employee.
You should customize the form to fit your firm’s goals and the metrics you want to track for your employees. Here’s what to include when creating an appraisal form:
- Develop a rating rubric - Think of the key objectives of an employee’s role and separate them into different categories. Then, rate how well they’re meeting each objective. Common rating systems include a 5-point rating scale (e.g., Excellent, Very Good, Good, Fair, or Poor) or a 3-point expectation scale (Exceeds, Meets, or Fails). Detail what each rating means to help managers in different departments rate employees consistently.
- The employee’s achievements - Most managers fill out a new appraisal form for each performance review. Always be positive first by highlighting the employee’s greatest achievements during the review period. Include what projects the employee successfully completed or what tasks they did well consistently.
- The employee’s areas for improvement - Discuss what areas the employee could improve on. Also discuss how their work affects others or the firm. Example, an employee might not communicate well with the team, or they might show up late frequently, hurting the team’s productivity.
- Improvements since the last review - If the employee has already had a performance review, note the things they have or haven’t improved since the last meeting. Detail what actions they took to meet their previous goals. By reflecting on what worked for them, they can use this insight for their new goals.
- The employee’s professional goals - Help the employee create new professional goals for their next review period. The goals should be specific, measurable, attainable, and relevant, so the employee is more likely to accomplish the goals in a set timeframe.
- Comments from the employee - The employee comments section lets employees explain themselves in writing. Here they can include why their work isn’t meeting company standards, or they could express what they enjoy about their position. They can also discuss what technology or training they need to better accomplish their tasks.
A great performance appraisal form should help you thoroughly evaluate your employees. It’s a useful tool to keep track of their efforts and achievements. It can also help you compensate your best performers accordingly and incentivize others to perform better in the future.