1、WHITE PAPERAI at Work:Navigating the Legal Landscape of Automated Decision-Making Tools in EmploymentFederal and state regulators are increasingly focusing their attention on artificial intelli-gence(“AI”)tools,including the use of automated decision-making tools in employment.This White Paper explo

2、res current uses of AI in the workplace,focusing on the use of automated decision-making tools by employers during the recruiting and hiring process;examines the legal and regulatory risks associated with increased use of AI in employ-ment;discusses employment policy considerations associated with e

3、mployee use of AI-powered chatbots;and offers tangible solutions for employers seeking to reduce litiga-tion risk and stay one step ahead while remaining in compliance with existing laws and emerging legislation.July 20231Jones Day White PaperThe growing use of AI to make employment decisions has dr

4、awn the attention of lawmakers and regulators,who are con-cerned about privacy,the possibility of algorithmic bias,and the impacts of automation.On October 28,2021,the Equal Employment Opportunity Commission(“EEOC”)announced the launch of a new initiative to“ensure that AI and other emerging tools u

5、sed in hiring and other employment deci-sions comply with federal civil rights laws that the agency enforces.”1 In May 2022,the DOJ Civil Rights Division and the EEOC each issued a technical assistance document regarding AI and the potential for disability discrimination in the employ-ment context.2

6、 In April of this year,officials from the EEOC and other agencies that enforce employment,fair lending,and fair housing laws issued a joint statement pledging“to use their enforcement authorities to ensure AI does not become a high-tech pathway to discrimination.”On May 18 of this year,the EEOC rele

7、ased a technical assistance document that explains the EEOCs views about the application of Title VII of the Civil Rights Act(“Title VII”)to an employers use of automated systems,includ-ing those that incorporate AI.3 Several proposals for com-prehensive AI-related legislation,including the Algorith

8、mic Accountability Act,have been proposed in Congress.4 And some lawmakers have called for the creation of an expert federal agency focused on regulating the development and use of AI.5Federal officials are not alone in voicing concerns regarding AI.On July 5,2023,New York City began enforcing a law

9、 that governs employers use of AI to make hiring and promotion decisions.In California,a new agencythe California Privacy Protection Agencyis preparing to write new rules to address the uses and abuses caused by automated decision-making technology.Other states and localities are considering similar

10、 legislation and regulations.The New York City law,as well as proposed state laws,require employers to disclose how they are using AI and identify any disparate impacts on race,gen-der,and other protected categories.By drawing attention to the use of AI tools,these required disclosures could spur li

11、tiga-tion asserting discrimination under Title VII,the Americans with Disabilities Act(“ADA”),the Age Discrimination in Employment Act(“ADEA”),and other employment laws.ARTIFICIAL INTELLIGENCE IN THE WORKPLACE Now more than ever,employers are relying upon AI.It is used in nearly every stage of the e

12、mployment process,including recruiting,hiring,training,retention,promotion,compensation,and firing.In December 2021,EEOC Chair Charlotte Burrows reported that“83%of employers”and“90%of Fortune 500 companies”rely on AI during hiring.6 Within hiring and recruit-ing,employers use AI tools to target job

13、 postings to specific groups,screen applicants to move forward in the hiring pro-cess,administer automated interviews,and analyze candidate responses.Two common types of AI are predictive algorithms(which can use labeled datasets to train algorithms to classify data or predict outcomes)and natural l

14、anguage processing(which helps machines process and understand human language).Both technologies may be used in a single AI tool.For exam-ple,an AI tool that screens applicant resumes may use natural language processing to scan the resume for key words and use predictive algorithms to select candida

15、tes for interviews.Such a tool might be“trained”using resumes from current employees who are high performers so that the tool,without human intervention,can decide what factors predict an appli-cants success at the company.The AI tools output is a short-list of prescreened resumes that,in theory,ref

16、lects candidates who have similar attributes to successful employees.In short,the tool is making decisions that would previously have been made by humans.In addition to screening resumes,employers are using AI tools to evaluate candidates through video interviews.Live or recorded video interviews ca

17、n be run through software utiliz-ing a combination of machine learning,computer vision,and natural language processing to evaluate candidates based on their facial expressions and speech patterns,and then provide a score or assessment of the applicants attributes or fitness for a job.Other applicati

18、ons evaluate applicants personalities,aptitudes,cognitive skills,or“cultural fit.”The efficiencies obtained by application of AI to human resources functions can be profound.By one measure,85%of HR professionals reported that AI tools save them time and/or increase their efficiency.7 Nearly 50%said

19、that such tools 2Jones Day White Paperimprove their ability to identify top candidates.8 By stream-lining repetitive tasks like screening resumes,recruiters have more time to provide a personalized experience to candidates and increase their competitive edge.LEGAL RISKS OF USING ARTIFICIAL INTELLIGE

20、NCE IN HIRING AND RECRUITMENT AI vendors often promise that their products will reduce or eliminate unconscious bias in recruiting and hiring decisions.However,critics express concern that AI tools perpetuate and can even exacerbate biases that are embedded in the training data.An often-cited exampl

21、e is Amazons attempt to build an AI recruitment tool,which was abandoned in 2018 when engi-neers found that the algorithm discriminated against female candidates.9 The companys AI-driven model reportedly down-graded resumes containing the word“womens”and filtered out resumes with terms related to wo

22、men,including candi-dates who had attended women-only colleges.This reportedly occurred because the tool was trained primarily on resumes submitted to the company over the past 10 years,the majority of which were from male candidates.More recently,Workdays AI-powered screening tools are being challe

23、nged in a class action lawsuit filed in a California federal court in February 2023.The plaintiff alleges that these AI tools disqualify Black,over-forty,and disabled applicants.10 The plaintiff alleges he has been rejected from 80100 posi-tions that purportedly use Workday as a screening tool for a

24、pplicants.Workdays AI-dependent tools,he argues,“allow its customers to use discriminatory and subjective judgements in reviewing and evaluating employees for hire”and“caused disparate impact and disparate treatment”against African-Americans,individuals with disability,and individuals over the age o

25、f 40 in violation of Title VII,the ADA,and ADEA.This section considers legal risks tied to AI adoption,including potential claims under discrimination laws,privacy laws,and newly adopted state and local legislation.Disparate Treatment and Disparate Impact Claims While AI technology is relatively new

26、,the use of selection pro-cedures or tools in making employment decisions is not.Well before the AI-era,employers used a variety of selection tools and procedures,including written aptitude tests,strength tests,and personality assessments.Those types of selection procedures have been repeatedly chal

27、lenged in court under Title VII and other antidiscrimination laws.As early as 1978,the EEOC adopted the Uniform Guidelines on Employee Selection Procedures(the“Guidelines”),which provide guidance on how to assess bias in selection procedures.More recently,the EEOC has stated that these Guidelines ap

28、ply squarely to AI tools that are used to make employment decisions.11 Selection tools are typically challenged through a dispa-rate impact theory of discrimination.Unlike disparate treat-ment claims,disparate impact claims do not require proof of intentional discrimination.Rather,they require proof

29、 that a facially neutral employment policy or practice caused a dis-parate impact on a protected group without relevant justifica-tion.Employers who use biased AI tools could have liability under this theory without knowing or intending that the tool disadvantage a protected group.To demonstrate how

30、 this might occur,we first explain how courts analyze disparate impact claims,and then compare how this analysis typically applies outside the context of AI to how it might apply to an AI-powered selection tool.Disparate impact claims arising under Title VII generally pro-ceed in three parts.12 Firs

31、t,the plaintiff must identify a facially neutral employment practice or policy that caused a dispa-rate impact on the basis of race,color,religion,sex,national origin,age,disability,or other protected category.13 Second,the employer can defend against a showing of disparate impact by demonstrating t

32、hat the practice or policy is both“job-related and consistent with a business necessity.”14 And third,the plaintiff can rebut the employers“job-relatedness”defense by establishing that the employer failed to adopt a less discriminatory practice that would have equally met the employers legitimate ne

33、ed.15Statistical analysis plays an important role in litigating Title VII disparate impact claims.One statistical approach is to com-pare the selection rates of a particular protected group(e.g.,White,Black,Latino,Asian,Native Hawaiian or Pacific Islander,Native American or Alaska Native)to the sele

34、ction rate of another protected group.Because differences in selection rates can be caused by chance,it is important to measure whether the difference is significant enough to rule out chance.In the Guidelines,the EEOC uses a“four-fifths rule”as a rule of thumb for screening out matters it is less l

35、ikely to pursue.This 3Jones Day White Papermetric measures whether the selection rate for one protected group is less than 80%of the selection rate for the protected group with the highest selection rate.16 Courts have expressed skepticism toward the four-fifths rule,however,noting that it is inhere

36、ntly unreliable,especially when analyzing small sample sizes.17 Even the EEOC has noted that“the four-fifths rule may be inappropriate under certain circumstances.”18 Another approach,articulated by the U.S.Supreme Court in Hazelwood School District v.United States,19 utilizes standard deviations.Th

37、ere,the Supreme Court noted that a disparity of more than two or three standard deviations“would undercut the hypothesis that decisions were being made randomly with respect to the protected group.”Given the specialized nature of these inquiries,it is important that statistical analyses be supported

38、 by relevant and reliable expert analyses.A proper statistical analysis must first identify the pool from which to assess adverse impactin other words,the denomi-nator of the selection rate formula.The pool should be aligned to the challenged employment decision.For example,if a plaintiff alleges th

39、at a written exam administered to appli-cants had a disparate impact on women,the pool of similarly situated applicants consists of all applicants who took that examination during the relevant time frame.If the pool is too broadfor example,if it includes applicants who took a dif-ferent test or were

40、 screened out based on some other criteria at a different stage of the hiring process,the analysis will not be meaningful.Determining the proper pool becomes difficult with certain AI tools.Take,for example,an AI tool that administers an AI-based pre-employment examination that changes based on dyna

41、mic data sets.These algorithms utilize machine learn-ing to“learn”or“improve”over time.Thus,in our example,the AI-based examination an employer deploys in Week 1 may be different from the examination administered in Week 4,as the tool“learns”that certain questions are less likely to produce a desire

42、d result.It may be difficult to produce a meaning-ful statistical analysis without knowing how the AI tool works,e.g.,how frequently the algorithm changes or whether the algorithm is different for different applicant pools.For similar reasons,in cases involving AI tools,it may be difficult to estab-

43、lish the causation prongi.e.,that the employment practice“caused”a disparate impact.Certain AI tools could present challenges for employers to for-mulate and advance their“job-relatedness”defense.As noted above,an employer can defend against a disparate impact claim by showing that the selection cri

44、teria used by the AI tool is“job-related for the position in question and consistent with business necessity,”20 but the employer may not have complete visibility into the selection criteria used by the AI tool.Employers may be able to strengthen a job-relatedness defense with proof that the algorit

45、hm is programmed to utilize job-related criteria and by demonstrating how the algorithm applies that criteria.Impermissible Reliance on Regulated Data SourcesEmployers reliance on AI tools also implicates state privacy laws.Many AI tools derive their efficacy and efficiency,in part,by relying on ext

46、remely large data sets.Generally speaking,the larger the data set,the more accurate an AI tools predic-tions and/or recommendations will be.However,not all data is fair game for employers to use in connection with AI tools.Some datasuch as criminal history,salary history,and bio-metric dataare subje

47、ct to regulation in certain jurisdictions when used in the employment context.Further,employers who rely on certain data about candidates as part of the hir-ing process must comply with federal and state background checks laws.AI Tools and Protected Data Sets.While antidiscrimination laws forbid emp

48、loyers from hiring employees based upon their race,sex,age,and other categories,employers also must be cautious when considering other types of protected data,including criminal history,salary history,and/or biometric data in the hiring process.Various jurisdictions restrict employ-ers from using su

49、ch data outright,while others require that employers abide by specific disclosure and other require-ments before doing so.Criminal and Salary History.Federal law does not ban employ-ers from considering an applicants criminal history,although EEOC guidance asserts that excluding all applicants with

50、an arrest record will run afoul of Title VII if doing so results in discrimination based on race or another protected charac-teristic.21 Many state and local laws,however,explicitly restrict employers from considering applicants or current employees criminal history,either completely or unless certa

51、in conditions are met.For example,laws in California and New York limit the types of criminal records that may be considered and prohibit 4Jones Day White Paperinquiry into an applicants criminal history until after a condi-tional offer has been made.Similarly,employers should be cautious when consi

52、dering an applicants salary history in the hiring process.The EEOC and some federal courts have determined that an applicants prior salary cannot,by itself,justify a compensation dispari-ty.22 Additionally,in an effort to close the wage gap,numerous states and localities prohibit employers from seek

53、ing out or relying upon salary history information in determining whether to hire an applicant or at what salary.23 Employers in these jurisdictions should ensure that any AI products they are rely-ing upon are not considering an applicants criminal or sal-ary history.Biometric Data.A few statesIlli

54、nois,Texas,Washington,and Marylandregulate employers use of biometric infor-mation.The Illinois Biometric Information Privacy Act(“BIPA”)has garnered the most attention as it is the only state law that provides a private right of action,and offers actual dam-ages,statutory damages,attorneys fees,and

55、 injunctive relief,which has brought an onslaught of litigation and sizeable settlements.24 Under BIPA,prior to obtaining an applicants or employees biometric identifiers or information,an employer must:(i)notify the individual of the specific reason for col-lecting the information and how long the

56、employer will use or retain it;(ii)receive a written release from the individual to use the information;and(iii)develop a publicly available written policy including a retention policy and guidelines for perma-nently destroying the information.Some AI tools may scan an individuals facial geometry or

57、 pro-duce a voiceprint of an individuals voice,which are“biomet-ric identifiers”under BIPA.Further,as discussed below,the Illinois Artificial Intelligence Video Interview Act places certain obligations and restrictions on employers who use AI to ana-lyze applicant-submitted videos.Accordingly,an emp

58、loyer in Illinois who utilizes an AI tool that analyzes video interviews and/or captures the unique biological characteristics of an employee or applicant must be cautious to comply with Illinois law or risk severe penalties.AI Tools and Background Checks.Background check compa-nies that scan social

59、 media platforms and produce reports of compiled information to employers are considered consumer reporting agencies(“CRA”)and thus are governed by the Fair Credit Reporting Act(“FCRA”)and analogous state laws.25 When employers obtain a consumer report through a CRA to make employment decisions,they

60、 likewise must comply with the FCRA.A“consumer report”is broadly defined as any writ-ten,oral,or other communication of any information by a CRA bearing on an individuals creditworthiness,credit standing,credit capacity,character,general reputation,personal char-acteristics,or mode of living that is

61、 used or expected to be used or collected in whole or in part for the purpose of serving as a factor in establishing the individuals eligibility for employ-ment purposes.26 The FCRA typically requires that employers:(i)first obtain consent from an applicant to conduct a background check through a th

62、ird party using a form with statutorily mandated disclosures;(ii)notify the consumer when adverse action is taken on the basis of that background check;and(iii)identify the CRA that provided the background check.27 Passed and Emerging Legislation Regulating AI inEmploymentFederal agencies have issue

63、d regulatory guidance explaining how employers should comply with existing federal civil rights laws in connection with their use of AI.28 States and localities have gone even further and passed new laws that impose new requirementssuch as notice and disclosure obligationson the use of AI in employm

64、ent decisions.Federal Efforts to Regulate AI.In recent years,federal reg-ulators have increasingly focused their attention on AI.On January 1,2021,Congress passed the National Artificial Intelligence Initiative Act of 2020(“NAIIA”),which established,among other things,several new federal offices to

65、oversee and implement a national AI strategy.Pursuant to the NAIIA,President Biden formed the National Artificial Intelligence Advisory Committee,led by the Secretary of Commerce,within the White House Office of Science and Technology Policy(“OSTP”).In October 2022,OSTP published“Blueprint for an AI

66、 Bill of Rights:Making Automated Systems Work for the American People”with the stated purpose of protecting the public from harmful outcomes or harmful use of technologies that imple-ment AI.29 The Blueprint is a nonbinding white paper laying out 5Jones Day White Paperfive principles to guide the de

67、sign,development,and deploy-ment of AI and other automated technologies,including practi-cal guidance for developers.30 On October 28,2021,the EEOC announced the launch of a new initiative on AI and algorithmic fairness to“ensure that AIand other emerging tools used in hiring and other employ-ment d

68、ecisions comply with federal civil rights laws that the agency enforces.”31 On May 18,2023,the EEOC published a technical assistance document that states the EEOCs views about the applica-tion of Title VII to an employers use of automated systems,including those that incorporate artificial intellige

69、nce.32 In part,the EEOCs technical assistance document asserts that if an employers“use of an algorithmic decision-making tool has an adverse impact on individuals of a particular race,color,religion,sex,or national origin,or on individuals with a particu-lar combination of such characteristics(e.g.

70、,a combination of race and sex,such as for applicants who are Asian women),then use of the tool will violate Title VII unless the employer can show that such use is job related and consistent with business necessity pursuant to Title VII.”33 The EEOCs docu-ment does not address“other stages of the T

71、itle VII disparate impact analysis,such as whether a tool is a valid measure of important job-related traits or characteristics.The document also does not address Title VIIs prohibitions against intentional discrimination(called“disparate treatment”)or the protections against discrimination afforded

72、 by other federal employment discrimination statutes.”34Other federal agencies have also taken action about the use of AI.For example,in May 2022,the DOJ Civil Rights Division and EEOC issued technical assistance documents regarding AI and the potential for disability discrimination in the employ-me

73、nt context.35 One month later,in June 2022,the Civil Rights Division announced the settlement of its Fair Housing Act law-suit that alleged unlawful algorithmic discrimination in adver-tising.36 Then,on January 9,2023,the Civil Rights Division and the U.S.Department of Housing and Urban Development

74、also filed a statement of interest in support of allegations of unlawful discrimination by an algorithm-based tenant screen-ing system.37 Federal Trade Commission(“FTC”)guidance issued in April 2021 explained how the agency would enforce transpar-ency and fairness in algorithmic decision-making by b

75、ringing enforcement actions under section 5 of the FTC Act,the Fair Credit Reporting Act,and the Equal Credit Opportunity Act.Federal agencies have been involved in challenges to employ-ers use of AI-powered employment tools.For example,in November 2019,the Electronic Privacy Information Center(“EPI

76、C”)filed a complaint with the FTC against HireVue,a startup that initially used AI-driven facial recognition software to assess a candidates effectiveness.38 EPIC alleged that HireVues AI toolswhich the company claimed could mea-sure the“cognitive ability,”“psychological traits,”“emotional intellige

77、nce,”and“social aptitudes”of job candidateswere unproven,invasive,and prone to bias.EPIC also challenged HireVues allegedly deceptive claim that it did not use facial recognition in its assessments.Fourteen months later,HireVue removed its facial recognition tools from its hiring assessment software

78、.In May 2022,the EEOC filed its first algorithmic discrimina-tion case against an English-language tutoring service com-pany.In the class action,the EEOC alleges that the employer“intentionally discriminated against older applicants because of their age by programing their software to automatically

79、reject female applicants aged 55 or older and male appli-cants aged 60 or older”in violation of the Age Discrimination in Employment Act.39 As of May 2023,the case remains pend-ing in federal court.New York City.On January 1,2023,New York Citys law reg-ulating the use of AI tools in hiring and promo

80、tion became effective,40 but soon thereafter the City postponed enforce-ment until April 15,2023,due to the high volume of public com-ments in response to the Proposed Rules.41 On April 6,2023,the City released its Final Rules and further delayed enforce-ment until July 5,2023.42 New York Citys law

81、is the first of its kind to regulate the use of automated employment decision tools(“AEDT”),which the law defines as“any computational process,derived from machine learning,statistical modeling,data analytics,or artificial intel-ligence,that issues simplified output,including a score,classi-fication

82、,or recommendation,that is used to substantially assist or replace discretionary decision-making”in employment decisions.It excludes tools that do not“automate,support,or substantially assist or replace discretionary decision-making 6Jones Day White Paperprocesses”such as junk email filters,calculat

83、ors,databases,or data sets.The Final Rules provide much-needed clarification on the scope of AEDT.The regulations clarify that the law covers only AEDTs that:“(1)generate a prediction,meaning an expected outcome for an observation,such as an assessment of a can-didates fit or likelihood of success,o

84、r that generate a classi-fication,meaning an assignment of an observation to a group,such as categorizations based on skill sets or aptitude;and(2)for which a computer at least in part identifies the inputs,the relative importance placed on those inputs,and,if applicable,other parameters for the mod

85、els in order to improve the accu-racy of the prediction or classification.”43 An employer must comply with the laws substantive require-ments if it uses an AEDT to make a hiring and/or promotional decision and relies upon the tools“simplified output”:(i)solely with no other factors considered;(ii)mo

86、re than any other cri-terion;or(iii)to overrule conclusions derived from other fac-tors,including human decision-making.“Simplified output”means“a prediction or classification,”which can take the form of a score,(e.g.,rating a candidates estimated technical skills),tag or categorization(e.g.,categor

87、izing a candidates resume based on key words,assigning a skill or trait to a can-didate),recommendation(e.g.,whether a candidate should be given an interview),or ranking(e.g.,arranging a list of candi-dates based on how well their cover letters match the job description).When an employer uses an AED

88、T in hiring or promotions,the employer must:(i)conduct a bias audit on the tool;(ii)pub-lish a summary of the results of the most recent bias audit on the employers website before using the tool;and(iii)provide applicants notice of the employers use of the tool.44 A“bias audit”is an impartial evalua

89、tion by an independent auditor that must assess the tools disparate impact on persons in the EEO categories of race,ethnicity,and sex,as well as intersec-tional categories.An“independent auditor”means a person or group that is not involved in using or developing the AI tool and is not an employee of

90、 the employer or the vendor who developed the tool.The Final Rules prescribe the minimum requirements for the bias audit,including:(i)how to calculate the selection rate,impact rate,or scoring rate;(ii)the types of datahistorical or test dataused to conduct the audit;and(iii)the criteria for selecti

91、ng an independent auditor.Multiple employers using the same AEDT may rely on the same bias audit if they provide historical data for the audit.The New York City Department of Consumer and Worker Protection began enforcing the law on July 5,2023.The Department can issue fines between$500 and$1,500 pe

92、r vio-lation,per day.Specifically,“each day on which an employer uses an automated employment decision tool”or each time an employer fails to provide the required notice to a candidate or employee will constitute a separate violation.Although the law does not expressly provide a private right of act

93、ion,the law will require disclosures that could prompt private litigants to challenge AEDTs under other laws,such as federal and state antidiscrimination laws.45Illinois and Maryland.In 2019,Illinois became the first state to pass legislation addressing the use of AI in the hiring process,specifical

94、ly the use of AI in evaluating job interview videos.46 The Artificial Intelligence Video Interview Act,which took effect in January 2020,requires employers who use AI to analyze applicant-submitted videos to:(i)notify each applicant that AI may be used to analyze the applicants video interview;(ii)p

95、rovide each applicant with information before the inter-view explaining how the AI works and what general types of characteristics it uses to evaluate applicants;and(iii)obtain,before the interview,consent from the applicant to be eval-uated by the AI program.Under the law,employers cannot use AI to

96、 evaluate applicants who have not consented to its use and cannot share applicant videos,except with those whose expertise is necessary to evaluate the applicants fit-ness.Additionally,employers must delete an applicants video interview,including all copies and backups,within 30 days of receiving a

97、request to delete.Effective January 1,2022,the Illinois Legislature amended the Artificial Intelligence Video Interview Act to require employ-ers who rely solely upon an AI analysis of a video interview to determine whether an applicant will be selected for an in-person interview to collect and repo

98、rt demographic data to the Department of Commerce and Economic Opportunity.The report must include both:(i)the race and ethnicity of appli-cants who are and are not afforded the opportunity for an in-person interview after the AI analysis;and(ii)the race and 7Jones Day White Paperethnicity of applic

99、ants who are hired.The burden of analyz-ing the data for racial bias falls to the Department,which must prepare an annual report for the Governor and General Assembly disclosing its findings.In May 2020,Maryland passed a related law,which prohib-its employers from using a“facial recognition service”

100、for the purpose of creating a“facial template”during an applicants interview unless the applicant consents by signing a waiver.47 The law defines“facial recognition service”as“technology that analyzes facial features and is used for recognition or persis-tent tracking of individuals in still or vide

101、o images.”A“facial template”means“the machine-interpretable pattern of facial features that is extracted from one or more images of an indi-vidual by a facial recognition service.”Although the law does not explicitly address AI,AI tools may utilize facial recognition technology.Neither the Illinois

102、nor Maryland laws provide a private right of action,enforcement mechanism,or penalties for noncompli-ance.Even without these features,employers should antici-pate that states may amend these laws to give them more teeth in coming years.California Draft Regulations.On March 15,2022,Californias Civil

103、Rights Department(“CRD”)(formerly the California Department of Fair Employment and Housing)issued pro-posed regulations regarding the use of AI in employment deci-sions.48 Unlike state laws that focus on notice,consent,and bias reporting,Californias proposed regulations make clear that employers and

104、 other companies that use,administer,or create AI tools that impact applicants or employees can face liability under the Fair Employment and Housing Act(“FEHA”),the states antidiscrimination law.In July 2022,the CRD proposed additional modifications to its proposed regulations,which make it unlawful

105、 for an employer or other covered entity to use“qualification standards,employ-ment tests,automated-decision systems,proxies or other selection criteria if such criteria have a disparate impact on or constitute disparate treatment of an applicant,employee,or class of applicants or employees,on the b

106、asis of protected characteristics unless the criteria are shown to be both(1)job related for the position and(2)consistent with business necessity.”49 Under the proposed regulations,an“automated-decision sys-tem”means“a computational process,including one derived from machine-learning,statistics,or

107、other data processing or artificial intelligence techniques,that screens,evaluates,categorizes,recommends,or otherwise makes a decision or facilitates human decision-making that impacts employees or applicants.”Examples of“automated-decision systems”include directing job advertisements to targeted g

108、roups;screening resumes for particular terms or patterns;analyzing facial expressions,word choices,and voices in online inter-views;and measuring aptitude,cognitive capabilities,or cul-tural fit through tests,questionnaires,games,puzzles,or other challenges.The draft regulations provide an illustrat

109、ive example as to how an employers use of AI tools may violate FEHA.“An automated-decision system that measures an applicants reaction time may unlawfully screen out individuals with certain disabilities.Unless an affirmative defense applies(e.g.,an employer dem-onstrates that a quick reaction time

110、while using an electronic device is job-related and consistent with business necessity),an employers decisions made or facilitated by automated-decision systems may constitute unlawful discrimination.”50 Interestingly,under the proposed regulations,companies that sell AI tools may be liable under an

111、 aiding-and-abetting theory for advertising,selling,or promoting their automated-decision system if the system unlawfully limits,screens out,or discrimi-nates against applicants or employees based on protected characteristics.51 Separately,the California Legislature introduced SB 313 on February 6,2

112、023,to create an“Office of Artificial Intelligence”within the states Department of Technology that would,among other things,guide the design,use,or deployment of auto-mated systems by a state agency to ensure such AI tools are used in compliance with state and federal laws.Other States and Localitie

113、s.Other states and localities throughout the United States are also seeking to regulate AI,facial recognition software,and algorithms in the employment context.For example,in late 2021,the District of Columbia introduced Bill 24-558,the“Stop Discrimination by Algorithms Act of 2021,”which seeks to p

114、rohibit employers from using algorithms that make decisions based on protected catego-ries(i.e.,race,sex,ethnicity,etc.).52 Others states,including 8Jones Day White PaperColorado,53 Illinois,54 Massachusetts,55 New Jersey,56 New York,57 Pennsylvania,58 Vermont,59 and Washington,60 have proposed or c

115、reated tasks forces or committees to research and advise on the use of AI in employment and other industries.Task forces are just the beginning,and employers should expect more states to regulate in this area in years to come.AI CHATBOTS AT WORKGenerative AI systems create text and/or photos in resp

116、onse to human input,such as a question posed to a chatbot.In recent months,the widespread availability of generative AI with user-friendly interfaces has spurred what some are call-ing an“AI race.”61 Employers response to generative AI has been mixed.Some employers are encouraging employees to utili

117、ze such genera-tive AI to produce content,perform research,and respond to customer inquiries.Others have taken a different approach,banning employees from using generative AI at work,citing privacy and confidentiality concerns.Going forward,employ-ers should consider both the accuracy of generative

118、AI outputs,the potential for plagiarism,and the potential for generative AI to compromise company confidential information and trade secrets if such information is disclosed in the query posed to the AI chatbot.62 Generative AI is impressive,but not infallible.Put simply,its output is only as accura

119、te as its source material,which itself may be inaccurate.Sometimes,generative AI produces responses with errors,factually incorrect information,or pla-giarized content.63 OpenAI acknowledges the limitations of its generative AI tool,ChatGPT,which“can occasionally produce incorrect answers”and“may al

120、so occasionally produce harm-ful instructions or biased content.”64 Indeed,ChatGPT was trained on data sets available through 2021,so there are gaps in its knowledge base.65 At work,problems and risks may arise if employees rely on outputs without fact-checking the output or determining whether it a

121、ppears plagiarized.Aware of this concern,a federal judges standing order requires all attor-neys appearing before the court to file a certificate attesting either that no portion of their filing was drafted by genera-tive AI or that any language drafted by generative AI was checked for accuracy by a

122、 human being.66 Where employees use AI chatbots at work,they should be required to closely review chatbot outputs before incorporating the result into work product.Protecting confidential trade secrets and other propri-etary information is another area of concern for employers.Employers may worry th

123、at employees may enter confidential or proprietary information into the chatbot.Assessing this risk is difficult.On the one hand,depending on the nature of the applicable end-user license agreement,conversations may be reviewed by AI trainers and the chatbot may“learn”and refine its answers based on

124、 user input,which could compromise confidential information.67 But on the other hand,an employee disclosure into a chatbot is arguably not a disclosure to a“per-son”or the public.Employers who allow employees to utilize AI chatbots should consider updating their employee handbooks and/or proprietary

125、 information protection policies to address the possibility of employees entering trade secret or confiden-tial information into online chatbots.PRACTICAL SOLUTIONS Traditional legal frameworks are notoriously slow at adapt-ing to new technology.And employers may feel hamstrung between the need to a

126、dopt AI technologies to stay competi-tive and the potential for legal risks.But no employer wants to be the test subject in a case applying employment laws to their use of this developing technology.There are several tan-gible solutions and best practices employers might consider adopting to mitigat

127、e legal risks associated with using AI in hiring and recruiting.Identify and Vet AI ToolsAs a threshold matter,employers would be wise to identify what products currently in use rely on AI and adopt a uniform policy for registering and tracking the workforces utilization of AI tools.Employers should

128、 not limit this inquiry to whether and how they use AI in the hiring process;instead,they may wish to examine if they use these tools“to monitor and track employees and track performance”for purposes such as pro-motions,demotions,or other potential actionable employment decisions.68 Employers may wa

129、nt to form an AI committee that develops policies and vets risks associated with the compa-nys use of AI tools.9Jones Day White PaperKnow the AI Product The contract law of caveat emptor,or“buyer beware,”is a fit-ting maxim for employers to abide by when purchasing an AI tool from a third-party vend

130、or or using an AI tool developed in-house.Before implementing the tool,employers should take steps to understand both what the AI tool relies upon to make its assessment and how it makes the assessment.As a starting point,with respect to employment-law issues,employers can ask third-party vendors or

131、 in-house engi-neers some or all of the following questions when evaluating the AI tool:Did you attempt to determine whether use of the algorithm disadvantages individuals based on protected categories such as race,gender,age,disability,etc.?For example,did you assess whether any of the traits or ch

132、aracteristics that the tool relies upon for its output are highly correlated with a particular race,gender,age,disability,etc.?Was that assessment conducted under legal privilege?What data was the tool“trained”on?What data does the tool use to evaluate its subjects?For example,does the tool rely upo

133、n criminal records,salary history,or other restricted data in the employment context?How does the tool reach its recommendations and/or conclusions?Is the tools interface accessible to as many individuals with disabilities as possible?Are the materials presented to job applicants or employees in alt

134、ernative formats?If so,which formats?Are there any kinds of disabilities for which there will not be accessible formats?Are there mechanisms for ongoing monitoring and evalua-tion of the AI tools performance such as impact on HR pro-cesses and employee experiences to identify any issues or areas for

135、 improvement?How does the tool adjust or adapt in response to identified issues?Is the tool subject to any state or local laws that impose disclosure or notice obligations,and what work has been done to prepare for those requirements?To the extent state or local law requires a bias audit,who will sh

136、oulder the costs(e.g.,as between the employer and the vendor)?Prior to conducting an audit,what assurances are there that the results will not create legal liability for use of the tool?Has a prior audit been conducted under privilege that provides reassurance that the tool is not biased?Review Cont

137、racts with AI Vendors According to a 2022 study from the Society for Human Resource Management,92%of employers who use AI to sup-port human resources functions source some or all of these tools directly from a third-party vendor.69 Employers who use AI tools provided by third-party vendors should ca

138、refully review their contracts and consider how they allocate liability for employment claims based upon use of the product.Conduct Bias Audits Given the current environment,with states and localities pass-ing new laws,and governmental and private litigants pursuing enforcement actions under existin

139、g laws,employers should consider conducting a bias audit of AI tools if it has not been done.If the audit identifies problems,the employer will then have time to correct the problems in an orderly fashion,with-out the time pressure created by impending enforcement or litigation-related deadlines.Con

140、ducting a proactive bias audit can be particularly ben-eficial given how long it can take to bring AI tools into com-pliance.Employers subject to the New York City Automated Employment Decision Tools law are finding that coming into compliance can take months of preparation and coordination with var

141、ious stakeholders.As an initial matter,it can be diffi-cult to identify all tools that rely on AI unless the employer has already developed a centralized method to track such tools.Even once the tools are identified,understanding how the tools work and whether they are covered by these new laws take

142、s effort,particularly given the need to coordinate with ven-dors who may have better knowledge of how the tools operate.Once these hurdles are cleared,the employer will want to coordinate with legal counsel before conducting a bias audit.Employers may want to conduct these audits under privilege to

143、promote candid,objective assessments of whether the tool might create a risk of discrimination findings.Involving coun-sel will also help the employer understand how to design the bias audit.Employment lawyers who are familiar with court decisions and agency guidance on disparate impact analyses can

144、 ensure that the audit is structured in a way that tracks the applicable legal requirements.10Jones Day White PaperCONCLUSION Artificial intelligence is here to stay,and its uses will only con-tinue to expand.AI has already and will continue to revolution-ize the way organizations hire and recruit e

145、mployees,among many other employment functions.Every revolution brings with it new challenges for employers and employees alike.By keeping abreast of the latest developments and implement-ing best practices,employers can implement these emerging technologies while mitigating legal risk.ENDNOTES1 EEO

146、C Press Release,“EEOC Launches Initiative on Artificial Intelligence and Algorithmic Fairness”(Oct.28,2021).2 DOJ Civil Rights Division,Algorithms,Artificial Intelligence,and Disability Discrimination in Hiring(May 12,2022);EEOC,The Americans with Disabilities Act and the Use of Software,Algorithms,

147、and Artificial Intelligence to Assess Job Applicants and Employees(May 12,2022).At least one EEOC Commissioner has publicly voiced concern that the EEOCs May 12,2022,technical assistance doc-ument“was not voted on by the full Commission,and did not go through the administrative law process involving

148、 noticed and com-ment.”Keith E.Sonderling,Bradford J.Kelly,and Lance Casimar,The Promise and The Peril:Artificial Intelligence and Employment Discrimination,77 U.Mia.L.Rev.1,42(2022).3 EEOC Press Release,“EEOC Releases New Resource on Artificial Intelligence and Title VII”(May 18,2023);EEOC,Select I

149、ssues:Assessing Adverse Impact in Software,Algorithms,and Artificial Intelligence Used in Employment Selection Procedures Under Title VII of the Civil Rights Act of 1964(May 18,2023).4 Keith E.Sonderling&Bradford J.Kelley,Filling the Void:Artificial Intelligence and Private Initiatives,North Carolin

150、a Journal of Law&Technology(Vol.24,Issue 4:May 2023).5 Digital Platform Commission Act,S.1671,118th Congress(2023).6 Workforce Matters,EEOC working to stop artificial intelligence from perpetuating bias in hiring(Dec.5,2021).7 Society for Human Resource Management,Fresh SHRM Research Explores Use of

151、 Automation and AI in HR(April 13,2022).8 Id.9 Jeffrey Dastin,Amazon scraps secret AI recruiting tool that showed bias against women,Reuters(Oct.10,2018).10 Mobley v.Workday,Inc.,No.3:23-CV-00770(N.D.Cal.Feb.21,2023).11 EEOC,Select Issues,supra note 3.12 The disparate impact proof structure under Ti

152、tle VII,in part due to congressional amendments of that statute in 1991,differs in impor-tant ways from the proof structure under other federal antidiscrimina-tion provisions that codify disparate impact.13 42 U.S.C.2000e-2(k)(1)(A).14 Id.15 Id.2000e-2(k)(1)(A)(ii).16 EEOCs Uniform Guidelines on Emp

153、loyee Selection Procedures,codi-fied at 29 C.F.R.1607.1,1607.4(D).17 See,e.g.,Watson v.Fort Worth Bank&Trust,487 U.S.977,995(1988)(noting that the rule“has been criticized on technical grounds.and has not provided more than a rule of thumb for courts”);Stagi v.Natl R.R.Passenger Corp.,391 Fed.Appx.1

154、33,138(3d Cir.2010)(unpub-lished)(“The four-fifths rule has come under substantial criticism,and has not been particularly persuasive.”).18 See EEOC,Select Issues,supra note 3.19 433 U.S.299(1977)20 42 U.S.C.2000e-2(k)(1)(A).See also Griggs v.Duke Power Co.,401 U.S.424(1971).LAWYER CONTACTSRick Berg

155、stromSan Diego+Wendy C.ButlerNew York+Eric S.DreibandWashington+Jonathan M.LinasChicago+Alexander V.MaugeriNew York+Efrat R.SchulmanChicago+Lauren Ball,an associate in the San Diego Office,contributed to this White Paper.2023 Jones Day.All rights reserved.Printed in the U.S.A.Jones Day publications

156、should not be construed as legal advice on any specific facts or circumstances.The contents are intended for general information purposes only and may not be quoted or referred to in any other publication or proceeding without the prior written consent of the Firm,to be given or withheld at our disc

157、retion.To request reprint permission for any of our publications,please use our“Contact Us”form,which can be found on our website at .The mailing of this publication is not intended to create,and receipt of it does not constitute,an attorney-client relationship.The views set forth herein are the per

158、sonal views of the authors and do not necessarily reflect those of the Firm.21 EEOC,Enforcement Guidance on the Consideration of Arrest and Conviction Records in Employment Decisions under Title VII of the Civil Rights Act(April 25,2021).22 EEOC Compliance Manual,Compensation Discrimination,No.915.0

159、03,at 10-IV(F)(2)(G)(Dec.5,2000)(collecting cases);Rizo v.Yovino,950 F.3d 1217,1219(9th Cir.2020)(“Prior rate of pay is not a factor other than sex that allows an employer pay a female employee less than male employees who perform the same work.”(quoting 29 U.S.C.206(d)(1)(iv).But see Boyer v.United

160、 States,159 Fed.Cl.387,402(2022)(explaining that courts are divided about whether prior pay alone is a legitimate“factor other than sex”for purposes of the Equal Pay Act);Wernsing v.Dept of Hum.Servs.,State of Ill.,427 F.3d 466,468(7th Cir.2005)(“Wages at ones prior employer are a factor other than

161、sex and so.an employer may use them to set pay consistently with the Act.”);Spencer v.Va.State Univ.,919 F.3d 199,206(4th Cir.2019)(denying Equal Pay Act claim in which“the wage difference at issue resulted from the University setting claimants and comparators pay at 75%of their previous salaries as

162、 administrators”).23 See,e.g.,Cal.Lab.Code 432.3(a).24 Biometric privacy bills have been introduced in various states(i.e.,Arizona,Hawaii,Maryland,Massachusetts,Minnesota,Mississippi,Missouri,New York,Tennessee,Vermont,and Washington)since the start of the 2023 legislative session.Most are modeled a

163、fter Illinoiss BIPA.25 Letter to FTC,2011 WL 2110608(May 9,2011).26 15 U.S.C.1681a(d)(1).27 See 15 U.S.C.1681b.28 See,e.g.,EEOC,Select Issues,supra note 3.29 The White House,Blueprint for an AI Bill of Rights.30 Id.31 EEOC,Press Release,supra note 1.32 EEOC,Press Release,supra note 3;EEOC,Select Iss

164、ues,supra note 3.33 EEOC,Select Issues,supra note 3.34 Id.35 Id.36 U.S.Department of Justice,Press Release,“Justice Department Secures Groundbreaking Settlement Agreement with Meta Platforms,Formerly Known as Facebook,to Resolve Allegations of Discriminatory Advertising”(June 21,2022).37 Statement o

165、f Interest of the United States,Louis v.Saferent Solutions,LLC,Case 1:22-cv-10800-AK(D.Mass.Jan.9,2023).38 Electronic Privacy Information Center,Consumer Cases,In re HireVue.39 EEOC v.iTutorGroup,Inc.,et al.,Case No.1:22-cv-02565(E.D.N.Y.)40 N.Y.C.Admin.Code 20-870,et seq.41 New Laws and Rules,Autom

166、ated Employment Decision Tools,December Update,NYC Consumer and Worker Protection.42 New York City Department of Consumer and Worker Protection,Notice of Adoption of Final Rule.43 Rules of the City of New York 5-300,Definitions(Proposed).44 N.Y.C.Admin.Code 20-871(a).45 See,e.g.,New York City Human

167、Rights Law,NYC Administrative Code 8-101,et seq.(the NYCHRL protects employees,applicants,and unpaid interns from discrimination on the basis of actual or per-ceived race,gender,disability,and other protected characteristics).46 Public Act 101-0260.47 H.B.1202.48 Fair Employment&Housing Counsel Draf

168、t Modifications to Employment Regulations Regarding Automated-Decision Systems(March 15,2022).49 Cal.Code Regs.11008(f)(Proposed).50 Cal.Code Regs.11008(d)(Proposed).51 Cal.Code Reg.11020(a)(1)-(2)(Proposed).52 B24-0058.53 CO S.B.113.54 IL H.B.0645.55 MA H.B.4512(Proposed)56 NJ A.B.168(Proposed)57 N

169、Y A.B.2414(Proposed)58 PA H.B.1338(Proposed).59 VT H.B.410.60 WA S.B.5092.61 Associated Press,Tech leaders urge a pause in the out-of-control artificial intelligence race,NPR(March 29,2023).62 See e.g.,Siladitya Ray,Samsung Bans ChatGPT Among Employees After Sensitive Code Leak,Forbes(May 2,2023);Ju

170、lia Horowitz,JPMorgan restricts employee use of ChatGPT,CNN(Feb.22,2023).63 Danielle Abril,Can ChatGPT help me at the office?We put the AI Chatbot to the test.,Wash.Post(Feb.2,2023).64 OpenAI,ChatGPT General FAQ.65 Id.66 Judge Brantley Starr,Judge Specific RequirementsMandatory Certification Regarding Generative Artificial Intelligence.67 See OpenAI,supra note 63.68 Keith E.Sonderling,Bradford J.Kelly,and Lance Casimar,The Promise and The Peril:Artificial Intelligence and Employment Discrimination,77 U.Mia.L.Rev.1,74(2022).69 Society for Human Resource Management,supra note 7.