AI Now report on bias in algorithms & recommendations

For those interested in AI and algorithms, AI now has just published an excellent report as an outcome of their symposium. See attached - recommendations 1, 7, and 9 are perhaps areas we could work on.

 Recommendations from the Report

These​ ​recommendations​ ​reflect​ ​the​ ​views​ ​and​ ​research​ ​of​ ​the​​ ​​AI​ ​Now​ ​Institute​ ​at​ ​New​ ​York
University.​ ​We​ ​thank​ ​the​ ​experts​ ​who​ ​contributed​ ​to​ ​the​ ​​AI​ ​Now​ ​2017​ ​Symposium​ ​and
Workshop​​ ​for​ ​informing​ ​these​ ​perspectives,​ ​and​ ​our​ ​research​ ​team​ ​for​ ​helping​ ​shape​ ​the​​ ​AI
Now​ ​2017​ ​Report.


1. Core​ ​public​ ​agencies,​ ​such​ ​as​ ​those​ ​responsible​ ​for​ ​criminal​ ​justice,​ ​healthcare,
welfare,​ ​and​ ​education​ ​(e.g​ ​“high​ ​stakes”​ ​domains)​ ​should​ ​no​ ​longer​ ​use​ ​“black​ ​box”
AI​ ​and​ ​algorithmic​ ​systems.​​ ​This​ ​includes​ ​the​ ​unreviewed​ ​or​ ​unvalidated​ ​use​ ​of
pre-trained​ ​models,​ ​AI​ ​systems​ ​licensed​ ​from​ ​third​ ​party​ ​vendors,​ ​and​ ​algorithmic
processes​ ​created​ ​in-house.​ ​The​ ​use​ ​of​ ​such​ ​systems​ ​by​ ​public​ ​agencies​ ​raises​ ​serious
due​ ​process​ ​concerns,​ ​and​ ​at​ ​a​ ​minimum​ ​they​ ​should​ ​be​ ​available​ ​for​ ​public​ ​auditing,
testing,​ ​and​ ​review,​ ​and​ ​subject​ ​to​ ​accountability​ ​standards.


2. Before​ ​releasing​ ​an​ ​AI​ ​system,​ ​companies​ ​should​ ​run​ ​rigorous​ ​pre-release​ ​trials​ ​to
ensure​ ​that​ ​they​ ​will​ ​not​ ​amplify​ ​biases​ ​and​ ​errors​ ​​​due​ ​to​ ​any​ ​issues​ ​with​ ​the​ ​training
data,​ ​algorithms,​ ​or​ ​other​ ​elements​ ​of​ ​system​ ​design.​ ​​As​ ​this​ ​is​ ​a​ ​rapidly​ ​changing​ ​field,
the​ ​methods​ ​and​ ​assumptions​ ​by​ ​which​ ​such​ ​testing​ ​is​ ​conducted,​ ​along​ ​with​ ​the
results,​ ​should​ ​be​ ​openly​ ​documented​ ​and​ ​publicly​ ​available,​ ​with​ ​clear​ ​versioning​ ​to
accommodate​ ​updates​ ​and​ ​new​ ​findings.

3. After​ ​releasing​ ​an​ ​AI​ ​system,​ ​companies​ ​should​ ​continue​ ​to​ ​monitor​ ​its​ ​use​ ​across
different​ ​contexts​ ​and​ ​communities.​​ ​The​ ​methods​ ​and​ ​outcomes​ ​of​ ​monitoring​ ​should
be​ ​defined​ ​through​ ​open,​ ​academically​ ​rigorous​ ​processes,​ ​and​ ​should​ ​be​ ​accountable
to​ ​the​ ​public.​ ​Particularly​ ​in​ ​high​ ​stakes​ ​decision-making​ ​contexts,​ ​the​ ​views​ ​and
experiences​ ​of​ ​traditionally​ ​marginalized​ ​communities​ ​should​ ​be​ ​prioritized.

4. More​ ​research​ ​and​ ​policy​ ​making​ ​is​ ​needed​ ​on​ ​the​ ​use​ ​of​ ​AI​ ​systems​ ​in​ ​workplace
management​ ​and​ ​monitoring,​ ​including​ ​hiring​ ​and​ ​HR.​ ​​This​ ​research​ ​will​ ​complement
the​ ​existing​ ​focus​ ​on​ ​worker​ ​replacement​ ​via​ ​automation.​ ​Specific​ ​attention​ ​should​ ​be
given​ ​to​ ​the​ ​potential​ ​impact​ ​on​ ​labor​ ​rights​ ​and​ ​practices,​ ​and​ ​should​ ​focus​ ​especially
on​ ​the​ ​potential​ ​for​ ​behavioral​ ​manipulation​ ​and​ ​the​ ​unintended​ ​reinforcement​ ​of​ ​bias
in​ ​hiring​ ​and​ ​promotion.


5. Develop​ ​standards​ ​to​ ​track​ ​the​ ​provenance,​ ​development,​ ​and​ ​use​ ​of​ ​training​ ​datasets
throughout​ ​their​ ​life​ ​cycle.​ ​​This​ ​is​ ​necessary​ ​to​ ​better​ ​understand​ ​and​ ​monitor​ ​issues​ ​of
bias​ ​and​ ​representational​ ​skews.​ ​In​ ​addition​ ​to​ ​developing​ ​better​ ​records​ ​for​ ​how​ ​a
training​ ​dataset​ ​was​ ​created​ ​and​ ​maintained,​ ​social​ ​scientists​ ​and​ ​measurement
researchers​ ​within​ ​the​ ​AI​ ​bias​ ​research​ ​field​ ​should​ ​continue​ ​to​ ​examine​ ​existing​ ​training
datasets,​ ​and​ ​work​ ​to​ ​understand​ ​potential​ ​blind​ ​spots​ ​and​ ​biases​ ​that​ ​may​ ​already​ ​be
at​ ​work.

6. Expand​ ​AI​ ​bias​ ​research​ ​and​ ​mitigation​ ​strategies​ ​beyond​ ​a​ ​narrowly​ ​technical
approach.​​ ​Bias​ ​issues​ ​are​ ​long​ ​term​ ​and​ ​structural,​ ​and​ ​contending​ ​with​ ​them
necessitates​ ​deep​ ​interdisciplinary​ ​research.​ ​Technical​ ​approaches​ ​that​ ​look​ ​for​ ​a
one-time​ ​“fix”​ ​for​ ​fairness​ ​risk​ ​oversimplifying​ ​the​ ​complexity​ ​of​ ​social​ ​systems.​ ​Within
each​ ​domain​ ​–​ ​such​ ​as​ ​education,​ ​healthcare​ ​or​ ​criminal​ ​justice​ ​–​ ​legacies​ ​of​ ​bias​ ​and
movements​ ​toward​ ​equality​ ​have​ ​their​ ​own​ ​histories​ ​and​ ​practices.​ ​Legacies​ ​of​ ​bias
cannot​ ​be​ ​“solved”​ ​without​ ​drawing​ ​on​ ​domain​ ​expertise.​ ​Addressing​ ​fairness
meaningfully​ ​will​ ​require​ ​interdisciplinary​ ​collaboration​ ​and​ ​methods​ ​of​ ​listening​ ​across
different​ ​disciplines.

7. Strong​ ​standards​ ​for​ ​auditing​ ​and​ ​understanding​ ​the​ ​use​ ​of​ ​AI​ ​systems​ ​“in​ ​the​ ​wild”
are​ ​urgently​ ​needed.​​ ​Creating​ ​such​ ​standards​ ​will​ ​require​ ​the​ ​perspectives​ ​of​ ​diverse
disciplines​ ​and​ ​coalitions.​ ​The​ ​process​ ​by​ ​which​ ​such​ ​standards​ ​are​ ​developed​ ​should​ ​be
publicly​ ​accountable,​ ​academically​ ​rigorous​ ​and​ ​subject​ ​to​ ​periodic​ ​review​ ​and​ ​revision.


8. Companies,​ ​universities,​ ​conferences​ ​and​ ​other​ ​stakeholders​ ​in​ ​the​ ​AI​ ​field​ ​should
release​ ​data​ ​on​ ​the​ ​participation​ ​of​ ​women,​ ​minorities​ ​and​ ​other​ ​marginalized​ ​groups
within​ ​AI​ ​research​ ​and​ ​development.​​ ​Many​ ​now​ ​recognize​ ​that​ ​the​ ​current​ ​lack​ ​of
diversity​ ​in​ ​AI​ ​is​ ​a​ ​serious​ ​issue,​ ​yet​ ​there​ ​is​ ​insufficiently​ ​granular​ ​data​ ​on​ ​the​ ​scope​ ​of
the​ ​problem,​ ​which​ ​is​ ​needed​ ​to​ ​measure​ ​progress.​ ​Beyond​ ​this,​ ​we​ ​need​ ​a​ ​deeper
assessment​ ​of​ ​workplace​ ​cultures​ ​in​ ​the​ ​technology​ ​industry,​ ​which​ ​requires​ ​going
beyond​ ​simply​ ​hiring​ ​more​ ​women​ ​and​ ​minorities,​ ​toward​ ​building​ ​more​ ​genuinely
inclusive​ ​workplaces.

9. The​ ​AI​ ​industry​ ​should​ ​hire​ ​experts​ ​from​ ​disciplines​ ​beyond​ ​computer​ ​science​ ​and
engineering​ ​and​ ​ensure​ ​they​ ​have​ ​decision​ ​making​ ​power.​ ​​​ ​As​ ​AI​ ​moves​ ​into​ ​diverse
social​ ​and​ ​institutional​ ​domains,​ ​influencing​ ​increasingly​ ​high​ ​stakes​ ​decisions,​ ​efforts
must​ ​be​ ​made​ ​to​ ​integrate​ ​social​ ​scientists,​ ​legal​ ​scholars,​ ​and​ ​others​ ​with​ ​domain
expertise​ ​that​ ​can​ ​guide​ ​the​ ​creation​ ​and​ ​integration​ ​of​ ​AI​ ​into​ ​long-standing​ ​systems
with​ ​established​ ​practices​ ​and​ ​norms.

 
10. Ethical​ ​codes​ ​meant​ ​to​ ​steer​ ​the​ ​AI​ ​field​ ​should​ ​be​ ​accompanied​ ​by​ ​strong​ ​oversight
and​ ​accountability​ ​mechanisms.​​ ​More​ ​work​ ​is​ ​needed​ ​on​ ​how​ ​to​ ​substantively​ ​connect
high​ ​level​ ​ethical​ ​principles​ ​and​ ​guidelines​ ​for​ ​best​ ​practices​ ​to​ ​everyday​ ​development
processes,​ ​promotion​ ​and​ ​product​ ​release​ ​cycles.