Algocracia en el Poder Judicial: un desafío a la confianza en el sistema
No. 93 (2025-07-25)Autor/a(es/as)
-
Maria Lorena Flórez RojasUniversity of Groningen, Países BajosIdentificador ORCID: https://orcid.org/0000-0002-8264-8227
Resumen
Este artículo analiza la manera en que la algocracia —la delegación de la toma de decisiones judiciales a sistemas algorítmicos— afecta la confianza ciudadana en los tribunales que funcionan dentro de estructuras de gobernanza multinivel cada vez más mediadas por la inteligencia artificial. El artículo sostiene que la legitimidad legal depende de la ética normativa y de la aceptación social que se desprende de la confianza. A partir de una metodología cualitativa que incorpora el análisis doctrinal, la evaluación regulatoria comparada y el estudio de casos en México, Australia, Alemania, Países Bajos y Dinamarca, esta investigación desarrolla un marco interpretativo que se compone de cinco principios: la transparencia, la responsabilidad, la comprensión, la justicia social y la supervisión confiable. El marco es implementado en una tipología de ocho herramientas de toma de decisiones basadas en algoritmos, orientadas a evaluar su impacto en la confianza individual, institucional y social. Los hallazgos indican que la confianza se erosiona cuando los sistemas funcionan de manera opaca, involucran la delegación sin control y reproducen el sesgo de datos, condiciones que pueden desencadenar crisis de legitimidad y resistencia cívica. De esta manera, el marco propuesto proporciona unas directrices prácticas para el diseño, la auditoría y la implementación de herramientas algorítmicas capaces de asegurar la deliberación humana y el acceso igualitario a la justicia. Este trabajo tiende puentes entre los debates filosóficos sobre algocracia y los análisis legales centrados en la confianza, con el fin de aportar una herramienta original e interdisciplinaria para asistir a los tribunales en la gestión de la innovación tecnológica sin comprometer la legitimidad democrática.
Referencias
Alvarez, Jose M., Alejandra Bringas Colmenarejo, Alaa Elobaid, Simone Fabbrizzi, Miriam Fahimi, Antonio Ferrara, et al. 2024. “Policy Advice and Best Practices on Bias and Fairness in AI”. Ethics and Information Technology 26 (31): 1-26. https://doi.org/10.1007/s10676-024-09746-w
Amnesty International. 2021. Xenophobic Machines: Discrimination through Unregulated Use of Algorithms in the Dutch Childcare Benefits Scandal. London: Amnesty International Ltd. https://www.amnesty.org/en/documents/eur35/4686/2021/en/
Amnesty International. 2024. Denmark: Coded Injustice: Surveillance and Discrimination in Denmark’s Automated Welfare State. London: Amnesty International Ltd. https://www.amnesty.org/en/documents/eur18/8709/2024/en/
Aneesh, Aneesh. 2009. “Global Labor: Algocratic Modes of Organization”. Sociological Theory 27 (4): 347-370. https://doi.org/10.1111/J.1467-9558.2009.01352.X
Azzutti, Alessio, Wolf-Georg Ringe, and H. Siegfried Stiehl. 2023. “Artificial Intelligence in Finance: Challenges, Opportunities and Regulatory Developments”. In Artificial Intelligence in Finance: Challenges, Opportunities and Regulatory Developments, edited by Nydia Remolina and Aurelio Gurrea-Martinez, 198-242. London: Edward Elgar Publishing Ltd. https://doi.org/10.4337/9781803926179
Baier, Annette. 1986. “Trust and Antitrust”. Ethics 96 (2): 231-260. https://doi.org/10.1086/292745
Barysė, Dovilė, and Roee Sarel. 2024. “Algorithms in the Court: Does It Matter Which Part of the Judicial Decision- Making Is Automated?”. Artificial Intelligence and Law 32 (1): 117-146. https://doi.org/10.1007/S10506-022-09343-6
Bayer, Judit. 2024. “Post-Truth and Post-Trust: How to Re-Define Trust in the Judicial System and the Media”. ERA Forum 25 (2): 165-179. https://doi.org/10.1007/S12027-024-00795-8
Beckman, Ludvig, Jenny Hultin Rosenberg, and Karim Jebari. 2024. “Artificial Intelligence and Democratic Legitimacy: The Problem of Publicity in Public Authority”. AI & Society 39 (2): 975-984. https://doi.org/10.1007/s00146-022-01493-0
Bennett Moses, Lyria, Monika Zalnieriute, Michael Legg, Felicity Bell, and Jake Silove. 2022. “AI Decision- Making and the Courts: A Guide for Judges, Tribunal Members and Court Administrators”. Research Reports. Sydney: The Australian Institute of Judicial Administration Limited (AIJA). https://aija.org.au/publications/ai-decision-making-and-the-courts-a-guide-for-judges-tribunal-members-and-court-administrators/
Buijsman, Stefan, and Herman Veluwenkamp. 2023. “Spotting When Algorithms Are Wrong”. Minds and Machines 33 (4): 541-562. https://doi.org/10.1007/S11023-022-09591-0
BVerfG. 2023. Judgment of the First Senate of 16 February 2023 -1 BvR 1547/19-, paras. 1-178. https://www.bverfg.de/e/rs20230216_1bvr154719en
Cáceres, Enrique. 2008. “EXPERTIUS: A Mexican Judicial Decision-Support System in the Field of Family Law”. In Legal Knowledge and Information Systems, edited by Enrico Francesconi, Giovanni Sartor, and Daniela Tiscornia, 78-87. https://doi.org/10.3233/978-1-58603-952-3-78
Cantero Gamito, Marta, and Martin Ebers. 2021. “Algorithmic Governance and Governance of Algorithms: An Introduction”. In Algorithmic Governance and Governance of Algorithms. Legal and Ethical Challenges, edited by Martin Ebers and Marta Cantero Gamito, 1-22. Cham: Springer. https://doi.org/10.1007/978-3-030-50559-2_1
CEPEJ (European Commission for the Efficiency of Justice). 2024. European Judicial Systems. CEPEJ Evaluation Report. Part 1 General Analyses. Strasbourg: Council of Europe. https://rm.coe.int/cepej-evaluation-report-part-1-en-/1680b272ac
16. Ceva, Emanuela, and María Carolina Jiménez. 2022. “Automating Anticorruption?”. Ethics and Information Technology 24 (48): 1-14. https://doi.org/10.1007/s10676-022-09670-x
Cheong, Ben Chester. 2024. “Transparency and Accountability in AI Systems: Safeguarding Wellbeing in the Age of Algorithmic Decision-Making”. Frontiers in Human Dynamics 6: 1421273. https://doi.org/10.3389/FHUMD.2024.1421273
Colombi Ciacchi, Aurelia, María Lorena Flórez Rojas, Lottie Lane, and Tobias Nowak, eds. 2025. AI and Public Administration: The (Legal) Limits of Algorithmic Governance. Groningen: Faculty of Law / University of Groningen. https://www.julia-project.eu/resources
Constitutional Court of Colombia. 2024. Case T-323/24. https://www.corteconstitucional.gov.co/relatoria/2024/t-323-24.htm
Cram, Frederick. 2024. “Fairness, Relationships and Perceptions of Police Legitimacy in the Context of Integrated Offender Management”. Policing and Society 34 (4): 250-267. https://doi.org/10.1080/10439463.2023.2267733
Danaher, John. 2016. “The Threat of Algocracy: Reality, Resistance and Accommodation”. Philosophy and Technology 29 (3): 245-268. https://doi.org/10.1007/S13347-015-0211-1
Dávila, K.D. 2020. Please Hold. Scavenger Entertainment, United States.
De Brito Duarte, Regina, Filipa Correia, Patrícia Arriaga, and Ana Paiva. 2023. “AI Trust: Can Explainable AI Enhance Warranted Trust?”. Human Behavior and Emerging Technologies 2023 (1): 1-12. https://doi.org/10.1155/2023/4637678
Díaz-Rodríguez, Natalia, Javier Del Ser, Mark Coeckelbergh, Marcos López de Prado, Enrique Herrera-Viedma, and Francisco Herrera. 2023. “Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation”. Information Fusion 99: 101896. https://doi.org/10.1016/J.INFFUS.2023.101896
European Commission. 2024. Perceived Independence of the National Justice Systems in the EU among the General Public. Brussels: European Commission. https://europa.eu/eurobarometer/surveys/detail/3193
Felzmann, Heike, Eduard Fosch-Villaronga, Christoph Lutz, and Aurelia Tamò-Larrieux. 2020. “Towards Trans-parency by Design for Artificial Intelligence”. Science and Engineering Ethics 26 (6): 3333-3361. https://doi.org/10.1007/S11948-020-00276-4
Fine, Anna, and Shawn Marsh. 2024. “Judicial Leadership Matters (yet Again): The Association between Judge and Public Trust for Artificial Intelligence in Courts”. Discover Artificial Intelligence 4 (44): 1-18. https://doi.org/10.1007/S44163-024-00142-3
Flórez Rojas, María Lorena. 2024. “Artificial Intelligence in Judicial Decision-Making: A Comparative Analysis of Recent Rulings in Colombia and The Netherlands”. Journal of AI Law and Regulation 1 (3): 361-366. https://doi. org/10.21552/aire/2024/3/11
Fukuyama, Francis. 1995. Trust: The Social Virtues and the Creation of Prosperity. New York: Free Press.
Gaspar, Walter, and Yasmin Curzi de Mendonça. 2021. Artificial Intelligence in Brazil Still Lacks a Strategy. Rio de Janerio: Center for Technology and Society at FGV Law School. https://cyberbrics.info/wp-content/uploads/ 2021/05/EBIA-en-2.pdf
Gelderland District Court. 2024. Case 8060619 CV EXPL 19-4029 ECLI:NL:RBGEL:2024:3636. https://linkeddata.overheid.nl/front/portal/document-viewer?ext-id=ECLI:NL:RBGEL:2024:3636
Gutiérrez, Juan David. 2024a. “Critical Appraisal of Large Language Models in Judicial Decision-Making”. In Handbook on Public Policy and Artificial Intelligence, edited by Paul Regine, Emma Carmel, and Jennifer Cobbe, 323-339. London: Edward Elgar Publishing.
Gutiérrez, Juan David. 2024b. UNESCO Global Judges’ Initiative: Survey on the Use of AI Systems by Judicial Operators. Paris: UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000389786
Haitsma, Lucas Michael. 2023. “Regulating Algorithmic Discrimination through Adjudication: The Court of Justice of the European Union on Discrimination in Algorithmic Profiling Based on PNR Data”. Frontiers in Political Science 5: 1232601. https://doi.org/10.3389/FPOS.2023.1232601
Hardin, Russell. 2002. Trust and Trustworthiness. New York: Russell Sage Foundation.
Hough, Mike, Jonathan Jackson, and Ben Bradford. 2013. “Legitimacy, Trust, and Compliance: An Empirical Test of Procedural Justice Theory Using the European Social Survey”. In Legitimacy and Criminal Justice: An International Exploration, edited by Justice Tankebe and Alison Liebling, 326-352. Oxford: Oxford University Press. https://doi.org/10.1093/ACPROF:OSO/9780198701996.003.0017
Jones, Karen. 2012. “Trustworthiness”. Ethics 123 (1): 61-85. https://doi.org/10.1086/667838
Karhu, Mari, Jonna Häkkilä, and Eija Timonen. 2019. “Collaborative Media as a Platform for Community Powered Ecological Sustainability Transformations: A Case Study”. In MUM’19: Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia, 1-5. November 26-29, Pisa, Italy. https://doi.org/10.1145/3365610.3368413
Kinchin, Niamh. 2024. “‘Voiceless’: The Procedural Gap in Algorithmic Justice”. International Journal of Law and Information Technology 32: eaae024. https://doi.org/10.1093/IJLIT/EAAE024
Kolkman, Daan, Floris Bex, Nitin Narayan, and Manuella van der Put. 2024. “Justitia Ex Machina: The Impact of an AI System on Legal Decision-Making and Discretionary Authority”. Big Data and Society 11 (2): online. https://doi.org/10.1177/20539517241255101
Krause, Josua, Adam Perer, and Kenney Ng. 2016. “Interacting with Predictions: Visual Inspection of Black-Box Machine Learning Models”. In CHI’16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 5686-5697. May 7-12, San Jose, CA, United States. https://doi.org/10.1145/2858036.2858529
Laato, Samuli, Teemu Birkstedt, Matti Mantymaki, Matti Minkkinen, and Tommi Mikkonen. 2022. “AI Governance in the System Development Life Cycle: Insights on Responsible Machine Learning Engineering”. In CAIN’22: Proceedings of the 1st International Conference on AI Engineering: Software Engineering for AI, 113-123. May 16-24, Pittsburgh, United States. https://doi.org/10.1145/3522664.3528598
Marabelli, Marco, Sue Newell, and Valerie Handunge. 2021. “The Lifecycle of Algorithmic Decision-Making Systems: Organizational Choices and Ethical Challenges”. The Journal of Strategic Information Systems 30 (3): 101683. https://doi.org/10.1016/J.JSIS.2021.101683
Mayer, Roger C., James H. Davis, and F. David Schoorman. 1995. “An Integrative Model of Organizational Trust”. The Academy of Management Review 20 (3): 709-734. https://doi.org/10.2307/258792
Medvedeva, Masha, and Pauline McBride. 2023. “Legal Judgment Prediction: If You Are Going to Do It, Do It Right”. In Proceedings of the Natural Legal Language Processing Workshop 2023, 73-84. Singapore: Association for Computational Linguistics. https://doi.org/10.18653/V1/2023.NLLP-1.9
Meyer-Resende, Michael, and Marlene Straub. 2022. “The Rule of Law versus the Rule of the Algorithm”. Verfassungsblog, March 28. https://verfassungsblog.de/rule-of-the-algorithm/
Murrer, Adina. 2023. “Hesse and Brandenburg Cooperate on the AI Project ‘FraUKe’”. Hessen Ministry of Justice, November 13. https://hessen.de/presse/pressearchiv/hessen-und-brandenburg-kooperieren-beim-ki-projekt-frauke
NSW Government. 2024. “Bail and Accommodation Support Service”. Youth Justice NSW. https://www.nsw.gov.au/legal-and-justice/youth-justice/bail-and-accommodation-support-service
Pakulski, Jan. 2017. “Trust and Legitimacy”. Policy, Organization and Society 5 (1): 25-31. https://doi.org/10.1080/10349952.1992.11876776
Perona, Riccardo, and Yezid Carrillo de la Rosa. 2024. “Unveiling AI in the Courtroom: Exploring ChatGPT’s Impact on Judicial Decision-Making through a Pilot Colombian Case Study”. AI & Society 40: 2533-2540. https://doi.org/10.1007/s00146-024-01943-x
Popelier, Patricia, Monika Glavina, Federica Baldan, and Esther van Zimmeren. 2022. “A Research Agenda for Trust and Distrust in a Multilevel Judicial System”. Maastricht Journal of European and Comparative Law 29 (3): 351-374. https://doi.org/10.1177/1023263X221096026
Purves, Duncan, and Jeremy Davis. 2022. “Public Trust, Institutional Legitimacy, and the Use of Algorithms in Criminal Justice”. Public Affairs Quarterly 36 (3): 136-162. https://doi.org/10.5406/21520542.36.2.03
Rechtbank Den Haag. 2020. Case C-09-550982-HA ZA 18-388 ECLI:NL:RBDHA:2020:1878. https://uitspraken.rechtspraak.nl/details?id=ECLI:NL:RBDHA:2020:1878
Reisman, Dillon, Jason Schultz, Kate Crawford, and Meredith Whittaker. 2018. Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability. AI Now Institute, April 9. https://ainowinstitute.org/publication/algorithmic-impact-assessments-report-2
Richards, Neil M., and Woodrow Hartzog. 2015. “Taking Trust Seriously in Privacy Law”. Stanford Technology Law Review 19: 431-472. https://doi.org/10.2139/SSRN.2655719
Rouvroy, Antoinette, Thomas Berns, and Liz Carey-Libbrecht. 2013. “Algorithmic Governmentality and Prospects of Emancipation: Disparateness as a Precondition for Individuation through Relationships?”. Réseaux 177 (1): 163-196. https://shs.cairn.info/article/E_RES_177_0163?lang=en
Sgueo, Gianluca. 2020. The Practice of Democracy: A Selection of Civic Engagement Initiatives. Brussels: European Parliament Research Service (EPRS). https://www.europarl.europa.eu/thinktank/en/document/EPRS_STU(2020)651970
Stankovich, Miriam, Ivana Feldfeber, Yasmín Quiroga, Marianela Ciolfi Felice, and Vukosi Marivate. 2023. Global Toolkit on AI and the Rule of Law for the Judiciary. Paris: UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000387331
Svare, Helge, Anne Haugen Gausdal, and Guido Möllering. 2020. “The Function of Ability, Benevolence, and Integrity-Based Trust in Innovation Networks”. Industry and Innovation 27 (6): 585-604. https://doi.org/10.1080/13662716.2019.1632695
Terzidou, Kalliopi. 2023. “Automated Anonymization of Court Decisions: Facilitating the Publication of Court Decisions through Algorithmic Systems”. In ICAIL’23: Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, 297-305. June 19-23, Braga, Portugal. https://doi.org/10.1145/3594536.3595151
Veluwenkamp, Herman, and Frank Hindriks. 2024. “Artificial Agents: Responsibility & Control Gaps”. Inquiry, 1-25. https://doi.org/10.1080/0020174X.2024.2410995
Wallace, Anne, and Jane Goodman-Delahunty. 2021. “Measuring Trust and Confidence in Courts”. International Journal for Court Administration 12 (3): 1-17. https://doi.org/10.36745/IJCA.418
Williamson, Ben. 2015. “Governing Software: Networks, Databases and Algorithmic Power in the Digital Gover-nance of Public Education”. Learning, Media and Technology 40 (1): 83-105. https://doi.org/10.1080/17439884.2014.924527
Yassine, Saliha, Mustapha Esghir, and Ouafaa Ibrihich. 2023. “Using Artificial Intelligence Tools in the Judicial Domain and the Evaluation of Their Impact on the Prediction of Judgments”. Procedia Computer Science 220: 1021-1026. https://doi.org/10.1016/J.PROCS.2023.03.142
Yeung, Douglas, Inez Khan, Nidhi Kalra, and Osonde A. Osoba. 2021. Identifying Systemic Bias in the Acquisition of Machine Learning Decision Aids for Law Enforcement Applications. Santa Monica, CA: Rand Corporation. https://doi.org/10.7249/PEA862-1
Yeung, Karen. 2019. “Algorithmic Regulation: An Introduction”. In Algorithmic Regulation, edited by Karen Yeung and Martin Lodge, 1-18. Oxford: Oxford University Press. https://doi.org/10.1093/OSO/9780198838494.003.0001
Yu, Liangru, and Yi Li. 2022. “Artificial Intelligence Decision-Making Transparency and Employees’ Trust: The Parallel Multiple Mediating Effect of Effectiveness and Discomfort”. Behavioral Sciences 12 (5): 1-17. https://doi.org/10.3390/BS12050127
Zerilli, John, Alistair Knott, James Maclaurin, and Colin Gavaghan. 2019. “Algorithmic Decision-Making and the Control Problem”. Minds and Machines 29: 555-578. https://doi.org/10.1007/s11023-019-09513-7
Zimmermann, Annette, and Chad Lee-Stronach. 2022. “Proceed with Caution”. Canadian Journal of Philosophy 52 (1): 6-25. https://doi.org/10.1017/CAN.2021.17
Licencia
Derechos de autor 2025 Maria Lorena Flórez Rojas

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial-SinDerivadas 4.0.