Sobre o uso de evidências e validade externa na avaliação das intervenções sociais: um olhar crítico
No. 105 (2021-01-01)Autor(es)
-
Juan David ParraUniversidad del Norte (Colombia)
Resumo
Objetivo/contexto: há cada vez mais vozes críticas sobre a adequação das práticas dominantes na avaliação das intervenções sociais (por exemplo, RCT) versus o objetivo de informar políticas baseadas em evidências. Este artigo enfoca dois elementos centrais de tal reflexão: i) a evidência, como um conceito e como resultado de um processo de raciocínio e ii) a noção de validade externa. Metodologia: o uso da literatura nas áreas de avaliação e filosofia do conhecimento me permite fazer uma desconstrução do conceito de causalidade nas ciências sociais. A partir desse exercício e da distinção entre teorias de causalidade sucessória e generativa, identifico critérios para examinar criticamente alguns postulados epistemológicos implícitos em argumentos de expoentes de técnicas de avaliação experimental. Conclusões: as técnicas experimentais, por si só, não informam as decisões sobre como investir recursos de forma eficiente. Apesar de sua força em quantificar possíveis efeitos causais, é necessário complementar a análise estatística contrafactual com formas de raciocínio qualitativo que conduzam à solução de questões sobre as causas efetivas por trás do resultado das intervenções sociais e os fatores de apoio que tornem possível pensar sobre a extrapolação de políticas ou programas sociais entre diferentes contextos. Originalidade: a literatura, principalmente em espanhol, sobre críticas e alternativas às técnicas de avaliação de impacto é escassa. Em vez de apresentar um resumo dos argumentos de outros autores, este artigo constrói uma narrativa coerente para repensar o papel da avaliação na sociedade.
Referências
Bernal, Raquel y XimenaPeña. 2011. Guía práctica para la evaluación de impacto. Bogotá: Ediciones Uniandes.
Blamey, Avril y MhairiMackenzie. 2007. “Theories of Change and Realistic Evaluation: Peas in a Pod or Apples and Oranges?”. Evaluation 13 (4): 439-455.
Bowles, Samuel. 2008. “Endogenous Preferences: The Cultural Consequences of Markets and Other Economic Institutions”. Journal of Economic Literature 36 (1): 75-111.
Brouselle, Astrid y Jean-MarieBurejeya. 2018. “Theory-based Evaluations: Framing the Existence of the New Theory Evaluation and the Rise of the 5th Generation”. Evaluation 24 (2): 153-168.
Cardozo, Myriam. 2013. “De la evaluación a la reformulación de políticas públicas”. Política y Cultura 40: 123-149.
Cartwright, Nancy. 2012. “Presidential Address: Will This Policy Work for You? Predicting Effectiveness Better: How Philoshophy Helps”. Philosophy of Science 79 (5): 973-989.
Cartwright, Nancy. 2013. “Knowing What We Are Talking About: Why Evidence Doesn’t Always Travel”. Evidence & Policy 9 (1): 97-112.
Cartwright, Nancy. 2017. “Single Case Causes: What is Evidence and Why”. En Philosophy of Science in Practice: Nancy Cartwright and the Nature of Scientific Reasoning, editado por Hsiang-KeChao y JulianReiss, 11-24. Cham: Springer.
Cartwright, Nancy y JeremieHardie. 2012. Evidence-Based Policy: A Practical Guide to Doing It Better. Oxford: Oxford University Press.
Cartwright, Nancy y JeremieHardie. 2017. “Predicting What Will Happen When You Intervene”. Clinical Social Work Journal 45 (3): 270-279.
Danermark, Berth, MatsEkström, LiselotteJakobsen y Jan ChKarlsson. 2002. Explaining Society: Critical Realism in the Social Sciences. Nueva York: Psychology Press.
Deaton, Angus. 2010. “Instruments, Randomization, and Learning about Development”. Journal of Economic Literature 48: 424-455.
Deaton, Angus y NancyCartwright. 2018. “Understanding and Misunderstanding Randomized Controlled Trials”. Social Science & Medicine 210: 2-21.
Ellis, George. 2005. “Physics, Complexity and Causality”. Nature 435 (743).
Gertler, Paul J., SebastiánMartínez, PatrickPremand, Laura B.Rawlings y Christel M. J.Vermeersch. 2011. Evaluación de impacto en la práctica. Washington D. C.: Banco Mundial.
Goodman, Lisa, DeborahEpstein y CrisSullivan. 2018. “Beyond the RCT: Integrating Rigor and Relevance to Evaluate the Outcomes of Domestic Violence Programs”. American Journal of Evaluation 39 (1): 58-70.
Greenhalgh, Trish y CraigRussell. 2009. “Evidence-Based Policymaking: A Critique”. Perspectives in Biology and Medicine 52 (2): 304-318.
Hallam, Susan. 2010. “The Power of Music: Its Impact on the Intellectual, Social and Personal Development of Children and Young People”. International Journal of Music Education 28 (3): 269-289.
Harman, Graham. 2018. Object-Oriented Ontology. Londres: Pelican Books.
Hausmann, Ricardo. 2016. “El problema con las políticas basadas en evidencia”. La Nación, 3 de marzo. https://www.nacion.com/opinion/foros/el-problema-con-las-politicas-basadas-en-evidencia/ISW3U6CZYNHEVND43YO2XZWFBY/story/
Head, Brian. 2016. “Toward More ‘Evidence‐Informed’ Policy Making?”. Public Administration Review 76 (3): 472-484.
Henrich, Joseph, RobertBoyd, SamuelBowles, ColinCamerer, ErnstFehr, HerbertGintis y RichardMcElreath. 2001. “In Search of Homo Economicus: Behavioral Experiments in 15 Small-Scale Societies”. American Economic Review 91 (2): 73-78.
Joyce, Kathryn y NancyCartwright. 2020. “Bridging the Gap Between Research and Practice: Predicting What Will Work Locally”. American Educational Research Journal 57 (3): 1045-1082.
Krause, Philipp y GonzaloHernández. 2020. “Commentary: From Experimental Findings to Evidence-Based Policy”. World Development 127. https://doi.org/10.1016/j.worlddev.2019.104812
Krauss, Alexander. 2015. “The Scientific Limits of Understanding the (Potential) Relationship Between Complex Social Phenomena: The Case of Democracy and Inequality”. Journal of Economic Methodology 23 (1): 97-109.
Krauss, Alexander. 2018. “Why All Randomised Controlled Trials Produce Biased Results”. Annals of Medicine 50 (4): 312-322.
Lawson, Tony. 2009. “Applied Economics, Contrast Explanation and Asymmetric Information”. Cambridge Journal of Economics 33 (3): 405-419.
Manzano, Ana. 2010. “El análisis del contexto local en un programa multidisciplinario (sanidad y servicios sociales) usando el enfoque de la evaluación realista”. E-valuacion 3 (10): 24-27.
Masino, Serena y MiguelNiño-Zarazúa. 2016. “What Works to Improve the Quality of Student Learning in Developing Countries?”. International Journal of Educational Development 48: 53-65.
Monaghan, Mark, RayPawson y KateWicker. 2012. “The Precautionary Principle and Evidence-Based Policy Making”. Evidence & Policy 8 (2): 171-191.
Munro, Eileen, NancyCartwright, JeremyHardie y EleonoraMontuschi. 2016. Improving Child Safety: Deliberation, Judgement and Empirical Research. Durham: Centre for Humanities Engaging Science and Society (Chess). Philosophy Department, Durham University.
Parra, Juan David. 2013. “Preferencias endógenas, prosocialidad y políticas públicas”. Divergencia 15: 64-71.
Parra, Juan David. 2016. “Realismo crítico: una alternativa en el análisis social”. Sociedad y Economía 31: 215-238.
Parra, Juan David. 2017. “¿Qué funciona, para quién, en qué aspectos, hasta qué punto, en qué contexto y cómo? Una introducción a la evaluación realista y sus métodos”. Economía & Región 11 (2): 11-44.
Parra, Juan David. 2018. “Critical Realism and School Effectiveness Research in Colombia: The Difference It Should Make”. British Journal of Sociology of Education 39 (1): 107-125.
Parra, Juan David. 2019. “El arte del muestreo cualitativo y su importancia para la evaluación y la investigación de políticas públicas: una aproximación realista”. Opera 25: 119-136.
Pawson, Ray. 2013. The Science of Evaluation: A Realist Manifesto.Londres: Sage.
Pawson, Ray y NickTilley. 1997. Realistic Evaluation. Londres; Nueva Delhi: Thousand Oaks.
Pawson, Ray, GeoffWong y LesleyOwen. 2011. “Known Knowns, Known Unknowns, Unknown Unknowns: The Predicament of Evidence-Based Policy”. American Journal of Evaluation 32 (4): 518-546.
Porter, Sam, TraceyMcConnell y JoanneReidcor. 2017. “The Possibility of Critical Realist Randomised Controlled Trials”. Trials 18: 133.
Reiss, Julian. 2018. “Against External Validity”. Synthese 196 (8): 3103-3121.
Saltelli, Andrea y MarioGiampietro. 2017. “What Is Wrong with Evidence Based Policy, and How Can It Be Improved?”. Futures 91: 62-71.
Van Belle, Sara, Geoff Wong, Gill Westhorp, Mark Pearson, Nick Emmel, AnaManzano y BrunoMarchal. 2016. “Can ‘Realist’ Randomised Controlled Trials Be Genuinely Realist?”. Trials 17 (1): 2-6.
Verger, Antoni, XavierBonal y AdriánZancajo. 2016. “What Are the Role and Impact of Public-Private Partnerships in Education? A Realist Evaluation of the Chilean Education Quasi-Market”. Comparative Education Review 60 (2): 223-248.
Licença
Este trabalho está licenciado sob uma licença Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.