Loss of control and “black box”
This section aims to highlight the nature of the “black box” in many Artificial Intelligence (AI) systems and analyze how this feature can generate a sense of loss of control among employers and employees.
We will explore the concept of the “black box”, which can be defined as an intrinsic feature of many Artificial Intelligence systems. This characteristic not only makes it challenging to understand decision-making processes but also poses problems in predicting both their choices and outcomes (Bathaee, 2018).
The term “black box” is used when an algorithm is so complex that it is incomprehensible to humans yet it is trusted to generate a correct response to a given input. This term reflects the lack of transparency of its inner workings, even if its performance is reliable (Giró & Gil, 2022).
In the previous chapter, we talked about how neural networks work, but the question here is: where does a neural network stop? We want to answer this question because once a final neural network has been trained, it becomes a black box. This can lead to problems, as it is not always possible to identify the cause of an unexpected result produced by the neural network (Scheire, 2023).
Figure 1 shows that the degree of knowledge about the operation of decision-making “black boxes” can vary at different levels. At a first level of transparency, it would be possible to access the code of the algorithms responsible for processing the data. At a second level, it would be likely to understand the behavior of the algorithm, particularly in the case of those based on machine learning, which would enable its operation to be analyzed. Finally, at a third level, it would be possible to know the justification for each decision taken by the algorithm in a given context (Cotino Hueso & Castellanos Claramunt, 2022).
“Black-box” models, such as neural networks, are characterized by their high accuracy and ability to handle complex use cases. However, due to their fuzzy structure, they cannot be easily interpreted. This lack of explicability may lead to their replacement by white-box models, which are directly interpretable. This is particularly relevant in critical, high-risk applications, where it is essential to guarantee the stability of their operation, or in regulated sectors, where transparency in decision-making is an indispensable requirement (Cotino Hueso & Castellanos Claramunt, 2022).
The explicability of “black box” models remains an ongoing research challenge. One of the current strategies for improving their understanding is to use algorithms that examine the neural network, evaluating the inputs and corresponding outputs it generates. This analysis enables the construction of a directly interpretable model that helps to explain, albeit not wholly, the behavior of the “black box” and its decision-making process (Cotino Hueso & Castellanos Claramunt, 2022).
The explicability of “black box” models remains an ongoing research challenge.
One of the current strategies for improving their understanding is to use algorithms that examine the neural network, evaluating the inputs and corresponding outputs it generates
. This analysis enables the construction of a directly interpretable model that helps to explain, albeit not wholly, the behavior of the “black box” and its decision-making process (Cotino Hueso & Castellanos Claramunt, 2022).
Some AI systems operate as “black boxes”, meaning that their internal processes are not easily understood by users or even by their developers. This can raise concerns about the reliability, ethics, and accountability of their decisions. To solve this problem, it is essential to apply explicability measures, i.e., strategies aimed at better understanding how these systems work and how they arrive at their results:
- Firstly, traceability involves recording the data and processes involved in AI decision-making to track its progress and detect any errors.
- Secondly, auditability enables experts or regulators to examine the system to check that it complies with the principles of transparency and fairness.
- Thirdly, transparent communication by clearly informing users of AI’s capabilities and limitations, avoiding exaggerations or false expectations about what it can do. These measures aim to make AI systems more transparent, accountable, and trustworthy, particularly in areas such as healthcare and justice, where their decisions can have a significant impact.
References
- BATHAEE, Y. (2018). “The Artificial Intelligence Black Box and the Failure of Intent and Causation”. En Harvard Journal of Law & Technology, vol. 31(2) pág. 905. Véase también, PARALAMENTO EUROPEO (2020). Resolución 2020/2012(INL), Op. cit., apartado 23.
- Costino Hueso, L., & Castellanos Claramunt, J. (Eds.). (2022). Transparencia y explicabilidad de la inteligencia artificial. Tirant lo Blanch.
- Giró Gràcia, X., & Sancho Gil, J. M. (2022). La Inteligencia Artificial en la educación: Big data, cajas negras y solucionismo tecnológico.
0 Comments