Generative AI models offer limited transparency into the data that is used to train them and the date ranges of the data used. When using any generative AI tool or interface, always make efforts to carefully assess the accuracy, relevance, and veracity of the generative AI system’s outputs.
Generative models can also perpetuate biases present in their design, build and training data which can be amplified by the prompts input by the user. This has the potential to manifest in harmful ways including but not limited to, perpetuating stereotypes, reinforcing misinformation or creating unequal representation of different groups. When using generative AI, it is essential to be aware of this potential for bias, thoughtfully develop prompts, and critically review and assess the system’s output.
Below are some examples of different types of bias that might exist in generative AI, and should be taken into consideration when using various tools, or resources that utilize generative AI.
Machine bias refers to the biases that are present in the training data used to build the tools. Generative AI models learn from large human-generated datasets and will ingest the biases present in the text.
Confirmation bias when individuals seek information that confirms their existing beliefs and disregard alternative information. This can be demonstrated either in the training data or in the way that the prompt is written. When users seek information on a particular subject, the AI might selectively generate content that reinforces their viewpoints.
Selection bias when certain groups or perspectives are underrepresented or not present in the training data, the model will not have the information to generate comprehensive answers.
Contextual bias can happen when the model is unable to understand or interpret the context of a conversation or prompt accurately.
Linguistic bias Language models may exhibit preferences for certain dialects or languages, making it challenging for individuals who speak other dialects to access information or engage with AI interfaces.
Automation bias is the propensity for humans to favor suggestions from automated systems such as generative AI and to ignore contradictory information made without automation, even if it is correct.
Generative AI does sometimes provide citations/attributions to works it has used in providing its answers but these should be verified using external sources as there are examples of AI models creating “fake” references.
Below is an example of a generated citation to an article that does not exist.
Kallajoki M, et al. Homocysteine and bone metabolism. Osteoporos Int. 2002 Oct;13(10):822-7. PMID: 12352394