LLMs require significant computational resources for training and inference, which can translate into higher operational costs, especially for large-scale document processing.
Specialized personnel is required to implement extraction models with LLMs which also needs to be factored into the overall cost of resulting document processing solution.
Because of these contigencies, the overall costs of LLMs are difficult to plan.

Processing large amounts of documents with an LLM takes a lot of time and computational resources. Depending on how much processing power is used (with the required GPUs it is expensive for LLMs) a single LLM extraction may take 15 seconds. On average a group insurance document may have about 50 extractions. Then 50 extraction x 15 seconds = 12.5 minutes. That is a long time waiting for a document to process.

For some document processing tasks, especially those requiring simple extraction of basic text, using an LLM might be an overkill. Deploying such a powerful model for simple tasks can introduce unnecessary complexity into the system and increase computational costs.

LLMs sometimes invent content: they generate answers that may appear plausible but lack factual basis, making them difficult to discern as false. This phenomenon is commonly referred to as hallucination.

LLMs lack transparency for several reasons:

• Black-box nature: Training data is mostly not disclosed making it difficult for users to understand how they arrive at specific conclusions or output. The intricate layers of neural networks involved in LLMs obscure the decision-making process.
• Lack of explainability: models do not provide explanations for their predictions or decisions. This makes it difficult to identify and understand potential biases and errors.

Pre-trained LLM models are very general and are required to be trained on a downstream data set to solve specific use cases. They generate text based on learned patterns from vast amounts of data, but they may produce outputs that are difficult to interpret or control. Users have no possibility to inspect or modify the inner workings of LLMs, which limits the possibility to customize or rectify issues.

The lack of control over text generated by LLMs may raise concerns about compliance, accuracy, and accountability. Deploying LLMs in document processing workflows may cause issues with privacy and security, particularly regarding data exposure or potential misuse of information. Local deployments of LLMs are a more secure option but also more expensive.

Training a productive extraction model for complex tasks like insurance policies using LLMs typically requires expertise in AI, machine learning, and data science.

Considering the complexity of the task, the need for extensive data preparation and model training, and the iterative nature of the process, implementing a reliable data extraction use case for insurance policies using an LLM can take a long time. The larger the model and the more fine-tuning required for your specific task, the longer training will take. Data preprocessing and cleaning can be time-consuming, especially for unstructured text data like insurance policies. This step is crucial for ensuring the quality of the training data and the performance of the model. The availability and speed of required GPUs can also impact training time.