Optimización del proceso de clasificación, distribución y orden de solicitudes de clientes mediante modelos de lenguaje en Andritz Chile.
Loading...
Date
2025
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Universidad de Concepción
Abstract
Esta memoria de título presenta el diseño e implementación de una solución para optimizar la organización de solicitudes de clientes en el área de ventas de repuestos de Andritz Chile. El trabajo se divide en dos ejes principales: el desarrollo de un sistema automatizado que mejora el proceso operativo actual y un estudio experimental que evalúa el desempeño de distintos modelos de lenguaje y técnicas de prompting en la extracción de información técnica.
El problema radica en la alta carga operativa asociada a la revisión manual de solicitudes, provocando pérdida de tiempo y duplicación de esfuerzos, que se puede dedicar a tareas más relevantes.
La solución desarrollada integra herramientas como Power Automate, Python, SharePoint y modelos de lenguaje (LLMs), automatizando la detección de correos, extracción de datos en formato XML y su visualización en Excel complementado con datos históricos desde archivos maestros.
Paralelamente, se comparan cuatro modelos de lenguaje (GPT-4, Gemini 2.0 Flash, LLaMA 3 8B y Mistral 7B) combinados con cuatro técnicas de prompting (uso de roles, Chain of Thought, formato estructurado y ejemplos (few-shot)), evaluados con documentos reales y métricas como exactitud, consistencia, percepción de exactitud, tokens generados, costo y tiempo de inferencia.
Los resultados destacan a GPT-4 como el modelo más preciso (98,5 %), con una consistencia del 100 % y una percepción de exactitud del 95 %, aunque con el mayor costo. Gemini 2.0 Flash se consolida como la alternativa más eficiente, alcanzando un rendimiento del 94,6 %, misma consistencia, percepción del 89,2 % y costos mínimos. Por otro lado, los modelos de código abierto ejecutados localmente (LLaMA 3 8B y Mistral 7B) ofrecen ventajas en autonomía y costos directos nulos, pero presentan limitaciones claras en precisión, estabilidad y velocidad. En cuanto a las técnicas de prompting, las más efectivas fueron el formato estructurado y los ejemplos, que permitieron guiar mejor la salida de los modelos mejorando la exactitud y percepción subjetiva.
Finalmente, el sistema fue implementado utilizando el modelo Gemini 2.0 Flash en conjunto con la técnica de prompting de formato estructurado, combinación que ofreció un excelente equilibrio entre exactitud, velocidad y costo. Se concluye que es posible automatizar de forma confiable procesos operativos complejos mediante el uso de LLMs, siempre que se cuente con una arquitectura modular, modelos bien seleccionados y una adecuada ingeniería de prompts. Esta experiencia abre el camino hacia una adopción más amplia de inteligencia artificial en procesos industriales cotidianos.
This thesis presents the design and implementation of a solution to optimize the organization of customer requests in the spare parts sales area of Andritz Chile. The work is divided into two main areas: the development of an automated system that improves the current operational process, and an experimental study that evaluates the performance of different language models and prompting techniques in the extraction of technical information. The problem lies in the high operational burden associated with manually reviewing applications, causing wasted time and duplication of effort that could be devoted to more important tasks. The solution developed integrates tools such as Power Automate, Python, SharePoint, and language models (LLMs), automating email detection, data extraction in XML format, and visualization in Excel, complemented by historical data from master files. At the same time, four language models (GPT-4, Gemini 2.0 Flash, LLaMA 3 8B, and Mistral 7B) are compared in combination with four prompting techniques (use of roles, Chain of Thought, structured format, and examples (few-shot)), evaluated with real documents and metrics such as accuracy, consistency, perceived accuracy, generated tokens, monetary cost, and inference time. The results highlight GPT-4 as the most accurate model (98.5%), with 100% consistency and 95% perceived accuracy, although it has the highest cost. Gemini 2.0 Flash stands out as the most efficient alternative, achieving 94.6% performance, the same consistency, 89.2% perception, and minimal costs. On the other hand, locally executed open-source models (LLaMA 3 8B and Mistral 7B) offer advantages in autonomy and zero direct costs, but have clear limitations in accuracy, stability, and speed. In terms of prompting techniques, the most effective were structured format and examples, which allowed for better guidance of model output and improved accuracy and subjective perception. Finally, the system was implemented using the Gemini 2.0 Flash model in conjunction with the structured format prompting technique, a combination that offered an excellent balance between accuracy, speed, and cost. It is concluded that it is possible to reliably automate complex operational processes using LLMs, provided that a modular architecture, well-selected models, and adequate prompt engineering are in place. This experience paves the way for wider adoption of artificial intelligence in everyday industrial processes.
This thesis presents the design and implementation of a solution to optimize the organization of customer requests in the spare parts sales area of Andritz Chile. The work is divided into two main areas: the development of an automated system that improves the current operational process, and an experimental study that evaluates the performance of different language models and prompting techniques in the extraction of technical information. The problem lies in the high operational burden associated with manually reviewing applications, causing wasted time and duplication of effort that could be devoted to more important tasks. The solution developed integrates tools such as Power Automate, Python, SharePoint, and language models (LLMs), automating email detection, data extraction in XML format, and visualization in Excel, complemented by historical data from master files. At the same time, four language models (GPT-4, Gemini 2.0 Flash, LLaMA 3 8B, and Mistral 7B) are compared in combination with four prompting techniques (use of roles, Chain of Thought, structured format, and examples (few-shot)), evaluated with real documents and metrics such as accuracy, consistency, perceived accuracy, generated tokens, monetary cost, and inference time. The results highlight GPT-4 as the most accurate model (98.5%), with 100% consistency and 95% perceived accuracy, although it has the highest cost. Gemini 2.0 Flash stands out as the most efficient alternative, achieving 94.6% performance, the same consistency, 89.2% perception, and minimal costs. On the other hand, locally executed open-source models (LLaMA 3 8B and Mistral 7B) offer advantages in autonomy and zero direct costs, but have clear limitations in accuracy, stability, and speed. In terms of prompting techniques, the most effective were structured format and examples, which allowed for better guidance of model output and improved accuracy and subjective perception. Finally, the system was implemented using the Gemini 2.0 Flash model in conjunction with the structured format prompting technique, a combination that offered an excellent balance between accuracy, speed, and cost. It is concluded that it is possible to reliably automate complex operational processes using LLMs, provided that a modular architecture, well-selected models, and adequate prompt engineering are in place. This experience paves the way for wider adoption of artificial intelligence in everyday industrial processes.
Description
Tesis presentada para optar al título de Ingeniero/a Civil Industrial.
Keywords
Simplificación de procesos industriales, Automatización, Inteligencia artificial