Desarrollo de una interfaz para la consulta y visualización de datos de salud pública de los resúmenes estadísticos mensuales del MINSAL.
Loading...
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
Universidad de Concepción
Abstract
El objetivo de esta memoria es desarrollar una interfaz para la consulta y visualización de los datos de salud pública contenidos en los Resúmenes Estadísticos Mensuales (REM) emitidos por el Ministerio de Salud de Chile (MINSAL). Esta
memoria está patrocinada por la empresa privada HealthTracker.
El proyecto se enmarca en la necesidad de mejorar la gestión y análisis de los datos de salud, los cuales se encuentran dispersos y en distintos formatos, dificultando su utilización. Para abordar este desafío, se plantearon varias etapas que incluyen la descarga y consolidación de archivos REM, el análisis de los diccionarios de datos, la generación de un formato decodificador en JSON, y la implementación de una interfaz para la consulta de estos datos, la cual busca mejorar la accesibilidad y manejo de los datos de la Serie A de los REM, que abarcan desde 2009 hasta 2023.
Durante el desarrollo, se utilizaron técnicas como el método de chunks para manejar grandes volúmenes de datos, lo que permitió procesar múltiples cortes que conformaban hasta 500 MB cada uno, sin agotar la memoria del sistema. Además, se emplearon herramientas avanzadas de LLM como Chat GPT 4-O para la generación de descriptores y junto a métodos de cruce para la normalización de datos, lo que agilizó el proceso de decodificación.
Los resultados obtenidos incluyen una base de datos consolidada de los archivos REM de la Serie A, un formato decodificador en JSON que facilita la consulta y análisis de los datos a través de los años, y una interfaz interactiva desarrollada
con Streamlit, que permite realizar consultas SQL.
Se lograron los objetivos, como la integración de datos de diferentes años y formatos, y la implementación de filtros avanzados para asegurar consultas. Significando un desarrollo en la accesibilidad de los datos abiertos de salud pública en Chile.
The objective of this thesis is to develop an interface for querying and visualizing public health data contained in the Monthly Statistical Summaries (REM) issued by the Ministry of Health of Chile (MINSAL). This thesis is sponsored by the private company HealthTracker. The project addresses the need to improve the management and analysis of health data, which are currently dispersed and in various formats, making their use challenging. To tackle this challenge, several stages were outlined, including the downloading and consolidation of REM files, the analysis of data dictionaries, the generation of a JSON decoding format, and the implementation of an interface for querying this data. The aim is to enhance the accessibility and management of the Series A REM data, covering the years 2009 to 2023. During the development, techniques such as the chunk method were used to handle large volumes of data, allowing the processing of multiple segments, each up to 500 MB, without exhausting system memory. Additionally, advanced LLM tools like Chat GPT 4-O were employed for descriptor generation, along with cross-referencing methods for data normalization, which streamlined the decoding process. The results include a consolidated database of Series A REM files, a JSON decoding format that facilitates the querying and analysis of data over the years, and an interactive interface developed with Streamlit that allows for SQL queries. The objectives were achieved, including the integration of data from different years and formats, and the implementation of advanced filters to ensure precise queries, marking a significant advancement in the accessibility of public health open data in Chile.
The objective of this thesis is to develop an interface for querying and visualizing public health data contained in the Monthly Statistical Summaries (REM) issued by the Ministry of Health of Chile (MINSAL). This thesis is sponsored by the private company HealthTracker. The project addresses the need to improve the management and analysis of health data, which are currently dispersed and in various formats, making their use challenging. To tackle this challenge, several stages were outlined, including the downloading and consolidation of REM files, the analysis of data dictionaries, the generation of a JSON decoding format, and the implementation of an interface for querying this data. The aim is to enhance the accessibility and management of the Series A REM data, covering the years 2009 to 2023. During the development, techniques such as the chunk method were used to handle large volumes of data, allowing the processing of multiple segments, each up to 500 MB, without exhausting system memory. Additionally, advanced LLM tools like Chat GPT 4-O were employed for descriptor generation, along with cross-referencing methods for data normalization, which streamlined the decoding process. The results include a consolidated database of Series A REM files, a JSON decoding format that facilitates the querying and analysis of data over the years, and an interactive interface developed with Streamlit that allows for SQL queries. The objectives were achieved, including the integration of data from different years and formats, and the implementation of advanced filters to ensure precise queries, marking a significant advancement in the accessibility of public health open data in Chile.
Description
Tesis presentada para optar al título de Ingeniero/a Civil Biomédica
Keywords
Interfaces de computadores, Visualización Procesamiento de datos, Salud pública