{"id":5406,"date":"2026-08-30T18:20:38","date_gmt":"2026-08-30T09:20:38","guid":{"rendered":"https:\/\/donguri3.net\/server-tech\/openwebui-markitdown-mcp-integration-2\/"},"modified":"2026-08-30T18:20:39","modified_gmt":"2026-08-30T09:20:39","slug":"openwebui-markitdown-mcp-integration","status":"publish","type":"post","link":"https:\/\/donguri3.net\/en\/server-tech\/ai-local-llm\/openwebui-markitdown-mcp-integration\/","title":{"rendered":"Document Analysis with Open WebUI and MarkItDown"},"content":{"rendered":"

In recent times, building local LLM environments using Ollama and Podman has become extremely easy.<\/a> However, when having AI read rich documents such as PDFs and Excel files, high hurdles still remain. While many LLMs excel at handling text data, there is a limit to their accuracy when directly interpreting binary-format files.<\/p>\n

As a means to solve this problem, Microsoft’s “MarkItDown” has been gaining attention. This is a tool that converts PDFs and Office files into Markdown format, which is easiest for AI to understand. By seamlessly integrating this with Open WebUI\u2014a chat interface\u2014we aimed to build an environment where local materials can be analyzed instantly.<\/p>\n

Full Automation of the Conversion Process and Pursuit of Accuracy<\/h2>\n

The objective of this build is to complete a pipeline where simply drag-and-dropping a PDF into the Open WebUI chat screen executes high-precision Markdown conversion via MarkItDown, allowing the LLM to instantly summarize and analyze the contents.<\/p>\n

We set the following two quantitative goals:<\/p>\n

    \n
  1. Reduction of manual operations: Eliminate text extraction and copy-and-paste tasks using external tools by 100%, aiming for completion entirely within the chat UI.<\/li>\n
  2. Improvement of analysis accuracy: Compared to text extraction using Open WebUI’s built-in standard parser, routing through MarkItDown\u2014which excels in retaining charts, tables, and structured data\u2014significantly boosts reading comprehension accuracy for complex data sheets and similar files.<\/li>\n<\/ol>\n

    The Barriers of “UUID Renaming” and “Caching” That Hinder Development<\/h2>\n

    As implementation progressed, we faced two technical challenges unique to containerized environments.<\/p>\n

      \n
    1. The “loss of file storage path and name.” When a file is dropped into the Open WebUI chat screen, it is saved on the server (inside the container) with a filename assigned a random UUID (e.g., da99462c…_datasheet.pdf). Because the LLM cannot recognize this random string, it cannot pass the correct file path to MarkItDown, causing analysis errors.<\/li>\n
    2. Podman’s “missing dependencies due to build cache.” Analyzing PDFs with MarkItDown requires installing an extension library called markitdown[all]. However, even when modifying requirements.txt afterward, the caching function kicked in, resulting in a phenomenon where the heavy PDF analysis package was actually not installed.<\/li>\n<\/ol>\n

      Build Method: Breakthrough via Addition of MCPO Container and Wildcard Search<\/h2>\n

      To address these challenges, we sought solutions from both the infrastructure configuration and Python script perspectives.<\/p>\n

      File Structure<\/h3>\n
      Working Directory\/\n \u251c\u2500\u2500 docker-compose.yml\n \u251c\u2500\u2500 webui-data\/             # *Automatically generated (Open WebUI data storage destination)\n \u2514\u2500\u2500 mcpo-markitdown\/        # Folder containing the following three files\n      \u251c\u2500\u2500 requirements.txt\n      \u251c\u2500\u2500 Dockerfile\n      \u2514\u2500\u2500 server.py<\/pre>\n
      Sharing Volume Mounts and Forced Rebuild<\/h3>\n
      To ensure that the directory where Open WebUI saves files and the directory read by the MCPO container are identical, we added the following configuration to docker-compose.yml. Furthermore, to overcome the caching barrier, we forcefully rebuilt the container using podman-compose build –no-cache, reliably installing the PDF analysis library.<\/p>\n
      \nservices:\n  ollama:\n    image: docker.io\/ollama\/ollama:0.20.6\n    container_name: ollama\n    ports:\n      - "11434:11434"\n    environment:\n      - "OLLAMA_KEEP_ALIVE=-1"  # Keep in VRAM with -1\n      - "OLLAMA_MAX_LOADED_MODELS=2"    # Allow loading 2 models simultaneously\n    volumes:\n      - ollama_data:\/root\/.ollama\n    devices:\n      - nvidia.com\/gpu=all\n  open-webui:\n    image: ghcr.io\/open-webui\/open-webui:main\n    container_name: open-webui\n    ports:\n      - "3000:8080"\n    volumes:\n      # Mount host-side .\/webui-data to container's \/app\/backend\/data\n      - .\/webui-data:\/app\/backend\/data\n    restart: always\n\n  mcpo-markitdown:\n    build: .\/mcpo-markitdown\n    container_name: mcpo-markitdown\n    ports:\n      - "8000:8000"\n    volumes:\n      # Mount the exact same host directory as Open WebUI as "read-only (ro)"\n      - .\/webui-data:\/app\/backend\/data:ro\n    restart: always\n<\/pre>\n
      Python Dependency Packages<\/h3>\n
      These are the Python dependency packages.<\/p>\n
      mcpo\nmarkitdown[all]\nmcp<\/pre>\n
      Dockerfile<\/h3>\n
      This Dockerfile avoids the trap of the build cache and ensures the heavy libraries for PDF analysis are installed.<\/p>\n
      FROM python:3.12-slim\n\nWORKDIR \/app\n\n# First, copy requirement files and perform basic installation\nCOPY requirements.txt .\nRUN pip install --no-cache-dir -r requirements.txt\n\n# A reassuring extra step to ensure the [all] option (PDF and Office parsing engines) is installed\nRUN pip install --no-cache-dir "markitdown[all]"\n\n# Copy script group\nCOPY . .\n\n# Convert standard I\/O to Web API (OpenAPI) via mcpo and start up\nCMD ["mcpo", "--host", "0.0.0.0", "--port", "8000", "--", "python", "server.py"]<\/pre>\n
      Path Auto-Identification Script Using Wildcard Search<\/h3>\n
      To solve the UUID issue, we modified the script (server.py) running on the MCPO side. We implemented a mechanism where the LLM is only made to pass the “original filename,” and Python’s glob module is used to perform a wildcard search within the upload directory.<\/p>\n
      import os\nimport glob\nfrom mcp.server.fastmcp import FastMCP\nfrom markitdown import MarkItDown\n\n# Initialize MCP Server and MarkItDown\nmcp = FastMCP("MarkItDown Converter")\nmd = MarkItDown()\n\n@mcp.tool()\ndef convert_to_markdown(filename: str) -> str:\n"""\nConverts the file uploaded to the chat into Markdown.\nArgs:\nfilename: The name of the uploaded file, or a part of it (e.g., datasheet.pdf)\n"""\n# Upload destination path of the shared mount directory in docker-compose\nupload_dir = "\/app\/backend\/data\/uploads"\n\n# Perform a wildcard search among files assigned with UUIDs for one containing the filename\nsearch_pattern = os.path.join(upload_dir, f"*{filename}*")\nmatched_files = glob.glob(search_pattern)\n\nif not matched_files:\nreturn f"Error: No file containing '{filename}' was found in the upload folder."\n\n# If multiple files with the same name exist, target the most recently uploaded one\ntarget_file = max(matched_files, key=os.path.getmtime)\n\ntry:\n# Execute conversion with MarkItDown using the discovered full path\nresult = md.convert(target_file)\nreturn result.text_content\n\nexcept Exception as e:\nreturn f"Error: An issue occurred during conversion: {str(e)}"\n\nif __name__ == "__main__":\n# Since mcpo wraps and communicates, it internally starts via stdio\nmcp.run()<\/pre>\n
      This allows users to simply instruct, “Please read this datasheet.pdf,” enabling the system to automatically identify the full path with the UUID and execute the analysis.<\/p>\n
      Startup and Integration Procedure (Grand Finale)<\/h3>\n
      Once the file placement is complete, run the following commands to perform a clean build without using the cache and start up the services. (*Run podman-compose or docker-compose according to your environment)<\/p>\n
      podman-compose down\npodman-compose build --no-cache\npodman-compose up -d<\/pre>\n
      Execution Results<\/h2>\n
      We loaded the ESP32-WROOM-32E Datasheet and verified whether it could be analyzed successfully.<\/p>\n
      The chat response’s sources include the uploaded file and a source named tool_convert_to_markdown_post, where the latter is the Markdown converted by MarkItDown.<\/p>\n
      An example of extracting information from the cover page is shown below:<\/p>\n
      \"ESP32\u00adWROOM\u00ad32E<\/a><\/p>\n
      ESP32\u00adWROOM\u00ad32E\nESP32\u00adWROOM\u00ad32UE\nDatasheet\n\n2.4 GHz\nWi\u00adFi + Bluetooth\u00ae\n+ Bluetooth\nLE module\nXtensa\u00ae\nBuilt around ESP32 series of SoCs, dual\u00adcore 32\u00adbit LX6 microprocessor\n4\/8\/16\nMB flash available\n---------\n------------------\n--------------\n-----------------\n26 GPIOs,\nrich set\nof peripherals\nOn\u00adboard\nPCB antenna\nor external\nantenna connector<\/pre>\n
      Evolution Into a Practical Personal AI Assistant<\/h2>\n
      Through this setup, an environment has been established where simply dropping complex English datasheets like the ESP32 into the chat allows local models like Qwen3 or Gemma3 to accurately summarize the contents based on the structured data parsed by MarkItDown.<\/p>\n
      The integration of Open WebUI and MCP tools transforms a simple chatbot into an agent that autonomously utilizes tools. The “file search method that hides UUIDs from the user” established this time is applicable not only to MarkItDown but to any MCP server handling local files. Even without a vast space, I am convinced that accumulating such techniques is the key to unlocking infinite possibilities from a limited space like a storeroom (Nando).<\/p>\n","protected":false},"excerpt":{"rendered":"
      In recent times, building local LLM environments using Ollama and Podman has become extremely easy. However, w […]<\/p>\n","protected":false},"author":4,"featured_media":3001,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"en_US","_original_post":"https:\/\/donguri3.net\/?p=3000","footnotes":""},"categories":[1169],"tags":[18,19,1126,1125,47,291,931,10,1127,1040],"class_list":["post-5406","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-local-llm","tag-docker","tag-docker-compose","tag-llm","tag-podman","tag-python","tag-yaml","tag-931","tag-server","tag-1040","en-US"],"_links":{"self":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/5406","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/comments?post=5406"}],"version-history":[{"count":1,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/5406\/revisions"}],"predecessor-version":[{"id":5409,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/posts\/5406\/revisions\/5409"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/media\/3001"}],"wp:attachment":[{"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/media?parent=5406"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/categories?post=5406"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/donguri3.net\/wp-json\/wp\/v2\/tags?post=5406"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}