Local LLM Notepad: A Portable Tool for Running Local Large Language Models Offline

Local LLM Notepad is an open source offline application that allows users to run Local Large Language Models on any Windows computer via a USB device without an Internet connection and without installation. Users can simply copy a single executable (EXE) and a model file (e.g. GGUF format) to a USB flash drive for chatting, brainstorming, or document drafting anytime, anywhere. It is designed with the goal of simplicity and ease of use, without the need for administrator privileges, and is suitable for users of varying skill levels in a variety of scenarios, such as client access or temporary workstations. The tool emphasizes portability and privacy protection, and all data processing is done locally without relying on cloud or API services. The interface is simple and intuitive, with a dual-window layout, support for cue word highlighting and JSON format export dialogs, making it easy for users to manage and trace content.

Function List

• Running Large Language Models Offline: No internet connection required, loads GGUF format models, suitable for privacy-sensitive scenarios.
• Portable design: Single file EXE, no installation required, run directly from USB flash drive.
• Dual Window Interactive Interface: Enter the prompt word below and the model answer will be displayed in real time above, the interface is simple and free of unnecessary decorations.
• Cue word highlighting and traceability: Words entered by the user are automatically bolded and underlined in the model answer, Ctrl+Click to see the history of the relevant prompt word.
• JSON dialog export: Supports one-click export of dialog records for easy migration across devices.
• Flexible model switching: Load different GGUF models through "File > Select Model" to meet various needs.
• High performance: Supports fast CPU models (e.g., 0.8GB models) and achieves generation speeds of about 20 tokens/second on common hardware.

Using Help

Installation and Preparation

Local LLM Notepad does not require installation in the traditional sense and is designed to be plug and play. Below is a detailed preparation process:

1. download program: Visit the Releases page of the GitHub repository (https://github.com/runzhouye/Local_LLM_Notepad) to download the latest version of the Local_LLM_Notepad-portable.exeThe
2. Getting the model file: Download compatible model files in GGUF format, e.g., the recommended gemma-3-1b-it-Q4_K_M.gguf(~0.8GB). Available at Hugging Face or other modeling communities.
3. Copy to USB flash drive: Copy the downloaded EXE files and GGUF model files to a USB flash drive or other removable storage device.
4. running program: Double-click on any Windows computer Local_LLM_Notepad-portable.exe Startup. The first time the program runs, it loads the model into memory (RAM), which may take a few seconds; subsequent operations will be smoother.

Main Functions

1. Operation and interaction

After launching the program, the interface is divided into two parts:

• bottom half: Enter a prompt (prompt), supporting text, questions or instructions.
• top half: Displays the model's answers in real time, with a generation speed of about 20 tokens/second (depending on hardware performance).
  Users can directly enter a question or task, such as "Write me an email" or "Explain quantum mechanics", and the model will instantly generate the answer. The interface is intuitive with no complex menus, making it easy to get started.

2. Cue highlighting and tracing

One of the features of Local LLM Notepad is prompt word highlighting. Words or numbers that the user enters in the cue word are highlighted in the model response with abold underlineFormat. For example, if you enter "technology trends in 2025", the part of the model response that refers to "2025" or "technology" will be highlighted. The user can highlight the part of the model response that refers to "2025" or "technology" by pressing and holding the Ctrl and clicking on the highlighted word opens a side window to view all historical cue words containing that word. This feature is ideal for scenarios where you need to validate model output or trace information back, such as data extraction or summary verification.

3. Model switching

If you need to change the model, you can click on the menu bar's 文件 > 选择模型If you want to use the program, select another GGUF format model file on the USB flash drive. The program will automatically load the new model without rebooting. It is recommended that first-time users use the recommended lightweight models (e.g., 0.8 GB of gemma-3-1b-it-Q4_K_M.gguf) to ensure a smooth experience on common hardware.

4. Exporting dialogues

Users can save the current dialog in JSON format to a USB flash drive or other storage path by clicking the Export button within the interface. The exported file contains the complete prompts and model responses, making it easy to continue using or archiving on other devices. Steps:

• Click on the menu bar of the 导出 Button.
• Select the save path and name the file (default extension is .json).
• Once saved, the file can be viewed in a text editor or imported into other tools that support JSON.

5. Performance optimization

The program is optimized for common hardware and does not require GPU support. When loading the model for the first time, the program will cache the model to RAM to speed up the subsequent response. If your computer has low RAM, it is recommended to choose a smaller model file (less than 1GB). In testing, the program achieves a generation speed of about 20 tokens/second on the i7-10750H CPU, which is suitable for most daily tasks.

caveat

• hardware requirement: Minimum 4GB RAM, 8GB or more recommended to ensure smooth operation.
• Model Compatibility: Only GGUF format models are supported, not other formats such as PyTorch or ONNX.
• operating system: Currently only Windows is supported, no Mac or Linux version is available at this time.
• Privacy: All processing is done locally with no data uploads to ensure information security.

application scenario

1. Temporary workstation utilization
   Quickly draft documents, brainstorm or answer questions by running Local LLM Notepad from a USB flash drive on computers with no network or restricted access, such as at customer sites or public facilities.
2. Privacy-sensitive mandates
   For scenarios that require a high degree of confidentiality, such as legal document drafting or sensitive data analysis, running offline ensures that data is not leaked, making it suitable for lawyers, journalists or researchers.
3. Education and learning
   Students or teachers can use this tool in the classroom to access models to explain complex concepts by typing in questions, or to generate study notes, suitable for web-free teaching environments.
4. Creation while traveling
   Writers or creators on the go use a USB drive to run the program and record inspiration, draft articles, or generate creative content at any time without relying on a cloud-based service.

QA

1. Do I need to network?
   No. Local LLM Notepad runs completely offline, all calculations are done locally, no internet connection is required.
2. What models are supported?
   Currently, GGUF format models are supported, and it is recommended to use lightweight models such as gemma-3-1b-it-Q4_K_M.ggufUsers can download compatible models from platforms such as Hug16.ging Face. Users can download compatible models from platforms such as Hug16. ging Face.
3. Will it work on a Mac?
   Currently only Windows is supported. Mac or Linux users will have to wait for a later version or run it using a virtual machine.
4. How to ensure the speed of generation?
   The program is optimized for the CPU and caches to RAM when the model is first loaded. it is recommended to use a device with more than 8GB of RAM and select a model smaller than 1GB for best performance.
5. How are conversation logs saved?
   Save the conversation as a JSON file via the export function in the menu bar and store it on a USB flash drive or other device for easy migration and archiving.