LocalAI

Midori AI photo Midori AI photo

Here is a link to LocalAI Github

Installing LocalAI: A Step-by-Step Guide

This guide will walk you through the process of installing LocalAI on your system. Please follow the steps carefully for a successful installation.

Step 1: Initiate Installation

  1. From the main menu, enter the option 2 to begin the installation process.
  2. You will be prompted with a visual confirmation.

Step 2: Confirm GPU Backend

  1. Respond to the prompt with either yes or no to proceed with GPU support or CPU support only, respectively.

Step 3: Confirm LocalAI installation

  1. Type localai into the menu and press Enter to start the LocalAI installation.

Step 4: Wait for Setup Completion

  1. LocalAI will now automatically configure itself. This process may take approximately 10 to 30 minutes.
  2. Important: Please do not restart your system or attempt to send requests to LocalAI during this setup phase.

Step 5: Access LocalAI

  1. Once the setup is complete, you can access LocalAI on port 38080.
Important Notes
  • Remember to use your computer’s IP address instead of localhost when accessing LocalAI. For example, you would use 192.168.10.10:38080/v1 or 192.168.1.3:38080/v1 depending on your network configuration.

Support and Assistance

If you encounter any issues or require further assistance, please feel free to reach out through the following channels:

Subsections of LocalAI

Install LocalAI Models

Midori AI photo Midori AI photo

Install a Model from the Midori AI Model Repo

Step 1:

  • Start the Midori AI Subsystem

Step 2:

  • On the Main Menu, Type 5 to Enter the Backend Program Menu

Step 3:

  • On the Backend Program Menu, Type 10 to Enter the LocalAI Model Installer

Step 4a:

  • If you have LocalAI installed in the subsystem, skip this step.
  • If you do not have LocalAI installed in the subsystem, the program will ask you to enter the LocalAI docker’s name. It will look something like localai-api-1, but not always. If you need help, reach out on the Midori AI Discord / Email.

Step 4b:

  • If you have GPU support installed in that image, type yes.
  • If you do not have GPU support installed in that image, type no.

Step 5:

  • Type in the size you would like for your LLM and then follow the prompts in the manager!

Step 6:

  • Sit Back and Let the Model Download from Midori AI’s Model Repo
  • Don’t forget to note the name of the model you just installed so you can request it for OpenAI V1 later.

Need help on how to do that? Stop by - How to send OpenAI request to LocalAI

Install a Hugging Face Model from the Midori AI Model Repo

Step 1:

  • Start the Midori AI Subsystem

Step 2:

  • On the Main Menu, Type 5 to Enter the Backend Program Menu

Step 3:

  • On the Backend Program Menu, Type 10 to Enter the LocalAI Model Installer

Step 4a:

  • If you have LocalAI installed in the subsystem, skip this step.
  • If you do not have LocalAI installed in the subsystem, the program will ask you to enter the LocalAI docker’s name. It will look something like localai-api-1, but not always. If you need help, reach out on the Midori AI Discord / Email.

Step 4b:

  • If you have GPU support installed in that image, type yes.
  • If you do not have GPU support installed in that image, type no.

Step 5:

  • Type huggingface when asked what size of model you would like.

Step 6:

  • Copy and Paste the Hugging Face Download URL That You Wish to Use
  • For example: https://huggingface.co/mlabonne/gemma-7b-it-GGUF/resolve/main/gemma-7b-it.Q2_K.gguf?download=true midori ai photo midori ai photo
  • Or you can use the huggingface naming from their api
  • For example: mlabonne/gemma-7b-it-GGUF/gemma-7b-it.Q2_K.gguf

Step 7:

  • Sit Back and Let the Model Download from Midori AI’s Model Repo
  • Don’t forget to note the name of the model you just installed so you can request it for OpenAI V1 later.

Need help on how to do that? Stop by - How to send OpenAI request to LocalAI