Interact with local models

Intro

LlamaGPT-Chat is a command line chat application. It integrates with the LLM via C++. This means that it will only work with LLM’s that have been in the same language. There are few available today that are free to download and use. Because they are in C++, the models are able to run locally on your computer, even if there is no GPU available. Most of the models are quite fast in their responses. By integrating with LlamaGPT-Chat, able to access multiple types of LLM’s with only one additional back-end within chattr.

Installation

To get LlamaGPT-Chat working you machine you will need two things:

A version of LlamaGPT-Chat that works in your computer
An LLM file that works with with the chat program

Install LLamaGPT-Chat

LlamaGPT-Chat will need a “compiled binary” that is specific to your Operating System. For example, for Windows, a compiled binary should be an .exe file.

The GitHub repository offers pre-compiled binaries that you can download and use: Releases. Depending on the system’s security, the pre-compiled program may blocked from running. If that is the case, you will need to “build” the program in your computer. The instructions to do this are here.

LLM file

The GitHub repository contains a list of compatible models. Download at least one of the models, and make note of where it was saved to in your computer.

Setup

To start, instruct chattr to switch the LLamaGPT back-end using chattr_use()

library(chattr)

chattr_use("llamagpt")
#> 
#> ── chattr
#> • Provider: LlamaGPT
#> • Path/URL: ~/LlamaGPTJ-chat/build/bin/chat
#> • Model: ~/ggml-gpt4all-j-v1.3-groovy.bin
#> • Label: GPT4ALL 1.3 (LlamaGPT)

If this is the first time you use this interface, confirm that the path of to the compiled program, and the model matches to what you have in your machine. To check that use chattr_defaults()

chattr_defaults()
#> 
#> ── chattr ──────────────────────────────────────────────────────────────────────
#> 
#> ── Defaults for: Default ──
#> 
#> ── Prompt:
#> • Use the R language, the tidyverse, and tidymodels
#> 
#> ── Model
#> • Provider: LlamaGPT
#> • Path/URL: ~/LlamaGPTJ-chat/build/bin/chat
#> • Model: ~/ggml-gpt4all-j-v1.3-groovy.bin
#> • Label: GPT4ALL 1.3 (LlamaGPT)
#> 
#> ── Model Arguments:
#> • threads: 4
#> • temp: 0.01
#> • n_predict: 1000
#> 
#> ── Context:
#> Max Data Files: 0
#> Max Data Frames: 0
#> ✖ Chat History
#> ✖ Document contents

If either, or both, paths are incorrect, correct them by updating the path, and model arguments in chattr_defaults()

chattr_defaults(path = "[path to compiled program]", model = "[path to model]")
#> 
#> ── chattr ──────────────────────────────────────────────────────────────────────
#> 
#> ── Defaults for: Default ──
#> 
#> ── Prompt:
#> • Use the R language, the tidyverse, and tidymodels
#> 
#> ── Model
#> • Provider: LlamaGPT
#> • Path/URL: [path to compiled program]
#> • Model: [path to model]
#> • Label: GPT4ALL 1.3 (LlamaGPT)
#> 
#> ── Model Arguments:
#> • threads: 4
#> • temp: 0.01
#> • n_predict: 1000
#> 
#> ── Context:
#> Max Data Files: 0
#> Max Data Frames: 0
#> ✖ Chat History
#> ✖ Document contents

So that you do not need to change those defaults every time you start a new R session, use chattr_defaults_save(). That will create a YAML file in your working directory. chattr will use that file to override the defaults.

chattr_defaults_save()

Use chattr_test() to confirm that the chattr is able to communicate with LLamaGPT-Chat, and that the model it will use with your R session is accessible

chattr_test()
#> ✔ Model started sucessfully
#> ✔ Model session closed sucessfully

Model Arguments

The arguments sent to the model can be modified in chattr_defaults() by modifying the model_arguments argument. It expects a list object. Here are the arguments it sets by default:

chattr_defaults()$model_arguments
#> $threads
#> [1] 4
#> 
#> $temp
#> [1] 0.01
#> 
#> $n_predict
#> [1] 1000

Here is an example of adding batch_size to the defaults:

chattr_defaults(model_arguments = list(batch_size = 40))
#> 
#> ── chattr ──────────────────────────────────────────────────────────────────────
#> 
#> ── Defaults for: Default ──
#> 
#> ── Prompt:
#> • Use the R language, the tidyverse, and tidymodels
#> 
#> ── Model
#> • Provider: LlamaGPT
#> • Path/URL: [path to compiled program]
#> • Model: [path to model]
#> • Label: GPT4ALL 1.3 (LlamaGPT)
#> 
#> ── Model Arguments:
#> • batch_size: 40
#> • threads: 4
#> • temp: 0.01
#> • n_predict: 1000
#> 
#> ── Context:
#> Max Data Files: 0
#> Max Data Frames: 0
#> ✖ Chat History
#> ✖ Document contents

To see the most current list of available model arguments go to: Detailed command list.

IMPORTANT - chattr passes the arguments directly to LLamaGPT-chat as a command line flag, except model. chattr will use the value in chattr_defaults(model = "[path to model]") instead.