Since the appearance of ChatGPT, a lot of tools have emerged whose respect for user privacy is dubious. Luckily, several open source projects appeared to remedy this. Ollama AI is a tool to use artificial intelligence in the terminal
The app will allows you to use a list of LLMs on your computer terminal or single board device as long as your hardware is powerful enough. The good thing is that their library has options for all sizes.
LLMs (Large Scale Language Models) are used to understand generating and translating language at a level that is close to natural language. They are trained with enormous amounts of text data. These models can be used for a wide variety of language-related tasks, such as answering questions, summarizing text, translating into other languages, producing coherent and creative content, finishing sentences, and finding language patterns.
The expression "Large scale" refers to both the enormous amount of data (Billions) and the parameters that the model uses to establish relationships and find patterns among the data.
The parameters are a factor to take into account. Although there are those who report having used Ollama AI on a Raspberry Pi 4, not all models can be used if you do not have enough memory. I have used models with 7 billion parameters with 6 GB of RAM and without a graphics card without hardly affecting the performance of the system, but it couldn't handle one with 13.
The models with fewer parameters speak Spanish at the Yankee tourist parody level and are not very precise in their responses, so you should be careful when using them.
Ollama AI: Use artificial intelligence in the terminal
Before continuing, let's start with a brief explanation. LlaMa is a machine learning algorithm that allows an Artificial Intelligence to learn to recognize and classify images of objects.
To this type of models They are trained with labeled images so that they learn to recognize specific characteristics such as texture, shape and color that allow them to be identified in other contexts.
Since the name of the algorithm is pronounced the same as the name of the animal, some libraries were named after similar species such as vicuna (Vicuña in languages that do not use the ñ) or Alpaca.
Coming back to Ollama AI, It allows us to download and run different open source models from the terminal of our Linux distribution. It is installed with the command:
curl https://ollama.ai/install.sh | sh
To launch a model we write the command:
ollama run nombre_del_modelo
We can see the list of models here
For example, to install TinyLlaama, a model with just over a terabyte of parameters, we use the command:
ollama run tinyllama
The first time this command is executed the model is downloaded to the computer. Please note that some take up several gigabytes of disk space.
You can delete a model with the command:
ollama rm nombre_del modelo
To see the list of installed models use the command:
ollama list
You can see the information about each of the installed models with:
ollama show
An interesting option from the Ollama model library are the so-called “Uncensored” ones.. The best-known Artificial Intelligence models place restrictions on certain types of questions to comply not only with the law but with the social pressures of political correctness.
I remember that a long time ago I asked Copilot, Microsoft's ChatGPT-based assistant, to tell me iceberg jokes. He told me that ecological catastrophes were too serious a topic to make jokes about.
The uncensored models They identify occasions when AIs refuse to respond or give biased responses from the base models, remove them, and train the system to respond correctly.
I would have to do more tests to know how useful Ollama AI can be on computers with a limited amount of RAM and without a dedicated graphics card. In any case, it is good to know that open source alternatives are making their way and allowing censorship to be eliminated.