Ollama大模型部署工具使用笔记

Last updated on Sep 25, 2024 7 min read LLM

Ollama

Ollama 是一个开源的大型语言模型服务工具，旨在帮助用户在本地环境中部署和运行大规模预训练语言模型（LLM）。它通过简化模型的下载、安装和管理流程，使得用户能够快速地在本地运行这些模型，并且支持多种流行模型如 GPT-4 和 LLaMA 等。

Ollama 提供了多种功能，包括：

模型管理：用户可以通过命令行工具（如 ollama serve、ollama run、ollama pull 等）来启动服务、运行模型、拉取模型、推送模型等操作。
多 GPU 支持：Ollama 支持多 GPU 配置，可以优化模型的运行效率，适应不同的计算资源需求。
热加载模型：用户可以在不重新启动服务的情况下切换不同的模型，这使得 Ollama 在灵活性和效率上具有优势。
API 兼容性：Ollama 提供了与 OpenAI 兼容的 API 接口，方便用户通过 API 调用模型进行任务处理。
Web 界面和客户端支持：Ollama 支持通过 WebUI 访问，并提供了多种客户端库（如 ollama-js 和 ollama-python），方便开发者在不同平台和语言环境中使用。

Ollama 的核心优势:

简单易用。用户只需通过简单的安装指令即可在本地运行复杂的 AI 模型。
通用性。可以在多种应用中使用，包括web方式部署。
安全性和私密性。模型、数据处理流程运行在本地机器。
支持并行请求和并发处理，进一步提升了其在多任务场景下的性能。

源码地址：GitHub - ollama/ollama: Get up and running with Llama 3.3, Mistral, Gemma 2, and other large language models.

大模型文件

GGUF：GPT-Generated Unified Format
一种用于数据交换的大模型文件格式。GGUF包含加载模型所需的所有信息，无需依赖外部文件。
GGUF格式的大模型文件是什么意思？gguf是什么格式？如何使用？为什么有GGUF格式的大模型文件？GGUF大模型文件与GGML的差异是啥？ | 数据学习者官方网站(Datalearner)

安装及基本使用

各系统安装方法

系统级安装

Linux

curl -fsSL https://ollama.com/install.sh | sh

Mac/Windows
下载桌面端软件
注：对于windows而言，默认安装在c盘，模型也会拉取到c盘中，为避免c盘空间不够，可设置系统变量OLLAMA_MODELS后重启，使模型拉取到指定目录下。

官方文档：Download Ollama on Linux

docker化部署

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Nvidia GPU

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Ollama is now available as an official Docker image · Ollama Blog

Ollama Docker 镜像指南 | Ollama 中文文档

环境变量

OLLAMA_DEBUG 显示其他调试信息（例如 OLLAMA_DEBUG=1）
OLLAMA_HOST ollama 服务器的 IP 地址（默认 127.0.0.1:11434）
OLLAMA_KEEP_ALIVE 模型在内存中保持加载状态的持续时间（默认5min）
OLLAMA_MAX_LOADED_MODELS 每个 GPU 加载的最大模型数量
OLLAMA_MAX_QUEUE 排队请求的最大数量
OLLAMA_MODELS 模型目录的路径
OLLAMA_NUM_PARALLEL 并行请求的最大数量
OLLAMA_NOPRUNE 启动时不修剪模型 blob
OLLAMA_ORIGINS 允许来源的逗号分隔列表
OLLAMA_TMPDIR 临时文件的位置
OLLAMA_FLASH_ATTENTION 启用闪存注意力机制
OLLAMA_LLM_LIBRARY 设置 LLM 库以绕过自动检测

模型镜像

拉取镜像

使用方式类似docker

ollama pull bge-m3:567m

可拉取huggingface.co上的模型。示例：

# 拉取无量化的模型
ollama run hf-mirror.com/CompendiumLabs/bge-large-zh-v1.5-gguf:F32

常用模型列表

查看模型镜像

ollama list

查看指定镜像信息

模型架构、参数量、上下文长度、向量维度、量化等级。

ollama show bge-m3:567m

运行指定模型镜像

嵌入模型使用

官方文档介绍：嵌入模型 · Ollama 博客 - Ollama 框架

通过web接口使用(CURL)

curl http://127.0.0.1:11434/api/embeddings -d '{
  "model": "bge-m3:567m",
  "prompt": "Llamas are members of the camelid family"
}'

通过web接口使用（Python requests）

import requests
model = "bge-m3:567m"
data = {
  "model": "bge-m3:567m",
  "prompt": "Llamas are members of the camelid family"
}
url = "http://127.0.0.1:11434/api/embeddings"
resp = requests.post(url, json=data)
print(resp.status_code)
print(resp.json())

# 200
# {'embedding': [-0.48389095067977905, 0.1629858911037445, -1.2879666090011597, -0.4249868392944336, 0.06189827620983124, -1.894164800643921, -0.05759759992361069, -0.37197133898735046, -0.7300008535385132, -0.2554854452610016, -0.877233624458313, 0.1477876901626587, -1.3568150997161865, -0.3967072069644928, 0.2673203647136688, 0.45496314764022827, -0.21816010773181915, -0.30262428522109985, -0.4199124872684479, -0.864885687828064, -0.8550161719322205, -0.338420569896698, -1.6880122423171997, -0.6067222356796265, -0.6625216603279114, 0.7664051651954651, -0.5339586138725281, 0.031786322593688965, 0.1257663369178772, -1.2599785327911377, 0.04856197535991669, 0.4365246295928955, -0.24520611763000488, -1.288858413696289, -0.6384986639022827, -0.8468897938728333, 0.19501514732837677, 0.6735445261001587, -0.8204600214958191, -0.11305803805589676, -0.5183432698249817, -0.1329708695411682, -0.12623924016952515, -0.4765018820762634, 0.010443715378642082, 0.6672123074531555, -0.3428826332092285, -0.8787519931793213, 0.9061988592147827, -0.1680990606546402, -0.5252246856689453, 0.30696871876716614, 0.3571414351463318, 1.3839727640151978, -0.35037967562675476, -0.3252205550670624, -0.050703853368759155, -1.2032965421676636, -1.0453174114227295, 0.8841867446899414, -0.5340055823326111, 1.6697667837142944, -0.12002211809158325, -0.6885624527931213, 0.52571702003479, 2.4142980575561523, -0.9816556572914124, 0.031116757541894913, -0.8211243152618408, -0.1276358813047409, -0.37229105830192566, 0.3456203043460846, -0.40211620926856995, -0.7581859827041626, -1.1912188529968262, 1.1245625019073486, -0.14072367548942566, 0.1796419769525528, -0.7918760776519775, 0.7440110445022583, 0.4012685716152191, 0.33956584334373474, 0.09141135215759277, 0.26180267333984375, -1.1351662874221802, 1.5037933588027954, 0.13359855115413666, -1.134125828742981, -0.3231596350669861, -0.5057533979415894, 0.3898926079273224, -0.5854844450950623, 0.31405889987945557, -0.12119662761688232, -1.4858416318893433, -0.9023401141166687, -0.7150736451148987, 0.7029703855514526, 1.318637728691101, 0.9904419183731079, 0.04628058522939682, 0.8020230531692505, -0.7286942005157471, 1.3178064823150635, 0.2129446417093277, -0.9833354353904724, 0.006343025714159012, -0.07465913891792297, -0.09912257641553879, -0.1754705011844635, 1.1729116439819336, 1.270808458328247, -1.2898138761520386, -0.7459997534751892, -1.2078202962875366, -0.29605934023857117, 0.057505376636981964, -1.2475160360336304, -0.0822816789150238, -0.1509387195110321, 0.19790568947792053, -0.3700128197669983, 0.8258823752403259, -1.4422955513000488, -0.31156960129737854, 0.09654010087251663, 0.10952077805995941, -0.196611225605011, 1.3652459383010864, -0.5213243961334229, -0.05336098372936249, 0.24806123971939087, -0.9901211261749268, 0.9068143963813782, -0.20537859201431274, -1.3204782009124756, 0.9759248495101929, 0.056110210716724396, -0.7407263517379761, 0.32144758105278015, -0.039979949593544006, 0.3706381916999817, 0.01982523873448372, -2.053661346435547, -0.2762638032436371, -0.04722151160240173, -0.10801201313734055, 0.4196217954158783, 0.7176979780197144, -0.05409080907702446, -1.4919244050979614, -0.5315126180648804, 0.3348028063774109, 0.9908292293548584, 1.3872123956680298, -0.48270487785339355, -2.3281936645507812, 1.2646150588989258, 0.43967413902282715, 2.2732536792755127, 1.1881746053695679, 1.2016551494598389, 0.383813738822937, -0.6256048679351807, 0.5411356687545776, 0.5889014601707458, 0.2853587865829468, 2.32376766204834, 0.6299901008605957, -0.6772042512893677, -0.5504119992256165, -0.9248839020729065, -0.5500874519348145, 0.8805500268936157, 0.9225412607192993, 0.6374692320823669, 0.11973302066326141, -0.05772602930665016, 0.19341659545898438, -1.1111313104629517, -0.7513657808303833, -1.5892566442489624, -0.6885318756103516, -1.9779391288757324, -0.6286582350730896, -0.2722617983818054, 0.19061094522476196, -1.435196876525879, 0.27162250876426697, -0.3080448508262634, -0.11701054871082306, -1.3302035331726074, 0.872103750705719, -0.4459778368473053, 0.8661269545555115, -0.291522741317749, -1.0445623397827148, -1.4062522649765015, -1.020662546157837, 0.6032674312591553, -1.391931176185608, 1.1222949028015137, -0.38415995240211487, 0.7606028318405151, -1.011482834815979, -0.2167990654706955, -2.3548789024353027, -0.7761564254760742, 0.6241523623466492, 0.4363159239292145, 0.7824066877365112, 1.5523499250411987, -0.6260149478912354, -1.1249802112579346, 0.40284663438796997, -0.3003946840763092, -0.1630946546792984, 1.039842128753662, -0.29646024107933044, 0.15052631497383118, 0.34122294187545776, 0.3863466680049896, 0.9002049565315247, 1.5289305448532104, 1.2288819551467896, -1.2961223125457764, 1.575148344039917, -0.23363909125328064, -0.5276688933372498, -0.9197537899017334, -1.0664429664611816, 0.7786262035369873, -0.6545555591583252, -0.01195970457047224, -0.7407588362693787, 0.3266623318195343, -1.011655569076538, 0.9380229711532593, 0.6970893144607544, -0.2440209835767746, -2.1250221729278564, 0.29284560680389404, 0.05085933208465576, -0.7229329943656921, 0.5164413452148438, -1.2202954292297363, 0.7292686104774475, 1.2441515922546387, -0.4444581866264343, 0.15960593521595, 0.48850172758102417, -1.2230074405670166, -1.1194137334823608, -0.36882302165031433, -1.5303047895431519, 1.4107462167739868, 0.5172622203826904, -0.8728272318840027, 1.3676211833953857, -0.21578697860240936, 0.035039231181144714, 0.664027750492096, 1.098572015762329, 1.1513136625289917, 0.3596438765525818, 0.35524892807006836, -0.7183171510696411, -1.339687705039978, 1.0057482719421387, -0.29071277379989624, -0.9573113918304443, -0.445783793926239, 0.11905526369810104, 0.215439110994339, -0.45394423604011536, 0.45929333567619324, 0.24423661828041077, -0.21359869837760925, 0.53621506690979, -0.2946447730064392, -0.9282507300376892, 0.7658008337020874, 0.43154916167259216, -1.014953374862671, -0.11654475331306458, 0.16619645059108734, -1.3857553005218506, -0.37362900376319885, 0.24885837733745575, -0.8228291869163513, 0.24077779054641724, -0.8728160262107849, 1.3874666690826416, 0.27450475096702576, 0.9550139307975769, 1.7118417024612427, -0.5531263947486877, -3.841129779815674, 0.2598555386066437, -0.7621341943740845, -0.16707825660705566, 0.4923422038555145, -0.7297694683074951, 0.6409592628479004, -1.3195465803146362, 0.914095938205719, 0.9269043803215027, 0.20966579020023346, -1.4014893770217896, 0.22388288378715515, 0.15204374492168427, -1.3223536014556885, -0.1783973127603531, 0.2684626877307892, -0.2605661451816559, 0.8373638987541199, -0.9207277297973633, -0.8719792366027832, -1.939489722251892, 0.3787740170955658, -0.06400647759437561, 0.40199294686317444, 0.23852740228176117, -1.472751498222351, -0.7568486928939819, -0.7294861078262329, -0.15985587239265442, 1.140151858329773, 1.2499442100524902, -0.19350507855415344, 0.7957978844642639, -0.32460227608680725, 0.7127668857574463, 0.48049014806747437, -0.11277987062931061, 0.8896826505661011, 0.40263834595680237, 1.253379225730896, 1.3455395698547363, -0.3411444425582886, 0.03447602689266205, -0.47524788975715637, -2.3980557918548584, -0.2932802736759186, 0.21891197562217712, 2.2351622581481934, 0.4272134304046631, 0.49982666969299316, 0.7709869742393494, -0.022764012217521667, -0.42780801653862, -0.5912521481513977, -0.6674491763114929, -0.5571022629737854, 0.008947830647230148, 1.3874914646148682, -0.6490374803543091, 0.9244760870933533, -0.9663845300674438, 0.5826146006584167, -0.5499289631843567, 0.2708287537097931, -0.5268988609313965, 0.047928810119628906, 0.41516828536987305, 0.3335105776786804, -1.1862213611602783, 0.648356020450592, -0.07660266011953354, 0.7585784196853638, -0.6259901523590088, 0.17375971376895905, 1.2746256589889526, 0.6661895513534546, 0.45807796716690063, -1.1792011260986328, -2.067671775817871, -2.1423487663269043, -0.8159367442131042, 0.49140775203704834, 0.37510475516319275, -0.6751887798309326, -0.06905323266983032, -0.4252377152442932, 0.14085279405117035, 2.273564338684082, 4.374858856201172, 0.8568570017814636, -0.1486024558544159, -1.1394788026809692, 0.7987928986549377, -0.502269983291626, 1.0873501300811768, -0.7543830275535583, 1.0823357105255127, -1.126244068145752, 0.015512105077505112, -1.6732574701309204, -0.08897897601127625, -0.11131826788187027, -0.22474148869514465, 0.8963887691497803, -0.21610161662101746, 0.4772183895111084, 2.3766326904296875, -0.18505465984344482, 0.06916872411966324, -0.5725727677345276, -0.042843665927648544, 0.6541853547096252, 0.20124579966068268, -1.0198312997817993, -0.4806182384490967, 0.8910837173461914, -0.6163516640663147, -0.3703957796096802, -0.18910782039165497, -0.626382052898407, -0.4345147907733917, 0.5857327580451965, 0.7833346724510193, -0.5581040978431702, 0.01850747875869274, -0.33377954363822937, -0.8964641094207764, 0.7714489698410034, -1.7408682107925415, -0.7150954008102417, 0.5365071892738342, 0.46195167303085327, -0.7281333804130554, -1.4020211696624756, 1.3868874311447144, 0.6546242237091064, 0.9242591261863708, 0.6575019955635071, 1.016343355178833, -0.40921545028686523, -1.2088876962661743, 1.3184688091278076, -0.7345009446144104, 0.2393958568572998, -0.20745331048965454, -1.1442959308624268, 0.013941999524831772, 0.16471493244171143, -0.3787882626056671, -0.3148609697818756, 0.8737255334854126, -0.34712737798690796, -1.1366064548492432, 1.4119541645050049, -0.5862771272659302, -0.0833955705165863, 0.254255086183548, -0.6454010605812073, 0.392562597990036, -1.145630955696106, -0.35809120535850525, -0.19380570948123932, 0.138933464884758, 0.22295784950256348, 0.974608838558197, 0.5948063135147095, 0.6698558330535889, 1.453622817993164, -0.20721052587032318, 0.9429672956466675, -0.19915491342544556, -0.2525172233581543, 0.3783726394176483, 1.099809169769287, 0.23500335216522217, 1.1995747089385986, -0.33503562211990356, -0.7516640424728394, 0.2653375566005707, 0.7513218522071838, -0.18856637179851532, 0.3886350095272064, -0.0776878297328949, -0.9533616304397583, 0.8250511288642883, -0.9892752766609192, -1.1322112083435059, 0.334240585565567, -0.15351955592632294, -0.5575529932975769, 0.411653071641922, -0.7441359758377075, -0.22734123468399048, -0.600610077381134, 0.2553544342517853, 0.9224490523338318, -0.4594558775424957, 0.6269127130508423, -0.08593116700649261, -0.18045832216739655, -0.33589667081832886, 0.37196120619773865, 1.0858609676361084, -0.4756281077861786, -0.3095019459724426, 0.7028302550315857, 1.4592286348342896, 0.9338569641113281, 0.14455312490463257, 1.0953871011734009, -0.024741563946008682, 0.34132254123687744, 0.3612818121910095, 0.19038526713848114, 0.5416169762611389, 1.4395241737365723, -0.9690583348274231, 1.0187524557113647, 0.5114442110061646, -0.33944278955459595, -1.2973610162734985, 0.5144263505935669, 0.5794021487236023, 1.040926218032837, 0.17142854630947113, 0.13562457263469696, -0.15474654734134674, 0.7504505515098572, -0.6694148182868958, 0.5218936800956726, -1.00209379196167, 0.6513790488243103, -0.2623409926891327, -1.1654722690582275, -0.9594823122024536, -0.2689248025417328, 0.5088849663734436, -0.315811425447464, -1.144059181213379, -0.4386417269706726, 0.04228678345680237, -0.13890372216701508, 0.08983086049556732, -0.2892051637172699, 0.10661523044109344, 0.32713583111763, 0.39429858326911926, -0.1117032989859581, 0.4808030426502228, -0.17176799476146698, 1.8583332300186157, 1.0257786512374878, 0.5378902554512024, 0.7901830077171326, -0.9154506921768188, 1.5570508241653442, 0.6575223207473755, -0.10238159447908401, -0.5354354381561279, -1.0949883460998535, -0.6410660147666931, 1.3545423746109009, 0.4747467041015625, -1.0997564792633057, -0.9008767604827881, -0.9321032166481018, -0.04706874489784241, 1.2818530797958374, 2.1697137355804443, 1.4837135076522827, 0.41118136048316956, 0.32722699642181396, 0.5643124580383301, 0.6267502307891846, -0.1449054628610611, -0.5619372129440308, 0.49184393882751465, 0.4118610620498657, -2.0120034217834473, 1.7257705926895142, -1.9534697532653809, -0.25044578313827515, 1.3506834506988525, 0.763012707233429, -1.6158742904663086, 0.16273142397403717, 0.3495383858680725, -1.6734611988067627, 0.141712486743927, -0.702837347984314, -0.4471498727798462, 0.007544901221990585, -0.17757470905780792, 1.095269799232483, -0.5200486183166504, 1.5710393190383911, -0.13629013299942017, -0.6435431241989136, 0.9233338832855225, -0.501850962638855, -0.00806130189448595, -0.07507804036140442, -0.06853725016117096, 1.104569673538208, -0.13101615011692047, 1.343636155128479, -1.0531611442565918, -0.5977267622947693, 0.21137170493602753, -1.0766499042510986, 0.7170558571815491, -1.019190788269043, -1.0984108448028564, 1.3423140048980713, 0.35710081458091736, 0.8959841728210449, -0.5184237957000732, 0.08741726726293564, 0.2648735046386719, 0.2583630383014679, 0.31615209579467773, 1.8552192449569702, 0.18373729288578033, 0.31858131289482117, 1.2174822092056274, 0.0002594403922557831, -0.2711639702320099, -0.06542174518108368, -0.4343301057815552, -0.396636962890625, -0.4865146577358246, 1.6044085025787354, 0.2498398721218109, 2.1292014122009277, -0.37401124835014343, 0.3378225564956665, -0.05598484352231026, 0.5166763067245483, 0.3022933006286621, 0.09485073387622833, 0.8038163185119629, 0.6531191468238831, -0.11651826649904251, 0.1481170654296875, -0.5797003507614136, 0.4908023178577423, -1.3707826137542725, 0.6044203042984009, -0.2063351720571518, 0.8987088799476624, -0.1532531976699829, -0.9236035943031311, -0.5937836766242981, -0.3412547707557678, 0.4465879797935486, 0.3720558285713196, -0.6115177869796753, -0.7166279554367065, -0.6698778867721558, -0.08633238822221756, -2.1518402099609375, -1.8085110187530518, 0.8992704749107361, 0.4295235276222229, -1.1189333200454712, 1.6854535341262817, -0.18284085392951965, 0.9078053832054138, -0.38644203543663025, 0.225868821144104, 0.31985318660736084, 0.7112962007522583, -0.44675272703170776, -0.096546970307827, 0.15549083054065704, 0.7993689775466919, 0.9356825351715088, -0.3974955677986145, -0.22645516693592072, 0.039798520505428314, -0.10044921189546585, 0.18548843264579773, -0.8468291759490967, 0.6533968448638916, -0.7577924728393555, -0.3526364266872406, -0.14121688902378082, 1.1495893001556396, -0.14410509169101715, 1.04098379611969, -0.5193806290626526, -0.5166260600090027, 0.07788616418838501, -0.33017614483833313, -0.6555951237678528, -1.6482337713241577, 0.3829895853996277, 0.595099687576294, 0.6765041947364807, -0.026130538433790207, -0.39853397011756897, 0.4448527991771698, -0.7093629837036133, -0.08623401075601578, -0.12741562724113464, -0.0543740950524807, -0.11028140038251877, 0.2727143466472626, 0.4323011338710785, 0.5170363783836365, 0.6991905570030212, 0.43977248668670654, -0.26055943965911865, -0.41749659180641174, -0.37892115116119385, 0.26712533831596375, -1.1590168476104736, 0.13999031484127045, -1.2830641269683838, 0.31746649742126465, -0.699725866317749, 0.4502127170562744, -0.7030218839645386, -0.6288986802101135, -0.10577888041734695, 0.4197461009025574, 1.1889363527297974, 1.0995092391967773, -0.25288307666778564, 0.43505924940109253, -0.38185006380081177, 0.0245368629693985, 1.3544762134552002, -0.7132253646850586, 0.13584789633750916, -0.5469449758529663, 0.08268885314464569, 0.7937004566192627, 0.1427004039287567, 1.0121901035308838, -0.2695789933204651, -0.9273400902748108, -0.32654085755348206, -0.2924230694770813, 1.0390599966049194, -1.2600510120391846, 0.3527325391769409, -0.07651464641094208, -1.046594500541687, 0.5937753319740295, -0.4779147207736969, 0.612679660320282, -0.3113807141780853, -0.17392639815807343, 0.1908830851316452, -0.02696373127400875, -0.2900051772594452, -0.4194733798503876, -0.7049617171287537, 0.18955166637897491, 0.8208848237991333, 0.7151224613189697, -0.45836585760116577, -0.28675204515457153, -0.9628690481185913, 0.29398950934410095, 0.8706287145614624, 0.47204336524009705, 0.5006674528121948, -0.22978398203849792, -1.1493273973464966, -1.0882036685943604, 0.9270052313804626, -0.1585390716791153, -0.7612862586975098, -0.6218879818916321, -0.6159817576408386, 0.36547791957855225, -0.37269479036331177, 0.7049635052680969, -0.8601799607276917, -0.08814814686775208, -2.7143328189849854, 0.08826959133148193, -0.46739956736564636, 1.0577058792114258, -1.0583564043045044, -0.04607272893190384, -0.47055381536483765, 0.3596341013908386, 0.3922647535800934, -1.6438339948654175, 1.2529239654541016, 0.7920860648155212, 1.0432029962539673, -0.032906174659729004, 0.4608609080314636, 1.1496167182922363, 0.715035617351532, -0.05857841297984123, 0.9109014868736267, -0.004223208874464035, -0.8470831513404846, -0.193971648812294, -0.6667903065681458, 1.2107670307159424, -0.7552700042724609, 1.1727250814437866, -0.4634220004081726, -0.4212205410003662, -0.409645140171051, 0.39958178997039795, 0.3843447268009186, -0.985909104347229, 0.6418947577476501, 0.8980875611305237, 0.6091064214706421, 0.47374922037124634, -0.3871939480304718, -1.1100895404815674, -0.3502974808216095, 1.1468558311462402, -1.4893337488174438, 0.011540520936250687, 0.2538171410560608, 0.6053327918052673, -0.27761298418045044, 0.8587223887443542, -0.7669300436973572, 0.2324858158826828, -0.47888466715812683, -0.510636568069458, 0.025440365076065063, 0.24266070127487183, -0.7128787636756897, 1.7400859594345093, 0.9517950415611267, 0.3655461370944977, -1.2001410722732544, 0.0916493684053421, -0.19792652130126953, 0.660355806350708, -0.4828328490257263, -0.1066732406616211, 0.3789789080619812, -2.241586208343506, -0.07087761163711548, 0.42019397020339966, -0.6749734878540039, -0.23759256303310394, -0.03265907242894173, 1.3034744262695312, -0.5104959011077881, -0.5472404956817627, 0.8820847868919373, -0.7575673460960388, 0.7382651567459106, -0.49096378684043884, 0.16328629851341248, 0.4601089358329773, -0.1880638599395752, -0.3282238841056824, -0.4477576017379761, 0.34469860792160034, 0.7963706254959106, -1.869153618812561, 0.8098132014274597, -0.14547166228294373, -0.7249885201454163, 0.71927809715271, -1.231347680091858, 0.21596068143844604, -0.9943387508392334, -0.31990307569503784, -0.9600954055786133, -1.124435544013977, 0.00236291391775012, 0.45672914385795593, -0.8256598114967346, -0.4172733724117279, -0.4362039864063263, -0.09346774965524673, -1.9135421514511108, 0.15227016806602478, -0.0452173613011837, 0.037314098328351974, -0.5001634359359741, 0.21376919746398926, 0.5416676998138428, -0.6216533780097961, 0.4845426678657532, 0.7794995903968811, 0.6219879388809204, 0.5843859314918518, 0.4008253812789917, -1.083384394645691, -0.2938498258590698, 0.20412977039813995, 1.4202282428741455, -0.2975914478302002, 0.06236982345581055, 0.2859801650047302, 0.7802537679672241, 0.7437810897827148, -0.3486124277114868, -0.4976764917373657, 1.1670929193496704, -0.6601022481918335, 1.1865148544311523, 0.5924584865570068, 0.5811538100242615, -1.324131965637207, 0.6601008176803589, -1.023046851158142, -0.9697758555412292, -0.14466354250907898, 0.41825252771377563, -0.1761837601661682, 0.6032886505126953, 1.3474615812301636, -0.9316960573196411, -0.20060226321220398, 0.6700326204299927, -0.2170209288597107, 0.4497944712638855, -0.5349551439285278, -1.2508801221847534, 0.026875004172325134, 0.7398591041564941, -0.7845653891563416, -1.917072057723999, -0.4466893970966339, 0.3931063413619995, 0.31019526720046997, 1.0334419012069702, -0.5943686366081238, 0.02615933120250702, -0.17425522208213806, 0.3417590856552124, -1.444457769393921, 0.09899255633354187, 0.1041608601808548, 0.8462392091751099, -0.2718645930290222, -0.22917187213897705, -0.8862298727035522, -0.6211874485015869, 0.45963579416275024, -0.09948010742664337, -0.6371907591819763, -1.0364465713500977, -0.1916862428188324, 0.01696515455842018, 0.5381381511688232, -0.262356162071228, -2.2822928428649902, 0.7177671790122986, -0.5404238700866699, 0.15031853318214417, 1.237904667854309, 0.5731261372566223, 0.3420713543891907, 0.5901159048080444, 1.3740406036376953, 0.3963102102279663, 0.4374891519546509, 0.48124414682388306, 0.9687537550926208, 0.4497835040092468, -0.3311283588409424, 0.18091653287410736, -0.6760132908821106, -0.5325351357460022, -1.2046533823013306, -0.6017568707466125, 0.7201565504074097, -0.16314469277858734, 0.12975934147834778, 0.04345990717411041, -0.1981891244649887, -1.062495470046997, 0.11409696936607361, 0.052117668092250824, -0.1114358976483345, 0.10844367742538452, -0.5243117809295654, -0.03415350615978241, -1.1313976049423218, 0.6405377984046936, -1.0357269048690796, -1.164151668548584, 0.9725601673126221, -1.0447090864181519, -0.3242415189743042, 0.15080960094928741, 0.8158973455429077, 0.0001805536448955536, 0.19246873259544373, 0.735846996307373, 0.6445034742355347, -0.4716704785823822, 1.5978564023971558, -0.4781534969806671, 1.478446125984192, -0.7744138240814209, -0.18896900117397308, 0.5071486234664917, -0.04415427893400192, -1.108320713043213, -1.130005955696106, -0.30516430735588074, 0.045964114367961884, 0.24080491065979004, 0.6340259313583374, -0.208756223320961, 1.3827378749847412, 0.5032235383987427, 1.162843108177185, 1.2213060855865479, 0.8414467573165894, 0.38811609148979187, -1.237191081047058, 0.6385774612426758, -0.9437369704246521, -0.42809122800827026, 0.09981541335582733]}

如果尚未拉取模型，接口会返回404响应码并提示你先拉取模型:

404
{'error': 'model "bge-m3:567m1" not found, try pulling it first'}

通过python库使用模型

import ollama
ollama.embeddings(
  model='mxbai-embed-large',
  prompt='Llamas are members of the camelid family',
)

……

大语言模型（指令问答）使用

终端对话

ollama run llama3.2

要求模型具有generate接口。

补充测试

Mac上运行大模型

ollama run qwen:7b-q8_0 # M2 Pro运行时有异常

ollama run qwen:4b # 正常运行

ollama run llama3.2:latest

可见llama8B以下版本模型在中文知识积累欠缺。

Python调用

from ollama import chat, ChatResponse
text = """
   <br>米其林集团(MICHELIN)公布2024年前九个月业绩。当期集团总销售额201.71亿欧元(约213亿美元)，上年同期为211.52亿欧元，同比下降4.6%。其中，汽车业务103.56亿欧元，同比下降2.4%。公路运输业务销售额49.33亿欧元，同比下降4.6%。特种业务销售额48.82亿欧元，同比下降9.1%。
   <br>普利司通(Bridgestone)公布2024年前九个月业绩。当期营收32694.11亿日元(约213亿美元)，上年同期为31981.44亿日元。当期营业利润3771.25亿日元，上年同期为3757.95亿日元。当期归属母公司所有者的净利润2524.52亿日元，上年同期为2656.65亿日元。<br>固特异(Goodyear Tire & Rubber)公布2024年第三季度业绩。季度净销售额48.24亿美元，上年同期为51.42亿美元，同比下降6.2%。季度营业利润3.47亿美元，上年同期为3.336亿美元。季度公司净亏损3400万美元，上年同期净亏损8900万美元。<br>德国汽车零部件供应商大陆集团(Continental)公布2024年第三季度业绩。季度集团销售额98.33亿欧元，上年同期为102.4亿欧元，同比下降4%。季度调整后的EBIT利润8.73亿欧元，上年同期为6.42亿欧元，同比增长36%。季度归属母公司股东净利润4.86亿欧元，上年同期为2.99亿欧元，同比增长62.8%。其中，轮胎子集团销售额34.95亿欧元(约36.95亿美元)，上年同期为34.29亿欧元；EBIT利润5亿欧元，上年同期为4.4亿欧元。<br>住友橡胶(Sumitomo Rubber Industries)公布2024年前九个月业绩。当期营收8749.42亿日元(约57亿美元)，上年同期为8507亿日元。当期营业利润114.85亿日元，上年同期为414.43亿日元，同比下降72.3%。当期归属母公司所有者的净利润40.5亿日元，上年同期为272.05亿日元，同比下降85.1%。<br>倍耐力(Pirelli & C. S.p.A)公布2024年前九个月业绩。当期集团净销售额51.85亿欧元(约54.82亿美元)，上年同期51.6亿欧元。调整后的EBITDA利润11.57亿欧元，上年同期为11.16亿欧元。归属母公司所有者的净利润3.46亿欧元，上年同期为3.93亿欧元。<br>韩泰轮胎(Hankook Tire Co.)公布2024年第三季度业绩。季度销售额24352亿韩元(约17.48亿美元)，上年同期为23400亿韩元。季度营业利润4702亿韩元，上年同期为3963亿韩元。<br>优科豪马(Yokohama Rubber Co., Ltd.)公布2024年前九个月业绩。当期营收7829亿日元(约51亿美元)，上年同期为6925亿日元，同比增长13.1%。当期营业利润857.72亿日元，上年同期为523.66亿日元，同比增长63.8%。当期归属母公司所有者的净利润608.41亿日元，上年同期为445.83亿日元，同比增长36.5%。
   <br>赛轮轮胎发布2024年三季报。前三季度实现营业收入236.28亿元(约32.64亿美元)，同比增长24.3%；净利润32.44亿元，同比增长60.2%。其中，第三季度实现营收84.74亿元(约11.7亿美元)，同比增长14.8%；净利润10.92亿元，同比增长11.5%。
   <br>通伊欧轮胎(Toyo Tire Corporation)公布2024年前九个月业绩。当期净销售额4206.08亿日元(约27.39亿美元)，上年同期为4092.21亿日元。当期营业利润762.37亿日元，上年同期为503.24亿日元。当期归属母公司所有者的净利润551.96亿日元，上年同期为567.68亿日元。
   <br>文章来源：轮胎观察网，不代表本平台观点，仅供参考。感恩原创作者，版权归原作者所有，如若侵权，烦请平台留言删除。<br>欢迎轮胎产业链业界人士进入社群交流商机。包括轮胎原材料供应商群、轮胎经销商1群、轮胎经销商2群、轮胎经销商3群，轮胎区域群、轮胎汽修技师群、卡友群、轮胎商务网车主群、汽车后市场轮胎圈、越野玩家、新能源玩车讨论群等几十个群，加群请扫管理员微信（请注明您企业或职业名称）。<br>▲请打开图片，长按屏幕识别二维码<br>原创投稿/新闻线索tirechina@tirechina.net<br>商务合作 | 林老师 13229655098（微信同号）<br>精彩文章推荐<br>✦【轮胎周报】ApexTire2024盛大启动；轮胎产量破九亿条；10月轮胎出口大幅增长；锦湖轮胎大手笔扩产；普利司通重组业务<br>✦嘉峪关途虎养车工场店里的兄弟搭档：把活儿干好、让车主信任是最大成就<br>✦轮胎行业：贡献超3300亿元！<br>✦涨价潮降温！轮胎原材料全线下滑<br>✦“智启新质，绿色领航” ApexTire2024中国轮胎年度盛大启幕<br>✦智启新质，绿色领航：ApexTire2024中国轮胎年度大选盛大启动！<br>滑动查看下一张图片<br>善养车油轮矩阵抖音平台<br>保存图片并打开抖音关注<br>温馨提醒<br>点亮“在看”，点亮财运<br>
"""
model = 'qwen:4b'

response: ChatResponse = chat(model=model, messages=[
    {
        'role': 'system',
        'content': '你需要对新闻进行总结，总结必须基于事实，不多于200字。直接输出结果即可。',
    },
  {
    'role': 'user',
    'content': text,
  },
])
print(response['message']['content'])
# or access fields directly from the response object
print(response.message.content)

……

构建模型镜像

准备一个gguf模型文件
可以从其他模型文件转格式，也可以直接在huggingface下载开源模型文件。比如：
hfl/llama-3-chinese-8b-instruct-v3-gguf at main
编写Modelfile
参考ollama/docs/modelfile.md at main · ollama/ollama · GitHub
构建模型镜像

ollama create weird-llama3-chinese -f ./Modelfile

【Tips：可以通过ollama show命令查看其他模型的Modelfile，如】

ollama show --modelfile llama3.2

删除模型镜像

ollama rm [MODEL]

工具集成

GitHub - kevinhermawan/Ollamac: Mac app for Ollama

应用案例

https://github.com/HKUDS/LightRAG

Building a Local RAG api with LlamaIndex, Qdrant, Ollama and FastAPI | Otmane Boughaba.

RAG 应用基础示例 - MatrixOne 中文文档

Ollama LLM 模型部署