高级功能附录
更新时间:2024-11-29
高级功能附录
附录1:extension插件说明
功能:extension插件可以提前设定对话的“背景信息”或“语气”,帮助 AI 理解要用什么风格、角色或知识背景来回应用户。
使用方法:
- 准备vllm_extension.py 和 template.json 两个文件(文件示例见下)。两个文件目录层级需要一致。如有其他具体实现可以修改text_to_tokens 函数的逻辑 和 extension/template.json文件。
# !/usr/bin/env python3
"""vllm_extension"""
from transformers import AutoTokenizer
from typing import List, Union
import json
class VllmExtension:
"""VllmExtension"""
def __init__(self): #不能有构造参数,因为自动构建实例的时候不能传入参数;但可以从当前目录获取config
import pathlib
folder_path = pathlib.Path(__file__).parent.resolve()
with open(f'{folder_path}/template.json', encoding='utf-8') as f:
self.template = json.load(f)
def get_tokenizer(self, tokenizer_path: str) -> AutoTokenizer:
"""get_tokenizer"""
return AutoTokenizer.from_pretrained(tokenizer_path, trust_remote_code=True, use_fast=False)
def text_to_tokens(self, tokenizer: AutoTokenizer, text: str) -> List[int]:
"""text_to_tokens"""
text = self.template['prompt'].replace('#CONTENT#', text)
prompt_tokens = tokenizer.encode(text)
return prompt_tokens
{
"prompt": "[Round 1]\n\n问:#CONTENT#\n\n答:"
}
- 启动服务端,使用--tokenizer-extension-path 指定 vllm_extension.py 文件所在路径
python3 -m vllm.entrypoints.openai.api_server \
--model=/path/to/models/Llama-2-7b-hf/ \
--tensor-parallel-size=1 \
--dtype=float16 \
--gpu_memory_utilization=0.8 \
--tokenizer-extension-path=/path/to/vllm_extension.py
- 客户端正常发送请求
客户端发送请求示例:1 + 1
服务端实际处理请求:[Round 1]\n\n问:1 + 1 =\n\n答:
附录2:extension插件替代用法(chat接口使用方法)
- 编写模板文件
以下是一个简单的聊天模板示例:
{% for message in messages %}
{{ '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n' }}
{% endfor %}
{% if add_generation_prompt %}
{{ '<|im_start|>assistant\n' }}
{% endif %}
将上述模板内容保存为 chat_template.jinja2 文件。
假设我们有以下消息:
messages = [
{'role': 'user', 'content': '你好,能给我讲个笑话吗?'}
]
使用上述模板渲染后,得到的格式化输入为:
<|im_start|>user
你好,能给我讲个笑话吗?<|im_end|>
<|im_start|>assistant
- 在启动服务端的时候传入模板文件
python -m vllm.entrypoints.openai.api_server \
--model <您的模型路径或名称> \
--chat-template path_to_chat_template.jinja2 ...
- 使用openai的chat接口,和非chat接口略有区别。
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "your-model-name",
"messages": [
{"role": "user", "content": "你好,能给我讲个笑话吗?"}
]
}'
消息结构:Chat 接口使用包含 role 和 content 的消息列表,而非 Chat 接口通常只发送纯文本。
上下文管理:Chat 接口能够维护对话的上下文,支持多轮交互。
角色区分:通过 role 指定消息的发送者,如 user、assistant、system 等。
附录3: 自定义量化数据集
smooth_tokenids.txt 格式示例
1, 6991, 3034, 675, 278, 1667, 7014, 310, 12208, 19512, 29915, 29879, 10969
997, 3322, 25515, 964, 24334, 3291, 408, 372, 639, 2408, 29879, 304, 263, 14321, 9999
2825, 491, 12208, 19512, 29892, 1058, 756, 1063, 6826, 292, 9316, 7395
具体转化过程示例:
1.原始数据示例:
'It\'s official: U.S. President Barack Obama wants lawmakers to weigh in on whether to use military force in Syria. Obama sent a letter to the heads of the House and Senate on Saturday night, hours after announcing that he believes military action against Syrian targets is the right step to take over the alleged use of chemical weapons. The proposed legislation from Obama asks Congress to approve the use of military force "to deter, disrupt, prevent and degrade the potential for future uses of chemical weapons or other weapons of mass destruction." It\'s a step that is set to turn an international crisis into a fierce domestic political battle. There are key questions looming over the debate: What did U.N. weapons inspectors find in Syria? What happens if Congress votes no? And how will the Syrian government react? In a televised address from the White House Rose Garden earlier Saturday, the president said he would take his case to Congress, not because he has to -- but because he wants to. "While I believe I have the authority to carry out this military action without specific congressional authorization, I know that the country will be stronger if we take this course, and our actions will be even more effective," he said. "We should have this debate, because the issues are too big for business as usual." Obama said top congressional leaders had agreed to schedule a debate when the body returns to Washington on September 9. The Senate Foreign Relations Committee will hold a hearing over the matter on Tuesday, Sen. Robert Menendez said. Transcript: Read Obama\'s full remarks . Syrian crisis: Latest developments . U.N. inspectors leave Syria . Obama\'s remarks came shortly after U.N. inspectors left Syria, carrying evidence that will determine whether chemical weapons were used in an attack early last week in a Damascus suburb. "The aim of the game here, the mandate, is very clear -- and that is to ascertain whether chemical weapons were used -- and not by whom," U.N. spokesman Martin Nesirky told reporters on Saturday. But who used the weapons in the reported toxic gas attack in a Damascus suburb on August 21 has been a key point of global debate over the Syrian crisis. Top U.S. officials have said there\'s no doubt that the Syrian government was behind it, while Syrian officials have denied responsibility and blamed jihadists fighting with the rebels. British and U.S. intelligence reports say the attack involved chemical weapons, but U.N. officials have stressed the importance of waiting for an official report from inspectors. The inspectors will share their findings with U.N. Secretary-General Ban Ki-moon Ban, who has said he wants to wait until the U.N. team\'s final report is completed before presenting it to the U.N. Security Council. The Organization for the Prohibition of Chemical Weapons, which nine of the inspectors belong to, said Saturday that it could take up to three weeks to analyze the evidence they collected. "It needs time to be able to analyze the information and the samples," Nesirky said. He noted that Ban has repeatedly said there is no alternative to a political solution to the crisis in Syria, and that "a military solution is not an option." Bergen: Syria is a problem from hell for the U.S. Obama: \'This menace must be confronted\' Obama\'s senior advisers have debated the next steps to take, and the president\'s comments Saturday came amid mounting political pressure over the situation in Syria. Some U.S. lawmakers have called for immediate action while others warn of stepping into what could become a quagmire. Some global leaders have expressed support, but the British Parliament\'s vote against military action earlier this week was a blow to Obama\'s hopes of getting strong backing from key NATO allies. On Saturday, Obama proposed what he said would be a limited military action against Syrian President Bashar al-Assad. Any military attack would not be open-ended or include U.S. ground forces, he said. Syria\'s alleged use of chemical weapons earlier this month "is an assault on human dignity," the president said. A failure to respond with force, Obama argued, "could lead to escalating use of chemical weapons or their proliferation to terrorist groups who would do our people harm. In a world with many dangers, this menace must be confronted." Syria missile strike: What would happen next? Map: U.S. and allied assets around Syria . Obama decision came Friday night . On Friday night, the president made a last-minute decision to consult lawmakers. What will happen if they vote no? It\'s unclear. A senior administration official told CNN that Obama has the authority to act without Congress -- even if Congress rejects his request for authorization to use force. Obama on Saturday continued to shore up support for a strike on the al-Assad government. He spoke by phone with French President Francois Hollande before his Rose Garden speech. "The two leaders agreed that the international community must deliver a resolute message to the Assad regime -- and others who would consider using chemical weapons -- that these crimes are unacceptable and those who violate this international norm will be held accountable by the world," the White House said. Meanwhile, as uncertainty loomed over how Congress would weigh in, U.S. military officials said they remained at the ready. 5 key assertions: U.S. intelligence report on Syria . Syria: Who wants what after chemical weapons horror . Reactions mixed to Obama\'s speech . A spokesman for the Syrian National Coalition said that the opposition group was disappointed by Obama\'s announcement. "Our fear now is that the lack of action could embolden the regime and they repeat his attacks in a more serious way," said spokesman Louay Safi. "So we are quite concerned." Some members of Congress applauded Obama\'s decision. House Speaker John Boehner, Majority Leader Eric Cantor, Majority Whip Kevin McCarthy and Conference Chair Cathy McMorris Rodgers issued a statement Saturday praising the president. "Under the Constitution, the responsibility to declare war lies with Congress," the Republican lawmakers said. "We are glad the president is seeking authorization for any military action in Syria in response to serious, substantive questions being raised." More than 160 legislators, including 63 of Obama\'s fellow Democrats, had signed letters calling for either a vote or at least a "full debate" before any U.S. action. British Prime Minister David Cameron, whose own attempt to get lawmakers in his country to support military action in Syria failed earlier this week, responded to Obama\'s speech in a Twitter post Saturday. "I understand and support Barack Obama\'s position on Syria," Cameron said. An influential lawmaker in Russia -- which has stood by Syria and criticized the United States -- had his own theory. "The main reason Obama is turning to the Congress: the military operation did not get enough support either in the world, among allies of the US or in the United States itself," Alexei Pushkov, chairman of the international-affairs committee of the Russian State Duma, said in a Twitter post. In the United States, scattered groups of anti-war protesters around the country took to the streets Saturday. "Like many other Americans...we\'re just tired of the United States getting involved and invading and bombing other countries," said Robin Rosecrans, who was among hundreds at a Los Angeles demonstration. What do Syria\'s neighbors think? Why Russia, China, Iran stand by Assad . Syria\'s government unfazed . After Obama\'s speech, a military and political analyst on Syrian state TV said Obama is "embarrassed" that Russia opposes military action against Syria, is "crying for help" for someone to come to his rescue and is facing two defeats -- on the political and military levels. Syria\'s prime minister appeared unfazed by the saber-rattling. "The Syrian Army\'s status is on maximum readiness and fingers are on the trigger to confront all challenges," Wael Nader al-Halqi said during a meeting with a delegation of Syrian expatriates from Italy, according to a banner on Syria State TV that was broadcast prior to Obama\'s address. An anchor on Syrian state television said Obama "appeared to be preparing for an aggression on Syria based on repeated lies." A top Syrian diplomat told the state television network that Obama was facing pressure to take military action from Israel, Turkey, some Arabs and right-wing extremists in the United States. "I think he has done well by doing what Cameron did in terms of taking the issue to Parliament," said Bashar Jaafari, Syria\'s ambassador to the United Nations. Both Obama and Cameron, he said, "climbed to the top of the tree and don\'t know how to get down." The Syrian government has denied that it used chemical weapons in the August 21 attack, saying that jihadists fighting with the rebels used them in an effort to turn global sentiments against it. British intelligence had put the number of people killed in the attack at more than 350. On Saturday, Obama said "all told, well over 1,000 people were murdered." U.S. Secretary of State John Kerry on Friday cited a death toll of 1,429, more than 400 of them children. No explanation was offered for the discrepancy. Iran: U.S. military action in Syria would spark \'disaster\' Opinion: Why strikes in Syria are a bad idea . TL;DR:'
'(CNN) -- Usain Bolt rounded off the world championships Sunday by claiming his third gold in Moscow as he anchored Jamaica to victory in the men\'s 4x100m relay. The fastest man in the world charged clear of United States rival Justin Gatlin as the Jamaican quartet of Nesta Carter, Kemar Bailey-Cole, Nickel Ashmeade and Bolt won in 37.36 seconds. The U.S finished second in 37.56 seconds with Canada taking the bronze after Britain were disqualified for a faulty handover. The 26-year-old Bolt has now collected eight gold medals at world championships, equaling the record held by American trio Carl Lewis, Michael Johnson and Allyson Felix, not to mention the small matter of six Olympic titles. The relay triumph followed individual successes in the 100 and 200 meters in the Russian capital. "I\'m proud of myself and I\'ll continue to work to dominate for as long as possible," Bolt said, having previously expressed his intention to carry on until the 2016 Rio Olympics. Victory was never seriously in doubt once he got the baton safely in hand from Ashmeade, while Gatlin and the United States third leg runner Rakieem Salaam had problems. Gatlin strayed out of his lane as he struggled to get full control of their baton and was never able to get on terms with Bolt. Earlier, Jamaica\'s women underlined their dominance in the sprint events by winning the 4x100m relay gold, anchored by Shelly-Ann Fraser-Pryce, who like Bolt was completing a triple. Their quartet recorded a championship record of 41.29 seconds, well clear of France, who crossed the line in second place in 42.73 seconds. Defending champions, the United States, were initially back in the bronze medal position after losing time on the second handover between Alexandria Anderson and English Gardner, but promoted to silver when France were subsequently disqualified for an illegal handover. The British quartet, who were initially fourth, were promoted to the bronze which eluded their men\'s team. Fraser-Pryce, like Bolt aged 26, became the first woman to achieve three golds in the 100-200 and the relay. In other final action on the last day of the championships, France\'s Teddy Tamgho became the third man to leap over 18m in the triple jump, exceeding the mark by four centimeters to take gold. Germany\'s Christina Obergfoll finally took gold at global level in the women\'s javelin after five previous silvers, while Kenya\'s Asbel Kiprop easily won a tactical men\'s 1500m final. Kiprop\'s compatriot Eunice Jepkoech Sum was a surprise winner of the women\'s 800m. Bolt\'s final dash for golden glory brought the eight-day championship to a rousing finale, but while the hosts topped the medal table from the United States there was criticism of the poor attendances in the Luzhniki Stadium. There was further concern when their pole vault gold medalist Yelena Isinbayeva made controversial remarks in support of Russia\'s new laws, which make "the propagandizing of non-traditional sexual relations among minors" a criminal offense. She later attempted to clarify her comments, but there were renewed calls by gay rights groups for a boycott of the 2014 Winter Games in Sochi, the next major sports event in Russia. TL;DR:'
'Kansas City, Missouri (CNN) -- The General Services Administration, already under investigation for lavish spending, allowed an employee to telecommute from Hawaii even though he is based at the GSA\'s Kansas City, Missouri, office, a CNN investigation has found. It cost more than $24,000 for the business development specialist to travel to and from the mainland United States over the past year. He is among several hundred GSA "virtual" workers who also travel to various conferences and their home offices, costing the agency millions of dollars over the past three years. Under the program, employees work from home and may live in another state from the region in which they\'re actually assigned. The Kansas City employee, who started his job in January 2011, is paid $84,440 and works from his home in Honolulu, a GSA representative confirmed. In the past year, according to GSA travel records, the employee has flown back to the mainland nine times for conferences and meetings. Four of those trips were to St. Louis; four were to Washington, with a side trip to Cincinnati; and one was to San Diego. The total cost to taxpayers was $24,221. Jason Klumb, the GSA\'s regional administrator for Kansas City, defended the hire. "The cost of that travel was included in the consideration of his candidacy as an employee as compared with the other applicants," Klumb said. "And when factoring all of those in, it was determined that he was the best candidate, even in light of the cost that would be incurred." Klumb called the GSA\'s teleworking program "a successful program that\'s going to lead to cost savings for taxpayers." But a GSA spokeswoman said, "We are not going to defend this type of travel." And a GSA employee in Kansas City, who requested anonymity, said that hiring someone in Hawaii to work for the Kansas City region was ludicrous. "It doesn\'t make sense," the employee said. "When you consider everything you need when you hire someone, it would have been better to look for someone in the Kansas City area. It would have reduced the cost of travel by at least 70 percent when you look at just the airfare of what it takes to from Honolulu to Washington, D.C., where a lot of business is done." Dan Tangherlini, who was appointed acting GSA administrator this year, said the agency was examining the cost of the entire teleworking program. "I think the most important part for the GSA to think about is make sure we open ourselves up, avail ourselves to all the smart people in the country, but then also make sure we have a clear business case," he said. "If we have someone who is working in Nebraska but reporting to Boston, there has to be a clear explanation for what value they\'re providing, and you\'ve got to give me the business case. You\'ve got to explain to me why that\'s a cost-effective move for the American people, and that\'s a new standard that we\'re asking everyone at GSA to adhere to." The GSA "virtual employee" program is different from telework programs offered by many private companies including CNN\'s parent company, Turner Broadcasting, in which some employees are encouraged to work from home some days of the week, partially to reduce traffic congestion. The House Committee on Oversight and Government Reform requested details about the GSA\'s teleworking program in June. That followed disclosures that 95 virtual employees, including 12 in supervisory positions, spent nearly $750,000 in travel costs between October 2010 and June 2011. "The American people have a right to know that federal bureaucrats who enjoy the benefits of virtual work are eligible and responsible stewards of the taxpayer dollars that support the program," according to a letter from committee Chairman Rep. Darrell Issa, R-California, to the GSA. The details requested by Issa about the GSA program have not been provided to the committee. CNN also requested the information more than two months ago through the federal Freedom of Information Act but has been repeatedly told by the GSA that FOIA staff members have not finished compiling the material. The General Services Administration, which has more than 12,600 employees and a $26.3 billion budget, is a relatively obscure federal agency that handles government real estate and other non-military procurement. Congress launched an investigation into the GSA after a scathing inspector general\'s report issued this year showed lavish spending -- $823,000 -- at the agency\'s Western Regions Conference in Las Vegas in October 2010. The controversy became politically toxic after reports and video clips of the lavish conference were released. The revelation prompted taxpayer indignation, embarrassed the administration and put a spotlight on wasteful spending by the GSA. Jeff Neely, the GSA official who organized the conference, resigned, as did the agency\'s administrator, Martha Johnson. Two of Johnson\'s deputies were fired, and eight other employees left the agency. Tangherlini, a former Treasury Department official, took over as acting GSA administrator. In addition to the Las Vegas conference, the GSA apparently spent $330,000 to relocate an employee from Denver to Hawaii and probably millions more on other employees over a two-year period, according to a transcript of an interview with a GSA event planner. And 84 GSA employees, most of them supervisors or other senior staff -- all subjects of inspector general investigations -- are still collecting their bonuses, totaling more than $1 million in taxpayer money. In July, a CNN investigation revealed that the GSA\'s Kansas City office spent more than $20,000 to send employees to cooking classes to build team spirit. While the classes do not amount to a significant sum of money in the world of trillion-dollar government budgets, insiders said it was part of the free-spending culture that went on for years at the GSA\'s Kansas City regional headquarters. GSA spokeswoman Betsaida Alcantara said in a statement this year that all the agency\'s practices are under a top-down review. CNN\'s Sara Anwar, Elizabeth M. Nunez and Tom Cohen contributed to this report. Watch Erin Burnett weekdays 7pm ET. For the latest from Erin Burnett click here. TL;DR:'
2.转化脚本示例
from datasets import load_dataset
from transformers import AutoTokenizer
from tqdm import tqdm
import torch
model_path = "/ssd2/models/huggingface.co/meta-llama/Llama-2-7b-hf/"
data_path = "./data.txt"
output_path = "./output.txt"
tokenizer = AutoTokenizer.from_pretrained(model_path)
dataset = load_dataset('text', data_files=data_path)
list_token_ids = []
for i in tqdm(range(len(dataset['train'])), desc="calibrating model"):
datapoint = dataset['train'][i:i+1]
datapoint=datapoint['text'][0]
line_encoded = tokenizer(datapoint,
return_tensors="pt",
truncation=True)["input_ids"].type(torch.int64)
print(line_encoded.tolist()[0])
list_token_ids.append(line_encoded.tolist()[0])
with open(output_path, "w") as file:
for line in list_token_ids:
file.write(str(line).strip("[]") + "\n")
3.得到转化后的结果示例
1, 525, 3112, 20333, 29879, 6221, 29901, 501, 29889, 29903, 29889, 7178, 2261, 547, 4250, 3304, 10753, 4307, 29885, 21079, 304, 591, 1141, 297, 373, 3692, 304, 671, 9121, 4889, 297, 8713, 2849, 29889, 4250, 3304, 2665, 263, 5497, 304, 278, 15883, 310, 278, 5619, 322, 18148, 373, 24211, 4646, 29892, 6199, 1156, 7475, 3277, 393, 540, 1339, 17180, 9121, 3158, 2750, 8713, 6392, 22525, 338, 278, 1492, 4331, 304, 2125, 975, 278, 16831, 287, 671, 310, 22233, 25340, 29889, 450, 7972, 13332, 362, 515, 4250, 3304, 19514, 11559, 304, 2134, 345, 278, 671, 310, 9121, 4889, 376, 517, 270, 1308, 29892, 766, 6685, 29892, 5557, 322, 3587, 15464, 278, 7037, 363, 5434, 3913, 310, 22233, 25340, 470, 916, 25340, 310, 4158, 22104, 1213, 739, 20333, 29879, 263, 4331, 393, 338, 731, 304, 2507, 385, 6121, 24161, 964, 263, 21334, 346, 21849, 8604, 10555, 29889, 1670, 526, 1820, 5155, 658, 28826, 975, 278, 27836, 29901, 1724, 1258, 501, 29889, 29940, 29889, 25340, 16096, 943, 1284, 297, 8713, 2849, 29973, 1724, 5930, 565, 11559, 18952, 694, 29973, 1126, 920, 674, 278, 8713, 6392, 5874, 7657, 29973, 512, 263, 4382, 11292, 3211, 515, 278, 8037, 5619, 14008, 19906, 8859, 24211, 29892, 278, 6673, 1497, 540, 723, 2125, 670, 1206, 304, 11559, 29892, 451, 1363, 540, 756, 304, 1192, 541, 1363, 540, 10753, 304, 29889, 376, 8809, 488, 306, 4658, 306, 505, 278, 14329, 304, 8677, 714, 445, 9121, 3158, 1728, 2702, 378, 11476, 284, 28733, 29892, 306, 1073, 393, 278, 4234, 674, 367, 23505, 565, 591, 2125, 445, 3236, 29892, 322, 1749, 8820, 674, 367, 1584, 901, 11828, 1699, 540, 1497, 29889, 376, 4806, 881, 505, 445, 27836, 29892, 1363, 278, 5626, 526, 2086, 4802, 363, 5381, 408, 9670, 1213, 4250, 3304, 1497, 2246, 378, 11476, 284, 20251, 750, 15502, 304, 20410, 263, 27836, 746, 278, 3573, 3639, 304, 7660, 373, 3839, 29871, 29929, 29889, 450, 18148, 19358, 6376, 800, 12930, 674, 4808, 263, 22514, 975, 278, 4383, 373, 323, 1041, 3250, 29892, 5811, 29889, 4755, 7567, 3324, 29920, 1497, 29889, 4103, 924, 29901, 7523, 4250, 3304, 20333, 29879, 2989, 29360, 869, 8713, 6392, 24161, 29901, 7053, 342, 2693, 1860, 869, 501, 29889, 29940, 29889, 16096, 943, 5967, 8713, 2849, 869, 4250, 3304, 20333, 29879, 29360, 2996, 21734, 1156, 501, 29889, 29940, 29889, 16096, 943, 2175, 8713, 2849, 29892, 19436, 10757, 393, 674, 8161, 3692, 22233, 25340, 892, 1304, 297, 385, 5337, 4688, 1833, 4723, 297, 263, 9865, 6151, 375, 1014, 9265, 29889, 376, 1576, 12242, 310, 278, 3748, 1244, 29892, 278, 9619, 403, 29892, 338, 1407, 2821, 1192, 322, 393, 338, 304, 408, 14082, 3692, 22233, 25340, 892, 1304, 1192, 322, 451, 491, 6029, 1699, 501, 29889, 29940, 29889, 805, 23195, 1171, 6502, 405, 267, 381, 3459, 5429, 1634, 272, 2153, 373, 24211, 29889, 1205, 1058, 1304, 278, 25340, 297, 278, 8967, 304, 27375, 10489, 5337, 297, 263, 9865, 6151, 375, 1014, 9265, 373, 3111, 29871, 29906, 29896, 756, 1063, 263, 1820, 1298, 310, 5534, 27836, 975, 278, 8713, 6392, 24161, 29889, 7488, 501, 29889, 29903, 29889, 24921, 505, 1497, 727, 20333, 29879, 694, 7404, 393, 278, 8713, 6392, 5874, 471, 5742, 372, 29892, 1550, 8713, 6392, 24921, 505, 17935, 23134, 322, 1999, 2795, 432, 4861, 328, 2879, 17770, 411, 278, 15121, 1379, 29889, 4908, 322, 501, 29889, 29903, 29889, 21082, 13676, 1827, 278, 5337, 9701, 22233, 25340, 29892, 541, 501, 29889, 29940, 29889, 24921, 505, 851, 11517, 278, 13500, 310, 10534, 363, 385, 6221, 3461, 515, 16096, 943, 29889, 450, 16096, 943, 674, 6232, 1009, 1284, 886, 411, 501, 29889, 29940, 29889, 17719, 29899, 15263, 10765, 16540, 29899, 4346, 265, 10765, 29892, 1058, 756, 1497, 540, 10753, 304, 4480, 2745, 278, 501, 29889, 29940, 29889, 3815, 20333, 29879, 2186, 3461, 338, 8676, 1434, 2198, 292, 372, 304, 278, 501, 29889, 29940, 29889, 14223, 8831, 29889, 450, 9205, 2133, 363, 278, 1019, 6335, 654, 310, 12677, 936, 1334, 481, 787, 29892, 607, 14183, 310, 278, 16096, 943, 6852, 304, 29892, 1497, 24211, 393, 372, 1033, 2125, 701, 304, 2211, 11405, 304, 27599, 278, 10757, 896, 16531, 29889, 376, 3112, 4225, 931, 304, 367, 2221, 304, 27599, 278, 2472, 322, 278, 11916, 1699, 405, 267, 381, 3459, 1497, 29889, 940, 11682, 393, 10765, 756, 28424, 1497, 727, 338, 694, 8671, 304, 263, 8604, 1650, 304, 278, 24161, 297, 8713, 2849, 29892, 322, 393, 376, 29874, 9121, 1650, 338, 451, 385, 2984, 1213, 2292, 1885, 29901, 29871, 8713, 2849, 338, 263, 1108, 515, 23927, 363, 278, 501, 29889, 29903, 29889, 4250, 3304, 29901, 320, 29915, 4013, 1757, 815, 1818, 367, 21751, 287, 20333, 4250, 3304, 20333, 29879, 16336, 25228, 414, 505, 2553, 630, 278, 2446, 6576, 304, 2125, 29892, 322, 278, 6673, 20333, 29879, 6589, 24211, 2996, 28655, 5766, 292, 8604, 12959, 975, 278, 6434, 297, 8713, 2849, 29889, 3834, 501, 29889, 29903, 29889, 4307, 29885, 21079, 505, 2000, 363, 16800, 3158, 1550, 4045, 29383, 310, 1886, 3262, 964, 825, 1033, 4953, 263, 439, 351, 29885, 533, 29889, 3834, 5534, 20251, 505, 13384, 2304, 29892, 541, 278, 4908, 13212, 20333, 29879, 11719, 2750, 9121, 3158, 8859, 445, 4723, 471, 263, 13031, 304, 4250, 3304, 20333, 29879, 26926, 310, 2805, 4549, 27436, 515, 1820, 26038, 29949, 394, 3687, 29889, 1551, 24211, 29892, 4250, 3304, 7972, 825, 540, 1497, 723, 367, 263, 9078, 9121, 3158, 2750, 8713, 6392, 7178, 29677, 279, 394, 29899, 7900, 328, 29889, 3139, 9121, 5337, 723, 451, 367, 1722, 29899, 2760, 470, 3160, 501, 29889, 29903, 29889, 5962, 8249, 29892, 540, 1497, 29889, 8713, 2849, 20333, 29879, 16831, 287, 671, 310, 22233, 25340, 8859, 445, 4098, 376, 275, 385, 29159, 373, 5199, 18085, 537, 1699, 278, 6673, 1497, 29889, 319, 10672, 304, 10049, 411, 4889, 29892, 4250, 3304, 28705, 29892, 29871, 376, 26680, 3275, 304, 831, 1052, 1218, 671, 310, 22233, 25340, 470, 1009, 410, 29880, 9633, 362, 304, 15115, 391, 6471, 1058, 723, 437, 1749, 2305, 10311, 29889, 512, 263, 3186, 411, 1784, 270, 13873, 29892, 445, 1757, 815, 1818, 367, 21751, 287, 1213, 8713, 2849, 3052, 488, 21283, 29901, 1724, 723, 3799, 2446, 29973, 7315, 29901, 501, 29889, 29903, 29889, 322, 394, 2957, 21608, 2820, 8713, 2849, 869, 4250, 3304, 10608, 2996, 28728, 4646, 869, 1551, 28728, 4646, 29892, 278, 6673, 1754, 263, 1833, 29899, 1195, 1082, 10608, 304, 8799, 4307, 29885, 21079, 29889, 1724, 674, 3799, 565, 896, 11719, 694, 29973, 739, 20333, 29879, 20871, 29889, 319, 16336, 17517, 6221, 5429, 29696, 393, 4250, 3304, 756, 278, 14329, 304, 1044, 1728, 11559, 1192, 1584, 565, 11559, 12560, 29879, 670, 2009, 363, 28733, 304, 671, 4889, 29889, 4250, 3304, 373, 24211, 7572, 304, 19055, 701, 2304, 363, 263, 21283, 373, 278, 394, 29899, 7900, 328, 5874, 29889, 940, 12707, 491, 9008, 411, 5176, 7178, 20923, 275, 379, 3028, 4182, 1434, 670, 14008, 19906, 12032, 29889, 376, 1576, 1023, 20251, 15502, 393, 278, 6121, 7881, 1818, 12021, 263, 3770, 1082, 2643, 304, 278, 4007, 328, 22384, 1192, 322, 4045, 1058, 723, 2050, 773, 22233, 25340, 1192, 393, 1438, 2181, 1355, 526, 443, 16044, 519, 322, 1906, 1058, 5537, 403, 445, 6121, 6056, 674, 367, 4934, 3633, 519, 491, 278, 3186, 1699, 278, 8037, 5619, 1497, 29889, 25065, 29892, 408, 25812, 658, 27067, 975, 920, 11559, 723, 591, 1141, 297, 29892, 501, 29889, 29903, 29889, 9121, 24921, 1497, 896, 9488, 472, 278, 7960, 29889, 29871, 29945, 1820, 4974, 1080, 29901, 501, 29889, 29903, 29889, 21082, 3461, 373, 8713, 2849, 869, 8713, 2849, 29901, 11644, 10753, 825, 1156, 22233, 25340, 23026, 869, 830, 7387, 12849, 304, 4250, 3304, 20333, 29879, 12032, 869, 319, 805, 23195, 1171, 363, 278, 8713, 6392, 3086, 3189, 284, 654, 1497, 393, 278, 19626, 2318, 471, 23451, 287, 491, 4250, 3304, 20333, 29879, 7475, 13561, 29889, 376, 29949, 332, 8866, 1286, 338, 393, 278, 10225, 310, 3158, 1033, 7232, 1025, 264, 278, 22384, 322, 896, 12312, 670, 16661, 297, 263, 901, 10676, 982, 1699, 1497, 805, 23195, 1171, 4562, 388, 14795, 29875, 29889, 376, 6295, 591, 526, 3755, 15041, 1213, 3834, 5144, 310, 11559, 623, 433, 566, 287, 4250, 3304, 20333, 29879, 10608, 29889, 5619, 5013, 5790, 2259, 1952, 14797, 1089, 29892, 11019, 537, 951, 1664, 14713, 15498, 272, 29892, 11019, 537, 806, 666, 19323, 4052, 8179, 21155, 322, 16377, 24193, 315, 493, 29891, 4052, 28581, 3780, 7733, 5743, 16610, 263, 3229, 24211, 7213, 5921, 278, 6673, 29889, 376, 29177, 278, 20063, 29892, 278, 23134, 304, 9607, 1370, 12185, 411, 11559, 1699, 278, 21178, 4307, 29885, 21079, 1497, 29889, 376, 4806, 526, 10932, 278, 6673, 338, 25738, 28733, 363, 738, 9121, 3158, 297, 8713, 2849, 297, 2933, 304, 10676, 29892, 20446, 573, 5155, 1641, 10425, 1213, 5853, 1135, 29871, 29896, 29953, 29900, 13332, 4097, 29892, 3704, 29871, 29953, 29941, 310, 4250, 3304, 20333, 29879, 10404, 14189, 1446, 29892, 750, 8794, 8721, 5432, 363, 2845, 263, 11719, 470, 472, 3203, 263, 376, 8159, 27836, 29908, 1434, 738, 501, 29889, 29903, 29889, 3158, 29889, 4908, 15512, 7668, 4699, 20939, 265, 29892, 5069, 1914, 4218, 304, 679, 4307, 29885, 21079, 297, 670, 4234, 304, 2304, 9121, 3158, 297, 8713, 2849, 5229, 8859, 445, 4723, 29892, 10049, 287, 304, 4250, 3304, 20333, 29879, 12032, 297, 263, 20147, 1400, 24211, 29889, 376, 29902, 2274, 322, 2304, 2261, 547, 4250, 3304, 20333, 29879, 2602, 373, 8713, 2849, 1699, 20939, 265, 1497, 29889, 530, 7112, 2556, 4307, 28107, 297, 12710, 1192, 607, 756, 8389, 491, 8713, 2849, 322, 11164, 1891, 278, 3303, 3900, 1192, 750, 670, 1914, 6368, 29889, 376, 1576, 1667, 2769, 4250, 3304, 338, 14712, 304, 278, 11559, 29901, 29871, 278, 9121, 5858, 1258, 451, 679, 3307, 2304, 2845, 297, 278, 3186, 29892, 4249, 394, 3687, 310, 278, 3148, 470, 297, 278, 3303, 3900, 3528, 1699, 4827, 10096, 349, 1878, 9756, 29892, 28942, 310, 278, 6121, 29899, 3470, 7121, 21118, 310, 278, 10637, 4306, 360, 10859, 29892, 1497, 297, 263, 20147, 1400, 29889, 512, 278, 3303, 3900, 29892, 29574, 6471, 310, 9418, 29899, 4495, 10021, 414, 2820, 278, 4234, 3614, 304, 278, 19756, 24211, 29889, 376, 27552, 1784, 916, 23035, 856, 705, 20333, 276, 925, 23407, 310, 278, 3303, 3900, 2805, 9701, 322, 2437, 9382, 322, 13585, 292, 916, 10916, 1699, 1497, 13104, 14008, 7283, 550, 29892, 1058, 471, 4249, 21006, 472, 263, 4602, 10722, 9004, 362, 29889, 1724, 437, 8713, 2849, 20333, 29879, 22092, 943, 1348, 29973, 3750, 12710, 29892, 7551, 29892, 14883, 2317, 491, 4007, 328, 869, 8713, 2849, 20333, 29879, 5874, 29395, 834, 287, 869, 2860, 4250, 3304, 20333, 29879, 12032, 29892, 263, 9121, 322, 8604, 3483, 858, 373, 8713, 6392, 2106, 5648, 1497, 4250, 3304, 338, 376, 1590, 2749, 465, 287, 29908, 393, 12710, 9209, 267, 9121, 3158, 2750, 8713, 2849, 29892, 338, 376, 29883, 719, 292, 363, 1371, 29908, 363, 4856, 304, 2041, 304, 670, 26429, 322, 338, 14870, 1023, 8686, 1446, 1192, 373, 278, 8604, 322, 9121, 11174, 29889, 8713, 2849, 20333, 29879, 6019, 11050, 7470, 29395, 834, 287, 491, 278, 15296, 261, 29899, 29878, 1131, 1847, 29889, 376, 1576, 8713, 6392, 8811, 20333, 29879, 4660, 338, 373, 7472, 1303, 3335, 322, 23915, 526, 373, 278, 7135, 304, 21751, 599, 18066, 267, 1699, 399, 4271, 405, 1664, 394, 29899, 29950, 284, 26461, 1497, 2645, 263, 11781, 411, 263, 16000, 362, 310, 8713, 6392, 1518, 25054, 1078, 515, 12730, 29892, 5034, 304, 263, 289, 7310, 373, 8713, 2849, 4306, 5648, 393, 471, 12672, 7536, 304, 4250, 3304, 20333, 29879, 3211, 29889, 530, 17360, 373, 8713, 6392, 2106, 11456, 1497, 4250, 3304, 376, 932, 21128, 304, 367, 10223, 292, 363, 385, 946, 11476, 373, 8713, 2849, 2729, 373, 10324, 12185, 1213, 319, 2246, 8713, 6392, 13487, 271, 5429, 278, 2106, 11456, 3564, 393, 4250, 3304, 471, 14870, 12959, 304, 2125, 9121, 3158, 515, 11996, 29892, 26459, 29892, 777, 10387, 29879, 322, 1492, 29899, 16958, 9413, 2879, 297, 278, 3303, 3900, 29889, 376, 29902, 1348, 540, 756, 2309, 1532, 491, 2599, 825, 20939, 265, 1258, 297, 4958, 310, 5622, 278, 2228, 304, 13212, 1699, 1497, 29677, 279, 14021, 2142, 1306, 29892, 8713, 2849, 20333, 29879, 3181, 28760, 304, 278, 3303, 18269, 29889, 9134, 4250, 3304, 322, 20939, 265, 29892, 540, 1497, 29892, 376, 695, 326, 2580, 304, 278, 2246, 310, 278, 5447, 322, 1016, 20333, 29873, 1073, 920, 304, 679, 1623, 1213, 450, 8713, 6392, 5874, 756, 17935, 393, 372, 1304, 22233, 25340, 297, 278, 3111, 29871, 29906, 29896, 5337, 29892, 5934, 393, 432, 4861, 328, 2879, 17770, 411, 278, 15121, 1379, 1304, 963, 297, 385, 7225, 304, 2507, 5534, 2665, 7862, 2750, 372, 29889, 4908, 21082, 750, 1925, 278, 1353, 310, 2305, 9445, 297, 278, 5337, 472, 901, 1135, 29871, 29941, 29945, 29900, 29889, 1551, 24211, 29892, 4250, 3304, 1497, 376, 497, 5429, 29892, 1532, 975, 29871, 29896, 29892, 29900, 29900, 29900, 2305, 892, 13406, 287, 1213, 501, 29889, 29903, 29889, 17719, 310, 4306, 2259, 12693, 719, 373, 28728, 274, 1573, 263, 4892, 304, 645, 310, 29871, 29896, 29892, 29946, 29906, 29929, 29892, 901, 1135, 29871, 29946, 29900, 29900, 310, 963, 4344, 29889, 1939, 8252, 471, 12520, 363, 278, 766, 1037, 29886, 6906, 29889, 14883, 29901, 501, 29889, 29903, 29889, 9121, 3158, 297, 8713, 2849, 723, 16267, 320, 29915, 2218, 1901, 20333, 6461, 262, 291, 29901, 3750, 19492, 267, 297, 8713, 2849, 526, 263, 4319, 2969, 869, 323, 29931, 29936, 8353, 11283
1, 525, 29898, 29907, 10262, 29897, 1192, 10783, 475, 8922, 29873, 28240, 1283, 278, 3186, 8064, 9981, 16340, 491, 5995, 292, 670, 4654, 7684, 297, 25820, 408, 540, 23791, 4395, 435, 3304, 983, 304, 15354, 297, 278, 1757, 20333, 29879, 29871, 29946, 29916, 29896, 29900, 29900, 29885, 29856, 29889, 450, 5172, 342, 767, 297, 278, 3186, 20139, 2821, 310, 3303, 3900, 17055, 26408, 402, 271, 1915, 408, 278, 435, 3304, 2185, 10256, 300, 310, 405, 4405, 22264, 29892, 476, 331, 279, 6000, 15168, 29899, 29907, 1772, 29892, 13853, 295, 12835, 1004, 1943, 322, 8922, 29873, 2113, 297, 29871, 29941, 29955, 29889, 29941, 29953, 6923, 29889, 450, 501, 29889, 29903, 7743, 1473, 297, 29871, 29941, 29955, 29889, 29945, 29953, 6923, 411, 7400, 5622, 278, 18449, 1156, 14933, 892, 766, 15380, 2164, 363, 263, 12570, 29891, 1361, 957, 29889, 450, 29871, 29906, 29953, 29899, 6360, 29899, 1025, 8922, 29873, 756, 1286, 16531, 9475, 7684, 1612, 1338, 472, 3186, 8064, 9981, 29892, 5186, 292, 278, 2407, 4934, 491, 3082, 3367, 29877, 8965, 15037, 29892, 5765, 11717, 322, 838, 368, 1100, 26346, 29892, 451, 304, 3585, 278, 2319, 4383, 310, 4832, 19025, 17735, 29889, 450, 29856, 24124, 5643, 5375, 2551, 267, 297, 278, 29871, 29896, 29900, 29900, 322, 29871, 29906, 29900, 29900, 27881, 297, 278, 10637, 7483, 29889, 376, 29902, 20333, 29885, 22314, 310, 6142, 322, 306, 20333, 645, 6773, 304, 664, 304, 8022, 403, 363, 408, 1472, 408, 1950, 1699, 8922, 29873, 1497, 29892, 2534, 9251, 13384, 670, 16392, 304, 8677, 373, 2745, 278, 29871, 29906, 29900, 29896, 29953, 11611, 16373, 29889, 7229, 706, 471, 2360, 25798, 297, 7404, 2748, 540, 2355, 278, 289, 14114, 23511, 297, 1361, 515, 12835, 1004, 1943, 29892, 1550, 402, 271, 1915, 322, 278, 3303, 3900, 4654, 2814, 28877, 29350, 347, 331, 317, 2883, 314, 750, 4828, 29889, 402, 271, 1915, 380, 25724, 714, 310, 670, 301, 1662, 408, 540, 10205, 839, 304, 679, 2989, 2761, 310, 1009, 289, 14114, 322, 471, 2360, 2221, 304, 679, 373, 4958, 411, 8922, 29873, 29889, 5290, 4926, 29892, 435, 3304, 983, 20333, 29879, 5866, 1090, 21354, 1009, 8022, 749, 297, 278, 269, 2158, 4959, 491, 15613, 278, 29871, 29946, 29916, 29896, 29900, 29900, 29885, 29856, 7684, 29892, 23791, 4395, 491, 1383, 14112, 29899, 2744, 29876, 383, 3417, 261, 29899, 29925, 719, 346, 29892, 1058, 763, 8922, 29873, 471, 1614, 1259, 263, 21954, 29889, 11275, 10256, 300, 10478, 263, 22401, 2407, 310, 29871, 29946, 29896, 29889, 29906, 29929, 6923, 29892, 1532, 2821, 310, 3444, 29892, 1058, 21692, 278, 1196, 297, 1473, 2058, 297, 29871, 29946, 29906, 29889, 29955, 29941, 6923, 29889, 5282, 2548, 29314, 29892, 278, 3303, 3900, 29892, 892, 12919, 1250, 297, 278, 18449, 12807, 2602, 1156, 19035, 931, 373, 278, 1473, 1361, 957, 1546, 13878, 2849, 17483, 322, 4223, 15971, 1089, 29892, 541, 21201, 304, 13283, 746, 3444, 892, 17602, 766, 15380, 2164, 363, 385, 27302, 1361, 957, 29889, 450, 4908, 10256, 300, 29892, 1058, 892, 12919, 11582, 29892, 892, 21201, 304, 278, 18449, 607, 560, 566, 287, 1009, 1757, 20333, 29879, 3815, 29889, 383, 3417, 261, 29899, 29925, 719, 346, 29892, 763, 8922, 29873, 26552, 29871, 29906, 29953, 29892, 3897, 278, 937, 6114, 304, 6176, 2211, 330, 3361, 297, 278, 29871, 29896, 29900, 29900, 29899, 29906, 29900, 29900, 322, 278, 29856, 29889, 512, 916, 2186, 3158, 373, 278, 1833, 2462, 310, 278, 8064, 9981, 29892, 3444, 20333, 29879, 22040, 4518, 16939, 29887, 1251, 3897, 278, 4654, 767, 304, 454, 481, 975, 29871, 29896, 29947, 29885, 297, 278, 21954, 12500, 29892, 13461, 292, 278, 2791, 491, 3023, 1644, 326, 2699, 304, 2125, 7684, 29889, 9556, 20333, 29879, 2819, 1099, 438, 2552, 29888, 3028, 7146, 3614, 7684, 472, 5534, 3233, 297, 278, 5866, 20333, 29879, 432, 6447, 262, 1156, 5320, 3517, 4047, 874, 29892, 1550, 10015, 3761, 20333, 29879, 1094, 6596, 476, 666, 1336, 5948, 2113, 263, 28476, 936, 1757, 20333, 29879, 29871, 29896, 29945, 29900, 29900, 29885, 2186, 29889, 476, 666, 1336, 20333, 29879, 10007, 374, 327, 382, 348, 625, 435, 1022, 2901, 5309, 6991, 471, 263, 16671, 19576, 310, 278, 5866, 20333, 29879, 29871, 29947, 29900, 29900, 29885, 29889, 8922, 29873, 20333, 29879, 2186, 12569, 363, 22843, 26080, 6296, 278, 9475, 29899, 3250, 22401, 304, 263, 364, 681, 292, 10996, 29892, 541, 1550, 278, 18982, 304, 2986, 278, 12807, 1591, 515, 278, 3303, 3900, 727, 471, 29035, 310, 278, 6460, 14333, 2925, 297, 278, 3182, 29920, 3123, 10058, 17213, 29889, 1670, 471, 4340, 5932, 746, 1009, 22775, 325, 1292, 7684, 12807, 391, 612, 295, 2386, 1317, 262, 29890, 15802, 1564, 1754, 19341, 616, 29360, 297, 2304, 310, 12710, 20333, 29879, 716, 14243, 29892, 607, 1207, 376, 1552, 13089, 392, 5281, 310, 1661, 29899, 29576, 3245, 18287, 5302, 4249, 1375, 943, 29908, 263, 22161, 1283, 1947, 29889, 2296, 2678, 16388, 304, 15544, 902, 6589, 29892, 541, 727, 892, 23011, 287, 5717, 491, 23852, 10462, 6471, 363, 263, 8023, 29883, 1501, 310, 278, 29871, 29906, 29900, 29896, 29946, 12267, 12482, 297, 1105, 4161, 29892, 278, 2446, 4655, 14717, 1741, 297, 12710, 29889, 323, 29931, 29936, 8353, 11283
1, 525, 29968, 13353, 4412, 29892, 19171, 313, 29907, 10262, 29897, 1192, 450, 4593, 15538, 23303, 29892, 2307, 1090, 22522, 363, 22181, 728, 805, 2548, 29892, 6068, 385, 19001, 304, 4382, 2055, 1082, 515, 26901, 29875, 1584, 2466, 540, 338, 2729, 472, 278, 402, 8132, 20333, 29879, 20029, 4412, 29892, 19171, 29892, 8034, 29892, 263, 29696, 22522, 756, 1476, 29889, 739, 3438, 901, 1135, 395, 29906, 29946, 29892, 29900, 29900, 29900, 363, 278, 5381, 5849, 4266, 391, 304, 9850, 304, 322, 515, 278, 1667, 1049, 3303, 3900, 975, 278, 4940, 1629, 29889, 940, 338, 4249, 3196, 6893, 402, 8132, 376, 18714, 29908, 17162, 1058, 884, 9850, 304, 5164, 378, 10662, 322, 1009, 3271, 25222, 29892, 3438, 292, 278, 946, 3819, 14746, 310, 17208, 975, 278, 4940, 2211, 2440, 29889, 7634, 278, 1824, 29892, 22873, 664, 515, 3271, 322, 1122, 5735, 297, 1790, 2106, 515, 278, 5120, 297, 607, 896, 20333, 276, 2869, 9859, 29889, 450, 20029, 4412, 19001, 29892, 1058, 4687, 670, 4982, 297, 5490, 29871, 29906, 29900, 29896, 29896, 29892, 338, 12530, 395, 29947, 29946, 29892, 29946, 29946, 29900, 322, 1736, 515, 670, 3271, 297, 7906, 324, 21528, 29892, 263, 402, 8132, 21097, 16725, 29889, 512, 278, 4940, 1629, 29892, 5034, 304, 402, 8132, 9850, 6475, 29892, 278, 19001, 756, 4972, 29876, 1250, 304, 278, 1667, 1049, 14183, 3064, 363, 378, 10662, 322, 5870, 886, 29889, 12458, 310, 1906, 3367, 567, 892, 304, 624, 29889, 5899, 29936, 3023, 892, 304, 7660, 29892, 411, 263, 2625, 17487, 304, 315, 3742, 2559, 2219, 29936, 322, 697, 471, 304, 3087, 16879, 29889, 450, 3001, 3438, 304, 8818, 10472, 414, 471, 395, 29906, 29946, 29892, 29906, 29906, 29896, 29889, 21776, 12043, 3774, 29892, 278, 402, 8132, 20333, 29879, 14014, 27443, 363, 20029, 4412, 29892, 822, 2760, 278, 298, 533, 29889, 376, 1576, 3438, 310, 393, 9850, 471, 5134, 297, 278, 19220, 310, 670, 7105, 4135, 408, 385, 19001, 408, 9401, 411, 278, 916, 15576, 1934, 1699, 12043, 3774, 1497, 29889, 376, 2855, 746, 7329, 292, 599, 310, 1906, 297, 29892, 372, 471, 10087, 393, 540, 471, 278, 1900, 14020, 29892, 1584, 297, 3578, 310, 278, 3438, 393, 723, 367, 297, 2764, 1127, 1213, 12043, 3774, 2000, 278, 402, 8132, 20333, 29879, 4382, 22899, 1824, 376, 29874, 9150, 1824, 393, 20333, 29879, 2675, 304, 3275, 304, 3438, 4048, 886, 363, 8818, 10472, 414, 1213, 1205, 263, 402, 8132, 805, 23195, 29893, 2480, 1497, 29892, 376, 4806, 526, 451, 2675, 304, 24663, 445, 1134, 310, 9850, 1213, 1126, 263, 402, 8132, 19001, 297, 20029, 4412, 29892, 1058, 13877, 385, 4735, 537, 29892, 1497, 393, 298, 8491, 4856, 297, 26901, 29875, 304, 664, 363, 278, 20029, 4412, 5120, 471, 21615, 293, 29878, 681, 29889, 376, 3112, 1838, 20333, 29873, 1207, 4060, 1699, 278, 19001, 1497, 29889, 376, 10401, 366, 2050, 4129, 366, 817, 746, 366, 298, 533, 4856, 29892, 372, 723, 505, 1063, 2253, 304, 1106, 363, 4856, 297, 278, 20029, 4412, 4038, 29889, 739, 723, 505, 12212, 278, 3438, 310, 9850, 491, 472, 3203, 29871, 29955, 29900, 10151, 746, 366, 1106, 472, 925, 278, 4799, 24658, 310, 825, 372, 4893, 304, 515, 7906, 324, 21528, 304, 7660, 29892, 360, 29889, 29907, 1696, 988, 263, 3287, 310, 5381, 338, 2309, 1213, 3951, 27378, 2276, 1915, 29875, 29892, 1058, 471, 10658, 16684, 402, 8132, 27443, 445, 1629, 29892, 1497, 278, 946, 3819, 471, 4392, 2827, 278, 3438, 310, 278, 4152, 4382, 22899, 1824, 29889, 376, 29902, 1348, 278, 1556, 4100, 760, 363, 278, 402, 8132, 304, 1348, 1048, 338, 1207, 1854, 591, 1722, 20278, 701, 29892, 20847, 20278, 304, 599, 278, 15040, 2305, 297, 278, 4234, 29892, 541, 769, 884, 1207, 1854, 591, 505, 263, 2821, 5381, 1206, 1699, 540, 1497, 29889, 376, 3644, 591, 505, 4856, 1058, 338, 1985, 297, 19288, 29878, 16191, 541, 23415, 304, 12115, 29892, 727, 756, 304, 367, 263, 2821, 8252, 363, 825, 995, 896, 20333, 276, 13138, 29892, 322, 366, 20333, 345, 2355, 304, 2367, 592, 278, 5381, 1206, 29889, 887, 20333, 345, 2355, 304, 5649, 304, 592, 2020, 393, 20333, 29879, 263, 3438, 29899, 15987, 573, 4337, 363, 278, 3082, 2305, 29892, 322, 393, 20333, 29879, 263, 716, 3918, 393, 591, 20333, 276, 6721, 14332, 472, 402, 8132, 304, 594, 4150, 304, 1213, 450, 402, 8132, 376, 18714, 19001, 29908, 1824, 338, 1422, 515, 4382, 1287, 11104, 12520, 491, 1784, 2024, 14582, 3704, 29696, 20333, 29879, 3847, 5001, 29892, 24215, 25484, 292, 29892, 297, 607, 777, 22873, 526, 18443, 287, 304, 664, 515, 3271, 777, 3841, 310, 278, 4723, 29892, 22039, 304, 10032, 12469, 378, 29887, 602, 29889, 450, 5619, 12930, 373, 438, 874, 523, 322, 10354, 23933, 13877, 4902, 1048, 278, 402, 8132, 20333, 29879, 4382, 22899, 1824, 297, 5306, 29889, 2193, 5643, 766, 11291, 1973, 393, 29871, 29929, 29945, 6901, 22873, 29892, 3704, 29871, 29896, 29906, 297, 2428, 1730, 706, 11909, 29892, 10398, 8886, 395, 29955, 29945, 29900, 29892, 29900, 29900, 29900, 297, 9850, 21544, 1546, 5533, 29871, 29906, 29900, 29896, 29900, 322, 5306, 29871, 29906, 29900, 29896, 29896, 29889, 376, 1576, 3082, 2305, 505, 263, 1492, 304, 1073, 393, 17097, 289, 13411, 7283, 1446, 1058, 13389, 278, 23633, 310, 6901, 664, 526, 560, 335, 1821, 322, 14040, 1886, 2935, 310, 278, 8818, 29886, 2747, 17208, 393, 2304, 278, 1824, 1699, 5034, 304, 263, 5497, 515, 21118, 24193, 1171, 10088, 29889, 7335, 15044, 16982, 29874, 29892, 390, 29899, 7856, 6557, 423, 29892, 304, 278, 402, 8132, 29889, 450, 4902, 13877, 491, 16982, 29874, 1048, 278, 402, 8132, 1824, 505, 451, 1063, 4944, 304, 278, 21118, 29889, 29696, 884, 13877, 278, 2472, 901, 1135, 1023, 7378, 8020, 1549, 278, 17097, 3878, 11607, 310, 10343, 3185, 541, 756, 1063, 28424, 5429, 491, 278, 402, 8132, 393, 18322, 10764, 13925, 5144, 505, 451, 7743, 22520, 278, 5518, 29889, 450, 4593, 15538, 23303, 29892, 607, 756, 901, 1135, 29871, 29896, 29906, 29892, 29953, 29900, 29900, 22873, 322, 263, 395, 29906, 29953, 29889, 29941, 24464, 23562, 29892, 338, 263, 13774, 19726, 545, 17097, 946, 3819, 393, 17766, 5874, 1855, 19989, 322, 916, 1661, 29899, 23853, 277, 653, 9580, 545, 358, 29889, 11559, 15241, 385, 22522, 964, 278, 402, 8132, 1156, 263, 885, 493, 292, 16096, 272, 2498, 20333, 29879, 3461, 16610, 445, 1629, 10018, 22181, 728, 805, 2548, 1192, 395, 29947, 29906, 29941, 29892, 29900, 29900, 29900, 1192, 472, 278, 946, 3819, 20333, 29879, 10504, 2169, 1080, 16377, 297, 7413, 17236, 294, 297, 5533, 29871, 29906, 29900, 29896, 29900, 29889, 450, 19341, 29891, 3897, 2832, 1711, 304, 27375, 1156, 13676, 322, 4863, 9335, 567, 310, 278, 22181, 728, 21362, 892, 5492, 29889, 450, 20474, 362, 9508, 287, 8818, 29886, 2747, 1399, 25072, 29892, 21620, 26771, 287, 278, 17517, 322, 1925, 263, 9758, 4366, 373, 19863, 1319, 805, 2548, 491, 278, 402, 8132, 29889, 12208, 2448, 873, 29892, 278, 402, 8132, 6221, 1058, 19098, 278, 21362, 29892, 620, 12961, 29892, 408, 1258, 278, 946, 3819, 20333, 29879, 27443, 29892, 1085, 15457, 11717, 29889, 7803, 310, 11717, 20333, 29879, 20723, 583, 892, 17285, 29892, 322, 9475, 916, 22873, 2175, 278, 946, 3819, 29889, 27378, 2276, 1915, 29875, 29892, 263, 4642, 6479, 294, 2857, 10317, 6221, 29892, 3614, 975, 408, 16684, 402, 8132, 27443, 29889, 512, 6124, 304, 278, 7413, 17236, 294, 21362, 29892, 278, 402, 8132, 13229, 10398, 395, 29941, 29941, 29900, 29892, 29900, 29900, 29900, 304, 337, 2029, 403, 385, 19001, 515, 3384, 369, 304, 26901, 29875, 322, 3117, 14746, 901, 373, 916, 22873, 975, 263, 1023, 29899, 6360, 3785, 29892, 5034, 304, 263, 1301, 924, 310, 385, 15593, 411, 263, 402, 8132, 1741, 715, 7310, 29889, 1126, 29871, 29947, 29946, 402, 8132, 22873, 29892, 1556, 310, 963, 2428, 1730, 943, 470, 916, 16336, 13925, 1192, 599, 17800, 310, 16096, 272, 2498, 7405, 800, 1192, 526, 1603, 6314, 292, 1009, 10814, 6394, 29892, 3001, 292, 901, 1135, 395, 29896, 7284, 297, 8818, 29886, 2747, 6909, 29889, 512, 5468, 29892, 263, 29696, 22522, 17845, 393, 278, 402, 8132, 20333, 29879, 20029, 4412, 8034, 10398, 901, 1135, 395, 29906, 29900, 29892, 29900, 29900, 29900, 304, 3638, 22873, 304, 7984, 292, 4413, 304, 2048, 3815, 8548, 29889, 5806, 278, 4413, 437, 451, 5253, 304, 263, 7282, 2533, 310, 6909, 297, 278, 3186, 310, 534, 453, 291, 29899, 29881, 26810, 5874, 8619, 20078, 29892, 1663, 11376, 1497, 372, 471, 760, 310, 278, 3889, 29899, 1028, 2548, 9257, 393, 3512, 373, 363, 2440, 472, 278, 402, 8132, 20333, 29879, 20029, 4412, 14014, 26463, 29889, 402, 8132, 805, 23195, 29893, 2480, 350, 1691, 29874, 1458, 25761, 424, 2518, 1497, 297, 263, 3229, 445, 1629, 393, 599, 278, 946, 3819, 20333, 29879, 23274, 526, 1090, 263, 2246, 29899, 3204, 9076, 29889, 29696, 20333, 29879, 317, 2518, 530, 4495, 29892, 13061, 341, 29889, 405, 1540, 29920, 322, 4335, 315, 14899, 26869, 304, 445, 3461, 29889, 24274, 1425, 262, 16640, 1803, 4723, 16700, 29871, 29955, 3358, 382, 29911, 29889, 1152, 278, 9281, 515, 1425, 262, 16640, 1803, 2828, 1244, 29889, 323, 29931, 29936, 8353, 11283
附录4:MultiLoRA使用示例
MultiLoRA功能不支持不同模型混用(如LoRA部分分别为Qwen和Llama),只支持一个base模型扩展多个LoRA部分(包括系列和参数量,也就是Llama2-7B的不同LoRA模型只能跟这个base的部署到一起)。
在启动服务端时使用:请求服务指定参数 --enable-lora --lora-modules {lora_name}={lora_file_path}
python3 -m vllm.entrypoints.openai.api_server \
--host 0.0.0.0 \
--port 27253 \
--trust-remote-code \
--tensor-parallel-size 1 \
--dtype float16 \
--model path_to_model/Qwen2.5-0.5B \
--enable-lora \
--lora-modules lora-1=/path_to_lora/Qwen2.5-0.5B
客户端请求:给lora_name 为 lora-1发起请求
curl http://localhost:27253/v1/completions \
-H "Content-Type: application/json" \
-d '{
"model": "lora-1",
"prompt": "San Francisco is a",
"temperature": 0
}'
目前线上模型支持MultiLoRA详细列表如下:
系列 | 模型 | 支持加速芯片数量 | 机器环境 |
---|---|---|---|
Llama2 | Llama2-7B | 1 | A800 |
Llama2-13B | 1 | A800 | |
Llama2-70B | 4、8 | A800 | |
Llama3 | Llama2-8B | 1 | A800 |
Qwen1.5 | Qwen1.5-32B | 1、2 | A800 |
Qwen1.5-72B | 4、8 | A800 | |
Qwen 2 | Qwen2-0.5B | 1、2 | A800 |
Qwen2-1.5B | 1、2 | A800 | |
Qwen2-7B | 1、2 | A800 | |
Qwen2-72B | 4、8 | A800 | |
Qwen2.5 | Qwen2.5-0.5B | 1、2 | A800 |
Qwen2.5-1.5B | 1、2 | A800 | |
Qwen2.5-7B | 1、2 | A800 | |
Qwen2.5-14B | 1、2 | A800 | |
Qwen2.5-32B | 1、2 | A800 | |
Qwen2.5-72B | 4、8 | A800 |
附录5:自定义监控大盘效果
可以参考vLLM社区输出的监控项指标及参数说明中样例,配置监控大盘。
附录6:服务端完整参数
用户配置参数项 | 描述 | 取值类型 | 是否必选 | 是否AIAK特有 | 默认值 | 取值范围 |
---|---|---|---|---|---|---|
–host | 主机名 | str | 否 | 否 | ||
–port | 端口号 | int | 否 | 否 | 8000 | |
–uvicorn-log-level | uvicorn的日志级别 | str | 否 | 否 | info | debug,info,warning,error,critical,trace |
–allow-credentials | 是否允许凭证 | bool | 否 | 否 | FALSE | |
–allowed-origins | 允许的来源 | list | 否 | 否 | ['*'] | |
–allowed-methods | 允许的方法 | list | 否 | 否 | ['*'] | |
–allowed-headers | 允许的头部 | list | 否 | 否 | ['*'] | |
–api-key | API密钥,如果提供,服务器将要求在请求头中提供此密钥 | str | 否 | 否 | ||
–lora-modules | LoRA模块配置,格式为'name=path'或JSON格式 | str | 否 | 否 | ||
–prompt-adapters | 提示适配器配置,格式为name=path,可以指定多个适配器 | str | 否 | 否 | ||
–chat-template | 聊天模板的文件路径,或指定模型的单行模板 | str | 否 | 否 | ||
–response-role | 如果request.add_generation_prompt=true,返回的角色名称 | str | 否 | 否 | assistant | |
–ssl-keyfile | SSL密钥文件的文件路径 | str | 否 | 否 | ||
–ssl-certfile | SSL证书文件的文件路径 | str | 否 | 否 | ||
–ssl-ca-certs | CA证书文件 | str | 否 | 否 | ||
–ssl-cert-reqs | 是否需要客户端证书(参见stdlib的ssl模块) | int | 否 | 否 | 0 | |
–root-path | 当应用程序在基于路径的代理后面时,FastAPI的root_path | str | 否 | 否 | ||
–middleware | 应用程序的额外ASGI中间件,可多次指定--middleware参数,值应为导入路径 | str | 否 | 否 | [] | |
–return-tokens-as-token-ids | 当指定--max-logprobs时,将单个标记表示为'token_id:{token_id}'形式的字符串,以便无法JSON编码的标记可以被识别 | bool | 否 | 否 | FALSE | |
–disable-frontend-multiprocessing | 如果指定,将在与模型服务引擎相同的进程中运行OpenAI前端服务器 | bool | 否 | 否 | FALSE | |
–enable-auto-tool-choice | 为支持的模型启用自动工具选择,使用--tool-call-parser指定要使用的解析器 | bool | 否 | 否 | FALSE | |
–tool-call-parser | 根据您使用的模型选择工具调用解析器,用于将模型生成的工具调用解析为OpenAIAPI格式,使用--enable-auto-tool-choice时需要指定 | str | 视情况而定 | 否 | ||
–tool-parser-plugin | 指定用于解析模型生成的工具的工具解析器插件,在--tool-call-parser中可以使用该插件中注册的名称 | str | 否 | 否 | ||
–model | 要使用的huggingface模型的名称或路径 | str | 否 | 否 | facebook/opt-125m | |
–task | 使用模型的任务,每个vLLM实例只支持一个任务,即使相同的模型可以用于多个任务 | str | 否 | 否 | auto | auto,generate,embedding |
–tokenizer | 要使用的huggingface分词器的名称或路径,如果未指定,将使用模型名称或路径 | str | 否 | 否 | ||
–skip-tokenizer-init | 跳过分词器和反分词器的初始化 | bool | 否 | 否 | FALSE | |
–revision | 要使用的特定模型版本,可以是分支名、标签名或提交ID,如果未指定,将使用默认版本 | str | 否 | 否 | ||
–code-revision | 在HuggingFaceHub上使用的模型代码的特定修订版本,可以是分支名、标签名或提交ID,如果未指定,将使用默认版本 | str | 否 | 否 | ||
–tokenizer-revision | 要使用的huggingface分词器的修订版本,可以是分支名、标签名或提交ID,如果未指定,将使用默认版本 | str | 否 | 否 | ||
–tokenizer-mode | 分词器模式 | str | 否 | 否 | auto | auto,slow,mistral |
–chat-template-text-format | 在聊天模板中呈现文本内容的格式 | str | 否 | 否 | string | string,openai |
–trust-remote-code | 信任来自huggingface的远程代码 | bool | 否 | 否 | FALSE | |
–download-dir | 下载和加载权重的目录 | str | 否 | 否 | ||
–load-format | 要加载的模型权重的格式 | str | 否 | 否 | auto | auto,pt,safetensors,npcache,dummy,tensorizer,sharded_state,gguf,bitsandbytes,mistral |
–config-format | 要加载的模型配置的格式 | string | 否 | 否 | auto | auto,hf,mistral |
–dtype | 模型权重和激活的数据显示类型 | str | 否 | 否 | auto | auto,half,float16,bfloat16,float,float32 |
–kv-cache-dtype | kv缓存存储的数据类型 | str | 否 | 否 | auto | auto,fp8,fp8_e5m2,fp8_e4m3 |
–quantization-param-path | 包含KV缓存缩放因子的JSON文件路径,通常在KV缓存dtype为FP8时提供,否则KV缓存缩放因子默认为1.0,可能会导致准确性问题 | str | 否 | 否 | ||
–max-model-len | 模型的上下文长度 | int | 否 | 否 | ||
–guided-decoding-backend | 用于引导解码的引擎(JSONschema/正则表达式等) | str | 否 | 否 | outlines | outlines,lm-format-enforcer |
–distributed-executor-backend | 用于分布式服务的后端,当使用多个GPU时,如果安装了ray,将自动设置为'ray',否则为'mp'(多进程) | str | 否 | 否 | ray,mp | |
–worker-use-ray | 已弃用,使用--distributed-executor-backend=ray | bool | 否 | 否 | FALSE | |
–pipeline-parallel-size | 管道并行阶段的数量 | int | 否 | 否 | 1 | |
–tensor-parallel-size | 张量并行副本的数量 | int | 否 | 否 | 1 | |
–max-parallel-loading-workers | 在多个批次中顺序加载模型,以避免在使用张量并行和大型模型时出现RAM不足 | int | 否 | 否 | ||
–ray-workers-use-nsight | 如果指定,使用nsight来分析Ray工作者 | bool | 否 | 否 | FALSE | |
–block-size | 连续标记块的令牌块大小 | int | 否 | 否 | 16 | 8,16,32 |
–enable-prefix-caching | 启用自动前缀缓存 | bool | 否 | 否 | FALSE | |
–disable-sliding-window | 禁用滑动窗口 | bool | 否 | 否 | FALSE | |
–num-lookahead-slots | 推测解码的实验性调度配置,将来会被推测配置取代 | int | 否 | 否 | 0 | |
–seed | 操作的随机种子 | int | 否 | 否 | 0 | |
–swap-space | 每个GPU的CPU交换空间大小(GiB) | int | 否 | 否 | 4 | |
–cpu-offload-gb | 每个GPU要卸载到CPU的空间(GiB) | int | 否 | 否 | 0 | |
–gpu-memory-utilization | 用于模型执行器的GPU内存占用率,范围从0到1 | float | 否 | 否 | 0.9 | 0~1 |
–num-gpu-blocks-override | 如果指定,忽略GPU分析结果并使用此GPU块数 | int | 否 | 否 | ||
–max-num-batched-tokens | 每次迭代的最大批处理标记数 | int | 否 | 否 | ||
–max-num-seqs | 每次迭代的最大序列数 | int | 否 | 否 | 256 | |
–max-logprobs | 返回的最大对数概率数量,如果在SamplingParams中指定了logprobs | int | 否 | 否 | 20 | |
–disable-log-stats | 禁用日志统计 | bool | 否 | 否 | FALSE | |
--tokenizer-extension-path | 自定义tokenizer中vllm_extension.py的路径 | str | 否 | 是 | ||
–quantization | 用于量化权重的方法 | str | 否 | 部分 | aqlm;awq;deepspeedfp;tpu_int8;fp8;fbgemm_fp8;modelopt;marlin;gguf;gptq_marlin_24;gptq_marlin;awq_marlin;gptq;compressed-tensors;bitsandbytes;qqq;experts_int8;neuron_quant;ipex;weight_only_int8;smooth_quant;None | |
–rope-scaling | RoPE缩放配置(JSON格式) | json | 否 | 否 | ||
–rope-theta | RoPEtheta,与rope_scaling一起使用 | float | 否 | 否 | ||
–enforce-eager | 始终使用eager模式的PyTorch | bool | 否 | 否 | FALSE | |
–max-seq-len-to-capture | CUDA图覆盖的最大序列长度,超过此值回退到eager模式 | int | 否 | 否 | 8192 | |
–disable-custom-all-reduce | 参见ParallelConfig | bool | 否 | 否 | FALSE | |
–tokenizer-pool-size | 用于异步分词的分词器池大小,为0时使用同步分词 | int | 否 | 否 | 0 | |
–tokenizer-pool-type | 用于异步分词的分词器池类型 | str | 否 | 否 | ray | |
–tokenizer-pool-extra-config | 分词器池的额外配置(JSON格式) | json | 否 | 否 | ||
–limit-mm-per-prompt | 限制每个提示的多模态插件输入实例数量,如image=16,video=2 | str | 否 | 否 | ||
–mm-processor-kwargs | 多模态输入处理的覆盖参数(JSON格式) | json | 否 | 否 | ||
–enable-lora | 启用LoRA适配器的处理 | bool | 否 | 否 | FALSE | |
–max-loras | 单个批次中的最大LoRA数量 | int | 否 | 否 | 1 | |
–max-lora-rank | 最大LoRA秩 | int | 否 | 否 | 16 | |
–lora-extra-vocab-size | LoRA适配器中额外词汇的最大大小 | int | 否 | 否 | 256 | |
–lora-dtype | LoRA的数据类型,'auto'默认为基础模型的数据类型 | str | 否 | 否 | auto | auto,float16,bfloat16,float32 |
–long-lora-scaling-factors | 指定多个缩放因子以允许使用多个LoRA适配器 | str | 否 | 否 | ||
–max-cpu-loras | 要存储在CPU内存中的最大LoRA数量,必须≥max_num_seqs | int | 否 | 否 | ||
–fully-sharded-loras | 启用完全分片的LoRA计算 | bool | 否 | 否 | FALSE | |
–enable-prompt-adapter | 启用PromptAdapters的处理 | bool | 否 | 否 | FALSE | |
–max-prompt-adapters | 批处理中PromptAdapters的最大数量 | int | 否 | 否 | 1 | |
–max-prompt-adapter-token | PromptAdapters的最大令牌数 | int | 否 | 否 | 0 | |
–device | vLLM执行的设备类型 | str | 否 | 否 | auto | auto,cuda,neuron,cpu,openvino,tpu,xpu |
–num-scheduler-steps | 每个调度器调用的最大前向步数 | int | 否 | 否 | 1 | |
–scheduler-delay-factor | 调度下一个提示前的延迟因子 | float | 否 | 否 | 0 | |
–enable-chunked-prefill | 设置后,预填充请求可根据max_num_batched_tokens分块 | bool | 否 | 否 | ||
–ngram-prompt-lookup-max | 推测解码中ngram提示查找的最大窗口大小 | int | 否 | 否 | ||
–ngram-prompt-lookup-min | 推测解码中ngram提示查找的最小窗口大小 | int | 否 | 否 | ||
–spec-decoding-acceptance-method | 推测解码期间草稿标记验证的接受方法 | str | 否 | 否 | rejection_sampler | rejection_sampler,typical_acceptance_sampler |
–typical-acceptance-sampler-posterior-threshold | 典型采样器中令牌接受的后验概率下限阈值,默认0.09 | float | 否 | 否 | 0.09 | |
–typical-acceptance-sampler-posterior-alpha | 基于熵的阈值缩放因子,默认0.3 | float | 否 | 否 | 0.3 | |
–disable-logprobs-during-spec-decoding | 为True时,在推测解码期间不返回对数概率 | bool | 否 | 否 | TRUE | |
–model-loader-extra-config | 模型加载器的额外配置(JSON格式) | json | 否 | 否 | ||
–ignore-patterns | 加载模型时要忽略的模式,默认['original/*/'] | str | 否 | 否 | [] | |
–preemption-mode | recompute'或'swap',指定引擎的抢占模式 | str | 否 | 否 | ||
–served-model-name | API中使用的模型名称,可提供多个名称 | str | 否 | 否 | ||
–qlora-adapter-name-or-path | QLoRA适配器的名称或路径 | str | 否 | 否 | ||
–otlp-traces-endpoint | 发送OpenTelemetry跟踪的目标URL | str | 否 | 否 | ||
–collect-detailed-traces | 有效选项为model,worker,all,仅在设置--otlp-traces-endpoint时有意义 | str | 否 | 否 | ||
–disable-async-output-proc | 禁用异步输出处理 | bool | 否 | 否 | FALSE | |
–override-neuron-config | 覆盖或设置神经元设备配置(JSON格式) | json | 否 | 否 | ||
–scheduling-policy | 调度策略 | str | 否 | 否 | fcfs | fcfs,priority |
–pooling-type | 配置嵌入模型的池化方法 | str | 否 | 否 | LAST,ALL,CLS,STEP | |
–pooling-norm | 是否在嵌入模型中标准化池化数据 | bool | 否 | 否 | ||
–no-pooling-norm | 是否在嵌入模型中标准化池化数据 | bool | 否 | 否 | ||
–pooling-softmax | 是否在嵌入模型中对池化数据进行softmax | bool | 否 | 否 | ||
–no-pooling-softmax | 是否在嵌入模型中对池化数据进行softmax | bool | 否 | 否 | ||
–pooling-step-tag-id | 当不为-1时,返回对应step-tag-ids的得分 | int | 否 | 否 | ||
–pooling-returned-token-ids | 要提取的词汇表维度索引列表 | int | 否 | 否 | ||
–disable-log-requests | 禁用请求日志记录 | bool | 否 | 否 | FALSE | |
–max-log-len | 日志中打印的最大提示字符数或ID数量 | int | 否 | 否 | ||
–disable-fastapi-docs | 禁用FastAPI的OpenAPI模式、SwaggerUI和ReDoc端点 | bool | 否 | 否 | FALSE |