Commit Graph

19 Commits

Author SHA1 Message Date
bigbrother666sh
3e07d63757 llm wrapper and prompt opz, base url bug, mp scraper opz 2025-01-18 13:47:47 +08:00
bigbrother666sh
dd7d92476e 0.3.7 release 2025-01-17 23:28:22 +08:00
Tusik
eec15ba037
feat: openai SDK使用异步客户端提升效率 (#182)
* feat: async tasks for get_more_related_urls

* feat: max LLM concurrent number

* fix(core/agents/get_info.py): 降低默认并发数量

* fix(get_info.py): 在部分模型指定response_format为json的情况下需要在prompt中显式的指明json格式

* fix: text长度不足的情况

* feat: 并发部分移动到openai_wapper

* ♻️ refactor(openai_wrapper.py): 重构异步LLM调用逻辑,优化异常处理和日志记录

- 将响应结果提取到`resp`变量中,避免重复代码
- 简化异常处理逻辑,确保`finally`块中释放信号量
- 优化日志记录位置,确保在返回结果前记录调试信息

* Update openai_wrapper.py

to resolve error raise by 'logger is None'
(This problem existed in the previous version. It was not caused by your code. I just modified it.)

Signed-off-by: bigbrother666 <96130569+bigbrother666sh@users.noreply.github.com>

---------

Signed-off-by: bigbrother666 <96130569+bigbrother666sh@users.noreply.github.com>
Co-authored-by: bigbrother666 <96130569+bigbrother666sh@users.noreply.github.com>
2025-01-08 09:56:08 +08:00
bigbrother666sh
1f79cb3c4d v0.3.6fix 2025-01-05 21:54:06 +08:00
bigbrother666sh
35fbff0f27 v0.3.6 release 2025-01-05 18:12:36 +08:00
bigbrother666sh
1f9b6d5d6c v0.3.6 mockup 2025-01-04 23:36:18 +08:00
bigbrother666sh
b4da3cc853 v0.3.6test 2025-01-02 22:05:51 +08:00
bigbrother666sh
dc8391c357 deep scraper 2025-01-02 10:14:33 +08:00
bigbrother666sh
ae7b5d7f65 v0.3.6 2024-12-27 14:07:37 +08:00
bigbrother666sh
fd9d9f9a4e add test for v0.3.6 2024-12-23 10:12:52 +08:00
bigbrother666sh
1416ab29c8 add test 2024-12-18 22:45:20 +08:00
bigbrother666sh
7752b4b3b4 V0.3.5 2024-12-10 14:18:03 +08:00
bigbrother666sh
cad383b0fe feat:docker file 2024-12-09 18:18:10 +08:00
bigbrother666sh
de549c6334 fix: erros 2024-12-08 21:30:39 +08:00
bigbrother666sh
ec514b49dd refactor(core): the new general crawler 2024-12-08 18:03:34 +08:00
bigbrother666sh
8c64749ba7 feat(core): update pb data sheet structure 2024-12-06 14:14:28 +08:00
bigbrother666sh
3e4454a33b add batch process 2024-12-06 12:16:02 +08:00
bigbrother666sh
b83ca2369a second commit for V0.3.22 2024-12-06 11:42:22 +08:00
bigbrother666sh
61251547a0 first commit for V0.3.22 2024-12-05 12:11:28 +08:00