From 0159fdf14921ba4202875bdecc7991c803545b3d Mon Sep 17 00:00:00 2001 From: Ke Sun <33739460+keeees@users.noreply.github.com> Date: Fri, 30 Jan 2026 14:51:34 +0800 Subject: [PATCH] Release/v0.2.2 (#258) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 读取的接口，去掉全局锁 * 输出数组 * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化测试接口 * 反思优化测试接口 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察) * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 把group_id替换end_user_id * 把group_id替换end_user_id_ * 把group_id替换end_user_id_ * config_config替换成memory_config * config_config替换成memory_config * [fix]Fix the memory interface to use end_user_id. * config_config替换成memory_config * config_config替换成memory_config * config_config替换成memory_config * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID，与develop校对恢复 * 检查项目，修复group_id的遗留问题 * 检查项目，修复group_id的遗留问题 * Fix/interface home (#182) * [fix]Fix the interface for statistics of recent activities and applications * [changes]Modify the code based on the AI review 1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter. 2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls. * [fix]Fix the interface for statistics of recent activities and applications * [changes]Modify the code based on the AI review 1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter. 2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls. * Fix/optimize inerface (#183) * [changes]Optimize the time consumption of the "/end_users" interface * [fix]Optimize the time consumption of the "/hot_memory_tags" interface * [changes]Optimize the time consumption of the "/end_users" interface * [fix]Optimize the time consumption of the "/hot_memory_tags" interface * [changes]Improve the code based on AI review * Fix/memory mcp2 1 (#184) * 优化快速检索的回复内容 * 优化快速检索的回复内容 * Fix/memory mcp2 1 (#185) * 优化快速检索的回复内容 * 优化快速检索的回复内容 * 路径的BUG修复 * 路径的BUG修复 * 路径的BUG修复 * 路径的BUG修复 * 路径的BUG修复 * Fix/memory mcp2 1 (#188) * 优化快速检索的回复内容 * 优化快速检索的回复内容 * 路径的BUG修复 * 路径的BUG修复 * 路径的BUG修复 * 路径的BUG修复 * 路径的BUG修复 * LLM生存缺少config_id认证，修复BUG * LLM生存缺少config_id认证，修复BUG * LLM生存缺少config_id认证，修复BUG * 解决冲突 * 解决冲突 * feat(home page): version description update * Fix/memory mcp2 1 (#190) * 优化快速检索的回复内容 * 优化快速检索的回复内容 * 路径的BUG修复 * 路径的BUG修复 * 路径的BUG修复 * 路径的BUG修复 * 路径的BUG修复 * LLM生存缺少config_id认证，修复BUG * LLM生存缺少config_id认证，修复BUG * LLM生存缺少config_id认证，修复BUG * 深度检索优化，搜索不到数据/提问的概念过于蘑菇，以引导的方式继续提问 * 深度检索优化，搜索不到数据/提问的概念过于蘑菇，以引导的方式继续提问 * 深度检索优化，搜索不到数据/提问的概念过于蘑菇，以引导的方式继续提问 * end_user_id清理干净 * end_user_id清理干净 * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * feat(web): memory related interface parameter transfer adjustment * 感知meta_data字段BUG修复 * Fix/memory bug fix (#171) * feat(sandbox): add Python 3 code execution sandbox support * feat(workflow): emit SSE events for node exception output * perf(sandbox): optimize code encryption handling * perf(workflow): update standard node output structure * [add] migration script * [modify] migration script * feat(web): add workflow runtime info * fix(web): handleSSE bugfix * fix(sandbox): prevent imports from being blocked when network is disabled * user_id->现实为config_id_old * user_id->显示为config_id_old传输 * Fix/memory bug fix (#199) * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 读取的接口，去掉全局锁 * 输出数组 * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化测试接口 * 反思优化测试接口 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察) * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 把group_id替换end_user_id * 把group_id替换end_user_id_ * 把group_id替换end_user_id_ * config_config替换成memory_config * config_config替换成memory_config * [fix]Fix the memory interface to use end_user_id. * config_config替换成memory_config * config_config替换成memory_config * config_config替换成memory_config * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID，与develop校对恢复 * 检查项目，修复group_id的遗留问题 * 检查项目，修复group_id的遗留问题 * 解决冲突 * 解决冲突 * end_user_id清理干净 * end_user_id清理干净 * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 感知meta_data字段BUG修复 * user_id->现实为config_id_old * user_id->显示为config_id_old传输 --------- Co-authored-by: lanceyq <1982376970@qq.com> * user_id->显示为config_id_old传输 * feat(web): update read_all_config select valueKey * user_id->显示为config_id_old传输 * feat(workflow): Add a new node for executing code * fix(web): KnowledgeConfigModal bugfix * fix(web): iteration's variable add parameter-extractor node * fix(sandbox): treat non-zero exit codes as errors instead of relying only on stderr * Fix/memory bug fix (#200) * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 读取的接口，去掉全局锁 * 输出数组 * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化测试接口 * 反思优化测试接口 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察) * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 把group_id替换end_user_id * 把group_id替换end_user_id_ * 把group_id替换end_user_id_ * config_config替换成memory_config * config_config替换成memory_config * [fix]Fix the memory interface to use end_user_id. * config_config替换成memory_config * config_config替换成memory_config * config_config替换成memory_config * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID，与develop校对恢复 * 检查项目，修复group_id的遗留问题 * 检查项目，修复group_id的遗留问题 * 解决冲突 * 解决冲突 * end_user_id清理干净 * end_user_id清理干净 * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 感知meta_data字段BUG修复 * user_id->现实为config_id_old * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 --------- Co-authored-by: lanceyq <1982376970@qq.com> * Refactor/benchmark test (#196) * [changes]refactor locomo_test * [fix]Fix the circular import of ModelParameters * [changes]The benchmark test can run stably. * [fix]Complete end-to-end LoCoMo repair * [fix]Complete the end-to-end longmemeval and memsciqa fixes * [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect. * [changes]refactor locomo_test * [fix]Fix the circular import of ModelParameters * [changes]The benchmark test can run stably. * [fix]Complete end-to-end LoCoMo repair * [fix]Complete the end-to-end longmemeval and memsciqa fixes * [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect. * [changes]Benchmark test adaptation for end_user_id * [changes]refactor locomo_test * [fix]Fix the circular import of ModelParameters * [changes]The benchmark test can run stably. * [fix]Complete end-to-end LoCoMo repair * [fix]Complete the end-to-end longmemeval and memsciqa fixes * [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect. * [fix]Complete the end-to-end longmemeval and memsciqa fixes * [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect. * [changes]Benchmark test adaptation for end_user_id * [modify] migration script * delete benchmark-test (#204) * Refactor: Move evaluation folder to redbear-mem-benchmark submodule * [changes]Restore .gitmodules * feat(web): workflow add code node * 检查需要更改的格式问题 * Fix/redbear benchmark (#205) * Refactor: Move evaluation folder to redbear-mem-benchmark submodule * [changes]Update submodule reference * Refactor: Move evaluation folder to redbear-mem-benchmark submodule * [changes]Update submodule reference * Remove duplicate evaluation submodule, use redbear-mem-benchmark instead * Fix/memory bug fix (#207) * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 读取的接口，去掉全局锁 * 输出数组 * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化测试接口 * 反思优化测试接口 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察) * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 把group_id替换end_user_id * 把group_id替换end_user_id_ * 把group_id替换end_user_id_ * config_config替换成memory_config * config_config替换成memory_config * [fix]Fix the memory interface to use end_user_id. * config_config替换成memory_config * config_config替换成memory_config * config_config替换成memory_config * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID，与develop校对恢复 * 检查项目，修复group_id的遗留问题 * 检查项目，修复group_id的遗留问题 * 解决冲突 * 解决冲突 * end_user_id清理干净 * end_user_id清理干净 * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 感知meta_data字段BUG修复 * user_id->现实为config_id_old * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * 检查需要更改的格式问题 --------- Co-authored-by: lanceyq <1982376970@qq.com> * fix(web): remove URI decode and encode * [add] plugin system and base sso module * 修复宿主列表获取memory_config_idBUG * Fix/memory bug fix (#209) * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 读取的接口，去掉全局锁 * 输出数组 * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化测试接口 * 反思优化测试接口 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察) * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 把group_id替换end_user_id * 把group_id替换end_user_id_ * 把group_id替换end_user_id_ * config_config替换成memory_config * config_config替换成memory_config * [fix]Fix the memory interface to use end_user_id. * config_config替换成memory_config * config_config替换成memory_config * config_config替换成memory_config * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID，与develop校对恢复 * 检查项目，修复group_id的遗留问题 * 检查项目，修复group_id的遗留问题 * 解决冲突 * 解决冲突 * end_user_id清理干净 * end_user_id清理干净 * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 感知meta_data字段BUG修复 * user_id->现实为config_id_old * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * 检查需要更改的格式问题 * 修复宿主列表获取memory_config_idBUG --------- Co-authored-by: lanceyq <1982376970@qq.com> * [modify] file local server url * [add] migration script * fix(workflow): fix activation and branch control issues in streaming output * fix(workflow): fix function cache not taking effect and potential list index overflow * style(workflow): enforce PEP8 style and remove redundant imports * fix(workflow): fix streaming output error when variable is not a string * [fix]remove aspose-slides * perf(workflow): enhance streaming output node activation performance * feat(workflow): store token usage in message table * feat(web): add PageEmpty component * feat(web): add PageTabs component * perf(workflow): make memory configuration backward compatible * feat(web): update model management * config_id做映射 * config_id做映射 * Fix/memory bug fix (#211) * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 读取的接口，去掉全局锁 * 输出数组 * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化测试接口 * 反思优化测试接口 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察) * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 把group_id替换end_user_id * 把group_id替换end_user_id_ * 把group_id替换end_user_id_ * config_config替换成memory_config * config_config替换成memory_config * [fix]Fix the memory interface to use end_user_id. * config_config替换成memory_config * config_config替换成memory_config * config_config替换成memory_config * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID，与develop校对恢复 * 检查项目，修复group_id的遗留问题 * 检查项目，修复group_id的遗留问题 * 解决冲突 * 解决冲突 * end_user_id清理干净 * end_user_id清理干净 * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 感知meta_data字段BUG修复 * user_id->现实为config_id_old * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * 检查需要更改的格式问题 * 修复宿主列表获取memory_config_idBUG * config_id做映射 * config_id做映射 --------- Co-authored-by: lanceyq <1982376970@qq.com> * feat(web): getModelListUrl add is_active param * config_id做映射+1 * config_id做映射+1 * config_id做映射+1 * feat(web): remove file url replace * Fix/memory bug fix (#212) * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 读取的接口，去掉全局锁 * 输出数组 * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化测试接口 * 反思优化测试接口 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察) * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 把group_id替换end_user_id * 把group_id替换end_user_id_ * 把group_id替换end_user_id_ * config_config替换成memory_config * config_config替换成memory_config * [fix]Fix the memory interface to use end_user_id. * config_config替换成memory_config * config_config替换成memory_config * config_config替换成memory_config * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID，与develop校对恢复 * 检查项目，修复group_id的遗留问题 * 检查项目，修复group_id的遗留问题 * 解决冲突 * 解决冲突 * end_user_id清理干净 * end_user_id清理干净 * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 感知meta_data字段BUG修复 * user_id->现实为config_id_old * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * 检查需要更改的格式问题 * 修复宿主列表获取memory_config_idBUG * config_id做映射 * config_id做映射 * config_id做映射+1 * config_id做映射+1 * config_id做映射+1 --------- Co-authored-by: lanceyq <1982376970@qq.com> * feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics * feat(web): model logo update * 应用层memory_content->memory_config * fix(web): correct spelling * 应用层memory_content->memory_config * 应用层memory_content->memory_config * Fix/memory bug fix (#215) * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 读取的接口，去掉全局锁 * 输出数组 * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化测试接口 * 反思优化测试接口 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察) * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 把group_id替换end_user_id * 把group_id替换end_user_id_ * 把group_id替换end_user_id_ * config_config替换成memory_config * config_config替换成memory_config * [fix]Fix the memory interface to use end_user_id. * config_config替换成memory_config * config_config替换成memory_config * config_config替换成memory_config * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID，与develop校对恢复 * 检查项目，修复group_id的遗留问题 * 检查项目，修复group_id的遗留问题 * 解决冲突 * 解决冲突 * end_user_id清理干净 * end_user_id清理干净 * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 感知meta_data字段BUG修复 * user_id->现实为config_id_old * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * 检查需要更改的格式问题 * 修复宿主列表获取memory_config_idBUG * config_id做映射 * config_id做映射 * config_id做映射+1 * config_id做映射+1 * config_id做映射+1 * 应用层memory_content->memory_config * 应用层memory_content->memory_config * 应用层memory_content->memory_config --------- Co-authored-by: lanceyq <1982376970@qq.com> * feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics * fix(web): model loading update * 统一字段为config_id_old * 统一字段为config_id_old * feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics * 统一字段为config_id_old * 统一字段为config_id_old * memory_content暂时不修改 * memory_content暂时不修改 * Fix/memory bug fix (#217) * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 图谱数据量限制数量去掉 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 用户详情优化 * 读取的接口，去掉全局锁 * 输出数组 * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化1.0（优化隐私输出、时间检索） * 反思优化测试接口 * 反思优化测试接口 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 读取接口内层嵌套BUG修复 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察) * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 新增中翻英功能（记忆时间线）(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 * 把group_id替换end_user_id * 把group_id替换end_user_id_ * 把group_id替换end_user_id_ * config_config替换成memory_config * config_config替换成memory_config * [fix]Fix the memory interface to use end_user_id. * config_config替换成memory_config * config_config替换成memory_config * config_config替换成memory_config * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID * config_id字段改成UUID，与develop校对恢复 * 检查项目，修复group_id的遗留问题 * 检查项目，修复group_id的遗留问题 * 解决冲突 * 解决冲突 * end_user_id清理干净 * end_user_id清理干净 * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 修复遗留合并BUG * 感知meta_data字段BUG修复 * user_id->现实为config_id_old * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * user_id->显示为config_id_old传输 * 检查需要更改的格式问题 * 修复宿主列表获取memory_config_idBUG * config_id做映射 * config_id做映射 * config_id做映射+1 * config_id做映射+1 * config_id做映射+1 * 应用层memory_content->memory_config * 应用层memory_content->memory_config * 应用层memory_content->memory_config * 统一字段为config_id_old * 统一字段为config_id_old * 统一字段为config_id_old * 统一字段为config_id_old * memory_content暂时不修改 * memory_content暂时不修改 --------- Co-authored-by: lanceyq <1982376970@qq.com> * feat(web): add app statistics * fix(workflow): fix streaming output issues with multi-output End nodes End nodes with multiple output segments could cause cursor errors or leave some segments inactive, resulting in incorrect final outputs. Unified _emit_active_chunks and _update_scope_activate to ensure all segments are activated in order and streamed correctly. * feat(web): add apps statistics api * fix(web): agent's knowledge_bases bugfix * Revert "feat(web): update read_all_config select valueKey" This reverts commit 46f0f3cee90f7cf852bf5bcf89866b57448f1ffa. * [add] migrations script * perf(workflow): make memory write node backward-compatible and defer config validation * 旧数据兼容 * 旧数据兼容 * 旧数据兼容 * 旧数据兼容 * fix(web): model bugfix * fix(web): model bugfix * 提交遗漏 (#228) * [fix] chat api for workflow * [fix] web search set for v1 api * fix(web): model bugfix * fix(web): model list remove is_active * fix(model): bug fix * [add]migration script * [fix] api * [fix] api * fix(web): model bugfix * fix(model): the model type does not allow modification, delete tts and speech2text type * fix(model): bug fix * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * Add/develop memory (#239) * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * feat(web): model ui update * feat(web): model ui update * Add/develop memory (#243) * 遗漏的历史映射 * 遗漏的历史映射 * fix(model): bug fix * feat(web): model ui update * Add/develop memory (#247) * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * 遗漏的历史映射 * [modify] migration script * [add] migration script * fix(web): change form message * fix(web): the memoryContent field is compatible with numbers and strings * feat(web): code node hidden * fix(model): 1. create a basic model to check if the name and provider are duplicated. 2. The result shows error models because the provider created API Keys for all matching models. --------- Co-authored-by: lixinyue <2569494688@qq.com> Co-authored-by: lanceyq <1982376970@qq.com> Co-authored-by: yujiangping Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com> Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com> Co-authored-by: yingzhao Co-authored-by: Timebomb2018 <18868801967@163.com> Co-authored-by: Mark Co-authored-by: zhaoying Co-authored-by: Eternity <1533512157@qq.com> Co-authored-by: lixiangcheng1 --- .gitignore | 3 + api/app/__init__.py | 0 api/app/controllers/app_controller.py | 41 + .../controllers/emotion_config_controller.py | 14 +- api/app/controllers/emotion_controller.py | 38 +- .../controllers/file_storage_controller.py | 2 +- .../controllers/implicit_memory_controller.py | 80 +- .../controllers/memory_agent_controller.py | 55 +- .../memory_dashboard_controller.py | 147 +- .../controllers/memory_forget_controller.py | 38 +- .../memory_perceptual_controller.py | 66 +- .../memory_reflection_controller.py | 51 +- .../controllers/memory_storage_controller.py | 99 +- .../controllers/memory_working_controller.py | 16 +- api/app/controllers/model_controller.py | 321 +++- .../controllers/public_share_controller.py | 7 +- .../controllers/service/app_api_controller.py | 12 +- .../service/memory_api_controller.py | 2 +- .../controllers/user_memory_controllers.py | 18 +- api/app/controllers/workflow_controller.py | 14 +- api/app/core/agent/langchain_agent.py | 99 +- api/app/core/config.py | 26 +- .../langgraph_graph/nodes/problem_nodes.py | 10 +- .../langgraph_graph/nodes/retrieve_nodes.py | 18 +- .../langgraph_graph/nodes/summary_nodes.py | 40 +- .../nodes/verification_nodes.py | 8 +- .../langgraph_graph/nodes/write_nodes.py | 17 +- .../agent/langgraph_graph/read_graph.py | 6 +- .../agent/langgraph_graph/tools/tool.py | 30 +- .../agent/langgraph_graph/write_graph.py | 23 +- .../agent/services/parameter_builder.py | 6 +- .../memory/agent/services/search_service.py | 8 +- .../memory/agent/services/session_service.py | 18 +- .../core/memory/agent/utils/get_dialogs.py | 32 +- api/app/core/memory/agent/utils/llm_tools.py | 13 +- .../utils/prompt/direct_summary_prompt.jinja2 | 61 + .../utils/prompt/fail_summary_prompt.jinja2 | 43 + api/app/core/memory/agent/utils/redis_tool.py | 26 +- .../core/memory/agent/utils/session_tools.py | 18 +- .../core/memory/agent/utils/write_tools.py | 16 +- .../core/memory/analytics/api_docs_parser.py | 3 +- .../core/memory/analytics/hot_memory_tags.py | 36 +- .../analytics/implicit_memory/data_source.py | 4 +- .../memory/analytics/recent_activity_stats.py | 88 +- api/app/core/memory/evaluation/__init__.py | 1 - api/app/core/memory/evaluation/benchmark.md | 30 - .../core/memory/evaluation/common/metrics.py | 100 -- .../memory/evaluation/dialogue_queries.py | 60 - .../memory/evaluation/extraction_utils.py | 341 ----- .../evaluation/locomo/locomo_benchmark.py | 575 ------- .../evaluation/locomo/locomo_metrics.py | 225 --- .../memory/evaluation/locomo/locomo_test.py | 810 ---------- .../memory/evaluation/locomo/locomo_utils.py | 626 -------- .../evaluation/locomo/qwen_search_eval.py | 878 ----------- .../longmemeval/qwen_search_eval.py | 1363 ----------------- .../evaluation/longmemeval/test_eval.py | 1330 ---------------- .../memory/evaluation/memsciqa/evaluate_qa.py | 324 ---- .../evaluation/memsciqa/memsciqa-test.py | 576 ------- api/app/core/memory/evaluation/run_eval.py | 150 -- .../core/memory/llm_tools/chunker_client.py | 24 +- api/app/core/memory/models/config_models.py | 4 +- api/app/core/memory/models/graph_models.py | 16 +- api/app/core/memory/models/message_models.py | 20 +- api/app/core/memory/src/search.py | 45 +- .../data_preprocessing/data_preprocessor.py | 10 +- .../deduplication/deduped_and_disamb.py | 18 +- .../deduplication/entity_dedup_llm.py | 18 +- .../deduplication/second_layer_dedup.py | 8 +- .../deduplication/two_stage_dedup.py | 14 +- .../extraction_orchestrator.py | 90 +- .../knowledge_extraction/memory_summary.py | 6 +- .../statement_extraction.py | 16 +- .../temporal_extraction.py | 2 +- .../triplet_extraction.py | 2 +- .../access_history_manager.py | 86 +- .../forgetting_engine/config_utils.py | 16 +- .../forgetting_engine/forgetting_scheduler.py | 31 +- .../forgetting_engine/forgetting_strategy.py | 31 +- .../storage_services/search/__init__.py | 6 +- .../storage_services/search/hybrid_search.py | 14 +- .../storage_services/search/keyword_search.py | 12 +- .../search/search_strategy.py | 10 +- .../search/semantic_search.py | 12 +- api/app/core/memory/utils/config/get_data.py | 4 +- api/app/core/memory/utils/log/audit_logger.py | 12 +- api/app/core/models/scripts/__init__.py | 1 + .../core/models/scripts/bedrock_models.yaml | 174 +++ .../core/models/scripts/dashscope_models.yaml | 820 ++++++++++ api/app/core/models/scripts/loader.py | 143 ++ .../core/models/scripts/openai_models.yaml | 294 ++++ api/app/core/rag/app/presentation.py | 165 -- api/app/core/rag/vdb/field.py | 2 +- api/app/core/storage/url_signer.py | 2 +- .../core/tools/builtin/baidu_search_tool.py | 4 +- .../validators/memory_config_validators.py | 10 +- api/app/core/workflow/executor.py | 501 +++--- api/app/core/workflow/graph_builder.py | 397 ++++- api/app/core/workflow/nodes/base_node.py | 93 +- api/app/core/workflow/nodes/code/__init__.py | 3 + api/app/core/workflow/nodes/code/config.py | 50 + api/app/core/workflow/nodes/code/node.py | 121 ++ api/app/core/workflow/nodes/configs.py | 14 +- .../workflow/nodes/cycle_graph/iteration.py | 1 - .../core/workflow/nodes/cycle_graph/node.py | 1 - api/app/core/workflow/nodes/end/node.py | 291 +--- api/app/core/workflow/nodes/if_else/node.py | 2 +- api/app/core/workflow/nodes/llm/node.py | 49 +- api/app/core/workflow/nodes/memory/config.py | 7 +- api/app/core/workflow/nodes/memory/node.py | 7 +- api/app/core/workflow/nodes/node_factory.py | 5 +- .../nodes/question_classifier/config.py | 3 +- .../nodes/question_classifier/node.py | 19 +- api/app/core/workflow/nodes/tool/__init__.py | 2 +- api/app/core/workflow/nodes/tool/node.py | 18 +- api/app/main.py | 11 + api/app/models/__init__.py | 8 +- api/app/models/agent_app_config_model.py | 2 +- api/app/models/data_config_model.py | 88 -- api/app/models/memory_config_model.py | 119 +- api/app/models/memory_perceptual_model.py | 2 +- api/app/models/models_model.py | 106 +- api/app/models/multi_agent_model.py | 2 +- api/app/models/tenant_model.py | 4 + api/app/models/user_model.py | 4 + api/app/plugins/__init__.py | 74 + api/app/repositories/app_repository.py | 10 +- api/app/repositories/home_page_repository.py | 30 +- ...ository.py => memory_config_repository.py} | 214 +-- .../memory_perceptual_repository.py | 4 +- api/app/repositories/model_repository.py | 287 +++- api/app/repositories/neo4j/add_edges.py | 4 +- api/app/repositories/neo4j/add_nodes.py | 22 +- .../neo4j/base_neo4j_repository.py | 2 +- api/app/repositories/neo4j/cypher_queries.py | 171 +-- .../repositories/neo4j/dialog_repository.py | 34 +- .../repositories/neo4j/emotion_repository.py | 24 +- api/app/repositories/neo4j/graph_saver.py | 12 +- api/app/repositories/neo4j/graph_search.py | 236 ++- .../neo4j/memory_summary_repository.py | 48 +- api/app/repositories/neo4j/neo4j_connector.py | 48 +- .../neo4j/statement_repository.py | 2 +- api/app/repositories/user_repository.py | 4 +- api/app/repositories/workflow_repository.py | 2 +- api/app/repositories/workspace_repository.py | 24 +- api/app/schemas/app_schema.py | 12 + api/app/schemas/emotion_schema.py | 11 +- api/app/schemas/memory_agent_schema.py | 6 +- api/app/schemas/memory_config_schema.py | 20 +- api/app/schemas/memory_perceptual_schema.py | 8 +- api/app/schemas/memory_reflection_schemas.py | 7 +- api/app/schemas/memory_storage_schema.py | 36 +- api/app/schemas/model_schema.py | 171 ++- api/app/schemas/multi_agent_schema.py | 2 +- api/app/schemas/release_share_schema.py | 14 +- api/app/services/agent_registry.py | 4 +- api/app/services/app_service.py | 10 +- api/app/services/app_statistics_service.py | 193 +++ api/app/services/draft_run_service.py | 41 +- api/app/services/emotion_analytics_service.py | 14 +- api/app/services/emotion_config_service.py | 16 +- .../services/emotion_extraction_service.py | 4 +- api/app/services/llm_router.py | 17 +- api/app/services/master_agent_router.py | 2 +- api/app/services/memory_agent_service.py | 208 ++- api/app/services/memory_api_service.py | 26 +- api/app/services/memory_base_service.py | 18 +- api/app/services/memory_config_service.py | 122 +- api/app/services/memory_dashboard_service.py | 87 +- .../memory_entity_relationship_service.py | 4 +- api/app/services/memory_episodic_service.py | 30 +- api/app/services/memory_explicit_service.py | 16 +- api/app/services/memory_forget_service.py | 81 +- api/app/services/memory_konwledges_server.py | 14 +- api/app/services/memory_perceptual_service.py | 26 +- api/app/services/memory_reflection_service.py | 106 +- api/app/services/memory_storage_service.py | 158 +- api/app/services/model_service.py | 427 +++++- api/app/services/multi_agent_orchestrator.py | 25 +- api/app/services/multi_agent_service.py | 4 +- api/app/services/pilot_run_service.py | 2 +- api/app/services/prompt_optimizer_service.py | 5 +- api/app/services/shared_chat_service.py | 54 +- api/app/services/user_memory_service.py | 42 +- api/app/services/workflow_service.py | 11 +- api/app/tasks.py | 171 ++- api/app/utils/app_config_utils.py | 25 +- api/app/utils/config_utils.py | 45 + api/app/version_info.json | 90 +- api/docker-compose.yml | 12 + api/env.example | 1 + api/migrations/env.py | 3 +- .../versions/325b759cd66b_2026011240.py | 61 + .../versions/5ca246ee7dd4_202601291352.py | 30 + .../versions/5de9b1e28509_20260129212722.py | 80 + .../versions/75f0ec80e50b_202601271517.py | 57 + .../versions/915bed077f8d_202601281340.py | 224 +++ api/pyproject.toml | 1 - api/requirements.txt | 1 - api/uv.lock | 2 +- api_key_mcp_server.py | 38 - basic_auth_mcp_server.py | 45 - bearer_token_mcp_server.py | 40 - mcp_base.py | 111 -- redbear-mem-benchmark | 2 +- sandbox/Dockerfile | 42 + sandbox/app/config.py | 134 ++ sandbox/app/controllers/__init__.py | 8 + sandbox/app/controllers/health_controller.py | 12 + sandbox/app/controllers/sandbox_controller.py | 59 + sandbox/app/core/__init__.py | 1 + sandbox/app/core/encryption.py | 33 + sandbox/app/core/executor.py | 47 + sandbox/app/core/runners/__init__.py | 1 + sandbox/app/core/runners/python/__init__.py | 4 + sandbox/app/core/runners/python/env.py | 50 + sandbox/app/core/runners/python/prescript.py | 56 + .../app/core/runners/python/python_runner.py | 154 ++ sandbox/app/core/runners/python/settings.py | 62 + sandbox/app/dependencies.py | 161 ++ sandbox/app/logger.py | 42 + sandbox/app/middleware/__init__.py | 1 + sandbox/app/middleware/auth.py | 15 + sandbox/app/middleware/concurrency.py | 48 + sandbox/app/models.py | 80 + sandbox/app/services/__init__.py | 1 + sandbox/app/services/python_service.py | 80 + sandbox/config.yaml | 20 + sandbox/dependencies/python-requirements.txt | 4 + sandbox/lib/seccomp_nodejs/Cargo.lock | 7 + sandbox/lib/seccomp_nodejs/Cargo.toml | 6 + sandbox/lib/seccomp_nodejs/src/lib.rs | 0 sandbox/lib/seccomp_python/Cargo.lock | 23 + sandbox/lib/seccomp_python/Cargo.toml | 12 + sandbox/lib/seccomp_python/src/lib.rs | 195 +++ sandbox/lib/seccomp_python/src/syscalls.rs | 85 + sandbox/main.py | 97 ++ sandbox/requirements.txt | 20 + sandbox/script/env.sh | 53 + simple_mcp_server.py | 130 -- web/src/api/application.ts | 6 +- web/src/api/fileStorage.ts | 25 + web/src/api/knowledgeBase.ts | 2 +- web/src/api/memory.ts | 48 +- web/src/api/models.ts | 71 +- web/src/assets/images/empty/pageEmpty.png | Bin 0 -> 161041 bytes web/src/assets/images/model/bedrock.svg | 15 + web/src/assets/images/model/dashscope.png | Bin 0 -> 2835 bytes web/src/assets/images/model/gpustack.png | Bin 0 -> 57988 bytes web/src/assets/images/model/ollama.svg | 15 + web/src/assets/images/model/openai.svg | 4 + web/src/assets/images/model/xinference.svg | 24 + web/src/components/Chat/ChatContent.tsx | 17 +- web/src/components/Chat/types.ts | 5 +- web/src/components/Empty/PageEmpty.tsx | 16 + web/src/components/Markdown/CodeBlock.tsx | 16 +- web/src/components/PageTabs/index.module.css | 13 + web/src/components/PageTabs/index.tsx | 18 + web/src/components/RbCard/Card.tsx | 10 +- web/src/components/Upload/UploadImages.tsx | 70 +- web/src/components/Upload/index.module.less | 7 + web/src/i18n/en.ts | 88 +- web/src/i18n/zh.ts | 91 +- web/src/styles/antdThemeConfig.ts | 5 +- web/src/utils/request.ts | 5 +- web/src/utils/stream.ts | 14 + web/src/views/ApplicationConfig/Agent.tsx | 20 +- web/src/views/ApplicationConfig/Cluster.tsx | 2 +- .../views/ApplicationConfig/Statistics.tsx | 86 ++ .../components/AiPromptModal.tsx | 2 +- .../components/ConfigHeader.tsx | 2 +- .../Knowledge/KnowledgeConfigModal.tsx | 2 +- .../Knowledge/KnowledgeGlobalConfigModal.tsx | 2 +- .../ApplicationConfig/components/LineCard.tsx | 127 ++ web/src/views/ApplicationConfig/index.tsx | 2 + web/src/views/ApplicationConfig/types.ts | 15 + web/src/views/EmotionEngine/index.tsx | 2 +- web/src/views/MemberManagement/index.tsx | 4 +- web/src/views/MemoryConversation/index.tsx | 6 +- .../views/MemoryExtractionEngine/constant.ts | 604 +------- .../views/MemoryExtractionEngine/index.tsx | 10 +- web/src/views/MemoryManagement/types.ts | 1 - web/src/views/ModelManagement/Group.tsx | 92 ++ web/src/views/ModelManagement/List.tsx | 86 ++ web/src/views/ModelManagement/Square.tsx | 104 ++ .../components/ConfigModal.tsx | 171 --- .../components/CustomModelModal.tsx | 168 ++ .../components/GroupModelModal.tsx | 173 +++ .../components/KeyConfigModal.tsx | 92 ++ .../ModelImplement/SubModelModal.tsx | 181 +++ .../components/ModelImplement/index.tsx | 99 ++ .../components/ModelImplement/types.ts | 17 + .../components/ModelListDetail.tsx | 142 ++ .../components/ModelSquareDetail.tsx | 106 ++ .../components/MultiKeyConfigModal.tsx | 122 ++ web/src/views/ModelManagement/index.tsx | 181 ++- web/src/views/ModelManagement/types.ts | 179 ++- web/src/views/ModelManagement/utils.ts | 26 + web/src/views/SelfReflectionEngine/index.tsx | 2 +- web/src/views/SpaceConfig/index.tsx | 6 +- .../SpaceManagement/components/SpaceModal.tsx | 12 +- .../components/PerceptualLastInfo.tsx | 11 +- .../views/Workflow/components/Chat/Chat.tsx | 206 ++- .../Workflow/components/Chat/chat.module.css | 45 + .../Workflow/components/Editor/index.tsx | 47 +- .../plugin/JavaScriptHighlightPlugin.tsx | 164 ++ .../Editor/plugin/Python3HighlightPlugin.tsx | 159 ++ .../Properties/CodeExecution/OutputList.tsx | 86 ++ .../Properties/CodeExecution/index.tsx | 128 ++ .../Properties/HttpRequest/EditableTable.tsx | 3 +- .../Properties/HttpRequest/index.tsx | 1 + .../Properties/JinjaRender/index.tsx | 4 +- .../Knowledge/KnowledgeConfigModal.tsx | 6 +- .../Knowledge/KnowledgeGlobalConfigModal.tsx | 2 +- .../Properties/MappingList/index.tsx | 30 +- .../components/Properties/MessageEditor.tsx | 22 +- .../Properties/hooks/useVariableList.ts | 7 +- .../Workflow/components/Properties/index.tsx | 10 +- .../Properties/properties.module.css | 3 + web/src/views/Workflow/constant.ts | 39 +- .../views/Workflow/hooks/useWorkflowGraph.ts | 85 +- 320 files changed, 11769 insertions(+), 11942 deletions(-) create mode 100644 api/app/__init__.py create mode 100644 api/app/core/memory/agent/utils/prompt/direct_summary_prompt.jinja2 create mode 100644 api/app/core/memory/agent/utils/prompt/fail_summary_prompt.jinja2 delete mode 100644 api/app/core/memory/evaluation/__init__.py delete mode 100644 api/app/core/memory/evaluation/benchmark.md delete mode 100644 api/app/core/memory/evaluation/common/metrics.py delete mode 100644 api/app/core/memory/evaluation/dialogue_queries.py delete mode 100644 api/app/core/memory/evaluation/extraction_utils.py delete mode 100644 api/app/core/memory/evaluation/locomo/locomo_benchmark.py delete mode 100644 api/app/core/memory/evaluation/locomo/locomo_metrics.py delete mode 100644 api/app/core/memory/evaluation/locomo/locomo_test.py delete mode 100644 api/app/core/memory/evaluation/locomo/locomo_utils.py delete mode 100644 api/app/core/memory/evaluation/locomo/qwen_search_eval.py delete mode 100644 api/app/core/memory/evaluation/longmemeval/qwen_search_eval.py delete mode 100644 api/app/core/memory/evaluation/longmemeval/test_eval.py delete mode 100644 api/app/core/memory/evaluation/memsciqa/evaluate_qa.py delete mode 100644 api/app/core/memory/evaluation/memsciqa/memsciqa-test.py delete mode 100644 api/app/core/memory/evaluation/run_eval.py create mode 100644 api/app/core/models/scripts/__init__.py create mode 100644 api/app/core/models/scripts/bedrock_models.yaml create mode 100644 api/app/core/models/scripts/dashscope_models.yaml create mode 100644 api/app/core/models/scripts/loader.py create mode 100644 api/app/core/models/scripts/openai_models.yaml delete mode 100644 api/app/core/rag/app/presentation.py create mode 100644 api/app/core/workflow/nodes/code/config.py create mode 100644 api/app/core/workflow/nodes/code/node.py delete mode 100644 api/app/models/data_config_model.py create mode 100644 api/app/plugins/__init__.py rename api/app/repositories/{data_config_repository.py => memory_config_repository.py} (72%) create mode 100644 api/app/services/app_statistics_service.py create mode 100644 api/app/utils/config_utils.py create mode 100644 api/migrations/versions/325b759cd66b_2026011240.py create mode 100644 api/migrations/versions/5ca246ee7dd4_202601291352.py create mode 100644 api/migrations/versions/5de9b1e28509_20260129212722.py create mode 100644 api/migrations/versions/75f0ec80e50b_202601271517.py create mode 100644 api/migrations/versions/915bed077f8d_202601281340.py delete mode 100644 api_key_mcp_server.py delete mode 100644 basic_auth_mcp_server.py delete mode 100644 bearer_token_mcp_server.py delete mode 100644 mcp_base.py create mode 100644 sandbox/Dockerfile create mode 100644 sandbox/app/config.py create mode 100644 sandbox/app/controllers/__init__.py create mode 100644 sandbox/app/controllers/health_controller.py create mode 100644 sandbox/app/controllers/sandbox_controller.py create mode 100644 sandbox/app/core/__init__.py create mode 100644 sandbox/app/core/encryption.py create mode 100644 sandbox/app/core/executor.py create mode 100644 sandbox/app/core/runners/__init__.py create mode 100644 sandbox/app/core/runners/python/__init__.py create mode 100644 sandbox/app/core/runners/python/env.py create mode 100644 sandbox/app/core/runners/python/prescript.py create mode 100644 sandbox/app/core/runners/python/python_runner.py create mode 100644 sandbox/app/core/runners/python/settings.py create mode 100644 sandbox/app/dependencies.py create mode 100644 sandbox/app/logger.py create mode 100644 sandbox/app/middleware/__init__.py create mode 100644 sandbox/app/middleware/auth.py create mode 100644 sandbox/app/middleware/concurrency.py create mode 100644 sandbox/app/models.py create mode 100644 sandbox/app/services/__init__.py create mode 100644 sandbox/app/services/python_service.py create mode 100644 sandbox/config.yaml create mode 100644 sandbox/dependencies/python-requirements.txt create mode 100644 sandbox/lib/seccomp_nodejs/Cargo.lock create mode 100644 sandbox/lib/seccomp_nodejs/Cargo.toml create mode 100644 sandbox/lib/seccomp_nodejs/src/lib.rs create mode 100644 sandbox/lib/seccomp_python/Cargo.lock create mode 100644 sandbox/lib/seccomp_python/Cargo.toml create mode 100644 sandbox/lib/seccomp_python/src/lib.rs create mode 100644 sandbox/lib/seccomp_python/src/syscalls.rs create mode 100644 sandbox/main.py create mode 100644 sandbox/requirements.txt create mode 100644 sandbox/script/env.sh delete mode 100644 simple_mcp_server.py create mode 100644 web/src/api/fileStorage.ts create mode 100644 web/src/assets/images/empty/pageEmpty.png create mode 100644 web/src/assets/images/model/bedrock.svg create mode 100644 web/src/assets/images/model/dashscope.png create mode 100644 web/src/assets/images/model/gpustack.png create mode 100644 web/src/assets/images/model/ollama.svg create mode 100644 web/src/assets/images/model/openai.svg create mode 100644 web/src/assets/images/model/xinference.svg create mode 100644 web/src/components/Empty/PageEmpty.tsx create mode 100644 web/src/components/PageTabs/index.module.css create mode 100644 web/src/components/PageTabs/index.tsx create mode 100644 web/src/components/Upload/index.module.less create mode 100644 web/src/views/ApplicationConfig/Statistics.tsx create mode 100644 web/src/views/ApplicationConfig/components/LineCard.tsx create mode 100644 web/src/views/ModelManagement/Group.tsx create mode 100644 web/src/views/ModelManagement/List.tsx create mode 100644 web/src/views/ModelManagement/Square.tsx delete mode 100644 web/src/views/ModelManagement/components/ConfigModal.tsx create mode 100644 web/src/views/ModelManagement/components/CustomModelModal.tsx create mode 100644 web/src/views/ModelManagement/components/GroupModelModal.tsx create mode 100644 web/src/views/ModelManagement/components/KeyConfigModal.tsx create mode 100644 web/src/views/ModelManagement/components/ModelImplement/SubModelModal.tsx create mode 100644 web/src/views/ModelManagement/components/ModelImplement/index.tsx create mode 100644 web/src/views/ModelManagement/components/ModelImplement/types.ts create mode 100644 web/src/views/ModelManagement/components/ModelListDetail.tsx create mode 100644 web/src/views/ModelManagement/components/ModelSquareDetail.tsx create mode 100644 web/src/views/ModelManagement/components/MultiKeyConfigModal.tsx create mode 100644 web/src/views/ModelManagement/utils.ts create mode 100644 web/src/views/Workflow/components/Chat/chat.module.css create mode 100644 web/src/views/Workflow/components/Editor/plugin/JavaScriptHighlightPlugin.tsx create mode 100644 web/src/views/Workflow/components/Editor/plugin/Python3HighlightPlugin.tsx create mode 100644 web/src/views/Workflow/components/Properties/CodeExecution/OutputList.tsx create mode 100644 web/src/views/Workflow/components/Properties/CodeExecution/index.tsx diff --git a/.gitignore b/.gitignore index c2648945..de160688 100644 --- a/.gitignore +++ b/.gitignore @@ -35,3 +35,6 @@ nltk_data/ tika-server*.jar* cl100k_base.tiktoken libssl*.deb + +sandbox/lib/seccomp_python/target +sandbox/lib/seccomp_nodejs/target diff --git a/api/app/__init__.py b/api/app/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/api/app/controllers/app_controller.py b/api/app/controllers/app_controller.py index 3b4e5a25..d57ee69d 100644 --- a/api/app/controllers/app_controller.py +++ b/api/app/controllers/app_controller.py @@ -872,3 +872,44 @@ async def update_workflow_config( workspace_id = current_user.current_workspace_id cfg = app_service.update_workflow_config(db, app_id=app_id, data=payload, workspace_id=workspace_id) return success(data=WorkflowConfigSchema.model_validate(cfg)) + + +@router.get("/{app_id}/statistics", summary="应用统计数据") +@cur_workspace_access_guard() +def get_app_statistics( + app_id: uuid.UUID, + start_date: int, + end_date: int, + db: Session = Depends(get_db), + current_user=Depends(get_current_user), +): + """获取应用统计数据 + + Args: + app_id: 应用ID + start_date: 开始时间戳（毫秒） + end_date: 结束时间戳（毫秒） + + Returns: + - daily_conversations: 每日会话数统计 + - total_conversations: 总会话数 + - daily_new_users: 每日新增用户数 + - total_new_users: 总新增用户数 + - daily_api_calls: 每日API调用次数 + - total_api_calls: 总API调用次数 + - daily_tokens: 每日token消耗 + - total_tokens: 总token消耗 + """ + workspace_id = current_user.current_workspace_id + + from app.services.app_statistics_service import AppStatisticsService + stats_service = AppStatisticsService(db) + + result = stats_service.get_app_statistics( + app_id=app_id, + workspace_id=workspace_id, + start_date=start_date, + end_date=end_date + ) + + return success(data=result) diff --git a/api/app/controllers/emotion_config_controller.py b/api/app/controllers/emotion_config_controller.py index 76450d8a..b1630ee6 100644 --- a/api/app/controllers/emotion_config_controller.py +++ b/api/app/controllers/emotion_config_controller.py @@ -7,11 +7,13 @@ Routes: GET /memory/config/emotion - 获取情绪引擎配置 POST /memory/config/emotion - 更新情绪引擎配置 """ +import uuid from fastapi import APIRouter, Depends, Query, HTTPException, status from pydantic import BaseModel, Field -from typing import Optional +from typing import Optional, Union from sqlalchemy.orm import Session +from uuid import UUID from app.core.response_utils import success from app.dependencies import get_current_user @@ -20,6 +22,7 @@ from app.schemas.response_schema import ApiResponse from app.services.emotion_config_service import EmotionConfigService from app.core.logging_config import get_api_logger from app.db import get_db +from app.utils.config_utils import resolve_config_id # 获取API专用日志器 api_logger = get_api_logger() @@ -32,11 +35,11 @@ router = APIRouter( class EmotionConfigQuery(BaseModel): """情绪配置查询请求模型""" - config_id: int = Field(..., description="配置ID") + config_id: UUID = Field(..., description="配置ID") class EmotionConfigUpdate(BaseModel): """情绪配置更新请求模型""" - config_id: int = Field(..., description="配置ID") + config_id: Union[uuid.UUID, int, str]= Field(..., description="配置ID") emotion_enabled: bool = Field(..., description="是否启用情绪提取") emotion_model_id: Optional[str] = Field(None, description="情绪分析专用模型ID") emotion_extract_keywords: bool = Field(..., description="是否提取情绪关键词") @@ -45,7 +48,7 @@ class EmotionConfigUpdate(BaseModel): @router.get("/read_config", response_model=ApiResponse) def get_emotion_config( - config_id: int = Query(..., description="配置ID"), + config_id: UUID|int = Query(..., description="配置ID"), db: Session = Depends(get_db), current_user: User = Depends(get_current_user), ): @@ -78,7 +81,7 @@ def get_emotion_config( f"用户 {current_user.username} 请求获取情绪配置", extra={"config_id": config_id} ) - + config_id=resolve_config_id(config_id, db) # 初始化服务 config_service = EmotionConfigService(db) @@ -157,6 +160,7 @@ def update_emotion_config( } } """ + config.config_id=resolve_config_id(config.config_id, db) try: api_logger.info( f"用户 {current_user.username} 请求更新情绪配置", diff --git a/api/app/controllers/emotion_controller.py b/api/app/controllers/emotion_controller.py index 154a3928..cd199aa7 100644 --- a/api/app/controllers/emotion_controller.py +++ b/api/app/controllers/emotion_controller.py @@ -53,7 +53,7 @@ async def get_emotion_tags( api_logger.info( f"用户 {current_user.username} 请求获取情绪标签统计", extra={ - "group_id": request.group_id, + "end_user_id": request.end_user_id, "emotion_type": request.emotion_type, "start_date": request.start_date, "end_date": request.end_date, @@ -63,7 +63,7 @@ async def get_emotion_tags( # 调用服务层 data = await emotion_service.get_emotion_tags( - end_user_id=request.group_id, + end_user_id=request.end_user_id, emotion_type=request.emotion_type, start_date=request.start_date, end_date=request.end_date, @@ -73,7 +73,7 @@ async def get_emotion_tags( api_logger.info( "情绪标签统计获取成功", extra={ - "group_id": request.group_id, + "end_user_id": request.end_user_id, "total_count": data.get("total_count", 0), "tags_count": len(data.get("tags", [])) } @@ -84,7 +84,7 @@ async def get_emotion_tags( except Exception as e: api_logger.error( f"获取情绪标签统计失败: {str(e)}", - extra={"group_id": request.group_id}, + extra={"end_user_id": request.end_user_id}, exc_info=True ) raise HTTPException( @@ -105,7 +105,7 @@ async def get_emotion_wordcloud( api_logger.info( f"用户 {current_user.username} 请求获取情绪词云数据", extra={ - "group_id": request.group_id, + "end_user_id": request.end_user_id, "emotion_type": request.emotion_type, "limit": request.limit } @@ -113,7 +113,7 @@ async def get_emotion_wordcloud( # 调用服务层 data = await emotion_service.get_emotion_wordcloud( - end_user_id=request.group_id, + end_user_id=request.end_user_id, emotion_type=request.emotion_type, limit=request.limit ) @@ -121,7 +121,7 @@ async def get_emotion_wordcloud( api_logger.info( "情绪词云数据获取成功", extra={ - "group_id": request.group_id, + "end_user_id": request.end_user_id, "total_keywords": data.get("total_keywords", 0) } ) @@ -131,7 +131,7 @@ async def get_emotion_wordcloud( except Exception as e: api_logger.error( f"获取情绪词云数据失败: {str(e)}", - extra={"group_id": request.group_id}, + extra={"end_user_id": request.end_user_id}, exc_info=True ) raise HTTPException( @@ -159,21 +159,21 @@ async def get_emotion_health( api_logger.info( f"用户 {current_user.username} 请求获取情绪健康指数", extra={ - "group_id": request.group_id, + "end_user_id": request.end_user_id, "time_range": request.time_range } ) # 调用服务层 data = await emotion_service.calculate_emotion_health_index( - end_user_id=request.group_id, + end_user_id=request.end_user_id, time_range=request.time_range ) api_logger.info( "情绪健康指数获取成功", extra={ - "group_id": request.group_id, + "end_user_id": request.end_user_id, "health_score": data.get("health_score", 0), "level": data.get("level", "未知") } @@ -186,7 +186,7 @@ async def get_emotion_health( except Exception as e: api_logger.error( f"获取情绪健康指数失败: {str(e)}", - extra={"group_id": request.group_id}, + extra={"end_user_id": request.end_user_id}, exc_info=True ) raise HTTPException( @@ -206,7 +206,7 @@ async def get_emotion_suggestions( """获取个性化情绪建议（从缓存读取） Args: - request: 包含 group_id 和可选的 config_id + request: 包含 end_user_id 和可选的 config_id db: 数据库会话 current_user: 当前用户 @@ -217,22 +217,22 @@ async def get_emotion_suggestions( api_logger.info( f"用户 {current_user.username} 请求获取个性化情绪建议（缓存）", extra={ - "group_id": request.group_id, + "end_user_id": request.end_user_id, "config_id": request.config_id } ) # 从缓存获取建议 data = await emotion_service.get_cached_suggestions( - end_user_id=request.group_id, + end_user_id=request.end_user_id, db=db ) if data is None: # 缓存不存在或已过期 api_logger.info( - f"用户 {request.group_id} 的建议缓存不存在或已过期", - extra={"group_id": request.group_id} + f"用户 {request.end_user_id} 的建议缓存不存在或已过期", + extra={"end_user_id": request.end_user_id} ) return fail( BizCode.NOT_FOUND, @@ -243,7 +243,7 @@ async def get_emotion_suggestions( api_logger.info( "个性化建议获取成功（缓存）", extra={ - "group_id": request.group_id, + "end_user_id": request.end_user_id, "suggestions_count": len(data.get("suggestions", [])) } ) @@ -253,7 +253,7 @@ async def get_emotion_suggestions( except Exception as e: api_logger.error( f"获取个性化建议失败: {str(e)}", - extra={"group_id": request.group_id}, + extra={"end_user_id": request.end_user_id}, exc_info=True ) raise HTTPException( diff --git a/api/app/controllers/file_storage_controller.py b/api/app/controllers/file_storage_controller.py index c28ffe6c..1a7e8ad2 100644 --- a/api/app/controllers/file_storage_controller.py +++ b/api/app/controllers/file_storage_controller.py @@ -310,7 +310,7 @@ async def get_file_url( try: if permanent: # Generate permanent URL (no expiration check) - server_url = f"http://{settings.SERVER_IP}:8000/api" + server_url = settings.FILE_LOCAL_SERVER_URL url = f"{server_url}/storage/permanent/{file_id}" return success( data={ diff --git a/api/app/controllers/implicit_memory_controller.py b/api/app/controllers/implicit_memory_controller.py index a53290e2..96e437d6 100644 --- a/api/app/controllers/implicit_memory_controller.py +++ b/api/app/controllers/implicit_memory_controller.py @@ -122,10 +122,10 @@ def validate_confidence_threshold(threshold: float) -> None: raise ValueError("confidence_threshold must be between 0.0 and 1.0") -@router.get("/preferences/{user_id}", response_model=ApiResponse) +@router.get("/preferences/{end_user_id}", response_model=ApiResponse) @cur_workspace_access_guard() async def get_preference_tags( - user_id: str, + end_user_id: str, confidence_threshold: float = Query(0.5, ge=0.0, le=1.0, description="Minimum confidence threshold"), tag_category: Optional[str] = Query(None, description="Filter by tag category"), start_date: Optional[datetime] = Query(None, description="Filter start date"), @@ -137,7 +137,7 @@ async def get_preference_tags( Get user preference tags from cache. Args: - user_id: Target user ID + end_user_id: Target end user ID confidence_threshold: Minimum confidence score (0.0-1.0) tag_category: Optional category filter start_date: Optional start date filter @@ -146,20 +146,20 @@ async def get_preference_tags( Returns: List of preference tags from cache """ - api_logger.info(f"Preference tags requested for user: {user_id} (from cache)") + api_logger.info(f"Preference tags requested for user: {end_user_id} (from cache)") try: # Validate inputs - validate_user_id(user_id) + validate_user_id(end_user_id) # Create service with user-specific config - service = ImplicitMemoryService(db=db, end_user_id=user_id) + service = ImplicitMemoryService(db=db, end_user_id=end_user_id) # Get cached profile - cached_profile = await service.get_cached_profile(end_user_id=user_id, db=db) + cached_profile = await service.get_cached_profile(end_user_id=end_user_id, db=db) if cached_profile is None: - api_logger.info(f"用户 {user_id} 的画像缓存不存在或已过期") + api_logger.info(f"用户 {end_user_id} 的画像缓存不存在或已过期") return fail( BizCode.NOT_FOUND, "画像缓存不存在或已过期，请右上角刷新生成新画像", @@ -192,17 +192,17 @@ async def get_preference_tags( filtered_preferences.append(pref) - api_logger.info(f"Retrieved {len(filtered_preferences)} preference tags for user: {user_id} (from cache)") + api_logger.info(f"Retrieved {len(filtered_preferences)} preference tags for user: {end_user_id} (from cache)") return success(data=filtered_preferences, msg="偏好标签获取成功（缓存）") except Exception as e: - return handle_implicit_memory_error(e, "偏好标签获取", user_id) + return handle_implicit_memory_error(e, "偏好标签获取", end_user_id) -@router.get("/portrait/{user_id}", response_model=ApiResponse) +@router.get("/portrait/{end_user_id}", response_model=ApiResponse) @cur_workspace_access_guard() async def get_dimension_portrait( - user_id: str, + end_user_id: str, include_history: bool = Query(False, description="Include historical trends"), db: Session = Depends(get_db), current_user: User = Depends(get_current_user) @@ -211,26 +211,26 @@ async def get_dimension_portrait( Get user's four-dimension personality portrait from cache. Args: - user_id: Target user ID + end_user_id: Target end user ID include_history: Whether to include historical trend data (ignored for cached data) Returns: Four-dimension personality portrait from cache """ - api_logger.info(f"Dimension portrait requested for user: {user_id} (from cache)") + api_logger.info(f"Dimension portrait requested for user: {end_user_id} (from cache)") try: # Validate inputs - validate_user_id(user_id) + validate_user_id(end_user_id) # Create service with user-specific config - service = ImplicitMemoryService(db=db, end_user_id=user_id) + service = ImplicitMemoryService(db=db, end_user_id=end_user_id) # Get cached profile - cached_profile = await service.get_cached_profile(end_user_id=user_id, db=db) + cached_profile = await service.get_cached_profile(end_user_id=end_user_id, db=db) if cached_profile is None: - api_logger.info(f"用户 {user_id} 的画像缓存不存在或已过期") + api_logger.info(f"用户 {end_user_id} 的画像缓存不存在或已过期") return fail( BizCode.NOT_FOUND, "画像缓存不存在或已过期，请右上角刷新生成新画像", @@ -240,17 +240,17 @@ async def get_dimension_portrait( # Extract portrait from cache portrait = cached_profile.get("portrait", {}) - api_logger.info(f"Dimension portrait retrieved for user: {user_id} (from cache)") + api_logger.info(f"Dimension portrait retrieved for user: {end_user_id} (from cache)") return success(data=portrait, msg="四维画像获取成功（缓存）") except Exception as e: - return handle_implicit_memory_error(e, "四维画像获取", user_id) + return handle_implicit_memory_error(e, "四维画像获取", end_user_id) -@router.get("/interest-areas/{user_id}", response_model=ApiResponse) +@router.get("/interest-areas/{end_user_id}", response_model=ApiResponse) @cur_workspace_access_guard() async def get_interest_area_distribution( - user_id: str, + end_user_id: str, include_trends: bool = Query(False, description="Include trend analysis"), db: Session = Depends(get_db), current_user: User = Depends(get_current_user) @@ -259,26 +259,26 @@ async def get_interest_area_distribution( Get user's interest area distribution from cache. Args: - user_id: Target user ID + end_user_id: Target end user ID include_trends: Whether to include trend analysis data (ignored for cached data) Returns: Interest area distribution from cache """ - api_logger.info(f"Interest area distribution requested for user: {user_id} (from cache)") + api_logger.info(f"Interest area distribution requested for user: {end_user_id} (from cache)") try: # Validate inputs - validate_user_id(user_id) + validate_user_id(end_user_id) # Create service with user-specific config - service = ImplicitMemoryService(db=db, end_user_id=user_id) + service = ImplicitMemoryService(db=db, end_user_id=end_user_id) # Get cached profile - cached_profile = await service.get_cached_profile(end_user_id=user_id, db=db) + cached_profile = await service.get_cached_profile(end_user_id=end_user_id, db=db) if cached_profile is None: - api_logger.info(f"用户 {user_id} 的画像缓存不存在或已过期") + api_logger.info(f"用户 {end_user_id} 的画像缓存不存在或已过期") return fail( BizCode.NOT_FOUND, "画像缓存不存在或已过期，请右上角刷新生成新画像", @@ -288,17 +288,17 @@ async def get_interest_area_distribution( # Extract interest areas from cache interest_areas = cached_profile.get("interest_areas", {}) - api_logger.info(f"Interest area distribution retrieved for user: {user_id} (from cache)") + api_logger.info(f"Interest area distribution retrieved for user: {end_user_id} (from cache)") return success(data=interest_areas, msg="兴趣领域分布获取成功（缓存）") except Exception as e: - return handle_implicit_memory_error(e, "兴趣领域分布获取", user_id) + return handle_implicit_memory_error(e, "兴趣领域分布获取", end_user_id) -@router.get("/habits/{user_id}", response_model=ApiResponse) +@router.get("/habits/{end_user_id}", response_model=ApiResponse) @cur_workspace_access_guard() async def get_behavior_habits( - user_id: str, + end_user_id: str, confidence_level: Optional[str] = Query(None, regex="^(high|medium|low)$", description="Filter by confidence level"), frequency_pattern: Optional[str] = Query(None, regex="^(daily|weekly|monthly|seasonal|occasional|event_triggered)$", description="Filter by frequency pattern"), time_period: Optional[str] = Query(None, regex="^(current|past)$", description="Filter by time period"), @@ -309,7 +309,7 @@ async def get_behavior_habits( Get user's behavioral habits from cache. Args: - user_id: Target user ID + end_user_id: Target end user ID confidence_level: Filter by confidence level (high, medium, low) frequency_pattern: Filter by frequency pattern (daily, weekly, monthly, seasonal, occasional, event_triggered) time_period: Filter by time period (current, past) @@ -317,20 +317,20 @@ async def get_behavior_habits( Returns: List of behavioral habits from cache """ - api_logger.info(f"Behavior habits requested for user: {user_id} (from cache)") + api_logger.info(f"Behavior habits requested for user: {end_user_id} (from cache)") try: # Validate inputs - validate_user_id(user_id) + validate_user_id(end_user_id) # Create service with user-specific config - service = ImplicitMemoryService(db=db, end_user_id=user_id) + service = ImplicitMemoryService(db=db, end_user_id=end_user_id) # Get cached profile - cached_profile = await service.get_cached_profile(end_user_id=user_id, db=db) + cached_profile = await service.get_cached_profile(end_user_id=end_user_id, db=db) if cached_profile is None: - api_logger.info(f"用户 {user_id} 的画像缓存不存在或已过期") + api_logger.info(f"用户 {end_user_id} 的画像缓存不存在或已过期") return fail( BizCode.NOT_FOUND, "画像缓存不存在或已过期，请右上角刷新生成新画像", @@ -368,11 +368,11 @@ async def get_behavior_habits( filtered_habits.append(habit) - api_logger.info(f"Retrieved {len(filtered_habits)} behavior habits for user: {user_id} (from cache)") + api_logger.info(f"Retrieved {len(filtered_habits)} behavior habits for user: {end_user_id} (from cache)") return success(data=filtered_habits, msg="行为习惯获取成功（缓存）") except Exception as e: - return handle_implicit_memory_error(e, "行为习惯获取", user_id) + return handle_implicit_memory_error(e, "行为习惯获取", end_user_id) diff --git a/api/app/controllers/memory_agent_controller.py b/api/app/controllers/memory_agent_controller.py index 78a5771f..61b16d9e 100644 --- a/api/app/controllers/memory_agent_controller.py +++ b/api/app/controllers/memory_agent_controller.py @@ -125,7 +125,7 @@ async def write_server( Write service endpoint - processes write operations synchronously Args: - user_input: Write request containing message and group_id + user_input: Write request containing message and end_user_id Returns: Response with write operation status @@ -160,19 +160,18 @@ async def write_server( api_logger.warning("workspace_id 为空，无法使用 rag 存储，将使用 neo4j 存储") storage_type = 'neo4j' - api_logger.info(f"Write service requested for group {user_input.group_id}, storage_type: {storage_type}, user_rag_memory_id: {user_rag_memory_id}") + api_logger.info(f"Write service requested for group {user_input.end_user_id}, storage_type: {storage_type}, user_rag_memory_id: {user_rag_memory_id}") try: - # 获取标准化的消息列表 messages_list = memory_agent_service.get_messages_list(user_input) - result = await memory_agent_service.write_memory( - user_input.group_id, - messages_list, # 传递结构化消息列表 + user_input.end_user_id, + messages_list, config_id, db, storage_type, user_rag_memory_id ) + return success(data=result, msg="写入成功") except BaseException as e: # Handle ExceptionGroup from TaskGroup (Python 3.11+) or BaseExceptionGroup @@ -196,7 +195,7 @@ async def write_server_async( Async write service endpoint - enqueues write processing to Celery Args: - user_input: Write request containing message and group_id + user_input: Write request containing message and end_user_id Returns: Task ID for tracking async operation @@ -226,10 +225,10 @@ async def write_server_async( try: # 获取标准化的消息列表 messages_list = memory_agent_service.get_messages_list(user_input) - + task = celery_app.send_task( "app.core.memory.agent.write_message", - args=[user_input.group_id, messages_list, config_id, storage_type, user_rag_memory_id] + args=[user_input.end_user_id, messages_list, config_id, storage_type, user_rag_memory_id] ) api_logger.info(f"Write task queued: {task.id}") @@ -255,16 +254,14 @@ async def read_server( - "2": Direct answer based on context Args: - user_input: Read request with message, history, search_switch, and group_id + user_input: Read request with message, history, search_switch, and end_user_id Returns: Response with query answer """ config_id = user_input.config_id workspace_id = current_user.current_workspace_id - api_logger.info(f"Read service: workspace_id={workspace_id}, config_id={config_id}") - # 获取 storage_type，如果为 None 则使用默认值 storage_type = workspace_service.get_workspace_storage_type( db=db, workspace_id=workspace_id, @@ -279,12 +276,13 @@ async def read_server( name="USER_RAG_MERORY", workspace_id=workspace_id ) - if knowledge: user_rag_memory_id = str(knowledge.id) + if knowledge: + user_rag_memory_id = str(knowledge.id) - api_logger.info(f"Read service: group={user_input.group_id}, storage_type={storage_type}, user_rag_memory_id={user_rag_memory_id}, workspace_id={workspace_id}") + api_logger.info(f"Read service: group={user_input.end_user_id}, storage_type={storage_type}, user_rag_memory_id={user_rag_memory_id}, workspace_id={workspace_id}") try: result = await memory_agent_service.read_memory( - user_input.group_id, + user_input.end_user_id, user_input.message, user_input.history, user_input.search_switch, @@ -295,17 +293,20 @@ async def read_server( ) if str(user_input.search_switch) == "2": retrieve_info = result['answer'] - history = await SessionService(store).get_history(user_input.group_id, user_input.group_id, user_input.group_id) + history = await SessionService(store).get_history(user_input.end_user_id, user_input.end_user_id, user_input.end_user_id) query = user_input.message - + # 调用 memory_agent_service 的方法生成最终答案 result['answer'] = await memory_agent_service.generate_summary_from_retrieve( + end_user_id=user_input.end_user_id, retrieve_info=retrieve_info, history=history, query=query, config_id=config_id, db=db ) + if "信息不足，无法回答" in result['answer']: + result['answer']=retrieve_info return success(data=result, msg="回复对话消息成功") except BaseException as e: # Handle ExceptionGroup from TaskGroup (Python 3.11+) or BaseExceptionGroup @@ -403,7 +404,7 @@ async def read_server_async( try: task = celery_app.send_task( "app.core.memory.agent.read_message", - args=[user_input.group_id, user_input.message, user_input.history, user_input.search_switch, + args=[user_input.end_user_id, user_input.message, user_input.history, user_input.search_switch, config_id, storage_type, user_rag_memory_id] ) api_logger.info(f"Read task queued: {task.id}") @@ -447,7 +448,7 @@ async def get_read_task_result( return success( data={ "result": task_result.get("result"), - "group_id": task_result.get("group_id"), + "end_user_id": task_result.get("end_user_id"), "elapsed_time": task_result.get("elapsed_time"), "task_id": task_id }, @@ -524,7 +525,7 @@ async def get_write_task_result( return success( data={ "result": task_result.get("result"), - "group_id": task_result.get("group_id"), + "end_user_id": task_result.get("end_user_id"), "elapsed_time": task_result.get("elapsed_time"), "task_id": task_id }, @@ -578,16 +579,16 @@ async def status_type( Determine the type of user message (read or write) Args: - user_input: Request containing user message and group_id + user_input: Request containing user message and end_user_id Returns: Type classification result """ - api_logger.info(f"Status type check requested for group {user_input.group_id}") + api_logger.info(f"Status type check requested for group {user_input.end_user_id}") try: # 获取标准化的消息列表 messages_list = memory_agent_service.get_messages_list(user_input) - + # 将消息列表转换为字符串用于分类 # 只取最后一条用户消息进行分类 last_user_message = "" @@ -595,11 +596,11 @@ async def status_type( if msg.get('role') == 'user': last_user_message = msg.get('content', '') break - + if not last_user_message: # 如果没有用户消息，使用所有消息的内容 last_user_message = " ".join([msg.get('content', '') for msg in messages_list]) - + result = await memory_agent_service.classify_message_type( last_user_message, user_input.config_id, @@ -624,7 +625,7 @@ async def get_knowledge_type_stats_api( 会对缺失类型补 0，返回字典形式。 可选按状态过滤。 - 知识库类型根据当前用户的 current_workspace_id 过滤 - - memory 是 Neo4j 中 Chunk 的数量，根据 end_user_id (group_id) 过滤 + - memory 是 Neo4j 中 Chunk 的数量，根据 end_user_id (end_user_id) 过滤 - 如果用户没有当前工作空间或未提供 end_user_id，对应的统计返回 0 """ api_logger.info(f"Knowledge type stats requested for workspace_id: {current_user.current_workspace_id}, end_user_id: {end_user_id}") @@ -697,7 +698,7 @@ async def get_user_profile_api( current_user: User = Depends(get_current_user) ): """ - 获取工作空间下Popular Memory Tags，包含： + 获取用户详情，包含： - name: 用户名字（直接使用 end_user_id） - tags: 3个用户特征标签（从语句和实体中LLM总结） - hot_tags: 4个热门记忆标签 diff --git a/api/app/controllers/memory_dashboard_controller.py b/api/app/controllers/memory_dashboard_controller.py index e03c1846..88684a39 100644 --- a/api/app/controllers/memory_dashboard_controller.py +++ b/api/app/controllers/memory_dashboard_controller.py @@ -49,63 +49,134 @@ async def get_workspace_end_users( current_user: User = Depends(get_current_user), ): """ - 获取工作空间的宿主列表 + 获取工作空间的宿主列表（高性能优化版本 v2） - 返回格式与原 memory_list 接口中的 end_users 字段相同， - 并包含每个用户的记忆配置信息（memory_config_id 和 memory_config_name） + 优化策略： + 1. 批量查询 end_users（一次查询而非循环） + 2. 并发查询所有用户的记忆数量（Neo4j） + 3. RAG 模式使用批量查询（一次 SQL） + 4. 只返回必要字段减少数据传输 + 5. 添加短期缓存减少重复查询 + 6. 并发执行配置查询和记忆数量查询 + + 返回格式： + { + "end_user": {"id": "uuid", "other_name": "名称"}, + "memory_num": {"total": 数量}, + "memory_config": {"memory_config_id": "id", "memory_config_name": "名称"} + } """ + import asyncio + import json + from app.aioRedis import aio_redis_get, aio_redis_set + workspace_id = current_user.current_workspace_id + + # 尝试从缓存获取（30秒缓存） + cache_key = f"end_users:workspace:{workspace_id}" + try: + cached_data = await aio_redis_get(cache_key) + if cached_data: + api_logger.info(f"从缓存获取宿主列表: workspace_id={workspace_id}") + return success(data=json.loads(cached_data), msg="宿主列表获取成功") + except Exception as e: + api_logger.warning(f"Redis 缓存读取失败: {str(e)}") + # 获取当前空间类型 current_workspace_type = memory_dashboard_service.get_current_workspace_type(db, workspace_id, current_user) api_logger.info(f"用户 {current_user.username} 请求获取工作空间 {workspace_id} 的宿主列表") + + # 获取 end_users（已优化为批量查询） end_users = memory_dashboard_service.get_workspace_end_users( db=db, workspace_id=workspace_id, current_user=current_user ) - - # 批量获取所有用户的记忆配置信息（优化：一次查询而非 N 次） - end_user_ids = [str(user.id) for user in end_users] - memory_configs_map = {} - if end_user_ids: + if not end_users: + api_logger.info("工作空间下没有宿主") + # 缓存空结果，避免重复查询 try: - memory_configs_map = get_end_users_connected_configs_batch(end_user_ids, db) + await aio_redis_set(cache_key, json.dumps([]), expire=30) + except Exception as e: + api_logger.warning(f"Redis 缓存写入失败: {str(e)}") + return success(data=[], msg="宿主列表获取成功") + + end_user_ids = [str(user.id) for user in end_users] + + # 并发执行两个独立的查询任务 + async def get_memory_configs(): + """获取记忆配置（在线程池中执行同步查询）""" + try: + return await asyncio.to_thread( + get_end_users_connected_configs_batch, + end_user_ids, db + ) except Exception as e: api_logger.error(f"批量获取记忆配置失败: {str(e)}") - # 失败时使用空字典，不影响其他数据返回 + return {} + async def get_memory_nums(): + """获取记忆数量""" + if current_workspace_type == "rag": + # RAG 模式：批量查询 + try: + chunk_map = await asyncio.to_thread( + memory_dashboard_service.get_users_total_chunk_batch, + end_user_ids, db, current_user + ) + return {uid: {"total": count} for uid, count in chunk_map.items()} + except Exception as e: + api_logger.error(f"批量获取 RAG chunk 数量失败: {str(e)}") + return {uid: {"total": 0} for uid in end_user_ids} + + elif current_workspace_type == "neo4j": + # Neo4j 模式：并发查询（带并发限制） + # 使用信号量限制并发数，避免大量用户时压垮 Neo4j + MAX_CONCURRENT_QUERIES = 10 + semaphore = asyncio.Semaphore(MAX_CONCURRENT_QUERIES) + + async def get_neo4j_memory_num(end_user_id: str): + async with semaphore: + try: + return await memory_storage_service.search_all(end_user_id) + except Exception as e: + api_logger.error(f"获取用户 {end_user_id} Neo4j 记忆数量失败: {str(e)}") + return {"total": 0} + + memory_nums_list = await asyncio.gather(*[get_neo4j_memory_num(uid) for uid in end_user_ids]) + return {end_user_ids[i]: memory_nums_list[i] for i in range(len(end_user_ids))} + + return {uid: {"total": 0} for uid in end_user_ids} + + # 并发执行配置查询和记忆数量查询 + memory_configs_map, memory_nums_map = await asyncio.gather( + get_memory_configs(), + get_memory_nums() + ) + + # 构建结果（优化：使用列表推导式） result = [] for end_user in end_users: - memory_num = {} - if current_workspace_type == "neo4j": - # EndUser 是 Pydantic 模型，直接访问属性而不是使用 .get() - memory_num = await memory_storage_service.search_all(str(end_user.id)) - elif current_workspace_type == "rag": - memory_num = { - "total":memory_dashboard_service.get_current_user_total_chunk(str(end_user.id), db, current_user) - } - - # 从批量查询结果中获取配置信息 user_id = str(end_user.id) - memory_config_info = memory_configs_map.get(user_id, { - "memory_config_id": None, - "memory_config_name": None - }) - - # 只保留需要的字段，移除 error 字段（如果有） - memory_config = { - "memory_config_id": memory_config_info.get("memory_config_id"), - "memory_config_name": memory_config_info.get("memory_config_name") - } - - result.append( - { - 'end_user': end_user, - 'memory_num': memory_num, - 'memory_config': memory_config + config_info = memory_configs_map.get(user_id, {}) + result.append({ + 'end_user': { + 'id': user_id, + 'other_name': end_user.other_name + }, + 'memory_num': memory_nums_map.get(user_id, {"total": 0}), + 'memory_config': { + "memory_config_id": config_info.get("memory_config_id"), + "memory_config_name": config_info.get("memory_config_name") } - ) - + }) + + # 写入缓存（30秒过期） + try: + await aio_redis_set(cache_key, json.dumps(result), expire=30) + except Exception as e: + api_logger.warning(f"Redis 缓存写入失败: {str(e)}") + api_logger.info(f"成功获取 {len(end_users)} 个宿主记录") return success(data=result, msg="宿主列表获取成功") diff --git a/api/app/controllers/memory_forget_controller.py b/api/app/controllers/memory_forget_controller.py index ca628d0c..2b5ef72f 100644 --- a/api/app/controllers/memory_forget_controller.py +++ b/api/app/controllers/memory_forget_controller.py @@ -11,6 +11,7 @@ """ from typing import Optional +from uuid import UUID from fastapi import APIRouter, Depends from sqlalchemy.orm import Session @@ -33,7 +34,7 @@ from app.schemas.memory_storage_schema import ( ) from app.schemas.response_schema import ApiResponse from app.services.memory_forget_service import MemoryForgetService - +from app.utils.config_utils import resolve_config_id # 获取API专用日志器 api_logger = get_api_logger() @@ -83,7 +84,8 @@ async def trigger_forgetting_cycle( connected_config = get_end_user_connected_config(end_user_id, db) config_id = connected_config.get("memory_config_id") - + config_id = resolve_config_id((config_id), db) + if config_id is None: api_logger.warning(f"终端用户 {end_user_id} 未关联记忆配置") return fail(BizCode.INVALID_PARAMETER, f"终端用户 {end_user_id} 未关联记忆配置", "memory_config_id is None") @@ -106,7 +108,7 @@ async def trigger_forgetting_cycle( # 调用服务层执行遗忘周期 report = await forget_service.trigger_forgetting_cycle( db=db, - group_id=end_user_id, # 服务层方法的参数名是 group_id + end_user_id=end_user_id, # 服务层方法的参数名是 end_user_id max_merge_batch_size=payload.max_merge_batch_size, min_days_since_access=payload.min_days_since_access, config_id=config_id @@ -128,7 +130,7 @@ async def trigger_forgetting_cycle( @router.get("/read_config", response_model=ApiResponse) async def read_forgetting_config( - config_id: int, + config_id: UUID|int, current_user: User = Depends(get_current_user), db: Session = Depends(get_db) ): @@ -157,6 +159,7 @@ async def read_forgetting_config( ) try: + config_id=resolve_config_id(config_id, db) # 调用服务层读取配置 config = forget_service.read_forgetting_config(db=db, config_id=config_id) @@ -194,6 +197,8 @@ async def update_forgetting_config( ApiResponse: 包含更新结果的响应 """ workspace_id = current_user.current_workspace_id + payload.config_id=resolve_config_id((payload.config_id), db) + # 检查用户是否已选择工作空间 if workspace_id is None: @@ -236,7 +241,7 @@ async def update_forgetting_config( @router.get("/stats", response_model=ApiResponse) async def get_forgetting_stats( - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, current_user: User = Depends(get_current_user), db: Session = Depends(get_db) ): @@ -246,7 +251,7 @@ async def get_forgetting_stats( 返回知识层节点统计、激活值分布等信息。 Args: - group_id: 组ID（即 end_user_id，可选） + end_user_id: 组ID（即 end_user_id，可选） current_user: 当前用户 db: 数据库会话 @@ -254,26 +259,25 @@ async def get_forgetting_stats( ApiResponse: 包含统计信息的响应 """ workspace_id = current_user.current_workspace_id - # 检查用户是否已选择工作空间 if workspace_id is None: api_logger.warning(f"用户 {current_user.username} 尝试获取遗忘引擎统计但未选择工作空间") return fail(BizCode.INVALID_PARAMETER, "请先切换到一个工作空间", "current_workspace_id is None") - - # 如果提供了 group_id，通过它获取 config_id + # 如果提供了 end_user_id，通过它获取 config_id config_id = None - if group_id: + if end_user_id: try: from app.services.memory_agent_service import get_end_user_connected_config - connected_config = get_end_user_connected_config(group_id, db) + connected_config = get_end_user_connected_config(end_user_id, db) config_id = connected_config.get("memory_config_id") + config_id = resolve_config_id(config_id, db) if config_id is None: - api_logger.warning(f"终端用户 {group_id} 未关联记忆配置") - return fail(BizCode.INVALID_PARAMETER, f"终端用户 {group_id} 未关联记忆配置", "memory_config_id is None") + api_logger.warning(f"终端用户 {end_user_id} 未关联记忆配置") + return fail(BizCode.INVALID_PARAMETER, f"终端用户 {end_user_id} 未关联记忆配置", "memory_config_id is None") - api_logger.debug(f"通过 group_id={group_id} 获取到 config_id={config_id}") + api_logger.debug(f"通过 end_user_id={end_user_id} 获取到 config_id={config_id}") except ValueError as e: api_logger.warning(f"获取终端用户配置失败: {str(e)}") return fail(BizCode.INVALID_PARAMETER, str(e), "ValueError") @@ -283,14 +287,14 @@ async def get_forgetting_stats( api_logger.info( f"用户 {current_user.username} 在工作空间 {workspace_id} 请求获取遗忘引擎统计: " - f"group_id={group_id}, config_id={config_id}" + f"end_user_id={end_user_id}, config_id={config_id}" ) try: # 调用服务层获取统计信息 stats = await forget_service.get_forgetting_stats( db=db, - group_id=group_id, + end_user_id=end_user_id, config_id=config_id ) @@ -324,7 +328,7 @@ async def get_forgetting_curve( ApiResponse: 包含遗忘曲线数据的响应 """ workspace_id = current_user.current_workspace_id - + request.config_id = resolve_config_id((request.config_id), db) # 检查用户是否已选择工作空间 if workspace_id is None: api_logger.warning(f"用户 {current_user.username} 尝试获取遗忘曲线但未选择工作空间") diff --git a/api/app/controllers/memory_perceptual_controller.py b/api/app/controllers/memory_perceptual_controller.py index 5154c763..44750808 100644 --- a/api/app/controllers/memory_perceptual_controller.py +++ b/api/app/controllers/memory_perceptual_controller.py @@ -27,27 +27,27 @@ router = APIRouter( ) -@router.get("/{group_id}/count", response_model=ApiResponse) +@router.get("/{end_user_id}/count", response_model=ApiResponse) def get_memory_count( - group_id: uuid.UUID, + end_user_id: uuid.UUID, current_user: User = Depends(get_current_user), db: Session = Depends(get_db) ): """Retrieve perceptual memory statistics for a user group. Args: - group_id: ID of the user group (usually end_user_id in this context) + end_user_id: ID of the user group (usually end_user_id in this context) current_user: Current authenticated user db: Database session Returns: ApiResponse: Response containing memory count statistics """ - api_logger.info(f"Fetching perceptual memory statistics: user={current_user.username}, group_id={group_id}") + api_logger.info(f"Fetching perceptual memory statistics: user={current_user.username}, end_user_id={end_user_id}") try: service = MemoryPerceptualService(db) - count_stats = service.get_memory_count(group_id) + count_stats = service.get_memory_count(end_user_id) api_logger.info(f"Memory statistics fetched successfully: total={count_stats.get('total', 0)}") @@ -57,37 +57,37 @@ def get_memory_count( ) except Exception as e: - api_logger.error(f"Failed to fetch memory statistics: group_id={group_id}, error={str(e)}") + api_logger.error(f"Failed to fetch memory statistics: end_user_id={end_user_id}, error={str(e)}") return fail( code=BizCode.INTERNAL_ERROR, msg="Failed to fetch memory statistics", ) -@router.get("/{group_id}/last_visual", response_model=ApiResponse) +@router.get("/{end_user_id}/last_visual", response_model=ApiResponse) def get_last_visual_memory( - group_id: uuid.UUID, + end_user_id: uuid.UUID, current_user: User = Depends(get_current_user), db: Session = Depends(get_db) ): """Retrieve the most recent VISION-type memory for a user. Args: - group_id: ID of the user group + end_user_id: ID of the user group current_user: Current authenticated user db: Database session Returns: ApiResponse: Metadata of the latest visual memory """ - api_logger.info(f"Fetching latest visual memory: user={current_user.username}, group_id={group_id}") + api_logger.info(f"Fetching latest visual memory: user={current_user.username}, end_user_id={end_user_id}") try: service = MemoryPerceptualService(db) - visual_memory = service.get_latest_visual_memory(group_id) + visual_memory = service.get_latest_visual_memory(end_user_id) if visual_memory is None: - api_logger.info(f"No visual memory found: group_id={group_id}") + api_logger.info(f"No visual memory found: end_user_id={end_user_id}") return success( data=None, msg="No visual memory available" @@ -101,37 +101,37 @@ def get_last_visual_memory( ) except Exception as e: - api_logger.error(f"Failed to fetch latest visual memory: group_id={group_id}, error={str(e)}") + api_logger.error(f"Failed to fetch latest visual memory: end_user_id={end_user_id}, error={str(e)}") return fail( code=BizCode.INTERNAL_ERROR, msg="Failed to fetch latest visual memory", ) -@router.get("/{group_id}/last_listen", response_model=ApiResponse) +@router.get("/{end_user_id}/last_listen", response_model=ApiResponse) def get_last_memory_listen( - group_id: uuid.UUID, + end_user_id: uuid.UUID, current_user: User = Depends(get_current_user), db: Session = Depends(get_db) ): """Retrieve the most recent AUDIO-type memory for a user. Args: - group_id: ID of the user group + end_user_id: ID of the user group current_user: Current authenticated user db: Database session Returns: ApiResponse: Metadata of the latest audio memory """ - api_logger.info(f"Fetching latest audio memory: user={current_user.username}, group_id={group_id}") + api_logger.info(f"Fetching latest audio memory: user={current_user.username}, end_user_id={end_user_id}") try: service = MemoryPerceptualService(db) - audio_memory = service.get_latest_audio_memory(group_id) + audio_memory = service.get_latest_audio_memory(end_user_id) if audio_memory is None: - api_logger.info(f"No audio memory found: group_id={group_id}") + api_logger.info(f"No audio memory found: end_user_id={end_user_id}") return success( data=None, msg="No audio memory available" @@ -145,38 +145,38 @@ def get_last_memory_listen( ) except Exception as e: - api_logger.error(f"Failed to fetch latest audio memory: group_id={group_id}, error={str(e)}") + api_logger.error(f"Failed to fetch latest audio memory: end_user_id={end_user_id}, error={str(e)}") return fail( code=BizCode.INTERNAL_ERROR, msg="Failed to fetch latest audio memory", ) -@router.get("/{group_id}/last_text", response_model=ApiResponse) +@router.get("/{end_user_id}/last_text", response_model=ApiResponse) def get_last_text_memory( - group_id: uuid.UUID, + end_user_id: uuid.UUID, current_user: User = Depends(get_current_user), db: Session = Depends(get_db) ): """Retrieve the most recent TEXT-type memory for a user. Args: - group_id: ID of the user group + end_user_id: ID of the user group current_user: Current authenticated user db: Database session Returns: ApiResponse: Metadata of the latest text memory """ - api_logger.info(f"Fetching latest text memory: user={current_user.username}, group_id={group_id}") + api_logger.info(f"Fetching latest text memory: user={current_user.username}, end_user_id={end_user_id}") try: # 调用服务层获取最近的文本记忆 service = MemoryPerceptualService(db) - text_memory = service.get_latest_text_memory(group_id) + text_memory = service.get_latest_text_memory(end_user_id) if text_memory is None: - api_logger.info(f"No text memory found: group_id={group_id}") + api_logger.info(f"No text memory found: end_user_id={end_user_id}") return success( data=None, msg="No text memory available" @@ -190,16 +190,16 @@ def get_last_text_memory( ) except Exception as e: - api_logger.error(f"Failed to fetch latest text memory: group_id={group_id}, error={str(e)}") + api_logger.error(f"Failed to fetch latest text memory: end_user_id={end_user_id}, error={str(e)}") return fail( code=BizCode.INTERNAL_ERROR, msg="Failed to fetch latest text memory", ) -@router.get("/{group_id}/timeline", response_model=ApiResponse) +@router.get("/{end_user_id}/timeline", response_model=ApiResponse) def get_memory_time_line( - group_id: uuid.UUID, + end_user_id: uuid.UUID, perceptual_type: Optional[PerceptualType] = Query(None, description="感知类型过滤"), page: int = Query(1, ge=1, description="页码"), page_size: int = Query(10, ge=1, le=100, description="每页大小"), @@ -209,7 +209,7 @@ def get_memory_time_line( """Retrieve a timeline of perceptual memories for a user group. Args: - group_id: ID of the user group + end_user_id: ID of the user group perceptual_type: Optional filter for perceptual type page: Page number for pagination page_size: Number of items per page @@ -221,7 +221,7 @@ def get_memory_time_line( """ api_logger.info( f"Fetching perceptual memory timeline: user={current_user.username}, " - f"group_id={group_id}, type={perceptual_type}, page={page}" + f"end_user_id={end_user_id}, type={perceptual_type}, page={page}" ) try: @@ -232,7 +232,7 @@ def get_memory_time_line( ) service = MemoryPerceptualService(db) - timeline_data = service.get_time_line(group_id, query) + timeline_data = service.get_time_line(end_user_id, query) api_logger.info( f"Perceptual memory timeline retrieved successfully: total={timeline_data.total}, " @@ -246,7 +246,7 @@ def get_memory_time_line( except Exception as e: api_logger.error( - f"Failed to fetch perceptual memory timeline: group_id={group_id}, " + f"Failed to fetch perceptual memory timeline: end_user_id={end_user_id}, " f"error={str(e)}" ) return fail( diff --git a/api/app/controllers/memory_reflection_controller.py b/api/app/controllers/memory_reflection_controller.py index abd50a33..7941be35 100644 --- a/api/app/controllers/memory_reflection_controller.py +++ b/api/app/controllers/memory_reflection_controller.py @@ -1,6 +1,7 @@ import asyncio import time import uuid +from uuid import UUID from app.core.logging_config import get_api_logger from app.core.memory.storage_services.reflection_engine.self_reflexion import ( @@ -11,7 +12,7 @@ from app.core.response_utils import success from app.db import get_db from app.dependencies import get_current_user from app.models.user_model import User -from app.repositories.data_config_repository import DataConfigRepository +from app.repositories.memory_config_repository import MemoryConfigRepository from app.repositories.neo4j.neo4j_connector import Neo4jConnector from app.schemas.memory_reflection_schemas import Memory_Reflection from app.services.memory_reflection_service import ( @@ -24,6 +25,8 @@ from fastapi import APIRouter, Depends, HTTPException, status,Header from sqlalchemy import text from sqlalchemy.orm import Session +from app.utils.config_utils import resolve_config_id + load_dotenv() api_logger = get_api_logger() @@ -42,6 +45,7 @@ async def save_reflection_config( """Save reflection configuration to data_comfig table""" try: config_id = request.config_id + config_id = resolve_config_id(config_id, db) if not config_id: raise HTTPException( status_code=status.HTTP_400_BAD_REQUEST, @@ -50,7 +54,7 @@ async def save_reflection_config( api_logger.info(f"用户 {current_user.username} 保存反思配置，config_id: {config_id}") - data_config = DataConfigRepository.update_reflection_config( + memory_config = MemoryConfigRepository.update_reflection_config( db, config_id=config_id, enable_self_reflexion=request.reflection_enabled, @@ -63,17 +67,17 @@ async def save_reflection_config( ) db.commit() - db.refresh(data_config) + db.refresh(memory_config) reflection_result={ - "config_id": data_config.config_id, - "enable_self_reflexion": data_config.enable_self_reflexion, - "iteration_period": data_config.iteration_period, - "reflexion_range": data_config.reflexion_range, - "baseline": data_config.baseline, - "reflection_model_id": data_config.reflection_model_id, - "memory_verify": data_config.memory_verify, - "quality_assessment": data_config.quality_assessment} + "config_id": memory_config.config_id, + "enable_self_reflexion": memory_config.enable_self_reflexion, + "iteration_period": memory_config.iteration_period, + "reflexion_range": memory_config.reflexion_range, + "baseline": memory_config.baseline, + "reflection_model_id": memory_config.reflection_model_id, + "memory_verify": memory_config.memory_verify, + "quality_assessment": memory_config.quality_assessment} return success(data=reflection_result, msg="反思配置成功") @@ -111,14 +115,14 @@ async def start_workspace_reflection( reflection_results = [] for data in result['apps_detailed_info']: - if data['data_configs'] == []: + if data['memory_configs'] == []: continue releases = data['releases'] - data_configs = data['data_configs'] + memory_configs = data['memory_configs'] end_users = data['end_users'] - for base, config, user in zip(releases, data_configs, end_users): + for base, config, user in zip(releases, memory_configs, end_users): # 安全地转换为整数，处理空字符串和None的情况 print(base['config']) try: @@ -156,17 +160,20 @@ async def start_workspace_reflection( @router.get("/reflection/configs") async def start_reflection_configs( - config_id: int, + config_id: uuid.UUID|int, current_user: User = Depends(get_current_user), db: Session = Depends(get_db), ) -> dict: - """通过config_id查询data_config表中的反思配置信息""" + """通过config_id查询memory_config表中的反思配置信息""" + config_id = resolve_config_id(config_id, db) try: + config_id=resolve_config_id(config_id,db) api_logger.info(f"用户 {current_user.username} 查询反思配置，config_id: {config_id}") - result = DataConfigRepository.query_reflection_config_by_id(db, config_id) + result = MemoryConfigRepository.query_reflection_config_by_id(db, config_id) + memory_config_id = resolve_config_id(result.config_id, db) # 构建返回数据 reflection_config = { - "config_id": result.config_id, + "config_id": memory_config_id, "reflection_enabled": result.enable_self_reflexion, "reflection_period_in_hours": result.iteration_period, "reflexion_range": result.reflexion_range, @@ -191,7 +198,7 @@ async def start_reflection_configs( @router.get("/reflection/run") async def reflection_run( - config_id: int, + config_id: UUID|int, language_type: str = Header(default="zh", alias="X-Language-Type"), current_user: User = Depends(get_current_user), db: Session = Depends(get_db), @@ -199,9 +206,9 @@ async def reflection_run( """Activate the reflection function for all matching applications in the workspace""" api_logger.info(f"用户 {current_user.username} 查询反思配置，config_id: {config_id}") - - # 使用DataConfigRepository查询反思配置 - result = DataConfigRepository.query_reflection_config_by_id(db, config_id) + config_id = resolve_config_id(config_id, db) + # 使用MemoryConfigRepository查询反思配置 + result = MemoryConfigRepository.query_reflection_config_by_id(db, config_id) if not result: raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, diff --git a/api/app/controllers/memory_storage_controller.py b/api/app/controllers/memory_storage_controller.py index f4175923..ae372d3b 100644 --- a/api/app/controllers/memory_storage_controller.py +++ b/api/app/controllers/memory_storage_controller.py @@ -1,5 +1,6 @@ import os from typing import Optional +from uuid import UUID from app.core.error_codes import BizCode from app.core.logging_config import get_api_logger @@ -34,6 +35,8 @@ from fastapi import APIRouter, Depends from fastapi.responses import StreamingResponse from sqlalchemy.orm import Session +from app.utils.config_utils import resolve_config_id + # Get API logger api_logger = get_api_logger() @@ -140,7 +143,6 @@ def create_config( db: Session = Depends(get_db), ) -> dict: workspace_id = current_user.current_workspace_id - # 检查用户是否已选择工作空间 if workspace_id is None: api_logger.warning(f"用户 {current_user.username} 尝试创建配置但未选择工作空间") @@ -160,12 +162,12 @@ def create_config( @router.delete("/delete_config", response_model=ApiResponse) # 删除数据库中的内容（按配置名称） def delete_config( - config_id: str, + config_id: UUID|int, current_user: User = Depends(get_current_user), db: Session = Depends(get_db), ) -> dict: workspace_id = current_user.current_workspace_id - + config_id=resolve_config_id(config_id, db) # 检查用户是否已选择工作空间 if workspace_id is None: api_logger.warning(f"用户 {current_user.username} 尝试删除配置但未选择工作空间") @@ -187,7 +189,7 @@ def update_config( db: Session = Depends(get_db), ) -> dict: workspace_id = current_user.current_workspace_id - + payload.config_id = resolve_config_id(payload.config_id, db) # 检查用户是否已选择工作空间 if workspace_id is None: api_logger.warning(f"用户 {current_user.username} 尝试更新配置但未选择工作空间") @@ -210,7 +212,7 @@ def update_config_extracted( db: Session = Depends(get_db), ) -> dict: workspace_id = current_user.current_workspace_id - + payload.config_id = resolve_config_id(payload.config_id, db) # 检查用户是否已选择工作空间 if workspace_id is None: api_logger.warning(f"用户 {current_user.username} 尝试更新提取配置但未选择工作空间") @@ -232,12 +234,12 @@ def update_config_extracted( @router.get("/read_config_extracted", response_model=ApiResponse) # 通过查询参数读取某条配置（固定路径） 没有意义的话就删除 def read_config_extracted( - config_id: str, + config_id: UUID | int, current_user: User = Depends(get_current_user), db: Session = Depends(get_db), ) -> dict: workspace_id = current_user.current_workspace_id - + config_id = resolve_config_id(config_id, db) # 检查用户是否已选择工作空间 if workspace_id is None: api_logger.warning(f"用户 {current_user.username} 尝试读取提取配置但未选择工作空间") @@ -285,6 +287,7 @@ async def pilot_run( f"Pilot run requested: config_id={payload.config_id}, " f"dialogue_text_length={len(payload.dialogue_text)}" ) + payload.config_id = resolve_config_id(payload.config_id, db) svc = DataConfigService(db) return StreamingResponse( svc.pilot_run_stream(payload), @@ -420,15 +423,95 @@ async def get_hot_memory_tags_api( db: Session = Depends(get_db), current_user: User = Depends(get_current_user), ) -> dict: - api_logger.info(f"Hot memory tags requested for current_user: {current_user.id}") + """ + 获取热门记忆标签（带Redis缓存） + + 缓存策略： + - 缓存键：workspace_id + limit + - 过期时间：5分钟（300秒） + - 缓存命中：~50ms + - 缓存未命中：~600-800ms（取决于LLM速度） + """ + workspace_id = current_user.current_workspace_id + + # 构建缓存键 + cache_key = f"hot_memory_tags:{workspace_id}:{limit}" + + api_logger.info(f"Hot memory tags requested for workspace: {workspace_id}, limit: {limit}") + try: + # 尝试从Redis缓存获取 + from app.aioRedis import aio_redis_get, aio_redis_set + import json + + cached_result = await aio_redis_get(cache_key) + if cached_result: + api_logger.info(f"Cache hit for key: {cache_key}") + try: + data = json.loads(cached_result) + return success(data=data, msg="查询成功（缓存）") + except json.JSONDecodeError: + api_logger.warning(f"Failed to parse cached data, will refresh") + + # 缓存未命中，执行查询 + api_logger.info(f"Cache miss for key: {cache_key}, executing query") result = await analytics_hot_memory_tags(db, current_user, limit) + + # 写入缓存（过期时间：5分钟） + # 注意：result是列表，需要转换为JSON字符串 + try: + cache_data = json.dumps(result, ensure_ascii=False) + await aio_redis_set(cache_key, cache_data, expire=300) + api_logger.info(f"Cached result for key: {cache_key}") + except Exception as cache_error: + # 缓存写入失败不影响主流程 + api_logger.warning(f"Failed to cache result: {str(cache_error)}") + return success(data=result, msg="查询成功") + except Exception as e: api_logger.error(f"Hot memory tags failed: {str(e)}") return fail(BizCode.INTERNAL_ERROR, "热门标签查询失败", str(e)) +@router.delete("/analytics/hot_memory_tags/cache", response_model=ApiResponse) +async def clear_hot_memory_tags_cache( + current_user: User = Depends(get_current_user), + ) -> dict: + """ + 清除热门标签缓存 + + 用于： + - 手动刷新数据 + - 调试和测试 + - 数据更新后立即生效 + """ + workspace_id = current_user.current_workspace_id + + api_logger.info(f"Clear hot memory tags cache requested for workspace: {workspace_id}") + + try: + from app.aioRedis import aio_redis_delete + + # 清除所有limit的缓存（常见的limit值） + cleared_count = 0 + for limit in [5, 10, 15, 20, 30, 50]: + cache_key = f"hot_memory_tags:{workspace_id}:{limit}" + result = await aio_redis_delete(cache_key) + if result: + cleared_count += 1 + api_logger.info(f"Cleared cache for key: {cache_key}") + + return success( + data={"cleared_count": cleared_count}, + msg=f"成功清除 {cleared_count} 个缓存" + ) + + except Exception as e: + api_logger.error(f"Clear cache failed: {str(e)}") + return fail(BizCode.INTERNAL_ERROR, "清除缓存失败", str(e)) + + @router.get("/analytics/recent_activity_stats", response_model=ApiResponse) async def get_recent_activity_stats_api( current_user: User = Depends(get_current_user), diff --git a/api/app/controllers/memory_working_controller.py b/api/app/controllers/memory_working_controller.py index dfd64044..e5de3c04 100644 --- a/api/app/controllers/memory_working_controller.py +++ b/api/app/controllers/memory_working_controller.py @@ -20,18 +20,18 @@ router = APIRouter( ) -@router.get("/{group_id}/count", response_model=ApiResponse) +@router.get("/{end_user_id}/count", response_model=ApiResponse) def get_memory_count( - group_id: uuid.UUID, + end_user_id: uuid.UUID, current_user: User = Depends(get_current_user), db: Session = Depends(get_db) ): pass -@router.get("/{group_id}/conversations", response_model=ApiResponse) +@router.get("/{end_user_id}/conversations", response_model=ApiResponse) def get_conversations( - group_id: uuid.UUID, + end_user_id: uuid.UUID, current_user: User = Depends(get_current_user), db: Session = Depends(get_db) ): @@ -39,7 +39,7 @@ def get_conversations( Retrieve all conversations for the current user in a specific group. Args: - group_id (UUID): The group identifier. + end_user_id (UUID): The group identifier. current_user (User, optional): The authenticated user. db (Session, optional): SQLAlchemy session. @@ -53,7 +53,7 @@ def get_conversations( """ conversation_service = ConversationService(db) conversations = conversation_service.get_user_conversations( - group_id + end_user_id ) return success(data=[ { @@ -63,7 +63,7 @@ def get_conversations( ], msg="get conversations success") -@router.get("/{group_id}/messages", response_model=ApiResponse) +@router.get("/{end_user_id}/messages", response_model=ApiResponse) def get_messages( conversation_id: uuid.UUID, current_user: User = Depends(get_current_user), @@ -100,7 +100,7 @@ def get_messages( return success(data=messages, msg="get conversation history success") -@router.get("/{group_id}/detail", response_model=ApiResponse) +@router.get("/{end_user_id}/detail", response_model=ApiResponse) async def get_conversation_detail( conversation_id: uuid.UUID, current_user: User = Depends(get_current_user), diff --git a/api/app/controllers/model_controller.py b/api/app/controllers/model_controller.py index 42d59664..83753744 100644 --- a/api/app/controllers/model_controller.py +++ b/api/app/controllers/model_controller.py @@ -3,15 +3,17 @@ from sqlalchemy.orm import Session from typing import Optional import uuid - +from app.core.error_codes import BizCode +from app.core.exceptions import BusinessException from app.db import get_db from app.dependencies import get_current_user -from app.models.models_model import ModelProvider, ModelType +from app.models.models_model import ModelProvider, ModelType, LoadBalanceStrategy from app.models.user_model import User +from app.repositories.model_repository import ModelConfigRepository from app.schemas import model_schema from app.core.response_utils import success from app.schemas.response_schema import ApiResponse, PageData -from app.services.model_service import ModelConfigService, ModelApiKeyService +from app.services.model_service import ModelConfigService, ModelApiKeyService, ModelBaseService from app.core.logging_config import get_api_logger # 获取API专用日志器 @@ -24,24 +26,83 @@ router = APIRouter( @router.get("/type", response_model=ApiResponse) def get_model_types(): - return success(msg="获取模型类型成功", data=list(ModelType)) @router.get("/provider", response_model=ApiResponse) def get_model_providers(): - return success(msg="获取模型提供商成功", data=list(ModelProvider)) + providers = [p for p in ModelProvider if p != ModelProvider.COMPOSITE] + return success(msg="获取模型提供商成功", data=providers) + +@router.get("/strategy", response_model=ApiResponse) +def get_model_strategies(): + return success(msg="获取模型策略成功", data=list(LoadBalanceStrategy)) @router.get("", response_model=ApiResponse) def get_model_list( - type: Optional[str] = Query(None, description="模型类型筛选（支持多个，如 ?type=LLM 或 ?type=LLM,EMBEDDING）"), - provider: Optional[model_schema.ModelProvider] = Query(None, description="提供商筛选(基于API Key)"), + type: Optional[list[str]] = Query(None, description="模型类型筛选（支持多个，如 ?type=LLM 或 ?type=LLM,EMBEDDING）"), + provider: Optional[model_schema.ModelProvider] = Query(None, description="提供商筛选(基于API Key)"), + is_active: Optional[bool] = Query(None, description="激活状态筛选"), + is_public: Optional[bool] = Query(None, description="公开状态筛选"), + search: Optional[str] = Query(None, description="搜索关键词"), + page: int = Query(1, ge=1, description="页码"), + pagesize: int = Query(10, ge=1, le=100, description="每页数量"), + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """ + 获取模型配置列表 + + 支持多个 type 参数： + - 单个：?type=LLM + - 多个（逗号分隔）：?type=LLM,EMBEDDING + - 多个（重复参数）：?type=LLM&type=EMBEDDING + """ + api_logger.info( + f"获取模型配置列表请求: type={type}, provider={provider}, page={page}, pagesize={pagesize}, tenant_id={current_user.tenant_id}") + + try: + # 解析 type 参数（支持逗号分隔） + type_list = [] + if type is not None: + flat_type = [] + for item in type: + split_items = [t.strip() for t in item.split(',') if t.strip()] + flat_type.extend(split_items) + + unique_flat_type = list(dict.fromkeys(flat_type)) + type_list = [ModelType(t.lower()) for t in unique_flat_type] + + api_logger.error(f"获取模型type_list: {type_list}") + query = model_schema.ModelConfigQuery( + type=type_list, + provider=provider, + is_active=is_active, + is_public=is_public, + search=search, + page=page, + pagesize=pagesize + ) + + api_logger.debug(f"开始获取模型配置列表: {query.dict()}") + result_orm = ModelConfigService.get_model_list(db=db, query=query, tenant_id=current_user.tenant_id) + result = PageData.model_validate(result_orm) + api_logger.info(f"模型配置列表获取成功: 总数={result.page.total}, 当前页={len(result.items)}") + return success(data=result, msg="模型配置列表获取成功") + except Exception as e: + api_logger.error(f"获取模型配置列表失败: {str(e)}") + raise + + +@router.get("/new", response_model=ApiResponse) +def get_model_list_new( + type: Optional[list[str]] = Query(None, description="模型类型筛选（支持多个，如 ?type=LLM 或 ?type=LLM,EMBEDDING）"), + provider: Optional[model_schema.ModelProvider] = Query(None, description="提供商筛选(基于ModelConfig)"), is_active: Optional[bool] = Query(None, description="激活状态筛选"), is_public: Optional[bool] = Query(None, description="公开状态筛选"), search: Optional[str] = Query(None, description="搜索关键词"), - page: int = Query(1, ge=1, description="页码"), - pagesize: int = Query(10, ge=1, le=100, description="每页数量"), + is_composite: Optional[bool] = Query(None, description="组合模型筛选"), db: Session = Depends(get_db), current_user: User = Depends(get_current_user) ): @@ -53,36 +114,127 @@ def get_model_list( - 多个（逗号分隔）：?type=LLM,EMBEDDING - 多个（重复参数）：?type=LLM&type=EMBEDDING """ - api_logger.info(f"获取模型配置列表请求: type={type}, provider={provider}, page={page}, pagesize={pagesize}, tenant_id={current_user.tenant_id}") + api_logger.info(f"获取模型配置列表请求: type={type}, provider={provider}, tenant_id={current_user.tenant_id}") try: # 解析 type 参数（支持逗号分隔） - type_list = None - if type: - type_values = [t.strip() for t in type.split(',')] - type_list = [model_schema.ModelType(t.lower()) for t in type_values if t] + type_list = [] + if type is not None: + flat_type = [] + for item in type: + split_items = [t.strip() for t in item.split(',') if t.strip()] + flat_type.extend(split_items) + + unique_flat_type = list(dict.fromkeys(flat_type)) + type_list = [ModelType(t.lower()) for t in unique_flat_type] - api_logger.error(f"获取模型type_list: {type_list}") - query = model_schema.ModelConfigQuery( + api_logger.info(f"获取模型type_list: {type_list}") + query = model_schema.ModelConfigQueryNew( type=type_list, provider=provider, is_active=is_active, is_public=is_public, - search=search, - page=page, - pagesize=pagesize + is_composite=is_composite, + search=search ) - api_logger.debug(f"开始获取模型配置列表: {query.dict()}") - result_orm = ModelConfigService.get_model_list(db=db, query=query, tenant_id=current_user.tenant_id) - result = PageData.model_validate(result_orm) - api_logger.info(f"模型配置列表获取成功: 总数={result.page.total}, 当前页={len(result.items)}") + api_logger.debug(f"开始获取模型配置列表: {query.model_dump()}") + result = ModelConfigService.get_model_list_new(db=db, query=query, tenant_id=current_user.tenant_id) + api_logger.info(f"模型配置列表获取成功: 分组数={len(result)}, 总模型数={sum(len(item['models']) for item in result)}") return success(data=result, msg="模型配置列表获取成功") except Exception as e: api_logger.error(f"获取模型配置列表失败: {str(e)}") raise +@router.get("/model_plaza", response_model=ApiResponse) +def get_model_plaza_list( + type: Optional[ModelType] = Query(None, description="模型类型"), + provider: Optional[ModelProvider] = Query(None, description="供应商"), + is_official: Optional[bool] = Query(None, description="是否官方模型"), + is_deprecated: Optional[bool] = Query(None, description="是否弃用"), + search: Optional[str] = Query(None, description="搜索关键词"), + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """模型广场查询接口（按供应商分组）""" + + query = model_schema.ModelBaseQuery( + type=type, + provider=provider, + is_official=is_official, + is_deprecated=is_deprecated, + search=search + ) + result = ModelBaseService.get_model_base_list(db=db, query=query, tenant_id=current_user.tenant_id) + return success(data=result, msg="模型广场列表获取成功") + + +@router.get("/model_plaza/{model_base_id}", response_model=ApiResponse) +def get_model_base_by_id( + model_base_id: uuid.UUID, + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """获取基础模型详情""" + + result = ModelBaseService.get_model_base_by_id(db=db, model_base_id=model_base_id) + return success(data=model_schema.ModelBase.model_validate(result), msg="基础模型获取成功") + + +@router.post("/model_plaza", response_model=ApiResponse) +def create_model_base( + data: model_schema.ModelBaseCreate, + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """创建基础模型""" + + result = ModelBaseService.create_model_base(db=db, data=data) + return success(data=model_schema.ModelBase.model_validate(result), msg="基础模型创建成功") + + +@router.put("/model_plaza/{model_base_id}", response_model=ApiResponse) +def update_model_base( + model_base_id: uuid.UUID, + data: model_schema.ModelBaseUpdate, + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """更新基础模型""" + + # 不允许更改type类型 + if data.type is not None or data.provider is not None: + raise BusinessException("不允许更改模型类型和供应商", BizCode.INVALID_PARAMETER) + + result = ModelBaseService.update_model_base(db=db, model_base_id=model_base_id, data=data) + return success(data=model_schema.ModelBase.model_validate(result), msg="基础模型更新成功") + + +@router.delete("/model_plaza/{model_base_id}", response_model=ApiResponse) +def delete_model_base( + model_base_id: uuid.UUID, + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """删除基础模型""" + + ModelBaseService.delete_model_base(db=db, model_base_id=model_base_id) + return success(msg="基础模型删除成功") + + +@router.post("/model_plaza/{model_base_id}/add", response_model=ApiResponse) +def add_model_from_plaza( + model_base_id: uuid.UUID, + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """从模型广场添加模型到模型列表""" + + result = ModelBaseService.add_model_from_plaza(db=db, model_base_id=model_base_id, tenant_id=current_user.tenant_id) + return success(data=model_schema.ModelConfig.model_validate(result), msg="模型添加成功") + + @router.get("/{model_id}", response_model=ApiResponse) def get_model_by_id( model_id: uuid.UUID, @@ -138,6 +290,73 @@ async def create_model( raise +@router.post("/composite", response_model=ApiResponse) +async def create_composite_model( + model_data: model_schema.CompositeModelCreate, + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """ + 创建组合模型 + + - 绑定一个或多个现有的 API Key + - 所有 API Key 必须来自非组合模型 + - 所有 API Key 关联的模型类型必须与组合模型类型一致 + """ + api_logger.info(f"创建组合模型请求: {model_data.name}, 用户: {current_user.username}, tenant_id={current_user.tenant_id}") + + try: + result_orm = await ModelConfigService.create_composite_model(db=db, model_data=model_data, tenant_id=current_user.tenant_id) + api_logger.info(f"组合模型创建成功: {result_orm.name} (ID: {result_orm.id})") + + result = model_schema.ModelConfig.model_validate(result_orm) + return success(data=result, msg="组合模型创建成功") + except Exception as e: + api_logger.error(f"创建组合模型失败: {model_data.name} - {str(e)}") + raise + + +@router.put("/composite/{model_id}", response_model=ApiResponse) +async def update_composite_model( + model_id: uuid.UUID, + model_data: model_schema.CompositeModelCreate, + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """更新组合模型""" + api_logger.info(f"更新组合模型请求: model_id={model_id}, 用户: {current_user.username}") + + try: + if model_data.type is not None: + raise BusinessException("不允许更改模型类型和供应商", BizCode.INVALID_PARAMETER) + result_orm = await ModelConfigService.update_composite_model(db=db, model_id=model_id, model_data=model_data, tenant_id=current_user.tenant_id) + api_logger.info(f"组合模型更新成功: {result_orm.name} (ID: {model_id})") + + result = model_schema.ModelConfig.model_validate(result_orm) + return success(data=result, msg="组合模型更新成功") + except Exception as e: + api_logger.error(f"更新组合模型失败: model_id={model_id} - {str(e)}") + raise + + +@router.delete("/composite/{model_id}", response_model=ApiResponse) +def delete_composite_model( + model_id: uuid.UUID, + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """删除组合模型""" + api_logger.info(f"删除组合模型请求: model_id={model_id}, 用户: {current_user.username}") + + try: + ModelConfigService.delete_model(db=db, model_id=model_id, tenant_id=current_user.tenant_id) + api_logger.info(f"组合模型删除成功: model_id={model_id}") + return success(msg="组合模型删除成功") + except Exception as e: + api_logger.error(f"删除组合模型失败: model_id={model_id} - {str(e)}") + raise + + @router.put("/{model_id}", response_model=ApiResponse) def update_model( model_id: uuid.UUID, @@ -214,6 +433,53 @@ def get_model_api_keys( raise +@router.post("/provider/apikeys", response_model=ApiResponse) +async def create_model_api_key_by_provider( + api_key_data: model_schema.ModelApiKeyCreateByProvider, + db: Session = Depends(get_db), + current_user: User = Depends(get_current_user) +): + """ + 根据供应商为所有匹配的模型创建API Key + """ + api_logger.info(f"创建API Key请求: provider={api_key_data.provider}, 用户: {current_user.username}") + + try: + # 根据tenant_id和provider筛选model_config_id列表 + model_config_ids = api_key_data.model_config_ids + if not model_config_ids: + model_config_ids = ModelConfigRepository.get_model_config_ids_by_provider( + db=db, + tenant_id=current_user.tenant_id, + provider=api_key_data.provider + ) + + if not model_config_ids: + raise BusinessException(f"未找到供应商 {api_key_data.provider} 的模型配置", BizCode.MODEL_NOT_FOUND) + + # 构造schema并调用service + create_data = model_schema.ModelApiKeyCreateByProvider( + provider=api_key_data.provider, + api_key=api_key_data.api_key, + api_base=api_key_data.api_base, + description=api_key_data.description, + config=api_key_data.config, + is_active=api_key_data.is_active, + priority=api_key_data.priority, + model_config_ids=model_config_ids + ) + created_keys, failed_models = await ModelApiKeyService.create_api_key_by_provider(db=db, data=create_data) + + api_logger.info(f"API Key创建成功: 关联{len(created_keys)}个模型") + # result_list = [model_schema.ModelApiKey.model_validate(key) for key in created_keys] + result = "API Key已存在" if len(created_keys) == 0 and len(failed_models) == 0 else \ + f"成功为 {len(created_keys)} 个模型创建API Key, 失败模型列表{failed_models}" + return success(data=result, msg=f"成功为 {len(created_keys)} 个模型创建API Key") + except Exception as e: + api_logger.error(f"创建API Key失败: {str(e)}") + raise + + @router.post("/{model_id}/apikeys", response_model=ApiResponse, status_code=status.HTTP_201_CREATED) async def create_model_api_key( model_id: uuid.UUID, @@ -228,11 +494,12 @@ async def create_model_api_key( try: # 设置模型配置ID - api_key_data.model_config_id = model_id + api_key_data.model_config_ids = [model_id] api_logger.debug(f"开始创建模型API Key: {api_key_data.model_name}") - result = await ModelApiKeyService.create_api_key(db=db, api_key_data=api_key_data) - api_logger.info(f"模型API Key创建成功: {result.model_name} (ID: {result.id})") + result_orm = await ModelApiKeyService.create_api_key(db=db, api_key_data=api_key_data) + api_logger.info(f"模型API Key创建成功: {result_orm.model_name} (ID: {result_orm.id})") + result = model_schema.ModelApiKey.model_validate(result_orm) return success(data=result, msg="模型API Key创建成功") except Exception as e: api_logger.error(f"创建模型API Key失败: {api_key_data.model_name} - {str(e)}") @@ -334,5 +601,3 @@ async def validate_model_config( return success(data=model_schema.ModelValidateResponse(**result), msg="验证完成") - - diff --git a/api/app/controllers/public_share_controller.py b/api/app/controllers/public_share_controller.py index 17ad70a7..6e2d383c 100644 --- a/api/app/controllers/public_share_controller.py +++ b/api/app/controllers/public_share_controller.py @@ -317,9 +317,12 @@ async def chat( appid = share.app_id """获取存储类型和工作空间的ID""" - # 直接通过 SQLAlchemy 查询 app + # 直接通过 SQLAlchemy 查询 app（仅查询未删除的应用） from app.models.app_model import App - app = db.query(App).filter(App.id == appid).first() + app = db.query(App).filter( + App.id == appid, + App.is_active.is_(True) + ).first() if not app: raise BusinessException("应用不存在", BizCode.APP_NOT_FOUND) diff --git a/api/app/controllers/service/app_api_controller.py b/api/app/controllers/service/app_api_controller.py index 677e1623..31e799d2 100644 --- a/api/app/controllers/service/app_api_controller.py +++ b/api/app/controllers/service/app_api_controller.py @@ -235,11 +235,11 @@ async def chat( message=payload.message, conversation_id=conversation.id, # 使用已创建的会话 ID - user_id=new_end_user.id, # 转换为字符串 + user_id=end_user_id, # 转换为字符串 variables=payload.variables, config=config, - web_search=payload.web_search, - memory=payload.memory, + web_search=web_search, + memory=memory, storage_type=storage_type, user_rag_memory_id=user_rag_memory_id, app_id=app.id, @@ -268,11 +268,11 @@ async def chat( message=payload.message, conversation_id=conversation.id, # 使用已创建的会话 ID - user_id=new_end_user.id, # 转换为字符串 + user_id=end_user_id, # 转换为字符串 variables=payload.variables, config=config, - web_search=payload.web_search, - memory=payload.memory, + web_search=web_search, + memory=memory, storage_type=storage_type, user_rag_memory_id=user_rag_memory_id, app_id=app.id, diff --git a/api/app/controllers/service/memory_api_controller.py b/api/app/controllers/service/memory_api_controller.py index 30ca1306..accd749e 100644 --- a/api/app/controllers/service/memory_api_controller.py +++ b/api/app/controllers/service/memory_api_controller.py @@ -39,7 +39,7 @@ async def write_memory_api_service( Stores memory content for the specified end user using the Memory API Service. """ - logger.info(f"Memory write request - end_user_id: {payload.end_user_id}") + logger.info(f"Memory write request - end_user_id: {payload.end_user_id}, tenant_id: {api_key_auth.tenant_id}") memory_api_service = MemoryAPIService(db) diff --git a/api/app/controllers/user_memory_controllers.py b/api/app/controllers/user_memory_controllers.py index 6f02f8f9..39cbe523 100644 --- a/api/app/controllers/user_memory_controllers.py +++ b/api/app/controllers/user_memory_controllers.py @@ -135,27 +135,27 @@ async def generate_cache_api( api_logger.warning(f"用户 {current_user.username} 尝试生成缓存但未选择工作空间") return fail(BizCode.INVALID_PARAMETER, "请先切换到一个工作空间", "current_workspace_id is None") - group_id = request.end_user_id + end_user_id = request.end_user_id api_logger.info( f"缓存生成请求: user={current_user.username}, workspace={workspace_id}, " - f"end_user_id={group_id if group_id else '全部用户'}" + f"end_user_id={end_user_id if end_user_id else '全部用户'}" ) try: - if group_id: + if end_user_id: # 为单个用户生成 - api_logger.info(f"开始为单个用户生成缓存: end_user_id={group_id}") + api_logger.info(f"开始为单个用户生成缓存: end_user_id={end_user_id}") # 生成记忆洞察 - insight_result = await user_memory_service.generate_and_cache_insight(db, group_id, workspace_id) + insight_result = await user_memory_service.generate_and_cache_insight(db, end_user_id, workspace_id) # 生成用户摘要 - summary_result = await user_memory_service.generate_and_cache_summary(db, group_id, workspace_id) + summary_result = await user_memory_service.generate_and_cache_summary(db, end_user_id, workspace_id) # 构建响应 result = { - "end_user_id": group_id, + "end_user_id": end_user_id, "insight_success": insight_result["success"], "summary_success": summary_result["success"], "errors": [] @@ -175,9 +175,9 @@ async def generate_cache_api( # 记录结果 if result["insight_success"] and result["summary_success"]: - api_logger.info(f"成功为用户 {group_id} 生成缓存") + api_logger.info(f"成功为用户 {end_user_id} 生成缓存") else: - api_logger.warning(f"用户 {group_id} 的缓存生成部分失败: {result['errors']}") + api_logger.warning(f"用户 {end_user_id} 的缓存生成部分失败: {result['errors']}") return success(data=result, msg="生成完成") diff --git a/api/app/controllers/workflow_controller.py b/api/app/controllers/workflow_controller.py index c6d9ddab..8a15f717 100644 --- a/api/app/controllers/workflow_controller.py +++ b/api/app/controllers/workflow_controller.py @@ -54,7 +54,7 @@ async def create_workflow_config( app = db.query(App).filter( App.id == app_id, App.workspace_id == current_user.current_workspace_id, - App.is_active == True + App.is_active.is_(True) ).first() if not app: @@ -214,7 +214,7 @@ async def delete_workflow_config( app = db.query(App).filter( App.id == app_id, App.workspace_id == current_user.current_workspace_id, - App.is_active == True + App.is_active.is_(True) ).first() if not app: @@ -259,7 +259,7 @@ async def validate_workflow_config( app = db.query(App).filter( App.id == app_id, App.workspace_id == current_user.current_workspace_id, - App.is_active == True + App.is_active.is_(True) ).first() if not app: @@ -329,7 +329,7 @@ async def get_workflow_executions( app = db.query(App).filter( App.id == app_id, App.workspace_id == current_user.current_workspace_id, - App.is_active == True + App.is_active.is_(True) ).first() if not app: @@ -389,7 +389,7 @@ async def get_workflow_execution( app = db.query(App).filter( App.id == execution.app_id, App.workspace_id == current_user.current_workspace_id, - App.is_active == True + App.is_active.is_(True) ).first() if not app: @@ -440,7 +440,7 @@ async def run_workflow( app = db.query(App).filter( App.id == app_id, App.workspace_id == current_user.current_workspace_id, - App.is_active == True + App.is_active.is_(True) ).first() if not app: @@ -578,7 +578,7 @@ async def cancel_workflow_execution( app = db.query(App).filter( App.id == execution.app_id, App.workspace_id == current_user.current_workspace_id, - App.is_active == True + App.is_active.is_(True) ).first() if not app: diff --git a/api/app/core/agent/langchain_agent.py b/api/app/core/agent/langchain_agent.py index 87b46e6f..a34c781f 100644 --- a/api/app/core/agent/langchain_agent.py +++ b/api/app/core/agent/langchain_agent.py @@ -28,6 +28,8 @@ from langchain.agents import create_agent from langchain_core.messages import AIMessage, BaseMessage, HumanMessage, SystemMessage from langchain_core.tools import BaseTool +from app.utils.config_utils import resolve_config_id + logger = get_business_logger() @@ -155,13 +157,13 @@ class LangChainAgent: # userid=end_user_end, # messages=messages, # apply_id=end_user_end, - # group_id=end_user_end, + # end_user_id=end_user_end, # aimessages=aimessages # ) # store.delete_duplicate_sessions() # # logger.info(f'Redis_Agent:{end_user_end};{session_id}') # return session_id - + # TODO 乐力齐 - 累积多组对话批量写入功能已禁用 # async def term_memory_redis_read(self,end_user_end): # end_user_end = f"Term_{end_user_end}" @@ -175,11 +177,10 @@ class LangChainAgent: # messagss_list.append(f'用户:{query}。AI回复:{aimessages}') # retrieved_content.append({query: aimessages}) # return messagss_list,retrieved_content - async def write(self, storage_type, end_user_id, user_message, ai_message, user_rag_memory_id, actual_end_user_id, actual_config_id): """ 写入记忆（支持结构化消息） - + Args: storage_type: 存储类型 (neo4j/rag) end_user_id: 终端用户ID @@ -188,7 +189,7 @@ class LangChainAgent: user_rag_memory_id: RAG 记忆ID actual_end_user_id: 实际用户ID actual_config_id: 配置ID - + 逻辑说明： - RAG 模式：组合 user_message 和 ai_message 为字符串格式，保持原有逻辑不变 - Neo4j 模式：使用结构化消息列表 @@ -196,48 +197,54 @@ class LangChainAgent: 2. 如果只有 user_message：创建单条用户消息 [user]（用于历史记忆场景） 3. 每条消息会被转换为独立的 Chunk，保留 speaker 字段 """ - if storage_type == "rag": - # RAG 模式：组合消息为字符串格式（保持原有逻辑） - combined_message = f"user: {user_message}\nassistant: {ai_message}" - await write_rag(end_user_id, combined_message, user_rag_memory_id) - logger.info(f'RAG_Agent:{end_user_id};{user_rag_memory_id}') - else: - # Neo4j 模式：使用结构化消息列表 - structured_messages = [] - - # 始终添加用户消息（如果不为空） - if user_message: - structured_messages.append({"role": "user", "content": user_message}) - - # 只有当 AI 回复不为空时才添加 assistant 消息 - if ai_message: - structured_messages.append({"role": "assistant", "content": ai_message}) - - # 如果没有消息，直接返回 - if not structured_messages: - logger.warning(f"No messages to write for user {actual_end_user_id}") - return - - # 调用 Celery 任务，传递结构化消息列表 - # 数据流： - # 1. structured_messages 传递给 write_message_task - # 2. write_message_task 调用 memory_agent_service.write_memory - # 3. write_memory 调用 write_tools.write，传递 messages 参数 - # 4. write_tools.write 调用 get_chunked_dialogs，传递 messages 参数 - # 5. get_chunked_dialogs 为每条消息创建独立的 Chunk，设置 speaker 字段 - # 6. 每个 Chunk 保存到 Neo4j，包含 speaker 字段 - logger.info(f"[WRITE] Submitting Celery task - user={actual_end_user_id}, messages={len(structured_messages)}, config={actual_config_id}") - write_id = write_message_task.delay( - actual_end_user_id, # group_id: 用户ID - structured_messages, # message: 结构化消息列表 [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}] - actual_config_id, # config_id: 配置ID - storage_type, # storage_type: "neo4j" - user_rag_memory_id # user_rag_memory_id: RAG记忆ID（Neo4j模式下不使用） - ) - logger.info(f"[WRITE] Celery task submitted - task_id={write_id}") - write_status = get_task_memory_write_result(str(write_id)) - logger.info(f'[WRITE] Task result - user={actual_end_user_id}, status={write_status}') + db = next(get_db()) + try: + actual_config_id=resolve_config_id(actual_config_id, db) + + if storage_type == "rag": + # RAG 模式：组合消息为字符串格式（保持原有逻辑） + combined_message = f"user: {user_message}\nassistant: {ai_message}" + await write_rag(end_user_id, combined_message, user_rag_memory_id) + logger.info(f'RAG_Agent:{end_user_id};{user_rag_memory_id}') + else: + # Neo4j 模式：使用结构化消息列表 + structured_messages = [] + + # 始终添加用户消息（如果不为空） + if user_message: + structured_messages.append({"role": "user", "content": user_message}) + + # 只有当 AI 回复不为空时才添加 assistant 消息 + if ai_message: + structured_messages.append({"role": "assistant", "content": ai_message}) + + # 如果没有消息，直接返回 + if not structured_messages: + logger.warning(f"No messages to write for user {actual_end_user_id}") + return + + # 调用 Celery 任务，传递结构化消息列表 + # 数据流： + # 1. structured_messages 传递给 write_message_task + # 2. write_message_task 调用 memory_agent_service.write_memory + # 3. write_memory 调用 write_tools.write，传递 messages 参数 + # 4. write_tools.write 调用 get_chunked_dialogs，传递 messages 参数 + # 5. get_chunked_dialogs 为每条消息创建独立的 Chunk，设置 speaker 字段 + # 6. 每个 Chunk 保存到 Neo4j，包含 speaker 字段 + logger.info(f"[WRITE] Submitting Celery task - user={actual_end_user_id}, messages={len(structured_messages)}, config={actual_config_id}") + write_id = write_message_task.delay( + actual_end_user_id, # end_user_id: 用户ID + structured_messages, # message: 结构化消息列表 [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}] + actual_config_id, # config_id: 配置ID + storage_type, # storage_type: "neo4j" + user_rag_memory_id # user_rag_memory_id: RAG记忆ID（Neo4j模式下不使用） + ) + logger.info(f"[WRITE] Celery task submitted - task_id={write_id}") + write_status = get_task_memory_write_result(str(write_id)) + logger.info(f'[WRITE] Task result - user={actual_end_user_id}, status={write_status}') + finally: + db.close() async def chat( self, message: str, diff --git a/api/app/core/config.py b/api/app/core/config.py index 59c6ff5f..a8981054 100644 --- a/api/app/core/config.py +++ b/api/app/core/config.py @@ -9,6 +9,25 @@ load_dotenv() class Settings: + # ======================================================================== + # Deployment Mode Configuration + # ======================================================================== + # community: 社区版（开源，功能受限） + # cloud: SaaS 云服务版（全功能，按量计费） + # enterprise: 企业私有化版（License 控制） + DEPLOYMENT_MODE: str = os.getenv("DEPLOYMENT_MODE", "community") + + # License 配置（企业版） + LICENSE_FILE: str = os.getenv("LICENSE_FILE", "/etc/app/license.json") + LICENSE_SERVER_URL: str = os.getenv("LICENSE_SERVER_URL", "https://license.yourcompany.com") + + # 计费服务配置（SaaS 版） + BILLING_SERVICE_URL: str = os.getenv("BILLING_SERVICE_URL", "") + + # 基础 URL（用于 SSO 回调等） + BASE_URL: str = os.getenv("BASE_URL", "http://localhost:8000") + FRONTEND_URL: str = os.getenv("FRONTEND_URL", "http://localhost:3000") + ENABLE_SINGLE_WORKSPACE: bool = os.getenv("ENABLE_SINGLE_WORKSPACE", "true").lower() == "true" # API Keys Configuration OPENAI_API_KEY: str = os.getenv("OPENAI_API_KEY", "") @@ -72,6 +91,10 @@ class Settings: # Single Sign-On configuration ENABLE_SINGLE_SESSION: bool = os.getenv("ENABLE_SINGLE_SESSION", "false").lower() == "true" + + # SSO 免登配置 + SSO_TOKEN_EXPIRE_SECONDS: int = int(os.getenv("SSO_TOKEN_EXPIRE_SECONDS", "300")) + SSO_TRUSTED_SOURCES_CONFIG: str = os.getenv("SSO_TRUSTED_SOURCES_CONFIG", "{}") # File Upload MAX_FILE_SIZE: int = int(os.getenv("MAX_FILE_SIZE", "52428800")) @@ -107,6 +130,7 @@ class Settings: # Server Configuration SERVER_IP: str = os.getenv("SERVER_IP", "127.0.0.1") + FILE_LOCAL_SERVER_URL : str = os.getenv("FILE_LOCAL_SERVER_URL", "http://localhost:8000/api") # ======================================================================== # Internal Configuration (not in .env, used by application code) @@ -184,7 +208,7 @@ class Settings: ENABLE_TOOL_MANAGEMENT: bool = os.getenv("ENABLE_TOOL_MANAGEMENT", "true").lower() == "true" # official environment system version - SYSTEM_VERSION: str = os.getenv("SYSTEM_VERSION", "v0.2.0") + SYSTEM_VERSION: str = os.getenv("SYSTEM_VERSION", "v0.2.1") # workflow config WORKFLOW_NODE_TIMEOUT: int = int(os.getenv("WORKFLOW_NODE_TIMEOUT", 600)) diff --git a/api/app/core/memory/agent/langgraph_graph/nodes/problem_nodes.py b/api/app/core/memory/agent/langgraph_graph/nodes/problem_nodes.py index 697a13bd..ac1fb9a6 100644 --- a/api/app/core/memory/agent/langgraph_graph/nodes/problem_nodes.py +++ b/api/app/core/memory/agent/langgraph_graph/nodes/problem_nodes.py @@ -14,7 +14,7 @@ from app.core.memory.agent.utils.session_tools import SessionService from app.core.memory.agent.utils.template_tools import TemplateService from app.core.memory.agent.services.optimized_llm_service import LLMServiceMixin -template_root = os.path.join(PROJECT_ROOT_, 'agent', 'utils', 'prompt') +template_root = os.path.join(PROJECT_ROOT_, 'memory', 'agent', 'utils', 'prompt') db_session = next(get_db()) logger = get_agent_logger(__name__) @@ -35,10 +35,10 @@ async def Split_The_Problem(state: ReadState) -> ReadState: """问题分解节点""" # 从状态中获取数据 content = state.get('data', '') - group_id = state.get('group_id', '') + end_user_id = state.get('end_user_id', '') memory_config = state.get('memory_config', None) - history = await SessionService(store).get_history(group_id, group_id, group_id) + history = await SessionService(store).get_history(end_user_id, end_user_id, end_user_id) # 生成 JSON schema 以指导 LLM 输出正确格式 json_schema = ProblemExtensionResponse.model_json_schema() @@ -140,7 +140,7 @@ async def Problem_Extension(state: ReadState) -> ReadState: start = time.time() content = state.get('data', '') data = state.get('spit_data', '')['context'] - group_id = state.get('group_id', '') + end_user_id = state.get('end_user_id', '') storage_type = state.get('storage_type', '') user_rag_memory_id = state.get('user_rag_memory_id', '') memory_config = state.get('memory_config', None) @@ -156,7 +156,7 @@ async def Problem_Extension(state: ReadState) -> ReadState: databasets = {} data = [] - history = await SessionService(store).get_history(group_id, group_id, group_id) + history = await SessionService(store).get_history(end_user_id, end_user_id, end_user_id) # 生成 JSON schema 以指导 LLM 输出正确格式 json_schema = ProblemExtensionResponse.model_json_schema() diff --git a/api/app/core/memory/agent/langgraph_graph/nodes/retrieve_nodes.py b/api/app/core/memory/agent/langgraph_graph/nodes/retrieve_nodes.py index 14f8fa8b..1880357c 100644 --- a/api/app/core/memory/agent/langgraph_graph/nodes/retrieve_nodes.py +++ b/api/app/core/memory/agent/langgraph_graph/nodes/retrieve_nodes.py @@ -52,9 +52,9 @@ async def rag_config(state): return kb_config async def rag_knowledge(state,question): kb_config = await rag_config(state) - group_id = state.get('group_id', '') + end_user_id = state.get('end_user_id', '') user_rag_memory_id=state.get("user_rag_memory_id",'') - retrieve_chunks_result = knowledge_retrieval(question, kb_config, [str(group_id)]) + retrieve_chunks_result = knowledge_retrieval(question, kb_config, [str(end_user_id)]) try: retrieval_knowledge = [i.page_content for i in retrieve_chunks_result] clean_content = '\n\n'.join(retrieval_knowledge) @@ -159,7 +159,7 @@ async def retrieve_nodes(state: ReadState) -> ReadState: problem_extension=state.get('problem_extension', '')['context'] storage_type=state.get('storage_type', '') user_rag_memory_id=state.get('user_rag_memory_id', '') - group_id=state.get('group_id', '') + end_user_id=state.get('end_user_id', '') memory_config = state.get('memory_config', None) original=state.get('data', '') problem_list=[] @@ -172,7 +172,7 @@ async def retrieve_nodes(state: ReadState) -> ReadState: try: # Prepare search parameters based on storage type search_params = { - "group_id": group_id, + "end_user_id": end_user_id, "question": question, "return_raw_results": True } @@ -263,13 +263,13 @@ async def retrieve_nodes(state: ReadState) -> ReadState: async def retrieve(state: ReadState) -> ReadState: - # 从state中获取group_id + # 从state中获取end_user_id import time start=time.time() problem_extension = state.get('problem_extension', '')['context'] storage_type = state.get('storage_type', '') user_rag_memory_id = state.get('user_rag_memory_id', '') - group_id = state.get('group_id', '') + end_user_id = state.get('end_user_id', '') memory_config = state.get('memory_config', None) original = state.get('data', '') problem_list = [] @@ -295,13 +295,13 @@ async def retrieve(state: ReadState) -> ReadState: temperature=0.2, ) - time_retrieval_tool = create_time_retrieval_tool(group_id) - search_params = { "group_id": group_id, "return_raw_results": True } + time_retrieval_tool = create_time_retrieval_tool(end_user_id) + search_params = { "end_user_id": end_user_id, "return_raw_results": True } hybrid_retrieval=create_hybrid_retrieval_tool_sync(memory_config, **search_params) agent = create_agent( llm, tools=[time_retrieval_tool,hybrid_retrieval], - system_prompt=f"我是检索专家，可以根据适合的工具进行检索。当前使用的group_id是: {group_id}" + system_prompt=f"我是检索专家，可以根据适合的工具进行检索。当前使用的end_user_id是: {end_user_id}" ) # 创建异步任务处理单个问题 diff --git a/api/app/core/memory/agent/langgraph_graph/nodes/summary_nodes.py b/api/app/core/memory/agent/langgraph_graph/nodes/summary_nodes.py index 44f89c6a..0144c0e9 100644 --- a/api/app/core/memory/agent/langgraph_graph/nodes/summary_nodes.py +++ b/api/app/core/memory/agent/langgraph_graph/nodes/summary_nodes.py @@ -19,7 +19,7 @@ from app.core.memory.agent.utils.session_tools import SessionService from app.core.memory.agent.utils.template_tools import TemplateService from app.db import get_db -template_root = os.path.join(PROJECT_ROOT_, 'agent', 'utils', 'prompt') +template_root = os.path.join(PROJECT_ROOT_, 'memory', 'agent', 'utils', 'prompt') logger = get_agent_logger(__name__) db_session = next(get_db()) @@ -34,8 +34,8 @@ class SummaryNodeService(LLMServiceMixin): summary_service = SummaryNodeService() async def summary_history(state: ReadState) -> ReadState: - group_id = state.get("group_id", '') - history = await SessionService(store).get_history(group_id, group_id, group_id) + end_user_id = state.get("end_user_id", '') + history = await SessionService(store).get_history(end_user_id, end_user_id, end_user_id) return history async def summary_llm(state: ReadState, history, retrieve_info, template_name, operation_name, response_model,search_mode) -> str: @@ -122,12 +122,12 @@ async def summary_llm(state: ReadState, history, retrieve_info, template_name, o async def summary_redis_save(state: ReadState,aimessages) -> ReadState: data = state.get("data", '') - group_id = state.get("group_id", '') + end_user_id = state.get("end_user_id", '') await SessionService(store).save_session( - user_id=group_id, + user_id=end_user_id, query=data, - apply_id=group_id, - group_id=group_id, + apply_id=end_user_id, + end_user_id=end_user_id, ai_response=aimessages ) await SessionService(store).cleanup_duplicates() @@ -175,11 +175,11 @@ async def Input_Summary(state: ReadState) -> ReadState: memory_config = state.get('memory_config', None) user_rag_memory_id=state.get("user_rag_memory_id",'') data=state.get("data", '') - group_id=state.get("group_id", '') + end_user_id=state.get("end_user_id", '') logger.info(f"Input_Summary: storage_type={storage_type}, user_rag_memory_id={user_rag_memory_id}") history = await summary_history( state) search_params = { - "group_id": group_id, + "end_user_id": end_user_id, "question": data, "return_raw_results": True, "include": ["summaries"] # Only search summary nodes for faster performance @@ -236,7 +236,7 @@ async def Retrieve_Summary(state: ReadState)-> ReadState: retrieve_info_str='\n'.join(retrieve_info_str) aimessages=await summary_llm(state,history,retrieve_info_str, - 'Retrieve_Summary_prompt.jinja2','retrieve_summary',RetrieveSummaryResponse,"1") + 'direct_summary_prompt.jinja2','retrieve_summary',RetrieveSummaryResponse,"1") if '信息不足，无法回答' not in str(aimessages) or str(aimessages) != "": await summary_redis_save(state, aimessages) if aimessages == '': @@ -276,7 +276,6 @@ async def Summary(state: ReadState)-> ReadState: aimessages=await summary_llm(state,history,data, 'summary_prompt.jinja2','summary',SummaryResponse,0) - if '信息不足，无法回答' not in str(aimessages) or str(aimessages) != "": await summary_redis_save(state, aimessages) if aimessages == '': @@ -295,9 +294,26 @@ async def Summary(state: ReadState)-> ReadState: async def Summary_fails(state: ReadState)-> ReadState: storage_type=state.get("storage_type", '') user_rag_memory_id=state.get("user_rag_memory_id", '') + history = await summary_history(state) + query = state.get("data", '') + verify = state.get("verify", '') + verify_expansion_issue = verify.get("verified_data", '') + retrieve_info_str = '' + for data in verify_expansion_issue: + for key, value in data.items(): + if key == 'answer_small': + for i in value: + retrieve_info_str += i + '\n' + data = { + "query": query, + "history": history, + "retrieve_info": retrieve_info_str + } + aimessages = await summary_llm(state, history, data, + 'fail_summary_prompt.jinja2', 'summary', SummaryResponse, 0) result= { "status": "success", - "summary_result": "没有相关数据", + "summary_result": aimessages, "storage_type": storage_type, "user_rag_memory_id": user_rag_memory_id } diff --git a/api/app/core/memory/agent/langgraph_graph/nodes/verification_nodes.py b/api/app/core/memory/agent/langgraph_graph/nodes/verification_nodes.py index dac7ea14..b809faf2 100644 --- a/api/app/core/memory/agent/langgraph_graph/nodes/verification_nodes.py +++ b/api/app/core/memory/agent/langgraph_graph/nodes/verification_nodes.py @@ -12,7 +12,7 @@ from app.core.memory.agent.utils.session_tools import SessionService from app.core.memory.agent.utils.template_tools import TemplateService from app.core.memory.agent.services.optimized_llm_service import LLMServiceMixin -template_root = os.path.join(PROJECT_ROOT_, 'agent', 'utils', 'prompt') +template_root = os.path.join(PROJECT_ROOT_, 'memory', 'agent', 'utils', 'prompt') db_session = next(get_db()) logger = get_agent_logger(__name__) @@ -62,12 +62,12 @@ async def Verify(state: ReadState): logger.info("=== Verify 节点开始执行 ===") try: content = state.get('data', '') - group_id = state.get('group_id', '') + end_user_id = state.get('end_user_id', '') memory_config = state.get('memory_config', None) - logger.info(f"Verify: content={content[:50] if content else 'empty'}..., group_id={group_id}") + logger.info(f"Verify: content={content[:50] if content else 'empty'}..., end_user_id={end_user_id}") - history = await SessionService(store).get_history(group_id, group_id, group_id) + history = await SessionService(store).get_history(end_user_id, end_user_id, end_user_id) logger.info(f"Verify: 获取历史记录完成，history length={len(history)}") retrieve = state.get("retrieve", {}) diff --git a/api/app/core/memory/agent/langgraph_graph/nodes/write_nodes.py b/api/app/core/memory/agent/langgraph_graph/nodes/write_nodes.py index 6af313c3..b85130ad 100644 --- a/api/app/core/memory/agent/langgraph_graph/nodes/write_nodes.py +++ b/api/app/core/memory/agent/langgraph_graph/nodes/write_nodes.py @@ -1,23 +1,24 @@ - -from app.core.memory.agent.utils.llm_tools import WriteState +from app.core.memory.agent.utils.llm_tools import WriteState from app.core.memory.agent.utils.write_tools import write from app.core.logging_config import get_agent_logger logger = get_agent_logger(__name__) + + async def write_node(state: WriteState) -> WriteState: """ Write data to the database/file system. Args: - state: WriteState containing messages, group_id, and memory_config + state: WriteState containing messages, end_user_id, and memory_config Returns: dict: Contains 'write_result' with status and data fields """ messages = state.get('messages', []) - group_id = state.get('group_id', '') + end_user_id = state.get('end_user_id', '') memory_config = state.get('memory_config', '') - + # Convert LangChain messages to structured format expected by write() structured_messages = [] for msg in messages: @@ -28,13 +29,11 @@ async def write_node(state: WriteState) -> WriteState: "role": role, "content": msg.content # content is now guaranteed to be a string }) - + try: result = await write( messages=structured_messages, - user_id=group_id, - apply_id=group_id, - group_id=group_id, + end_user_id=end_user_id, memory_config=memory_config, ) logger.info(f"Write completed successfully! Config: {memory_config.config_name}") diff --git a/api/app/core/memory/agent/langgraph_graph/read_graph.py b/api/app/core/memory/agent/langgraph_graph/read_graph.py index 19011a5f..3476d0ec 100644 --- a/api/app/core/memory/agent/langgraph_graph/read_graph.py +++ b/api/app/core/memory/agent/langgraph_graph/read_graph.py @@ -79,7 +79,7 @@ async def make_read_graph(): async def main(): """主函数 - 运行工作流""" message = "昨天有什么好看的电影" - group_id = '88a459f5_text09' # 组ID + end_user_id = '88a459f5_text09' # 组ID storage_type = 'neo4j' # 存储类型 search_switch = '1' # 搜索开关 user_rag_memory_id = 'wwwwwwww' # 用户RAG记忆ID @@ -95,9 +95,9 @@ async def main(): start=time.time() try: async with make_read_graph() as graph: - config = {"configurable": {"thread_id": group_id}} + config = {"configurable": {"thread_id": end_user_id}} # 初始状态 - 包含所有必要字段 - initial_state = {"messages": [HumanMessage(content=message)] ,"search_switch":search_switch,"group_id":group_id + initial_state = {"messages": [HumanMessage(content=message)] ,"search_switch":search_switch,"end_user_id":end_user_id ,"storage_type":storage_type,"user_rag_memory_id":user_rag_memory_id,"memory_config":memory_config} # 获取节点更新信息 _intermediate_outputs = [] diff --git a/api/app/core/memory/agent/langgraph_graph/tools/tool.py b/api/app/core/memory/agent/langgraph_graph/tools/tool.py index ce6d5dd4..c4814de1 100644 --- a/api/app/core/memory/agent/langgraph_graph/tools/tool.py +++ b/api/app/core/memory/agent/langgraph_graph/tools/tool.py @@ -48,11 +48,11 @@ def extract_tool_message_content(response): class TimeRetrievalInput(BaseModel): """时间检索工具的输入模式""" context: str = Field(description="用户输入的查询内容") - group_id: str = Field(default="88a459f5_text09", description="组ID，用于过滤搜索结果") + end_user_id: str = Field(default="88a459f5_text09", description="组ID，用于过滤搜索结果") -def create_time_retrieval_tool(group_id: str): +def create_time_retrieval_tool(end_user_id: str): """ - 创建一个带有特定group_id的TimeRetrieval工具（同步版本），用于按时间范围搜索语句(Statements) + 创建一个带有特定end_user_id的TimeRetrieval工具（同步版本），用于按时间范围搜索语句(Statements) """ def clean_temporal_result_fields(data): @@ -93,26 +93,26 @@ def create_time_retrieval_tool(group_id: str): return data @tool - def TimeRetrievalWithGroupId(context: str, start_date: str = None, end_date: str = None, group_id_param: str = None, clean_output: bool = True) -> str: + def TimeRetrievalWithGroupId(context: str, start_date: str = None, end_date: str = None, end_user_id_param: str = None, clean_output: bool = True) -> str: """ 优化的时间检索工具，只结合时间范围搜索（同步版本），自动过滤不需要的元数据字段 显式接收参数： - context: 查询上下文内容 - start_date: 开始时间（可选，格式：YYYY-MM-DD） - end_date: 结束时间（可选，格式：YYYY-MM-DD） - - group_id_param: 组ID（可选，用于覆盖默认组ID） + - end_user_id_param: 组ID（可选，用于覆盖默认组ID） - clean_output: 是否清理输出中的元数据字段 -end_date 需要根据用户的描述获取结束的时间，输出格式用strftime("%Y-%m-%d") """ async def _async_search(): # 使用传入的参数或默认值 - actual_group_id = group_id_param or group_id + actual_end_user_id = end_user_id_param or end_user_id actual_end_date = end_date or datetime.now().strftime("%Y-%m-%d") actual_start_date = start_date or (datetime.now() - timedelta(days=7)).strftime("%Y-%m-%d") # 基本时间搜索 results = await search_by_temporal( - group_id=actual_group_id, + end_user_id=actual_end_user_id, start_date=actual_start_date, end_date=actual_end_date, limit=10 @@ -147,7 +147,7 @@ def create_time_retrieval_tool(group_id: str): # 关键词时间搜索 results = await search_by_keyword_temporal( query_text=context, - group_id=group_id, + end_user_id=end_user_id, start_date=actual_start_date, end_date=actual_end_date, limit=15 @@ -172,7 +172,7 @@ def create_hybrid_retrieval_tool_async(memory_config, **search_params): Args: memory_config: 内存配置对象 - **search_params: 搜索参数，包含group_id, limit, include等 + **search_params: 搜索参数，包含end_user_id, limit, include等 """ def clean_result_fields(data): @@ -211,7 +211,7 @@ def create_hybrid_retrieval_tool_async(memory_config, **search_params): context: str, search_type: str = "hybrid", limit: int = 10, - group_id: str = None, + end_user_id: str = None, rerank_alpha: float = 0.6, use_forgetting_rerank: bool = False, use_llm_rerank: bool = False, @@ -224,7 +224,7 @@ def create_hybrid_retrieval_tool_async(memory_config, **search_params): context: 查询内容 search_type: 搜索类型 ('keyword', 'embedding', 'hybrid') limit: 结果数量限制 - group_id: 组ID，用于过滤搜索结果 + end_user_id: 组ID，用于过滤搜索结果 rerank_alpha: 重排序权重参数 use_forgetting_rerank: 是否使用遗忘重排序 use_llm_rerank: 是否使用LLM重排序 @@ -238,7 +238,7 @@ def create_hybrid_retrieval_tool_async(memory_config, **search_params): final_params = { "query_text": context, "search_type": search_type, - "group_id": group_id or search_params.get("group_id"), + "end_user_id": end_user_id or search_params.get("end_user_id"), "limit": limit or search_params.get("limit", 10), "include": search_params.get("include", ["summaries", "statements", "chunks", "entities"]), "output_path": None, # 不保存到文件 @@ -291,7 +291,7 @@ def create_hybrid_retrieval_tool_sync(memory_config, **search_params): context: str, search_type: str = "hybrid", limit: int = 10, - group_id: str = None, + end_user_id: str = None, clean_output: bool = True ) -> str: """ @@ -301,7 +301,7 @@ def create_hybrid_retrieval_tool_sync(memory_config, **search_params): context: 查询内容 search_type: 搜索类型 ('keyword', 'embedding', 'hybrid') limit: 结果数量限制 - group_id: 组ID，用于过滤搜索结果 + end_user_id: 组ID，用于过滤搜索结果 clean_output: 是否清理输出中的元数据字段 """ async def _async_search(): @@ -311,7 +311,7 @@ def create_hybrid_retrieval_tool_sync(memory_config, **search_params): "context": context, "search_type": search_type, "limit": limit, - "group_id": group_id, + "end_user_id": end_user_id, "clean_output": clean_output }) diff --git a/api/app/core/memory/agent/langgraph_graph/write_graph.py b/api/app/core/memory/agent/langgraph_graph/write_graph.py index fe281a23..8b5de444 100644 --- a/api/app/core/memory/agent/langgraph_graph/write_graph.py +++ b/api/app/core/memory/agent/langgraph_graph/write_graph.py @@ -14,6 +14,7 @@ from app.db import get_db from app.core.logging_config import get_agent_logger from app.core.memory.agent.utils.llm_tools import WriteState from app.core.memory.agent.langgraph_graph.nodes.write_nodes import write_node +from app.core.memory.agent.langgraph_graph.nodes.data_nodes import content_input_write from app.services.memory_config_service import MemoryConfigService warnings.filterwarnings("ignore", category=RuntimeWarning) @@ -26,9 +27,21 @@ async def make_write_graph(): """ Create a write graph workflow for memory operations. - The workflow directly processes messages from the initial state - and saves them to Neo4j storage. + Args: + user_id: User identifier + tools: MCP tools loaded from session + apply_id: Application identifier + end_user_id: Group identifier + memory_config: MemoryConfig object containing all configuration """ + # workflow = StateGraph(WriteState) + # workflow.add_node("content_input", content_input_write) + # workflow.add_node("save_neo4j", write_node) + # workflow.add_edge(START, "content_input") + # workflow.add_edge("content_input", "save_neo4j") + # workflow.add_edge("save_neo4j", END) + # + # graph = workflow.compile() workflow = StateGraph(WriteState) workflow.add_node("save_neo4j", write_node) workflow.add_edge(START, "save_neo4j") @@ -42,7 +55,7 @@ async def make_write_graph(): async def main(): """主函数 - 运行工作流""" message = "今天周一" - group_id = 'new_2025test1103' # 组ID + end_user_id = 'new_2025test1103' # 组ID # 获取数据库会话 @@ -54,9 +67,9 @@ async def main(): ) try: async with make_write_graph() as graph: - config = {"configurable": {"thread_id": group_id}} + config = {"configurable": {"thread_id": end_user_id}} # 初始状态 - 包含所有必要字段 - initial_state = {"messages": [HumanMessage(content=message)], "group_id": group_id, "memory_config": memory_config} + initial_state = {"messages": [HumanMessage(content=message)], "end_user_id": end_user_id, "memory_config": memory_config} # 获取节点更新信息 async for update_event in graph.astream( diff --git a/api/app/core/memory/agent/services/parameter_builder.py b/api/app/core/memory/agent/services/parameter_builder.py index a58fcf1a..74382ade 100644 --- a/api/app/core/memory/agent/services/parameter_builder.py +++ b/api/app/core/memory/agent/services/parameter_builder.py @@ -24,7 +24,7 @@ class ParameterBuilder: tool_call_id: str, search_switch: str, apply_id: str, - group_id: str, + end_user_id: str, storage_type: Optional[str] = None, user_rag_memory_id: Optional[str] = None ) -> Dict[str, Any]: @@ -44,7 +44,7 @@ class ParameterBuilder: tool_call_id: Extracted tool call identifier search_switch: Search routing parameter apply_id: Application identifier - group_id: Group identifier + end_user_id: Group identifier storage_type: Storage type for the workspace (optional) user_rag_memory_id: User RAG memory ID for knowledge base retrieval (optional) @@ -55,7 +55,7 @@ class ParameterBuilder: base_args = { "usermessages": tool_call_id, "apply_id": apply_id, - "group_id": group_id + "end_user_id": end_user_id } # Always add storage_type and user_rag_memory_id (with defaults if None) diff --git a/api/app/core/memory/agent/services/search_service.py b/api/app/core/memory/agent/services/search_service.py index 8a2e7cfe..4fc4256e 100644 --- a/api/app/core/memory/agent/services/search_service.py +++ b/api/app/core/memory/agent/services/search_service.py @@ -91,7 +91,7 @@ class SearchService: async def execute_hybrid_search( self, - group_id: str, + end_user_id: str, question: str, limit: int = 5, search_type: str = "hybrid", @@ -105,7 +105,7 @@ class SearchService: Execute hybrid search and return clean content. Args: - group_id: Group identifier for filtering results + end_user_id: Group identifier for filtering results question: Search query text limit: Maximum number of results to return (default: 5) search_type: Type of search - "hybrid", "keyword", or "embedding" (default: "hybrid") @@ -130,7 +130,7 @@ class SearchService: answer = await run_hybrid_search( query_text=cleaned_query, search_type=search_type, - group_id=group_id, + end_user_id=end_user_id, limit=limit, include=include, output_path=output_path, @@ -186,7 +186,7 @@ class SearchService: except Exception as e: logger.error( - f"Search failed for query '{question}' in group '{group_id}': {e}", + f"Search failed for query '{question}' in group '{end_user_id}': {e}", exc_info=True ) # Return empty results on failure diff --git a/api/app/core/memory/agent/services/session_service.py b/api/app/core/memory/agent/services/session_service.py index b2d4f0ff..f7389984 100644 --- a/api/app/core/memory/agent/services/session_service.py +++ b/api/app/core/memory/agent/services/session_service.py @@ -59,7 +59,7 @@ class SessionService: self, user_id: str, apply_id: str, - group_id: str + end_user_id: str ) -> List[dict]: """ Retrieve conversation history from Redis. @@ -67,20 +67,20 @@ class SessionService: Args: user_id: User identifier apply_id: Application identifier - group_id: Group identifier + end_user_id: Group identifier Returns: List of conversation history items with Query and Answer keys Returns empty list if no history found or on error """ try: - history = self.store.find_user_apply_group(user_id, apply_id, group_id) + history = self.store.find_user_apply_group(user_id, apply_id, end_user_id) # Validate history structure if not isinstance(history, list): logger.warning( f"Invalid history format for user {user_id}, " - f"apply {apply_id}, group {group_id}: expected list, got {type(history)}" + f"apply {apply_id}, group {end_user_id}: expected list, got {type(history)}" ) return [] @@ -89,7 +89,7 @@ class SessionService: except Exception as e: logger.error( f"Failed to retrieve history for user {user_id}, " - f"apply {apply_id}, group {group_id}: {e}", + f"apply {apply_id}, group {end_user_id}: {e}", exc_info=True ) # Return empty list on error to allow execution to continue @@ -100,7 +100,7 @@ class SessionService: user_id: str, query: str, apply_id: str, - group_id: str, + end_user_id: str, ai_response: str ) -> Optional[str]: """ @@ -110,7 +110,7 @@ class SessionService: user_id: User identifier query: User query/message apply_id: Application identifier - group_id: Group identifier + end_user_id: Group identifier ai_response: AI response/answer Returns: @@ -131,7 +131,7 @@ class SessionService: userid=user_id, messages=query, apply_id=apply_id, - group_id=group_id, + end_user_id=end_user_id, aimessages=ai_response ) @@ -152,7 +152,7 @@ class SessionService: Duplicates are identified by matching: - sessionid - user_id (id field) - - group_id + - end_user_id - messages - aimessages diff --git a/api/app/core/memory/agent/utils/get_dialogs.py b/api/app/core/memory/agent/utils/get_dialogs.py index 82a41773..bfb0f675 100644 --- a/api/app/core/memory/agent/utils/get_dialogs.py +++ b/api/app/core/memory/agent/utils/get_dialogs.py @@ -9,9 +9,7 @@ from app.core.memory.models.message_models import DialogData, ConversationContex async def get_chunked_dialogs( chunker_strategy: str = "RecursiveChunker", - group_id: str = "group_1", - user_id: str = "user1", - apply_id: str = "applyid", + end_user_id: str = "group_1", messages: list = None, ref_id: str = "wyl_20251027", config_id: str = None @@ -20,9 +18,7 @@ async def get_chunked_dialogs( Args: chunker_strategy: The chunking strategy to use (default: RecursiveChunker) - group_id: Group identifier - user_id: User identifier - apply_id: Application identifier + end_user_id: Group identifier messages: Structured message list [{"role": "user", "content": "..."}, ...] ref_id: Reference identifier config_id: Configuration ID for processing @@ -32,42 +28,40 @@ async def get_chunked_dialogs( """ from app.core.logging_config import get_agent_logger logger = get_agent_logger(__name__) - + if not messages or not isinstance(messages, list) or len(messages) == 0: raise ValueError("messages parameter must be a non-empty list") - + conversation_messages = [] - + for idx, msg in enumerate(messages): if not isinstance(msg, dict) or 'role' not in msg or 'content' not in msg: raise ValueError(f"Message {idx} format error: must contain 'role' and 'content' fields") - + role = msg['role'] content = msg['content'] - + if role not in ['user', 'assistant']: raise ValueError(f"Message {idx} role must be 'user' or 'assistant', got: {role}") - + if content.strip(): conversation_messages.append(ConversationMessage(role=role, msg=content.strip())) - + if not conversation_messages: raise ValueError("Message list cannot be empty after filtering") - + conversation_context = ConversationContext(msgs=conversation_messages) dialog_data = DialogData( context=conversation_context, ref_id=ref_id, - group_id=group_id, - user_id=user_id, - apply_id=apply_id, + end_user_id=end_user_id, config_id=config_id ) - + chunker = DialogueChunker(chunker_strategy) extracted_chunks = await chunker.process_dialogue(dialog_data) dialog_data.chunks = extracted_chunks - + logger.info(f"DialogData created with {len(extracted_chunks)} chunks") return [dialog_data] diff --git a/api/app/core/memory/agent/utils/llm_tools.py b/api/app/core/memory/agent/utils/llm_tools.py index 8dd2f1d3..7f1041cb 100644 --- a/api/app/core/memory/agent/utils/llm_tools.py +++ b/api/app/core/memory/agent/utils/llm_tools.py @@ -1,24 +1,23 @@ import os from collections import defaultdict +from pathlib import Path from typing import Annotated, TypedDict from langchain_core.messages import AnyMessage from langgraph.graph import add_messages -PROJECT_ROOT_ = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) +PROJECT_ROOT_ = str(Path(__file__).resolve().parents[3]) class WriteState(TypedDict): ''' Langgrapg Writing TypedDict ''' messages: Annotated[list[AnyMessage], add_messages] - user_id:str - apply_id:str - group_id:str + end_user_id: str errors: list[dict] # Track errors: [{"tool": "tool_name", "error": "message"}] memory_config: object write_result: dict - data:str + data: str class ReadState(TypedDict): """ @@ -28,7 +27,7 @@ class ReadState(TypedDict): messages: 消息列表，支持自动追加 loop_count: 遍历次数 search_switch: 搜索类型开关 - group_id: 组标识 + end_user_id: 组标识 config_id: 配置ID，用于过滤结果 data: 从content_input_node传递的内容数据 spit_data: 从Split_The_Problem传递的分解结果 @@ -39,7 +38,7 @@ class ReadState(TypedDict): messages: Annotated[list[AnyMessage], add_messages] # 消息追加模式 loop_count: int search_switch: str - group_id: str + end_user_id: str config_id: str data: str # 新增字段用于传递内容 spit_data: dict # 新增字段用于传递问题分解结果 diff --git a/api/app/core/memory/agent/utils/prompt/direct_summary_prompt.jinja2 b/api/app/core/memory/agent/utils/prompt/direct_summary_prompt.jinja2 new file mode 100644 index 00000000..1e0690bf --- /dev/null +++ b/api/app/core/memory/agent/utils/prompt/direct_summary_prompt.jinja2 @@ -0,0 +1,61 @@ +# 角色 +你是一个智能问答助手，基于检索信息和历史对话回答用户问题。 +# 任务 +根据提供的上下文信息回答用户的问题。 +# 输入信息 +- 历史对话：{{history}} +- 检索信息：{{retrieve_info}} +# 用户问题 +{{query}} +# 回答指南 +## 1. 仔细阅读检索信息 +- 答案可能直接或间接地出现在检索信息中 +- 如果检索信息中提到"小曼会使用Python"，说明用户名是"小曼" +- 第三人称描述的偏好、行为通常指用户本人 + +## 2. 判断信息相关性 +**情况A：信息匹配问题** +- 直接回答，像自然对话一样 +- 例：检索到"小曼会使用Python" → 问"我叫什么" → 答"你叫小曼" + +**情况B：信息部分相关** +- 先回答已知部分，再自然地询问更多信息 +- 例：检索到"用户去过上海的面包店" → 问"我吃过哪家面包" → 答"我记得你去过上海的面包店，但具体是哪家我不太清楚，是哪家呢？" + +**情况C：信息完全不相关** +- 自然地表达不知道，但可以提及检索到的相关信息，让对话更连贯 +- 使用友好的表达： + - "你好像没和我说过...，但是我知道你[检索到的相关信息]" + - "关于这个我不太清楚，不过我记得你[检索到的相关信息]，能告诉我更多吗？" + - "我不记得你提到过...，但你[检索到的相关信息]" +- 即使检索信息不直接回答问题，也可以自然地融入对话中 +- 避免僵硬的"信息不足，无法回答" +## 3. 回答要求 +- 像人类对话一样自然流畅 +- 不要提及"检索信息"、"搜索结果"、"根据资料"等技术术语 +- 不要解释推理过程或引用信息来源 +- 保持友好、乐于助人的语气 +- 使用与问题相同的语言回答 +# 关键示例 +**示例1 - 直接匹配：** +- 检索信息："小曼会使用Python..." +- 问题："我叫什么" +- ✓ 正确："你叫小曼" +- ✗ 错误："你没有告诉我你的名字" +**示例2 - 间接匹配：** +- 检索信息："用户很喜欢吃星巴克的甜品" +- 问题："我喜欢什么" +- ✓ 正确："你很喜欢吃星巴克的甜品" +- ✗ 错误："信息不足" +**示例3 - 信息不匹配（推荐做法）：** +- 检索信息："用户只喝拿铁咖啡，认为美式咖啡太苦" +- 问题："我吃过哪家面包" +- ✓ 最佳："你好像没和我说过吃过哪家面包，但是我知道你喜欢喝拿铁，能跟我分享一下吗？" +- ✓ 可以："你好像没和我说过吃过哪家面包，能跟我分享一下吗？" +- ✗ 错误："用户只喝拿铁咖啡，认为美式咖啡太苦。"（答非所问） +- ✗ 错误："信息不足，无法回答。"（太僵硬） +# 重要提醒 +- 检索信息中描述用户行为/偏好时提到的名字，就是用户的名字 +- 信息不匹配时，不要强行回答无关内容，但可以自然地提及检索到的信息，让对话更有温度 +- 用对话式语言表达"不知道"，而非机械模板 +- 检索信息代表你对用户的了解，即使不直接回答问题，也能体现你对用户的记忆 diff --git a/api/app/core/memory/agent/utils/prompt/fail_summary_prompt.jinja2 b/api/app/core/memory/agent/utils/prompt/fail_summary_prompt.jinja2 new file mode 100644 index 00000000..3744f99b --- /dev/null +++ b/api/app/core/memory/agent/utils/prompt/fail_summary_prompt.jinja2 @@ -0,0 +1,43 @@ +{# 角色定义 #} +你是专业的问题解答专家+引导学者 + +{# 输入数据展示 #} +{% if data %} +## 输入数据 +上下文信息: +{% for item in data.history %} +- {{ item }} +{% endfor %} +检索到的所有信息: +{% for item in data.retrieve_info %} +- {{ item }} +{% endfor %} +{% endif %} + +## User Query +{{ query }} + +{# 问题回答标准 #} +## 问题回答核心标准 +根据上下文信息(history)和检索到的所有信息(retrieve_info)准确回答用户的问题(query)。 +注意，仔细阅读检索信息，答案可能直接或间接地出现在检索信息中或者历史上下文消息中，同时需要 判断信息相关性 +**情况A：信息匹配问题** +- 直接回答，像自然对话一样 +- 例：检索到"小曼会使用Python" → 问"我叫什么" → 答"你叫小曼" + +**情况B：信息部分相关** +- 先回答已知部分，再自然地询问更多信息 +- 例：检索到"用户去过上海的面包店" → 问"我吃过哪家面包" → 答"我记得你去过上海的面包店，但具体是哪家我不太清楚，是哪家呢？" + +**情况C：信息完全不相关** +- 自然地表达不知道，但可以提及检索到的相关信息，让对话更连贯 +- 使用友好的表达： + - "你好像没和我说过...，但是我知道你[检索到的相关信息]" + - "关于这个我不太清楚，不过我记得你[检索到的相关信息]，能告诉我更多吗？" + - "我不记得你提到过...，但你[检索到的相关信息]" +- 即使检索信息不直接回答问题，也可以自然地融入对话中 +- 避免僵硬的"信息不足，无法回答" + +{# 重要提醒 #} +当检索以及上下文的历史信息都无法回答的时候，可引导对方进行提问/回答，或者进行其他引导 +当检索或者上下文中出现了，相似的问题，可以委婉，提醒对方，我记得刚刚提过这个问题，但是我自己不记得了，能在描述一次吗～以此为例 diff --git a/api/app/core/memory/agent/utils/redis_tool.py b/api/app/core/memory/agent/utils/redis_tool.py index 31a76a11..505545b3 100644 --- a/api/app/core/memory/agent/utils/redis_tool.py +++ b/api/app/core/memory/agent/utils/redis_tool.py @@ -28,7 +28,7 @@ class RedisSessionStore: return text # 修改后的 save_session 方法 - def save_session(self, userid, messages, aimessages, apply_id, group_id): + def save_session(self, userid, messages, aimessages, apply_id, end_user_id): """ 写入一条会话数据，返回 session_id 优化版本：确保写入时间不超过1秒 @@ -46,7 +46,7 @@ class RedisSessionStore: "id": self.uudi, "sessionid": userid, "apply_id": apply_id, - "group_id": group_id, + "end_user_id": end_user_id, "messages": messages, "aimessages": aimessages, "starttime": starttime @@ -67,7 +67,7 @@ class RedisSessionStore: def save_sessions_batch(self, sessions_data): """ 批量写入多条会话数据，返回 session_id 列表 - sessions_data: list of dict, 每个 dict 包含 userid, messages, aimessages, apply_id, group_id + sessions_data: list of dict, 每个 dict 包含 userid, messages, aimessages, apply_id, end_user_id 优化版本：批量操作，大幅提升性能 """ try: @@ -83,7 +83,7 @@ class RedisSessionStore: "id": self.uudi, "sessionid": session.get('userid'), "apply_id": session.get('apply_id'), - "group_id": session.get('group_id'), + "end_user_id": session.get('end_user_id'), "messages": session.get('messages'), "aimessages": session.get('aimessages'), "starttime": starttime @@ -108,9 +108,9 @@ class RedisSessionStore: data = self.r.hgetall(key) return data if data else None - def get_session_apply_group(self, sessionid, apply_id, group_id): + def get_session_apply_group(self, sessionid, apply_id, end_user_id): """ - 根据 sessionid、apply_id 和 group_id 三个条件查询会话数据 + 根据 sessionid、apply_id 和 end_user_id 三个条件查询会话数据 """ result_items = [] @@ -124,7 +124,7 @@ class RedisSessionStore: # 检查三个条件是否都匹配 if (data.get('sessionid') == sessionid and data.get('apply_id') == apply_id and - data.get('group_id') == group_id): + data.get('end_user_id') == end_user_id): result_items.append(data) return result_items @@ -172,7 +172,7 @@ class RedisSessionStore: def delete_duplicate_sessions(self): """ 删除重复会话数据，条件： - "sessionid"、"user_id"、"group_id"、"messages"、"aimessages" 五个字段都相同的只保留一个，其他删除 + "sessionid"、"user_id"、"end_user_id"、"messages"、"aimessages" 五个字段都相同的只保留一个，其他删除 优化版本：使用 pipeline 批量操作，确保在1秒内完成 """ import time @@ -202,12 +202,12 @@ class RedisSessionStore: # 获取五个字段的值 sessionid = data.get('sessionid', '') user_id = data.get('id', '') - group_id = data.get('group_id', '') + end_user_id = data.get('end_user_id', '') messages = data.get('messages', '') aimessages = data.get('aimessages', '') # 用五元组作为唯一标识 - identifier = (sessionid, user_id, group_id, messages, aimessages) + identifier = (sessionid, user_id, end_user_id, messages, aimessages) if identifier in seen: # 重复，标记为待删除 @@ -248,9 +248,9 @@ class RedisSessionStore: result_items = [] return (result_items) - def find_user_apply_group(self, sessionid, apply_id, group_id): + def find_user_apply_group(self, sessionid, apply_id, end_user_id): """ - 根据 sessionid、apply_id 和 group_id 三个条件查询会话数据，返回最新的6条 + 根据 sessionid、apply_id 和 end_user_id 三个条件查询会话数据，返回最新的6条 """ import time start_time = time.time() @@ -276,7 +276,7 @@ class RedisSessionStore: # 检查是否符合三个条件 if (data.get('apply_id') == apply_id and - data.get('group_id') == group_id): + data.get('end_user_id') == end_user_id): # 支持模糊匹配 sessionid 或者完全匹配 if sessionid in data.get('sessionid', '') or data.get('sessionid') == sessionid: matched_items.append({ diff --git a/api/app/core/memory/agent/utils/session_tools.py b/api/app/core/memory/agent/utils/session_tools.py index b2d4f0ff..f7389984 100644 --- a/api/app/core/memory/agent/utils/session_tools.py +++ b/api/app/core/memory/agent/utils/session_tools.py @@ -59,7 +59,7 @@ class SessionService: self, user_id: str, apply_id: str, - group_id: str + end_user_id: str ) -> List[dict]: """ Retrieve conversation history from Redis. @@ -67,20 +67,20 @@ class SessionService: Args: user_id: User identifier apply_id: Application identifier - group_id: Group identifier + end_user_id: Group identifier Returns: List of conversation history items with Query and Answer keys Returns empty list if no history found or on error """ try: - history = self.store.find_user_apply_group(user_id, apply_id, group_id) + history = self.store.find_user_apply_group(user_id, apply_id, end_user_id) # Validate history structure if not isinstance(history, list): logger.warning( f"Invalid history format for user {user_id}, " - f"apply {apply_id}, group {group_id}: expected list, got {type(history)}" + f"apply {apply_id}, group {end_user_id}: expected list, got {type(history)}" ) return [] @@ -89,7 +89,7 @@ class SessionService: except Exception as e: logger.error( f"Failed to retrieve history for user {user_id}, " - f"apply {apply_id}, group {group_id}: {e}", + f"apply {apply_id}, group {end_user_id}: {e}", exc_info=True ) # Return empty list on error to allow execution to continue @@ -100,7 +100,7 @@ class SessionService: user_id: str, query: str, apply_id: str, - group_id: str, + end_user_id: str, ai_response: str ) -> Optional[str]: """ @@ -110,7 +110,7 @@ class SessionService: user_id: User identifier query: User query/message apply_id: Application identifier - group_id: Group identifier + end_user_id: Group identifier ai_response: AI response/answer Returns: @@ -131,7 +131,7 @@ class SessionService: userid=user_id, messages=query, apply_id=apply_id, - group_id=group_id, + end_user_id=end_user_id, aimessages=ai_response ) @@ -152,7 +152,7 @@ class SessionService: Duplicates are identified by matching: - sessionid - user_id (id field) - - group_id + - end_user_id - messages - aimessages diff --git a/api/app/core/memory/agent/utils/write_tools.py b/api/app/core/memory/agent/utils/write_tools.py index 1df0b336..446ab86a 100644 --- a/api/app/core/memory/agent/utils/write_tools.py +++ b/api/app/core/memory/agent/utils/write_tools.py @@ -29,20 +29,18 @@ logger = get_agent_logger(__name__) async def write( - user_id: str, - apply_id: str, - group_id: str, + end_user_id: str, memory_config: MemoryConfig, messages: list, ref_id: str = "wyl20251027", ) -> None: """ Execute the complete knowledge extraction pipeline. - + Args: user_id: User identifier apply_id: Application identifier - group_id: Group identifier + end_user_id: Group identifier memory_config: MemoryConfig object containing all configuration messages: Structured message list [{"role": "user", "content": "..."}, ...] ref_id: Reference ID, defaults to "wyl20251027" @@ -51,14 +49,14 @@ async def write( embedding_model_id = str(memory_config.embedding_model_id) chunker_strategy = memory_config.chunker_strategy config_id = str(memory_config.config_id) - + logger.info("=== MemSci Knowledge Extraction Pipeline ===") logger.info(f"Config: {memory_config.config_name} (ID: {config_id})") logger.info(f"Workspace: {memory_config.workspace_name}") logger.info(f"LLM model: {memory_config.llm_model_name}") logger.info(f"Embedding model: {memory_config.embedding_model_name}") logger.info(f"Chunker strategy: {chunker_strategy}") - logger.info(f"Group ID: {group_id}") + logger.info(f"end_user_id ID: {end_user_id}") # Construct clients from memory_config using factory pattern with db session with get_db_context() as db: @@ -83,9 +81,7 @@ async def write( step_start = time.time() chunked_dialogs = await get_chunked_dialogs( chunker_strategy=chunker_strategy, - group_id=group_id, - user_id=user_id, - apply_id=apply_id, + end_user_id=end_user_id, messages=messages, ref_id=ref_id, config_id=config_id, diff --git a/api/app/core/memory/analytics/api_docs_parser.py b/api/app/core/memory/analytics/api_docs_parser.py index 94ed0f00..4a116520 100644 --- a/api/app/core/memory/analytics/api_docs_parser.py +++ b/api/app/core/memory/analytics/api_docs_parser.py @@ -139,7 +139,8 @@ def parse_api_docs(file_path: str) -> Dict[str, Any]: def get_default_docs_path() -> str: - project_root = os.path.dirname(os.path.dirname(os.path.dirname(__file__))) + from pathlib import Path + project_root = str(Path(__file__).resolve().parents[2]) return os.path.join(project_root, "src", "analytics", "API接口.md") diff --git a/api/app/core/memory/analytics/hot_memory_tags.py b/api/app/core/memory/analytics/hot_memory_tags.py index cab6cacd..95302726 100644 --- a/api/app/core/memory/analytics/hot_memory_tags.py +++ b/api/app/core/memory/analytics/hot_memory_tags.py @@ -16,13 +16,13 @@ class FilteredTags(BaseModel): """用于接收LLM筛选后的核心标签列表的模型。""" meaningful_tags: List[str] = Field(..., description="从原始列表中筛选出的具有核心代表意义的名词列表。") -async def filter_tags_with_llm(tags: List[str], group_id: str) -> List[str]: +async def filter_tags_with_llm(tags: List[str], end_user_id: str) -> List[str]: """ 使用LLM筛选标签列表，仅保留具有代表性的核心名词。 Args: tags: 原始标签列表 - group_id: 用户组ID，用于获取配置 + end_user_id: 用户组ID，用于获取配置 Returns: 筛选后的标签列表 @@ -37,12 +37,12 @@ async def filter_tags_with_llm(tags: List[str], group_id: str) -> List[str]: get_end_user_connected_config, ) - connected_config = get_end_user_connected_config(group_id, db) + connected_config = get_end_user_connected_config(end_user_id, db) config_id = connected_config.get("memory_config_id") if not config_id: raise ValueError( - f"No memory_config_id found for group_id: {group_id}. " + f"No memory_config_id found for end_user_id: {end_user_id}. " "Please ensure the user has a valid memory configuration." ) @@ -87,7 +87,7 @@ async def filter_tags_with_llm(tags: List[str], group_id: str) -> List[str]: async def get_raw_tags_from_db( connector: Neo4jConnector, - group_id: str, + end_user_id: str, limit: int, by_user: bool = False ) -> List[Tuple[str, int]]: @@ -99,9 +99,9 @@ async def get_raw_tags_from_db( Args: connector: Neo4j连接器实例 - group_id: 如果by_user=False，则为group_id；如果by_user=True，则为user_id + end_user_id: 如果by_user=False，则为end_user_id；如果by_user=True，则为user_id limit: 返回的标签数量限制 - by_user: 是否按user_id查询（默认False，按group_id查询） + by_user: 是否按user_id查询（默认False，按end_user_id查询） Returns: List[Tuple[str, int]]: 标签名称和频率的元组列表 @@ -119,7 +119,7 @@ async def get_raw_tags_from_db( else: query = ( "MATCH (e:ExtractedEntity) " - "WHERE e.group_id = $id AND e.entity_type <> '人物' AND e.name IS NOT NULL AND NOT e.name IN $names_to_exclude " + "WHERE e.end_user_id = $id AND e.entity_type <> '人物' AND e.name IS NOT NULL AND NOT e.name IN $names_to_exclude " "RETURN e.name AS name, count(e) AS frequency " "ORDER BY frequency DESC " "LIMIT $limit" @@ -128,44 +128,44 @@ async def get_raw_tags_from_db( # 使用项目的Neo4jConnector执行查询 results = await connector.execute_query( query, - id=group_id, + id=end_user_id, limit=limit, names_to_exclude=names_to_exclude ) return [(record["name"], record["frequency"]) for record in results] -async def get_hot_memory_tags(group_id: str, limit: int = 40, by_user: bool = False) -> List[Tuple[str, int]]: +async def get_hot_memory_tags(end_user_id: str, limit: int = 40, by_user: bool = False) -> List[Tuple[str, int]]: """ 获取原始标签，然后使用LLM进行筛选，返回最终的热门标签列表。 查询更多的标签(limit=40)给LLM提供更丰富的上下文进行筛选。 Args: - group_id: 必需参数。如果by_user=False，则为group_id；如果by_user=True，则为user_id + end_user_id: 必需参数。如果by_user=False，则为end_user_id；如果by_user=True，则为user_id limit: 返回的标签数量限制 - by_user: 是否按user_id查询（默认False，按group_id查询） + by_user: 是否按user_id查询（默认False，按end_user_id查询） Raises: - ValueError: 如果group_id未提供或为空 + ValueError: 如果end_user_id未提供或为空 """ - # 验证group_id必须提供且不为空 - if not group_id or not group_id.strip(): + # 验证end_user_id必须提供且不为空 + if not end_user_id or not end_user_id.strip(): raise ValueError( - "group_id is required. Please provide a valid group_id or user_id." + "end_user_id is required. Please provide a valid end_user_id or user_id." ) # 使用项目的Neo4jConnector connector = Neo4jConnector() try: # 1. 从数据库获取原始排名靠前的标签 - raw_tags_with_freq = await get_raw_tags_from_db(connector, group_id, limit, by_user=by_user) + raw_tags_with_freq = await get_raw_tags_from_db(connector, end_user_id, limit, by_user=by_user) if not raw_tags_with_freq: return [] raw_tag_names = [tag for tag, freq in raw_tags_with_freq] # 2. 初始化LLM客户端并使用LLM筛选出有意义的标签 - meaningful_tag_names = await filter_tags_with_llm(raw_tag_names, group_id) + meaningful_tag_names = await filter_tags_with_llm(raw_tag_names, end_user_id) # 3. 根据LLM的筛选结果，构建最终的标签列表（保留原始频率和顺序） final_tags = [] diff --git a/api/app/core/memory/analytics/implicit_memory/data_source.py b/api/app/core/memory/analytics/implicit_memory/data_source.py index d277a05e..18678a55 100644 --- a/api/app/core/memory/analytics/implicit_memory/data_source.py +++ b/api/app/core/memory/analytics/implicit_memory/data_source.py @@ -75,8 +75,8 @@ class MemoryDataSource: start_date = time_range.start_date if time_range else None end_date = time_range.end_date if time_range else None - summary_dicts = await self.memory_summary_repo.find_by_group_id( - group_id=user_id, + summary_dicts = await self.memory_summary_repo.find_by_end_user_id( + end_user_id=user_id, limit=limit, start_date=start_date, end_date=end_date diff --git a/api/app/core/memory/analytics/recent_activity_stats.py b/api/app/core/memory/analytics/recent_activity_stats.py index c41f4208..71f70c09 100644 --- a/api/app/core/memory/analytics/recent_activity_stats.py +++ b/api/app/core/memory/analytics/recent_activity_stats.py @@ -2,13 +2,16 @@ import os import re import glob import json +from pathlib import Path from typing import Tuple try: from app.core.memory.utils.config.definitions import PROJECT_ROOT except Exception: # Fallback: derive project root from this file location - PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + # 当前文件在 api/app/core/memory/analytics/recent_activity_stats.py + # 需要向上 5 级到达 api/ 目录 + PROJECT_ROOT = str(Path(__file__).resolve().parents[4]) def _get_latest_prompt_log_path() -> str | None: @@ -67,44 +70,43 @@ def parse_stats_from_log(log_path: str) -> dict: triplet_relations_count = 0 temporal_count = 0 - # Patterns + # 正则表达式模式 - 匹配当前日志格式 pat_chunk_render = re.compile(r"===\s*RENDERED\s*STATEMENT\s*EXTRACTION\s*PROMPT\s*===") - pat_triplet_start = re.compile(r"\[Triplet\].*statements_to_process\s*=\s*(\d+)") - pat_triplet_done = re.compile( - r"\[Triplet\].*completed,\s*total_triplets\s*=\s*(\d+),\s*total_entities\s*=\s*(\d+)" + pat_triplet_started = re.compile(r"\[Triplet\]\s+Started\s+-\s+statement_id=") + pat_triplet_completed = re.compile( + r"\[Triplet\]\s+Completed\s+-\s+statement_id=[^,]+,\s+triplets=(\d+),\s+entities=(\d+)" ) - pat_temporal_done = re.compile( - r"\[Temporal\].*completed,\s*extracted_valid_ranges\s*=\s*(\d+)" + pat_temporal_completed = re.compile( + r"\[Temporal\]\s+Completed\s+-\s+statement_id=[^,]+,\s+valid_ranges=(\d+)" ) with open(log_path, "r", encoding="utf-8", errors="ignore") as f: for line in f: - # Chunk prompts count (each chunk triggers one statement-extraction prompt render) + # 文本块数量（每个块触发一次陈述提取提示） if pat_chunk_render.search(line): chunk_count += 1 continue - m1 = pat_triplet_start.search(line) - if m1: + # 陈述数量（每个 Triplet Started 代表一个陈述被处理） + if pat_triplet_started.search(line): + statements_count += 1 + continue + + # 三元组完成：[Triplet] Completed - statement_id=xxx, triplets=X, entities=Y + m_triplet = pat_triplet_completed.search(line) + if m_triplet: try: - statements_count += int(m1.group(1)) + triplet_relations_count += int(m_triplet.group(1)) + triplet_entities_count += int(m_triplet.group(2)) except Exception: pass continue - m2 = pat_triplet_done.search(line) - if m2: + # 时间信息完成：[Temporal] Completed - statement_id=xxx, valid_ranges=X + m_temporal = pat_temporal_completed.search(line) + if m_temporal: try: - triplet_relations_count += int(m2.group(1)) - triplet_entities_count += int(m2.group(2)) - except Exception: - pass - continue - - m3 = pat_temporal_done.search(line) - if m3: - try: - temporal_count += int(m3.group(1)) + temporal_count += int(m_temporal.group(1)) except Exception: pass continue @@ -120,15 +122,20 @@ def parse_stats_from_log(log_path: str) -> dict: def get_recent_activity_stats() -> Tuple[dict, str]: - """Get aggregated stats from all prompt logs in logs/. + """Get stats from the latest prompt log file only. Returns (stats_dict, message). """ - all_logs = _get_all_prompt_logs() - # Fallback to recursive search if none found in logs/ - if not all_logs: + # 获取最新的日志文件 + latest_log = _get_latest_prompt_log_path() + + # 如果没有找到，尝试递归搜索 + if not latest_log: all_logs = _get_any_logs_recursive() - if not all_logs: + if all_logs: + latest_log = all_logs[-1] # 取最新的 + + if not latest_log: return ( { "chunk_count": 0, @@ -141,24 +148,13 @@ def get_recent_activity_stats() -> Tuple[dict, str]: "未找到日志文件，请确认已运行过提取流程。", ) - agg = { - "chunk_count": 0, - "statements_count": 0, - "triplet_entities_count": 0, - "triplet_relations_count": 0, - "temporal_count": 0, - } - for path in all_logs: - s = parse_stats_from_log(path) - agg["chunk_count"] += s.get("chunk_count", 0) - agg["statements_count"] += s.get("statements_count", 0) - agg["triplet_entities_count"] += s.get("triplet_entities_count", 0) - agg["triplet_relations_count"] += s.get("triplet_relations_count", 0) - agg["temporal_count"] += s.get("temporal_count", 0) - - # Attach a summary of files combined - agg["log_path"] = f"{len(all_logs)} 个日志文件，最新：{all_logs[-1]}" - return agg, "成功汇总 logs 目录中所有提示日志。" + # 只解析最新的日志文件 + stats = parse_stats_from_log(latest_log) + + # 添加日志文件路径信息 + stats["log_path"] = f"最新：{latest_log}" + + return stats, "成功读取最近一次记忆活动统计。" def _format_summary(stats: dict) -> str: diff --git a/api/app/core/memory/evaluation/__init__.py b/api/app/core/memory/evaluation/__init__.py deleted file mode 100644 index e9d6aa6c..00000000 --- a/api/app/core/memory/evaluation/__init__.py +++ /dev/null @@ -1 +0,0 @@ -"""Evaluation package with dataset-specific pipelines and a unified runner.""" diff --git a/api/app/core/memory/evaluation/benchmark.md b/api/app/core/memory/evaluation/benchmark.md deleted file mode 100644 index 2853b22b..00000000 --- a/api/app/core/memory/evaluation/benchmark.md +++ /dev/null @@ -1,30 +0,0 @@ -⏬数据集下载地址： - Locomo10.json：https://github.com/snap-research/locomo/tree/main/data - LongMemEval_oracle.json：https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned - msc_self_instruct.jsonl:https://huggingface.co/datasets/MemGPT/MSC-Self-Instruct - 上方数据集下载好后全部放入app/core/memory/data文件夹中 - -全流程基准测试运行： - locomo： - python -m app.core.memory.evaluation.run_eval --dataset locomo --sample-size 1 --reset-group --group-id yyw1 --search-type hybrid --search-limit 8 --context-char-budget 12000 --llm-max-tokens 32 - LongMemEval： - python -m app.core.memory.evaluation.run_eval --dataset longmemeval --sample-size 10 --start-index 0 --group-id longmemeval_zh_bak_2 --search-limit 8 --context-char-budget 4000 --search-type hybrid --max-contexts-per-item 2 --reset-group - memsciqa： - python -m app.core.memory.evaluation.run_eval --dataset memsciqa --sample-size 10 --reset-group --group-id group_memsci - -单独检索评估运行命令： - python -m app.core.memory.evaluation.locomo.locomo_test - python -m app.core.memory.evaluation.longmemeval.test_eval - python -m app.core.memory.evaluation.memsciqa.memsciqa-test - 需要先在项目中修改需要检测评估的group_id。 - -参数及解释： - ● --dataset longmemeval - 指定数据集 - ● --sample-size 10 - 评估10个样本 - ● --start-index 0 - 从第0个样本开始 - ● --group-id longmemeval_zh_bak_2 - 使用指定的组ID - ● --search-limit 8 - 检索限制8条 - ● --context-char-budget 4000 - 上下文字符预算4000 - ● --search-type hybrid - 使用混合检索 - ● --max-contexts-per-item 2 - 每个样本最多摄入2个上下文 - ● --reset-group - 运行前清空组数据 \ No newline at end of file diff --git a/api/app/core/memory/evaluation/common/metrics.py b/api/app/core/memory/evaluation/common/metrics.py deleted file mode 100644 index acc27fb9..00000000 --- a/api/app/core/memory/evaluation/common/metrics.py +++ /dev/null @@ -1,100 +0,0 @@ -import math -import re -from typing import List, Dict - - -def _normalize(text: str) -> List[str]: - """Lowercase, strip punctuation, and split into tokens.""" - text = text.lower().strip() - # Python's re doesn't support \p classes; use a simple non-word filter - text = re.sub(r"[^\w\s]", " ", text) - tokens = [t for t in text.split() if t] - return tokens - - -def exact_match(pred: str, ref: str) -> float: - return float(_normalize(pred) == _normalize(ref)) - - -def jaccard(pred: str, ref: str) -> float: - p = set(_normalize(pred)) - r = set(_normalize(ref)) - if not p and not r: - return 1.0 - if not p or not r: - return 0.0 - return len(p & r) / len(p | r) - - -def f1_score(pred: str, ref: str) -> float: - p_tokens = _normalize(pred) - r_tokens = _normalize(ref) - if not p_tokens and not r_tokens: - return 1.0 - if not p_tokens or not r_tokens: - return 0.0 - p_set = set(p_tokens) - r_set = set(r_tokens) - tp = len(p_set & r_set) - precision = tp / len(p_set) if p_set else 0.0 - recall = tp / len(r_set) if r_set else 0.0 - if precision + recall == 0: - return 0.0 - return 2 * precision * recall / (precision + recall) - - -def bleu1(pred: str, ref: str) -> float: - """Unigram BLEU (BLEU-1) with clipping and brevity penalty.""" - p_tokens = _normalize(pred) - r_tokens = _normalize(ref) - if not p_tokens: - return 0.0 - # Clipped count - r_counts: Dict[str, int] = {} - for t in r_tokens: - r_counts[t] = r_counts.get(t, 0) + 1 - clipped = 0 - p_counts: Dict[str, int] = {} - for t in p_tokens: - p_counts[t] = p_counts.get(t, 0) + 1 - for t, c in p_counts.items(): - clipped += min(c, r_counts.get(t, 0)) - precision = clipped / max(len(p_tokens), 1) - # Brevity penalty - ref_len = len(r_tokens) - pred_len = len(p_tokens) - if pred_len > ref_len or pred_len == 0: - bp = 1.0 - else: - bp = math.exp(1 - ref_len / max(pred_len, 1)) - return bp * precision - - -def percentile(values: List[float], p: float) -> float: - if not values: - return 0.0 - vals = sorted(values) - k = (len(vals) - 1) * p - f = math.floor(k) - c = math.ceil(k) - if f == c: - return vals[int(k)] - return vals[f] + (k - f) * (vals[c] - vals[f]) - - -def latency_stats(latencies_ms: List[float]) -> Dict[str, float]: - """Return basic latency stats: mean, p50, p95, iqr (p75-p25).""" - if not latencies_ms: - return {"mean": 0.0, "p50": 0.0, "p95": 0.0, "iqr": 0.0} - p25 = percentile(latencies_ms, 0.25) - p50 = percentile(latencies_ms, 0.50) - p75 = percentile(latencies_ms, 0.75) - p95 = percentile(latencies_ms, 0.95) - mean = sum(latencies_ms) / max(len(latencies_ms), 1) - return {"mean": mean, "p50": p50, "p95": p95, "iqr": p75 - p25} - - -def avg_context_tokens(contexts: List[str]) -> float: - if not contexts: - return 0.0 - return sum(len(_normalize(c)) for c in contexts) / len(contexts) diff --git a/api/app/core/memory/evaluation/dialogue_queries.py b/api/app/core/memory/evaluation/dialogue_queries.py deleted file mode 100644 index fd7fa671..00000000 --- a/api/app/core/memory/evaluation/dialogue_queries.py +++ /dev/null @@ -1,60 +0,0 @@ -""" -Dialogue search queries for evaluation purposes. -This file contains Cypher queries for searching dialogues, entities, and chunks. -Placed in evaluation directory to avoid circular imports with src modules. -""" - -# Entity search queries -SEARCH_ENTITIES_BY_NAME = """ -MATCH (e:Entity) -WHERE e.name = $name -RETURN e -""" - -SEARCH_ENTITIES_BY_NAME_FALLBACK = """ -MATCH (e:Entity) -WHERE e.name CONTAINS $name -RETURN e -""" - -# Chunk search queries -SEARCH_CHUNKS_BY_CONTENT = """ -MATCH (c:Chunk) -WHERE c.content CONTAINS $content -RETURN c -""" - -# Dialogue search queries -SEARCH_DIALOGUE_BY_DIALOG_ID = """ -MATCH (d:Dialogue) -WHERE d.dialog_id = $dialog_id -RETURN d -""" - -SEARCH_DIALOGUES_BY_CONTENT = """ -MATCH (d:Dialogue) -WHERE d.content CONTAINS $q -RETURN d -""" - -DIALOGUE_EMBEDDING_SEARCH = """ -WITH $embedding AS q -MATCH (d:Dialogue) -WHERE d.dialog_embedding IS NOT NULL - AND ($group_id IS NULL OR d.group_id = $group_id) -WITH d, q, d.dialog_embedding AS v -WITH d, - reduce(dot = 0.0, i IN range(0, size(q)-1) | dot + toFloat(q[i]) * toFloat(v[i])) AS dot, - sqrt(reduce(qs = 0.0, i IN range(0, size(q)-1) | qs + toFloat(q[i]) * toFloat(q[i]))) AS qnorm, - sqrt(reduce(vs = 0.0, i IN range(0, size(v)-1) | vs + toFloat(v[i]) * toFloat(v[i]))) AS vnorm -WITH d, CASE WHEN qnorm = 0 OR vnorm = 0 THEN 0.0 ELSE dot / (qnorm * vnorm) END AS score -WHERE score > $threshold -RETURN d.id AS dialog_id, - d.group_id AS group_id, - d.content AS content, - d.created_at AS created_at, - d.expired_at AS expired_at, - score -ORDER BY score DESC -LIMIT $limit -""" diff --git a/api/app/core/memory/evaluation/extraction_utils.py b/api/app/core/memory/evaluation/extraction_utils.py deleted file mode 100644 index 9afa228c..00000000 --- a/api/app/core/memory/evaluation/extraction_utils.py +++ /dev/null @@ -1,341 +0,0 @@ -import asyncio -import json -import os -import re -from datetime import datetime -from typing import Any, Dict, List, Optional - -from app.core.memory.llm_tools.openai_client import LLMClient -from app.core.memory.models.message_models import ( - ConversationContext, - ConversationMessage, - DialogData, -) - -# 使用新的模块化架构 -from app.core.memory.storage_services.extraction_engine.extraction_orchestrator import ( - ExtractionOrchestrator, -) -from app.core.memory.storage_services.extraction_engine.knowledge_extraction.chunk_extraction import ( - DialogueChunker, -) -from app.core.memory.utils.config.definitions import ( - SELECTED_CHUNKER_STRATEGY, - SELECTED_EMBEDDING_ID, -) -from app.core.memory.utils.llm.llm_utils import MemoryClientFactory -from app.db import get_db_context - -# Import from database module -from app.repositories.neo4j.graph_saver import save_dialog_and_statements_to_neo4j -from app.repositories.neo4j.neo4j_connector import Neo4jConnector - -# Cypher queries for evaluation -# Note: Entity, chunk, and dialogue search queries have been moved to evaluation/dialogue_queries.py - - -async def ingest_contexts_via_full_pipeline( - contexts: List[str], - group_id: str, - chunker_strategy: str | None = None, - embedding_name: str | None = None, - save_chunk_output: bool = False, - save_chunk_output_path: str | None = None, -) -> bool: - """DEPRECATED: 此函数使用旧的流水线架构，建议使用新的 ExtractionOrchestrator - - Run the full extraction pipeline on provided dialogue contexts and save to Neo4j. - This function mirrors the steps in main(), but starts from raw text contexts. - Args: - contexts: List of dialogue texts, each containing lines like "role: message". - group_id: Group ID to assign to generated DialogData and graph nodes. - chunker_strategy: Optional chunker strategy; defaults to SELECTED_CHUNKER_STRATEGY. - embedding_name: Optional embedding model ID; defaults to SELECTED_EMBEDDING_ID. - save_chunk_output: If True, write chunked DialogData list to a JSON file for debugging. - save_chunk_output_path: Optional output path; defaults to src/chunker_test_output.txt. - Returns: - True if data saved successfully, False otherwise. - """ - chunker_strategy = chunker_strategy or SELECTED_CHUNKER_STRATEGY - embedding_name = embedding_name or SELECTED_EMBEDDING_ID - - # Initialize llm client with graceful fallback - llm_client = None - llm_available = True - try: - from app.core.memory.utils.config import definitions as config_defs - with get_db_context() as db: - factory = MemoryClientFactory(db) - llm_client = factory.get_llm_client(config_defs.SELECTED_LLM_ID) - except Exception as e: - print(f"[Ingestion] LLM client unavailable, will skip LLM-dependent steps: {e}") - llm_available = False - - # Step A: Build DialogData list from contexts with robust parsing - chunker = DialogueChunker(chunker_strategy) - dialog_data_list: List[DialogData] = [] - - for idx, ctx in enumerate(contexts): - messages: List[ConversationMessage] = [] - - # Improved parsing: capture multi-line message blocks, normalize roles - pattern = r"^\s*(用户|AI|assistant|user)\s*[：:]\s*(.+?)(?=\n\s*(?:用户|AI|assistant|user)\s*[：:]|\Z)" - matches = list(re.finditer(pattern, ctx, flags=re.MULTILINE | re.DOTALL)) - - if matches: - for m in matches: - raw_role = m.group(1).strip() - content = m.group(2).strip() - norm_role = "AI" if raw_role.lower() in ("ai", "assistant") else "用户" - messages.append(ConversationMessage(role=norm_role, msg=content)) - else: - # Fallback: line-by-line parsing - for raw in ctx.split("\n"): - line = raw.strip() - if not line: - continue - m = re.match(r'^\s*([^:：]+)\s*[：:]\s*(.+)$', line) - if m: - role = m.group(1).strip() - msg = m.group(2).strip() - norm_role = "AI" if role.lower() in ("ai", "assistant") else "用户" - messages.append(ConversationMessage(role=norm_role, msg=msg)) - else: - # Final fallback: treat as user message - default_role = "AI" if re.match(r'^\s*(assistant|AI)\b', line, flags=re.IGNORECASE) else "用户" - messages.append(ConversationMessage(role=default_role, msg=line)) - - context_model = ConversationContext(msgs=messages) - dialog = DialogData( - context=context_model, - ref_id=f"pipeline_item_{idx}", - group_id=group_id, - user_id="default_user", - apply_id="default_application", - ) - # Generate chunks - dialog.chunks = await chunker.process_dialogue(dialog) - dialog_data_list.append(dialog) - - if not dialog_data_list: - print("No dialogs to process for ingestion.") - return False - - # Optionally save chunking outputs for debugging - if save_chunk_output: - try: - def _serialize_datetime(obj): - if isinstance(obj, datetime): - return obj.isoformat() - raise TypeError(f"Object of type {obj.__class__.__name__} is not JSON serializable") - - from app.core.config import settings - settings.ensure_memory_output_dir() - default_path = settings.get_memory_output_path("chunker_test_output.txt") - out_path = save_chunk_output_path or default_path - - combined_output = [dd.model_dump() for dd in dialog_data_list] - with open(out_path, "w", encoding="utf-8") as f: - json.dump(combined_output, f, ensure_ascii=False, indent=4, default=_serialize_datetime) - print(f"Saved chunking results to: {out_path}") - except Exception as e: - print(f"Failed to save chunking results: {e}") - - # Step B-G: 使用新的 ExtractionOrchestrator 执行完整的提取流水线 - if not llm_available: - print("[Ingestion] Skipping extraction pipeline (no LLM).") - return False - - # 初始化 embedder 客户端 - from app.core.memory.llm_tools.openai_embedder import OpenAIEmbedderClient - from app.core.models.base import RedBearModelConfig - from app.services.memory_config_service import MemoryConfigService - - try: - with get_db_context() as db: - embedder_config_dict = MemoryConfigService(db).get_embedder_config(embedding_name or SELECTED_EMBEDDING_ID) - embedder_config = RedBearModelConfig(**embedder_config_dict) - embedder_client = OpenAIEmbedderClient(embedder_config) - except Exception as e: - print(f"[Ingestion] Failed to initialize embedder client: {e}") - print("[Ingestion] Skipping extraction pipeline (embedder initialization failed).") - return False - - connector = Neo4jConnector() - - # 初始化并运行 ExtractionOrchestrator - from app.core.memory.utils.config.config_utils import get_pipeline_config - config = get_pipeline_config() - - orchestrator = ExtractionOrchestrator( - llm_client=llm_client, - embedder_client=embedder_client, - connector=connector, - config=config, - ) - - # 创建一个包装的 orchestrator 来修复时间提取器的输出 - # 保存原始的 _assign_extracted_data 方法 - original_assign = orchestrator._assign_extracted_data - - def clean_temporal_value(value): - """清理 temporal_validity 字段的值，将无效值转换为 None""" - if value is None: - return None - if isinstance(value, str): - # 处理字符串形式的 'null', 'None', 空字符串等 - if value.lower() in ('null', 'none', '') or value.strip() == '': - return None - return value - - async def patched_assign_extracted_data(*args, **kwargs): - """包装方法：在赋值后清理 temporal_validity 中的无效字符串""" - result = await original_assign(*args, **kwargs) - - # 清理返回的 dialog_data_list 中的 temporal_validity - for dialog in result: - if hasattr(dialog, 'chunks') and dialog.chunks: - for chunk in dialog.chunks: - if hasattr(chunk, 'statements') and chunk.statements: - for statement in chunk.statements: - if hasattr(statement, 'temporal_validity') and statement.temporal_validity: - tv = statement.temporal_validity - # 清理 valid_at 和 invalid_at - if hasattr(tv, 'valid_at'): - tv.valid_at = clean_temporal_value(tv.valid_at) - if hasattr(tv, 'invalid_at'): - tv.invalid_at = clean_temporal_value(tv.invalid_at) - return result - - # 替换方法 - orchestrator._assign_extracted_data = patched_assign_extracted_data - - # 同时包装 _create_nodes_and_edges 方法，在创建节点前再次清理 - original_create = orchestrator._create_nodes_and_edges - - async def patched_create_nodes_and_edges(dialog_data_list_arg): - """包装方法：在创建节点前再次清理 temporal_validity""" - # 最后一次清理，确保万无一失 - for dialog in dialog_data_list_arg: - if hasattr(dialog, 'chunks') and dialog.chunks: - for chunk in dialog.chunks: - if hasattr(chunk, 'statements') and chunk.statements: - for statement in chunk.statements: - if hasattr(statement, 'temporal_validity') and statement.temporal_validity: - tv = statement.temporal_validity - if hasattr(tv, 'valid_at'): - tv.valid_at = clean_temporal_value(tv.valid_at) - if hasattr(tv, 'invalid_at'): - tv.invalid_at = clean_temporal_value(tv.invalid_at) - - return await original_create(dialog_data_list_arg) - - orchestrator._create_nodes_and_edges = patched_create_nodes_and_edges - - # 运行完整的提取流水线 - # orchestrator.run 返回 7 个元素的元组 - result = await orchestrator.run(dialog_data_list, is_pilot_run=False) - ( - dialogue_nodes, - chunk_nodes, - statement_nodes, - entity_nodes, - statement_chunk_edges, - statement_entity_edges, - entity_entity_edges, - ) = result - - # statement_chunk_edges 已经由 orchestrator 创建，无需重复创建 - - # Step G: 生成记忆摘要 - print("[Ingestion] Generating memory summaries...") - try: - from app.core.memory.storage_services.extraction_engine.knowledge_extraction.memory_summary import ( - memory_summary_generation, - ) - from app.repositories.neo4j.add_edges import add_memory_summary_statement_edges - from app.repositories.neo4j.add_nodes import add_memory_summary_nodes - - summaries = await memory_summary_generation( - chunked_dialogs=dialog_data_list, - llm_client=llm_client, - embedder_client=embedder_client - ) - print(f"[Ingestion] Generated {len(summaries)} memory summaries") - except Exception as e: - print(f"[Ingestion] Warning: Failed to generate memory summaries: {e}") - summaries = [] - - # Step H: Save to Neo4j - try: - success = await save_dialog_and_statements_to_neo4j( - dialogue_nodes=dialogue_nodes, - chunk_nodes=chunk_nodes, - statement_nodes=statement_nodes, - entity_nodes=entity_nodes, - entity_edges=entity_entity_edges, - statement_chunk_edges=statement_chunk_edges, - statement_entity_edges=statement_entity_edges, - connector=connector - ) - - # Save memory summaries separately - if summaries: - try: - await add_memory_summary_nodes(summaries, connector) - await add_memory_summary_statement_edges(summaries, connector) - print(f"Successfully saved {len(summaries)} memory summary nodes to Neo4j") - except Exception as e: - print(f"Warning: Failed to save summary nodes: {e}") - - await connector.close() - if success: - print("Successfully saved extracted data to Neo4j!") - else: - print("Failed to save data to Neo4j") - return success - except Exception as e: - print(f"Failed to save data to Neo4j: {e}") - return False - - -async def handle_context_processing(args): - """Handle context-based processing from command line arguments.""" - contexts = [] - - if args.contexts: - contexts.extend(args.contexts) - - if args.context_file: - try: - with open(args.context_file, 'r', encoding='utf-8') as f: - contexts.extend(line.strip() for line in f if line.strip()) - except Exception as e: - print(f"Error reading context file: {e}") - return False - - if not contexts: - print("No contexts provided for processing.") - return False - - return await main_from_contexts(contexts, args.context_group_id) - - -async def main_from_contexts(contexts: List[str], group_id: str): - """Run the pipeline from provided dialogue contexts instead of test data.""" - print("=== Running pipeline from provided contexts ===") - - success = await ingest_contexts_via_full_pipeline( - contexts=contexts, - group_id=group_id, - chunker_strategy=SELECTED_CHUNKER_STRATEGY, - embedding_name=SELECTED_EMBEDDING_ID, - save_chunk_output=True - ) - - if success: - print("Successfully processed and saved contexts to Neo4j!") - else: - print("Failed to process contexts.") - - return success diff --git a/api/app/core/memory/evaluation/locomo/locomo_benchmark.py b/api/app/core/memory/evaluation/locomo/locomo_benchmark.py deleted file mode 100644 index b7d988c5..00000000 --- a/api/app/core/memory/evaluation/locomo/locomo_benchmark.py +++ /dev/null @@ -1,575 +0,0 @@ -""" -LoCoMo Benchmark Script - -This module provides the main entry point for running LoCoMo benchmark evaluations. -It orchestrates data loading, ingestion, retrieval, LLM inference, and metric calculation -in a clean, maintainable way. - -Usage: - python locomo_benchmark.py --sample_size 20 --search_type hybrid -""" - -import argparse -import asyncio -import json -import os -import time -from datetime import datetime -from typing import Any, Dict, List, Optional - -try: - from dotenv import load_dotenv -except ImportError: - def load_dotenv(): - pass - -from app.core.memory.evaluation.common.metrics import ( - avg_context_tokens, - bleu1, - f1_score, - jaccard, - latency_stats, -) -from app.core.memory.evaluation.locomo.locomo_metrics import ( - get_category_name, - locomo_f1_score, - locomo_multi_f1, -) -from app.core.memory.evaluation.locomo.locomo_utils import ( - extract_conversations, - ingest_conversations_if_needed, - load_locomo_data, - resolve_temporal_references, - retrieve_relevant_information, - select_and_format_information, -) -from app.core.memory.llm_tools.openai_embedder import OpenAIEmbedderClient -from app.core.memory.utils.definitions import ( - PROJECT_ROOT, - SELECTED_EMBEDDING_ID, - SELECTED_GROUP_ID, - SELECTED_LLM_ID, -) -from app.core.memory.utils.llm.llm_utils import MemoryClientFactory -from app.core.models.base import RedBearModelConfig -from app.db import get_db_context -from app.repositories.neo4j.neo4j_connector import Neo4jConnector -from app.services.memory_config_service import MemoryConfigService - - -async def run_locomo_benchmark( - sample_size: int = 20, - group_id: Optional[str] = None, - search_type: str = "hybrid", - search_limit: int = 12, - context_char_budget: int = 8000, - reset_group: bool = False, - skip_ingest: bool = False, - output_dir: Optional[str] = None -) -> Dict[str, Any]: - """ - Run LoCoMo benchmark evaluation. - - This function orchestrates the complete evaluation pipeline: - 1. Load LoCoMo dataset (only QA pairs from first conversation) - 2. Check/ingest conversations into database (only first conversation, unless skip_ingest=True) - 3. For each question: - - Retrieve relevant information - - Generate answer using LLM - - Calculate metrics - 4. Aggregate results and save to file - - Note: By default, only the first conversation is ingested into the database, - and only QA pairs from that conversation are evaluated. This ensures that - all questions have corresponding memory in the database for retrieval. - - Args: - sample_size: Number of QA pairs to evaluate (from first conversation) - group_id: Database group ID for retrieval (uses default if None) - search_type: "keyword", "embedding", or "hybrid" - search_limit: Max documents to retrieve per query - context_char_budget: Max characters for context - reset_group: Whether to clear and re-ingest data (not implemented) - skip_ingest: If True, skip data ingestion and use existing data in Neo4j - output_dir: Directory to save results (uses default if None) - - Returns: - Dictionary with evaluation results including metrics, timing, and samples - """ - # Use default group_id if not provided - group_id = group_id or SELECTED_GROUP_ID - - # Determine data path - data_path = os.path.join(PROJECT_ROOT, "data", "locomo10.json") - if not os.path.exists(data_path): - # Fallback to current directory - data_path = os.path.join(os.getcwd(), "data", "locomo10.json") - - print(f"\n{'='*60}") - print("🚀 Starting LoCoMo Benchmark Evaluation") - print(f"{'='*60}") - print("📊 Configuration:") - print(f" Sample size: {sample_size}") - print(f" Group ID: {group_id}") - print(f" Search type: {search_type}") - print(f" Search limit: {search_limit}") - print(f" Context budget: {context_char_budget} chars") - print(f" Data path: {data_path}") - print(f"{'='*60}\n") - - # Step 1: Load LoCoMo data - print("📂 Loading LoCoMo dataset...") - try: - # Only load QA pairs from the first conversation (index 0) - # since we only ingest the first conversation into the database - qa_items = load_locomo_data(data_path, sample_size, conversation_index=0) - print(f"✅ Loaded {len(qa_items)} QA pairs from conversation 0\n") - except Exception as e: - print(f"❌ Failed to load data: {e}") - return { - "error": f"Data loading failed: {e}", - "timestamp": datetime.now().isoformat() - } - - # Step 2: Extract conversations and ingest if needed - if skip_ingest: - print("⏭️ Skipping data ingestion (using existing data in Neo4j)") - print(f" Group ID: {group_id}\n") - else: - print("💾 Checking database ingestion...") - try: - conversations = extract_conversations(data_path, max_dialogues=1) - print(f"📝 Extracted {len(conversations)} conversations") - - # Always ingest for now (ingestion check not implemented) - print(f"🔄 Ingesting conversations into group '{group_id}'...") - success = await ingest_conversations_if_needed( - conversations=conversations, - group_id=group_id, - reset=reset_group - ) - - if success: - print("✅ Ingestion completed successfully\n") - else: - print("⚠️ Ingestion may have failed, continuing anyway\n") - - except Exception as e: - print(f"❌ Ingestion failed: {e}") - print("⚠️ Continuing with evaluation (database may be empty)\n") - - # Step 3: Initialize clients - print("🔧 Initializing clients...") - connector = Neo4jConnector() - - # Initialize LLM client with database context - with get_db_context() as db: - factory = MemoryClientFactory(db) - llm_client = factory.get_llm_client(SELECTED_LLM_ID) - - # Initialize embedder - with get_db_context() as db: - config_service = MemoryConfigService(db) - cfg_dict = config_service.get_embedder_config(SELECTED_EMBEDDING_ID) - embedder = OpenAIEmbedderClient( - model_config=RedBearModelConfig.model_validate(cfg_dict) - ) - print("✅ Clients initialized\n") - - # Step 4: Process questions - print(f"🔍 Processing {len(qa_items)} questions...") - print(f"{'='*60}\n") - - # Tracking variables - latencies_search: List[float] = [] - latencies_llm: List[float] = [] - context_counts: List[int] = [] - context_chars: List[int] = [] - context_tokens: List[int] = [] - - # Metric lists - f1_scores: List[float] = [] - bleu1_scores: List[float] = [] - jaccard_scores: List[float] = [] - locomo_f1_scores: List[float] = [] - - # Per-category tracking - category_counts: Dict[str, int] = {} - category_f1: Dict[str, List[float]] = {} - category_bleu1: Dict[str, List[float]] = {} - category_jaccard: Dict[str, List[float]] = {} - category_locomo_f1: Dict[str, List[float]] = {} - - # Detailed samples - samples: List[Dict[str, Any]] = [] - - # Fixed anchor date for temporal resolution - anchor_date = datetime(2023, 5, 8) - - try: - for idx, item in enumerate(qa_items, 1): - question = item.get("question", "") - ground_truth = item.get("answer", "") - category = get_category_name(item) - - # Ensure ground truth is a string - ground_truth_str = str(ground_truth) if ground_truth is not None else "" - - print(f"[{idx}/{len(qa_items)}] Category: {category}") - print(f"❓ Question: {question}") - print(f"✅ Ground Truth: {ground_truth_str}") - - # Step 4a: Retrieve relevant information - t_search_start = time.time() - try: - retrieved_info = await retrieve_relevant_information( - question=question, - group_id=group_id, - search_type=search_type, - search_limit=search_limit, - connector=connector, - embedder=embedder - ) - t_search_end = time.time() - search_latency = (t_search_end - t_search_start) * 1000 - latencies_search.append(search_latency) - - print(f"🔍 Retrieved {len(retrieved_info)} documents ({search_latency:.1f}ms)") - - except Exception as e: - print(f"❌ Retrieval failed: {e}") - retrieved_info = [] - search_latency = 0.0 - latencies_search.append(search_latency) - - # Step 4b: Select and format context - context_text = select_and_format_information( - retrieved_info=retrieved_info, - question=question, - max_chars=context_char_budget - ) - - # Resolve temporal references - context_text = resolve_temporal_references(context_text, anchor_date) - - # Add reference date to context - if context_text: - context_text = f"Reference date: {anchor_date.date().isoformat()}\n\n{context_text}" - else: - context_text = "No relevant context found." - - # Track context statistics - context_counts.append(len(retrieved_info)) - context_chars.append(len(context_text)) - context_tokens.append(len(context_text.split())) - - print(f"📝 Context: {len(context_text)} chars, {len(retrieved_info)} docs") - - # Step 4c: Generate answer with LLM - messages = [ - { - "role": "system", - "content": ( - "You are a precise QA assistant. Answer following these rules:\n" - "1) Extract the EXACT information mentioned in the context\n" - "2) For time questions: calculate actual dates from relative times\n" - "3) Return ONLY the answer text in simplest form\n" - "4) For dates, use format 'DD Month YYYY' (e.g., '7 May 2023')\n" - "5) If no clear answer found, respond with 'Unknown'" - ) - }, - { - "role": "user", - "content": f"Question: {question}\n\nContext:\n{context_text}" - } - ] - - t_llm_start = time.time() - try: - response = await llm_client.chat(messages=messages) - t_llm_end = time.time() - llm_latency = (t_llm_end - t_llm_start) * 1000 - latencies_llm.append(llm_latency) - - # Extract prediction from response - if hasattr(response, 'content'): - prediction = response.content.strip() - elif isinstance(response, dict): - prediction = response["choices"][0]["message"]["content"].strip() - else: - prediction = "Unknown" - - print(f"🤖 Prediction: {prediction} ({llm_latency:.1f}ms)") - - except Exception as e: - print(f"❌ LLM failed: {e}") - prediction = "Unknown" - llm_latency = 0.0 - latencies_llm.append(llm_latency) - - # Step 4d: Calculate metrics - f1_val = f1_score(prediction, ground_truth_str) - bleu1_val = bleu1(prediction, ground_truth_str) - jaccard_val = jaccard(prediction, ground_truth_str) - - # LoCoMo-specific F1: use multi-answer for category 1 (Multi-Hop) - if item.get("category") == 1: - locomo_f1_val = locomo_multi_f1(prediction, ground_truth_str) - else: - locomo_f1_val = locomo_f1_score(prediction, ground_truth_str) - - # Accumulate metrics - f1_scores.append(f1_val) - bleu1_scores.append(bleu1_val) - jaccard_scores.append(jaccard_val) - locomo_f1_scores.append(locomo_f1_val) - - # Track by category - category_counts[category] = category_counts.get(category, 0) + 1 - category_f1.setdefault(category, []).append(f1_val) - category_bleu1.setdefault(category, []).append(bleu1_val) - category_jaccard.setdefault(category, []).append(jaccard_val) - category_locomo_f1.setdefault(category, []).append(locomo_f1_val) - - print(f"📊 Metrics - F1: {f1_val:.3f}, BLEU-1: {bleu1_val:.3f}, " - f"Jaccard: {jaccard_val:.3f}, LoCoMo F1: {locomo_f1_val:.3f}") - print() - - # Save sample details - samples.append({ - "question": question, - "ground_truth": ground_truth_str, - "prediction": prediction, - "category": category, - "metrics": { - "f1": f1_val, - "bleu1": bleu1_val, - "jaccard": jaccard_val, - "locomo_f1": locomo_f1_val - }, - "retrieval": { - "num_docs": len(retrieved_info), - "context_length": len(context_text) - }, - "timing": { - "search_ms": search_latency, - "llm_ms": llm_latency - } - }) - - finally: - # Close connector - await connector.close() - - # Step 5: Aggregate results - print(f"\n{'='*60}") - print("📊 Aggregating Results") - print(f"{'='*60}\n") - - # Overall metrics - overall_metrics = { - "f1": sum(f1_scores) / max(len(f1_scores), 1) if f1_scores else 0.0, - "bleu1": sum(bleu1_scores) / max(len(bleu1_scores), 1) if bleu1_scores else 0.0, - "jaccard": sum(jaccard_scores) / max(len(jaccard_scores), 1) if jaccard_scores else 0.0, - "locomo_f1": sum(locomo_f1_scores) / max(len(locomo_f1_scores), 1) if locomo_f1_scores else 0.0 - } - - # Per-category metrics - by_category: Dict[str, Dict[str, Any]] = {} - for cat in category_counts: - f1_list = category_f1.get(cat, []) - b1_list = category_bleu1.get(cat, []) - j_list = category_jaccard.get(cat, []) - lf_list = category_locomo_f1.get(cat, []) - - by_category[cat] = { - "count": category_counts[cat], - "f1": sum(f1_list) / max(len(f1_list), 1) if f1_list else 0.0, - "bleu1": sum(b1_list) / max(len(b1_list), 1) if b1_list else 0.0, - "jaccard": sum(j_list) / max(len(j_list), 1) if j_list else 0.0, - "locomo_f1": sum(lf_list) / max(len(lf_list), 1) if lf_list else 0.0 - } - - # Latency statistics - latency = { - "search": latency_stats(latencies_search), - "llm": latency_stats(latencies_llm) - } - - # Context statistics - context_stats = { - "avg_retrieved_docs": sum(context_counts) / max(len(context_counts), 1) if context_counts else 0.0, - "avg_context_chars": sum(context_chars) / max(len(context_chars), 1) if context_chars else 0.0, - "avg_context_tokens": sum(context_tokens) / max(len(context_tokens), 1) if context_tokens else 0.0 - } - - # Build result dictionary - result = { - "dataset": "locomo", - "sample_size": len(qa_items), - "timestamp": datetime.now().isoformat(), - "params": { - "group_id": group_id, - "search_type": search_type, - "search_limit": search_limit, - "context_char_budget": context_char_budget, - "llm_id": SELECTED_LLM_ID, - "embedding_id": SELECTED_EMBEDDING_ID - }, - "overall_metrics": overall_metrics, - "by_category": by_category, - "latency": latency, - "context_stats": context_stats, - "samples": samples - } - - # Step 6: Save results - if output_dir is None: - output_dir = os.path.join( - os.path.dirname(__file__), - "results" - ) - - os.makedirs(output_dir, exist_ok=True) - - # Generate timestamped filename - timestamp_str = datetime.now().strftime("%Y%m%d_%H%M%S") - output_path = os.path.join(output_dir, f"locomo_{timestamp_str}.json") - - try: - with open(output_path, "w", encoding="utf-8") as f: - json.dump(result, f, ensure_ascii=False, indent=2) - print(f"✅ Results saved to: {output_path}\n") - except Exception as e: - print(f"❌ Failed to save results: {e}") - print("📊 Printing results to console instead:\n") - print(json.dumps(result, ensure_ascii=False, indent=2)) - - return result - - -def main(): - """ - Parse command-line arguments and run benchmark. - - This function provides a CLI interface for running LoCoMo benchmarks - with configurable parameters. - """ - parser = argparse.ArgumentParser( - description="Run LoCoMo benchmark evaluation", - formatter_class=argparse.ArgumentDefaultsHelpFormatter - ) - - parser.add_argument( - "--sample_size", - type=int, - default=20, - help="Number of QA pairs to evaluate" - ) - parser.add_argument( - "--group_id", - type=str, - default=None, - help="Database group ID for retrieval (uses default if not specified)" - ) - parser.add_argument( - "--search_type", - type=str, - default="hybrid", - choices=["keyword", "embedding", "hybrid"], - help="Search strategy to use" - ) - parser.add_argument( - "--search_limit", - type=int, - default=12, - help="Maximum number of documents to retrieve per query" - ) - parser.add_argument( - "--context_char_budget", - type=int, - default=8000, - help="Maximum characters for context" - ) - parser.add_argument( - "--reset_group", - action="store_true", - help="Clear and re-ingest data (not implemented)" - ) - parser.add_argument( - "--skip_ingest", - action="store_true", - help="Skip data ingestion and use existing data in Neo4j" - ) - parser.add_argument( - "--output_dir", - type=str, - default=None, - help="Directory to save results (uses default if not specified)" - ) - - args = parser.parse_args() - - # Load environment variables - load_dotenv() - - # Run benchmark - result = asyncio.run(run_locomo_benchmark( - sample_size=args.sample_size, - group_id=args.group_id, - search_type=args.search_type, - search_limit=args.search_limit, - context_char_budget=args.context_char_budget, - reset_group=args.reset_group, - skip_ingest=args.skip_ingest, - output_dir=args.output_dir - )) - - # Print summary - print(f"\n{'='*60}") - - # Check if there was an error - if 'error' in result: - print("❌ Benchmark Failed!") - print(f"{'='*60}") - print(f"Error: {result['error']}") - return - - print("🎉 Benchmark Complete!") - print(f"{'='*60}") - print("📊 Final Results:") - print(f" Sample size: {result.get('sample_size', 0)}") - print(f" F1: {result['overall_metrics']['f1']:.3f}") - print(f" BLEU-1: {result['overall_metrics']['bleu1']:.3f}") - print(f" Jaccard: {result['overall_metrics']['jaccard']:.3f}") - print(f" LoCoMo F1: {result['overall_metrics']['locomo_f1']:.3f}") - - if result.get('context_stats'): - print("\n📈 Context Statistics:") - print(f" Avg retrieved docs: {result['context_stats']['avg_retrieved_docs']:.1f}") - print(f" Avg context chars: {result['context_stats']['avg_context_chars']:.0f}") - print(f" Avg context tokens: {result['context_stats']['avg_context_tokens']:.0f}") - - if result.get('latency'): - print("\n⏱️ Latency Statistics:") - print(f" Search - Mean: {result['latency']['search']['mean']:.1f}ms, " - f"P50: {result['latency']['search']['p50']:.1f}ms, " - f"P95: {result['latency']['search']['p95']:.1f}ms") - print(f" LLM - Mean: {result['latency']['llm']['mean']:.1f}ms, " - f"P50: {result['latency']['llm']['p50']:.1f}ms, " - f"P95: {result['latency']['llm']['p95']:.1f}ms") - - if result.get('by_category'): - print("\n📂 Results by Category:") - for cat, metrics in result['by_category'].items(): - print(f" {cat}:") - print(f" Count: {metrics['count']}") - print(f" F1: {metrics['f1']:.3f}") - print(f" LoCoMo F1: {metrics['locomo_f1']:.3f}") - print(f" Jaccard: {metrics['jaccard']:.3f}") - - print(f"\n{'='*60}\n") - - -if __name__ == "__main__": - main() diff --git a/api/app/core/memory/evaluation/locomo/locomo_metrics.py b/api/app/core/memory/evaluation/locomo/locomo_metrics.py deleted file mode 100644 index 20d5f2b5..00000000 --- a/api/app/core/memory/evaluation/locomo/locomo_metrics.py +++ /dev/null @@ -1,225 +0,0 @@ -""" -LoCoMo-specific metric calculations. - -This module provides clean, simplified implementations of metrics used for -LoCoMo benchmark evaluation, including text normalization and F1 score variants. -""" - -import re -from typing import Dict, Any - - -def normalize_text(text: str) -> str: - """ - Normalize text for LoCoMo evaluation. - - Normalization steps: - - Convert to lowercase - - Remove commas - - Remove stop words (a, an, the, and) - - Remove punctuation - - Normalize whitespace - - Args: - text: Input text to normalize - - Returns: - Normalized text string with consistent formatting - - Examples: - >>> normalize_text("The cat, and the dog") - 'cat dog' - >>> normalize_text("Hello, World!") - 'hello world' - """ - # Ensure input is a string - text = str(text) if text is not None else "" - - # Convert to lowercase - text = text.lower() - - # Remove commas - text = re.sub(r"[\,]", " ", text) - - # Remove stop words - text = re.sub(r"\b(a|an|the|and)\b", " ", text) - - # Remove punctuation (keep only word characters and whitespace) - text = re.sub(r"[^\w\s]", " ", text) - - # Normalize whitespace (collapse multiple spaces to single space) - text = " ".join(text.split()) - - return text - - -def locomo_f1_score(prediction: str, ground_truth: str) -> float: - """ - Calculate LoCoMo F1 score for single-answer questions. - - Uses token-level precision and recall based on normalized text. - Treats tokens as sets (no duplicate counting). - - Args: - prediction: Model's predicted answer - ground_truth: Correct answer - - Returns: - F1 score between 0.0 and 1.0 - - Examples: - >>> locomo_f1_score("Paris", "Paris") - 1.0 - >>> locomo_f1_score("The cat", "cat") - 1.0 - >>> locomo_f1_score("dog", "cat") - 0.0 - """ - # Ensure inputs are strings - pred_str = str(prediction) if prediction is not None else "" - truth_str = str(ground_truth) if ground_truth is not None else "" - - # Normalize and tokenize - pred_tokens = normalize_text(pred_str).split() - truth_tokens = normalize_text(truth_str).split() - - # Handle empty cases - if not pred_tokens or not truth_tokens: - return 0.0 - - # Convert to sets for comparison - pred_set = set(pred_tokens) - truth_set = set(truth_tokens) - - # Calculate true positives (intersection) - true_positives = len(pred_set & truth_set) - - # Calculate precision and recall - precision = true_positives / len(pred_set) if pred_set else 0.0 - recall = true_positives / len(truth_set) if truth_set else 0.0 - - # Calculate F1 score - if precision + recall == 0: - return 0.0 - - f1 = 2 * precision * recall / (precision + recall) - return f1 - - -def locomo_multi_f1(prediction: str, ground_truth: str) -> float: - """ - Calculate LoCoMo F1 score for multi-answer questions. - - Handles comma-separated answers by: - 1. Splitting both prediction and ground truth by commas - 2. For each ground truth answer, finding the best matching prediction - 3. Averaging the F1 scores across all ground truth answers - - Args: - prediction: Model's predicted answer (may contain multiple comma-separated answers) - ground_truth: Correct answer (may contain multiple comma-separated answers) - - Returns: - Average F1 score across all ground truth answers (0.0 to 1.0) - - Examples: - >>> locomo_multi_f1("Paris, London", "Paris, London") - 1.0 - >>> locomo_multi_f1("Paris", "Paris, London") - 0.5 - >>> locomo_multi_f1("Paris, Berlin", "Paris, London") - 0.5 - """ - # Ensure inputs are strings - pred_str = str(prediction) if prediction is not None else "" - truth_str = str(ground_truth) if ground_truth is not None else "" - - # Split by commas and strip whitespace - predictions = [p.strip() for p in pred_str.split(',') if p.strip()] - ground_truths = [g.strip() for g in truth_str.split(',') if g.strip()] - - # Handle empty cases - if not predictions or not ground_truths: - return 0.0 - - # For each ground truth, find the best matching prediction - f1_scores = [] - for gt in ground_truths: - # Calculate F1 with each prediction and take the maximum - best_f1 = max(locomo_f1_score(pred, gt) for pred in predictions) - f1_scores.append(best_f1) - - # Return average F1 across all ground truths - return sum(f1_scores) / len(f1_scores) - - -def get_category_name(item: Dict[str, Any]) -> str: - """ - Extract and normalize category name from QA item. - - Handles both numeric categories (1-4) and string categories with various formats. - Supports multiple field names: "cat", "category", "type". - - Category mapping: - - 1 or "multi-hop" -> "Multi-Hop" - - 2 or "temporal" -> "Temporal" - - 3 or "open domain" -> "Open Domain" - - 4 or "single-hop" -> "Single-Hop" - - Args: - item: QA item dictionary containing category information - - Returns: - Standardized category name or "unknown" if not found - - Examples: - >>> get_category_name({"category": 1}) - 'Multi-Hop' - >>> get_category_name({"cat": "temporal"}) - 'Temporal' - >>> get_category_name({"type": "Single-Hop"}) - 'Single-Hop' - """ - # Numeric category mapping - CATEGORY_MAP = { - 1: "Multi-Hop", - 2: "Temporal", - 3: "Open Domain", - 4: "Single-Hop", - } - - # String category aliases (case-insensitive) - TYPE_ALIASES = { - "single-hop": "Single-Hop", - "singlehop": "Single-Hop", - "single hop": "Single-Hop", - "multi-hop": "Multi-Hop", - "multihop": "Multi-Hop", - "multi hop": "Multi-Hop", - "open domain": "Open Domain", - "opendomain": "Open Domain", - "temporal": "Temporal", - } - - # Try "cat" field first (string category) - cat = item.get("cat") - if isinstance(cat, str) and cat.strip(): - name = cat.strip() - lower = name.lower() - return TYPE_ALIASES.get(lower, name) - - # Try "category" field (can be int or string) - cat_num = item.get("category") - if isinstance(cat_num, int): - return CATEGORY_MAP.get(cat_num, "unknown") - elif isinstance(cat_num, str) and cat_num.strip(): - lower = cat_num.strip().lower() - return TYPE_ALIASES.get(lower, cat_num.strip()) - - # Try "type" field as fallback - cat_type = item.get("type") - if isinstance(cat_type, str) and cat_type.strip(): - lower = cat_type.strip().lower() - return TYPE_ALIASES.get(lower, cat_type.strip()) - - return "unknown" diff --git a/api/app/core/memory/evaluation/locomo/locomo_test.py b/api/app/core/memory/evaluation/locomo/locomo_test.py deleted file mode 100644 index b5ad5820..00000000 --- a/api/app/core/memory/evaluation/locomo/locomo_test.py +++ /dev/null @@ -1,810 +0,0 @@ -# file name: check_neo4j_connection_fixed.py -import asyncio -import json -import math -import os -import re -import sys -import time -from datetime import datetime, timedelta -from typing import Any, Dict, List - -from dotenv import load_dotenv - -# 1 -# 添加项目根目录到路径 -current_dir = os.path.dirname(os.path.abspath(__file__)) -project_root = os.path.dirname(current_dir) -if project_root not in sys.path: - sys.path.insert(0, project_root) -# 关键：将 src 目录置于最前，确保从当前仓库加载模块 -src_dir = os.path.join(project_root, "src") -if src_dir not in sys.path: - sys.path.insert(0, src_dir) - -load_dotenv() - -# 首先定义 _loc_normalize 函数，因为其他函数依赖它 -def _loc_normalize(text: str) -> str: - text = str(text) if text is not None else "" - text = text.lower() - text = re.sub(r"[\,]", " ", text) - text = re.sub(r"\b(a|an|the|and)\b", " ", text) - text = re.sub(r"[^\w\s]", " ", text) - text = " ".join(text.split()) - return text - -# 尝试从 metrics.py 导入基础指标 -try: - from common.metrics import bleu1, f1_score, jaccard - print("✅ 从 metrics.py 导入基础指标成功") -except ImportError as e: - print(f"❌ 从 metrics.py 导入失败: {e}") - # 回退到本地实现 - def f1_score(pred: str, ref: str) -> float: - pred_str = str(pred) if pred is not None else "" - ref_str = str(ref) if ref is not None else "" - - p_tokens = _loc_normalize(pred_str).split() - r_tokens = _loc_normalize(ref_str).split() - if not p_tokens and not r_tokens: - return 1.0 - if not p_tokens or not r_tokens: - return 0.0 - p_set = set(p_tokens) - r_set = set(r_tokens) - tp = len(p_set & r_set) - precision = tp / len(p_set) if p_set else 0.0 - recall = tp / len(r_set) if r_set else 0.0 - if precision + recall == 0: - return 0.0 - return 2 * precision * recall / (precision + recall) - - def bleu1(pred: str, ref: str) -> float: - pred_str = str(pred) if pred is not None else "" - ref_str = str(ref) if ref is not None else "" - - p_tokens = _loc_normalize(pred_str).split() - r_tokens = _loc_normalize(ref_str).split() - if not p_tokens: - return 0.0 - - r_counts = {} - for t in r_tokens: - r_counts[t] = r_counts.get(t, 0) + 1 - - clipped = 0 - p_counts = {} - for t in p_tokens: - p_counts[t] = p_counts.get(t, 0) + 1 - - for t, c in p_counts.items(): - clipped += min(c, r_counts.get(t, 0)) - - precision = clipped / max(len(p_tokens), 1) - ref_len = len(r_tokens) - pred_len = len(p_tokens) - - if pred_len > ref_len or pred_len == 0: - bp = 1.0 - else: - bp = math.exp(1 - ref_len / max(pred_len, 1)) - - return bp * precision - - def jaccard(pred: str, ref: str) -> float: - pred_str = str(pred) if pred is not None else "" - ref_str = str(ref) if ref is not None else "" - - p = set(_loc_normalize(pred_str).split()) - r = set(_loc_normalize(ref_str).split()) - if not p and not r: - return 1.0 - if not p or not r: - return 0.0 - return len(p & r) / len(p | r) - -# 尝试从 qwen_search_eval.py 导入 LoCoMo 特定指标 -try: - # 添加 evaluation 目录路径 - evaluation_dir = os.path.join(project_root, "evaluation") - if evaluation_dir not in sys.path: - sys.path.insert(0, evaluation_dir) - - # 尝试从不同位置导入 - try: - from locomo.qwen_search_eval import ( - _resolve_relative_times, - loc_f1_score, - loc_multi_f1, - ) - print("✅ 从 locomo.qwen_search_eval 导入 LoCoMo 特定指标成功") - except ImportError: - from qwen_search_eval import _resolve_relative_times, loc_f1_score, loc_multi_f1 - print("✅ 从 qwen_search_eval 导入 LoCoMo 特定指标成功") - -except ImportError as e: - print(f"❌ 从 qwen_search_eval.py 导入失败: {e}") - # 回退到本地实现 LoCoMo 特定函数 - def _resolve_relative_times(text: str, anchor: datetime) -> str: - t = str(text) if text is not None else "" - t = re.sub(r"\btoday\b", anchor.date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\byesterday\b", (anchor - timedelta(days=1)).date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\btomorrow\b", (anchor + timedelta(days=1)).date().isoformat(), t, flags=re.IGNORECASE) - - def _ago_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor - timedelta(days=n)).date().isoformat() - def _in_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor + timedelta(days=n)).date().isoformat() - - t = re.sub(r"\b(\d+)\s+days\s+ago\b", _ago_repl, t, flags=re.IGNORECASE) - t = re.sub(r"\bin\s+(\d+)\s+days\b", _in_repl, t, flags=re.IGNORECASE) - t = re.sub(r"\blast\s+week\b", (anchor - timedelta(days=7)).date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\bnext\s+week\b", (anchor + timedelta(days=7)).date().isoformat(), t, flags=re.IGNORECASE) - return t - - def loc_f1_score(prediction: str, ground_truth: str) -> float: - p_tokens = _loc_normalize(prediction).split() - g_tokens = _loc_normalize(ground_truth).split() - if not p_tokens or not g_tokens: - return 0.0 - p = set(p_tokens) - g = set(g_tokens) - tp = len(p & g) - precision = tp / len(p) if p else 0.0 - recall = tp / len(g) if g else 0.0 - return (2 * precision * recall / (precision + recall)) if (precision + recall) > 0 else 0.0 - - def loc_multi_f1(prediction: str, ground_truth: str) -> float: - predictions = [p.strip() for p in str(prediction).split(',') if p.strip()] - ground_truths = [g.strip() for g in str(ground_truth).split(',') if g.strip()] - if not predictions or not ground_truths: - return 0.0 - def _f1(a: str, b: str) -> float: - return loc_f1_score(a, b) - vals = [] - for gt in ground_truths: - vals.append(max(_f1(pred, gt) for pred in predictions)) - return sum(vals) / len(vals) - - -def smart_context_selection(contexts: List[str], question: str, max_chars: int = 8000) -> str: - """基于问题关键词智能选择上下文""" - if not contexts: - return "" - - # 提取问题关键词（只保留有意义的词） - question_lower = question.lower() - stop_words = {'what', 'when', 'where', 'who', 'why', 'how', 'did', 'do', 'does', 'is', 'are', 'was', 'were', 'the', 'a', 'an', 'and', 'or', 'but'} - question_words = set(re.findall(r'\b\w+\b', question_lower)) - question_words = {word for word in question_words if word not in stop_words and len(word) > 2} - - print(f"🔍 问题关键词: {question_words}") - - # 给每个上下文打分 - scored_contexts = [] - for i, context in enumerate(contexts): - context_lower = context.lower() - score = 0 - - # 关键词匹配得分 - keyword_matches = 0 - for word in question_words: - if word in context_lower: - keyword_matches += 1 - # 关键词出现次数越多，得分越高 - score += context_lower.count(word) * 2 - - # 上下文长度得分（适中的长度更好） - context_len = len(context) - if 100 < context_len < 2000: # 理想长度范围 - score += 5 - elif context_len >= 2000: # 太长可能包含无关信息 - score += 2 - - # 如果是前几个上下文，给予额外分数（通常相关性更高） - if i < 3: - score += 3 - - scored_contexts.append((score, context, keyword_matches)) - - # 按得分排序 - scored_contexts.sort(key=lambda x: x[0], reverse=True) - - # 选择高得分的上下文，直到达到字符限制 - selected = [] - total_chars = 0 - selected_count = 0 - - print("📊 上下文相关性分析:") - for score, context, matches in scored_contexts[:5]: # 只显示前5个 - print(f" - 得分: {score}, 关键词匹配: {matches}, 长度: {len(context)}") - - for score, context, matches in scored_contexts: - if total_chars + len(context) <= max_chars: - selected.append(context) - total_chars += len(context) - selected_count += 1 - else: - # 如果这个上下文得分很高但放不下，尝试截取 - if score > 10 and total_chars < max_chars - 500: - remaining = max_chars - total_chars - # 找到包含关键词的部分 - lines = context.split('\n') - relevant_lines = [] - current_chars = 0 - - for line in lines: - line_lower = line.lower() - line_relevance = any(word in line_lower for word in question_words) - - if line_relevance and current_chars < remaining - 100: - relevant_lines.append(line) - current_chars += len(line) - - if relevant_lines: - truncated = '\n'.join(relevant_lines) - if len(truncated) > 100: # 确保有足够内容 - selected.append(truncated + "\n[相关内容截断...]") - total_chars += len(truncated) - selected_count += 1 - break # 不再尝试添加更多上下文 - - result = "\n\n".join(selected) - print(f"✅ 智能选择: {selected_count}个上下文, 总长度: {total_chars}字符") - return result - - -def get_dynamic_search_params(question: str, question_index: int, total_questions: int): - """根据问题复杂度和进度动态调整检索参数""" - - # 分析问题复杂度 - word_count = len(question.split()) - has_temporal = any(word in question.lower() for word in ['when', 'date', 'time', 'ago']) - has_multi_hop = any(word in question.lower() for word in ['and', 'both', 'also', 'while']) - - # 根据进度调整 - 后期问题可能需要更精确的检索 - progress_factor = question_index / total_questions - - base_limit = 12 - if has_temporal and has_multi_hop: - base_limit = 20 - elif word_count > 8: - base_limit = 16 - - # 随着测试进行，逐渐收紧检索范围 - adjusted_limit = max(8, int(base_limit * (1 - progress_factor * 0.3))) - - # 动态调整最大字符数 - max_chars = 8000 + 4000 * (1 - progress_factor) - - return { - "limit": adjusted_limit, - "max_chars": int(max_chars) - } - - -class EnhancedEvaluationMonitor: - def __init__(self, reset_interval=5, performance_threshold=0.6): - self.question_count = 0 - self.reset_interval = reset_interval - self.performance_threshold = performance_threshold - self.consecutive_low_scores = 0 - self.performance_history = [] - self.recent_f1_scores = [] - - def should_reset_connections(self, current_f1=None): - """基于计数和性能双重判断""" - # 定期重置 - if self.question_count % self.reset_interval == 0: - return True - - # 性能驱动的重置 - if current_f1 is not None and current_f1 < self.performance_threshold: - self.consecutive_low_scores += 1 - if self.consecutive_low_scores >= 2: # 连续2个低分就重置 - print("🚨 连续低分，触发紧急重置") - self.consecutive_low_scores = 0 - return True - else: - self.consecutive_low_scores = 0 - - return False - - def record_performance(self, question_index, metrics, context_length, retrieved_docs): - """记录性能指标，检测衰减""" - self.performance_history.append({ - 'index': question_index, - 'metrics': metrics, - 'context_length': context_length, - 'retrieved_docs': retrieved_docs, - 'timestamp': time.time() - }) - - # 记录最近的F1分数 - self.recent_f1_scores.append(metrics['f1']) - if len(self.recent_f1_scores) > 5: - self.recent_f1_scores.pop(0) - - def get_recent_performance(self): - """获取近期平均性能""" - if not self.recent_f1_scores: - return 0.5 - return sum(self.recent_f1_scores) / len(self.recent_f1_scores) - - def get_performance_trend(self): - """分析性能趋势""" - if len(self.performance_history) < 2: - return "stable" - - recent_metrics = [item['metrics']['f1'] for item in self.performance_history[-5:]] - earlier_metrics = [item['metrics']['f1'] for item in self.performance_history[-10:-5]] - - if len(recent_metrics) < 2 or len(earlier_metrics) < 2: - return "stable" - - recent_avg = sum(recent_metrics) / len(recent_metrics) - earlier_avg = sum(earlier_metrics) / len(earlier_metrics) - - if recent_avg < earlier_avg * 0.8: - return "degrading" - elif recent_avg > earlier_avg * 1.1: - return "improving" - else: - return "stable" - - -def get_enhanced_search_params(question: str, question_index: int, total_questions: int, recent_performance: float): - """基于问题复杂度和近期性能动态调整检索参数""" - - # 基础参数 - base_params = get_dynamic_search_params(question, question_index, total_questions) - - # 性能自适应调整 - if recent_performance < 0.5: # 近期表现差 - # 增加检索范围，尝试获取更多上下文 - base_params["limit"] = min(base_params["limit"] + 5, 25) - base_params["max_chars"] = min(base_params["max_chars"] + 2000, 12000) - print(f"📈 性能自适应：增加检索范围 (limit={base_params['limit']}, max_chars={base_params['max_chars']})") - - elif recent_performance > 0.8: # 近期表现好 - # 收紧检索，提高精度 - base_params["limit"] = max(base_params["limit"] - 2, 8) - base_params["max_chars"] = max(base_params["max_chars"] - 1000, 6000) - print(f"🎯 性能自适应：提高检索精度 (limit={base_params['limit']}, max_chars={base_params['max_chars']})") - - # 中间阶段特殊处理 - mid_sequence_factor = abs(question_index / total_questions - 0.5) - if mid_sequence_factor < 0.2: # 在中间30%的问题 - print("🎯 中间阶段：使用更精确的检索策略") - base_params["limit"] = max(base_params["limit"] - 2, 10) # 减少数量，提高质量 - base_params["max_chars"] = max(base_params["max_chars"] - 1000, 7000) - - return base_params - - -def enhanced_context_selection(contexts: List[str], question: str, question_index: int, total_questions: int, max_chars: int = 8000) -> str: - """考虑问题序列位置的智能选择""" - - if not contexts: - return "" - - # 在序列中间阶段使用更严格的筛选 - mid_sequence_factor = abs(question_index / total_questions - 0.5) # 距离中心的距离 - - if mid_sequence_factor < 0.2: # 在中间30%的问题 - print("🎯 中间阶段：使用严格上下文筛选") - - # 提取问题关键词 - question_lower = question.lower() - stop_words = {'what', 'when', 'where', 'who', 'why', 'how', 'did', 'do', 'does', 'is', 'are', 'was', 'were', 'the', 'a', 'an', 'and', 'or', 'but'} - question_words = set(re.findall(r'\b\w+\b', question_lower)) - question_words = {word for word in question_words if word not in stop_words and len(word) > 2} - - # 只保留高度相关的上下文 - filtered_contexts = [] - for context in contexts: - context_lower = context.lower() - relevance_score = sum(3 if word in context_lower else 0 for word in question_words) - - # 额外加分给包含数字、日期的上下文（对事实性问题更重要） - if any(char.isdigit() for char in context): - relevance_score += 2 - - # 提高阈值：只有得分>=3的上下文才保留 - if relevance_score >= 3: - filtered_contexts.append(context) - else: - print(f" - 过滤低分上下文: 得分={relevance_score}") - - contexts = filtered_contexts - print(f"🔍 严格筛选后保留 {len(contexts)} 个上下文") - - # 使用原有的智能选择逻辑 - return smart_context_selection(contexts, question, max_chars) - - -async def run_enhanced_evaluation(): - """使用增强方法进行完整评估 - 解决中间性能衰减问题""" - try: - from dotenv import load_dotenv - except Exception: - def load_dotenv(): - return None - - # 修正导入路径：使用 app.core.memory.src 前缀 - from app.core.memory.llm_tools.openai_embedder import OpenAIEmbedderClient - from app.core.memory.utils.config.definitions import ( - SELECTED_EMBEDDING_ID, - SELECTED_LLM_ID, - ) - from app.core.memory.utils.llm.llm_utils import MemoryClientFactory - from app.core.models.base import RedBearModelConfig - from app.db import get_db_context - from app.repositories.neo4j.graph_search import search_graph_by_embedding - from app.repositories.neo4j.neo4j_connector import Neo4jConnector - from app.services.memory_config_service import MemoryConfigService - - # 加载数据 - # 获取项目根目录 - current_file = os.path.abspath(__file__) - evaluation_dir = os.path.dirname(os.path.dirname(current_file)) # evaluation目录 - memory_dir = os.path.dirname(evaluation_dir) # memory目录 - data_path = os.path.join(memory_dir, "data", "locomo10.json") - with open(data_path, "r", encoding="utf-8") as f: - raw = json.load(f) - - qa_items = [] - if isinstance(raw, list): - for entry in raw: - qa_items.extend(entry.get("qa", [])) - else: - qa_items.extend(raw.get("qa", [])) - - items = qa_items[:20] # 测试多少个问题 - - # 初始化增强监控器 - monitor = EnhancedEvaluationMonitor(reset_interval=5, performance_threshold=0.6) - - with get_db_context() as db: - factory = MemoryClientFactory(db) - llm = factory.get_llm_client(SELECTED_LLM_ID) - - # 初始化embedder - with get_db_context() as db: - config_service = MemoryConfigService(db) - cfg_dict = config_service.get_embedder_config(SELECTED_EMBEDDING_ID) - embedder = OpenAIEmbedderClient( - model_config=RedBearModelConfig.model_validate(cfg_dict) - ) - - # 初始化连接器 - connector = Neo4jConnector() - - # 初始化结果字典 - results = { - "questions": [], - "overall_metrics": {"f1": 0.0, "b1": 0.0, "j": 0.0, "loc_f1": 0.0}, - "category_metrics": {}, - "retrieval_stats": {"total_questions": len(items), "avg_context_length": 0, "avg_retrieved_docs": 0}, - "performance_trend": "stable", - "timestamp": datetime.now().isoformat(), - "enhanced_strategy": True - } - - total_f1 = 0.0 - total_bleu1 = 0.0 - total_jaccard = 0.0 - total_loc_f1 = 0.0 - total_context_length = 0 - total_retrieved_docs = 0 - category_stats = {} - - try: - for i, item in enumerate(items): - monitor.question_count += 1 - - # 获取近期性能用于重置判断 - recent_performance = monitor.get_recent_performance() - - # 增强的重置判断 - should_reset = monitor.should_reset_connections(current_f1=recent_performance) - if should_reset and i > 0: - print(f"🔄 重置Neo4j连接 (问题 {i+1}/{len(items)}, 近期性能: {recent_performance:.3f})...") - await connector.close() - connector = Neo4jConnector() # 创建新连接 - print("✅ 连接重置完成") - - q = item.get("question", "") - ref = item.get("answer", "") - ref_str = str(ref) if ref is not None else "" - - print(f"\n🔍 [{i+1}/{len(items)}] 问题: {q}") - print(f"✅ 真实答案: {ref_str}") - - # 分类别统计 - category = "Unknown" - if item.get("category") == 1: - category = "Multi-Hop" - elif item.get("category") == 2: - category = "Temporal" - elif item.get("category") == 3: - category = "Open Domain" - elif item.get("category") == 4: - category = "Single-Hop" - - # 增强的检索参数 - search_params = get_enhanced_search_params(q, i, len(items), recent_performance) - search_limit = search_params["limit"] - max_chars = search_params["max_chars"] - - print(f"🏷️ 类别: {category}, 检索参数: limit={search_limit}, max_chars={max_chars}") - - # 使用项目标准的混合检索方法 - t0 = time.time() - contexts_all = [] - - try: - # 使用统一的搜索服务 - from app.core.memory.storage_services.search import run_hybrid_search - - print("🔀 使用混合搜索服务...") - - search_results = await run_hybrid_search( - query_text=q, - search_type="hybrid", - group_id="locomo_sk", - limit=20, - include=["statements", "chunks", "entities", "summaries"], - alpha=0.6, # BM25权重 - embedding_id=SELECTED_EMBEDDING_ID - ) - - # 处理搜索结果 - 新的搜索服务返回统一的结构 - chunks = search_results.get("chunks", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - summaries = search_results.get("summaries", []) - - print(f"✅ 混合检索成功: {len(chunks)} chunks, {len(statements)} 条陈述, {len(entities)} 个实体, {len(summaries)} 个摘要") - - # 构建上下文：优先使用 chunks、statements 和 summaries - for c in chunks: - content = str(c.get("content", "")).strip() - if content: - contexts_all.append(content) - - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - - for sm in summaries: - summary_text = str(sm.get("summary", "")).strip() - if summary_text: - contexts_all.append(summary_text) - - # 实体摘要：最多加入前3个高分实体，避免噪声 - scored = [e for e in entities if e.get("score") is not None] - top_entities = sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] if scored else entities[:3] - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append(f"EntitySummary: {name}{(' [' + ' '.join(meta) + ']') if meta else ''}") - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - - print(f"📊 有效上下文数量: {len(contexts_all)}") - except Exception as e: - print(f"❌ 检索失败: {e}") - contexts_all = [] - - t1 = time.time() - search_time = (t1 - t0) * 1000 - - # 增强的上下文选择 - context_text = "" - if contexts_all: - # 使用增强的上下文选择 - context_text = enhanced_context_selection(contexts_all, q, i, len(items), max_chars=max_chars) - - # 如果智能选择后仍然过长，进行最终保护性截断 - if len(context_text) > max_chars: - print(f"⚠️ 智能选择后仍然过长 ({len(context_text)}字符)，进行最终截断") - context_text = context_text[:max_chars] + "\n\n[最终截断...]" - - # 时间解析 - anchor_date = datetime(2023, 5, 8) # 使用固定日期确保一致性 - context_text = _resolve_relative_times(context_text, anchor_date) - - context_text = f"Reference date: {anchor_date.date().isoformat()}\n\n" + context_text - - print(f"📝 最终上下文长度: {len(context_text)} 字符") - - # 显示不同上下文的预览（不只是第一条） - print("🔍 上下文预览:") - for j, context in enumerate(contexts_all[:3]): # 显示前3个上下文 - preview = context[:150].replace('\n', ' ') - print(f" 上下文{j+1}: {preview}...") - - # 🔍 调试：检查答案是否在上下文中 - if ref_str and ref_str.strip(): - answer_found = any(ref_str.lower() in ctx.lower() for ctx in contexts_all) - print(f"🔍 调试：答案 '{ref_str}' 是否在检索到的上下文中？ {'✅ 是' if answer_found else '❌ 否'}") - - else: - print("❌ 没有检索到有效上下文") - context_text = "No relevant context found." - - # LLM 回答 - messages = [ - {"role": "system", "content": ( - "You are a precise QA assistant. Answer following these rules:\n" - "1) Extract the EXACT information mentioned in the context\n" - "2) For time questions: calculate actual dates from relative times\n" - "3) Return ONLY the answer text in simplest form\n" - "4) For dates, use format 'DD Month YYYY' (e.g., '7 May 2023')\n" - "5) If no clear answer found, respond with 'Unknown'" - )}, - {"role": "user", "content": f"Question: {q}\n\nContext:\n{context_text}"}, - ] - - t2 = time.time() - try: - # 使用异步调用 - resp = await llm.chat(messages=messages) - # 兼容不同的响应格式 - pred = resp.content.strip() if hasattr(resp, 'content') else (resp["choices"][0]["message"]["content"].strip() if isinstance(resp, dict) else "Unknown") - except Exception as e: - print(f"❌ LLM 生成失败: {e}") - pred = "Unknown" - t3 = time.time() - llm_time = (t3 - t2) * 1000 - - # 计算指标 - 使用导入的指标函数 - f1_val = f1_score(pred, ref_str) - bleu1_val = bleu1(pred, ref_str) - jaccard_val = jaccard(pred, ref_str) - loc_f1_val = loc_f1_score(pred, ref_str) - - print(f"🤖 LLM 回答: {pred}") - print(f"📈 指标 - F1: {f1_val:.3f}, BLEU-1: {bleu1_val:.3f}, Jaccard: {jaccard_val:.3f}, LoCoMo F1: {loc_f1_val:.3f}") - print(f"⏱️ 时间 - 检索: {search_time:.1f}ms, LLM: {llm_time:.1f}ms") - - # 更新统计 - total_f1 += f1_val - total_bleu1 += bleu1_val - total_jaccard += jaccard_val - total_loc_f1 += loc_f1_val - total_context_length += len(context_text) - total_retrieved_docs += len(contexts_all) - - if category not in category_stats: - category_stats[category] = {"count": 0, "f1_sum": 0.0, "b1_sum": 0.0, "j_sum": 0.0, "loc_f1_sum": 0.0} - - category_stats[category]["count"] += 1 - category_stats[category]["f1_sum"] += f1_val - category_stats[category]["b1_sum"] += bleu1_val - category_stats[category]["j_sum"] += jaccard_val - category_stats[category]["loc_f1_sum"] += loc_f1_val - - # 记录性能指标 - metrics = {"f1": f1_val, "bleu1": bleu1_val, "jaccard": jaccard_val, "loc_f1": loc_f1_val} - monitor.record_performance(i, metrics, len(context_text), len(contexts_all)) - - # 保存结果 - question_result = { - "question": q, - "ground_truth": ref_str, - "prediction": pred, - "category": category, - "metrics": metrics, - "retrieval": { - "retrieved_documents": len(contexts_all), - "context_length": len(context_text), - "search_limit": search_limit, - "max_chars": max_chars, - "recent_performance": recent_performance - }, - "timing": { - "search_ms": search_time, - "llm_ms": llm_time - } - } - - results["questions"].append(question_result) - - print("="*60) - - except Exception as e: - print(f"❌ 评估过程中发生错误: {e}") - # 即使出错，也返回已有的结果 - import traceback - traceback.print_exc() - - finally: - await connector.close() - - # 计算总体指标 - n = len(items) - if n > 0: - results["overall_metrics"] = { - "f1": total_f1 / n, - "b1": total_bleu1 / n, - "j": total_jaccard / n, - "loc_f1": total_loc_f1 / n - } - - for category, stats in category_stats.items(): - count = stats["count"] - results["category_metrics"][category] = { - "count": count, - "f1": stats["f1_sum"] / count, - "bleu1": stats["b1_sum"] / count, - "jaccard": stats["j_sum"] / count, - "loc_f1": stats["loc_f1_sum"] / count - } - - results["retrieval_stats"]["avg_context_length"] = total_context_length / n - results["retrieval_stats"]["avg_retrieved_docs"] = total_retrieved_docs / n - - # 分析性能趋势 - results["performance_trend"] = monitor.get_performance_trend() - results["reset_interval"] = monitor.reset_interval - results["total_questions_processed"] = monitor.question_count - - return results - - -if __name__ == "__main__": - print("🚀 运行增强版完整评估（解决中间性能衰减问题）...") - print("📋 增强特性:") - print(" - 双重重置策略：定期重置 + 性能驱动重置") - print(" - 动态检索参数：基于近期性能自适应调整") - print(" - 中间阶段严格筛选：提高上下文质量要求") - print(" - 连续性能监控：实时检测性能衰减") - - result = asyncio.run(run_enhanced_evaluation()) - - print("\n📊 最终评估结果:") - print("总体指标:") - print(f" F1: {result['overall_metrics']['f1']:.4f}") - print(f" BLEU-1: {result['overall_metrics']['b1']:.4f}") - print(f" Jaccard: {result['overall_metrics']['j']:.4f}") - print(f" LoCoMo F1: {result['overall_metrics']['loc_f1']:.4f}") - - print("\n分类别指标:") - for category, metrics in result['category_metrics'].items(): - print(f" {category}: F1={metrics['f1']:.4f}, BLEU-1={metrics['bleu1']:.4f}, Jaccard={metrics['jaccard']:.4f}, LoCoMo F1={metrics['loc_f1']:.4f} (样本数: {metrics['count']})") - - print("\n检索统计:") - stats = result['retrieval_stats'] - print(f" 平均上下文长度: {stats['avg_context_length']:.0f} 字符") - print(f" 平均检索文档数: {stats['avg_retrieved_docs']:.1f}") - - print(f"\n性能趋势: {result['performance_trend']}") - print(f"重置间隔: 每{result['reset_interval']}个问题") - print(f"处理问题总数: {result['total_questions_processed']}") - print(f"增强策略: {'启用' if result.get('enhanced_strategy', False) else '未启用'}") - - - # 保存结果到指定目录 - # 使用代码文件所在目录的绝对路径 - current_file_dir = os.path.dirname(os.path.abspath(__file__)) - output_dir = os.path.join(current_file_dir, "results") - os.makedirs(output_dir, exist_ok=True) - output_file = os.path.join(output_dir, "enhanced_evaluation_results.json") - with open(output_file, "w", encoding="utf-8") as f: - json.dump(result, f, ensure_ascii=False, indent=2) - print(f"\n详细结果已保存到: {output_file}") diff --git a/api/app/core/memory/evaluation/locomo/locomo_utils.py b/api/app/core/memory/evaluation/locomo/locomo_utils.py deleted file mode 100644 index 69be5da9..00000000 --- a/api/app/core/memory/evaluation/locomo/locomo_utils.py +++ /dev/null @@ -1,626 +0,0 @@ -""" -LoCoMo Utilities Module - -This module provides helper functions for the LoCoMo benchmark evaluation: -- Data loading from JSON files -- Conversation extraction for ingestion -- Temporal reference resolution -- Context selection and formatting -- Retrieval wrapper functions -- Ingestion wrapper functions -""" - -import os -import json -import re -from datetime import datetime, timedelta -from typing import List, Dict, Any, Optional - -from app.core.memory.utils.definitions import PROJECT_ROOT -from app.core.memory.evaluation.extraction_utils import ingest_contexts_via_full_pipeline - - -def load_locomo_data( - data_path: str, - sample_size: int, - conversation_index: int = 0 -) -> List[Dict[str, Any]]: - """ - Load LoCoMo dataset from JSON file. - - The LoCoMo dataset structure is a list of conversation objects, where each - object contains a "qa" list of question-answer pairs. - - Args: - data_path: Path to locomo10.json file - sample_size: Number of QA pairs to load (limits total QA items returned) - conversation_index: Which conversation to load QA pairs from (default: 0 for first) - - Returns: - List of QA item dictionaries, each containing: - - question: str - - answer: str - - category: int (1-4) - - evidence: List[str] - - Raises: - FileNotFoundError: If data_path does not exist - json.JSONDecodeError: If file is not valid JSON - IndexError: If conversation_index is out of range - """ - if not os.path.exists(data_path): - raise FileNotFoundError(f"LoCoMo data file not found: {data_path}") - - with open(data_path, "r", encoding="utf-8") as f: - raw = json.load(f) - - # LoCoMo data structure: list of objects, each with a "qa" list - qa_items: List[Dict[str, Any]] = [] - - if isinstance(raw, list): - # Only load QA pairs from the specified conversation - if conversation_index < len(raw): - entry = raw[conversation_index] - if isinstance(entry, dict) and "qa" in entry: - qa_items.extend(entry.get("qa", [])) - else: - raise IndexError( - f"Conversation index {conversation_index} out of range. " - f"Dataset has {len(raw)} conversations." - ) - else: - # Fallback: single object with qa list - if conversation_index == 0: - qa_items.extend(raw.get("qa", [])) - else: - raise IndexError( - f"Conversation index {conversation_index} out of range. " - f"Dataset has only 1 conversation." - ) - - # Return only the requested sample size - return qa_items[:sample_size] - - -def extract_conversations(data_path: str, max_dialogues: int = 1) -> List[str]: - """ - Extract conversation texts from LoCoMo data for ingestion. - - This function extracts the raw conversation dialogues from the LoCoMo dataset - so they can be ingested into the memory system. Each conversation is formatted - as a multi-line string with "role: message" format. - - Args: - data_path: Path to locomo10.json file - max_dialogues: Maximum number of dialogues to extract (default: 1) - - Returns: - List of conversation strings formatted for ingestion. - Each string contains multiple lines in format "role: message" - - Example output: - [ - "User: I went to the store yesterday.\\nAI: What did you buy?\\n...", - "User: I love hiking.\\nAI: Where do you like to hike?\\n..." - ] - """ - if not os.path.exists(data_path): - raise FileNotFoundError(f"LoCoMo data file not found: {data_path}") - - with open(data_path, "r", encoding="utf-8") as f: - raw = json.load(f) - - # Ensure we have a list of entries - entries = raw if isinstance(raw, list) else [raw] - - contents: List[str] = [] - - for i, entry in enumerate(entries[:max_dialogues]): - if not isinstance(entry, dict): - continue - - conv = entry.get("conversation", {}) - - if not isinstance(conv, dict): - continue - - lines: List[str] = [] - - # Collect all session_* messages - for key, val in sorted(conv.items()): - if isinstance(val, list) and key.startswith("session_"): - for msg in val: - if not isinstance(msg, dict): - continue - - role = msg.get("speaker") or "User" - text = msg.get("text") or "" - text = str(text).strip() - - if not text: - continue - - lines.append(f"{role}: {text}") - - if lines: - contents.append("\n".join(lines)) - - return contents - - -def resolve_temporal_references(text: str, anchor_date: datetime) -> str: - """ - Resolve relative temporal references to absolute dates. - - This function converts relative time expressions (like "today", "yesterday", - "3 days ago") into absolute ISO date strings based on an anchor date. - - Supported patterns: - - today, yesterday, tomorrow - - X days ago, in X days - - last week, next week - - Args: - text: Text containing temporal references - anchor_date: Reference date for resolution (datetime object) - - Returns: - Text with temporal references replaced by ISO dates (YYYY-MM-DD format) - - Example: - >>> anchor = datetime(2023, 5, 8) - >>> resolve_temporal_references("I saw him yesterday", anchor) - "I saw him 2023-05-07" - """ - # Ensure input is a string - t = str(text) if text is not None else "" - - # today / yesterday / tomorrow - t = re.sub( - r"\btoday\b", - anchor_date.date().isoformat(), - t, - flags=re.IGNORECASE - ) - t = re.sub( - r"\byesterday\b", - (anchor_date - timedelta(days=1)).date().isoformat(), - t, - flags=re.IGNORECASE - ) - t = re.sub( - r"\btomorrow\b", - (anchor_date + timedelta(days=1)).date().isoformat(), - t, - flags=re.IGNORECASE - ) - - # X days ago - def _ago_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor_date - timedelta(days=n)).date().isoformat() - - # in X days - def _in_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor_date + timedelta(days=n)).date().isoformat() - - t = re.sub( - r"\b(\d+)\s+days?\s+ago\b", - _ago_repl, - t, - flags=re.IGNORECASE - ) - t = re.sub( - r"\bin\s+(\d+)\s+days?\b", - _in_repl, - t, - flags=re.IGNORECASE - ) - - # last week / next week (approximate as 7 days) - t = re.sub( - r"\blast\s+week\b", - (anchor_date - timedelta(days=7)).date().isoformat(), - t, - flags=re.IGNORECASE - ) - t = re.sub( - r"\bnext\s+week\b", - (anchor_date + timedelta(days=7)).date().isoformat(), - t, - flags=re.IGNORECASE - ) - - return t - - -def select_and_format_information( - retrieved_info: List[str], - question: str, - max_chars: int = 8000 -) -> str: - """ - Intelligently select and format most relevant retrieved information for LLM prompt. - - This function scores each piece of retrieved information based on keyword matching - with the question, then selects the highest-scoring pieces up to the character limit. - - Scoring criteria: - - Keyword matches (higher weight for multiple occurrences) - - Context length (moderate length preferred) - - Position (earlier contexts get bonus points) - - Args: - retrieved_info: List of retrieved information strings (chunks, statements, entities) - question: Question being answered - max_chars: Maximum total characters to include in final prompt - - Returns: - Formatted string combining the most relevant information for LLM prompt. - Contexts are separated by double newlines. - - Example: - >>> contexts = ["Alice went to Paris", "Bob likes pizza", "Alice visited the Eiffel Tower"] - >>> question = "Where did Alice go?" - >>> select_and_format_information(contexts, question, max_chars=100) - "Alice went to Paris\\n\\nAlice visited the Eiffel Tower" - """ - if not retrieved_info: - return "" - - # Extract question keywords (filter out stop words and short words) - question_lower = question.lower() - stop_words = { - 'what', 'when', 'where', 'who', 'why', 'how', - 'did', 'do', 'does', 'is', 'are', 'was', 'were', - 'the', 'a', 'an', 'and', 'or', 'but', 'in', 'on', 'at' - } - question_words = set(re.findall(r'\b\w+\b', question_lower)) - question_words = { - word for word in question_words - if word not in stop_words and len(word) > 2 - } - - # Score each context - scored_contexts = [] - for i, context in enumerate(retrieved_info): - context_lower = context.lower() - score = 0 - - # Keyword matching score - keyword_matches = 0 - for word in question_words: - if word in context_lower: - keyword_matches += 1 - # Multiple occurrences increase score - score += context_lower.count(word) * 2 - - # Length score (prefer moderate length) - context_len = len(context) - if 100 < context_len < 2000: - score += 5 - elif context_len >= 2000: - score += 2 - - # Position bonus (earlier contexts often more relevant) - if i < 3: - score += 3 - - scored_contexts.append((score, context, keyword_matches)) - - # Sort by score (descending) - scored_contexts.sort(key=lambda x: x[0], reverse=True) - - # Select contexts up to character limit - selected = [] - total_chars = 0 - - for score, context, matches in scored_contexts: - if total_chars + len(context) <= max_chars: - selected.append(context) - total_chars += len(context) - else: - # Try to include high-scoring context by truncating - if score > 10 and total_chars < max_chars - 500: - remaining = max_chars - total_chars - # Find lines with keywords - lines = context.split('\n') - relevant_lines = [] - current_chars = 0 - - for line in lines: - line_lower = line.lower() - line_relevance = any(word in line_lower for word in question_words) - - if line_relevance and current_chars < remaining - 100: - relevant_lines.append(line) - current_chars += len(line) - - if relevant_lines and len('\n'.join(relevant_lines)) > 100: - truncated = '\n'.join(relevant_lines) - selected.append(truncated + "\n[Content truncated...]") - total_chars += len(truncated) - break - - return "\n\n".join(selected) - - -async def retrieve_relevant_information( - question: str, - group_id: str, - search_type: str, - search_limit: int, - connector: Any, - embedder: Any -) -> List[str]: - """ - Retrieve relevant information from memory graph for a question. - - This function searches the Neo4j memory graph (populated during ingestion) and - returns relevant chunks, statements, and entity information that might help - answer the question. - - The function supports three search types: - - "keyword": Full-text search using Cypher queries - - "embedding": Vector similarity search using embeddings - - "hybrid": Combination of keyword and embedding search with reranking - - Args: - question: Question to search for - group_id: Database group ID (identifies which conversation memory to search) - search_type: "keyword", "embedding", or "hybrid" - search_limit: Max memory pieces to retrieve - connector: Neo4j connector instance - embedder: Embedder client instance - - Returns: - List of text strings (chunks, statements, entity summaries) from memory graph. - Each string represents a piece of retrieved information. - - Raises: - Exception: If search fails (caught and returns empty list) - """ - from app.repositories.neo4j.graph_search import ( - search_graph, - search_graph_by_embedding - ) - from app.core.memory.storage_services.search import run_hybrid_search - - contexts_all: List[str] = [] - - try: - if search_type == "embedding": - # Embedding-based search - search_results = await search_graph_by_embedding( - connector=connector, - embedder_client=embedder, - query_text=question, - group_id=group_id, - limit=search_limit, - include=["chunks", "statements", "entities", "summaries"], - ) - - chunks = search_results.get("chunks", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - summaries = search_results.get("summaries", []) - - # Build context from chunks - for c in chunks: - content = str(c.get("content", "")).strip() - if content: - contexts_all.append(content) - - # Add statements - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - - # Add summaries - for sm in summaries: - summary_text = str(sm.get("summary", "")).strip() - if summary_text: - contexts_all.append(summary_text) - - # Add top entities (limit to 3 to avoid noise) - if entities: - scored = [e for e in entities if e.get("score") is not None] - top_entities = ( - sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] - if scored else entities[:3] - ) - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append( - f"EntitySummary: {name}" - f"{(' [' + '; '.join(meta) + ']') if meta else ''}" - ) - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - - elif search_type == "keyword": - # Keyword-based search - search_results = await search_graph( - connector=connector, - q=question, - group_id=group_id, - limit=search_limit - ) - - dialogs = search_results.get("dialogues", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - - # Build context from dialogues - for d in dialogs: - content = str(d.get("content", "")).strip() - if content: - contexts_all.append(content) - - # Add statements - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - - # Add entity names - if entities: - entity_names = [ - str(e.get("name", "")).strip() - for e in entities[:5] - if e.get("name") - ] - if entity_names: - contexts_all.append(f"EntitySummary: {', '.join(entity_names)}") - - else: # hybrid - # Hybrid search with fallback to embedding - try: - search_results = await run_hybrid_search( - query_text=question, - search_type=search_type, - group_id=group_id, - limit=search_limit, - include=["chunks", "statements", "entities", "summaries"], - output_path=None, - ) - - # Handle flat structure (new API format) - if search_results and isinstance(search_results, dict): - chunks = search_results.get("chunks", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - summaries = search_results.get("summaries", []) - - # Check if we got results - if not (chunks or statements or entities or summaries): - # Try nested structure (backward compatibility) - reranked = search_results.get("reranked_results", {}) - if reranked and isinstance(reranked, dict): - chunks = reranked.get("chunks", []) - statements = reranked.get("statements", []) - entities = reranked.get("entities", []) - summaries = reranked.get("summaries", []) - else: - raise ValueError("Hybrid search returned empty results") - else: - raise ValueError("Hybrid search returned empty results") - - except Exception as e: - # Fallback to embedding search - search_results = await search_graph_by_embedding( - connector=connector, - embedder_client=embedder, - query_text=question, - group_id=group_id, - limit=search_limit, - include=["chunks", "statements", "entities", "summaries"], - ) - chunks = search_results.get("chunks", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - summaries = search_results.get("summaries", []) - - # Build context (same for both hybrid and fallback) - for c in chunks: - content = str(c.get("content", "")).strip() - if content: - contexts_all.append(content) - - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - - for sm in summaries: - summary_text = str(sm.get("summary", "")).strip() - if summary_text: - contexts_all.append(summary_text) - - # Add top entities - if entities: - scored = [e for e in entities if e.get("score") is not None] - top_entities = ( - sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] - if scored else entities[:3] - ) - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append( - f"EntitySummary: {name}" - f"{(' [' + '; '.join(meta) + ']') if meta else ''}" - ) - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - - except Exception as e: - # Return empty list on error - contexts_all = [] - - return contexts_all - - -async def ingest_conversations_if_needed( - conversations: List[str], - group_id: str, - reset: bool = False -) -> bool: - """ - Wrapper for conversation ingestion using external extraction pipeline. - - This function populates the Neo4j database with processed conversation data - (chunks, statements, entities) so that the retrieval system has memory to search. - - The ingestion process: - 1. Parses conversation text into dialogue messages - 2. Chunks the dialogues into semantic units - 3. Extracts statements and entities using LLM - 4. Generates embeddings for all content - 5. Stores everything in Neo4j graph database - - Args: - conversations: List of raw conversation texts from LoCoMo dataset - Example: ["User: I went to Paris. AI: When was that?", ...] - group_id: Target group ID for database storage - reset: Whether to clear existing data first (not implemented in wrapper) - - Returns: - True if successful, False otherwise - - Note: - The external function uses "contexts" to mean "conversation texts". - This runs the full extraction pipeline: chunking → entity extraction → - statement extraction → embedding → Neo4j storage. - """ - try: - success = await ingest_contexts_via_full_pipeline( - contexts=conversations, - group_id=group_id, - save_chunk_output=True - ) - return success - except Exception as e: - print(f"[Ingestion] Failed to ingest conversations: {e}") - return False diff --git a/api/app/core/memory/evaluation/locomo/qwen_search_eval.py b/api/app/core/memory/evaluation/locomo/qwen_search_eval.py deleted file mode 100644 index 87a70a29..00000000 --- a/api/app/core/memory/evaluation/locomo/qwen_search_eval.py +++ /dev/null @@ -1,878 +0,0 @@ -import argparse -import asyncio -import json -import os -import statistics -import time -from datetime import datetime, timedelta -from typing import Any, Dict, List - -try: - from dotenv import load_dotenv -except Exception: - def load_dotenv(): - return None - -import re - -from app.core.memory.evaluation.common.metrics import ( - avg_context_tokens, - bleu1, - jaccard, - latency_stats, -) -from app.core.memory.evaluation.common.metrics import f1_score as common_f1 -from app.core.memory.evaluation.extraction_utils import ( - ingest_contexts_via_full_pipeline, -) -from app.core.memory.llm_tools.openai_embedder import OpenAIEmbedderClient -from app.core.memory.storage_services.search import run_hybrid_search -from app.core.memory.utils.config.definitions import ( - PROJECT_ROOT, - SELECTED_EMBEDDING_ID, - SELECTED_GROUP_ID, - SELECTED_LLM_ID, -) -from app.core.memory.utils.llm.llm_utils import MemoryClientFactory -from app.core.models.base import RedBearModelConfig -from app.db import get_db_context -from app.repositories.neo4j.graph_search import search_graph, search_graph_by_embedding -from app.repositories.neo4j.neo4j_connector import Neo4jConnector -from app.services.memory_config_service import MemoryConfigService - - -# 参考 evaluation/locomo/evaluation.py 的 F1 计算逻辑（移除外部依赖，内联实现） -def _loc_normalize(text: str) -> str: - import re - # 确保输入是字符串 - text = str(text) if text is not None else "" - text = text.lower() - text = re.sub(r"[\,]", " ", text) # 去掉逗号 - text = re.sub(r"\b(a|an|the|and)\b", " ", text) - text = re.sub(r"[^\w\s]", " ", text) - text = " ".join(text.split()) - return text - -# 追加：相对时间归一化为绝对日期（有限支持：today/yesterday/tomorrow/X days ago/in X days/last week/next week） -def _resolve_relative_times(text: str, anchor: datetime) -> str: - import re - # 确保输入是字符串 - t = str(text) if text is not None else "" - # today / yesterday / tomorrow - t = re.sub(r"\btoday\b", anchor.date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\byesterday\b", (anchor - timedelta(days=1)).date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\btomorrow\b", (anchor + timedelta(days=1)).date().isoformat(), t, flags=re.IGNORECASE) - # X days ago / in X days - def _ago_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor - timedelta(days=n)).date().isoformat() - def _in_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor + timedelta(days=n)).date().isoformat() - t = re.sub(r"\b(\d+)\s+days\s+ago\b", _ago_repl, t, flags=re.IGNORECASE) - t = re.sub(r"\bin\s+(\d+)\s+days\b", _in_repl, t, flags=re.IGNORECASE) - # last week / next week（以7天近似） - t = re.sub(r"\blast\s+week\b", (anchor - timedelta(days=7)).date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\bnext\s+week\b", (anchor + timedelta(days=7)).date().isoformat(), t, flags=re.IGNORECASE) - return t - -def loc_f1_score(prediction: str, ground_truth: str) -> float: - # 单答案 F1：按词集合计算（近似原始实现，去除词干依赖） - # 确保输入是字符串 - pred_str = str(prediction) if prediction is not None else "" - truth_str = str(ground_truth) if ground_truth is not None else "" - - p_tokens = _loc_normalize(pred_str).split() - g_tokens = _loc_normalize(truth_str).split() - if not p_tokens or not g_tokens: - return 0.0 - p = set(p_tokens) - g = set(g_tokens) - tp = len(p & g) - precision = tp / len(p) if p else 0.0 - recall = tp / len(g) if g else 0.0 - return (2 * precision * recall / (precision + recall)) if (precision + recall) > 0 else 0.0 - -def loc_multi_f1(prediction: str, ground_truth: str) -> float: - # 多答案 F1：prediction 与 ground_truth 以逗号分隔，逐一匹配取最大，再对多个 GT 取平均 - # 确保输入是字符串 - pred_str = str(prediction) if prediction is not None else "" - truth_str = str(ground_truth) if ground_truth is not None else "" - - predictions = [p.strip() for p in str(pred_str).split(',') if p.strip()] - ground_truths = [g.strip() for g in str(truth_str).split(',') if g.strip()] - if not predictions or not ground_truths: - return 0.0 - def _f1(a: str, b: str) -> float: - return loc_f1_score(a, b) - vals = [] - for gt in ground_truths: - vals.append(max(_f1(pred, gt) for pred in predictions)) - return sum(vals) / len(vals) - -# 标准化 LoCoMo 类别名：支持数字 category 与字符串 cat/type -CATEGORY_MAP_NUM_TO_NAME = { - 4: "Single-Hop", - 1: "Multi-Hop", - 3: "Open Domain", - 2: "Temporal", -} - -_TYPE_ALIASES = { - "single-hop": "Single-Hop", - "singlehop": "Single-Hop", - "single hop": "Single-Hop", - "multi-hop": "Multi-Hop", - "multihop": "Multi-Hop", - "multi hop": "Multi-Hop", - "open domain": "Open Domain", - "opendomain": "Open Domain", - "temporal": "Temporal", -} - -def get_category_label(item: Dict[str, Any]) -> str: - # 1) 直接用字符串 cat - cat = item.get("cat") - if isinstance(cat, str) and cat.strip(): - name = cat.strip() - lower = name.lower() - return _TYPE_ALIASES.get(lower, name) - # 2) 数字 category 转名称 - cat_num = item.get("category") - if isinstance(cat_num, int): - return CATEGORY_MAP_NUM_TO_NAME.get(cat_num, "unknown") - # 3) 备用 type 字段 - t = item.get("type") - if isinstance(t, str) and t.strip(): - lower = t.strip().lower() - return _TYPE_ALIASES.get(lower, t.strip()) - return "unknown" - - -def smart_context_selection(contexts: List[str], question: str, max_chars: int = 12000) -> str: - """基于问题关键词智能选择上下文""" - if not contexts: - return "" - - # 提取问题关键词（只保留有意义的词） - question_lower = question.lower() - stop_words = {'what', 'when', 'where', 'who', 'why', 'how', 'did', 'do', 'does', 'is', 'are', 'was', 'were', 'the', 'a', 'an', 'and', 'or', 'but'} - question_words = set(re.findall(r'\b\w+\b', question_lower)) - question_words = {word for word in question_words if word not in stop_words and len(word) > 2} - - print(f"🔍 问题关键词: {question_words}") - - # 给每个上下文打分 - scored_contexts = [] - for i, context in enumerate(contexts): - context_lower = context.lower() - score = 0 - - # 关键词匹配得分 - keyword_matches = 0 - for word in question_words: - if word in context_lower: - keyword_matches += 1 - # 关键词出现次数越多，得分越高 - score += context_lower.count(word) * 2 - - # 上下文长度得分（适中的长度更好） - context_len = len(context) - if 100 < context_len < 2000: # 理想长度范围 - score += 5 - elif context_len >= 2000: # 太长可能包含无关信息 - score += 2 - - # 如果是前几个上下文，给予额外分数（通常相关性更高） - if i < 3: - score += 3 - - scored_contexts.append((score, context, keyword_matches)) - - # 按得分排序 - scored_contexts.sort(key=lambda x: x[0], reverse=True) - - # 选择高得分的上下文，直到达到字符限制 - selected = [] - total_chars = 0 - selected_count = 0 - - print("📊 上下文相关性分析:") - for score, context, matches in scored_contexts[:5]: # 只显示前5个 - print(f" - 得分: {score}, 关键词匹配: {matches}, 长度: {len(context)}") - - for score, context, matches in scored_contexts: - if total_chars + len(context) <= max_chars: - selected.append(context) - total_chars += len(context) - selected_count += 1 - else: - # 如果这个上下文得分很高但放不下，尝试截取 - if score > 10 and total_chars < max_chars - 500: - remaining = max_chars - total_chars - # 找到包含关键词的部分 - lines = context.split('\n') - relevant_lines = [] - current_chars = 0 - - for line in lines: - line_lower = line.lower() - line_relevance = any(word in line_lower for word in question_words) - - if line_relevance and current_chars < remaining - 100: - relevant_lines.append(line) - current_chars += len(line) - - if relevant_lines: - truncated = '\n'.join(relevant_lines) - if len(truncated) > 100: # 确保有足够内容 - selected.append(truncated + "\n[相关内容截断...]") - total_chars += len(truncated) - selected_count += 1 - break # 不再尝试添加更多上下文 - - result = "\n\n".join(selected) - print(f"✅ 智能选择: {selected_count}个上下文, 总长度: {total_chars}字符") - return result - - -def get_search_params_by_category(category: str): - """根据问题类别调整检索参数""" - params_map = { - "Multi-Hop": {"limit": 20, "max_chars": 15000}, - "Temporal": {"limit": 16, "max_chars": 10000}, - "Open Domain": {"limit": 24, "max_chars": 18000}, - "Single-Hop": {"limit": 12, "max_chars": 8000}, - } - return params_map.get(category, {"limit": 16, "max_chars": 12000}) - - -async def run_locomo_eval( - sample_size: int = 1, - group_id: str | None = None, - search_limit: int = 8, - context_char_budget: int = 4000, # 保持默认值不变 - llm_temperature: float = 0.0, - llm_max_tokens: int = 32, - search_type: str = "hybrid", # 保持默认值不变 - output_path: str | None = None, - skip_ingest_if_exists: bool = True, - llm_timeout: float = 10.0, - llm_max_retries: int = 1 -) -> Dict[str, Any]: - - # 函数内部使用三路检索逻辑，但保持参数签名不变 - group_id = group_id or SELECTED_GROUP_ID - data_path = os.path.join(PROJECT_ROOT, "data", "locomo10.json") - if not os.path.exists(data_path): - data_path = os.path.join(os.getcwd(), "data", "locomo10.json") - with open(data_path, "r", encoding="utf-8") as f: - raw = json.load(f) - # LoCoMo 数据结构：顶层为若干对象，每个对象下有 qa 列表 - qa_items: List[Dict[str, Any]] = [] - if isinstance(raw, list): - for entry in raw: - qa_items.extend(entry.get("qa", [])) - else: - qa_items.extend(raw.get("qa", [])) - items: List[Dict[str, Any]] = qa_items[:sample_size] - - # === 保持原来的数据摄入逻辑 === - entries = raw if isinstance(raw, list) else [raw] - - # 只摄入前1条对话（保持原样） - max_dialogues_to_ingest = 1 - contents: List[str] = [] - print(f"📊 找到 {len(entries)} 个对话对象，只摄入前 {max_dialogues_to_ingest} 条") - - for i, entry in enumerate(entries[:max_dialogues_to_ingest]): - if not isinstance(entry, dict): - continue - - conv = entry.get("conversation", {}) - sample_id = entry.get("sample_id", f"unknown_{i}") - - print(f"🔍 处理对话 {i+1}: {sample_id}") - - lines: List[str] = [] - if isinstance(conv, dict): - # 收集所有 session_* 的消息 - session_count = 0 - for key, val in conv.items(): - if isinstance(val, list) and key.startswith("session_"): - session_count += 1 - for msg in val: - role = msg.get("speaker") or "用户" - text = msg.get("text") or "" - text = str(text).strip() - if not text: - continue - lines.append(f"{role}: {text}") - - print(f" - 包含 {session_count} 个session, {len(lines)} 条消息") - - if not lines: - print(f"⚠️ 警告: 对话 {sample_id} 没有对话内容，跳过摄入") - continue - - contents.append("\n".join(lines)) - - print(f"📥 总共摄入 {len(contents)} 个对话的conversation内容") - - # 选择要评测的QA对（从所有对话中选取） - indexed_items: List[tuple[int, Dict[str, Any]]] = [] - if isinstance(raw, list): - for e_idx, entry in enumerate(raw): - for qa in entry.get("qa", []): - indexed_items.append((e_idx, qa)) - else: - for qa in raw.get("qa", []): - indexed_items.append((0, qa)) - - # 这里使用sample_size来限制评测的QA数量 - selected = indexed_items[:sample_size] - items: List[Dict[str, Any]] = [qa for _, qa in selected] - - print(f"🎯 将评测 {len(items)} 个QA对，数据库中只包含 {len(contents)} 个对话") - # === 修改结束 === - - connector = Neo4jConnector() - - # 关键修复：强制重新摄入纯净的对话数据 - print("🔄 强制重新摄入纯净的对话数据...") - await ingest_contexts_via_full_pipeline(contents, group_id, save_chunk_output=True) - - # 使用异步LLM客户端 - with get_db_context() as db: - factory = MemoryClientFactory(db) - llm_client = factory.get_llm_client(SELECTED_LLM_ID) - # 初始化embedder用于直接调用 - with get_db_context() as db: - config_service = MemoryConfigService(db) - cfg_dict = config_service.get_embedder_config(SELECTED_EMBEDDING_ID) - embedder = OpenAIEmbedderClient( - model_config=RedBearModelConfig.model_validate(cfg_dict) - ) - - # connector initialized above - latencies_llm: List[float] = [] - latencies_search: List[float] = [] - # 上下文诊断收集 - per_query_context_counts: List[int] = [] - per_query_context_avg_tokens: List[float] = [] - per_query_context_chars: List[int] = [] - per_query_context_tokens_total: List[int] = [] - # 详细样本调试信息 - samples: List[Dict[str, Any]] = [] - # 通用指标 - f1s: List[float] = [] - b1s: List[float] = [] - jss: List[float] = [] - # 参考 LoCoMo 评测的类别专用 F1（multi-hop 使用多答案 F1） - loc_f1s: List[float] = [] - # Per-category aggregation - cat_counts: Dict[str, int] = {} - cat_f1s: Dict[str, List[float]] = {} - cat_b1s: Dict[str, List[float]] = {} - cat_jss: Dict[str, List[float]] = {} - cat_loc_f1s: Dict[str, List[float]] = {} - try: - for item in items: - q = item.get("question", "") - ref = item.get("answer", "") - # 确保答案是字符串 - ref_str = str(ref) if ref is not None else "" - cat = get_category_label(item) - - print(f"\n=== 处理问题: {q} ===") - - # 根据类别调整检索参数 - search_params = get_search_params_by_category(cat) - adjusted_limit = search_params["limit"] - max_chars = search_params["max_chars"] - - print(f"🏷️ 类别: {cat}, 检索参数: limit={adjusted_limit}, max_chars={max_chars}") - - # 改进的检索逻辑：使用三路检索（statements, dialogues, entities） - t0 = time.time() - contexts_all: List[str] = [] - search_results = None # 保存完整的检索结果 - - try: - if search_type == "embedding": - # 直接调用嵌入检索，包含三路数据 - search_results = await search_graph_by_embedding( - connector=connector, - embedder_client=embedder, - query_text=q, - group_id=group_id, - limit=adjusted_limit, - include=["chunks", "statements", "entities", "summaries"], # 修复：使用正确的类型 - ) - chunks = search_results.get("chunks", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - summaries = search_results.get("summaries", []) - - print(f"✅ 嵌入检索成功: {len(chunks)} chunks, {len(statements)} 条陈述, {len(entities)} 个实体, {len(summaries)} 个摘要") - - # 构建上下文：优先使用 chunks、statements 和 summaries - for c in chunks: - content = str(c.get("content", "")).strip() - if content: - contexts_all.append(content) - - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - - for sm in summaries: - summary_text = str(sm.get("summary", "")).strip() - if summary_text: - contexts_all.append(summary_text) - - # 实体摘要：最多加入前3个高分实体，避免噪声 - scored = [e for e in entities if e.get("score") is not None] - top_entities = sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] if scored else entities[:3] - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append(f"EntitySummary: {name}{(' [' + '; '.join(meta) + ']') if meta else ''}") - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - - elif search_type == "keyword": - # 直接调用关键词检索 - search_results = await search_graph( - connector=connector, - q=q, - group_id=group_id, - limit=adjusted_limit - ) - dialogs = search_results.get("dialogues", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - print(f"🔤 关键词检索找到 {len(dialogs)} 条对话, {len(statements)} 条陈述, {len(entities)} 个实体") - - # 构建上下文 - for d in dialogs: - content = str(d.get("content", "")).strip() - if content: - contexts_all.append(content) - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - # 实体处理（关键词检索的实体可能没有分数） - if entities: - entity_names = [str(e.get("name", "")).strip() for e in entities[:5] if e.get("name")] - if entity_names: - contexts_all.append(f"EntitySummary: {', '.join(entity_names)}") - - else: # hybrid - # 🎯 关键修复：混合检索使用更严格的回退机制 - print("🔀 使用混合检索（带回退机制）...") - try: - search_results = await run_hybrid_search( - query_text=q, - search_type=search_type, - group_id=group_id, - limit=adjusted_limit, - include=["chunks", "statements", "entities", "summaries"], - output_path=None, - ) - - # 🎯 关键修复：正确处理混合检索的扁平结构 - # 新的API返回扁平结构，直接从顶层获取结果 - if search_results and isinstance(search_results, dict): - # 新API返回扁平结构：直接从顶层获取 - chunks = search_results.get("chunks", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - summaries = search_results.get("summaries", []) - - # 检查是否有有效结果 - if chunks or statements or entities or summaries: - print(f"✅ 混合检索成功: {len(chunks)} chunks, {len(statements)} 陈述, {len(entities)} 实体, {len(summaries)} 摘要") - else: - # 如果顶层没有结果，尝试旧的嵌套结构（向后兼容） - reranked = search_results.get("reranked_results", {}) - if reranked and isinstance(reranked, dict): - chunks = reranked.get("chunks", []) - statements = reranked.get("statements", []) - entities = reranked.get("entities", []) - summaries = reranked.get("summaries", []) - print(f"✅ 混合检索成功（使用旧格式reranked结果）: {len(chunks)} chunks, {len(statements)} 陈述") - else: - raise ValueError("混合检索返回空结果") - else: - raise ValueError("混合检索返回空结果") - - except Exception as e: - print(f"❌ 混合检索失败: {e}，回退到嵌入检索") - search_results = await search_graph_by_embedding( - connector=connector, - embedder_client=embedder, - query_text=q, - group_id=group_id, - limit=adjusted_limit, - include=["chunks", "statements", "entities", "summaries"], - ) - chunks = search_results.get("chunks", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - summaries = search_results.get("summaries", []) - print(f"✅ 回退嵌入检索成功: {len(chunks)} chunks, {len(statements)} 陈述") - - # 🎯 统一处理：构建上下文（所有检索类型共用） - for c in chunks: - content = str(c.get("content", "")).strip() - if content: - contexts_all.append(content) - - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - - for sm in summaries: - summary_text = str(sm.get("summary", "")).strip() - if summary_text: - contexts_all.append(summary_text) - - # 实体摘要：最多加入前3个高分实体 - if entities: - scored = [e for e in entities if e.get("score") is not None] - top_entities = sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] if scored else entities[:3] - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append(f"EntitySummary: {name}{(' [' + '; '.join(meta) + ']') if meta else ''}") - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - - # 关键修复：过滤掉包含当前问题答案的上下文 - filtered_contexts = [] - for context in contexts_all: - content = str(context) - # 排除包含当前问题标准答案的上下文 - if ref_str and ref_str.strip() and ref_str.strip() in content: - print("🚫 过滤掉包含标准答案的上下文") - continue - filtered_contexts.append(context) - - print(f"📊 过滤后保留 {len(filtered_contexts)} 个上下文 (原 {len(contexts_all)} 个)") - contexts_all = filtered_contexts - - # 输出完整的检索结果信息 - print("🔍 检索结果详情:") - if search_results: - output_data = { - "statements": [ - { - "statement": s.get("statement", "")[:200] + "..." if len(s.get("statement", "")) > 200 else s.get("statement", ""), - "score": s.get("score", 0.0) - } - for s in (statements[:2] if 'statements' in locals() else []) - ], - "dialogues": [ - { - "uuid": d.get("uuid", ""), - "group_id": d.get("group_id", ""), - "content": d.get("content", "")[:200] + "..." if len(d.get("content", "")) > 200 else d.get("content", ""), - "score": d.get("score", 0.0) - } - for d in (dialogs[:2] if 'dialogs' in locals() else []) - ], - "entities": [ - { - "name": e.get("name", ""), - "entity_type": e.get("entity_type", ""), - "score": e.get("score", 0.0) - } - for e in (entities[:2] if 'entities' in locals() else []) - ] - } - print(json.dumps(output_data, ensure_ascii=False, indent=2)) - else: - print(" 无检索结果") - - except Exception as e: - print(f"❌ {search_type}检索失败: {e}") - contexts_all = [] - search_results = None - - t1 = time.time() - latencies_search.append((t1 - t0) * 1000) - - # 使用智能上下文选择 - context_text = "" - if contexts_all: - context_text = smart_context_selection(contexts_all, q, max_chars=max_chars) - - # 如果智能选择后仍然过长，进行最终保护性截断 - if len(context_text) > max_chars: - print(f"⚠️ 智能选择后仍然过长 ({len(context_text)}字符)，进行最终截断") - context_text = context_text[:max_chars] + "\n\n[最终截断...]" - - # 时间解析 - anchor_date = datetime(2023, 5, 8) # 使用固定日期确保一致性 - context_text = _resolve_relative_times(context_text, anchor_date) - - context_text = f"Reference date: {anchor_date.date().isoformat()}\n\n" + context_text - - print(f"📝 最终上下文长度: {len(context_text)} 字符") - - # 显示不同上下文的预览 - print("🔍 上下文预览:") - for j, context in enumerate(contexts_all[:3]): # 显示前3个上下文 - preview = context[:150].replace('\n', ' ') - print(f" 上下文{j+1}: {preview}...") - - else: - print("❌ 没有检索到有效上下文") - context_text = "No relevant context found." - - # 记录上下文诊断信息 - per_query_context_counts.append(len(contexts_all)) - per_query_context_avg_tokens.append(avg_context_tokens([context_text])) - per_query_context_chars.append(len(context_text)) - per_query_context_tokens_total.append(len(_loc_normalize(context_text).split())) - - # LLM 提示词 - messages = [ - {"role": "system", "content": ( - "You are a precise QA assistant. Answer following these rules:\n" - "1) Extract the EXACT information mentioned in the context\n" - "2) For time questions: calculate actual dates from relative times\n" - "3) Return ONLY the answer text in simplest form\n" - "4) For dates, use format 'DD Month YYYY' (e.g., '7 May 2023')\n" - "5) If no clear answer found, respond with 'Unknown'" - )}, - {"role": "user", "content": f"Question: {q}\n\nContext:\n{context_text}"}, - ] - - t2 = time.time() - # 使用异步调用 - resp = await llm_client.chat(messages=messages) - t3 = time.time() - latencies_llm.append((t3 - t2) * 1000) - - # 兼容不同的响应格式 - pred = resp.content.strip() if hasattr(resp, 'content') else (resp["choices"][0]["message"]["content"].strip() if isinstance(resp, dict) else "Unknown") - - # 计算指标（确保使用字符串） - f1_val = common_f1(str(pred), ref_str) - b1_val = bleu1(str(pred), ref_str) - j_val = jaccard(str(pred), ref_str) - - f1s.append(f1_val) - b1s.append(b1_val) - jss.append(j_val) - - # Accumulate by category - cat_counts[cat] = cat_counts.get(cat, 0) + 1 - cat_f1s.setdefault(cat, []).append(f1_val) - cat_b1s.setdefault(cat, []).append(b1_val) - cat_jss.setdefault(cat, []).append(j_val) - - # LoCoMo 专用 F1：multi-hop(1) 使用多答案 F1，其它(2/3/4)使用单答案 F1 - if item.get("category") in [2, 3, 4]: - loc_val = loc_f1_score(str(pred), ref_str) - elif item.get("category") in [1]: - loc_val = loc_multi_f1(str(pred), ref_str) - else: - loc_val = loc_f1_score(str(pred), ref_str) - loc_f1s.append(loc_val) - cat_loc_f1s.setdefault(cat, []).append(loc_val) - - # 保存完整的检索结果信息 - samples.append({ - "question": q, - "answer": ref_str, - "category": cat, - "prediction": pred, - "metrics": { - "f1": f1_val, - "b1": b1_val, - "j": j_val, - "loc_f1": loc_val - }, - "retrieval": { - "retrieved_documents": len(contexts_all), - "context_length": len(context_text), - "search_limit": adjusted_limit, - "max_chars": max_chars - }, - "timing": { - "search_ms": (t1 - t0) * 1000, - "llm_ms": (t3 - t2) * 1000 - } - }) - - print(f"🤖 LLM 回答: {pred}") - print(f"✅ 正确答案: {ref_str}") - print(f"📈 当前指标 - F1: {f1_val:.3f}, BLEU-1: {b1_val:.3f}, Jaccard: {j_val:.3f}, LoCoMo F1: {loc_val:.3f}") - - # Compute per-category averages and dispersion (std, iqr) - def _percentile(sorted_vals: List[float], p: float) -> float: - if not sorted_vals: - return 0.0 - if len(sorted_vals) == 1: - return sorted_vals[0] - k = (len(sorted_vals) - 1) * p - f = int(k) - c = f + 1 if f + 1 < len(sorted_vals) else f - if f == c: - return sorted_vals[f] - return sorted_vals[f] + (sorted_vals[c] - sorted_vals[f]) * (k - f) - - by_category: Dict[str, Dict[str, float | int]] = {} - for c in cat_counts: - f_list = cat_f1s.get(c, []) - b_list = cat_b1s.get(c, []) - j_list = cat_jss.get(c, []) - lf_list = cat_loc_f1s.get(c, []) - j_sorted = sorted(j_list) - j_std = statistics.stdev(j_list) if len(j_list) > 1 else 0.0 - j_q75 = _percentile(j_sorted, 0.75) - j_q25 = _percentile(j_sorted, 0.25) - by_category[c] = { - "count": cat_counts[c], - "f1": (sum(f_list) / max(len(f_list), 1)) if f_list else 0.0, - "b1": (sum(b_list) / max(len(b_list), 1)) if b_list else 0.0, - "j": (sum(j_list) / max(len(j_list), 1)) if j_list else 0.0, - "j_std": j_std, - "j_iqr": (j_q75 - j_q25) if j_list else 0.0, - # 参考 LoCoMo 评测的类别专用 F1 - "loc_f1": (sum(lf_list) / max(len(lf_list), 1)) if lf_list else 0.0, - } - - # 累加命中（cum accuracy by category）：与 evaluation_stats.py 输出形式相仿 - cum_accuracy_by_category = {c: sum(cat_loc_f1s.get(c, [])) for c in cat_counts} - - result = { - "dataset": "locomo", - "items": len(items), - "metrics": { - "f1": sum(f1s) / max(len(f1s), 1), - "b1": sum(b1s) / max(len(b1s), 1), - "j": sum(jss) / max(len(jss), 1), - # LoCoMo 类别专用 F1 的总体 - "loc_f1": sum(loc_f1s) / max(len(loc_f1s), 1), - }, - "by_category": by_category, - "category_counts": cat_counts, - "cum_accuracy_by_category": cum_accuracy_by_category, - "context": { - "avg_tokens": (sum(per_query_context_avg_tokens) / max(len(per_query_context_avg_tokens), 1)) if per_query_context_avg_tokens else 0.0, - "avg_chars": (sum(per_query_context_chars) / max(len(per_query_context_chars), 1)) if per_query_context_chars else 0.0, - "count_avg": (sum(per_query_context_counts) / max(len(per_query_context_counts), 1)) if per_query_context_counts else 0.0, - "avg_memory_tokens": (sum(per_query_context_tokens_total) / max(len(per_query_context_tokens_total), 1)) if per_query_context_tokens_total else 0.0, - }, - "latency": { - "search": latency_stats(latencies_search), - "llm": latency_stats(latencies_llm), - }, - "samples": samples, - "params": { - "group_id": group_id, - "search_limit": search_limit, - "context_char_budget": context_char_budget, - "search_type": search_type, - "llm_id": SELECTED_LLM_ID, - "retrieval_embedding_id": SELECTED_EMBEDDING_ID, - "skip_ingest_if_exists": skip_ingest_if_exists, - "llm_timeout": llm_timeout, - "llm_max_retries": llm_max_retries, - "llm_temperature": llm_temperature, - "llm_max_tokens": llm_max_tokens - }, - "timestamp": datetime.now().isoformat() - } - if output_path: - try: - os.makedirs(os.path.dirname(output_path), exist_ok=True) - with open(output_path, "w", encoding="utf-8") as f: - json.dump(result, f, ensure_ascii=False, indent=2) - print(f"✅ 结果已保存到: {output_path}") - except Exception as e: - print(f"❌ 保存结果失败: {e}") - return result - finally: - await connector.close() - - -def main(): - parser = argparse.ArgumentParser(description="Run LoCoMo evaluation with Qwen search") - parser.add_argument("--sample_size", type=int, default=1, help="Number of samples to evaluate") - parser.add_argument("--group_id", type=str, default=None, help="Group ID for retrieval") - parser.add_argument("--search_limit", type=int, default=8, help="Search limit per query") - parser.add_argument("--context_char_budget", type=int, default=12000, help="Max characters for context") - parser.add_argument("--llm_temperature", type=float, default=0.0, help="LLM temperature") - parser.add_argument("--llm_max_tokens", type=int, default=32, help="LLM max tokens") - parser.add_argument("--search_type", type=str, default="embedding", choices=["keyword", "embedding", "hybrid"], help="Search type") - parser.add_argument("--output_path", type=str, default=None, help="Output path for results") - parser.add_argument("--skip_ingest_if_exists", action="store_true", help="Skip ingest if group exists") - parser.add_argument("--llm_timeout", type=float, default=10.0, help="LLM timeout in seconds") - parser.add_argument("--llm_max_retries", type=int, default=1, help="LLM max retries") - args = parser.parse_args() - - load_dotenv() - - result = asyncio.run(run_locomo_eval( - sample_size=args.sample_size, - group_id=args.group_id, - search_limit=args.search_limit, - context_char_budget=args.context_char_budget, - llm_temperature=args.llm_temperature, - llm_max_tokens=args.llm_max_tokens, - search_type=args.search_type, - output_path=args.output_path, - skip_ingest_if_exists=args.skip_ingest_if_exists, - llm_timeout=args.llm_timeout, - llm_max_retries=args.llm_max_retries - )) - - print("\n" + "="*50) - print("📊 最终评测结果:") - print(f" 样本数量: {result['items']}") - print(f" F1: {result['metrics']['f1']:.3f}") - print(f" BLEU-1: {result['metrics']['b1']:.3f}") - print(f" Jaccard: {result['metrics']['j']:.3f}") - print(f" LoCoMo F1: {result['metrics']['loc_f1']:.3f}") - print(f" 平均上下文长度: {result['context']['avg_chars']:.0f} 字符") - print(f" 平均检索延迟: {result['latency']['search']['mean']:.1f}ms") - print(f" 平均LLM延迟: {result['latency']['llm']['mean']:.1f}ms") - - if result['by_category']: - print("\n📈 按类别细分:") - for cat, metrics in result['by_category'].items(): - print(f" {cat}:") - print(f" 样本数: {metrics['count']}") - print(f" F1: {metrics['f1']:.3f}") - print(f" LoCoMo F1: {metrics['loc_f1']:.3f}") - print(f" Jaccard: {metrics['j']:.3f} (±{metrics['j_std']:.3f}, IQR={metrics['j_iqr']:.3f})") - - -if __name__ == "__main__": - main() diff --git a/api/app/core/memory/evaluation/longmemeval/qwen_search_eval.py b/api/app/core/memory/evaluation/longmemeval/qwen_search_eval.py deleted file mode 100644 index 53c5ce19..00000000 --- a/api/app/core/memory/evaluation/longmemeval/qwen_search_eval.py +++ /dev/null @@ -1,1363 +0,0 @@ -import argparse -import asyncio -import json -import os -import re -import statistics -import time -from datetime import datetime, timedelta -from typing import Any, Dict, List - -try: - from dotenv import load_dotenv -except Exception: - def load_dotenv(): - return None - -# 确保可以找到 src 及项目根路径 -import sys - -_THIS_DIR = os.path.dirname(os.path.abspath(__file__)) -_PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.dirname(_THIS_DIR))) -_SRC_DIR = os.path.join(_PROJECT_ROOT, "src") -for _p in (_SRC_DIR, _PROJECT_ROOT): - if _p not in sys.path: - sys.path.insert(0, _p) - -# 与现有评估脚本保持一致的导入方式 -from app.repositories.neo4j.neo4j_connector import Neo4jConnector - -try: - # 优先从 extraction_utils1 导入 - from app.core.memory.evaluation.extraction_utils import ( - ingest_contexts_via_full_pipeline, # type: ignore - ) -except Exception: - ingest_contexts_via_full_pipeline = None # 在运行时做兜底检查 -from app.core.memory.evaluation.common.metrics import ( - avg_context_tokens, - jaccard, - latency_stats, -) -from app.core.memory.evaluation.common.metrics import f1_score as common_f1 -from app.core.memory.evaluation.dialogue_queries import SEARCH_ENTITIES_BY_NAME -from app.core.memory.llm_tools.openai_embedder import OpenAIEmbedderClient -from app.core.memory.utils.config.definitions import ( - PROJECT_ROOT, - SELECTED_EMBEDDING_ID, - SELECTED_LLM_ID, -) -from app.core.memory.utils.llm.llm_utils import MemoryClientFactory -from app.core.models.base import RedBearModelConfig -from app.db import get_db_context -from app.repositories.neo4j.graph_search import search_graph, search_graph_by_embedding -from app.services.memory_config_service import MemoryConfigService - -try: - from app.core.memory.evaluation.common.metrics import exact_match -except Exception: - # 兜底：简单的大小写不敏感比较 - def exact_match(pred: str, ref: str) -> bool: - return str(pred).strip().lower() == str(ref).strip().lower() - - -def load_dataset_any(path: str) -> List[Dict[str, Any]]: - """健壮地加载数据集（兼容 list 或多段 JSON）。""" - with open(path, "r", encoding="utf-8") as f: - s = f.read().strip() - try: - obj = json.loads(s) - if isinstance(obj, list): - return obj - elif isinstance(obj, dict): - return [obj] - except json.JSONDecodeError: - pass - dec = json.JSONDecoder() - idx = 0 - items: List[Dict[str, Any]] = [] - while idx < len(s): - while idx < len(s) and s[idx].isspace(): - idx += 1 - if idx >= len(s): - break - try: - obj, end = dec.raw_decode(s, idx) - if isinstance(obj, list): - for it in obj: - if isinstance(it, dict): - items.append(it) - elif isinstance(obj, dict): - items.append(obj) - idx = end - except json.JSONDecodeError: - nl = s.find("\n", idx) - if nl == -1: - break - idx = nl + 1 - return items - - -def is_chinese_text(s: str) -> bool: - return bool(re.search(r"[\u4e00-\u9fff]", s or "")) - - -def build_context_from_sessions(item: Dict[str, Any]) -> List[str]: - """从数据项的 haystack_sessions 构建上下文片段。 - - 优先返回包含 has_answer 的消息 - - 其次返回拼接后的整段会话 - """ - contexts: List[str] = [] - sessions = item.get("haystack_sessions", []) or item.get("sessions", []) - for session in sessions: - parts: List[str] = [] - if isinstance(session, list): - for msg in session: - role = msg.get("role", "") - content = msg.get("content", "") or msg.get("text", "") - if content: - parts.append(f"{role}: {content}" if role else str(content)) - if msg.get("has_answer", False): - contexts.append(f"{role}: {content}" if role else str(content)) - elif isinstance(session, dict): - role = session.get("role", "") - content = session.get("content", "") or session.get("text", "") - if content: - parts.append(f"{role}: {content}" if role else str(content)) - if session.get("has_answer", False): - contexts.append(f"{role}: {content}" if role else str(content)) - if parts: - contexts.append("\n".join(parts)) - # 兜底：存在单字段上下文 - if not contexts: - single_ctx = item.get("context") or item.get("dialogue") or item.get("conversation") - if isinstance(single_ctx, str) and single_ctx.strip(): - contexts.append(single_ctx.strip()) - return contexts - - -def extract_candidate_options(question: str) -> List[str]: - """从问题中提取候选选项（A-or-B 类问题）。""" - q = (question or "").strip() - options: List[str] = [] - - # 1) 引号包裹的片段 - for pat in [r"'([^']+)'", r'\"([^\"]+)\"', r'“([^”]+)”', r'‘([^’]+)’']: - for m in re.findall(pat, q): - val = (m or "").strip() - if val: - options.append(val) - - # 2) or/还是/或者 连接词 - if len(options) < 2: - pats = [ - r"([^,;，；]+?)\s+or\s+([^,;，；\?\.!.。！]+)", - r"([^,;，；]+?)\s+还是\s+([^,;，；\?\.!.。！]+)", - r"([^,;，；]+?)\s+或者\s+([^,;，；\?\.!.。！]+)", - ] - for pat in pats: - matches = list(re.finditer(pat, q, flags=re.IGNORECASE)) - if matches: - m = matches[-1] - cand1 = m.group(1).strip().strip("?？.,，;； ") - cand2 = m.group(2).strip().strip("?？.,，;； ") - options.extend([cand1, cand2]) - break - - # 去重 - seen = set() - uniq: List[str] = [] - for o in options: - o2 = o.strip() - key = o2.lower() if not is_chinese_text(o2) else o2 - if o2 and key not in seen: - uniq.append(o2) - seen.add(key) - return uniq - - -def extract_time_entities(text: str) -> List[Dict[str, Any]]: - """增强时间实体提取，专门用于时间推理问题""" - time_entities = [] - - # 日期模式 - date_patterns = [ - (r'\b(\d{4})-(\d{1,2})-(\d{1,2})\b', 'date'), # YYYY-MM-DD - (r'\b(\d{1,2})月(\d{1,2})日\b', 'date'), # 中文日期 - (r'\b(January|February|March|April|May|June|July|August|September|October|November|December)\s+(\d{1,2}),?\s+(\d{4})?', 'date'), # 英文月份 - (r'\b(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\s+(\d{1,2}),?\s+(\d{4})?', 'date'), # 英文月份缩写 - ] - - # 时间间隔模式 - duration_patterns = [ - (r'(\d+)\s*天', 'days'), - (r'(\d+)\s*周', 'weeks'), - (r'(\d+)\s*个月', 'months'), - (r'(\d+)\s*年', 'years'), - (r'(\d+)\s*days?', 'days'), - (r'(\d+)\s*weeks?', 'weeks'), - (r'(\d+)\s*months?', 'months'), - (r'(\d+)\s*years?', 'years'), - ] - - # 事件时间关系模式 - temporal_relation_patterns = [ - (r'(之前|以前|前)\s*(\d+)\s*天', 'days_before'), - (r'(之后|以后|后)\s*(\d+)\s*天', 'days_after'), - (r'(\d+)\s*天\s*(之前|以前|前)', 'days_before'), - (r'(\d+)\s*天\s*(之后|以后|后)', 'days_after'), - (r'(\d+)\s*days?\s*(before|ago)', 'days_before'), - (r'(\d+)\s*days?\s*(after|later)', 'days_after'), - ] - - # 提取日期 - for pattern, entity_type in date_patterns: - matches = re.finditer(pattern, text, re.IGNORECASE) - for match in matches: - time_entities.append({ - 'text': match.group(), - 'type': entity_type, - 'start': match.start(), - 'end': match.end() - }) - - # 提取时间间隔 - for pattern, entity_type in duration_patterns: - matches = re.finditer(pattern, text, re.IGNORECASE) - for match in matches: - time_entities.append({ - 'text': match.group(), - 'type': entity_type, - 'value': int(match.group(1)), - 'start': match.start(), - 'end': match.end() - }) - - # 提取时间关系 - for pattern, entity_type in temporal_relation_patterns: - matches = re.finditer(pattern, text, re.IGNORECASE) - for match in matches: - time_entities.append({ - 'text': match.group(), - 'type': entity_type, - 'value': int(match.group(2)) if match.groups() >= 2 else int(match.group(1)), - 'start': match.start(), - 'end': match.end() - }) - - return time_entities - - -def calculate_time_difference(date1: str, date2: str) -> int: - """计算两个日期之间的天数差""" - try: - # 解析日期格式 - def parse_date(date_str: str) -> datetime: - # 尝试多种日期格式 - formats = [ - '%Y-%m-%d', - '%m月%d日', - '%B %d, %Y', - '%b %d, %Y', - '%Y年%m月%d日' - ] - - for fmt in formats: - try: - return datetime.strptime(date_str, fmt) - except ValueError: - continue - - # 如果都无法解析，返回当前日期 - return datetime.now() - - d1 = parse_date(date1) - d2 = parse_date(date2) - - # 计算天数差（绝对值） - return abs((d2 - d1).days) - except Exception: - return -1 # 表示计算失败 - - -def smart_context_selection(contexts: List[str], question: str, max_chars: int = 4000) -> str: - """增强版上下文选择：特别优化时间推理问题的处理""" - if not contexts: - return "" - - # 检测是否为时间推理问题 - is_temporal_question = any(keyword in question.lower() for keyword in - ['days', 'day', 'before', 'after', 'first', '先后', '顺序', '间隔', '多久', '多少天']) - - # 提取时间实体从问题中 - question_time_entities = extract_time_entities(question) - - # 英文关键词（去停用词） - question_lower = question.lower() - stop_words = { - 'what','when','where','who','why','how','did','do','does','is','are','was','were', - 'the','a','an','and','or','but','many','which','first' - } - eng_words = [w for w in set(re.findall(r'\b\w+\b', question_lower)) - if w not in stop_words and len(w) > 2] - - # 中文片段与候选选项 - cn_tokens = generate_query_keywords_cn(question) - options = extract_candidate_options(question) - - # 时间推理问题的特殊处理 - if is_temporal_question: - # 为时间问题添加时间相关关键词 - time_keywords = ['天', '日', '月', '年', 'before', 'after', 'days', 'first', '先后'] - eng_words = [w for w in eng_words if w not in ['days', 'first']] # 避免重复 - cn_tokens.extend([kw for kw in time_keywords if kw not in cn_tokens]) - - # 限制关键词数量，优先时间相关 - tokens = time_keywords[:2] + cn_tokens[:2] + eng_words[:1] + options[:1] - else: - # 常规问题处理 - tokens = cn_tokens[:3] + options[:2] + eng_words[:1] - - # 去重 - seen = set() - final_tokens: List[str] = [] - for t in tokens: - t2 = t.strip() - if t2 and t2 not in seen: - final_tokens.append(t2) - seen.add(t2) - - scored_contexts: List[tuple[float, str]] = [] - - # 时间推理问题的权重映射 - temporal_weight_map = { - "天": 2.0, "日": 2.0, "月": 1.8, "年": 1.8, "days": 2.0, - "before": 1.5, "after": 1.5, "first": 1.5, "先后": 1.5 - } - - # 常规问题的权重映射 - normal_weight_map = { - "问题": 2.0, "故障": 2.0, "异常": 1.8, "不正常": 1.8, "坏了": 1.8, - "系统": 1.3, "GPS": 1.5, "保养": 1.4, "设备": 1.2, "模块": 1.2, "功能": 1.1 - } - - weight_map = temporal_weight_map if is_temporal_question else normal_weight_map - - for i, context in enumerate(contexts): - context_str = str(context) - lines = re.split(r'[\r\n]+', context_str) - hit_lines: List[str] = [] - kw_hits: float = 0.0 - time_entity_count = 0 - - for line in lines: - ln = line.strip() - if not ln: - continue - - has_keyword = False - # 关键词匹配 - for tok in final_tokens: - if tok and tok in ln: - w = weight_map.get(tok, 1.0) - kw_hits += ln.count(tok) * w - has_keyword = True - - # 时间实体检测（特别针对时间推理问题） - if is_temporal_question: - time_entities = extract_time_entities(ln) - time_entity_count += len(time_entities) - if time_entities: - has_keyword = True - - if has_keyword: - # 对于时间推理问题，保留包含时间信息的完整行 - hit_lines.append(ln) - - snippet = "\n".join(hit_lines) if hit_lines else context_str.strip() - - # 限制单段长度，但对时间推理问题稍微放宽限制 - max_snippet_len = 600 if is_temporal_question else 500 - if len(snippet) > max_snippet_len: - snippet = snippet[:max_snippet_len] - - # 评分逻辑 - has_number = 1 if re.search(r'\d', snippet) else 0 - has_date = 1 if (re.search(r'\b\d{4}-\d{1,2}-\d{1,2}\b', snippet) or - re.search(r'\d{1,2}月\d{1,2}日', snippet)) else 0 - - # 时间推理问题的特殊评分 - if is_temporal_question: - time_bonus = time_entity_count * 2.0 # 时间实体奖励 - temporal_coherence = 3 if (has_date and time_entity_count >= 2) else 0 - else: - time_bonus = 0 - temporal_coherence = 0 - - length_bonus = 5 if 50 < len(snippet) < 1000 else (2 if len(snippet) >= 1000 else 0) - pos_bonus = 3 if i < 3 else 0 - - score = (kw_hits * 0.8 + (has_number + has_date) * 1.5 + - length_bonus + pos_bonus + time_bonus + temporal_coherence) - - scored_contexts.append((score, snippet)) - - # 选择累计至总字符预算 - scored_contexts.sort(key=lambda x: x[0], reverse=True) - selected: List[str] = [] - total_chars = 0 - - for score, snippet in scored_contexts: - if total_chars + len(snippet) <= max_chars: - selected.append(snippet) - total_chars += len(snippet) - else: - if not selected and len(snippet) > max_chars: - selected.append(snippet[:max_chars]) - break - - final_context = "\n\n".join(selected) - - # 对于时间推理问题，添加时间计算提示 - if is_temporal_question and question_time_entities: - time_prompt = "\n\n[时间推理提示：请仔细分析上述上下文中的日期和时间关系，计算时间间隔或确定事件顺序]" - if total_chars + len(time_prompt) <= max_chars: - final_context += time_prompt - - return final_context - - -# 中文关键词提取（短语级，含数词/日期/常见领域词） -def _extract_cn_tokens(text: str) -> List[str]: - if not text: - return [] - t = str(text) - # 去掉常见功能词（粗略，不依赖分词库） - stop_words = [ - "我","我们","你","他","她","它","这","那","哪","一个","一次","一些","什么","怎么","是否","吗","呢", - "很","更","最","已经","正在","将要","马上","尽快","最近","关于","有关","以及","并且","或者","还是", - "因为","所以","如果","但是","而且","然后","之后","之前","同时","另外","并","但","却","被","把","让","给", - "和","与","跟","及","还有","就","都","在","对","对于","的","了","着","过","到","于","从","以","为","向","至","是" - ] - for sw in stop_words: - t = t.replace(sw, " ") - # 去标点 - t = re.sub(r"[，。！？、；：,.!?;:\"'（）()［］\[\]\-—…·]", " ", t) - # 基础中文片段（>=2） - base = re.findall(r"[\u4e00-\u9fff]{2,}", t) - # 特殊组合：第X次XXXX - specials = re.findall(r"第[一二三四五六七八九十]+次[\u4e00-\u9fff]{2,6}", text) - # 领域词（简单词典） - # 日期与数字 - dates = re.findall(r"\d{4}年\d{1,2}月\d{1,2}日|\d{1,2}月\d{1,2}日|\d{4}-\d{1,2}-\d{1,2}", text) - numbers = re.findall(r"\b\d+\b", text) - - tokens: List[str] = specials + base + dates + numbers - - generic = {"建议","推荐","帮助","提升","技能","有效","团队","参与度","喜欢","开始"} - tokens: List[str] = specials + base + dates + numbers - uniq: List[str] = [] - seen = set() - for tok in tokens: - tok2 = tok.strip() - if len(tok2) < 2 or len(tok2) > 6: - continue - if tok2 in generic: - continue - if tok2 not in seen: - uniq.append(tok2) - seen.add(tok2) - # 排除常见疑问型短语 - blacklist_exact = {"是什么","多少","多少天","哪个","哪些","之间","先","后","之前","之后"} - uniq2: List[str] = [u for u in uniq if u not in blacklist_exact] - return uniq2[:12] - - -# 面向检索的中文关键词生成：强调"短语、核心名词、问题/故障" -def generate_query_keywords_cn(question: str) -> List[str]: - if not question: - return [] - raw = _extract_cn_tokens(question) - core: List[str] = [] - seen = set() - - def push(x: str): - x2 = x.strip() - if not x2: - return - if 2 <= len(x2) <= 6 and x2 not in seen: - core.append(x2) - seen.add(x2) - - # 检测时间推理问题 - is_temporal = any(keyword in question for keyword in ['天', '日', 'before', 'after', 'first', '先后', '间隔']) - if is_temporal: - push("天") - push("日") - push("先后") - - # 明确优先的核心词 - if "新车" in question: - push("新车") - # 第X次保养/维修 - specials = re.findall(r"第[一二三四五六七八九十]+次[\u4e00-\u9fff]{2,6}", question) - for s in specials: - if "保养" in s or "维修" in s: - push(s) - if "保养" in question: - push("保养") - # 问题/故障类词，如题含"问题"则扩展同义词 - if "问题" in question: - for w in ["问题","故障","异常","不正常"]: - push(w) - - # 补充：从原始片段筛更短的名词短语（过滤疑问型词） - blacklist = {"是什么","多少","哪个","还是","或者","之间","先","后","之前","之后"} - for tok in raw: - if tok in blacklist: - continue - push(tok) - - # 限制数量，避免过长列表影响检索稳定性 - return core[:4] # 稍微增加限制 - - -# 通过别名匹配进行实体关键词检索（多token合并） -async def _search_entities_by_aliases(connector: Neo4jConnector, tokens: List[str], group_id: str | None, limit: int) -> List[Dict[str, Any]]: - results: List[Dict[str, Any]] = [] - try: - for tok in tokens: - rows = await connector.execute_query(SEARCH_ENTITIES_BY_NAME, q=tok, group_id=group_id, limit=limit) - if rows: - results.extend(rows) - except Exception: - pass - - # 按 name 去重 - deduped: List[Dict[str, Any]] = [] - seen = set() - for r in results: - k = str(r.get("name", "")) - if k and k not in seen: - deduped.append(r) - seen.add(k) - return deduped - - -# 通过对话/陈述中的entity_ids反查实体名称 -_FETCH_ENTITIES_BY_IDS = """ -MATCH (e:ExtractedEntity) -WHERE e.id IN $ids AND ($group_id IS NULL OR e.group_id = $group_id) -RETURN e.id AS id, e.name AS name, e.group_id AS group_id, e.entity_type AS entity_type -""" - -async def _fetch_entities_by_ids(connector: Neo4jConnector, ids: List[str], group_id: str | None) -> List[Dict[str, Any]]: - if not ids: - return [] - try: - rows = await connector.execute_query(_FETCH_ENTITIES_BY_IDS, ids=list({i for i in ids if i}), group_id=group_id) - return rows or [] - except Exception: - return [] - - -# 增强的时间实体检索 -_TIME_ENTITY_SEARCH = """ -MATCH (e:ExtractedEntity) -WHERE e.entity_type CONTAINS "TIME" OR e.entity_type CONTAINS "DATE" OR e.name =~ $date_pattern -AND ($group_id IS NULL OR e.group_id = $group_id) -RETURN e.id AS id, e.name AS name, e.group_id AS group_id, e.entity_type AS entity_type -LIMIT $limit -""" - -async def _search_time_entities(connector: Neo4jConnector, group_id: str | None, limit: int = 5) -> List[Dict[str, Any]]: - """专门搜索时间相关的实体""" - try: - date_pattern = r".*\d{4}.*|.*\d{1,2}月\d{1,2}日.*" - rows = await connector.execute_query(_TIME_ENTITY_SEARCH, - date_pattern=date_pattern, - group_id=group_id, - limit=limit) - return rows or [] - except Exception: - return [] - - -# 中英相对时间解析：today/昨天/上周/3天后 等简单归一化为日期 -def _resolve_relative_times_cn_en(text: str, anchor: datetime) -> str: - t = str(text) if text is not None else "" - # 英文 today/yesterday/tomorrow - t = re.sub(r"\btoday\b", anchor.date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\byesterday\b", (anchor - timedelta(days=1)).date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\btomorrow\b", (anchor + timedelta(days=1)).date().isoformat(), t, flags=re.IGNORECASE) - - # 英文 X days ago / in X days - def _ago_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor - timedelta(days=n)).date().isoformat() - def _in_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor + timedelta(days=n)).date().isoformat() - t = re.sub(r"\b(\d+)\s+days\s+ago\b", _ago_repl, t, flags=re.IGNORECASE) - t = re.sub(r"\bin\s+(\d+)\s+days\b", _in_repl, t, flags=re.IGNORECASE) - t = re.sub(r"\blast\s+week\b", (anchor - timedelta(days=7)).date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\bnext\s+week\b", (anchor + timedelta(days=7)).date().isoformat(), t, flags=re.IGNORECASE) - - # 中文 今天/昨天/明天 - t = re.sub(r"今天", anchor.date().isoformat(), t) - t = re.sub(r"昨日|昨天", (anchor - timedelta(days=1)).date().isoformat(), t) - t = re.sub(r"明天", (anchor + timedelta(days=1)).date().isoformat(), t) - # 中文 X天前 / X天后 - t = re.sub(r"(\d+)天前", lambda m: (anchor - timedelta(days=int(m.group(1)))).date().isoformat(), t) - t = re.sub(r"(\d+)天后", lambda m: (anchor + timedelta(days=int(m.group(1)))).date().isoformat(), t) - # 中文 上周 / 下周（近似7天） - t = re.sub(r"上周", (anchor - timedelta(days=7)).date().isoformat(), t) - t = re.sub(r"下周", (anchor + timedelta(days=7)).date().isoformat(), t) - # 中文 月日（无年份）补全年份 - def _md_repl(m: re.Match[str]) -> str: - mon = int(m.group(1)); day = int(m.group(2)) - return f"{anchor.year}-{mon:02d}-{day:02d}" - t = re.sub(r"(\d{1,2})月(\d{1,2})日", _md_repl, t) - return t - - -async def run_longmemeval_test( - sample_size: int = 3, - group_id: str = "longmemeval_zh_bak_3", - search_limit: int = 8, - context_char_budget: int = 4000, - llm_temperature: float = 0.0, - llm_max_tokens: int = 16, - search_type: str = "hybrid", - data_path: str | None = None, - start_index: int = 0, - max_contexts_per_item: int = 2, - save_chunk_output: bool = True, - save_chunk_output_path: str | None = None, - reset_group_before_ingest: bool = False, - skip_ingest: bool = False, -) -> Dict[str, Any]: - """LongMemEval 评估测试：增强时间推理能力""" - - # 数据路径 - if not data_path: - # 固定使用中文数据集：data/longmemeval_oracle_zh.json - zh_proj = os.path.join(PROJECT_ROOT, "data", "longmemeval_oracle_zh.json") - zh_cwd = os.path.join(os.getcwd(), "data", "longmemeval_oracle_zh.json") - if os.path.exists(zh_proj): - data_path = zh_proj - elif os.path.exists(zh_cwd): - data_path = zh_cwd - else: - raise FileNotFoundError("未找到数据集: data/longmemeval_oracle_zh.json，请确保其存在于项目根目录或当前工作目录的 data 目录下。") - - qa_list: List[Dict[str, Any]] = load_dataset_any(data_path) - # 支持评估全部样本：当 sample_size <= 0 时，取从 start_index 到末尾 - if sample_size is None or sample_size <= 0: - items = qa_list[start_index:] - else: - items = qa_list[start_index:start_index + sample_size] - - # 可选：摄入上下文（默认启用） - if not skip_ingest: - # 选择上下文并限量 - contexts: List[str] = [] - for it in items: - built = build_context_from_sessions(it) - full_transcripts = [c for c in built if "\n" in c] - evidence_msgs = [c for c in built if "\n" not in c] - selected: List[str] = [] - take_e = min(len(evidence_msgs), max_contexts_per_item) - selected.extend(evidence_msgs[:take_e]) - remain = max_contexts_per_item - len(selected) - if remain > 0 and full_transcripts: - selected.extend(full_transcripts[:remain]) - if not selected and built: - selected.append(built[0]) - contexts.extend(selected) - - print(f"📥 摄入 {len(contexts)} 个上下文到数据库") - if reset_group_before_ingest and group_id: - try: - _tmp_conn = Neo4jConnector() - await _tmp_conn.delete_group(group_id) - print(f"🧹 已清空组 {group_id} 的历史图数据") - except Exception as _e: - print(f"⚠️ 清空组数据失败（忽略继续）: {group_id} - {_e}") - finally: - try: - await _tmp_conn.close() - except Exception: - pass - _ingest_fn = ingest_contexts_via_full_pipeline - if _ingest_fn is None: - print("⚠️ 摄入函数不可用，已跳过摄入。请确认 PYTHONPATH 包含 'src' 或从项目根运行。") - else: - await _ingest_fn( - contexts, - group_id, - save_chunk_output=save_chunk_output, - save_chunk_output_path=save_chunk_output_path, - ) - - # 初始化组件（摄入后再初始化连接器）- 使用异步LLM客户端 - with get_db_context() as db: - factory = MemoryClientFactory(db) - llm_client = factory.get_llm_client(SELECTED_LLM_ID) - connector = Neo4jConnector() - with get_db_context() as db: - config_service = MemoryConfigService(db) - cfg_dict = config_service.get_embedder_config(SELECTED_EMBEDDING_ID) - embedder = OpenAIEmbedderClient( - model_config=RedBearModelConfig.model_validate(cfg_dict) - ) - - # 指标收集 - latencies_llm: List[float] = [] - latencies_search: List[float] = [] - per_query_context_counts: List[int] = [] - per_query_context_avg_tokens: List[float] = [] - per_query_context_chars: List[int] = [] - - type_correct: Dict[str, List[float]] = {} - type_f1: Dict[str, List[float]] = {} - type_jacc: Dict[str, List[float]] = {} - - samples: List[Dict[str, Any]] = [] - # 统计重复的上下文预览（跨样本），便于诊断"相同上下文"问题 - preview_counter: Dict[str, int] = {} - - try: - for item in items: - question = item.get("question", "") - reference = item.get("answer", "") - qtype = item.get("question_type") or item.get("type", "unknown") - - print(f"\n=== 处理问题: {question} ===") - - # 检测问题类型 - is_temporal = any(keyword in question.lower() for keyword in - ['days', 'day', 'before', 'after', 'first', '先后', '顺序', '间隔', '多久', '多少天']) - - # 检索 - t0 = time.time() - contexts_all: List[str] = [] - dialogs, statements, entities = [], [], [] - - try: - if search_type == "embedding": - search_results = await search_graph_by_embedding( - connector=connector, - embedder_client=embedder, - query_text=question, - group_id=group_id, - limit=search_limit, - include=["chunks", "statements", "entities", "summaries"], - ) - chunks = search_results.get("chunks", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - - for d in dialogs: - content = str(d.get("content", "")).strip() - if content: - contexts_all.append(content) - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - - # for sm in summaries: - # summary_text = str(sm.get("summary", "")).strip() - # if summary_text: - # contexts_all.append(summary_text) - - # 实体摘要（最多3个） - scored = [e for e in entities if e.get("score") is not None] - top_entities = sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] if scored else entities[:3] - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append(f"EntitySummary: {name}{(' [' + '; '.join(meta) + ']') if meta else ''}") - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - - elif search_type == "keyword": - search_results = await search_graph( - connector=connector, - q=question, - group_id=group_id, - limit=search_limit, - ) - chunks = search_results.get("chunks", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - summaries = search_results.get("summaries", []) - - for c in chunks: - content = str(c.get("content", "")).strip() - if content: - contexts_all.append(content) - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - for sm in summaries: - summary_text = str(sm.get("summary", "")).strip() - if summary_text: - contexts_all.append(summary_text) - if entities: - entity_names = [str(e.get("name", "")).strip() for e in entities[:5] if e.get("name")] - if entity_names: - contexts_all.append(f"EntitySummary: {', '.join(entity_names)}") - - else: # hybrid（增强版：特别优化时间推理问题） - emb_chunks, emb_statements, emb_entities, emb_summaries, emb_dialogs = [], [], [], [], [] - kw_dialogs, kw_statements, kw_entities = [], [], [] - - # 1) 嵌入检索 - try: - emb_res = await search_graph_by_embedding( - connector=connector, - embedder_client=embedder, - query_text=question, - group_id=group_id, - limit=search_limit, - include=["chunks", "statements", "entities", "summaries"], - ) - if isinstance(emb_res, dict): - emb_chunks = emb_res.get("chunks", []) or [] - emb_statements = emb_res.get("statements", []) or [] - emb_entities = emb_res.get("entities", []) or [] - emb_summaries = emb_res.get("summaries", []) or [] - emb_dialogs = emb_res.get("dialogues", []) or [] - except Exception as e: - print(f"⚠️ 嵌入检索失败，将继续进行关键词检索: {e}") - - # 2) 关键词检索（增强版） - try: - kw_res = await search_graph( - connector=connector, - q=question, - group_id=group_id, - limit=search_limit, - ) - if isinstance(kw_res, dict): - kw_dialogs = kw_res.get("dialogues", []) or [] - kw_statements = kw_res.get("statements", []) or [] - kw_entities = kw_res.get("entities", []) or [] - - # 时间推理问题的特殊处理 - if is_temporal: - # 专门搜索时间实体 - time_entities = await _search_time_entities(connector, group_id, search_limit//2) - if time_entities: - kw_entities.extend(time_entities) - # 添加时间相关关键词检索 - time_keywords = ['天', '日', '月', '年', 'before', 'after', 'first'] - for tk in time_keywords: - try: - time_res = await search_graph( - connector=connector, - q=tk, - group_id=group_id, - limit=2, - ) - if isinstance(time_res, dict): - kw_dialogs.extend(time_res.get("dialogues", []) or []) - kw_statements.extend(time_res.get("statements", []) or []) - except Exception: - pass - - # 中文关键词拆分后做别名匹配 - cn_tokens = _extract_cn_tokens(question) - alias_entities = await _search_entities_by_aliases(connector, cn_tokens, group_id, search_limit) - if alias_entities: - kw_entities.extend(alias_entities) - - # 从对话/陈述中的 entity_ids 反查实体 - ids = [] - try: - for d in kw_dialogs: - ids.extend(d.get("entity_ids", []) or []) - for s in kw_statements: - ids.extend(s.get("entity_ids", []) or []) - except Exception: - pass - if ids: - id_entities = await _fetch_entities_by_ids(connector, ids, group_id) - if id_entities: - kw_entities.extend(id_entities) - - # 多关键词检索 - try: - eng_words = [w for w in set(re.findall(r"\b\w+\b", question.lower())) if len(w) > 2] - kw_list = generate_query_keywords_cn(question)[:3] + eng_words[:1] - for kw in kw_list: - if not kw: - continue - sub_res = await search_graph( - connector=connector, - q=str(kw), - group_id=group_id, - limit=max(3, search_limit // 2), - ) - if isinstance(sub_res, dict): - kw_dialogs.extend(sub_res.get("dialogues", []) or []) - kw_statements.extend(sub_res.get("statements", []) or []) - kw_entities.extend(sub_res.get("entities", []) or []) - except Exception: - pass - - # 选项参与关键词检索 - try: - opt_list = extract_candidate_options(question)[:2] - for opt in opt_list: - if not opt: - continue - opt_res = await search_graph( - connector=connector, - q=str(opt), - group_id=group_id, - limit=max(3, search_limit // 2), - ) - if isinstance(opt_res, dict): - kw_dialogs.extend(opt_res.get("dialogues", []) or []) - kw_statements.extend(opt_res.get("statements", []) or []) - kw_entities.extend(opt_res.get("entities", []) or []) - except Exception: - pass - except Exception as e: - print(f"❌ 关键词检索失败: {e}") - - # 3) 合并、排序并去重 - all_dialogs = emb_dialogs + kw_dialogs - all_statements = emb_statements + kw_statements - all_entities = emb_entities + kw_entities - - def dedup(items: List[Dict[str, Any]], key_field: str = "uuid") -> List[Dict[str, Any]]: - seen = set() - out = [] - for it in items: - key = str(it.get(key_field, "")) + str(it.get("content", "") + str(it.get("statement", ""))) - if key not in seen: - out.append(it) - seen.add(key) - return out - - # 时间推理问题优先排序包含时间信息的文档 - if is_temporal: - def temporal_score(item: Dict[str, Any]) -> float: - base_score = float(item.get("score", 0.0)) - content = str(item.get("content", "") + str(item.get("statement", ""))) - time_entities = extract_time_entities(content) - time_bonus = len(time_entities) * 0.5 - return base_score + time_bonus - - dialogs = dedup(sorted(all_dialogs, key=temporal_score, reverse=True)) - statements = dedup(sorted(all_statements, key=temporal_score, reverse=True)) - else: - dialogs = dedup(sorted(all_dialogs, key=lambda d: float(d.get("score", 0.0)), reverse=True)) - statements = dedup(sorted(all_statements, key=lambda s: float(s.get("score", 0.0)), reverse=True)) - - entities = dedup(all_entities, key_field="name") - - # 4) 构建上下文 - for d in dialogs: - content = str(d.get("content", "")).strip() - if content: - contexts_all.append(content) - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - # 实体摘要 - try: - scored = [e for e in entities if e.get("score") is not None] - top_entities = sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] if scored else entities[:3] - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append(f"EntitySummary: {name}{(' [' + '; '.join(meta) + ']') if meta else ''}") - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - except Exception: - pass - - # 全局回退 - if not contexts_all and search_type in ("embedding", "hybrid"): - try: - print("🔁 检索为空，回退到关键词检索...") - kw_fallback = await search_graph( - connector=connector, - q=question, - group_id=group_id, - limit=max(search_limit, 5), - ) - fb_dialogs = kw_fallback.get("dialogues", []) or [] - fb_statements = kw_fallback.get("statements", []) or [] - fb_entities = kw_fallback.get("entities", []) or [] - - for d in fb_dialogs: - content = str(d.get("content", "")).strip() - if content: - contexts_all.append(content) - for s in fb_statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - if fb_entities: - entity_names = [str(e.get("name", "")).strip() for e in fb_entities[:5] if e.get("name")] - if entity_names: - contexts_all.append(f"EntitySummary: {', '.join(entity_names)}") - - dialogs = fb_dialogs if fb_dialogs else dialogs - statements = fb_statements if fb_statements else statements - entities = fb_entities if fb_entities else entities - print(f"↩️ 回退到关键词检索: {len(fb_dialogs)} 对话, {len(fb_statements)} 条陈述, {len(fb_entities)} 个实体") - except Exception as fe: - print(f"❌ 关键词回退失败: {fe}") - - ent_count = len(entities) if isinstance(entities, list) else 0 - print(f"✅ {search_type}检索成功: {len(dialogs)} 对话, {len(statements)} 条陈述, {ent_count} 个实体") - if is_temporal: - print("⏰ 检测为时间推理问题，已启用时间优化检索") - - except Exception as e: - print(f"❌ {search_type}检索失败: {e}") - contexts_all = [] - - t1 = time.time() - latencies_search.append((t1 - t0) * 1000) - - # 智能上下文选择 - context_text = "" - if contexts_all: - context_text = smart_context_selection(contexts_all, question, max_chars=context_char_budget) - # 相对时间解析 - try: - context_text = _resolve_relative_times_cn_en(context_text, anchor=datetime.now()) - except Exception: - pass - # 诊断信息 - try: - cn_diag = generate_query_keywords_cn(question)[:3] - opts = extract_candidate_options(question)[:2] - qlw = [w for w in set(re.findall(r'\b\w+\b', question.lower())) if len(w) > 2][:1] - diag_tokens: List[str] = [] - for t in cn_diag + opts + qlw: - if t and t not in diag_tokens: - diag_tokens.append(t) - print(f"🔍 关键词/选项: {', '.join(diag_tokens)}") - preview = context_text[:200].replace('\n', ' ') - print(f"🔎 上下文预览: {preview}...") - key_preview = preview.strip() - if key_preview: - preview_counter[key_preview] = preview_counter.get(key_preview, 0) + 1 - except Exception: - pass - else: - print("❌ 没有检索到有效上下文") - context_text = "No relevant context found." - - # 记录上下文诊断信息 - per_query_context_counts.append(len(contexts_all)) - per_query_context_avg_tokens.append(avg_context_tokens([context_text])) - per_query_context_chars.append(len(context_text)) - - # LLM 推理（增强时间推理提示） - options = extract_candidate_options(question) - if len(options) >= 2: - opt_lines = "\n".join(f"- {o}" for o in options) - # 时间推理问题的特殊提示 - if is_temporal: - system_prompt = ( - "You are a QA assistant specializing in temporal reasoning. Analyze the dates and time relationships in the context carefully. " - "Return ONLY one string: exactly one option from the provided candidates. If the context is insufficient, respond with 'Unknown'. " - "Pay special attention to date sequences and time intervals." - ) - else: - system_prompt = ( - "You are a QA assistant. Respond in the same language as the question. Return ONLY one string: exactly one option from the provided candidates. " - "If the context is insufficient, respond with 'Unknown'. If the context expresses a synonym or paraphrase of a candidate, return the closest candidate. " - "Do not include explanations." - ) - - messages = [ - {"role": "system", "content": system_prompt}, - { - "role": "user", - "content": ( - f"Question: {question}\n\nCandidates:\n{opt_lines}\n\nContext:\n{context_text}\n\nReturn EXACTLY one candidate string (or 'Unknown')." - ), - }, - ] - else: - # 时间推理问题的特殊提示 - if is_temporal: - system_prompt = ( - "You are a QA assistant specializing in temporal reasoning. Analyze the dates and time relationships in the context carefully. " - "If the context contains the answer, return a concise answer phrase focusing on temporal information. " - "If the answer cannot be determined from the context, respond with 'Unknown'. Return ONLY the final answer string, no explanations." - ) - else: - system_prompt = ( - "You are a QA assistant. Respond in the same language as the question. If the context contains the answer, return a concise answer phrase. " - "If the answer cannot be determined from the context, respond with 'Unknown'. Return ONLY the final answer string, no explanations." - ) - - messages = [ - {"role": "system", "content": system_prompt}, - { - "role": "user", - "content": f"Question: {question}\n\nContext:\n{context_text}\n\nReturn ONLY the answer (or 'Unknown').", - }, - ] - - t2 = time.time() - # 使用异步调用 - resp = await llm_client.chat(messages=messages) - t3 = time.time() - latencies_llm.append((t3 - t2) * 1000) - - # 兼容不同的响应格式 - pred_raw = resp.content.strip() if hasattr(resp, 'content') else (resp["choices"][0]["message"]["content"].strip() if isinstance(resp, dict) else "Unknown") - - # 选项题输出规范化 - pred = pred_raw - if len(options) >= 2 and not pred_raw.lower().startswith("unknown"): - def _basic_norm(s: str) -> str: - s = s.lower().strip() - return re.sub(r"[^\w\s]", " ", s) - def _jaccard(a: str, b: str) -> float: - ta = set(t for t in _basic_norm(a).split() if t) - tb = set(t for t in _basic_norm(b).split() if t) - if not ta and not tb: - return 1.0 - if not ta or not tb: - return 0.0 - return len(ta & tb) / len(ta | tb) - best = None - best_score = -1.0 - for o in options: - score = _jaccard(pred_raw, o) - if score > best_score: - best = o - best_score = score - if best is not None and best_score > 0.0: - pred = best - - # 指标 - flag = exact_match(pred, reference) - f1_val = common_f1(str(pred), str(reference)) - j_val = jaccard(str(pred), str(reference)) - - type_correct.setdefault(qtype, []).append(flag) - type_f1.setdefault(qtype, []).append(f1_val) - type_jacc.setdefault(qtype, []).append(j_val) - - samples.append({ - "question": question, - "prediction": pred, - "answer": reference, - "question_type": qtype, - "is_temporal": is_temporal, - "question_id": item.get("question_id"), - "options": options, - "context_count": len(contexts_all), - "context_chars": len(context_text), - "retrieved_dialogue_count": len(dialogs), - "retrieved_statement_count": len(statements), - "metrics": { - "exact_match": bool(flag), - "f1": f1_val, - "jaccard": j_val - }, - "timing": { - "search_ms": (t1 - t0) * 1000, - "llm_ms": (t3 - t2) * 1000 - } - }) - - print(f"🤖 LLM 回答: {pred}") - print(f"✅ 正确答案: {reference}") - print(f"📈 当前指标 - Exact Match: {flag}, F1: {f1_val:.3f}, Jaccard: {j_val:.3f}") - - # 聚合结果 - type_acc = {t: (sum(v) / max(len(v), 1)) for t, v in type_correct.items()} - f1_by_type = {t: (sum(v) / max(len(v), 1)) for t, v in type_f1.items()} - jacc_by_type = {t: (sum(v) / max(len(v), 1)) for t, v in type_jacc.items()} - - result = { - "dataset": "longmemeval", - "items": len(items), - "accuracy_by_type": type_acc, - "f1_by_type": f1_by_type, - "jaccard_by_type": jacc_by_type, - "samples": samples, - "latency": { - "search": latency_stats(latencies_search), - "llm": latency_stats(latencies_llm), - }, - "context": { - "avg_tokens": statistics.mean(per_query_context_avg_tokens) if per_query_context_avg_tokens else 0.0, - "avg_chars": statistics.mean(per_query_context_chars) if per_query_context_chars else 0.0, - "count_avg": statistics.mean(per_query_context_counts) if per_query_context_counts else 0.0, - }, - "params": { - "group_id": group_id, - "search_limit": search_limit, - "context_char_budget": context_char_budget, - "search_type": search_type, - "llm_id": SELECTED_LLM_ID, - "embedding_id": SELECTED_EMBEDDING_ID, - "sample_size": sample_size, - "start_index": start_index, - }, - "timestamp": datetime.now().isoformat() - } - - # 计算汇总指标 - try: - total_items = max(len(samples), 1) - correct_count = sum(1 for s in samples if s.get("metrics", {}).get("exact_match")) - score_accuracy = (correct_count / total_items) * 100.0 - - total_latencies_ms = [] - for s in samples: - t = s.get("timing", {}) - total_latencies_ms.append(float(t.get("search_ms", 0.0)) + float(t.get("llm_ms", 0.0))) - total_lat_stats = latency_stats(total_latencies_ms) if total_latencies_ms else {"p50": 0.0, "iqr": 0.0} - latency_median_s = total_lat_stats.get("p50", 0.0) / 1000.0 - latency_iqr_s = total_lat_stats.get("iqr", 0.0) / 1000.0 - - avg_ctx_tokens = statistics.mean(per_query_context_avg_tokens) if per_query_context_avg_tokens else 0.0 - avg_ctx_tokens_k = avg_ctx_tokens / 1000.0 - - result["metric_summary"] = { - "score_accuracy": score_accuracy, - "latency_median_s": latency_median_s, - "latency_iqr_s": latency_iqr_s, - "avg_context_tokens_k": avg_ctx_tokens_k, - } - except Exception: - result["metric_summary"] = { - "score_accuracy": 0.0, - "latency_median_s": 0.0, - "latency_iqr_s": 0.0, - "avg_context_tokens_k": 0.0, - } - - # 诊断信息 - try: - dups = sorted([(k, c) for k, c in preview_counter.items() if c > 1], key=lambda x: -x[1])[:5] - result["diagnostics"] = { - "duplicate_previews_top": [{"count": c, "preview": k[:120]} for k, c in dups], - "unique_preview_count": len(preview_counter), - } - except Exception: - pass - - return result - - finally: - await connector.close() - -def main(): - load_dotenv() - parser = argparse.ArgumentParser(description="LongMemEval 评估测试脚本（增强时间推理版）") - parser.add_argument("--sample-size", type=int, default=3, help="样本数量（<=0 表示全部）") - parser.add_argument("--all", action="store_true", help="评估全部样本（覆盖 --sample-size）") - parser.add_argument("--start-index", type=int, default=0, help="起始样本索引") - parser.add_argument("--group-id", type=str, default="longmemeval_zh_bak_3", help="图数据库 Group ID") - parser.add_argument("--search-limit", type=int, default=8, help="检索条数上限") - parser.add_argument("--context-char-budget", type=int, default=4000, help="上下文字符预算") - parser.add_argument("--llm-temperature", type=float, default=0.0, help="LLM 温度") - parser.add_argument("--llm-max-tokens", type=int, default=16, help="LLM 最大输出 token") - parser.add_argument("--search-type", type=str, default="hybrid", choices=["embedding","keyword","hybrid"], help="检索类型") - parser.add_argument("--data-path", type=str, default=None, help="数据集路径") - parser.add_argument("--max-contexts-per-item", type=int, default=2, help="每条样本最多摄入的上下文段数") - parser.add_argument("--no-save-chunk-output", action="store_true", help="不保存分块结果（默认保存）") - parser.add_argument("--save-chunk-output-path", type=str, default=None, help="自定义分块输出路径") - parser.add_argument("--reset-group-before-ingest", action="store_true", help="摄入前清空该 Group 在图数据库中的历史数据") - parser.add_argument("--skip-ingest", action="store_true", help="跳过摄入，仅检索评估") - args = parser.parse_args() - - sample_size = 0 if args.all else args.sample_size - - result = asyncio.run( - run_longmemeval_test( - sample_size=sample_size, - group_id=args.group_id, - search_limit=args.search_limit, - context_char_budget=args.context_char_budget, - llm_temperature=args.llm_temperature, - llm_max_tokens=args.llm_max_tokens, - search_type=args.search_type, - data_path=args.data_path, - start_index=args.start_index, - max_contexts_per_item=args.max_contexts_per_item, - save_chunk_output=(not args.no_save_chunk_output), - save_chunk_output_path=args.save_chunk_output_path, - reset_group_before_ingest=args.reset_group_before_ingest, - skip_ingest=args.skip_ingest, - ) - ) - - # 打印结果 - print("\n" + "="*50) - print("📊 LongMemEval 测试结果:") - print(f" 样本数量: {result['items']}") - - if result['accuracy_by_type']: - print("\n📈 按问题类型细分:") - for qtype, acc in result['accuracy_by_type'].items(): - print(f" {qtype}:") - print(f" Score (Accuracy): {acc:.3f}") - - print(f"\n📊 指标总览:") - ms = result.get('metric_summary', {}) - print(f" Score (Accuracy): {ms.get('score_accuracy', 0.0):.1f}%") - print(f" Latency (s): median {ms.get('latency_median_s', 0.0):.3f}s") - print(f" Latency IQR (s): {ms.get('latency_iqr_s', 0.0):.3f}s") - print(f" Avg Context Tokens (k): {ms.get('avg_context_tokens_k', 0.0):.3f}k") - - print(f"\n⏱️ 细分性能指标:") - print(f" 检索延迟(均值): {result['latency']['search']['mean']:.1f}ms") - print(f" LLM延迟(均值): {result['latency']['llm']['mean']:.1f}ms") - print(f" 上下文长度(均值): {result['context']['avg_chars']:.0f} 字符") - - - # 保存结果到文件 - try: - out_dir = os.path.join(PROJECT_ROOT, "evaluation", "longmemeval", "results") - os.makedirs(out_dir, exist_ok=True) - ts = datetime.now().strftime("%Y%m%d_%H%M%S") - out_path = os.path.join(out_dir, f"longmemeval_{result['params']['search_type']}_{ts}.json") - with open(out_path, "w", encoding="utf-8") as f: - json.dump(result, f, ensure_ascii=False, indent=2) - print(f"\n💾 结果已保存: {out_path}") - except Exception as e: - print(f"⚠️ 结果保存失败: {e}") - - -if __name__ == "__main__": - main() diff --git a/api/app/core/memory/evaluation/longmemeval/test_eval.py b/api/app/core/memory/evaluation/longmemeval/test_eval.py deleted file mode 100644 index 08a763e3..00000000 --- a/api/app/core/memory/evaluation/longmemeval/test_eval.py +++ /dev/null @@ -1,1330 +0,0 @@ -import argparse -import asyncio -import json -import os -import re -import statistics -import time -from datetime import datetime, timedelta -from typing import Any, Dict, List - -try: - from dotenv import load_dotenv -except Exception: - def load_dotenv(): - return None - -# 与现有评估脚本保持一致的导入方式 -from app.core.memory.evaluation.common.metrics import ( - avg_context_tokens, - jaccard, - latency_stats, -) -from app.core.memory.evaluation.common.metrics import f1_score as common_f1 -from app.core.memory.evaluation.dialogue_queries import SEARCH_ENTITIES_BY_NAME -from app.core.memory.llm_tools.openai_embedder import OpenAIEmbedderClient -from app.core.memory.utils.config.definitions import ( - PROJECT_ROOT, - SELECTED_EMBEDDING_ID, - SELECTED_LLM_ID, -) -from app.core.memory.utils.llm.llm_utils import MemoryClientFactory -from app.core.models.base import RedBearModelConfig -from app.db import get_db_context -from app.repositories.neo4j.graph_search import search_graph, search_graph_by_embedding -from app.repositories.neo4j.neo4j_connector import Neo4jConnector -from app.services.memory_config_service import MemoryConfigService - -try: - from app.core.memory.evaluation.common.metrics import exact_match -except Exception: - # 兜底：简单的大小写不敏感比较 - def exact_match(pred: str, ref: str) -> bool: - return str(pred).strip().lower() == str(ref).strip().lower() - - -def load_dataset_any(path: str) -> List[Dict[str, Any]]: - """健壮地加载数据集（兼容 list 或多段 JSON）。""" - with open(path, "r", encoding="utf-8") as f: - s = f.read().strip() - try: - obj = json.loads(s) - if isinstance(obj, list): - return obj - elif isinstance(obj, dict): - return [obj] - except json.JSONDecodeError: - pass - dec = json.JSONDecoder() - idx = 0 - items: List[Dict[str, Any]] = [] - while idx < len(s): - while idx < len(s) and s[idx].isspace(): - idx += 1 - if idx >= len(s): - break - try: - obj, end = dec.raw_decode(s, idx) - if isinstance(obj, list): - for it in obj: - if isinstance(it, dict): - items.append(it) - elif isinstance(obj, dict): - items.append(obj) - idx = end - except json.JSONDecodeError: - nl = s.find("\n", idx) - if nl == -1: - break - idx = nl + 1 - return items - - -def is_chinese_text(s: str) -> bool: - return bool(re.search(r"[\u4e00-\u9fff]", s or "")) - - -def extract_candidate_options(question: str) -> List[str]: - """从问题中提取候选选项（A-or-B 类问题）。""" - q = (question or "").strip() - options: List[str] = [] - - # 1) 引号包裹的片段 - for pat in [r"'([^']+)'", r'\"([^\"]+)\"', r'“([^”]+)”', r'‘([^’]+)’']: - for m in re.findall(pat, q): - val = (m or "").strip() - if val: - options.append(val) - - # 2) or/还是/或者 连接词 - if len(options) < 2: - pats = [ - r"([^,;，；]+?)\s+or\s+([^,;，；\?\.!.。！]+)", - r"([^,;，；]+?)\s+还是\s+([^,;，；\?\.!.。！]+)", - r"([^,;，；]+?)\s+或者\s+([^,;，；\?\.!.。！]+)", - ] - for pat in pats: - matches = list(re.finditer(pat, q, flags=re.IGNORECASE)) - if matches: - m = matches[-1] - cand1 = m.group(1).strip().strip("?？.,，;； ") - cand2 = m.group(2).strip().strip("?？.,，;； ") - options.extend([cand1, cand2]) - break - - # 去重 - seen = set() - uniq: List[str] = [] - for o in options: - o2 = o.strip() - key = o2.lower() if not is_chinese_text(o2) else o2 - if o2 and key not in seen: - uniq.append(o2) - seen.add(key) - return uniq - - -def extract_time_entities(text: str) -> List[Dict[str, Any]]: - """增强时间实体提取，专门用于时间推理问题""" - time_entities = [] - - # 日期模式 - date_patterns = [ - (r'\b(\d{4})-(\d{1,2})-(\d{1,2})\b', 'date'), # YYYY-MM-DD - (r'\b(\d{1,2})月(\d{1,2})日\b', 'date'), # 中文日期 - (r'\b(January|February|March|April|May|June|July|August|September|October|November|December)\s+(\d{1,2}),?\s+(\d{4})?', 'date'), # 英文月份 - (r'\b(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\s+(\d{1,2}),?\s+(\d{4})?', 'date'), # 英文月份缩写 - ] - - # 时间间隔模式 - duration_patterns = [ - (r'(\d+)\s*天', 'days'), - (r'(\d+)\s*周', 'weeks'), - (r'(\d+)\s*个月', 'months'), - (r'(\d+)\s*年', 'years'), - (r'(\d+)\s*days?', 'days'), - (r'(\d+)\s*weeks?', 'weeks'), - (r'(\d+)\s*months?', 'months'), - (r'(\d+)\s*years?', 'years'), - ] - - # 事件时间关系模式 - temporal_relation_patterns = [ - (r'(之前|以前|前)\s*(\d+)\s*天', 'days_before'), - (r'(之后|以后|后)\s*(\d+)\s*天', 'days_after'), - (r'(\d+)\s*天\s*(之前|以前|前)', 'days_before'), - (r'(\d+)\s*天\s*(之后|以后|后)', 'days_after'), - (r'(\d+)\s*days?\s*(before|ago)', 'days_before'), - (r'(\d+)\s*days?\s*(after|later)', 'days_after'), - ] - - # 提取日期 - for pattern, entity_type in date_patterns: - matches = re.finditer(pattern, text, re.IGNORECASE) - for match in matches: - time_entities.append({ - 'text': match.group(), - 'type': entity_type, - 'start': match.start(), - 'end': match.end() - }) - - # 提取时间间隔 - for pattern, entity_type in duration_patterns: - matches = re.finditer(pattern, text, re.IGNORECASE) - for match in matches: - time_entities.append({ - 'text': match.group(), - 'type': entity_type, - 'value': int(match.group(1)), - 'start': match.start(), - 'end': match.end() - }) - - # 提取时间关系 - for pattern, entity_type in temporal_relation_patterns: - matches = re.finditer(pattern, text, re.IGNORECASE) - for match in matches: - time_entities.append({ - 'text': match.group(), - 'type': entity_type, - 'value': int(match.group(2)) if match.groups() >= 2 else int(match.group(1)), - 'start': match.start(), - 'end': match.end() - }) - - return time_entities - - -def calculate_time_difference(date1: str, date2: str) -> int: - """计算两个日期之间的天数差""" - try: - # 解析日期格式 - def parse_date(date_str: str) -> datetime: - # 尝试多种日期格式 - formats = [ - '%Y-%m-%d', - '%m月%d日', - '%B %d, %Y', - '%b %d, %Y', - '%Y年%m月%d日' - ] - - for fmt in formats: - try: - return datetime.strptime(date_str, fmt) - except ValueError: - continue - - # 如果都无法解析，返回当前日期 - return datetime.now() - - d1 = parse_date(date1) - d2 = parse_date(date2) - - # 计算天数差（绝对值） - return abs((d2 - d1).days) - except Exception: - return -1 # 表示计算失败 - - -def _extract_cn_tokens(text: str) -> List[str]: - """中文关键词提取（短语级，含数词/日期/常见领域词）""" - if not text: - return [] - t = str(text) - # 去掉常见功能词（粗略，不依赖分词库） - stop_words = [ - "我","我们","你","他","她","它","这","那","哪","一个","一次","一些","什么","怎么","是否","吗","呢", - "很","更","最","已经","正在","将要","马上","尽快","最近","关于","有关","以及","并且","或者","还是", - "因为","所以","如果","但是","而且","然后","之后","之前","同时","另外","并","但","却","被","把","让","给", - "和","与","跟","及","还有","就","都","在","对","对于","的","了","着","过","到","于","从","以","为","向","至","是" - ] - for sw in stop_words: - t = t.replace(sw, " ") - # 去标点 - t = re.sub(r"[，。！？、；：,.!?;:\"'（）()［］\[\]\-—…·]", " ", t) - # 基础中文片段（>=2） - base = re.findall(r"[\u4e00-\u9fff]{2,}", t) - # 特殊组合：第X次XXXX - specials = re.findall(r"第[一二三四五六七八九十]+次[\u4e00-\u9fff]{2,6}", text) - # 日期与数字 - dates = re.findall(r"\d{4}年\d{1,2}月\d{1,2}日|\d{1,2}月\d{1,2}日|\d{4}-\d{1,2}-\d{1,2}", text) - numbers = re.findall(r"\b\d+\b", text) - - generic = {"建议","推荐","帮助","提升","技能","有效","团队","参与度","喜欢","开始"} - tokens: List[str] = specials + base + dates + numbers - uniq: List[str] = [] - seen = set() - for tok in tokens: - tok2 = tok.strip() - if len(tok2) < 2 or len(tok2) > 6: - continue - if tok2 in generic: - continue - if tok2 not in seen: - uniq.append(tok2) - seen.add(tok2) - # 排除常见疑问型短语 - blacklist_exact = {"是什么","多少","多少天","哪个","哪些","之间","先","后","之前","之后"} - uniq2: List[str] = [u for u in uniq if u not in blacklist_exact] - return uniq2[:12] - - -def generate_query_keywords_cn(question: str) -> List[str]: - """增强版关键词提取，特别关注技术术语和专有名词""" - if not question: - return [] - - # 提取专有名词（带引号的内容） - quoted_terms = re.findall(r'["""]([^"""]+)["""]', question) - - # 提取技术术语（中英文混合） - tech_terms = re.findall(r'[A-Z][a-zA-Z]+\s+[A-Z][a-zA-Z]+|[A-Za-z]+[\u4e00-\u9fff]+|[\u4e00-\u9fff]+[A-Za-z]+', question) - - # 提取核心名词短语 - core_nouns = re.findall(r'[\u4e00-\u9fff]{2,5}系统|[\u4e00-\u9fff]{2,5}管理|[\u4e00-\u9fff]{2,5}分析|[\u4e00-\u9fff]{2,5}工作坊|[\u4e00-\u9fff]{2,5}研讨会', question) - - # 基础中文片段 - base_tokens = _extract_cn_tokens(question) - - # 特定领域关键词增强 - domain_keywords = [] - # GPS相关 - if any(term in question for term in ["GPS", "导航", "定位系统", "系统运行"]): - domain_keywords.extend(["GPS", "导航系统", "定位", "系统故障", "功能异常"]) - # 活动相关 - if any(term in question for term in ["工作坊", "研讨会", "网络研讨会", "活动"]): - domain_keywords.extend(["工作坊", "研讨会", "参加", "参与", "活动"]) - # 时间顺序相关 - if any(term in question for term in ["先", "后", "第一个", "之前", "首先"]): - domain_keywords.extend(["先", "后", "之前", "之后", "第一次", "首先"]) - # 设备相关 - if any(term in question for term in ["设备", "手机", "电脑", "笔记本电脑"]): - domain_keywords.extend(["设备", "手机", "电脑", "笔记本电脑", "购买"]) - - # 合并并去重 - all_tokens = quoted_terms + tech_terms + core_nouns + base_tokens + domain_keywords - seen = set() - final_tokens = [] - - for token in all_tokens: - token = token.strip() - if len(token) >= 2 and token not in seen: - final_tokens.append(token) - seen.add(token) - - return final_tokens[:8] - - -def smart_context_selection(contexts: List[str], question: str, max_chars: int = 4000) -> str: - """增强版上下文选择：特别优化技术术语和精确匹配""" - if not contexts: - return "" - - # 检测是否为时间推理问题 - is_temporal_question = any(keyword in question.lower() for keyword in - ['days', 'day', 'before', 'after', 'first', '先后', '顺序', '间隔', '多久', '多少天']) - - # 提取时间实体从问题中 - question_time_entities = extract_time_entities(question) - - # 提取关键技术实体 - key_entities = [] - # GPS相关 - if any(term in question for term in ["GPS", "导航", "定位系统", "系统运行"]): - key_entities.extend(["GPS", "导航", "定位", "系统", "功能", "问题", "故障"]) - # 活动相关 - if any(term in question for term in ["工作坊", "研讨会", "网络研讨会", "活动"]): - key_entities.extend(["工作坊", "研讨会", "参加", "参与", "活动", "时间"]) - # 时间顺序相关 - if any(term in question for term in ["先", "后", "第一个", "之前", "首先"]): - key_entities.extend(["先", "后", "之前", "之后", "第一次", "首先"]) - - # 英文关键词（去停用词） - question_lower = question.lower() - stop_words = { - 'what','when','where','who','why','how','did','do','does','is','are','was','were', - 'the','a','an','and','or','but','many','which','first' - } - eng_words = [w for w in set(re.findall(r'\b\w+\b', question_lower)) - if w not in stop_words and len(w) > 2] - - # 中文片段与候选选项 - cn_tokens = generate_query_keywords_cn(question) - options = extract_candidate_options(question) - - # 时间推理问题的特殊处理 - if is_temporal_question: - # 为时间问题添加时间相关关键词 - time_keywords = ['天', '日', '月', '年', 'before', 'after', 'days', 'first', '先后'] - eng_words = [w for w in eng_words if w not in ['days', 'first']] # 避免重复 - cn_tokens.extend([kw for kw in time_keywords if kw not in cn_tokens]) - - # 限制关键词数量，优先时间相关 - tokens = time_keywords[:2] + key_entities[:3] + cn_tokens[:2] + eng_words[:1] + options[:1] - else: - # 常规问题处理，优先关键技术实体 - tokens = key_entities[:4] + cn_tokens[:3] + options[:2] + eng_words[:1] - - # 去重 - seen = set() - final_tokens: List[str] = [] - for t in tokens: - t2 = t.strip() - if t2 and t2 not in seen: - final_tokens.append(t2) - seen.add(t2) - - scored_contexts: List[tuple[float, str]] = [] - - # 关键技术实体权重映射 - key_entity_weights = { - "GPS": 3.0, "导航": 2.5, "系统": 2.0, "功能": 2.0, "问题": 2.0, "故障": 2.5, - "工作坊": 2.5, "研讨会": 2.5, "参加": 2.0, "参与": 2.0, - "先": 2.0, "后": 2.0, "之前": 2.0, "之后": 2.0, "第一次": 2.5 - } - - # 时间推理问题的权重映射 - temporal_weight_map = { - "天": 2.0, "日": 2.0, "月": 1.8, "年": 1.8, "days": 2.0, - "before": 1.5, "after": 1.5, "first": 1.5, "先后": 1.5 - } - - # 常规问题的权重映射 - normal_weight_map = { - "问题": 2.0, "故障": 2.0, "异常": 1.8, "不正常": 1.8, "坏了": 1.8, - "系统": 1.3, "GPS": 1.5, "保养": 1.4, "设备": 1.2, "模块": 1.2, "功能": 1.1 - } - - # 合并权重映射 - weight_map = {**normal_weight_map, **temporal_weight_map, **key_entity_weights} - - for i, context in enumerate(contexts): - context_str = str(context) - lines = re.split(r'[\r\n]+', context_str) - hit_lines: List[str] = [] - kw_hits: float = 0.0 - time_entity_count = 0 - key_entity_hits = 0 - - for line in lines: - ln = line.strip() - if not ln: - continue - - has_keyword = False - # 关键词匹配 - for tok in final_tokens: - if tok and tok in ln: - w = weight_map.get(tok, 1.0) - hit_count = ln.count(tok) - kw_hits += hit_count * w - # 关键技术实体额外奖励 - if tok in key_entity_weights: - key_entity_hits += hit_count - has_keyword = True - - # 时间实体检测（特别针对时间推理问题） - if is_temporal_question: - time_entities = extract_time_entities(ln) - time_entity_count += len(time_entities) - if time_entities: - has_keyword = True - - # 精确匹配奖励（完整问题关键词出现在上下文中） - for q_word in question.split(): - if len(q_word) > 3 and q_word in ln: - kw_hits += 0.5 # 精确匹配奖励 - - if has_keyword: - # 对于包含关键信息的行，保留完整行 - hit_lines.append(ln) - - snippet = "\n".join(hit_lines) if hit_lines else context_str.strip() - - # 限制单段长度，但对包含关键信息的上下文稍微放宽限制 - max_snippet_len = 600 if (key_entity_hits > 0 or time_entity_count > 0) else 500 - if len(snippet) > max_snippet_len: - snippet = snippet[:max_snippet_len] - - # 评分逻辑 - has_number = 1 if re.search(r'\d', snippet) else 0 - has_date = 1 if (re.search(r'\b\d{4}-\d{1,2}-\d{1,2}\b', snippet) or - re.search(r'\d{1,2}月\d{1,2}日', snippet)) else 0 - - # 关键技术实体奖励 - key_entity_bonus = key_entity_hits * 1.0 - - # 时间推理问题的特殊评分 - if is_temporal_question: - time_bonus = time_entity_count * 2.0 # 时间实体奖励 - temporal_coherence = 3 if (has_date and time_entity_count >= 2) else 0 - else: - time_bonus = 0 - temporal_coherence = 0 - - length_bonus = 5 if 50 < len(snippet) < 1000 else (2 if len(snippet) >= 1000 else 0) - pos_bonus = 3 if i < 3 else 0 - - score = (kw_hits * 0.8 + (has_number + has_date) * 1.5 + - length_bonus + pos_bonus + time_bonus + temporal_coherence + key_entity_bonus) - - scored_contexts.append((score, snippet)) - - # 选择累计至总字符预算 - scored_contexts.sort(key=lambda x: x[0], reverse=True) - selected: List[str] = [] - total_chars = 0 - - for score, snippet in scored_contexts: - if total_chars + len(snippet) <= max_chars: - selected.append(snippet) - total_chars += len(snippet) - else: - if not selected and len(snippet) > max_chars: - selected.append(snippet[:max_chars]) - break - - final_context = "\n\n".join(selected) - - # 对于时间推理问题，添加时间计算提示 - if is_temporal_question and question_time_entities: - time_prompt = "\n\n[时间推理提示：请仔细分析上述上下文中的日期和时间关系，计算时间间隔或确定事件顺序]" - if total_chars + len(time_prompt) <= max_chars: - final_context += time_prompt - - return final_context - - -# 通过别名匹配进行实体关键词检索（多token合并） -async def _search_entities_by_aliases(connector: Neo4jConnector, tokens: List[str], group_id: str | None, limit: int) -> List[Dict[str, Any]]: - results: List[Dict[str, Any]] = [] - try: - for tok in tokens: - rows = await connector.execute_query(SEARCH_ENTITIES_BY_NAME, q=tok, group_id=group_id, limit=limit) - if rows: - results.extend(rows) - except Exception: - pass - - # 按 name 去重 - deduped: List[Dict[str, Any]] = [] - seen = set() - for r in results: - k = str(r.get("name", "")) - if k and k not in seen: - deduped.append(r) - seen.add(k) - return deduped - - -# 通过对话/陈述中的entity_ids反查实体名称 -_FETCH_ENTITIES_BY_IDS = """ -MATCH (e:ExtractedEntity) -WHERE e.id IN $ids AND ($group_id IS NULL OR e.group_id = $group_id) -RETURN e.id AS id, e.name AS name, e.group_id AS group_id, e.entity_type AS entity_type -""" - -async def _fetch_entities_by_ids(connector: Neo4jConnector, ids: List[str], group_id: str | None) -> List[Dict[str, Any]]: - if not ids: - return [] - try: - rows = await connector.execute_query(_FETCH_ENTITIES_BY_IDS, ids=list({i for i in ids if i}), group_id=group_id) - return rows or [] - except Exception: - return [] - - -# 增强的时间实体检索 -_TIME_ENTITY_SEARCH = """ -MATCH (e:ExtractedEntity) -WHERE e.entity_type CONTAINS "TIME" OR e.entity_type CONTAINS "DATE" OR e.name =~ $date_pattern -AND ($group_id IS NULL OR e.group_id = $group_id) -RETURN e.id AS id, e.name AS name, e.group_id AS group_id, e.entity_type AS entity_type -LIMIT $limit -""" - -async def _search_time_entities(connector: Neo4jConnector, group_id: str | None, limit: int = 5) -> List[Dict[str, Any]]: - """专门搜索时间相关的实体""" - try: - date_pattern = r".*\d{4}.*|.*\d{1,2}月\d{1,2}日.*" - rows = await connector.execute_query(_TIME_ENTITY_SEARCH, - date_pattern=date_pattern, - group_id=group_id, - limit=limit) - return rows or [] - except Exception: - return [] - - -# 技术术语专门检索 -async def _search_tech_terms(connector: Neo4jConnector, question: str, group_id: str | None, limit: int = 3) -> List[Dict[str, Any]]: - """专门搜索技术术语相关的实体""" - tech_entities = [] - try: - # GPS相关 - if any(term in question for term in ["GPS", "导航", "定位系统"]): - gps_rows = await connector.execute_query(SEARCH_ENTITIES_BY_NAME, q="GPS", group_id=group_id, limit=limit) - if gps_rows: - tech_entities.extend(gps_rows) - - # 活动相关 - if any(term in question for term in ["工作坊", "研讨会", "网络研讨会"]): - workshop_rows = await connector.execute_query(SEARCH_ENTITIES_BY_NAME, q="工作坊", group_id=group_id, limit=limit) - if workshop_rows: - tech_entities.extend(workshop_rows) - - # 时间顺序相关 - if any(term in question for term in ["先", "后", "第一个"]): - time_rows = await connector.execute_query(SEARCH_ENTITIES_BY_NAME, q="第一次", group_id=group_id, limit=limit) - if time_rows: - tech_entities.extend(time_rows) - - except Exception: - pass - - return tech_entities - - -# 中英相对时间解析：today/昨天/上周/3天后 等简单归一化为日期 -def _resolve_relative_times_cn_en(text: str, anchor: datetime) -> str: - t = str(text) if text is not None else "" - # 英文 today/yesterday/tomorrow - t = re.sub(r"\btoday\b", anchor.date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\byesterday\b", (anchor - timedelta(days=1)).date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\btomorrow\b", (anchor + timedelta(days=1)).date().isoformat(), t, flags=re.IGNORECASE) - - # 英文 X days ago / in X days - def _ago_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor - timedelta(days=n)).date().isoformat() - def _in_repl(m: re.Match[str]) -> str: - n = int(m.group(1)) - return (anchor + timedelta(days=n)).date().isoformat() - t = re.sub(r"\b(\d+)\s+days\s+ago\b", _ago_repl, t, flags=re.IGNORECASE) - t = re.sub(r"\bin\s+(\d+)\s+days\b", _in_repl, t, flags=re.IGNORECASE) - t = re.sub(r"\blast\s+week\b", (anchor - timedelta(days=7)).date().isoformat(), t, flags=re.IGNORECASE) - t = re.sub(r"\bnext\s+week\b", (anchor + timedelta(days=7)).date().isoformat(), t, flags=re.IGNORECASE) - - # 中文 今天/昨天/明天 - t = re.sub(r"今天", anchor.date().isoformat(), t) - t = re.sub(r"昨日|昨天", (anchor - timedelta(days=1)).date().isoformat(), t) - t = re.sub(r"明天", (anchor + timedelta(days=1)).date().isoformat(), t) - # 中文 X天前 / X天后 - t = re.sub(r"(\d+)天前", lambda m: (anchor - timedelta(days=int(m.group(1)))).date().isoformat(), t) - t = re.sub(r"(\d+)天后", lambda m: (anchor + timedelta(days=int(m.group(1)))).date().isoformat(), t) - # 中文 上周 / 下周（近似7天） - t = re.sub(r"上周", (anchor - timedelta(days=7)).date().isoformat(), t) - t = re.sub(r"下周", (anchor + timedelta(days=7)).date().isoformat(), t) - # 中文 月日（无年份）补全年份 - def _md_repl(m: re.Match[str]) -> str: - mon = int(m.group(1)); day = int(m.group(2)) - return f"{anchor.year}-{mon:02d}-{day:02d}" - t = re.sub(r"(\d{1,2})月(\d{1,2})日", _md_repl, t) - return t - - -async def run_longmemeval_test( - sample_size: int = 3, - group_id: str = "longmemeval_zh_bak_2", - search_limit: int = 8, - context_char_budget: int = 4000, - llm_temperature: float = 0.0, - llm_max_tokens: int = 16, - search_type: str = "hybrid", - data_path: str | None = None, - start_index: int = 0, -) -> Dict[str, Any]: - """LongMemEval 评估测试：增强技术术语检索能力""" - - # 数据路径 - if not data_path: - # 固定使用中文数据集：data/longmemeval_oracle_zh.json - zh_proj = os.path.join(PROJECT_ROOT, "data", "longmemeval_oracle_zh.json") - zh_cwd = os.path.join(os.getcwd(), "data", "longmemeval_oracle_zh.json") - if os.path.exists(zh_proj): - data_path = zh_proj - elif os.path.exists(zh_cwd): - data_path = zh_cwd - else: - raise FileNotFoundError("未找到数据集: data/longmemeval_oracle_zh.json，请确保其存在于项目根目录或当前工作目录的 data 目录下。") - - qa_list: List[Dict[str, Any]] = load_dataset_any(data_path) - # 支持评估全部样本：当 sample_size <= 0 时，取从 start_index 到末尾 - if sample_size is None or sample_size <= 0: - items = qa_list[start_index:] - else: - items = qa_list[start_index:start_index + sample_size] - - # 初始化组件 - 使用异步LLM客户端 - with get_db_context() as db: - factory = MemoryClientFactory(db) - llm_client = factory.get_llm_client(SELECTED_LLM_ID) - connector = Neo4jConnector() - with get_db_context() as db: - config_service = MemoryConfigService(db) - cfg_dict = config_service.get_embedder_config(SELECTED_EMBEDDING_ID) - embedder = OpenAIEmbedderClient( - model_config=RedBearModelConfig.model_validate(cfg_dict) - ) - - # 指标收集 - latencies_llm: List[float] = [] - latencies_search: List[float] = [] - per_query_context_counts: List[int] = [] - per_query_context_avg_tokens: List[float] = [] - per_query_context_chars: List[int] = [] - - type_correct: Dict[str, List[float]] = {} - type_f1: Dict[str, List[float]] = {} - type_jacc: Dict[str, List[float]] = {} - - samples: List[Dict[str, Any]] = [] - # 统计重复的上下文预览（跨样本），便于诊断"相同上下文"问题 - preview_counter: Dict[str, int] = {} - - try: - for item in items: - question = item.get("question", "") - reference = item.get("answer", "") - qtype = item.get("question_type") or item.get("type", "unknown") - - print(f"\n=== 处理问题: {question} ===") - - # 检测问题类型 - is_temporal = any(keyword in question.lower() for keyword in - ['days', 'day', 'before', 'after', 'first', '先后', '顺序', '间隔', '多久', '多少天']) - - # 检索 - t0 = time.time() - contexts_all: List[str] = [] - dialogs, statements, entities = [], [], [] - - try: - if search_type == "embedding": - search_results = await search_graph_by_embedding( - connector=connector, - embedder_client=embedder, - query_text=question, - group_id=group_id, - limit=search_limit, - include=["dialogues", "statements", "entities"], - ) - dialogs = search_results.get("dialogues", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - - for d in dialogs: - content = str(d.get("content", "")).strip() - if content: - contexts_all.append(content) - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - # 实体摘要（最多3个） - scored = [e for e in entities if e.get("score") is not None] - top_entities = sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] if scored else entities[:3] - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append(f"EntitySummary: {name}{(' [' + '; '.join(meta) + ']') if meta else ''}") - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - - elif search_type == "keyword": - search_results = await search_graph( - connector=connector, - q=question, - group_id=group_id, - limit=search_limit, - ) - dialogs = search_results.get("dialogues", []) - statements = search_results.get("statements", []) - entities = search_results.get("entities", []) - - for d in dialogs: - content = str(d.get("content", "")).strip() - if content: - contexts_all.append(content) - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - if entities: - entity_names = [str(e.get("name", "")).strip() for e in entities[:5] if e.get("name")] - if entity_names: - contexts_all.append(f"EntitySummary: {', '.join(entity_names)}") - - else: # hybrid（增强版：特别优化技术术语检索） - emb_dialogs, emb_statements, emb_entities = [], [], [] - kw_dialogs, kw_statements, kw_entities = [], [], [] - - # 1) 嵌入检索 - try: - emb_res = await search_graph_by_embedding( - connector=connector, - embedder_client=embedder, - query_text=question, - group_id=group_id, - limit=search_limit, - include=["dialogues", "statements", "entities"], - ) - if isinstance(emb_res, dict): - emb_dialogs = emb_res.get("dialogues", []) or [] - emb_statements = emb_res.get("statements", []) or [] - emb_entities = emb_res.get("entities", []) or [] - except Exception as e: - print(f"⚠️ 嵌入检索失败，将继续进行关键词检索: {e}") - - # 2) 关键词检索（增强版） - try: - kw_res = await search_graph( - connector=connector, - q=question, - group_id=group_id, - limit=search_limit, - ) - if isinstance(kw_res, dict): - kw_dialogs = kw_res.get("dialogues", []) or [] - kw_statements = kw_res.get("statements", []) or [] - kw_entities = kw_res.get("entities", []) or [] - - # 技术术语专门检索 - tech_entities = await _search_tech_terms(connector, question, group_id, search_limit//2) - if tech_entities: - kw_entities.extend(tech_entities) - - # 时间推理问题的特殊处理 - if is_temporal: - # 专门搜索时间实体 - time_entities = await _search_time_entities(connector, group_id, search_limit//2) - if time_entities: - kw_entities.extend(time_entities) - # 添加时间相关关键词检索 - time_keywords = ['天', '日', '月', '年', 'before', 'after', 'first'] - for tk in time_keywords: - try: - time_res = await search_graph( - connector=connector, - q=tk, - group_id=group_id, - limit=2, - ) - if isinstance(time_res, dict): - kw_dialogs.extend(time_res.get("dialogues", []) or []) - kw_statements.extend(time_res.get("statements", []) or []) - except Exception: - pass - - # 中文关键词拆分后做别名匹配 - cn_tokens = generate_query_keywords_cn(question) # 使用增强版关键词提取 - alias_entities = await _search_entities_by_aliases(connector, cn_tokens, group_id, search_limit) - if alias_entities: - kw_entities.extend(alias_entities) - - # 从对话/陈述中的 entity_ids 反查实体 - ids = [] - try: - for d in kw_dialogs: - ids.extend(d.get("entity_ids", []) or []) - for s in kw_statements: - ids.extend(s.get("entity_ids", []) or []) - except Exception: - pass - if ids: - id_entities = await _fetch_entities_by_ids(connector, ids, group_id) - if id_entities: - kw_entities.extend(id_entities) - - # 多关键词检索（使用增强版关键词） - try: - eng_words = [w for w in set(re.findall(r"\b\w+\b", question.lower())) if len(w) > 2] - kw_list = generate_query_keywords_cn(question)[:4] # 使用更多关键词 - for kw in kw_list: - if not kw: - continue - sub_res = await search_graph( - connector=connector, - q=str(kw), - group_id=group_id, - limit=max(3, search_limit // 2), - ) - if isinstance(sub_res, dict): - kw_dialogs.extend(sub_res.get("dialogues", []) or []) - kw_statements.extend(sub_res.get("statements", []) or []) - kw_entities.extend(sub_res.get("entities", []) or []) - except Exception: - pass - - # 选项参与关键词检索 - try: - opt_list = extract_candidate_options(question)[:2] - for opt in opt_list: - if not opt: - continue - opt_res = await search_graph( - connector=connector, - q=str(opt), - group_id=group_id, - limit=max(3, search_limit // 2), - ) - if isinstance(opt_res, dict): - kw_dialogs.extend(opt_res.get("dialogues", []) or []) - kw_statements.extend(opt_res.get("statements", []) or []) - kw_entities.extend(opt_res.get("entities", []) or []) - except Exception: - pass - except Exception as e: - print(f"❌ 关键词检索失败: {e}") - - # 3) 合并、排序并去重 - all_dialogs = emb_dialogs + kw_dialogs - all_statements = emb_statements + kw_statements - all_entities = emb_entities + kw_entities - - def dedup(items: List[Dict[str, Any]], key_field: str = "uuid") -> List[Dict[str, Any]]: - seen = set() - out = [] - for it in items: - key = str(it.get(key_field, "")) + str(it.get("content", "") + str(it.get("statement", ""))) - if key not in seen: - out.append(it) - seen.add(key) - return out - - # 关键技术实体优先排序 - def enhanced_score(item: Dict[str, Any]) -> float: - score_val = item.get("score", 0.0) - base_score = float(score_val) if score_val is not None else 0.0 - content = str(item.get("content", "") + str(item.get("statement", ""))) - - # 关键技术实体奖励 - key_entities = [] - if any(term in question for term in ["GPS", "导航", "系统"]): - key_entities.extend(["GPS", "导航", "系统", "功能"]) - if any(term in question for term in ["工作坊", "研讨会", "活动"]): - key_entities.extend(["工作坊", "研讨会", "参加"]) - - key_bonus = 0 - for key_ent in key_entities: - if key_ent in content: - key_bonus += 1.0 - - # 时间实体奖励 - time_bonus = 0 - if is_temporal: - time_entities = extract_time_entities(content) - time_bonus = len(time_entities) * 0.5 - - return base_score + key_bonus + time_bonus - - dialogs = dedup(sorted(all_dialogs, key=enhanced_score, reverse=True)) - statements = dedup(sorted(all_statements, key=enhanced_score, reverse=True)) - entities = dedup(all_entities, key_field="name") - - # 4) 构建上下文 - for d in dialogs: - content = str(d.get("content", "")).strip() - if content: - contexts_all.append(content) - for s in statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - # 实体摘要 - try: - scored = [e for e in entities if e.get("score") is not None] - top_entities = sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] if scored else entities[:3] - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append(f"EntitySummary: {name}{(' [' + '; '.join(meta) + ']') if meta else ''}") - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - except Exception: - pass - - # 全局回退 - if not contexts_all and search_type in ("embedding", "hybrid"): - try: - print("🔁 检索为空，回退到关键词检索...") - kw_fallback = await search_graph( - connector=connector, - q=question, - group_id=group_id, - limit=max(search_limit, 5), - ) - fb_dialogs = kw_fallback.get("dialogues", []) or [] - fb_statements = kw_fallback.get("statements", []) or [] - fb_entities = kw_fallback.get("entities", []) or [] - - for d in fb_dialogs: - content = str(d.get("content", "")).strip() - if content: - contexts_all.append(content) - for s in fb_statements: - stmt_text = str(s.get("statement", "")).strip() - if stmt_text: - contexts_all.append(stmt_text) - if fb_entities: - entity_names = [str(e.get("name", "")).strip() for e in fb_entities[:5] if e.get("name")] - if entity_names: - contexts_all.append(f"EntitySummary: {', '.join(entity_names)}") - - dialogs = fb_dialogs if fb_dialogs else dialogs - statements = fb_statements if fb_statements else statements - entities = fb_entities if fb_entities else entities - print(f"↩️ 回退到关键词检索: {len(fb_dialogs)} 对话, {len(fb_statements)} 条陈述, {len(fb_entities)} 个实体") - except Exception as fe: - print(f"❌ 关键词回退失败: {fe}") - - ent_count = len(entities) if isinstance(entities, list) else 0 - print(f"✅ {search_type}检索成功: {len(dialogs)} 对话, {len(statements)} 条陈述, {ent_count} 个实体") - if is_temporal: - print("⏰ 检测为时间推理问题，已启用时间优化检索") - - except Exception as e: - print(f"❌ {search_type}检索失败: {e}") - contexts_all = [] - - t1 = time.time() - latencies_search.append((t1 - t0) * 1000) - - # 智能上下文选择 - context_text = "" - if contexts_all: - context_text = smart_context_selection(contexts_all, question, max_chars=context_char_budget) - # 相对时间解析 - try: - context_text = _resolve_relative_times_cn_en(context_text, anchor=datetime.now()) - except Exception: - pass - # 诊断信息 - try: - cn_diag = generate_query_keywords_cn(question)[:4] # 显示更多关键词 - opts = extract_candidate_options(question)[:2] - qlw = [w for w in set(re.findall(r'\b\w+\b', question.lower())) if len(w) > 2][:1] - diag_tokens: List[str] = [] - for t in cn_diag + opts + qlw: - if t and t not in diag_tokens: - diag_tokens.append(t) - print(f"🔍 关键词/选项: {', '.join(diag_tokens)}") - preview = context_text[:200].replace('\n', ' ') - print(f"🔎 上下文预览: {preview}...") - key_preview = preview.strip() - if key_preview: - preview_counter[key_preview] = preview_counter.get(key_preview, 0) + 1 - except Exception: - pass - else: - print("❌ 没有检索到有效上下文") - context_text = "No relevant context found." - - # 记录上下文诊断信息 - per_query_context_counts.append(len(contexts_all)) - per_query_context_avg_tokens.append(avg_context_tokens([context_text])) - per_query_context_chars.append(len(context_text)) - - # LLM 推理（增强技术术语提示） - options = extract_candidate_options(question) - if len(options) >= 2: - opt_lines = "\n".join(f"- {o}" for o in options) - # 技术术语问题的特殊提示 - if any(term in question for term in ["GPS", "系统", "功能", "工作坊", "研讨会"]): - system_prompt = ( - "You are a QA assistant specializing in technical and activity-related questions. " - "Pay special attention to technical terms like GPS, systems, functions, workshops, and seminars. " - "Return ONLY one string: exactly one option from the provided candidates. If the context is insufficient, respond with 'Unknown'. " - "Focus on matching technical details and activity sequences accurately." - ) - elif is_temporal: - system_prompt = ( - "You are a QA assistant specializing in temporal reasoning. Analyze the dates and time relationships in the context carefully. " - "Return ONLY one string: exactly one option from the provided candidates. If the context is insufficient, respond with 'Unknown'. " - "Pay special attention to date sequences and time intervals." - ) - else: - system_prompt = ( - "You are a QA assistant. Respond in the same language as the question. Return ONLY one string: exactly one option from the provided candidates. " - "If the context is insufficient, respond with 'Unknown'. If the context expresses a synonym or paraphrase of a candidate, return the closest candidate. " - "Do not include explanations." - ) - - messages = [ - {"role": "system", "content": system_prompt}, - { - "role": "user", - "content": ( - f"Question: {question}\n\nCandidates:\n{opt_lines}\n\nContext:\n{context_text}\n\nReturn EXACTLY one candidate string (or 'Unknown')." - ), - }, - ] - else: - # 技术术语问题的特殊提示 - if any(term in question for term in ["GPS", "系统", "功能", "工作坊", "研讨会"]): - system_prompt = ( - "You are a QA assistant specializing in technical and activity-related questions. " - "Pay special attention to technical terms like GPS, systems, functions, workshops, and seminars. " - "If the context contains the answer, return a concise answer phrase focusing on technical details. " - "If the answer cannot be determined from the context, respond with 'Unknown'. Return ONLY the final answer string, no explanations." - ) - elif is_temporal: - system_prompt = ( - "You are a QA assistant specializing in temporal reasoning. Analyze the dates and time relationships in the context carefully. " - "If the context contains the answer, return a concise answer phrase focusing on temporal information. " - "If the answer cannot be determined from the context, respond with 'Unknown'. Return ONLY the final answer string, no explanations." - ) - else: - system_prompt = ( - "You are a QA assistant. Respond in the same language as the question. If the context contains the answer, return a concise answer phrase. " - "If the answer cannot be determined from the context, respond with 'Unknown'. Return ONLY the final answer string, no explanations." - ) - - messages = [ - {"role": "system", "content": system_prompt}, - { - "role": "user", - "content": f"Question: {question}\n\nContext:\n{context_text}\n\nReturn ONLY the answer (or 'Unknown').", - }, - ] - - t2 = time.time() - # 使用异步调用 - resp = await llm_client.chat(messages=messages) - t3 = time.time() - latencies_llm.append((t3 - t2) * 1000) - - # 兼容不同的响应格式 - pred_raw = resp.content.strip() if hasattr(resp, 'content') else (resp["choices"][0]["message"]["content"].strip() if isinstance(resp, dict) else "Unknown") - - # 选项题输出规范化 - pred = pred_raw - if len(options) >= 2 and not pred_raw.lower().startswith("unknown"): - def _basic_norm(s: str) -> str: - s = s.lower().strip() - return re.sub(r"[^\w\s]", " ", s) - def _jaccard(a: str, b: str) -> float: - ta = set(t for t in _basic_norm(a).split() if t) - tb = set(t for t in _basic_norm(b).split() if t) - if not ta and not tb: - return 1.0 - if not ta or not tb: - return 0.0 - return len(ta & tb) / len(ta | tb) - best = None - best_score = -1.0 - for o in options: - score = _jaccard(pred_raw, o) - if score > best_score: - best = o - best_score = score - if best is not None and best_score > 0.0: - pred = best - - # 指标 - flag = exact_match(pred, reference) - f1_val = common_f1(str(pred), str(reference)) - j_val = jaccard(str(pred), str(reference)) - - type_correct.setdefault(qtype, []).append(flag) - type_f1.setdefault(qtype, []).append(f1_val) - type_jacc.setdefault(qtype, []).append(j_val) - - samples.append({ - "question": question, - "prediction": pred, - "answer": reference, - "question_type": qtype, - "is_temporal": is_temporal, - "question_id": item.get("question_id"), - "options": options, - "context_count": len(contexts_all), - "context_chars": len(context_text), - "retrieved_dialogue_count": len(dialogs), - "retrieved_statement_count": len(statements), - "metrics": { - "exact_match": bool(flag), - "f1": f1_val, - "jaccard": j_val - }, - "timing": { - "search_ms": (t1 - t0) * 1000, - "llm_ms": (t3 - t2) * 1000 - } - }) - - print(f"🤖 LLM 回答: {pred}") - print(f"✅ 正确答案: {reference}") - print(f"📈 当前指标 - Exact Match: {flag}, F1: {f1_val:.3f}, Jaccard: {j_val:.3f}") - - # 聚合结果 - type_acc = {t: (sum(v) / max(len(v), 1)) for t, v in type_correct.items()} - f1_by_type = {t: (sum(v) / max(len(v), 1)) for t, v in type_f1.items()} - jacc_by_type = {t: (sum(v) / max(len(v), 1)) for t, v in type_jacc.items()} - - result = { - "dataset": "longmemeval", - "items": len(items), - "accuracy_by_type": type_acc, - "f1_by_type": f1_by_type, - "jaccard_by_type": jacc_by_type, - "samples": samples, - "latency": { - "search": latency_stats(latencies_search), - "llm": latency_stats(latencies_llm), - }, - "context": { - "avg_tokens": statistics.mean(per_query_context_avg_tokens) if per_query_context_avg_tokens else 0.0, - "avg_chars": statistics.mean(per_query_context_chars) if per_query_context_chars else 0.0, - "count_avg": statistics.mean(per_query_context_counts) if per_query_context_counts else 0.0, - }, - "params": { - "group_id": group_id, - "search_limit": search_limit, - "context_char_budget": context_char_budget, - "search_type": search_type, - "llm_id": SELECTED_LLM_ID, - "embedding_id": SELECTED_EMBEDDING_ID, - "sample_size": sample_size, - "start_index": start_index, - }, - "timestamp": datetime.now().isoformat() - } - - # 计算汇总指标 - try: - total_items = max(len(samples), 1) - correct_count = sum(1 for s in samples if s.get("metrics", {}).get("exact_match")) - score_accuracy = (correct_count / total_items) * 100.0 - - total_latencies_ms = [] - for s in samples: - t = s.get("timing", {}) - total_latencies_ms.append(float(t.get("search_ms", 0.0)) + float(t.get("llm_ms", 0.0))) - total_lat_stats = latency_stats(total_latencies_ms) if total_latencies_ms else {"p50": 0.0, "iqr": 0.0} - latency_median_s = total_lat_stats.get("p50", 0.0) / 1000.0 - latency_iqr_s = total_lat_stats.get("iqr", 0.0) / 1000.0 - - avg_ctx_tokens = statistics.mean(per_query_context_avg_tokens) if per_query_context_avg_tokens else 0.0 - avg_ctx_tokens_k = avg_ctx_tokens / 1000.0 - - result["metric_summary"] = { - "score_accuracy": score_accuracy, - "latency_median_s": latency_median_s, - "latency_iqr_s": latency_iqr_s, - "avg_context_tokens_k": avg_ctx_tokens_k, - } - except Exception: - result["metric_summary"] = { - "score_accuracy": 0.0, - "latency_median_s": 0.0, - "latency_iqr_s": 0.0, - "avg_context_tokens_k": 0.0, - } - - # 诊断信息 - try: - dups = sorted([(k, c) for k, c in preview_counter.items() if c > 1], key=lambda x: -x[1])[:5] - result["diagnostics"] = { - "duplicate_previews_top": [{"count": c, "preview": k[:120]} for k, c in dups], - "unique_preview_count": len(preview_counter), - } - except Exception: - pass - - return result - - finally: - await connector.close() - - -def main(): - load_dotenv() - parser = argparse.ArgumentParser(description="LongMemEval 评估测试脚本（增强技术术语检索版）") - parser.add_argument("--sample-size", type=int, default=3, help="样本数量（<=0 表示全部）") - parser.add_argument("--all", action="store_true", help="评估全部样本（覆盖 --sample-size）") - parser.add_argument("--start-index", type=int, default=0, help="起始样本索引") - parser.add_argument("--group-id", type=str, default="longmemeval_zh_bak_3", help="图数据库 Group ID") - parser.add_argument("--search-limit", type=int, default=8, help="检索条数上限") - parser.add_argument("--context-char-budget", type=int, default=4000, help="上下文字符预算") - parser.add_argument("--llm-temperature", type=float, default=0.0, help="LLM 温度") - parser.add_argument("--llm-max-tokens", type=int, default=16, help="LLM 最大输出 token") - parser.add_argument("--search-type", type=str, default="hybrid", choices=["embedding","keyword","hybrid"], help="检索类型") - parser.add_argument("--data-path", type=str, default=None, help="数据集路径") - args = parser.parse_args() - - sample_size = 0 if args.all else args.sample_size - - result = asyncio.run( - run_longmemeval_test( - sample_size=sample_size, - group_id=args.group_id, - search_limit=args.search_limit, - context_char_budget=args.context_char_budget, - llm_temperature=args.llm_temperature, - llm_max_tokens=args.llm_max_tokens, - search_type=args.search_type, - data_path=args.data_path, - start_index=args.start_index, - ) - ) - - # 打印结果 - print("\n" + "="*50) - print("📊 LongMemEval 测试结果:") - print(f" 样本数量: {result['items']}") - - if result['accuracy_by_type']: - print("\n📈 按问题类型细分:") - for qtype, acc in result['accuracy_by_type'].items(): - print(f" {qtype}:") - print(f" Score (Accuracy): {acc:.3f}") - - print(f"\n📊 指标总览:") - ms = result.get('metric_summary', {}) - print(f" Score (Accuracy): {ms.get('score_accuracy', 0.0):.1f}%") - print(f" Latency (s): median {ms.get('latency_median_s', 0.0):.3f}s") - print(f" Latency IQR (s): {ms.get('latency_iqr_s', 0.0):.3f}s") - print(f" Avg Context Tokens (k): {ms.get('avg_context_tokens_k', 0.0):.3f}k") - - print(f"\n⏱️ 细分性能指标:") - print(f" 检索延迟(均值): {result['latency']['search']['mean']:.1f}ms") - print(f" LLM延迟(均值): {result['latency']['llm']['mean']:.1f}ms") - print(f" 上下文长度(均值): {result['context']['avg_chars']:.0f} 字符") - - - # 保存结果到文件 - try: - out_dir = os.path.join(PROJECT_ROOT, "evaluation", "longmemeval", "results") - os.makedirs(out_dir, exist_ok=True) - ts = datetime.now().strftime("%Y%m%d_%H%M%S") - out_path = os.path.join(out_dir, f"longmemeval_{result['params']['search_type']}_{ts}.json") - with open(out_path, "w", encoding="utf-8") as f: - json.dump(result, f, ensure_ascii=False, indent=2) - print(f"\n💾 结果已保存: {out_path}") - except Exception as e: - print(f"⚠️ 结果保存失败: {e}") - - -if __name__ == "__main__": - main() diff --git a/api/app/core/memory/evaluation/memsciqa/evaluate_qa.py b/api/app/core/memory/evaluation/memsciqa/evaluate_qa.py deleted file mode 100644 index 6efb66ff..00000000 --- a/api/app/core/memory/evaluation/memsciqa/evaluate_qa.py +++ /dev/null @@ -1,324 +0,0 @@ -import argparse -import asyncio -import json -import os -import time -from datetime import datetime -from typing import TYPE_CHECKING, Any, Dict, List - -if TYPE_CHECKING: - from app.schemas.memory_config_schema import MemoryConfig - -try: - from dotenv import load_dotenv -except Exception: - def load_dotenv(): - return None - -from app.core.memory.evaluation.common.metrics import ( - avg_context_tokens, - exact_match, - latency_stats, -) -from app.core.memory.evaluation.extraction_utils import ( - ingest_contexts_via_full_pipeline, -) -from app.core.memory.storage_services.search import run_hybrid_search -from app.core.memory.utils.config.definitions import ( - PROJECT_ROOT, - SELECTED_EMBEDDING_ID, - SELECTED_GROUP_ID, - SELECTED_LLM_ID, -) -from app.core.memory.utils.llm.llm_utils import MemoryClientFactory -from app.db import get_db_context -from app.repositories.neo4j.neo4j_connector import Neo4jConnector - - -def smart_context_selection(contexts: List[str], question: str, max_chars: int = 4000) -> str: - """基于问题关键词对上下文进行评分选择，并在预算内拼接文本。""" - if not contexts: - return "" - import re - # 提取问题关键词（移除停用词） - question_lower = (question or "").lower() - stop_words = { - 'what','when','where','who','why','how','did','do','does','is','are','was','were', - 'the','a','an','and','or','but' - } - question_words = set(re.findall(r"\b\w+\b", question_lower)) - question_words = {w for w in question_words if w not in stop_words and len(w) > 2} - - # 评分 - scored = [] - for i, ctx in enumerate(contexts): - ctx_lower = (ctx or "").lower() - score = 0 - matches = 0 - for w in question_words: - if w in ctx_lower: - matches += 1 - score += ctx_lower.count(w) * 2 - length = len(ctx) - if 100 < length < 2000: - score += 5 - elif length >= 2000: - score += 2 - if i < 3: - score += 3 - scored.append((score, ctx, matches)) - - scored.sort(key=lambda x: x[0], reverse=True) - - # 选择直到达到字符限制，必要时截断包含关键词的段落 - selected: List[str] = [] - total = 0 - for score, ctx, _ in scored: - if total + len(ctx) <= max_chars: - selected.append(ctx) - total += len(ctx) - else: - if score > 10 and total < max_chars - 200: - remaining = max_chars - total - lines = ctx.split('\n') - rel_lines: List[str] = [] - cur = 0 - for line in lines: - l = line.lower() - if any(w in l for w in question_words) and cur < remaining - 50: - rel_lines.append(line) - cur += len(line) - if rel_lines: - truncated = '\n'.join(rel_lines) - if len(truncated) > 50: - selected.append(truncated + "\n[相关内容截断...]") - total += len(truncated) - break - return "\n\n".join(selected) - - -def build_context_from_dialog(dialog_obj: Dict[str, Any]) -> str: - """Compose a text context from `dialog` list in msc_self_instruct item.""" - parts: List[str] = [] - for turn in dialog_obj.get("dialog", []): - speaker = turn.get("speaker", "") - text = turn.get("text", "") - if text: - parts.append(f"{speaker}: {text}") - return "\n".join(parts) - - -def _combine_dialogues_for_hybrid(results: Dict[str, Any]) -> List[Dict[str, Any]]: - """Combine dialogues from embedding and keyword searches (embedding first).""" - if results is None: - return [] - emb = [] - kw = [] - if isinstance(results.get("embedding_search"), dict): - emb = results.get("embedding_search", {}).get("dialogues", []) or [] - elif isinstance(results.get("dialogues"), list): - emb = results.get("dialogues", []) or [] - if isinstance(results.get("keyword_search"), dict): - kw = results.get("keyword_search", {}).get("dialogues", []) or [] - seen = set() - merged: List[Dict[str, Any]] = [] - for d in emb: - k = (str(d.get("uuid", "")), str(d.get("content", ""))) - if k not in seen: - merged.append(d) - seen.add(k) - for d in kw: - k = (str(d.get("uuid", "")), str(d.get("content", ""))) - if k not in seen: - merged.append(d) - seen.add(k) - return merged - - -async def run_memsciqa_eval(sample_size: int = 1, group_id: str | None = None, search_limit: int = 8, context_char_budget: int = 4000, llm_temperature: float = 0.0, llm_max_tokens: int = 64, search_type: str = "hybrid", memory_config: "MemoryConfig" = None) -> Dict[str, Any]: - group_id = group_id or SELECTED_GROUP_ID - # Load data - data_path = os.path.join(PROJECT_ROOT, "data", "msc_self_instruct.jsonl") - if not os.path.exists(data_path): - data_path = os.path.join(os.getcwd(), "data", "msc_self_instruct.jsonl") - with open(data_path, "r", encoding="utf-8") as f: - lines = f.readlines() - items: List[Dict[str, Any]] = [json.loads(l) for l in lines[:sample_size]] - # 改为：每条样本仅摄入一个上下文（完整对话转录），避免多上下文摄入 - # 说明：memsciqa 数据集的每个样本天然只有一个对话，保持按样本一上下文的策略 - contexts: List[str] = [build_context_from_dialog(item) for item in items] - await ingest_contexts_via_full_pipeline(contexts, group_id) - - # LLM client (使用异步调用) - with get_db_context() as db: - factory = MemoryClientFactory(db) - llm_client = factory.get_llm_client(SELECTED_LLM_ID) - - # Evaluate each item - connector = Neo4jConnector() - latencies_llm: List[float] = [] - latencies_search: List[float] = [] - contexts_used: List[str] = [] - correct_flags: List[float] = [] - f1s: List[float] = [] - b1s: List[float] = [] - jss: List[float] = [] - try: - for item in items: - question = item.get("self_instruct", {}).get("B", "") or item.get("question", "") - reference = item.get("self_instruct", {}).get("A", "") or item.get("answer", "") - # 检索：对齐 locomo 的三路检索（dialogues/statements/entities） - t0 = time.time() - try: - results = await run_hybrid_search( - query_text=question, - search_type=search_type, - group_id=group_id, - limit=search_limit, - include=["dialogues", "statements", "entities"], - output_path=None, - memory_config=memory_config, - ) - except Exception: - results = None - t1 = time.time() - latencies_search.append((t1 - t0) * 1000) - - # 构建上下文：包含对话、陈述和实体摘要，并智能选择 - contexts_all: List[str] = [] - if results: - if search_type == "hybrid": - emb = results.get("embedding_search", {}) if isinstance(results.get("embedding_search"), dict) else {} - kw = results.get("keyword_search", {}) if isinstance(results.get("keyword_search"), dict) else {} - emb_dialogs = emb.get("dialogues", []) - emb_statements = emb.get("statements", []) - emb_entities = emb.get("entities", []) - kw_dialogs = kw.get("dialogues", []) - kw_statements = kw.get("statements", []) - kw_entities = kw.get("entities", []) - all_dialogs = emb_dialogs + kw_dialogs - all_statements = emb_statements + kw_statements - all_entities = emb_entities + kw_entities - - # 简单去重与限制 - seen_texts = set() - for d in all_dialogs: - text = str(d.get("content", "")).strip() - if text and text not in seen_texts: - contexts_all.append(text) - seen_texts.add(text) - if len(contexts_all) >= search_limit: - break - for s in all_statements: - text = str(s.get("statement", "")).strip() - if text and text not in seen_texts: - contexts_all.append(text) - seen_texts.add(text) - if len(contexts_all) >= search_limit: - break - # 实体摘要（最多3个） - names = [] - merged_entities = all_entities[:] - for e in merged_entities: - name = str(e.get("name", "")).strip() - if name and name not in names: - names.append(name) - if len(names) >= 3: - break - if names: - contexts_all.append("EntitySummary: " + ", ".join(names)) - else: - dialogs = results.get("dialogues", []) - statements = results.get("statements", []) - entities = results.get("entities", []) - for d in dialogs: - text = str(d.get("content", "")).strip() - if text: - contexts_all.append(text) - for s in statements: - text = str(s.get("statement", "")).strip() - if text: - contexts_all.append(text) - names = [str(e.get("name", "")).strip() for e in entities[:3] if e.get("name")] - if names: - contexts_all.append("EntitySummary: " + ", ".join(names)) - - # 智能选择并截断到预算 - context_text = smart_context_selection(contexts_all, question, max_chars=context_char_budget) if contexts_all else "" - if not context_text: - context_text = "No relevant context found." - contexts_used.append(context_text[:200]) - - # Call LLM (使用异步调用) - messages = [ - {"role": "system", "content": "You are a QA assistant. Answer in English. Strictly follow: 1) If the context contains the answer, copy the shortest exact span from the context as the answer; 2) If the answer cannot be determined from the context, respond with 'Unknown'; 3) Return ONLY the answer text, no explanations."}, - {"role": "user", "content": f"Question: {question}\n\nContext:\n{context_text}"}, - ] - t2 = time.time() - resp = await llm_client.chat(messages=messages) - t3 = time.time() - latencies_llm.append((t3 - t2) * 1000) - pred = resp.content.strip() if hasattr(resp, 'content') else (resp["choices"][0]["message"]["content"].strip() if isinstance(resp, dict) else str(resp).strip()) - # Metrics: F1, BLEU-1, Jaccard; keep exact match for reference - correct_flags.append(exact_match(pred, reference)) - from app.core.memory.evaluation.common.metrics import ( - bleu1, - f1_score, - jaccard, - ) - f1s.append(f1_score(str(pred), str(reference))) - b1s.append(bleu1(str(pred), str(reference))) - jss.append(jaccard(str(pred), str(reference))) - - # Aggregate metrics - acc = sum(correct_flags) / max(len(correct_flags), 1) - ctx_avg_tokens = avg_context_tokens(contexts_used) - result = { - "dataset": "memsciqa", - "items": len(items), - "metrics": { - "accuracy": acc, - # Placeholders for extensibility - "f1": (sum(f1s) / max(len(f1s), 1)) if f1s else 0.0, - "bleu1": (sum(b1s) / max(len(b1s), 1)) if b1s else 0.0, - "jaccard": (sum(jss) / max(len(jss), 1)) if jss else 0.0, - }, - "latency": { - "search": latency_stats(latencies_search), - "llm": latency_stats(latencies_llm), - }, - "avg_context_tokens": ctx_avg_tokens, - } - return result - finally: - await connector.close() - - -def main(): - load_dotenv() - parser = argparse.ArgumentParser(description="Evaluate DMR (memsciqa) with graph search and Qwen") - parser.add_argument("--sample-size", type=int, default=1, help="评测样本数量") - parser.add_argument("--group-id", type=str, default=None, help="可选 group_id，默认取 runtime.json") - parser.add_argument("--search-limit", type=int, default=8, help="每类检索最大返回数") - parser.add_argument("--context-char-budget", type=int, default=4000, help="上下文字符预算") - parser.add_argument("--llm-temperature", type=float, default=0.0, help="LLM 温度") - parser.add_argument("--llm-max-tokens", type=int, default=64, help="LLM 最大生成长度") - parser.add_argument("--search-type", type=str, choices=["keyword","embedding","hybrid"], default="hybrid", help="检索类型") - args = parser.parse_args() - - result = asyncio.run( - run_memsciqa_eval( - sample_size=args.sample_size, - group_id=args.group_id, - search_limit=args.search_limit, - context_char_budget=args.context_char_budget, - llm_temperature=args.llm_temperature, - llm_max_tokens=args.llm_max_tokens, - search_type=args.search_type, - ) - ) - print(json.dumps(result, ensure_ascii=False, indent=2)) - - -if __name__ == "__main__": - main() diff --git a/api/app/core/memory/evaluation/memsciqa/memsciqa-test.py b/api/app/core/memory/evaluation/memsciqa/memsciqa-test.py deleted file mode 100644 index 279f4042..00000000 --- a/api/app/core/memory/evaluation/memsciqa/memsciqa-test.py +++ /dev/null @@ -1,576 +0,0 @@ -import argparse -import asyncio -import json -import os -import re -import time -from datetime import datetime -from typing import Any, Dict, List - -try: - from dotenv import load_dotenv -except Exception: - def load_dotenv(): - return None - -# 路径与模块导入保持与现有评估脚本一致 -import sys - -_THIS_DIR = os.path.dirname(os.path.abspath(__file__)) -_PROJECT_ROOT = os.path.dirname(os.path.dirname(_THIS_DIR)) -_SRC_DIR = os.path.join(_PROJECT_ROOT, "src") -for _p in (_SRC_DIR, _PROJECT_ROOT): - if _p not in sys.path: - sys.path.insert(0, _p) - -# 对齐 locomo_test 的检索逻辑：直接使用 graph_search 与 Neo4jConnector/Embedder1 -from app.core.memory.evaluation.common.metrics import ( - avg_context_tokens, - exact_match, - latency_stats, -) -from app.core.memory.llm_tools.openai_embedder import OpenAIEmbedderClient -from app.core.memory.utils.config.definitions import ( - PROJECT_ROOT, - SELECTED_EMBEDDING_ID, - SELECTED_GROUP_ID, - SELECTED_LLM_ID, -) -from app.core.memory.utils.llm.llm_utils import MemoryClientFactory -from app.core.models.base import RedBearModelConfig -from app.db import get_db_context -from app.repositories.neo4j.graph_search import search_graph, search_graph_by_embedding -from app.repositories.neo4j.neo4j_connector import Neo4jConnector -from app.services.memory_config_service import MemoryConfigService - -try: - from app.core.memory.evaluation.common.metrics import bleu1, f1_score, jaccard -except Exception: - # 兜底：简单实现（必要时） - def f1_score(pred: str, ref: str) -> float: - ps = pred.lower().split() - rs = ref.lower().split() - if not ps or not rs: - return 0.0 - tp = len(set(ps) & set(rs)) - if tp == 0: - return 0.0 - precision = tp / len(ps) - recall = tp / len(rs) - if precision + recall == 0: - return 0.0 - return 2 * precision * recall / (precision + recall) - - def bleu1(pred: str, ref: str) -> float: - ps = pred.lower().split() - rs = ref.lower().split() - if not ps or not rs: - return 0.0 - overlap = len([w for w in ps if w in rs]) - return overlap / max(len(ps), 1) - - def jaccard(pred: str, ref: str) -> float: - ps = set(pred.lower().split()) - rs = set(ref.lower().split()) - union = len(ps | rs) - if union == 0: - return 0.0 - return len(ps & rs) / union - - -def smart_context_selection(contexts: List[str], question: str, max_chars: int = 4000) -> str: - """基于问题关键词对上下文进行评分选择，并在预算内拼接文本。 - - 参考 evaluation/memsciqa/evaluate_qa.py 的实现，避免路径导入带来的不稳定。 - """ - if not contexts: - return "" - question_lower = (question or "").lower() - stop_words = { - 'what','when','where','who','why','how','did','do','does','is','are','was','were', - 'the','a','an','and','or','but' - } - question_words = set(re.findall(r"\b\w+\b", question_lower)) - question_words = {w for w in question_words if w not in stop_words and len(w) > 2} - - scored = [] - for i, ctx in enumerate(contexts): - ctx_lower = (ctx or "").lower() - score = 0 - matches = 0 - for w in question_words: - if w in ctx_lower: - matches += 1 - score += ctx_lower.count(w) * 2 - length = len(ctx) - if 100 < length < 2000: - score += 5 - elif length >= 2000: - score += 2 - if i < 3: - score += 3 - scored.append((score, ctx, matches)) - - scored.sort(key=lambda x: x[0], reverse=True) - - selected: List[str] = [] - total = 0 - for score, ctx, _ in scored: - if total + len(ctx) <= max_chars: - selected.append(ctx) - total += len(ctx) - else: - if score > 10 and total < max_chars - 200: - remaining = max_chars - total - lines = ctx.split('\n') - rel_lines: List[str] = [] - cur = 0 - for line in lines: - l = line.lower() - if any(w in l for w in question_words) and cur < remaining - 50: - rel_lines.append(line) - cur += len(line) - if rel_lines: - truncated = '\n'.join(rel_lines) - if len(truncated) > 50: - selected.append(truncated + "\n[相关内容截断...]") - total += len(truncated) - break - return "\n\n".join(selected) - - -def extract_question_keywords(question: str, max_keywords: int = 8) -> List[str]: - """提取问题中的关键词（简单英文分词，去停用词，长度>=3）。""" - ql = (question or "").lower() - stop_words = { - 'what','when','where','who','why','how','did','do','does','is','are','was','were', - 'the','a','an','and','or','but','of','to','in','on','for','with','from','that','this' - } - words = re.findall(r"\b[\w-]+\b", ql) - kws = [w for w in words if w not in stop_words and len(w) >= 3] - # 去重保序 - seen = set() - uniq = [] - for w in kws: - if w not in seen: - uniq.append(w) - seen.add(w) - if len(uniq) >= max_keywords: - break - return uniq - - -def analyze_contexts_simple(contexts: List[str], keywords: List[str], top_n: int = 5) -> List[Dict[str, int | float]]: - """对上下文进行简单相关性打分，仅用于控制台可视化。 - - 评分: score = match_count*200 + min(len(text), 100000)/100 - """ - results = [] - for ctx in contexts: - tl = (ctx or "").lower() - match_count = sum(1 for k in keywords if k in tl) - length = len(ctx) - score = match_count * 200 + min(length, 100000) / 100.0 - results.append({"score": float(f"{score:.0f}"), "match": match_count, "length": length}) - results.sort(key=lambda x: (x["score"], x["match"], x["length"]), reverse=True) - return results[:max(top_n, 0)] - - -# 纯测试脚本不进行摄入；若需摄入请使用 evaluate_qa.py - - -def load_dataset_memsciqa(data_path: str) -> List[Dict[str, Any]]: - if not os.path.exists(data_path): - raise FileNotFoundError(f"未找到数据集: {data_path}") - items: List[Dict[str, Any]] = [] - with open(data_path, "r", encoding="utf-8") as f: - for line in f: - line = line.strip() - if not line: - continue - try: - items.append(json.loads(line)) - except Exception: - # 跳过坏行但不中断 - continue - return items - - -async def run_memsciqa_test( - sample_size: int = 3, - group_id: str | None = None, - search_limit: int = 8, - context_char_budget: int = 4000, - llm_temperature: float = 0.0, - llm_max_tokens: int = 64, - search_type: str = "embedding", - data_path: str | None = None, - start_index: int = 0, - verbose: bool = True, -) -> Dict[str, Any]: - """memsciqa 增强测试脚本：结合 evaluate_qa 的三路检索与智能上下文选择。 - - - 支持从指定索引开始与评估全部样本（sample_size<=0） - - 支持在摄入前重置组（清空图）与跳过摄入 - - 支持 keyword / embedding / hybrid 三种检索 - """ - - # 默认使用指定的 memsci 组 ID - group_id = group_id or "group_memsci" - - # 数据路径解析（项目根与当前工作目录兜底） - if not data_path: - proj_path = os.path.join(PROJECT_ROOT, "data", "msc_self_instruct.jsonl") - cwd_path = os.path.join(os.getcwd(), "data", "msc_self_instruct.jsonl") - if os.path.exists(proj_path): - data_path = proj_path - elif os.path.exists(cwd_path): - data_path = cwd_path - else: - raise FileNotFoundError("未找到数据集: data/msc_self_instruct.jsonl，请确保其存在于项目根目录或当前工作目录的 data 目录下。") - - # 加载数据 - all_items = load_dataset_memsciqa(data_path) - if sample_size is None or sample_size <= 0: - items = all_items[start_index:] - else: - items = all_items[start_index:start_index + sample_size] - - # 初始化 LLM（纯测试：不进行摄入） - with get_db_context() as db: - factory = MemoryClientFactory(db) - llm = factory.get_llm_client(SELECTED_LLM_ID) - - # 初始化 Neo4j 连接与向量检索 Embedder（对齐 locomo_test） - connector = Neo4jConnector() - embedder = None - if search_type in ("embedding", "hybrid"): - with get_db_context() as db: - config_service = MemoryConfigService(db) - cfg_dict = config_service.get_embedder_config(SELECTED_EMBEDDING_ID) - embedder = OpenAIEmbedderClient( - model_config=RedBearModelConfig.model_validate(cfg_dict) - ) - - # 评估循环 - latencies_llm: List[float] = [] - latencies_search: List[float] = [] - # 存储完整上下文文本用于统计 - contexts_used: List[str] = [] - per_query_context_chars: List[int] = [] - per_query_context_counts: List[int] = [] - correct_flags: List[float] = [] - f1s: List[float] = [] - b1s: List[float] = [] - jss: List[float] = [] - samples: List[Dict[str, Any]] = [] - - total_items = len(items) - for idx, item in enumerate(items): - if verbose: - print(f"\n🧪 评估样本: {idx+1}/{total_items}") - question = item.get("self_instruct", {}).get("B", "") or item.get("question", "") - reference = item.get("self_instruct", {}).get("A", "") or item.get("answer", "") - - # 三路检索：chunks/statements/entities/summaries（对齐 qwen_search_eval.py） - t0 = time.time() - results = None - try: - if search_type in ("embedding", "hybrid"): - # 使用嵌入检索（与 qwen_search_eval 对齐） - results = await search_graph_by_embedding( - connector=connector, - embedder_client=embedder, - query_text=question, - group_id=group_id, - limit=search_limit, - include=["chunks", "statements", "entities", "summaries"], # 使用 chunks 而不是 dialogues - ) - elif search_type == "keyword": - # 关键词检索（直接调用 graph_search） - results = await search_graph( - connector=connector, - q=question, - group_id=group_id, - limit=search_limit, - include=["chunks", "statements", "entities", "summaries"], # 使用 chunks 而不是 dialogues - ) - except Exception: - results = None - t1 = time.time() - search_ms = (t1 - t0) * 1000 - latencies_search.append(search_ms) - - # 构建上下文：包含 chunks、陈述、摘要和实体（对齐 qwen_search_eval.py） - contexts_all: List[str] = [] - retrieved_counts: Dict[str, int] = {} - if results: - chunks = results.get("chunks", []) - statements = results.get("statements", []) - entities = results.get("entities", []) - summaries = results.get("summaries", []) - retrieved_counts = { - "chunks": len(chunks), - "statements": len(statements), - "entities": len(entities), - "summaries": len(summaries), - } - # 优先使用 chunks - for c in chunks: - text = str(c.get("content", "")).strip() - if text: - contexts_all.append(text) - # 然后是 statements - for s in statements: - text = str(s.get("statement", "")).strip() - if text: - contexts_all.append(text) - # 然后是 summaries - for sm in summaries: - text = str(sm.get("summary", "")).strip() - if text: - contexts_all.append(text) - # 实体摘要：最多加入前3个高分实体（对齐 qwen_search_eval.py） - scored = [e for e in entities if e.get("score") is not None] - top_entities = sorted(scored, key=lambda x: x.get("score", 0), reverse=True)[:3] if scored else entities[:3] - if top_entities: - summary_lines = [] - for e in top_entities: - name = str(e.get("name", "")).strip() - etype = str(e.get("entity_type", "")).strip() - score = e.get("score") - if name: - meta = [] - if etype: - meta.append(f"type={etype}") - if isinstance(score, (int, float)): - meta.append(f"score={score:.3f}") - summary_lines.append(f"EntitySummary: {name}{(' [' + '; '.join(meta) + ']') if meta else ''}") - if summary_lines: - contexts_all.append("\n".join(summary_lines)) - - if verbose: - if retrieved_counts: - print(f"✅ 检索成功: {retrieved_counts.get('chunks',0)} chunks, {retrieved_counts.get('statements',0)} 条陈述, {retrieved_counts.get('entities',0)} 个实体, {retrieved_counts.get('summaries',0)} 个摘要") - print(f"📊 有效上下文数量: {len(contexts_all)}") - q_keywords = extract_question_keywords(question, max_keywords=8) - if q_keywords: - print(f"🔍 问题关键词: {set(q_keywords)}") - if contexts_all: - analysis = analyze_contexts_simple(contexts_all, q_keywords, top_n=5) - if analysis: - print("📊 上下文相关性分析:") - for a in analysis: - print(f" - 得分: {int(a['score'])}, 关键词匹配: {a['match']}, 长度: {a['length']}") - # 打印检索到的上下文预览，便于定位为何为 Unknown - print("🔎 上下文预览（最多前10条，每条截断展示）:") - for i, ctx in enumerate(contexts_all[:10]): - preview = str(ctx).replace("\n", " ") - if len(preview) > 300: - preview = preview[:300] + "..." - print(f" [{i+1}] 长度: {len(ctx)} | 片段: {preview}") - # 标注参考答案是否出现在任一上下文中 - ref_lower = (str(reference) or "").lower() - if ref_lower: - hits = [] - for i, ctx in enumerate(contexts_all): - if ref_lower in str(ctx).lower(): - hits.append(i+1) - print(f"🔗 参考答案命中上下文条数: {len(hits)}" + (f" | 命中索引: {hits}" if hits else "")) - - context_text = smart_context_selection(contexts_all, question, max_chars=context_char_budget) if contexts_all else "" - if not context_text: - context_text = "No relevant context found." - contexts_used.append(context_text) - per_query_context_chars.append(len(context_text)) - per_query_context_counts.append(len(contexts_all)) - - if verbose: - selected_count = (context_text.count("\n\n") + 1) if context_text else 0 - print(f"✅ 智能选择: {selected_count}个上下文, 总长度: {len(context_text)}字符") - # 展示拼接后的上下文片段，便于核查是否包含答案 - concat_preview = context_text.replace("\n", " ") - if len(concat_preview) > 600: - concat_preview = concat_preview[:600] + "..." - print(f"🧵 拼接上下文预览: {concat_preview}") - - messages = [ - { - "role": "system", - "content": ( - "You are a QA assistant. Answer in English. Follow these guidelines:\n" - "1) If the context contains information to answer the question, provide a concise answer based on the context;\n" - "2) If the context does not contain enough information to answer the question, respond with 'Unknown';\n" - "3) Keep your answer brief and to the point;\n" - "4) Do not add explanations or additional text beyond the answer." - ), - }, - {"role": "user", "content": f"Question: {question}\n\nContext:\n{context_text}"}, - ] - - t2 = time.time() - try: - # 使用异步调用 - resp = await llm.chat(messages=messages) - # 更健壮的响应解析，处理不同的LLM响应格式 - if hasattr(resp, 'content'): - pred = resp.content.strip() - elif isinstance(resp, dict) and "choices" in resp and len(resp["choices"]) > 0: - pred = resp["choices"][0]["message"]["content"].strip() - elif isinstance(resp, dict) and "content" in resp: - pred = resp["content"].strip() - elif isinstance(resp, str): - pred = resp.strip() - else: - pred = "Unknown" - print(f"⚠️ LLM响应格式异常: {type(resp)} - {resp}") - - # 检查预测是否为"Unknown"或空，如果是则检查上下文是否真的没有答案 - if pred.lower() in ["unknown", ""]: - # 如果参考答案在上下文中存在，但LLM返回Unknown，可能是提示词问题 - ref_lower = (str(reference) or "").lower() - if ref_lower and any(ref_lower in ctx.lower() for ctx in contexts_all): - print("⚠️ 参考答案在上下文中存在但LLM返回Unknown，检查提示词") - except Exception as e: - # 更详细的错误处理 - pred = "Unknown" - print(f"⚠️ LLM调用异常: {e}") - t3 = time.time() - llm_ms = (t3 - t2) * 1000 - latencies_llm.append(llm_ms) - - exact = exact_match(pred, reference) - correct_flags.append(exact) - f1_val = f1_score(str(pred), str(reference)) - b1_val = bleu1(str(pred), str(reference)) - j_val = jaccard(str(pred), str(reference)) - f1s.append(f1_val) - b1s.append(b1_val) - jss.append(j_val) - - if verbose: - print(f"🤖 LLM 回答: {pred}") - print(f"✅ 正确答案: {reference}") - print(f"📈 当前指标 - F1: {f1_val:.3f}, BLEU-1: {b1_val:.3f}, Jaccard: {j_val:.3f}") - print(f"⏱️ 延迟 - 检索: {search_ms:.0f}ms, LLM: {llm_ms:.0f}ms") - - # 对齐 locomo/qwen_search_eval.py 的样本输出结构 - samples.append({ - "question": str(question), - "answer": str(reference), - "prediction": str(pred), - "metrics": { - "f1": f1_val, - "b1": b1_val, - "j": j_val - }, - "retrieval": { - "retrieved_documents": len(contexts_all), - "context_length": len(context_text), - "search_limit": search_limit, - "max_chars": context_char_budget - }, - "timing": { - "search_ms": search_ms, - "llm_ms": llm_ms - } - }) - - # 计算总体指标与聚合 - acc = sum(correct_flags) / max(len(correct_flags), 1) - ctx_avg_tokens = avg_context_tokens(contexts_used) - result = { - "dataset": "memsciqa", - "items": len(items), - "metrics": { - "f1": (sum(f1s) / max(len(f1s), 1)) if f1s else 0.0, - "b1": (sum(b1s) / max(len(b1s), 1)) if b1s else 0.0, - "j": (sum(jss) / max(len(jss), 1)) if jss else 0.0, - }, - "context": { - "avg_tokens": ctx_avg_tokens, - "avg_chars": (sum(per_query_context_chars) / max(len(per_query_context_chars), 1)) if per_query_context_chars else 0.0, - "count_avg": (sum(per_query_context_counts) / max(len(per_query_context_counts), 1)) if per_query_context_counts else 0.0, - "avg_memory_tokens": 0.0 - }, - "latency": { - "search": latency_stats(latencies_search), - "llm": latency_stats(latencies_llm), - }, - "samples": samples, - "params": { - "group_id": group_id, - "search_limit": search_limit, - "context_char_budget": context_char_budget, - "llm_temperature": llm_temperature, - "llm_max_tokens": llm_max_tokens, - "search_type": search_type, - "start_index": start_index, - "llm_id": SELECTED_LLM_ID, - "retrieval_embedding_id": SELECTED_EMBEDDING_ID - }, - "timestamp": datetime.now().isoformat(), - } - try: - await connector.close() - except Exception: - pass - return result - - -def main(): - load_dotenv() - parser = argparse.ArgumentParser(description="memsciqa 测试脚本（三路检索 + 智能上下文选择）") - parser.add_argument("--sample-size", type=int, default=30, help="样本数量（<=0 表示全部）") - parser.add_argument("--all", action="store_true", help="评估全部样本（覆盖 --sample-size）") - parser.add_argument("--start-index", type=int, default=0, help="起始样本索引") - parser.add_argument("--group-id", type=str, default="group_memsci", help="图数据库 Group ID（默认 group_memsci）") - parser.add_argument("--search-limit", type=int, default=8, help="检索条数上限") - parser.add_argument("--context-char-budget", type=int, default=4000, help="上下文字符预算") - parser.add_argument("--llm-temperature", type=float, default=0.0, help="LLM 温度") - parser.add_argument("--llm-max-tokens", type=int, default=64, help="LLM 最大输出 token") - parser.add_argument("--search-type", type=str, default="embedding", choices=["embedding","keyword","hybrid"], help="检索类型（hybrid 等同于 embedding）") - parser.add_argument("--data-path", type=str, default=None, help="数据集路径（默认 data/msc_self_instruct.jsonl）") - parser.add_argument("--output", type=str, default=None, help="将评估结果保存到指定文件路径（JSON）") - parser.add_argument("--verbose", action="store_true", default=True, help="打印过程日志（默认开启）") - parser.add_argument("--quiet", action="store_true", help="关闭过程日志") - args = parser.parse_args() - - sample_size = 0 if args.all else args.sample_size - - verbose_flag = False if args.quiet else args.verbose - result = asyncio.run( - run_memsciqa_test( - sample_size=sample_size, - group_id=args.group_id, - search_limit=args.search_limit, - context_char_budget=args.context_char_budget, - llm_temperature=args.llm_temperature, - llm_max_tokens=args.llm_max_tokens, - search_type=args.search_type, - data_path=args.data_path, - start_index=args.start_index, - verbose=verbose_flag, - ) - ) - - print(json.dumps(result, ensure_ascii=False, indent=2)) - - # 结果保存 - out_path = args.output - if not out_path: - eval_dir = os.path.dirname(os.path.abspath(__file__)) - dataset_results_dir = os.path.join(eval_dir, "results") - ts = datetime.now().strftime("%Y%m%d_%H%M%S") - out_path = os.path.join(dataset_results_dir, f"memsciqa_{result['params']['search_type']}_{ts}.json") - try: - os.makedirs(os.path.dirname(out_path), exist_ok=True) - with open(out_path, "w", encoding="utf-8") as f: - json.dump(result, f, ensure_ascii=False, indent=2) - print(f"\n💾 结果已保存: {out_path}") - except Exception as e: - print(f"⚠️ 结果保存失败: {e}") - - -if __name__ == "__main__": - main() diff --git a/api/app/core/memory/evaluation/run_eval.py b/api/app/core/memory/evaluation/run_eval.py deleted file mode 100644 index 1de3de89..00000000 --- a/api/app/core/memory/evaluation/run_eval.py +++ /dev/null @@ -1,150 +0,0 @@ -import argparse -import asyncio -import json -import os -import sys -from typing import Any, Dict - -# Add src directory to Python path for proper imports when running from evaluation directory -sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'src')) - -try: - from dotenv import load_dotenv -except Exception: - def load_dotenv(): - return None - -from app.repositories.neo4j.neo4j_connector import Neo4jConnector -from app.core.memory.utils.config.definitions import SELECTED_GROUP_ID, PROJECT_ROOT - -from app.core.memory.evaluation.memsciqa.evaluate_qa import run_memsciqa_eval -from app.core.memory.evaluation.longmemeval.qwen_search_eval import run_longmemeval_test -from app.core.memory.evaluation.locomo.qwen_search_eval import run_locomo_eval - - -async def run( - dataset: str, - sample_size: int, - reset_group: bool, - group_id: str | None, - judge_model: str | None = None, - search_limit: int | None = None, - context_char_budget: int | None = None, - llm_temperature: float | None = None, - llm_max_tokens: int | None = None, - search_type: str | None = None, - start_index: int | None = None, - max_contexts_per_item: int | None = None, -) -> Dict[str, Any]: - # 恢复原始风格：统一入口做路由，并沿用各数据集既有默认 - group_id = group_id or SELECTED_GROUP_ID - - if reset_group: - connector = Neo4jConnector() - try: - await connector.delete_group(group_id) - finally: - await connector.close() - - if dataset == "locomo": - kwargs: Dict[str, Any] = {"sample_size": sample_size, "group_id": group_id} - if search_limit is not None: - kwargs["search_limit"] = search_limit - if context_char_budget is not None: - kwargs["context_char_budget"] = context_char_budget - if llm_temperature is not None: - kwargs["llm_temperature"] = llm_temperature - if llm_max_tokens is not None: - kwargs["llm_max_tokens"] = llm_max_tokens - if search_type is not None: - kwargs["search_type"] = search_type - return await run_locomo_eval(**kwargs) - - if dataset == "memsciqa": - kwargs: Dict[str, Any] = {"sample_size": sample_size, "group_id": group_id} - if search_limit is not None: - kwargs["search_limit"] = search_limit - if context_char_budget is not None: - kwargs["context_char_budget"] = context_char_budget - if llm_temperature is not None: - kwargs["llm_temperature"] = llm_temperature - if llm_max_tokens is not None: - kwargs["llm_max_tokens"] = llm_max_tokens - if search_type is not None: - kwargs["search_type"] = search_type - return await run_memsciqa_eval(**kwargs) - - if dataset == "longmemeval": - kwargs: Dict[str, Any] = {"sample_size": sample_size, "group_id": group_id} - if search_limit is not None: - kwargs["search_limit"] = search_limit - if context_char_budget is not None: - kwargs["context_char_budget"] = context_char_budget - if llm_temperature is not None: - kwargs["llm_temperature"] = llm_temperature - if llm_max_tokens is not None: - kwargs["llm_max_tokens"] = llm_max_tokens - if search_type is not None: - kwargs["search_type"] = search_type - if start_index is not None: - kwargs["start_index"] = start_index - if max_contexts_per_item is not None: - kwargs["max_contexts_per_item"] = max_contexts_per_item - return await run_longmemeval_test(**kwargs) - raise ValueError(f"未知数据集: {dataset}") - - -def main(): - load_dotenv() - parser = argparse.ArgumentParser(description="统一评估入口：memsciqa / longmemeval / locomo") - parser.add_argument("--dataset", choices=["memsciqa", "longmemeval", "locomo"], required=True) - parser.add_argument("--sample-size", type=int, default=1, help="先用一条数据跑通") - parser.add_argument("--reset-group", action="store_true", help="运行前清空当前 group_id 的图数据") - parser.add_argument("--group-id", type=str, default=None, help="可选 group_id，默认取 runtime.json") - parser.add_argument("--judge-model", type=str, default=None, help="可选：longmemeval 判别式评测模型名") - parser.add_argument("--search-limit", type=int, default=None, help="检索返回的对话节点数量上限（不提供则使用各脚本默认）") - parser.add_argument("--context-char-budget", type=int, default=None, help="上下文字符预算（不提供则使用各脚本默认）") - parser.add_argument("--llm-temperature", type=float, default=None, help="生成温度（不提供则使用各脚本默认）") - parser.add_argument("--llm-max-tokens", type=int, default=None, help="最大生成 tokens（不提供则使用各脚本默认）") - parser.add_argument("--search-type", type=str, default=None, choices=["keyword", "embedding", "hybrid"], help="检索类型（不提供则使用各脚本默认）") - # 仅透传到 longmemeval；其他数据集忽略 - parser.add_argument("--start-index", type=int, default=None, help="仅 longmemeval：起始样本索引（不提供则用脚本默认）") - parser.add_argument("--max-contexts-per-item", type=int, default=None, help="仅 longmemeval：每条样本摄入的上下文数量上限（不提供则用脚本默认）") - parser.add_argument("--output", type=str, default=None, help="可选：将评估结果保存到指定文件路径（JSON）；不提供时默认保存到 evaluation//results 目录") - args = parser.parse_args() - - result = asyncio.run(run( - args.dataset, - args.sample_size, - args.reset_group, - args.group_id, - args.judge_model, - args.search_limit, - args.context_char_budget, - args.llm_temperature, - args.llm_max_tokens, - args.search_type, - args.start_index, - args.max_contexts_per_item, - )) - print(json.dumps(result, ensure_ascii=False, indent=2)) - - # 结果输出逻辑保持不变 - if args.output: - out_path = args.output - else: - eval_dir = os.path.dirname(os.path.abspath(__file__)) - dataset_results_dir = os.path.join(eval_dir, args.dataset, "results") - out_filename = f"{args.dataset}_{args.sample_size}.json" - out_path = os.path.join(dataset_results_dir, out_filename) - - out_dir = os.path.dirname(out_path) - if out_dir and not os.path.exists(out_dir): - os.makedirs(out_dir, exist_ok=True) - with open(out_path, "w", encoding="utf-8") as f: - json.dump(result, f, ensure_ascii=False, indent=2) - print(f"\n结果已保存到: {out_path}") - - -if __name__ == "__main__": - main() diff --git a/api/app/core/memory/llm_tools/chunker_client.py b/api/app/core/memory/llm_tools/chunker_client.py index 87cdb9f4..93a2df82 100644 --- a/api/app/core/memory/llm_tools/chunker_client.py +++ b/api/app/core/memory/llm_tools/chunker_client.py @@ -187,11 +187,11 @@ class ChunkerClient: async def generate_chunks(self, dialogue: DialogData): """ Generate chunks following 1 Message = 1 Chunk strategy. - + Each message creates one chunk, directly inheriting role information. If a message is too long, it will be split into multiple sub-chunks, each maintaining the same speaker. - + Raises: ValueError: If dialogue has no messages or chunking fails """ @@ -201,9 +201,9 @@ class ChunkerClient: f"Dialogue {dialogue.ref_id} has no messages. " f"Cannot generate chunks from empty dialogue." ) - + dialogue.chunks = [] - + # 按消息分块：每个消息创建一个或多个 chunk，直接继承角色 for msg_idx, msg in enumerate(dialogue.context.msgs): # Validate message has required attributes @@ -212,13 +212,13 @@ class ChunkerClient: f"Message {msg_idx} in dialogue {dialogue.ref_id} " f"missing 'role' or 'msg' attribute" ) - + msg_content = msg.msg.strip() - + # Skip empty messages if not msg_content: continue - + # 如果消息太长，可以进一步分块 if len(msg_content) > self.chunk_size: # 对单个消息的内容进行分块 @@ -228,14 +228,14 @@ class ChunkerClient: raise ValueError( f"Failed to chunk long message {msg_idx} in dialogue {dialogue.ref_id}: {e}" ) - + for idx, sub_chunk in enumerate(sub_chunks): sub_chunk_text = sub_chunk.text if hasattr(sub_chunk, 'text') else str(sub_chunk) sub_chunk_text = sub_chunk_text.strip() - + if len(sub_chunk_text) < (self.min_characters_per_chunk or 50): continue - + chunk = Chunk( content=f"{msg.role}: {sub_chunk_text}", speaker=msg.role, # 直接继承角色 @@ -260,7 +260,7 @@ class ChunkerClient: }, ) dialogue.chunks.append(chunk) - + # Validate we generated at least one chunk if not dialogue.chunks: raise ValueError( @@ -268,7 +268,7 @@ class ChunkerClient: f"All messages were either empty or too short. " f"Messages count: {len(dialogue.context.msgs)}" ) - + return dialogue def evaluate_chunking(self, dialogue: DialogData) -> dict: diff --git a/api/app/core/memory/models/config_models.py b/api/app/core/memory/models/config_models.py index f3341cc5..ca1780aa 100644 --- a/api/app/core/memory/models/config_models.py +++ b/api/app/core/memory/models/config_models.py @@ -72,7 +72,7 @@ class TemporalSearchParams(BaseModel): """Parameters for temporal search queries in the knowledge graph. Attributes: - group_id: Group ID to filter search results (default: 'test') + end_user_id: Group ID to filter search results (default: 'test') apply_id: Application ID to filter search results user_id: User ID to filter search results start_date: Start date for temporal filtering (format: 'YYYY-MM-DD') @@ -81,7 +81,7 @@ class TemporalSearchParams(BaseModel): invalid_date: Date when memory should be invalid (format: 'YYYY-MM-DD') limit: Maximum number of results to return (default: 3) """ - group_id: Optional[str] = Field("test", description="The group ID to filter the search.") + end_user_id: Optional[str] = Field("test", description="The group ID to filter the search.") apply_id: Optional[str] = Field(None, description="The apply ID to filter the search.") user_id: Optional[str] = Field(None, description="The user ID to filter the search.") start_date: Optional[str] = Field(None, description="The start date for the search.") diff --git a/api/app/core/memory/models/graph_models.py b/api/app/core/memory/models/graph_models.py index 7a48d6cb..79b88fdc 100644 --- a/api/app/core/memory/models/graph_models.py +++ b/api/app/core/memory/models/graph_models.py @@ -103,9 +103,7 @@ class Edge(BaseModel): id: Unique identifier for the edge source: ID of the source node target: ID of the target node - group_id: Group ID for multi-tenancy - user_id: User ID for user-specific data - apply_id: Application ID for application-specific data + end_user_id: End user ID for multi-tenancy run_id: Unique identifier for the pipeline run that created this edge created_at: Timestamp when the edge was created (system perspective) expired_at: Optional timestamp when the edge expires (system perspective) @@ -113,9 +111,7 @@ class Edge(BaseModel): id: str = Field(default_factory=lambda: uuid4().hex, description="A unique identifier for the edge.") source: str = Field(..., description="The ID of the source node.") target: str = Field(..., description="The ID of the target node.") - group_id: str = Field(..., description="The group ID of the edge.") - user_id: str = Field(..., description="The user ID of the edge.") - apply_id: str = Field(..., description="The apply ID of the edge.") + end_user_id: str = Field(..., description="The end user ID of the edge.") run_id: str = Field(default_factory=lambda: uuid4().hex, description="Unique identifier for this pipeline run.") created_at: datetime = Field(..., description="The valid time of the edge from system perspective.") expired_at: Optional[datetime] = Field(None, description="The expired time of the edge from system perspective.") @@ -185,18 +181,14 @@ class Node(BaseModel): Attributes: id: Unique identifier for the node name: Name of the node - group_id: Group ID for multi-tenancy - user_id: User ID for user-specific data - apply_id: Application ID for application-specific data + end_user_id: End user ID for multi-tenancy run_id: Unique identifier for the pipeline run that created this node created_at: Timestamp when the node was created (system perspective) expired_at: Optional timestamp when the node expires (system perspective) """ id: str = Field(..., description="The unique identifier for the node.") name: str = Field(..., description="The name of the node.") - group_id: str = Field(..., description="The group ID of the node.") - user_id: str = Field(..., description="The user ID of the edge.") - apply_id: str = Field(..., description="The apply ID of the edge.") + end_user_id: str = Field(..., description="The end user ID of the node.") run_id: str = Field(default_factory=lambda: uuid4().hex, description="Unique identifier for this pipeline run.") created_at: datetime = Field(..., description="The valid time of the node from system perspective.") expired_at: Optional[datetime] = Field(None, description="The expired time of the node from system perspective.") diff --git a/api/app/core/memory/models/message_models.py b/api/app/core/memory/models/message_models.py index bcf08999..2f8660af 100644 --- a/api/app/core/memory/models/message_models.py +++ b/api/app/core/memory/models/message_models.py @@ -55,7 +55,7 @@ class Statement(BaseModel): Attributes: id: Unique identifier for the statement chunk_id: ID of the parent chunk this statement belongs to - group_id: Optional group ID for multi-tenancy + end_user_id: Optional group ID for multi-tenancy statement: The actual statement text content speaker: Optional speaker identifier ('用户' for user, 'AI' for AI responses) statement_embedding: Optional embedding vector for the statement @@ -73,7 +73,7 @@ class Statement(BaseModel): """ id: str = Field(default_factory=lambda: uuid4().hex, description="A unique identifier for the statement.") chunk_id: str = Field(..., description="ID of the parent chunk this statement belongs to.") - group_id: Optional[str] = Field(None, description="ID of the group this statement belongs to.") + end_user_id: Optional[str] = Field(None, description="ID of the group this statement belongs to.") statement: str = Field(..., description="The text content of the statement.") speaker: Optional[str] = Field(None, description="Speaker identifier: 'user' for user messages, 'assistant' for AI responses") statement_embedding: Optional[List[float]] = Field(None, description="The embedding vector of the statement.") @@ -159,9 +159,7 @@ class DialogData(BaseModel): context: Full conversation context dialog_embedding: Optional embedding vector for the entire dialog ref_id: Reference ID linking to external dialog system - group_id: Group ID for multi-tenancy - user_id: User ID for user-specific data - apply_id: Application ID for application-specific data + end_user_id: End user ID for multi-tenancy created_at: Timestamp when the dialog was created expired_at: Timestamp when the dialog expires (default: far future) metadata: Additional metadata as key-value pairs @@ -175,9 +173,7 @@ class DialogData(BaseModel): context: ConversationContext = Field(..., description="The full conversation context as a single string.") dialog_embedding: Optional[List[float]] = Field(None, description="The embedding vector of the dialog.") ref_id: str = Field(..., description="Refer to external dialog id. This is used to link to the original dialog.") - group_id: str = Field(default=..., description="Group ID of dialogue data") - user_id: str = Field(..., description="USER ID of dialogue data") - apply_id: str = Field(..., description="APPLY ID of dialogue data") + end_user_id: str = Field(default=..., description="End user ID of dialogue data") run_id: str = Field(default_factory=lambda: uuid4().hex, description="Unique identifier for this pipeline run.") created_at: datetime = Field(default_factory=datetime.now, description="The timestamp when the dialog was created.") expired_at: datetime = Field(default_factory=lambda: datetime(9999, 12, 31), description="The timestamp when the dialog expires.") @@ -250,11 +246,11 @@ class DialogData(BaseModel): return [] def assign_group_id_to_statements(self) -> None: - """Assign this dialog's group_id to all statements in all chunks. + """Assign this dialog's end_user_id to all statements in all chunks. - This method updates statements that don't have a group_id set. + This method updates statements that don't have a end_user_id set. """ for chunk in self.chunks: for statement in chunk.statements: - if statement.group_id is None: - statement.group_id = self.group_id + if statement.end_user_id is None: + statement.end_user_id = self.end_user_id diff --git a/api/app/core/memory/src/search.py b/api/app/core/memory/src/search.py index 91e47eae..0e1d8424 100644 --- a/api/app/core/memory/src/search.py +++ b/api/app/core/memory/src/search.py @@ -6,6 +6,7 @@ import os import time from datetime import datetime from typing import TYPE_CHECKING, Any, Dict, List, Optional +from uuid import UUID if TYPE_CHECKING: from app.schemas.memory_config_schema import MemoryConfig @@ -396,13 +397,13 @@ def rerank_with_activation( return reranked -def log_search_query(query_text: str, search_type: str, group_id: str | None, limit: int, include: List[str], log_file: str = None): +def log_search_query(query_text: str, search_type: str, end_user_id: str | None, limit: int, include: List[str], log_file: str = None): """Log search query information using the logger. Args: query_text: The search query text search_type: Type of search (keyword, embedding, hybrid) - group_id: Group identifier for filtering + end_user_id: Group identifier for filtering limit: Maximum number of results include: List of result types to include log_file: Deprecated parameter, kept for backward compatibility @@ -413,7 +414,7 @@ def log_search_query(query_text: str, search_type: str, group_id: str | None, li # Log using the standard logger logger.info( f"Search query: query='{cleaned_query}', type={search_type}, " - f"group_id={group_id}, limit={limit}, include={include}" + f"end_user_id={end_user_id}, limit={limit}, include={include}" ) @@ -672,7 +673,7 @@ def apply_reranker_placeholder( async def run_hybrid_search( query_text: str, search_type: str, - group_id: str | None, + end_user_id: str | None, limit: int, include: List[str], output_path: str | None, @@ -715,7 +716,7 @@ async def run_hybrid_search( } # Log the search query - log_search_query(query_text, search_type, group_id, limit, include) + log_search_query(query_text, search_type, end_user_id, limit, include) connector = Neo4jConnector() results = {} @@ -732,7 +733,7 @@ async def run_hybrid_search( search_graph( connector=connector, q=query_text, - group_id=group_id, + end_user_id=end_user_id, limit=limit, include=include ) @@ -769,7 +770,7 @@ async def run_hybrid_search( connector=connector, embedder_client=embedder, query_text=query_text, - group_id=group_id, + end_user_id=end_user_id, limit=limit, include=include, ) @@ -916,9 +917,7 @@ async def run_hybrid_search( async def search_by_temporal( - group_id: Optional[str] = "test", - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = "test", start_date: Optional[str] = None, end_date: Optional[str] = None, valid_date: Optional[str] = None, @@ -929,7 +928,7 @@ async def search_by_temporal( Temporal search across Statements. - Matches statements created between start_date and end_date - - Optionally filters by group_id + - Optionally filters by end_user_id - Returns up to 'limit' statements """ connector = Neo4jConnector() @@ -939,9 +938,7 @@ async def search_by_temporal( end_date = normalize_date_safe(end_date) params = TemporalSearchParams.model_validate({ - "group_id": group_id, - "apply_id": apply_id, - "user_id": user_id, + "end_user_id": end_user_id, "start_date": start_date, "end_date": end_date, "valid_date": valid_date, @@ -950,9 +947,7 @@ async def search_by_temporal( }) statements = await search_graph_by_temporal( connector=connector, - group_id=params.group_id, - apply_id=params.apply_id, - user_id=params.user_id, + end_user_id=params.end_user_id, start_date=params.start_date, end_date=params.end_date, valid_date=params.valid_date, @@ -964,9 +959,7 @@ async def search_by_temporal( async def search_by_keyword_temporal( query_text: str, - group_id: Optional[str] = "test", - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = "test", start_date: Optional[str] = None, end_date: Optional[str] = None, valid_date: Optional[str] = None, @@ -987,9 +980,7 @@ async def search_by_keyword_temporal( invalid_date = normalize_date_safe(invalid_date) params = TemporalSearchParams.model_validate({ - "group_id": group_id, - "apply_id": apply_id, - "user_id": user_id, + "end_user_id": end_user_id, "start_date": start_date, "end_date": end_date, "valid_date": valid_date, @@ -999,9 +990,7 @@ async def search_by_keyword_temporal( statements = await search_graph_by_keyword_temporal( connector=connector, query_text=query_text, - group_id=params.group_id, - apply_id=params.apply_id, - user_id=params.user_id, + end_user_id=params.end_user_id, start_date=params.start_date, end_date=params.end_date, valid_date=params.valid_date, @@ -1013,7 +1002,7 @@ async def search_by_keyword_temporal( async def search_chunk_by_chunk_id( chunk_id: str, - group_id: Optional[str] = "test", + end_user_id: Optional[str] = "test", limit: int = 1, ): """ @@ -1023,7 +1012,7 @@ async def search_chunk_by_chunk_id( chunks = await search_graph_by_chunk_id( connector=connector, chunk_id=chunk_id, - group_id=group_id, + end_user_id=end_user_id, limit=limit ) return {"chunks": chunks} diff --git a/api/app/core/memory/storage_services/extraction_engine/data_preprocessing/data_preprocessor.py b/api/app/core/memory/storage_services/extraction_engine/data_preprocessing/data_preprocessor.py index f5e72517..4dafd3ed 100644 --- a/api/app/core/memory/storage_services/extraction_engine/data_preprocessing/data_preprocessor.py +++ b/api/app/core/memory/storage_services/extraction_engine/data_preprocessing/data_preprocessor.py @@ -555,8 +555,8 @@ class DataPreprocessor: dialog_id = item.get('dialog_id', item.get('ref_id', item.get('id', f'dialog_{i}'))) - # 获取group_id，如果不存在则生成默认值 - group_id = item.get('group_id', f'group_default_{i}') + # 获取end_user_id，如果不存在则生成默认值 + end_user_id = item.get('end_user_id', f'group_default_{i}') user_id = item.get('user_id', f'user_default_{i}') apply_id = item.get('apply_id', f'apply_default_{i}') @@ -574,7 +574,7 @@ class DataPreprocessor: dialog_data = DialogData( context=context, ref_id=dialog_id, - group_id=group_id, + end_user_id=end_user_id, user_id=user_id, apply_id=apply_id, metadata=metadata @@ -644,7 +644,7 @@ class DataPreprocessor: context = ConversationContext(msgs=messages) dialog_id = item.get('dialog_id', item.get('ref_id', item.get('id', f'dialog_{i}'))) - group_id = item.get('group_id', f'group_default_{i}') + end_user_id = item.get('end_user_id', f'group_default_{i}') user_id = item.get('user_id', f'user_default_{i}') apply_id = item.get('apply_id', f'apply_default_{i}') @@ -657,7 +657,7 @@ class DataPreprocessor: dialog_data = DialogData( context=context, ref_id=dialog_id, - group_id=group_id, + end_user_id=end_user_id, user_id=user_id, apply_id=apply_id, metadata=metadata diff --git a/api/app/core/memory/storage_services/extraction_engine/deduplication/deduped_and_disamb.py b/api/app/core/memory/storage_services/extraction_engine/deduplication/deduped_and_disamb.py index 62b656b0..a425e0ed 100644 --- a/api/app/core/memory/storage_services/extraction_engine/deduplication/deduped_and_disamb.py +++ b/api/app/core/memory/storage_services/extraction_engine/deduplication/deduped_and_disamb.py @@ -199,7 +199,7 @@ def accurate_match( entity_nodes: List[ExtractedEntityNode] ) -> Tuple[List[ExtractedEntityNode], Dict[str, str], Dict[str, Dict]]: """ - 精确匹配：按 (group_id, name, entity_type) 合并实体并建立重定向与合并记录。 + 精确匹配：按 (end_user_id, name, entity_type) 合并实体并建立重定向与合并记录。 返回: (deduped_entities, id_redirect, exact_merge_map) """ exact_merge_map: Dict[str, Dict] = {} @@ -210,8 +210,8 @@ def accurate_match( for ent in entity_nodes: name_norm = (getattr(ent, "name", "") or "").strip() type_norm = (getattr(ent, "entity_type", "") or "").strip() - key = f"{getattr(ent, 'group_id', None)}|{name_norm}|{type_norm}" - # 为避免跨业务组误并，明确以 group_id 为范围边界 + key = f"{getattr(ent, 'end_user_id', None)}|{name_norm}|{type_norm}" + # 为避免跨业务组误并，明确以 end_user_id 为范围边界 if key not in canonical_map: canonical_map[key] = ent id_redirect[ent.id] = ent.id @@ -223,11 +223,11 @@ def accurate_match( id_redirect[ent.id] = canonical.id # 记录精确匹配的合并项（使用规范化键，避免外层变量误用） try: - k = f"{canonical.group_id}|{(canonical.name or '').strip()}|{(canonical.entity_type or '').strip()}" + k = f"{canonical.end_user_id}|{(canonical.name or '').strip()}|{(canonical.entity_type or '').strip()}" if k not in exact_merge_map: exact_merge_map[k] = { "canonical_id": canonical.id, - "group_id": canonical.group_id, + "end_user_id": canonical.end_user_id, "name": canonical.name, "entity_type": canonical.entity_type, "merged_ids": set(), @@ -596,7 +596,7 @@ def fuzzy_match( b = deduped_entities[j] # 跳过不同业务组的实体 - if getattr(a, "group_id", None) != getattr(b, "group_id", None): + if getattr(a, "end_user_id", None) != getattr(b, "end_user_id", None): j += 1 continue @@ -671,7 +671,7 @@ def fuzzy_match( merge_reason = "[别名匹配]" if alias_match_merge else "[模糊]" merge_reason = "[别名匹配]" if alias_match_merge else "[模糊]" fuzzy_merge_records.append( - f"{merge_reason} 规范实体 {a.id} ({a.group_id}|{a.name}|{a.entity_type}) <- 合并实体 {b.id} ({b.group_id}|{b.name}|{b.entity_type}) | " + f"{merge_reason} 规范实体 {a.id} ({a.end_user_id}|{a.name}|{a.entity_type}) <- 合并实体 {b.id} ({b.end_user_id}|{b.name}|{b.entity_type}) | " f"s_name={s_name:.3f}, s_type={s_type:.3f}, overall={overall:.3f}, exact_alias={has_exact_match}" ) except Exception: @@ -779,7 +779,7 @@ async def LLM_decision( # 决策中包含去重和消歧的功能 # 记录 LLM 融合日志 try: llm_records.append( - f"[LLM融合] 规范实体 {a.id} ({a.group_id}|{a.name}|{a.entity_type}) <- 合并实体 {b.id} ({b.group_id}|{b.name}|{b.entity_type})" + f"[LLM融合] 规范实体 {a.id} ({a.end_user_id}|{a.name}|{a.entity_type}) <- 合并实体 {b.id} ({b.end_user_id}|{b.name}|{b.entity_type})" ) # 详细的“同类名称相似”记录改由 LLM 去重模块统一生成以携带 conf/reason except Exception: @@ -847,7 +847,7 @@ async def LLM_disamb_decision( id_redirect[k] = a.id try: disamb_records.append( - f"[DISAMB合并应用] 规范实体 {a.id} ({a.group_id}|{a.name}|{a.entity_type}) <- 合并实体 {b.id} ({b.group_id}|{b.name}|{b.entity_type})" + f"[DISAMB合并应用] 规范实体 {a.id} ({a.end_user_id}|{a.name}|{a.entity_type}) <- 合并实体 {b.id} ({b.end_user_id}|{b.name}|{b.entity_type})" ) except Exception: pass diff --git a/api/app/core/memory/storage_services/extraction_engine/deduplication/entity_dedup_llm.py b/api/app/core/memory/storage_services/extraction_engine/deduplication/entity_dedup_llm.py index 734f7b69..0249ac1f 100644 --- a/api/app/core/memory/storage_services/extraction_engine/deduplication/entity_dedup_llm.py +++ b/api/app/core/memory/storage_services/extraction_engine/deduplication/entity_dedup_llm.py @@ -174,7 +174,7 @@ async def _judge_pair( pass # 3. 构建LLM判断的“上下文信息”（规则层计算的所有特征） 判断上下文特征有助于实体消歧首先判断的类型关系 ctx = { - "same_group": getattr(a, "group_id", None) == getattr(b, "group_id", None), + "same_group": getattr(a, "end_user_id", None) == getattr(b, "end_user_id", None), "type_ok": _simple_type_ok(getattr(a, "entity_type", None), getattr(b, "entity_type", None)), "type_similarity": _type_similarity(getattr(a, "entity_type", None), getattr(b, "entity_type", None)), "name_text_sim": name_text_sim, @@ -235,7 +235,7 @@ async def _judge_pair_disamb( except Exception: pass ctx = { - "same_group": getattr(a, "group_id", None) == getattr(b, "group_id", None), + "same_group": getattr(a, "end_user_id", None) == getattr(b, "end_user_id", None), "type_ok": _simple_type_ok(getattr(a, "entity_type", None), getattr(b, "entity_type", None)), "name_text_sim": name_text_sim, "name_embed_sim": name_embed_sim, @@ -317,8 +317,8 @@ async def llm_dedup_entities( # 保留对偶判断作为子流程，是为了 a = entity_nodes[i] for j in range(i + 1, len(entity_nodes)): b = entity_nodes[j] - # 规则1：必须属于同一组（group_id相同，不同组的实体不重复） - if getattr(a, "group_id", None) != getattr(b, "group_id", None): + # 规则1：必须属于同一组（end_user_id相同，不同组的实体不重复） + if getattr(a, "end_user_id", None) != getattr(b, "end_user_id", None): continue # 规则2：类型必须兼容（调用_simple_type_ok判断） if not _simple_type_ok(getattr(a, "entity_type", None), getattr(b, "entity_type", None)): @@ -474,7 +474,7 @@ async def llm_dedup_entities_iterative_blocks( # 迭代分块并发 LLM 去重 - max_rounds: upper bound for iterative passes (default 3) - auto_merge_threshold: decision confidence for auto-merge when no co-occurrence (default 0.90) - co_ctx_threshold: lower threshold when co-occurrence is detected (default 0.83) - - shuffle_each_round: whether to shuffle entities within group_id each round to vary block composition + - shuffle_each_round: whether to shuffle entities within end_user_id each round to vary block composition Returns: - global_redirect: dict losing_id -> canonical_id accumulated across rounds @@ -509,7 +509,7 @@ async def llm_dedup_entities_iterative_blocks( # 迭代分块并发 LLM 去重 def _partition_blocks(nodes: List[ExtractedEntityNode]) -> List[List[ExtractedEntityNode]]: """ - 按 group_id 分块，避免跨组实体在同一块，减少无效候选对 + 按 end_user_id 分块，避免跨组实体在同一块，减少无效候选对 Args: nodes: 实体节点列表 @@ -519,7 +519,7 @@ async def llm_dedup_entities_iterative_blocks( # 迭代分块并发 LLM 去重 """ groups: Dict[str, List[ExtractedEntityNode]] = {} for e in nodes: - gid = getattr(e, "group_id", None) + gid = getattr(e, "end_user_id", None) groups.setdefault(str(gid), []).append(e) blocks: List[List[ExtractedEntityNode]] = [] for gid, arr in groups.items(): @@ -559,7 +559,7 @@ async def llm_dedup_entities_iterative_blocks( # 迭代分块并发 LLM 去重 # Collapse nodes to canonical reps before each round to avoid redundant comparisons # 步骤1：折叠实体（合并已确定的重复实体，减少后续计算量） current_nodes = _collapse_nodes(current_nodes) - # 步骤2：分块（按group_id分块，避免跨组处理） + # 步骤2：分块（按end_user_id分块，避免跨组处理） blocks = _partition_blocks(current_nodes) if not blocks: # 无块可处理（实体已全部折叠），退出循环 break @@ -645,7 +645,7 @@ async def llm_disambiguate_pairs_iterative( a = entity_nodes[i] b = entity_nodes[j] # 必须同组 - if getattr(a, "group_id", None) != getattr(b, "group_id", None): + if getattr(a, "end_user_id", None) != getattr(b, "end_user_id", None): continue ta = getattr(a, "entity_type", None) tb = getattr(b, "entity_type", None) diff --git a/api/app/core/memory/storage_services/extraction_engine/deduplication/second_layer_dedup.py b/api/app/core/memory/storage_services/extraction_engine/deduplication/second_layer_dedup.py index b41f35a4..dbc697d9 100644 --- a/api/app/core/memory/storage_services/extraction_engine/deduplication/second_layer_dedup.py +++ b/api/app/core/memory/storage_services/extraction_engine/deduplication/second_layer_dedup.py @@ -61,7 +61,7 @@ def _row_to_entity(row: Dict[str, Any]) -> ExtractedEntityNode: return ExtractedEntityNode( id=row.get("id"), name=row.get("name") or "", - group_id=row.get("group_id") or "", + end_user_id=row.get("end_user_id") or "", user_id=row.get("user_id") or "", apply_id=row.get("apply_id") or "", created_at=_parse_dt(row.get("created_at")), @@ -79,7 +79,7 @@ def _row_to_entity(row: Dict[str, Any]) -> ExtractedEntityNode: async def second_layer_dedup_and_merge_with_neo4j( # 二层去重的核心逻辑，与 Neo4j 中同组实体联合去重 connector: Neo4jConnector, - group_id: str, # 用于定位neo4j中同一组的实体，确保只在同组内去重 + end_user_id: str, # 用于定位neo4j中同一组的实体，确保只在同组内去重 entity_nodes: List[ExtractedEntityNode], # 输入的实体节点列表，包含待去重的实体 statement_entity_edges: List[StatementEntityEdge], # 输入的语句实体边列表，用于处理实体之间的关系 entity_entity_edges: List[EntityEntityEdge], # 输入的实体实体边列表，用于处理实体之间的关系 @@ -88,7 +88,7 @@ async def second_layer_dedup_and_merge_with_neo4j( # 二层去重的核心逻辑 ) -> Tuple[List[ExtractedEntityNode], List[StatementEntityEdge], List[EntityEntityEdge]]: """ 第二层去重消歧： - - 以第一层结果为索引，检索相同 group_id 下的 DB 候选实体 + - 以第一层结果为索引，检索相同 end_user_id 下的 DB 候选实体 - 将 DB 候选与当前实体集合联合，按既有精确/模糊/LLM 决策进行融合 - 返回融合后的实体与重定向后的边（边已指向规范 ID，优先 DB ID） """ @@ -102,7 +102,7 @@ async def second_layer_dedup_and_merge_with_neo4j( # 二层去重的核心逻辑 ] candidates_map = await get_dedup_candidates_for_entities( # 从 Neo4j 中查询候选实体，并将结果赋值给candidates_map（等待异步操作完成）。 - connector=connector, group_id=group_id, + connector=connector, end_user_id=end_user_id, entities=incoming_rows, # 传入参数：第一层实体的核心信息（作为查询索引） use_contains_fallback=True # 传入参数：启用 “包含关系” 作为匹配失败的降级策略（若精确匹配无结果，用包含关系召回候选），与src\database\cypher_queries.py的307产生联动 ) diff --git a/api/app/core/memory/storage_services/extraction_engine/deduplication/two_stage_dedup.py b/api/app/core/memory/storage_services/extraction_engine/deduplication/two_stage_dedup.py index 11845d7d..f28b8a5f 100644 --- a/api/app/core/memory/storage_services/extraction_engine/deduplication/two_stage_dedup.py +++ b/api/app/core/memory/storage_services/extraction_engine/deduplication/two_stage_dedup.py @@ -57,11 +57,11 @@ async def dedup_layers_and_merge_and_return( if pipeline_config is None: raise ValueError("pipeline_config is required for dedup_layers_and_merge_and_return") - # 先探测 group_id，决定报告写入策略 - group_id: Optional[str] = None + # 先探测 end_user_id，决定报告写入策略 + end_user_id: Optional[str] = None for dd in dialog_data_list: - group_id = getattr(dd, "group_id", None) - if group_id: + end_user_id = getattr(dd, "end_user_id", None) + if end_user_id: break # 第一层去重消歧 @@ -82,11 +82,11 @@ async def dedup_layers_and_merge_and_return( # 第二层去重消歧：与 Neo4j 中同组实体联合融合 try: - if group_id: + if end_user_id: if connector: fused_entity_nodes, fused_statement_entity_edges, fused_entity_entity_edges = await second_layer_dedup_and_merge_with_neo4j( connector=connector, - group_id=group_id, + end_user_id=end_user_id, entity_nodes=dedup_entity_nodes, statement_entity_edges=dedup_statement_entity_edges, entity_entity_edges=dedup_entity_entity_edges, @@ -96,7 +96,7 @@ async def dedup_layers_and_merge_and_return( else: print("Skip second-layer dedup: missing connector") else: - print("Skip second-layer dedup: missing group_id") + print("Skip second-layer dedup: missing end_user_id") except Exception as e: print(f"Second-layer dedup failed: {e}") diff --git a/api/app/core/memory/storage_services/extraction_engine/extraction_orchestrator.py b/api/app/core/memory/storage_services/extraction_engine/extraction_orchestrator.py index 46ba1dde..7b7e854b 100644 --- a/api/app/core/memory/storage_services/extraction_engine/extraction_orchestrator.py +++ b/api/app/core/memory/storage_services/extraction_engine/extraction_orchestrator.py @@ -287,7 +287,7 @@ class ExtractionOrchestrator: for d_idx, dialog in enumerate(dialog_data_list): dialogue_content = dialog.content if self.config.statement_extraction.include_dialogue_context else None for c_idx, chunk in enumerate(dialog.chunks): - all_chunks.append((chunk, dialog.group_id, dialogue_content)) + all_chunks.append((chunk, dialog.end_user_id, dialogue_content)) chunk_metadata.append((d_idx, c_idx)) logger.info(f"收集到 {len(all_chunks)} 个分块，开始全局并行提取") @@ -299,9 +299,9 @@ class ExtractionOrchestrator: # 全局并行处理所有分块 async def extract_for_chunk(chunk_data, chunk_index): nonlocal completed_chunks - chunk, group_id, dialogue_content = chunk_data + chunk, end_user_id, dialogue_content = chunk_data try: - statements = await self.statement_extractor._extract_statements(chunk, group_id, dialogue_content) + statements = await self.statement_extractor._extract_statements(chunk, end_user_id, dialogue_content) # 流式输出：每提取完一个分块的陈述句，立即发送进度 # 注意：只在试运行模式下发送陈述句详情，正式模式不发送 @@ -569,32 +569,32 @@ class ExtractionOrchestrator: if dialog_data_list and hasattr(dialog_data_list[0], 'config_id'): config_id = dialog_data_list[0].config_id - # 加载DataConfig - data_config = None + # 加载MemoryConfig + memory_config = None if config_id: try: from app.db import SessionLocal - from app.repositories.data_config_repository import DataConfigRepository + from app.repositories.memory_config_repository import MemoryConfigRepository db = SessionLocal() try: - data_config = DataConfigRepository.get_by_id(db, config_id) + memory_config = MemoryConfigRepository.get_by_id(db, config_id) finally: db.close() - if data_config and not data_config.emotion_enabled: + if memory_config and not memory_config.emotion_enabled: logger.info("情绪提取已在配置中禁用，跳过情绪提取") return [{} for _ in dialog_data_list] except Exception as e: - logger.warning(f"加载DataConfig失败: {e}，将跳过情绪提取") + logger.warning(f"加载MemoryConfig失败: {e}，将跳过情绪提取") return [{} for _ in dialog_data_list] else: logger.info("未找到config_id，跳过情绪提取") return [{} for _ in dialog_data_list] # 如果配置未启用情绪提取，直接返回空映射 - if not data_config or not data_config.emotion_enabled: + if not memory_config or not memory_config.emotion_enabled: logger.info("情绪提取未启用，跳过") return [{} for _ in dialog_data_list] @@ -608,7 +608,7 @@ class ExtractionOrchestrator: total_statements += 1 # 只处理用户的陈述句 (role 为 "user") if hasattr(statement, 'speaker') and statement.speaker == "user": - all_statements.append((statement, data_config)) + all_statements.append((statement, memory_config)) statement_metadata.append((d_idx, statement.id)) filtered_statements += 1 @@ -617,7 +617,7 @@ class ExtractionOrchestrator: # 初始化情绪提取服务 from app.services.emotion_extraction_service import EmotionExtractionService emotion_service = EmotionExtractionService( - llm_id=data_config.emotion_model_id if data_config.emotion_model_id else None + llm_id=memory_config.emotion_model_id if memory_config.emotion_model_id else None ) # 全局并行处理所有陈述句 @@ -992,9 +992,7 @@ class ExtractionOrchestrator: id=dialog_data.id, name=f"Dialog_{dialog_data.id}", # 添加必需的 name 字段 ref_id=dialog_data.ref_id, - group_id=dialog_data.group_id, - user_id=dialog_data.user_id, - apply_id=dialog_data.apply_id, + end_user_id=dialog_data.end_user_id, run_id=dialog_data.run_id, # 使用 dialog_data 的 run_id content=dialog_data.context.content if dialog_data.context else "", dialog_embedding=dialog_data.dialog_embedding if hasattr(dialog_data, 'dialog_embedding') else None, @@ -1012,9 +1010,7 @@ class ExtractionOrchestrator: id=chunk.id, name=f"Chunk_{chunk.id}", # 添加必需的 name 字段 dialog_id=dialog_data.id, - group_id=dialog_data.group_id, - user_id=dialog_data.user_id, - apply_id=dialog_data.apply_id, + end_user_id=dialog_data.end_user_id, run_id=dialog_data.run_id, # 使用 dialog_data 的 run_id content=chunk.content, chunk_embedding=chunk.chunk_embedding, @@ -1035,9 +1031,7 @@ class ExtractionOrchestrator: stmt_type=getattr(statement, 'stmt_type', 'general'), # 添加必需的 stmt_type 字段 temporal_info=getattr(statement, 'temporal_info', TemporalInfo.ATEMPORAL), # 添加必需的 temporal_info 字段 connect_strength=statement.connect_strength if statement.connect_strength is not None else 'Strong', # 添加必需的 connect_strength 字段 - group_id=dialog_data.group_id, - user_id=dialog_data.user_id, - apply_id=dialog_data.apply_id, + end_user_id=dialog_data.end_user_id, run_id=dialog_data.run_id, # 使用 dialog_data 的 run_id statement=statement.statement, speaker=getattr(statement, 'speaker', None), # 添加 speaker 字段 @@ -1060,9 +1054,7 @@ class ExtractionOrchestrator: statement_chunk_edge = StatementChunkEdge( source=statement.id, target=chunk.id, - group_id=dialog_data.group_id, - user_id=dialog_data.user_id, - apply_id=dialog_data.apply_id, + end_user_id=dialog_data.end_user_id, run_id=dialog_data.run_id, # 使用 dialog_data 的 run_id created_at=dialog_data.created_at, ) @@ -1072,13 +1064,16 @@ class ExtractionOrchestrator: if statement.triplet_extraction_info: triplet_info = statement.triplet_extraction_info - # 创建实体索引到ID的映射 + # 创建实体索引到ID的映射（支持多种索引方式） entity_idx_to_id = {} # 创建实体节点 for entity_idx, entity in enumerate(triplet_info.entities): - # 映射实体索引到实体ID + # 映射实体索引到实体ID（使用多个键以提高容错性） + # 1. 使用实体自己的 entity_idx entity_idx_to_id[entity.entity_idx] = entity.id + # 2. 使用枚举索引（从0开始） + entity_idx_to_id[entity_idx] = entity.id if entity.id not in entity_id_set: entity_connect_strength = getattr(entity, 'connect_strength', 'Strong') @@ -1095,9 +1090,7 @@ class ExtractionOrchestrator: aliases=getattr(entity, 'aliases', []) or [], # 传递从三元组提取阶段获取的aliases name_embedding=getattr(entity, 'name_embedding', None), is_explicit_memory=getattr(entity, 'is_explicit_memory', False), # 新增：传递语义记忆标记 - group_id=dialog_data.group_id, - user_id=dialog_data.user_id, - apply_id=dialog_data.apply_id, + end_user_id=dialog_data.end_user_id, run_id=dialog_data.run_id, # 使用 dialog_data 的 run_id created_at=dialog_data.created_at, expired_at=dialog_data.expired_at, @@ -1112,9 +1105,7 @@ class ExtractionOrchestrator: source=statement.id, target=entity.id, connect_strength=entity_connect_strength if entity_connect_strength is not None else 'Strong', - group_id=dialog_data.group_id, - user_id=dialog_data.user_id, - apply_id=dialog_data.apply_id, + end_user_id=dialog_data.end_user_id, run_id=dialog_data.run_id, # 使用 dialog_data 的 run_id created_at=dialog_data.created_at, ) @@ -1134,9 +1125,7 @@ class ExtractionOrchestrator: relation_type=triplet.predicate, statement=statement.statement, source_statement_id=statement.id, - group_id=dialog_data.group_id, - user_id=dialog_data.user_id, - apply_id=dialog_data.apply_id, + end_user_id=dialog_data.end_user_id, run_id=dialog_data.run_id, # 使用 dialog_data 的 run_id created_at=dialog_data.created_at, expired_at=dialog_data.expired_at, @@ -1163,9 +1152,18 @@ class ExtractionOrchestrator: relationship_result ) else: - logger.warning( - f"跳过三元组 - 无法找到实体ID: subject_id={triplet.subject_id}, " - f"object_id={triplet.object_id}, statement_id={statement.id}" + # 改进的警告信息，包含更多调试信息 + missing_subject = "subject" if not subject_entity_id else "" + missing_object = "object" if not object_entity_id else "" + missing_both = " and " if (not subject_entity_id and not object_entity_id) else "" + + logger.debug( + f"跳过三元组 - 无法找到{missing_subject}{missing_both}{missing_object}实体ID: " + f"subject_id={triplet.subject_id} ({triplet.subject_name}), " + f"object_id={triplet.object_id} ({triplet.object_name}), " + f"predicate={triplet.predicate}, " + f"statement_id={statement.id}, " + f"available_indices={sorted(entity_idx_to_id.keys())}" ) logger.info( @@ -1763,14 +1761,14 @@ class ExtractionOrchestrator: async def get_chunked_dialogs( chunker_strategy: str = "RecursiveChunker", - group_id: str = "group_1", + end_user_id: str = "group_1", indices: Optional[List[int]] = None, ) -> List[DialogData]: """从测试数据生成分块对话 Args: chunker_strategy: 分块策略（默认: RecursiveChunker） - group_id: 组ID + end_user_id: 组ID indices: 要处理的数据索引列表（可选） Returns: @@ -1834,7 +1832,7 @@ async def get_chunked_dialogs( dialog_data = DialogData( context=conversation_context, ref_id=data['id'], - group_id=group_id, + end_user_id=end_user_id, metadata=dialog_metadata, ) @@ -1936,7 +1934,7 @@ async def get_chunked_dialogs_from_preprocessed( async def get_chunked_dialogs_with_preprocessing( chunker_strategy: str = "RecursiveChunker", - group_id: str = "default", + end_user_id: str = "default", user_id: str = "default", apply_id: str = "default", indices: Optional[List[int]] = None, @@ -1948,7 +1946,7 @@ async def get_chunked_dialogs_with_preprocessing( Args: chunker_strategy: 分块策略 - group_id: 组ID + end_user_id: 组ID user_id: 用户ID apply_id: 应用ID indices: 要处理的数据索引列表 @@ -1976,11 +1974,9 @@ async def get_chunked_dialogs_with_preprocessing( indices=indices, ) - # 设置 group_id, user_id, apply_id + # 设置 end_user_id for dd in preprocessed_data: - dd.group_id = group_id - dd.user_id = user_id - dd.apply_id = apply_id + dd.end_user_id = end_user_id # 步骤2: 语义剪枝 try: diff --git a/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/memory_summary.py b/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/memory_summary.py index 7e75fd2d..f39313a8 100644 --- a/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/memory_summary.py +++ b/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/memory_summary.py @@ -193,9 +193,9 @@ async def _process_chunk_summary( node = MemorySummaryNode( id=uuid4().hex, name=title if title else f"MemorySummaryChunk_{chunk.id}", - group_id=dialog.group_id, - user_id=dialog.user_id, - apply_id=dialog.apply_id, + end_user_id=dialog.end_user_id, + user_id=dialog.end_user_id, + apply_id=dialog.end_user_id, run_id=dialog.run_id, # 使用 dialog 的 run_id created_at=datetime.now(), expired_at=datetime(9999, 12, 31), diff --git a/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/statement_extraction.py b/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/statement_extraction.py index fb1b539a..b06bd70f 100644 --- a/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/statement_extraction.py +++ b/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/statement_extraction.py @@ -82,12 +82,12 @@ class StatementExtractor: logger.warning(f"Chunk {getattr(chunk, 'id', 'unknown')} has no speaker field or is empty") return None - async def _extract_statements(self, chunk, group_id: Optional[str] = None, dialogue_content: str = None) -> List[Statement]: + async def _extract_statements(self, chunk, end_user_id: Optional[str] = None, dialogue_content: str = None) -> List[Statement]: """Process a single chunk and return extracted statements Args: chunk: Chunk object to process - group_id: Group ID to assign to all statements in this chunk + end_user_id: Group ID to assign to all statements in this chunk dialogue_content: Full dialogue content to provide as context Returns: @@ -158,7 +158,7 @@ class StatementExtractor: temporal_info=temporal_type, relevence_info=relevence_info, chunk_id=chunk.id, - group_id=group_id, + end_user_id=end_user_id, speaker=chunk_speaker, ) @@ -184,10 +184,10 @@ class StatementExtractor: logger.info(f"Processing {len(chunks_to_process)} chunks for statement extraction") - # Process all chunks concurrently, passing the group_id and dialogue content from dialog_data + # Process all chunks concurrently, passing the end_user_id and dialogue content from dialog_data dialogue_content = dialog_data.content if self.config.include_dialogue_context else None results = await asyncio.gather( - *[self._extract_statements(chunk, dialog_data.group_id, dialogue_content) for chunk in chunks_to_process], + *[self._extract_statements(chunk, dialog_data.end_user_id, dialogue_content) for chunk in chunks_to_process], return_exceptions=True ) @@ -225,7 +225,7 @@ class StatementExtractor: for i, statement in enumerate(statements, 1): f.write(f"Statement {i}:\n") f.write(f"Id: {statement.id}\n") - f.write(f"Group Id: {statement.group_id}\n") + f.write(f"Group Id: {statement.end_user_id}\n") f.write(f"Content: {statement.statement}\n") f.write(f"Type: {statement.stmt_type.value}\n") f.write(f"Temporal Info: {statement.temporal_info.value}\n") @@ -298,7 +298,7 @@ class StatementExtractor: dialog_sections.append({ "dialog_id": dialog.ref_id, - "group_id": dialog.group_id, + "end_user_id": dialog.end_user_id, "content": dialog.content if getattr(dialog, "content", None) else "", "strong": strong_relations, "weak": weak_relations, @@ -312,7 +312,7 @@ class StatementExtractor: for idx, section in enumerate(dialog_sections, 1): f.write(f"Dialog {idx}:\n") f.write(f"Dialog ID: {section.get('dialog_id', '')}\n") - f.write(f"Group ID: {section.get('group_id', '')}\n") + f.write(f"Group ID: {section.get('end_user_id', '')}\n") f.write("Content:\n") f.write(f"{section.get('content', '')}\n") f.write("-" * 40 + "\n\n") diff --git a/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/temporal_extraction.py b/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/temporal_extraction.py index 9528e638..499027a4 100644 --- a/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/temporal_extraction.py +++ b/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/temporal_extraction.py @@ -132,7 +132,7 @@ class TemporalExtractor: prompt_logger.info("") prompt_logger.info("=== TEMPORAL EXTRACTION RESULTS ===") prompt_logger.info( - f"[Temporal] Dialog ref_id={getattr(dialog_data, 'ref_id', None)}, group_id={getattr(dialog_data, 'group_id', None)}" + f"[Temporal] Dialog ref_id={getattr(dialog_data, 'ref_id', None)}, end_user_id={getattr(dialog_data, 'end_user_id', None)}" ) except Exception: pass diff --git a/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/triplet_extraction.py b/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/triplet_extraction.py index d3d059b0..bfc0bc88 100644 --- a/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/triplet_extraction.py +++ b/api/app/core/memory/storage_services/extraction_engine/knowledge_extraction/triplet_extraction.py @@ -116,7 +116,7 @@ class TripletExtractor: logger.info(f"Processing {len(all_statements)} statements for triplet extraction...") try: prompt_logger.info( - f"[Triplet] Dialog ref_id={getattr(dialog_data, 'ref_id', None)}, group_id={getattr(dialog_data, 'group_id', None)}, statements_to_process={len(all_statements)}" + f"[Triplet] Dialog ref_id={getattr(dialog_data, 'ref_id', None)}, end_user_id={getattr(dialog_data, 'end_user_id', None)}, statements_to_process={len(all_statements)}" ) except Exception: pass diff --git a/api/app/core/memory/storage_services/forgetting_engine/access_history_manager.py b/api/app/core/memory/storage_services/forgetting_engine/access_history_manager.py index 5722769a..a71c0957 100644 --- a/api/app/core/memory/storage_services/forgetting_engine/access_history_manager.py +++ b/api/app/core/memory/storage_services/forgetting_engine/access_history_manager.py @@ -75,7 +75,7 @@ class AccessHistoryManager: self, node_id: str, node_label: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, current_time: Optional[datetime] = None ) -> Dict[str, Any]: """ @@ -91,7 +91,7 @@ class AccessHistoryManager: Args: node_id: 节点ID node_label: 节点标签（Statement, ExtractedEntity, MemorySummary） - group_id: 组ID（可选，用于过滤） + end_user_id: 组ID（可选，用于过滤） current_time: 当前时间（可选，默认使用系统时间） Returns: @@ -123,7 +123,7 @@ class AccessHistoryManager: for attempt in range(self.max_retries): try: # 步骤1：读取当前节点状态 - node_data = await self._fetch_node(node_id, node_label, group_id) + node_data = await self._fetch_node(node_id, node_label, end_user_id) if not node_data: raise ValueError( @@ -142,7 +142,7 @@ class AccessHistoryManager: node_id=node_id, node_label=node_label, update_data=update_data, - group_id=group_id + end_user_id=end_user_id ) logger.info( @@ -172,7 +172,7 @@ class AccessHistoryManager: self, node_ids: List[str], node_label: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, current_time: Optional[datetime] = None ) -> List[Dict[str, Any]]: """ @@ -184,7 +184,7 @@ class AccessHistoryManager: Args: node_ids: 节点ID列表 node_label: 节点标签（所有节点必须是同一类型） - group_id: 组ID（可选） + end_user_id: 组ID（可选） current_time: 当前时间（可选） Returns: @@ -202,7 +202,7 @@ class AccessHistoryManager: task = self.record_access( node_id=node_id, node_label=node_label, - group_id=group_id, + end_user_id=end_user_id, current_time=current_time ) tasks.append(task) @@ -235,7 +235,7 @@ class AccessHistoryManager: self, node_id: str, node_label: str, - group_id: Optional[str] = None + end_user_id: Optional[str] = None ) -> Tuple[ConsistencyCheckResult, Optional[str]]: """ 检查节点数据的一致性 @@ -249,14 +249,14 @@ class AccessHistoryManager: Args: node_id: 节点ID node_label: 节点标签 - group_id: 组ID（可选） + end_user_id: 组ID（可选） Returns: Tuple[ConsistencyCheckResult, Optional[str]]: - 一致性检查结果枚举 - 错误描述（如果不一致） """ - node_data = await self._fetch_node(node_id, node_label, group_id) + node_data = await self._fetch_node(node_id, node_label, end_user_id) if not node_data: return ConsistencyCheckResult.CONSISTENT, None @@ -305,7 +305,7 @@ class AccessHistoryManager: async def check_batch_consistency( self, node_label: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, limit: int = 1000 ) -> Dict[str, Any]: """ @@ -313,7 +313,7 @@ class AccessHistoryManager: Args: node_label: 节点标签 - group_id: 组ID（可选） + end_user_id: 组ID（可选） limit: 检查的最大节点数 Returns: @@ -329,16 +329,16 @@ class AccessHistoryManager: MATCH (n:{node_label}) WHERE n.access_history IS NOT NULL """ - if group_id: - query += " AND n.group_id = $group_id" + if end_user_id: + query += " AND n.end_user_id = $end_user_id" query += """ RETURN n.id as id LIMIT $limit """ params = {"limit": limit} - if group_id: - params["group_id"] = group_id + if end_user_id: + params["end_user_id"] = end_user_id results = await self.connector.execute_query(query, **params) node_ids = [r['id'] for r in results] @@ -351,7 +351,7 @@ class AccessHistoryManager: result, message = await self.check_consistency( node_id=node_id, node_label=node_label, - group_id=group_id + end_user_id=end_user_id ) if result == ConsistencyCheckResult.CONSISTENT: @@ -387,7 +387,7 @@ class AccessHistoryManager: self, node_id: str, node_label: str, - group_id: Optional[str] = None + end_user_id: Optional[str] = None ) -> bool: """ 自动修复节点的数据不一致问题 @@ -401,7 +401,7 @@ class AccessHistoryManager: Args: node_id: 节点ID node_label: 节点标签 - group_id: 组ID（可选） + end_user_id: 组ID（可选） Returns: bool: 修复成功返回True，否则返回False @@ -411,7 +411,7 @@ class AccessHistoryManager: result, message = await self.check_consistency( node_id=node_id, node_label=node_label, - group_id=group_id + end_user_id=end_user_id ) if result == ConsistencyCheckResult.CONSISTENT: @@ -419,7 +419,7 @@ class AccessHistoryManager: return True # 获取节点数据 - node_data = await self._fetch_node(node_id, node_label, group_id) + node_data = await self._fetch_node(node_id, node_label, end_user_id) if not node_data: logger.error(f"节点不存在，无法修复: {node_label}[{node_id}]") return False @@ -457,8 +457,8 @@ class AccessHistoryManager: query = f""" MATCH (n:{node_label} {{id: $node_id}}) """ - if group_id: - query += " WHERE n.group_id = $group_id" + if end_user_id: + query += " WHERE n.end_user_id = $end_user_id" query += """ SET n += $repair_data RETURN n @@ -468,8 +468,8 @@ class AccessHistoryManager: 'node_id': node_id, 'repair_data': repair_data } - if group_id: - params['group_id'] = group_id + if end_user_id: + params['end_user_id'] = end_user_id await self.connector.execute_query(query, **params) @@ -491,7 +491,7 @@ class AccessHistoryManager: self, node_id: str, node_label: str, - group_id: Optional[str] = None + end_user_id: Optional[str] = None ) -> Optional[Dict[str, Any]]: """ 获取节点数据 @@ -499,7 +499,7 @@ class AccessHistoryManager: Args: node_id: 节点ID node_label: 节点标签 - group_id: 组ID（可选） + end_user_id: 组ID（可选） Returns: Optional[Dict[str, Any]]: 节点数据，如果不存在返回None @@ -507,8 +507,8 @@ class AccessHistoryManager: query = f""" MATCH (n:{node_label} {{id: $node_id}}) """ - if group_id: - query += " WHERE n.group_id = $group_id" + if end_user_id: + query += " WHERE n.end_user_id = $end_user_id" query += """ RETURN n.id as id, n.importance_score as importance_score, @@ -519,8 +519,8 @@ class AccessHistoryManager: """ params = {'node_id': node_id} - if group_id: - params['group_id'] = group_id + if end_user_id: + params['end_user_id'] = end_user_id results = await self.connector.execute_query(query, **params) @@ -585,7 +585,7 @@ class AccessHistoryManager: node_id: str, node_label: str, update_data: Dict[str, Any], - group_id: Optional[str] = None + end_user_id: Optional[str] = None ) -> Dict[str, Any]: """ 原子性更新节点（使用乐观锁） @@ -597,7 +597,7 @@ class AccessHistoryManager: node_id: 节点ID node_label: 节点标签 update_data: 更新数据 - group_id: 组ID（可选） + end_user_id: 组ID（可选） Returns: Dict[str, Any]: 更新后的节点数据 @@ -606,13 +606,13 @@ class AccessHistoryManager: RuntimeError: 如果更新失败或发生版本冲突 """ # 定义事务函数 - async def update_transaction(tx, node_id, node_label, update_data, group_id): + async def update_transaction(tx, node_id, node_label, update_data, end_user_id): # 步骤1：读取当前节点并获取版本号 read_query = f""" MATCH (n:{node_label} {{id: $node_id}}) """ - if group_id: - read_query += " WHERE n.group_id = $group_id" + if end_user_id: + read_query += " WHERE n.end_user_id = $end_user_id" read_query += """ RETURN n.id as id, n.version as version, @@ -624,8 +624,8 @@ class AccessHistoryManager: """ read_params = {'node_id': node_id} - if group_id: - read_params['group_id'] = group_id + if end_user_id: + read_params['end_user_id'] = end_user_id read_result = await tx.run(read_query, **read_params) current_node = await read_result.single() @@ -656,8 +656,8 @@ class AccessHistoryManager: # 构建 WHERE 子句 where_conditions = [] - if group_id: - where_conditions.append("n.group_id = $group_id") + if end_user_id: + where_conditions.append("n.end_user_id = $end_user_id") # 添加版本检查 if current_version > 0: @@ -695,8 +695,8 @@ class AccessHistoryManager: 'last_access_time': update_data['last_access_time'], 'access_count': update_data['access_count'] } - if group_id: - update_params['group_id'] = group_id + if end_user_id: + update_params['end_user_id'] = end_user_id update_result = await tx.run(update_query, **update_params) updated_node = await update_result.single() @@ -720,7 +720,7 @@ class AccessHistoryManager: node_id=node_id, node_label=node_label, update_data=update_data, - group_id=group_id + end_user_id=end_user_id ) return result except Exception as e: diff --git a/api/app/core/memory/storage_services/forgetting_engine/config_utils.py b/api/app/core/memory/storage_services/forgetting_engine/config_utils.py index ea9a6358..25daa968 100644 --- a/api/app/core/memory/storage_services/forgetting_engine/config_utils.py +++ b/api/app/core/memory/storage_services/forgetting_engine/config_utils.py @@ -11,9 +11,10 @@ Functions: import logging from typing import Optional, Dict, Any +from uuid import UUID from sqlalchemy.orm import Session -from app.repositories.data_config_repository import DataConfigRepository +from app.repositories.memory_config_repository import MemoryConfigRepository from app.core.memory.storage_services.forgetting_engine.actr_calculator import ACTRCalculator @@ -61,12 +62,12 @@ def calculate_forgetting_rate(lambda_time: float, lambda_mem: float) -> float: def load_actr_config_from_db( db: Session, - config_id: Optional[int] = None + config_id: Optional[UUID] = None ) -> Dict[str, Any]: """ 从数据库加载 ACT-R 配置参数 - 从 PostgreSQL 的 data_config 表读取配置参数， + 从 PostgreSQL 的 memory_config 表读取配置参数， 并计算派生参数（如 forgetting_rate）。 Args: @@ -99,7 +100,7 @@ def load_actr_config_from_db( # 从数据库加载配置 try: - repository = DataConfigRepository() + repository = MemoryConfigRepository() db_config = repository.get_by_id(db, config_id) if db_config is None: @@ -150,7 +151,7 @@ def load_actr_config_from_db( def create_actr_calculator_from_config( db: Session, - config_id: Optional[int] = None + config_id: Optional[UUID] = None ) -> ACTRCalculator: """ 从数据库配置创建 ACTRCalculator 实例 @@ -168,11 +169,6 @@ def create_actr_calculator_from_config( ValueError: 如果指定的 config_id 不存在 Examples: - >>> from sqlalchemy.orm import Session - >>> db = Session() - >>> calculator = create_actr_calculator_from_config(db, config_id=1) - >>> # 使用计算器 - >>> activation = calculator.calculate_memory_activation(...) """ # 加载配置 config = load_actr_config_from_db(db, config_id) diff --git a/api/app/core/memory/storage_services/forgetting_engine/forgetting_scheduler.py b/api/app/core/memory/storage_services/forgetting_engine/forgetting_scheduler.py index 6d42af53..072d587c 100644 --- a/api/app/core/memory/storage_services/forgetting_engine/forgetting_scheduler.py +++ b/api/app/core/memory/storage_services/forgetting_engine/forgetting_scheduler.py @@ -16,6 +16,7 @@ Classes: import logging from typing import Dict, Any, Optional +from uuid import UUID from datetime import datetime from app.core.memory.storage_services.forgetting_engine.forgetting_strategy import ForgettingStrategy @@ -66,10 +67,10 @@ class ForgettingScheduler: async def run_forgetting_cycle( self, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, max_merge_batch_size: int = 100, min_days_since_access: int = 30, - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, db = None ) -> Dict[str, Any]: """ @@ -77,7 +78,7 @@ class ForgettingScheduler: Args: - group_id: 组 ID（可选，用于过滤特定组的节点） + end_user_id: 组 ID（可选，用于过滤特定组的节点） max_merge_batch_size: 单次最大融合节点对数（默认 100） min_days_since_access: 最小未访问天数（默认 30 天） config_id: 配置ID（可选，用于获取 llm_id） @@ -107,19 +108,19 @@ class ForgettingScheduler: start_time_iso = start_time.isoformat() logger.info( - f"开始遗忘周期: group_id={group_id}, " + f"开始遗忘周期: end_user_id={end_user_id}, " f"max_batch={max_merge_batch_size}, " f"min_days={min_days_since_access}" ) try: # 步骤1：统计遗忘前的节点数量 - nodes_before = await self._count_knowledge_nodes(group_id) + nodes_before = await self._count_knowledge_nodes(end_user_id) logger.info(f"遗忘前节点总数: {nodes_before}") # 步骤2：识别可遗忘的节点对 forgettable_pairs = await self.forgetting_strategy.find_forgettable_nodes( - group_id=group_id, + end_user_id=end_user_id, min_days_since_access=min_days_since_access ) @@ -213,7 +214,7 @@ class ForgettingScheduler: 'statement_text': pair['statement_text'], 'statement_activation': pair['statement_activation'], 'statement_importance': pair['statement_importance'], - 'group_id': group_id + 'end_user_id': end_user_id } entity_node = { @@ -222,7 +223,7 @@ class ForgettingScheduler: 'entity_type': pair['entity_type'], 'entity_activation': pair['entity_activation'], 'entity_importance': pair['entity_importance'], - 'group_id': group_id + 'end_user_id': end_user_id } # 融合节点 @@ -262,7 +263,7 @@ class ForgettingScheduler: continue # 步骤6：统计遗忘后的节点数量 - nodes_after = await self._count_knowledge_nodes(group_id) + nodes_after = await self._count_knowledge_nodes(end_user_id) logger.info(f"遗忘后节点总数: {nodes_after}") # 步骤7：生成遗忘报告 @@ -315,7 +316,7 @@ class ForgettingScheduler: async def _count_knowledge_nodes( self, - group_id: Optional[str] = None + end_user_id: Optional[str] = None ) -> int: """ 统计知识层节点总数 @@ -323,7 +324,7 @@ class ForgettingScheduler: 统计 Statement、ExtractedEntity 和 MemorySummary 节点的总数。 Args: - group_id: 组 ID（可选，用于过滤特定组的节点） + end_user_id: 组 ID（可选，用于过滤特定组的节点） Returns: int: 知识层节点总数 @@ -333,16 +334,16 @@ class ForgettingScheduler: WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary) """ - if group_id: - query += " AND n.group_id = $group_id" + if end_user_id: + query += " AND n.end_user_id = $end_user_id" query += """ RETURN count(n) as total """ params = {} - if group_id: - params['group_id'] = group_id + if end_user_id: + params['end_user_id'] = end_user_id results = await self.connector.execute_query(query, **params) diff --git a/api/app/core/memory/storage_services/forgetting_engine/forgetting_strategy.py b/api/app/core/memory/storage_services/forgetting_engine/forgetting_strategy.py index ccd8d2ca..a8c62dd4 100644 --- a/api/app/core/memory/storage_services/forgetting_engine/forgetting_strategy.py +++ b/api/app/core/memory/storage_services/forgetting_engine/forgetting_strategy.py @@ -13,6 +13,7 @@ Classes: import logging from typing import List, Dict, Any, Optional +from uuid import UUID from datetime import datetime, timedelta from app.repositories.neo4j.neo4j_connector import Neo4jConnector @@ -90,7 +91,7 @@ class ForgettingStrategy: async def find_forgettable_nodes( self, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, min_days_since_access: int = 30 ) -> List[Dict[str, Any]]: """ @@ -102,7 +103,7 @@ class ForgettingStrategy: 3. Statement 和 Entity 之间存在关系边 Args: - group_id: 组 ID（可选，用于过滤特定组的节点） + end_user_id: 组 ID（可选，用于过滤特定组的节点） min_days_since_access: 最小未访问天数（默认 30 天） Returns: @@ -136,8 +137,8 @@ class ForgettingStrategy: AND (e.entity_type IS NULL OR e.entity_type <> 'Person') """ - if group_id: - query += " AND s.group_id = $group_id AND e.group_id = $group_id" + if end_user_id: + query += " AND s.end_user_id = $end_user_id AND e.end_user_id = $end_user_id" query += """ RETURN s.id as statement_id, @@ -159,8 +160,8 @@ class ForgettingStrategy: 'threshold': self.forgetting_threshold, 'cutoff_time': cutoff_time_iso } - if group_id: - params['group_id'] = group_id + if end_user_id: + params['end_user_id'] = end_user_id results = await self.connector.execute_query(query, **params) @@ -176,7 +177,7 @@ class ForgettingStrategy: self, statement_node: Dict[str, Any], entity_node: Dict[str, Any], - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, db = None ) -> str: """ @@ -247,8 +248,8 @@ class ForgettingStrategy: entity_activation = entity_node['entity_activation'] entity_importance = entity_node['entity_importance'] - # 获取 group_id（从 statement 或 entity 节点） - group_id = statement_node.get('group_id') or entity_node.get('group_id') + # 获取 end_user_id（从 statement 或 entity 节点） + end_user_id = statement_node.get('end_user_id') or entity_node.get('end_user_id') # 生成摘要内容 summary_text = await self._generate_summary( @@ -325,7 +326,7 @@ class ForgettingStrategy: last_access_time: $current_time, access_count: 1, version: 1, - group_id: $group_id, + end_user_id: $end_user_id, created_at: datetime($current_time), merged_at: datetime($current_time) }) @@ -423,7 +424,7 @@ class ForgettingStrategy: 'inherited_activation': inherited_activation, 'inherited_importance': inherited_importance, 'current_time': current_time_iso, - 'group_id': group_id + 'end_user_id': end_user_id } try: @@ -462,7 +463,7 @@ class ForgettingStrategy: statement_text: str, entity_name: str, entity_type: str, - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, db = None ) -> str: """ @@ -527,7 +528,7 @@ class ForgettingStrategy: statement_text, entity_name, entity_type ) - async def _get_llm_client(self, db, config_id: int): + async def _get_llm_client(self, db, config_id: UUID): """ 从数据库获取 LLM 客户端 @@ -539,11 +540,11 @@ class ForgettingStrategy: LLM 客户端实例，如果无法获取则返回 None """ try: - from app.repositories.data_config_repository import DataConfigRepository + from app.repositories.memory_config_repository import MemoryConfigRepository from app.core.memory.utils.llm.llm_utils import MemoryClientFactory # 从数据库读取配置 - repository = DataConfigRepository() + repository = MemoryConfigRepository() db_config = repository.get_by_id(db, config_id) if db_config is None or db_config.llm_id is None: diff --git a/api/app/core/memory/storage_services/search/__init__.py b/api/app/core/memory/storage_services/search/__init__.py index 2bec5bf1..c12c39b0 100644 --- a/api/app/core/memory/storage_services/search/__init__.py +++ b/api/app/core/memory/storage_services/search/__init__.py @@ -37,7 +37,7 @@ __all__ = [ async def run_hybrid_search( query_text: str, search_type: str = "hybrid", - group_id: str | None = None, + end_user_id: str | None = None, apply_id: str | None = None, user_id: str | None = None, limit: int = 50, @@ -54,7 +54,7 @@ async def run_hybrid_search( Args: query_text: 查询文本 search_type: 搜索类型（"hybrid", "keyword", "semantic"） - group_id: 组ID过滤 + end_user_id: 组ID过滤 apply_id: 应用ID过滤 user_id: 用户ID过滤 limit: 每个类别的最大结果数 @@ -104,7 +104,7 @@ async def run_hybrid_search( # 执行搜索 result = await strategy.search( query_text=query_text, - group_id=group_id, + end_user_id=end_user_id, limit=limit, include=include, alpha=alpha, diff --git a/api/app/core/memory/storage_services/search/hybrid_search.py b/api/app/core/memory/storage_services/search/hybrid_search.py index 43215df5..4111b09c 100644 --- a/api/app/core/memory/storage_services/search/hybrid_search.py +++ b/api/app/core/memory/storage_services/search/hybrid_search.py @@ -77,7 +77,7 @@ # async def search( # self, # query_text: str, -# group_id: Optional[str] = None, +# end_user_id: Optional[str] = None, # limit: int = 50, # include: Optional[List[str]] = None, # **kwargs @@ -86,7 +86,7 @@ # Args: # query_text: 查询文本 -# group_id: 可选的组ID过滤 +# end_user_id: 可选的组ID过滤 # limit: 每个类别的最大结果数 # include: 要包含的搜索类别列表 # **kwargs: 其他搜索参数（如alpha, use_forgetting_curve） @@ -94,7 +94,7 @@ # Returns: # SearchResult: 搜索结果对象 # """ -# logger.info(f"执行混合搜索: query='{query_text}', group_id={group_id}, limit={limit}") +# logger.info(f"执行混合搜索: query='{query_text}', end_user_id={end_user_id}, limit={limit}") # # 从kwargs中获取参数 # alpha = kwargs.get("alpha", self.alpha) @@ -107,14 +107,14 @@ # # 并行执行关键词搜索和语义搜索 # keyword_result = await self.keyword_strategy.search( # query_text=query_text, -# group_id=group_id, +# end_user_id=end_user_id, # limit=limit, # include=include_list # ) # semantic_result = await self.semantic_strategy.search( # query_text=query_text, -# group_id=group_id, +# end_user_id=end_user_id, # limit=limit, # include=include_list # ) @@ -139,7 +139,7 @@ # metadata = self._create_metadata( # query_text=query_text, # search_type="hybrid", -# group_id=group_id, +# end_user_id=end_user_id, # limit=limit, # include=include_list, # alpha=alpha, @@ -165,7 +165,7 @@ # metadata=self._create_metadata( # query_text=query_text, # search_type="hybrid", -# group_id=group_id, +# end_user_id=end_user_id, # limit=limit, # error=str(e) # ) diff --git a/api/app/core/memory/storage_services/search/keyword_search.py b/api/app/core/memory/storage_services/search/keyword_search.py index 95dd0581..d2591945 100644 --- a/api/app/core/memory/storage_services/search/keyword_search.py +++ b/api/app/core/memory/storage_services/search/keyword_search.py @@ -44,7 +44,7 @@ class KeywordSearchStrategy(SearchStrategy): async def search( self, query_text: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, limit: int = 50, include: Optional[List[str]] = None, **kwargs @@ -53,7 +53,7 @@ class KeywordSearchStrategy(SearchStrategy): Args: query_text: 查询文本 - group_id: 可选的组ID过滤 + end_user_id: 可选的组ID过滤 limit: 每个类别的最大结果数 include: 要包含的搜索类别列表 **kwargs: 其他搜索参数 @@ -61,7 +61,7 @@ class KeywordSearchStrategy(SearchStrategy): Returns: SearchResult: 搜索结果对象 """ - logger.info(f"执行关键词搜索: query='{query_text}', group_id={group_id}, limit={limit}") + logger.info(f"执行关键词搜索: query='{query_text}', end_user_id={end_user_id}, limit={limit}") # 获取有效的搜索类别 include_list = self._get_include_list(include) @@ -75,7 +75,7 @@ class KeywordSearchStrategy(SearchStrategy): results_dict = await search_graph( connector=self.connector, q=query_text, - group_id=group_id, + end_user_id=end_user_id, limit=limit, include=include_list ) @@ -84,7 +84,7 @@ class KeywordSearchStrategy(SearchStrategy): metadata = self._create_metadata( query_text=query_text, search_type="keyword", - group_id=group_id, + end_user_id=end_user_id, limit=limit, include=include_list ) @@ -115,7 +115,7 @@ class KeywordSearchStrategy(SearchStrategy): metadata=self._create_metadata( query_text=query_text, search_type="keyword", - group_id=group_id, + end_user_id=end_user_id, limit=limit, error=str(e) ) diff --git a/api/app/core/memory/storage_services/search/search_strategy.py b/api/app/core/memory/storage_services/search/search_strategy.py index 27c02c89..3a670dd6 100644 --- a/api/app/core/memory/storage_services/search/search_strategy.py +++ b/api/app/core/memory/storage_services/search/search_strategy.py @@ -58,7 +58,7 @@ class SearchStrategy(ABC): async def search( self, query_text: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, limit: int = 50, include: Optional[List[str]] = None, **kwargs @@ -67,7 +67,7 @@ class SearchStrategy(ABC): Args: query_text: 查询文本 - group_id: 可选的组ID过滤 + end_user_id: 可选的组ID过滤 limit: 每个类别的最大结果数 include: 要包含的搜索类别列表（statements, chunks, entities, summaries） **kwargs: 其他搜索参数 @@ -81,7 +81,7 @@ class SearchStrategy(ABC): self, query_text: str, search_type: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, limit: int = 50, **kwargs ) -> Dict[str, Any]: @@ -90,7 +90,7 @@ class SearchStrategy(ABC): Args: query_text: 查询文本 search_type: 搜索类型 - group_id: 组ID + end_user_id: 组ID limit: 结果限制 **kwargs: 其他元数据 @@ -100,7 +100,7 @@ class SearchStrategy(ABC): metadata = { "query": query_text, "search_type": search_type, - "group_id": group_id, + "end_user_id": end_user_id, "limit": limit, "timestamp": datetime.now().isoformat() } diff --git a/api/app/core/memory/storage_services/search/semantic_search.py b/api/app/core/memory/storage_services/search/semantic_search.py index b20f90a5..8d4eb05f 100644 --- a/api/app/core/memory/storage_services/search/semantic_search.py +++ b/api/app/core/memory/storage_services/search/semantic_search.py @@ -85,7 +85,7 @@ class SemanticSearchStrategy(SearchStrategy): async def search( self, query_text: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, limit: int = 50, include: Optional[List[str]] = None, **kwargs @@ -94,7 +94,7 @@ class SemanticSearchStrategy(SearchStrategy): Args: query_text: 查询文本 - group_id: 可选的组ID过滤 + end_user_id: 可选的组ID过滤 limit: 每个类别的最大结果数 include: 要包含的搜索类别列表 **kwargs: 其他搜索参数 @@ -102,7 +102,7 @@ class SemanticSearchStrategy(SearchStrategy): Returns: SearchResult: 搜索结果对象 """ - logger.info(f"执行语义搜索: query='{query_text}', group_id={group_id}, limit={limit}") + logger.info(f"执行语义搜索: query='{query_text}', end_user_id={end_user_id}, limit={limit}") # 获取有效的搜索类别 include_list = self._get_include_list(include) @@ -119,7 +119,7 @@ class SemanticSearchStrategy(SearchStrategy): connector=self.connector, embedder_client=self.embedder_client, query_text=query_text, - group_id=group_id, + end_user_id=end_user_id, limit=limit, include=include_list ) @@ -128,7 +128,7 @@ class SemanticSearchStrategy(SearchStrategy): metadata = self._create_metadata( query_text=query_text, search_type="semantic", - group_id=group_id, + end_user_id=end_user_id, limit=limit, include=include_list ) @@ -159,7 +159,7 @@ class SemanticSearchStrategy(SearchStrategy): metadata=self._create_metadata( query_text=query_text, search_type="semantic", - group_id=group_id, + end_user_id=end_user_id, limit=limit, error=str(e) ) diff --git a/api/app/core/memory/utils/config/get_data.py b/api/app/core/memory/utils/config/get_data.py index 1de6f6aa..e37ad723 100644 --- a/api/app/core/memory/utils/config/get_data.py +++ b/api/app/core/memory/utils/config/get_data.py @@ -23,7 +23,7 @@ async def _load_(data: List[Any]) -> List[Dict]: target_keys = [ "id", "statement", - "group_id", + "end_user_id", "chunk_id", "created_at", "expired_at", @@ -75,7 +75,7 @@ async def get_data(result): """ EXCLUDE_FIELDS = { "user_id", - "group_id", + "end_user_id", "entity_type", "connect_strength", "relationship_type", diff --git a/api/app/core/memory/utils/log/audit_logger.py b/api/app/core/memory/utils/log/audit_logger.py index 9010aad5..f80ad4d5 100644 --- a/api/app/core/memory/utils/log/audit_logger.py +++ b/api/app/core/memory/utils/log/audit_logger.py @@ -62,7 +62,7 @@ class ConfigAuditLogger: self, config_id: str, user_id: Optional[str] = None, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, success: bool = True, details: Optional[Dict[str, Any]] = None ): @@ -72,14 +72,14 @@ class ConfigAuditLogger: Args: config_id: 配置 ID user_id: 用户 ID（可选） - group_id: 组 ID（可选） + end_user_id: 组 ID（可选） success: 是否成功 details: 详细信息（可选） """ result = "SUCCESS" if success else "FAILED" msg = ( f"CONFIG_LOAD config_id={config_id} " - f"user={user_id or 'N/A'} group={group_id or 'N/A'} " + f"user={user_id or 'N/A'} group={end_user_id or 'N/A'} " f"result={result}" ) if details: @@ -121,7 +121,7 @@ class ConfigAuditLogger: self, operation: str, config_id: str, - group_id: str, + end_user_id: str, success: bool = True, duration: Optional[float] = None, error: Optional[str] = None, @@ -133,7 +133,7 @@ class ConfigAuditLogger: Args: operation: 操作类型（WRITE, READ 等） config_id: 配置 ID - group_id: 组 ID + end_user_id: 组 ID success: 是否成功 duration: 操作耗时（秒） error: 错误信息（可选） @@ -142,7 +142,7 @@ class ConfigAuditLogger: result = "SUCCESS" if success else "FAILED" msg = ( f"{operation.upper()} config_id={config_id} " - f"group={group_id} result={result}" + f"group={end_user_id} result={result}" ) if duration is not None: msg += f" duration={duration:.2f}s" diff --git a/api/app/core/models/scripts/__init__.py b/api/app/core/models/scripts/__init__.py new file mode 100644 index 00000000..657b12fd --- /dev/null +++ b/api/app/core/models/scripts/__init__.py @@ -0,0 +1 @@ +"""模型配置脚本模块""" diff --git a/api/app/core/models/scripts/bedrock_models.yaml b/api/app/core/models/scripts/bedrock_models.yaml new file mode 100644 index 00000000..e561310d --- /dev/null +++ b/api/app/core/models/scripts/bedrock_models.yaml @@ -0,0 +1,174 @@ +provider: bedrock +enabled: true +models: +- name: ai21 + type: llm + provider: bedrock + description: AI21 Labs大语言模型，completion生成模式，256000上下文窗口 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + logo: bedrock +- name: amazon nova + type: llm + provider: bedrock + description: Amazon Nova大语言模型，支持智能体思考、工具调用、流式工具调用、视觉能力，300000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - stream-tool-call + - vision + logo: bedrock +- name: anthropic claude + type: llm + provider: bedrock + description: Anthropic Claude大语言模型，支持智能体思考、视觉能力、工具调用、流式工具调用、文档处理，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - vision + - tool-call + - stream-tool-call + - document + logo: bedrock +- name: cohere + type: llm + provider: bedrock + description: Cohere大语言模型，支持智能体思考、工具调用、流式工具调用，128000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - stream-tool-call + logo: bedrock +- name: deepseek + type: llm + provider: bedrock + description: DeepSeek大语言模型，支持智能体思考、视觉能力、工具调用、流式工具调用，32768上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - vision + - tool-call + - stream-tool-call + logo: bedrock +- name: meta + type: llm + provider: bedrock + description: Meta Llama大语言模型，支持智能体思考、工具调用，128000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + logo: bedrock +- name: mistral + type: llm + provider: bedrock + description: Mistral AI大语言模型，支持智能体思考、工具调用，32000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + logo: bedrock +- name: openai + type: llm + provider: bedrock + description: OpenAI大语言模型，支持智能体思考、工具调用、流式工具调用，32768上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - stream-tool-call + logo: bedrock +- name: qwen + type: llm + provider: bedrock + description: Qwen大语言模型，支持智能体思考、工具调用、流式工具调用，32768上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - stream-tool-call + logo: bedrock +- name: amazon.rerank-v1:0 + type: rerank + provider: bedrock + description: amazon.rerank-v1:0重排序模型，5120上下文窗口 + is_deprecated: false + is_official: true + tags: + - 重排序模型 + logo: bedrock +- name: cohere.rerank-v3-5:0 + type: rerank + provider: bedrock + description: cohere.rerank-v3-5:0重排序模型，5120上下文窗口 + is_deprecated: false + is_official: true + tags: + - 重排序模型 + logo: bedrock +- name: amazon.nova-2-multimodal-embeddings-v1:0 + type: embedding + provider: bedrock + description: amazon.nova-2-multimodal-embeddings-v1:0文本嵌入模型，支持视觉能力，8192上下文窗口 + is_deprecated: false + is_official: true + tags: + - 文本嵌入模型 + - vision + logo: bedrock +- name: amazon.titan-embed-text-v1 + type: embedding + provider: bedrock + description: amazon.titan-embed-text-v1文本嵌入模型，8192上下文窗口 + is_deprecated: false + is_official: true + tags: + - 文本嵌入模型 + logo: bedrock +- name: amazon.titan-embed-text-v2:0 + type: embedding + provider: bedrock + description: amazon.titan-embed-text-v2:0文本嵌入模型，8192上下文窗口 + is_deprecated: false + is_official: true + tags: + - 文本嵌入模型 + logo: bedrock +- name: cohere.embed-english-v3 + type: embedding + provider: bedrock + description: Cohere Embed 3 English文本嵌入模型，512上下文窗口 + is_deprecated: false + is_official: true + tags: + - 文本嵌入模型 + logo: bedrock +- name: cohere.embed-multilingual-v3 + type: embedding + provider: bedrock + description: Cohere Embed 3 Multilingual文本嵌入模型，512上下文窗口 + is_deprecated: false + is_official: true + tags: + - 文本嵌入模型 + logo: bedrock diff --git a/api/app/core/models/scripts/dashscope_models.yaml b/api/app/core/models/scripts/dashscope_models.yaml new file mode 100644 index 00000000..c02ca2cb --- /dev/null +++ b/api/app/core/models/scripts/dashscope_models.yaml @@ -0,0 +1,820 @@ +provider: dashscope +enabled: true +models: +- name: deepseek-r1-distill-qwen-14b + type: llm + provider: dashscope + description: DeepSeek-R1-Distill-Qwen-14B大语言模型，支持智能体思考，32000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + logo: dashscope +- name: deepseek-r1-distill-qwen-32b + type: llm + provider: dashscope + description: DeepSeek-R1-Distill-Qwen-32B大语言模型，支持智能体思考，32000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + logo: dashscope +- name: deepseek-r1 + type: llm + provider: dashscope + description: DeepSeek-R1大语言模型，支持智能体思考，131072超大上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + logo: dashscope +- name: deepseek-v3.1 + type: llm + provider: dashscope + description: DeepSeek-V3.1大语言模型，支持智能体思考，131072超大上下文窗口，对话模式，支持丰富生成参数调节 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + logo: dashscope +- name: deepseek-v3.2-exp + type: llm + provider: dashscope + description: DeepSeek-V3.2-exp实验版大语言模型，支持智能体思考，131072超大上下文窗口，对话模式，支持丰富生成参数调节 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + logo: dashscope +- name: deepseek-v3.2 + type: llm + provider: dashscope + description: DeepSeek-V3.2大语言模型，支持智能体思考，131072超大上下文窗口，对话模式，支持丰富生成参数调节 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + logo: dashscope +- name: deepseek-v3 + type: llm + provider: dashscope + description: DeepSeek-V3大语言模型，支持智能体思考，64000上下文窗口，对话模式，支持文本与JSON格式输出 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + logo: dashscope +- name: farui-plus + type: llm + provider: dashscope + description: farui-plus大语言模型，支持多工具调用、智能体思考、流式工具调用，12288上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: glm-4.7 + type: llm + provider: dashscope + description: GLM-4.7大语言模型，支持多工具调用、智能体思考、流式工具调用，202752超大上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qvq-max-latest + type: llm + provider: dashscope + description: qvq-max-latest大语言模型，支持视觉、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - vision + - agent-thought + - stream-tool-call + logo: dashscope +- name: qvq-max + type: llm + provider: dashscope + description: qvq-max大语言模型，支持视觉、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - vision + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-coder-turbo-0919 + type: llm + provider: dashscope + description: qwen-coder-turbo-0919代码专用大语言模型，支持智能体思考，131072上下文窗口，对话模式，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - 代码模型 + - agent-thought + logo: dashscope +- name: qwen-max-latest + type: llm + provider: dashscope + description: qwen-max-latest大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式，支持联网搜索 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-max-longcontext + type: llm + provider: dashscope + description: qwen-max-longcontext长上下文大语言模型，支持多工具调用、智能体思考、流式工具调用，32000上下文窗口，对话模式，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-max + type: llm + provider: dashscope + description: qwen-max大语言模型，支持多工具调用、智能体思考、流式工具调用，32768上下文窗口，对话模式，支持联网搜索 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-mt-plus + type: llm + provider: dashscope + description: qwen-mt-plus多语言翻译大语言模型，支持智能体思考，16384上下文窗口，对话模式，支持多语种互译与领域翻译适配 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 翻译模型 + - agent-thought + logo: dashscope +- name: qwen-mt-turbo + type: llm + provider: dashscope + description: qwen-mt-turbo轻量化多语言翻译大语言模型，支持智能体思考，16384上下文窗口，对话模式，支持多语种互译与领域翻译适配 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 翻译模型 + - agent-thought + logo: dashscope +- name: qwen-plus-0112 + type: llm + provider: dashscope + description: qwen-plus-0112大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式，支持联网搜索，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-plus-0125 + type: llm + provider: dashscope + description: qwen-plus-0125大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式，支持联网搜索，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-plus-0723 + type: llm + provider: dashscope + description: qwen-plus-0723大语言模型，支持多工具调用、智能体思考、流式工具调用，32000上下文窗口，对话模式，支持联网搜索，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-plus-0806 + type: llm + provider: dashscope + description: qwen-plus-0806大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式，支持联网搜索，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-plus-0919 + type: llm + provider: dashscope + description: qwen-plus-0919大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式，支持联网搜索，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-plus-1125 + type: llm + provider: dashscope + description: qwen-plus-1125大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式，支持联网搜索，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-plus-1127 + type: llm + provider: dashscope + description: qwen-plus-1127大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式，支持联网搜索，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-plus-1220 + type: llm + provider: dashscope + description: qwen-plus-1220大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen-vl-max + type: llm + provider: dashscope + description: qwen-vl-max多模态大模型，支持视觉理解、智能体思考、视频理解，131072上下文窗口，对话模式，未废弃 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - vision + - agent-thought + - video + logo: dashscope +- name: qwen-vl-plus-0809 + type: llm + provider: dashscope + description: qwen-vl-plus-0809多模态大模型，支持视觉理解、智能体思考、视频理解，32768上下文窗口，对话模式，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - vision + - agent-thought + - video + logo: dashscope +- name: qwen-vl-plus-2025-01-02 + type: llm + provider: dashscope + description: qwen-vl-plus-2025-01-02多模态大模型，支持视觉理解、智能体思考、视频理解，32768上下文窗口，对话模式，未废弃 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - vision + - agent-thought + - video + logo: dashscope +- name: qwen-vl-plus-2025-01-25 + type: llm + provider: dashscope + description: qwen-vl-plus-2025-01-25多模态大模型，支持视觉理解、智能体思考、视频理解，131072上下文窗口，对话模式，未废弃 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - vision + - agent-thought + - video + logo: dashscope +- name: qwen-vl-plus-latest + type: llm + provider: dashscope + description: qwen-vl-plus-latest多模态大模型，支持视觉理解、智能体思考、视频理解，131072上下文窗口，对话模式，未废弃 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - vision + - agent-thought + - video + logo: dashscope +- name: qwen-vl-plus + type: llm + provider: dashscope + description: qwen-vl-plus多模态大模型，支持视觉理解、智能体思考、视频理解，131072上下文窗口，对话模式，未废弃 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - vision + - agent-thought + - video + logo: dashscope +- name: qwen2.5-0.5b-instruct + type: llm + provider: dashscope + description: qwen2.5-0.5b-instruct大语言模型，支持多工具调用、智能体思考、流式工具调用，32768上下文窗口，对话模式，未废弃 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-14b + type: llm + provider: dashscope + description: qwen3-14b大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-235b-a22b-instruct-2507 + type: llm + provider: dashscope + description: qwen3-235b-a22b-instruct-2507大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-235b-a22b-thinking-2507 + type: llm + provider: dashscope + description: qwen3-235b-a22b-thinking-2507大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-235b-a22b + type: llm + provider: dashscope + description: qwen3-235b-a22b大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-30b-a3b-instruct-2507 + type: llm + provider: dashscope + description: qwen3-30b-a3b-instruct-2507大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-30b-a3b + type: llm + provider: dashscope + description: qwen3-30b-a3b大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-32b + type: llm + provider: dashscope + description: qwen3-32b大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-4b + type: llm + provider: dashscope + description: qwen3-4b大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-8b + type: llm + provider: dashscope + description: qwen3-8b大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-coder-30b-a3b-instruct + type: llm + provider: dashscope + description: qwen3-coder-30b-a3b-instruct大语言模型，支持智能体思考，262144上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 代码模型 + - agent-thought + logo: dashscope +- name: qwen3-coder-480b-a35b-instruct + type: llm + provider: dashscope + description: qwen3-coder-480b-a35b-instruct大语言模型，支持智能体思考，262144上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 代码模型 + - agent-thought + logo: dashscope +- name: qwen3-coder-plus-2025-09-23 + type: llm + provider: dashscope + description: qwen3-coder-plus-2025-09-23大语言模型，支持智能体思考，1000000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 代码模型 + - agent-thought + logo: dashscope +- name: qwen3-coder-plus + type: llm + provider: dashscope + description: qwen3-coder-plus大语言模型，支持智能体思考，1000000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 代码模型 + - agent-thought + logo: dashscope +- name: qwen3-max-2025-09-23 + type: llm + provider: dashscope + description: qwen3-max-2025-09-23大语言模型，支持多工具调用、智能体思考、流式工具调用，262144上下文窗口，对话模式，支持联网搜索 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - 联网搜索 + logo: dashscope +- name: qwen3-max-2026-01-23 + type: llm + provider: dashscope + description: qwen3-max-2026-01-23大语言模型，支持多工具调用、智能体思考、流式工具调用，262144上下文窗口，对话模式，支持联网搜索 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - 联网搜索 + logo: dashscope +- name: qwen3-max-preview + type: llm + provider: dashscope + description: qwen3-max-preview大语言模型，支持多工具调用、智能体思考、流式工具调用，262144上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-max + type: llm + provider: dashscope + description: qwen3-max大语言模型，支持多工具调用、智能体思考、流式工具调用，262144上下文窗口，对话模式，支持联网搜索 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - 联网搜索 + logo: dashscope +- name: qwen3-next-80b-a3b-instruct + type: llm + provider: dashscope + description: qwen3-next-80b-a3b-instruct大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-next-80b-a3b-thinking + type: llm + provider: dashscope + description: qwen3-next-80b-a3b-thinking大语言模型，支持多工具调用、智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwen3-omni-flash-2025-12-01 + type: llm + provider: dashscope + description: qwen3-omni-flash-2025-12-01多模态大语言模型，支持视觉、智能体思考、视频、音频能力，65536上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - vision + - agent-thought + - video + - audio + logo: dashscope +- name: qwen3-vl-235b-a22b-instruct + type: llm + provider: dashscope + description: qwen3-vl-235b-a22b-instruct多模态大语言模型，支持多工具调用、智能体思考、流式工具调用、视觉、视频能力，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - vision + - video + logo: dashscope +- name: qwen3-vl-235b-a22b-thinking + type: llm + provider: dashscope + description: qwen3-vl-235b-a22b-thinking多模态大语言模型，支持多工具调用、智能体思考、流式工具调用、视觉、视频能力，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - vision + - video + logo: dashscope +- name: qwen3-vl-30b-a3b-instruct + type: llm + provider: dashscope + description: qwen3-vl-30b-a3b-instruct多模态大语言模型，支持多工具调用、智能体思考、流式工具调用、视觉、视频能力，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - vision + - video + logo: dashscope +- name: qwen3-vl-30b-a3b-thinking + type: llm + provider: dashscope + description: qwen3-vl-30b-a3b-thinking多模态大语言模型，支持多工具调用、智能体思考、流式工具调用、视觉、视频能力，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - vision + - video + logo: dashscope +- name: qwen3-vl-flash + type: llm + provider: dashscope + description: qwen3-vl-flash多模态大语言模型，支持多工具调用、智能体思考、流式工具调用、视觉、视频能力，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - vision + - video + logo: dashscope +- name: qwen3-vl-plus-2025-09-23 + type: llm + provider: dashscope + description: qwen3-vl-plus-2025-09-23多模态大语言模型，支持视觉、智能体思考、视频能力，262144上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - vision + - agent-thought + - video + logo: dashscope +- name: qwen3-vl-plus + type: llm + provider: dashscope + description: qwen3-vl-plus多模态大语言模型，支持视觉、智能体思考、视频能力，262144上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - 多模态模型 + - vision + - agent-thought + - video + logo: dashscope +- name: qwq-32b + type: llm + provider: dashscope + description: qwq-32b大语言模型，支持智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwq-plus-0305 + type: llm + provider: dashscope + description: qwq-plus-0305大语言模型，支持智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - stream-tool-call + logo: dashscope +- name: qwq-plus + type: llm + provider: dashscope + description: qwq-plus大语言模型，支持智能体思考、流式工具调用，131072上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - stream-tool-call + logo: dashscope +- name: gte-rerank-v2 + type: rerank + provider: dashscope + description: gte-rerank-v2重排序模型，4000上下文窗口 + is_deprecated: false + is_official: true + tags: + - 重排序模型 + logo: dashscope +- name: gte-rerank + type: rerank + provider: dashscope + description: gte-rerank重排序模型，4000上下文窗口 + is_deprecated: false + is_official: true + tags: + - 重排序模型 + logo: dashscope +- name: multimodal-embedding-v1 + type: embedding + provider: dashscope + description: multimodal-embedding-v1多模态嵌入模型，支持视觉能力，8192上下文窗口，最大分块数10 + is_deprecated: false + is_official: true + tags: + - 嵌入模型 + - 多模态模型 + - vision + logo: dashscope +- name: text-embedding-v1 + type: embedding + provider: dashscope + description: text-embedding-v1文本嵌入模型，2048上下文窗口，最大分块数25 + is_deprecated: false + is_official: true + tags: + - 嵌入模型 + - 文本嵌入 + logo: dashscope +- name: text-embedding-v2 + type: embedding + provider: dashscope + description: text-embedding-v2文本嵌入模型，2048上下文窗口，最大分块数25 + is_deprecated: false + is_official: true + tags: + - 嵌入模型 + - 文本嵌入 + logo: dashscope +- name: text-embedding-v3 + type: embedding + provider: dashscope + description: text-embedding-v3文本嵌入模型，8192上下文窗口，最大分块数10 + is_deprecated: false + is_official: true + tags: + - 嵌入模型 + - 文本嵌入 + logo: dashscope +- name: text-embedding-v4 + type: embedding + provider: dashscope + description: text-embedding-v4文本嵌入模型，8192上下文窗口，最大分块数10 + is_deprecated: false + is_official: true + tags: + - 嵌入模型 + - 文本嵌入 + logo: dashscope diff --git a/api/app/core/models/scripts/loader.py b/api/app/core/models/scripts/loader.py new file mode 100644 index 00000000..6469656c --- /dev/null +++ b/api/app/core/models/scripts/loader.py @@ -0,0 +1,143 @@ +"""模型配置加载器 - 用于将预定义模型批量导入到数据库""" + +import os +from pathlib import Path +from typing import Callable + +import yaml +from sqlalchemy.orm import Session +from app.models.models_model import ModelBase, ModelProvider + + +def _load_yaml_config(provider: ModelProvider) -> list[dict]: + """从YAML文件加载指定供应商的模型配置""" + config_dir = Path(__file__).parent + config_file = config_dir / f"{provider.value}_models.yaml" + + if not config_file.exists(): + return [] + + with open(config_file, 'r', encoding='utf-8') as f: + data = yaml.safe_load(f) + + # 检查是否需要加载（默认为 true） + if not data.get('enabled', True): + return [] + + return data.get('models', []) + + +def _disable_yaml_config(provider: ModelProvider) -> None: + """将YAML文件的enabled标志设置为false""" + config_dir = Path(__file__).parent + config_file = config_dir / f"{provider.value}_models.yaml" + + if not config_file.exists(): + return + + with open(config_file, 'r', encoding='utf-8') as f: + data = yaml.safe_load(f) + + data['enabled'] = False + + with open(config_file, 'w', encoding='utf-8') as f: + yaml.dump(data, f, allow_unicode=True, sort_keys=False) + + +def load_models(db: Session, providers: list[str] = None, silent: bool = False) -> dict: + """ + 加载模型配置到数据库 + + Args: + db: 数据库会话 + providers: 要加载的供应商列表，None表示加载所有 + silent: 是否静默模式（不输出详细日志） + + Returns: + dict: 加载结果统计 {"success": int, "skipped": int, "failed": int} + """ + result = {"success": 0, "skipped": 0, "failed": 0} + + # 确定要加载的供应商 + if providers: + target_providers = [ModelProvider(p) if isinstance(p, str) else p for p in providers] + else: + target_providers = [p for p in ModelProvider if p != ModelProvider.COMPOSITE] + + for provider in target_providers: + # 从YAML文件加载模型配置 + models = _load_yaml_config(provider) + + if not models: + if not silent: + print(f"警告: 供应商 '{provider.value}' 暂无预定义模型") + continue + + if not silent: + print(f"\n正在加载 {provider.value} 的 {len(models)} 个模型...") + + # provider_success = 0 + for model_data in models: + try: + # 检查模型是否已存在 + existing = db.query(ModelBase).filter( + ModelBase.name == model_data["name"], + ModelBase.provider == model_data["provider"] + ).first() + + if existing: + # 更新现有模型配置 + for key, value in model_data.items(): + setattr(existing, key, value) + db.commit() + if not silent: + print(f"更新成功: {model_data['name']}") + result["success"] += 1 + # provider_success += 1 + else: + # 创建新模型 + model = ModelBase(**model_data) + db.add(model) + db.commit() + if not silent: + print(f"添加成功: {model_data['name']}") + result["success"] += 1 + # provider_success += 1 + + except Exception as e: + db.rollback() + if not silent: + print(f"添加失败: {model_data['name']} - {str(e)}") + result["failed"] += 1 + + # 如果该供应商的模型全部加载成功，将enabled设置为false + # if provider_success == len(models): + _disable_yaml_config(provider) + + return result + + +def load_models_by_provider(db: Session, provider: str) -> dict: + """ + 加载指定供应商的模型配置 + + Args: + db: 数据库会话 + provider: 供应商名称（字符串或ModelProvider枚举） + + Returns: + dict: 加载结果统计 + """ + provider_enum = ModelProvider(provider) if isinstance(provider, str) else provider + return load_models(db, providers=[provider_enum]) + + +def get_available_providers() -> list[Callable[[], str]]: + """获取所有可用的供应商列表（从ModelProvider枚举获取，排除COMPOSITE）""" + return [p.value for p in ModelProvider if p != ModelProvider.COMPOSITE] + + +def get_models_by_provider(provider: str) -> list[dict]: + """获取指定供应商的模型配置列表""" + provider_enum = ModelProvider(provider) if isinstance(provider, str) else provider + return _load_yaml_config(provider_enum) diff --git a/api/app/core/models/scripts/openai_models.yaml b/api/app/core/models/scripts/openai_models.yaml new file mode 100644 index 00000000..c114d53f --- /dev/null +++ b/api/app/core/models/scripts/openai_models.yaml @@ -0,0 +1,294 @@ +provider: openai +enabled: true +models: +- name: chatgpt-4o-latest + type: llm + provider: openai + description: chatgpt-4o-latest大语言模型，支持多工具调用、智能体思考、流式工具调用、视觉能力，128000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - vision + logo: openai +- name: gpt-3.5-turbo-0125 + type: llm + provider: openai + description: gpt-3.5-turbo-0125大语言模型，支持多工具调用、智能体思考、流式工具调用，16385上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: openai +- name: gpt-3.5-turbo-1106 + type: llm + provider: openai + description: gpt-3.5-turbo-1106大语言模型，支持多工具调用、智能体思考、流式工具调用，16385上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: openai +- name: gpt-3.5-turbo-16k + type: llm + provider: openai + description: gpt-3.5-turbo-16k大语言模型，支持多工具调用、智能体思考、流式工具调用，16385上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: openai +- name: gpt-3.5-turbo-instruct + type: llm + provider: openai + description: gpt-3.5-turbo-instruct大语言模型，4096上下文窗口，文本补全模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + logo: openai +- name: gpt-3.5-turbo + type: llm + provider: openai + description: gpt-3.5-turbo大语言模型，支持多工具调用、智能体思考、流式工具调用，16385上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: openai +- name: gpt-4-0125-preview + type: llm + provider: openai + description: gpt-4-0125-preview大语言模型，支持多工具调用、智能体思考、流式工具调用，128000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: openai +- name: gpt-4-1106-preview + type: llm + provider: openai + description: gpt-4-1106-preview大语言模型，支持多工具调用、智能体思考、流式工具调用，128000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: openai +- name: gpt-4-turbo-2024-04-09 + type: llm + provider: openai + description: gpt-4-turbo-2024-04-09大语言模型，支持多工具调用、智能体思考、流式工具调用、视觉能力，128000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - vision + logo: openai +- name: gpt-4-turbo-preview + type: llm + provider: openai + description: gpt-4-turbo-preview大语言模型，支持多工具调用、智能体思考、流式工具调用，128000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + logo: openai +- name: gpt-4-turbo + type: llm + provider: openai + description: gpt-4-turbo大语言模型，支持多工具调用、智能体思考、流式工具调用、视觉能力，128000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - vision + logo: openai +- name: o1-preview + type: llm + provider: openai + description: o1-preview大语言模型，支持智能体思考，128000上下文窗口，对话模式，已废弃 + is_deprecated: true + is_official: true + tags: + - 大语言模型 + - agent-thought + logo: openai +- name: o1 + type: llm + provider: openai + description: o1大语言模型，支持多工具调用、智能体思考、流式工具调用、视觉能力、结构化输出，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - multi-tool-call + - agent-thought + - stream-tool-call + - vision + - structured-output + logo: openai +- name: o3-2025-04-16 + type: llm + provider: openai + description: o3-2025-04-16大语言模型，支持智能体思考、工具调用、视觉能力、流式工具调用、结构化输出，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - vision + - stream-tool-call + - structured-output + logo: openai +- name: o3-mini-2025-01-31 + type: llm + provider: openai + description: o3-mini-2025-01-31大语言模型，支持智能体思考、工具调用、流式工具调用、结构化输出，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - stream-tool-call + - structured-output + logo: openai +- name: o3-mini + type: llm + provider: openai + description: o3-mini大语言模型，支持智能体思考、工具调用、流式工具调用、结构化输出，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - stream-tool-call + - structured-output + logo: openai +- name: o3-pro-2025-06-10 + type: llm + provider: openai + description: o3-pro-2025-06-10大语言模型，支持智能体思考、工具调用、视觉能力、结构化输出，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - vision + - structured-output + logo: openai +- name: o3-pro + type: llm + provider: openai + description: o3-pro大语言模型，支持智能体思考、工具调用、视觉能力、结构化输出，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - vision + - structured-output + logo: openai +- name: o3 + type: llm + provider: openai + description: o3大语言模型，支持智能体思考、视觉能力、工具调用、流式工具调用、结构化输出，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - vision + - tool-call + - stream-tool-call + - structured-output + logo: openai +- name: o4-mini-2025-04-16 + type: llm + provider: openai + description: o4-mini-2025-04-16大语言模型，支持智能体思考、工具调用、视觉能力、流式工具调用、结构化输出，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - vision + - stream-tool-call + - structured-output + logo: openai +- name: o4-mini + type: llm + provider: openai + description: o4-mini大语言模型，支持智能体思考、工具调用、视觉能力、流式工具调用、结构化输出，200000上下文窗口，对话模式 + is_deprecated: false + is_official: true + tags: + - 大语言模型 + - agent-thought + - tool-call + - vision + - stream-tool-call + - structured-output + logo: openai +- name: text-embedding-3-large + type: embedding + provider: openai + description: text-embedding-3-large文本向量模型，8191上下文窗口，最大分块数32 + is_deprecated: false + is_official: true + tags: + - 文本向量模型 + logo: openai +- name: text-embedding-3-small + type: embedding + provider: openai + description: text-embedding-3-small文本向量模型，8191上下文窗口，最大分块数32 + is_deprecated: false + is_official: true + tags: + - 文本向量模型 + logo: openai +- name: text-embedding-ada-002 + type: embedding + provider: openai + description: text-embedding-ada-002文本向量模型，8097上下文窗口，最大分块数32 + is_deprecated: false + is_official: true + tags: + - 文本向量模型 + logo: openai diff --git a/api/app/core/rag/app/presentation.py b/api/app/core/rag/app/presentation.py deleted file mode 100644 index d62e0096..00000000 --- a/api/app/core/rag/app/presentation.py +++ /dev/null @@ -1,165 +0,0 @@ -import copy -import re -from io import BytesIO -from PIL import Image - -from app.core.rag.nlp import tokenize, is_english -from app.core.rag.nlp import rag_tokenizer -from app.core.rag.deepdoc.parser import PdfParser, PlainParser -from app.core.rag.deepdoc.parser.ppt_parser import RAGPptParser as PptParser -from PyPDF2 import PdfReader as pdf2_read -from app.core.rag.app.naive import by_plaintext, PARSERS - -class Ppt(PptParser): - def __call__(self, fnm, from_page, to_page, callback=None): - txts = super().__call__(fnm, from_page, to_page) - - callback(0.5, "Text extraction finished.") - import aspose.slides as slides - import aspose.pydrawing as drawing - imgs = [] - with slides.Presentation(BytesIO(fnm)) as presentation: - for i, slide in enumerate(presentation.slides[from_page: to_page]): - try: - with BytesIO() as buffered: - slide.get_thumbnail( - 0.1, 0.1).save( - buffered, drawing.imaging.ImageFormat.jpeg) - buffered.seek(0) - imgs.append(Image.open(buffered).copy()) - except RuntimeError as e: - raise RuntimeError(f'ppt parse error at page {i+1}, original error: {str(e)}') from e - assert len(imgs) == len( - txts), "Slides text and image do not match: {} vs. {}".format(len(imgs), len(txts)) - callback(0.9, "Image extraction finished") - self.is_english = is_english(txts) - return [(txts[i], imgs[i]) for i in range(len(txts))] - -class Pdf(PdfParser): - def __init__(self): - super().__init__() - - def __garbage(self, txt): - txt = txt.lower().strip() - if re.match(r"[0-9\.,%/-]+$", txt): - return True - if len(txt) < 3: - return True - return False - - def __call__(self, filename, binary=None, from_page=0, - to_page=100000, zoomin=3, callback=None): - from timeit import default_timer as timer - start = timer() - callback(msg="OCR started") - self.__images__(filename if not binary else binary, - zoomin, from_page, to_page, callback) - callback(msg="Page {}~{}: OCR finished ({:.2f}s)".format(from_page, min(to_page, self.total_page), timer() - start)) - assert len(self.boxes) == len(self.page_images), "{} vs. {}".format( - len(self.boxes), len(self.page_images)) - res = [] - for i in range(len(self.boxes)): - lines = "\n".join([b["text"] for b in self.boxes[i] - if not self.__garbage(b["text"])]) - res.append((lines, self.page_images[i])) - callback(0.9, "Page {}~{}: Parsing finished".format( - from_page, min(to_page, self.total_page))) - return res, [] - - -class PlainPdf(PlainParser): - def __call__(self, filename, binary=None, from_page=0, - to_page=100000, callback=None, **kwargs): - self.pdf = pdf2_read(filename if not binary else BytesIO(binary)) - page_txt = [] - for page in self.pdf.pages[from_page: to_page]: - page_txt.append(page.extract_text()) - callback(0.9, "Parsing finished") - return [(txt, None) for txt in page_txt], [] - - -def chunk(filename, binary=None, from_page=0, to_page=100000, - lang="Chinese", callback=None, vision_model=None, parser_config=None, **kwargs): - """ - The supported file formats are pdf, pptx. - Every page will be treated as a chunk. And the thumbnail of every page will be stored. - PPT file will be parsed by using this method automatically, setting-up for every PPT file is not necessary. - """ - if parser_config is None: - parser_config = {} - eng = lang.lower() == "english" - doc = { - "docnm_kwd": filename, - "title_tks": rag_tokenizer.tokenize(re.sub(r"\.[a-zA-Z]+$", "", filename)) - } - doc["title_sm_tks"] = rag_tokenizer.fine_grained_tokenize(doc["title_tks"]) - res = [] - if re.search(r"\.pptx?$", filename, re.IGNORECASE): - if not binary: - with open(filename, "rb") as f: - binary = f.read() - ppt_parser = Ppt() - for pn, (txt, img) in enumerate(ppt_parser( - filename if not binary else binary, from_page, 1000000, callback)): - d = copy.deepcopy(doc) - pn += from_page - d["image"] = img - d["doc_type_kwd"] = "image" - d["page_num_int"] = [pn + 1] - d["top_int"] = [0] - d["position_int"] = [(pn + 1, 0, img.size[0], 0, img.size[1])] - tokenize(d, txt, eng) - res.append(d) - return res - elif re.search(r"\.pdf$", filename, re.IGNORECASE): - layout_recognizer = parser_config.get("layout_recognize", "DeepDOC") - - if isinstance(layout_recognizer, bool): - layout_recognizer = "DeepDOC" if layout_recognizer else "Plain Text" - - name = layout_recognizer.strip().lower() - parser = PARSERS.get(name, by_plaintext) - callback(0.1, "Start to parse.") - - sections, _, _ = parser( - filename=filename, - binary=binary, - from_page=from_page, - to_page=to_page, - lang=lang, - callback=callback, - vision_model=vision_model, - pdf_cls=Pdf, - **kwargs - ) - - if not sections: - return [] - - if name in ["tcadp", "docling", "mineru"]: - parser_config["chunk_token_num"] = 0 - - callback(0.8, "Finish parsing.") - - for pn, (txt, img) in enumerate(sections): - d = copy.deepcopy(doc) - pn += from_page - if img: - d["image"] = img - d["page_num_int"] = [pn + 1] - d["top_int"] = [0] - d["position_int"] = [(pn + 1, 0, img.size[0] if img else 0, 0, img.size[1] if img else 0)] - tokenize(d, txt, eng) - res.append(d) - return res - - raise NotImplementedError( - "file type not supported yet(pptx, pdf supported)") - - -if __name__ == "__main__": - import sys - - def dummy(a, b): - pass - chunk(sys.argv[1], callback=dummy) diff --git a/api/app/core/rag/vdb/field.py b/api/app/core/rag/vdb/field.py index 86d39060..99d872c2 100644 --- a/api/app/core/rag/vdb/field.py +++ b/api/app/core/rag/vdb/field.py @@ -4,7 +4,7 @@ from enum import StrEnum, auto class Field(StrEnum): CONTENT_KEY = "page_content" METADATA_KEY = "metadata" - GROUP_KEY = "group_id" + GROUP_KEY = "end_user_id" VECTOR = auto() # Sparse Vector aims to support full text search SPARSE_VECTOR = auto() diff --git a/api/app/core/storage/url_signer.py b/api/app/core/storage/url_signer.py index 480c8ef4..712b298e 100644 --- a/api/app/core/storage/url_signer.py +++ b/api/app/core/storage/url_signer.py @@ -36,7 +36,7 @@ def generate_signed_url( """ if base_url is None: # Use SERVER_IP or default to localhost - server_url = f"http://{settings.SERVER_IP}:8000/api" + server_url = settings.FILE_LOCAL_SERVER_URL base_url = server_url # Calculate expiration timestamp diff --git a/api/app/core/tools/builtin/baidu_search_tool.py b/api/app/core/tools/builtin/baidu_search_tool.py index 02431aed..45d4c359 100644 --- a/api/app/core/tools/builtin/baidu_search_tool.py +++ b/api/app/core/tools/builtin/baidu_search_tool.py @@ -16,7 +16,7 @@ class BaiduSearchTool(BuiltinTool): @property def description(self) -> str: - return "百度搜索 - 搜索引擎服务：网页搜索、新闻搜索、图片搜索、实时结果" + return "百度搜索 - 搜索引擎服务：网页搜索、新闻搜索、图片搜索、视频搜索" def get_required_config_parameters(self) -> List[str]: return ["api_key"] @@ -33,7 +33,7 @@ class BaiduSearchTool(BuiltinTool): ToolParameter( name="search_type", type=ParameterType.STRING, - description="搜索类型", + description="搜索类型, web: 网页搜索；news：新闻搜索；image：图片搜索；video视频搜索", required=False, default="web", enum=["web", "news", "image", "video"] diff --git a/api/app/core/validators/memory_config_validators.py b/api/app/core/validators/memory_config_validators.py index 333572e6..ba26c5f2 100644 --- a/api/app/core/validators/memory_config_validators.py +++ b/api/app/core/validators/memory_config_validators.py @@ -26,7 +26,7 @@ logger = get_config_logger() def _parse_model_id(model_id: Union[str, UUID, None], model_type: str, - config_id: Optional[int] = None, workspace_id: Optional[UUID] = None) -> Optional[UUID]: + config_id: Optional[UUID] = None, workspace_id: Optional[UUID] = None) -> Optional[UUID]: """Parse model ID from string or UUID.""" if model_id is None: return None @@ -59,7 +59,7 @@ def validate_model_exists_and_active( model_type: str, db: Session, tenant_id: Optional[UUID] = None, - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, workspace_id: Optional[UUID] = None ) -> tuple[str, bool]: """Validate that a model exists and is active. @@ -166,7 +166,7 @@ def validate_and_resolve_model_id( db: Session, tenant_id: Optional[UUID] = None, required: bool = False, - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, workspace_id: Optional[UUID] = None ) -> tuple[Optional[UUID], Optional[str]]: """Validate and resolve a model ID, checking existence and active status. @@ -204,7 +204,7 @@ def validate_and_resolve_model_id( def validate_embedding_model( - config_id: int, + config_id: UUID, embedding_id: Union[str, UUID, None], db: Session, tenant_id: Optional[UUID] = None, @@ -256,7 +256,7 @@ def validate_embedding_model( def validate_llm_model( - config_id: int, + config_id: UUID, llm_id: Union[str, UUID, None], db: Session, tenant_id: Optional[UUID] = None, diff --git a/api/app/core/workflow/executor.py b/api/app/core/workflow/executor.py index b719091c..b7abf659 100644 --- a/api/app/core/workflow/executor.py +++ b/api/app/core/workflow/executor.py @@ -11,17 +11,12 @@ from typing import Any from langchain_core.runnables import RunnableConfig from langgraph.graph.state import CompiledStateGraph -from app.core.workflow.graph_builder import GraphBuilder +from app.core.workflow.expression_evaluator import evaluate_expression +from app.core.workflow.graph_builder import GraphBuilder, StreamOutputConfig from app.core.workflow.nodes import WorkflowState from app.core.workflow.nodes.base_config import VariableType from app.core.workflow.nodes.enums import NodeType -# from app.core.tools.registry import ToolRegistry -# from app.core.tools.executor import ToolExecutor -# from app.core.tools.langchain_adapter import LangchainAdapter -# TOOL_MANAGEMENT_AVAILABLE = True -# from app.db import get_db - logger = logging.getLogger(__name__) @@ -55,6 +50,8 @@ class WorkflowExecutor: self.execution_config = workflow_config.get("execution_config", {}) self.start_node_id = None + self.end_outputs: dict[str, StreamOutputConfig] = {} + self.activate_end: str | None = None self.checkpoint_config = RunnableConfig( configurable={ @@ -127,7 +124,6 @@ class WorkflowExecutor: "user_id": self.user_id, "error": None, "error_node": None, - "streaming_buffer": {}, # 流式缓冲区 "cycle_nodes": [ node.get("id") for node in self.workflow_config.get("nodes") @@ -139,9 +135,8 @@ class WorkflowExecutor: } } - def _build_final_output(self, result, elapsed_time): + def _build_final_output(self, result, elapsed_time, final_output): node_outputs = result.get("node_outputs", {}) - final_output = self._extract_final_output(node_outputs) token_usage = self._aggregate_token_usage(node_outputs) conversation_id = None for node_id, node_output in node_outputs.items(): @@ -161,6 +156,146 @@ class WorkflowExecutor: "error": result.get("error"), } + def _update_scope_activate(self, scope, status=None): + """ + Update the activation state of all End nodes based on a completed scope (node or variable). + + Iterates over all End nodes in `self.end_outputs` and calls + `update_activate` on each, which may: + - Activate variable segments that depend on the completed node/scope. + - Activate the entire End node output if all control conditions are met. + + If any End node becomes active and `self.activate_end` is not yet set, + this node will be marked as the currently active End node. + + Args: + scope (str): The node ID or scope that has completed execution. + status (str | None): Optional status of the node (used for branch/control nodes). + """ + for node in self.end_outputs.keys(): + self.end_outputs[node].update_activate(scope, status) + if self.end_outputs[node].activate and self.activate_end is None: + self.activate_end = node + + def _update_stream_output_status(self, activate, data): + """ + Update the stream output state of End nodes based on workflow state updates. + + This method checks which nodes/scopes are activated and propagates + activation to End nodes accordingly. + + Args: + activate (dict): Mapping of node_id -> bool indicating which nodes/scopes are activated. + data (dict): Mapping of node_id -> node runtime data, including outputs. + + Behavior: + For each node in `data`: + 1. If the node is activated (`activate[node_id]` is True), + retrieve its output status from `runtime_vars`. + 2. Call `_update_scope_activate` to propagate the activation + to all relevant End nodes and update `self.activate_end`. + """ + for node_id in data.keys(): + if activate.get(node_id): + node_output_status = ( + data[node_id] + .get('runtime_vars', {}) + .get(node_id) + .get("output") + ) + self._update_scope_activate(node_id, status=node_output_status) + + async def _emit_active_chunks( + self, + node_outputs: dict, + variables: dict, + force=False + ): + """ + Process and yield all currently active output segments for the currently active End node. + + This method handles stream-mode output for an End node by iterating through its output segments + (`OutputContent`). Only segments marked as active (`activate=True`) are processed, unless + `force=True`, which allows all segments to be processed regardless of their activation state. + + Behavior: + 1. Iterates from the current `cursor` position to the end of the outputs list. + 2. For each segment: + - If the segment is literal text (`is_variable=False`), append it directly. + - If the segment is a variable (`is_variable=True`), evaluate it using + `evaluate_expression` with the given `node_outputs` and `variables`, + then transform the result with `_trans_output_string`. + 3. Yield a stream event of type "message" containing the processed chunk. + 4. Move the `cursor` forward after processing each segment. + 5. When all segments have been processed, remove this End node from `end_outputs` + and reset `activate_end` to None. + + Args: + node_outputs (dict): Current runtime node outputs, used for variable evaluation. + variables (dict): Current runtime variables, used for variable evaluation. + force (bool, default=False): If True, process segments even if `activate=False`. + + Yields: + dict: A stream event of type "message" containing the processed chunk. + + Notes: + - Segments that fail evaluation (ValueError) are skipped with a warning logged. + - This method only processes the currently active End node (`self.activate_end`). + - Use `force=True` for final emission regardless of activation state. + """ + + end_info = self.end_outputs[self.activate_end] + + while end_info.cursor < len(end_info.outputs): + final_chunk = '' + current_segment = end_info.outputs[end_info.cursor] + + if not current_segment.activate and not force: + # Stop processing until this segment becomes active + break + + # Literal segment + if not current_segment.is_variable: + final_chunk += current_segment.literal + else: + # Variable segment: evaluate and transform + try: + chunk = evaluate_expression( + current_segment.literal, + variables=variables, + node_outputs=node_outputs + ) + chunk = self._trans_output_string(chunk) + final_chunk += chunk + except ValueError: + # Log failed evaluation but continue streaming + logger.warning(f"[STREAM] Failed to evaluate segment: {current_segment.literal}") + + if final_chunk: + yield { + "event": "message", + "data": { + "chunk": final_chunk + } + } + + # Advance cursor after processing + end_info.cursor += 1 + + # Remove End node from active tracking if all segments have been processed + if end_info.cursor >= len(end_info.outputs): + self.end_outputs.pop(self.activate_end) + self.activate_end = None + + @staticmethod + def _trans_output_string(content): + if isinstance(content, str): + return content + elif isinstance(content, list): + return "\n".join(content) + else: + return str(content) + def build_graph(self, stream=False) -> CompiledStateGraph: """构建 LangGraph @@ -173,6 +308,7 @@ class WorkflowExecutor: stream=stream, ) self.start_node_id = builder.start_node_id + self.end_outputs = builder.end_node_map graph = builder.build() logger.info(f"工作流图构建完成: execution_id={self.execution_id}") @@ -205,14 +341,28 @@ class WorkflowExecutor: try: result = await graph.ainvoke(initial_state, config=self.checkpoint_config) - + full_content = '' + for end_id in self.end_outputs.keys(): + full_content += result.get('runtime_vars', {}).get(end_id, {}).get('output', '') + result["messages"].extend( + [ + { + "role": "user", + "content": input_data.get("message", '') + }, + { + "role": "assistant", + "content": full_content + } + ] + ) # 计算耗时 end_time = datetime.datetime.now() elapsed_time = (end_time - start_time).total_seconds() logger.info(f"工作流执行完成: execution_id={self.execution_id}, elapsed_time={elapsed_time:.2f}s") - return self._build_final_output(result, elapsed_time) + return self._build_final_output(result, elapsed_time, full_content) except Exception as e: # 计算耗时（即使失败也记录） @@ -261,7 +411,7 @@ class WorkflowExecutor: "data": { "execution_id": self.execution_id, "workspace_id": self.workspace_id, - "timestamp": start_time.isoformat() + "timestamp": int(start_time.timestamp() * 1000) } } @@ -273,7 +423,8 @@ class WorkflowExecutor: # 3. Execute workflow try: chunk_count = 0 - + full_content = '' + self._update_scope_activate("sys") async for event in graph.astream( initial_state, stream_mode=["updates", "debug", "custom"], # Use updates + debug + custom mode @@ -293,20 +444,42 @@ class WorkflowExecutor: # Handle custom streaming events (chunks from nodes via stream writer) chunk_count += 1 event_type = data.get("type", "node_chunk") # "message" or "node_chunk" - logger.info(f"[CUSTOM] ✅ 收到 {event_type} #{chunk_count} from {data.get('node_id')}" - f"- execution_id: {self.execution_id}") - yield { - "event": event_type, # "message" or "node_chunk" - "data": { - "node_id": data.get("node_id"), - "chunk": data.get("chunk"), - "full_content": data.get("full_content"), - "chunk_index": data.get("chunk_index"), - "is_prefix": data.get("is_prefix"), - "is_suffix": data.get("is_suffix"), - "conversation_id": input_data.get("conversation_id"), + if event_type == "node_chunk": + node_id = data.get("node_id") + if self.activate_end: + end_info = self.end_outputs.get(self.activate_end) + if not end_info or end_info.cursor >= len(end_info.outputs): + continue + current_output = end_info.outputs[end_info.cursor] + if current_output.is_variable and current_output.depends_on_scope(node_id): + if data.get("done"): + end_info.cursor += 1 + if end_info.cursor >= len(end_info.outputs): + self.end_outputs.pop(self.activate_end) + self.activate_end = None + else: + full_content += data.get("chunk") + yield { + "event": "message", + "data": { + "chunk": data.get("chunk") + } + } + logger.info(f"[CUSTOM] ✅ 收到 {event_type} #{chunk_count} from {data.get('node_id')}" + f"- execution_id: {self.execution_id}") + + elif event_type == "node_error": + yield { + "event": event_type, # "message" or "node_chunk" + "data": { + "node_id": data.get("node_id"), + "status": "failed", + "input": data.get("input_data"), + "elapsed_time": data.get("elapsed_time"), + "output": None, + "error": data.get("error") + } } - } elif mode == "debug": # Handle debug information (node execution status) @@ -325,14 +498,15 @@ class WorkflowExecutor: conversation_id = input_data.get("conversation_id") logger.info(f"[NODE-START] Node starts execution: {node_name} " f"- execution_id: {self.execution_id}") - yield { "event": "node_start", "data": { "node_id": node_name, "conversation_id": conversation_id, "execution_id": self.execution_id, - "timestamp": data.get("timestamp"), + "timestamp": int(datetime.datetime.fromisoformat( + data.get("timestamp") + ).timestamp() * 1000), } } elif event_type == "task_result": @@ -351,21 +525,82 @@ class WorkflowExecutor: "node_id": node_name, "conversation_id": conversation_id, "execution_id": self.execution_id, - "timestamp": data.get("timestamp"), - "state": result.get("node_outputs", {}).get(node_name), + "timestamp": int(datetime.datetime.fromisoformat( + data.get("timestamp") + ).timestamp() * 1000), + "input": result.get("node_outputs", {}).get(node_name, {}).get("input"), + "output": result.get("node_outputs", {}).get(node_name, {}).get("output"), + "elapsed_time": result.get("node_outputs", {}).get(node_name, {}).get("elapsed_time"), } } elif mode == "updates": # Handle state updates - store final state - # TODO:流式输出点 + state = graph.get_state(config=self.checkpoint_config).values + node_outputs = state.get("runtime_vars", {}) + variables = state.get("variables", {}) + activate = state.get("activate", {}) + for _, node_data in data.items(): + node_outputs |= node_data.get("runtime_vars", {}) + variables |= node_data.get("variables", {}) + + self._update_stream_output_status(activate, data) + wait = False + while self.activate_end and not wait: + async for msg_event in self._emit_active_chunks( + node_outputs=node_outputs, + variables=variables + ): + full_content += msg_event["data"]['chunk'] + yield msg_event + + if self.activate_end: + wait = True + else: + self._update_stream_output_status(activate, data) + logger.debug(f"[UPDATES] 收到 state 更新 from {list(data.keys())} " f"- execution_id: {self.execution_id}") + result = graph.get_state(self.checkpoint_config).values + node_outputs = result.get("runtime_vars", {}) + variables = result.get("variables", {}) + self.end_outputs = { + node_id: node_info + for node_id, node_info in self.end_outputs.items() + if node_info.activate + } + + if self.end_outputs or self.activate_end: + while self.activate_end: + async for msg_event in self._emit_active_chunks( + node_outputs=node_outputs, + variables=variables, + force=True + ): + full_content += msg_event["data"]['chunk'] + yield msg_event + + if not self.activate_end and self.end_outputs: + self.activate_end = list(self.end_outputs.keys())[0] + # 计算耗时 end_time = datetime.datetime.now() elapsed_time = (end_time - start_time).total_seconds() result = graph.get_state(self.checkpoint_config).values + logger.info(result) + result["messages"].extend( + [ + { + "role": "user", + "content": input_data.get("message", '') + }, + { + "role": "assistant", + "content": full_content + } + ] + ) logger.info( f"Workflow execution completed (streaming), " f"total chunks: {chunk_count}, elapsed: {elapsed_time:.2f}s, execution_id: {self.execution_id}" @@ -374,7 +609,7 @@ class WorkflowExecutor: # 发送 workflow_end 事件 yield { "event": "workflow_end", - "data": self._build_final_output(result, elapsed_time) + "data": self._build_final_output(result, elapsed_time, full_content) } except Exception as e: @@ -396,31 +631,6 @@ class WorkflowExecutor: } } - @staticmethod - def _extract_final_output(node_outputs: dict[str, Any]) -> str | None: - """从节点输出中提取最终输出 - - 优先级： - 1. 最后一个执行的非 start/end 节点的 output - 2. 如果没有节点输出，返回 None - - Args: - node_outputs: 所有节点的输出 - - Returns: - 最终输出字符串或 None - """ - if not node_outputs: - return None - - # 获取最后一个节点的输出 - last_node_output = list(node_outputs.values())[-1] if node_outputs else None - - if last_node_output and isinstance(last_node_output, dict): - return last_node_output.get("output") - - return None - @staticmethod def _aggregate_token_usage(node_outputs: dict[str, Any]) -> dict[str, int] | None: """聚合所有节点的 token 使用情况 @@ -511,178 +721,3 @@ async def execute_workflow_stream( ) async for event in executor.execute_stream(input_data): yield event - -# ==================== 工具管理系统集成 ==================== - -# def get_workflow_tools(workspace_id: str, user_id: str) -> list: -# """获取工作流可用的工具列表 -# -# Args: -# workspace_id: 工作空间ID -# user_id: 用户ID -# -# Returns: -# 可用工具列表 -# """ -# if not TOOL_MANAGEMENT_AVAILABLE: -# logger.warning("工具管理系统不可用") -# return [] -# -# try: -# db = next(get_db()) -# -# # 创建工具注册表 -# registry = ToolRegistry(db) -# -# # 注册内置工具类 -# from app.core.tools.builtin import ( -# DateTimeTool, JsonTool, BaiduSearchTool, MinerUTool, TextInTool -# ) -# registry.register_tool_class(DateTimeTool) -# registry.register_tool_class(JsonTool) -# registry.register_tool_class(BaiduSearchTool) -# registry.register_tool_class(MinerUTool) -# registry.register_tool_class(TextInTool) -# -# # 获取活跃的工具 -# import uuid -# tools = registry.list_tools(workspace_id=uuid.UUID(workspace_id)) -# active_tools = [tool for tool in tools if tool.status.value == "active"] -# -# # 转换为Langchain工具 -# langchain_tools = [] -# for tool_info in active_tools: -# try: -# tool_instance = registry.get_tool(tool_info.id) -# if tool_instance: -# langchain_tool = LangchainAdapter.convert_tool(tool_instance) -# langchain_tools.append(langchain_tool) -# except Exception as e: -# logger.error(f"转换工具失败: {tool_info.name}, 错误: {e}") -# -# logger.info(f"为工作流获取了 {len(langchain_tools)} 个工具") -# return langchain_tools -# -# except Exception as e: -# logger.error(f"获取工作流工具失败: {e}") -# return [] -# -# -# class ToolWorkflowNode: -# """工具工作流节点 - 在工作流中执行工具""" -# -# def __init__(self, node_config: dict, workflow_config: dict): -# """初始化工具节点 -# -# Args: -# node_config: 节点配置 -# workflow_config: 工作流配置 -# """ -# self.node_config = node_config -# self.workflow_config = workflow_config -# self.tool_id = node_config.get("tool_id") -# self.tool_parameters = node_config.get("parameters", {}) -# -# async def run(self, state: WorkflowState) -> WorkflowState: -# """执行工具节点""" -# if not TOOL_MANAGEMENT_AVAILABLE: -# logger.error("工具管理系统不可用") -# state["error"] = "工具管理系统不可用" -# return state -# -# try: -# from sqlalchemy.orm import Session -# db = next(get_db()) -# -# # 创建工具执行器 -# registry = ToolRegistry(db) -# executor = ToolExecutor(db, registry) -# -# # 准备参数（支持变量替换） -# parameters = self._prepare_parameters(state) -# -# # 执行工具 -# result = await executor.execute_tool( -# tool_id=self.tool_id, -# parameters=parameters, -# user_id=uuid.UUID(state["user_id"]), -# workspace_id=uuid.UUID(state["workspace_id"]) -# ) -# -# # 更新状态 -# node_id = self.node_config.get("id") -# if result.success: -# state["node_outputs"][node_id] = { -# "type": "tool", -# "tool_id": self.tool_id, -# "output": result.data, -# "execution_time": result.execution_time, -# "token_usage": result.token_usage -# } -# -# # 更新运行时变量 -# if isinstance(result.data, dict): -# for key, value in result.data.items(): -# state["runtime_vars"][f"{node_id}.{key}"] = value -# else: -# state["runtime_vars"][f"{node_id}.result"] = result.data -# else: -# state["error"] = result.error -# state["error_node"] = node_id -# state["node_outputs"][node_id] = { -# "type": "tool", -# "tool_id": self.tool_id, -# "error": result.error, -# "execution_time": result.execution_time -# } -# -# return state -# -# except Exception as e: -# logger.error(f"工具节点执行失败: {e}") -# state["error"] = str(e) -# state["error_node"] = self.node_config.get("id") -# return state -# -# def _prepare_parameters(self, state: WorkflowState) -> dict: -# """准备工具参数（支持变量替换）""" -# parameters = {} -# -# for key, value in self.tool_parameters.items(): -# if isinstance(value, str) and value.startswith("${") and value.endswith("}"): -# # 变量替换 -# var_path = value[2:-1] -# -# # 支持多层级变量访问，如 ${sys.message} 或 ${node1.result} -# if "." in var_path: -# parts = var_path.split(".") -# current = state.get("variables", {}) -# -# for part in parts: -# if isinstance(current, dict) and part in current: -# current = current[part] -# else: -# # 尝试从运行时变量获取 -# runtime_key = ".".join(parts) -# current = state.get("runtime_vars", {}).get(runtime_key, value) -# break -# -# parameters[key] = current -# else: -# # 简单变量 -# variables = state.get("variables", {}) -# parameters[key] = variables.get(var_path, value) -# else: -# parameters[key] = value -# -# return parameters -# -# -# # 注册工具节点到NodeFactory（如果存在） -# try: -# from app.core.workflow.nodes import NodeFactory -# if hasattr(NodeFactory, 'register_node_type'): -# NodeFactory.register_node_type("tool", ToolWorkflowNode) -# logger.info("工具节点已注册到工作流系统") -# except Exception as e: -# logger.warning(f"注册工具节点失败: {e}") diff --git a/api/app/core/workflow/graph_builder.py b/api/app/core/workflow/graph_builder.py index 5b9388fc..b1d43e08 100644 --- a/api/app/core/workflow/graph_builder.py +++ b/api/app/core/workflow/graph_builder.py @@ -1,12 +1,15 @@ import logging +import re import uuid from collections import defaultdict +from functools import lru_cache from typing import Any from langgraph.checkpoint.memory import InMemorySaver from langgraph.graph import START, END from langgraph.graph.state import CompiledStateGraph, StateGraph from langgraph.types import Send +from pydantic import BaseModel, Field from app.core.workflow.expression_evaluator import evaluate_condition from app.core.workflow.nodes import WorkflowState, NodeFactory @@ -15,6 +18,149 @@ from app.core.workflow.nodes.enums import NodeType, BRANCH_NODES logger = logging.getLogger(__name__) +class OutputContent(BaseModel): + """ + Represents a single output segment of an End node. + + An output segment can be either: + - literal text (static string) + - a variable placeholder (e.g. {{ node.field }}) + + Each segment has its own activation state, which is especially + important in stream mode. + """ + + literal: str = Field( + ..., + description="Raw output content. Can be literal text or a variable placeholder." + ) + + activate: bool = Field( + ..., + description=( + "Whether this output segment is currently active.\n" + "- True: allowed to be emitted/output\n" + "- False: blocked until activated by branch control" + ) + ) + + is_variable: bool = Field( + ..., + description=( + "Whether this segment represents a variable placeholder.\n" + "True -> variable (e.g. {{ node.field }})\n" + "False -> literal text" + ) + ) + + def depends_on_scope(self, scope: str) -> bool: + """ + Check if this segment depends on a given scope. + + Args: + scope (str): Node ID or special variable prefix (e.g., "sys"). + + Returns: + bool: True if this segment references the given scope. + """ + pattern = rf"\{{\{{\s*{re.escape(scope)}\.[a-zA-Z0-9_]+\s*\}}\}}" + return bool(re.search(pattern, self.literal)) + + +class StreamOutputConfig(BaseModel): + """ + Streaming output configuration for an End node. + + This configuration describes how the End node output behaves in streaming mode, + including: + - whether output emission is globally activated + - which upstream branch/control nodes gate the activation + - how each parsed output segment is streamed and activated + """ + + activate: bool = Field( + ..., + description=( + "Global activation flag for the End node output.\n" + "When False, output segments should not be emitted even if available.\n" + "This flag typically becomes True once required control branch conditions " + "are satisfied." + ) + ) + + control_nodes: dict[str, str] = Field( + ..., + description=( + "Control branch conditions for this End node output.\n" + "Mapping of `branch_node_id -> expected_branch_label`.\n" + "The End node output becomes globally active when a controlling branch node " + "reports a matching completion status." + ) + ) + + outputs: list[OutputContent] = Field( + ..., + description=( + "Ordered list of output segments parsed from the output template.\n" + "Each segment represents either a literal text block or a variable placeholder " + "that may be activated independently." + ) + ) + + cursor: int = Field( + ..., + description=( + "Streaming cursor index.\n" + "Indicates the next output segment index to be emitted.\n" + "Segments with index < cursor are considered already streamed." + ) + ) + + def update_activate(self, scope: str, status=None): + """ + Update streaming activation state based on an upstream node or special variable. + + Args: + scope (str): + Identifier of the completed upstream entity. + - If a control branch node, it should match a key in `control_nodes`. + - If a variable placeholder (e.g., "sys.xxx"), it may appear in output segments. + status (optional): + Completion status of the control branch node. + Required when `scope` refers to a control node. + + Behavior: + 1. Control branch nodes: + - If `scope` matches a key in `control_nodes` and `status` matches the expected + branch label, the End node output becomes globally active (`activate = True`). + + 2. Variable output segments: + - For each segment that is a variable (`is_variable=True`): + - If the segment literal references `scope`, mark the segment as active. + - This applies both to regular node variables (e.g., "node_id.field") + and special system variables (e.g., "sys.xxx"). + + Notes: + - This method does not emit output or advance the streaming cursor. + - It only updates activation flags based on upstream events or special variables. + """ + + # Case 1: resolve control branch dependency + if scope in self.control_nodes.keys(): + if status is None: + raise RuntimeError("[Stream Output] Control node activation status not provided") + if status == self.control_nodes[scope]: + self.activate = True + + # Case 2: activate variable segments related to this node + for i in range(len(self.outputs)): + if ( + self.outputs[i].is_variable + and self.outputs[i].depends_on_scope(scope) + ): + self.outputs[i].activate = True + + class GraphBuilder: def __init__( self, @@ -29,10 +175,16 @@ class GraphBuilder: self.start_node_id = None self.end_node_ids = [] + self.node_map = {node["id"]: node for node in self.nodes} + self.end_node_map: dict[str, StreamOutputConfig] = {} + self._find_upstream_branch_node = lru_cache( + maxsize=len(self.nodes) * 2 + )(self._find_upstream_branch_node) self.graph = StateGraph(WorkflowState) self.add_nodes() self.add_edges() + self._analyze_end_node_output() # EDGES MUST BE ADDED AFTER NODES ARE ADDED. @property @@ -43,79 +195,207 @@ class GraphBuilder: def edges(self) -> list[dict[str, Any]]: return self.workflow_config.get("edges", []) - def _analyze_end_node_prefixes(self) -> tuple[dict[str, str], set[str]]: - """ - Analyze the prefix configuration for End nodes. + def get_node_type(self, node_id: str) -> str: + """Retrieve the type of node given its ID. - This function scans each End node's output template, identifies - references to its direct upstream nodes, and extracts the prefix - string appearing before the first reference. + Args: + node_id (str): The unique identifier of the node. Returns: - tuple: - - dict[str, str]: Mapping from upstream node ID to its End node prefix - - set[str]: Set of node IDs that are directly adjacent to End nodes and referenced + str: The type of the node. + + Raises: + RuntimeError: If no node with the given `node_id` exists. """ - import re + try: + return self.node_map[node_id]["type"] + except KeyError: + raise RuntimeError(f"Node not found: Id={node_id}") - prefixes = {} - adjacent_and_referenced = set() # Record nodes directly adjacent to End and referenced + def _find_upstream_branch_node(self, target_node: str) -> tuple[bool, tuple[tuple[str, str]]]: + """ + Recursively find all upstream branch (control) nodes that influence the execution + of the given target node. - # 找到所有 End 节点 + This method walks upstream along the workflow graph starting from `target_node`. + It distinguishes between: + - branch nodes (node types listed in `BRANCH_NODES`) + - non-branch nodes (ordinary processing nodes) + + Traversal rules: + 1. For each immediate upstream node: + - If it is a branch node, it is recorded as an affecting control node. + - If it is a non-branch node, the traversal continues recursively upstream. + 2. If ANY upstream path reaches a START / CYCLE_START node without encountering + a branch node, the traversal is considered invalid: + - `has_branch` will be False + - no branch nodes are returned. + 3. Only when ALL upstream non-branch paths eventually lead to at least one + branch node will `has_branch` be True. + + Special case: + - If `target_node` has no upstream nodes AND its type is START or CYCLE_START, + it is considered directly reachable from the workflow entry, and therefore + has no controlling branch nodes. + + Args: + target_node (str): + The identifier of the node whose upstream control branches + are to be resolved. + + Returns: + tuple[bool, tuple[tuple[str, str]]]: + - has_branch (bool): + True if every upstream path from `target_node` encounters + at least one branch node. + False if any path reaches a start node without a branch. + - branch_nodes (tuple[tuple[str, str]]): + A deduplicated tuple of `(branch_node_id, branch_label)` pairs + representing all branch nodes that can influence `target_node`. + Returns an empty tuple if `has_branch` is False. + """ + source_nodes = [ + { + "id": edge.get("source"), + "branch": edge.get("label") + } + for edge in self.edges + if edge.get("target") == target_node + ] + if not source_nodes and self.get_node_type(target_node) in [NodeType.START, NodeType.CYCLE_START]: + return False, tuple() + + branch_nodes = [] + non_branch_nodes = [] + + for node_info in source_nodes: + if self.get_node_type(node_info["id"]) in BRANCH_NODES: + branch_nodes.append( + (node_info["id"], node_info["branch"]) + ) + else: + non_branch_nodes.append(node_info["id"]) + + has_branch = True + for node_id in non_branch_nodes: + node_has_branch, nodes = self._find_upstream_branch_node(node_id) + has_branch = has_branch and node_has_branch + if not has_branch: + break + branch_nodes.extend(nodes) + if not has_branch: + branch_nodes = [] + + return has_branch, tuple(set(branch_nodes)) + + def _analyze_end_node_output(self): + """ + Analyze output templates of all End nodes and generate StreamOutputConfig. + + This method is responsible for parsing the `output` field of End nodes, + splitting literal text and variable placeholders (e.g. {{ node.field }}), + and determining whether each output segment should be activated immediately + or controlled by upstream branch nodes. + + In stream mode: + - If the End node is controlled by any upstream branch node, the output + will be initially inactive and controlled by those branch nodes. + - Otherwise, the output is activated immediately. + + In non-stream mode: + - All outputs are activated by default. + """ + + # Collect all End nodes in the workflow end_nodes = [node for node in self.nodes if node.get("type") == "end"] logger.info(f"[Prefix Analysis] Found {len(end_nodes)} End nodes") + # Iterate through each End node to analyze its output for end_node in end_nodes: end_node_id = end_node.get("id") - output_template = end_node.get("config", {}).get("output") + config = end_node.get("config", {}) + output = config.get("output") - logger.info(f"[Prefix Analysis] End node {end_node_id} template: {output_template}") - - if not output_template: + # Skip End nodes without output configuration + if not output: continue - # Find all node references in the template - # Matches {{node_id.xxx}} or {{ node_id.xxx }} format (allowing spaces) - pattern = r'\{\{\s*([a-zA-Z0-9_-]+)\.[a-zA-Z0-9_]+\s*\}\}' - matches = list(re.finditer(pattern, output_template)) + # Regex to split output into: + # - variable placeholders: {{ ... }} + # - normal literal text + # + # Example: + # "Hello {{user.name}}!" -> + # ["Hello ", "{{user.name}}", "!"] + pattern = r'\{\{.*?\}\}|[^{}]+' - logger.info(f"[Prefix Analysis] 模板中找到 {len(matches)} 个节点引用") + # Strict variable format: {{ node_id.field_name }} + variable_pattern_string = r'\{\{\s*[a-zA-Z0-9_]+\.[a-zA-Z0-9_]+\s*\}\}' + variable_pattern = re.compile(variable_pattern_string) - # Identify all direct upstream nodes connected to the End node - direct_upstream_nodes = [] - for edge in self.edges: - if edge.get("target") == end_node_id: - source_node_id = edge.get("source") - direct_upstream_nodes.append(source_node_id) + # Split output into ordered segments + output_template = list(re.findall(pattern, output)) - logger.info(f"[Prefix Analysis] Direct upstream nodes of End node: {direct_upstream_nodes}") + # Determine whether each segment is literal text + # True -> literal (can be directly output) + # False -> variable placeholder (needs runtime value) + output_flag = [ + not bool(variable_pattern.match(item)) + for item in output_template + ] - # 找到第一个直接上游节点的引用 - for match in matches: - referenced_node_id = match.group(1) - logger.info(f"[Prefix Analysis] Checking reference: {referenced_node_id}") + # Stream mode: output activation depends on upstream branch nodes + if self.stream: + # Find upstream branch nodes that can control this End node + has_branch, control_nodes = self._find_upstream_branch_node(end_node_id) - if referenced_node_id in direct_upstream_nodes: - # 这是直接上游节点的引用，提取前缀 - prefix = output_template[:match.start()] + # Build StreamOutputConfig for this End node + self.end_node_map[end_node_id] = StreamOutputConfig( + # If there is no upstream branch, output is active immediately + activate=not has_branch, - logger.info(f"[Prefix Analysis] " - f"✅ Found reference to direct upstream node {referenced_node_id}, prefix: '{prefix}'") + # Branch nodes that control activation of this End node + control_nodes=dict(control_nodes), - # 标记这个节点为"相邻且被引用" - adjacent_and_referenced.add(referenced_node_id) + # Convert output segments into OutputContent objects + outputs=list( + [ + OutputContent( + literal=output_string, + # Literal text can be activated immediately unless blocked by branch + activate=activate, + # Variable segments are marked explicitly + is_variable=not activate + ) + for output_string, activate in zip(output_template, output_flag) + ] + ), + # Cursor for streaming output (initially 0) + cursor=0 + ) + logger.info(f"[Stream Analysis] end_id: {end_node_id}, " + f"activate: {not has_branch}, " + f"control_nodes: {control_nodes}," + f"output: {output_template}," + f"output_activate: {output_flag}") - if prefix: - prefixes[referenced_node_id] = prefix - logger.info(f"[Prefix Analysis] " - f"✅ Assign prefix for node {referenced_node_id}: '{prefix[:50]}...'") - - # 只处理第一个直接上游节点的引用 - break - - logger.info(f"[Prefix Analysis] Final prefixes: {prefixes}") - logger.info(f"[Prefix Analysis] Nodes adjacent to End and referenced: {adjacent_and_referenced}") - return prefixes, adjacent_and_referenced + # Non-stream mode: all outputs are activated by default + else: + self.end_node_map[end_node_id] = StreamOutputConfig( + activate=True, + control_nodes={}, + outputs=list( + [ + OutputContent( + literal=output_string, + activate=True, + is_variable=not activate + ) + for output_string, activate in zip(output_template, output_flag) + ] + ), + cursor=0 + ) def add_nodes(self): """Add all nodes from the workflow configuration to the state graph. @@ -135,9 +415,6 @@ class GraphBuilder: Returns: None """ - # Analyze End node prefixes if in stream mode - end_prefixes, adjacent_and_referenced = self._analyze_end_node_prefixes() if self.stream else ({}, set()) - for node in self.nodes: node_type = node.get("type") node_id = node.get("id") @@ -171,17 +448,6 @@ class GraphBuilder: related_edge[idx]['condition'] = f"node.{node_id}.output == '{related_edge[idx]['label']}'" if node_instance: - # Inject End node prefix configuration if in stream mode - if self.stream and node_id in end_prefixes: - node_instance._end_node_prefix = end_prefixes[node_id] - logger.info(f"Injected End prefix for node {node_id}") - - # Mark nodes as adjacent and referenced to End node in stream mode - if self.stream: - node_instance._is_adjacent_to_end = node_id in adjacent_and_referenced - if node_id in adjacent_and_referenced: - logger.info(f"Node {node_id} marked as adjacent and referenced to End node") - # Wrap node's run method to avoid closure issues if self.stream: # Stream mode: create an async generator function @@ -261,6 +527,7 @@ class GraphBuilder: for source_node, branches in conditional_edges.items(): def make_router(src, branch_list): """reate a router function for each source node that routes to a NOP node for later merging.""" + def make_branch_node(node_name, targets): def node(s): # NOTE: NOP NODE MUST NOT MODIFY STATE diff --git a/api/app/core/workflow/nodes/base_node.py b/api/app/core/workflow/nodes/base_node.py index 1ebeb378..4dcdf2bb 100644 --- a/api/app/core/workflow/nodes/base_node.py +++ b/api/app/core/workflow/nodes/base_node.py @@ -67,10 +67,6 @@ class WorkflowState(TypedDict): error: str | None error_node: str | None - # Streaming buffer (stores real-time streaming output of nodes) - # Format: {node_id: {"chunks": [...], "full_content": "..."}} - streaming_buffer: Annotated[dict[str, Any], lambda x, y: {**x, **y}] - # node activate status activate: Annotated[dict[str, bool], merge_activate_state] @@ -300,7 +296,7 @@ class BaseNode(ABC): """ if not self.check_activate(state): yield self.trans_activate(state) - logger.info(f"跳过节点{self.node_id}") + logger.info(f"jump node: {self.node_id}") return import time @@ -313,19 +309,6 @@ class BaseNode(ABC): # Get LangGraph's stream writer for sending custom data writer = get_stream_writer() - # Check if this is an End node - # End nodes CAN send chunks (for suffix), but only after LLM content - is_end_node = self.node_type == "end" - - # Check if this node is adjacent to End node (for message type) - is_adjacent_to_end = getattr(self, '_is_adjacent_to_end', False) - - # Determine chunk type: "message" for End and adjacent nodes, "node_chunk" for others - chunk_type = "message" if (is_end_node or is_adjacent_to_end) else "node_chunk" - - logger.debug( - f"节点 {self.node_id} chunk 类型: {chunk_type} (is_end={is_end_node}, adjacent={is_adjacent_to_end})") - # Accumulate complete result (for final wrapping) chunks = [] final_result = None @@ -340,66 +323,25 @@ class BaseNode(ABC): raise TimeoutError() # Check if it's a completion marker - if isinstance(item, dict) and item.get("__final__"): + if item.get("__final__"): final_result = item["result"] - elif isinstance(item, str): - # String is a chunk + else: chunk_count += 1 - chunks.append(item) - full_content = "".join(chunks) + content = str(item.get("chunk")) + done = item.get("done", False) + chunks.append(content) # Send chunks for all nodes (including End nodes for suffix) - logger.debug(f"节点 {self.node_id} 发送 chunk #{chunk_count}: {item[:50]}...") + logger.debug(f"节点 {self.node_id} 发送 chunk #{chunk_count}: {content[:50]}...") # 1. Send via stream writer (for real-time client updates) writer({ - "type": chunk_type, # "message" or "node_chunk" + "type": "node_chunk", "node_id": self.node_id, - "chunk": item, - "full_content": full_content, - "chunk_index": chunk_count + "chunk": content, + "done": done }) - # 2. Update streaming buffer in state (for downstream nodes) - # Only non-End nodes need streaming buffer - if not is_end_node: - yield { - "streaming_buffer": { - self.node_id: { - "full_content": full_content, - "chunk_count": chunk_count, - "is_complete": False - } - } - } - else: - # Other types are also treated as chunks - chunk_count += 1 - chunk_str = str(item) - chunks.append(chunk_str) - full_content = "".join(chunks) - - # Send chunks for all nodes - writer({ - "type": chunk_type, # "message" or "node_chunk" - "node_id": self.node_id, - "chunk": chunk_str, - "full_content": full_content, - "chunk_index": chunk_count - }) - - # Only non-End nodes need streaming buffer - if not is_end_node: - yield { - "streaming_buffer": { - self.node_id: { - "full_content": full_content, - "chunk_count": chunk_count, - "is_complete": False - } - } - } - elapsed_time = time.time() - start_time logger.info(f"节点 {self.node_id} 流式执行完成，耗时: {elapsed_time:.2f}s, chunks: {chunk_count}") @@ -426,16 +368,6 @@ class BaseNode(ABC): "looping": state["looping"] } - # Add streaming buffer for non-End nodes - if not is_end_node: - state_update["streaming_buffer"] = { - self.node_id: { - "full_content": "".join(chunks), - "chunk_count": chunk_count, - "is_complete": True # Mark as complete - } - } - # Finally yield state update # LangGraph will merge this into state yield state_update | self.trans_activate(state) @@ -544,6 +476,11 @@ class BaseNode(ABC): "error_node": self.node_id } else: + writer = get_stream_writer() + writer({ + "type": "node_error", + **node_output + }) # 无错误边：抛出异常停止工作流 logger.error(f"节点 {self.node_id} 执行失败，停止工作流: {error_message}") raise Exception(f"节点 {self.node_id} 执行失败: {error_message}") diff --git a/api/app/core/workflow/nodes/code/__init__.py b/api/app/core/workflow/nodes/code/__init__.py index e69de29b..758ab3a5 100644 --- a/api/app/core/workflow/nodes/code/__init__.py +++ b/api/app/core/workflow/nodes/code/__init__.py @@ -0,0 +1,3 @@ +from app.core.workflow.nodes.code.node import CodeNode + +__all__ = ["CodeNode"] diff --git a/api/app/core/workflow/nodes/code/config.py b/api/app/core/workflow/nodes/code/config.py new file mode 100644 index 00000000..8af13f12 --- /dev/null +++ b/api/app/core/workflow/nodes/code/config.py @@ -0,0 +1,50 @@ +from typing import Literal +from pydantic import Field, BaseModel + +from app.core.workflow.nodes.base_config import BaseNodeConfig, VariableType + + +class InputVariable(BaseModel): + name: str = Field( + ..., + description="variable name" + ) + + variable: str = Field( + ..., + description="variable selector" + ) + + +class OutputVariable(BaseModel): + name: str = Field( + ..., + description="variable name" + ) + + type: VariableType = Field( + ..., + description="variable selector" + ) + + +class CodeNodeConfig(BaseNodeConfig): + input_variables: list[InputVariable] = Field( + default_factory=list, + description="input variables" + ) + + output_variables: list[OutputVariable] = Field( + default_factory=list, + description="output variables" + ) + + code: str = Field( + default="", + description="code content" + ) + + language: Literal['python3', 'nodejs'] = Field( + ..., + description="language" + ) diff --git a/api/app/core/workflow/nodes/code/node.py b/api/app/core/workflow/nodes/code/node.py new file mode 100644 index 00000000..b2a4da32 --- /dev/null +++ b/api/app/core/workflow/nodes/code/node.py @@ -0,0 +1,121 @@ +import base64 +import json +import logging +import re +from string import Template +from textwrap import dedent +from typing import Any + +import httpx + +from app.core.workflow.nodes import BaseNode, WorkflowState +from app.core.workflow.nodes.base_config import VariableType +from app.core.workflow.nodes.code.config import CodeNodeConfig + +logger = logging.getLogger(__name__) + +SCRIPT_TEMPLATE = Template(dedent(""" +$code + +import json +from base64 import b64decode + +# decode and prepare input dict +inputs_obj = json.loads(b64decode('$inputs_variable').decode('utf-8')) + +# execute main function +output_obj = main(**inputs_obj) + +# convert output to json and print +output_json = json.dumps(output_obj, indent=4) +result = "<>" + output_json + "<>" +print(result) +""")) + + +class CodeNode(BaseNode): + def __init__(self, node_config: dict[str, Any], workflow_config: dict[str, Any]): + super().__init__(node_config, workflow_config) + self.typed_config: CodeNodeConfig | None = None + + def extract_result(self, content: str): + match = re.search(r'<>(.*?)<>', content, re.DOTALL) + if match: + extracted = match.group(1) + exec_result = json.loads(extracted) + result = {} + for output in self.typed_config.output_variables: + value = exec_result.get(output.name) + if value is None: + raise RuntimeError(f"Return value {output.name} does not exist") + match output.type: + case VariableType.STRING: + if not isinstance(value, str): + raise RuntimeError(f"Return value {output.name} should be a string") + case VariableType.BOOLEAN: + if not isinstance(value, bool): + raise RuntimeError(f"Return value {output.name} should be a boolean") + case VariableType.NUMBER: + if not isinstance(value, (int, float)): + raise RuntimeError(f"Return value {output.name} should be a number") + case VariableType.OBJECT: + if not isinstance(value, dict): + raise RuntimeError(f"Return value {output.name} should be a dictionary") + case VariableType.ARRAY_STRING: + if not isinstance(value, list) or not all(isinstance(v, str) for v in value): + raise RuntimeError(f"Return value {output.name} should be a list of strings") + case VariableType.ARRAY_NUMBER: + if not isinstance(value, list) or not all(isinstance(v, (int, float)) for v in value): + raise RuntimeError(f"Return value {output.name} should be a list of numbers") + case VariableType.ARRAY_OBJECT: + if not isinstance(value, list) or not all(isinstance(v, dict) for v in value): + raise RuntimeError(f"Return value {output.name} should be a list of dictionaries") + case VariableType.ARRAY_BOOLEAN: + if not isinstance(value, list) or not all(isinstance(v, bool) for v in value): + raise RuntimeError(f"Return value {output.name} should be a list of booleans") + result[output.name] = value + return result + else: + raise RuntimeError("The output of main must be a dictionary") + + async def execute(self, state: WorkflowState) -> Any: + self.typed_config = CodeNodeConfig(**self.config) + input_variable_dict = {} + for input_variable in self.typed_config.input_variables: + input_variable_dict[input_variable.name] = self.get_variable(input_variable.variable, state) + code = base64.b64decode( + self.typed_config.code + ).decode("utf-8") + + input_variable_dict = base64.b64encode( + json.dumps(input_variable_dict).encode("utf-8") + ).decode("utf-8") + + final_script = SCRIPT_TEMPLATE.substitute( + code=code, + inputs_variable=input_variable_dict, + ) + + async with httpx.AsyncClient() as client: + response = await client.post( + "http://sandbox:8194/v1/sandbox/run", + headers={ + "x-api-key": 'redbear-sandbox' + }, + json={ + "language": self.typed_config.language, + "code": base64.b64encode(final_script.encode("utf-8")).decode("utf-8"), + "options": { + "enable_network": True + } + } + ) + resp = response.json() + + match resp['code']: + case 31: + raise RuntimeError("Operation not permitted") + case 0: + return self.extract_result(resp["data"]["stdout"]) + case _: + raise Exception(resp["message"]) diff --git a/api/app/core/workflow/nodes/configs.py b/api/app/core/workflow/nodes/configs.py index 4d31efaa..d73754f6 100644 --- a/api/app/core/workflow/nodes/configs.py +++ b/api/app/core/workflow/nodes/configs.py @@ -10,21 +10,22 @@ from app.core.workflow.nodes.base_config import ( VariableDefinition, VariableType, ) +from app.core.workflow.nodes.code.config import CodeNodeConfig +from app.core.workflow.nodes.cycle_graph.config import LoopNodeConfig, IterationNodeConfig from app.core.workflow.nodes.end.config import EndNodeConfig from app.core.workflow.nodes.http_request.config import HttpRequestNodeConfig from app.core.workflow.nodes.if_else.config import IfElseNodeConfig from app.core.workflow.nodes.jinja_render.config import JinjaRenderNodeConfig from app.core.workflow.nodes.knowledge.config import KnowledgeRetrievalNodeConfig from app.core.workflow.nodes.llm.config import LLMNodeConfig, MessageConfig -from app.core.workflow.nodes.start.config import StartNodeConfig -from app.core.workflow.nodes.transform.config import TransformNodeConfig -from app.core.workflow.nodes.variable_aggregator.config import VariableAggregatorNodeConfig +from app.core.workflow.nodes.memory.config import MemoryReadNodeConfig, MemoryWriteNodeConfig from app.core.workflow.nodes.parameter_extractor.config import ParameterExtractorNodeConfig from app.core.workflow.nodes.question_classifier.config import QuestionClassifierNodeConfig +from app.core.workflow.nodes.start.config import StartNodeConfig from app.core.workflow.nodes.tool.config import ToolNodeConfig -from app.core.workflow.nodes.memory.config import MemoryReadNodeConfig, MemoryWriteNodeConfig +from app.core.workflow.nodes.transform.config import TransformNodeConfig +from app.core.workflow.nodes.variable_aggregator.config import VariableAggregatorNodeConfig -from app.core.workflow.nodes.cycle_graph.config import LoopNodeConfig, IterationNodeConfig __all__ = [ # 基础类 "BaseNodeConfig", @@ -49,5 +50,6 @@ __all__ = [ "QuestionClassifierNodeConfig", "ToolNodeConfig", "MemoryReadNodeConfig", - "MemoryWriteNodeConfig" + "MemoryWriteNodeConfig", + "CodeNodeConfig" ] diff --git a/api/app/core/workflow/nodes/cycle_graph/iteration.py b/api/app/core/workflow/nodes/cycle_graph/iteration.py index e9174df8..cd63d233 100644 --- a/api/app/core/workflow/nodes/cycle_graph/iteration.py +++ b/api/app/core/workflow/nodes/cycle_graph/iteration.py @@ -1,5 +1,4 @@ import asyncio -import copy import logging import re from typing import Any diff --git a/api/app/core/workflow/nodes/cycle_graph/node.py b/api/app/core/workflow/nodes/cycle_graph/node.py index 1f550b0b..82782658 100644 --- a/api/app/core/workflow/nodes/cycle_graph/node.py +++ b/api/app/core/workflow/nodes/cycle_graph/node.py @@ -6,7 +6,6 @@ from langgraph.graph.state import CompiledStateGraph from app.core.workflow.nodes import WorkflowState from app.core.workflow.nodes.base_node import BaseNode -from app.core.workflow.nodes.cycle_graph.config import LoopNodeConfig, IterationNodeConfig from app.core.workflow.nodes.cycle_graph.iteration import IterationRuntime from app.core.workflow.nodes.cycle_graph.loop import LoopRuntime from app.core.workflow.nodes.enums import NodeType diff --git a/api/app/core/workflow/nodes/end/node.py b/api/app/core/workflow/nodes/end/node.py index 0cbd9e8e..3a5153a9 100644 --- a/api/app/core/workflow/nodes/end/node.py +++ b/api/app/core/workflow/nodes/end/node.py @@ -5,10 +5,8 @@ End 节点实现 """ import logging -import re from app.core.workflow.nodes.base_node import BaseNode, WorkflowState -from app.core.workflow.nodes.enums import NodeType logger = logging.getLogger(__name__) @@ -37,24 +35,8 @@ class EndNode(BaseNode): # 如果配置了输出模板，使用模板渲染；否则使用默认输出 if output_template: output = self._render_template(output_template, state, strict=False) - state['messages'].extend([ - { - "role": "user", - "content": self.get_variable("sys.message", state) - }, - { - "role": "assistant", - "content": output - } - ]) else: - state['messages'].extend([ - { - "role": "user", - "content": self.get_variable("sys.message", state) - }, - ]) - output = "工作流已完成" + output = "" # 统计信息（用于日志） node_outputs = state.get("node_outputs", {}) @@ -63,274 +45,3 @@ class EndNode(BaseNode): logger.info(f"节点 {self.node_id} (End) 执行完成，共执行 {total_nodes} 个节点") return output - - def _extract_referenced_nodes(self, template: str) -> list[str]: - """从模板中提取引用的节点 ID - - 例如：'结果：{{llm_qa.output}}' -> ['llm_qa'] - - Args: - template: 模板字符串 - - Returns: - 引用的节点 ID 列表 - """ - # 匹配 {{node_id.xxx}} 格式 - pattern = r'\{\{([a-zA-Z0-9_]+)\.[a-zA-Z0-9_]+\}\}' - matches = re.findall(pattern, template) - return list(set(matches)) # 去重 - - def _parse_template_parts(self, template: str, state: WorkflowState) -> list[dict]: - """解析模板，分离静态文本和动态引用 - - 例如：'你好 {{llm.output}}, 这是后缀' - 返回：[ - {"type": "static", "content": "你好 "}, - {"type": "dynamic", "node_id": "llm", "field": "output"}, - {"type": "static", "content": ", 这是后缀"} - ] - - Args: - template: 模板字符串 - state: 工作流状态 - - Returns: - 模板部分列表 - """ - import re - - parts = [] - last_end = 0 - - # 匹配 {{xxx}} 或 {{ xxx }} 格式（支持空格） - pattern = r'\{\{\s*([^}]+?)\s*\}\}' - - for match in re.finditer(pattern, template): - start, end = match.span() - - # 添加前面的静态文本 - if start > last_end: - static_text = template[last_end:start] - if static_text: - parts.append({"type": "static", "content": static_text}) - - # 解析动态引用 - ref = match.group(1).strip() - - # 检查是否是节点引用（如 llm.output 或 llm_qa.output） - if '.' in ref: - node_id, field = ref.split('.', 1) - parts.append({ - "type": "dynamic", - "node_id": node_id, - "field": field, - "raw": ref - }) - else: - # 其他引用（如 {{var.xxx}}），当作静态处理 - # 直接渲染这部分 - rendered = self._render_template(f"{{{{{ref}}}}}", state) - parts.append({"type": "static", "content": rendered}) - - last_end = end - - # 添加最后的静态文本 - if last_end < len(template): - static_text = template[last_end:] - if static_text: - parts.append({"type": "static", "content": static_text}) - - return parts - - async def execute_stream(self, state: WorkflowState): - """Execute End node business logic (streaming) - - Smart output strategy: - 1. Check if template references a direct upstream LLM node - 2. If yes, only output the part AFTER that reference (suffix) - 3. Prefix and LLM content have already been sent during LLM node streaming - - Note: Only LLM nodes get this special treatment. Other node types output normally. - - Example: '{{start.test}}hahaha {{ llm_qa.output }} lalalalala a' - - Direct upstream LLM node is llm_qa - - Prefix '{{start.test}}hahaha ' was sent before LLM node streaming - - LLM content was streamed during LLM node execution - - End node only outputs ' lalalalala a' (suffix, sent as one chunk) - - Args: - state: Workflow state - - Yields: - Completion marker - """ - logger.info(f"节点 {self.node_id} (End) 开始执行（流式）") - - # 获取配置的输出模板 - output_template = self.config.get("output") - - if not output_template: - output = "工作流已完成" - from langgraph.config import get_stream_writer - writer = get_stream_writer() - writer({ - "type": "message", # End node output uses message type - "node_id": self.node_id, - "chunk": "", - "full_content": output, - "chunk_index": 1, - "is_suffix": False - }) - state['messages'].extend([ - { - "role": "user", - "content": self.get_variable("sys.message", state) - } - ]) - yield {"__final__": True, "result": output} - return - - # Find direct upstream LLM nodes - direct_upstream_llm_nodes = [] - for edge in self.workflow_config.get("edges", []): - if edge.get("target") == self.node_id: - source_node_id = edge.get("source") - # Check if the source node is an LLM node - for node in self.workflow_config.get("nodes", []): - logger.info(f"节点 {self.node_id} 的类型 {node.get("type")}") - if node.get("id") == source_node_id and node.get("type") == NodeType.LLM: - direct_upstream_llm_nodes.append(source_node_id) - break - - logger.info(f"节点 {self.node_id} 的直接上游 LLM 节点: {direct_upstream_llm_nodes}") - - # Parse template parts - parts = self._parse_template_parts(output_template, state) - logger.info(f"节点 {self.node_id} 解析模板，共 {len(parts)} 个部分") - for i, part in enumerate(parts): - logger.info(f"[模板解析] part[{i}]: {part}") - - # Find the first reference to a direct upstream LLM node - upstream_llm_ref_index = None - for i, part in enumerate(parts): - if part["type"] == "dynamic" and part["node_id"] in direct_upstream_llm_nodes: - upstream_llm_ref_index = i - logger.info(f"节点 {self.node_id} 找到直接上游 LLM 节点 {part['node_id']} 的引用，索引: {i}") - break - - if upstream_llm_ref_index is None: - # No reference to direct upstream LLM node, output complete template content - output = self._render_template(output_template, state, strict=False) - logger.info(f"节点 {self.node_id} 没有引用直接上游 LLM 节点，输出完整内容: '{output[:50]}...'") - - # Send complete content via writer (as a single message chunk) - from langgraph.config import get_stream_writer - writer = get_stream_writer() - writer({ - "type": "message", # End node output uses message type - "node_id": self.node_id, - "chunk": output, - "full_content": output, - "chunk_index": 1, - "is_suffix": False - }) - logger.info(f"节点 {self.node_id} 已通过 writer 发送完整内容") - - state['messages'].extend([ - { - "role": "user", - "content": self.get_variable("sys.message", state) - }, - { - "role": "assistant", - "content": output - } - ]) - - # yield completion marker - yield {"__final__": True, "result": output} - return - - # Has reference to direct upstream LLM node, only output the part after that reference (suffix) - logger.info( - f"节点 {self.node_id} 检测到直接上游 LLM 节点引用，只输出后缀部分（从索引 {upstream_llm_ref_index + 1} 开始）") - - # Collect suffix parts - suffix_parts = [] - logger.info(f"[后缀调试] 开始收集后缀，从索引 {upstream_llm_ref_index + 1} 到 {len(parts) - 1}") - for i in range(upstream_llm_ref_index + 1, len(parts)): - part = parts[i] - logger.info(f"[后缀调试] 处理 part[{i}]: {part}") - if part["type"] == "static": - # 静态文本 - logger.info(f"[后缀调试] 添加静态文本: '{part['content']}'") - suffix_parts.append(part["content"]) - - elif part["type"] == "dynamic": - # Other dynamic references (if there are multiple references) - node_id = part["node_id"] - field = part["field"] - - # Use VariablePool to get variable value - pool = self.get_variable_pool(state) - try: - # Try to get variable value with default empty string - content = pool.get([node_id, field], default="") - logger.info(f"[后缀调试] 获取变量 {node_id}.{field} 成功: '{content}'") - except Exception as e: - logger.warning(f"[后缀调试] 获取变量 {node_id}.{field} 失败: {e}") - content = "" - - # Convert to string if not None - suffix_parts.append(str(content) if content is not None else "") - - # 拼接后缀 - suffix = "".join(suffix_parts) - - # 构建完整输出（用于返回，包含前缀 + 动态内容 + 后缀） - full_output = self._render_template(output_template, state, strict=False) - - state['messages'].extend([ - { - "role": "user", - "content": self.get_variable("sys.message", state) - }, - { - "role": "assistant", - "content": full_output - } - ]) - - logger.info(f"[后缀调试] 节点 {self.node_id} 后缀部分数量: {len(suffix_parts)}") - logger.info(f"[后缀调试] 后缀内容: '{suffix}'") - logger.info(f"[后缀调试] 后缀长度: {len(suffix)}") - logger.info(f"[后缀调试] 后缀是否为空: {not suffix}") - - if suffix: - logger.info(f"节点 {self.node_id} 输出后缀: '{suffix}...' (长度: {len(suffix)})") - # 一次性输出后缀（作为单个 chunk） - # 注意：不要直接 yield 字符串，因为 base_node 会逐字符处理 - # 而是通过 writer 直接发送 - from langgraph.config import get_stream_writer - writer = get_stream_writer() - writer({ - "type": "message", # End 节点的输出使用 message 类型 - "node_id": self.node_id, - "chunk": suffix, - "full_content": full_output, # full_content 是完整的渲染结果（前缀+LLM+后缀） - "chunk_index": 1, - "is_suffix": True - }) - logger.info(f"节点 {self.node_id} 已通过 writer 发送后缀，full_content 长度: {len(full_output)}") - else: - logger.warning(f"[后缀调试] 节点 {self.node_id} 后缀为空，不发送！" - f"upstream_llm_ref_index={upstream_llm_ref_index}, parts数量={len(parts)}") - - # 统计信息 - node_outputs = state.get("node_outputs", {}) - total_nodes = len(node_outputs) - - logger.info(f"节点 {self.node_id} (End) 执行完成（流式），共执行了 {total_nodes} 个节点") - - # yield 完成标记（包含完整输出） - yield {"__final__": True, "result": full_output} diff --git a/api/app/core/workflow/nodes/if_else/node.py b/api/app/core/workflow/nodes/if_else/node.py index 41f1138b..cf5a1499 100644 --- a/api/app/core/workflow/nodes/if_else/node.py +++ b/api/app/core/workflow/nodes/if_else/node.py @@ -13,7 +13,7 @@ logger = logging.getLogger(__name__) class IfElseNode(BaseNode): def __init__(self, node_config: dict[str, Any], workflow_config: dict[str, Any]): super().__init__(node_config, workflow_config) - self.typed_config: IfElseNodeConfig | None= None + self.typed_config: IfElseNodeConfig | None = None @staticmethod def _evaluate(operator, instance: CompareOperatorInstance) -> Any: diff --git a/api/app/core/workflow/nodes/llm/node.py b/api/app/core/workflow/nodes/llm/node.py index a74e0b60..f315b238 100644 --- a/api/app/core/workflow/nodes/llm/node.py +++ b/api/app/core/workflow/nodes/llm/node.py @@ -7,18 +7,18 @@ LLM 节点实现 import logging import re from typing import Any -from langchain_core.messages import AIMessage, SystemMessage, HumanMessage -from app.core.workflow.nodes.base_node import BaseNode, WorkflowState +from langchain_core.messages import AIMessage + +from app.core.error_codes import BizCode +from app.core.exceptions import BusinessException from app.core.models import RedBearLLM, RedBearModelConfig +from app.core.workflow.nodes.base_node import BaseNode, WorkflowState from app.core.workflow.nodes.llm.config import LLMNodeConfig from app.db import get_db_context from app.models import ModelType from app.services.model_service import ModelConfigService -from app.core.exceptions import BusinessException -from app.core.error_codes import BizCode - logger = logging.getLogger(__name__) @@ -231,42 +231,14 @@ class LLMNode(BaseNode): 文本片段（chunk）或完成标记 """ self.typed_config = LLMNodeConfig(**self.config) - from langgraph.config import get_stream_writer llm, prompt_or_messages = self._prepare_llm(state, True) logger.info(f"节点 {self.node_id} 开始执行 LLM 调用（流式）") logger.debug(f"LLM 配置: streaming={getattr(llm._model, 'streaming', 'unknown')}") - # 检查是否有注入的 End 节点前缀配置 - writer = get_stream_writer() - end_prefix = getattr(self, '_end_node_prefix', None) - - logger.info(f"[LLM前缀] 节点 {self.node_id} 检查前缀配置: {end_prefix is not None}") - if end_prefix: - logger.info(f"[LLM前缀] 前缀内容: '{end_prefix}'") - - if end_prefix: - # 渲染前缀（可能包含其他变量） - try: - rendered_prefix = self._render_template(end_prefix, state) - logger.info(f"节点 {self.node_id} 提前发送 End 节点前缀: '{rendered_prefix[:50]}...'") - - # 提前发送 End 节点的前缀（使用 "message" 类型） - writer({ - "type": "message", # End 相关的内容都是 message 类型 - "node_id": "end", # 标记为 end 节点的输出 - "chunk": rendered_prefix, - "full_content": rendered_prefix, - "chunk_index": 0, - "is_prefix": True # 标记这是前缀 - }) - except Exception as e: - logger.warning(f"渲染/发送 End 节点前缀失败: {e}") - # 累积完整响应 full_response = "" - last_chunk = None chunk_count = 0 # 调用 LLM（流式，支持字符串或消息列表） @@ -284,12 +256,19 @@ class LLMNode(BaseNode): # 只有当内容不为空时才处理 if content: full_response += content - last_chunk = chunk chunk_count += 1 # 流式返回每个文本片段 - yield content + yield { + "__final__": False, + "chunk": content + } + yield { + "__final__": False, + "chunk": "", + "done": True + } logger.info(f"节点 {self.node_id} LLM 调用完成，输出长度: {len(full_response)}, 总 chunks: {chunk_count}") # 构建完整的 AIMessage（包含元数据） diff --git a/api/app/core/workflow/nodes/memory/config.py b/api/app/core/workflow/nodes/memory/config.py index 987230c1..31881e24 100644 --- a/api/app/core/workflow/nodes/memory/config.py +++ b/api/app/core/workflow/nodes/memory/config.py @@ -1,7 +1,6 @@ -import uuid +from uuid import UUID from pydantic import Field -from typing import Literal from app.core.workflow.nodes.base_config import BaseNodeConfig @@ -11,7 +10,7 @@ class MemoryReadNodeConfig(BaseNodeConfig): ... ) - config_id: int = Field( + config_id: UUID | int = Field( ... ) @@ -26,6 +25,6 @@ class MemoryWriteNodeConfig(BaseNodeConfig): ... ) - config_id: int = Field( + config_id: UUID | int = Field( ... ) diff --git a/api/app/core/workflow/nodes/memory/node.py b/api/app/core/workflow/nodes/memory/node.py index 08a2b280..13860bec 100644 --- a/api/app/core/workflow/nodes/memory/node.py +++ b/api/app/core/workflow/nodes/memory/node.py @@ -22,9 +22,9 @@ class MemoryReadNode(BaseNode): raise RuntimeError("End user id is required") return await MemoryAgentService().read_memory( - group_id=end_user_id, + end_user_id=end_user_id, message=self._render_template(self.typed_config.message, state), - config_id=str(self.typed_config.config_id), + config_id=self.typed_config.config_id, search_switch=self.typed_config.search_switch, history=[], db=db, @@ -36,9 +36,10 @@ class MemoryReadNode(BaseNode): class MemoryWriteNode(BaseNode): def __init__(self, node_config: dict[str, Any], workflow_config: dict[str, Any]): super().__init__(node_config, workflow_config) - self.typed_config = MemoryWriteNodeConfig(**self.config) + self.typed_config: MemoryWriteNodeConfig | None = None async def execute(self, state: WorkflowState) -> Any: + self.typed_config = MemoryWriteNodeConfig(**self.config) end_user_id = self.get_variable("sys.user_id", state) if not end_user_id: diff --git a/api/app/core/workflow/nodes/node_factory.py b/api/app/core/workflow/nodes/node_factory.py index 9fca8d7a..fb2fe00f 100644 --- a/api/app/core/workflow/nodes/node_factory.py +++ b/api/app/core/workflow/nodes/node_factory.py @@ -10,6 +10,7 @@ from typing import Any, Union from app.core.workflow.nodes.agent import AgentNode from app.core.workflow.nodes.assigner import AssignerNode from app.core.workflow.nodes.base_node import BaseNode +from app.core.workflow.nodes.code import CodeNode from app.core.workflow.nodes.cycle_graph.node import CycleGraphNode from app.core.workflow.nodes.end import EndNode from app.core.workflow.nodes.enums import NodeType @@ -49,7 +50,8 @@ WorkflowNode = Union[ QuestionClassifierNode, ToolNode, MemoryReadNode, - MemoryWriteNode + MemoryWriteNode, + CodeNode ] @@ -81,6 +83,7 @@ class NodeFactory: NodeType.TOOL: ToolNode, NodeType.MEMORY_READ: MemoryReadNode, NodeType.MEMORY_WRITE: MemoryWriteNode, + NodeType.CODE: CodeNode, } @classmethod diff --git a/api/app/core/workflow/nodes/question_classifier/config.py b/api/app/core/workflow/nodes/question_classifier/config.py index 998e2fb4..2dd8d28a 100644 --- a/api/app/core/workflow/nodes/question_classifier/config.py +++ b/api/app/core/workflow/nodes/question_classifier/config.py @@ -5,6 +5,7 @@ from pydantic import Field, BaseModel from app.core.workflow.nodes.base_config import BaseNodeConfig + class ClassifierConfig(BaseModel): """分类器节点配置""" @@ -13,7 +14,7 @@ class ClassifierConfig(BaseModel): class QuestionClassifierNodeConfig(BaseNodeConfig): """问题分类器节点配置""" - + model_id: uuid.UUID = Field(..., description="LLM模型ID") input_variable: str = Field(default="{{sys.message}}", description="输入变量选择器(用户问题)") user_supplement_prompt: Optional[str] = Field(default=None, description="用户补充提示词，额外分类指令") diff --git a/api/app/core/workflow/nodes/question_classifier/node.py b/api/app/core/workflow/nodes/question_classifier/node.py index aee72eda..6df410cb 100644 --- a/api/app/core/workflow/nodes/question_classifier/node.py +++ b/api/app/core/workflow/nodes/question_classifier/node.py @@ -18,30 +18,30 @@ DEFAULT_EMPTY_QUESTION_CASE = f"{DEFAULT_CASE_PREFIX}1" class QuestionClassifierNode(BaseNode): """问题分类器节点""" - + def __init__(self, node_config: dict[str, Any], workflow_config: dict[str, Any]): super().__init__(node_config, workflow_config) self.typed_config: QuestionClassifierNodeConfig | None = None self.category_to_case_map = {} - + def _get_llm_instance(self) -> RedBearLLM: """获取LLM实例""" with get_db_read() as db: config = ModelConfigService.get_model_by_id(db=db, model_id=self.typed_config.model_id) - + if not config: raise BusinessException("配置的模型不存在", BizCode.NOT_FOUND) - + if not config.api_keys or len(config.api_keys) == 0: raise BusinessException("模型配置缺少 API Key", BizCode.INVALID_PARAMETER) - + api_config = config.api_keys[0] model_name = api_config.model_name provider = api_config.provider api_key = api_config.api_key base_url = api_config.api_base model_type = config.type - + return RedBearLLM( RedBearModelConfig( model_name=model_name, @@ -64,7 +64,7 @@ class QuestionClassifierNode(BaseNode): case_tag = f"{DEFAULT_CASE_PREFIX}{idx}" category_map[category_name] = case_tag return category_map - + async def execute(self, state: WorkflowState) -> dict: """执行问题分类""" self.typed_config = QuestionClassifierNodeConfig(**self.config) @@ -74,11 +74,12 @@ class QuestionClassifierNode(BaseNode): categories = self.typed_config.categories or [] category_names = [class_item.class_name.strip() for class_item in categories] category_count = len(category_names) - + if not question: logger.warning( f"节点 {self.node_id} 未获取到输入问题，使用默认分支" - f"（默认分支：{DEFAULT_EMPTY_QUESTION_CASE}，分类总数：{category_count}）" + f"(默认分支:{DEFAULT_EMPTY_QUESTION_CASE}" + f"分类总数: {category_count})" ) # 若分类列表为空，返回默认unknown分支，否则返回CASE1 if category_count > 0: diff --git a/api/app/core/workflow/nodes/tool/__init__.py b/api/app/core/workflow/nodes/tool/__init__.py index 8392f05c..a311139e 100644 --- a/api/app/core/workflow/nodes/tool/__init__.py +++ b/api/app/core/workflow/nodes/tool/__init__.py @@ -1,4 +1,4 @@ from app.core.workflow.nodes.tool.config import ToolNodeConfig from app.core.workflow.nodes.tool.node import ToolNode -__all__ = ["ToolNode", "ToolNodeConfig"] \ No newline at end of file +__all__ = ["ToolNode", "ToolNodeConfig"] diff --git a/api/app/core/workflow/nodes/tool/node.py b/api/app/core/workflow/nodes/tool/node.py index 3e79b075..aba96303 100644 --- a/api/app/core/workflow/nodes/tool/node.py +++ b/api/app/core/workflow/nodes/tool/node.py @@ -16,11 +16,11 @@ TEMPLATE_PATTERN = re.compile(r"\{\{.*?\}\}") class ToolNode(BaseNode): """工具节点""" - + def __init__(self, node_config: dict[str, Any], workflow_config: dict[str, Any]): super().__init__(node_config, workflow_config) self.typed_config: ToolNodeConfig | None = None - + async def execute(self, state: WorkflowState) -> dict[str, Any]: """执行工具""" self.typed_config = ToolNodeConfig(**self.config) @@ -28,21 +28,21 @@ class ToolNode(BaseNode): tenant_id = self.get_variable("sys.tenant_id", state) user_id = self.get_variable("sys.user_id", state) workspace_id = self.get_variable("sys.workspace_id", state) - + # 如果没有租户ID，尝试从工作流ID获取 if not tenant_id: if workspace_id: from app.repositories.tool_repository import ToolRepository with get_db_read() as db: tenant_id = ToolRepository.get_tenant_id_by_workspace_id(db, workspace_id) - + if not tenant_id: logger.error(f"节点 {self.node_id} 缺少租户ID") return { "success": False, "data": "缺少租户ID" } - + # 渲染工具参数 rendered_parameters = {} for param_name, param_template in self.typed_config.tool_parameters.items(): @@ -55,9 +55,9 @@ class ToolNode(BaseNode): # 非模板参数（数字/布尔/普通字符串）直接保留原值 rendered_value = param_template rendered_parameters[param_name] = rendered_value - + logger.info(f"节点 {self.node_id} 执行工具 {self.typed_config.tool_id}，参数: {rendered_parameters}") - + # 执行工具 with get_db_read() as db: tool_service = ToolService(db) @@ -79,7 +79,7 @@ class ToolNode(BaseNode): else: logger.error(f"节点 {self.node_id} 工具执行失败: {result.error}") return { - "data": result.error if isinstance(result.error, str) else json.dumps(result.error, ensure_ascii=False), + "data": result.error if isinstance(result.error, str) else json.dumps(result.error, ensure_ascii=False), "error_code": result.error_code, "execution_time": result.execution_time - } \ No newline at end of file + } diff --git a/api/app/main.py b/api/app/main.py index 87bfecf8..7e16d2c0 100644 --- a/api/app/main.py +++ b/api/app/main.py @@ -16,6 +16,8 @@ from app.core.error_codes import BizCode, HTTP_MAPPING from app.core.exceptions import BusinessException from app.core.logging_config import LoggingConfig, get_logger from app.core.response_utils import fail +from app.core.models.scripts.loader import load_models +from app.db import get_db_context # Initialize logging system LoggingConfig.setup_logging() @@ -47,6 +49,15 @@ async def lifespan(app: FastAPI): else: logger.info("自动数据库升级已禁用 (DB_AUTO_UPGRADE=false)") + # 加载预定义模型 + logger.info("开始加载预定义模型...") + try: + with get_db_context() as db: + result = load_models(db, silent=True) + logger.info(f"预定义模型加载完成: 成功{result['success']}个, 跳过{result['skipped']}个, 失败{result['failed']}个") + except Exception as e: + logger.warning(f"加载预定义模型时出错: {str(e)}") + logger.info("应用程序启动完成") yield # 应用关闭事件 diff --git a/api/app/models/__init__.py b/api/app/models/__init__.py index bf3a1b3d..a429dd8e 100644 --- a/api/app/models/__init__.py +++ b/api/app/models/__init__.py @@ -6,7 +6,7 @@ from .document_model import Document from .file_model import File from .file_metadata_model import FileMetadata from .generic_file_model import GenericFile -from .models_model import ModelConfig, ModelProvider, ModelType, ModelApiKey +from .models_model import ModelConfig, ModelProvider, ModelType, ModelApiKey, ModelBase, LoadBalanceStrategy from .memory_short_model import ShortTermMemory, LongTermMemory from .knowledgeshare_model import KnowledgeShare from .app_model import App @@ -18,7 +18,7 @@ from .appshare_model import AppShare from .release_share_model import ReleaseShare from .conversation_model import Conversation, Message from .api_key_model import ApiKey, ApiKeyLog, ApiKeyType -from .data_config_model import DataConfig +from .memory_config_model import MemoryConfig from .multi_agent_model import MultiAgentConfig, AgentInvocation from .workflow_model import WorkflowConfig, WorkflowExecution, WorkflowNodeExecution from .retrieval_info import RetrievalInfo @@ -57,7 +57,7 @@ __all__ = [ "ApiKey", "ApiKeyLog", "ApiKeyType", - "DataConfig", + "MemoryConfig", "MultiAgentConfig", "AgentInvocation", "WorkflowConfig", @@ -79,4 +79,6 @@ __all__ = [ "AuthType", "ExecutionStatus", "MemoryPerceptualModel", + "ModelBase", + "LoadBalanceStrategy" ] diff --git a/api/app/models/agent_app_config_model.py b/api/app/models/agent_app_config_model.py index 0a7a5935..96752c8e 100644 --- a/api/app/models/agent_app_config_model.py +++ b/api/app/models/agent_app_config_model.py @@ -6,7 +6,7 @@ from sqlalchemy.orm import relationship from app.base.type import PydanticType from app.db import Base -from app.schemas import ModelParameters +from app.schemas.app_schema import ModelParameters class AgentConfig(Base): diff --git a/api/app/models/data_config_model.py b/api/app/models/data_config_model.py deleted file mode 100644 index 06f87cb2..00000000 --- a/api/app/models/data_config_model.py +++ /dev/null @@ -1,88 +0,0 @@ -import datetime -from sqlalchemy import Column, String, Boolean, DateTime, Integer, Float -from sqlalchemy.dialects.postgresql import UUID -from app.db import Base - - -class DataConfig(Base): - """数据配置表 - 用于存储记忆系统的配置参数""" - __tablename__ = "data_config" - - # 主键 - config_id = Column(Integer, primary_key=True, autoincrement=True, comment="配置ID") - - # 基本信息 - config_name = Column(String, nullable=False, comment="配置名称") - config_desc = Column(String, nullable=True, comment="配置描述") - - # 组织信息 - workspace_id = Column(UUID(as_uuid=True), nullable=True, comment="工作空间ID") - group_id = Column(String, nullable=True, comment="组ID") - user_id = Column(String, nullable=True, comment="用户ID") - apply_id = Column(String, nullable=True, comment="应用ID") - - # 模型选择（从workspace继承） - llm_id = Column(String, nullable=True, comment="LLM模型配置ID") - embedding_id = Column(String, nullable=True, comment="嵌入模型配置ID") - rerank_id = Column(String, nullable=True, comment="重排序模型配置ID") - - # 记忆萃取引擎配置 - enable_llm_dedup_blockwise = Column(Boolean, default=True, comment="启用LLM决策去重") - enable_llm_disambiguation = Column(Boolean, default=True, comment="启用LLM决策消歧") - deep_retrieval = Column(Boolean, default=True, comment="深度检索开关") - - # 阈值配置 (0-1 之间的浮点数) - t_type_strict = Column(Float, default=0.8, comment="类型严格阈值") - t_name_strict = Column(Float, default=0.8, comment="名称严格阈值") - t_overall = Column(Float, default=0.8, comment="综合阈值") - - # 状态配置 - state = Column(Boolean, default=False, comment="配置使用状态") - - # 分块策略 - chunker_strategy = Column(String, default="RecursiveChunker", comment="分块策略") - - # 剪枝配置 - pruning_enabled = Column(Boolean, default=False, comment="是否启动智能语义剪枝") - pruning_scene = Column(String, nullable=True, comment="智能剪枝场景：education/online_service/outbound") - pruning_threshold = Column(Float, nullable=True, comment="智能语义剪枝阈值（0-0.9）") - - # 自我反思配置 - enable_self_reflexion = Column(Boolean, default=False, comment="是否启用自我反思") - iteration_period = Column(String, default="3", comment="反思迭代周期") - reflexion_range = Column(String, default="partial", comment="反思范围：部分/全部") - baseline = Column(String, default="TIME", comment="基线：时间/事实/时间和事实") - reflection_model_id = Column(String, nullable=True, comment="反思模型ID") - memory_verify = Column(Boolean, default=True, comment="记忆验证") - quality_assessment = Column(Boolean, default=True, comment="质量评估") - - # 遗忘引擎配置 - statement_granularity = Column(Integer, default=2, comment="陈述提取颗粒度，挡位 1/2/3") - include_dialogue_context = Column(Boolean, default=False, comment="是否包含对话上下文") - max_context = Column(Integer, default=1000, comment="对话语境中包含字符的最大数量") - lambda_time = Column("lambda_time", Float, default=0.5, comment="最低保持度，0-1 小数") - lambda_mem = Column("lambda_mem", Float, default=0.5, comment="遗忘率，0-1 小数") - offset = Column("offset", Float, default=0.0, comment="偏移度，0-1 小数") - - # ACT-R 遗忘引擎配置 - decay_constant = Column(Float, default=0.5, comment="ACT-R衰减常数d，默认0.5") - forgetting_threshold = Column(Float, default=0.3, comment="遗忘阈值，默认0.3") - forgetting_interval_hours = Column(Integer, default=24, comment="遗忘周期间隔（小时），默认24") - enable_llm_summary = Column(Boolean, default=True, comment="是否使用LLM生成摘要，默认True") - max_merge_batch_size = Column(Integer, default=100, comment="单次最大融合节点对数，默认100") - max_history_length = Column(Integer, default=100, comment="访问历史最大长度，默认100") - min_days_since_access = Column(Integer, default=30, comment="最小未访问天数，默认30") - - # 情绪引擎配置 - emotion_enabled = Column(Boolean, default=True, comment="是否启用情绪提取") - emotion_model_id = Column(String, nullable=True, comment="情绪分析专用模型ID") - emotion_extract_keywords = Column(Boolean, default=True, comment="是否提取情绪关键词") - emotion_min_intensity = Column(Float, default=0.1, comment="最小情绪强度阈值") - emotion_enable_subject = Column(Boolean, default=True, comment="是否启用主体分类") - - # 时间戳 - created_at = Column(DateTime, default=datetime.datetime.now, comment="创建时间") - updated_at = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now, comment="更新时间") - - def __repr__(self): - return f"" diff --git a/api/app/models/memory_config_model.py b/api/app/models/memory_config_model.py index d47c3b52..454b1b48 100644 --- a/api/app/models/memory_config_model.py +++ b/api/app/models/memory_config_model.py @@ -1,39 +1,88 @@ -# -*- coding: utf-8 -*- -"""Memory Configuration Model - Backward Compatibility +import datetime +from sqlalchemy import Column, String, Boolean, DateTime, Integer, Float +from sqlalchemy.dialects.postgresql import UUID +from app.db import Base -This module provides backward compatibility for imports. -All classes have been moved to app.schemas.memory_config_schema. -DEPRECATED: Import from app.schemas.memory_config_schema instead. -""" +class MemoryConfig(Base): + """记忆配置表 - 用于存储记忆系统的配置参数""" + __tablename__ = "memory_config" -# Re-export for backward compatibility -from app.schemas.memory_config_schema import ( - ConfigurationError, - InvalidConfigError, - MemoryConfig, - MemoryConfigValidation, - ModelInactiveError, - ModelNotFoundError, - ModelValidation, - WorkspaceNotFoundError, - WorkspaceValidation, - validate_memory_config_data, - validate_model_data, - validate_workspace_data, -) + # 主键 + config_id = Column(UUID(as_uuid=True), primary_key=True, comment="配置ID") + config_id_old = Column(Integer, nullable=True, comment="备份的配置ID") + # 基本信息 + config_name = Column(String, nullable=False, comment="配置名称") + config_desc = Column(String, nullable=True, comment="配置描述") -__all__ = [ - "ConfigurationError", - "InvalidConfigError", - "MemoryConfig", - "MemoryConfigValidation", - "ModelInactiveError", - "ModelNotFoundError", - "ModelValidation", - "WorkspaceNotFoundError", - "WorkspaceValidation", - "validate_memory_config_data", - "validate_model_data", - "validate_workspace_data", -] + # 组织信息 + workspace_id = Column(UUID(as_uuid=True), nullable=True, comment="工作空间ID") + end_user_id = Column(String, nullable=True, comment="组ID") + user_id = Column(String, nullable=True, comment="用户ID") + apply_id = Column(String, nullable=True, comment="应用ID") + + # 模型选择（从workspace继承） + llm_id = Column(String, nullable=True, comment="LLM模型配置ID") + embedding_id = Column(String, nullable=True, comment="嵌入模型配置ID") + rerank_id = Column(String, nullable=True, comment="重排序模型配置ID") + + # 记忆萃取引擎配置 + enable_llm_dedup_blockwise = Column(Boolean, default=True, comment="启用LLM决策去重") + enable_llm_disambiguation = Column(Boolean, default=True, comment="启用LLM决策消歧") + deep_retrieval = Column(Boolean, default=True, comment="深度检索开关") + + # 阈值配置 (0-1 之间的浮点数) + t_type_strict = Column(Float, default=0.8, comment="类型严格阈值") + t_name_strict = Column(Float, default=0.8, comment="名称严格阈值") + t_overall = Column(Float, default=0.8, comment="综合阈值") + + # 状态配置 + state = Column(Boolean, default=False, comment="配置使用状态") + + # 分块策略 + chunker_strategy = Column(String, default="RecursiveChunker", comment="分块策略") + + # 剪枝配置 + pruning_enabled = Column(Boolean, default=False, comment="是否启动智能语义剪枝") + pruning_scene = Column(String, nullable=True, comment="智能剪枝场景：education/online_service/outbound") + pruning_threshold = Column(Float, nullable=True, comment="智能语义剪枝阈值（0-0.9）") + + # 自我反思配置 + enable_self_reflexion = Column(Boolean, default=False, comment="是否启用自我反思") + iteration_period = Column(String, default="3", comment="反思迭代周期") + reflexion_range = Column(String, default="partial", comment="反思范围：部分/全部") + baseline = Column(String, default="TIME", comment="基线：时间/事实/时间和事实") + reflection_model_id = Column(String, nullable=True, comment="反思模型ID") + memory_verify = Column(Boolean, default=True, comment="记忆验证") + quality_assessment = Column(Boolean, default=True, comment="质量评估") + + # 遗忘引擎配置 + statement_granularity = Column(Integer, default=2, comment="陈述提取颗粒度，挡位 1/2/3") + include_dialogue_context = Column(Boolean, default=False, comment="是否包含对话上下文") + max_context = Column(Integer, default=1000, comment="对话语境中包含字符的最大数量") + lambda_time = Column("lambda_time", Float, default=0.5, comment="最低保持度，0-1 小数") + lambda_mem = Column("lambda_mem", Float, default=0.5, comment="遗忘率，0-1 小数") + offset = Column("offset", Float, default=0.0, comment="偏移度，0-1 小数") + + # ACT-R 遗忘引擎配置 + decay_constant = Column(Float, default=0.5, comment="ACT-R衰减常数d，默认0.5") + forgetting_threshold = Column(Float, default=0.3, comment="遗忘阈值，默认0.3") + forgetting_interval_hours = Column(Integer, default=24, comment="遗忘周期间隔（小时），默认24") + enable_llm_summary = Column(Boolean, default=True, comment="是否使用LLM生成摘要，默认True") + max_merge_batch_size = Column(Integer, default=100, comment="单次最大融合节点对数，默认100") + max_history_length = Column(Integer, default=100, comment="访问历史最大长度，默认100") + min_days_since_access = Column(Integer, default=30, comment="最小未访问天数，默认30") + + # 情绪引擎配置 + emotion_enabled = Column(Boolean, default=True, comment="是否启用情绪提取") + emotion_model_id = Column(String, nullable=True, comment="情绪分析专用模型ID") + emotion_extract_keywords = Column(Boolean, default=True, comment="是否提取情绪关键词") + emotion_min_intensity = Column(Float, default=0.1, comment="最小情绪强度阈值") + emotion_enable_subject = Column(Boolean, default=True, comment="是否启用主体分类") + + # 时间戳 + created_at = Column(DateTime, default=datetime.datetime.now, comment="创建时间") + updated_at = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now, comment="更新时间") + + def __repr__(self): + return f"" diff --git a/api/app/models/memory_perceptual_model.py b/api/app/models/memory_perceptual_model.py index 59eb0222..cafb18d4 100644 --- a/api/app/models/memory_perceptual_model.py +++ b/api/app/models/memory_perceptual_model.py @@ -16,7 +16,7 @@ class PerceptualType(IntEnum): CONVERSATION = 4 -class FileStorageType(IntEnum): +class FileStorageService(IntEnum): LOCAL = 1 REMOTE = 2 diff --git a/api/app/models/models_model.py b/api/app/models/models_model.py index 2e60ef1c..3e378f17 100644 --- a/api/app/models/models_model.py +++ b/api/app/models/models_model.py @@ -1,19 +1,34 @@ import datetime import uuid from enum import StrEnum -from typing import Optional, List -from sqlalchemy import Column, String, Boolean, DateTime, Text, ForeignKey, Enum as SQLEnum + +from sqlalchemy import Column, String, Boolean, DateTime, Text, ForeignKey, Enum as SQLEnum, UniqueConstraint, Integer, ARRAY, Table from sqlalchemy.dialects.postgresql import UUID, JSON from sqlalchemy.orm import relationship +from sqlalchemy.sql import func from app.db import Base +class BaseModel(Base): + """基础模型（抽象类，提取公共字段）""" + __abstract__ = True # 标记为抽象类，不生成表 + id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4, index=True) + created_at = Column(DateTime, default=datetime.datetime.now, comment="创建时间") + updated_at = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now, comment="更新时间") + is_active = Column(Boolean, default=True, nullable=False, comment="是否激活") + + class ModelType(StrEnum): """模型类型枚举""" LLM = "llm" CHAT = "chat" EMBEDDING = "embedding" RERANK = "rerank" + # TTS = "tts" + # SPEECH2TEXT = "speech2text" + # IMAGE = "image" + # AUDIO = "audio" + # VISION = "vision" class ModelProvider(StrEnum): @@ -30,16 +45,36 @@ class ModelProvider(StrEnum): XINFERENCE = "xinference" GPUSTACK = "gpustack" BEDROCK = "bedrock" + COMPOSITE = "composite" -class ModelConfig(Base): +class LoadBalanceStrategy(StrEnum): + """API Key负载均衡策略枚举""" + ROUND_ROBIN = "round_robin" # 轮询 + NONE = "none" # 无 + + +# 多对多关联表 +model_config_api_key_association = Table( + 'model_config_api_key_association', + Base.metadata, + Column('model_config_id', UUID(as_uuid=True), ForeignKey('model_configs.id'), primary_key=True), + Column('api_key_id', UUID(as_uuid=True), ForeignKey('model_api_keys.id'), primary_key=True), + Column('created_at', DateTime, default=datetime.datetime.now) +) + + +class ModelConfig(BaseModel): """模型配置表""" __tablename__ = "model_configs" - id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4, index=True) + model_id = Column(UUID(as_uuid=True), ForeignKey("model_bases.id"), nullable=True, index=True, comment="基础模型ID") tenant_id = Column(UUID(as_uuid=True), ForeignKey("tenants.id"), nullable=False, index=True, comment="租户ID") + logo = Column(String(255), nullable=True, comment="模型logo图片URL") name = Column(String, nullable=False, comment="模型显示名称") + provider = Column(String, nullable=False, comment="供应商", server_default=ModelProvider.COMPOSITE) type = Column(String, nullable=False, index=True, comment="模型类型") + is_composite = Column(Boolean, default=False, server_default="true", nullable=False, comment="是否为组合模型") description = Column(String, comment="模型描述") # 模型配置参数 @@ -56,29 +91,29 @@ class ModelConfig(Base): # context_length = Column(String, comment="上下文长度") # 状态管理 - is_active = Column(Boolean, default=True, nullable=False, comment="是否激活") is_public = Column(Boolean, default=False, nullable=False, comment="是否公开") - - # 时间戳 - created_at = Column(DateTime, default=datetime.datetime.now, comment="创建时间") - updated_at = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now, comment="更新时间") + load_balance_strategy = Column(String, nullable=True, comment="负载均衡策略", default=LoadBalanceStrategy.NONE, + server_default=LoadBalanceStrategy.NONE) # 关联关系 - api_keys = relationship("ModelApiKey", back_populates="model_config", cascade="all, delete-orphan") + model_base = relationship("ModelBase", back_populates="configs") + api_keys = relationship( + "ModelApiKey", + secondary=model_config_api_key_association, + back_populates="model_configs" + ) def __repr__(self): return f"" -class ModelApiKey(Base): +class ModelApiKey(BaseModel): """模型API密钥表""" __tablename__ = "model_api_keys" - - id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4, index=True) - model_config_id = Column(UUID(as_uuid=True), ForeignKey("model_configs.id"), nullable=False, comment="模型配置ID") # API Key 信息 model_name = Column(String, nullable=False, comment="模型实际名称") + description = Column(String, comment="备注") provider = Column(String, nullable=False, comment="API Key提供商") api_key = Column(String, nullable=False, comment="API密钥") api_base = Column(String, comment="API基础URL") @@ -91,15 +126,42 @@ class ModelApiKey(Base): last_used_at = Column(DateTime, comment="最后使用时间") # 状态管理 - is_active = Column(Boolean, default=True, nullable=False, comment="是否激活") priority = Column(String, default="1", comment="优先级") - - # 时间戳 - created_at = Column(DateTime, default=datetime.datetime.now, comment="创建时间") - updated_at = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now, comment="更新时间") - + # 关联关系 - model_config = relationship("ModelConfig", back_populates="api_keys") + model_configs = relationship( + "ModelConfig", + secondary=model_config_api_key_association, + back_populates="api_keys" + ) + def __repr__(self): - return f"" + return f"" + + +class ModelBase(Base): + """基础模型信息表（模型广场）""" + __tablename__ = "model_bases" + + id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4, index=True) + logo = Column(String(255), nullable=True, comment="模型logo图片URL") + name = Column(String, nullable=False, comment="模型唯一标识（如gpt-3.5-turbo）") + type = Column(String, nullable=False, index=True, comment="模型类型") + provider = Column(String, nullable=False, index=True) + description = Column(Text, comment="模型描述") + is_deprecated = Column(Boolean, default=False, nullable=False, comment="是否弃用") + is_official = Column(Boolean, default=True, comment="是否供应商官方模型（区分自定义）") + tags = Column(ARRAY(String), default=list, nullable=False, comment="模型标签（如['聊天', '创作']）") + add_count = Column(Integer, default=0, nullable=False, comment="模型被用户添加的次数") + created_at = Column(DateTime, default=datetime.datetime.now, comment="创建时间", server_default=func.now()) + + # 关联关系 + configs = relationship("ModelConfig", back_populates="model_base", cascade="all, delete-orphan") + + __table_args__ = ( + UniqueConstraint("name", "provider", name="uk_model_name_provider"), + ) + + def __repr__(self): + return f"" \ No newline at end of file diff --git a/api/app/models/multi_agent_model.py b/api/app/models/multi_agent_model.py index 544ddb27..400c05ad 100644 --- a/api/app/models/multi_agent_model.py +++ b/api/app/models/multi_agent_model.py @@ -10,7 +10,7 @@ from sqlalchemy.orm import relationship from app.base.type import PydanticType from app.db import Base -from app.schemas import ModelParameters +from app.schemas.app_schema import ModelParameters class OrchestrationMode(StrEnum): diff --git a/api/app/models/tenant_model.py b/api/app/models/tenant_model.py index 552e87b5..54a3e347 100644 --- a/api/app/models/tenant_model.py +++ b/api/app/models/tenant_model.py @@ -16,6 +16,10 @@ class Tenants(Base): updated_at = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now) is_active = Column(Boolean, default=True) + # SSO 外部关联字段 + external_id = Column(String(100), nullable=True, index=True) # 外部企业ID + external_source = Column(String(50), nullable=True) # 来源系统 + # Relationship to users - one tenant has many users users = relationship("User", back_populates="tenant") diff --git a/api/app/models/user_model.py b/api/app/models/user_model.py index 89971a3a..663bfc71 100644 --- a/api/app/models/user_model.py +++ b/api/app/models/user_model.py @@ -18,6 +18,10 @@ class User(Base): updated_at = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now) last_login_at = Column(DateTime, nullable=True) # 最后登录时间，可为空 + # SSO 外部关联字段 + external_id = Column(String(100), nullable=True) # 外部用户ID + external_source = Column(String(50), nullable=True) # 来源系统 + current_workspace_id = Column(UUID(as_uuid=True), ForeignKey("workspaces.id"), nullable=True) # 当前工作空间ID，可为空 # Foreign key to tenant - each user belongs to exactly one tenant diff --git a/api/app/plugins/__init__.py b/api/app/plugins/__init__.py new file mode 100644 index 00000000..e9ef92fd --- /dev/null +++ b/api/app/plugins/__init__.py @@ -0,0 +1,74 @@ +# app/plugins/__init__.py +""" +插件系统 - 支持开源核心 + 闭源增值模块 + +使用方式： +1. 开源版（community）：基础功能 +2. 商业版（enterprise）：加载 premium 包中的高级实现 +""" +import os +from typing import Dict, Any, Optional +from app.core.logging_config import get_logger + +logger = get_logger(__name__) + +# 版本标识 +EDITION = os.environ.get("EDITION", "community") +IS_ENTERPRISE = EDITION == "enterprise" + +# 插件注册表 +_plugins: Dict[str, Any] = {} + +# 路由注册表（用于动态注册闭源模块的路由） +_routers: list = [] + + +def is_enterprise() -> bool: + """是否为商业版""" + return IS_ENTERPRISE + + +def list_plugins() -> list: + """列出所有已注册插件""" + return list(_plugins.keys()) + + +def register_plugin(name: str, instance: Any): + """注册插件""" + _plugins[name] = instance + logger.info(f"插件已注册: {name}") + + +def get_plugin(name: str) -> Optional[Any]: + """获取插件实例""" + return _plugins.get(name) + + +def register_router(router, prefix: str = "", tags: list = None): + """注册路由（供闭源模块使用）""" + _routers.append({ + "router": router, + "prefix": prefix, + "tags": tags or [] + }) + logger.info(f"路由已注册: {prefix}") + + +def get_registered_routers() -> list: + """获取所有注册的路由""" + return _routers + + +def register_premium_routers(app): + """ + 注册 premium 模块的路由到 FastAPI app + + 在商业版 main.py 中调用 + """ + for router_info in _routers: + app.include_router( + router_info["router"], + prefix=f"/api{router_info['prefix']}", + tags=router_info["tags"] + ) + logger.info(f"Premium 路由已挂载: /api{router_info['prefix']}") diff --git a/api/app/repositories/app_repository.py b/api/app/repositories/app_repository.py index 11a2ea3e..0c7ba6a4 100644 --- a/api/app/repositories/app_repository.py +++ b/api/app/repositories/app_repository.py @@ -15,9 +15,13 @@ class AppRepository: self.db = db def get_apps_by_workspace_id(self, workspace_id: uuid.UUID) -> list[App]: - """根据工作空间ID查询应用""" + """根据工作空间ID查询应用（仅返回未删除的应用）""" try: - apps = self.db.query(App).filter(App.workspace_id == workspace_id).all() + apps = ( + self.db.query(App) + .filter(App.workspace_id == workspace_id, App.is_active.is_(True)) + .all() + ) db_logger.info(f"成功查询工作空间 {workspace_id} 下的 {len(apps)} 个应用") return apps except Exception as e: @@ -26,7 +30,7 @@ class AppRepository: def get_apps_by_id(self, app_id: uuid.UUID) -> App: try: - app = self.db.query(App).filter(App.id == app_id, App.is_active == True).first() + app = self.db.query(App).filter(App.id == app_id, App.is_active.is_(True)).first() return app except Exception as e: raise diff --git a/api/app/repositories/home_page_repository.py b/api/app/repositories/home_page_repository.py index 888071ac..bcb3b622 100644 --- a/api/app/repositories/home_page_repository.py +++ b/api/app/repositories/home_page_repository.py @@ -17,24 +17,24 @@ class HomePageRepository: """获取模型统计数据""" total_models = db.query(ModelConfig).filter( ModelConfig.tenant_id == tenant_id, - ModelConfig.is_active == True + ModelConfig.is_active.is_(True) ).count() total_llm = db.query(ModelConfig).filter( ModelConfig.tenant_id == tenant_id, - ModelConfig.is_active == True, + ModelConfig.is_active.is_(True), ModelConfig.type == "llm" ).count() total_embedding = db.query(ModelConfig).filter( ModelConfig.tenant_id == tenant_id, - ModelConfig.is_active == True, + ModelConfig.is_active.is_(True), ModelConfig.type == "embedding" ).count() new_models_this_week = db.query(ModelConfig).filter( ModelConfig.tenant_id == tenant_id, - ModelConfig.is_active == True, + ModelConfig.is_active.is_(True), ModelConfig.created_at >= week_start ).count() @@ -56,12 +56,12 @@ class HomePageRepository: """获取工作空间统计数据""" active_workspaces = db.query(Workspace).filter( Workspace.tenant_id == tenant_id, - Workspace.is_active == True + Workspace.is_active.is_(True) ).count() new_workspaces_this_week = db.query(Workspace).filter( Workspace.tenant_id == tenant_id, - Workspace.is_active == True, + Workspace.is_active.is_(True), Workspace.created_at >= week_start ).count() @@ -83,7 +83,7 @@ class HomePageRepository: """获取用户统计数据""" workspace_ids = db.query(Workspace.id).filter( Workspace.tenant_id == tenant_id, - Workspace.is_active == True + Workspace.is_active.is_(True) ).subquery() total_users = db.query(EndUser).join( @@ -91,7 +91,7 @@ class HomePageRepository: EndUser.app_id == App.id ).filter( App.workspace_id.in_(workspace_ids), - App.is_active == True, + App.is_active.is_(True), App.status == "active" ).count() @@ -100,7 +100,7 @@ class HomePageRepository: EndUser.app_id == App.id ).filter( App.workspace_id.in_(workspace_ids), - App.is_active == True, + App.is_active.is_(True), App.status == "active", EndUser.created_at >= week_start ).count() @@ -123,18 +123,18 @@ class HomePageRepository: """获取应用统计数据""" workspace_ids = db.query(Workspace.id).filter( Workspace.tenant_id == tenant_id, - Workspace.is_active == True + Workspace.is_active.is_(True) ).subquery() running_apps = db.query(App).filter( App.workspace_id.in_(workspace_ids), - App.is_active == True, + App.is_active.is_(True), App.status == "active" ).count() new_apps_this_week = db.query(App).filter( App.workspace_id.in_(workspace_ids), - App.is_active == True, + App.is_active.is_(True), App.status == "active", App.created_at >= week_start ).count() @@ -158,7 +158,7 @@ class HomePageRepository: # 获取工作空间列表 workspaces = db.query(Workspace).filter( Workspace.tenant_id == tenant_id, - Workspace.is_active == True + Workspace.is_active.is_(True) ).all() workspace_ids = [ws.id for ws in workspaces] @@ -169,7 +169,7 @@ class HomePageRepository: func.count(App.id).label('count') ).filter( App.workspace_id.in_(workspace_ids), - App.is_active, + App.is_active.is_(True), App.status == "active" ).group_by(App.workspace_id).all() @@ -184,7 +184,7 @@ class HomePageRepository: EndUser.app_id == App.id ).filter( App.workspace_id.in_(workspace_ids), - App.is_active, + App.is_active.is_(True), App.status == "active" ).group_by(App.workspace_id).all() diff --git a/api/app/repositories/data_config_repository.py b/api/app/repositories/memory_config_repository.py similarity index 72% rename from api/app/repositories/data_config_repository.py rename to api/app/repositories/memory_config_repository.py index 3df7f800..fbc04f2e 100644 --- a/api/app/repositories/data_config_repository.py +++ b/api/app/repositories/memory_config_repository.py @@ -1,18 +1,19 @@ # -*- coding: utf-8 -*- -"""数据配置Repository模块 +"""记忆配置Repository模块 -本模块提供data_config表的数据访问层，使用SQLAlchemy ORM进行数据库操作。 +本模块提供memory_config表的数据访问层，使用SQLAlchemy ORM进行数据库操作。 包括CRUD操作和Neo4j Cypher查询常量。 Classes: - DataConfigRepository: 数据配置仓储类，提供CRUD操作 + MemoryConfigRepository: 记忆配置仓储类，提供CRUD操作 """ import uuid +from uuid import UUID from typing import Dict, List, Optional, Tuple from app.core.exceptions import BusinessException from app.core.logging_config import get_config_logger, get_db_logger -from app.models.data_config_model import DataConfig +from app.models.memory_config_model import MemoryConfig from app.schemas.memory_storage_schema import ( ConfigKey, ConfigParamsCreate, @@ -23,16 +24,18 @@ from app.schemas.memory_storage_schema import ( from sqlalchemy import desc, select from sqlalchemy.orm import Session +from app.utils.config_utils import resolve_config_id + # 获取数据库专用日志器 db_logger = get_db_logger() # 获取配置专用日志器 config_logger = get_config_logger() -TABLE_NAME = "data_config" -class DataConfigRepository: - """数据配置Repository +TABLE_NAME = "memory_config" +class MemoryConfigRepository: + """记忆配置Repository - 提供data_config表的数据访问方法，包括： + 提供memory_config表的数据访问方法，包括： - SQLAlchemy ORM 数据库操作 - Neo4j Cypher查询常量 """ @@ -41,48 +44,48 @@ class DataConfigRepository: # Dialogue count by group SEARCH_FOR_DIALOGUE = """ - MATCH (n:Dialogue) WHERE n.group_id = $group_id RETURN COUNT(n) AS num + MATCH (n:Dialogue) WHERE n.end_user_id = $end_user_id RETURN COUNT(n) AS num """ # Chunk count by group SEARCH_FOR_CHUNK = """ - MATCH (n:Chunk) WHERE n.group_id = $group_id RETURN COUNT(n) AS num + MATCH (n:Chunk) WHERE n.end_user_id = $end_user_id RETURN COUNT(n) AS num """ # Statement count by group SEARCH_FOR_STATEMENT = """ - MATCH (n:Statement) WHERE n.group_id = $group_id RETURN COUNT(n) AS num + MATCH (n:Statement) WHERE n.end_user_id = $end_user_id RETURN COUNT(n) AS num """ # ExtractedEntity count by group SEARCH_FOR_ENTITY = """ - MATCH (n:ExtractedEntity) WHERE n.group_id = $group_id RETURN COUNT(n) AS num + MATCH (n:ExtractedEntity) WHERE n.end_user_id = $end_user_id RETURN COUNT(n) AS num """ # All counts by label and total SEARCH_FOR_ALL = """ - OPTIONAL MATCH (n:Dialogue) WHERE n.group_id = $group_id RETURN 'Dialogue' AS Label, COUNT(n) AS Count + OPTIONAL MATCH (n:Dialogue) WHERE n.end_user_id = $end_user_id RETURN 'Dialogue' AS Label, COUNT(n) AS Count UNION ALL - OPTIONAL MATCH (n:Chunk) WHERE n.group_id = $group_id RETURN 'Chunk' AS Label, COUNT(n) AS Count + OPTIONAL MATCH (n:Chunk) WHERE n.end_user_id = $end_user_id RETURN 'Chunk' AS Label, COUNT(n) AS Count UNION ALL - OPTIONAL MATCH (n:Statement) WHERE n.group_id = $group_id RETURN 'Statement' AS Label, COUNT(n) AS Count + OPTIONAL MATCH (n:Statement) WHERE n.end_user_id = $end_user_id RETURN 'Statement' AS Label, COUNT(n) AS Count UNION ALL - OPTIONAL MATCH (n:ExtractedEntity) WHERE n.group_id = $group_id RETURN 'ExtractedEntity' AS Label, COUNT(n) AS Count + OPTIONAL MATCH (n:ExtractedEntity) WHERE n.end_user_id = $end_user_id RETURN 'ExtractedEntity' AS Label, COUNT(n) AS Count UNION ALL - OPTIONAL MATCH (n) WHERE n.group_id = $group_id RETURN 'ALL' AS Label, COUNT(n) AS Count + OPTIONAL MATCH (n) WHERE n.end_user_id = $end_user_id RETURN 'ALL' AS Label, COUNT(n) AS Count """ # Extracted entity details within group/app/user SEARCH_FOR_DETIALS = """ MATCH (n:ExtractedEntity) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id RETURN n.entity_idx AS entity_idx, n.connect_strength AS connect_strength, n.description AS description, n.entity_type AS entity_type, n.name AS name, COALESCE(n.fact_summary, '') AS fact_summary, - n.group_id AS group_id, + n.end_user_id AS end_user_id, n.apply_id AS apply_id, n.user_id AS user_id, n.id AS id @@ -91,9 +94,9 @@ class DataConfigRepository: # Edges between extracted entities within group/app/user SEARCH_FOR_EDGES = """ MATCH (n:ExtractedEntity)-[r]->(m:ExtractedEntity) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id RETURN - r.group_id AS group_id, + r.end_user_id AS end_user_id, r.apply_id AS apply_id, r.user_id AS user_id, elementId(r) AS rel_id, @@ -107,7 +110,7 @@ class DataConfigRepository: @staticmethod def update_reflection_config( db: Session, - config_id: int, + config_id: uuid.UUID, enable_self_reflexion: bool, iteration_period: str, reflexion_range: str, @@ -115,7 +118,7 @@ class DataConfigRepository: reflection_model_id: str, memory_verify: bool, quality_assessment: bool - ) -> DataConfig: + ) -> MemoryConfig: """构建反思配置更新语句（SQLAlchemy text() 命名参数） Args: @@ -130,28 +133,28 @@ class DataConfigRepository: config_id: 配置ID Returns: - Data + MemoryConfig Raises: ValueError: 没有字段需要更新时抛出 """ db_logger.debug(f"构建反思配置更新语句: config_id={config_id}") - stmt = select(DataConfig).where(DataConfig.config_id == config_id) - data_config_obj = db.scalars(stmt).first() - if not data_config_obj: + stmt = select(MemoryConfig).where(MemoryConfig.config_id == config_id) + memory_config_obj = db.scalars(stmt).first() + if not memory_config_obj: raise BusinessException - data_config_obj.enable_self_reflexion = enable_self_reflexion - data_config_obj.iteration_period = iteration_period - data_config_obj.reflexion_range = reflexion_range - data_config_obj.baseline = baseline - data_config_obj.reflection_model_id = reflection_model_id - data_config_obj.memory_verify = memory_verify - data_config_obj.quality_assessment = quality_assessment + memory_config_obj.enable_self_reflexion = enable_self_reflexion + memory_config_obj.iteration_period = iteration_period + memory_config_obj.reflexion_range = reflexion_range + memory_config_obj.baseline = baseline + memory_config_obj.reflection_model_id = reflection_model_id + memory_config_obj.memory_verify = memory_verify + memory_config_obj.quality_assessment = quality_assessment - return data_config_obj + return memory_config_obj @staticmethod - def query_reflection_config_by_id(db: Session, config_id: int) -> DataConfig: + def query_reflection_config_by_id(db: Session, config_id: uuid.UUID) -> MemoryConfig: """构建反思配置查询语句，通过config_id查询反思配置（SQLAlchemy text() 命名参数） Args: @@ -162,13 +165,13 @@ class DataConfigRepository: Tuple[str, Dict]: (SQL查询字符串, 参数字典) """ db_logger.debug(f"构建反思配置查询语句: config_id={config_id}") - stmt = select(DataConfig).where(DataConfig.config_id == config_id) - data_config = db.scalars(stmt).first() - if not data_config: + stmt = select(MemoryConfig).where(MemoryConfig.config_id == config_id) + memory_config = db.scalars(stmt).first() + if not memory_config: raise RuntimeError("reflection config not found") - return data_config + return memory_config @staticmethod - def query_reflection_config_by_workspace_id(db: Session, workspace_id: uuid.UUID) -> DataConfig: + def query_reflection_config_by_workspace_id(db: Session, workspace_id: uuid.UUID) -> MemoryConfig: """构建查询所有配置的语句（SQLAlchemy text() 命名参数） Args: @@ -180,11 +183,11 @@ class DataConfigRepository: """ db_logger.debug(f"构建查询所有配置语句: workspace_id={workspace_id}") - stmt = select(DataConfig).where(DataConfig.workspace_id == workspace_id) - data_config = db.scalars(stmt).first() - if not data_config: + stmt = select(MemoryConfig).where(MemoryConfig.workspace_id == workspace_id) + memory_config = db.scalars(stmt).first() + if not memory_config: raise RuntimeError("reflection config not found") - return data_config + return memory_config @staticmethod @@ -208,20 +211,21 @@ class DataConfigRepository: return query, params @staticmethod - def create(db: Session, params: ConfigParamsCreate) -> DataConfig: - """创建数据配置 + def create(db: Session, params: ConfigParamsCreate) -> MemoryConfig: + """创建记忆配置 Args: db: 数据库会话 params: 配置参数创建模型 Returns: - DataConfig: 创建的配置对象 + MemoryConfig: 创建的配置对象 """ - db_logger.debug(f"创建数据配置: config_name={params.config_name}, workspace_id={params.workspace_id}") + db_logger.debug(f"创建记忆配置: config_name={params.config_name}, workspace_id={params.workspace_id}") try: - db_config = DataConfig( + db_config = MemoryConfig( + config_id=uuid.uuid4(), config_name=params.config_name, config_desc=params.config_desc, workspace_id=params.workspace_id, @@ -232,16 +236,16 @@ class DataConfigRepository: db.add(db_config) db.flush() # 获取自增ID但不提交事务 - db_logger.info(f"数据配置已添加到会话: {db_config.config_name} (ID: {db_config.config_id})") + db_logger.info(f"记忆配置已添加到会话: {db_config.config_name} (ID: {db_config.config_id})") return db_config except Exception as e: db.rollback() - db_logger.error(f"创建数据配置失败: {params.config_name} - {str(e)}") + db_logger.error(f"创建记忆配置失败: {params.config_name} - {str(e)}") raise @staticmethod - def update(db: Session, update: ConfigUpdate) -> Optional[DataConfig]: + def update(db: Session, update: ConfigUpdate) -> Optional[MemoryConfig]: """更新基础配置 Args: @@ -249,17 +253,17 @@ class DataConfigRepository: update: 配置更新模型 Returns: - Optional[DataConfig]: 更新后的配置对象，不存在则返回None + Optional[MemoryConfig]: 更新后的配置对象，不存在则返回None Raises: ValueError: 没有字段需要更新时抛出 """ - db_logger.debug(f"更新数据配置: config_id={update.config_id}") + db_logger.debug(f"更新记忆配置: config_id={update.config_id}") try: - db_config = db.query(DataConfig).filter(DataConfig.config_id == update.config_id).first() + db_config = db.query(MemoryConfig).filter(MemoryConfig.config_id == update.config_id).first() if not db_config: - db_logger.warning(f"数据配置不存在: config_id={update.config_id}") + db_logger.warning(f"记忆配置不存在: config_id={update.config_id}") return None # 更新字段 @@ -277,17 +281,17 @@ class DataConfigRepository: db.commit() db.refresh(db_config) - db_logger.info(f"数据配置更新成功: {db_config.config_name} (ID: {update.config_id})") + db_logger.info(f"记忆配置更新成功: {db_config.config_name} (ID: {update.config_id})") return db_config except Exception as e: db.rollback() - db_logger.error(f"更新数据配置失败: config_id={update.config_id} - {str(e)}") + db_logger.error(f"更新记忆配置失败: config_id={update.config_id} - {str(e)}") raise @staticmethod - def update_extracted(db: Session, update: ConfigUpdateExtracted) -> Optional[DataConfig]: + def update_extracted(db: Session, update: ConfigUpdateExtracted) -> Optional[MemoryConfig]: """更新记忆萃取引擎配置 Args: @@ -295,7 +299,7 @@ class DataConfigRepository: update: 萃取配置更新模型 Returns: - Optional[DataConfig]: 更新后的配置对象，不存在则返回None + Optional[MemoryConfig]: 更新后的配置对象，不存在则返回None Raises: ValueError: 没有字段需要更新时抛出 @@ -303,9 +307,9 @@ class DataConfigRepository: db_logger.debug(f"更新萃取配置: config_id={update.config_id}") try: - db_config = db.query(DataConfig).filter(DataConfig.config_id == update.config_id).first() + db_config = db.query(MemoryConfig).filter(MemoryConfig.config_id == update.config_id).first() if not db_config: - db_logger.warning(f"数据配置不存在: config_id={update.config_id}") + db_logger.warning(f"记忆配置不存在: config_id={update.config_id}") return None # 更新字段映射 @@ -360,7 +364,7 @@ class DataConfigRepository: raise @staticmethod - def update_forget(db: Session, update: ConfigUpdateForget) -> Optional[DataConfig]: + def update_forget(db: Session, update: ConfigUpdateForget) -> Optional[MemoryConfig]: """更新遗忘引擎配置 Args: @@ -368,7 +372,7 @@ class DataConfigRepository: update: 遗忘配置更新模型 Returns: - Optional[DataConfig]: 更新后的配置对象，不存在则返回None + Optional[MemoryConfig]: 更新后的配置对象，不存在则返回None Raises: ValueError: 没有字段需要更新时抛出 @@ -376,9 +380,9 @@ class DataConfigRepository: db_logger.debug(f"更新遗忘配置: config_id={update.config_id}") try: - db_config = db.query(DataConfig).filter(DataConfig.config_id == update.config_id).first() + db_config = db.query(MemoryConfig).filter(MemoryConfig.config_id == update.config_id).first() if not db_config: - db_logger.warning(f"数据配置不存在: config_id={update.config_id}") + db_logger.warning(f"记忆配置不存在: config_id={update.config_id}") return None # 更新字段 @@ -408,7 +412,7 @@ class DataConfigRepository: raise @staticmethod - def get_extracted_config(db: Session, config_id: int) -> Optional[Dict]: + def get_extracted_config(db: Session, config_id: UUID |int) -> Optional[Dict]: """获取萃取配置，通过主键查询某条配置 Args: @@ -418,10 +422,10 @@ class DataConfigRepository: Returns: Optional[Dict]: 萃取配置字典，不存在则返回None """ + config_id=resolve_config_id(config_id,db) db_logger.debug(f"查询萃取配置: config_id={config_id}") - try: - db_config = db.query(DataConfig).filter(DataConfig.config_id == config_id).first() + db_config = db.query(MemoryConfig).filter(MemoryConfig.config_id == config_id).first() if not db_config: db_logger.debug(f"萃取配置不存在: config_id={config_id}") return None @@ -457,7 +461,7 @@ class DataConfigRepository: raise @staticmethod - def get_forget_config(db: Session, config_id: int) -> Optional[Dict]: + def get_forget_config(db: Session, config_id: UUID) -> Optional[Dict]: """获取遗忘配置，通过主键查询某条配置 Args: @@ -470,7 +474,7 @@ class DataConfigRepository: db_logger.debug(f"查询遗忘配置: config_id={config_id}") try: - db_config = db.query(DataConfig).filter(DataConfig.config_id == config_id).first() + db_config = db.query(MemoryConfig).filter(MemoryConfig.config_id == config_id).first() if not db_config: db_logger.debug(f"遗忘配置不存在: config_id={config_id}") return None @@ -489,39 +493,39 @@ class DataConfigRepository: raise @staticmethod - def get_by_id(db: Session, config_id: int) -> Optional[DataConfig]: - """根据ID获取数据配置 + def get_by_id(db: Session, config_id: uuid.UUID) -> Optional[MemoryConfig]: + """根据ID获取记忆配置 Args: db: 数据库会话 config_id: 配置ID Returns: - Optional[DataConfig]: 配置对象，不存在则返回None + Optional[MemoryConfig]: 配置对象，不存在则返回None """ - db_logger.debug(f"根据ID查询数据配置: config_id={config_id}") + db_logger.debug(f"根据ID查询记忆配置: config_id={config_id}") try: - config = db.query(DataConfig).filter(DataConfig.config_id == config_id).first() + config = db.query(MemoryConfig).filter(MemoryConfig.config_id == config_id).first() if config: - db_logger.debug(f"数据配置查询成功: {config.config_name} (ID: {config_id})") + db_logger.debug(f"记忆配置查询成功: {config.config_name} (ID: {config_id})") else: - db_logger.debug(f"数据配置不存在: config_id={config_id}") + db_logger.debug(f"记忆配置不存在: config_id={config_id}") return config except Exception as e: - db_logger.error(f"根据ID查询数据配置失败: config_id={config_id} - {str(e)}") + db_logger.error(f"根据ID查询记忆配置失败: config_id={config_id} - {str(e)}") raise @staticmethod - def get_config_with_workspace(db: Session, config_id: int) -> Optional[tuple]: - """Get data config and its associated workspace information + def get_config_with_workspace(db: Session, config_id: uuid.UUID) -> Optional[tuple]: + """Get memory config and its associated workspace information Args: db: Database session config_id: Configuration ID Returns: - Optional[tuple]: (DataConfig, Workspace) tuple, None if not found + Optional[tuple]: (MemoryConfig, Workspace) tuple, None if not found Raises: ValueError: Raised when config exists but workspace doesn't @@ -541,19 +545,19 @@ class DataConfigRepository: } ) - db_logger.debug(f"Querying data config and workspace: config_id={config_id}") + db_logger.debug(f"Querying memory config and workspace: config_id={config_id}") try: # Use join query to get both config and workspace - result = db.query(DataConfig, Workspace).join( - Workspace, DataConfig.workspace_id == Workspace.id - ).filter(DataConfig.config_id == config_id).first() + result = db.query(MemoryConfig, Workspace).join( + Workspace, MemoryConfig.workspace_id == Workspace.id + ).filter(MemoryConfig.config_id == config_id).first() elapsed_ms = (time.time() - start_time) * 1000 if not result: # Check if config exists but workspace is missing - config_only = db.query(DataConfig).filter(DataConfig.config_id == config_id).first() + config_only = db.query(MemoryConfig).filter(MemoryConfig.config_id == config_id).first() if config_only: if config_only.workspace_id is None: config_logger.error( @@ -566,7 +570,7 @@ class DataConfigRepository: "elapsed_ms": elapsed_ms } ) - db_logger.error(f"Data config {config_id} has no associated workspace ID") + db_logger.error(f"Memory config {config_id} has no associated workspace ID") raise ValueError(f"Configuration {config_id} has no associated workspace") else: config_logger.error( @@ -579,7 +583,7 @@ class DataConfigRepository: "elapsed_ms": elapsed_ms } ) - db_logger.error(f"Data config {config_id} references non-existent workspace {config_only.workspace_id}") + db_logger.error(f"Memory config {config_id} references non-existent workspace {config_only.workspace_id}") raise ValueError(f"Workspace {config_only.workspace_id} not found for configuration {config_id}") config_logger.debug( @@ -591,7 +595,7 @@ class DataConfigRepository: "elapsed_ms": elapsed_ms } ) - db_logger.debug(f"Data config not found: config_id={config_id}") + db_logger.debug(f"Memory config not found: config_id={config_id}") return None config, workspace = result @@ -611,7 +615,7 @@ class DataConfigRepository: } ) - db_logger.debug(f"Data config and workspace query successful: config={config.config_name}, workspace={workspace.name}") + db_logger.debug(f"Memory config and workspace query successful: config={config.config_name}, workspace={workspace.name}") return (config, workspace) except ValueError: @@ -633,10 +637,10 @@ class DataConfigRepository: exc_info=True ) - db_logger.error(f"Failed to query data config and workspace: config_id={config_id} - {str(e)}") + db_logger.error(f"Failed to query memory config and workspace: config_id={config_id} - {str(e)}") raise @staticmethod - def get_all(db: Session, workspace_id: Optional[uuid.UUID] = None) -> List[DataConfig]: + def get_all(db: Session, workspace_id: Optional[uuid.UUID] = None) -> List[MemoryConfig]: """获取所有配置参数 Args: @@ -644,17 +648,17 @@ class DataConfigRepository: workspace_id: 工作空间ID，用于过滤查询结果 Returns: - List[DataConfig]: 配置列表 + List[MemoryConfig]: 配置列表 """ db_logger.debug(f"查询所有配置: workspace_id={workspace_id}") try: - query = db.query(DataConfig) + query = db.query(MemoryConfig) if workspace_id: - query = query.filter(DataConfig.workspace_id == workspace_id) + query = query.filter(MemoryConfig.workspace_id == workspace_id) - configs = query.order_by(desc(DataConfig.updated_at)).all() + configs = query.order_by(desc(MemoryConfig.updated_at)).all() db_logger.debug(f"配置列表查询成功: 数量={len(configs)}") return configs @@ -664,8 +668,8 @@ class DataConfigRepository: raise @staticmethod - def delete(db: Session, config_id: int) -> bool: - """删除数据配置 + def delete(db: Session, config_id: uuid.UUID) -> bool: + """删除记忆配置 Args: db: 数据库会话 @@ -674,22 +678,22 @@ class DataConfigRepository: Returns: bool: 删除成功返回True，配置不存在返回False """ - db_logger.debug(f"删除数据配置: config_id={config_id}") + db_logger.debug(f"删除记忆配置: config_id={config_id}") try: - db_config = db.query(DataConfig).filter(DataConfig.config_id == config_id).first() + db_config = db.query(MemoryConfig).filter(MemoryConfig.config_id == config_id).first() if not db_config: - db_logger.warning(f"数据配置不存在: config_id={config_id}") + db_logger.warning(f"记忆配置不存在: config_id={config_id}") return False db.delete(db_config) db.commit() - db_logger.info(f"数据配置删除成功: config_id={config_id}") + db_logger.info(f"记忆配置删除成功: config_id={config_id}") return True except Exception as e: db.rollback() - db_logger.error(f"删除数据配置失败: config_id={config_id} - {str(e)}") + db_logger.error(f"删除记忆配置失败: config_id={config_id} - {str(e)}") raise diff --git a/api/app/repositories/memory_perceptual_repository.py b/api/app/repositories/memory_perceptual_repository.py index 8415c2d0..9fa9536e 100644 --- a/api/app/repositories/memory_perceptual_repository.py +++ b/api/app/repositories/memory_perceptual_repository.py @@ -6,7 +6,7 @@ from sqlalchemy import and_, desc from sqlalchemy.orm import Session from app.core.logging_config import get_db_logger -from app.models.memory_perceptual_model import MemoryPerceptualModel, PerceptualType, FileStorageType +from app.models.memory_perceptual_model import MemoryPerceptualModel, PerceptualType, FileStorageService from app.schemas.memory_perceptual_schema import PerceptualQuerySchema db_logger = get_db_logger() @@ -28,7 +28,7 @@ class MemoryPerceptualRepository: file_ext: str, summary: Optional[str] = None, meta_data: Optional[dict] = None, - storage_service: FileStorageType = FileStorageType.LOCAL + storage_service: FileStorageService = FileStorageService.LOCAL ) -> MemoryPerceptualModel: diff --git a/api/app/repositories/model_repository.py b/api/app/repositories/model_repository.py index 1fe29d66..3d66964a 100644 --- a/api/app/repositories/model_repository.py +++ b/api/app/repositories/model_repository.py @@ -1,12 +1,12 @@ -from sqlalchemy.orm import Session, joinedload -from sqlalchemy import and_, or_, func, desc +from sqlalchemy.orm import Session, joinedload, selectinload +from sqlalchemy import and_, or_, func, desc, select from typing import List, Optional, Dict, Any, Tuple import uuid -from app.models.models_model import ModelConfig, ModelApiKey, ModelType +from app.models.models_model import ModelConfig, ModelApiKey, ModelType, ModelBase, model_config_api_key_association from app.schemas.model_schema import ( ModelConfigUpdate, ModelApiKeyCreate, ModelApiKeyUpdate, - ModelConfigQuery + ModelConfigQuery, ModelConfigQueryNew ) from app.core.logging_config import get_db_logger @@ -107,6 +107,80 @@ class ModelConfigRepository: def get_list(db: Session, query: ModelConfigQuery, tenant_id: uuid.UUID | None = None) -> Tuple[List[ModelConfig], int]: """获取模型配置列表""" db_logger.debug(f"查询模型配置列表: {query.dict()}, tenant_id={tenant_id}") + + try: + # 构建查询条件 + filters = [] + + # 添加租户过滤（查询本租户的模型或公开模型） + if tenant_id: + filters.append( + or_( + ModelConfig.tenant_id == tenant_id, + ModelConfig.is_public + ) + ) + + # 支持多个 type 值（使用 IN 查询） + # 兼容 chat 和 llm 类型：如果查询包含其中一个，则同时匹配两者 + if query.type: + type_values = list(query.type) + # 如果包含 chat 或 llm，则同时包含两者 + if ModelType.CHAT in type_values or ModelType.LLM in type_values: + if ModelType.CHAT not in type_values: + type_values.append(ModelType.CHAT) + if ModelType.LLM not in type_values: + type_values.append(ModelType.LLM) + filters.append(ModelConfig.type.in_(type_values)) + + if query.is_active is not None: + filters.append(ModelConfig.is_active == query.is_active) + + if query.is_public is not None: + filters.append(ModelConfig.is_public == query.is_public) + + if query.search: + # 搜索逻辑需要join ModelApiKey表来搜索model_name + search_filter = or_( + ModelConfig.name.ilike(f"%{query.search}%"), + # ModelConfig.description.ilike(f"%{query.search}%") + ) + filters.append(search_filter) + + # 构建基础查询 + base_query = db.query(ModelConfig).options( + joinedload(ModelConfig.api_keys) + ) + + # 如果需要按provider筛选，需要join ModelApiKey表 + if query.provider: + base_query = base_query.join(ModelApiKey).filter( + ModelApiKey.provider == query.provider + ).distinct() + + if filters: + base_query = base_query.filter(and_(*filters)) + + # 获取总数 + total = base_query.count() + + # 分页查询 + models = base_query.order_by(desc(ModelConfig.created_at)).offset( + (query.page - 1) * query.pagesize + ).limit(query.pagesize).all() + + db_logger.debug(f"模型配置列表查询成功: 总数={total}, 当前页={len(models)}, type筛选={query.type}") + return models, total + + except Exception as e: + db_logger.error(f"查询模型配置列表失败: {str(e)}") + raise + + @staticmethod + def get_list_new(db: Session, query: ModelConfigQueryNew, tenant_id: uuid.UUID | None = None) -> tuple[ + dict[str, list[ModelConfig]], Any]: + """获取模型配置列表""" + db_logger.debug(f"查询模型配置列表: {query.model_dump()}, tenant_id={tenant_id}") try: # 构建查询条件 @@ -138,13 +212,15 @@ class ModelConfigRepository: if query.is_public is not None: filters.append(ModelConfig.is_public == query.is_public) + + if query.is_composite is not None: + filters.append(ModelConfig.is_composite == query.is_composite) + + if query.provider: + filters.append(ModelConfig.provider == query.provider) if query.search: - # 搜索逻辑需要join ModelApiKey表来搜索model_name - search_filter = or_( - ModelConfig.name.ilike(f"%{query.search}%"), - # ModelConfig.description.ilike(f"%{query.search}%") - ) + search_filter = ModelConfig.name.ilike(f"%{query.search}%") filters.append(search_filter) # 构建基础查询 @@ -152,28 +228,30 @@ class ModelConfigRepository: joinedload(ModelConfig.api_keys) ) - # 如果需要按provider筛选，需要join ModelApiKey表 - if query.provider: - base_query = base_query.join(ModelApiKey).filter( - ModelApiKey.provider == query.provider - ).distinct() - if filters: base_query = base_query.filter(and_(*filters)) # 获取总数 total = base_query.count() + + query_results = base_query.order_by(desc(ModelConfig.created_at)).all() + + provider_groups: Dict[str, List[ModelConfig]] = {} + for model_config in query_results: + provider = model_config.provider + if provider not in provider_groups: + provider_groups[provider] = [] + provider_groups[provider].append(model_config) - # 分页查询 - models = base_query.order_by(desc(ModelConfig.updated_at)).offset( - (query.page - 1) * query.pagesize - ).limit(query.pagesize).all() - - db_logger.debug(f"模型配置列表查询成功: 总数={total}, 当前页={len(models)}, type筛选={query.type}") - return models, total + db_logger.debug( + f"模型配置列表查询成功: 总数={total}, " + f"分组数={len(provider_groups)}, " + f"各分组模型数={[len(v) for v in provider_groups.values()]}, " + f"type筛选={query.type}") + return provider_groups, total except Exception as e: - db_logger.error(f"查询模型配置列表失败: {str(e)}") + db_logger.error(f"查询模型配置列表失败(按provider分组/无分页): {str(e)}") raise @staticmethod @@ -241,7 +319,7 @@ class ModelConfigRepository: return None # 更新字段 - update_data = model_data.dict(exclude_unset=True) + update_data = model_data.model_dump(exclude_unset=True) for field, value in update_data.items(): setattr(db_model, field, value) @@ -303,8 +381,18 @@ class ModelConfigRepository: # 按提供商统计 - 现在从ModelApiKey表获取 provider_stats = {} provider_results = db.query( - ModelApiKey.provider, func.count(func.distinct(ModelApiKey.model_config_id)) - ).group_by(ModelApiKey.provider).all() + # 保留 provider 字段 + ModelApiKey.provider, + # 统计中间表中 唯一的 model_config_id 数量（替换原 ModelApiKey.model_config_id） + func.count(func.distinct(model_config_api_key_association.c.model_config_id)) + ).join( + # 联表：ModelApiKey <-> 中间表（多对多关联） + model_config_api_key_association, + ModelApiKey.id == model_config_api_key_association.c.api_key_id + ).group_by( + # 按 provider 分组（保留原有逻辑） + ModelApiKey.provider + ).all() for provider, count in provider_results: provider_stats[provider.value] = count @@ -325,6 +413,38 @@ class ModelConfigRepository: db_logger.error(f"获取模型统计信息失败: {str(e)}") raise + @staticmethod + def get_model_config_ids_by_provider( + db: Session, + tenant_id: uuid.UUID, + provider: Any + ) -> List[uuid.UUID]: + """根据tenant_id和provider获取model_config_id列表""" + db_logger.debug(f"查询model_config_id列表: tenant_id={tenant_id}, provider={provider}") + + try: + # 查询ModelConfig关联的ModelApiKey，筛选出匹配的model_config_id + model_config_ids = db.query(ModelConfig.id).join( + ModelBase, ModelConfig.model_id == ModelBase.id + ).filter( + and_( + or_( + ModelConfig.tenant_id == tenant_id, + ModelConfig.is_public + ), + ModelBase.provider == provider, + ModelConfig.is_active, + ~ModelConfig.is_composite + ) + ).distinct().all() + + db_logger.debug(f"查询成功: 数量={len(model_config_ids)}") + return [row[0] for row in model_config_ids] + + except Exception as e: + db_logger.error(f"查询model_config_id列表失败: {str(e)}") + raise + class ModelApiKeyRepository: """模型API Key Repository""" @@ -349,7 +469,14 @@ class ModelApiKeyRepository: db_logger.debug(f"根据模型配置ID查询API Key: model_config_id={model_config_id}") try: - query = db.query(ModelApiKey).filter(ModelApiKey.model_config_id == model_config_id) + from app.models.models_model import ModelConfig, model_config_api_key_association + + query = db.query(ModelApiKey).join( + model_config_api_key_association, + ModelApiKey.id == model_config_api_key_association.c.api_key_id + ).filter( + model_config_api_key_association.c.model_config_id == model_config_id + ) if is_active: query = query.filter(ModelApiKey.is_active) @@ -368,8 +495,20 @@ class ModelApiKeyRepository: db_logger.debug(f"创建API Key: {api_key_data.provider}") try: - db_api_key = ModelApiKey(**api_key_data.dict()) + from app.models.models_model import ModelConfig + + # 创建API Key，不包含model_config_ids + api_key_dict = api_key_data.model_dump(exclude={"model_config_ids"}) + db_api_key = ModelApiKey(**api_key_dict) db.add(db_api_key) + db.flush() # 获取生成的ID + + # 关联ModelConfig + if api_key_data.model_config_ids: + for model_config_id in api_key_data.model_config_ids: + model_config = db.query(ModelConfig).filter(ModelConfig.id == model_config_id).first() + if model_config: + db_api_key.model_configs.append(model_config) db_logger.info(f"API Key已添加到会话: {db_api_key.provider}") return db_api_key @@ -391,7 +530,7 @@ class ModelApiKeyRepository: return None # 更新字段 - update_data = api_key_data.dict(exclude_unset=True) + update_data = api_key_data.model_dump(exclude_unset=True) for field, value in update_data.items(): setattr(db_api_key, field, value) @@ -451,4 +590,92 @@ class ModelApiKeyRepository: except Exception as e: db.rollback() db_logger.error(f"更新API Key使用统计失败: api_key_id={api_key_id} - {str(e)}") - raise \ No newline at end of file + raise + + +class ModelBaseRepository: + """基础模型Repository""" + + @staticmethod + def get_by_id(db: Session, model_base_id: uuid.UUID) -> Optional['ModelBase']: + return db.query(ModelBase).filter(ModelBase.id == model_base_id).first() + + @staticmethod + def get_list(db: Session, query: 'ModelBaseQuery') -> List['ModelBase']: + + filters = [] + if query.type: + filters.append(ModelBase.type == query.type) + if query.provider: + filters.append(ModelBase.provider == query.provider) + if query.is_official is not None: + filters.append(ModelBase.is_official == query.is_official) + if query.is_deprecated is not None: + filters.append(ModelBase.is_deprecated == query.is_deprecated) + if query.search: + filters.append(or_( + ModelBase.name.ilike(f"%{query.search}%"), + # ModelBase.description.ilike(f"%{query.search}%") + )) + + q = db.query(ModelBase) + if filters: + q = q.filter(and_(*filters)) + + return q.order_by(ModelBase.add_count.desc(), ModelBase.created_at.desc()).all() + + @staticmethod + def create(db: Session, data: dict) -> 'ModelBase': + model_base = ModelBase(**data) + db.add(model_base) + return model_base + + @staticmethod + def get_by_name_and_provider(db: Session, name: str, provider: str) -> Optional['ModelBase']: + return db.query(ModelBase).filter( + ModelBase.name == name, + ModelBase.provider == provider + ).first() + + @staticmethod + def update(db: Session, model_base_id: uuid.UUID, data: dict) -> Optional['ModelBase']: + model_base = db.query(ModelBase).filter(ModelBase.id == model_base_id).first() + if not model_base: + return None + for key, value in data.items(): + setattr(model_base, key, value) + + # 同步更新绑定的非组合模型配置 + if any(k in data for k in ['name', 'description', 'logo']): + db.query(ModelConfig).filter( + ModelConfig.model_id == model_base_id, + ModelConfig.is_composite == False + ).update({ + k: v for k, v in data.items() + if k in ['name', 'description', 'logo'] + }, synchronize_session=False) + + return model_base + + @staticmethod + def delete(db: Session, model_base_id: uuid.UUID) -> bool: + model_base = db.query(ModelBase).filter(ModelBase.id == model_base_id).first() + if not model_base: + return False + db.delete(model_base) + return True + + @staticmethod + def increment_add_count(db: Session, model_base_id: uuid.UUID) -> bool: + model_base = db.query(ModelBase).filter(ModelBase.id == model_base_id).first() + if not model_base: + return False + model_base.add_count += 1 + return True + + @staticmethod + def check_added_by_tenant(db: Session, model_base_id: uuid.UUID, tenant_id: uuid.UUID) -> bool: + return db.query(ModelConfig).filter( + ModelConfig.model_id == model_base_id, + ModelConfig.tenant_id == tenant_id + ).first() is not None diff --git a/api/app/repositories/neo4j/add_edges.py b/api/app/repositories/neo4j/add_edges.py index 3b45867e..162bf411 100644 --- a/api/app/repositories/neo4j/add_edges.py +++ b/api/app/repositories/neo4j/add_edges.py @@ -32,7 +32,7 @@ async def add_chunk_statement_edges(chunks: List[Chunk], connector: Neo4jConnect "id": stable_edge_id, "source": chunk.id, "target": stmt.id, - "group_id": getattr(stmt, 'group_id', None), + "end_user_id": getattr(stmt, 'end_user_id', None), "user_id":getattr(stmt, 'user_id', None), "apply_id": getattr(stmt, 'apply_id', None), "run_id": getattr(stmt, 'run_id', None) or getattr(chunk, 'run_id', None), @@ -83,7 +83,7 @@ async def add_memory_summary_statement_edges(summaries: List[MemorySummaryNode], edges.append({ "summary_id": s.id, "chunk_id": chunk_id, - "group_id": s.group_id, + "end_user_id": s.end_user_id, "run_id": s.run_id, "created_at": s.created_at.isoformat() if s.created_at else None, "expired_at": s.expired_at.isoformat() if s.expired_at else None, diff --git a/api/app/repositories/neo4j/add_nodes.py b/api/app/repositories/neo4j/add_nodes.py index cf60a773..fcf700b5 100644 --- a/api/app/repositories/neo4j/add_nodes.py +++ b/api/app/repositories/neo4j/add_nodes.py @@ -6,10 +6,10 @@ from app.core.memory.models.graph_models import DialogueNode, StatementNode, Chu from app.repositories.neo4j.neo4j_connector import Neo4jConnector -async def delete_all_nodes(group_id: str, connector: Neo4jConnector): +async def delete_all_nodes(end_user_id: str, connector: Neo4jConnector): """Delete all nodes in the database.""" - result = await connector.execute_query(f"MATCH (n {{group_id: '{group_id}'}}) DETACH DELETE n") - print(f"All group_id: {group_id} node and edge deleted successfully") + result = await connector.execute_query(f"MATCH (n {{end_user_id: '{end_user_id}'}}) DETACH DELETE n") + print(f"All end_user_id: {end_user_id} node and edge deleted successfully") return result async def add_dialogue_nodes(dialogues: List[DialogueNode], connector: Neo4jConnector) -> Optional[List[str]]: @@ -32,9 +32,7 @@ async def add_dialogue_nodes(dialogues: List[DialogueNode], connector: Neo4jConn for dialogue in dialogues: flattened_dialogues.append({ "id": dialogue.id, - "group_id": dialogue.group_id, - "user_id": dialogue.user_id, - "apply_id": dialogue.apply_id, + "end_user_id": dialogue.end_user_id, "run_id": dialogue.run_id, "ref_id": dialogue.ref_id, "name": dialogue.name, @@ -79,9 +77,7 @@ async def add_statement_nodes(statements: List[StatementNode], connector: Neo4jC flattened_statement = { "id": statement.id, "name": statement.name, - "group_id": statement.group_id, - "user_id": statement.user_id, - "apply_id": statement.apply_id, + "end_user_id": statement.end_user_id, "run_id": statement.run_id, "chunk_id": statement.chunk_id, # "created_at": statement.created_at.isoformat(), @@ -154,9 +150,7 @@ async def add_chunk_nodes(chunks: List[ChunkNode], connector: Neo4jConnector) -> flattened_chunk = { "id": chunk.id, "name": chunk.name, - "group_id": chunk.group_id, - "user_id": chunk.user_id, - "apply_id": chunk.apply_id, + "end_user_id": chunk.end_user_id, "run_id": chunk.run_id, "created_at": chunk.created_at.isoformat() if chunk.created_at else None, "expired_at": chunk.expired_at.isoformat() if chunk.expired_at else None, @@ -206,9 +200,7 @@ async def add_memory_summary_nodes(summaries: List[MemorySummaryNode], connector flattened.append({ "id": s.id, "name": s.name, - "group_id": s.group_id, - "user_id": s.user_id, - "apply_id": s.apply_id, + "end_user_id": s.end_user_id, "run_id": s.run_id, "created_at": s.created_at.isoformat() if s.created_at else None, "expired_at": s.expired_at.isoformat() if s.expired_at else None, diff --git a/api/app/repositories/neo4j/base_neo4j_repository.py b/api/app/repositories/neo4j/base_neo4j_repository.py index 959a1e68..df953eb9 100644 --- a/api/app/repositories/neo4j/base_neo4j_repository.py +++ b/api/app/repositories/neo4j/base_neo4j_repository.py @@ -152,7 +152,7 @@ class BaseNeo4jRepository(BaseRepository[T]): Example: >>> results = await repository.find( - ... {"group_id": "group_123", "user_id": "user_456"}, + ... {"end_user_id": "group_123", "user_id": "user_456"}, ... limit=50 ... ) """ diff --git a/api/app/repositories/neo4j/cypher_queries.py b/api/app/repositories/neo4j/cypher_queries.py index cd3cbed7..c93e75b3 100644 --- a/api/app/repositories/neo4j/cypher_queries.py +++ b/api/app/repositories/neo4j/cypher_queries.py @@ -3,9 +3,7 @@ DIALOGUE_NODE_SAVE = """ UNWIND $dialogues AS dialogue MERGE (n:Dialogue {id: dialogue.id}) SET n.uuid = coalesce(n.uuid, dialogue.id), - n.group_id = dialogue.group_id, - n.user_id = dialogue.user_id, - n.apply_id = dialogue.apply_id, + n.end_user_id = dialogue.end_user_id, n.run_id = dialogue.run_id, n.ref_id = dialogue.ref_id, n.created_at = dialogue.created_at, @@ -22,9 +20,7 @@ SET s += { id: statement.id, run_id: statement.run_id, chunk_id: statement.chunk_id, - group_id: statement.group_id, - user_id: statement.user_id, - apply_id: statement.apply_id, + end_user_id: statement.end_user_id, stmt_type: statement.stmt_type, statement: statement.statement, emotion_intensity: statement.emotion_intensity, @@ -54,9 +50,7 @@ MERGE (c:Chunk {id: chunk.id}) SET c += { id: chunk.id, name: chunk.name, - group_id: chunk.group_id, - user_id: chunk.user_id, - apply_id: chunk.apply_id, + end_user_id: chunk.end_user_id, run_id: chunk.run_id, created_at: chunk.created_at, expired_at: chunk.expired_at, @@ -76,9 +70,7 @@ EXTRACTED_ENTITY_NODE_SAVE = """ UNWIND $entities AS entity MERGE (e:ExtractedEntity {id: entity.id}) SET e.name = CASE WHEN entity.name IS NOT NULL AND entity.name <> '' THEN entity.name ELSE e.name END, - e.group_id = CASE WHEN entity.group_id IS NOT NULL AND entity.group_id <> '' THEN entity.group_id ELSE e.group_id END, - e.user_id = CASE WHEN entity.user_id IS NOT NULL AND entity.user_id <> '' THEN entity.user_id ELSE e.user_id END, - e.apply_id = CASE WHEN entity.apply_id IS NOT NULL AND entity.apply_id <> '' THEN entity.apply_id ELSE e.apply_id END, + e.end_user_id = CASE WHEN entity.end_user_id IS NOT NULL AND entity.end_user_id <> '' THEN entity.end_user_id ELSE e.end_user_id END, e.run_id = CASE WHEN entity.run_id IS NOT NULL AND entity.run_id <> '' THEN entity.run_id ELSE e.run_id END, e.created_at = CASE WHEN entity.created_at IS NOT NULL AND (e.created_at IS NULL OR entity.created_at < e.created_at) @@ -134,9 +126,9 @@ RETURN e.id AS uuid # Add back ENTITY_RELATIONSHIP_SAVE to be used by graph_saver.save_entities_and_relationships ENTITY_RELATIONSHIP_SAVE = """ UNWIND $relationships AS rel -// Match entities by stable id within group, do not constrain by run_id -MATCH (subject:ExtractedEntity {id: rel.source_id, group_id: rel.group_id}) -MATCH (object:ExtractedEntity {id: rel.target_id, group_id: rel.group_id}) +// Match entities by stable id within end_user_id, do not constrain by run_id +MATCH (subject:ExtractedEntity {id: rel.source_id, end_user_id: rel.end_user_id}) +MATCH (object:ExtractedEntity {id: rel.target_id, end_user_id: rel.end_user_id}) // Avoid duplicate edges across runs for the same endpoints MERGE (subject)-[r:EXTRACTED_RELATIONSHIP]->(object) SET r.predicate = rel.predicate, @@ -148,7 +140,7 @@ SET r.predicate = rel.predicate, r.created_at = rel.created_at, r.expired_at = rel.expired_at, r.run_id = rel.run_id, - r.group_id = rel.group_id + r.end_user_id = rel.end_user_id RETURN elementId(r) AS uuid """ @@ -160,7 +152,7 @@ UNWIND $weak_entities AS entity MERGE (e:ExtractedEntity {id: entity.id, run_id: entity.run_id}) SET e += { name: entity.name, - group_id: entity.group_id, + end_user_id: entity.end_user_id, run_id: entity.run_id, description: entity.description, chunk_id: entity.chunk_id, @@ -175,11 +167,11 @@ RETURN e.id AS id SAVE_STRONG_TRIPLE_ENTITIES = """ UNWIND $items AS item MERGE (s:ExtractedEntity {id: item.source_id, run_id: item.run_id}) -SET s += {name: item.subject, group_id: item.group_id, run_id: item.run_id} +SET s += {name: item.subject, end_user_id: item.end_user_id, run_id: item.run_id} // Independent strong flag SET s.is_strong = true MERGE (o:ExtractedEntity {id: item.target_id, run_id: item.run_id}) -SET o += {name: item.object, group_id: item.group_id, run_id: item.run_id} +SET o += {name: item.object, end_user_id: item.end_user_id, run_id: item.run_id} // Independent strong flag SET o.is_strong = true """ @@ -194,7 +186,7 @@ DIALOGUE_STATEMENT_EDGE_SAVE = """ // 仅按端点去重，关系属性可更新 MERGE (dialogue)-[e:MENTIONS]->(statement) SET e.uuid = edge.id, - e.group_id = edge.group_id, + e.end_user_id = edge.end_user_id, e.created_at = edge.created_at, e.expired_at = edge.expired_at RETURN e.uuid AS uuid @@ -208,7 +200,7 @@ CHUNK_STATEMENT_EDGE_SAVE = """ MATCH (statement:Statement {id: edge.source, run_id: edge.run_id}) MATCH (chunk:Chunk {id: edge.target, run_id: edge.run_id}) MERGE (chunk)-[e:CONTAINS {id: edge.id}]->(statement) - SET e.group_id = edge.group_id, + SET e.end_user_id = edge.end_user_id, e.run_id = edge.run_id, e.created_at = edge.created_at, e.expired_at = edge.expired_at @@ -218,13 +210,12 @@ CHUNK_STATEMENT_EDGE_SAVE = """ STATEMENT_ENTITY_EDGE_SAVE = """ UNWIND $relationships AS rel // Statement nodes are per-run; keep run_id constraint on statements -// Statement nodes are per-run; keep run_id constraint on statements MATCH (statement:Statement {id: rel.source, run_id: rel.run_id}) -// Entities are shared across runs within a group; do not constrain by run_id -MATCH (entity:ExtractedEntity {id: rel.target, group_id: rel.group_id}) +// Entities are shared across runs within end_user_id; do not constrain by run_id +MATCH (entity:ExtractedEntity {id: rel.target, end_user_id: rel.end_user_id}) // Avoid duplicate edges across runs for same endpoints MERGE (statement)-[r:REFERENCES_ENTITY]->(entity) -SET r.group_id = rel.group_id, +SET r.end_user_id = rel.end_user_id, r.run_id = rel.run_id, r.created_at = rel.created_at, r.expired_at = rel.expired_at, @@ -236,10 +227,10 @@ ENTITY_EMBEDDING_SEARCH = """ CALL db.index.vector.queryNodes('entity_embedding_index', $limit * 100, $embedding) YIELD node AS e, score WHERE e.name_embedding IS NOT NULL - AND ($group_id IS NULL OR e.group_id = $group_id) + AND ($end_user_id IS NULL OR e.end_user_id = $end_user_id) RETURN e.id AS id, e.name AS name, - e.group_id AS group_id, + e.end_user_id AS end_user_id, e.entity_type AS entity_type, COALESCE(e.activation_value, e.importance_score, 0.5) AS activation_value, COALESCE(e.importance_score, 0.5) AS importance_score, @@ -254,10 +245,10 @@ STATEMENT_EMBEDDING_SEARCH = """ CALL db.index.vector.queryNodes('statement_embedding_index', $limit * 100, $embedding) YIELD node AS s, score WHERE s.statement_embedding IS NOT NULL - AND ($group_id IS NULL OR s.group_id = $group_id) + AND ($end_user_id IS NULL OR s.end_user_id = $end_user_id) RETURN s.id AS id, s.statement AS statement, - s.group_id AS group_id, + s.end_user_id AS end_user_id, s.chunk_id AS chunk_id, s.created_at AS created_at, s.expired_at AS expired_at, @@ -277,9 +268,9 @@ CHUNK_EMBEDDING_SEARCH = """ CALL db.index.vector.queryNodes('chunk_embedding_index', $limit * 100, $embedding) YIELD node AS c, score WHERE c.chunk_embedding IS NOT NULL - AND ($group_id IS NULL OR c.group_id = $group_id) + AND ($end_user_id IS NULL OR c.end_user_id = $end_user_id) RETURN c.id AS chunk_id, - c.group_id AS group_id, + c.end_user_id AS end_user_id, c.content AS content, c.dialog_id AS dialog_id, COALESCE(c.activation_value, 0.5) AS activation_value, @@ -292,12 +283,12 @@ LIMIT $limit SEARCH_STATEMENTS_BY_KEYWORD = """ CALL db.index.fulltext.queryNodes("statementsFulltext", $q) YIELD node AS s, score -WHERE ($group_id IS NULL OR s.group_id = $group_id) +WHERE ($end_user_id IS NULL OR s.end_user_id = $end_user_id) OPTIONAL MATCH (c:Chunk)-[:CONTAINS]->(s) OPTIONAL MATCH (s)-[:REFERENCES_ENTITY]->(e:ExtractedEntity) RETURN s.id AS id, s.statement AS statement, - s.group_id AS group_id, + s.end_user_id AS end_user_id, s.chunk_id AS chunk_id, s.created_at AS created_at, s.expired_at AS expired_at, @@ -316,15 +307,13 @@ LIMIT $limit # 查询实体名称包含指定字符串的实体 SEARCH_ENTITIES_BY_NAME = """ CALL db.index.fulltext.queryNodes("entitiesFulltext", $q) YIELD node AS e, score -WHERE ($group_id IS NULL OR e.group_id = $group_id) +WHERE ($end_user_id IS NULL OR e.end_user_id = $end_user_id) OPTIONAL MATCH (s:Statement)-[:REFERENCES_ENTITY]->(e) OPTIONAL MATCH (c:Chunk)-[:CONTAINS]->(s) RETURN e.id AS id, e.name AS name, - e.group_id AS group_id, + e.end_user_id AS end_user_id, e.entity_type AS entity_type, - e.apply_id AS apply_id, - e.user_id AS user_id, e.created_at AS created_at, e.expired_at AS expired_at, e.entity_idx AS entity_idx, @@ -347,11 +336,11 @@ LIMIT $limit SEARCH_CHUNKS_BY_CONTENT = """ CALL db.index.fulltext.queryNodes("chunksFulltext", $q) YIELD node AS c, score -WHERE ($group_id IS NULL OR c.group_id = $group_id) +WHERE ($end_user_id IS NULL OR c.end_user_id = $end_user_id) OPTIONAL MATCH (c)-[:CONTAINS]->(s:Statement) OPTIONAL MATCH (s)-[:REFERENCES_ENTITY]->(e:ExtractedEntity) RETURN c.id AS chunk_id, - c.group_id AS group_id, + c.end_user_id AS end_user_id, c.content AS content, c.dialog_id AS dialog_id, c.sequence_number AS sequence_number, @@ -413,10 +402,10 @@ LIMIT $limit SEARCH_DIALOGUE_BY_DIALOG_ID = """ MATCH (d:Dialogue) -WHERE ($group_id IS NULL OR d.group_id = $group_id) +WHERE ($end_user_id IS NULL OR d.end_user_id = $end_user_id) AND d.id = $dialog_id RETURN d.id AS dialog_id, - d.group_id AS group_id, + d.end_user_id AS end_user_id, d.content AS content, d.created_at AS created_at, d.expired_at AS expired_at @@ -426,10 +415,10 @@ LIMIT $limit SEARCH_CHUNK_BY_CHUNK_ID = """ MATCH (c:Chunk) -WHERE ($group_id IS NULL OR c.group_id = $group_id) +WHERE ($end_user_id IS NULL OR c.end_user_id = $end_user_id) AND c.id = $chunk_id RETURN c.id AS chunk_id, - c.group_id AS group_id, + c.end_user_id AS end_user_id, c.content AS content, c.dialog_id AS dialog_id, c.created_at AS created_at, @@ -441,18 +430,14 @@ LIMIT $limit SEARCH_STATEMENTS_BY_TEMPORAL = """ MATCH (s:Statement) -WHERE ($group_id IS NULL OR s.group_id = $group_id) - AND ($apply_id IS NULL OR s.apply_id = $apply_id) - AND ($user_id IS NULL OR s.user_id = $user_id) +WHERE ($end_user_id IS NULL OR s.end_user_id = $end_user_id) AND ((($start_date IS NULL OR datetime(s.created_at) >= datetime($start_date)) AND ($end_date IS NULL OR datetime(s.created_at) <= datetime($end_date))) OR (($valid_date IS NULL OR (s.valid_at IS NOT NULL AND datetime(s.valid_at) >= datetime($valid_date))) AND ($invalid_date IS NULL OR (s.invalid_at IS NOT NULL AND datetime(s.invalid_at) <= datetime($invalid_date))))) RETURN s.id AS id, s.statement AS statement, - s.group_id AS group_id, - s.apply_id AS apply_id, - s.user_id AS user_id, + s.end_user_id AS end_user_id, s.chunk_id AS chunk_id, s.created_at AS created_at, s.valid_at AS valid_at, @@ -468,9 +453,7 @@ LIMIT $limit SEARCH_STATEMENTS_BY_KEYWORD_TEMPORAL = """ CALL db.index.fulltext.queryNodes("statementsFulltext", $q) YIELD node AS s, score -WHERE ($group_id IS NULL OR s.group_id = $group_id) - AND ($apply_id IS NULL OR s.apply_id = $apply_id) - AND ($user_id IS NULL OR s.user_id = $user_id) +WHERE ($end_user_id IS NULL OR s.end_user_id = $end_user_id) AND ((($start_date IS NULL OR (s.created_at IS NOT NULL AND datetime(s.created_at) >= datetime($start_date))) AND ($end_date IS NULL OR (s.created_at IS NOT NULL AND datetime(s.created_at) <= datetime($end_date)))) OR (($valid_date IS NULL OR (s.valid_at IS NOT NULL AND datetime(s.valid_at) >= datetime($valid_date))) @@ -479,9 +462,7 @@ OPTIONAL MATCH (c:Chunk)-[:CONTAINS]->(s) OPTIONAL MATCH (s)-[:REFERENCES_ENTITY]->(e:ExtractedEntity) RETURN s.id AS id, s.statement AS statement, - s.group_id AS group_id, - s.apply_id AS apply_id, - s.user_id AS user_id, + s.end_user_id AS end_user_id, s.chunk_id AS chunk_id, s.created_at AS created_at, s.valid_at AS valid_at, @@ -499,15 +480,11 @@ LIMIT $limit SEARCH_STATEMENTS_BY_CREATED_AT = """ MATCH (n:Statement) -WHERE ($group_id IS NULL OR n.group_id = $group_id) - AND ($apply_id IS NULL OR n.apply_id = $apply_id) - AND ($user_id IS NULL OR n.user_id = $user_id) +WHERE ($end_user_id IS NULL OR n.end_user_id = $end_user_id) AND ($created_at IS NOT NULL AND date(substring(n.created_at, 0, 10)) = date($created_at)) RETURN n.id AS id, n.statement AS statement, - n.group_id AS group_id, - n.apply_id AS apply_id, - n.user_id AS user_id, + n.end_user_id AS end_user_id, n.chunk_id AS chunk_id, n.created_at AS created_at, n.valid_at AS valid_at, @@ -519,15 +496,11 @@ LIMIT $limit SEARCH_STATEMENTS_BY_VALID_AT = """ MATCH (n:Statement) -WHERE ($group_id IS NULL OR n.group_id = $group_id) - AND ($apply_id IS NULL OR n.apply_id = $apply_id) - AND ($user_id IS NULL OR n.user_id = $user_id) +WHERE ($end_user_id IS NULL OR n.end_user_id = $end_user_id) AND ($valid_at IS NOT NULL AND date(substring(n.valid_at, 0, 10)) = date($valid_at)) RETURN n.id AS id, n.statement AS statement, - n.group_id AS group_id, - n.apply_id AS apply_id, - n.user_id AS user_id, + n.end_user_id AS end_user_id, n.chunk_id AS chunk_id, n.created_at AS created_at, n.valid_at AS valid_at, @@ -539,15 +512,11 @@ LIMIT $limit SEARCH_STATEMENTS_G_CREATED_AT = """ MATCH (n:Statement) -WHERE ($group_id IS NULL OR n.group_id = $group_id) - AND ($apply_id IS NULL OR n.apply_id = $apply_id) - AND ($user_id IS NULL OR n.user_id = $user_id) +WHERE ($end_user_id IS NULL OR n.end_user_id = $end_user_id) AND ($created_at IS NOT NULL AND date(substring(n.created_at, 0, 19)) = date($created_at)) RETURN n.id AS id, n.statement AS statement, - n.group_id AS group_id, - n.apply_id AS apply_id, - n.user_id AS user_id, + n.end_user_id AS end_user_id, n.chunk_id AS chunk_id, n.created_at AS created_at, n.valid_at AS valid_at, @@ -559,15 +528,11 @@ LIMIT $limit SEARCH_STATEMENTS_L_CREATED_AT = """ MATCH (n:Statement) -WHERE ($group_id IS NULL OR n.group_id = $group_id) - AND ($apply_id IS NULL OR n.apply_id = $apply_id) - AND ($user_id IS NULL OR n.user_id = $user_id) +WHERE ($end_user_id IS NULL OR n.end_user_id = $end_user_id) AND ($created_at IS NOT NULL AND date(substring(n.created_at, 0, 19)) < date($created_at)) RETURN n.id AS id, n.statement AS statement, - n.group_id AS group_id, - n.apply_id AS apply_id, - n.user_id AS user_id, + n.end_user_id AS end_user_id, n.chunk_id AS chunk_id, n.created_at AS created_at, n.valid_at AS valid_at, @@ -579,15 +544,11 @@ LIMIT $limit SEARCH_STATEMENTS_G_VALID_AT = """ MATCH (n:Statement) -WHERE ($group_id IS NULL OR n.group_id = $group_id) - AND ($apply_id IS NULL OR n.apply_id = $apply_id) - AND ($user_id IS NULL OR n.user_id = $user_id) +WHERE ($end_user_id IS NULL OR n.end_user_id = $end_user_id) AND ($valid_at IS NOT NULL AND date(substring(n.valid_at, 0, 10)) > date($valid_at)) RETURN n.id AS id, n.statement AS statement, - n.group_id AS group_id, - n.apply_id AS apply_id, - n.user_id AS user_id, + n.end_user_id AS end_user_id, n.chunk_id AS chunk_id, n.created_at AS created_at, n.valid_at AS valid_at, @@ -599,15 +560,11 @@ LIMIT $limit SEARCH_STATEMENTS_L_VALID_AT = """ MATCH (n:Statement) -WHERE ($group_id IS NULL OR n.group_id = $group_id) - AND ($apply_id IS NULL OR n.apply_id = $apply_id) - AND ($user_id IS NULL OR n.user_id = $user_id) +WHERE ($end_user_id IS NULL OR n.end_user_id = $end_user_id) AND ($valid_at IS NOT NULL AND date(substring(n.valid_at, 0, 10)) < date($valid_at)) RETURN n.id AS id, n.statement AS statement, - n.group_id AS group_id, - n.apply_id AS apply_id, - n.user_id AS user_id, + n.end_user_id AS end_user_id, n.chunk_id AS chunk_id, n.created_at AS created_at, n.valid_at AS valid_at, @@ -665,18 +622,18 @@ LIMIT $limit # 根据id修改句子的invalid_at的值 UPDATE_STATEMENT_INVALID_AT = """ -MATCH (n:Statement {group_id: $group_id, id: $id}) +MATCH (n:Statement {end_user_id: $end_user_id, id: $id}) SET n.invalid_at = $new_invalid_at """ # MemorySummary keyword search using fulltext index SEARCH_MEMORY_SUMMARIES_BY_KEYWORD = """ CALL db.index.fulltext.queryNodes("summariesFulltext", $q) YIELD node AS m, score -WHERE ($group_id IS NULL OR m.group_id = $group_id) +WHERE ($end_user_id IS NULL OR m.end_user_id = $end_user_id) OPTIONAL MATCH (m)-[:DERIVED_FROM_STATEMENT]->(s:Statement) RETURN m.id AS id, m.name AS name, - m.group_id AS group_id, + m.end_user_id AS end_user_id, m.dialog_id AS dialog_id, m.chunk_ids AS chunk_ids, m.content AS content, @@ -695,10 +652,10 @@ MEMORY_SUMMARY_EMBEDDING_SEARCH = """ CALL db.index.vector.queryNodes('summary_embedding_index', $limit * 100, $embedding) YIELD node AS m, score WHERE m.summary_embedding IS NOT NULL - AND ($group_id IS NULL OR m.group_id = $group_id) + AND ($end_user_id IS NULL OR m.end_user_id = $end_user_id) RETURN m.id AS id, m.name AS name, - m.group_id AS group_id, + m.end_user_id AS end_user_id, m.dialog_id AS dialog_id, m.chunk_ids AS chunk_ids, m.content AS content, @@ -718,9 +675,7 @@ MERGE (m:MemorySummary {id: summary.id}) SET m += { id: summary.id, name: summary.name, - group_id: summary.group_id, - user_id: summary.user_id, - apply_id: summary.apply_id, + end_user_id: summary.end_user_id, run_id: summary.run_id, created_at: summary.created_at, expired_at: summary.expired_at, @@ -745,7 +700,7 @@ MATCH (ms:MemorySummary {id: e.summary_id, run_id: e.run_id}) MATCH (c:Chunk {id: e.chunk_id, run_id: e.run_id}) MATCH (c)-[:CONTAINS]->(s:Statement {run_id: e.run_id}) MERGE (ms)-[r:DERIVED_FROM_STATEMENT]->(s) -SET r.group_id = e.group_id, +SET r.end_user_id = e.end_user_id, r.run_id = e.run_id, r.created_at = e.created_at, r.expired_at = e.expired_at @@ -774,7 +729,7 @@ FOREACH (rel IN CASE WHEN r IS NOT NULL THEN [r] ELSE [] END | source_statement_id: rel.source_statement_id, valid_at: rel.valid_at, invalid_at: rel.invalid_at, - group_id: rel.group_id, + end_user_id: rel.end_user_id, user_id: rel.user_id, apply_id: rel.apply_id, run_id: rel.run_id, @@ -796,7 +751,7 @@ FOREACH (rel IN CASE WHEN r IS NOT NULL THEN [r] ELSE [] END | source_statement_id: rel.source_statement_id, valid_at: rel.valid_at, invalid_at: rel.invalid_at, - group_id: rel.group_id, + end_user_id: rel.end_user_id, user_id: rel.user_id, apply_id: rel.apply_id, run_id: rel.run_id, @@ -814,7 +769,7 @@ RETURN count(losing) as deleted neo4j_statement_part = ''' MATCH (n:Statement) -WHERE n.group_id = "{}" +WHERE n.end_user_id = "{}" AND datetime(n.created_at) >= datetime() - duration('P3D') RETURN n.statement as statement_name, @@ -824,7 +779,7 @@ RETURN ''' neo4j_statement_all = ''' MATCH (n:Statement) -WHERE n.group_id = "{}" +WHERE n.end_user_id = "{}" RETURN n.statement as statement_name, n.id as statement_id @@ -832,7 +787,7 @@ RETURN ''' neo4j_query_part = """ MATCH (n)-[r]-(m:ExtractedEntity) - WHERE n.group_id = "{}" + WHERE n.end_user_id = "{}" AND datetime(n.created_at) >= datetime() - duration('P3D') WITH DISTINCT m OPTIONAL MATCH (m)-[rel]-(other:ExtractedEntity) @@ -853,7 +808,7 @@ neo4j_query_part = """ """ neo4j_query_all = """ MATCH (n)-[r]-(m:ExtractedEntity) - WHERE n.group_id = "{}" + WHERE n.end_user_id = "{}" WITH DISTINCT m OPTIONAL MATCH (m)-[rel]-(other:ExtractedEntity) RETURN @@ -1027,14 +982,14 @@ RETURN DISTINCT Memory_Space_User=""" MATCH (n)-[r]->(m) -WHERE n.group_id = $group_id AND m.name="用户" +WHERE n.end_user_id = $end_user_id AND m.name="用户" return DISTINCT elementId(m) as id """ Memory_Space_Entity=""" MATCH (n)-[]-(m) WHERE elementId(m) = $id AND m.entity_type = "Person" RETURN -DISTINCT m.name as name,m.group_id as group_id +DISTINCT m.name as name,m.end_user_id as end_user_id """ Memory_Space_Associative=""" MATCH (u)-[]-(x)-[]-(h) diff --git a/api/app/repositories/neo4j/dialog_repository.py b/api/app/repositories/neo4j/dialog_repository.py index ccb3d94c..020e7346 100644 --- a/api/app/repositories/neo4j/dialog_repository.py +++ b/api/app/repositories/neo4j/dialog_repository.py @@ -19,7 +19,7 @@ class DialogRepository(BaseNeo4jRepository[DialogueNode]): """对话仓储 管理对话节点的创建、查询、更新和删除操作。 - 提供按group_id、user_id、ref_id等条件查询对话的方法。 + 提供按end_user_id、user_id、ref_id等条件查询对话的方法。 Attributes: connector: Neo4j连接器实例 @@ -54,17 +54,17 @@ class DialogRepository(BaseNeo4jRepository[DialogueNode]): return DialogueNode(**n) - async def find_by_group_id(self, group_id: str, limit: int = 100) -> List[DialogueNode]: - """根据group_id查询对话 + async def find_by_end_user_id(self, end_user_id: str, limit: int = 100) -> List[DialogueNode]: + """根据end_user_id查询对话 Args: - group_id: 组ID + end_user_id: 组ID limit: 返回结果的最大数量 Returns: List[DialogueNode]: 对话列表 """ - return await self.find({"group_id": group_id}, limit=limit) + return await self.find({"end_user_id": end_user_id}, limit=limit) async def find_by_user_id(self, user_id: str, limit: int = 100) -> List[DialogueNode]: """根据user_id查询对话 @@ -94,14 +94,14 @@ class DialogRepository(BaseNeo4jRepository[DialogueNode]): async def find_by_group_and_user( self, - group_id: str, + end_user_id: str, user_id: str, limit: int = 100 ) -> List[DialogueNode]: - """根据group_id和user_id查询对话 + """根据end_user_id和user_id查询对话 Args: - group_id: 组ID + end_user_id: 组ID user_id: 用户ID limit: 返回结果的最大数量 @@ -109,20 +109,20 @@ class DialogRepository(BaseNeo4jRepository[DialogueNode]): List[DialogueNode]: 对话列表 """ return await self.find( - {"group_id": group_id, "user_id": user_id}, + {"end_user_id": end_user_id, "user_id": user_id}, limit=limit ) async def find_recent_dialogs( self, - group_id: str, + end_user_id: str, days: int = 7, limit: int = 100 ) -> List[DialogueNode]: """查询最近的对话 Args: - group_id: 组ID + end_user_id: 组ID days: 查询最近多少天的对话 limit: 返回结果的最大数量 @@ -131,7 +131,7 @@ class DialogRepository(BaseNeo4jRepository[DialogueNode]): """ query = f""" MATCH (n:{self.node_label}) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id AND n.created_at >= datetime() - duration({{days: $days}}) RETURN n ORDER BY n.created_at DESC @@ -139,7 +139,7 @@ class DialogRepository(BaseNeo4jRepository[DialogueNode]): """ results = await self.connector.execute_query( query, - group_id=group_id, + end_user_id=end_user_id, days=days, limit=limit ) @@ -164,22 +164,22 @@ class DialogRepository(BaseNeo4jRepository[DialogueNode]): async def find_by_config_and_group( self, config_id: str, - group_id: str, + end_user_id: str, limit: int = 100 ) -> List[DialogueNode]: - """根据config_id和group_id查询对话 + """根据config_id和end_user_id查询对话 支持按配置ID和组ID同时过滤,确保只返回使用特定配置处理的对话。 Args: config_id: 配置ID - group_id: 组ID + end_user_id: 组ID limit: 返回结果的最大数量 Returns: List[DialogueNode]: 对话列表 """ return await self.find( - {"config_id": config_id, "group_id": group_id}, + {"config_id": config_id, "end_user_id": end_user_id}, limit=limit ) diff --git a/api/app/repositories/neo4j/emotion_repository.py b/api/app/repositories/neo4j/emotion_repository.py index d445c8d4..e39968ac 100644 --- a/api/app/repositories/neo4j/emotion_repository.py +++ b/api/app/repositories/neo4j/emotion_repository.py @@ -40,7 +40,7 @@ class EmotionRepository: async def get_emotion_tags( self, - group_id: str, + end_user_id: str, emotion_type: Optional[str] = None, start_date: Optional[str] = None, end_date: Optional[str] = None, @@ -51,7 +51,7 @@ class EmotionRepository: 查询指定用户的情绪类型分布，包括计数、百分比和平均强度。 Args: - group_id: 用户组ID（宿主ID） + end_user_id: 用户组ID（宿主ID） emotion_type: 可选的情绪类型过滤（joy/sadness/anger/fear/surprise/neutral） start_date: 可选的开始日期（ISO格式字符串） end_date: 可选的结束日期（ISO格式字符串） @@ -65,8 +65,8 @@ class EmotionRepository: - avg_intensity: 平均强度 """ # 构建查询条件 - where_clauses = ["s.group_id = $group_id", "s.emotion_type IS NOT NULL"] - params = {"group_id": group_id, "limit": limit} + where_clauses = ["s.end_user_id = $end_user_id", "s.emotion_type IS NOT NULL"] + params = {"end_user_id": end_user_id, "limit": limit} if emotion_type: where_clauses.append("s.emotion_type = $emotion_type") @@ -119,7 +119,7 @@ class EmotionRepository: async def get_emotion_wordcloud( self, - group_id: str, + end_user_id: str, emotion_type: Optional[str] = None, limit: int = 50 ) -> List[Dict[str, Any]]: @@ -128,7 +128,7 @@ class EmotionRepository: 查询情绪关键词及其频率，用于生成词云可视化。 Args: - group_id: 用户组ID（宿主ID） + end_user_id: 用户组ID（宿主ID） emotion_type: 可选的情绪类型过滤 limit: 返回关键词的最大数量 @@ -140,8 +140,8 @@ class EmotionRepository: - avg_intensity: 平均强度 """ # 构建查询条件 - where_clauses = ["s.group_id = $group_id", "s.emotion_keywords IS NOT NULL"] - params = {"group_id": group_id, "limit": limit} + where_clauses = ["s.end_user_id = $end_user_id", "s.emotion_keywords IS NOT NULL"] + params = {"end_user_id": end_user_id, "limit": limit} if emotion_type: where_clauses.append("s.emotion_type = $emotion_type") @@ -186,7 +186,7 @@ class EmotionRepository: async def get_emotions_in_range( self, - group_id: str, + end_user_id: str, time_range: str = "30d" ) -> List[Dict[str, Any]]: """获取时间范围内的情绪数据 @@ -194,7 +194,7 @@ class EmotionRepository: 查询指定时间范围内的所有情绪数据，用于健康指数计算。 Args: - group_id: 用户组ID（宿主ID） + end_user_id: 用户组ID（宿主ID） time_range: 时间范围（7d/30d/90d） Returns: @@ -214,7 +214,7 @@ class EmotionRepository: # 优化的 Cypher 查询：使用字符串比较避免时区问题 query = """ MATCH (s:Statement) - WHERE s.group_id = $group_id + WHERE s.end_user_id = $end_user_id AND s.emotion_type IS NOT NULL AND s.created_at >= $start_date RETURN s.id as statement_id, @@ -227,7 +227,7 @@ class EmotionRepository: try: results = await self.connector.execute_query( query, - group_id=group_id, + end_user_id=end_user_id, start_date=start_date ) formatted_results = [ diff --git a/api/app/repositories/neo4j/graph_saver.py b/api/app/repositories/neo4j/graph_saver.py index 13215e0f..1575315f 100644 --- a/api/app/repositories/neo4j/graph_saver.py +++ b/api/app/repositories/neo4j/graph_saver.py @@ -44,9 +44,7 @@ async def save_entities_and_relationships( 'created_at': edge.created_at.isoformat(), 'expired_at': edge.expired_at.isoformat(), 'run_id': edge.run_id, - 'group_id': edge.group_id, - 'user_id': edge.user_id, - 'apply_id': edge.apply_id, + 'end_user_id': edge.end_user_id, } all_relationships.append(relationship) @@ -101,9 +99,7 @@ async def save_statement_chunk_edges( "id": edge.id, "source": edge.source, "target": edge.target, - "group_id": edge.group_id, - "user_id": edge.user_id, - "apply_id": edge.apply_id, + "end_user_id": edge.end_user_id, "run_id": edge.run_id, "created_at": edge.created_at.isoformat() if edge.created_at else None, "expired_at": edge.expired_at.isoformat() if edge.expired_at else None, @@ -132,9 +128,7 @@ async def save_statement_entity_edges( edge_data = { "source": edge.source, "target": edge.target, - "group_id": edge.group_id, - "user_id": edge.user_id, - "apply_id": edge.apply_id, + "end_user_id": edge.end_user_id, "run_id": edge.run_id, "connect_strength": edge.connect_strength, "created_at": edge.created_at.isoformat() if edge.created_at else None, diff --git a/api/app/repositories/neo4j/graph_search.py b/api/app/repositories/neo4j/graph_search.py index 6f5764b4..e8f52535 100644 --- a/api/app/repositories/neo4j/graph_search.py +++ b/api/app/repositories/neo4j/graph_search.py @@ -33,7 +33,7 @@ async def _update_activation_values_batch( connector: Neo4jConnector, nodes: List[Dict[str, Any]], node_label: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, max_retries: int = 3 ) -> List[Dict[str, Any]]: """ @@ -46,7 +46,7 @@ async def _update_activation_values_batch( connector: Neo4j连接器 nodes: 节点列表，每个节点必须包含 'id' 字段 node_label: 节点标签（Statement, ExtractedEntity, MemorySummary） - group_id: 组ID（可选） + end_user_id: 组ID（可选） max_retries: 最大重试次数 Returns: @@ -97,7 +97,7 @@ async def _update_activation_values_batch( updated_nodes = await access_manager.record_batch_access( node_ids=unique_node_ids, node_label=node_label, - group_id=group_id + end_user_id=end_user_id ) logger.info( @@ -118,7 +118,7 @@ async def _update_activation_values_batch( async def _update_search_results_activation( connector: Neo4jConnector, results: Dict[str, List[Dict[str, Any]]], - group_id: Optional[str] = None + end_user_id: Optional[str] = None ) -> Dict[str, List[Dict[str, Any]]]: """ 更新搜索结果中所有知识节点的激活值 @@ -129,7 +129,7 @@ async def _update_search_results_activation( Args: connector: Neo4j连接器 results: 搜索结果字典，包含不同类型节点的列表 - group_id: 组ID（可选） + end_user_id: 组ID（可选） Returns: Dict[str, List[Dict[str, Any]]]: 更新后的搜索结果 @@ -152,7 +152,7 @@ async def _update_search_results_activation( connector=connector, nodes=results[key], node_label=label, - group_id=group_id + end_user_id=end_user_id ) ) update_keys.append(key) @@ -218,7 +218,7 @@ async def _update_search_results_activation( async def search_graph( connector: Neo4jConnector, q: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, limit: int = 50, include: List[str] = None, ) -> Dict[str, List[Dict[str, Any]]]: @@ -236,7 +236,7 @@ async def search_graph( Args: connector: Neo4j connector q: Query text - group_id: Optional group filter + end_user_id: Optional group filter limit: Max results per category include: List of categories to search (default: all) @@ -254,7 +254,7 @@ async def search_graph( tasks.append(connector.execute_query( SEARCH_STATEMENTS_BY_KEYWORD, q=q, - group_id=group_id, + end_user_id=end_user_id, limit=limit, )) task_keys.append("statements") @@ -263,7 +263,7 @@ async def search_graph( tasks.append(connector.execute_query( SEARCH_ENTITIES_BY_NAME, q=q, - group_id=group_id, + end_user_id=end_user_id, limit=limit, )) task_keys.append("entities") @@ -272,7 +272,7 @@ async def search_graph( tasks.append(connector.execute_query( SEARCH_CHUNKS_BY_CONTENT, q=q, - group_id=group_id, + end_user_id=end_user_id, limit=limit, )) task_keys.append("chunks") @@ -281,7 +281,7 @@ async def search_graph( tasks.append(connector.execute_query( SEARCH_MEMORY_SUMMARIES_BY_KEYWORD, q=q, - group_id=group_id, + end_user_id=end_user_id, limit=limit, )) task_keys.append("summaries") @@ -310,12 +310,12 @@ async def search_graph( key in include and key in results and results[key] for key in ['statements', 'entities', 'chunks'] ) - + if needs_activation_update: results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) return results @@ -325,7 +325,7 @@ async def search_graph_by_embedding( connector: Neo4jConnector, embedder_client, query_text: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, limit: int = 50, include: List[str] = ["statements", "chunks", "entities","summaries"], ) -> Dict[str, List[Dict[str, Any]]]: @@ -337,7 +337,7 @@ async def search_graph_by_embedding( - Computes query embedding with the provided embedder_client - Ranks by cosine similarity in Cypher - - Filters by group_id if provided + - Filters by end_user_id if provided - Returns up to 'limit' per included type """ import time @@ -346,7 +346,7 @@ async def search_graph_by_embedding( embed_start = time.time() embeddings = await embedder_client.response([query_text]) embed_time = time.time() - embed_start - logger.info(f"[PERF] Embedding generation took: {embed_time:.4f}s") + print(f"[PERF] Embedding generation took: {embed_time:.4f}s") if not embeddings or not embeddings[0]: return {"statements": [], "chunks": [], "entities": [], "summaries": []} @@ -361,7 +361,7 @@ async def search_graph_by_embedding( tasks.append(connector.execute_query( STATEMENT_EMBEDDING_SEARCH, embedding=embedding, - group_id=group_id, + end_user_id=end_user_id, limit=limit, )) task_keys.append("statements") @@ -371,7 +371,7 @@ async def search_graph_by_embedding( tasks.append(connector.execute_query( CHUNK_EMBEDDING_SEARCH, embedding=embedding, - group_id=group_id, + end_user_id=end_user_id, limit=limit, )) task_keys.append("chunks") @@ -381,7 +381,7 @@ async def search_graph_by_embedding( tasks.append(connector.execute_query( ENTITY_EMBEDDING_SEARCH, embedding=embedding, - group_id=group_id, + end_user_id=end_user_id, limit=limit, )) task_keys.append("entities") @@ -391,7 +391,7 @@ async def search_graph_by_embedding( tasks.append(connector.execute_query( MEMORY_SUMMARY_EMBEDDING_SEARCH, embedding=embedding, - group_id=group_id, + end_user_id=end_user_id, limit=limit, )) task_keys.append("summaries") @@ -400,7 +400,7 @@ async def search_graph_by_embedding( query_start = time.time() task_results = await asyncio.gather(*tasks, return_exceptions=True) query_time = time.time() - query_start - logger.info(f"[PERF] Neo4j queries (parallel) took: {query_time:.4f}s") + print(f"[PERF] Neo4j queries (parallel) took: {query_time:.4f}s") # Build results dictionary results: Dict[str, List[Dict[str, Any]]] = { @@ -429,13 +429,13 @@ async def search_graph_by_embedding( key in include and key in results and results[key] for key in ['statements', 'entities', 'chunks'] ) - + if needs_activation_update: update_start = time.time() results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) update_time = time.time() - update_start logger.info(f"[PERF] Activation value updates took: {update_time:.4f}s") @@ -445,7 +445,7 @@ async def search_graph_by_embedding( return results async def get_dedup_candidates_for_entities( # 适配新版查询：使用全文索引按名称检索候选实体 connector: Neo4jConnector, - group_id: str, + end_user_id: str, entities: List[Dict[str, Any]], use_contains_fallback: bool = True, batch_size: int = 500, @@ -453,7 +453,7 @@ async def get_dedup_candidates_for_entities( # 适配新版查询：使用全 ) -> Dict[str, List[Dict[str, Any]]]: """ 为第二层去重消歧批量检索候选实体（适配新版 cypher_queries）： - - 使用全文索引查询 `SEARCH_ENTITIES_BY_NAME` 按 (group_id, name) 检索候选； + - 使用全文索引查询 `SEARCH_ENTITIES_BY_NAME` 按 (end_user_id, name) 检索候选； - 保留并发控制与返回结构（incoming_id -> [db_entity_props...]）； - 若提供 `entity_type`，在本地对返回结果做类型过滤； - `use_contains_fallback` 保留形参以兼容，必要时可扩展二次查询策略。 @@ -477,7 +477,7 @@ async def get_dedup_candidates_for_entities( # 适配新版查询：使用全 rows = await connector.execute_query( SEARCH_ENTITIES_BY_NAME, q=name, - group_id=group_id, + end_user_id=end_user_id, limit=100, ) except Exception: @@ -501,7 +501,7 @@ async def get_dedup_candidates_for_entities( # 适配新版查询：使用全 rows = await connector.execute_query( SEARCH_ENTITIES_BY_NAME, q=name.lower(), - group_id=group_id, + end_user_id=end_user_id, limit=100, ) for r in rows: @@ -532,9 +532,7 @@ async def get_dedup_candidates_for_entities( # 适配新版查询：使用全 async def search_graph_by_keyword_temporal( connector: Neo4jConnector, query_text: str, - group_id: Optional[str] = None, - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = None, start_date: Optional[str] = None, end_date: Optional[str] = None, valid_date: Optional[str] = None, @@ -547,32 +545,30 @@ async def search_graph_by_keyword_temporal( INTEGRATED: Updates activation values for Statement nodes before returning results - Matches statements containing query_text created between start_date and end_date - - Optionally filters by group_id, apply_id, user_id + - Optionally filters by end_user_id, apply_id, user_id - Returns up to 'limit' statements """ if not query_text: - logger.warning(f"query_text cannot be empty") + print(f"query_text不能为空") return {"statements": []} statements = await connector.execute_query( SEARCH_STATEMENTS_BY_KEYWORD_TEMPORAL, q=query_text, - group_id=group_id, - apply_id=apply_id, - user_id=user_id, + end_user_id=end_user_id, start_date=start_date, end_date=end_date, valid_date=valid_date, invalid_date=invalid_date, limit=limit, ) - logger.debug(f"Temporal keyword search results: {len(statements)} statements found") + print(f"查询结果为：\n{statements}") # 更新 Statement 节点的激活值 results = {"statements": statements} results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) return results @@ -580,9 +576,7 @@ async def search_graph_by_keyword_temporal( async def search_graph_by_temporal( connector: Neo4jConnector, - group_id: Optional[str] = None, - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = None, start_date: Optional[str] = None, end_date: Optional[str] = None, valid_date: Optional[str] = None, @@ -595,14 +589,12 @@ async def search_graph_by_temporal( INTEGRATED: Updates activation values for Statement nodes before returning results - Matches statements created between start_date and end_date - - Optionally filters by group_id, apply_id, user_id + - Optionally filters by end_user_id - Returns up to 'limit' statements """ statements = await connector.execute_query( SEARCH_STATEMENTS_BY_TEMPORAL, - group_id=group_id, - apply_id=apply_id, - user_id=user_id, + end_user_id=end_user_id, start_date=start_date, end_date=end_date, valid_date=valid_date, @@ -610,16 +602,16 @@ async def search_graph_by_temporal( limit=limit, ) - logger.debug(f"Temporal search query: {SEARCH_STATEMENTS_BY_TEMPORAL}") - logger.debug(f"Query params: group_id={group_id}, apply_id={apply_id}, user_id={user_id}, start_date={start_date}, end_date={end_date}, valid_date={valid_date}, invalid_date={invalid_date}, limit={limit}") - logger.debug(f"Temporal search results: {len(statements)} statements found") + print(f"查询语句为：\n{SEARCH_STATEMENTS_BY_TEMPORAL}") + print(f"查询参数为：\n{{end_user_id: {end_user_id}, start_date: {start_date}, end_date: {end_date}, valid_date: {valid_date}, invalid_date: {invalid_date}, limit: {limit}}}") + print(f"查询结果为：\n{statements}") # 更新 Statement 节点的激活值 results = {"statements": statements} results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) return results @@ -628,23 +620,23 @@ async def search_graph_by_temporal( async def search_graph_by_dialog_id( connector: Neo4jConnector, dialog_id: str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, limit: int = 1, ) -> Dict[str, List[Dict[str, Any]]]: """ Temporal search across Dialogues. - Matches dialogues with dialog_id - - Optionally filters by group_id + - Optionally filters by end_user_id - Returns up to 'limit' dialogues """ if not dialog_id: - logger.warning(f"dialog_id cannot be empty") + print(f"dialog_id不能为空") return {"dialogues": []} dialogues = await connector.execute_query( SEARCH_DIALOGUE_BY_DIALOG_ID, - group_id=group_id, + end_user_id=end_user_id, dialog_id=dialog_id, limit=limit, ) @@ -654,15 +646,15 @@ async def search_graph_by_dialog_id( async def search_graph_by_chunk_id( connector: Neo4jConnector, chunk_id : str, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, limit: int = 1, ) -> Dict[str, List[Dict[str, Any]]]: if not chunk_id: - logger.warning(f"chunk_id cannot be empty") + print(f"chunk_id不能为空") return {"chunks": []} chunks = await connector.execute_query( SEARCH_CHUNK_BY_CHUNK_ID, - group_id=group_id, + end_user_id=end_user_id, chunk_id=chunk_id, limit=limit, ) @@ -671,9 +663,9 @@ async def search_graph_by_chunk_id( async def search_graph_by_created_at( connector: Neo4jConnector, - group_id: Optional[str] = None, - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = None, + + created_at: Optional[str] = None, limit: int = 1, ) -> Dict[str, List[Dict[str, Any]]]: @@ -683,37 +675,37 @@ async def search_graph_by_created_at( INTEGRATED: Updates activation values for Statement nodes before returning results - Matches statements created at created_at - - Optionally filters by group_id, apply_id, user_id + - Optionally filters by end_user_id, apply_id, user_id - Returns up to 'limit' statements """ statements = await connector.execute_query( SEARCH_STATEMENTS_BY_CREATED_AT, - group_id=group_id, - apply_id=apply_id, - user_id=user_id, + end_user_id=end_user_id, + + created_at=created_at, limit=limit, ) - logger.debug(f"Search by created_at query: {SEARCH_STATEMENTS_BY_CREATED_AT}") - logger.debug(f"Query params: group_id={group_id}, apply_id={apply_id}, user_id={user_id}, created_at={created_at}, limit={limit}") - logger.debug(f"Search results: {len(statements)} statements found") + print(f"查询语句为：\n{SEARCH_STATEMENTS_BY_CREATED_AT}") + print(f"查询参数为：\n{{end_user_id: {end_user_id} created_at: {created_at}, limit: {limit}}}") + print(f"查询结果为：\n{statements}") # 更新 Statement 节点的激活值 results = {"statements": statements} results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) return results async def search_graph_by_valid_at( connector: Neo4jConnector, - group_id: Optional[str] = None, - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = None, + + valid_at: Optional[str] = None, limit: int = 1, ) -> Dict[str, List[Dict[str, Any]]]: @@ -723,37 +715,37 @@ async def search_graph_by_valid_at( INTEGRATED: Updates activation values for Statement nodes before returning results - Matches statements valid at valid_at - - Optionally filters by group_id, apply_id, user_id + - Optionally filters by end_user_id, apply_id, user_id - Returns up to 'limit' statements """ statements = await connector.execute_query( SEARCH_STATEMENTS_BY_VALID_AT, - group_id=group_id, - apply_id=apply_id, - user_id=user_id, + end_user_id=end_user_id, + + valid_at=valid_at, limit=limit, ) - logger.debug(f"Search by valid_at query: {SEARCH_STATEMENTS_BY_VALID_AT}") - logger.debug(f"Query params: group_id={group_id}, apply_id={apply_id}, user_id={user_id}, valid_at={valid_at}, limit={limit}") - logger.debug(f"Search results: {len(statements)} statements found") + print(f"查询语句为：\n{SEARCH_STATEMENTS_BY_VALID_AT}") + print(f"查询参数为：\n{{end_user_id: {end_user_id}， valid_at: {valid_at}, limit: {limit}}}") + print(f"查询结果为：\n{statements}") # 更新 Statement 节点的激活值 results = {"statements": statements} results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) return results async def search_graph_g_created_at( connector: Neo4jConnector, - group_id: Optional[str] = None, - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = None, + + created_at: Optional[str] = None, limit: int = 1, ) -> Dict[str, List[Dict[str, Any]]]: @@ -763,37 +755,37 @@ async def search_graph_g_created_at( INTEGRATED: Updates activation values for Statement nodes before returning results - Matches statements created at created_at - - Optionally filters by group_id, apply_id, user_id + - Optionally filters by end_user_id, apply_id, user_id - Returns up to 'limit' statements """ statements = await connector.execute_query( SEARCH_STATEMENTS_G_CREATED_AT, - group_id=group_id, - apply_id=apply_id, - user_id=user_id, + end_user_id=end_user_id, + + created_at=created_at, limit=limit, ) - logger.debug(f"Search greater than created_at query: {SEARCH_STATEMENTS_G_CREATED_AT}") - logger.debug(f"Query params: group_id={group_id}, apply_id={apply_id}, user_id={user_id}, created_at={created_at}, limit={limit}") - logger.debug(f"Search results: {len(statements)} statements found") + print(f"查询语句为：\n{SEARCH_STATEMENTS_G_CREATED_AT}") + print(f"查询参数为：\n{{end_user_id: {end_user_id}, created_at: {created_at}, limit: {limit}}}") + print(f"查询结果为：\n{statements}") # 更新 Statement 节点的激活值 results = {"statements": statements} results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) return results async def search_graph_g_valid_at( connector: Neo4jConnector, - group_id: Optional[str] = None, - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = None, + + valid_at: Optional[str] = None, limit: int = 1, ) -> Dict[str, List[Dict[str, Any]]]: @@ -803,37 +795,37 @@ async def search_graph_g_valid_at( INTEGRATED: Updates activation values for Statement nodes before returning results - Matches statements valid at valid_at - - Optionally filters by group_id, apply_id, user_id + - Optionally filters by end_user_id, apply_id, user_id - Returns up to 'limit' statements """ statements = await connector.execute_query( SEARCH_STATEMENTS_G_VALID_AT, - group_id=group_id, - apply_id=apply_id, - user_id=user_id, + end_user_id=end_user_id, + + valid_at=valid_at, limit=limit, ) - logger.debug(f"Search greater than valid_at query: {SEARCH_STATEMENTS_G_VALID_AT}") - logger.debug(f"Query params: group_id={group_id}, apply_id={apply_id}, user_id={user_id}, valid_at={valid_at}, limit={limit}") - logger.debug(f"Search results: {len(statements)} statements found") + print(f"查询语句为：\n{SEARCH_STATEMENTS_G_VALID_AT}") + print(f"查询参数为：\n{{end_user_id: {end_user_id}, valid_at: {valid_at}, limit: {limit}}}") + print(f"查询结果为：\n{statements}") # 更新 Statement 节点的激活值 results = {"statements": statements} results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) return results async def search_graph_l_created_at( connector: Neo4jConnector, - group_id: Optional[str] = None, - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = None, + + created_at: Optional[str] = None, limit: int = 1, ) -> Dict[str, List[Dict[str, Any]]]: @@ -843,37 +835,37 @@ async def search_graph_l_created_at( INTEGRATED: Updates activation values for Statement nodes before returning results - Matches statements created at created_at - - Optionally filters by group_id, apply_id, user_id + - Optionally filters by end_user_id, apply_id, user_id - Returns up to 'limit' statements """ statements = await connector.execute_query( SEARCH_STATEMENTS_L_CREATED_AT, - group_id=group_id, - apply_id=apply_id, - user_id=user_id, + end_user_id=end_user_id, + + created_at=created_at, limit=limit, ) - logger.debug(f"Search less than created_at query: {SEARCH_STATEMENTS_L_CREATED_AT}") - logger.debug(f"Query params: group_id={group_id}, apply_id={apply_id}, user_id={user_id}, created_at={created_at}, limit={limit}") - logger.debug(f"Search results: {len(statements)} statements found") + print(f"查询语句为：\n{SEARCH_STATEMENTS_L_CREATED_AT}") + print(f"查询参数为：\n{{end_user_id: {end_user_id}, created_at: {created_at}, limit: {limit}}}") + print(f"查询结果为：\n{statements}") # 更新 Statement 节点的激活值 results = {"statements": statements} results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) return results async def search_graph_l_valid_at( connector: Neo4jConnector, - group_id: Optional[str] = None, - apply_id: Optional[str] = None, - user_id: Optional[str] = None, + end_user_id: Optional[str] = None, + + valid_at: Optional[str] = None, limit: int = 1, ) -> Dict[str, List[Dict[str, Any]]]: @@ -883,28 +875,28 @@ async def search_graph_l_valid_at( INTEGRATED: Updates activation values for Statement nodes before returning results - Matches statements valid at valid_at - - Optionally filters by group_id, apply_id, user_id + - Optionally filters by end_user_id, apply_id, user_id - Returns up to 'limit' statements """ statements = await connector.execute_query( SEARCH_STATEMENTS_L_VALID_AT, - group_id=group_id, - apply_id=apply_id, - user_id=user_id, + end_user_id=end_user_id, + + valid_at=valid_at, limit=limit, ) - logger.debug(f"Search less than valid_at query: {SEARCH_STATEMENTS_L_VALID_AT}") - logger.debug(f"Query params: group_id={group_id}, apply_id={apply_id}, user_id={user_id}, valid_at={valid_at}, limit={limit}") - logger.debug(f"Search results: {len(statements)} statements found") + print(f"查询语句为：\n{SEARCH_STATEMENTS_L_VALID_AT}") + print(f"查询参数为：\n{{end_user_id: {end_user_id}, valid_at: {valid_at}, limit: {limit}}}") + print(f"查询结果为：\n{statements}") # 更新 Statement 节点的激活值 results = {"statements": statements} results = await _update_search_results_activation( connector=connector, results=results, - group_id=group_id + end_user_id=end_user_id ) return results diff --git a/api/app/repositories/neo4j/memory_summary_repository.py b/api/app/repositories/neo4j/memory_summary_repository.py index fc743f33..d7cd4fd4 100644 --- a/api/app/repositories/neo4j/memory_summary_repository.py +++ b/api/app/repositories/neo4j/memory_summary_repository.py @@ -18,7 +18,7 @@ class MemorySummaryRepository(BaseNeo4jRepository): """Memory Summary Repository Manages CRUD operations for MemorySummary nodes. - Provides methods to query summaries by group_id, user_id, and time ranges. + Provides methods to query summaries by end_user_id, user_id, and time ranges. Attributes: connector: Neo4j connector instance @@ -51,17 +51,17 @@ class MemorySummaryRepository(BaseNeo4jRepository): return dict(n) - async def find_by_group_id( + async def find_by_end_user_id( self, - group_id: str, + end_user_id: str, limit: int = 1000, start_date: Optional[datetime] = None, end_date: Optional[datetime] = None ) -> List[Dict[str, Any]]: - """Query memory summaries by group_id + """Query memory summaries by end_user_id Args: - group_id: Group ID to filter by + end_user_id: Group ID to filter by limit: Maximum number of results to return start_date: Optional start date filter end_date: Optional end date filter @@ -71,10 +71,10 @@ class MemorySummaryRepository(BaseNeo4jRepository): """ query = f""" MATCH (n:{self.node_label}) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id """ - params = {"group_id": group_id, "limit": limit} + params = {"end_user_id": end_user_id, "limit": limit} # Add date range filters if provided if start_date: @@ -139,16 +139,16 @@ class MemorySummaryRepository(BaseNeo4jRepository): async def find_by_group_and_user( self, - group_id: str, + end_user_id: str, user_id: str, limit: int = 1000, start_date: Optional[datetime] = None, end_date: Optional[datetime] = None ) -> List[Dict[str, Any]]: - """Query memory summaries by both group_id and user_id + """Query memory summaries by both end_user_id and user_id Args: - group_id: Group ID to filter by + end_user_id: Group ID to filter by user_id: User ID to filter by limit: Maximum number of results to return start_date: Optional start date filter @@ -159,10 +159,10 @@ class MemorySummaryRepository(BaseNeo4jRepository): """ query = f""" MATCH (n:{self.node_label}) - WHERE n.group_id = $group_id AND n.user_id = $user_id + WHERE n.end_user_id = $end_user_id AND n.user_id = $user_id """ - params = {"group_id": group_id, "user_id": user_id, "limit": limit} + params = {"end_user_id": end_user_id, "user_id": user_id, "limit": limit} # Add date range filters if provided if start_date: @@ -184,14 +184,14 @@ class MemorySummaryRepository(BaseNeo4jRepository): async def find_recent_summaries( self, - group_id: str, + end_user_id: str, days: int = 7, limit: int = 1000 ) -> List[Dict[str, Any]]: """Query recent memory summaries Args: - group_id: Group ID to filter by + end_user_id: Group ID to filter by days: Number of recent days to query limit: Maximum number of results to return @@ -200,7 +200,7 @@ class MemorySummaryRepository(BaseNeo4jRepository): """ query = f""" MATCH (n:{self.node_label}) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id AND n.created_at >= datetime() - duration({{days: $days}}) RETURN n ORDER BY n.created_at DESC @@ -209,7 +209,7 @@ class MemorySummaryRepository(BaseNeo4jRepository): results = await self.connector.execute_query( query, - group_id=group_id, + end_user_id=end_user_id, days=days, limit=limit ) @@ -217,14 +217,14 @@ class MemorySummaryRepository(BaseNeo4jRepository): async def find_by_content_keywords( self, - group_id: str, + end_user_id: str, keywords: List[str], limit: int = 100 ) -> List[Dict[str, Any]]: """Query memory summaries by content keywords Args: - group_id: Group ID to filter by + end_user_id: Group ID to filter by keywords: List of keywords to search for in content limit: Maximum number of results to return @@ -233,7 +233,7 @@ class MemorySummaryRepository(BaseNeo4jRepository): """ # Build keyword search conditions keyword_conditions = [] - params = {"group_id": group_id, "limit": limit} + params = {"end_user_id": end_user_id, "limit": limit} for i, keyword in enumerate(keywords): keyword_conditions.append(f"toLower(n.content) CONTAINS toLower($keyword_{i})") @@ -243,7 +243,7 @@ class MemorySummaryRepository(BaseNeo4jRepository): query = f""" MATCH (n:{self.node_label}) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id AND ({keyword_filter}) RETURN n ORDER BY n.created_at DESC @@ -253,21 +253,21 @@ class MemorySummaryRepository(BaseNeo4jRepository): results = await self.connector.execute_query(query, **params) return [self._map_to_dict(r) for r in results] - async def get_summary_count_by_group(self, group_id: str) -> int: + async def get_summary_count_by_group(self, end_user_id: str) -> int: """Get count of memory summaries for a group Args: - group_id: Group ID to count summaries for + end_user_id: Group ID to count summaries for Returns: int: Number of memory summaries """ query = f""" MATCH (n:{self.node_label}) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id RETURN count(n) as count """ - results = await self.connector.execute_query(query, group_id=group_id) + results = await self.connector.execute_query(query, end_user_id=end_user_id) return results[0]['count'] if results else 0 \ No newline at end of file diff --git a/api/app/repositories/neo4j/neo4j_connector.py b/api/app/repositories/neo4j/neo4j_connector.py index 7c4b43b5..d96e4431 100644 --- a/api/app/repositories/neo4j/neo4j_connector.py +++ b/api/app/repositories/neo4j/neo4j_connector.py @@ -70,11 +70,7 @@ class Neo4jConnector: List[Dict[str, Any]]: 查询结果列表，每个元素是一个字典 Example: - >>> connector = Neo4jConnector() - >>> results = await connector.execute_query( - ... "MATCH (n:Person {name: $name}) RETURN n", - ... name="Alice" - ... ) + """ result = await self.driver.execute_query( query, @@ -98,17 +94,7 @@ class Neo4jConnector: Any: 事务函数的返回值 Example: - >>> async def create_node(tx, name): - ... result = await tx.run( - ... "CREATE (n:Person {name: $name}) RETURN n", - ... name=name - ... ) - ... return await result.single() - >>> - >>> connector = Neo4jConnector() - >>> result = await connector.execute_write_transaction( - ... create_node, name="Alice" - ... ) + """ async with self.driver.session(database="neo4j") as session: return await session.execute_write(transaction_func, **kwargs) @@ -126,45 +112,33 @@ class Neo4jConnector: Any: 事务函数的返回值 Example: - >>> async def get_node(tx, name): - ... result = await tx.run( - ... "MATCH (n:Person {name: $name}) RETURN n", - ... name=name - ... ) - ... return await result.single() - >>> - >>> connector = Neo4jConnector() - >>> result = await connector.execute_read_transaction( - ... get_node, name="Alice" - ... ) + """ async with self.driver.session(database="neo4j") as session: return await session.execute_read(transaction_func, **kwargs) - async def delete_group(self, group_id: str): + async def delete_group(self, end_user_id: str): """删除指定组的所有数据 - 删除所有属于指定group_id的节点和边。 + 删除所有属于指定end_user_id的节点和边。 这是一个危险操作，会永久删除数据。 Args: - group_id: 要删除的组ID + end_user_id: 要删除的组ID Example: - >>> connector = Neo4jConnector() - >>> await connector.delete_group("group_123") Group group_123 deleted. """ # 删除节点（DETACH DELETE会同时删除相关的边） await self.driver.execute_query( - "MATCH (n) WHERE n.group_id = $group_id DETACH DELETE n", + "MATCH (n) WHERE n.end_user_id = $end_user_id DETACH DELETE n", database="neo4j", - group_id=group_id + end_user_id=end_user_id ) # 删除独立的边（如果有的话） await self.driver.execute_query( - "MATCH ()-[r]->() WHERE r.group_id = $group_id DELETE r", + "MATCH ()-[r]->() WHERE r.end_user_id = $end_user_id DELETE r", database="neo4j", - group_id=group_id + end_user_id=end_user_id ) - print(f"Group {group_id} deleted.") + print(f"Group {end_user_id} deleted.") diff --git a/api/app/repositories/neo4j/statement_repository.py b/api/app/repositories/neo4j/statement_repository.py index cd9f2fac..4f12af83 100644 --- a/api/app/repositories/neo4j/statement_repository.py +++ b/api/app/repositories/neo4j/statement_repository.py @@ -20,7 +20,7 @@ class StatementRepository(BaseNeo4jRepository[StatementNode]): """陈述句仓储 管理陈述句节点的创建、查询、更新和删除操作。 - 提供按chunk_id、group_id、向量相似度等条件查询陈述句的方法。 + 提供按chunk_id、end_user_id、向量相似度等条件查询陈述句的方法。 Attributes: connector: Neo4j连接器实例 diff --git a/api/app/repositories/user_repository.py b/api/app/repositories/user_repository.py index a43c5869..b4c11aa4 100644 --- a/api/app/repositories/user_repository.py +++ b/api/app/repositories/user_repository.py @@ -68,7 +68,7 @@ class UserRepository: db_logger.debug("查询超级用户") try: - user = self.db.query(User).options(joinedload(User.tenant)).filter(User.is_active == True).filter(User.is_superuser == True).first() + user = self.db.query(User).options(joinedload(User.tenant)).filter(User.is_active.is_(True)).filter(User.is_superuser.is_(True)).first() if user: db_logger.debug(f"超级用户查询成功: {user.username}") else: @@ -82,7 +82,7 @@ class UserRepository: db_logger.debug("检查是否只有一个超级用户") try: - count = self.db.query(User).options(joinedload(User.tenant)).filter(User.is_active == True).filter(User.is_superuser == True).count() + count = self.db.query(User).options(joinedload(User.tenant)).filter(User.is_active.is_(True)).filter(User.is_superuser.is_(True)).count() return count == 1 except Exception as e: db_logger.error(f"检查超级用户数量失败: {str(e)}") diff --git a/api/app/repositories/workflow_repository.py b/api/app/repositories/workflow_repository.py index 04734640..b22673e6 100644 --- a/api/app/repositories/workflow_repository.py +++ b/api/app/repositories/workflow_repository.py @@ -33,7 +33,7 @@ class WorkflowConfigRepository: """ return self.db.query(WorkflowConfig).filter( WorkflowConfig.app_id == app_id, - WorkflowConfig.is_active == True + WorkflowConfig.is_active.is_(True) ).first() def create_or_update( diff --git a/api/app/repositories/workspace_repository.py b/api/app/repositories/workspace_repository.py index 106830be..70ed7521 100644 --- a/api/app/repositories/workspace_repository.py +++ b/api/app/repositories/workspace_repository.py @@ -103,7 +103,7 @@ class WorkspaceRepository: workspaces = ( self.db.query(Workspace) .filter(Workspace.tenant_id == user.tenant_id) - .filter(Workspace.is_active == True) + .filter(Workspace.is_active.is_(True)) .order_by(Workspace.updated_at.desc()) .all() ) @@ -115,7 +115,7 @@ class WorkspaceRepository: self.db.query(Workspace) .join(WorkspaceMember, Workspace.id == WorkspaceMember.workspace_id) .filter(WorkspaceMember.user_id == user_id) - .filter(Workspace.is_active == True) + .filter(Workspace.is_active.is_(True)) .order_by(Workspace.updated_at.desc()) .all() ) @@ -134,7 +134,7 @@ class WorkspaceRepository: workspaces = ( self.db.query(Workspace) .filter(Workspace.tenant_id == tenant_id) - .filter(Workspace.is_active == True) + .filter(Workspace.is_active.is_(True)) .all() ) db_logger.debug(f"租户工作空间查询成功: tenant_id={tenant_id}, 数量={len(workspaces)}") @@ -169,7 +169,7 @@ class WorkspaceRepository: member = self.db.query(WorkspaceMember).filter( WorkspaceMember.user_id == user_id, WorkspaceMember.workspace_id == workspace_id, - WorkspaceMember.is_active == True, + WorkspaceMember.is_active.is_(True), ).first() if member: db_logger.debug(f"工作空间成员查询成功: user_id={user_id}, workspace_id={workspace_id}, role={member.role}") @@ -189,8 +189,8 @@ class WorkspaceRepository: .join(User, WorkspaceMember.user_id == User.id) .options(joinedload(WorkspaceMember.user), joinedload(WorkspaceMember.workspace)) .filter(WorkspaceMember.workspace_id == workspace_id) - .filter(WorkspaceMember.is_active == True) - .filter(User.is_active == True) + .filter(WorkspaceMember.is_active.is_(True)) + .filter(User.is_active.is_(True)) .all() ) db_logger.debug(f"成员列表查询成功: workspace_id={workspace_id}, 数量={len(members)}") @@ -208,8 +208,8 @@ class WorkspaceRepository: .join(User, WorkspaceMember.user_id == User.id) .options(joinedload(WorkspaceMember.user), joinedload(WorkspaceMember.workspace)) .filter(WorkspaceMember.id == member_id) - .filter(WorkspaceMember.is_active == True) - .filter(User.is_active == True) + .filter(WorkspaceMember.is_active.is_(True)) + .filter(User.is_active.is_(True)) .first() ) if member: @@ -226,7 +226,7 @@ class WorkspaceRepository: member = self.db.query(WorkspaceMember).filter( WorkspaceMember.workspace_id == workspace_id, WorkspaceMember.user_id == user_id, - WorkspaceMember.is_active == True, + WorkspaceMember.is_active.is_(True), ).first() if not member: return None @@ -243,7 +243,7 @@ class WorkspaceRepository: member = self.db.query(WorkspaceMember).filter( WorkspaceMember.workspace_id == workspace_id, WorkspaceMember.user_id == user_id, - WorkspaceMember.is_active == True, + WorkspaceMember.is_active.is_(True), ).first() if not member: return None @@ -259,7 +259,7 @@ class WorkspaceRepository: try: member = self.db.query(WorkspaceMember).filter( WorkspaceMember.id == member_id, - WorkspaceMember.is_active == True, + WorkspaceMember.is_active.is_(True), ).first() if not member: return None @@ -275,7 +275,7 @@ class WorkspaceRepository: try: member = self.db.query(WorkspaceMember).filter( WorkspaceMember.id == id, - WorkspaceMember.is_active == True, + WorkspaceMember.is_active.is_(True), ).first() if not member: return None diff --git a/api/app/schemas/app_schema.py b/api/app/schemas/app_schema.py index 35d2e424..09410091 100644 --- a/api/app/schemas/app_schema.py +++ b/api/app/schemas/app_schema.py @@ -299,6 +299,18 @@ class AppRelease(BaseModel): created_at: datetime.datetime updated_at: datetime.datetime + @field_validator("config", mode="before") + @classmethod + def parse_config(cls, v): + """处理 config 字段，如果是字符串则解析为字典""" + if isinstance(v, str): + import json + try: + return json.loads(v) + except json.JSONDecodeError: + return {} + return v if v is not None else {} + @field_serializer("created_at", when_used="json") def _serialize_created_at(self, dt: datetime.datetime): return int(dt.timestamp() * 1000) if dt else None diff --git a/api/app/schemas/emotion_schema.py b/api/app/schemas/emotion_schema.py index c48fbd41..13c802b5 100644 --- a/api/app/schemas/emotion_schema.py +++ b/api/app/schemas/emotion_schema.py @@ -1,11 +1,12 @@ """情绪分析相关的请求和响应模型""" from typing import Optional +from uuid import UUID from pydantic import BaseModel, Field class EmotionTagsRequest(BaseModel): """获取情绪标签统计请求""" - group_id: str = Field(..., description="组ID") + end_user_id: str = Field(..., description="组ID") emotion_type: Optional[str] = Field(None, description="情绪类型过滤（joy/sadness/anger/fear/surprise/neutral）") start_date: Optional[str] = Field(None, description="开始日期（ISO格式，如：2024-01-01）") end_date: Optional[str] = Field(None, description="结束日期（ISO格式，如：2024-12-31）") @@ -14,14 +15,14 @@ class EmotionTagsRequest(BaseModel): class EmotionWordcloudRequest(BaseModel): """获取情绪词云数据请求""" - group_id: str = Field(..., description="组ID") + end_user_id: str = Field(..., description="组ID") emotion_type: Optional[str] = Field(None, description="情绪类型过滤（joy/sadness/anger/fear/surprise/neutral）") limit: int = Field(50, ge=1, le=200, description="返回词语数量") class EmotionHealthRequest(BaseModel): """获取情绪健康指数请求""" - group_id: str = Field(..., description="组ID") + end_user_id: str = Field(..., description="组ID") time_range: str = Field("30d", description="时间范围（7d/30d/90d）") @@ -29,8 +30,8 @@ class EmotionHealthRequest(BaseModel): class EmotionSuggestionsRequest(BaseModel): """获取个性化情绪建议请求""" - group_id: str = Field(..., description="组ID") - config_id: Optional[int] = Field(None, description="配置ID（用于指定LLM模型）") + end_user_id: str = Field(..., description="组ID") + config_id: Optional[UUID] = Field(None, description="配置ID（用于指定LLM模型）") class EmotionGenerateSuggestionsRequest(BaseModel): diff --git a/api/app/schemas/memory_agent_schema.py b/api/app/schemas/memory_agent_schema.py index d4354c40..b6f50dd7 100644 --- a/api/app/schemas/memory_agent_schema.py +++ b/api/app/schemas/memory_agent_schema.py @@ -7,11 +7,11 @@ class UserInput(BaseModel): message: str history: list[dict] search_switch: str - group_id: str + end_user_id: str config_id: Optional[str] = None class Write_UserInput(BaseModel): messages: list[dict] - group_id: str - config_id: Optional[str] = None + end_user_id: str + config_id: Optional[str] = None \ No newline at end of file diff --git a/api/app/schemas/memory_config_schema.py b/api/app/schemas/memory_config_schema.py index 0443dcc4..76acee5c 100644 --- a/api/app/schemas/memory_config_schema.py +++ b/api/app/schemas/memory_config_schema.py @@ -35,7 +35,7 @@ class ConfigurationError(Exception): def __init__( self, message: str, - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, workspace_id: Optional[UUID] = None, context: Optional[Dict[str, Any]] = None, ): @@ -72,7 +72,7 @@ class WorkspaceNotFoundError(ConfigurationError): def __init__( self, workspace_id: UUID, - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, message: Optional[str] = None, ): if message is None: @@ -89,7 +89,7 @@ class ModelNotFoundError(ConfigurationError): self, model_id: Union[str, UUID], model_type: str, - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, workspace_id: Optional[UUID] = None, message: Optional[str] = None, ): @@ -112,7 +112,7 @@ class ModelInactiveError(ConfigurationError): model_id: Union[str, UUID], model_name: str, model_type: str, - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, workspace_id: Optional[UUID] = None, message: Optional[str] = None, ): @@ -136,7 +136,7 @@ class InvalidConfigError(ConfigurationError): message: str, field_name: Optional[str] = None, invalid_value: Optional[Any] = None, - config_id: Optional[int] = None, + config_id: Optional[UUID] = None, workspace_id: Optional[UUID] = None, ): context = {} @@ -155,7 +155,7 @@ class InvalidConfigError(ConfigurationError): class MemoryConfigValidation(BaseModel): """Pydantic model for validating memory configuration data from database.""" - config_id: int = Field(..., gt=0, description="Configuration ID must be positive") + config_id: UUID = Field(..., description="Configuration ID (UUID)") config_name: str = Field(..., min_length=1, max_length=255) workspace_id: UUID = Field(..., description="Workspace UUID") workspace_name: str = Field(..., min_length=1, max_length=255) @@ -275,7 +275,7 @@ class ModelValidation(BaseModel): def validate_memory_config_data( - config_data: Dict[str, Any], config_id: Optional[int] = None + config_data: Dict[str, Any], config_id: Optional[UUID] = None ) -> MemoryConfigValidation: """Validate memory configuration data using Pydantic model.""" try: @@ -302,7 +302,7 @@ def validate_memory_config_data( def validate_workspace_data( - workspace_data: Dict[str, Any], config_id: Optional[int] = None + workspace_data: Dict[str, Any], config_id: Optional[UUID] = None ) -> WorkspaceValidation: """Validate workspace data using Pydantic model.""" try: @@ -331,7 +331,7 @@ def validate_workspace_data( def validate_model_data( - model_data: Dict[str, Any], config_id: Optional[int] = None + model_data: Dict[str, Any], config_id: Optional[UUID] = None ) -> ModelValidation: """Validate model data using Pydantic model.""" try: @@ -364,7 +364,7 @@ def validate_model_data( class MemoryConfig: """Immutable memory configuration loaded from database.""" - config_id: int + config_id: UUID config_name: str workspace_id: UUID workspace_name: str diff --git a/api/app/schemas/memory_perceptual_schema.py b/api/app/schemas/memory_perceptual_schema.py index 05e01d2a..7dfefe01 100644 --- a/api/app/schemas/memory_perceptual_schema.py +++ b/api/app/schemas/memory_perceptual_schema.py @@ -4,7 +4,7 @@ from typing import Optional from pydantic import BaseModel, Field -from app.models.memory_perceptual_model import PerceptualType, FileStorageType +from app.models.memory_perceptual_model import PerceptualType, FileStorageService class PerceptualFilter(BaseModel): @@ -38,12 +38,14 @@ class PerceptualMemoryItem(BaseModel): """感知记忆项""" id: uuid.UUID = Field(..., description="Unique memory ID") perceptual_type: PerceptualType = Field(..., description="Type of perception, e.g., text, audio, or video") + storage_service: FileStorageService = Field(..., description="Storage service for file") file_path: str = Field(..., description="File path in the storage service") - file_ext: str = Field(..., description="File extension") file_name: str = Field(..., description="File name") + file_ext: str = Field(..., description="File extension") summary: Optional[str] = Field(None, description="summary") - storage_type: FileStorageType = Field(..., description="Storage type for file") + meta_data: Optional[dict] = Field(None, description="Metadata information") created_time: int = Field(..., description="create time") + topic: str = Field(..., description="topic") domain: str = Field(..., description="domain") keywords: list[str] = Field(..., description="keywords") diff --git a/api/app/schemas/memory_reflection_schemas.py b/api/app/schemas/memory_reflection_schemas.py index 860f1ef1..88454364 100644 --- a/api/app/schemas/memory_reflection_schemas.py +++ b/api/app/schemas/memory_reflection_schemas.py @@ -1,5 +1,8 @@ +import uuid + from pydantic import BaseModel, Field -from typing import Optional +from typing import Optional, Union +from uuid import UUID from enum import Enum @@ -9,7 +12,7 @@ class OptimizationStrategy(str, Enum): ACCURACY_FIRST = "accuracy_first" BALANCED = "balanced" class Memory_Reflection(BaseModel): - config_id: Optional[int] = None + config_id: Union[uuid.UUID, int, str] = None reflection_enabled: bool reflection_period_in_hours: str reflexion_range: Optional[str] = "partial" diff --git a/api/app/schemas/memory_storage_schema.py b/api/app/schemas/memory_storage_schema.py index d17a9f2c..5fda0a1d 100644 --- a/api/app/schemas/memory_storage_schema.py +++ b/api/app/schemas/memory_storage_schema.py @@ -1,5 +1,5 @@ """ -所有的内容是放错误地方了，应该放在models + """ from typing import Any, Optional, List, Dict, Literal, Union @@ -8,20 +8,8 @@ import uuid from pydantic import BaseModel, Field, ConfigDict, field_validator, model_validator -# ============================================================================ -# 原 UserInput 相关 Schema (保留原有功能) -# ============================================================================ -class UserInput(BaseModel): - message: str - history: list[dict] - search_switch: str - group_id: str -class Write_UserInput(BaseModel): - message: str - group_id: str - # ============================================================================ # 从 json_schema.py 迁移的 Schema @@ -159,7 +147,7 @@ class ReflexionResultSchema(BaseModel): # Composite key identifying a config row class ConfigKey(BaseModel): # 配置参数键模型 model_config = ConfigDict(populate_by_name=True, extra="forbid") - config_id: int = Field("config_id", description="配置唯一标识（字符串）") + config_id:Union[uuid.UUID, int, str] = Field(..., description="配置唯一标识（UUID或int)") user_id: str = Field("user_id", description="用户标识（字符串）") apply_id: str = Field("apply_id", description="应用或场景标识（字符串）") @@ -250,17 +238,17 @@ class ConfigParamsCreate(BaseModel): # 创建配置参数模型（仅 body， class ConfigParamsDelete(BaseModel): # 删除配置参数模型（请求体） model_config = ConfigDict(populate_by_name=True, extra="forbid") # config_name: str = Field("配置名称", description="配置名称（字符串）") - config_id: int = Field("配置ID", description="配置ID（字符串）") + config_id:Union[uuid.UUID, int, str] = Field(..., description="配置ID（支持UUID、整数或字符串）") class ConfigUpdate(BaseModel): # 更新记忆萃取引擎配置参数时使用的模型 - config_id: Optional[int] = None + config_id: Union[uuid.UUID, int, str] = None config_name: str = Field("配置名称", description="配置名称（字符串）") config_desc: str = Field("配置描述", description="配置描述（字符串）") class ConfigUpdateExtracted(BaseModel): # 更新记忆萃取引擎配置参数时使用的模型 - config_id: Optional[int] = None + config_id:Union[uuid.UUID, int, str] = None llm_id: Optional[str] = Field(None, description="LLM模型配置ID") embedding_id: Optional[str] = Field(None, description="嵌入模型配置ID") rerank_id: Optional[str] = Field(None, description="重排序模型配置ID") @@ -327,14 +315,14 @@ class ConfigUpdateExtracted(BaseModel): # 更新记忆萃取引擎配置参数 class ConfigUpdateForget(BaseModel): # 更新遗忘引擎配置参数时使用的模型 # 遗忘引擎配置参数更新模型 - config_id: Optional[int] = None + config_id:Union[uuid.UUID, int, str] = None lambda_time: Optional[float] = Field(0.5, ge=0.0, le=1.0, description="最低保持度，0-1 小数；默认 0.5") lambda_mem: Optional[float] = Field(0.5, ge=0.0, le=1.0, description="遗忘率，0-1 小数；默认 0.5") offset: Optional[float] = Field(0.0, ge=0.0, le=1.0, description="偏移度，0-1 小数；默认 0.0") class ConfigPilotRun(BaseModel): # 试运行触发请求模型 - config_id: int = Field(..., description="配置ID（唯一）") + config_id:Union[uuid.UUID, int, str] = Field(..., description="配置ID（唯一，支持UUID、整数或字符串）") dialogue_text: str = Field(..., description="前端传入的对话文本，格式如 '用户: ...\nAI: ...' 可多行，试运行必填") model_config = ConfigDict(populate_by_name=True, extra="forbid") @@ -342,7 +330,7 @@ class ConfigPilotRun(BaseModel): # 试运行触发请求模型 class ConfigFilter(BaseModel): # 查询配置参数时使用的模型 model_config = ConfigDict(populate_by_name=True, extra="forbid") - config_id: Optional[int] = None + config_id: Union[uuid.UUID, int, str] = None user_id: Optional[str] = None apply_id: Optional[str] = None @@ -418,7 +406,7 @@ class ForgettingConfigResponse(BaseModel): """遗忘引擎配置响应模型""" model_config = ConfigDict(populate_by_name=True, extra="forbid") - config_id: int = Field(..., description="配置ID") + config_id: Union[uuid.UUID, int, str] = Field(..., description="配置ID（支持UUID、整数或字符串）") decay_constant: float = Field(..., description="衰减常数 d") lambda_time: float = Field(..., description="时间衰减参数") lambda_mem: float = Field(..., description="记忆衰减参数") @@ -435,8 +423,8 @@ class ForgettingConfigResponse(BaseModel): class ForgettingConfigUpdateRequest(BaseModel): """遗忘引擎配置更新请求模型""" model_config = ConfigDict(populate_by_name=True, extra="forbid") - - config_id: int = Field(..., description="配置ID") + + config_id: Union[uuid.UUID, int,str] = Field(..., description="配置唯一标识（UUID或int)") decay_constant: Optional[float] = Field(None, ge=0.0, le=1.0, description="衰减常数 d") lambda_time: Optional[float] = Field(None, ge=0.0, le=1.0, description="时间衰减参数") lambda_mem: Optional[float] = Field(None, ge=0.0, le=1.0, description="记忆衰减参数") @@ -511,7 +499,7 @@ class ForgettingCurveRequest(BaseModel): importance_score: float = Field(0.5, ge=0.0, le=1.0, description="重要性分数（0-1）") days: int = Field(60, ge=1, le=365, description="模拟天数（默认60天）") - config_id: Optional[int] = Field(None, description="配置ID（可选，如果为None则使用默认配置）") + config_id: Union[uuid.UUID, int, str] = Field(..., description="配置唯一标识（UUID或int)") class ForgettingCurveResponse(BaseModel): diff --git a/api/app/schemas/model_schema.py b/api/app/schemas/model_schema.py index 5b1fe6d9..a2d3650a 100644 --- a/api/app/schemas/model_schema.py +++ b/api/app/schemas/model_schema.py @@ -1,10 +1,12 @@ -from pydantic import BaseModel, Field, field_serializer, ConfigDict +from pydantic import BaseModel, Field, field_serializer, field_validator, ConfigDict from typing import Optional, List, Dict, Any import datetime import uuid -from app.models.models_model import ModelProvider, ModelType +from app.models.models_model import ModelProvider, ModelType, LoadBalanceStrategy +from app.core.logging_config import get_business_logger +schema_logger = get_business_logger() # ModelConfig Schemas @@ -12,15 +14,19 @@ class ModelConfigBase(BaseModel): """模型配置基础Schema""" name: str = Field(..., description="模型显示名称", max_length=255) type: ModelType = Field(..., description="模型类型") + logo: Optional[str] = Field(None, description="模型logo图片URL", max_length=255) description: Optional[str] = Field(None, description="模型描述") + provider: str = Field(..., description="供应商") config: Optional[Dict[str, Any]] = Field({}, description="模型配置参数") is_active: bool = Field(True, description="是否激活") is_public: bool = Field(False, description="是否公开") + load_balance_strategy: Optional[str] = Field(LoadBalanceStrategy.NONE.value, description="负载均衡策略") class ApiKeyCreateNested(BaseModel): """用于在创建模型时内嵌创建API Key的Schema""" model_name: str = Field(..., description="模型实际名称", max_length=255) + description: Optional[str] = Field(None, description="备注") provider: ModelProvider = Field(..., description="API Key提供商") api_key: str = Field(..., description="API密钥", max_length=500) api_base: Optional[str] = Field(None, description="API基础URL", max_length=500) @@ -30,10 +36,23 @@ class ApiKeyCreateNested(BaseModel): class ModelConfigCreate(ModelConfigBase): """创建模型配置Schema""" - api_keys: Optional[ApiKeyCreateNested] = Field(None, description="同时创建的API Key配置") + api_keys: Optional[List[ApiKeyCreateNested]] = Field(None, description="同时创建的API Key配置") skip_validation: Optional[bool] = Field(False, description="是否跳过配置验证") +class CompositeModelCreate(BaseModel): + """创建组合模型Schema""" + name: str = Field(..., description="组合模型名称", max_length=255) + type: Optional[ModelType] = Field(None, description="模型类型") + logo: Optional[str] = Field(None, description="模型logo图片URL", max_length=255) + description: Optional[str] = Field(None, description="模型描述") + config: Optional[Dict[str, Any]] = Field({}, description="模型配置参数") + is_active: bool = Field(True, description="是否激活") + is_public: bool = Field(False, description="是否公开") + api_key_ids: List[uuid.UUID] = Field(..., description="绑定的API Key ID列表") + load_balance_strategy: Optional[str] = Field(default=LoadBalanceStrategy.NONE.value, description="负载均衡策略") + + class ModelConfigUpdate(BaseModel): """更新模型配置Schema""" name: Optional[str] = Field(None, description="模型显示名称", max_length=255) @@ -53,22 +72,48 @@ class ModelConfig(ModelConfigBase): updated_at: datetime.datetime api_keys: List["ModelApiKey"] = [] + @field_validator("api_keys", mode="after") + @classmethod + def filter_active_api_keys(cls, api_keys: List["ModelApiKey"]) -> List["ModelApiKey"]: + return [key for key in api_keys if key.is_active] + + @field_serializer("created_at", when_used="json") + def _serialize_created_at(self, dt: datetime.datetime | None): + return int(dt.timestamp() * 1000) if dt else None + + @field_serializer("updated_at", when_used="json") + def _serialize_updated_at(self, dt: datetime.datetime): + return int(dt.timestamp() * 1000) if dt else None + # ModelApiKey Schemas -class ModelApiKeyBase(BaseModel): - """API Key基础Schema""" - model_name: str = Field(..., description="模型实际名称", max_length=255) +class ModelApiKeyCreateByProvider(BaseModel): + """基于供应商创建API Key Schema""" provider: ModelProvider = Field(..., description="API Key提供商") api_key: str = Field(..., description="API密钥", max_length=500) api_base: Optional[str] = Field(None, description="API基础URL", max_length=500) - config: Optional[Dict[str, Any]] = Field(None, description="API Key特定配置") + description: Optional[str] = Field(None, description="备注") + config: Optional[Dict[str, Any]] = Field({}, description="API Key特定配置") + is_active: bool = Field(True, description="是否激活") + priority: str = Field("1", description="优先级", max_length=10) + model_config_ids: Optional[List[uuid.UUID]] = Field(None, description="关联的模型配置ID列表") + + +class ModelApiKeyBase(BaseModel): + """API Key基础Schema""" + model_name: str = Field(..., description="模型实际名称", max_length=255) + description: Optional[str] = Field(None, description="备注") + provider: ModelProvider = Field(..., description="API Key提供商") + api_key: str = Field(..., description="API密钥", max_length=500) + api_base: Optional[str] = Field(None, description="API基础URL", max_length=500) + config: Optional[Dict[str, Any]] = Field({}, description="API Key特定配置") is_active: bool = Field(True, description="是否激活") priority: str = Field("1", description="优先级", max_length=10) class ModelApiKeyCreate(ModelApiKeyBase): """创建API Key Schema""" - model_config_id: uuid.UUID = Field(..., description="模型配置ID") + model_config_ids: Optional[List[uuid.UUID]] = Field(None, description="关联的模型配置ID列表") class ModelApiKeyUpdate(BaseModel): @@ -85,11 +130,54 @@ class ModelApiKeyUpdate(BaseModel): class ModelApiKey(ModelApiKeyBase): """API Key Schema""" id: uuid.UUID - model_config_id: uuid.UUID usage_count: str last_used_at: Optional[datetime.datetime] created_at: datetime.datetime updated_at: datetime.datetime + model_configs: Any = Field(default=None, exclude=True) + model_config_ids: List[uuid.UUID] = Field(default_factory=list, description="关联的模型配置ID列表") + + def model_post_init(self, __context: Any) -> None: + """实例化后强制提取 model_configs 的ID到 model_config_ids""" + # 如果手动传入了 model_config_ids，不覆盖 + if self.model_config_ids and len(self.model_config_ids) > 0: + return + + # 从 model_configs 提取ID（只提取与 model_name 相同的非组合模型） + if self.model_configs is not None: + try: + # 情况1：ORM 对象列表（SQLAlchemy 关联） + if hasattr(self.model_configs, '__iter__') and not isinstance(self.model_configs, dict): + self.model_config_ids = [ + mc.id for mc in self.model_configs + if hasattr(mc, 'id') + and not getattr(mc, 'is_composite', False) + and getattr(mc, 'name', None) == self.model_name + ] + # 情况2：字典列表 + elif isinstance(self.model_configs, list): + self.model_config_ids = [ + mc['id'] if isinstance(mc, dict) else mc.id + for mc in self.model_configs + if ((isinstance(mc, dict) + and 'id' in mc + and not mc.get('is_composite', False) + and mc.get('name') == self.model_name) or + (hasattr(mc, 'id') + and not getattr(mc, 'is_composite', False) + and getattr(mc, 'name', None) == self.model_name)) + ] + except Exception as e: + schema_logger.warning(f"提取 model_config_ids 失败：{e}") + self.model_config_ids = [] + + model_config = ConfigDict( + from_attributes=True, # 支持从 ORM 解析 + arbitrary_types_allowed=True, # 允许任意类型（ORM 对象） + populate_by_name=True, # 按属性名匹配字段 + validate_assignment=True # 确保赋值触发校验 + ) + @field_serializer("created_at", when_used="json") def _serialize_created_at(self, dt: datetime.datetime): @@ -98,15 +186,12 @@ class ModelApiKey(ModelApiKeyBase): @field_serializer("updated_at", when_used="json") def _serialize_updated_at(self, dt: datetime.datetime): return int(dt.timestamp() * 1000) if dt else None - - model_config = ConfigDict(from_attributes=True) @field_serializer("last_used_at", when_used="json") def _serialize_last_used_at(self, dt: datetime.datetime): return int(dt.timestamp() * 1000) if dt else None -# 查询和响应Schemas class ModelConfigQuery(BaseModel): """模型配置查询Schema""" type: Optional[List[ModelType]] = Field(None, description="模型类型筛选（支持多个）") @@ -117,6 +202,17 @@ class ModelConfigQuery(BaseModel): page: int = Field(1, description="页码", ge=1) pagesize: int = Field(10, description="每页数量", ge=1, le=100) + +# 查询和响应Schemas +class ModelConfigQueryNew(BaseModel): + """模型配置查询Schema""" + type: Optional[List[ModelType]] = Field(None, description="模型类型筛选（支持多个）") + provider: Optional[ModelProvider] = Field(None, description="提供商筛选(通过API Key)") + is_active: Optional[bool] = Field(None, description="激活状态筛选") + is_public: Optional[bool] = Field(None, description="公开状态筛选") + is_composite: Optional[bool] = Field(None, description="组合模型筛选") + search: Optional[str] = Field(None, description="搜索关键词", max_length=255) + class ModelMarketplace(BaseModel): """模型广场响应Schema""" llm_models: List[ModelConfig] = [] @@ -159,4 +255,53 @@ class ModelValidateResponse(BaseModel): # 更新前向引用 -ModelConfig.model_rebuild() \ No newline at end of file +ModelConfig.model_rebuild() + + +# ModelBase Schemas +class ModelBaseCreate(BaseModel): + """创建基础模型Schema""" + name: str = Field(..., description="模型唯一标识", max_length=255) + type: ModelType = Field(..., description="模型类型") + provider: ModelProvider = Field(..., description="提供商") + logo: Optional[str] = Field(None, description="模型logo图片URL", max_length=255) + description: Optional[str] = Field(None, description="模型描述") + is_official: bool = Field(True, description="是否供应商官方模型") + tags: List[str] = Field(default_factory=list, description="模型标签") + + +class ModelBaseUpdate(BaseModel): + """更新基础模型Schema""" + name: Optional[str] = Field(None, description="模型唯一标识", max_length=255) + type: Optional[ModelType] = Field(None, description="模型类型") + provider: Optional[ModelProvider] = Field(None, description="提供商") + logo: Optional[str] = Field(None, description="模型logo图片URL", max_length=255) + description: Optional[str] = Field(None, description="模型描述") + is_deprecated: Optional[bool] = Field(None, description="是否弃用") + is_official: Optional[bool] = Field(None, description="是否供应商官方模型") + tags: Optional[List[str]] = Field(None, description="模型标签") + + +class ModelBase(BaseModel): + """基础模型Schema""" + model_config = ConfigDict(from_attributes=True) + + id: uuid.UUID + name: str + type: str + provider: str + logo: Optional[str] + description: Optional[str] + is_deprecated: bool + is_official: bool + tags: List[str] + add_count: int + + +class ModelBaseQuery(BaseModel): + """基础模型查询Schema""" + type: Optional[ModelType] = Field(None, description="模型类型") + provider: Optional[ModelProvider] = Field(None, description="提供商") + is_official: Optional[bool] = Field(None, description="是否官方模型") + is_deprecated: Optional[bool] = Field(None, description="是否弃用") + search: Optional[str] = Field(None, description="搜索关键词", max_length=255) diff --git a/api/app/schemas/multi_agent_schema.py b/api/app/schemas/multi_agent_schema.py index c0d72cdd..8fba2929 100644 --- a/api/app/schemas/multi_agent_schema.py +++ b/api/app/schemas/multi_agent_schema.py @@ -4,7 +4,7 @@ import datetime from typing import Optional, List, Dict, Any, Union from pydantic import BaseModel, Field, ConfigDict, field_serializer -from app.schemas import ModelParameters +from app.schemas.app_schema import ModelParameters # ==================== 子 Agent 配置 ==================== diff --git a/api/app/schemas/release_share_schema.py b/api/app/schemas/release_share_schema.py index 069b78a9..47897847 100644 --- a/api/app/schemas/release_share_schema.py +++ b/api/app/schemas/release_share_schema.py @@ -1,7 +1,7 @@ import uuid import datetime from typing import Optional, List, Dict, Any -from pydantic import BaseModel, Field, ConfigDict, field_serializer +from pydantic import BaseModel, Field, ConfigDict, field_serializer, field_validator # ---------- Input Schemas ---------- @@ -88,6 +88,18 @@ class SharedReleaseInfo(BaseModel): # 嵌入配置 allow_embed: bool + @field_validator("config", mode="before") + @classmethod + def parse_config(cls, v): + """处理 config 字段，如果是字符串则解析为字典""" + if isinstance(v, str): + import json + try: + return json.loads(v) + except json.JSONDecodeError: + return {} + return v if v is not None else {} + class EmbedCode(BaseModel): """嵌入代码""" diff --git a/api/app/services/agent_registry.py b/api/app/services/agent_registry.py index 2b6d92e3..d221bbf5 100644 --- a/api/app/services/agent_registry.py +++ b/api/app/services/agent_registry.py @@ -55,8 +55,8 @@ class AgentRegistry: """ # 构建查询 stmt = select(AgentConfig).join(App).where( - AgentConfig.is_active == True, - App.is_active == True + AgentConfig.is_active.is_(True), + App.is_active.is_(True) ) # 工作空间过滤（同工作空间或公开） diff --git a/api/app/services/app_service.py b/api/app/services/app_service.py index 68acab1d..7ec4bc0e 100644 --- a/api/app/services/app_service.py +++ b/api/app/services/app_service.py @@ -758,7 +758,7 @@ class AppService: ) # 构建查询条件 - filters = [App.is_active == True] + filters = [App.is_active.is_(True)] if type: filters.append(App.type == type) if visibility: @@ -873,7 +873,7 @@ class AppService: self._validate_workspace_access(app, workspace_id) - stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active == True).order_by( + stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active.is_(True)).order_by( AgentConfig.updated_at.desc()) agent_cfg: Optional[AgentConfig] = self.db.scalars(stmt).first() now = datetime.datetime.now() @@ -1204,7 +1204,7 @@ class AppService: default_model_config_id = None if app.type == AppType.AGENT: - stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active == True).order_by( + stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active.is_(True)).order_by( AgentConfig.updated_at.desc()) agent_cfg = self.db.scalars(stmt).first() if not agent_cfg: @@ -1226,7 +1226,7 @@ class AppService: select(MultiAgentConfig) .where( MultiAgentConfig.app_id == app_id, - MultiAgentConfig.is_active == True + MultiAgentConfig.is_active.is_(True) ) .order_by(MultiAgentConfig.updated_at.desc()) ) @@ -1380,7 +1380,7 @@ class AppService: stmt = ( select(AppRelease) - .where(AppRelease.app_id == app_id, AppRelease.is_active == True) + .where(AppRelease.app_id == app_id, AppRelease.is_active.is_(True)) .order_by(AppRelease.version.desc()) ) return list(self.db.scalars(stmt).all()) diff --git a/api/app/services/app_statistics_service.py b/api/app/services/app_statistics_service.py new file mode 100644 index 00000000..c164924a --- /dev/null +++ b/api/app/services/app_statistics_service.py @@ -0,0 +1,193 @@ +"""应用统计服务""" +from datetime import datetime, timedelta +from typing import Dict, Any, List +import uuid +from sqlalchemy import func, and_, cast, Date +from sqlalchemy.orm import Session + +from app.models.conversation_model import Conversation, Message +from app.models.end_user_model import EndUser +from app.models.api_key_model import ApiKey, ApiKeyLog +from app.core.exceptions import BusinessException +from app.core.error_codes import BizCode + + +class AppStatisticsService: + """应用统计服务""" + + def __init__(self, db: Session): + self.db = db + + def get_app_statistics( + self, + app_id: uuid.UUID, + workspace_id: uuid.UUID, + start_date: int, + end_date: int + ) -> Dict[str, Any]: + """获取应用统计数据 + + Args: + app_id: 应用ID + workspace_id: 工作空间ID + start_date: 开始时间戳（毫秒） + end_date: 结束时间戳（毫秒） + + Returns: + 统计数据字典 + """ + # 将毫秒时间戳转换为 datetime + start_dt = datetime.fromtimestamp(start_date / 1000) + end_dt = datetime.fromtimestamp(end_date / 1000) + timedelta(days=1) + + # 1. 会话统计 + conversations_stats = self._get_conversations_statistics(app_id, workspace_id, start_dt, end_dt) + + # 2. 新增用户统计 + users_stats = self._get_new_users_statistics(app_id, start_dt, end_dt) + + # 3. API调用统计 + api_stats = self._get_api_calls_statistics(app_id, start_dt, end_dt) + + # 4. Token消耗统计 + token_stats = self._get_token_statistics(app_id, start_dt, end_dt) + + return { + "daily_conversations": conversations_stats["daily"], + "total_conversations": conversations_stats["total"], + "daily_new_users": users_stats["daily"], + "total_new_users": users_stats["total"], + "daily_api_calls": api_stats["daily"], + "total_api_calls": api_stats["total"], + "daily_tokens": token_stats["daily"], + "total_tokens": token_stats["total"] + } + + def _get_conversations_statistics( + self, + app_id: uuid.UUID, + workspace_id: uuid.UUID, + start_dt: datetime, + end_dt: datetime + ) -> Dict[str, Any]: + """获取会话统计""" + # 每日会话数 + daily_query = self.db.query( + cast(Conversation.created_at, Date).label('date'), + func.count(Conversation.id).label('count') + ).filter( + and_( + Conversation.app_id == app_id, + Conversation.workspace_id == workspace_id, + Conversation.created_at >= start_dt, + Conversation.created_at < end_dt + ) + ).group_by(cast(Conversation.created_at, Date)).all() + + daily_data = [{"date": str(row.date), "count": row.count} for row in daily_query] + total = sum(row["count"] for row in daily_data) + + return {"daily": daily_data, "total": total} + + def _get_new_users_statistics( + self, + app_id: uuid.UUID, + start_dt: datetime, + end_dt: datetime + ) -> Dict[str, Any]: + """获取新增用户统计""" + # 每日新增用户数 + daily_query = self.db.query( + cast(EndUser.created_at, Date).label('date'), + func.count(EndUser.id).label('count') + ).filter( + and_( + EndUser.app_id == app_id, + EndUser.created_at >= start_dt, + EndUser.created_at < end_dt + ) + ).group_by(cast(EndUser.created_at, Date)).all() + + daily_data = [{"date": str(row.date), "count": row.count} for row in daily_query] + total = sum(row["count"] for row in daily_data) + + return {"daily": daily_data, "total": total} + + def _get_api_calls_statistics( + self, + app_id: uuid.UUID, + start_dt: datetime, + end_dt: datetime + ) -> Dict[str, Any]: + """获取API调用统计""" + # 每日API调用次数 + daily_query = self.db.query( + cast(ApiKeyLog.created_at, Date).label('date'), + func.count(ApiKeyLog.id).label('count') + ).join( + ApiKey, ApiKeyLog.api_key_id == ApiKey.id + ).filter( + and_( + ApiKey.resource_id == app_id, + ApiKeyLog.created_at >= start_dt, + ApiKeyLog.created_at < end_dt + ) + ).group_by(cast(ApiKeyLog.created_at, Date)).all() + + daily_data = [{"date": str(row.date), "count": row.count} for row in daily_query] + total = sum(row["count"] for row in daily_data) + + return {"daily": daily_data, "total": total} + + def _get_token_statistics( + self, + app_id: uuid.UUID, + start_dt: datetime, + end_dt: datetime + ) -> Dict[str, Any]: + """获取Token消耗统计（从Message的meta_data中提取）""" + from sqlalchemy import text + + # 查询所有相关消息的token使用情况 + # meta_data中可能包含: {"usage": {"total_tokens": 100}} 或 {"tokens": 100} + daily_query = self.db.query( + cast(Message.created_at, Date).label('date'), + Message.meta_data + ).join( + Conversation, Message.conversation_id == Conversation.id + ).filter( + and_( + Conversation.app_id == app_id, + Message.created_at >= start_dt, + Message.created_at < end_dt, + Message.meta_data.isnot(None) + ) + ).all() + + # 按日期聚合token + daily_tokens = {} + for row in daily_query: + date_str = str(row.date) + meta = row.meta_data or {} + + # 提取token数量（支持多种格式） + tokens = 0 + if isinstance(meta, dict): + # 格式1: {"usage": {"total_tokens": 100}} + if "usage" in meta and isinstance(meta["usage"], dict): + tokens = meta["usage"].get("total_tokens", 0) + # 格式2: {"tokens": 100} + elif "tokens" in meta: + tokens = meta.get("tokens", 0) + # 格式3: {"total_tokens": 100} + elif "total_tokens" in meta: + tokens = meta.get("total_tokens", 0) + + if date_str not in daily_tokens: + daily_tokens[date_str] = 0 + daily_tokens[date_str] += int(tokens) + + daily_data = [{"date": date, "tokens": tokens} for date, tokens in sorted(daily_tokens.items()) if tokens != 0] + total = sum(row["tokens"] for row in daily_data) + + return {"daily": daily_data, "total": total} diff --git a/api/app/services/draft_run_service.py b/api/app/services/draft_run_service.py index 46bda5f6..524c9ff6 100644 --- a/api/app/services/draft_run_service.py +++ b/api/app/services/draft_run_service.py @@ -16,6 +16,7 @@ from app.core.exceptions import BusinessException from app.core.logging_config import get_business_logger from app.core.rag.nlp.search import knowledge_retrieval from app.models import AgentConfig, ModelApiKey, ModelConfig +from app.repositories.model_repository import ModelApiKeyRepository from app.repositories.tool_repository import ToolRepository from app.schemas.prompt_schema import PromptMessageRole, render_prompt_message from app.services import task_service @@ -56,7 +57,7 @@ def create_long_term_memory_tool(memory_config: Dict[str, Any], end_user_id: str 长期记忆工具 """ # search_switch = memory_config.get("search_switch", "2") - config_id= memory_config.get("memory_content",None) + config_id= memory_config.get("memory_content") or memory_config.get("memory_config",None) logger.info(f"创建长期记忆工具，配置: end_user_id={end_user_id}, config_id={config_id}, storage_type={storage_type}") @tool(args_schema=LongTermMemoryInput) def long_term_memory(question: str) -> str: @@ -92,7 +93,7 @@ def create_long_term_memory_tool(memory_config: Dict[str, Any], end_user_id: str try: memory_content = asyncio.run( MemoryAgentService().read_memory( - group_id=end_user_id, + end_user_id=end_user_id, message=question, history=[], search_switch="2", @@ -106,9 +107,9 @@ def create_long_term_memory_tool(memory_config: Dict[str, Any], end_user_id: str "app.core.memory.agent.read_message", args=[end_user_id, question, [], "1", config_id, storage_type, user_rag_memory_id] ) - # result = task_service.get_task_memory_read_result(task.id) - # status = result.get("status") - # logger.info(f"读取任务状态：{status}") + result = task_service.get_task_memory_read_result(task.id) + status = result.get("status") + logger.info(f"读取任务状态：{status}") finally: db.close() @@ -418,7 +419,7 @@ class DraftRunService: ) memory_config_= agent_config.memory - config_id = memory_config_.get("memory_content") + config_id = memory_config_.get("memory_content") or memory_config_.get("memory_config",None) # 7. 调用 Agent result = await agent.chat( @@ -644,7 +645,7 @@ class DraftRunService: }) memory_config_ = agent_config.memory - config_id = memory_config_.get("memory_content") + config_id = memory_config_.get("memory_content") or memory_config_.get("memory_config",None) # 9. 流式调用 Agent full_content = "" @@ -724,17 +725,21 @@ class DraftRunService: Raises: BusinessException: 当没有可用的 API Key 时 """ - stmt = ( - select(ModelApiKey) - .where( - ModelApiKey.model_config_id == model_config_id, - ModelApiKey.is_active == True - ) - .order_by(ModelApiKey.priority.desc()) - .limit(1) - ) - - api_key = self.db.scalars(stmt).first() + api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config_id) + # stmt = ( + # select(ModelApiKey).join( + # ModelConfig, ModelApiKey.model_configs + # ) + # .where( + # ModelConfig.id == model_config_id, + # ModelApiKey.is_active.is_(True) + # ) + # .order_by(ModelApiKey.priority.desc()) + # .limit(1) + # ) + # + # api_key = self.db.scalars(stmt).first() + api_key = api_keys[0] if api_keys else None if not api_key: raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING) diff --git a/api/app/services/emotion_analytics_service.py b/api/app/services/emotion_analytics_service.py index 601d2921..af98fb52 100644 --- a/api/app/services/emotion_analytics_service.py +++ b/api/app/services/emotion_analytics_service.py @@ -75,7 +75,7 @@ class EmotionAnalyticsService: # 调用仓储层查询 tags = await self.emotion_repo.get_emotion_tags( - group_id=end_user_id, + end_user_id=end_user_id, emotion_type=emotion_type, start_date=start_date, end_date=end_date, @@ -157,7 +157,7 @@ class EmotionAnalyticsService: # 调用仓储层查询 keywords = await self.emotion_repo.get_emotion_wordcloud( - group_id=end_user_id, + end_user_id=end_user_id, emotion_type=emotion_type, limit=limit ) @@ -339,7 +339,7 @@ class EmotionAnalyticsService: # 获取时间范围内的情绪数据 emotions = await self.emotion_repo.get_emotions_in_range( - group_id=end_user_id, + end_user_id=end_user_id, time_range=time_range ) @@ -505,7 +505,7 @@ class EmotionAnalyticsService: ) config_service = MemoryConfigService(db) memory_config = config_service.load_memory_config( - config_id=int(config_id), + config_id=(config_id), service_name="EmotionAnalyticsService.generate_emotion_suggestions" ) from app.core.memory.utils.llm.llm_utils import MemoryClientFactory @@ -519,7 +519,7 @@ class EmotionAnalyticsService: # 3. 获取情绪数据用于模式分析 emotions = await self.emotion_repo.get_emotions_in_range( - group_id=end_user_id, + end_user_id=end_user_id, time_range="30d" ) @@ -598,13 +598,13 @@ class EmotionAnalyticsService: # 查询用户的实体和标签 query = """ MATCH (e:Entity) - WHERE e.group_id = $group_id + WHERE e.end_user_id = $end_user_id RETURN e.name as name, e.type as type ORDER BY e.created_at DESC LIMIT 20 """ - entities = await connector.execute_query(query, group_id=end_user_id) + entities = await connector.execute_query(query, end_user_id=end_user_id) # 提取兴趣标签 interests = [e["name"] for e in entities if e.get("type") in ["INTEREST", "HOBBY"]][:5] diff --git a/api/app/services/emotion_config_service.py b/api/app/services/emotion_config_service.py index 37171640..9880d4e1 100644 --- a/api/app/services/emotion_config_service.py +++ b/api/app/services/emotion_config_service.py @@ -8,9 +8,11 @@ Classes: """ from typing import Dict, Any +from uuid import UUID + from sqlalchemy.orm import Session -from app.models.data_config_model import DataConfig +from app.models.memory_config_model import MemoryConfig from app.core.logging_config import get_business_logger logger = get_business_logger() @@ -37,7 +39,7 @@ class EmotionConfigService: self.db = db logger.info("情绪配置服务初始化完成") - def get_emotion_config(self, config_id: int) -> Dict[str, Any]: + def get_emotion_config(self, config_id: UUID) -> Dict[str, Any]: """获取情绪引擎配置 查询指定配置ID的情绪相关配置字段。 @@ -61,8 +63,8 @@ class EmotionConfigService: logger.info(f"获取情绪配置: config_id={config_id}") # 查询配置 - config = self.db.query(DataConfig).filter( - DataConfig.config_id == config_id + config = self.db.query(MemoryConfig).filter( + MemoryConfig.config_id == config_id ).first() if not config: @@ -144,7 +146,7 @@ class EmotionConfigService: def update_emotion_config( self, - config_id: int, + config_id: UUID, config_data: Dict[str, Any] ) -> Dict[str, Any]: """更新情绪引擎配置 @@ -173,8 +175,8 @@ class EmotionConfigService: self.validate_emotion_config(config_data) # 查询配置 - config = self.db.query(DataConfig).filter( - DataConfig.config_id == config_id + config = self.db.query(MemoryConfig).filter( + MemoryConfig.config_id == config_id ).first() if not config: diff --git a/api/app/services/emotion_extraction_service.py b/api/app/services/emotion_extraction_service.py index d134251d..6b596a80 100644 --- a/api/app/services/emotion_extraction_service.py +++ b/api/app/services/emotion_extraction_service.py @@ -14,7 +14,7 @@ from app.core.memory.llm_tools.llm_client import LLMClientException from app.core.memory.models.emotion_models import EmotionExtraction from app.core.memory.utils.llm.llm_utils import MemoryClientFactory from app.db import get_db_context -from app.models.data_config_model import DataConfig +from app.models.memory_config_model import MemoryConfig logger = logging.getLogger(__name__) @@ -60,7 +60,7 @@ class EmotionExtractionService: async def extract_emotion( self, statement: str, - config: DataConfig + config: MemoryConfig ) -> Optional[EmotionExtraction]: """Extract emotion information from a statement. diff --git a/api/app/services/llm_router.py b/api/app/services/llm_router.py index 9ef9dbb1..9e102ac3 100644 --- a/api/app/services/llm_router.py +++ b/api/app/services/llm_router.py @@ -5,6 +5,7 @@ import uuid from typing import Dict, Any, List, Optional, Tuple from sqlalchemy.orm import Session +from app.repositories.model_repository import ModelApiKeyRepository from app.services.conversation_state_manager import ConversationStateManager from app.models import ModelConfig, AgentConfig from app.core.logging_config import get_business_logger @@ -382,11 +383,14 @@ class LLMRouter: from app.core.models.base import RedBearModelConfig from app.models import ModelApiKey, ModelType - # 获取 API Key 配置 - api_key_config = self.db.query(ModelApiKey).filter( - ModelApiKey.model_config_id == self.routing_model_config.id, - ModelApiKey.is_active - ).first() + # 获取 API Key 配置（通过关联关系） + # api_key_config = self.db.query(ModelApiKey).join( + # ModelConfig, ModelApiKey.model_configs + # ).filter(ModelConfig.id == self.routing_model_config.id, + # ModelApiKey.is_active == True + # ).first() + api_keys = ModelApiKeyRepository.get_by_model_config(self.db, self.routing_model_config.id) + api_key_config = api_keys[0] if api_keys else None if not api_key_config: raise Exception("路由模型没有可用的 API Key") @@ -419,6 +423,9 @@ class LLMRouter: # 调用模型 response = await llm.ainvoke(prompt) + + from app.services.model_service import ModelApiKeyService + ModelApiKeyService.record_api_key_usage(self.db, api_key_config.id) # 提取响应内容 if hasattr(response, 'content'): diff --git a/api/app/services/master_agent_router.py b/api/app/services/master_agent_router.py index 3971aab7..87fdb22c 100644 --- a/api/app/services/master_agent_router.py +++ b/api/app/services/master_agent_router.py @@ -5,7 +5,7 @@ import uuid from typing import Dict, Any, List, Optional, Tuple from sqlalchemy.orm import Session -from app.schemas import ModelParameters +from app.schemas.app_schema import ModelParameters from app.services.conversation_state_manager import ConversationStateManager from app.models import ModelConfig, AgentConfig from app.core.logging_config import get_business_logger diff --git a/api/app/services/memory_agent_service.py b/api/app/services/memory_agent_service.py index 8170bdd8..823d5d43 100644 --- a/api/app/services/memory_agent_service.py +++ b/api/app/services/memory_agent_service.py @@ -9,6 +9,7 @@ import os import re import time import uuid +from uuid import UUID from typing import Any, AsyncGenerator, Dict, List, Optional import redis @@ -27,6 +28,7 @@ from app.core.memory.analytics.hot_memory_tags import get_hot_memory_tags from app.core.memory.utils.llm.llm_utils import MemoryClientFactory from app.db import get_db_context from app.models.knowledge_model import Knowledge, KnowledgeType +from app.repositories.memory_short_repository import ShortTermMemoryRepository from app.repositories.neo4j.neo4j_connector import Neo4jConnector from app.schemas.memory_agent_schema import Write_UserInput from app.schemas.memory_config_schema import ConfigurationError @@ -35,6 +37,7 @@ from app.services.memory_config_service import MemoryConfigService from app.services.memory_konwledges_server import ( write_rag, ) +from langchain_core.messages import AIMessage from langchain_core.messages import HumanMessage from pydantic import BaseModel, Field from sqlalchemy import func @@ -54,25 +57,24 @@ _neo4j_connector = Neo4jConnector() class MemoryAgentService: """Service for memory agent operations""" - def writer_messages_deal(self, messages, start_time, group_id, config_id, message, context): + def writer_messages_deal(self, messages, start_time, end_user_id, config_id, message, context): duration = time.time() - start_time - if str(messages) == 'success': - logger.info(f"Write operation successful for group {group_id} with config_id {config_id}") + logger.info(f"Write operation successful for group {end_user_id} with config_id {config_id}") # 记录成功的操作 if audit_logger: - audit_logger.log_operation(operation="WRITE", config_id=config_id, group_id=group_id, success=True, + audit_logger.log_operation(operation="WRITE", config_id=config_id, end_user_id=end_user_id, success=True, duration=duration, details={"message_length": len(message)}) return context else: - logger.warning(f"Write operation failed for group {group_id}") + logger.warning(f"Write operation failed for group {end_user_id}") # 记录失败的操作 if audit_logger: audit_logger.log_operation( operation="WRITE", config_id=config_id, - group_id=group_id, + end_user_id=end_user_id, success=False, duration=duration, error=f"写入失败: {messages[:100]}" @@ -173,10 +175,9 @@ class MemoryAgentService: """ logger.info("Reading log file") - - current_file = os.path.abspath(__file__) # app/services/memory_agent_service.py - app_dir = os.path.dirname(os.path.dirname(current_file)) # app directory - project_root = os.path.dirname(app_dir) # redbear-mem directory + # Get log file path - use project root directory + from pathlib import Path + project_root = str(Path(__file__).resolve().parents[2]) # api directory log_path = os.path.join(project_root, "logs", "agent_service.log") summer = '' @@ -215,9 +216,8 @@ class MemoryAgentService: logger.info("Starting log content streaming") # Get log file path - use project root directory - current_file = os.path.abspath(__file__) # app/services/memory_agent_service.py - app_dir = os.path.dirname(os.path.dirname(current_file)) # app directory - project_root = os.path.dirname(app_dir) # redbear-mem directory + from pathlib import Path + project_root = str(Path(__file__).resolve().parents[2]) # api directory log_path = os.path.join(project_root, "logs", "agent_service.log") # Check if file exists before starting stream @@ -265,13 +265,13 @@ class MemoryAgentService: logger.info("Log streaming completed, cleaning up resources") # LogStreamer uses context manager for file handling, so cleanup is automatic - async def write_memory(self, group_id: str, messages: list[dict], config_id: Optional[str], db: Session, storage_type: str, user_rag_memory_id: str) -> str: + async def write_memory(self, end_user_id: str, messages: list[dict], config_id: Optional[uuid.UUID]|int, db: Session, storage_type: str, user_rag_memory_id: str) -> str: """ Process write operation with config_id Args: - group_id: Group identifier (also used as end_user_id) - messages: Structured message list [{"role": "user", "content": "..."}, ...] + end_user_id: Group identifier (also used as end_user_id) + message: Message to write config_id: Configuration ID from database db: SQLAlchemy database session storage_type: Storage type (neo4j or rag) @@ -286,15 +286,15 @@ class MemoryAgentService: # Resolve config_id if None using end_user's connected config if config_id is None: try: - connected_config = get_end_user_connected_config(group_id, db) + connected_config = get_end_user_connected_config(end_user_id, db) config_id = connected_config.get("memory_config_id") if config_id is None: - raise ValueError(f"No memory configuration found for end_user {group_id}. Please ensure the user has a connected memory configuration.") + raise ValueError(f"No memory configuration found for end_user {end_user_id}. Please ensure the user has a connected memory configuration.") except Exception as e: if "No memory configuration found" in str(e): - raise - logger.error(f"Failed to get connected config for end_user {group_id}: {e}") - raise ValueError(f"Unable to determine memory configuration for end_user {group_id}: {e}") + raise # Re-raise our specific error + logger.error(f"Failed to get connected config for end_user {end_user_id}: {e}") + raise ValueError(f"Unable to determine memory configuration for end_user {end_user_id}: {e}") import time start_time = time.time() @@ -314,7 +314,7 @@ class MemoryAgentService: # Log failed operation if audit_logger: duration = time.time() - start_time - audit_logger.log_operation(operation="WRITE", config_id=config_id, group_id=group_id, success=False, duration=duration, error=error_msg) + audit_logger.log_operation(operation="WRITE", config_id=config_id, end_user_id=end_user_id, success=False, duration=duration, error=error_msg) raise ValueError(error_msg) @@ -322,24 +322,25 @@ class MemoryAgentService: if storage_type == "rag": # For RAG storage, convert messages to single string message_text = "\n".join([f"{msg['role']}: {msg['content']}" for msg in messages]) - result = await write_rag(group_id, message_text, user_rag_memory_id) + result = await write_rag(end_user_id, message_text, user_rag_memory_id) return result else: async with make_write_graph() as graph: - config = {"configurable": {"thread_id": group_id}} + config = {"configurable": {"thread_id": end_user_id}} # Convert structured messages to LangChain messages langchain_messages = [] for msg in messages: if msg['role'] == 'user': langchain_messages.append(HumanMessage(content=msg['content'])) elif msg['role'] == 'assistant': - from langchain_core.messages import AIMessage langchain_messages.append(AIMessage(content=msg['content'])) - + print(100*'-') + print(langchain_messages) + print(100*'-') # 初始状态 - 包含所有必要字段 initial_state = { "messages": langchain_messages, - "group_id": group_id, + "end_user_id": end_user_id, "memory_config": memory_config } @@ -356,14 +357,14 @@ class MemoryAgentService: contents = massages.get('write_result') # Convert messages back to string for logging message_text = "\n".join([f"{msg['role']}: {msg['content']}" for msg in messages]) - return self.writer_messages_deal(massagesstatus, start_time, group_id, config_id, message_text, contents) + return self.writer_messages_deal(massagesstatus, start_time, end_user_id, config_id, message_text, contents) except Exception as e: # Ensure proper error handling and logging error_msg = f"Write operation failed: {str(e)}" logger.error(error_msg) if audit_logger: duration = time.time() - start_time - audit_logger.log_operation(operation="WRITE", config_id=config_id, group_id=group_id, success=False, duration=duration, error=error_msg) + audit_logger.log_operation(operation="WRITE", config_id=config_id, end_user_id=end_user_id, success=False, duration=duration, error=error_msg) raise ValueError(error_msg) @@ -371,15 +372,14 @@ class MemoryAgentService: async def read_memory( self, - group_id: str, + end_user_id: str, message: str, history: List[Dict], search_switch: str, - config_id: Optional[str], + config_id: Optional[uuid.UUID]|int, db: Session, storage_type: str, - user_rag_memory_id: str - ) -> Dict: + user_rag_memory_id: str) -> Dict: """ Process read operation with config_id @@ -389,7 +389,7 @@ class MemoryAgentService: - "2": Direct answer based on context Args: - group_id: Group identifier (also used as end_user_id) + end_user_id: Group identifier (also used as end_user_id) message: User message history: Conversation history search_switch: Search mode switch @@ -407,22 +407,22 @@ class MemoryAgentService: import time start_time = time.time() - logger.info(f"[PERF] read_memory started for group_id={group_id}, search_switch={search_switch}") + ori_message= message # Resolve config_id if None using end_user's connected config if config_id is None: try: - connected_config = get_end_user_connected_config(group_id, db) + connected_config = get_end_user_connected_config(end_user_id, db) config_id = connected_config.get("memory_config_id") if config_id is None: - raise ValueError(f"No memory configuration found for end_user {group_id}. Please ensure the user has a connected memory configuration.") + raise ValueError(f"No memory configuration found for end_user {end_user_id}. Please ensure the user has a connected memory configuration.") except Exception as e: if "No memory configuration found" in str(e): raise # Re-raise our specific error - logger.error(f"Failed to get connected config for end_user {group_id}: {e}") - raise ValueError(f"Unable to determine memory configuration for end_user {group_id}: {e}") + logger.error(f"Failed to get connected config for end_user {end_user_id}: {e}") + raise ValueError(f"Unable to determine memory configuration for end_user {end_user_id}: {e}") - logger.info(f"Read operation for group {group_id} with config_id {config_id}") + logger.info(f"Read operation for group {end_user_id} with config_id {config_id}") # 导入审计日志记录器 try: @@ -450,7 +450,7 @@ class MemoryAgentService: audit_logger.log_operation( operation="READ", config_id=config_id, - group_id=group_id, + end_user_id=end_user_id, success=False, duration=duration, error=error_msg @@ -460,16 +460,16 @@ class MemoryAgentService: # Step 2: Prepare history history.append({"role": "user", "content": message}) - logger.debug(f"Group ID:{group_id}, Message:{message}, History:{history}, Config ID:{config_id}") + logger.debug(f"Group ID:{end_user_id}, Message:{message}, History:{history}, Config ID:{config_id}") # Step 3: Initialize MCP client and execute read workflow graph_exec_start = time.time() try: async with make_read_graph() as graph: - config = {"configurable": {"thread_id": group_id}} + config = {"configurable": {"thread_id": end_user_id}} # 初始状态 - 包含所有必要字段 initial_state = {"messages": [HumanMessage(content=message)], "search_switch": search_switch, - "group_id": group_id + "end_user_id": end_user_id , "storage_type": storage_type, "user_rag_memory_id": user_rag_memory_id, "memory_config": memory_config} # 获取节点更新信息 @@ -544,9 +544,8 @@ class MemoryAgentService: if intermediate_type == "search_result": query = intermediate.get('query', '') raw_results = intermediate.get('raw_results', {}) - reranked_results = raw_results.get('reranked_results', []) - try: + reranked_results = raw_results.get('reranked_results', []) statements = [statement['statement'] for statement in reranked_results.get('statements', [])] except Exception: statements = [] @@ -565,13 +564,13 @@ class MemoryAgentService: if '信息不足，无法回答。' != str(summary) and str(search_switch).strip() != "2": # 使用 upsert 方法 repo.upsert( - end_user_id=group_id, - messages=message, + end_user_id=end_user_id, + messages=ori_message, aimessages=summary, retrieved_content=retrieved_content, search_switch=str(search_switch) ) - logger.info(f"成功保存短期记忆: group_id={group_id}, search_switch={search_switch}") + logger.info(f"成功保存短期记忆: end_user_id={end_user_id}, search_switch={search_switch}") else: logger.debug(f"跳过保存短期记忆: summary={summary[:50] if summary else 'None'}, search_switch={search_switch}") @@ -587,7 +586,7 @@ class MemoryAgentService: audit_logger.log_operation( operation="READ", config_id=config_id, - group_id=group_id, + end_user_id=end_user_id, success=True, duration=duration ) @@ -599,20 +598,20 @@ class MemoryAgentService: except Exception as e: # Ensure proper error handling and logging error_msg = f"Read operation failed: {str(e)}" - total_time = time.time() - start_time - logger.error(f"[PERF] read_memory failed after {total_time:.4f}s: {error_msg}") + logger.error(error_msg) if audit_logger: duration = time.time() - start_time audit_logger.log_operation( operation="READ", config_id=config_id, - group_id=group_id, + end_user_id=end_user_id, success=False, duration=duration, error=error_msg ) raise ValueError(error_msg) + def get_messages_list(self, user_input: Write_UserInput) -> list[dict]: """ Get standardized message list from user input. @@ -657,7 +656,7 @@ class MemoryAgentService: logger.info(f"Validation successful: Structured message list, count: {len(user_input.messages)}") return user_input.messages - async def classify_message_type(self, message: str, config_id: int, db: Session) -> Dict: + async def classify_message_type(self, message: str, config_id: UUID, db: Session) -> Dict: """ Determine the type of user message (read or write) Updated to eliminate global variables in favor of explicit parameters. @@ -672,6 +671,8 @@ class MemoryAgentService: """ logger.info("Classifying message type") + + # Load configuration to get LLM model ID config_service = MemoryConfigService(db) memory_config = config_service.load_memory_config( @@ -682,9 +683,9 @@ class MemoryAgentService: status = await status_typle(message, memory_config.llm_model_id) logger.debug(f"Message type: {status}") return status - async def generate_summary_from_retrieve( self, + end_user_id: str, retrieve_info: str, history: List[Dict], query: str, @@ -706,6 +707,18 @@ class MemoryAgentService: Returns: 生成的答案文本 """ + if config_id is None: + try: + config_id = get_end_user_connected_config(end_user_id, db) + config_id = config_id.get('memory_config_id') + if config_id is None: + raise ValueError( + f"No memory configuration found for end_user {end_user_id}. Please ensure the user has a connected memory configuration.") + except Exception as e: + if "No memory configuration found" in str(e): + raise # Re-raise our specific error + logger.error(f"Failed to get connected config for end_user {end_user_id}: {e}") + raise ValueError(f"Unable to determine memory configuration for end_user {end_user_id}: {e}") logger.info(f"Generating summary from retrieve info for query: {query[:50]}...") try: @@ -731,7 +744,7 @@ class MemoryAgentService: state=state, history=history, retrieve_info=retrieve_info, - template_name='Retrieve_Summary_prompt.jinja2', + template_name='direct_summary_prompt.jinja2', operation_name='retrieve_summary', response_model=RetrieveSummaryResponse, search_mode="1" @@ -755,7 +768,7 @@ class MemoryAgentService: """ 统计知识库类型分布，包含： 1. PostgreSQL 中的知识库类型：General, Web, Third-party, Folder（根据 workspace_id 过滤） - 2. Neo4j 中的 memory 类型（仅统计 Chunk 数量，根据 end_user_id/group_id 过滤） + 2. Neo4j 中的 memory 类型（仅统计 Chunk 数量，根据 end_user_id/end_user_id 过滤） 3. total: 所有类型的总和 参数： @@ -841,11 +854,11 @@ class MemoryAgentService: for end_user in end_users: end_user_id_str = str(end_user.id) memory_query = """ - MATCH (n:Chunk) WHERE n.group_id = $group_id RETURN count(n) AS Count + MATCH (n:Chunk) WHERE n.end_user_id = $end_user_id RETURN count(n) AS Count """ neo4j_result = await _neo4j_connector.execute_query( memory_query, - group_id=end_user_id_str, + end_user_id=end_user_id_str, ) chunk_count = neo4j_result[0]["Count"] if neo4j_result else 0 total_chunks += chunk_count @@ -885,7 +898,7 @@ class MemoryAgentService: 获取指定用户的热门记忆标签 参数： - - end_user_id: 用户ID（可选），对应Neo4j中的group_id字段 + - end_user_id: 用户ID（可选），对应Neo4j中的end_user_id字段 - limit: 返回标签数量限制 返回格式： @@ -895,7 +908,7 @@ class MemoryAgentService: ] """ try: - # by_user=False 表示按 group_id 查询（在Neo4j中，group_id就是用户维度） + # by_user=False 表示按 end_user_id 查询（在Neo4j中，end_user_id就是用户维度） tags = await get_hot_memory_tags(end_user_id, limit=limit, by_user=False) payload=[] for tag, freq in tags: @@ -970,21 +983,21 @@ class MemoryAgentService: # 查询该用户的语句 query = ( "MATCH (s:Statement) " - "WHERE ($group_id IS NULL OR s.group_id = $group_id) AND s.statement IS NOT NULL " + "WHERE ($end_user_id IS NULL OR s.end_user_id = $end_user_id) AND s.statement IS NOT NULL " "RETURN s.statement AS statement " "ORDER BY s.created_at DESC LIMIT 100" ) - rows = await connector.execute_query(query, group_id=end_user_id) + rows = await connector.execute_query(query, end_user_id=end_user_id) statements = [r.get("statement", "") for r in rows if r.get("statement")] # 查询该用户的热门实体 entity_query = ( "MATCH (e:ExtractedEntity) " - "WHERE ($group_id IS NULL OR e.group_id = $group_id) AND e.entity_type <> '人物' AND e.name IS NOT NULL " + "WHERE ($end_user_id IS NULL OR e.end_user_id = $end_user_id) AND e.entity_type <> '人物' AND e.name IS NOT NULL " "RETURN e.name AS name, count(e) AS frequency " "ORDER BY frequency DESC LIMIT 20" ) - entity_rows = await connector.execute_query(entity_query, group_id=end_user_id) + entity_rows = await connector.execute_query(entity_query, end_user_id=end_user_id) entities = [f"{r['name']} ({r['frequency']})" for r in entity_rows] await connector.close() @@ -1037,14 +1050,14 @@ class MemoryAgentService: names_to_exclude = ['AI', 'Caroline', 'Melanie', 'Jon', 'Gina', '用户', 'AI助手', 'John', 'Maria'] hot_tag_query = ( "MATCH (e:ExtractedEntity) " - "WHERE ($group_id IS NULL OR e.group_id = $group_id) AND e.entity_type <> '人物' " + "WHERE ($end_user_id IS NULL OR e.end_user_id = $end_user_id) AND e.entity_type <> '人物' " "AND e.name IS NOT NULL AND NOT e.name IN $names_to_exclude " "RETURN e.name AS name, count(e) AS frequency " "ORDER BY frequency DESC LIMIT 4" ) hot_tag_rows = await connector.execute_query( hot_tag_query, - group_id=end_user_id, + end_user_id=end_user_id, names_to_exclude=names_to_exclude ) await connector.close() @@ -1079,9 +1092,8 @@ class MemoryAgentService: logger.info("Starting log content streaming") # Get log file path - use project root directory - current_file = os.path.abspath(__file__) # app/services/memory_agent_service.py - app_dir = os.path.dirname(os.path.dirname(current_file)) # app directory - project_root = os.path.dirname(app_dir) # redbear-mem directory + from pathlib import Path + project_root = str(Path(__file__).resolve().parents[2]) # api directory log_path = os.path.join(project_root, "logs", "agent_service.log") # Check if file exists before starting stream @@ -1179,6 +1191,16 @@ def get_end_user_connected_config(end_user_id: str, db: Session) -> Dict[str, An # 3. 从 config 中提取 memory_config_id config = latest_release.config or {} + + # 如果 config 是字符串，解析为字典 + if isinstance(config, str): + import json + try: + config = json.loads(config) + except json.JSONDecodeError: + logger.warning(f"Failed to parse config JSON for release {latest_release.id}") + config = {} + memory_obj = config.get('memory', {}) memory_config_id = memory_obj.get('memory_content') if isinstance(memory_obj, dict) else None @@ -1217,7 +1239,7 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session) """ from app.models.app_release_model import AppRelease from app.models.end_user_model import EndUser - from app.models.data_config_model import DataConfig + from app.models.memory_config_model import MemoryConfig from sqlalchemy import select logger.info(f"Batch getting connected configs for {len(end_user_ids)} end_users") @@ -1230,10 +1252,10 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session) # 1. 批量查询所有 end_user 及其 app_id end_users = db.query(EndUser).filter(EndUser.id.in_(end_user_ids)).all() - + # 创建 end_user_id -> app_id 的映射 user_to_app = {str(eu.id): eu.app_id for eu in end_users} - + # 记录未找到的用户 found_user_ids = set(user_to_app.keys()) missing_user_ids = set(end_user_ids) - found_user_ids @@ -1243,7 +1265,7 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session) result[user_id] = {"memory_config_id": None, "memory_config_name": None} # 2. 批量获取所有相关应用的最新发布版本 - app_ids = list(user_to_app.values()) + app_ids = list(set(user_to_app.values())) if not app_ids: return result @@ -1263,6 +1285,8 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session) # 3. 收集所有 memory_config_id 并批量查询配置名称 memory_config_ids = [] + old_config_ids = [] # 存储旧的整数ID + for end_user_id, app_id in user_to_app.items(): release = app_to_release.get(app_id) if release: @@ -1270,18 +1294,42 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session) memory_obj = config.get('memory', {}) memory_config_id = memory_obj.get('memory_content') if isinstance(memory_obj, dict) else None if memory_config_id: - memory_config_ids.append(memory_config_id) - + # 判断是否为UUID格式 + if len(str(memory_config_id))>=5: + uuid.UUID(str(memory_config_id)) + memory_config_ids.append(memory_config_id) + else: + old_config_ids.append(str(memory_config_id)) + # 批量查询 memory_config_name config_id_to_name = {} + + # 记录分类结果 + if memory_config_ids or old_config_ids: + logger.info(f"Collected {len(memory_config_ids)} UUID config_ids and {len(old_config_ids)} old integer config_ids") + if old_config_ids: + logger.debug(f"Old config IDs: {old_config_ids}") + + # 查询新的UUID格式的config_id if memory_config_ids: - memory_configs = db.query(DataConfig).filter(DataConfig.config_id.in_(memory_config_ids)).all() - config_id_to_name = {str(mc.config_id): mc.config_name for mc in memory_configs} + memory_configs = db.query(MemoryConfig).filter(MemoryConfig.config_id.in_(memory_config_ids)).all() + config_id_to_name.update({str(mc.config_id): mc.config_name for mc in memory_configs}) + + # 查询旧的整数ID（通过config_id_old字段） + if old_config_ids: + old_memory_configs = db.query(MemoryConfig).filter(MemoryConfig.config_id_old.in_(old_config_ids)).all() + # 使用config_id_old作为key，这样后面查找时能匹配上 + config_id_to_name.update({str(mc.config_id_old): mc.config_name for mc in old_memory_configs}) + # 同时也添加config_id作为key，方便后续使用 + for mc in old_memory_configs: + if mc.config_id_old: + config_id_to_name[str(mc.config_id)] = mc.config_name + logger.info(f"Found {len(old_memory_configs)} configs for old IDs") # 4. 构建最终结果 for end_user_id, app_id in user_to_app.items(): release = app_to_release.get(app_id) - + if not release: logger.warning(f"No active release found for app: {app_id} (end_user: {end_user_id})") result[end_user_id] = {"memory_config_id": None, "memory_config_name": None} @@ -1292,7 +1340,7 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session) memory_obj = config.get('memory', {}) memory_config_id = memory_obj.get('memory_content') if isinstance(memory_obj, dict) else None - # 获取配置名称 + # 获取配置名称（使用字符串形式的ID进行查找，兼容新旧格式） memory_config_name = config_id_to_name.get(str(memory_config_id)) if memory_config_id else None result[end_user_id] = { diff --git a/api/app/services/memory_api_service.py b/api/app/services/memory_api_service.py index 0ae2b965..a8c39a5a 100644 --- a/api/app/services/memory_api_service.py +++ b/api/app/services/memory_api_service.py @@ -25,7 +25,7 @@ class MemoryAPIService: This service provides a thin layer that: 1. Validates end_user exists and belongs to the authorized workspace - 2. Maps end_user_id to group_id for memory operations + 2. Maps end_user_id to end_user_id for memory operations 3. Delegates to MemoryAgentService for actual memory read/write operations """ @@ -68,7 +68,7 @@ class MemoryAPIService: ) end_user = self.db.query(EndUser).filter(EndUser.id == end_user_uuid).first() - + if not end_user: logger.warning(f"End user not found: {end_user_id}") raise ResourceNotFoundException( @@ -77,7 +77,10 @@ class MemoryAPIService: ) # Verify end_user belongs to the workspace via App relationship - app = self.db.query(App).filter(App.id == end_user.app_id).first() + app = self.db.query(App).filter( + App.id == end_user.app_id, + App.is_active.is_(True) + ).first() if not app: logger.warning(f"App not found for end_user: {end_user_id}") @@ -115,7 +118,7 @@ class MemoryAPIService: Args: workspace_id: Workspace ID for resource validation - end_user_id: End user identifier (used as group_id) + end_user_id: End user identifier (used as end_user_id) message: Message content to store config_id: Optional memory configuration ID storage_type: Storage backend (neo4j or rag) @@ -133,14 +136,13 @@ class MemoryAPIService: # Validate end_user exists and belongs to workspace self.validate_end_user(end_user_id, workspace_id) - # Use end_user_id as group_id for memory operations - group_id = end_user_id + # Use end_user_id as end_user_id for memory operations try: # Delegate to MemoryAgentService result = await MemoryAgentService().write_memory( - group_id=group_id, - message=message, + end_user_id=end_user_id, + messages=message, config_id=config_id, db=self.db, storage_type=storage_type, @@ -186,7 +188,7 @@ class MemoryAPIService: Args: workspace_id: Workspace ID for resource validation - end_user_id: End user identifier (used as group_id) + end_user_id: End user identifier (used as end_user_id) message: Query message search_switch: Search mode (0=deep search with verification, 1=deep search, 2=fast search) config_id: Optional memory configuration ID @@ -205,13 +207,13 @@ class MemoryAPIService: # Validate end_user exists and belongs to workspace self.validate_end_user(end_user_id, workspace_id) - # Use end_user_id as group_id for memory operations - group_id = end_user_id + # Use end_user_id as end_user_id for memory operations + try: # Delegate to MemoryAgentService result = await MemoryAgentService().read_memory( - group_id=group_id, + end_user_id=end_user_id, message=message, history=[], search_switch=search_switch, diff --git a/api/app/services/memory_base_service.py b/api/app/services/memory_base_service.py index 25a8281d..bc647752 100644 --- a/api/app/services/memory_base_service.py +++ b/api/app/services/memory_base_service.py @@ -326,7 +326,7 @@ class MemoryBaseService: Args: summary_id: Summary节点的ID - end_user_id: 终端用户ID (group_id) + end_user_id: 终端用户ID (end_user_id) Returns: 最大emotion_intensity对应的emotion_type，如果没有则返回None @@ -334,7 +334,7 @@ class MemoryBaseService: try: query = """ MATCH (s:MemorySummary) - WHERE elementId(s) = $summary_id AND s.group_id = $group_id + WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id MATCH (s)-[:DERIVED_FROM_STATEMENT]->(stmt:Statement) WHERE stmt.emotion_type IS NOT NULL AND stmt.emotion_intensity IS NOT NULL @@ -347,7 +347,7 @@ class MemoryBaseService: result = await self.neo4j_connector.execute_query( query, summary_id=summary_id, - group_id=end_user_id + end_user_id=end_user_id ) if result and len(result) > 0: @@ -381,10 +381,10 @@ class MemoryBaseService: if end_user_id: query = """ MATCH (n:MemorySummary) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id RETURN count(n) as count """ - result = await self.neo4j_connector.execute_query(query, group_id=end_user_id) + result = await self.neo4j_connector.execute_query(query, end_user_id=end_user_id) else: query = """ MATCH (n:MemorySummary) @@ -423,12 +423,12 @@ class MemoryBaseService: if end_user_id: semantic_query = """ MATCH (e:ExtractedEntity) - WHERE e.group_id = $group_id AND e.is_explicit_memory = true + WHERE e.end_user_id = $end_user_id AND e.is_explicit_memory = true RETURN count(e) as count """ semantic_result = await self.neo4j_connector.execute_query( semantic_query, - group_id=end_user_id + end_user_id=end_user_id ) else: semantic_query = """ @@ -519,7 +519,7 @@ class MemoryBaseService: """ if end_user_id: - query += " AND n.group_id = $group_id" + query += " AND n.end_user_id = $end_user_id" query += """ RETURN sum(CASE WHEN n.activation_value IS NOT NULL AND n.activation_value < $threshold THEN 1 ELSE 0 END) as low_activation_nodes @@ -528,7 +528,7 @@ class MemoryBaseService: # 设置查询参数 params = {'threshold': forgetting_threshold} if end_user_id: - params['group_id'] = end_user_id + params['end_user_id'] = end_user_id # 执行查询 result = await self.neo4j_connector.execute_query(query, **params) diff --git a/api/app/services/memory_config_service.py b/api/app/services/memory_config_service.py index 0099eb18..e09cf67f 100644 --- a/api/app/services/memory_config_service.py +++ b/api/app/services/memory_config_service.py @@ -7,14 +7,15 @@ This service eliminates code duplication between MemoryAgentService and MemorySt import time from datetime import datetime - +from app.models.memory_config_model import MemoryConfig as MemoryConfigModel +from sqlalchemy import select from app.core.logging_config import get_config_logger, get_logger from app.core.validators.memory_config_validators import ( validate_and_resolve_model_id, validate_embedding_model, validate_model_exists_and_active, ) -from app.repositories.data_config_repository import DataConfigRepository +from app.repositories.memory_config_repository import MemoryConfigRepository from app.schemas.memory_config_schema import ( ConfigurationError, InvalidConfigError, @@ -23,20 +24,24 @@ from app.schemas.memory_config_schema import ( ModelNotFoundError, ) from sqlalchemy.orm import Session +from uuid import UUID logger = get_logger(__name__) config_logger = get_config_logger() +import uuid - -def _validate_config_id(config_id): - """Validate configuration ID format.""" +def _validate_config_id(config_id, db: Session = None): + """Validate configuration ID format (supports both UUID and integer).""" + if isinstance(config_id, uuid.UUID): + return config_id + if config_id is None: raise InvalidConfigError( "Configuration ID cannot be None", field_name="config_id", invalid_value=config_id, ) - + if isinstance(config_id, int): if config_id <= 0: raise InvalidConfigError( @@ -44,27 +49,56 @@ def _validate_config_id(config_id): field_name="config_id", invalid_value=config_id, ) + # 如果提供了数据库会话，尝试通过 user_id 查询 config_id + if db is not None: + # 查询 user_id 匹配的记录 + stmt = select(MemoryConfigModel).where(MemoryConfigModel.config_id_old == str(config_id)) + result = db.execute(stmt).scalars().first() + if result: + logger.info(f"Found config_id {result.config_id} for user_id {config_id}") + return result.config_id + return config_id - + if isinstance(config_id, str): + config_id_stripped = config_id.strip() + + # Try parsing as UUID first try: - parsed_id = int(config_id.strip()) + return uuid.UUID(config_id_stripped) + except ValueError: + pass + + # Fall back to integer parsing + try: + parsed_id = int(config_id_stripped) if parsed_id <= 0: raise InvalidConfigError( f"Configuration ID must be positive: {parsed_id}", field_name="config_id", invalid_value=config_id, ) + + # 如果提供了数据库会话，尝试通过 user_id 查询 config_id + if db is not None: + # 查询 user_id 匹配的记录 + stmt = select(MemoryConfigModel).where(MemoryConfigModel.user_id == str(parsed_id)) + result = db.execute(stmt).scalars().first() + + if result: + logger.info(f"Found config_id {result.config_id} for user_id {parsed_id}") + return result.config_id + return parsed_id except ValueError: raise InvalidConfigError( - f"Invalid configuration ID format: '{config_id}'", + f"Invalid configuration ID format: '{config_id}' (must be UUID or positive integer)", field_name="config_id", invalid_value=config_id, ) - + raise InvalidConfigError( - f"Invalid type for configuration ID: expected int or str, got {type(config_id).__name__}", + f"Invalid type for configuration ID: expected UUID, int or str, got {type(config_id).__name__}", field_name="config_id", invalid_value=config_id, ) @@ -73,61 +107,61 @@ def _validate_config_id(config_id): class MemoryConfigService: """ Centralized service for memory configuration loading and validation. - + This class provides a single implementation of configuration loading logic that can be shared across multiple services, eliminating code duplication. - + Usage: config_service = MemoryConfigService(db) memory_config = config_service.load_memory_config(config_id) model_config = config_service.get_model_config(model_id) """ - + def __init__(self, db: Session): """Initialize the service with a database session. - + Args: db: SQLAlchemy database session """ self.db = db - + def load_memory_config( self, - config_id: int, + config_id: UUID, service_name: str = "MemoryConfigService", ) -> MemoryConfig: """ Load memory configuration from database by config_id. - + Args: - config_id: Configuration ID from database + config_id: Configuration ID (UUID) from database service_name: Name of the calling service (for logging purposes) - + Returns: MemoryConfig: Immutable configuration object - + Raises: ConfigurationError: If validation fails """ start_time = time.time() - + config_logger.info( "Starting memory configuration loading", extra={ "operation": "load_memory_config", "service": service_name, - "config_id": config_id, + "config_id": str(config_id), }, ) - + logger.info(f"Loading memory configuration from database: config_id={config_id}") - + try: - validated_config_id = _validate_config_id(config_id) - + validated_config_id = _validate_config_id(config_id, self.db) + # Step 1: Get config and workspace db_query_start = time.time() - result = DataConfigRepository.get_config_with_workspace(self.db, validated_config_id) + result = MemoryConfigRepository.get_config_with_workspace(self.db, validated_config_id) db_query_time = time.time() - db_query_start logger.info(f"[PERF] Config+Workspace query: {db_query_time:.4f}s") if not result: @@ -136,18 +170,18 @@ class MemoryConfigService: "Configuration not found in database", extra={ "operation": "load_memory_config", - "config_id": validated_config_id, + "config_id": str(config_id), "load_result": "not_found", "elapsed_ms": elapsed_ms, "service": service_name, }, ) raise ConfigurationError( - f"Configuration {validated_config_id} not found in database" + f"Configuration {config_id} not found in database" ) - + memory_config, workspace = result - + # Step 2: Validate embedding model (returns both UUID and name) embed_start = time.time() embedding_uuid, embedding_name = validate_embedding_model( @@ -159,7 +193,7 @@ class MemoryConfigService: ) embed_time = time.time() - embed_start logger.info(f"[PERF] Embedding validation: {embed_time:.4f}s") - + # Step 3: Resolve LLM model llm_start = time.time() llm_uuid, llm_name = validate_and_resolve_model_id( @@ -173,7 +207,7 @@ class MemoryConfigService: ) llm_time = time.time() - llm_start logger.info(f"[PERF] LLM validation: {llm_time:.4f}s") - + # Step 4: Resolve optional rerank model rerank_start = time.time() rerank_uuid = None @@ -191,10 +225,10 @@ class MemoryConfigService: rerank_time = time.time() - rerank_start if memory_config.rerank_id: logger.info(f"[PERF] Rerank validation: {rerank_time:.4f}s") - + # Note: embedding_name is now returned from validate_embedding_model above # No need for redundant query! - + # Create immutable MemoryConfig object config = MemoryConfig( config_id=memory_config.config_id, @@ -235,9 +269,9 @@ class MemoryConfigService: pruning_scene=memory_config.pruning_scene or "education", pruning_threshold=float(memory_config.pruning_threshold) if memory_config.pruning_threshold is not None else 0.5, ) - + elapsed_ms = (time.time() - start_time) * 1000 - + config_logger.info( "Memory configuration loaded successfully", extra={ @@ -250,13 +284,13 @@ class MemoryConfigService: "elapsed_ms": elapsed_ms, }, ) - + logger.info(f"Memory configuration loaded successfully: {config.config_name}") return config - + except Exception as e: elapsed_ms = (time.time() - start_time) * 1000 - + config_logger.error( "Failed to load memory configuration", extra={ @@ -270,7 +304,7 @@ class MemoryConfigService: }, exc_info=True, ) - + logger.error(f"Failed to load memory configuration {config_id}: {e}") if isinstance(e, (ConfigurationError, ValueError)): raise @@ -304,7 +338,7 @@ class MemoryConfigService: "provider": api_config.provider, "api_key": api_config.api_key, "base_url": api_config.api_base, - "model_config_id": api_config.model_config_id, + "model_config_id": str(config.id), "type": config.type, "timeout": settings.LLM_TIMEOUT, "max_retries": settings.LLM_MAX_RETRIES, @@ -336,7 +370,7 @@ class MemoryConfigService: "provider": api_config.provider, "api_key": api_config.api_key, "base_url": api_config.api_base, - "model_config_id": api_config.model_config_id, + "model_config_id": str(config.id), "type": config.type, "timeout": 120.0, "max_retries": 5, diff --git a/api/app/services/memory_dashboard_service.py b/api/app/services/memory_dashboard_service.py index a774647e..06a94060 100644 --- a/api/app/services/memory_dashboard_service.py +++ b/api/app/services/memory_dashboard_service.py @@ -53,18 +53,28 @@ def get_workspace_end_users( workspace_id: uuid.UUID, current_user: User ) -> List[EndUser]: - """获取工作空间的所有宿主""" + """获取工作空间的所有宿主（优化版本：减少数据库查询次数）""" business_logger.info(f"获取工作空间宿主列表: workspace_id={workspace_id}, 操作者: {current_user.username}") try: - # 查询应用（ORM）并转换为 Pydantic 模型 + # 查询应用（ORM） apps_orm = app_repository.get_apps_by_workspace_id(db, workspace_id) - apps = [AppSchema.model_validate(h) for h in apps_orm] - app_ids = [app.id for app in apps] - end_users = [] - for app_id in app_ids: - end_user_orm_list = end_user_repository.get_end_users_by_app_id(db, app_id) - end_users.extend([EndUserSchema.model_validate(h) for h in end_user_orm_list]) + + if not apps_orm: + business_logger.info("工作空间下没有应用") + return [] + + # 提取所有 app_id + app_ids = [app.id for app in apps_orm] + + # 批量查询所有 end_users（一次查询而非循环查询） + from app.models.end_user_model import EndUser as EndUserModel + end_users_orm = db.query(EndUserModel).filter( + EndUserModel.app_id.in_(app_ids) + ).all() + + # 转换为 Pydantic 模型（只在需要时转换） + end_users = [EndUserSchema.model_validate(eu) for eu in end_users_orm] business_logger.info(f"成功获取 {len(end_users)} 个宿主记录") return end_users @@ -414,6 +424,67 @@ def get_current_user_total_chunk( business_logger.error(f"获取用户总chunk数失败: end_user_id={end_user_id} - {str(e)}") raise + +def get_users_total_chunk_batch( + end_user_ids: List[str], + db: Session, + current_user: User +) -> dict: + """ + 批量获取多个用户的总chunk数（性能优化版本） + + Args: + end_user_ids: 用户ID列表 + db: 数据库会话 + current_user: 当前用户 + + Returns: + 字典，key为end_user_id，value为chunk总数 + 格式: {"user_id_1": 100, "user_id_2": 50, ...} + """ + business_logger.info(f"批量获取 {len(end_user_ids)} 个用户的总chunk数, 操作者: {current_user.username}") + + try: + from app.models.document_model import Document + from sqlalchemy import func, case + + if not end_user_ids: + return {} + + # 构造所有文件名 + file_names = [f"{user_id}.txt" for user_id in end_user_ids] + + # 一次查询获取所有用户的chunk总数 + # 使用 GROUP BY file_name 来分组统计 + results = db.query( + Document.file_name, + func.sum(Document.chunk_num).label('total_chunk') + ).filter( + Document.file_name.in_(file_names) + ).group_by( + Document.file_name + ).all() + + # 构建结果字典 + chunk_map = {} + for file_name, total_chunk in results: + # 从文件名中提取 end_user_id (去掉 .txt 后缀) + user_id = file_name.replace('.txt', '') + chunk_map[user_id] = int(total_chunk or 0) + + # 对于没有记录的用户，设置为0 + for user_id in end_user_ids: + if user_id not in chunk_map: + chunk_map[user_id] = 0 + + business_logger.info(f"成功批量获取 {len(chunk_map)} 个用户的总chunk数") + return chunk_map + + except Exception as e: + business_logger.error(f"批量获取用户总chunk数失败: {str(e)}") + raise + + def get_rag_content( end_user_id: str, limit: int, diff --git a/api/app/services/memory_entity_relationship_service.py b/api/app/services/memory_entity_relationship_service.py index 9b5f3c99..7081d28b 100644 --- a/api/app/services/memory_entity_relationship_service.py +++ b/api/app/services/memory_entity_relationship_service.py @@ -717,8 +717,8 @@ class MemoryInteraction: ori_data= await self.connector.execute_query(Memory_Space_Entity, id=self.id) if ori_data!=[]: # name = ori_data[0]['name'] - group_id = [i['group_id'] for i in ori_data][0] - Space_User = await self.connector.execute_query(Memory_Space_User, group_id=group_id) + end_user_id = [i['end_user_id'] for i in ori_data][0] + Space_User = await self.connector.execute_query(Memory_Space_User, end_user_id=end_user_id) if not Space_User: return [] user_id=Space_User[0]['id'] diff --git a/api/app/services/memory_episodic_service.py b/api/app/services/memory_episodic_service.py index 12eeff6e..08751fd1 100644 --- a/api/app/services/memory_episodic_service.py +++ b/api/app/services/memory_episodic_service.py @@ -34,7 +34,7 @@ class MemoryEpisodicService(MemoryBaseService): Args: summary_id: Summary节点的ID - end_user_id: 终端用户ID (group_id) + end_user_id: 终端用户ID (end_user_id) Returns: (标题, 类型)元组，如果不存在则返回默认值 @@ -43,14 +43,14 @@ class MemoryEpisodicService(MemoryBaseService): # 查询Summary节点的name(作为title)和memory_type(作为type) query = """ MATCH (s:MemorySummary) - WHERE elementId(s) = $summary_id AND s.group_id = $group_id + WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id RETURN s.name AS title, s.memory_type AS type """ result = await self.neo4j_connector.execute_query( query, summary_id=summary_id, - group_id=end_user_id + end_user_id=end_user_id ) if not result or len(result) == 0: @@ -77,7 +77,7 @@ class MemoryEpisodicService(MemoryBaseService): Args: summary_id: Summary节点的ID - end_user_id: 终端用户ID (group_id) + end_user_id: 终端用户ID (end_user_id) Returns: 前3个实体的name属性列表 @@ -87,7 +87,7 @@ class MemoryEpisodicService(MemoryBaseService): # 按activation_value降序排序,返回前3个 query = """ MATCH (s:MemorySummary) - WHERE elementId(s) = $summary_id AND s.group_id = $group_id + WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id MATCH (s)-[:DERIVED_FROM_STATEMENT]->(stmt:Statement) MATCH (stmt)-[:REFERENCES_ENTITY]->(entity:ExtractedEntity) WHERE entity.activation_value IS NOT NULL @@ -99,7 +99,7 @@ class MemoryEpisodicService(MemoryBaseService): result = await self.neo4j_connector.execute_query( query, summary_id=summary_id, - group_id=end_user_id + end_user_id=end_user_id ) # 提取实体名称 @@ -123,7 +123,7 @@ class MemoryEpisodicService(MemoryBaseService): Args: summary_id: Summary节点的ID - end_user_id: 终端用户ID (group_id) + end_user_id: 终端用户ID (end_user_id) Returns: 所有Statement节点的statement属性内容列表 @@ -132,7 +132,7 @@ class MemoryEpisodicService(MemoryBaseService): # 查询Summary节点指向的所有Statement节点 query = """ MATCH (s:MemorySummary) - WHERE elementId(s) = $summary_id AND s.group_id = $group_id + WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id MATCH (s)-[:DERIVED_FROM_STATEMENT]->(stmt:Statement) WHERE stmt.statement IS NOT NULL AND stmt.statement <> '' RETURN stmt.statement AS statement @@ -141,7 +141,7 @@ class MemoryEpisodicService(MemoryBaseService): result = await self.neo4j_connector.execute_query( query, summary_id=summary_id, - group_id=end_user_id + end_user_id=end_user_id ) # 提取statement内容 @@ -214,12 +214,12 @@ class MemoryEpisodicService(MemoryBaseService): # 1. 先查询所有情景记忆的总数（不受筛选条件限制） total_all_query = """ MATCH (s:MemorySummary) - WHERE s.group_id = $group_id + WHERE s.end_user_id = $end_user_id RETURN count(s) AS total_all """ total_all_result = await self.neo4j_connector.execute_query( total_all_query, - group_id=end_user_id + end_user_id=end_user_id ) total_all = total_all_result[0]["total_all"] if total_all_result else 0 @@ -229,7 +229,7 @@ class MemoryEpisodicService(MemoryBaseService): # 3. 构建Cypher查询 query = """ MATCH (s:MemorySummary) - WHERE s.group_id = $group_id + WHERE s.end_user_id = $end_user_id """ # 添加时间范围过滤 @@ -248,7 +248,7 @@ class MemoryEpisodicService(MemoryBaseService): ORDER BY s.created_at DESC """ - params = {"group_id": end_user_id} + params = {"end_user_id": end_user_id} if time_filter: params["time_filter"] = time_filter if title_keyword: @@ -333,14 +333,14 @@ class MemoryEpisodicService(MemoryBaseService): # 1. 查询指定的MemorySummary节点 query = """ MATCH (s:MemorySummary) - WHERE elementId(s) = $summary_id AND s.group_id = $group_id + WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id RETURN elementId(s) AS id, s.created_at AS created_at """ result = await self.neo4j_connector.execute_query( query, summary_id=summary_id, - group_id=end_user_id + end_user_id=end_user_id ) # 2. 如果节点不存在，返回错误 diff --git a/api/app/services/memory_explicit_service.py b/api/app/services/memory_explicit_service.py index 713215c3..f8d39ae8 100644 --- a/api/app/services/memory_explicit_service.py +++ b/api/app/services/memory_explicit_service.py @@ -60,7 +60,7 @@ class MemoryExplicitService(MemoryBaseService): # ========== 1. 查询情景记忆（MemorySummary节点） ========== episodic_query = """ MATCH (s:MemorySummary) - WHERE s.group_id = $group_id + WHERE s.end_user_id = $end_user_id RETURN elementId(s) AS id, s.name AS title, s.content AS content, @@ -70,7 +70,7 @@ class MemoryExplicitService(MemoryBaseService): episodic_result = await self.neo4j_connector.execute_query( episodic_query, - group_id=end_user_id + end_user_id=end_user_id ) # 处理情景记忆数据 @@ -96,7 +96,7 @@ class MemoryExplicitService(MemoryBaseService): # ========== 2. 查询语义记忆（ExtractedEntity节点） ========== semantic_query = """ MATCH (e:ExtractedEntity) - WHERE e.group_id = $group_id + WHERE e.end_user_id = $end_user_id AND e.is_explicit_memory = true RETURN elementId(e) AS id, e.name AS name, @@ -107,7 +107,7 @@ class MemoryExplicitService(MemoryBaseService): semantic_result = await self.neo4j_connector.execute_query( semantic_query, - group_id=end_user_id + end_user_id=end_user_id ) # 处理语义记忆数据 @@ -189,7 +189,7 @@ class MemoryExplicitService(MemoryBaseService): # ========== 1. 先尝试查询情景记忆 ========== episodic_query = """ MATCH (s:MemorySummary) - WHERE elementId(s) = $memory_id AND s.group_id = $group_id + WHERE elementId(s) = $memory_id AND s.end_user_id = $end_user_id RETURN s.name AS title, s.content AS content, s.created_at AS created_at @@ -198,7 +198,7 @@ class MemoryExplicitService(MemoryBaseService): episodic_result = await self.neo4j_connector.execute_query( episodic_query, memory_id=memory_id, - group_id=end_user_id + end_user_id=end_user_id ) if episodic_result and len(episodic_result) > 0: @@ -229,7 +229,7 @@ class MemoryExplicitService(MemoryBaseService): semantic_query = """ MATCH (e:ExtractedEntity) WHERE elementId(e) = $memory_id - AND e.group_id = $group_id + AND e.end_user_id = $end_user_id AND e.is_explicit_memory = true RETURN e.name AS name, e.description AS core_definition, @@ -240,7 +240,7 @@ class MemoryExplicitService(MemoryBaseService): semantic_result = await self.neo4j_connector.execute_query( semantic_query, memory_id=memory_id, - group_id=end_user_id + end_user_id=end_user_id ) if semantic_result and len(semantic_result) > 0: diff --git a/api/app/services/memory_forget_service.py b/api/app/services/memory_forget_service.py index 2db4cdc7..e1030b24 100644 --- a/api/app/services/memory_forget_service.py +++ b/api/app/services/memory_forget_service.py @@ -12,6 +12,7 @@ from typing import Optional, Dict, Any, Tuple from datetime import datetime, timezone +from uuid import UUID from sqlalchemy.orm import Session @@ -23,7 +24,7 @@ from app.core.memory.storage_services.forgetting_engine.config_utils import ( load_actr_config_from_db, ) from app.repositories.neo4j.neo4j_connector import Neo4jConnector -from app.repositories.data_config_repository import DataConfigRepository +from app.repositories.memory_config_repository import MemoryConfigRepository from app.repositories.forgetting_cycle_history_repository import ForgettingCycleHistoryRepository @@ -70,7 +71,7 @@ class MemoryForgetService: def __init__(self): """初始化服务""" - self.config_repository = DataConfigRepository() + self.config_repository = MemoryConfigRepository() self.history_repository = ForgettingCycleHistoryRepository() def _get_neo4j_connector(self) -> Neo4jConnector: @@ -87,7 +88,7 @@ class MemoryForgetService: async def _get_forgetting_components( self, db: Session, - config_id: Optional[int] = None + config_id: Optional[UUID] = None ) -> Tuple[ACTRCalculator, ForgettingStrategy, ForgettingScheduler, Dict[str, Any]]: """ 获取遗忘引擎组件（计算器、策略、调度器） @@ -132,7 +133,7 @@ class MemoryForgetService: async def _get_knowledge_stats( self, connector: Neo4jConnector, - group_id: Optional[str] = None, + end_user_id: Optional[str] = None, forgetting_threshold: float = 0.3 ) -> Dict[str, Any]: """ @@ -140,7 +141,7 @@ class MemoryForgetService: Args: connector: Neo4j 连接器 - group_id: 组ID（可选） + end_user_id: 组ID（可选） forgetting_threshold: 遗忘阈值 Returns: @@ -152,8 +153,8 @@ class MemoryForgetService: WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary) """ - if group_id: - query += " AND n.group_id = $group_id" + if end_user_id: + query += " AND n.end_user_id = $end_user_id" query += """ WITH n, @@ -172,8 +173,8 @@ class MemoryForgetService: """ params = {'threshold': forgetting_threshold} - if group_id: - params['group_id'] = group_id + if end_user_id: + params['end_user_id'] = end_user_id results = await connector.execute_query(query, **params) @@ -200,7 +201,7 @@ class MemoryForgetService: async def _get_pending_forgetting_nodes( self, connector: Neo4jConnector, - group_id: str, + end_user_id: str, forgetting_threshold: float, min_days_since_access: int, limit: int = 20 @@ -212,7 +213,7 @@ class MemoryForgetService: Args: connector: Neo4j 连接器 - group_id: 组ID + end_user_id: 组ID forgetting_threshold: 遗忘阈值 min_days_since_access: 最小未访问天数 limit: 返回节点数量限制 @@ -229,7 +230,7 @@ class MemoryForgetService: query = """ MATCH (n) WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary) - AND n.group_id = $group_id + AND n.end_user_id = $end_user_id AND n.activation_value IS NOT NULL AND n.activation_value < $threshold AND n.last_access_time IS NOT NULL @@ -250,7 +251,7 @@ class MemoryForgetService: """ params = { - 'group_id': group_id, + 'end_user_id': end_user_id, 'threshold': forgetting_threshold, 'min_access_time_str': min_access_time_str, 'limit': limit @@ -291,10 +292,10 @@ class MemoryForgetService: async def trigger_forgetting_cycle( self, db: Session, - group_id: str, + end_user_id: str, max_merge_batch_size: Optional[int] = None, min_days_since_access: Optional[int] = None, - config_id: Optional[int] = None + config_id: Optional[UUID] = None ) -> Dict[str, Any]: """ 手动触发遗忘周期 @@ -303,10 +304,10 @@ class MemoryForgetService: Args: db: 数据库会话 - group_id: 组ID（即终端用户ID，必填） + end_user_id: 组ID（即终端用户ID，必填） max_merge_batch_size: 最大融合批次大小（可选） min_days_since_access: 最小未访问天数（可选） - config_id: 配置ID（必填，由控制器层通过 group_id 获取） + config_id: 配置ID（必填，由控制器层通过 end_user_id 获取） Returns: dict: 遗忘报告 @@ -319,7 +320,7 @@ class MemoryForgetService: # 运行遗忘周期（LLM 客户端将在需要时由 forgetting_strategy 内部获取） report = await forgetting_scheduler.run_forgetting_cycle( - group_id=group_id, + end_user_id=end_user_id, max_merge_batch_size=max_merge_batch_size, min_days_since_access=min_days_since_access, config_id=config_id, @@ -338,7 +339,7 @@ class MemoryForgetService: stats_query = """ MATCH (n) WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary OR n:Chunk) - AND n.group_id = $group_id + AND n.end_user_id = $end_user_id RETURN count(n) as total_nodes, avg(n.activation_value) as average_activation, @@ -347,7 +348,7 @@ class MemoryForgetService: stats_results = await connector.execute_query( stats_query, - group_id=group_id, + end_user_id=end_user_id, threshold=config['forgetting_threshold'] ) @@ -364,7 +365,7 @@ class MemoryForgetService: # 保存历史记录到数据库 self.history_repository.create( db=db, - end_user_id=group_id, + end_user_id=end_user_id, execution_time=execution_time, merged_count=report['merged_count'], failed_count=report['failed_count'], @@ -376,7 +377,7 @@ class MemoryForgetService: ) api_logger.info( - f"已保存遗忘周期历史记录: end_user_id={group_id}, " + f"已保存遗忘周期历史记录: end_user_id={end_user_id}, " f"merged_count={report['merged_count']}" ) @@ -389,7 +390,7 @@ class MemoryForgetService: def read_forgetting_config( self, db: Session, - config_id: int + config_id: UUID ) -> Dict[str, Any]: """ 获取遗忘引擎配置 @@ -416,7 +417,7 @@ class MemoryForgetService: def update_forgetting_config( self, db: Session, - config_id: int, + config_id: UUID, update_fields: Dict[str, Any] ) -> Dict[str, Any]: """ @@ -465,8 +466,8 @@ class MemoryForgetService: async def get_forgetting_stats( self, db: Session, - group_id: Optional[str] = None, - config_id: Optional[int] = None + end_user_id: Optional[str] = None, + config_id: Optional[UUID] = None ) -> Dict[str, Any]: """ 获取遗忘引擎统计信息 @@ -475,7 +476,7 @@ class MemoryForgetService: Args: db: 数据库会话 - group_id: 组ID（可选） + end_user_id: 组ID（可选） config_id: 配置ID（可选，用于获取遗忘阈值） Returns: @@ -493,8 +494,8 @@ class MemoryForgetService: WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary OR n:Chunk) """ - if group_id: - activation_query += " AND n.group_id = $group_id" + if end_user_id: + activation_query += " AND n.end_user_id = $end_user_id" activation_query += """ RETURN @@ -506,8 +507,8 @@ class MemoryForgetService: """ params = {'threshold': forgetting_threshold} - if group_id: - params['group_id'] = group_id + if end_user_id: + params['end_user_id'] = end_user_id activation_results = await connector.execute_query(activation_query, **params) @@ -539,8 +540,8 @@ class MemoryForgetService: WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary OR n:Chunk) """ - if group_id: - distribution_query += " AND n.group_id = $group_id" + if end_user_id: + distribution_query += " AND n.end_user_id = $end_user_id" distribution_query += """ WITH n, @@ -558,8 +559,8 @@ class MemoryForgetService: """ dist_params = {} - if group_id: - dist_params['group_id'] = group_id + if end_user_id: + dist_params['end_user_id'] = end_user_id distribution_results = await connector.execute_query(distribution_query, **dist_params) @@ -582,11 +583,11 @@ class MemoryForgetService: # 获取最近7个日期的历史趋势数据（每天取最后一次执行） recent_trends = [] try: - if group_id: + if end_user_id: # 查询所有历史记录 history_records = self.history_repository.get_recent_by_end_user( db=db, - end_user_id=group_id + end_user_id=end_user_id ) # 按日期分组（一天可能有多次执行，取最后一次） @@ -632,7 +633,7 @@ class MemoryForgetService: # 获取待遗忘节点列表（前20个满足遗忘条件的节点） pending_nodes = [] try: - if group_id: + if end_user_id: # 验证 min_days_since_access 配置值 min_days = config.get('min_days_since_access') if min_days is None or not isinstance(min_days, (int, float)) or min_days < 0: @@ -643,7 +644,7 @@ class MemoryForgetService: pending_nodes = await self._get_pending_forgetting_nodes( connector=connector, - group_id=group_id, + end_user_id=end_user_id, forgetting_threshold=forgetting_threshold, min_days_since_access=int(min_days), limit=20 @@ -677,7 +678,7 @@ class MemoryForgetService: db: Session, importance_score: float, days: int, - config_id: Optional[int] = None + config_id: Optional[UUID] = None ) -> Dict[str, Any]: """ 获取遗忘曲线数据 diff --git a/api/app/services/memory_konwledges_server.py b/api/app/services/memory_konwledges_server.py index c6297e12..420f7ca1 100644 --- a/api/app/services/memory_konwledges_server.py +++ b/api/app/services/memory_konwledges_server.py @@ -450,12 +450,12 @@ async def create_document_chunk( return success(data=chunk, msg="文档块创建成功") -async def write_rag(group_id, message, user_rag_memory_id): +async def write_rag(end_user_id, message, user_rag_memory_id): """ 将消息写入 RAG 知识库 Args: - group_id: 组ID，用作文件标题 + end_user_id: 组ID，用作文件标题 message: 消息内容 user_rag_memory_id: 知识库ID（必须是有效的UUID） @@ -487,10 +487,10 @@ async def write_rag(group_id, message, user_rag_memory_id): db = next(db_gen) try: - create_data = CustomTextFileCreate(title=group_id, content=message) + create_data = CustomTextFileCreate(title=end_user_id, content=message) current_user = SimpleUser(user_rag_memory_id) # 检查文档是否已存在 - document = find_document_id_by_kb_and_filename(db=db, kb_id=user_rag_memory_id, file_name=f"{group_id}.txt") + document = find_document_id_by_kb_and_filename(db=db, kb_id=user_rag_memory_id, file_name=f"{end_user_id}.txt") print('======',document) api_logger.info(f"查找文档结果: document_id={document}") if document is not None: @@ -508,7 +508,7 @@ async def write_rag(group_id, message, user_rag_memory_id): return result else: # 文档不存在，创建新文档 - api_logger.info(f"文档不存在，创建新文档: group_id={group_id}") + api_logger.info(f"文档不存在，创建新文档: end_user_id={end_user_id}") result = await memory_konwledges_up( kb_id=user_rag_memory_id, parent_id=user_rag_memory_id, @@ -520,13 +520,13 @@ async def write_rag(group_id, message, user_rag_memory_id): new_document_id = find_document_id_by_kb_and_filename( db=db, kb_id=user_rag_memory_id, - file_name=f"{group_id}.txt" + file_name=f"{end_user_id}.txt" ) if new_document_id: await parse_document_by_id(new_document_id, db=db, current_user=current_user) else: - api_logger.error(f"创建文档后无法找到文档ID: group_id={group_id}") + api_logger.error(f"创建文档后无法找到文档ID: end_user_id={end_user_id}") return result finally: # 确保数据库会话被关闭 diff --git a/api/app/services/memory_perceptual_service.py b/api/app/services/memory_perceptual_service.py index d257e80f..b9d96a0b 100644 --- a/api/app/services/memory_perceptual_service.py +++ b/api/app/services/memory_perceptual_service.py @@ -6,7 +6,7 @@ from sqlalchemy.orm import Session from app.core.error_codes import BizCode from app.core.exceptions import BusinessException from app.core.logging_config import get_business_logger -from app.models.memory_perceptual_model import PerceptualType, FileStorageType +from app.models.memory_perceptual_model import PerceptualType, FileStorageService from app.repositories.memory_perceptual_repository import MemoryPerceptualRepository from app.schemas.memory_perceptual_schema import ( PerceptualQuerySchema, @@ -137,8 +137,19 @@ class MemoryPerceptualService: memory_items = [] for memory in memories: meta_data = memory.meta_data or {} - content = meta_data.get("content") - content = Content(**content) + content = meta_data.get("content", {}) + + # 安全地提取 content 字段，提供默认值 + if content: + content_obj = Content(**content) + topic = content_obj.topic + domain = content_obj.domain + keywords = content_obj.keywords + else: + topic = "Unknown" + domain = "Unknown" + keywords = [] + memory_item = PerceptualMemoryItem( id=memory.id, perceptual_type=PerceptualType(memory.perceptual_type), @@ -146,11 +157,12 @@ class MemoryPerceptualService: file_name=memory.file_name, file_ext=memory.file_ext, summary=memory.summary, - topic=content.topic, - domain=content.domain, - keywords=content.keywords, + meta_data=meta_data, + topic=topic, + domain=domain, + keywords=keywords, created_time=int(memory.created_time.timestamp()*1000), - storage_type=FileStorageType(memory.storage_service), + storage_service=FileStorageService(memory.storage_service), ) memory_items.append(memory_item) diff --git a/api/app/services/memory_reflection_service.py b/api/app/services/memory_reflection_service.py index 46e42b46..b92a5d06 100644 --- a/api/app/services/memory_reflection_service.py +++ b/api/app/services/memory_reflection_service.py @@ -13,11 +13,12 @@ from app.db import get_db from app.core.logging_config import get_api_logger from app.core.memory.storage_services.reflection_engine import ReflectionConfig, ReflectionEngine from app.core.memory.storage_services.reflection_engine.self_reflexion import ReflectionRange, ReflectionBaseline -from app.repositories.data_config_repository import DataConfigRepository +from app.repositories.memory_config_repository import MemoryConfigRepository from app.repositories.neo4j.neo4j_connector import Neo4jConnector from app.models.app_model import App from app.models.app_release_model import AppRelease from app.models.end_user_model import EndUser +from app.utils.config_utils import resolve_config_id api_logger = get_api_logger() @@ -38,7 +39,10 @@ class WorkspaceAppService: Returns: Dictionary containing detailed application information """ - apps = self.db.query(App).filter(App.workspace_id == workspace_id).all() + apps = self.db.query(App).filter( + App.workspace_id == workspace_id, + App.is_active.is_(True) + ).all() app_ids = [str(app.id) for app in apps] apps_detailed_info = [] @@ -70,7 +74,7 @@ class WorkspaceAppService: "created_at": app.created_at.isoformat() if app.created_at else None, "updated_at": app.updated_at.isoformat() if app.updated_at else None, "releases": [], - "data_configs": [], + "memory_configs": [], "end_users": [] } @@ -85,76 +89,76 @@ class WorkspaceAppService: for release in app_releases: memory_content = self._extract_memory_content(release.config) - - + memory_content=resolve_config_id(memory_content, self.db) if memory_content and memory_content in processed_configs: continue - + release_info = { "app_id": str(release.app_id), "config": memory_content } - + if memory_content: processed_configs.add(memory_content) - data_config_info = self._get_data_config(memory_content) - - if data_config_info: - if not any(dc["config_id"] == data_config_info["config_id"] for dc in app_info["data_configs"]): - app_info["data_configs"].append(data_config_info) - + memory_config_info = self._get_memory_config(memory_content) + if memory_config_info: + if not any(dc["config_id"] == memory_config_info["config_id"] for dc in app_info["memory_configs"]): + app_info["memory_configs"].append(memory_config_info) + app_info["releases"].append(release_info) - + def _extract_memory_content(self, config: Any) -> str: """Extract memory_comtent from config""" if not config or not isinstance(config, dict): return None - + memory_obj = config.get('memory') if memory_obj and isinstance(memory_obj, dict): return memory_obj.get('memory_content') - - return None - - def _get_data_config(self, memory_content: str) -> Dict[str, Any]: - """Retrieve data_comfig information based on memory_comtent""" - try: - data_config_result = DataConfigRepository.query_reflection_config_by_id(self.db, int(memory_content)) - # data_config_query, data_config_params = DataConfigRepository.build_select_reflection(memory_content) - # data_config_result = self.db.execute(text(data_config_query), data_config_params).fetchone() - # if data_config_result is None: + return None + + def _get_memory_config(self, memory_content: str) -> Dict[str, Any]: + """Retrieve memory_config information based on memory_content""" + try: + memory_config_result = MemoryConfigRepository.query_reflection_config_by_id(self.db, int(memory_content)) + + # memory_config_query, memory_config_params = MemoryConfigRepository.build_select_reflection(memory_content) + # memory_config_result = self.db.execute(text(memory_config_query), memory_config_params).fetchone() + # if memory_config_result is None: # return None - - if data_config_result: + + if memory_config_result: return { - "config_id": data_config_result.config_id, - "enable_self_reflexion": data_config_result.enable_self_reflexion, - "iteration_period": data_config_result.iteration_period, - "reflexion_range": data_config_result.reflexion_range, - "baseline": data_config_result.baseline, - "reflection_model_id": data_config_result.reflection_model_id, - "memory_verify": data_config_result.memory_verify, - "quality_assessment": data_config_result.quality_assessment, - "user_id": data_config_result.user_id + "config_id": memory_config_result.config_id, + "enable_self_reflexion": memory_config_result.enable_self_reflexion, + "iteration_period": memory_config_result.iteration_period, + "reflexion_range": memory_config_result.reflexion_range, + "baseline": memory_config_result.baseline, + "reflection_model_id": memory_config_result.reflection_model_id, + "memory_verify": memory_config_result.memory_verify, + "quality_assessment": memory_config_result.quality_assessment, + "user_id": memory_config_result.user_id } except Exception as e: - api_logger.warning(f"查询data_config失败，memory_content: {memory_content}, 错误: {str(e)}") - + api_logger.warning(f"查询memory_config失败，memory_content: {memory_content}, 错误: {str(e)}") + return None - + def _process_end_users(self, app: App, app_info: Dict[str, Any]) -> None: """Processing end-user information for applications""" end_users = self.db.query(EndUser).filter(EndUser.app_id == app.id).all() - + for end_user in end_users: end_user_info = { "id": str(end_user.id), "app_id": str(end_user.app_id) } app_info["end_users"].append(end_user_info) - + print(100*'-') + print(app_info) + def get_end_user_reflection_time(self, end_user_id: str) -> Optional[Any]: """ Read the reflection time of end users @@ -173,7 +177,7 @@ class WorkspaceAppService: except Exception as e: api_logger.error(f"读取用户反思时间失败，end_user_id: {end_user_id}, 错误: {str(e)}") return None - + def update_end_user_reflection_time(self, end_user_id: str) -> bool: """ Update the reflection time of end users to the current time @@ -186,7 +190,7 @@ class WorkspaceAppService: """ try: from datetime import datetime - + end_user = self.db.query(EndUser).filter(EndUser.id == end_user_id).first() if end_user: end_user.reflection_time = datetime.now() @@ -204,7 +208,7 @@ class WorkspaceAppService: class MemoryReflectionService: """Memory reflection service category""" - + def __init__(self,db: Session = Depends(get_db)): self.db=db @@ -223,7 +227,7 @@ class MemoryReflectionService: } config_data_id = config_data['config_id'] - reflection_config = WorkspaceAppService(self.db)._get_data_config(config_data_id) + reflection_config = WorkspaceAppService(self.db)._get_memory_config(config_data_id) if reflection_config is not None and reflection_config['enable_self_reflexion']: reflection_config = self._create_reflection_config_from_data(reflection_config) # 3. 执行反思引擎 @@ -249,22 +253,22 @@ class MemoryReflectionService: "end_user_id": end_user_id, "config_data": config_data } - + async def start_reflection_from_data(self, config_data: Dict[str, Any], end_user_id: str) -> Dict[str, Any]: """ Starting Reflection from Configuration Data - + Args: config_data: Configure data dictionary, including reflective configuration information end_user_id: end_user_id - + Returns: Reflect on the execution results """ try: config_id = config_data.get("config_id") api_logger.info(f"从配置数据启动反思，config_id: {config_id}, end_user_id: {end_user_id}") - + if not config_data.get("enable_self_reflexion", False): return { @@ -274,10 +278,10 @@ class MemoryReflectionService: "end_user_id": end_user_id, "config_data": config_data } - + config_data_id=config_data['config_id'] - reflection_config=WorkspaceAppService(self.db)._get_data_config(config_data_id) + reflection_config=WorkspaceAppService(self.db)._get_memory_config(config_data_id) if reflection_config is not None and reflection_config['enable_self_reflexion']: reflection_config= self._create_reflection_config_from_data(reflection_config) iteration_period = int(reflection_config.iteration_period) diff --git a/api/app/services/memory_storage_service.py b/api/app/services/memory_storage_service.py index 83d5923d..eec1007b 100644 --- a/api/app/services/memory_storage_service.py +++ b/api/app/services/memory_storage_service.py @@ -12,10 +12,14 @@ from datetime import datetime from typing import Any, AsyncGenerator, Dict, List, Optional from app.core.logging_config import get_config_logger, get_logger -from app.core.memory.analytics.hot_memory_tags import get_hot_memory_tags +from app.core.memory.analytics.hot_memory_tags import ( + get_hot_memory_tags, + get_raw_tags_from_db, + filter_tags_with_llm, +) from app.core.memory.analytics.recent_activity_stats import get_recent_activity_stats from app.models.user_model import User -from app.repositories.data_config_repository import DataConfigRepository +from app.repositories.memory_config_repository import MemoryConfigRepository from app.repositories.neo4j.neo4j_connector import Neo4jConnector from app.schemas.memory_config_schema import ConfigurationError from app.schemas.memory_storage_schema import ( @@ -125,7 +129,7 @@ class DataConfigService: # 数据配置服务类（PostgreSQL） if not params.rerank_id: params.rerank_id = configs.get('rerank') - config = DataConfigRepository.create(self.db, params) + config = MemoryConfigRepository.create(self.db, params) self.db.commit() return {"affected": 1, "config_id": config.config_id} @@ -142,20 +146,20 @@ class DataConfigService: # 数据配置服务类（PostgreSQL） # --- Delete --- def delete(self, key: ConfigParamsDelete) -> Dict[str, Any]: # 删除配置参数（按配置ID） - success = DataConfigRepository.delete(self.db, key.config_id) + success = MemoryConfigRepository.delete(self.db, key.config_id) if not success: raise ValueError("未找到配置") return {"affected": 1} # --- Update --- def update(self, update: ConfigUpdate) -> Dict[str, Any]: # 部分更新配置参数 - config = DataConfigRepository.update(self.db, update) + config = MemoryConfigRepository.update(self.db, update) if not config: raise ValueError("未找到配置") return {"affected": 1} def update_extracted(self, update: ConfigUpdateExtracted) -> Dict[str, Any]: # 更新记忆萃取引擎配置参数 - config = DataConfigRepository.update_extracted(self.db, update) + config = MemoryConfigRepository.update_extracted(self.db, update) if not config: raise ValueError("未找到配置") return {"affected": 1} @@ -166,25 +170,38 @@ class DataConfigService: # 数据配置服务类（PostgreSQL） # --- Read --- def get_extracted(self, key: ConfigKey) -> Dict[str, Any]: # 获取萃取配置参数 - result = DataConfigRepository.get_extracted_config(self.db, key.config_id) + result = MemoryConfigRepository.get_extracted_config(self.db, key.config_id) if not result: raise ValueError("未找到配置") return result # --- Read All --- def get_all(self, workspace_id = None) -> List[Dict[str, Any]]: # 获取所有配置参数 - configs = DataConfigRepository.get_all(self.db, workspace_id) + configs = MemoryConfigRepository.get_all(self.db, workspace_id) # 将 ORM 对象转换为字典列表 data_list = [] for config in configs: + # 安全地转换 user_id 为 int + config_id_old = None + if config.config_id_old: + try: + config_id_old = int(config.config_id_old) + except (ValueError, TypeError): + config_id_old = None + + + if config_id_old: + memory_config=config_id_old + else: + memory_config=config.config_id config_dict = { - "config_id": config.config_id, + "config_id": memory_config, "config_name": config.config_name, "config_desc": config.config_desc, "workspace_id": str(config.workspace_id) if config.workspace_id else None, - "group_id": config.group_id, - "user_id": config.user_id, + "end_user_id": config.end_user_id, + "config_id_old": config_id_old, "apply_id": config.apply_id, "llm_id": config.llm_id, "embedding_id": config.embedding_id, @@ -237,7 +254,8 @@ class DataConfigService: # 数据配置服务类（PostgreSQL） ValueError: 当配置无效或参数缺失时 RuntimeError: 当管线执行失败时 """ - project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + from pathlib import Path + project_root = str(Path(__file__).resolve().parents[2]) try: # 发出初始进度事件 @@ -263,7 +281,7 @@ class DataConfigService: # 数据配置服务类（PostgreSQL） try: config_service = MemoryConfigService(self.db) memory_config = config_service.load_memory_config( - config_id=int(cid), + config_id=str(cid), service_name="MemoryStorageService.pilot_run_stream" ) logger.info(f"Configuration loaded successfully: {memory_config.config_name}") @@ -390,8 +408,8 @@ _neo4j_connector = Neo4jConnector() async def search_dialogue(end_user_id: Optional[str] = None) -> Dict[str, Any]: result = await _neo4j_connector.execute_query( - DataConfigRepository.SEARCH_FOR_DIALOGUE, - group_id=end_user_id, + MemoryConfigRepository.SEARCH_FOR_DIALOGUE, + end_user_id=end_user_id, ) data = {"search_for": "dialogue", "num": result[0]["num"]} return data @@ -399,8 +417,8 @@ async def search_dialogue(end_user_id: Optional[str] = None) -> Dict[str, Any]: async def search_chunk(end_user_id: Optional[str] = None) -> Dict[str, Any]: result = await _neo4j_connector.execute_query( - DataConfigRepository.SEARCH_FOR_CHUNK, - group_id=end_user_id, + MemoryConfigRepository.SEARCH_FOR_CHUNK, + end_user_id=end_user_id, ) data = {"search_for": "chunk", "num": result[0]["num"]} return data @@ -408,8 +426,8 @@ async def search_chunk(end_user_id: Optional[str] = None) -> Dict[str, Any]: async def search_statement(end_user_id: Optional[str] = None) -> Dict[str, Any]: result = await _neo4j_connector.execute_query( - DataConfigRepository.SEARCH_FOR_STATEMENT, - group_id=end_user_id, + MemoryConfigRepository.SEARCH_FOR_STATEMENT, + end_user_id=end_user_id, ) data = {"search_for": "statement", "num": result[0]["num"]} return data @@ -417,8 +435,8 @@ async def search_statement(end_user_id: Optional[str] = None) -> Dict[str, Any]: async def search_entity(end_user_id: Optional[str] = None) -> Dict[str, Any]: result = await _neo4j_connector.execute_query( - DataConfigRepository.SEARCH_FOR_ENTITY, - group_id=end_user_id, + MemoryConfigRepository.SEARCH_FOR_ENTITY, + end_user_id=end_user_id, ) data = {"search_for": "entity", "num": result[0]["num"]} return data @@ -426,8 +444,8 @@ async def search_entity(end_user_id: Optional[str] = None) -> Dict[str, Any]: async def search_all(end_user_id: Optional[str] = None) -> Dict[str, Any]: result = await _neo4j_connector.execute_query( - DataConfigRepository.SEARCH_FOR_ALL, - group_id=end_user_id, + MemoryConfigRepository.SEARCH_FOR_ALL, + end_user_id=end_user_id, ) # 检查结果是否为空或长度不足 @@ -461,8 +479,8 @@ async def kb_type_distribution(end_user_id: Optional[str] = None) -> Dict[str, A 聚合 dialogue/chunk/statement/entity 四类计数，返回统一的分布结构，便于前端一次性消费。 """ result = await _neo4j_connector.execute_query( - DataConfigRepository.SEARCH_FOR_ALL, - group_id=end_user_id, + MemoryConfigRepository.SEARCH_FOR_ALL, + end_user_id=end_user_id, ) # 检查结果是否为空或长度不足 @@ -492,21 +510,19 @@ async def kb_type_distribution(end_user_id: Optional[str] = None) -> Dict[str, A async def search_detials(end_user_id: Optional[str] = None) -> List[Dict[str, Any]]: result = await _neo4j_connector.execute_query( - DataConfigRepository.SEARCH_FOR_DETIALS, - group_id=end_user_id, + MemoryConfigRepository.SEARCH_FOR_DETIALS, + end_user_id=end_user_id, ) return result async def search_edges(end_user_id: Optional[str] = None) -> List[Dict[str, Any]]: result = await _neo4j_connector.execute_query( - DataConfigRepository.SEARCH_FOR_EDGES, - group_id=end_user_id, + MemoryConfigRepository.SEARCH_FOR_EDGES, + end_user_id=end_user_id, ) return result - - async def analytics_hot_memory_tags( db: Session, current_user: User, @@ -514,27 +530,79 @@ async def analytics_hot_memory_tags( ) -> List[Dict[str, Any]]: """ 获取热门记忆标签，按数量排序并返回前N个 + + 优化策略： + 1. 先从所有用户收集原始标签（不调用LLM） + 2. 聚合并合并相同标签的频率 + 3. 排序后取前N个 + 4. 只调用一次LLM进行筛选 """ workspace_id = current_user.current_workspace_id # 获取更多标签供LLM筛选（获取limit*4个标签） raw_limit = limit * 4 from app.services.memory_dashboard_service import get_workspace_end_users - end_users = get_workspace_end_users(db, workspace_id, current_user) + # 使用 asyncio.to_thread 避免阻塞事件循环 + end_users = await asyncio.to_thread(get_workspace_end_users, db, workspace_id, current_user) - tags = [] - for end_user in end_users: - tag = await get_hot_memory_tags(str(end_user.id), limit=raw_limit) - if tag: - # 将每个用户的标签列表展平到总列表中 - tags.extend(tag) - - # 按频率降序排序（虽然数据库已经排序，但为了确保正确性再次排序） - sorted_tags = sorted(tags, key=lambda x: x[1], reverse=True) + if not end_users: + return [] - # 只返回前limit个 - top_tags = sorted_tags[:limit] - - return [{"name": t, "frequency": f} for t, f in top_tags] + # 步骤1: 收集所有用户的原始标签（不调用LLM） + connector = Neo4jConnector() + try: + all_raw_tags = [] + for end_user in end_users: + raw_tags = await get_raw_tags_from_db( + connector, + str(end_user.id), + limit=raw_limit, + by_user=False + ) + if raw_tags: + all_raw_tags.extend(raw_tags) + + if not all_raw_tags: + return [] + + # 步骤2: 聚合相同标签的频率 + tag_frequency_map = {} + for tag_name, frequency in all_raw_tags: + if tag_name in tag_frequency_map: + tag_frequency_map[tag_name] += frequency + else: + tag_frequency_map[tag_name] = frequency + + # 步骤3: 按频率降序排序，取前raw_limit个 + sorted_tags = sorted( + tag_frequency_map.items(), + key=lambda x: x[1], + reverse=True + )[:raw_limit] + + if not sorted_tags: + return [] + + # 步骤4: 只调用一次LLM进行筛选 + tag_names = [tag for tag, _ in sorted_tags] + + # 使用第一个用户的end_user_id来获取LLM配置 + # 因为同一工作空间下的用户应该使用相同的配置 + first_end_user_id = str(end_users[0].id) + filtered_tag_names = await filter_tags_with_llm(tag_names, first_end_user_id) + + # 步骤5: 根据LLM筛选结果构建最终列表（保留频率） + final_tags = [] + for tag, freq in sorted_tags: + if tag in filtered_tag_names: + final_tags.append((tag, freq)) + + # 步骤6: 只返回前limit个 + top_tags = final_tags[:limit] + + return [{"name": t, "frequency": f} for t, f in top_tags] + + finally: + await connector.close() async def analytics_recent_activity_stats() -> Dict[str, Any]: diff --git a/api/app/services/model_service.py b/api/app/services/model_service.py index e94a889b..dee6cd1d 100644 --- a/api/app/services/model_service.py +++ b/api/app/services/model_service.py @@ -1,3 +1,4 @@ +from datetime import datetime from sqlalchemy.orm import Session from typing import List, Optional, Dict, Any import uuid @@ -6,11 +7,11 @@ import time import asyncio from app.models.models_model import ModelConfig, ModelApiKey, ModelType -from app.repositories.model_repository import ModelConfigRepository, ModelApiKeyRepository +from app.repositories.model_repository import ModelConfigRepository, ModelApiKeyRepository, ModelBaseRepository from app.schemas import model_schema from app.schemas.model_schema import ( ModelConfigCreate, ModelConfigUpdate, ModelApiKeyCreate, ModelApiKeyUpdate, - ModelConfigQuery, ModelStats + ModelConfigQuery, ModelStats, ModelConfigQueryNew ) from app.core.logging_config import get_business_logger from app.schemas.response_schema import PageData, PageMeta @@ -47,6 +48,26 @@ class ModelConfigService: items=[model_schema.ModelConfig.model_validate(model) for model in models] ) + @staticmethod + def get_model_list_new(db: Session, query: ModelConfigQueryNew, tenant_id: uuid.UUID | None = None) -> List[dict]: + """获取模型配置列表""" + provider_groups, total = ModelConfigRepository.get_list_new(db, query, tenant_id=tenant_id) + + items = [] + for provider, models in provider_groups.items(): + # 验证每个模型并封装分组信息 + validated_models = [model_schema.ModelConfig.model_validate(model) for model in models] + tags = list({model.type for model in validated_models}) + group_item = { + "provider": provider, # 服务商名称 + "logo": validated_models[0].logo, + "tags": tags, + "models": validated_models # 该服务商下的所有模型 + } + items.append(group_item) + + return items + @staticmethod def get_model_by_name(db: Session, name: str, tenant_id: uuid.UUID | None = None) -> ModelConfig: """根据名称获取模型配置""" @@ -228,37 +249,39 @@ class ModelConfigService: # 验证配置 if not model_data.skip_validation and model_data.api_keys: - api_key_data = model_data.api_keys - validation_result = await ModelConfigService.validate_model_config( - db=db, - model_name=api_key_data.model_name, - provider=api_key_data.provider, - api_key=api_key_data.api_key, - api_base=api_key_data.api_base, - model_type=model_data.type, # 传递模型类型 - test_message="Hello" - ) - if not validation_result["valid"]: - raise BusinessException( - f"模型配置验证失败: {validation_result['error']}", - BizCode.INVALID_PARAMETER + api_key_data_list = model_data.api_keys + for api_key_data in api_key_data_list: + validation_result = await ModelConfigService.validate_model_config( + db=db, + model_name=api_key_data.model_name, + provider=api_key_data.provider, + api_key=api_key_data.api_key, + api_base=api_key_data.api_base, + model_type=model_data.type, # 传递模型类型 + test_message="Hello" ) + if not validation_result["valid"]: + raise BusinessException( + f"模型配置验证失败: {validation_result['error']}", + BizCode.INVALID_PARAMETER + ) # 事务处理 - api_key_data = model_data.api_keys - model_config_data = model_data.dict(exclude={"api_keys", "skip_validation"}) + api_key_datas = model_data.api_keys + model_config_data = model_data.model_dump(exclude={"api_keys", "skip_validation"}) # 添加租户ID model_config_data["tenant_id"] = tenant_id model = ModelConfigRepository.create(db, model_config_data) db.flush() # 获取生成的 ID - if api_key_data: - api_key_create_schema = ModelApiKeyCreate( - model_config_id=model.id, - **api_key_data.dict() - ) - ModelApiKeyRepository.create(db, api_key_create_schema) + if api_key_datas: + for api_key_data in api_key_datas: + api_key_create_schema = ModelApiKeyCreate( + model_config_ids=[model.id], + **api_key_data.model_dump() + ) + ModelApiKeyRepository.create(db, api_key_create_schema) db.commit() db.refresh(model) @@ -280,6 +303,116 @@ class ModelConfigService: db.refresh(model) return model + @staticmethod + async def create_composite_model(db: Session, model_data: model_schema.CompositeModelCreate, tenant_id: uuid.UUID) -> ModelConfig: + """创建组合模型""" + if ModelConfigRepository.get_by_name(db, model_data.name, tenant_id=tenant_id): + raise BusinessException("模型名称已存在", BizCode.DUPLICATE_NAME) + + # 验证所有 API Key 存在且类型匹配 + for api_key_id in model_data.api_key_ids: + api_key = ModelApiKeyRepository.get_by_id(db, api_key_id) + if not api_key: + raise BusinessException(f"API Key {api_key_id} 不存在", BizCode.NOT_FOUND) + + # 检查 API Key 关联的模型配置类型 + for model_config in api_key.model_configs: + # chat 和 llm 类型可以兼容 + compatible_types = {ModelType.LLM, ModelType.CHAT} + config_type = model_config.type + request_type = model_data.type + + if not (config_type == request_type or + (config_type in compatible_types and request_type in compatible_types)): + raise BusinessException( + f"API Key {api_key_id} 关联的模型类型 ({model_config.type}) 与组合模型类型 ({model_data.type}) 不匹配", + BizCode.INVALID_PARAMETER + ) + # if model_config.is_composite: + # raise BusinessException( + # f"API Key {api_key_id} 关联的模型是组合模型，不能用于创建新的组合模型", + # BizCode.INVALID_PARAMETER + # ) + + # 创建组合模型 + model_config_data = { + "tenant_id": tenant_id, + "name": model_data.name, + "type": model_data.type, + "logo": model_data.logo, + "description": model_data.description, + "provider": "composite", + "config": model_data.config, + "is_active": model_data.is_active, + "is_public": model_data.is_public, + "is_composite": True + } + if "load_balance_strategy" in model_data.model_fields_set: + model_config_data["load_balance_strategy"] = model_data.load_balance_strategy + + model = ModelConfigRepository.create(db, model_config_data) + db.flush() + + # 关联 API Keys + for api_key_id in model_data.api_key_ids: + api_key = ModelApiKeyRepository.get_by_id(db, api_key_id) + if api_key: + model.api_keys.append(api_key) + + db.commit() + db.refresh(model) + return model + + @staticmethod + async def update_composite_model(db: Session, model_id: uuid.UUID, model_data: model_schema.CompositeModelCreate, tenant_id: uuid.UUID) -> ModelConfig: + """更新组合模型""" + existing_model = ModelConfigRepository.get_by_id(db, model_id, tenant_id=tenant_id) + if not existing_model: + raise BusinessException("模型配置不存在", BizCode.MODEL_NOT_FOUND) + + if not existing_model.is_composite: + raise BusinessException("该模型不是组合模型", BizCode.INVALID_PARAMETER) + + # 验证所有 API Key 存在且类型匹配 + for api_key_id in model_data.api_key_ids: + api_key = ModelApiKeyRepository.get_by_id(db, api_key_id) + if not api_key: + raise BusinessException(f"API Key {api_key_id} 不存在", BizCode.NOT_FOUND) + + for model_config in api_key.model_configs: + compatible_types = {ModelType.LLM, ModelType.CHAT} + config_type = model_config.type + request_type = existing_model.type + + if not (config_type == request_type or + (config_type in compatible_types and request_type in compatible_types)): + raise BusinessException( + f"API Key {api_key_id} 关联的模型类型 ({model_config.type}) 与组合模型类型 ({model_data.type}) 不匹配", + BizCode.INVALID_PARAMETER + ) + + # 更新基本信息 + existing_model.name = model_data.name + # existing_model.type = model_data.type + existing_model.logo = model_data.logo + existing_model.description = model_data.description + existing_model.config = model_data.config + existing_model.is_active = model_data.is_active + existing_model.is_public = model_data.is_public + if "load_balance_strategy" in model_data.model_fields_set: + existing_model.load_balance_strategy = model_data.load_balance_strategy + + # 更新 API Keys 关联 + existing_model.api_keys.clear() + for api_key_id in model_data.api_key_ids: + api_key = ModelApiKeyRepository.get_by_id(db, api_key_id) + if api_key: + existing_model.api_keys.append(api_key) + + db.commit() + db.refresh(existing_model) + return existing_model + @staticmethod def delete_model(db: Session, model_id: uuid.UUID, tenant_id: uuid.UUID | None = None) -> bool: """删除模型配置""" @@ -324,27 +457,133 @@ class ModelApiKeyService: return ModelApiKeyRepository.get_by_model_config(db, model_config_id, is_active) @staticmethod - async def create_api_key(db: Session, api_key_data: ModelApiKeyCreate) -> ModelApiKey: - """创建API Key""" - model_config = ModelConfigRepository.get_by_id(db, api_key_data.model_config_id) - if not model_config: - raise BusinessException("模型配置不存在", BizCode.MODEL_NOT_FOUND) - - validation_result = await ModelConfigService.validate_model_config( + async def create_api_key_by_provider(db: Session, data: model_schema.ModelApiKeyCreateByProvider) -> tuple[ + list[Any], list[Any]]: + """根据provider为多个ModelConfig创建API Key""" + created_keys = [] + failed_models = [] # 记录验证失败的模型 + + for model_config_id in data.model_config_ids: + model_config = ModelConfigRepository.get_by_id(db, model_config_id) + if not model_config: + continue + + # 从ModelBase获取model_name + model_name = model_config.model_base.name if model_config.model_base else model_config.name + + # 检查是否存在API Key（包括软删除） + existing_key = db.query(ModelApiKey).filter( + ModelApiKey.api_key == data.api_key, + ModelApiKey.provider == data.provider, + ModelApiKey.model_name == model_name + ).first() + + if existing_key: + # 如果已存在，重新激活并更新 + if existing_key.is_active: + continue + existing_key.is_active = True + existing_key.api_base = data.api_base + existing_key.description = data.description + existing_key.config = data.config + existing_key.priority = data.priority + existing_key.model_name = model_name + + # 检查是否已关联该模型配置 + if model_config not in existing_key.model_configs: + existing_key.model_configs.append(model_config) + + created_keys.append(existing_key) + continue + + # 验证配置 + validation_result = await ModelConfigService.validate_model_config( db=db, - model_name=api_key_data.model_name, - provider=api_key_data.provider, - api_key=api_key_data.api_key, - api_base=api_key_data.api_base, - model_type=model_config.type, # 传递模型类型 + model_name=model_name, + provider=data.provider, + api_key=data.api_key, + api_base=data.api_base, + model_type=model_config.type, test_message="Hello" ) - print(validation_result) - if not validation_result["valid"]: - raise BusinessException( - f"模型配置验证失败: {validation_result['error']}", - BizCode.INVALID_PARAMETER + if not validation_result["valid"]: + # 记录验证失败的模型，但不抛出异常 + failed_models.append(model_name) + continue + + # 创建API Key + api_key_data = ModelApiKeyCreate( + model_config_ids=[model_config_id], + model_name=model_name, + description=data.description, + provider=data.provider, + api_key=data.api_key, + api_base=data.api_base, + config=data.config, + is_active=data.is_active, + priority=data.priority + ) + api_key_obj = ModelApiKeyRepository.create(db, api_key_data) + created_keys.append(api_key_obj) + + if created_keys: + db.commit() + for key in created_keys: + db.refresh(key) + + return created_keys, failed_models + + @staticmethod + async def create_api_key(db: Session, api_key_data: ModelApiKeyCreate) -> ModelApiKey: + # 验证所有关联的模型配置是否存在 + if api_key_data.model_config_ids: + for model_config_id in api_key_data.model_config_ids: + model_config = ModelConfigRepository.get_by_id(db, model_config_id) + if not model_config: + raise BusinessException("模型配置不存在", BizCode.MODEL_NOT_FOUND) + + # 检查API Key是否已存在(包括软删除) + existing_key = db.query(ModelApiKey).filter( + ModelApiKey.api_key == api_key_data.api_key, + ModelApiKey.provider == api_key_data.provider, + ModelApiKey.model_name == api_key_data.model_name + ).first() + + if existing_key: + if existing_key.is_active: + # 如果已激活，跳过 + raise BusinessException("该API Key已存在", BizCode.DUPLICATE_NAME) + # 如果已存在，重新激活并更新 + existing_key.is_active = True + existing_key.api_base = api_key_data.api_base + existing_key.description = api_key_data.description + existing_key.config = api_key_data.config + existing_key.priority = api_key_data.priority + existing_key.model_name = api_key_data.model_name + + # 检查是否已关联该模型配置 + if model_config not in existing_key.model_configs: + existing_key.model_configs.append(model_config) + + db.commit() + db.refresh(existing_key) + return existing_key + + # 验证配置 + validation_result = await ModelConfigService.validate_model_config( + db=db, + model_name=api_key_data.model_name, + provider=api_key_data.provider, + api_key=api_key_data.api_key, + api_base=api_key_data.api_base, + model_type=model_config.type, + test_message="Hello" ) + if not validation_result["valid"]: + raise BusinessException( + f"模型配置验证失败: {validation_result['error']}", + BizCode.INVALID_PARAMETER + ) api_key = ModelApiKeyRepository.create(db, api_key_data) db.commit() @@ -359,21 +598,19 @@ class ModelApiKeyService: raise BusinessException("API Key不存在", BizCode.NOT_FOUND) # 获取关联的模型配置以获取模型类型 - model_config = ModelConfigRepository.get_by_id(db, existing_api_key.model_config_id) - if not model_config: - raise BusinessException("关联的模型配置不存在", BizCode.MODEL_NOT_FOUND) - - validation_result = await ModelConfigService.validate_model_config( + if existing_api_key.model_configs: + model_config = existing_api_key.model_configs[0] + + validation_result = await ModelConfigService.validate_model_config( db=db, - model_name=api_key_data.model_name, - provider=api_key_data.provider, - api_key=api_key_data.api_key, - api_base=api_key_data.api_base, - model_type=model_config.type, # 传递模型类型 + model_name=api_key_data.model_name or existing_api_key.model_name, + provider=api_key_data.provider or existing_api_key.provider, + api_key=api_key_data.api_key or existing_api_key.api_key, + api_base=api_key_data.api_base or existing_api_key.api_base, + model_type=model_config.type, test_message="Hello" ) - print(validation_result) - if not validation_result["valid"]: + if not validation_result["valid"]: raise BusinessException( f"模型配置验证失败: {validation_result['error']}", BizCode.INVALID_PARAMETER @@ -417,3 +654,87 @@ class ModelApiKeyService: if api_kes and len(api_kes) > 0: return api_kes[0] raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING) + + + +class ModelBaseService: + """基础模型服务""" + + @staticmethod + def get_model_base_list(db: Session, query: model_schema.ModelBaseQuery, tenant_id: uuid.UUID = None) -> List: + models = ModelBaseRepository.get_list(db, query) + + provider_groups = {} + for m in models: + model_dict = model_schema.ModelBase.model_validate(m).model_dump() + if tenant_id: + model_dict['is_added'] = ModelBaseRepository.check_added_by_tenant(db, m.id, tenant_id) + + provider = m.provider + if provider not in provider_groups: + provider_groups[provider] = { + "provider": provider, + "models": [] + } + provider_groups[provider]["models"].append(model_dict) + + return list(provider_groups.values()) + + @staticmethod + def get_model_base_by_id(db: Session, model_base_id: uuid.UUID): + model = ModelBaseRepository.get_by_id(db, model_base_id) + if not model: + raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND) + return model + + @staticmethod + def create_model_base(db: Session, data: model_schema.ModelBaseCreate): + existing = ModelBaseRepository.get_by_name_and_provider(db, data.name, data.provider) + if existing: + raise BusinessException("模型已存在", BizCode.DUPLICATE_NAME) + model_base = ModelBaseRepository.create(db, data.model_dump()) + db.commit() + db.refresh(model_base) + return model_base + + @staticmethod + def update_model_base(db: Session, model_base_id: uuid.UUID, data: model_schema.ModelBaseUpdate): + model_base = ModelBaseRepository.update(db, model_base_id, data.model_dump(exclude_unset=True)) + if not model_base: + raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND) + db.commit() + db.refresh(model_base) + return model_base + + @staticmethod + def delete_model_base(db: Session, model_base_id: uuid.UUID) -> bool: + success = ModelBaseRepository.delete(db, model_base_id) + if not success: + raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND) + db.commit() + return success + + @staticmethod + def add_model_from_plaza(db: Session, model_base_id: uuid.UUID, tenant_id: uuid.UUID) -> ModelConfig: + model_base = ModelBaseRepository.get_by_id(db, model_base_id) + if not model_base: + raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND) + + if ModelBaseRepository.check_added_by_tenant(db, model_base_id, tenant_id): + raise BusinessException("模型已添加", BizCode.DUPLICATE_NAME) + + model_config_data = { + "model_id": model_base_id, + "tenant_id": tenant_id, + "name": model_base.name, + "provider": model_base.provider, + "type": model_base.type, + "logo": model_base.logo, + "description": model_base.description, + "is_composite": False + } + model_config = ModelConfigRepository.create(db, model_config_data) + ModelBaseRepository.increment_add_count(db, model_base_id) + db.commit() + db.refresh(model_config) + return model_config diff --git a/api/app/services/multi_agent_orchestrator.py b/api/app/services/multi_agent_orchestrator.py index 1972f344..d9062eaf 100644 --- a/api/app/services/multi_agent_orchestrator.py +++ b/api/app/services/multi_agent_orchestrator.py @@ -7,6 +7,7 @@ from sqlalchemy.orm import Session from app.models import MultiAgentConfig, AgentConfig, ModelConfig from app.models.multi_agent_model import AggregationStrategy, OrchestrationMode +from app.repositories.model_repository import ModelApiKeyRepository from app.services.agent_registry import AgentRegistry from app.services.master_agent_router import MasterAgentRouter from app.services.conversation_state_manager import ConversationStateManager @@ -2546,10 +2547,14 @@ class MultiAgentOrchestrator: return self._smart_merge_results(results, strategy) # 获取 API Key 配置 - api_key_config = self.db.query(ModelApiKey).filter( - ModelApiKey.model_config_id == default_model_config_id, - ModelApiKey.is_active == True - ).first() + # api_key_config = self.db.query(ModelApiKey).join( + # ModelConfig, ModelApiKey.model_configs + # ).filter( + # ModelConfig.id == default_model_config_id, + # ModelApiKey.is_active.is_(True) + # ).first() + api_keys = ModelApiKeyRepository.get_by_model_config(self.db, default_model_config_id) + api_key_config = api_keys[0] if api_keys else None if not api_key_config: logger.warning("Master Agent 没有可用的 API Key，使用简单整合") @@ -2703,10 +2708,14 @@ class MultiAgentOrchestrator: return # 获取 API Key 配置 - api_key_config = self.db.query(ModelApiKey).filter( - ModelApiKey.model_config_id == default_model_config_id, - ModelApiKey.is_active == True - ).first() + # api_key_config = self.db.query(ModelApiKey).join( + # ModelConfig, ModelApiKey.model_configs + # ).filter( + # ModelConfig.id == default_model_config_id, + # ModelApiKey.is_active.is_(True) + # ).first() + api_keys = ModelApiKeyRepository.get_by_model_config(self.db, default_model_config_id) + api_key_config = api_keys[0] if api_keys else None if not api_key_config: logger.warning("Master Agent 没有可用的 API Key，使用简单整合") diff --git a/api/app/services/multi_agent_service.py b/api/app/services/multi_agent_service.py index 1a08a5af..da984d16 100644 --- a/api/app/services/multi_agent_service.py +++ b/api/app/services/multi_agent_service.py @@ -74,7 +74,7 @@ class MultiAgentService: select(MultiAgentConfig) .where( MultiAgentConfig.app_id == app_id, - MultiAgentConfig.is_active == True + MultiAgentConfig.is_active.is_(True) ) .order_by(MultiAgentConfig.updated_at.desc()) ).first() @@ -144,7 +144,7 @@ class MultiAgentService: select(MultiAgentConfig) .where( MultiAgentConfig.app_id == app_id, - MultiAgentConfig.is_active == True + MultiAgentConfig.is_active.is_(True) ) .order_by(MultiAgentConfig.updated_at.desc()) ).first() diff --git a/api/app/services/pilot_run_service.py b/api/app/services/pilot_run_service.py index 17dfd7eb..755dda14 100644 --- a/api/app/services/pilot_run_service.py +++ b/api/app/services/pilot_run_service.py @@ -91,7 +91,7 @@ async def run_pilot_extraction( dialog = DialogData( context=context, ref_id="pilot_dialog_1", - group_id=str(memory_config.workspace_id), + end_user_id=str(memory_config.workspace_id), user_id=str(memory_config.tenant_id), apply_id=str(memory_config.config_id), metadata={"source": "pilot_run", "input_type": "frontend_text"}, diff --git a/api/app/services/prompt_optimizer_service.py b/api/app/services/prompt_optimizer_service.py index c6142c01..9e447214 100644 --- a/api/app/services/prompt_optimizer_service.py +++ b/api/app/services/prompt_optimizer_service.py @@ -16,7 +16,7 @@ from app.models.prompt_optimizer_model import ( PromptOptimizerSession, RoleType ) -from app.repositories.model_repository import ModelConfigRepository +from app.repositories.model_repository import ModelConfigRepository, ModelApiKeyRepository from app.repositories.prompt_optimizer_repository import ( PromptOptimizerSessionRepository ) @@ -168,7 +168,8 @@ class PromptOptimizerService: logger.info(f"Prompt optimization started, user_id={user_id}, session_id={session_id}") # Create LLM instance - api_config: ModelApiKey = model_config.api_keys[0] + api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config.id) + api_config: ModelApiKey = api_keys[0] if api_keys else None llm = RedBearLLM(RedBearModelConfig( model_name=api_config.model_name, provider=api_config.provider, diff --git a/api/app/services/shared_chat_service.py b/api/app/services/shared_chat_service.py index e5247e5e..1d012088 100644 --- a/api/app/services/shared_chat_service.py +++ b/api/app/services/shared_chat_service.py @@ -4,6 +4,8 @@ import time import asyncio from typing import Optional, Dict, Any, AsyncGenerator from sqlalchemy.orm import Session + +from app.repositories.model_repository import ModelApiKeyRepository from app.services.memory_konwledges_server import write_rag from app.models import ReleaseShare, AppRelease, Conversation from app.services.conversation_service import ConversationService @@ -164,16 +166,20 @@ class SharedChatService: raise ResourceNotFoundException("模型配置", str(model_config_id)) # 获取 API Key - stmt = ( - select(ModelApiKey) - .where( - ModelApiKey.model_config_id == model_config_id, - ModelApiKey.is_active == True - ) - .order_by(ModelApiKey.priority.desc()) - .limit(1) - ) - api_key_obj = self.db.scalars(stmt).first() + # stmt = ( + # select(ModelApiKey).join( + # ModelConfig, ModelApiKey.model_configs + # ) + # .where( + # ModelConfig.id == model_config_id, + # ModelApiKey.is_active.is_(True) + # ) + # .order_by(ModelApiKey.priority.desc()) + # .limit(1) + # ) + # api_key_obj = self.db.scalars(stmt).first() + api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config_id) + api_key_obj = api_keys[0] if api_keys else None if not api_key_obj: raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING) @@ -358,16 +364,20 @@ class SharedChatService: raise ResourceNotFoundException("模型配置", str(model_config_id)) # 获取 API Key - stmt = ( - select(ModelApiKey) - .where( - ModelApiKey.model_config_id == model_config_id, - ModelApiKey.is_active == True - ) - .order_by(ModelApiKey.priority.desc()) - .limit(1) - ) - api_key_obj = self.db.scalars(stmt).first() + # stmt = ( + # select(ModelApiKey).join( + # ModelConfig, ModelApiKey.model_configs + # ) + # .where( + # ModelConfig.id == model_config_id, + # ModelApiKey.is_active.is_(True) + # ) + # .order_by(ModelApiKey.priority.desc()) + # .limit(1) + # ) + # api_key_obj = self.db.scalars(stmt).first() + api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config_id) + api_key_obj = api_keys[0] if api_keys else None if not api_key_obj: raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING) @@ -598,7 +608,7 @@ class SharedChatService: # 获取多 Agent 配置 multi_agent_config = self.db.query(MultiAgentConfig).filter( MultiAgentConfig.app_id == release.app_id, - MultiAgentConfig.is_active == True + MultiAgentConfig.is_active.is_(True) ).first() if not multi_agent_config: @@ -695,7 +705,7 @@ class SharedChatService: # 获取多 Agent 配置 multi_agent_config = self.db.query(MultiAgentConfig).filter( MultiAgentConfig.app_id == release.app_id, - MultiAgentConfig.is_active == True + MultiAgentConfig.is_active.is_(True) ).first() if not multi_agent_config: diff --git a/api/app/services/user_memory_service.py b/api/app/services/user_memory_service.py index 863bccb0..3a90a821 100644 --- a/api/app/services/user_memory_service.py +++ b/api/app/services/user_memory_service.py @@ -155,10 +155,10 @@ class MemoryInsightHelper: """ query = """ MATCH (d:Dialogue) - WHERE d.group_id = $group_id AND d.created_at IS NOT NULL AND d.created_at <> '' + WHERE d.end_user_id = $end_user_id AND d.created_at IS NOT NULL AND d.created_at <> '' RETURN d.created_at AS creation_time """ - records = await self.neo4j_connector.execute_query(query, group_id=self.user_id) + records = await self.neo4j_connector.execute_query(query, end_user_id=self.user_id) if not records: return [] @@ -211,17 +211,17 @@ class MemoryInsightHelper: async def get_social_connections(self) -> dict | None: """Find the user with whom the most memories are shared.""" query = """ - MATCH (c1:Chunk {group_id: $group_id}) + MATCH (c1:Chunk {end_user_id: $end_user_id}) OPTIONAL MATCH (c1)-[:CONTAINS]->(s:Statement) OPTIONAL MATCH (s)<-[:CONTAINS]-(c2:Chunk) - WHERE c1.group_id <> c2.group_id AND s IS NOT NULL AND c2 IS NOT NULL - WITH c2.group_id AS other_user_id, COUNT(DISTINCT s) AS common_statements + WHERE c1.end_user_id <> c2.end_user_id AND s IS NOT NULL AND c2 IS NOT NULL + WITH c2.end_user_id AS other_user_id, COUNT(DISTINCT s) AS common_statements WHERE common_statements > 0 RETURN other_user_id, common_statements ORDER BY common_statements DESC LIMIT 1 """ - records = await self.neo4j_connector.execute_query(query, group_id=self.user_id) + records = await self.neo4j_connector.execute_query(query, end_user_id=self.user_id) if not records or not records[0].get("other_user_id"): return None @@ -230,7 +230,7 @@ class MemoryInsightHelper: time_range_query = """ MATCH (c:Chunk) - WHERE c.group_id IN [$user_id, $other_user_id] + WHERE c.end_user_id IN [$user_id, $other_user_id] RETURN min(c.created_at) AS start_time, max(c.created_at) AS end_time """ time_records = await self.neo4j_connector.execute_query( @@ -294,11 +294,11 @@ class UserSummaryHelper: """Fetch recent statements authored by the user/group for context.""" query = ( "MATCH (s:Statement) " - "WHERE s.group_id = $group_id AND s.statement IS NOT NULL " + "WHERE s.end_user_id = $end_user_id AND s.statement IS NOT NULL " "RETURN s.statement AS statement, s.created_at AS created_at " "ORDER BY created_at DESC LIMIT $limit" ) - rows = await self.connector.execute_query(query, group_id=self.user_id, limit=limit) + rows = await self.connector.execute_query(query, end_user_id=self.user_id, limit=limit) records = [] for r in rows: try: @@ -1152,7 +1152,7 @@ async def analytics_user_summary(end_user_id: Optional[str] = None) -> Dict[str, import re # 创建 UserSummaryHelper 实例 - user_summary_tool = UserSummaryHelper(end_user_id or os.getenv("SELECTED_GROUP_ID", "group_123")) + user_summary_tool = UserSummaryHelper(end_user_id or os.getenv("SELECTED_end_user_id", "group_123")) try: # 1) 收集上下文数据 @@ -1273,10 +1273,10 @@ async def analytics_node_statistics( if end_user_id: query = f""" MATCH (n:{node_type}) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id RETURN count(n) as count """ - result = await _neo4j_connector.execute_query(query, group_id=end_user_id) + result = await _neo4j_connector.execute_query(query, end_user_id=end_user_id) else: query = f""" MATCH (n:{node_type}) @@ -1387,10 +1387,10 @@ async def analytics_memory_types( # 查询 Statement 节点数量 query = """ MATCH (n:Statement) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id RETURN count(n) as count """ - result = await _neo4j_connector.execute_query(query, group_id=end_user_id) + result = await _neo4j_connector.execute_query(query, end_user_id=end_user_id) statement_count = result[0]["count"] if result and len(result) > 0 else 0 # 取三分之一作为隐性记忆数量 implicit_count = round(statement_count / 3) @@ -1504,7 +1504,7 @@ async def analytics_graph_data( 包含节点、边和统计信息的字典 """ try: - # 1. 获取 group_id + # 1. 获取 end_user_id user_uuid = uuid.UUID(end_user_id) repo = EndUserRepository(db) end_user = repo.get_by_id(user_uuid) @@ -1528,7 +1528,7 @@ async def analytics_graph_data( # 基于中心节点的扩展查询 node_query = f""" MATCH path = (center)-[*1..{depth}]-(connected) - WHERE center.group_id = $group_id + WHERE center.end_user_id = $end_user_id AND elementId(center) = $center_node_id WITH collect(DISTINCT center) + collect(DISTINCT connected) as all_nodes UNWIND all_nodes as n @@ -1539,7 +1539,7 @@ async def analytics_graph_data( LIMIT $limit """ node_params = { - "group_id": end_user_id, + "end_user_id": end_user_id, "center_node_id": center_node_id, "limit": limit } @@ -1547,7 +1547,7 @@ async def analytics_graph_data( # 按节点类型过滤查询 node_query = """ MATCH (n) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id AND labels(n)[0] IN $node_types RETURN elementId(n) as id, @@ -1556,7 +1556,7 @@ async def analytics_graph_data( LIMIT $limit """ node_params = { - "group_id": end_user_id, + "end_user_id": end_user_id, "node_types": node_types, "limit": limit } @@ -1564,7 +1564,7 @@ async def analytics_graph_data( # 查询所有节点 node_query = """ MATCH (n) - WHERE n.group_id = $group_id + WHERE n.end_user_id = $end_user_id RETURN elementId(n) as id, labels(n)[0] as label, @@ -1572,7 +1572,7 @@ async def analytics_graph_data( LIMIT $limit """ node_params = { - "group_id": end_user_id, + "end_user_id": end_user_id, "limit": limit } diff --git a/api/app/services/workflow_service.py b/api/app/services/workflow_service.py index b7d5df02..2958f4f9 100644 --- a/api/app/services/workflow_service.py +++ b/api/app/services/workflow_service.py @@ -528,7 +528,8 @@ class WorkflowService: self.conversation_service.add_message( conversation_id=conversation_id_uuid, role=message["role"], - content=message["content"] + content=message["content"], + meta_data=None if message["role"] == "user" else {"usage": token_usage} ) logger.info(f"Workflow Run Success, " f"execution_id: {execution.execution_id}, message count: {len(final_messages)}") @@ -678,7 +679,8 @@ class WorkflowService: self.conversation_service.add_message( conversation_id=conversation_id_uuid, role=message["role"], - content=message["content"] + content=message["content"], + meta_data=None if message["role"] == "user" else {"usage": token_usage} ) logger.info(f"Workflow Run Success, " f"execution_id: {execution.execution_id}, message count: {len(final_messages)}") @@ -761,7 +763,10 @@ class WorkflowService: # 4. 获取工作空间 ID（从 app 获取） from app.models import App - app = self.db.query(App).filter(App.id == app_id).first() + app = self.db.query(App).filter( + App.id == app_id, + App.is_active.is_(True) + ).first() if not app: raise BusinessException( code=BizCode.NOT_FOUND, diff --git a/api/app/tasks.py b/api/app/tasks.py index fa9d1fdf..cdd7945e 100644 --- a/api/app/tasks.py +++ b/api/app/tasks.py @@ -4,6 +4,7 @@ import os import re import time import uuid +from uuid import UUID from datetime import datetime, timezone from math import ceil from typing import Any, Dict, List, Optional @@ -382,16 +383,16 @@ def build_graphrag_for_kb(kb_id: uuid.UUID): @celery_app.task(name="app.core.memory.agent.read_message", bind=True) -def read_message_task(self, group_id: str, message: str, history: List[Dict[str, Any]], search_switch: str, config_id: str,storage_type:str,user_rag_memory_id:str) -> Dict[str, Any]: +def read_message_task(self, end_user_id: str, message: str, history: List[Dict[str, Any]], search_switch: str, config_id: str, storage_type:str, user_rag_memory_id:str) -> Dict[str, Any]: """Celery task to process a read message via MemoryAgentService. Args: - group_id: Group ID for the memory agent (also used as end_user_id) + end_user_id: Group ID for the memory agent (also used as end_user_id) message: User message to process history: Conversation history search_switch: Search switch parameter - config_id: Optional configuration ID + config_id: Configuration ID as string (will be converted to UUID) Returns: Dict containing the result and metadata @@ -401,14 +402,22 @@ def read_message_task(self, group_id: str, message: str, history: List[Dict[str, """ start_time = time.time() + # Convert config_id string to UUID + actual_config_id = None + if config_id: + try: + actual_config_id = uuid.UUID(config_id) if isinstance(config_id, str) else config_id + except (ValueError, AttributeError): + # If conversion fails, leave as None and try to resolve + pass + # Resolve config_id if None - actual_config_id = config_id if actual_config_id is None: try: from app.services.memory_agent_service import get_end_user_connected_config db = next(get_db()) try: - connected_config = get_end_user_connected_config(group_id, db) + connected_config = get_end_user_connected_config(end_user_id, db) actual_config_id = connected_config.get("memory_config_id") finally: db.close() @@ -420,24 +429,42 @@ def read_message_task(self, group_id: str, message: str, history: List[Dict[str, db = next(get_db()) try: service = MemoryAgentService() - return await service.read_memory(group_id, message, history, search_switch, actual_config_id, db, storage_type, user_rag_memory_id) + return await service.read_memory(end_user_id, message, history, search_switch, actual_config_id, db, storage_type, user_rag_memory_id) finally: db.close() try: - result = asyncio.run(_run()) + # 使用 nest_asyncio 来避免事件循环冲突 + try: + import nest_asyncio + nest_asyncio.apply() + except ImportError: + pass + + # 尝试获取现有事件循环，如果不存在则创建新的 + try: + loop = asyncio.get_event_loop() + if loop.is_closed(): + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + except RuntimeError: + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + + result = loop.run_until_complete(_run()) elapsed_time = time.time() - start_time return { "status": "SUCCESS", "result": result, - "group_id": group_id, + "end_user_id": end_user_id, "config_id": config_id, "elapsed_time": elapsed_time, "task_id": self.request.id } except BaseException as e: elapsed_time = time.time() - start_time + # Handle ExceptionGroup from TaskGroup if hasattr(e, 'exceptions'): error_messages = [f"{type(sub_e).__name__}: {str(sub_e)}" for sub_e in e.exceptions] detailed_error = "; ".join(error_messages) @@ -446,7 +473,7 @@ def read_message_task(self, group_id: str, message: str, history: List[Dict[str, return { "status": "FAILURE", "error": detailed_error, - "group_id": group_id, + "end_user_id": end_user_id, "config_id": config_id, "elapsed_time": elapsed_time, "task_id": self.request.id @@ -454,19 +481,13 @@ def read_message_task(self, group_id: str, message: str, history: List[Dict[str, @celery_app.task(name="app.core.memory.agent.write_message", bind=True) -def write_message_task(self, group_id: str, message, config_id: str, storage_type: str, user_rag_memory_id: str) -> Dict[str, Any]: +def write_message_task(self, end_user_id: str, message: str, config_id: str, storage_type:str, user_rag_memory_id:str) -> Dict[str, Any]: """Celery task to process a write message via MemoryAgentService. - 支持两种消息格式： - 1. 字符串格式（向后兼容）：message="user: xxx\nassistant: yyy" - 2. 结构化消息列表（推荐）：message=[{"role": "user", "content": "xxx"}, {"role": "assistant", "content": "yyy"}] - Args: - group_id: Group ID for the memory agent (also used as end_user_id) - message: Message to write (str or list[dict]) - config_id: Optional configuration ID - storage_type: Storage type (neo4j/rag) - user_rag_memory_id: RAG memory ID + end_user_id: Group ID for the memory agent (also used as end_user_id) + message: Message to write + config_id: Configuration ID as string (will be converted to UUID) Returns: Dict containing the result and metadata @@ -477,30 +498,46 @@ def write_message_task(self, group_id: str, message, config_id: str, storage_typ from app.core.logging_config import get_logger logger = get_logger(__name__) - logger.info(f"[CELERY WRITE] Starting write task - group_id={group_id}, config_id={config_id}, storage_type={storage_type}") + logger.info(f"[CELERY WRITE] Starting write task - end_user_id={end_user_id}, config_id={config_id}, storage_type={storage_type}") start_time = time.time() + # Convert config_id string to UUID + actual_config_id = None + if config_id: + try: + actual_config_id = uuid.UUID(config_id) if isinstance(config_id, str) else config_id + logger.info(f"[CELERY WRITE] Converted config_id to UUID: {actual_config_id} (type: {type(actual_config_id).__name__})") + except (ValueError, AttributeError) as e: + logger.error(f"[CELERY WRITE] Invalid config_id format: {config_id}, error: {e}") + return { + "status": "FAILURE", + "error": f"Invalid config_id format: {config_id}", + "end_user_id": end_user_id, + "config_id": config_id, + "elapsed_time": 0.0, + "task_id": self.request.id + } + # Resolve config_id if None - actual_config_id = config_id if actual_config_id is None: try: from app.services.memory_agent_service import get_end_user_connected_config db = next(get_db()) try: - connected_config = get_end_user_connected_config(group_id, db) + connected_config = get_end_user_connected_config(end_user_id, db) actual_config_id = connected_config.get("memory_config_id") finally: db.close() except Exception: # Log but continue - will fail later with proper error pass - + async def _run() -> str: db = next(get_db()) try: - logger.info(f"[CELERY WRITE] Executing MemoryAgentService.write_memory") + logger.info(f"[CELERY WRITE] Executing MemoryAgentService.write_memory with config_id={actual_config_id} (type: {type(actual_config_id).__name__})") service = MemoryAgentService() - result = await service.write_memory(group_id, message, actual_config_id, db, storage_type, user_rag_memory_id) + result = await service.write_memory(end_user_id, message, actual_config_id, db, storage_type, user_rag_memory_id) logger.info(f"[CELERY WRITE] Write completed successfully: {result}") return result except Exception as e: @@ -510,7 +547,24 @@ def write_message_task(self, group_id: str, message, config_id: str, storage_typ db.close() try: - result = asyncio.run(_run()) + # 使用 nest_asyncio 来避免事件循环冲突 + try: + import nest_asyncio + nest_asyncio.apply() + except ImportError: + pass + + # 尝试获取现有事件循环，如果不存在则创建新的 + try: + loop = asyncio.get_event_loop() + if loop.is_closed(): + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + except RuntimeError: + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + + result = loop.run_until_complete(_run()) elapsed_time = time.time() - start_time logger.info(f"[CELERY WRITE] Task completed successfully - elapsed_time={elapsed_time:.2f}s, task_id={self.request.id}") @@ -518,13 +572,14 @@ def write_message_task(self, group_id: str, message, config_id: str, storage_typ return { "status": "SUCCESS", "result": result, - "group_id": group_id, + "end_user_id": end_user_id, "config_id": config_id, "elapsed_time": elapsed_time, "task_id": self.request.id } except BaseException as e: elapsed_time = time.time() - start_time + # Handle ExceptionGroup from TaskGroup if hasattr(e, 'exceptions'): error_messages = [f"{type(sub_e).__name__}: {str(sub_e)}" for sub_e in e.exceptions] detailed_error = "; ".join(error_messages) @@ -536,7 +591,7 @@ def write_message_task(self, group_id: str, message, config_id: str, storage_typ return { "status": "FAILURE", "error": detailed_error, - "group_id": group_id, + "end_user_id": end_user_id, "config_id": config_id, "elapsed_time": elapsed_time, "task_id": self.request.id @@ -635,8 +690,11 @@ def write_total_memory_task(workspace_id: str) -> Dict[str, Any]: try: workspace_uuid = uuid.UUID(workspace_id) - # 1. 查询当前workspace下的所有app - apps = db.query(App).filter(App.workspace_id == workspace_uuid).all() + # 1. 查询当前workspace下的所有app（仅未删除的） + apps = db.query(App).filter( + App.workspace_id == workspace_uuid, + App.is_active.is_(True) + ).all() if not apps: # 如果没有app，总量为0 @@ -875,7 +933,24 @@ def regenerate_memory_cache(self) -> Dict[str, Any]: } try: - result = asyncio.run(_run()) + # 使用 nest_asyncio 来避免事件循环冲突 + try: + import nest_asyncio + nest_asyncio.apply() + except ImportError: + pass + + # 尝试获取现有事件循环，如果不存在则创建新的 + try: + loop = asyncio.get_event_loop() + if loop.is_closed(): + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + except RuntimeError: + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + + result = loop.run_until_complete(_run()) elapsed_time = time.time() - start_time result["elapsed_time"] = elapsed_time result["task_id"] = self.request.id @@ -948,7 +1023,7 @@ def workspace_reflection_task(self) -> Dict[str, Any]: end_users = data['end_users'] for base, config, user in zip(releases, data_configs, end_users): - if int(base['config']) == int(config['config_id']) and base['app_id'] == user['app_id']: + if str(base['config']) == str(config['config_id']) and str(base['app_id']) == str(user['app_id']): # 调用反思服务 api_logger.info(f"为用户 {user['id']} 启动反思，config_id: {config['config_id']}") @@ -1002,7 +1077,24 @@ def workspace_reflection_task(self) -> Dict[str, Any]: } try: - result = asyncio.run(_run()) + # 使用 nest_asyncio 来避免事件循环冲突 + try: + import nest_asyncio + nest_asyncio.apply() + except ImportError: + pass + + # 尝试获取现有事件循环，如果不存在则创建新的 + try: + loop = asyncio.get_event_loop() + if loop.is_closed(): + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + except RuntimeError: + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + + result = loop.run_until_complete(_run()) elapsed_time = time.time() - start_time result["elapsed_time"] = elapsed_time result["task_id"] = self.request.id @@ -1020,7 +1112,7 @@ def workspace_reflection_task(self) -> Dict[str, Any]: @celery_app.task(name="app.tasks.run_forgetting_cycle_task", bind=True) -def run_forgetting_cycle_task(self, config_id: Optional[int] = None) -> Dict[str, Any]: +def run_forgetting_cycle_task(self, config_id: Optional[uuid.UUID] = None) -> Dict[str, Any]: """定时任务：运行遗忘周期 定期执行遗忘周期，识别并融合低激活值的知识节点。 @@ -1048,7 +1140,7 @@ def run_forgetting_cycle_task(self, config_id: Optional[int] = None) -> Dict[str # 运行遗忘周期 report = await forget_service.trigger_forgetting( db=db, - group_id=None, # 处理所有组 + end_user_id=None, # 处理所有组 config_id=config_id ) @@ -1078,4 +1170,11 @@ def run_forgetting_cycle_task(self, config_id: Optional[int] = None) -> Dict[str "duration_seconds": duration } - return asyncio.run(_run()) + # 运行异步函数 + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + try: + result = loop.run_until_complete(_run()) + return result + finally: + loop.close() diff --git a/api/app/utils/app_config_utils.py b/api/app/utils/app_config_utils.py index 514e4565..06549989 100644 --- a/api/app/utils/app_config_utils.py +++ b/api/app/utils/app_config_utils.py @@ -57,7 +57,7 @@ def dict_to_model_parameters(data: Optional[Dict[str, Any]]) -> Optional[Any]: if data is None: return None - from app.schemas import ModelParameters + from app.schemas.app_schema import ModelParameters if isinstance(data, ModelParameters): return data @@ -83,6 +83,13 @@ class AgentConfigProxy: def agent_config_4_app_release(release: AppRelease) -> AgentConfig: config_dict = release.config + # 如果 config 是字符串，解析为字典 + if isinstance(config_dict, str): + import json + try: + config_dict = json.loads(config_dict) + except json.JSONDecodeError: + config_dict = {} agent_config = AgentConfig( app_id=release.app_id, @@ -100,6 +107,14 @@ def agent_config_4_app_release(release: AppRelease) -> AgentConfig: def multi_agent_config_4_app_release(release: AppRelease) -> MultiAgentConfig: config_dict = release.config + + # 如果 config 是字符串，解析为字典 + if isinstance(config_dict, str): + import json + try: + config_dict = json.loads(config_dict) + except json.JSONDecodeError: + config_dict = {} agent_config = MultiAgentConfig( app_id=release.app_id, @@ -120,6 +135,14 @@ def multi_agent_config_4_app_release(release: AppRelease) -> MultiAgentConfig: def workflow_config_4_app_release(release: AppRelease) -> WorkflowConfig: config_dict = release.config + + # 如果 config 是字符串，解析为字典 + if isinstance(config_dict, str): + import json + try: + config_dict = json.loads(config_dict) + except json.JSONDecodeError: + config_dict = {} config = WorkflowConfig( id=config_dict.get("id"), diff --git a/api/app/utils/config_utils.py b/api/app/utils/config_utils.py new file mode 100644 index 00000000..8863ea78 --- /dev/null +++ b/api/app/utils/config_utils.py @@ -0,0 +1,45 @@ +""" +Configuration utility functions + +Shared utilities for configuration handling to avoid circular imports. +""" +from uuid import UUID +from sqlalchemy.orm import Session + + +def resolve_config_id(config_id: UUID | int, db: Session) -> UUID: + """ + 解析 config_id，如果是整数则通过 config_id_old 查找对应的 UUID + + Args: + config_id: 配置ID（UUID 或整数） + db: 数据库会话 + + Returns: + UUID: 解析后的配置ID + + Raises: + ValueError: 当找不到对应的配置时 + """ + from app.models.memory_config_model import MemoryConfig + if isinstance(config_id, UUID): + return config_id + if isinstance(config_id, str) and len(config_id)<=6: + memory_config = db.query(MemoryConfig).filter( + MemoryConfig.config_id_old == config_id + ).first() + + if not memory_config: + raise ValueError(f"未找到 config_id_old={config_id} 对应的配置") + return memory_config.config_id + if isinstance(config_id, int): + memory_config = db.query(MemoryConfig).filter( + MemoryConfig.config_id_old == config_id + ).first() + + if not memory_config: + raise ValueError(f"未找到 config_id_old={config_id} 对应的配置") + + return memory_config.config_id + + return config_id diff --git a/api/app/version_info.json b/api/app/version_info.json index 20896845..86a5e33e 100644 --- a/api/app/version_info.json +++ b/api/app/version_info.json @@ -1,14 +1,46 @@ { + "v0.2.1": { + "introduction": { + "codeName": "启知", + "releaseDate": "2026-1-23", + "upgradePosition": "\uD83D\uDC3B 本次更新主要优化使用体验和修复已知问题，让系统更稳定、更好用。", + "coreUpgrades": [ + "1. 工作流更好用了
* 界面更清晰，一眼看懂怎么配置
* 新增节点输出变量展示，方便其他节点引用
* 修复了几个影响体验的bug", + "2. 智能体配置更简单
* 提示词和变量联动更顺畅
* 配置界面重新整理，找功能更方便", + "3. 记忆系统更稳定
* 优化了情绪记忆和隐性记忆的缓存更新
* 修复了记忆配置页面的报错问题
* 现在能自动识别用户和AI的身份了", + "4. 知识库体验提升
* 修复了文档解析异常的问题
* 上传文档时能看到处理进度了
* 取消了操作也不会报错了", + "5. 系统整体更可靠
* 修复了新用户访问跳转问题
* 流式接口更稳定，长对话不断线
* 调整了菜单顺序，操作更顺手", + "
", + "这次更新虽然不大，但让记忆熊的基础更扎实、体验更流畅。我们继续努力，让AI记忆更好用！", + "记忆熊，记得更牢，用得更好。\uD83D\uDC3B✨" + ] + }, + "introduction_en": { + "codeName": "Qizhi", + "releaseDate": "2026-1-23", + "upgradePosition": "\uD83D\uDC3B This update focuses on improving usability and fixing known issues, making the system more stable and easier to use overall.", + "coreUpgrades": [ + "1. Improved Workflow Experience
* Cleaner, more intuitive UI for easier configuration at a glance
* Added visibility of node output variables, making them easier to reference in downstream nodes
* Fixed several usability-related bugs that affected the workflow experience", + "2. Simpler Agent Configuration
* Smoother linkage between prompts and variables
* Reorganized configuration layout for easier navigation and better clarity", + "3. More Stable Memory System
* Optimized cache refresh for emotional memory and implicit memory
* Fixed error issues on the memory configuration page
* The system can now automatically distinguish between user and AI roles", + "4. Enhanced Knowledge Base Experience
* Fixed issues with document parsing failures
* Upload progress is now displayed during document processing
* Canceling an upload no longer triggers errors", + "5. Overall System Reliability Improvements
* Fixed redirect issues affecting new users
* Improved stability of streaming APIs to prevent interruptions during long conversations
* Adjusted menu ordering for a smoother and more intuitive workflow", + "
", + "Although this is a relatively small update, it strengthens MemoryBear’s foundation and delivers a noticeably smoother experience. We’ll keep refining the system to make AI memory more powerful and easier to use.", + "MemoryBear — remember better, work smarter. \uD83D\uDC3B✨" + ] + } + }, "v0.2.0": { "introduction": { "codeName": "启知", "releaseDate": "2026-1-16", "upgradePosition": "本次为架构升级，核心目标是把\"被动存储\"升级为\"主动认知\"，让系统具备情绪感知、情景理解与类人记忆机制，为后续多智能体协作与专业场景落地奠定底座。", "coreUpgrades": [ - "记忆详情：拟人记忆——情绪引擎、情景记忆、短期记忆、工作记忆、感知记忆、显性记忆、隐性记忆，并配套类脑遗忘机制，实现从感知→情绪→情景→长期沉淀的完整人类记忆闭环", - "可视化工作流：拖拽式节点编排（LLM、知识库、逻辑、工具），业务落地周期由天缩至小时。", - "多模态知识处理：PDF、PPT、MP3、MP4 一键解析，时间感知检索准确率 94.3%，问答对数据即插即用。", - "Agent集群内置\"记忆-知识-工具-审核\"四类角色模板，用户一键生成；主控Agent把复杂任务拆为子任务并行分发，再靠情景记忆统一消解冲突、校验一致性，输出完整报告。" + "1. 记忆详情：拟人记忆——情绪引擎、情景记忆、短期记忆、工作记忆、感知记忆、显性记忆、隐性记忆，并配套类脑遗忘机制，实现从感知→情绪→情景→长期沉淀的完整人类记忆闭环", + "2. 可视化工作流：拖拽式节点编排（LLM、知识库、逻辑、工具），业务落地周期由天缩至小时。", + "3. 多模态知识处理：PDF、PPT、MP3、MP4 一键解析，时间感知检索准确率 94.3%，问答对数据即插即用。", + "4. Agent集群内置\"记忆-知识-工具-审核\"四类角色模板，用户一键生成；主控Agent把复杂任务拆为子任务并行分发，再靠情景记忆统一消解冲突、校验一致性，输出完整报告。" ] }, "introduction_en": { @@ -16,10 +48,10 @@ "releaseDate": "2026-1-16", "upgradePosition": "This release marks a foundational upgrade to the system’s cognitive architecture. The core objective is to evolve the platform from passive information storage into active cognitive intelligence—enabling emotional awareness, situational understanding, and human-like memory mechanisms. This upgrade lays the groundwork for future multi-agent collaboration and domain-specific, production-grade AI applications.", "coreUpgrades": [ - "Human-Like Memory Architecture: A comprehensive, human-inspired memory system is introduced, encompassing emotional processing, situational memory, short-term and working memory, perceptual memory, as well as explicit and implicit memory. Combined with brain-inspired forgetting mechanisms, the system now supports a complete cognitive loop—from perception → emotion → context → long-term consolidation, closely mirroring human memory formation.", - "Visual Workflow Orchestration: A fully visual, drag-and-drop workflow enables modular composition of LLMs, knowledge bases, logic, and tools. This dramatically reduces the time required to move from experimentation to production—from days to hours.", - "Multimodal Knowledge Processing: The system now supports one-click parsing and ingestion of PDF, PPT, MP3, and MP4 content. With time-aware retrieval accuracy reaching 94.3%, structured Q&A data becomes instantly usable for downstream reasoning and generation.", - "Built-in Agent Clusters: Predefined role templates across four categories—Memory, Knowledge, Tools, and Review—can be generated with a single click. A Coordinator Agent decomposes complex tasks into parallel subtasks, while situational memory is used to resolve conflicts, validate consistency, and synthesize outputs into a coherent, end-to-end report." + "1. Human-Like Memory Architecture: A comprehensive, human-inspired memory system is introduced, encompassing emotional processing, situational memory, short-term and working memory, perceptual memory, as well as explicit and implicit memory. Combined with brain-inspired forgetting mechanisms, the system now supports a complete cognitive loop—from perception → emotion → context → long-term consolidation, closely mirroring human memory formation.", + "2. Visual Workflow Orchestration: A fully visual, drag-and-drop workflow enables modular composition of LLMs, knowledge bases, logic, and tools. This dramatically reduces the time required to move from experimentation to production—from days to hours.", + "3. Multimodal Knowledge Processing: The system now supports one-click parsing and ingestion of PDF, PPT, MP3, and MP4 content. With time-aware retrieval accuracy reaching 94.3%, structured Q&A data becomes instantly usable for downstream reasoning and generation.", + "4. Built-in Agent Clusters: Predefined role templates across four categories—Memory, Knowledge, Tools, and Review—can be generated with a single click. A Coordinator Agent decomposes complex tasks into parallel subtasks, while situational memory is used to resolve conflicts, validate consistency, and synthesize outputs into a coherent, end-to-end report." ] } }, @@ -29,16 +61,17 @@ "releaseDate": "2025-12-01", "upgradePosition": "这是一款专注于管理和利用AI记忆的工具，支持RAG和知识图谱两种主流存储方式，旨在为AI应用提供持久化、结构化的\"记忆\"能力。", "coreUpgrades": [ - "记忆空间：用户可以创建独立的空间来隔离不同记忆，并灵活选择存储方式。", - "记忆配置：简化了配置流程，内置自动提取关键信息的\"记忆萃取\"和管理生命周期的\"遗忘\"引擎。", - "知识检索：提供语义、分词和混合三种检索模式，并支持多种参数微调和结果重排序，以提升召回效果。", - "全局管理：支持统一设置默认检索参数，并可一键应用到所有知识库。", - "测试与调试：内置\"召回测试\"功能，方便用户实时验证检索效果并调整参数，支持通过分享码与他人协作。", - "记忆洞察：可查看详细的对话记录、用户画像和分析报告，帮助理解AI的\"记忆\"内容。", - "集成与管理：提供API Key用于系统集成，并包含基本的用户管理功能。", - "界面与体验：采用现代化的卡片式布局和渐变色设计，注重交互的流畅性和视觉美感。", - "起步与使用：文档中提供了清晰的基础使用流程，引导用户从创建空间、配置记忆到测试检索快速上手。", - "版本说明与限制: 记忆熊 v0.1.0 版本\"初心\"囊括智能记忆管理的核心思路和基础能力，为后续开发奠定了基础。", + "1. 记忆空间：用户可以创建独立的空间来隔离不同记忆，并灵活选择存储方式。", + "2. 记忆配置：简化了配置流程，内置自动提取关键信息的\"记忆萃取\"和管理生命周期的\"遗忘\"引擎。", + "3. 知识检索：提供语义、分词和混合三种检索模式，并支持多种参数微调和结果重排序，以提升召回效果。", + "4. 全局管理：支持统一设置默认检索参数，并可一键应用到所有知识库。", + "5. 测试与调试：内置\"召回测试\"功能，方便用户实时验证检索效果并调整参数，支持通过分享码与他人协作。", + "6. 记忆洞察：可查看详细的对话记录、用户画像和分析报告，帮助理解AI的\"记忆\"内容。", + "7. 集成与管理：提供API Key用于系统集成，并包含基本的用户管理功能。", + "8. 界面与体验：采用现代化的卡片式布局和渐变色设计，注重交互的流畅性和视觉美感。", + "9. 起步与使用：文档中提供了清晰的基础使用流程，引导用户从创建空间、配置记忆到测试检索快速上手。", + "10. 版本说明与限制: 记忆熊 v0.1.0 版本\"初心\"囊括智能记忆管理的核心思路和基础能力，为后续开发奠定了基础。", + "
", "文档资源：用户手册、API文档、FAQ", "问题反馈：GitHub Issues、邮件支持", "致谢：感谢所有参与测试和提供反馈的用户！" @@ -49,16 +82,17 @@ "releaseDate": "2025-12-01", "upgradePosition": "A tool focused on managing and utilizing AI memory, supporting both RAG and knowledge graph storage methods, aiming to provide persistent and structured 'memory' capabilities for AI applications.", "coreUpgrades": [ - "Memory Space: Users can create independent spaces to isolate different memories and flexibly choose storage methods.", - "Memory Configuration: Simplified configuration process with built-in 'memory extraction' for automatic key information extraction and 'forgetting' engine for lifecycle management.", - "Knowledge Retrieval: Provides semantic, tokenization, and hybrid retrieval modes with various parameter tuning and result reranking to improve recall.", - "Global Management: Supports unified default retrieval parameter settings with one-click application to all knowledge bases.", - "Testing & Debugging: Built-in 'recall testing' for real-time verification of retrieval effects and parameter adjustment, with sharing code support for collaboration.", - "Memory Insights: View detailed conversation records, user profiles, and analysis reports to understand AI 'memory' content.", - "Integration & Management: Provides API Key for system integration with basic user management features.", - "Interface & Experience: Modern card-based layout with gradient design, focusing on interaction fluidity and visual aesthetics.", - "Getting Started: Documentation provides clear basic usage flow, guiding users from creating spaces, configuring memory to testing retrieval.", - "Version Notes: MemoryBear v0.1.0 'Original Intent' encompasses core concepts and basic capabilities of intelligent memory management, laying foundation for future development.", + "1. Memory Space: Users can create independent spaces to isolate different memories and flexibly choose storage methods.", + "2. Memory Configuration: Simplified configuration process with built-in 'memory extraction' for automatic key information extraction and 'forgetting' engine for lifecycle management.", + "3. Knowledge Retrieval: Provides semantic, tokenization, and hybrid retrieval modes with various parameter tuning and result reranking to improve recall.", + "4. Global Management: Supports unified default retrieval parameter settings with one-click application to all knowledge bases.", + "5. Testing & Debugging: Built-in 'recall testing' for real-time verification of retrieval effects and parameter adjustment, with sharing code support for collaboration.", + "6. Memory Insights: View detailed conversation records, user profiles, and analysis reports to understand AI 'memory' content.", + "7. Integration & Management: Provides API Key for system integration with basic user management features.", + "8. Interface & Experience: Modern card-based layout with gradient design, focusing on interaction fluidity and visual aesthetics.", + "9. Getting Started: Documentation provides clear basic usage flow, guiding users from creating spaces, configuring memory to testing retrieval.", + "10. Version Notes: MemoryBear v0.1.0 'Original Intent' encompasses core concepts and basic capabilities of intelligent memory management, laying foundation for future development.", + "
", "Documentation: User Manual, API Documentation, FAQ", "Feedback: GitHub Issues, Email Support", "Acknowledgments: Thanks to all users who participated in testing and provided feedback!" diff --git a/api/docker-compose.yml b/api/docker-compose.yml index a7337689..f30220cb 100644 --- a/api/docker-compose.yml +++ b/api/docker-compose.yml @@ -15,6 +15,7 @@ services: networks: - default - celery + - sandbox depends_on: - worker-memory - worker-document @@ -63,5 +64,16 @@ services: depends_on: - worker-memory + sandbox: + image: redbear_sandbox:latest + container_name: sandbox + ports: + - "8194" + command: /code/.venv/bin/python main.py + restart: unless-stopped + networks: + - sandbox + networks: celery: + sandbox: diff --git a/api/env.example b/api/env.example index 45ab6c70..274049b9 100644 --- a/api/env.example +++ b/api/env.example @@ -75,6 +75,7 @@ ENABLE_SINGLE_SESSION= MAX_FILE_SIZE=52428800 # 50MB:10 * 1024 * 1024 FILE_PATH=/files +FILE_LOCAL_SERVER_URL="http://localhost:8000/api" # Storage Backend Configuration # Supported values: local, oss, s3 # Default: local diff --git a/api/migrations/env.py b/api/migrations/env.py index 95d74019..e4cd6dfb 100644 --- a/api/migrations/env.py +++ b/api/migrations/env.py @@ -46,7 +46,8 @@ def import_all_models_from_package(package_name: str): # Add the project root to sys.path if not already there # This is crucial for relative imports like 'app.db' to work - project_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..')) + from pathlib import Path + project_root = str(Path(__file__).resolve().parent.parent) if project_root not in sys.path: sys.path.insert(0, project_root) diff --git a/api/migrations/versions/325b759cd66b_2026011240.py b/api/migrations/versions/325b759cd66b_2026011240.py new file mode 100644 index 00000000..048b109b --- /dev/null +++ b/api/migrations/versions/325b759cd66b_2026011240.py @@ -0,0 +1,61 @@ +"""2026011240 + +Revision ID: 325b759cd66b +Revises: 9a936a9ebb20 +Create Date: 2026-01-26 12:37:35.946749 + +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa + + +revision: str = '325b759cd66b' +down_revision: Union[str, None] = '9a936a9ebb20' +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + # 1. 重命名表 data_config -> memory_config + op.rename_table('data_config', 'memory_config') + + # 2. 重命名列 group_id -> end_user_id + op.alter_column('memory_config', 'group_id', new_column_name='end_user_id') + + # 3. config_id: INTEGER -> UUID（保留旧值以便回滚） + op.drop_constraint('data_config_pkey', 'memory_config', type_='primary') + op.alter_column('memory_config', 'config_id', new_column_name='config_id_old', nullable=True) + op.add_column('memory_config', sa.Column('config_id', sa.UUID(), nullable=True)) + # Handle rows where apply_id might be NULL or invalid - generate new UUIDs for those + op.execute(""" + UPDATE memory_config + SET config_id = CASE + WHEN apply_id IS NOT NULL AND apply_id ~ '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$' + THEN apply_id::uuid + ELSE gen_random_uuid() + END + """) + op.alter_column('memory_config', 'config_id', nullable=False) + op.create_primary_key('memory_config_pkey', 'memory_config', ['config_id']) + op.execute("ALTER TABLE memory_config ALTER COLUMN config_id_old DROP DEFAULT") + op.execute("DROP SEQUENCE IF EXISTS data_config_config_id_seq") + + +def downgrade() -> None: + # 1. config_id: UUID -> INTEGER（恢复旧值，空值生成新ID） + op.execute("CREATE SEQUENCE IF NOT EXISTS data_config_config_id_seq") + op.execute("UPDATE memory_config SET config_id_old = nextval('data_config_config_id_seq') WHERE config_id_old IS NULL") + op.drop_constraint('memory_config_pkey', 'memory_config', type_='primary') + op.drop_column('memory_config', 'config_id') + op.alter_column('memory_config', 'config_id_old', new_column_name='config_id', nullable=False) + op.create_primary_key('data_config_pkey', 'memory_config', ['config_id']) + op.execute("ALTER SEQUENCE data_config_config_id_seq OWNED BY memory_config.config_id") + op.execute("SELECT setval('data_config_config_id_seq', COALESCE((SELECT MAX(config_id) FROM memory_config), 1))") + + # 2. 重命名列 end_user_id -> group_id + op.alter_column('memory_config', 'end_user_id', new_column_name='group_id') + + # 3. 重命名表 memory_config -> data_config + op.rename_table('memory_config', 'data_config') diff --git a/api/migrations/versions/5ca246ee7dd4_202601291352.py b/api/migrations/versions/5ca246ee7dd4_202601291352.py new file mode 100644 index 00000000..74931287 --- /dev/null +++ b/api/migrations/versions/5ca246ee7dd4_202601291352.py @@ -0,0 +1,30 @@ +"""202601291352 + +Revision ID: 5ca246ee7dd4 +Revises: 915bed077f8d +Create Date: 2026-01-29 13:52:47.647306 + +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa +from sqlalchemy.dialects import postgresql + +# revision identifiers, used by Alembic. +revision: str = '5ca246ee7dd4' +down_revision: Union[str, None] = '915bed077f8d' +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + # ### commands auto generated by Alembic - please adjust! ### + op.add_column('model_bases', sa.Column('created_at', sa.DateTime(), server_default=sa.text('now()'), nullable=True, comment='创建时间')) + # ### end Alembic commands ### + + +def downgrade() -> None: + # ### commands auto generated by Alembic - please adjust! ### + op.drop_column('model_bases', 'created_at') + # ### end Alembic commands ### diff --git a/api/migrations/versions/5de9b1e28509_20260129212722.py b/api/migrations/versions/5de9b1e28509_20260129212722.py new file mode 100644 index 00000000..cbffad68 --- /dev/null +++ b/api/migrations/versions/5de9b1e28509_20260129212722.py @@ -0,0 +1,80 @@ +"""20260129212722 + +Revision ID: 5de9b1e28509 +Revises: 5ca246ee7dd4 +Create Date: 2026-01-29 21:34:30.978031 + +""" +from typing import Sequence, Union + +import sqlalchemy as sa +from alembic import op +from sqlalchemy.dialects import postgresql + +# revision identifiers, used by Alembic. +revision: str = '5de9b1e28509' +down_revision: Union[str, None] = '5ca246ee7dd4' +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + # Neo4j migration: rename group_id to end_user_id + import asyncio + + from app.repositories.neo4j.neo4j_connector import Neo4jConnector + + async def run_neo4j_upgrade(): + connector = Neo4jConnector() + try: + async def transaction_func(tx): + result = await tx.run(""" + MATCH (n) + WHERE n.group_id IS NOT NULL + SET n.end_user_id = n.group_id + REMOVE n.group_id + WITH count(n) AS node_count + MATCH ()-[r]->() + WHERE r.group_id IS NOT NULL + SET r.end_user_id = r.group_id + REMOVE r.group_id + RETURN node_count, count(r) AS rel_count + """) + return await result.data() + + await connector.execute_write_transaction(transaction_func) + finally: + await connector.close() + + asyncio.run(run_neo4j_upgrade()) + + +def downgrade() -> None: + # Neo4j migration: rename end_user_id back to group_id + import asyncio + + from app.repositories.neo4j.neo4j_connector import Neo4jConnector + + async def run_neo4j_downgrade(): + connector = Neo4jConnector() + try: + async def transaction_func(tx): + result = await tx.run(""" + MATCH (n) + WHERE n.end_user_id IS NOT NULL + SET n.group_id = n.end_user_id + REMOVE n.end_user_id + WITH count(n) AS node_count + MATCH ()-[r]->() + WHERE r.end_user_id IS NOT NULL + SET r.group_id = r.end_user_id + REMOVE r.end_user_id + RETURN node_count, count(r) AS rel_count + """) + return await result.data() + + await connector.execute_write_transaction(transaction_func) + finally: + await connector.close() + + asyncio.run(run_neo4j_downgrade()) \ No newline at end of file diff --git a/api/migrations/versions/75f0ec80e50b_202601271517.py b/api/migrations/versions/75f0ec80e50b_202601271517.py new file mode 100644 index 00000000..a70d7315 --- /dev/null +++ b/api/migrations/versions/75f0ec80e50b_202601271517.py @@ -0,0 +1,57 @@ +"""202601271517 + +Revision ID: 75f0ec80e50b +Revises: 325b759cd66b +Create Date: 2026-01-27 15:26:48.696600 + +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa + + +# revision identifiers, used by Alembic. +revision: str = '75f0ec80e50b' +down_revision: Union[str, None] = '325b759cd66b' +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + # ### commands auto generated by Alembic - please adjust! ### + op.alter_column('memory_config', 'config_id', + existing_type=sa.UUID(), + comment='配置ID', + existing_nullable=False) + op.alter_column('memory_config', 'config_id_old', + existing_type=sa.INTEGER(), + comment='备份的配置ID', + existing_comment='配置ID', + existing_nullable=True) + op.add_column('tenants', sa.Column('external_id', sa.String(length=100), nullable=True)) + op.add_column('tenants', sa.Column('external_source', sa.String(length=50), nullable=True)) + op.create_index(op.f('ix_tenants_external_id'), 'tenants', ['external_id'], unique=False) + op.add_column('users', sa.Column('external_id', sa.String(length=100), nullable=True)) + op.add_column('users', sa.Column('external_source', sa.String(length=50), nullable=True)) + # ### end Alembic commands ### + + +def downgrade() -> None: + # ### commands auto generated by Alembic - please adjust! ### + op.drop_column('users', 'external_source') + op.drop_column('users', 'external_id') + op.drop_index(op.f('ix_tenants_external_id'), table_name='tenants') + op.drop_column('tenants', 'external_source') + op.drop_column('tenants', 'external_id') + op.alter_column('memory_config', 'config_id_old', + existing_type=sa.INTEGER(), + comment='配置ID', + existing_comment='备份的配置ID', + existing_nullable=True) + op.alter_column('memory_config', 'config_id', + existing_type=sa.UUID(), + comment=None, + existing_comment='配置ID', + existing_nullable=False) + # ### end Alembic commands ### diff --git a/api/migrations/versions/915bed077f8d_202601281340.py b/api/migrations/versions/915bed077f8d_202601281340.py new file mode 100644 index 00000000..022f0d25 --- /dev/null +++ b/api/migrations/versions/915bed077f8d_202601281340.py @@ -0,0 +1,224 @@ +"""202601281340 + +Revision ID: 915bed077f8d +Revises: 75f0ec80e50b +Create Date: 2026-01-28 13:38:49.471560 + +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa +from sqlalchemy.dialects import postgresql + +# revision identifiers, used by Alembic. +revision: str = '915bed077f8d' +down_revision: Union[str, None] = '75f0ec80e50b' +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + +BACKUP_TABLE_NAME = 'model_api_keys_backup_20260123' + +def get_temp_models(): + """创建临时模型，用于迁移过程中查询数据""" + metadata = sa.MetaData() + + # 临时ModelApiKey表（仅包含需要的字段） + ModelApiKey = sa.Table( + 'model_api_keys', metadata, + sa.Column('id', sa.UUID(), primary_key=True), + sa.Column('model_config_id', sa.UUID(), nullable=True), + ) + + # 临时关联表（和升级脚本创建的表结构一致） + ModelConfigApiKeyAssociation = sa.Table( + 'model_config_api_key_association', metadata, + sa.Column('model_config_id', sa.UUID(), nullable=False), + sa.Column('api_key_id', sa.UUID(), nullable=False), + sa.Column('created_at', sa.DateTime(), nullable=True), + ) + + ModelApiKeyBackup = sa.Table( + BACKUP_TABLE_NAME, metadata, + sa.Column('id', sa.UUID(), primary_key=True), + sa.Column('model_name', sa.String(), nullable=False), + sa.Column('description', sa.String(), nullable=True), + sa.Column('provider', sa.String(), nullable=False), + sa.Column('api_key', sa.String(), nullable=False), + sa.Column('api_base', sa.String(), nullable=True), + sa.Column('config', sa.JSON(), nullable=True), + sa.Column('usage_count', sa.String(), default="0"), + sa.Column('last_used_at', sa.DateTime(), nullable=True), + sa.Column('priority', sa.String(), default="1"), + sa.Column('model_config_id', sa.UUID(), nullable=True), + sa.Column('created_at', sa.DateTime(), nullable=True), + sa.Column('updated_at', sa.DateTime(), nullable=True), + sa.Column('is_active', sa.Boolean(), default=True), + ) + + return ModelApiKey, ModelConfigApiKeyAssociation, ModelApiKeyBackup + + +def backup_model_api_keys(): + """备份model_api_keys表的结构和数据""" + connection = op.get_bind() + + # 检查备份表是否已存在 + result = connection.execute(sa.text(f""" + SELECT EXISTS ( + SELECT FROM information_schema.tables + WHERE table_name = '{BACKUP_TABLE_NAME}' + ); + """)).scalar() + + if result: + # 备份表已存在，先删除再重建（确保结构一致） + op.execute(f"DROP TABLE IF EXISTS {BACKUP_TABLE_NAME};") + + # 直接复制表结构和数据（PostgreSQL专用，一步完成） + op.execute(f""" + CREATE TABLE {BACKUP_TABLE_NAME} AS + SELECT * FROM model_api_keys; + """) + + # 统计行数 + backup_count = connection.execute(sa.text(f"SELECT COUNT(*) FROM {BACKUP_TABLE_NAME}")).scalar() + original_count = connection.execute(sa.text("SELECT COUNT(*) FROM model_api_keys")).scalar() + + print( + f"已备份model_api_keys表到 {BACKUP_TABLE_NAME} \n" + f" 原表数据行数：{original_count} | 备份表数据行数：{backup_count}" + ) + +# def restore_model_api_keys_from_backup(): +# """从备份表恢复model_api_keys数据（可选，用于回滚失败时手动恢复）""" +# # 1. 清空原表（谨慎使用！） +# # op.execute("TRUNCATE TABLE model_api_keys;") +# +# # 2. 从备份表恢复数据 +# op.execute(f""" +# INSERT INTO model_api_keys +# SELECT * FROM {BACKUP_TABLE_NAME} +# ON CONFLICT (id) DO UPDATE SET +# model_name = EXCLUDED.model_name, +# description = EXCLUDED.description, +# provider = EXCLUDED.provider, +# api_key = EXCLUDED.api_key, +# api_base = EXCLUDED.api_base, +# config = EXCLUDED.config, +# usage_count = EXCLUDED.usage_count, +# last_used_at = EXCLUDED.last_used_at, +# priority = EXCLUDED.priority, +# model_config_id = EXCLUDED.model_config_id, +# created_at = EXCLUDED.created_at, +# updated_at = EXCLUDED.updated_at, +# is_active = EXCLUDED.is_active; +# """) +# print(f"✅ 已从 {BACKUP_TABLE_NAME} 恢复model_api_keys表数据") + +def upgrade() -> None: + backup_model_api_keys() + # ### commands auto generated by Alembic - please adjust! ### + op.create_table('model_bases', + sa.Column('id', sa.UUID(), nullable=False), + sa.Column('logo', sa.String(length=255), nullable=True, comment='模型logo图片URL'), + sa.Column('name', sa.String(), nullable=False, comment='模型唯一标识（如gpt-3.5-turbo）'), + sa.Column('type', sa.String(), nullable=False, comment='模型类型'), + sa.Column('provider', sa.String(), nullable=False), + sa.Column('description', sa.Text(), nullable=True, comment='模型描述'), + sa.Column('is_deprecated', sa.Boolean(), nullable=False, comment='是否弃用'), + sa.Column('is_official', sa.Boolean(), nullable=True, comment='是否供应商官方模型（区分自定义）'), + sa.Column('tags', sa.ARRAY(sa.String()), nullable=False, comment="模型标签（如['聊天', '创作']）"), + sa.Column('add_count', sa.Integer(), nullable=False, comment='模型被用户添加的次数'), + sa.PrimaryKeyConstraint('id'), + sa.UniqueConstraint('name', 'provider', name='uk_model_name_provider') + ) + op.create_index(op.f('ix_model_bases_id'), 'model_bases', ['id'], unique=False) + op.create_index(op.f('ix_model_bases_provider'), 'model_bases', ['provider'], unique=False) + op.create_index(op.f('ix_model_bases_type'), 'model_bases', ['type'], unique=False) + op.create_table('model_config_api_key_association', + sa.Column('model_config_id', sa.UUID(), nullable=False), + sa.Column('api_key_id', sa.UUID(), nullable=False), + sa.Column('created_at', sa.DateTime(), nullable=True), + sa.ForeignKeyConstraint(['api_key_id'], ['model_api_keys.id'], ), + sa.ForeignKeyConstraint(['model_config_id'], ['model_configs.id'], ), + sa.PrimaryKeyConstraint('model_config_id', 'api_key_id') + ) + op.add_column('model_api_keys', sa.Column('description', sa.String(), nullable=True, comment='备注')) + op.add_column('model_configs', sa.Column('model_id', sa.UUID(), nullable=True, comment='基础模型ID')) + op.add_column('model_configs', sa.Column('logo', sa.String(length=255), nullable=True, comment='模型logo图片URL')) + op.add_column('model_configs', sa.Column('provider', sa.String(), server_default='composite', nullable=False, comment='供应商')) + op.add_column('model_configs', sa.Column('is_composite', sa.Boolean(), server_default='true', nullable=False, comment='是否为组合模型')) + op.add_column('model_configs', sa.Column('load_balance_strategy', sa.String(), nullable=True, comment='负载均衡策略')) + op.create_index(op.f('ix_model_configs_model_id'), 'model_configs', ['model_id'], unique=False) + op.create_foreign_key("model_configs_model_id_fkey", 'model_configs', 'model_bases', ['model_id'], ['id']) + connection = op.get_bind() + ModelApiKey, ModelConfigApiKeyAssociation, _ = get_temp_models() + + # 查询所有有model_config_id的API Key + api_keys = connection.execute( + sa.select(ModelApiKey.c.id, ModelApiKey.c.model_config_id) + .where(ModelApiKey.c.model_config_id.isnot(None)) + ).fetchall() + + # 批量插入到多对多表 + if api_keys: + association_data = [ + { + 'model_config_id': row.model_config_id, + 'api_key_id': row.id + } + for row in api_keys + ] + connection.execute(ModelConfigApiKeyAssociation.insert(), association_data) + op.drop_constraint(op.f('model_api_keys_model_config_id_fkey'), 'model_api_keys', type_='foreignkey') + op.drop_column('model_api_keys', 'model_config_id') + # ### end Alembic commands ### + + +def downgrade() -> None: + # ### commands auto generated by Alembic - please adjust! ### + op.drop_constraint("model_configs_model_id_fkey", 'model_configs', type_='foreignkey') + op.drop_index(op.f('ix_model_configs_model_id'), table_name='model_configs') + op.drop_column('model_configs', 'load_balance_strategy') + op.drop_column('model_configs', 'is_composite') + op.drop_column('model_configs', 'provider') + op.drop_column('model_configs', 'logo') + op.drop_column('model_configs', 'model_id') + op.add_column('model_api_keys', sa.Column('model_config_id', sa.UUID(), autoincrement=False, nullable=True, comment='模型配置ID')) + connection = op.get_bind() + ModelApiKey, ModelConfigApiKeyAssociation, _ = get_temp_models() + + # 查询多对多表中的关联数据（取每个API Key的第一个关联的model_config_id） + association_data = connection.execute( + sa.select( + ModelConfigApiKeyAssociation.c.api_key_id, + ModelConfigApiKeyAssociation.c.model_config_id + ).distinct(ModelConfigApiKeyAssociation.c.api_key_id) + ).fetchall() + + # 批量更新model_api_keys表 + if association_data: + for api_key_id, model_config_id in association_data: + connection.execute( + sa.update(ModelApiKey) + .where(ModelApiKey.c.id == api_key_id) + .values(model_config_id=model_config_id) + ) + + op.execute( + "UPDATE model_api_keys SET model_config_id = '00000000-0000-0000-0000-000000000000' WHERE model_config_id IS NULL") + op.alter_column('model_api_keys', 'model_config_id', nullable=False) + op.create_foreign_key(op.f('model_api_keys_model_config_id_fkey'), 'model_api_keys', 'model_configs', ['model_config_id'], ['id']) + op.drop_column('model_api_keys', 'description') + op.drop_table('model_config_api_key_association') + # ### 可选：回滚时恢复备份（如需）### + # restore_model_api_keys_from_backup() + + print( + f"回滚完成！备份表 {BACKUP_TABLE_NAME} 仍保留，如需手动恢复可执行 restore_model_api_keys_from_backup() 函数") + op.drop_index(op.f('ix_model_bases_type'), table_name='model_bases') + op.drop_index(op.f('ix_model_bases_provider'), table_name='model_bases') + op.drop_index(op.f('ix_model_bases_id'), table_name='model_bases') + op.drop_table('model_bases') + # ### end Alembic commands ### diff --git a/api/pyproject.toml b/api/pyproject.toml index 81ac57a1..29597409 100644 --- a/api/pyproject.toml +++ b/api/pyproject.toml @@ -88,7 +88,6 @@ dependencies = [ "cachetools==6.2.1", "ruamel.yaml==0.18.10", "strenum==0.4.15", - "aspose-slides==24.12.0", "opencv-python==4.10.0.84", "numpy>=1.26.0,<2.0.0", "huggingface-hub==0.25.2", diff --git a/api/requirements.txt b/api/requirements.txt index 60e4d090..6cdae2d1 100644 --- a/api/requirements.txt +++ b/api/requirements.txt @@ -83,7 +83,6 @@ olefile==0.47 cachetools==6.2.1 ruamel.yaml==0.18.10 strenum==0.4.15 -aspose-slides==24.12.0 opencv-python==4.10.0.84 numpy>=1.26.0,<2.0.0 huggingface-hub==0.25.2 diff --git a/api/uv.lock b/api/uv.lock index bccaef2c..f3b23325 100644 --- a/api/uv.lock +++ b/api/uv.lock @@ -4462,4 +4462,4 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ff/8d/0309daffea4fcac7981021dbf21cdb2e3427a9e76bafbcdbdf5392ff99a4/zstandard-0.25.0-cp312-cp312-win32.whl", hash = "sha256:23ebc8f17a03133b4426bcc04aabd68f8236eb78c3760f12783385171b0fd8bd", size = 436922, upload-time = "2025-09-14T22:17:24.398Z" }, { url = "https://files.pythonhosted.org/packages/79/3b/fa54d9015f945330510cb5d0b0501e8253c127cca7ebe8ba46a965df18c5/zstandard-0.25.0-cp312-cp312-win_amd64.whl", hash = "sha256:ffef5a74088f1e09947aecf91011136665152e0b4b359c42be3373897fb39b01", size = 506276, upload-time = "2025-09-14T22:17:21.429Z" }, { url = "https://files.pythonhosted.org/packages/ea/6b/8b51697e5319b1f9ac71087b0af9a40d8a6288ff8025c36486e0c12abcc4/zstandard-0.25.0-cp312-cp312-win_arm64.whl", hash = "sha256:181eb40e0b6a29b3cd2849f825e0fa34397f649170673d385f3598ae17cca2e9", size = 462679, upload-time = "2025-09-14T22:17:23.147Z" }, -] +] \ No newline at end of file diff --git a/api_key_mcp_server.py b/api_key_mcp_server.py deleted file mode 100644 index f611dc59..00000000 --- a/api_key_mcp_server.py +++ /dev/null @@ -1,38 +0,0 @@ -#!/usr/bin/env python3 -"""API Key认证MCP服务器""" - -from fastapi import FastAPI, HTTPException, Depends, Header -from typing import Optional -import uvicorn -from mcp_base import MCPRequest, handle_mcp_request, TOOLS - -app = FastAPI(title="API Key MCP Server", version="1.0.0") - -# API Key配置 -API_KEYS = {"test-api-key", "demo-key-123"} - -def verify_api_key(x_api_key: Optional[str] = Header(None)): - """验证API Key""" - if x_api_key and x_api_key in API_KEYS: - return True - raise HTTPException(status_code=401, detail="Invalid API Key") - -@app.get("/") -async def root(): - return {"name": "API Key MCP Server", "version": "1.0.0", "auth_type": "api_key"} - -@app.get("/health") -async def health(): - return {"status": "healthy", "tools": len(TOOLS), "auth_type": "api_key"} - -@app.post("/mcp") -async def mcp_handler(request: MCPRequest, _: bool = Depends(verify_api_key)): - return await handle_mcp_request(request, "API Key MCP Server") - -if __name__ == "__main__": - print("启动API Key认证MCP服务器...") - print("访问 http://localhost:8004 查看服务状态") - print("MCP端点: http://localhost:8004/mcp") - print("认证方式: API Key (Header: X-API-Key)") - print("测试API Keys: test-api-key, demo-key-123") - uvicorn.run(app, host="0.0.0.0", port=8004) \ No newline at end of file diff --git a/basic_auth_mcp_server.py b/basic_auth_mcp_server.py deleted file mode 100644 index 11bb5595..00000000 --- a/basic_auth_mcp_server.py +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/env python3 -"""Basic Auth认证MCP服务器""" - -from fastapi import FastAPI, HTTPException, Depends, Header -from typing import Optional -import uvicorn -import base64 -from mcp_base import MCPRequest, handle_mcp_request, TOOLS - -app = FastAPI(title="Basic Auth MCP Server", version="1.0.0") - -# Basic Auth配置 -BASIC_AUTH_USERS = {"admin": "password", "user": "secret"} - -def verify_basic_auth(authorization: Optional[str] = Header(None)): - """验证Basic Auth""" - if authorization and authorization.startswith("Basic "): - try: - credentials = base64.b64decode(authorization.split(" ")[1]).decode() - username, password = credentials.split(":", 1) - if username in BASIC_AUTH_USERS and BASIC_AUTH_USERS[username] == password: - return True - except: - pass - raise HTTPException(status_code=401, detail="Invalid Basic Auth") - -@app.get("/") -async def root(): - return {"name": "Basic Auth MCP Server", "version": "1.0.0", "auth_type": "basic_auth"} - -@app.get("/health") -async def health(): - return {"status": "healthy", "tools": len(TOOLS), "auth_type": "basic_auth"} - -@app.post("/mcp") -async def mcp_handler(request: MCPRequest, _: bool = Depends(verify_basic_auth)): - return await handle_mcp_request(request, "Basic Auth MCP Server") - -if __name__ == "__main__": - print("启动Basic Auth认证MCP服务器...") - print("访问 http://localhost:8006 查看服务状态") - print("MCP端点: http://localhost:8006/mcp") - print("认证方式: Basic Auth (Header: Authorization: Basic )") - print("测试用户: admin:password, user:secret") - uvicorn.run(app, host="0.0.0.0", port=8006) \ No newline at end of file diff --git a/bearer_token_mcp_server.py b/bearer_token_mcp_server.py deleted file mode 100644 index 57d27f2f..00000000 --- a/bearer_token_mcp_server.py +++ /dev/null @@ -1,40 +0,0 @@ -#!/usr/bin/env python3 -"""Bearer Token认证MCP服务器""" - -from fastapi import FastAPI, HTTPException, Depends, Header -from typing import Optional -import uvicorn -from mcp_base import MCPRequest, handle_mcp_request, TOOLS - -app = FastAPI(title="Bearer Token MCP Server", version="1.0.0") - -# Bearer Token配置 -BEARER_TOKENS = {"bearer-token-123", "demo-bearer-token"} - -def verify_bearer_token(authorization: Optional[str] = Header(None)): - """验证Bearer Token""" - if authorization and authorization.startswith("Bearer "): - token = authorization.split(" ")[1] - if token in BEARER_TOKENS: - return True - raise HTTPException(status_code=401, detail="Invalid Bearer Token") - -@app.get("/") -async def root(): - return {"name": "Bearer Token MCP Server", "version": "1.0.0", "auth_type": "bearer_token"} - -@app.get("/health") -async def health(): - return {"status": "healthy", "tools": len(TOOLS), "auth_type": "bearer_token"} - -@app.post("/mcp") -async def mcp_handler(request: MCPRequest, _: bool = Depends(verify_bearer_token)): - return await handle_mcp_request(request, "Bearer Token MCP Server") - -if __name__ == "__main__": - print("启动Bearer Token认证MCP服务器...") - print("访问 http://localhost:8005 查看服务状态") - print("MCP端点: http://localhost:8005/mcp") - print("认证方式: Bearer Token (Header: Authorization: Bearer )") - print("测试Bearer Tokens: bearer-token-123, demo-bearer-token") - uvicorn.run(app, host="0.0.0.0", port=8005) \ No newline at end of file diff --git a/mcp_base.py b/mcp_base.py deleted file mode 100644 index f571e2fa..00000000 --- a/mcp_base.py +++ /dev/null @@ -1,111 +0,0 @@ -#!/usr/bin/env python3 -"""MCP服务器基础模块 - 共享的模型和处理逻辑""" - -from pydantic import BaseModel -from typing import Dict, Any - -class MCPRequest(BaseModel): - jsonrpc: str = "2.0" - id: str - method: str - params: Dict[str, Any] = {} - -class MCPResponse(BaseModel): - jsonrpc: str = "2.0" - id: str - result: Any = None - error: Dict[str, Any] = None - -# 工具定义 -TOOLS = [ - { - "name": "calculator", - "description": "简单计算器", - "inputSchema": { - "type": "object", - "properties": { - "expression": {"type": "string", "description": "数学表达式"} - }, - "required": ["expression"] - } - }, - { - "name": "echo", - "description": "回显工具", - "inputSchema": { - "type": "object", - "properties": { - "message": {"type": "string", "description": "要回显的消息"} - }, - "required": ["message"] - } - } -] - -async def handle_mcp_request(request: MCPRequest, server_name: str = "MCP Server"): - """处理MCP请求""" - try: - if request.method == "initialize": - return MCPResponse( - id=request.id, - result={ - "protocolVersion": "2024-11-05", - "capabilities": {"tools": {"listChanged": True}}, - "serverInfo": {"name": server_name, "version": "1.0.0"} - } - ) - - elif request.method == "tools/list": - return MCPResponse( - id=request.id, - result={"tools": TOOLS} - ) - - elif request.method == "tools/call": - tool_name = request.params.get("name") - arguments = request.params.get("arguments", {}) - - if tool_name == "calculator": - try: - expression = arguments.get("expression", "") - result = eval(expression) - return MCPResponse( - id=request.id, - result={"content": [{"type": "text", "text": f"结果: {result}"}]} - ) - except Exception as e: - return MCPResponse( - id=request.id, - error={"code": -1, "message": f"计算错误: {str(e)}"} - ) - - elif tool_name == "echo": - message = arguments.get("message", "") - return MCPResponse( - id=request.id, - result={"content": [{"type": "text", "text": f"Echo: {message}"}]} - ) - - else: - return MCPResponse( - id=request.id, - error={"code": -1, "message": f"未知工具: {tool_name}"} - ) - - elif request.method == "ping": - return MCPResponse( - id=request.id, - result={"status": "pong"} - ) - - else: - return MCPResponse( - id=request.id, - error={"code": -1, "message": f"未知方法: {request.method}"} - ) - - except Exception as e: - return MCPResponse( - id=request.id, - error={"code": -1, "message": str(e)} - ) \ No newline at end of file diff --git a/redbear-mem-benchmark b/redbear-mem-benchmark index d9a00be6..4b0257bb 160000 --- a/redbear-mem-benchmark +++ b/redbear-mem-benchmark @@ -1 +1 @@ -Subproject commit d9a00be62d974c0ad071c27e86f878b921c675b6 +Subproject commit 4b0257bb4e7dc384b2aaf849b0bd6eae4b39835d diff --git a/sandbox/Dockerfile b/sandbox/Dockerfile new file mode 100644 index 00000000..677b991c --- /dev/null +++ b/sandbox/Dockerfile @@ -0,0 +1,42 @@ +FROM python:3.12-slim +USER root +WORKDIR /code +LABEL authors="Eterntiy" + +ARG NEED_MIRROR=0 + +RUN --mount=type=cache,id=mem_apt,target=/var/cache/apt,sharing=locked \ + if [ "$NEED_MIRROR" == "1" ]; then \ + sed -i 's|https://ports.ubuntu.com|https://mirrors.tuna.tsinghua.edu.cn|g' /etc/apt/sources.list; \ + sed -i 's|https://archive.ubuntu.com|https://mirrors.tuna.tsinghua.edu.cn|g' /etc/apt/sources.list; \ + fi; \ + rm -f /etc/apt/apt.conf.d/docker-clean && \ + echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache && \ + chmod 1777 /tmp && \ + apt update && \ + apt --no-install-recommends install -y ca-certificates && \ + apt update && \ + apt install -y python3-pip pipx nginx unzip curl wget git vim less && \ + apt-get install -y --no-install-recommends tzdata libseccomp2 libseccomp-dev && \ + ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \ + echo "Asia/Shanghai" > /etc/timezone && \ + apt install -y cargo + +COPY ./app /code/app +COPY ./dependencies /code/dependencies +COPY ./lib /code/lib +COPY ./script /code/script +COPY ./config.yaml /code/config.yaml +COPY ./main.py /code/main.py +COPY ./requirements.txt /code/requirements.txt + +RUN python -m venv .venv +RUN .venv/bin/python3 -m pip install -r requirements.txt + +RUN cargo build --release --manifest-path lib/seccomp_python/Cargo.toml + +HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \ + CMD curl 127.0.0.1:8194/health + + +CMD [".venv/bin/python3", "main.py"] \ No newline at end of file diff --git a/sandbox/app/config.py b/sandbox/app/config.py new file mode 100644 index 00000000..3fa4cab5 --- /dev/null +++ b/sandbox/app/config.py @@ -0,0 +1,134 @@ +"""Configuration management""" +import os +from typing import List, Optional +from pydantic import BaseModel, Field +import yaml + +SANDBOX_USER_ID = 1000 +SANDBOX_GROUP_ID = 1000 + +DEFAULT_PYTHON_LIB_REQUIREMENTS_AMD = [ + "/usr/local/lib/python3.12", + "/usr/lib/python3", + "/usr/lib/x86_64-linux-gnu", + "/etc/ssl/certs/ca-certificates.crt", + "/etc/nsswitch.conf", + "/etc/hosts", + "/etc/resolv.conf", + "/run/systemd/resolve/stub-resolv.conf", + "/run/resolvconf/resolv.conf", + "/etc/localtime", + "/usr/share/zoneinfo", + "/etc/timezone", +] + + +class AppConfig(BaseModel): + """Application configuration""" + port: int = 8194 + debug: bool = True + key: str = "redbear-sandbox" + + +class ProxyConfig(BaseModel): + """Proxy configuration""" + socks5: str = "" + http: str = "" + https: str = "" + + +class Config(BaseModel): + """Global configuration""" + app: AppConfig = Field(default_factory=AppConfig) + max_workers: int = 4 + max_requests: int = 50 + worker_timeout: int = 30 + nodejs_path: str = "node" + enable_network: bool = True + enable_preload: bool = False + + python_path: str = "" + python_lib_paths: list = Field(default=DEFAULT_PYTHON_LIB_REQUIREMENTS_AMD) + python_deps_update_interval: str = "30m" + allowed_syscalls: List[int] = Field(default_factory=list) + proxy: ProxyConfig = Field(default_factory=ProxyConfig) + + +# Global configuration instance +_config: Optional[Config] = None + + +def load_config(config_path: str) -> Config: + """Load configuration from YAML file""" + global _config + + # Load from file + if os.path.exists(config_path): + with open(config_path, 'r') as f: + data = yaml.safe_load(f) + _config = Config(**data) + else: + _config = Config() + + # Override with environment variables + if os.getenv("DEBUG"): + _config.app.debug = os.getenv("DEBUG").lower() in ("true", "1", "yes") + + if os.getenv("MAX_WORKERS"): + _config.max_workers = int(os.getenv("MAX_WORKERS")) + + if os.getenv("MAX_REQUESTS"): + _config.max_requests = int(os.getenv("MAX_REQUESTS")) + + if os.getenv("SANDBOX_PORT"): + _config.app.port = int(os.getenv("SANDBOX_PORT")) + + if os.getenv("WORKER_TIMEOUT"): + _config.worker_timeout = int(os.getenv("WORKER_TIMEOUT")) + + if os.getenv("API_KEY"): + _config.app.key = os.getenv("API_KEY") + + if os.getenv("NODEJS_PATH"): + _config.nodejs_path = os.getenv("NODEJS_PATH") + + if os.getenv("ENABLE_NETWORK"): + _config.enable_network = os.getenv("ENABLE_NETWORK").lower() in ("true", "1", "yes") + + if os.getenv("ENABLE_PRELOAD"): + _config.enable_preload = os.getenv("ENABLE_PRELOAD").lower() in ("true", "1", "yes") + + if os.getenv("ALLOWED_SYSCALLS"): + _config.allowed_syscalls = [int(x) for x in os.getenv("ALLOWED_SYSCALLS").split(",")] + + if os.getenv("SOCKS5_PROXY"): + _config.proxy.socks5 = os.getenv("SOCKS5_PROXY") + + if os.getenv("HTTP_PROXY"): + _config.proxy.http = os.getenv("HTTP_PROXY") + + if os.getenv("HTTPS_PROXY"): + _config.proxy.https = os.getenv("HTTPS_PROXY") + + # python + if os.getenv("PYTHON_PATH"): + _config.python_path = os.getenv("PYTHON_PATH") + + if os.getenv("PYTHON_LIB_PATH"): + _config.python_lib_paths = os.getenv("PYTHON_LIB_PATH").split(',') + + if os.getenv("PYTHON_DEPS_UPDATE_INTERVAL"): + _config.python_deps_update_interval = os.getenv("PYTHON_DEPS_UPDATE_INTERVAL") + + return _config + + +config_path = os.getenv("CONFIG_PATH", "config.yaml") +load_config(config_path) + + +def get_config() -> Config: + """Get global configuration""" + if _config is None: + raise RuntimeError("Configuration not loaded. Call load_config() first.") + return _config diff --git a/sandbox/app/controllers/__init__.py b/sandbox/app/controllers/__init__.py new file mode 100644 index 00000000..b1d965ae --- /dev/null +++ b/sandbox/app/controllers/__init__.py @@ -0,0 +1,8 @@ +from fastapi import APIRouter + +from . import health_controller, sandbox_controller + +manager_router = APIRouter() + +manager_router.include_router(health_controller.router) +manager_router.include_router(sandbox_controller.router) diff --git a/sandbox/app/controllers/health_controller.py b/sandbox/app/controllers/health_controller.py new file mode 100644 index 00000000..4d872e58 --- /dev/null +++ b/sandbox/app/controllers/health_controller.py @@ -0,0 +1,12 @@ +"""Health check endpoint""" +from fastapi import APIRouter + +from app.models import HealthResponse + +router = APIRouter() + + +@router.get("/health", response_model=HealthResponse) +async def health_check(): + """Health check endpoint""" + return HealthResponse(status="healthy", version="2.0.0") diff --git a/sandbox/app/controllers/sandbox_controller.py b/sandbox/app/controllers/sandbox_controller.py new file mode 100644 index 00000000..1a713f52 --- /dev/null +++ b/sandbox/app/controllers/sandbox_controller.py @@ -0,0 +1,59 @@ +"""Sandbox API endpoints""" +from fastapi import APIRouter, Depends + +from app.middleware.auth import verify_api_key +from app.middleware.concurrency import check_max_requests, acquire_worker +from app.models import ( + RunCodeRequest, + ApiResponse, + UpdateDependencyRequest, + error_response +) +from app.services.python_service import ( + run_python_code, + list_python_dependencies, + update_python_dependencies +) + +router = APIRouter( + prefix="/v1/sandbox", + tags=["sandbox"], + dependencies=[Depends(verify_api_key)] +) + + +@router.post( + "/run", + response_model=ApiResponse, + dependencies=[Depends(check_max_requests), + Depends(acquire_worker)] +) +async def run_code(request: RunCodeRequest): + """Execute code in sandbox""" + if request.language == "python3": + return await run_python_code(request.code, request.preload, request.options) + elif request.language == "nodejs": + # TODO + return error_response(-400, "TODO") + else: + return error_response(-400, "unsupported language") + + +@router.get("/dependencies", response_model=ApiResponse) +async def get_dependencies(language: str): + """Get installed dependencies""" + if language == "python3": + return await list_python_dependencies() + else: + return error_response(-400, "unsupported language") + + +@router.post("/dependencies/update", response_model=ApiResponse) +async def update_dependencies(request: UpdateDependencyRequest): + """Update dependencies""" + if request.language == "python3": + return await update_python_dependencies() + else: + return error_response(-400, "unsupported language") + + diff --git a/sandbox/app/core/__init__.py b/sandbox/app/core/__init__.py new file mode 100644 index 00000000..e1abba12 --- /dev/null +++ b/sandbox/app/core/__init__.py @@ -0,0 +1 @@ +"""Core functionality package""" diff --git a/sandbox/app/core/encryption.py b/sandbox/app/core/encryption.py new file mode 100644 index 00000000..47a756c8 --- /dev/null +++ b/sandbox/app/core/encryption.py @@ -0,0 +1,33 @@ +"""Code encryption utilities""" +import base64 + + +def encrypt_code(code: bytes, key: bytes) -> str: + """Encrypt code using XOR cipher with base64 encoding + + Args: + code: Plain code string + key: Encryption key bytes + + Returns: + Base64 encoded encrypted code + """ + key_length = len(key) + encrypted_code = bytearray(len(code)) + for i in range(len(code)): + encrypted_code[i] = code[i] ^ key[i % key_length] + encoded_code = base64.b64encode(encrypted_code).decode("utf-8") + return encoded_code + + +def generate_key(length: int = 64) -> bytes: + """Generate random encryption key + + Args: + length: Key length in bytes (default 64 for 512 bits) + + Returns: + Random key bytes + """ + import secrets + return secrets.token_bytes(length) diff --git a/sandbox/app/core/executor.py b/sandbox/app/core/executor.py new file mode 100644 index 00000000..e87b510c --- /dev/null +++ b/sandbox/app/core/executor.py @@ -0,0 +1,47 @@ +"""Code execution engine""" +import os +from typing import Optional +from abc import ABC, abstractmethod + +from app.config import get_config +from app.logger import get_logger +from app.models import RunnerOptions + + +class ExecutionResult: + """Result of code execution""" + + def __init__(self, stdout: str = "", stderr: str = "", exit_code: int = 0, error: Optional[str] = None): + self.stdout = stdout + self.stderr = stderr + self.exit_code = exit_code + + +class CodeExecutor(ABC): + """Base code executor""" + + def __init__(self): + self.logger = get_logger() + self.config = get_config() + + @abstractmethod + async def run( + self, + code: str, + options: RunnerOptions, + preload: str = "", + timeout: Optional[int] = None + ) -> ExecutionResult: + pass + + def cleanup_temp_file(self, file_path: str) -> None: + """Remove temporary file + + Args: + file_path: Path to file to remove + """ + try: + if os.path.exists(file_path): + os.remove(file_path) + except Exception as e: + self.logger.warning(f"Failed to cleanup temp file {file_path}: {e}") diff --git a/sandbox/app/core/runners/__init__.py b/sandbox/app/core/runners/__init__.py new file mode 100644 index 00000000..96c5e380 --- /dev/null +++ b/sandbox/app/core/runners/__init__.py @@ -0,0 +1 @@ +"""Code runners package""" diff --git a/sandbox/app/core/runners/python/__init__.py b/sandbox/app/core/runners/python/__init__.py new file mode 100644 index 00000000..99a56ef7 --- /dev/null +++ b/sandbox/app/core/runners/python/__init__.py @@ -0,0 +1,4 @@ +# -*- coding: UTF-8 -*- +# Author: Eternity +# @Email: 1533512157@qq.com +# @Time : 2026/1/23 11:27 diff --git a/sandbox/app/core/runners/python/env.py b/sandbox/app/core/runners/python/env.py new file mode 100644 index 00000000..d82b0522 --- /dev/null +++ b/sandbox/app/core/runners/python/env.py @@ -0,0 +1,50 @@ +import asyncio +import tempfile +import stat +from pathlib import Path + +from app.config import get_config +from app.core.runners.python.settings import LIB_PATH +from app.logger import get_logger + +logger = get_logger() + + +async def prepare_python_dependencies_env(): + config = get_config() + + with tempfile.TemporaryDirectory(dir="/") as root_path: + root = Path(root_path) + + env_sh = root / "env.sh" + with open("script/env.sh") as f: + env_sh.write_text(f.read()) + env_sh.chmod(env_sh.stat().st_mode | stat.S_IXUSR) + + for lib_path in config.python_lib_paths: + lib_path = Path(lib_path) + + if not lib_path.exists(): + logger.warning("python lib path %s is not available", lib_path) + continue + + cmd = [ + "bash", + str(env_sh), + str(lib_path), + str(LIB_PATH), + ] + + process = await asyncio.create_subprocess_exec( + *cmd, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE + ) + + stdout, stderr = await process.communicate() + retcode = process.returncode + + if retcode != 0: + logger.error( + f"create env error for file {lib_path}: retcode={retcode}, stderr={stderr.decode()}" + ) diff --git a/sandbox/app/core/runners/python/prescript.py b/sandbox/app/core/runners/python/prescript.py new file mode 100644 index 00000000..950710ea --- /dev/null +++ b/sandbox/app/core/runners/python/prescript.py @@ -0,0 +1,56 @@ +import ctypes +import os +import sys +import traceback +from base64 import b64decode + + +# Setup exception hook +def excepthook(etype, value, tb): + sys.stderr.write("".join(traceback.format_exception(etype, value, tb))) + sys.stderr.flush() + sys.exit(-1) + + +sys.excepthook = excepthook + +# Load security library if available +lib = ctypes.CDLL("./libpython.so") +lib.init_seccomp.argtypes = [ctypes.c_uint32, ctypes.c_uint32, ctypes.c_bool] +lib.init_seccomp.restype = None # TODO: raise error info + +# Get running path +running_path = sys.argv[1] +if not running_path: + exit(-1) + +# Get decrypt key +key = sys.argv[2] +if not key: + exit(-1) + +key = b64decode(key) + +os.chdir(running_path) + +# Preload code +{{preload}} + +# Apply security if library is available +lib.init_seccomp({{uid}}, {{gid}}, {{enable_network}}) + +# Decrypt and execute code +code = b64decode("{{code}}") + + +def decrypt(code, key): + key_len = len(key) + code_len = len(code) + code = bytearray(code) + for i in range(code_len): + code[i] = code[i] ^ key[i % key_len] + return bytes(code) + + +code = decrypt(code, key) +exec(code) diff --git a/sandbox/app/core/runners/python/python_runner.py b/sandbox/app/core/runners/python/python_runner.py new file mode 100644 index 00000000..30792b91 --- /dev/null +++ b/sandbox/app/core/runners/python/python_runner.py @@ -0,0 +1,154 @@ +"""Python code runner""" +import asyncio +import base64 +import os +import uuid +from typing import Optional + +from app.config import SANDBOX_USER_ID, SANDBOX_GROUP_ID, get_config +from app.core.encryption import generate_key, encrypt_code +from app.core.executor import CodeExecutor, ExecutionResult +from app.core.runners.python.settings import check_lib_avaiable, release_lib_binary, LIB_PATH +from app.logger import get_logger +from app.models import RunnerOptions + +# Python sandbox prescript template +with open("app/core/runners/python/prescript.py") as f: + PYTHON_PRESCRIPT = f.read() + +logger = get_logger() + + +class PythonRunner(CodeExecutor): + """Python code runner with security isolation""" + + def __init__(self): + super().__init__() + + @staticmethod + def init_enviroment(code: bytes, preload, options: RunnerOptions) -> tuple[str, str]: + if not check_lib_avaiable(): + release_lib_binary(False) + config = get_config() + code_file_name = uuid.uuid4().hex.replace("-", "_") + + script = PYTHON_PRESCRIPT.replace("{{uid}}", str(SANDBOX_USER_ID), 1) + script = script.replace("{{gid}}", str(SANDBOX_GROUP_ID), 1) + script = script.replace( + "{{enable_network}}", + str(int(options.enable_network and config.enable_network) + ), + 1 + ) + script = script.replace("{{preload}}", f"{preload}\n", 1) + + key = generate_key(64) + + encoded_code = encrypt_code(code, key) + encoded_key = base64.b64encode(key).decode("utf-8") + + script = script.replace("{{code}}", encoded_code, 1) + + code_path = f"{LIB_PATH}/tmp/{code_file_name}.py" + try: + os.makedirs(os.path.dirname(code_path), mode=0o755, exist_ok=True) + with open(code_path, "w", encoding="utf-8") as f: + f.write(script) + os.chmod(code_path, 0o755) + + except OSError as e: + raise RuntimeError(f"Failed to write {code_path}") from e + + return code_path, encoded_key + + async def run( + self, + code: str, + options: RunnerOptions, + preload: str = "", + timeout: Optional[int] = None + ) -> ExecutionResult: + """Run Python code in sandbox + + Args: + options: + code: Base64 encoded encrypted code + preload: Preload code to execute before main code + timeout: Execution timeout in seconds + + Returns: + ExecutionResult with stdout, stderr, and exit code + """ + config = self.config + + if timeout is None: + timeout = config.worker_timeout + + # Check if preload is allowed + if not config.enable_preload: + preload = "" + code = base64.b64decode(code) + script_path, encoded_key = self.init_enviroment(code, preload, options=options) + + try: + # Setup environment + env = {} + + # Add proxy settings if configured + if config.proxy.socks5: + env["HTTPS_PROXY"] = config.proxy.socks5 + env["HTTP_PROXY"] = config.proxy.socks5 + elif config.proxy.https or config.proxy.http: + if config.proxy.https: + env["HTTPS_PROXY"] = config.proxy.https + if config.proxy.http: + env["HTTP_PROXY"] = config.proxy.http + + # Add allowed syscalls if configured + if config.allowed_syscalls: + env["ALLOWED_SYSCALLS"] = ",".join(map(str, config.allowed_syscalls)) + + # Execute with Python interpreter + logger.info(encoded_key) + + process = await asyncio.create_subprocess_exec( + config.python_path, + script_path, + LIB_PATH, + encoded_key, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + env=env, + cwd=LIB_PATH + ) + + # Wait for completion with timeout + try: + stdout, stderr = await asyncio.wait_for( + process.communicate(), + timeout=timeout + ) + + return ExecutionResult( + stdout=stdout.decode('utf-8', errors='replace'), + stderr=stderr.decode('utf-8', errors='replace'), + exit_code=process.returncode + ) + + except asyncio.TimeoutError: + # Kill process on timeout + try: + process.kill() + await process.wait() + except: + pass + + return ExecutionResult( + stdout="", + stderr="Execution timeout", + exit_code=-1, + ) + + finally: + # Cleanup temporary file + self.cleanup_temp_file(script_path) diff --git a/sandbox/app/core/runners/python/settings.py b/sandbox/app/core/runners/python/settings.py new file mode 100644 index 00000000..aee8827b --- /dev/null +++ b/sandbox/app/core/runners/python/settings.py @@ -0,0 +1,62 @@ +import os + +from app.logger import get_logger + +logger = get_logger() + +RELEASE_LIB_PATH = "./lib/seccomp_python/target/release/libpython.so" +LIB_PATH = "/var/sandbox/sandbox-python" +LIB_NAME = "libpython.so" + +try: + with open(RELEASE_LIB_PATH, "rb") as f: + _PYTHON_LIB = f.read() +except: + logger.critical("failed to load python lib") + raise + + +def check_lib_avaiable(): + return os.path.exists(os.path.join(LIB_PATH, LIB_NAME)) + + +def release_lib_binary(force_remove: bool): + logger.info("init runtime enviroment") + lib_file = os.path.join(LIB_PATH, LIB_NAME) + if os.path.exists(lib_file): + if force_remove: + try: + os.remove(lib_file) + except OSError: + logger.critical(f"failed to remove {os.path.join(LIB_PATH, LIB_NAME)}") + raise + + try: + os.makedirs(LIB_PATH, mode=0o755, exist_ok=True) + except OSError: + logger.critical(f"failed to create {LIB_PATH}") + raise + + try: + with open(lib_file, "wb") as f: + f.write(_PYTHON_LIB) + os.chmod(lib_file, 0o755) + except OSError: + logger.critical(f"failed to write {lib_file}") + raise + else: + try: + os.makedirs(LIB_PATH, mode=0o755, exist_ok=True) + except OSError: + logger.critical(f"failed to create {LIB_PATH}") + raise + + try: + with open(lib_file, "wb") as f: + f.write(_PYTHON_LIB) + os.chmod(lib_file, 0o755) + except OSError: + logger.critical(f"failed to write {lib_file}") + raise + + logger.info("python runner environment initialized") diff --git a/sandbox/app/dependencies.py b/sandbox/app/dependencies.py new file mode 100644 index 00000000..6e88aaf2 --- /dev/null +++ b/sandbox/app/dependencies.py @@ -0,0 +1,161 @@ +"""Dependency management""" +import asyncio +from pathlib import Path +from typing import List, Dict + +from app.config import get_config +from app.core.runners.python.env import prepare_python_dependencies_env +from app.logger import get_logger + + +async def setup_dependencies(): + """Setup initial dependencies""" + logger = get_logger() + + try: + logger.info("Installing Python dependencies...") + await install_python_dependencies() + logger.info("Python dependencies installed") + + logger.info("Preparing Python dependencies environment...") + await prepare_python_dependencies_env() + logger.info("Python dependencies environment ready") + + except Exception as e: + logger.error(f"Failed to setup dependencies: {e}") + + +async def update_dependencies(): + # TODO + return + + +async def install_python_dependencies(): + """Install Python dependencies from requirements file""" + logger = get_logger() + config = get_config() + + # Check if requirements file exists + req_file = Path("dependencies/python-requirements.txt") + if not req_file.exists(): + logger.warning("Python requirements file not found, skipping installation") + return + + # Read requirements + requirements = req_file.read_text().strip() + if not requirements: + logger.info("No Python requirements to install") + return + + # Install using pip + cmd = [ + config.python_path, + "-m", + "pip", + "install", + "--upgrade" + ] + + # Add packages from requirements + for line in requirements.split("\n"): + line = line.strip() + if line and not line.startswith("#"): + cmd.append(line) + + try: + process = await asyncio.create_subprocess_exec( + *cmd, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE + ) + + stdout, stderr = await process.communicate() + + if process.returncode != 0: + logger.error(f"Failed to install Python dependencies: {stderr.decode()}") + else: + logger.info("Python dependencies installed successfully") + + except Exception as e: + logger.error(f"Error installing Python dependencies: {e}") + + +async def list_dependencies(language: str) -> List[Dict[str, str]]: + """List installed dependencies + + Args: + language: Language (python or Node.js) + + Returns: + List of dependencies with name and version + """ + if language == "python": + return await list_python_packages() + else: + return [] + + +async def list_python_packages() -> List[Dict[str, str]]: + """List installed Python packages""" + config = get_config() + + try: + process = await asyncio.create_subprocess_exec( + config.python_path, + "-m", + "pip", + "list", + "--format=freeze", + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE + ) + + stdout, stderr = await process.communicate() + + if process.returncode != 0: + return [] + + # Parse output + packages = [] + for line in stdout.decode().split("\n"): + line = line.strip() + if line and "==" in line: + name, version = line.split("==", 1) + packages.append({"name": name, "version": version}) + + return packages + + except Exception as e: + get_logger().error(f"Failed to list Python packages: {e}") + return [] + + +async def update_dependencies_periodically(): + """Periodically update dependencies""" + logger = get_logger() + config = get_config() + + # Parse interval + interval_str = config.python_deps_update_interval + + # Convert to seconds + if interval_str.endswith("m"): + interval = int(interval_str[:-1]) * 60 + elif interval_str.endswith("h"): + interval = int(interval_str[:-1]) * 3600 + elif interval_str.endswith("s"): + interval = int(interval_str[:-1]) + else: + interval = 1800 # Default 30 minutes + + logger.info(f"Starting periodic dependency updates every {interval} seconds") + + while True: + await asyncio.sleep(interval) + + try: + logger.info("Updating Python dependencies...") + # TODO: await update_dependencies("python") + logger.info("Python dependencies updated successfully") + except Exception as e: + logger.error(f"Failed to update Python dependencies: {e}") diff --git a/sandbox/app/logger.py b/sandbox/app/logger.py new file mode 100644 index 00000000..de2ccc9e --- /dev/null +++ b/sandbox/app/logger.py @@ -0,0 +1,42 @@ +"""Logging configuration""" +import logging +import sys +from typing import Optional + +from app.config import get_config + +_logger: Optional[logging.Logger] = None + + +def setup_logger() -> logging.Logger: + """Setup application logger""" + global _logger + + config = get_config() + + # Create logger + _logger = logging.getLogger("sandbox") + _logger.setLevel(logging.DEBUG if config.app.debug else logging.INFO) + + # Create console handler + handler = logging.StreamHandler(sys.stdout) + handler.setLevel(logging.DEBUG if config.app.debug else logging.INFO) + + # Create formatter + formatter = logging.Formatter( + '%(asctime)s - %(name)s - %(levelname)s - %(message)s', + datefmt='%Y-%m-%d %H:%M:%S' + ) + handler.setFormatter(formatter) + + # Add handler to logger + _logger.addHandler(handler) + + return _logger + + +def get_logger() -> logging.Logger: + """Get application logger""" + if _logger is None: + return setup_logger() + return _logger diff --git a/sandbox/app/middleware/__init__.py b/sandbox/app/middleware/__init__.py new file mode 100644 index 00000000..77d6403c --- /dev/null +++ b/sandbox/app/middleware/__init__.py @@ -0,0 +1 @@ +"""Middleware package""" diff --git a/sandbox/app/middleware/auth.py b/sandbox/app/middleware/auth.py new file mode 100644 index 00000000..8a93a793 --- /dev/null +++ b/sandbox/app/middleware/auth.py @@ -0,0 +1,15 @@ +"""Authentication middleware""" +from fastapi import Header, HTTPException, status + +from app.config import get_config + + +async def verify_api_key(x_api_key: str = Header(..., alias="X-Api-Key")): + """Verify API key from request header""" + config = get_config() + if x_api_key != config.app.key: + raise HTTPException( + status_code=status.HTTP_401_UNAUTHORIZED, + detail="Invalid API key" + ) + return x_api_key diff --git a/sandbox/app/middleware/concurrency.py b/sandbox/app/middleware/concurrency.py new file mode 100644 index 00000000..8d8325a4 --- /dev/null +++ b/sandbox/app/middleware/concurrency.py @@ -0,0 +1,48 @@ +"""Concurrency control middleware""" +import asyncio +from fastapi import HTTPException, status + +from app.config import get_config +from app.models import error_response + + +# Global semaphores +_worker_semaphore: None | asyncio.Semaphore = None +_request_counter = 0 +_request_lock = asyncio.Lock() + + +def init_concurrency_control(): + """Initialize concurrency control""" + global _worker_semaphore + config = get_config() + _worker_semaphore = asyncio.Semaphore(config.max_workers) + + +async def check_max_requests(): + """Check if max requests limit is reached""" + global _request_counter + config = get_config() + + async with _request_lock: + if _request_counter >= config.max_requests: + raise HTTPException( + status_code=status.HTTP_503_SERVICE_UNAVAILABLE, + detail=error_response(-503, "Too many requests") + ) + _request_counter += 1 + + try: + yield + finally: + async with _request_lock: + _request_counter -= 1 + + +async def acquire_worker(): + """Acquire a worker slot""" + if _worker_semaphore is None: + init_concurrency_control() + + async with _worker_semaphore: + yield diff --git a/sandbox/app/models.py b/sandbox/app/models.py new file mode 100644 index 00000000..e7492b4c --- /dev/null +++ b/sandbox/app/models.py @@ -0,0 +1,80 @@ +"""Data models""" +from typing import Optional, Any + +from pydantic import BaseModel, Field + + +class RunnerOptions(BaseModel): + enable_network: bool = Field(default=False, description="Sandbox network flag") + + +class RunCodeRequest(BaseModel): + """Request model for code execution""" + language: str = Field(..., description="Programming language (python3 or nodejs)") + code: str = Field(..., description="Base64 encoded encrypted code") + preload: Optional[str] = Field(default="", description="Preload code") + options: RunnerOptions = Field(default_factory=RunnerOptions, description="Enable network access") + + +class RunCodeResponse(BaseModel): + """Response model for code execution""" + stdout: str = Field(default="", description="Standard output") + stderr: str = Field(default="", description="Standard error") + + +class DependencyRequest(BaseModel): + """Request model for dependency operations""" + language: str = Field(..., description="Programming language") + + +class UpdateDependencyRequest(BaseModel): + """Request model for updating dependencies""" + language: str = Field(..., description="Programming language") + packages: list[str] = Field(default_factory=list, description="Packages to install") + + +class Dependency(BaseModel): + """Dependency information""" + name: str + version: str + + +class ListDependenciesResponse(BaseModel): + """Response model for listing dependencies""" + dependencies: list[Dependency] = Field(default_factory=list) + + +class RefreshDependenciesResponse(BaseModel): + """Response model for refreshing dependencies""" + dependencies: list[Dependency] = Field(default_factory=list) + + +class UpdateDependenciesResponse(BaseModel): + """Response model for updating dependencies""" + success: bool = True + installed: list[str] = Field(default_factory=list) + + +class HealthResponse(BaseModel): + """Health check response""" + status: str = "healthy" + version: str = "2.0.0" + + +class ApiResponse(BaseModel): + """Standard API response wrapper""" + code: int = Field(default=0, description="Response code (0 for success, negative for error)") + message: str = Field(default="success", description="Response message") + data: Optional[Any] = Field(default=None, description="Response data") + + +def success_response(data: Any) -> ApiResponse: + """Create success response""" + return ApiResponse(code=0, message="success", data=data) + + +def error_response(code: int, message: str) -> ApiResponse: + """Create error response""" + if code >= 0: + code = -1 + return ApiResponse(code=code, message=message, data=None) diff --git a/sandbox/app/services/__init__.py b/sandbox/app/services/__init__.py new file mode 100644 index 00000000..e3726046 --- /dev/null +++ b/sandbox/app/services/__init__.py @@ -0,0 +1 @@ +"""Services package""" diff --git a/sandbox/app/services/python_service.py b/sandbox/app/services/python_service.py new file mode 100644 index 00000000..210b2086 --- /dev/null +++ b/sandbox/app/services/python_service.py @@ -0,0 +1,80 @@ +"""Python execution service""" +import signal + +from app.core.runners.python.python_runner import PythonRunner +from app.dependencies import ( + list_dependencies as list_deps, + update_dependencies as update_deps +) +from app.logger import get_logger +from app.models import ( + success_response, + error_response, + RunCodeResponse, + ListDependenciesResponse, + UpdateDependenciesResponse, + Dependency, + RunnerOptions +) + + +async def run_python_code(code: str, preload: str, options: RunnerOptions): + """Execute Python code in sandbox + + Args: + options: + code: Base64 encoded encrypted code + preload: Preload code + + Returns: + API response with execution result + """ + logger = get_logger() + + try: + runner = PythonRunner() + result = await runner.run(code, options, preload) + if result.exit_code == -signal.SIGSYS: + return error_response(31, "sandbox security policy violation") + + if result.stderr and result.exit_code != 0: + return error_response(500, result.stderr) + + return success_response(RunCodeResponse( + stdout=result.stdout, + stderr=result.stderr + )) + + except Exception as e: + logger.error(f"Python execution failed: {e}", exc_info=True) + return error_response(-500, str(e)) + + +async def list_python_dependencies(): + """List installed Python dependencies + + Returns: + API response with dependency list + """ + try: + deps = await list_deps("python") + dependencies = [ + Dependency(name=dep["name"], version=dep["version"]) + for dep in deps + ] + return success_response(ListDependenciesResponse(dependencies=dependencies)) + except Exception as e: + return error_response(500, str(e)) + + +async def update_python_dependencies(): + """Update Python dependencies + + Returns: + API response with update result + """ + try: + await update_deps() + return success_response(UpdateDependenciesResponse(success=True)) + except Exception as e: + return error_response(500, str(e)) diff --git a/sandbox/config.yaml b/sandbox/config.yaml new file mode 100644 index 00000000..d9581b34 --- /dev/null +++ b/sandbox/config.yaml @@ -0,0 +1,20 @@ +app: + port: 8194 + debug: true + key: redbear-sandbox + +max_workers: 4 +max_requests: 50 +worker_timeout: 30 +python_path: /usr/local/bin/python +nodejs_path: /usr/local/bin/node +enable_network: true +enable_preload: false +python_deps_update_interval: 30m + +allowed_syscalls: [] + +proxy: + socks5: '' + http: '' + https: '' diff --git a/sandbox/dependencies/python-requirements.txt b/sandbox/dependencies/python-requirements.txt new file mode 100644 index 00000000..1c3c2901 --- /dev/null +++ b/sandbox/dependencies/python-requirements.txt @@ -0,0 +1,4 @@ +requests==2.31.0 +# numpy==1.26.0 +# pandas==2.0.0 +jinja2==3.1.2 \ No newline at end of file diff --git a/sandbox/lib/seccomp_nodejs/Cargo.lock b/sandbox/lib/seccomp_nodejs/Cargo.lock new file mode 100644 index 00000000..b37698ee --- /dev/null +++ b/sandbox/lib/seccomp_nodejs/Cargo.lock @@ -0,0 +1,7 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "seccomp_nodejs" +version = "0.1.0" diff --git a/sandbox/lib/seccomp_nodejs/Cargo.toml b/sandbox/lib/seccomp_nodejs/Cargo.toml new file mode 100644 index 00000000..a8bd8932 --- /dev/null +++ b/sandbox/lib/seccomp_nodejs/Cargo.toml @@ -0,0 +1,6 @@ +[package] +name = "seccomp_nodejs" +version = "0.1.0" +edition = "2024" + +[dependencies] \ No newline at end of file diff --git a/sandbox/lib/seccomp_nodejs/src/lib.rs b/sandbox/lib/seccomp_nodejs/src/lib.rs new file mode 100644 index 00000000..e69de29b diff --git a/sandbox/lib/seccomp_python/Cargo.lock b/sandbox/lib/seccomp_python/Cargo.lock new file mode 100644 index 00000000..881ad177 --- /dev/null +++ b/sandbox/lib/seccomp_python/Cargo.lock @@ -0,0 +1,23 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "libc" +version = "0.2.180" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bcc35a38544a891a5f7c865aca548a982ccb3b8650a5b06d0fd33a10283c56fc" + +[[package]] +name = "libseccomp-sys" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "60276e2d41bbb68b323e566047a1bfbf952050b157d8b5cdc74c07c1bf4ca3b6" + +[[package]] +name = "seccomp_python" +version = "0.1.0" +dependencies = [ + "libc", + "libseccomp-sys", +] diff --git a/sandbox/lib/seccomp_python/Cargo.toml b/sandbox/lib/seccomp_python/Cargo.toml new file mode 100644 index 00000000..07037172 --- /dev/null +++ b/sandbox/lib/seccomp_python/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "seccomp_python" +version = "0.1.0" +edition = "2024" + +[lib] +name = "python" +crate-type = ["cdylib"] + +[dependencies] +libc = "0.2.180" +libseccomp-sys = "0.3.0" diff --git a/sandbox/lib/seccomp_python/src/lib.rs b/sandbox/lib/seccomp_python/src/lib.rs new file mode 100644 index 00000000..08b46c54 --- /dev/null +++ b/sandbox/lib/seccomp_python/src/lib.rs @@ -0,0 +1,195 @@ +mod syscalls; + +use crate::syscalls::*; +use libc::{chdir, chroot, gid_t, uid_t, c_int}; +use libseccomp_sys::*; +use std::env; +use std::ffi::CString; +use std::str::FromStr; + + +/* + * get_allowed_syscalls - retrieve allowed syscalls for the sandbox + * @enable_network: enable network-related syscalls if non-zero + * + * Syscall selection order: + * 1. ALLOWED_SYSCALLS environment variable + * 2. Built-in default allowlist + * 3. Optional network syscall extension + * + * Returns: + * (allowed_syscalls, allowed_not_kill_syscalls) + * allowed_syscalls: syscalls fully allowed + * allowed_not_kill_syscalls: syscalls returning EPERM + */ +pub fn get_allowed_syscalls(enable_network: bool) -> (Vec, Vec) { + let mut allowed_syscalls = Vec::new(); + let mut allowed_not_kill_syscalls = Vec::new(); + + /* Syscalls that return error instead of killing */ + allowed_not_kill_syscalls.extend(ALLOW_ERROR_SYSCALLS); + + /* Load from environment variable ALLOWED_SYSCALLS */ + if let Ok(env_val) = env::var("ALLOWED_SYSCALLS") { + if !env_val.is_empty() { + for s in env_val.split(',') { + if let Ok(sc) = i32::from_str(s) { + allowed_syscalls.push(sc); + } + } + } + } + + /* Fallback to default syscalls if env not set */ + if allowed_syscalls.is_empty() { + allowed_syscalls.extend(ALLOW_SYSCALLS); + if enable_network { + allowed_syscalls.extend(ALLOW_NETWORK_SYSCALLS); + } + } + + (allowed_syscalls, allowed_not_kill_syscalls) +} + +/* + * setup_root - setup restricted filesystem root + * + * Perform chroot(".") and change working directory to "/". + * + * Return: + * 0 on success + * negative error code on failure + */ +fn setup_root() -> Result<(), c_int> { + let root = CString::new(".").unwrap(); + if unsafe { chroot(root.as_ptr()) } != 0 { + return Err(-1); + } + + let root_dir = CString::new("/").unwrap(); + if unsafe { chdir(root_dir.as_ptr()) } != 0 { + return Err(-2); + } + + Ok(()) +} + +/* + * set_no_new_privs - enable PR_SET_NO_NEW_PRIVS + * + * Prevent privilege escalation via execve. + * + * Return: + * 0 on success + * negative error code on failure + */ +fn set_no_new_privs() -> Result<(), c_int> { + if unsafe { libc::prctl(libc::PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) } != 0 { + return Err(-3); + } + Ok(()) +} + +/* + * drop_privileges - drop process privileges + * @uid: target user ID + * @gid: target group ID + * + * Permanently reduce process privileges. + * + * Return: + * 0 on success + * negative error code on failure + */ +fn drop_privileges(uid: uid_t, gid: gid_t) -> Result<(), c_int> { + if unsafe { libc::setgid(gid) } != 0 { + return Err(-4); + } + if unsafe { libc::setuid(uid) } != 0 { + return Err(-5); + } + Ok(()) +} + +/* + * install_seccomp - install seccomp filter + * @enable_network: enable network-related syscalls if non-zero + * + * Default action is SCMP_ACT_KILL_PROCESS. + * Allowed syscalls are explicitly whitelisted. + * + * Return: + * 0 on success + * negative error code on failure + */ +fn install_seccomp(enable_network: bool) -> Result<(), c_int> { + unsafe { + let ctx = seccomp_init(SCMP_ACT_KILL_PROCESS); + if ctx.is_null() { + return Err(-6); /* failed to init seccomp context */ + } + + let (allowed_syscalls, allowed_not_kill_syscalls) = get_allowed_syscalls(enable_network); + + /* add fully allowed syscalls */ + for &sc in &allowed_syscalls { + if seccomp_rule_add(ctx, SCMP_ACT_ALLOW, sc, 0) != 0 { + seccomp_release(ctx); + return Err(-7); + } + } + + /* add syscalls returning EPERM */ + for &sc in &allowed_not_kill_syscalls { + if seccomp_rule_add(ctx, SCMP_ACT_ERRNO(libc::EPERM as u16), sc, 0) != 0 { + seccomp_release(ctx); + return Err(-8); + } + } + + if seccomp_load(ctx) != 0 { + seccomp_release(ctx); + return Err(-9); + } + + seccomp_release(ctx); + Ok(()) + } +} + +/* + * init_seccomp - initialize seccomp sandbox + * @uid: target user ID + * @gid: target group ID + * @enable_network: enable network syscalls if non-zero + * + * Initialize the sandbox and apply privilege restrictions + * in the following order: + * 1. setup_root() + * 2. set_no_new_privs() + * 3. drop_privileges() + * 4. install_seccomp() + * + * This function must be called before executing any untrusted code. + * It is not thread-safe and must be invoked once per process. + * + * Return: + * 0 on success + * negative error code on failure + */ +#[unsafe(no_mangle)] +pub unsafe extern "C" fn init_seccomp(uid: uid_t, gid: gid_t, enable_network: i32) -> c_int { + if let Err(code) = setup_root() { + return code; + } + if let Err(code) = set_no_new_privs() { + return code; + } + if let Err(code) = drop_privileges(uid, gid) { + return code; + } + match install_seccomp(enable_network != 0) { + Ok(_) => 0, + Err(code) => code, + } +} diff --git a/sandbox/lib/seccomp_python/src/syscalls.rs b/sandbox/lib/seccomp_python/src/syscalls.rs new file mode 100644 index 00000000..961fffac --- /dev/null +++ b/sandbox/lib/seccomp_python/src/syscalls.rs @@ -0,0 +1,85 @@ +// src/syscalls.rs + +pub static ALLOW_SYSCALLS: &[i32] = &[ + // file io + libc::SYS_read as i32, + libc::SYS_write as i32, + libc::SYS_openat as i32, + libc::SYS_close as i32, + libc::SYS_newfstatat as i32, + libc::SYS_ioctl as i32, + libc::SYS_lseek as i32, + libc::SYS_getdents64 as i32, + libc::SYS_fstat as i32, + + // thread + libc::SYS_futex as i32, + + // memory + libc::SYS_mmap as i32, + libc::SYS_brk as i32, + libc::SYS_mprotect as i32, + libc::SYS_munmap as i32, + libc::SYS_rt_sigreturn as i32, + libc::SYS_mremap as i32, + + // user / group + libc::SYS_setuid as i32, + libc::SYS_setgid as i32, + libc::SYS_getuid as i32, + + // process + libc::SYS_getpid as i32, + libc::SYS_getppid as i32, + libc::SYS_gettid as i32, + libc::SYS_exit as i32, + libc::SYS_exit_group as i32, + libc::SYS_tgkill as i32, + libc::SYS_rt_sigaction as i32, + libc::SYS_sched_yield as i32, + libc::SYS_set_robust_list as i32, + libc::SYS_get_robust_list as i32, + libc::SYS_rseq as i32, + + // time + libc::SYS_clock_gettime as i32, + libc::SYS_gettimeofday as i32, + libc::SYS_nanosleep as i32, + libc::SYS_epoll_create1 as i32, + libc::SYS_epoll_ctl as i32, + libc::SYS_clock_nanosleep as i32, + libc::SYS_pselect6 as i32, + libc::SYS_rt_sigprocmask as i32, + libc::SYS_sigaltstack as i32, + libc::SYS_getrandom as i32, + +]; + +pub static ALLOW_ERROR_SYSCALLS: &[i32] = &[ + libc::SYS_clone as i32, + libc::SYS_mkdirat as i32, + libc::SYS_mkdir as i32, +]; + +pub static ALLOW_NETWORK_SYSCALLS: &[i32] = &[ + libc::SYS_socket as i32, + libc::SYS_connect as i32, + libc::SYS_bind as i32, + libc::SYS_listen as i32, + libc::SYS_accept as i32, + libc::SYS_sendto as i32, + libc::SYS_recvfrom as i32, + libc::SYS_getsockname as i32, + libc::SYS_recvmsg as i32, + libc::SYS_getpeername as i32, + libc::SYS_setsockopt as i32, + libc::SYS_ppoll as i32, + libc::SYS_uname as i32, + libc::SYS_sendmsg as i32, + libc::SYS_sendmmsg as i32, + libc::SYS_getsockopt as i32, + libc::SYS_fcntl as i32, + libc::SYS_fstatfs as i32, + libc::SYS_poll as i32, + libc::SYS_epoll_pwait as i32, +]; diff --git a/sandbox/main.py b/sandbox/main.py new file mode 100644 index 00000000..fc417563 --- /dev/null +++ b/sandbox/main.py @@ -0,0 +1,97 @@ +""" +Redbear Sandbox - Main Entry Point +""" +import asyncio +import os +import sys +from contextlib import asynccontextmanager + +import uvicorn +from fastapi import FastAPI + +from app.config import get_config +from app.controllers import manager_router +from app.dependencies import setup_dependencies, update_dependencies_periodically +from app.logger import setup_logger, get_logger + +logger = get_logger() + + +@asynccontextmanager +async def lifespan(app: FastAPI): + """Application lifespan manager""" + logger = get_logger() + + # Startup + logger.info("Starting RedBear Sandbox...") + + # Setup dependencies in background + asyncio.create_task(setup_dependencies()) + + # Start periodic dependency updates + config = get_config() + if config.python_deps_update_interval: + asyncio.create_task(update_dependencies_periodically()) + + yield + + # Shutdown + logger.info("Shutting down Redbear Sandbox...") + + +def create_app() -> FastAPI: + """Create FastAPI application""" + config = get_config() + + app = FastAPI( + title="Sandbox", + description="Secure code execution sandbox", + version="2.0.0", + lifespan=lifespan, + debug=config.app.debug + ) + + app.include_router(manager_router) + + return app + + +def check_root_privileges(): + """Check if running with root privileges""" + if os.geteuid() != 0: + logger.info("Error: Sandbox must be run as root for security features (chroot, setuid)") + sys.exit(1) + + +def main(): + """Main entry point""" + # Check root privileges + check_root_privileges() + + # Setup logging + setup_logger() + + config = get_config() + logger = get_logger() + + logger.info(f"Starting server on port {config.app.port}") + logger.info(f"Debug mode: {config.app.debug}") + logger.info(f"Max workers: {config.max_workers}") + logger.info(f"Max requests: {config.max_requests}") + logger.info(f"Network enabled: {config.enable_network}") + + # Create app + app = create_app() + + # Run server + uvicorn.run( + app, + host="0.0.0.0", + port=config.app.port, + log_level="debug" if config.app.debug else "info", + access_log=config.app.debug + ) + + +if __name__ == "__main__": + main() diff --git a/sandbox/requirements.txt b/sandbox/requirements.txt new file mode 100644 index 00000000..0c91018a --- /dev/null +++ b/sandbox/requirements.txt @@ -0,0 +1,20 @@ +# Web Framework +fastapi==0.115.0 +uvicorn[standard]==0.32.0 +pydantic==2.9.0 +pydantic-settings==2.5.0 + +# Configuration +PyYAML==6.0.2 + +# Security +pyseccomp==0.1.2 + + +# Async & Concurrency +aiofiles==24.1.0 + +# Testing +pytest==8.3.0 +pytest-asyncio==0.24.0 +httpx==0.27.0 diff --git a/sandbox/script/env.sh b/sandbox/script/env.sh new file mode 100644 index 00000000..f44f7208 --- /dev/null +++ b/sandbox/script/env.sh @@ -0,0 +1,53 @@ +#!/bin/bash + +# Check if the correct number of arguments are provided +if [ "$#" -ne 2 ]; then + echo "Usage: $0 " + exit 1 +fi + +src="$1" +dest="$2" + +# Function to copy and link files +copy_and_link() { + local src_file="$1" + local dest_file="$2" + + if [ -L "$src_file" ]; then + # If src_file is a symbolic link, copy it without changing permissions + cp -P "$src_file" "$dest_file" + elif [ -b "$src_file" ] || [ -c "$src_file" ]; then + # If src_file is a device file, copy it and change permissions + cp "$src_file" "$dest_file" + chmod 444 "$dest_file" + else + # Otherwise, create a hard link and change the permissions to read-only + ln -f "$src_file" "$dest_file" 2>/dev/null || { cp "$src_file" "$dest_file" && chmod 444 "$dest_file"; } + fi +} + +# Check if src is a file or directory +if [ -f "$src" ]; then + # src is a file, create hard link directly in dest + mkdir -p "$(dirname "$dest/$src")" + copy_and_link "$src" "$dest/$src" +elif [ -d "$src" ]; then + # src is a directory, process as before + mkdir -p "$dest/$src" + + # Find all files in the source directory + find "$src" -type f,l | while read -r file; do + # Get the relative path of the file + rel_path="${file#$src/}" + # Get the directory of the relative path + rel_dir=$(dirname "$rel_path") + # Create the same directory structure in the destination + mkdir -p "$dest/$src/$rel_dir" + # Copy and link the file + copy_and_link "$file" "$dest/$src/$rel_path" + done +else + echo "Error: $src is neither a file nor a directory" + exit 1 +fi diff --git a/simple_mcp_server.py b/simple_mcp_server.py deleted file mode 100644 index fa299e37..00000000 --- a/simple_mcp_server.py +++ /dev/null @@ -1,130 +0,0 @@ -#!/usr/bin/env python3 -"""简化的MCP服务器 - 用于测试MCP工具集成""" - -from fastapi import FastAPI, HTTPException -from pydantic import BaseModel -from typing import Dict, Any, List -import uvicorn - -app = FastAPI(title="Simple MCP Server", version="1.0.0") - -class MCPRequest(BaseModel): - jsonrpc: str = "2.0" - id: str - method: str - params: Dict[str, Any] = {} - -class MCPResponse(BaseModel): - jsonrpc: str = "2.0" - id: str - result: Any = None - error: Dict[str, Any] = None - -# 可用工具定义 -TOOLS = [ - { - "name": "calculator", - "description": "简单计算器", - "inputSchema": { - "type": "object", - "properties": { - "expression": {"type": "string", "description": "数学表达式"} - }, - "required": ["expression"] - } - }, - { - "name": "echo", - "description": "回显工具", - "inputSchema": { - "type": "object", - "properties": { - "message": {"type": "string", "description": "要回显的消息"} - }, - "required": ["message"] - } - } -] - -@app.get("/") -async def root(): - return {"name": "Simple MCP Server", "version": "1.0.0"} - -@app.get("/health") -async def health(): - return {"status": "healthy", "tools": len(TOOLS)} - -@app.post("/mcp") -async def mcp_handler(request: MCPRequest): - """处理MCP请求""" - try: - if request.method == "initialize": - return MCPResponse( - id=request.id, - result={ - "protocolVersion": "2024-11-05", - "capabilities": {"tools": {"listChanged": True}}, - "serverInfo": {"name": "Simple MCP Server", "version": "1.0.0"} - } - ) - - elif request.method == "tools/list": - return MCPResponse( - id=request.id, - result={"tools": TOOLS} - ) - - elif request.method == "tools/call": - tool_name = request.params.get("name") - arguments = request.params.get("arguments", {}) - - if tool_name == "calculator": - try: - expression = arguments.get("expression", "") - result = eval(expression) # 注意：生产环境不要用eval - return MCPResponse( - id=request.id, - result={"content": [{"type": "text", "text": f"结果: {result}"}]} - ) - except Exception as e: - return MCPResponse( - id=request.id, - error={"code": -1, "message": f"计算错误: {str(e)}"} - ) - - elif tool_name == "echo": - message = arguments.get("message", "") - return MCPResponse( - id=request.id, - result={"content": [{"type": "text", "text": f"Echo: {message}"}]} - ) - - else: - return MCPResponse( - id=request.id, - error={"code": -1, "message": f"未知工具: {tool_name}"} - ) - - elif request.method == "ping": - return MCPResponse( - id=request.id, - result={"status": "pong"} - ) - - else: - return MCPResponse( - id=request.id, - error={"code": -1, "message": f"未知方法: {request.method}"} - ) - - except Exception as e: - return MCPResponse( - id=request.id, - error={"code": -1, "message": str(e)} - ) - -if __name__ == "__main__": - print("启动简化MCP服务器...") - print("访问 http://localhost:8002 查看服务状态") - print("MCP端点: http://localhost:8002/mcp") - uvicorn.run(app, host="0.0.0.0", port=8002) \ No newline at end of file diff --git a/web/src/api/application.ts b/web/src/api/application.ts index 69d27d44..1f20282e 100644 --- a/web/src/api/application.ts +++ b/web/src/api/application.ts @@ -108,4 +108,8 @@ export const getShareToken = (share_token: string, user_id: string) => { // 复制应用 export const copyApplication = (app_id: string, new_name: string) => { return request.post(`/apps/${app_id}/copy?new_name=${new_name}`) -} \ No newline at end of file +} +// 数据统计 +export const getAppStatistics = (app_id: string, data: { start_date: number; end_date: number; }) => { + return request.get(`/apps/${app_id}/statistics`, data) +} diff --git a/web/src/api/fileStorage.ts b/web/src/api/fileStorage.ts new file mode 100644 index 00000000..e7b476a3 --- /dev/null +++ b/web/src/api/fileStorage.ts @@ -0,0 +1,25 @@ +import { request, API_PREFIX } from '@/utils/request' + +// Upload file，file storage has expiration period +export const fileUploadUrl = `${API_PREFIX}/storage/files` +export const fileUpload = (formData?: unknown) => { + return request.uploadFile('/storage/files', formData) +} + +// Get file access URL (no token required) +export const getFileUrl = (file_id: string) => `/storage/files/${file_id}/url` +export const getFileLink = (fileId: string, data: { permanent?: boolean } = { permanent: true }) => { + return request.get(getFileUrl(fileId), data) +} + +// Get file internally +export const getInternalFileUrl = (file_id: string) => `/storage/files/${file_id}` +export const getInternalFile = (fileId: string) => { + return request.get(getInternalFileUrl(fileId)) +} + +// Delete file +export const deleteFileUrl = (file_id: string) => `/storage/files/${file_id}` +export const deleteFile = (fileId: string) => { + return request.delete(deleteFileUrl(fileId)) +} diff --git a/web/src/api/knowledgeBase.ts b/web/src/api/knowledgeBase.ts index 5f171a72..38a0d40d 100644 --- a/web/src/api/knowledgeBase.ts +++ b/web/src/api/knowledgeBase.ts @@ -65,7 +65,7 @@ export const getModelTypeList = async () => { }; // 获取模型列表 export const getModelList = async (pageInfo: PageRequest) => { - const response = await request.get(`${apiPrefix}/models`, pageInfo); + const response = await request.get(`${apiPrefix}/models`, { ...pageInfo, is_active: true }); return response as any; }; //获取模型提供者 diff --git a/web/src/api/memory.ts b/web/src/api/memory.ts index bbd9f6b0..ff8e0435 100644 --- a/web/src/api/memory.ts +++ b/web/src/api/memory.ts @@ -116,20 +116,20 @@ export const getRagContent = (end_user_id: string) => { return request.get(`/dashboard/rag_content`, { end_user_id, limit: 20 }) } // Emotion distribution analysis -export const getWordCloud = (group_id: string) => { - return request.post(`/memory/emotion-memory/wordcloud`, { group_id, limit: 20 }) +export const getWordCloud = (end_user_id: string) => { + return request.post(`/memory/emotion-memory/wordcloud`, { end_user_id, limit: 20 }) } // High-frequency emotion keywords -export const getEmotionTags = (group_id: string) => { - return request.post(`/memory/emotion-memory/tags`, { group_id, limit: 20 }) +export const getEmotionTags = (end_user_id: string) => { + return request.post(`/memory/emotion-memory/tags`, { end_user_id, limit: 20 }) } // Emotion health index -export const getEmotionHealth = (group_id: string) => { - return request.post(`/memory/emotion-memory/health`, { group_id, limit: 20 }) +export const getEmotionHealth = (end_user_id: string) => { + return request.post(`/memory/emotion-memory/health`, { end_user_id }) } // Personalized suggestions -export const getEmotionSuggestions = (group_id: string) => { - return request.post(`/memory/emotion-memory/suggestions`, { group_id, limit: 20 }) +export const getEmotionSuggestions = (end_user_id: string) => { + return request.post(`/memory/emotion-memory/suggestions`, { end_user_id }) } export const generateSuggestions = (end_user_id: string) => { return request.post(`/memory/emotion-memory/generate_suggestions`, { end_user_id }) @@ -138,8 +138,8 @@ export const analyticsRefresh = (end_user_id: string) => { return request.post('/memory-storage/analytics/generate_cache', { end_user_id }) } // Forgetting stats -export const getForgetStats = (group_id: string) => { - return request.get(`/memory/forget-memory/stats`, { group_id }) +export const getForgetStats = (end_user_id: string) => { + return request.get(`/memory/forget-memory/stats`, { end_user_id }) } // Implicit Memory - Preferences export const getImplicitPreferences = (end_user_id: string) => { @@ -165,20 +165,20 @@ export const getShortTerm = (end_user_id: string) => { return request.get(`/memory/short/short_term`, { end_user_id }) } // Perceptual Memory - Visual memory -export const getPerceptualLastVisual = (end_user: string) => { - return request.get(`/memory/perceptual/${end_user}/last_visual`) +export const getPerceptualLastVisual = (end_user_id: string) => { + return request.get(`/memory/perceptual/${end_user_id}/last_visual`) } // Perceptual Memory - Audio memory -export const getPerceptualLastListen = (end_user: string) => { - return request.get(`/memory/perceptual/${end_user}/last_listen`) +export const getPerceptualLastListen = (end_user_id: string) => { + return request.get(`/memory/perceptual/${end_user_id}/last_listen`) } // Perceptual Memory - Text memory -export const getPerceptualLastText = (end_user: string) => { - return request.get(`/memory/perceptual/${end_user}/last_text`) +export const getPerceptualLastText = (end_user_id: string) => { + return request.get(`/memory/perceptual/${end_user_id}/last_text`) } // Perceptual Memory - Perceptual memory timeline -export const getPerceptualTimeline = (end_user: string) => { - return request.get(`/memory/perceptual/${end_user}/timeline`) +export const getPerceptualTimeline = (end_user_id: string) => { + return request.get(`/memory/perceptual/${end_user_id}/timeline`) } // Episodic Memory - Overview export const getEpisodicOverview = (data: { end_user_id: string; time_range: string; episodic_type: string; } ) => { @@ -201,14 +201,14 @@ export const getExplicitMemory = (end_user_id: string) => { export const getExplicitMemoryDetails = (data: { end_user_id: string, memory_id: string; }) => { return request.post(`/memory/explicit-memory/details`, data) } -export const getConversations = (end_user: string) => { - return request.get(`/memory/work/${end_user}/conversations`) +export const getConversations = (end_user_id: string) => { + return request.get(`/memory/work/${end_user_id}/conversations`) } -export const getConversationMessages = (end_user: string, conversation_id: string) => { - return request.get(`/memory/work/${end_user}/messages`, { conversation_id }) +export const getConversationMessages = (end_user_id: string, conversation_id: string) => { + return request.get(`/memory/work/${end_user_id}/messages`, { conversation_id }) } -export const getConversationDetail = (end_user: string, conversation_id: string) => { - return request.get(`/memory/work/${end_user}/detail`, { conversation_id }) +export const getConversationDetail = (end_user_id: string, conversation_id: string) => { + return request.get(`/memory/work/${end_user_id}/detail`, { conversation_id }) } export const forgetTrigger = (data: { max_merge_batch_size: number; min_days_since_access: number; end_user_id: string;}) => { return request.post(`/memory/forget-memory/trigger`, data) diff --git a/web/src/api/models.ts b/web/src/api/models.ts index 20fdf91a..e5d0f339 100644 --- a/web/src/api/models.ts +++ b/web/src/api/models.ts @@ -1,23 +1,68 @@ import { request } from '@/utils/request' -import type { ModelFormData } from '@/views/ModelManagement/types' +import type { MultiKeyForm, Query, KeyConfigModalForm, CompositeModelForm, CustomModelForm } from '@/views/ModelManagement/types' -// 模型列表 +// Model list export const getModelListUrl = '/models' -export const getModelList = (data: { type: string; pagesize: number; page: number; }) => { +export const getModelList = (data: Query) => { return request.get(getModelListUrl, data) } -// 创建模型 -export const addModel = (data: ModelFormData) => { - return request.post('/models', data) -} -// 更新模型 -export const updateModel = (apiKeyId: string, data: ModelFormData) => { - return request.put(`/models/apikeys/${apiKeyId}`, data) -} -// 模型类型列表 +// Model type list export const modelTypeUrl = '/models/type' -// 模型供应商列表 +// Model provider list export const modelProviderUrl = '/models/provider' export const getModelProviderList = () => { return request.get(modelProviderUrl) +} +// New model list +export const getModelNewListUrl = '/models/new' +export const getModelNewList = (data: Query) => { + return request.get(getModelNewListUrl, data) +} +// Get model information +export const getModelInfo = (model_id: string) => { + return request.get(`/models/${model_id}`) +} +// Create composite model +export const addCompositeModel = (data: CompositeModelForm) => { + return request.post('/models/composite', data) +} +// Update composite model +export const updateCompositeModel = (model_id: string, data: CompositeModelForm) => { + return request.put(`/models/composite/${model_id}`, data) +} +// Delete composite model +export const deleteCompositeModel = (model_id: string) => { + return request.delete(`/models/composite/${model_id}`) +} +// Create API keys for all matching models by provider +export const updateProviderApiKeys = (data: KeyConfigModalForm) => { + return request.post('/models/provider/apikeys', data) +} +// Create model API key +export const addModelApiKey = (model_id: string, data: MultiKeyForm) => { + return request.post(`/models/${model_id}/apikeys`, data) +} +// Delete model API key +export const deleteModelApiKey = (api_key_id: string) => { + return request.delete(`/models/apikeys/${api_key_id}`) +} +// Update model status +export const updateModelStatus = (model_id: string, data: { is_active: boolean; }) => { + return request.put(`/models/${model_id}`, data) +} +// Model plaza list +export const getModelPlaza = (data: { search?: string; provider?: string; }) => { + return request.get('/models/model_plaza', data) +} +// Add model to plaza +export const addModelPlaza = (model_base_id: string) => { + return request.post(`/models/model_plaza/${model_base_id}/add`) +} +// Create custom model +export const addCustomModel = (data: CustomModelForm) => { + return request.post('/models/model_plaza', data) +} +// Update custom model +export const updateCustomModel = (model_base_id: string, data: CustomModelForm) => { + return request.put(`/models/model_plaza/${model_base_id}`, data) } \ No newline at end of file diff --git a/web/src/assets/images/empty/pageEmpty.png b/web/src/assets/images/empty/pageEmpty.png new file mode 100644 index 0000000000000000000000000000000000000000..f78cc42d0d5dc1cf149a1686fda2548a81fe5dec GIT binary patch literal 161041 zcmc$F^kvJwh58NVkN5NS8r3QX5^u2nk6ABvg=)?gl3a2GZRf z((#?X-k-nW`+V$&!B6*n&UMapUFSR(qo<=vMskY;4-bz_T}??J4-a?;4-ZfXCdB>c zcKS>c9?C$@>5wVe!bqhH|@0f7v^+Z~wo37G~of&?rBk=`qfvh@|EYE}F@m zeEy4k`si!R{OFZNFk-}6OS~fRS-{nnJfl*`7B*AV6B$S=^6%KEy!#z)n;+Y1>m7$@ ze1Be@69vS*Kk-{TsqW!@FZ%gqS(!#gAteKmIVAcxbGEMV!^~d%;042iTlY$lr|iv=48n#v*BI^PJM<^vx%->z5cl`j%>G_ocAiIY`=~kXw=o# z9-9dtg>1!e-#cW2SccI&T5gJwfmNB z|Hh@{1;>Qs+d8q8ecdY-`r%v^svBtsbWzcAO?~~|bBD{{q(aSZ#1rW-y*S?Axn1Ymc(Psdv*o)w!%)`O!AZiq)MYga`D_n6@gel|4Sp*$DSy&S zuLV|+VU$?inHJd|?{-QzIy({Q@SBPyx-hy_){V><8 z;jK2wx*6s?tt^G_&XH4}{d7v4*(AEf3d$cTG9;I}4R1L$D`_#uwXj6)E!E^^Hn?>g zCv(@qyx8Wv2$Frw^-=D=X=j;Zt~dg zru04DJ_$9?-)k($zurB+zHzwG_LV>C)jZteIAprf$-0=8eQGj2{`#SCw1Y^2c7kJ{ z+@h2D5=B5mEQZEChAskHX7={x zyKW8+mcIy8ApXnsCDBO=>!}RUkZY82-NzQs7r_T|W}=faId^|~%M^;U*&kPsXS&?l zA*V7DJKi$4xmq9$c-ZUY!ex^lv(o*$kN?}}%$6t@>$7)bpIW|q%x`&W3CVx;T7(RG zwy(iM89y^7U`{VWwb~8t&hov z)%ad~ySvp(*wuvtm3J@sj*e%0{I=;J1(T+#GkB4|?Uh8~`>)_^PyK!-QFH5D4vkkk zO861P{Fl$&ZPk0Ho)@}PCjz?-Zbj#;z7waGc`X?;mkZJZIiz8YcE; z93J{GP#?zttZ2Fb?_TR#4sTle(zl%u75go$dt%>-_PTc-z^Xq;YgO0rukfi_KdL22?%tMK0w@w0`Qa@I zy5);)uKx~I+rCVmqxAFAFq1WHQx?WJO;R}hZPC4-lWK1gNfB(>TKN(d1!HDbweEC9 zQcd9qST9O5uqfM8topP6nB3>!^-%-VAqn(XuY72-#?Wzo7`@#9k3R6U3(qwNW|#T2O30$t4Eu6_A~viYk&1p zDT{#0d-#EPJA%;xpbiGpG5oqq+7>O6u9>|B5^blWfxEBxvbxrX5{k~4yfafpw`E6);@Px-zM+h-lwVLDZ=1p+QI3@S zREjVvFdf(dgilK8j|ZV{Tx>1+NdUs(Hej^{zJ?|vqo2=TuMBiWQON(iY^07-={z*$`e6qK% z8%%c|0B&K!9aaU5yAK_L)Fhe#1}S5^D(f4H;8w^qPcA7V@m>9*&K~|L)iZa-cQ%Oi z{HR!d`aq=gU%#)gKAHBPolIbsIIa&_sMLU&*5G+aV(aT`(U9)ILz*Udk|$Sqbim4&uMX9I`=dq1 zc{08`EOXSTLy_pebkH+a@_G3)5joSnJ>T_9vO|ZN*{LDy#$rvYng-)Aak4G20bd9F zskUx-Vh~gQl(Dz}pnjn}7vSpuC1=XVN$C3@0SL#fA0umhYC?qPMuqgik@iq)2P7Uo z1k#1Y<6tnhPdjI>V`6~{R5{+<(#;7@AT?vIl!sGAkAT^zzCt6h2uP;8dHelBd4^A( z*Ti~63{5tWdKt?h&a$#cPqN0(dbW#iD&8-i0vpu_Jw-TXY$# z06!uiKEyVi08yS?{rO(eNA@ri;qsRexs{EQb2~Dw3rho8Ygzl}7gXJ^w!*`uYhrOcR$k;-TqVKjgcB#KoM8;ScCN84r5Q=b62`1G4%%hGIa zC%|?7n#e-gT*fS2a9&$e=vIsLw~BaP!WXYfZ1d^aK$MVhwsr)tS5VaUTZJi;VnE2=1mUnUBs2;S1|J2W z6$rv|O&F9^uG$A}@|Kq=g*%_m9^LvioY$4bBlA7YbkR$|%e2NE_D95-ij+Jhb5R8D zMubZ?_UR<{>@3yZ$7G$ec2#sEqQwxP?-B$0K+Q!7F(l$waeKfB>!NiFvqC@Hewaov zGt@gi9Lj{aX6NC=sfX6;YR}ty40BUfNq-&7=`Hrs>!Of7%^T+rof0o2uY#Q^)1T-k zF1JV=mhT3N(pyeANNX~vfK|Y6jk^$?`Sc9E?fI-G3?MopbdmegxB0s}FaB(Q`1nzO zOzwqu{&;=7x2TGo2?OJo#r{EktKmn9DZb2eJ57nO-Fm;Lidiq?F}I`N$8WLMF?Upa zf^yeL8BnM+ySaz`!ta1IbCz^5@f(`_?BZv~x9*2vKS4j~2{XMMmpx=$VuUu8m32;^ zc_f#0JlPwUXkH{s%syWFf{0Wq3Z>q;QJwxeg(@v`f>^(B9AQRW>S0? zZXuWwOUrBOl_CrXaQw7y`$g8p@+^)-)rNyw_ z{NJ`uo#iiFq&r)?qNMzez-&svt98Lq@~2^aUIZDmkU}cZSL@2?^7KZa4Rs zK^ZP{P4ls(+WS(rIo0oD z7*VqS7Kc{g^mumKA9YX1ty1=b35zj*BHzO_!uZI$d;w|eCq%gJNWUfHm1@4zIX!OA zwzp6{d^m4;L;qN^O*xCBzP^{h13pic%M8ddb{c-UJVRsFhV^Ssl+?yLFne?A5rLXF zVdN!*aVBlOS|A8hIGE7kE-BG&{;dzAI)ZaMg{cgXDC|1}ECd2yd%fb|J^d9&Oum6z zm3_GDDG+fekZ}ocg74#ETWyjgvEXo`=srLC@Yk>7^$i-Fh||IsTzxvv$(OY=Q9oKQ3qk z!IkIqhR(c&4_eDMLSzsqr9raJb!o>cxef>x{~=?q;C(|Vdk1g?M)+V1O;Qr}AVSF~ zW(4_R$9!%sNsdX%Cp_h3re`F}_3ha0xt+(hgjAM8*e4DK!8Z~iDW1i7j*Tf#43OmG z5#8QY493I*?ovA1i?+2{ni#XjbuN6*M^Ze>`nM7I_}Nm8RXQOxicA%FDq>yJC2y0K zP2{YW-vu^ ziNdYSYnsIBsNeP-8M?P0+sNLsWliT z+{yaQTp82J5EK+L;6BV?Y-*Jr{~TV)=S`M((Y@LVxHyeWyfN@fQI0xt5} zV=yxHBTM&;-|2+>s0^M5YJRyG&m4YYgipIePN;_U5np$;uzp$t61D>AL!tF5=<891 zNX?7!hD%u~$ElosKkU#DxiF}VArrfDba5r4S}<~Nj@Tt1`PV4~(k2^fe$Dw7Wcl!X znW0NQ^mMzte0Kv1gs}N!DhwqsLI~v)5fEzzNf4tEA|Ba}AdN53tqgxCGQ6&?Ho|+#IpuNf$lE>4!ur}ajUEW+3UZ;Q#r<8Zq1?K9&}PNQjU$~ zU7WHq9;&(O8yB33rRXU~}0%cIy5YY|ZVI1`u{MwiJ z$@hkLF@iuvQxign5AVWrf7On~g#YHm2~Ll-UuIH=Tbv#{m~hgmh$c7sL7lr(%r(cnK-71_m?%HN!spff$aomjg@(_%xpfbh z7K=?;B%>%QJT6b!$XVqF=d%(kPx^&lTmjUqBH`_LZ0Lx7If~Y#)!chsn(28RYtRJ8 zy{)A!JNlgjeHFo_1lg2n1@sWnR^jW$yx;Xz9Sg0(GFhuuf5^+Oqsw!wshT{7wwxYY zH~V%zu~$NHkchC~u_F1spzz{Kd1YNhBw83}KUTHCNeXR=9`7=DrVdJVgPu`lc!U}L z)3|)UO`q2M(%uzLq~O^rhi+Ubd=7|$MpkNqyNx;_7&cK%d;PoFUFx!ar(d6{93#$- z$<4kJILU;r7w6>UWX1Ku zNPCcxUc%YQi620Z=Z^osJ7UD-NamUKF9Zx*Br=SM@=0@VGXV&}Y*+{=1MfZ+VMe^U zBCO;+&NXJ#=2X@Rmw2k^uzPlQZq0c~dUTmD@i#6lG4%_I^mhC%ZmWs9 zwu%Nt_BSkcj*EI}+sw{~#j9vq(e7jhu=t^h^Ze30SL-VI$Ofp5Xw|D3rAUVp0#vA^ zWQ=A_SRgpPMEaxPC-AVQ116pFQM*?&Aj4Af>*;U&?r+7J! z_wSm(4DYfrbgq&0Zb^A@3(1|Bt{h5Lk7oxxf9~YslI-E|M&3$1XLQ-CS=htj1lT>NUIoRN{M#Pq*6-t6BwnvhNGdze6FY9|UY>fp9&hvcVHQLfJXNSDx z1?g*C6o3CnX*C7=H?iC&fqm6&oa{<&nU24B`HaBR(={m^(T)coq_Tpc-E$xi1Ug=b za-nhG4{c(cLGb!4TaXhAIg~PqdfyA|7ZFA>zj-d|hq|NHTi*CZYvmiy=CQULIe3-3LxUgpz z0>q%>2+;iI^$p8xsvIKL8Cuy8oS7JMS|Gc>MygeuSr`w^r54Ja<|^u*ODSFxiub>6 z@UAUp)=n817}&9+&t+x{$H?fnLj{pZ?U)831VlzDJ3A+1H2#sX}RlqnfD3$bJdaMQcD$e)HnyA-2c~% zz93&Sds6Ju9Se%?h{W0QCwbsYj)4)#$F%Ku#1IVLqGtSa5ChCT2MHrq+cKZ*S{S8D z47q-gFzq24Hjf%O1}#fD7)%l7_Y0mNMWX2t_GSbj zB**&Zdu-hEE_TGj>;TgQv4TwJYPl~?IywI7Pi43rxxXZ}Pvf8j$^cPu-7j;ubjYpH zi(=$jTM>P+$R^aXM?l_u%$3%6kpbZq=Wue7JakD2P}Pdb5Pj$kA_%=-6FXxS&$yJG z`E>=xhV~W66`=};s!SpOq1GM$>s@(M>RIm7hkKJR>%;B8SGB?%}QYA3VLI5lJ4zJN5HZFpPdnOjuR=`PJ`uK-Xhzn&MOK4PE3b z(b%DOk@O0aH_8j4{B`AUiV{+{{d(cP-Vd~Fyw`byBl?BD$F8sGUbHP(T zkzOZ?Kh@T6{Q2RQY5NqlBJ4W+v>F7asUNEE1u={fgTdA!7zEOz+?_~wX=%#ikTQ*q z{%Ip_lj7vMjcH5+Gl<5M*Ri~dh9)l(PdMo4C<@-atWlV)M%6#C!;tT^Woo`8({|{+Wp?}U%gv_z z@}NpsrY+0BBdc_1(LBGJWD4Houo667Z(tS}1UdnpNs+deQ{Z zn`^c?XRZXmeNngFc07ELO*+yh-5OOt)ZQ)FbxMWn$5@3(Hw-?IzF&cy=S{Y$`J;t) zbWnUB$537nr72Dl$#mR$y8 zefL!PAf6AynH^764-4R0*4oKlib;*%dRxxdc*f+5hN$=Vxal3%Ry3t^sGc0O-3+=` zwUQ^-yJTj%BJ$tRlX-BR#}~KlsqeJxD!AP9NZx;V0=SePB{RNUQpRV2528xl1uk~i#1ZuFRiyiZ2kC747JwS;7(cG0*M#+v7Lw9&=-$3F?m*$*Bg3 z8w&V6b`YXExv~Qj6Tv!dd;Gq*uPe>yY~sqUCm1HiegZp@d_h|fq<}P2&~AlCG_x){ z&VU^=4Jh6rnBWq1&5q}gkR?6Q^62yAlBo}~E96>agqNpSa}8OVtcG0oST61~9lNg| zaM6W3T2tvI)$~86=Cuzc(3i27-8zg(yb_O^@lOqQRjZh_efl)>$H&p3p^+My?s=d- zh%?l;-WLT%;%Qku=0rq6d2K!r{GW%pynR078+?J!$M;_TXsOh|Ro-`fA*uXJn4ouU zwOr9GtKe#O$P()f>&@xhiR@*$FP>|A=PkV~bO0gs7=zkl&Tc^f5RDuD#J>l^>>k_p zdQ*)_sr56#F?eYF!8uIyirs$SHK zq;-sQYB6Shaw^12%Vva9Qoey<3QRQzRg&3=PvYTIqC4!4Xqcla)lO%<1`zTXsx;;u zcrH4;_uIBOW0=hm%>*TkR(5Drkl*^&H$eVt%Wb}g+kYp5yM8mxnDV~ex83TR`tH_O z#H-~_lxfE(h}|5sxeNzhM|jM8B|I%0%zwNYbNk;Mq^&nf{&N1=sF>vqMey}%$myA= zV_UG~#_sLJ7cIOG#hlq!wbH5TrVgZWYm1Yh$*h^96<4=9YjA9&n{Rsu2nc~!Y69U1 zB%uQeL2YFP1CV4$jS)jT0jsxs^$5{WIGdCQMntWiRJ|~@{sgyMo2-i^SiC&*#3}c> z%v3Y%3M3#V?_&I9U$gDfEO2Lpet#*0F&P1w2jPcO!jsXN2;y`%M#Xv=s`@ZZoJe(( zfX@%Yh{C@Jqj@-}4p77sbQ8UlJEf}=dn{Yu|6lxqJgyRsaPCU_8rN_(2UQoXRka#LdW{|NVjEQNAc_0ay%ScWr zyZ?hH*YDt|pk*T#EyuTob(m8z`m+|O_63+2G|!G(diVFlnR!Vq0@FFrnsc4Y^`yQ@ zOjF7}Nmc!VRgOzKK7Ho(;Y}RGYHEzBfIb=q2)o2=a~@y11QJCG?WatbN7>r73j=lT z4@vVQBa~;3ze((E@A`391Z_Vzee~#EEWjG3ied|I$757Pg`HM+=OGaIOt$IB1YG3c zcAZRGKzl94@uYB&pQ2!Sk{hwQ8jOjlv39XzY-ZT?a>dmXHv`i5k>y@_szG~!{X9@{ zeb|>F0WA>SedIx&K@6P(i85JlQ86$m?$k38izo_WFyTuuH0gXsIDpESxYYI@X?F06 zEHnuuS&v7=r2g1(i%O2_i`GxQr#gAjtDr+|mW##N)#DUjw~LXQ}qAPL-J{{FOfo}W#snpN5LB>=EY&Z zp$eUGqAVFR@moj!hh{UK4e1s(Vl4Xw^7|>xHx<`LIy1pUUGo?QG6WI@zzwO`kg)q( zR81fTLauUQbhmft++*%`9CAE~q~l7Biqr@V*Xo7{>o z4M>vNksn~LXni>xd}*M@oOiaRn&I?OwB-4@_tzTd2R<(d!5a~@bSKO~4&J*-H=8~5 zxsv$NxAXsv#UD9Fjgik>Md{6%JV_VPQAg{N% zZEqel8L%Mj{*o9Jy{QFre_5{qMs#}vL0m`}8%PkD0rJ3LNE_NkaoK&`-=N}wU_-+( zVR-b(DV)7sfQpdmtH2e}y!ov-$C%2_SD8BE3__5S`t~VRBt<40THOj5UagsdW`kHq ze^gf?7et3Ip*X?^D2F~difQ)}sG+`NWuW=-cCc~}s4+3Xh&D#}yEv|of)Tvk7VlnD zk2_$?<0TA!w{}hpZS2gFc>sbyO7AW7Do+Bjrw~G0mz$!npJNDw6$8}oy(}zti3B^i2#XDqv)A$vO zeIXmgXOXUfG8f0J`D2@uSlki|fhebfA<(#&2B3;ybr?QlhBbr3f;0{&`SD9?3I4sZ z_lNFqriyGxr&2A2rz$X3aoEG~JE0!!5e>gNec@2b2xw^dqd_9Kbo!I6 z0Hh`e0&1;%0&Xh+38p=o+q^QxxMhR)wGL@CNFhH)tj5W@hx|ELvR?1f1W$sT8wKm- zZ%em5o`Cw@mD>#$)p@q9x2JqoU#9A>{kW@L++{VY&vVxRF#dyoyE(e&qs)$5{R
  • >m}U-p^$lczah`OzROG5j>b9J zvnpH~hN$f{TJNAxP|Ts-{Td{@`kf3b#oBKqbZR3&P6epKgLXK`E{udNBR|?skvD@z zA;rneXLsi7*Se|RZQtpy2L~dP;?6dRTi;j0EsDEJKHh1->qtex^7HenI0VOnzB8Erd9Vq2+i7#}y+OgqehKPX-9~|lpP{X4oQ`Z!Et0E0} zg|TnnPf1MXjGia)p5B`);PyYcn9(*dd2N|L=`0!KlbJ9|e5bv0C}^v3FUDtOHTVJ> zGxH}=_F%o!w$S14vvPa%?%3n+`~m{b7A7XYTX1bhyI6OZu`{M~*4o+mWTV2Q*DgGKi9eVlZ>A$%jDjFLgY{JqfUgkpdip1Q3uWg%X=0 zsADlP#JoxDRwO_|5zc&6Ad z?vkvtdWg!;#}KpW!9Z{*23eJZVASqs#|4a7BeMo(1*|HcB*p^OV3Y(apSH4UdK;1n zQ5B;EA!1&W;%u{7?Y1;Y%Iw+ia?caDF9HV)rI~-9GZoR0wO18Wo<~aqt7JrBdC_I( zuSU~$Egv&IZfkD7M1&B1ev|{LyLS^YyWg32J)Spty)kC-Wyo9xh8K1S#8hC&*r3oV zwv1$Nl72%V5J+$b8H%Lq56#5HDED{so4{$*4b_lHg%}VGfR+|OtBz3bp)pVrA?3|f ze@c2G*oe99+VZMtw>9eLD#}65oq4$7-6vEaT1)m-0w{=9LHQdLD{8IP z8=at!@2$f?{!r>3gzzhn^`=`jbJy#)?5v|KOJ#GjRxvs7j!}9Msi5ufbup&nLZ$1o zeDI!}s?I*cvyA`31PwUZnVt~)dOc!wro2k{e%9DDcP62)MP2=Etj+$qjL%m6^3p)c z>M6T7`ER1;we;Zm^k9bij8bg(QXCZ&3d9JJfE5rRHmJRZUTCMEznHsR)uo6^>l}#uYJM1Zvi>Hu)n%{`iBuB6Xcx44+Th3 zs<q7O{#*|fXGb)QfLMz&P~AR1)JeisB{XSR~ZznyI0^ts4PLC#`j zO&~8nr`^-?NIFed9(&~*obc1LZ%sVtZCQ*<-q%SAj}cDek*wVhckNHnDXX>cP#-^sHq$W&l zfDHxnytxB2IN;=EzYWn-g3{>egNyFmVdsUFX2kGtm1OAa^GCeTeZe_<{c*PQibA%3 zW02%3IPZyB?{;+X?gQzc4|^LJOH57877rz6`U(YW_Ks zcD>KOCmN2&@rjP4#0lIH4u+jh+L`l2C|H**G$Vn;`Gi3Be3EDqj9xye7lT6=V`B9H z@zou%u5CKOquYNmU;M7wR^&|yc0FB=wO3E<*- zn1`dRB(DyKP~)3<*6W#}ZJ9dxu3sUBcYl6IwbJDSftW)O$Sy~O^KW5<62ouv_ZyOM z!C+iZ&@=?1Xt-wRTc}!pIwvrcJs-^lep46y5M{n1@B!KiZlq>(e{#^D_y-aX00JryJfQZb z-$oLE3>2f)Xo-DF)`WS@(e*3|kdtOugP$*>CH~0&w<=)%p60Q@Mv5jRX_6}h-p#D^-7AMGneN(Ab z4Q3an#d+tA)muf43@aP1v~Df_w*;CUT3W)?AOad%N?O`ia6}Ic0j+@{R!KQoOPyNu z!2pe@aEeWGCJvOikftf}L~NVYdidDC%5u2^c6FfMl?|lUd@EI6xg!pxMML$BXuS=- z=VTSg)Mai@*5Tx3JE`9<(F&`Zo`CwLc0uyT&94G>r3MSQ(+6luVW`xI2o_G|FeE#P zH*;z-D%%*p!yf=r!aw?5=}K`W5Eq|w|5T$I{lDyO& z2c&*BNJKDwL@+@52T)It`=*)As(42+G{;sLXRd3lpF-iE?r zLP=s3lTEs{^bGacic;aC!uFDkd*(v^#A6`~DS-#y^WJDZDg%$&dHcOS>k+8wy7%!e zE z5_0XBxAJ417CLGrtCJ60l07pkyR|_O?Z4t{(e9ZS!WeRuNyJe8*{e+0kyUD7qk+~7 zm;|D=jw%ZQWNJbtRS~+CYH!Sp4HFo1TkEt`924#y#-03W@?^Dby-A>GT{V{g42-G$ zFOowWZhwTz;#XCcA{f?pbEk`9DW0FF3VEWY8z?^+vmXPcz7}+w0MdzoTgiU8jg)-zqXvr^oGR z$_Ts@`afnm~K-H%OY4PSX?$XST;Mz zrgxSK@-B}oXYk>cwq$?1rD&pzP|!Nxy^}X5ExQ~__nak}wq!id4Qq<3dXwb}bqt=Hp}R*{jrU?Agt z-T{rpWhnh#-)=B{*nBi^sNuvuNlQb4*G61MYGhd%o4w@Y#mL(`bMr)rhx=cI?ABN5 z#)CG*vc!e-ePv|#4}t=(-v!jK?x?yYqkb4gZC12}wKZOgSHamS(TaMMN!j$!j!548 zvgpRX&J`BpdqEPqE$39Vmdz=S_by1THx}y-=y%VU-oNMj;SHnxR#aXzNwy+An)l_x z*J*vW+HtI7(r0_-s5E$~xNfWYAgwDZ#IJuW)9N2(yRWyS@#(Yg1szyws}zhm&(_+1 z=d=7H*L>O|G+*|)pd^ieSwClXxV%Az99~1y@7VKq0BOviV&lb2f9IEt3Uh zp+MltbV%sch3}~0K?g-(nr+`Om4^jV=_QziBpU93=2xQ4HdX*p85%*owSSuD?+lHY zKVB-jIbGgrcFR?D35XtA;cVTD5pvrqC+Z*7SJkX|s#&~}GM&r1E)n?bVrK`}XGY_8 zuIbwjdaDC-NdE@vnk8t*G1y^#Er#0IuI;dJbh7!C)ux@leU;D7ltAJKGWiQp(m!EL zQu-NZ7YF8J78e^KXR*f33mf#!b7#|67v!T}(spz}OevlnuavHm_#I3(y%VlfKcS#p znK7}ZFaT*`A-e&O!ds_rb~z+f3e-3@0nt(#7%Cxz6~bxM)zp<0lobe)Y5L*XG&qvs zB+z(c_x5d!Vp=FC51j^It$;I64R!m{n?>$Z`Pt*Fsmlm6VNwFM(6S8!-(z9H$va*# zVD`{xa?dzVL+?-G3?rLAvEI*za&C*>B7cLk%?b!wD-ApqC}5hLpWjRJ&l-sW(ntYB zc~?nK;;vD#s&6YrPHbGoZ_V0>SJix2S5vqu^~5W-r6sP{2YCSktlNU|uCr~nljgfG z#$59j-QhSTeb(r@STv*RBpZU)9FCoc!?kMXgI#EnBrUA&^-6p7ox&|FrIP-_0pl^ycf2VpjR! zBO=kD?ONx_WACOG-fFkpOJ|SVXqW+zIC_u}kg1`^F90?{C`a(yiM+}HzQsvOFpg!2 zUw;s#hJpyxMF

    sc|6|8Avr013Y-3hg46&D)rFNYGdIBT1oJ@40Q)wd^veIg-J~d z?UFg4PJ))+P3|szY&)EkJ^I!K*Gq+$$Y|ybP}6N~_8EVu`s$jhlcnE`|j= zZ-bN5b>=IxT~@ML&tZIutARHElcrtkmG|%N-4zX~FDs)D)kT^7pc+gKjBo-&iSgAj zr66kPtLShL0i_0jQxgYRSYb>o53HYlK-%CJPc*E=Dl}Q-)t&r&J6zEZHu4kDYKtg? z6sQgKALt3EDBZycp=7R76Owls>aR1BGfOf;^+kjgQm}AsjVwF$(N0H~=7_qhsGIhE zYHmusryM04AN4Hz{i@m{!R*gIJR71;B_rX&7y>#B@hEYnp@1*~a2fR&LIFn1hLlRw z*smuUdAkpsGR(4z*L%WpEKWF#7Na)Q1E@#*=TgR6t{7m6o*g(%GuHjH`Hko5A-Ukt z%Uyr$ zeZ-M%9>mD4yTjL|Zj1~^ZF!gG`SJWP&k~waSe??)yFcEVxnC3@XWCnwC>MBS@P7OB z<~l_+vBmdUH+FVCXE)F>XzkmZRmYXhva%PoFZ`9TPv3f2puyr`_SFGGoGC7lDEk%n`K|KN#VH5h+hA!47DVacfLj54q%mt@vE+sdq zcpfS_-tHyPnqYw{8LAP0Kn7I`>psAz4?1FbdGpzy;Vf!+32|Xrp-mZiS>v0hyc_SV zpOy|UC{{l-+Anr4zCZ^TUk6T_2c8-Y2TxXZMX9%g zE!h@XKSjsj=QYj@-pW6{Y-LS2JJoZ-)XmA(1+gJcl7p6;CxYj{b9RFcMJ(^}f;cImnw~OLyZzT!uan(3hii;2yKnkS2DHthi!C}1 z{QAc)SnhO9$mqyiPp5P(&t9zKUf3ndIIFU}UbdV$H<15hAhhixGZs3GU3!a40`KcU zWdIPGB$OgV#g9@8rQ*@gC(y2fXJBF!lQVFr=7iY+(ZuNJeApcW++R3q3yqB>Apypk zswr?9h$WRK(~xGqeQQI)Q~D%16K-SHudi3BjRkSiLJbV{p$~}R28O>%Gm>L9a3zqb zE~D1z-QQ$QDa^h3N2v^g_#tKj8TZYu2*@usL2p-l)zw$ zU|6X1vy#wv9C4oJbZIN!?1Ee6w~nR9MowEvf!m#gBa)xIjZs#@DiMrh zaAJasZPgw}lxR3Rn(H;}PDy?Vs*srY9WLx8(HM302bJ&`6Q#e&NXGdt9vW%`CsJPC zLaVp=$+)JZbGr7e;A&i3qH?T0MnHthi3WYF+dJ?ebM-6JIy z4Tk>pP3*kVIfY}OJ0#)AZ$CzNe?k?qzU}_`UE`l1khPj7;c8w_zR=>Ry4W@|VJ6w= zY?k}tVsiH0QWg^D>kl)e+~!`B+LYcDTtzTf;C^&g?fU!IRflAO#gx4;h6FR>rdEru zQoQ3BxDvc^t>{&|TeQJH`rc-Ieopt-FP1aQbl0mz89&I?>htpT-IlF`=g&9tKGM>IY>VgO;_Cm*us%T*JFO{Ai_gh%MANHol!tTysL4n&3NHYdxGPL^;D z)YD8z6Y*)4%P&q;OuHSe5zQRToFtrb9b~%(P58`C( zXjZ54-3vkLVWE_-NQ0YBK6nQLQ4v((QPF9^s0Hi*{Fr1Cp4WE_hjsyovmSrCG`b-{{-GQH?_M+0wP!d_y*li4mNqnA z*L)_x(K;Z8D-l_TNXSKw`f63J;RUnSc~P0x^VI;)_{rGXns{zoc5c57mkm`H*wA!e zey(SY4^Ph5^wDGb^z-SSbbbK~uT_!j=h2H=dCXz+@(D-kBildhO`hm}9Hl8C z{>9m`?!+qFwIokgcHG4`y;ZJswaw|juE>)vWdHN%A<>7aKmU$WeU%O2bABs z@$|7UHRz}kKp7N~szf990LKwphWf&sIC0KjVyE#A9wT6tocV-}JH`~p-D(fim6fR= zN-#VH+yt11GeZPsm-B%Xj3mx4e^%xawdQqay^!o)tFOz=g-}2zwM*mGPLt(y`A6p0 z@!W@9we~*}20XIMOl@21B3yjY?q((L$)^b$z3+?NCDWIn^*js`7Z4oH+a2BXmH*+j zKoONP6Xm--asH&PsrJ0KuEn{t(RWX~6gOsE5q6)CFdiQBzG0jQQjvN0{O2uCPkw%9 zwn<_m-=rrg4ze~+wSndea8KgIKXq*l4Za5?DKiUmGz|7Lls%CRpt+5M30YC-~VoPmiRnk`axQ^IPXTb zujpbkiu~$2kyWayQ=-JU?fmEDZvFPJn~#$@0beJq>i-Z0&s{7q8k;MB_v+`rbOr)F za$(U(v=%}^nAE|Bi_7W_O$SgJF5Hufq=7#`s%L75*oP&DYS}Ati4^4*zJ)2`4qP}enNFgMp@+m_92-LZwa`TqS?-TR{vYk$~2 z+P!1aJGuJacm3{|nGFpyGc%8g)5Ejd7OT#8^kebhWIU=`2iDQYP74uq8;G`|9<6Y2 zd_PaDPSLFzY|-2!XZz>%k2a>>UAZM?Qi^M-{| zUD2J4IsL#r+_|=fPri!ndoCiXx?ZSym(!`=&{UnX|Mt+KcmLFpBLltxefG0A{(QgN zyt>zQTc@sh(TiWij-6W}lJf!#2m#q3s321b*@6icEJ*s@Hpezc=?6qhrTz z-~H4luikm*os$>8`sFQu+wsdTyX-9$lMl5c-@MT4T)XJ};&3=()V8$QG8~+u9iCzP z=56fUwVPcRY#;B~wd+F*TeiJ@b+Y`?PNzBjo4@v317`jU4Kp({kBRyI=Eb17Y&aY) zP-?o-qgNFwpcELdtx{)UlLPiwV>VYw-D$wWU^HY)JK+gc#TQ2B7}xXEHk<2a?rEi7 z+5w#6A6RER-V;$BbTaO@7OW(S6q70wt7KTopsMdav2Eu){2i`?_PV?8zW*mKyX>u# zw!MC3)V`@d;Cp(*vT5bmBi!4LiOE%LyYOP7=~Vqr^Gv#nZ)&4j`FAgW`G?;8-uI69 z#t=XH`ZrxMnw)v_$3OAcuN{v^+jO4&7w>1w=IsPi3Iv3JAR9!Gsf0{0(UxcnnT5hL`YGyyf231>oy%YaN^i|-qSwi4R2^8LjEi6yz|Zx@Y%~QyX?nYUH#QT=4-lh^WQ($ zG*7G}nr>Co>vibO&D~Tt@kfK<**`klF5I_c^96%nIsAVmGyks|W@cvo;az=oXY=v} zd&gz+gmGKd1dax4l&I*@Cprs+&@|GC2@Cxmu~t(EI@B!HUDk5ULK(87S>xgHX+}9& zYDZ_kb*<*<*IaYW-Pc`rUCTd^J6`_s-pKJzCv48iBv7IN29XK2kQI}JL1|aE9N4nL zf5ltwymJJ6{O6x^@qKpd&qrN;tiMt&S$p(k+<(hw5!iC!0V3+E+o@l$r9XdWxbxzZ z?>uzq)2};nr0_U(=+L2hI2`Vuo11&%i8BxWz(e=nf8pBNa^3B9Y1=WocJHR&@3BsT zAOKk>f=m@qNWlV`SXxR+Y+FhuWMWCuM1d@zBm#jZCvkf(WLB2nk^&LsOolaoL^Y{i-j$l-uI+E`)>NBx4w1wi@*4bB_iZY-uOno zO5SSGl`i^%D4ns2YRcV-+nfZtJ z=})(72Vc1yGO6k=o$fr*bXi$jrZcw*OG}k=WUq%+m(tt9+33->0~(-iy6mXuiA+}7 z|N7SZ9{IV;FaM3SOp5u>x&tpBo;nf7Zu>k< zQ?s!9Lh7oCChAvg>GzL4?AD{e1Ash^yx|RR=-heZjk_kJ!4H;6`=bvYKYrlssYk1| z)ittBtO})V=`PafFLA70wyXV znwX+uLRwJKBr{R4B${N3nPf@Q#FUaPwvt&eB}<}-3MDb6WJ@uzl~M}D#1u1GX9U4{ z$yY?MAtH!?pdx@kg1{OO1gxL}q6!Qo>Pc01O5N$4s_V{&<`=u~onP2`-I4da=lJ2n zhm*hI@ZrP!J;+Z(@`+D82t0V(U3c-Jk1_M#pkZca<}uM(sIqRCaxw|9X6u%13|3dj z(HhYSib>s{V@S=~+!mUJ3mCKw8q9UO^qLMA))kA9lrsZvU!9QcaBg*NaPhGRx{JUm zA>RH&Kh#^eceUwl8pUXN+0_FF+J)8C$?J|BDg2kJYYrc-xR<7GI-B=x>mR)LTwPxF zga32n?Psn$6Z?+8>@{y5eDu_PSMFFn_4FMb=Ewr$G1X*3^eTuTb3zprrAXUW3lGfC zF92uwD*Wngx1IZ$OLzbN(n7yUs~_tyymam4i8y}yja=02(A&0~x#%=>>Q_|@3%CFJ zo8I&rKmOLYuJAZ;_0?B*wr`(oz5lK|uI!Y`$p! zC!YM|C$BK`xX~~(GxOC7K>okQHP>8IGj-i{*R^ao9KHFwJM|>0bb= z?S#@6%mjj>R0I+A&e83U>;lg4Rr)t~9Xo#bX_vlZ5Y|a~V{eoPSB@Qv6P=sc{p2p) z?c15J>OIkPUNIVY!#fWh`pD~!94UMqyXKl}7DsF6UOpZy{PcLZeAUYGx$bBrmZ6Os(p;gG_|6vc#gMK~B z!@hs7`+k<+`tI*p-?L~v(m@Cb0fGPq5wqD8yh*X!ahAGHr=GYOXPP+EsVDD$Z8QBN zb~~MEW<1TbZkj<&ykcx@Y_Z5lXhRY@90{F$KkxoqpXa&n`?{(H3>Z5`5EBR9&qvij z!7O1Irl6*n8K|KOM#9v1nOefsu#`{?ONLn@CBrPC$xuy@6g5SYLR69|mJ~B3CB;-p zs+cBHN~ENus5Bu^r-sRR!eBCCV=&;{<*Ph@`Ye;_g!jJftsJ}Y2(6;TdqKn@P5_4! zL7c-mLDeJ9<2^VpI1!SF;iW>dY1S$?ebIV6%Zfkrt=1h7V6yssw1zv;IXHH#Tz4PXOos>>A`LD8OdSH0vqLl3pd&;}~7NPmAk81AX2Q({yE#pfB_?kr3D_S5Olv9+COchhZERm9-7SUv+m{3)cCNw6J8Yx9W2-LBmnpSKMhivQ& zIdka}&%SVqv*$11MCf$dT;JGWdobh!@BACQ?xq`PZcJ2DhZPSx9di$iZTaF$*`fF#; zp1sPe-0R-i7&TntCGaNCqJ!S9yeM}}}1f%qDa53Xu$M^CMKcfEl@AT!;`aLr< zbEI2tV=~jDdFqS!X@z%!i$)vhXKqfkyuTmQ{bjGmqdOB$Zd|6a4wD+17!aS~ixR;| zO`w`gh;>EPjJZ0gSdSI^z2jwBNLXx$DUlmU1m_c`@#e|2IkvjG`bVp)tIBJz`pk_jdhH_h;xheCmzM8%_5Ph{A-(IP_uv0-A9(P=#9t%#+;h+D zOqSieHXi(AjrK#s!Jr?S1`#3i8Qor={@fznnR&eTT)TRe7hiY*(@5?!oL}SAi(lrJ z8xGJZa#B(>B~%rQ3M4ccs){8gnPN#%Q!E(?!=hp+788nrf~uk@L?e-~Xkdw$6f-5I z2+2s6uxMZjO@^srYM=_HL<38SWQL_gQYEH@#z;zu7!x4|su0-T8M3ys#rbR3_~P@Y z`10AaOr{lcy*`Kb?q{yoV=|pG7>zl9`7*!p$vrMF;&OL+b#>J}_uO;xATXQuwUdr@$C0KX zdT|~R&(fYf)RPG_Gcz2L9I^s>eJ)?VhWU)TnNp9Z9Nag@d@FGLrg`o?(x?B_V~juc z2h#S1B`?IS>(q}wF1>fY-G(ds;LB&kC&R=rXT*?jF|gEBEXByNBIl387JqR4Jlj?> zih|*2wP(TQBr{AEi-Bo9;zBj#LTEVVg?5=!4Y#qPjg$q9Ymy1(GqNJzoBP&V{HGuN zJ>V3t(f@Y#^2vXCC>D#f04D#E!t(9<=J^=4$e?y8IqI~6F$pX=+9Es4cFH<_{3)(W_xFcn^z9fDRRV& zMk94JhE>JI^>t33y+(U>?std2`M<6`^URoC-?KcscJ2C>tPh9N*<$u`%-!uV$j@Y1 z=lQxGT^WzZE^}_VPImKjSWfBeH}A@~x>j$vqlfJL&}D~a-*wQu`(U28?#{iR1I>!E zO{d!>&k7QryetvPkgP;p&dkg#-QFDiUY~x}@Wz!6Kl$$4xqDVfm#+{9BlP?QoJPdJ z85DIRP2EU#`!Q^16NUpZB5DaQhKK-ykBQcBha0r!Xere7glnTI!|8;&o|00;49tu$ z9neg!G_qt9&a zT>03)_~%U)xveF#FAKqS$~LX}1zgb*$+9N*#mRs3*u!V|?h-%%InMEX{?_?@M~)o4 z$CCZMr0JJp-MnwOvpv&kwWMsd=(bxdEH2QWni_cWo# zOi3v+nM}C%t#4*&ejbepjR{Q&O$pV6X+UE_)vy>zDWM^fG(ikRO{5q|F_A(fghYsu z7#dQD#1POX66-*U0S$qs3B=G4V1UR+jg%e|ZnxvV})5Qw$IU?=nn`t>Fkmm{b$4 zu5U7!OxPKXIdkPI&z?TZqhEZEN5Ak4r!Q*m zX3A!2xP&nXf#sM}B%^UK+Pi@6U1T>u8-=aKE?ek-qKrBzS%*XtMQ@7YgpZi!;P zcSCH3_x$tsyytX~@#xE^uI>D*vu7**!;|~(zuy7=#EFZ&>GWDpq_SRbS&O20eXPTW ztx6v;4f~tA7LAeI35O0IpwnqH8jTo^r(C^qowc=1CZjQ`5hsEem?>s~loF~5Ou2aO zEPwb%pJHY2US?;ySd3r^i=Y^0iI@yaN-_mCR1<0mOGb)`sD^1G#e`}?W5QxWRk4^b zQ<5sCN{R}qs3``R8JMAFm>Fh9ObJpVCMASGRX1!6hFrRSo%2_(^5XgPoV|2~(P%CwZ0d3TY$7KJwmPTK#2e z&E7N#TMNJ%VC?#yl&VSIW-5 zH)Y+Kw>44Ulx5{jtxVn;n(FAzV9;LQ+R1l@L)qHeVQVmC+9-7s8BfNHcLt0{BgW$a z)2($@{Fwjj=6T++GSB4sFR}gypTV6!Pd=GolL_2%7%Mz(XA>6_3AnPeDUW~QDf#@# zr{(0uvz*yl=k(?pU*5RJOs9Zsw4Jv|LOz2*H+ZEjwj@ESdOeLQ~S z{7mJO9&Vf5ScE1k+a;|dD@ZrDdj0YelhGT^7jIZzS;{_g-@VsA_V}|E|6$0=%1V3x z{{1V-x#Oo_Ja_MCwEn}R(exvo-u$t$DBokr{(Fn*zPhUVF*H($qup+?Ff+rULx*r# z#_6-?c=5~`)~;Qro=h`7y>awQb?p2QBzbCNfWAynxaWb zDUm`T#7JraP17)*jJdkD$@6E=@%V`o{K?bL@WdCNV`Fohc2Tf@VUc|ci}YF@ym&AJ zF+?1Q;KYF#AiPQt=NVNsRihXMNDfkfZy|XH$(M1x6^gmT6x}6!QBagwx1;W%ojE(j z=4N8o_ax7*UAw*&!dDb~KG-XK>$r3ITawBRnJ-t0)@&@I3+3wTr&$Bo?%R6q|$Ira&Cf;&zmg(9W z!>67hzHpM<38E31jEFJfZ#alm4IAqlJhQXSBimbYYBWR>U|^<1ART#_%?=Pryhm+CSfA|ak&GQdE^pNoy{m}alukD;0wKL)M-5BScYeaKM8=GZod$=;w z>)z2leCTcS$8NiQW^VcNkG|twUw-hRhZ5fzdH;|7*ggsRzN~2f*BaB$U%htuzueg# z-e>5Y{cdlscli(0)nrkdnj&-byB+4{W>}b=rQh$dv%Ss6_6}FCUS(@*3rz|L-oQ}I z4Alf`7={=aLNc%v8IH&7-Mf#Q4<9D;LX3eFBPm5R1VV_!5D0Z5#y|))sR=Z7gElp> zj-)yeLPKmCLI`L`q!3A}Xfn(qNflK^Rne44s%VIWlt>{GQXL0CV635dnmk2`>>r01-e01jIp9$5gs#u>|D; zuDyio?8A2tko6DXyGvwchb+s;b5EJ&T4duZbM4vBZe2NDv+H}3XV

    e;(m$vbP^Q z*1fQ?5#fHf=h*y>?Q-rXe9^weqArVTA1(6v_Gl7!>V{HW=MG=?3d>uf<|HNcs^Yxw zkQXK0UXQZd$NL;j4Y6s6u_2~FG9wqbZO-$+>y|jyGp6Ga^@WR6k3C7D3ZkG2G2+Hk z8s|Av)jYO6=2_QawC6_258g_-d;s5_!DShO5kie&v;d!!?w-BOw7M`Hf(VA;y{Er0 z$GFV7JRVbrg!nf29_n(uH5|6YEzZ;T9;<5nzP(iQ3rw%CQHFqX4!S+Ufg9zb&t}#K z+m}N~PrdNM7bm<%51m*`Z(f?Mi(rRJ;g}nZi^2;F3rlp00=gXzzyAXqefJM6BuBis ze&NJ#e)6+xd8Gdrf4!$ z4YfqlL`sPim5?H_34|C4p&^8TX+pg4@Kew5^z$zu&a-E3j`^8c%A&v{2!@Em;SdBt zKtKcp0R%8!VZ20mnHk_RntYMeTE?{(aIJYLaYdUf%keJ9mw75&in+Iw-4>r= zXZZNW#)XDm-;*+S?b`MAve&=t={uD zikan3GHklN)wYZqbKe(w`MXS47TwuqiA)>^9+||bb9loR>TCH z2~Lf@W#s*b7kFJSQB7;oq^5b{G(}Y*-l4`<2=%n$^m8xr)XY2^2X4pD?!)_>m=dN9 z`ND|S=0$Yv46I!yRy)+8VHyLqDRtp+?HoV90CV$5R&Y>+_iYXcQF-LT4uhn)yp1o~ z483rszRdMxEXPFY9qG&&WPYCF@CxaL=gAOqfjx_}+&DW|T%Anba`Ec*yMRvu#%o~r z-|rv3e0lq^OP8)Uqfu;PWVE?Se>!GuoA(ye@J8SS-=_Dy?|ofx zAF>pFwwhKqHcd@cO%UfOJgn?L;0rGlnb2vs@Xk|BD|QCkY;JC|wY|k~Jf^N{e4Znj zM+`(2ONwB41c%@eULjy62u8q20vZ!v`tm7$|BoJGe&&an>2%Sgm@3H>O-hOpO$keh zloFN<6jT$2VFs9@W|)CtV1{Wz0kecgLk%=V)RdSK786Mm7>En52qp_z`z>CspKRYEl-RTESk?RKB` zY>#%UL%ZF>Oh_RxN5uyYZnT9P?~qT2L4+ZY)pH3nd0za!fm%f9kFSHZEledNAz}vynZ;K z-)&PaCmtFaV_)FgeX1hoIU1gf>l{dtMPI^&i}Y{2k;&p7ro}n>p(c-!*;sS;EyuH} z$nQGwr4#RX+xzc-@^ioc;3nS!TU~X(_2iS?&u7`2+xrgxZ;=C z;RCdC&-V6!&8i;Kbnt1OWpD5}+u-c!^+U z7{X{Y;@|$x$C>Z<`I~QhE3G^urhq0xV?xzXRZ!z)YKj>zQNGGhP*79U6jLRsK}tkb zG(|Kj$r5IU7Y|Now>v2>%cR~XC)t;9?ny24Cs$UEe7XMIV?X1>eF$egRKP$)c$Hxy z2(Jc**lmUD()r`0z>B6z_L={~t%n4VPzEl6Btf!uXwyZ}%)2t-?(5 zg4XO3brVR@%A!@4ow=5YE70kXlp0cq_^hC8cWAtW5YT$ec&(ytwQ2WyJG2h zNH;}t5vFHO!~8t{=n*EDE|DKSL~CUqi~SCV_wDa)Zm!>Z>e7pM0H5aH!$2NdUGbx_1bkt!y#+a2^TM1WOH+q$z+7q5j5ctoC#nU;DMlu zbDmO!OdKy$F_Ib)BNzmXgrR`)RR$ugZEW!GKK^^m_50j;>n(UyG#RRjp_uX#!$c6{ zB?e-s@)AW2Oe0ARjS00xj1fx#Gcds6@XnDJMJn=Ej7SjgPEh23BIds!?e>?{xr-MD zgSDUgrC$oiZ@TGtC(G^?vJRq75HJA@!FYuj4iE%2K}0cLW?(`zrK)QN<1wRZ!nB%_ zc!}Qu9OfJ^bFI$Ux#j=;@Z$d8{@mw&e+w{nec#B~wd=2!7OV(dOh5#bzhVat9O$g9 zEU#SJu$3?w&nofUcr(8+x4ihU`OYnx@}I)HQKy*gO{QV5**N*4ZJH8I)8L9W-Nn63 zM?-5wD6IO_gLG|x+Zs!=LI5y17>x~J?+4Ljija~>&#-w7@Rpv z$9q&1Q9&_AF5}AF0=oAonLwCMNjn?J)>ZP+CPkRwEP)s*yu@L=Br_xy)Z?1<=S~t& zou%r{vC^Morf88eA(~+Zxg~DRGw$gL=SB^iu|ZN~Dh1EEF3+SL4ra!@gLLsK+3O>% z``$ww4rw+A44*s6@Wo5S`Q^MUT6ZqYEZuqk{r5le;DZk~{5foO)jcyBEw#>^f3HsC zPf2RtMvMz7MAV?y@6(@|XKONIy179;nd0^wpj%abx$uipmBr8f=0_)do6O98=))hH z(V+d*L{e1X!lL;0AW`;Kq9GD_}m5~4ks440^0VAprO^F5( zBLT`^z>F9p&!0NOM}OyI6d(RDHyk{GsDgoEyi9nB2w)fn$xt)Y6ibPaBB}{ZhFauh z4ndrwmFFoh${5YnD)MZ|vd?CD`B=-jQzKtpj;;BP2OfAJ@pX2p>aljWdnOY*pduwE zyh;GaS1E>I2!g3$B1DBrJz{G(Vl*97g+PpfKNSOCFIpIFUf6#A(hJWF0Attpm5g1x z{^JtB18!z!rPuEIUl@;@8|!-E^LeZM%w)%Z2`&a8-Fxo```500sX2T0YycA9p?5>p z{@}fx^>I17xf8lER4z$zYticebbDd3lVXAS`JTl%7wTqTjH+r*LS1DoS6B=UMV6CP z$Xh)mYm+o#&M;L{sA-~-6)mznqil6)bu&WU5Zx3>ftV7xH58qko4UfieI=XhpwW?9 z5v?Zp!3bBDm;hoJuu-(Q-d{rV7IC;u+Ps2Wzla}ikz3+rY9Qhe2N)QFi6{7s42OoA z@np=oYQk_hVE_C)%l%ora~OgFJx#o+m2u7z57$$MQ4#Z8mNrk>gge#fdcnsU-PuAi zj~k3}k3Y@K<~E@}NB2hP&CD+BTR!m4S{EJ$p5o8=_X zL$=2QCX*>mXh^Dj1L127L_CHY=a~q*zAt6$+V$P%fC9*i{^G$TdXJ0RO{8;)#XEeF ze#le37wp>1-rm#gV&$XE&wO$m@}qUJ^mh*)nSb!ox#5%WV$FA8c+Wk;w03#)u8HlH zI%e+;Rk$xiJKAq`OYyCIJgHU9iIN!_oh36dkChQsMV!MK#1Qd$LE&3yYKSobhJ`7U z;WmgPFA99CMNzb9ce}*8!eSuvj*gAFMFI!akZOv{OOhtS+6HYes5nFbNmSxk@AepW zI;8Oy>H0-v<2+flMV1tC-kf*=A(=y6hGhot~?<+%mC z%OEL+vf`C@cOu(q!WSy2EHQO0PNo(o(v*Wu(Z(m(qeSURyb;4I%_~=L57e4p-59oCGb21v;>zmNmq(lRULYBw6$aj*@ z9^TIEOu*irp?8O!9Uto&*D7(tNvnWc-;(zZ0Wb1x5D`0e?AWGvZZm|i5Mv`d+dGJJ zRMRPS2nYt|$V|x0_<9N?j0jXf0u+cq(1ZYn@CxJW7_Sn*00RUtzN%>gk3I861mPp^ zyN{KHWiktMdt^-r*C2o6TmPO5MCuDaSSFyHg>idPbbt(0E}-Sd@}); zWq?*yMFfmp-*+;0?fUNWD=RBEYQxVCr}6eg>qyJD7o!BPCgbs>n#HnQXfNaP<=*^( z+q=E~TN|r?H+cUWrQdV!KnH(wcF*Y7cEUaXlC39p00$_q(GyQR5mr|j{Pu6(Gal88 zJ#ywVSKQc4$DN(6LCQ-r-&t^$#3X{H$}~*SdQ6t(&@+-o@^%l=M2Z0?8ClT=sYrDo zIq(@+3^a8>VnfWPWa7XJuR^BG2 zNVs$gw|N0yjqoTq56+Q*=rS}fu~v_)+rt$(NQMF_2By;qwtXENZ9<$dXeusjT}Mor zpIatdSfaguAN_uh`<#$p-)8#kDZaQiCgQm)CC_TbU8)?|yM*<7_;VL2H#f<>L&PXk zMYwX6Emul$c>n%Zk$w33Ab)Okb@fyKY<1N>^DDoybo$Xpf7Gq5|NU$*xRYtfqbguH z@3qeIX~@gZxmNrCc0PP|ZM4;OO`I*F+}Uhw`jl$&ZkKXg`CQ~?LVW}9BHyNK*ETQo zI{9f;J=#=teswZtZef8e XR1EMM-4lfEOB!qwwFlvN|5fO}UWW3DyX2MGhA>>&e z%`C*2xt%?q2O)zwwy+psJ3GJ7pE>ZlWOBFj`BF8Gw}nRT%5?LVth4y|{DJ+;t;HMTbn{ZpYxK}VFmvNi z@Aa+rkBWCcfom;Dt0zb@B#^A z-FePrIopFx?D}c)dW=I5??EzD98F$|_2%$%3l#lXe7B3VT1Z~tiWX9~k!~OQ^Q63u zH4PGKECfPQT6^{|cf*ZjlR)<3dD<7RaS)+1*X43OW~*vw5ZbBY&`gW{?|eIDtB;;| zfxKz(B4p0t^8&ZBmup?m`Sq(_9dmQN?nJ8Mh3vohKrf!Z{4?RowV(If+qY81T-6Y9 zMplF-_s=w${7O@{fBDGXxrgum)n8rzk={(*OzSt4%3E{OZ0^RJ$@?=@8=J27A`^>G z{rU?hzreStsw^Stw$LT2SA7^5kult0tgNWKm|-OYM=oUf0pnn0R#}f!hqzQ ztK04N<$iDW|LgU-|D!qgvCuSc0G4@$@G2pt#MDKD~n)>{cLHxfg^89~EzV)v`KK1;@&Wr!x7k_c{;~)Qc{m?@X8Q-R=sYi@h6J?w98lhMUb@AuC?rX?5GC8BNN}$Z+1{GDsm1Cqt+P)YFkowzj0q zTfGB^ZoS9RgvrhqsH(UJ?|!VvLus+T6zi*}fd2zXe9Kndt?#aH&0P3L&g8?%wU;d~ z92(8Jh)GQd6{Gblv}csg+%mIsw@@uCQV&Mdb&cdx+TIhx6xB$KkrWzIs4<`@T1b}D znq8o6GlV)&*A;2(h;^dXYSGJ#TYW__91vn8JJoAM|Kh*bb>!#~{w^Ubur!|@!)sVz2;(Zn?@2=L){+&qv|46m- z+y{T_*T;OtmD8t(<9g{5=Y`@i0G&PtZrOHglS_`YItJ&)6YUJB8y%K9ZVNSMC zYbS%jXhgf!#(5znBLNeX8c;w11B{mmV0;r1FbpAKB=5W{ihR9n6^|E1>-T`q=6QbU zmRoMwdEtd8=0aUuXzJz`Gcqu~UI>A6moD-D{`SZD^q$AL{pK6E`?gzIIk11=a`%Xu z{Cdj#qdCRZ<>SY<@4N558ZiFDYN|SXF7K50Wn#T)6a2_!@SqxjPB5rPO8oR#lWbE4YS4$4R<3FD(Mq58L zp4tBKX4KqOw0k{@*_IYFtvv70Zk41aQjLbBI*@xucezU(_ej+yqn$O&vI23r9lqs` zd}-x~4R?k_>-DlM`$#*dO3l@0TYY<`8lHa`F#a;{Ud@i@%^ha(pEgl{Sc+b+5Vg=W zMiWrikaS=k|_2*D`k7U|3{Vab?`Ce+gkoI_NZjt0c(7(sA(#&|lx7X^7y z;IopvRg#xou!JeZlqe-|&?e|)LNHKqBolnAgES3NmKXv~FpLnVRKrb-00`hn2uVDt z-N*H3$hvKOUf{gPi6D}I3<10YoJVkY?{Im6v=5{G8LIPbzDEIHBFvrG)OY{a~+JK1hDn%^fHrCiV@kNG54|3P*Z;@WN zxbf-7AOAZiuhnaB@3mWID$zT|m*c6k&*1r03&m$1{hilstg@I!iVQN_$i zK;q95!1z-F7z!4>bJ1mf&AII9BFj#Ru;85k^`>b)f7e}|t%n{uTLX6K(j^WYI4~*l z;qQm0Ic{bvyg~rus{~;*9`VB2iJiNASwH*O<6&lI_S~KW`+s$1<;W)iE%LG z-{<&haqHgf?xHOIyLPAZp(tgqu9c<_;=nmf9O`mnr%N_FN8am_<+)_uAr1sV#35n`4g^FD0i4Tl zna39ezBfxYzl0h1(EwLfgcQiSU2>m8j9^NZ68*ZOzp+jC{6(6JmvDy;(Op_XuC3uz zaRLs*A($mnmT}{KKfv36^dsDU{5D_a`P^V@W2vaBUMs0Z?@pNTD=x~vSiW}Y^Y8lf z9}XYlA>$?YgP$BdS{^RTV_ARxZ?8rE?zzP6&qwm-MiWkK?C{0y0ncn~=g)6$-nKa& z{@_f%zt}GFtHbGZ3ou@T_q^vlYuDGVFRSTWVu~F#NSeS5GvgZxUn68$W@T9>Ac&K? zEQ-^uPWc<{R{1~8^k@HPQ~fCLmK5Vh>ZX2cyIo!x4zu&a;c&`V42Q$Gw6t{5EFCdp zzlc-kTrkWdI9`(HIhpsoWL?C6HW~kFS~tIZ>eT5!IdS5|kne)&bXp&tmq$ico?ls- zKQ`0pEapCQnRf_+ri7}JVnhHhj(SqjZnY`OlF4+6n(`MZih{CLe!ACN{M?>9ek|sT z`?6tqz_Wdpmngfw=XrMR`m2+1^E(&&iwi$$fq&%7&YfD$IhS=PyR%ldXK8o)^!xMV zWzJyZDm&LN;4HH5$PLUNIDjt-rh|2M)?P$4(C#kC_WGL4&(E`d?Ex0mej3ZqXyN|6^J4iQi%Tu*`1&6pq_#S?u_>U7Cw=O}tTeC}}~h!f!z zh5=?ujDfnYX_}g(0mN`VBP$B>RzcpIp;%Z#)$qe@d{yDglGfZTnSzDL|IZ$r2YZ^G zg?;}%=bZO_f7|`t{r29o_3RqWNE+=DNF$Jf0U1o#OvNS@8x?^H6(>|ug_DwlsW>4v zRQ_>Uf25qHDrJeq1`OCpNHP+O(2VxcwA9nn)9byr@A_Nc_dVx4Pr7HYC1R%vOoLUE z`}rV%f@p@^5++kZzehfG0v99Y;RrFqL=X%yqsrn$u^`IEC_HZ-1V! z-xSYJZxoxq{ElDw$jwtXZm5C$+Us6deA{r?zdBrfeP!}rOY%SEGv7gyyP{gzst~gh zvl5e$vx%vcL5el+!|R>*k94~It?_hx2{8VIE?l^fA9>`F>EYqwsTkvfhIz~|Ge7|0 z-zJENbvo7N;=+9C7|VH9`Jzi=f|%fzx#>RZuiH$b5A&<5^gbv6A=E08DM|`j;3v?sxDR4VK$$k zs(|qSCIa(%_UMUwAOD{I;^KD)-+iiwJ-)DZeC7C=ySH|4T$*#_70q+x$erg{x_hy= zdfz*H(f$2U^?tNkK7NlIEO{3yQxTeaLOmbROsB-GNYSO!?UL&`qbnELeEu`+UVDzz zj3~rN?UdPQlX^DByOPP_E=5r=o7_Zw4|5Bc=5CJ7a#>Y}DNeow7+>euyMMNS_o?u$ z){g(dbTsjUltgUnmKVcA!#I=pei_VNP-h-#=DCNy_7KwaM;EM;hvMspZ+Wz z&6sxxj_ibORbFT#GmIPp*6UL&EmQUua3LTf-~8`W%27Q*-)~yM}=P%C2|LgL>!EgOXANm*V```b5*Bi#cKe_kJ z>0aHw^TxdW$+~WTGSBsGmTZko0Zvo_5k$ba#Tg)o69gw-q%6IhEP|{r40@OM$Kxyf zDZKA}@7ww8XTQ+qoX(|`PXLauC;Tza`PAulc80@+5A}Nezjyum^?yE_)i2Cuv+;B~ zjezn;Qc7P5WqGKY?lrZOv)TN0v5l|m^@cCh_3Q?a0o&c(&AYq1(}&MI>>6txY}@vA zu}!CdLPRh#P&#V<8ME=1fC!YYIro~C>cU0}*h7h=74{ivbQqm6#IUP8a6_^QOft^AZ6>Km=7I#aOl)R`SaIr_$i~ z!{x%MHd3cMV@tIrpRD{5MA_Jiat_9zzI)CpeF& z($42Jb%Vx4P6_7&Wv@%s9pK9z<^w8%IgbD-W@2o~O^e2aHj&)6Xw#5lB@Ls6F8 zrZJ(hCAV|5opDlYo+{>awMEoWap>9#?#wCDb6>&FTFg7Zp(5;A=EYnyGeNsu!qO7u z!T?tkIOhPvfRtL9&t^>P8r6*R9wz}@iSq$*4g_*CVr)q{Xi7+9YnzpoPalF-nPv9J$iSYp|`k1e=v}{ zA2@gL+PA&!6NjfRJomR>_t@Yw2QS=v*o1fQ?bh$x+@1W*!-Mt>v9U$X<}pwV3`88a znG*~F@qj}R1VaEPp$Os@G2I_@t1ll+r&IofKK8MX*$@5D4_-Wf{`@%Se1Fb)14KZ0 zi2wo!+50eBSQvhMb#>*Zy4~(SzIgHCbsz)Azni9%o*fR?K4sb-FMM&TZrXEgY~I@M z_g`$=c5`KArQF!qc*OhSC$4T^`JbEV>_6+2<$c+#%tf&Y#-YI9`_J zk8d4r|LoS*jUV6N-~XPbX`XVvr0o66M>>bYOr82!e=E7A41y zonSavpswebDhN2?7RRiq=~P_?{UOmZb<<#GfPfenh;fT&KG~%#hvM9lOPYL2657ZE zMLT)6&%*i9=xXA~E0X8Pkvq#`cj&6H@^*U5Z=B6rmrTr+9Xh=sy}=OY9p(e(g!z2V z=-?*vn^&1`e}#7UB58aXm&O#<;Bv%~5fmkZDViH{JE3Xz!7^eFO$i@Lpa?llma(ik z&p!e0*JI!P?dzwO{5z{q{a9IsZ%<}kC0Nzz(&=;&AJ8^aPshy1`{;azHZ6uycDfWr zg=WyKB*2A$_XQZ_Sd;20xv9~%#!|$2D5@@@+o9@oDY`veQQkpbm};6dylZQkNJQg7{*xOYRv38p;%g`>@5;6MNo%n1mH2*PcGh~qYo5WK4d zPu6^XcRGH7KZU2Cep(-W^wIM-Z|+|du_w%I24o2)%5MXUau2E@t;n{)4vmpcMZFp)x~~4cuZo< z#GEid1k6mTs(iHH?f%esHho{WQ@!(^yH2g;tYJ2<9kA}4m!b%t%Q^3Hhitf|@06T> zKs28gaKdeZIAL{djaR+uF;1L3#qr}C96NTLt*uSk7!klsNhx7?s$QSP#Wh+jsB6U# zFb@bXGeFT;6Bbs$3bI86aj|Wd8>#;JZ0E}pj=UlTjvTqO{CJ*@X*oNA1#z7vI)f#; z{Y4OA@8%6CDvF}Qp~QMhJKZNuxAEhvlH0yRcMAd|=fCZ3!|IF}Gh zn2S`sW2WtJ(KJ5Q?VkRF6Yuh0sABUsv~AxvI=HzQ&0CYzK(LfBF`PKaiEJ5Mfmw^@ zIk7d`$pPXDLeZz}_bJK(oZv)IGNGy{ih>ktw5icH5$BP5KF8q^7jPmNnH&?Q5hscZ z4zxm4P^UCGLe3z9$W{8g+UU!&LWl9O?GaKQCTSGaod64U)Dlj#mZBrLBGRyK%d z?xi|7K<6zpC-`1Pr3sIK87hLAkug61(U0+)(_KEdwMpC5ys9_k2R2UfhQSI&f~B3C zj6U{pZX93b^b?O^hug>E&5J)a=v0;Vmxs?^eSzQk>;j;K($bBT(%-C`=AEWCEX(3X zw_ASD`||&~dGqEb|DMjDKR*V3W?^CBOQjG0P0HytDF3}niIskjst6dyyqPna%$e3T zh9Ls|UZ1k6ID7I$H+a{5@#-~p_x9=a`ox&I%?zCL?+(HJ?@iNO1QK_G>cg7SaWV5E z0yw}J42Ily?i~GY4{{w?F>y5znI#jT9BQH7`mka-b+&U(CK&s6>E5drwQ@*M|lSPu#QPh`$XZJR)BwgrTD zMqX34-81a8D}>G&0FIpK6gMU+eGr?jM$&z5Km;^tr@p8dUGz zTb*--oJDXIbCWC62xDxb5)3n0>}V{GK`9b%e0c8q9i?Nhga9nA*v4SUH|fh&Q@gj|00A7Sdif_@ zH~3lFy8t;+i=rvUFQKm%#0s6)B=;?!e!U8bP{=(IQoLhOxXdfu>_11X69X|hDI%n* zMuw`7VJB?lBYwQu)V+yWunozX7|VV&IzKt^z<_@DEXltx7z}m^Io`rHgJE=4RgCWx zB3>Nry*%98U1r}pxhxbEJQpNfHV95Q9^0|4vkn=*bM8KLR-tAZDdOde?NNU~7uU9D zz1G8#tk669Q9$83H5sdrv5=^fRfp5vxtm`TW(SE&duU^@3~&uvjOPb&x!RYYh_RE) zlj1J%Xrl61B7QPD5s487dKh6vG0)bw=W!(IL2pCT;JHOCrD0-jdk6sK8z&h+{>3^Q zdT8(J+&ZcKdc|(}`d??h|2FL-KkUe{@nH6_y^i$mSq*)7T##9D6)AkWK{{7#Wydt3 zL}x610V8NhJ4v9}-hcX!H@J*w7@xG=$z`IHR7JQL+=UhNk@X%dlku+{_1YVOr~(>z zqeTl{|qbTBzQpJ`mdAYVV#heE$lMdJoc9U!8@ z{)Y-@{YR18}Yav(ePs%HU}VHwxArF!@^?9H1vgGtNbm=ztiqOr3| zA%$_p+aF>!E6WejxN5=y+4MyfT*gV;`Hx*u6i?LSlmKHJav)>^0#r^kLz+AIR^}(n zG^UNTt*a1I_iv~!F-jzooSfoWp}Yd5V=d=hzWxV;1HFL}M_g(-m4(WXgPG)$=(T&i zw!=UJe}AWfv%SudftSCUFTXd8ix!NA#{0bnAgKZQ8SKwzXRSILdYCDf(zTx$;R8e{ z=u!5rSW;9@ma}kTz8DgZh7c6Ht0k8Esqw9aH^#7$WUYSd%X+MelJD=aCjH^;}KGWmA^Mg-`@@L87uyR_VETMLPF8O{4X4 zQbKaqT-rTAKS;-nZ{QQ}KG5O^!Gl)XS0pF~Z=@!2O8{iI%y2TYktj57MgBRp_mfTO z71!o9s-o$y*5+R!N+-vSX(#8i%(vhnL48PbJP{h9trF0^1;Rep4SJV&TaA!Di~W*1 zjWrT>*gRgPZj0`wY$Jp>dP4_da*#xCX=f6fh{QMY+4U&{vP05G)HbwrE1Z#_=9VwY z7w6xgG{5-f(HaEa%aNly_F>QA{MLJ0GdhCh%jHmhJ_i?kOU1G9;4FXlvF=T% z?Zt8KLjIhM8<#FtTuDinGUnxno`tPyNZ2Oe+Bb}B;86DQrw861HUxHo=&w>66djjx zS+&0%cj04rNEk*}i|4J0P8xJ-TC@q3@aY@ERPjBSV%JX}EQA1?RtQj;kf?NteB1mQ z?(}wG?A_BBYTNH__y0)b@zAx_4^~|cWj!BKs^hj?ve@(X4X3793!~dpyuAs({ON*y zHK>ogC1z#?;eLd_J{A=_YJZ@PE=SW{1by*5OsR3_%hO~2@*~ljO3OYtxbW|cL_*TQ zv{IsTPTrDi%sS<;J(3Ltdat%Lb`nQdQMb(Y`*$f93gJ#dZ@+Bjp*Po0vk1j<^lhg~ z_)h*Wbw~;E<{UopN+;%4S_h~r&oJAMDNDB1W$SUl_IPe0{Fl}>r7S$8T3Ux{=;UX0 z#x({S42fO9{sg3Qh6Lk_n}{DVkEug%_up#HTHp1RdEfax3E*&nCg317-$$dwhB?Zu z>E6E+Pv?7favP{iJR&O1CH`oHW`ymF^BcE>Y=$u+wh;I-R|%|Z!w^zwN|i9W8v;i| z*HwtdiY+J0`;EKbIuhj4=n1&tIj7*-42pMUR7{){A(X$xGAGjiX8kT5MsWKtAp2~2 zH^08Z57|{N^my!B;F}#?5WYHtg?u|E_-^KC^Y_tk=~fBY#a>_v4$R= z#(qqb{O%S5g~A=Ur&$g&`m95jD1AT*w!8=xC;M8D)TbzR=Z-dfkxXwNi*sADT|1-x z?#1YB_*!$@yMZI=_^}V$x8BC4#?N_ej3{T|CyOlOPtnR^zCX;Ks>!WN_Y(1F@Jsd1 zi_XMG=!h>nY9s~7xMb1J;RECFUL7R5zqXgV zi8nDC&(}A4Cs`&&Kk@6_(&7hY!rSQmn6>-ff&K5sD3R5*vQxi{_(1+9pwO34;m=$y z)D`&6zf?o(56oD#M&k`ILEGdx;tfKA83m(kUiJ@20VZFXv6`lzbwmHbFtYx!Cn5c_ zvt-!8c$$<_7&wFSu$T168K*ERz>8(Le=x^loMg+1CWjKR=&EJwN}mqV!2Gw#(a-O1 zw+QQgVR^ZszB2-#XctacZn=86a#CuQiKvBzgB*);dpwh!t4K1e-Vpwx9=h%a)eT6n^MpIBc&%sqMF1f>1H`PuPY% z>Prj&*wN4<(Nx0{t_%RuvLv#(d=Vy~_G>os!T4}L#P2shmIHEc$HSALH1zhlJa3;# z@ZiBA+FaJXvYH!sx33foI&SsaPJa8AbIVIJA&;p_21%^%%kUX`P?dZ5437~7PU7KGKeP(wU zEK_K)GJkm-FfQLrV!iy>yw>*?b9}HM#5gTCNm)stv!~@}X4o_g$$R!Clo9 z6tW#Q|H@Yv>t_U2bGAOd(e_}B{pHyhsKfryW#-`s`N;{mFzDp1`+LsHy?K?aB5i9a zas20M+-iQ9t$gtD$S-NEM%_Rb{_SVSNom^^9=FyU3Ippex=}H^Ja%cAT{n5?8wyYl zS$MO~haV|UdfLqVT)kA#_3)@2lZVV{r z`CfO;ts#jwD&F$2Tust1H!nS~8nlt(tK_oub>`q>chm)`o^%eh^QN1`X13SC*p+}` zULLkBHXf1j_wV1U%AK-2y$?AbDsHMhe`VSIjh$1xy`(1W3~&$r(?Suxj@Mv~ZfPDU zOdBOo;IA@xF%azhYLMtNo4E=G>>z*n z;-O)$bRx$>*od?^u=*+7+iEVJqfRS|L-P=!LZ=%1>Fc2^Uhvi~ zWw{Bgs#2Z#EoLLDJi{WIYp^hzJK;JT7Pa$=9E(1Bq^$GcypC;&F)ubrpFYcFnhSE@ zBb$Ug>pjbk>j6p-gVsZGaF9FFpx-bV?I&PfpQq$YTZ_N>3(#Yo}bL z7(<#EUT^|=41&FJis1p1zej!gM36?U=cgD3`dFM@xg;t|uz8UyN__-!^E0GYlBFUh z3)Uk=)ZQ|DFhW%2G*HhZCBph`S2${jhpBAqg-7KmXmyI=dFXn4oP}(Ck~F1}M53Z& zTQ5);>_a(1Y8Rrv78pCQdJJn=$R4%}Wf5L1cirzLxRXf!+#HGE|5Ja$ilom=eHr-WU2W>jAeEq~5|jF`_1*TuMLA6< zkUoy-HzM&;gguBY{0+8%jyS!^W#9DUBz;&()d01vkXd=*;^cBxKX9ao_S+#|bE}oQ zVO7xiYfiGI3wV|D399{JbXsXt02enKGvk@}uYW*MB<#j|_0Kp<`_1ZF$JemqR|W8| z|JH7M*OLC>Ut#@CHpUvp6>pgovPY`n8Q+$KTFkry>hF3l26bY zZKYET>)+gRQ#oLk#k_nkdR)ey1UI~Wq#OLDhy`w;`I&24v^rWa+NUtxqtVFGYeI(d z16iintm`)o2lCK;BrR^a7EzH{z$6mvZ`HYhWrQ!d*QhKL) zuQa-^K@1R!k>?N0gvAop-JdztcoU6HQMvK^Ifmx=&pSmvdp&sgU@B;AmXkW|^CI6q zZPaJueC6Lzw{EM4k_=)!>DK9pPmW195Su}bXgp$@me^nK9{x$%ki`o(LLzV-?Ux+^ z`5c<+0?K1BO3vpzFGwzZ=)zyYlBo^T^TjG0>7lsfz0%>9SW%ZrwJJ|zk8jeBkE3GB z6D?owFd7yqg=du?1gyFU?ut;vLt%!mdMFQAp%g2sVTb%LC)myLpYu7}Zq+T@^SF># zE1f27Y$B*X7*HmtJQ-C3CT3wdWLU%Kes5d?E(C+Rmsa%N&)4cbQujQbV`4ag4pd#EC?u%)s^x!0jBCWb82nfcmoeYd$#t_ z;Vx+>tH2NB!fK>P%m8ImZc<(`E2XPkNp&l|C&oqGuEgA!utVAUF}u*=YO~eLuKgo} z(9fK&+Y4{j2^BBH)-EE}vaSd|OYda8z(ky_e5(qs^|f5+LYv3%ze^2}4rO?_ZD|){ z@M7)uZ`EnxyV)t8Ncgge@5`St4^E#>N#2Fz`4?~Ak8?Gf?(_u1HUxnK0!Rb^YL9(* zk1cT)BQJL%-wVi1cQe`i_0cEa(S1u{I#iql+W$kRQ;%f^6%`LI7_K$BGtR$>sQADB z6eT1{WlrAOZmVgWc0$baETKw?6P%E+kJ0-%$gUn4#l4zf668 zPOdeY|J9qpQe#+22qxZK#5Tt6tm3C;s&ul8VDhP^g8PzcO~9|bD3%Unsy_M4i0lE}Hq~u7tN>i3#ld+N-6Xu!FKx zb5hl&^zZiX=67HO-~+iS=X$b8v?iU*t={=OT3(9MCjH3!+?&Yysq-|^@x)W~34L?J zz&0NprCiVFP3PueZcv)O{oi>;(|r-F2@@U!>gWohrxz0QF#-cf{{v5iX(kgbHFWpM zAN=4Ks<#f2QCCuV#qK;>;j$P_dFZEdKO%!6X>n&`+?1yxo|aBbbjI!r0P7QTML-YN z{^Sy7>sq}$G8zBe+7#!~BdcjdlZ}@Jwcx5w5fD=~Ok67Anx=V_l$E^!)UT~O#a0JQ z+Md1`>0+|S+FQ{qmN9c%VC^y4wt4xI_0dzS8_k~e!>g8!HZvRHKA8@4VxPuf}8twnB?7F@(AFmJ6_`oM&y#J>lU>!DuCH zr@0l8+ru0b^d*PoAiy=WYrl-0qztT_wnbT-tG5?fxOjsMmIBu=qh9R>b^9Yl_nQDfLq3yku{XM%9P+K^jS9nT)!7*`r;%2weOL^#t|~$mS3+ zyeLySo$zbYSo?v|k_7`*;Zp z$a@ACZlumBXgAOGj3_~}*>R!v*(iEc6b^276t_K_J$(@Z$x{KPC!A}hFX#jjatyu9 z^4T~hY<07yrX%=sD61Y^|5`$TTy+RR0T33~;CC@p*Hz}&j@eZsFw3M1-HvJvxx_To z6k7D;mwH3><<&tnSmehDMTu5V_bO7_mw#QL&|bpnSspLY@x1NVZ-dha1Np(hmuv&5 zev6%SI@iGEtI5uPzsDoj=M4LzHUobLeqLT!9#D8!_ONE3&wm_y|PwM zGYvlc{w?Ib`{i#*TUiwdlP5E2>2tQaQS6gU{%_S;yP9EZ%^Q9NTfvBG0X9ME%CZYr zZT$;-vh8>#^v%ZNN#xxS zkK>du1GBJ&jf2Aumg3yey)nzxj)8hhsBNVX4~*`!7rr=D*C&%*ZkMxnsvmaTsK`TJ zHM+8R5(6n{Zpj5<{>=3?>&&)A;>CL;1TU4c$a2RXzi&lmJblldYPi`p5X$a}Q&$8g zrjSnk7@H4rd3E&XlRkxj4h=r(2{{jOf)7OiK-__zi`;Z#deh2KtVYkm3&Ds&0PxZH zHWt0K%)J>f3Zzo4(Oil_EqKk3)skwe?j?S(H?@x!2l2){@oV2mybr4va}~4Z^iM;S zzbM{@rqO+AY5iGm&R7z=n~3)r;EJ%P?-n44MbH2?8oyGV5b_$F#(Vqj|iN5(^! zV81d#oeG93WVxD$l`gFvz8tyQw<-&!$^1|Glz<~1tFWoO}=rv-J!DT!~yLM+*ip@q~Y0OCK6au1m!fTNna=|;DFxu#kRs2fBO4L zh4+cux+5nL+qHR`!_qGHHG2g=^x4RTuh+kozM{^epNk8Q_k3BlAsg3IHn0B?9w)KB z)O;4|dsWwLbbq%0Ja;kZ?kp}Ytn>NZzbieZD-ODsTT+bc%~+${yBX83opSC=0}sC> z(|?(F^H`Ex@k+UKv+bNqRESK(>F;!i19byrFN5lRU~RAo`~knZbRZHu`Ks%+J{IURLJL$ zAdm5xFRELu+$?*zyI66`8nZUo2njC>z9I_{p2(cFD$+h;^FDioR z;%-8^y%W-69(QHPC-_I@dpx%-hR2r2JdUD~SDEa23@Gm9cf$JjZB4(3yO6tLGl^Zx}`y z^l~ib-1K3T$Y9i2HjfeY7zN0bRWB#VEY0{muiQn=U+D(9^^I1kkY8yOegTo)O-!ve z=#V#OcJ1HZjCv2z@8?hsu`nMA>t(wPecm1%VBL%lGl06smZP}T)1bLBu2+Aa_n%zk zwI9Ev++i9~Tv&a{67p{8{E|Yw;}`Rh1?i3FJ-i_Az9dI#E?4rlr=Z}RgHGt3;`V*d z?(S}x9vT?_+}+*%Va$a4lG%TI;$~0YElSUaCFh)9IVILT%fxPY;u(GB?VyL44cE!v zk7Ox`hsJW;M^a*Ov+*CQtk)?Wzn&hg42qZyWVUW>?%=5uORlwOM4~mM(ht1W)_WHM zwlBHO3-6(6?bzK37QG()gf);~7OIk1{qs_t!CRiPiPep&8H&ZccqQV`DH6 zA)bWfmpR+E)#$mf>;>uZHnF3P;1}<1I_iG?tg$21zONecF?3xUCmhJs3A(FO-2PXl zQxm5Yb|RCuBo=F0+!>OyKPwsimox$(0SLBzgiwQk^L@d9U_U}g@S$H94nBODbh+!z zGQ~Uw`DsYJROEZ(<(+~pnH0M?*(kbCk5!HrPT75$TruFo(0j&`#hE)cnVEp`s{rWv|0v!(NcEyepY4*S8>`{v`5iguMMRsFm_lHfM8)tjtiZf^Tqgbgz-+rlxeH}Y()_?1SrfT3r@J3)aGc-I}mK-gs zf@#O6IwW?R6V%!KX05&XEmcwW{^Obq`ftaDz)KhIf8Q$N|1_1lxm^6L&JVp5i530S z8c^`j^)SOUzx4(pDjYBQR7==oW_S$&MvuVGhMMe&IZvk78Sv*y$2E!XhP9Tb7 zR~`L@#@2$~U_AfqC21a{l-XMevh^LQRsv%YCB>gXW+cI=)M8+asiu$59)6>OKnP8y zfyZc^N}fp~!o`mdA%svMNWMTodqST>&m_IwPKdeU;rZZ(lySSQkyh62 zYfp3Oh;caU8;uYg!d{$sGzf`qd5zwJq43D*i|omL)(OaQ`F(#+uA%jK^_%OXn(P}Q zzG)kMU~2yYe^uSwQZMTyQ*e40tnjUQRqWwM@^9== zSc|iVDa{YPFFLmG7|etc5f3%*=F`A&2H~5(2X{JaXCjO2Mmmi%?-T`x?y|-TBdd<@ zr^425R>yRtU*5Mr>v8C?;$`tn*8~OIe;)0AA-I#!&@EnCp05?v#62OIXsk2GrQ*fC%g^t((-jJL2#+YU8FO-l zlQ4NxYFT{L66XYxa#WP`P%&kyMKwy6Sr)R@s{-KoVw?nyq$o8%_IOH1py_gA1udU^ z6Sw4_kkJq~qA4CKDiQG*wro`}h@Bl8!Z4pLvvyZ|bJew^@9)1yQ2MO1yK-sH@}{el zUs6&lD=39VWFrZ&&W6C-v`?y``TJ1P+k)vMHH`|$zT0gz;JoN~lT~h1^+;ymJyWqM zd(@d8Iq=ZW1|sa&F^GZKPzEgHn6@zz&ee*wK4%u`(GYkX=G@X`|Cg00wlarAm2{$nr|H>wJwxTwz%Khqypq#s7a1JRirkq7aQ-f8tXpLWy^ z8PxgTed(LHU-W3Xz9FoRu2aCOj=oZicK_b+=38Wnzm44zWU4pz?_a$nZ{N=?EsqEB zWSe~&!ag~hiUu|%>-0LR#yfp&8cpupT2cr(=wZG45qYQ6`RBqGkjN3--kZwdoAPU} zPUz92q}x)sHOr*v{fXSjRD9T4pCNBU0aMJN%$HyDMdE3c){n*=$=eB^;D%N6R33h2 zam`nG{07Vk1tdh7KI=5MEqFFo$7nko^-I)|cD8a#?|VqJWZ<-MeRfK%Dhh3?OQNr& zLPE*?N-j@X44j+9MPMWLg*P2mZkadHNS2fq2WZA^DjKGBY3kL*yz=?1chH|rTgAI; zzZ=5_90fjwzo_2nNnRPgnZ?Vavvc}$BQ*6}prQe9cTXsF6n!72ICM8mxA2X2gPf8L zC#lt>7dG*3!C##odeIl#bn-x*R*JArOVR0elU6dh4t~c+XK}Nhep1oX4xK!>Wft$0*Vu-fR6nvXW;cENQ(zCc{_l^hM1|MCA#d=ciSKA8jPwC!W zcjGHvi7gVl%bSdA$jPY80sEiYM~FJg_%I-+Bm{W=U`>|0`Y>d8DnBGuY?dG>N(G|u zw;aKp|HT(#cPJOx9=bl+nbDYUIp%YBJl143Cil-)%GM_Ia3?~JQOb9ISe zw^2pKJ3*`KNCA4Ou*4rC@Kdb*KOAn^l3$SLFckHM;k(SL57lC9W(?Y47%v?)@IV=S zs72{UDb`y+k8dErPMjGwbc*v)zJ%a+I(l`(gPT?WpVe&I;Z-I}@T2D(i_>f#ZXeTk z6kKT^lTG5)>L=hO=QEzst=n!Q^IeToqedpQA|@T>O$ZG!^;azFQDSVeANs|KY;~}N1I_z}aAw#k+!CLd072d= zDmVfbIPpp-w5%yf8ELku(zligPso*4x#yPqCP?vQ+n}cXd&eoOfy&t&`Gabi`D3Y@ z(7Ps~@GBBo#2LXiD3K}ACJ>JGJ_mwQa${z-K}aBo3WUoGNJW8yLC|!rrvB9C#Qy?X zGtFPOdhqLfJpRwxm*Cqb!ZVL!>b=#`!LTETi=U?Fb+)c@Ip4jRHEXDJvlHhAM^O&- zoBr?oJ!ejzFO)jy&kSkz(`cj%D{p~3t&nQVv;{}0!?;ySxlqr6_(==~9L^om*?S}H z7%N3o$MBPi)is30kJNqEy_Jkp0+w2pLxayn?Z#xxyk%+>pi4HFu!PWID<`*crw{ot zot@lqYs=Ei@H^{1W%7{Jy7p5I5KF8grw^uIZW}!w8TNWPM<^X~E!{62pdK^6^G=AI zPqOhe-cVW@KYzX3vB_RxWFEp?`js6N3(k_a&Z;vS_d2_>yO-~{iR;XAdeqDYqQUej zF{`DZT62zNf|r@6F|_8<9@|%4ANR%^?wjT;3iack_S&{<<$eUyEf>IkYo$?^zY^uA zl+wlMJQODq1IJkr_&sVQD|Yu2(-jMq*7PhZ?$PrD0BW9yQ&cczl1SDVigAKOIHxnF zxe{Zfd(%@=xW7PSP}NSZ+xdgiX&>L}l#f4nuU*5|)-;F}Ell=4pS@RmI>V}%J!)4g zasP3y*5-Ny`#dVON5<)m%G3jbaoGt}62#dlIl(5JWlCUW*fs{BjMaZbPQbSXIN{sh zn6^7^4w-ywS#LJ18}0~~eh%oNZo|j!MjzgW7rqhmb^1LL?dInyRa?K<-tTt9U&&nD zSN}cjD>+%bDP#S2>h%P+Y*}b*o?{R3De1$FAVik+`b6Wem}$ozZx73j6#~&*DnHJL zUUEoL`yNu(^OLW@aW=B=1#I3cs*5t1kYBJgxMppJgTe>|{B2Gjk!8fkTzla(}<` zg}Z*U@pCw)e8hley@Vx&_Cw5i5*JP-J-5h^Bm^LW&EEDUFZmm&%)8^h`?H4dA6sjK z*Sn0pIGb8vn=T2qOJ-g)8?2n5%0h%*R95Ke`n9^k0U<~IE$(eD=`z4uSl|Mv@uGUtE8bS`hb~h=a-*UrcpFieB5lbZI03=!&D*<33`Ue#E%f;QQEZv z!s_3N-pi7_Q8wk*SjIP3_QzaH{~+cDTXA|*l@GYhHs^RTbG+|$ltxI405`AG!5rC! z*DA~}mb;psiIHL3lFXyfOTb}kQ0irCC&n#O!_ZHx7Y`RLyO|WvQ3bPC!haDnN4^_= zsAIPkN@ODxm62>jf|iv9b|`KB2C(?!^Hz;v9LHw+yR2mKO-2CGKQaz~>V4_>LmRZ^ z6uu#KhoA^Qkj>G51ja8>ey-#P@9i6{Ld*EXW+eQ5wPG%G_Rlvm_V#Q}I&AGW(KD*9 zU2SoR5GcY<(l{`eTJQHMAevjhGQQ0s)j0b70~Ix${$;m3aw1A5(b`wWX4$+{F>FNU zY%nvDCerB{T=SlfMGu{p9f%oIQzj%;CE92>VJt6fx}I{+aJwk!@mTSc70s{_jaHCo zP=~$_xoBNm`LC@an0{0kF6!g+?q!kp-?XN0amU9bBK>y%MzM6@VZ;f+&HTWZo)7m6 zzngYE9ssd7ItZ!-R*9jZ<^id2K2~OiA``Z-qT1U!7DN-OXvL)ai9uAsZTldv&6b`fq?&W1err%-kazTaQ!f}(uvjN- zf|PFh<2F{55!-EM61t}98Gf+ro`iL~znU2rv8@4_V(@#3K>B@fpdX=;VSYjYTa-Ee zJAE=cR^NKoxtt4~OqZz8K6n>BD10<#jdm-fs)i0vD%PQYY*iw6>td?!K?89KNdGu$ zaAz3B6!teF{JQXtfc3iL$kjXGjVp~ZS`~*dxPo>jAydVY{2=@mSLol^vz*TGR}FhC z_5v`uGu)GYmTUyQ55YkBJ8VPNzjcf*5? zvUvTl2yphJ4gcRv=btRDaU1mU(e10Fkb_v=ym9Ob-KrR4-~G!UH*v8F_rLqJ@AvEO zekyk2!+enrAqLrvbB~_V?5hhyz{{5el!t~80Gx&pVuSr%ewuzsZB3j>kWi-?&*5Ko zFpj)?Z;qaEvt|>ZaxSnQ(^%32AT;mL_L6erT2ga|GgX8Uf^qBJAA=4PGHHzxxm9Wd z5)SeBiu|>30QtTtF;aOk`$W#-_)<1}7>-7yxgT!{UWM|M2(~c)=ym@(s+U`i(e9Io z=9R4J5;aQH57Q4GiyJH0ZcS>gX+MSTWJTuf8yZ#vgMk3-*o#jA>{>PoPQX7eISxk% zNifq%9v1t^E0GH{+(wwSZ%86YHfaX{4jSuc3Y+iw*7EX;7)-Zj@5mO7hAaP0#iR(+ zh=8NqGWN?}_jY^;y?6R|E%XnM<>fz-)!X?}ac?Cg7(Bvj*09&7eJ=ELZ%0U>=kK$Q zu_ug&E%$)0xhH#eTr23L>`jF96Dj9_#N;Pw~IXCnj^xeIotnZ)?q zKJ-bI98?I}Ogi_YpLe^|1~$v+h?|??-%H~QWjcc!ZMd|4Nz0}&9Ga_;(vMpgw?&J` z&1bw=`pOoHgN&V1jd2+-BszxR0>=z45~G9?I9xB!{@6vguA>AXg!tX5;Vf_lQBcpa z^yd;$;fWBqX;v`36Whk`SmXd|ZRzC>K5&nS_7PQfR0UVGd{`@VxP6-$HL4Nn#&~%t zd$=bbq;&mLEd7kmP0vs&-knCs;q`XYDPtJ|3@Eve&Z zl3nazt=Cl+O^yH%&=+wg#;I;lJ79uCD9T|PbNc9u)c{DLS2SbN)-IYM$q6@5&LLz# z7L9JfTtzSC%U`g4IFeSK;`&+j?0!)%$hPB~Ux2w}Ueeam0KK>F<3RO`Q=LZhx@f+- z08%L@hK%DM3}1tpIesDc7eVax%Y|YK^Y))35`*MQ9t)JYIxZO|l=L8gm{AdlMrTLT z#-}>W1mYLj(q_a~Vr9;aG;NIa-b6-9xTEMEGBZ`K7*rl*Uebix?isDL`Ge}Hm*2J> zNo$`F)$q?ywcdNX^+aB*{p-FzArAgL}RPMsX0r?`LNwxX5Q=QfY%HD5#Gz*g)J9%VNz0(NqV!htSf6v zVG}9Yj-0VE;_-qSDqda1*~W20hA;dpr;WdvDdGJ@wpb^ezSa5a)@1w9buM<%uF5Ct z%Qz&=wd8u}gW~P}P-?3N7GIOn>RC{%5!7x{rI&Kha6Z0c#HxS zp?MDRt5tNJa?6V|X{`vA06T^_`Kpxk>~Q98ffmR+Y@ESWIJn(Xp*W;j2-zoe znO!^Fas@K)a`M+_cW={u@Ys!4+?I$!XE^FHZJV4B056sY-T#X>_3XiA)@CF3T0&Hh zhXwp<2*2CXA*_wOznBdMfR|bd#2ZBB#7Rn`@7s^|&Xsq*S4Z|9t_s$US;;z+*cM`I`NBW>B)>%}u2{a~6n7BziuPr_2yL2MLss zqRfwn*zq8Krp3aSKrGPwNX z8_>oK-<)>;n)b{=++{+W`GvCths_eaGf?Nda0<`{80-g8H4+Qr+0MSQp4F+QJHkG(9jMXomtJrRVaC7bxCx+EF|!URQ0?zL8`8#=&@= z;y#8<`4-+;WUCW>Q{2wyM|3U~`n855Jg(W?yDtq_g84mn51|HFLU-QuHT==7v&LH3 z&SFzzb7yj8anj?&IWEsMMaGZ&2R}_$4VUt^1deWkEJk={K8>9dLer(`P3&-5A3-qt^ zv>KA)D3M^!^&UneOkGlmSTcY1-vDpNLzY@rb zkwchc)sIO1*l=)Rc=3DBRrtof!X8DifxlJ1%c^Y~SxCF^^SA|;&LbHGV{uE8UaKMu zAIz6!Sw0OX5{Fc!@fD@j=2W}m7P)uR>;^!TVoga%fb+bA^{zos%G zov`92?{{UnzpNk7{}XfX2;ES$CckYrud{omkXL+I zQOf>}uhRRa*~P{EU<2Lxc}VoWAtMnm6^9=C{R9D&>|hjC3Wy5It_B7GiP&(Ba9MB} z@pu?b@gJ4=#Qe1ILk7e_GySTw`39qx%Hi9GZ~r@%E)i>XI4@mAHzH{zUv>vx1~*;v z1X~8{`q6zWDT75OV58S1&4~aYwFa(tW!tSIQ#wG^HYXDs>?&G7%46JjaO~`X=EV>@ zzG|Y8RyYhcpBd0a zo&x>)#VE^!p5O1*}zGIdvKJ&N@^e@FgBr;h4T??hnvvNZ0QBQ ztWzXi_gk?K$qh}?8r6SdFrc5#@wirTvuLtuP(~ z4~{MVK$-*AL^C~PQs(7R?FS;1&F=5KKh-P&m>Bl*OI^3T!l+B8y^Ej+ZJ2k6=`2(FH!??y z>k$%zw7LD2Ry>T#9PgCjBSjt$XR*)@UafX*hKKfGj&VlGD`Q&Ivab$5QEFJzhJJUy zk_s*D&x7e4!CBaG`u@eG|DY-Cl4;cWbR0SS$mX^+F~lZD;!}gGBmp2_6&u4S9Qrt1 zFr4nenf!wYRTpLkU$6@FJwiu0fP!3$Ee5AtoJLZ1Pxm>?N=z~&E{-DTc>E<9c_vo( zUu4q}g(nEUBnZF*N>~ug2W~i0hYE7pqL`a!^|!XDo!w6ku=D0ec|qJWs!QK_K|kPw!z>FT4T6fP!} zFLuX}G7}i$2OK9@s)JPwYn;U?rsdW}5D3@O+^LF9P`Sof20awDu4JmoybQ4s=~@<% z=n}z~U?Y&@{7mppdhK6EWNHgZ`1nQmxN2bJechv<$;+#)?eH*PRssC1b(Vi@ zv43)czTb{y@JT6BIIjYe@46^@s~a|Q-;~qQ>uL6sl&kO0F~kx9i6}YMN}Pi=xNpCj5gi|)cg-9f2R4Qo{}aQE&o&D9a`@t< zlM7+9v)y6WFHZ^u&8r7iynl`n91=mmgh*TPO2_xtmh z6mW2V=E7nA&DZYErpe=|c)y?5o#{-G!k2m;le#&nhmA&>Nm&QC^>2- z+e1ldEv9K<_81}s?!$m(VwZZQfaG6@orkrwn%Mpv1tEs;LazLQPoj z_*-5*%L+Ta8Gib`_o{jPokKg>cCHbTeJo86x(8nh6Ix($oj9!xR<4H99}CrvX@l8&2~5i>wpjekr+Ga9X*K3q65Jd~EUJRv&2hE*DV^Ti@{5&n~ToOKlgz z_d^f*$#H>H6Kt?ujbj8Q2scR;p=&S2KxeYqGJl%LDfnHaamgb8la*lVyej?icPE!F zkK`TNLk7bxz+MU-zzu-|ivxnYuBdTgk=+0Q5QQ*AwHM**gl5T{j8WD;TQD+EOH_k z=FbFLsW!xw?!r=z+A&;?1ZEjlyqxT@s8(u|UhLMuRDvN-pna2taLtQFNlN(=bvOx{ zuZkEr!VFh6Rf1L#kozRbvL)r&ljDMl4xgA;soOx4R;bVilUd8m&c4CTc7j=<3qtK? z8Qd7R?|od7ign6&S)NipHhX$pP;xmkxlyTzZ4TD!@6TS|2dq64-bgaDcZzt^@W=pu zJrCFS@@>FA2QG0|J;;2`5Ox!`{Iz}H&C1iDpbN|GD9ojkr>AG#^QRY6O|xsA5x;g% z&p+k`ZT#aKOKmm&Zyg}AkAIbF{L!iQxU?hy5TiO)j1Y!Om@M<1-I~MeP>tYn#6+*i zue=W}N0-@gb4(q9VyBnf@ZEkExqI3_=iX3(6r8<4jNX&|aj2X4z!v-)Z%f!~v<;t>8 z`JA#z-?2+(^lLo?Z?p@)O?pWaYT2_n8%FX=0Yy_Y_^g-&^$d!bJAPkAkD{y@uhP)9b^zAm6Ot{-5gWi$CZWd`^30aQ#7{ zkGaq|=^v8-ws$S;to0sX9cuZP)$}56Ykf=#s|Nz?2lvNJ7!Q$;5rcrZY1z^Nn?PUL zz|3Y}uj9+z^tiLPlXG^CAv?JCN;*Y7I2+iQBG_QaK#?G& z9#ivL>z;B0X-rpTwscfIOMWWML1MicuS!)&pQyXT^#Aen-r;QcU)Zn^J62FzgV;rc z+G?f5_@bz)su|R#T6^!Py;FO&)!KXSEvQjyQ>$u^*5=LcdYvPV1?sLw4 zdx*O)#K>n61~bPna!wFX40|x#p7kTY8V*zA1Z5Gfk;bWv1fnEc8bjvx8T=(~bWdFu zV=Ne5^>qpLBQ`rL1u_JR`H^y)O8U1gjlU5h2g&pIhc zLcZ$5v~GC7W-V~77mM8g&d+mY+M6yFA|sY0!qig^9$L??v^591k7aHU<2-@iG56_T znoGQ1YHn}8=p<%-y&qrEcJ-&~$6eVmdZ$*ts{O9=pZCZ42xu^mM9csii^Hy?b9s&Q z>%x7R0-S1l=;UmU9xul|@xs-w*<N+LyrRKyQc0gxyX zaIW4cNJWCFgUL{ogpiOJNQ@$<7vZZAsvtK$OR^}=UyO;b!$QTkltHzfz2U+#$#h7| zUc5Mc{zs*ivm(DATba7O;3RbG>$9_5?X9(TnD^$^cy0w|BZ!e*1ONb%x)nDodtI-w zWtT5k&JPcp@6P=z>T=H8Y&%9{Y_VQ3vFe@Xq3R}sVMpL%dUS$GG2L^6)XBHsqmz-` zsbBqvpA~O`N_fi*>L3EC5*+W{Hh;~>a`eJ}vRk?M?%@twul*HV9LlcS@9%g2EZ2YA zkcx@r@p+OvFQ8nD?6m^C|A;0jg1+L~#<-b?{UG%$v`(_(G6wT20@Y)Qz~E_SWNcKP zO&$@3n(?ccySRWl0e&zOljU$J=F&%A_WsJ{F1|zh)FttHVBmfh=eqLbc66aZE$`&@ z*`%h({)0b(jN7Lx&Sj=U`~RAam*SawLTj$m%0yEr35<6J)97@F(BVu3-JkO2dgX|d z(K8|Qs{L*@S+@OdC|T#9F|?VZ5pL{FZkx=nXO)02;^Lx`r)(ptPqh>t1+;5ShrJp5 z?w%&(@_N4Q%B$wv_|K&T7m%>1%8BXn_ zETP|E>o^lnoySV8{RKe(RL8il2JW>vc|hh_SdT6DM3p>({kv^G@+OjKw+e6V1OXBa}%7p*0p@q0k4xK_oN=oXJ&>l_?L* zRV|AUwSLoCBJn*(L4o8;?f%zUhkze{TT0Dt{IbUbtN&Yv{dg?YH4C;R(ZgngkuP3K=YF6dO-ny7k|(3oG5LNLzfRgMC~%nYbKG zCcN_BxNX77jeVU!=l=_h^d1+E#*B4#?x=l|9dJ3eFL}-`-~N>_2_F=rufbe&FuJy3=vTGZjk90};&%hKQ*H#lgp6cw9JP8neQaYMb^1 zv5K#5vMjsQxebZa~Se%j%K(gNsKMaK+OGFqH5Fy1w-=KkIJaydF=FUp7dEDXva0V?S zg_S)<-@~jdHJvKp?B1)_#eHo>cd0ZtyYklCe#TyrCv(3SH&yhPtLqr|XjvO=Id6J@ z!n`2!wzfviPLVi?gOq7744V*U3+7NPBIO9P2#G?8KKdE=>R_d1BvtSQ<`YIUQjIl! z01Lnuv#erO6}1mg){7+-`uO0`TZ!|2h!${&DJxnEDHpA8w8yzfT^ zDTW}E!blPzA$uSS!Re6SJ~>G6Q!cLjS9qive1xsMTnQ}j$Tz44#OTl)YA)1?6FylH zjZ=gS2cqzMlbF*CqD(Of%E>_sZ6EdGeBKEZM$sj9!#jDkyQ8YDt&pX52gb!lyNc?zm#+NnJF_4(B{y|W zpx-&yxaMk03;zz6S7$w1}b%8UX($Ns!$YD;iI*&__3O> z8p4c%PB5e3qp3}VCq+MoO#~kkGzGJqicf7RW+(H*lMR!-Kb8b>Dye>e7EBt0h0r8h zZ?JzFYwY|k*GSJB#3U&_ybi_E`NFXqEY6$BvK|+Yg>S-9T9FYRi%VbtI-N`^wc94k zPRNK-vs2nM7aW9fWx~n6C*fJZ!34q3qA)sqgk_(ivc@9$HZSj9=|4!ag~j6C76Ehj zHT~-jl^vhfL%(N~k!dV$Yfj%>*SgPtR@_aBunFfx2TH!_c<0gnB`O`!LbWH&sfZ2L|QBdqx_=o1)Y;1o{{fY2znq`$T2a&|o>m zi#ql(DT(X%r1Fjss&sXVMiFW@(=-$l-#2-!f}Cu<2bx>^AA9+FD2pF5rd^kgj~%sB zmn8}v>Ak@uD3XK>;I4xc7^xvDq#3$2SPIx+1k9C-J}49fEC`Z?93c*LgPqZw$QT!O zc83IY9te`myw(uF<*h|Qjf}zvCnex9(G6U2*PDbq*Dg4@8cJh`OA^E<*5h<5eK+{D z7Q!WF#Y12n{Y8n-ZjNymS)W&@F|F#Lgwh@Eq&;1vRt6`klfR*v4pTi5VnP7R2q%)} znI9*eQU~7s&KmbSrE$q|qAp;OF^1MkJ-r+FkbO`i8F<~mi;Ggs^K*&a8a1kToeGx* zE3}sIL-vBnXLefd+|UGl4K*{OPfZ2f-N9GY@jrr~M9O`!muPM8=JNlQkP z7_%V6)HPNKv9}hBt*sGDNRU2wZQABmDAeNg_Wwrf`)InUz}|kF;K1LGS>tnO&AG9+ zbAhJ^Y|zx!wb^URRpAg}$5a(`xa2{Cb?~A)VkbS7v}X6IE)f#^$D@-;*Rs zYTjZP1$n)G;XB_)qjQehi9)~Ab!vY46JzAP1}Ybe>pM5a1<(9-KJ>#Lc@@+%wVJ0@58t z2*)F)pTtj9?#CBOL*bVxL2H>TL+COR5H|Su_}xJWqKst21croQ*#pAA^7V|JA*XH| zO4&bunNv#DjUsq50ICkDO@?G~uRc8@RmOxHhuVFWQsvxwfUDuD%{us1ZdX|;wW6ztfe&z@r!e}*D9ExY1hH}&d|1beay8#Qm%A4hcE`)}?M zx07^?hj(ODJKUe`ja`Mlev!k}{1)+_vq{siWtDl3O~3IFP(+o2AoSF(Thc*-hP7h$ z%wLT^4x^&{LG@d#+ebzRYi3*H?2?I${$YK!>qpbU_^4?+%a6R2|EQ#X%0gye8AJ8D zqNm<_G>jfsC7&i!{J#YG`=V*PT*%>euKb-sozbJmn?L2fypmj&eT#B0zUv&htED%d z?C2y`1fUsNYFPQ7MQ_~m3b@+4l3yfeU06aZ1RHg#)f2b=Cvj3MvoTTlI5KkfXDLo> zq5JJ$@sT!`RZVzofKk3Pq@Ps1^Z~()w3DOJHlhpxU3!adnxY7oaCVK9(5{tP{}A$? zh>=Fsr{K|-(IR>+^?JxsR~Q(^TacgM39LmpqbOjBKs^90I)Mq11c!8n!5Dpx@P%|q ztGNyG$dDU^_{dHm5D|nv!#7Qk)&npJP!{Lq_cY^+Po`&WMiX0EW88qrbx zjoFx2J*B!1D&VI=w6k)O_7tU_0z=cx7OIkuI(to6Fy@9NkrFSAH9tvbsAI|WCgJEe z2;k#bm$5P96Sg!2xM`c_EvV_;DhMIG=`@X}Q^+fVOi-)(rKOfu= zUN5t>4Mj5nO`bF(W<9f9W@?SmbfiVADF6P#zf3EQBG^ts=l;noHpvIh2%x^EL|t`x zDt|r>I-+=}#&v$a7B-^LqeZ-*C%8q>S04@qsk6fZO`EtI=z@)3d3|n)JBFE0H_1Un6D_hIRr` zd7n6vXxs`m-V0PD`0@A7rpn>XXc)^Nkz@AWy%Q5Hj1m5ZvbxRPLpu(vuYJ;f-0A1V zRT)N9cShuR&#ww&Wh44Rnh2q&Zt?g*LV=X>@1Ef``}|u&SY*Ok@j&pSI8HClSDa!S z`1oLyGVc8aH+9NP0?@pAilWo_(a9Jj5qlnjAE`*$rz?X)c`!XO4FY=CP;t26PrQSu zjaQ3_p1NctDG;W{e9jHR);!>FXXx+CKgYek6+PYNIRd`l00dhl@EMF&l$Je#n1zs? z{-~X-XoGHg!B}hw&y1{LjLi7-d;$~yv+aJl$M$7f6P+=6$B0^0Cndh78j1t8ftvI* zllq815q~}Y!^Zw*n703~_?ojX@Nj-}y8{Qrdwp~6+8I8~$=jD3m&XoTx3WhP)Ir5MjU;pR_o)^$JIna$wK$Tp~-0jyN_onDQofSdpfW7 zvfNnbUQsM6EKrlu#Fi>~Y`&SjJa$kxQ^{?)JsH_H@p>hITTg1s-)(70uRLJb+EVbZ zXFC1Y^Y>^zlWHeT^T|c|w)a0lGt1Ldk^BcxRj3y9Gti9NJA74;)YCdE*^%2mIt3tD zx(E520(bq3@pQYrl9iT~wsJ9Y_3`N*n{a1kH7Oi==`zOH92_7D>1CuCdim%<%2-Ar z2e+N>0?;raG!q%z^yM+N>a46GLcmq2IXFpzSRjtR@G0cEp0YsW zNuclvP4oL8O_CC#g{$^Oia}_|Get5-#zALAA}qZ<_cZ_$Oxnc31kcwh0N`UdNJR+M zNh$-nZ00vvGBFhjWW%OCJyCS)tQ(78-1B}xo;eink|p-rWHC8d_K9(SYU2MeCGr4h z*_Mqlww`#8FzC1AF^0`(#P$*zB!;|E>hY?isDIR0^)Ag#d&%*qqiw&mtH`@s0sh;i zqs@B>)`P%b3b&&xSxrZ2ztjE&6z8b?seX*ZizC@=1&=>@qvHB$)qgGj7pWE*_+@z| z>(fQ&*3I4}R8I6!G8Q!!L7kz*L-L50mjrFWSZ~v8TU^?Y_F_J}#x8VC8Zc zq=zU&1DH}vn->q+ZqMo>M^>6g|Nl@f*>>FR-d|d0|4Em-4y=^U70LbYV|>GJFS^6a zcH0w`7lw{cuHaw4yQ(H#0|Ql0n35z@vp$KEhwu_5aF87kpMBdcB71Gt=yvOkbBrHr zTzb26GNt9tB{{U|kj`IxbTmN#PsZ_NHZkwJeF^AG-LYYrm3l|~Fnp#A>388I3G_vs zt*ft<`np)TjntcAe!r0fH6ixJmU*gfpC3QN4u#DKRL5cQ90tiW1_3a&5Nt$BP$Iu> z-a7~q4AxRbak~GOb!5zAQSAJl3}|N@q$ADX?)Fj!pzzU6x@1ipfC*O-@qbdzu?j8j zSev%BrASLLiLIsrEdk4CWbPCNME={TD7UPVvPmj(`%YF=gVFcD@(cT4yt5LdgEKz$ zi%W7Q&EsPx@a-u$gXy+~FhAU2LF1ntKV$41OFxrE1x%>S%4@BAzWIDVrEv4`iAOrQ z3q8e$HGlQC31)5r4=PF{5W(lq|NBA|G+0qFi+yi@X|eN#Y-WJJd%(YksQIsn21(*a z)exY2s8fuiofer%|97gv{ZE+Nik`7b_{E{vMexkmGD#M8c|Y9h$`e`tagRl6j_SwQ zXwRh+`Q(QkrXmU72SV9Mk=1t1+4m-GKG|(;Y@ggv zo`+E}&@FU!8=FSM>cs@XAGs={M~qiFP1-1j!K}&mXU+*-CW0jp3KXG8_#_k0vqBnZ zvDsv49w(NSvom~cFk74kJdAjY1^MBDq@mP8c)YBSLW+7c@_&#DYJbcpfyc)5vZj0o zYjG05DM)oKSr)0B0@NE|2Z7%ZY~VsV5iMvmstx@a~iVMsn?a1 zYuSDDC8lgrx=|Kxm8B}<4$gR0JQI;lsqR?+3Y%Td_?UT0ZriFc#_gkl!&9(+HilX1 zyD=9WjhxIXmwG99dYM)=ruE*YRIHBYL%>kw;*ws3j_v#*_Wo?DA!wuf)vt=%RVsFz z7xlamA%$RgVW`R?`MQVYM_YRbsJhL@;x87f`WMS?fB#%MRJ3_eLZMLuVT+;+8WuTk z9^lIaaZ-;kwc5v?S>*TS1`SnIK-ZpJdtQH;TUV-GK7Qi$`%rp^&EWY2C za=phR1J`N-?}k?7H@g?K|5q{#G9_JR%q~;Snra#vZV)Tm$5~6K8#i2YKUqAe+;NkQ zT`JSfy}z%tqSQJ&ijIk)73B6OMgU6PK*B0=e$=bM;pMy2vu_Ti9z;3KexxcfQ-gQQ zPY%LQYiP$Lw%=KINjd$Tc|6jWRKDvbY0NCp4eYn#fhW(~R8ksLwOjpM0?@;vWG0P! zWCmLBp-k!tUNT#RVn|6=u2QZu5VrF1BClOiEKL6RdH6U;3{!JR=)n}B}FL66Wh!KdX{>3g#jy}m!V zZ}+P%8C{Yo{hJVz!ChxKW3}m8QTP--CiYxiMD#zgGNN0{VC+vv)&Bhlm@h&ZpQO3; z%Xk)=I6b!>)G||(I%o8Hv{<*yB~4`5k5m~Nlcg3|*d{(o_bl8EbeU+siM*!=GAzTX z>OU*E+SQ#?TJKl$eAw)~>*5d!{p+Dp7MrGCs~RmKdOwHzcm)I;yr*1$m4rZcV_iKt zpW0C{KoY@voJojZ3)R+s_{YhyCiR9dT^h336;Sto9!H}Sp|qA+gK;X}=aN$L_lH6H z|Mz*a|6F=~Tlw_zZ?%;Tm-(s*IY!Jh=0tF=!XWyj-j6CIT!}190AVfdk52t&?K%sl9YYgIk>9$8)pY|2UZbv@>~0|G-dcWeiIi zEIQk7#Nwg5=i*>h>)vg7mi)zza{l`9H$Wq5WAwYxU?P|j$YL?<+PPLqr zoMIye#UsiK7o??5EsiJn{|wx%sk!93uL;#n2`x)D{EL&B6&CwBw`9%eBe9V9*8}y@ zmeU6R*#zT%x#e}vE#)l6W&I6|3341Du|(3e2rNty3WXeJT*MWQqz49G|DLEw(<{f; zeT|_V*xqKH?!T+v_LH45=Ar?d=(US{+Bfq(?qj$=f7Rd|i$g&CI0W=RR$OvK*3_u= zTlU7vi?Q(^e{9d=$Ej8t-JN+e`Db}qmc1&sUUtnE9DnX)fu{2MRBFESJ?xL%MX`xe z8BE|G3S-Rj805~SmBH{E_?p;>lQT*t36Qk5JYmcY0O%csVb)BMgh~*R3~3v-GKuK0 zm*FiNadu0oqES>geA=IRC;nVNNdNQ`M zqrjoD1F(D56WPz&6zMdHwP{%W1SVkDViL~X8XXjau+b{3S{|E5etLjV6zz$2sPYtF zczbv{%Fy9^ntPL5;ks64^X|^F-fMUXdrjC987VhkF0CMKmBnK$ZVZKNWeof{S&eT$ zDi(1(|M%}f*(X~KYQ+#zPYWAF$kSC}C=H{q3LGTXzJ8i}ej%=)0*_^N#IZh3jyQlt*U;f;UeV;osfjH7lu8HGFI7@Ag5m3xC5t|BlRwMv5tO zxjUsGLm*`;a0AdWFB){>D3Tik0b&}NI4po)f`BBcii~JJQ{8CQ zNLy~9Az15K5yBxsD1q59<_(2NI8+_L1F|?Uazi8mP?iVCJgpS<{wNYCF*hfS6;;$r zs}ThmTt1L*Wh6kqS3!wudVLw{wBpp%3@Wu2>#!^?7y-)`+x5h6=v8oC>_;^}DfpxSpJQ zLr2A)qw6r@_%ED)%#4sZe$+_<+5TS;Obiu*bhQwW%LgHQ7I#`{9eNx3Qufzn`xJo8 z8fHdp`=`t2_ptPGv(4=n3RG_;*0C%rt)ma`jvwyd4dvd&kh{o+lQXnyIxpY2 zoLlewp$gEJ-EPrzD+-zaq@+6dGY<8Pfp?PF$PuO0p=NDO@?;>%(3^+tI z$yD`FyKrI)O07N#`cavP)58aV5nxi(Dvz?NYEKyfw@y)~ zt0Ia~c#!Tvh+Y(uI)g=sr5x`uZW(Axl`{nmr-@)t*=m%<0CG0ykP#Rq19%d)H018&4reOcRegMZJ45Zmf~n1Rwu?Zuv2J9-M+8?>DsL=x?IINJTVr-*)|| z5s(f$_O^uT9wB$XM{zYN1|kQoV;p8Ln{w}%0yZL- z#|IRS?~h6X?;;mY*OTLCo0pCid;Bsn0 z({Y7#v(Kf~-*>lEasU2__^%Qul;+;f<*uuU+|p&9qDL?JGxxQg;>_p{*DG#*Je0oO z+cs_g`@2j0w%_Fjx4Gf_zx+HiRdG9C!Nxaszh(LJa<-%-=Ewvz0dTaO~r$%eCm+h^+_BSutsU9sGU0H${hQEbVihbf`oyGS5{?a zsv1mF`^9T#Cz3lUK^6sJpdke$1>unwaGQc)#uO=N3=-tYTQh0QC8lUTW`)o2px`k{ zr*#G?jCwl^-H*W2ddW&!_MBdRn5g|J+`CA_KJ^Z zx!BqRN&1_r*#zNM&I!+gA^81G@E=9idbEpd-b%99i~G65#HVXE*m_)k>^r;_={Uc7 zcvG`-TLa#=MLD@tnS3=9{&I?_jczh3BMP_~P{>I2yBw)lxy&X0w%y>o^ot?zm|<2j zN9JOk!CCJ7+_&b(l5Ydr+s4y2(%DG&6-+yI-)3X%RyJi5EGSPkHg2XMyRz)N9$Qft zA9%AacRV~&;kSj`kw4bC9k{<+@%4D=DC#m>sxv+(D~I|h@d#P(4)H8E4gAx!(tLcH zTe5z9#?sLh-hjakj$MRq|KM(=EL>U?q$}3bk4LGnFDYmAS#xroBKPYzd3tx6!BXiJ zPh#Jbf3n-yoAB@dD=Ml3`oNY({NkwnJC9Pg_$WfcgQ}v+y>_5c8i&^Ti(zA_Q&W*z z1D++jkw%i0KUr|zF6EC9?P)T%a{nz5ns z>IgkE5g(gP+?OJ+(u8GLSaGtDXjl*)3atm}0oFZ-Q;3IhcT;zppp<#y(NH(ZgkVyV zY9dZtv510-b%lU>@VxPZi$&Fn&&YrKz>_8gCPJ`yVbCNfZOCC|FEtd!%n0E%=p;YT zHV$@0g-1#s4SQvmNI%g!qfax1kgv0XM&cx%*Y75hyn$~uy~XVUIr45;IuBiX6xe*{ zC?FJK*3)xttL%_HD+Fjh zIPC51zjgh~Zy0&)827N+`-~(t*oi_r{vj-PvCx^F10Ar<{~&S}v*|d6L5W8)D}lq# zwJ_qibq8i8V~#13#|4ZO!RRlsT8Z!t!iPkDzmLY$lc1tZUY1)K40`H_o@hYG47KX$ z2098D^B{EwoXmly%a8^SLhOrALl!HSMVX=@4g{n%%Cnvs>_Has!CFulxW|5AZ(Xg zjZ;W98)T6)mfu}ko_gW*{#JmYUGQ#o{={3d?PrV7gKvZ%ZeQz-N$FtgCXBgWC6v#a zT>Tl%loVV!Zol=PIA=?=XF?F^WH2Rh(X%S?Ae4D%QX%<=mt7BUx;D&C}t?2O5eRY3p7?JzU z>ZYxxX7pNncHxJQW9b0OKK#34$4_orcDcrZxt?pf-pySc{QArPKvs5IqUwq7V6F7} z?~-pO?QWsj+47~&wGUdy=0dMimi+C3wgcyw@dHs8cIA+?$TPL^&dTviAD-uowu7!) zP0wgB1rm(g=5bJs{Er#0`e4+x4{C#&o+k}}hb<$WMIlTF;m`y?$Qv|AtCZb$d+s1$ zNCa>~fUm@fr}_k8a}tIXbvMR`!;=Rin$}HSne-GLP&`|N-d&oJFO}qM3O2*AijbtS zuuRzDadl^Nh(^9wJSG^8L<0D~-w&|2_U?VmejjkRbZ)%sy?icv(DuzE+r(pzXpvDx zWu7A&Xw1iic=iD?l=p}yF^LpWTks_n+s4aeLucJ7rAPEHOr3KRVnM}>Bv4WW#db+} zxL#JizCQ9l-0_}iJ`DW2$V5HDTkK%2x$>R~H&W*~QYH)I0G)<-ceN_p>5?&qkzIS_ zcEr56_%_9DHjnP`_Tc=&7{1Wc9q@84S9_7&FlRC6C)GDfMgG~5k6X-j?-eZ zr8406y`T*-^~gcm`te}voOxf#D8CI|Qds)-+qmd%g!1})A+OJm@g128lA) z+hJn6sfa1JfrEeiRSMOmyuzjd57F+Yd%LAAwi|8(EspJoL{g8Uzlv>}^m^b>$qlL|X zKZY@eI|>V1&(-u9UO7$`y}`HXEJA<};(lnn=cO$*%);MdjKGMd4#T^)pPWmt-)la+ z+K~(hJa@dm2`8RCpS7-9I<;**xjSE9{=%JMZ1I{<+H};i<05fk{~>PFq#z;SdC)3( zv%Fi8oBH0T>V;}tB>$Gv)?XSJ45K4!Bvq<3>O;`+SYYqmPaD2+A(*8BI8 z=>?Ukq7AF_vQ40G&)Qa+%y^c9Uv_C~rmfEJNH*{Je2`!BC(|~p-#tA_HOo-^zl|Vo z57kGHxH9IkjCmkEOFRMkHAdC0L3R|d9K!V`Q={=&o8=aYj6-o1B34SrM}Hdq7EfJZ z5ZKe%NtOqNE9LJPDQiIv5ECy6QnbONf$*c!kBda;;UtT zX|@h%-*y-XFR5^};`ZG>E}!ByNVa0oj2tzPdM713)Cozxm7(sRGl+8a``9!plpEN* zCe)$U;z?Miw^ie8eK))VyB%$kf8BVR(|qx^eZ}EM%Wl8gsors{ypwF#uDBu_3#CJB z(HdUg)MEbKu*E-A__yEYOKSt^3Hqa27WX5Bst4{SLg$XnvLxEtwETGCy!9=!VlSsI z@L6lSh}fs41EQA}A9=Lr+!&nE41uc_4;5Hfmhoi|Zhn4QY&xy@Gpdt%e-o<^$l z{vrN;+W!iP$?9>N*te)^aFK0@Ty1=p#vR2tZ3r`uNsU2h;MPOP7|stwog^N8=8|j} zmtv=Uw5zc+R{M=OMy9Hz$$E)x$U4$8cqiuVUxB|}g45ogaw$PXeYSj@Z@> zRMi-{lEn$cU<9FzT}IlCOfN&l2`UCE9j61K14#JhGJ^f5SvtIT_Kc>5(< z=f?QRQx2~=6|L0+Wb`VoSFO@W>2x>mb0Zdv!{{4ZSpKqlc%AXTOVLx$+soWQ9=NvH0-eY}tFV<(tpU=6F-voM;-GTXS%o3Wo0mRmr(ofi`W`fjW2%(gX9Mnt6}& zEkhs*keswg4Z$d&(!W%b7p0HGzPLXnGU}oz@H7_c{Y@tcQv<^bQt?E)SFm$c1a6rmJI4MDQs=U{9iXUEs6SU_eF}h%dt0k-=%DN^-NOr z+C{P+bR6kl{2<6#-eanwj+B4oi(1_l(G@o99!C6&<5cHPAtK@;V991hKSpvh0r;UX zv{om=rn6a@ClUP_ClU!9Q-pI8aX*a-9%otR4&Febe+RAqpw83cq)1kZMkqvxuV9++LaFez1c}CoK z9mG5MQvp1yA|tLuM7ia24sok})^7|(3%^c@n${W1WU42I&fg!jeh%1q==VF-_$<-* z&(lmBi3Rdjai0$BGi_D4Dj1_ybMkqlT5elb>l9_`+as_=C>Z{WBA3~e>^berr&Ssl zvV&AZ!9M#7ljOs0?3u#djLYq%#r=lE`C)3hs+oUaSnT3rmZ^muI{H{J!_;RYto;W^jqmnZ#q}3+5^H} zp7@WIP`#5&*ag^`FOU!S5olz)Qg9MVN4>}CD?*fj>$MPC)PSovQ8$d30|H@F<@F^N zgQ-AC8PPCRDj$#|NG>$07mFlSM?|Ge*b|Z^Dyix&iXzB3iO_HQs3W8y$4(k>;xHsg z%_~@IO?Z*~{a5N>CO!gdb7%87%vv1>A)K#o2o!^envi`jY{Z`z(ys`fqe7S+&)^NP zw$fFGU;y}}{E5=aUT%u!IPQfM&H*6c;(MZR%oXDzSggEMJrZtmzEREF&yExMi)7_{ z`Y8y&q!_|&q|_cks@t_ zb8_x-<#KIvYsJTtniZ}A0i(E6te-1#7Uz4U*9)IhjJ+?e)SA53crRy?U(JX@41c`~ zTegrH5HSrKWonEad2}P zBDs8csjc2|8p3He%}WJi6d4&~=zQW7l)EQ>k}%+Kh*Iz^O1LLEUj2=K&$ z+=*ihzPl&zFH)dZBTDf|rS$T|E#FO8L)LEO8) zNi2$Q9_nM-kB4*u^!y?n%9#wm5=2<^xq8}*Em9QzDpqtWmGJwjHEK6j9lh%7b+Vy8 zZFjYD{ZZDsa`E8zXRax^^*-0g8M0AYv9`p_QHPFG*$(~x8&~3*`;*xKr>}KpE1wiH zjAfHY1d21?NniQ+OO@H~el_;9@eSy|>?0J=7K|O8Bl@QM;Tcwnm1=R%xhv*{sSo`Y9b3Bq@c}^mL`%;V$3#rfug2?aB)V7 z4(^>Tzlr0`-i?mL$?pWsDIs4~>7Z76pG7n{|Q;M3K~H6#}N+ zm>)udV1kTsYe;I62}Z_&b@X7c9v>Ta2xKW;@HfmmUyHW*!6ym*g_t!Y8ZNqQK`5z9 z#3#zl9;~JaR8=OUHtb{tJ!@Spvn;dBjQCn;|8u%>l*zH8(t~oJ^%MD}zYpbsJtxUk$W%8}rN!NHeuHnSlUbebD{%;`{P(Sgn>LAhAnY6$aT%+tJAa3uMK&XKhtd>U@DLyl3o_8 z6L6CJtHtZ=_k>ra^!4p+^PSK^Rk<2223-pXCtB02CdRS6k58tx6ePzL<{JOc<0YQj zij~y1svUS3aQU~Ry86-&F;bNMqaHI}Z+rSc*7JvW*ojrq^nEl;KE z==cvv4Cr zODGaZofxycv#XI67c+%6k)h#KDZ+;#VXT0}pkTb>K@gM!VWZ3+)!Q1xB#Os9sR(a1 z=+q^$NnpxTZW-vQs6fs}hqo(Nq-#;8V=Y=^OWy>1uS1x)SI|?gil{ zk06x{F!WJ)Vg{I#W~il=aVw(fnM1z8z+xu|V?i2WQT(f{OT0^5Wl0ee+}#1~>}1s- z1lkm_z~7wmwABrK=6-nk``O!VdG~x`k=Z{k$a*^`y(LHBoBhxj)R{ICb<&p?cwCcv zyN5pfb#el>7GHEDz`99B%i7R;5|9;(FA8p@{)!vvuUSVQ4w!w+`CE7I{9ZQp$99?@}nzQw%<>%2PNS7-cI?+(mfPsq(ZJYQt@@G3D`ZixqJ`>(x|soHa{ z((~S_lWaU+eg02#(%MTXW*Pk#F{9dao=`16@L~NlrY1?JL$TwMGs)sqo{z!;arlPh zJi;=MBnn}LryPYv6skN#MgTM{C=m(JKWuCd9ueq+`BU;3V??8@t+V1_GBlw^3^P^W zE?F?YIyVJZmdc7wjmB_#LgpHTSlP19bz62)(MKyF%JcYc7=Q}lJ~E4*x*T1ZaSsB#+jmGtcA=O+!WW>ZTlP zuKzcP1NX`IS1Ypj7XuXvzuE88iUUN8b4{%lMYi65YZ`$dBHysMP(9+jGNt~^%|u}J$cc|~hPbj?PP4PU4U<)H1K zA1d>G+;RHbLgifR$7Ykcw~znumS$Txec;z6iv`pr;{OezGk)=wLKhz$u6xKw45r2h zH4rfsDUK-T{L7H#$){rqjR{U8?AJI8)-#H#@~{MwaB|RSSR^(Az=`OgXVF^i|B;kJ zaKj^RTu;hDU`GgKWJafusE|hYYIZHDm!Y7HC{VxAkQO{79|{!@smjsidGD^B+0e*T zM;A?Qx2gf!ejObQT_kHFDRiY`- zDd>F&JXm(soRaPfg~rp?9uU^`+pfXq&-Ul%|I4itIEOUD&&P>+`^~kg*}NLxN^g5* z9;i~mJEy9DoD5hUx@2N5gtb2>(;K*|Fv161TMbGLuw(~Bef&hu=9G08KJ`mP{pYyx z%`uDBdZxmwnPVXp=Fj2lcSyTbW1hA0U&8v9KgV$rSsWMUn`7~E+c+@WU8vU4emNJ} z;o7$0(URj>KD6lAE|=RdwY3+LxrDCpKjn|lU38Tr?28#05i|9^2r8lXxF*o?@SUMq z>}a>~t+)34d3^IPEi-c)@or3{!h5H9_NR0GBdb;G?qF-_ggW?Zg}m1$YsFPfGKCW_ zSFXsfrbP1pk`yboMM!WF@+E>Z1)A8aK)QL$O2J-gJb&1H)cBE|U_l5z|3q`a6vZF% zEkbo9=K#bKcZGBK(+Co3koP3&eV*1J01i%shvjq7kwQg@IALNby03~qh61D^VJUnu zAQ}Ko*LMh)SGsbFo+zj_5x_~x?aziFgQ-$lzAHjBB^8FgnDb zp(+XgGiU}3$DN1#CIEBB8dwq?L`V+6=<-A>L5gRhx*)v-4ojdY5r)mZXE&^cql-}~ z4ywwlrrVlF$Hduh9tMk%ar$yf)33Afh#td)~cROSKt6+mzhBtkFg`fXZ5 zU-H$wjS4?)oW4eWng&0;V0a21*JvVvhHeNswGDkP&lfQxfPfd_H(j!2p4PDpmKlvN zqdNWX?jD`Hu;8+*)9D|T!kg07?$2nMQB<@Mu@%l=+QvvQaPOqzz@w$>{4Ky~y$==f zyi-)HRm$rZ-6E?X;Ub@NqjX{bv(CUO@x?{v{d|e|eQE037~!}EU(Q|w+ZcRj12K9{ z$q(_P7?Hp~`ndwQC>s8+J2`F|zuaCvr=_ZJIY?PoI=KlzH$1Lrr~boDdj4)_cD!esHw4a|9iJh7pconq6kZKj+4u05-wk~yWanyV2onBUPkXTQfMSB9j2C(VfyPO7Pi+iDYQ2@68)Xg2`9IG@ z%eevSIb%H0iW*RkWsWtd0AVIiEjEvy&d4c>3}28illN$n>vFpd8~AP+jw59+ravz~ zVj6$)y*|CdxJu$bmU+3U^i)w9J|-?L60y6Dl+Ob$?Uyzu@-d>VMwl=V?N&q!0iom? zAt&y~oklYd6UfbcR`Pp@>iV|FQI);cU2X z)LDp~*jsExZEBSuMO8_aQls{&P3@Hst5#|&TBAjOt-AJs4!oqr;B9v;(b=R-9tm~Halh?t#Y&w_ID zMdLfE(;i}i;RfQJAW3W9_0Ia#hupl;`Um=+_q6Y1GQZY629Q2&bz;ifL2|3wS`zvc ze~Ioqok%CCr>2FeQGGoWpaaP##aY%-)|R}262wIsZ=^XGfzaWzRlGCrFKOe@*Ecs>zXDZX? z33+M198}V0jp~<-_shgGw$Bo1c40!sc&>F~Jr7L^Ls?lkXG{O97u!Zb@haXEKSanc z=P3R2b!iiqh(uEAs;Do_!mQ&NS^Ppj$ouVGSzcW2@AthbaqpfU%KkkuGd6h7!NKQU z^^b;z<(q4g*)WgyL1%;VGc0-0^Re8z?E-U_jTjAM3{~7`fmF!`I#qy$Cd37w|f)F0h zd9F9=w$v9+ToaLO!txAsK)9BrTl}B+9RiU}VzZ0K(pW=Sa@ivKk@9Hbklioaq+c*b z{j3BbG%~yn&Wup!x3wgGL2*w!ewnf+yr&2DNmb|#HZAH4H)(2So^dNpe0bVxX8${j zk@^DKcVcW;5xjzg(pmKkkLK^zl2@~&(qyag1$oa> zh-(9-z=Y2)=(wk|_?)F(>99@NbFQnAg`3@`u76)vyP8CH+}{^U7b)1Rr&`yNR0SLz zzjx0#UoO;4%K3lSvq|Ol+(9hyw!1&%cs<6{>Yt6vyruF( zZn_QEJ1wERB9CaS%yP^+7r9sofg8g7y3K9zlZ3m!`wM83uP^C@8=bJyvhEfZCO_ zZM#onc*dB8=_!yP>-M9lDp_BS*dmQZ~`BCmt+FZiow3N`Lj zM@TO(pGU>AAgU9|vKXafE1Jn{A1Q(AJ=A~V$sI95B2WCS#q^_&=`0zIAW>E~p{Zpq zvC(cAFwVAon%?zF^eE>z)eLu$z~ULZQv) z+0Avu)nN=-^%6gJ-Sy(vw45l<)bb^Q;91>QjDAC>oL9dsu79`Xaz0N}^lOT~U7X;| zZIspHtUSK0P`G7|n21oAV;gP0S&b4HN*dP`lFa;kxOfph7rL}wF|5uu(fX(J+;{nA zjjz^wa{qbIaK@kVnhrsDPKnbiiw<|lH4oLrr3kBT*E1IKWzo&bpLaj5 z`J0~A4AtHK>PvkVYj$W210S7^9vRI4>&<97B|&ZsCVe2xx)zpJIB!o*sc)A~$R!T@ ze(y|B+}=xFYnA-cbba;5BIsL9_BGbdyaQS$=D;yqzkXVA5Iix(so3Xa2QAw-_pbY- zGcdW!nE1#`?$zH{*2j@fqmyo8Sw)=zpQRtAH+|zx9T#-=Fw;A}hCfVoD$g&;^ znWqyxU0__{#IGlihv&gSAiVsreZ0CTg6rNanNXUfE9~Ka?Vfsj@71U>_aolh$Z;cx zxsezoaW?`erBy+P69?ILGyGBwiIl)A7!&l>aJBo)0fd)<7=-{ez`T@bbUL)~c*Ikf z^)a?ob9^*?vmu$7<`~F?;_pR$MEJ1@6snWmi=gYpNY98@fxg|V1t*9r17t}ucBc(557@^|_Aiyr zLhSbcvW_kG0lX*sIRF-v`?aRc2r#yy}vE;m5M276Y zqpI&~l65|`L}OqntNok5f7j~`x!%^#`p@W$HeS*mjuh377D(*NA|y1P+-J`E@KfQ! zCQHn9e=Z1=*^J@t0C3^Z5JVmqB@T&*C{UGN6%Wp>5gI=!cpK_H?{ecSdOiL-<(t|15ud^k$T`CGX99^1IFcGYZ}zF!?*t%C9yq^+IkU2ez6g zKAzcE53G6WP;+t?MQ23UHYj+6DER9i&TqV5IW6_(FlP(+M#nfI>@0TbP*WFDU7Vm= z*wVV@w03dP%_I=Wclq7B`jTDxO`+7icamw=WK4zM*%&)d;KH#^*01x#r~3tsIs-qr z`ii@jjP^EU{pq9Ek$S-Z%@-s#tQZJVAO1 z2Cu1K8MLDfL^vCe0?c3^gH;2j_pliFjIq>~gzJDiWevfd+5y93_&hVqr#6uzSSdOs znHZRAgiZ_7igpQZ3kiClL;JLUv4hgAlUSyZn64W`qb9B=eLALvf_cYu%H>>e!Y>&0 zZf2TqF|6*{xkvZ2osM`6=3iz@RoCCodUzUH(9LF?7qhTkyyk3i(YU~RMN}YTSN!lH zj*W&!OG8Km2E)MMm~f}jt@&KV^}81`)(M(;ZP+1u8; zSU>4|WNu~=@I}}`s^RQBXy=smMYG(&;#SA+5tsG*Gm54{aRN!Aatp+Meo~ClJd8#Y zY~|&o137a)9y??0m?DmQ%UrW&I`mtADNaO3j&5S?)+ZMwWn`?${-fTypfMR9bXy6Z z99;GKFwT9--30(?FM?QU#L=YE{gBa)N|i3zvcwFqvJ#NBNUDPZzNluWME2Q(g-a(K zB=4`mbeKxS!Gv$+kTVjG$rFbIoqA&_+HA99q#t0&ZETk$E&n5v7i^&^cq6;o7N{id ze&#rDNjg@tWk5+sYd4A!Qsp$@cfrk7gSgbbP7R`F#K%=n3!?|B{ zzofLI0U1g@xYF`z+Ww!CWFOp<;FlB2y3jK%cKu5%ci<>^W2ouoAGmP5xxM@SUh!w@ z$;*wXS7McOoVum=n;L$t{ifv{xb8()K5H;UR+^Mczg5hV`1N!pxV~o`Q9N4BN;qgFx7E{Z;S^JM zBTkhokYNzlV=G25Y}KD)4#7LEyAJzG={&IK0+Mw+6?LT5SMz0JhfMFpR!1_R!0jrJ zT|;g3!@3h{2DZb#1-L10XoBamW}ZA0{^(_+S%1wt$7C9uw0`8zOc5Jc>1=ai1!MnGvZ);TWjqlAZEIgC^r1VVFm51pn z_>U@ssDDL^y`Q}msqk81cUbQ9<7K_xxSdj|ZLYpB&B!QzKQ*+gnelU|+|K-kIY-UL z%-8I*Y?+O@pG$9EgzVB=ys(gb5HC_zo3l3g{4#cny;J*yeqeae z{qUd%JD)OlPE6fRV`FZ78LRX{D$yusLD43$^N)uyLt-@T8t_rEwe|D@_j;Cg`zA_- zZE|N{e1C|UCsw}O2WvLc@Ut<3A`P#}q$w>+h{ZM1AsG;Nb)1o+@MkMoAz~BFr6$5k;TY!^AG+No0rh@3r+63V5j@Z5Ac@8o(G&^|o7? zXxa<-+n!6z)#3_YFrUQx~xm7v8HM|s!3{et6|I+xivMBF4c~2V`OdG z49^GFBvrgmHgEqKx;2Ej9bTE$H#==ruXk47kC4pD?YCtYUQ=BXn>nN}iL9|4dY6(u zW%ocQ9VHEv@<$*sz=vAEmsB$rYoK3edu_~Y<;X1gVoZYdaY|IJjEZS5(UP$$;Uryt{#%z zB@umZ=GDmS;7|k8BZ^ldjQ2X1UWeB5Y7JJiLcL8GM9UI$#)-t(A)_~JyF=np4FwkW zt)_Fo*Ig>v4MX16h?7}I$*yb((*B13B@4!}p<_R{GERSgiseXTIO19zCEEXE^wVu|QNUIMZ))~vE*SIdVN{#*m zrvj4|xrvlWLQy3^7Yo<=k4?_+PJpex(=J)QrW%~K`>RaxK4?C{+rHB3uuvZ#8GrJd zor}OCIdz;CZww4g_)4yM7>o=y-*LWZdaN$iSgGM0R`F$9*nccg!At4Ih&}u*w4ZKK z8G88isNuY2L*@27_W#z@CZ6lw@+)nU8^}M}8+w)7*o>ih!_kG$d++OhDBs_ZvAH*( z6m}-+cd9sl)D}{-%e>I=CwTXShp{VB^GzStjJksDFtCl{nO>#r*9DhPW?9zjV~pcI z!_=Bv{zO&Z=60vtY7qWja-Ovci)@psGYi%VDgxi}DWgAarhWVNZpgbA-=*4f^Zx)% z<9N-8>X6M43qWLJf#hT!LYDT(P0y;qY-N=8PX=1klAOUHoG2?#Bb^i{`H)dR0%R-5 z!#p#&Xl1L;z#vCU7SnO3;LspEl4LQQIKC}wm|uxT7TY@shAp)bi5ALYG2-qOhkxSR zvDFLuRE}ygKw!TC{~q}&2I}&Lh^YHP9TEs+5_~g+kSAJ$31_|L)X#FrO9xn~JbxFI*-6gl7+cq|qlKB=@x?oY&bV0Cw_~UK0DNm6SIwHN-j7rH3<)v$Jx{d|43v9 z7l@y^H5BU1)*jye>kzp4Jmp|<>%q3NpI&sPSy_Q{dHX$kW<#-P+<(`4X^8V(?H)d4 z@dTGa9{fXX8~z_{%=wk*e&GJ)rs44m1HQh^4mtBD*DMp^Z!_2LddJL@7QzT`=IQsz9o)1oHZ z$@CQZ8{v&fV6UZ7^rTNCh0Yi-7!*8{GZLcBP<@bq?b5c>6taGcrez@XY;0O+Os}yT zy!%16TFw4k(~ls(u$S;r&#+KprRB~U?(*%>Y+04`pWa6ecGEjCu;=84xjGebEawTC z%a6KN4qw;&ksIi^{Sx0`2N#o&%t1{_b3#jY23dui5GT{dw1w%Ma5-M~MS&Nx-BwA{!BNmJi$G5J2VMO>Ra-xXFJ4LvflGy;{B8?P55y0*R(KdQNG`lE?L5Kf zkGKA2`v*W)AT%@IXBrNgsKwo@l>+~nM5dovOr~QkfY7nTvExZ)_sFZLj1{32wKPV{()<_< zivgcdMC4sTBnJ1nduNPEE+E47zt`V-gkJLMwxH#^2`KbGKP?;590MvZUZ+BSRG|o> zg;WOr_-P)4-0Mt~&M)aVZlf7VvlrzvWue~R>sIf0u88f{gc6{4bk*$v|N3PLzpJe2 zv{Fi8w}@nlvbN#3kqt4it_hXHSUrk)?yA}U9!Sf4KvL3S9WfOwS_qih2)-zJeR<}U zN0?o0Xmf1US>&2OLap*S6n+;$GP11hd=W%Y8rM_`4#!Pf+#a3@rUWdrmyjb03P zKT)w&x{$o?VPwt-qu8t5WuCd_h(s`2E46M5<1;V;DHRW_##p}I2giCg6i`tkI6)-y zV+1o@s&{XU5Qj_^?>ur_h#5N}8cZ&Ec%1~S@Eh7+cp^J9BB&uY(p-)l7T+OnNufln ziy$!|5h#O1Naa(=T8cB#itYe^bI}@h6+6GAh;JccdK%M$k+Px?BA-A4$*e$(2EH%N z4~#u2T?n*6S(`ioe_E92m=T`Ppy-`R%mYJJpKpF1`I&f-cxPq{6N}@h<54_!rk|4-ejpc8LX3ir>5WH5d}0C>y7El3wIYrL?#NGX@3Di0@d)EG6FcAs(|G8VPqp)P+j z5f*<0M8_B+FAYsSGAB$Dh|Q7V6kZ zWLVUh_lrptV%dVQVO+D88jy>Rs;u{!KdU)?G$=MPjk81)Fg#?{{i0(`x=?gq>%c6sr7-P|Y6eM8b6>;AFg9 zAe#?um^bL*qOC{roc9B5n<(Gk{v5Kn$+fs0S&;bA&Ehvc+M4#DpVov$t^e-xm*_FQ zy7T7qv0~*DfR_a^=*Xehw`D zvMyT5-a!%bMU_fI+eKEfXdo&Nor|E=HOgr^BsrdFZmGRd@CQ&hK(Solb9JT z3ez<)s+3TKIO_pKQft_Lw*2srzFV_}RBN6W;jaP8Sq+55cZPShNy#$Nf*k%56WOX# zb>l!ecy_Z%5c!Oy2bn{7D~}anKx4%&$7lxrubnWb(OA+pv&4OG(NMBmv5V9ec}Hdz zHFEeP{P7gR3Is#s#7ceI8T(U9_6ZoyQtS89{7}Jzg{-2bje-A37EA2+md-2#s_qU^ zaXn@Mxwu{|9~*6Q#ekG3l<3H<%RX$-t=U1UCi+MzB){i9d)LLB=UGH9t2^sVsaUYb z%fqG^*_z#S*?o_!uU`y&)4z68txtx&_*j15{~+&(;eVMDPvq7afUevC(hezpUOK{t zsIi*Fn~&4GMXNK%-(HlFh1nV0ypcXR@wYZtgwsmMyhT!`yZB^sn=e8XAA>?N;@H`!-P&-?LbY-yBmA+K$Y) zFCZMnU7TRiT>7qC*GiXWnV2XOqh9wmc?@y=WyKtPl)+?C@;{s;nK87Q;q6(8P6AO< zqPa+USY&&s@to9K=SxLOC!^K^eeOK~#>x0QmD%RV(u%jj79*Zdg13 zG%$2S57H7G&^86+As)jFFu1gcQ^??}kx!U3mLvdH^akPl9{JFmSUzz)dlfN{l?WOv zjDVz}kr)?rgiZ-iLKUj5RLCSpBCAJA=lG?TRhIn>$Uw}?1=d~t(1{3gsAMJt9+EpO zeMN@=I_ezW9#0GIW(5-{jWo5WBGx$uzNAe<-%eOae%Pg34o z2~gg0Iyls;O#WV4_I@Aev_`m(_d-|h4Ze2{7Fnzdc-J_$ms#*NvGosm2z$I0S^3AS zMQ*Vqo)dTWbwb<4>sIO=sN0%hLCCe;ig`AD^MuNBF z*iFcY(7Ei&x`rh=(so7b8H;i(y0mz=kg;K?Anw{93S5Vti2gg+@K*v6Th6BM$omph1n z0HnPrK=|S(A&Od^1S-%(%X2gllennI+n2=jWv#}Fg9Mb{@KEpZeD3?-zW07gYL>rf z4w^q+Xxvv0s}7A!V|D)=WU|-QaGllrL*=p5%pAv~v2XfWBHIPhw(CZLiDHdGVf%00 zTgk;5^4guts~i)U9zyBVxah*k=pxyf!2EQRK4_Ih>ac(9>oc#EI5_9#XDVy&0e(kns|b*Ryf~>E(?TOhpqmL z=#$}`kqdjdc`@l!>UXzmZLqNKYk*fkO#8{lni2nnII(XtbIX$?6Mdk!SvA-(U+rMr8P`)Dn8NUsXg%SK)B?ZGX9Xc;E&E5lq(G8yf zU{X3uoF{vW6cv~qX1R)0T9K@bq7{Ln4BO{UlF3k(4)wC`*o( z4s$0i4~d~=%xp``%d<6$C7QG(*xk#b3%tlZAV|EidWtxLUUCQP>`-ENn^->%l*V-4}kE_yuEuanI#`(<03!In}@?QT=v+uLLB z^5C+gyKmQ@bH%|5xInP=+uul1*=__rL}qSUjIdPgaPHeku)sQRzPcz6GfWU0>>%&z zYp={9&K@8pLcK;8m>eEx)*S8nh3$F2zm!DJ{u93Exl-Pkp>lJ$dEI zX@ZmOX(--j*7N7q{LI?Vl(~BrAYAiY$^Zqn}lRn z^Snhnr*^FUrtkZtPPG2X5n1ReP~@Z`r9-8`cNhE^ZS(U^9=z0(l;OoaNSUgv`CW z-dp+;c2mdcznh$)@^seU^tw+kEb!jx;mAz(bz=Y(VfrT_BbhSKC`7`;Wp1|}=iqQ8 z{J+@;$wIhT$mXMq`Lkxtd?bh$4=0rOcS1-)8Gsp9eVLfJ>DE-XzM-l9aYO8KTb?Pa z(j*$~B+oImcUtmzSTP@Q#&75x*+4tr8jG6*c0DZ&YJ|EJ9dk@vWj2c`Y(XOoDf$i7Jq(07EK>KoXPw>Lvzf zA%SgG+^kikrrqEq=L51TtTB4f|9*Y|803=I5=oj9OF>@*_XioNiZQY4AxQgTip|i( zs)pg@LLN-LlWH`}%n_99ciucV;2*?E>rg(Ha&30-+X$XpP&Spl{I24heYG&*bng8k zw66BBI;i&f*-+K)LVE3ttm66rGr^OK%2rI0Hp6oq>~&?;u}&ktzioo5dlH`!`tXrb|D`u1|go3LIr2XXlsFkK_OJ$2?q;_WIrY z_3(7UMdgjH+f^Ew+f~)s-@Y;0S~-rP7w4L{f13YoEi*QI5`sGR8|xQB7jmzsPTdLF zdw<(GZ$js9-|bt!t2q|>@gTd#Wg6nt`J-aaeJVFsRY#6uwBzK{_u~)mp9SxXy+u~k zhjgBodJmo|%4Xk{Ua~`H;U33nRFYO9Avh*_HnDO!^HP8BjDRz4*P!gpadWz1-yqqI z4Hx%0n`sqI=b*p;a3`;jGj}WR9^G(MQg%>z61%8M2_g%6Z7nJ&rva4EmjAuJL^QQv zVj6-v9wctiHT49N_8?$!OU?+9PnLrQGihnW+%pI#0m>P56KPdo0E}onDRh=_e+{e| zYZ0_&cyK|!V+=5))dtIwMKCf(y%edXE`c!C#t4a*Adz-U z(rW0Rti5<5*;G{w5ks}AWu9cdpBb1ZMq?&-duPwv#eG3JKH##r&;EAXwdtfebT7oB zdA0LaY_DT4Qg-0>ZAWRqcdL2VR%`QTW>$2 zLy`;nhPm+HqyJ4driy*H`)m$2>^3$plE3;^lB$7NylK6xjz)v3#exY7xP((o%dS0S zUs}ts5JkQx@T{k2?!WaoC(a#M{Z9917Q8OvpE5>2Ep~V!oBjNli-G#$N5#VX@Ap+~ zcEL;~13BYqE5GE;#a6#n5LR#O_huUh_QsjH1lxbSnx%c>GL3qm-#&?qafsh+^cWO> z&qbjtDD{|i)<}t{{=1@qRJ6c^fdzA;qkLxZ-9EDpx44Vf*3nfY%mmWqP4X7GJ@+*8 zAo%olfMd49Bt7>m)Of}nNds8a2MQudrf)V(zPuftNa;5K zRqocW$=#gll#)-J82p*|Xo;dQIuxV2mPA-0tRxbWeg|1VC;36-0ryWmnd~`pLqa_G z&{R-))%-m}A1An)23LJH!sorFnKu#x;0n(JvcncJ7>bb?V^vaam@ovAc0gnc1Nk8U zOLA&3cE?5C?);%)MpzPL5eZ`Rz^lq}$IKIG(_sb>8Y!5s6pjcKnC7@>qg~3=I3|XC zEpr6)i=(<>ZH@3j9_-kgoCD%{E(A9Qp@avMDk*GX{9<&qFfHs*Zrmyo38C?Kt~Ht! zMsvkUQ|J^6u&MaeKOQ}edn9&SYUsF9 z$Nrx=PosD9j9%YT%gCn0UA{KOZ!KdF)dR9e=H-VRZdqyxoJ6YU%22}l^Q`whe(u+i zVcC`EtbEiGF4d2itEHm@O8fiN03U6WCGUpYJ^z}MnOIXfeK-&0dr4kKm3AF!FTZm* zKgpIb8?sJSvp>INThEn&dWO~)r_>naPQ&$}X&AW8Ai6j>1%y}@ps^3s)&e%6i{41n zcY3P#60@V#ppbwZv?xwdQr!t9tHy)^kpf4Axp^odtd<}eGzP-k0*GFOgB5P0#z@O? zB>Gs7hH{whF&WSw`31Ql@ermFqZxjH1Qo!W{-O>jflwtt1PoE2h>O5E4zpD19wXB$ zQd_~};3sE`?BWPjXoI?6iSQQHUaev>GRmyPuR*lq2Tdtf5er3tpsEmmLP}JN;!*;X zgqj-!f^soiDiuMYON}EF+Q*1CG)ILKFQ+SG+OC7pCTETM?V9kYd;0DsmeoC`rqPv+ zt5K|A;hCg9GoHE8@TIfAGirZ~fpPv{djJQg7THZ0At_DuRTd7m1OnL-@E(X8kE6iD zt#npa8>mA4zn{OYzfBLfdq=IXBld^Nif1|(`#NM%<2!nKn4O(PN>y8?S&ydW z%xLZw9mMfrT|uH3(i2#c{>bGMtf)niDr) zJ~WV;pH_WvHF@eIuCJ#z55+{W5;Y&*1yZ-;_V`0jfrAtOeVvMdPm%spLyRd@34$OP3@I>8R!YmoOKlKSWr>VHgnZn*KS;j*o$B3h?e zm#$SD8Mkkl;1(Hl1gzRK`|k)@PT^h`5)^i5)P9uo@Q~_`+mN$Z^tHCD_4lj;s_433 zK@Oz}IX@3a;2af|+VquMUz}~8GnnkJCd=KIU)$M9o9$B9Dhd)Z zuUHALx$H#$*ii1>ID#lM-y^uLZ30LdkC_+Djz!`9KiH;HcmaiRLr4<&RZ4xB>$=2q zHEj$RNQ6g`-z0J#1CVV2K*bS+{lN)}HXup(0dXsBKAAyN4*(2-V6lpD;ba8c1(_Ub z`E3LZ3WMy4;ZS0DRRLK-DPr!WQl{aefB}5zj|w;mk(hFpFi?)b7D%2}o)*Ld=RlP& z>K@ZmtTMNe`W*li@*(yZxa6vC%Zx|}wVP7HwU;~Rp91EmR%$6qv4b&B?# zurY<(ZtmOO3FXs;CJ5x-yf`7RBJNZCX++_#Bbtacnr9lIVlnc0#ZpCGy#9G(c-Y?c ze%NhW%_3oUr(w@cmS3%v2Z+NWUvbkQVgSti>UW@iV&0_HgWIZ&!Saq4y7umK`@T*u z7>b?5}&sGC}9b=CRo67w@?eWQyCHyL|sn-gGo9wQpVJ>7D&Y z5B*G%TIT>ZY%~#kF)c)oe7F^|oz$7hqJR8S8VhT%Pa;Xopy%e6HHRQF&B8vQE;XJ=htTNpKf8 z6SQm5=HF<;ljuuUv^1(YXCm7%QFp%q#g)R5f_ieyf3(y#DRw9(LA2sPqHZLJIL~9C zjvs*#stA7;L+?>W3eywBBsKu)2@8sC!o%2(DhggGcpuZExCjGMpsj7IXj5|DbK8ty14#qcawJ}UR+Nugpfkr8uIt%ucv-ZUo@re z=ze2!%FzmFF)c2o^eg^dG~VXy{GK&x$?jpU`Lp21;ydM~PCtY7&a*Fsuu@O| zo94^+%?riHe)oEmho4|2S|W-uSkWBN9@hN*i>v8gmegCWhJ9Nlk+N&l&u<}f z^&v06hs+ouTNCchp0yu~Z+0n7@8-KOby31KN=cx_RT^=Kn4_GXxY>=Cnm_;k8ILL4 zvAYBEz@b5_uufVIq}o;{(OhY1L+Fbu#juSni!09C9ct%tDRJ|679FMadMP3!;uI3= z9L~Fc891j)jhdO@s-Gk_u}`~-eM&zWW*Rw^ z4LjR?XMgT&w?}@Yl=96pIJiTf)jOifhw4$}pIL?b%g3vkPW`Layg4E<`n3(Ut1ta} zf_;7l`waf5&l@B`rtXpkcxQ77g!fP9*yff*4P|_Z_Aa5$w*Zpsp@>P-vtFfcHa?ws z_mM-w^`04fSqsI!D#F^_(){4XS3tSZYD#hJfW)h=jMr=7QKSkT`LwQjA1Jj2V7e#_ zA6jh$N=@K0MUcc4(ny0maH|Xn8x#Qsfw%#1={!~BFqYUBNoWLN7^Kp#Jo0`dh-ix( zE(}9pDFyR5MGt&PUcx$MfSpwY?iY#N#>v={HqbVz?xUQQ;JJ>%dJq_piz5y&(#kq^ ze~G>UbEpO0?}k8VP}+iww#yRg#>`r=aY|GcT6P!u0%14gw>!n6i^X+^ov)!GTLf;l zF6XHJgIw=N6tkwSYOZjj{D30Iu({at4$iQEv$ZVu5&Q3NcBxXJa5b`u#n|pV05ynU zu%g9`7JGh3-k=OQl$a{!o@Pwhv<(Zj>Qj=UG!-d*Ri3FzRh-qh@=EEV)#B=>#WkqH zV?~kMdnKq>?|Mw{vEP2v<+*U!i3-89IH>WWwLSlRpn0@aCRy*!M&}L8Z12>iIMzyh z{ii(Jv(vtJGd1q{$+Zdw=_a3?9YkUaS9o-*S)+DfR7?KRpRd8b8yRLw{%LdVmtTw+ zkAH;Qb?)9RGWF|OfxD4@M{T#}h$S*ksxbv&Bat*}t4g#9O9`h!7m)liIP0oKB5WCob}geC88 z9;6=zl7t}aV6t%H|eQN7T>+1 zB?O+&oqiTrKKAjyGiX8BN6~9N<-c!|z68hl*5^`IadKV?dhil#Qqk|P^}|BM#A3Ae{_GE*8E2!wwHvPHz&fwnE9ArFPosDH z^yT~0K3~6E*^S!mxx2SdWp+2s$2B#1lB!c$VT}AVLvj5x`%y4L+Uxuvr#-^xQ^*5Z z4F6yIM;z7hetVKGi7<`iCDEwV=NVbD`z70;XTz@u<}Pe)q=ZW5@KWlkVsRA5_heg5 ztvlXU*Eq_z#hEYoXL1Z|*8{=nOW`n-w<7hH^iE1N#fv{=NR(pUSL;!RbcB|mUTp*n z1xSdYhb29DMTld63nYaiF&Ky(4oSs?h#~eu!brOb2hNLxVL(tQSSE&S3kDR*q{JCn zH}VD&L3jxjb^-*u<=pdk?}jw0et}UzpdL_~ZU6y~so5h1(uo52#8=s&P`D&rEW!i( zQD-nRC$73{rf+3siI3*2K5Zf1`t~K;Hdo+{!o7JqvBu--zgKhr9;w#52INuD-r-Ac z*RJd|$=zx-+cvbk9r8~WJC`L)^N~X?H9S#VU>bfd(l)}A0#E#dLDO(7-Ek$H=>Ozq zVmH&}SLL@RrML9PU32FFvvrr!4>j&ph<`gVHr47N--rv9=N9E9x;OtSi*@0*^2G=v zZLs6f#|(>r%Nw(>`%c#_R__YSt~uE=u6`f>`*;7_0C{Pb6r+*hnV4|7^6co_qw1O* zD`u+tUbnCZLb1jB$^*_uL852E+a{YnMFi5_^U!DfT77GY%T*p}(0j_z&xW z)a9?%8D<;dw*NHo30nLl+u?%)rWkp z{0FC2`D!58jti>jmzvL=7cN!gIQ;hVdalo>adpNcAmCGdwx{&57Dj=e3(0^Zr-S8* z+a_vheWLtGTrQP#qiA)SMz=|jOaEH0(~kn08mUoASuNF01xcM;D&G*CqUarG{YUNG zg748b_&1n^o?i|X?RwYM=!x{Lzt>gYmig@`zCmruI{$p`;`gt=*U{cT!oJ$Puy^B3 zQ1WS5Lo=40N;#J+9i`V-R#&`dEGs)zYPeigmX~i#FN_`D}|YWRHa z^*LXcfaWWi^hPO;+@(1WYDeKMT~Q4jmM`2gSB#0;h%I-PfZA<{ey}7xpXX+1*vo%` z@0@{PxwY)nOX*K^v|y3W!j=R*n--FU0*XW~>qP=Q@A~Ty81+~!WPKGli%u(o?c+@Jgy1iR55b?5oUb-$>-!S zxY;qxv72@1zPH98L2uwyx8SX$L-Ejz-Ja^Gw?Q`WZAD=XUT;2%U4>dR3@E#xseH+$ zyKu@;RQ#^1(t{FLVJ0__n90&i3IQh4!W`mg@Bl2^Se-ov z2!yDaZ7bz3z6Anbp-Jw-fKaVGIP~ED5~Vm2;Ol5#TMnMk<5F09vyt)Hq zKp-^)m*D={TngP4sd#3%svs9FjUa1)$f^^!Yc4q5Y;S*4ssQ zb5N^F{sMTTlXWkG^D1wx;*9>vmsx++leZYJuD=&<)5mvF?!lP2HjSZu8AYi~oY#yHsvGFQ=Lhzi$v8X`|4yH9Zc>k0Q5eUIBU% zDk0}LsvC?XCZ!F#d(3`A)r1-w&B)k))Gsx3|4|Ykeqe<3r+~)5myh{YU?<$TxKv`I zu3XTTEEmrm$LL69qo`epl}%>8gjA(P*9YMPhZ42l$2CgR4)Wr`KA(`9kkz|Sgdei= zK&6yOyEB=4BB0S6g03F%fs>x6W2T#~n(Sm-JaFt*@9ULDQ0x!0CufS)L-R#O5*8eP z8oB9R*m~qwc9Waljjm60uZ&9j75Iu!zT%)22LK_%SO5iqhK10@{I7U&kfML zBOY$gwc}SH{L`Ty&syd-M;M*Th^ru-41QTr6nx*(-|oKJloQAo0D z(|jf2P{`S1ca?1%c(H{Zop@1K`Sb3(#-MD@;62f)#ijFiB#b19H9P_{aB2i4+sop* zLj7(6JpguuG-K=1BRN_uTRH@(PdqE=rPq4TSd7 z%qz}6GmEU0S0(-{Y+VP6HK0i+)_$jD7oL;V317o1xjhVM-+MWpLTb1^NhjZU~(vfZq~8Mp!dp z5yVi?;jt9|hovhIhq{g0W{hPRj2U7=24hQEhS$D~C5D9LMaYst%3k&`W9PNZ$eKMQ zZ)7Q3+1J5DQKV#FBeIk5{oe2UF4yHB#$0}9p7We@pZnZ*k4PktNWnQqM1M!kU^nAG37kUs05EzCpf*28lj9r;YYHzr^ zw4ql>tj$r}H6VhSVyXK-$w+g?!vbf`?~6-!a_8n%m;oMgV4M1GnCpSA^>nE^rJoJ_ z_3eD?*nN>H5rV@&?w<-P`{1QC?}>ahi2-IZmrgd&l_4a8gjL~G9Kr4a(=>0v_)KxR z4Aop~y`Ibagm3d)cc;#rMw+hf4HL9%CvdcKYOBskR6>Mj-kX+ zd|wwQ(Wg^=zebOhaGHm3;WvFEZ5Ui{P*EfdL;TZ`LYB1rLb@M(>R5 zt%Y5c0YtDwx&qlxU(qphpqnvS#Dy2th%7aJ#M$~<<15yKMUqu_Hs4+#8%Iz2m?aSR zU1g5#SSLs{mA;JNXJl;9IM9vy$oJDn@b*gWYJMpMY(WHbq3NMC0EH#0Qh}(U|BIRc zGn&}3N{oUSp+WOc=|TTn1S7-Ij~SA*r076UU|d2EMmXY#Fbr4zFBWhY-F7sLKrEIe zb4JR>-HW!WA$p*-A4TF^Ao5+&$;gZ1Q+nCzD?5HBzuT+!6g-D#wl@n;V?=O36`~jY zZ}pbU>Qxonbf;&Fr0FRC+wN6$2d~DREqes@dWHm#sRpVu&((-d32@F>V=8cyC(J_S zjiH#dvL9j0({u)nh36&pjamos_2a{zH^Hy<>Y|NHF~%l<9gddI<3Ce9$x8VEUZahTn3Vk)-bE%w>AdTORDOOVE&|2^f&6SiCT3G&hb@+(<9c><DG&%sl>-t?@aU;|~Gu-bWPD(g$>|5hxL7tGd+!qsDlab+jt>zQ-y+Igj z+ww-EhyCV}ny>BJq6;G^A;*82Po_PW->K;YFcn%y8s%2k4*sIjVx3Sj!^qv+JPl*G z+NAVt<{wxjCcV-6DP8Tb)KGZ84NzHK`moN9lmK9~xmCsBP(HunDy9~HnC{#Q0y7BZ zLB|KUCF7W*WaDQ|B}oXt!mNYN_dRC{b7?HIo(@J9kVsjzL!xDc{NovU$;Z#xd4VGL zf*tEC2GbX4R8X8OGd+Y^79c1107Dh#V7e3d|yz}i^hd1syg<*&P9I_})E42T9*{*7RSu6HYzxlHydg`iw@nLKzTnJ--z0TBY z;-^Y=oLb;7+@%yQDvUygs>4(9#NB8Zl;3+jWHT+Owe?pQ>ubKkfE8^$=jl5iY;`B` zyijL6%ba%X5^u}j`4hdB%VNO>an z`pxjt(xri*&9!$hKVg>uZeZB*TuraTMs1Efh`)qgP`E29FC#x4HD>GPkM^qW7B-qI>hq?UKB+}Fcw-sz$I zq^*rSMmEi#eYr~dxh;|u1(n~jxyVWfu&%t*D9@!dYWnxtS!EK3{+8g_b4dl; zJczdP5ijowc*<+^+$wH*(t|PfrnN$>`Ik!q6mwU5*YV?yIoVq)^y_ssw5&JX4F2JG zjj8epCcQgE%A2s1?Y-MZC{!$zoAVL@DwPwD&VoHsPZHu@9v`??**e!Mf?^T>VZ4{J z6F+p*f-5pTVch!A>OT}HrfoCcKlUdyljYmOw>LJEGZ8t{d;?NwbzCq6$qAK=UsZoj zfHPdtwQywk3qz0GqZ92A3hE)@;yXzCbl5033p1!w1ZWl+5&yLs6Bj>_z|68myKbTd zfHVy!?omMv9k4Rx$OsHmSUb?F(Pmf)2D}7Hm>Hqy>G&0}o-PGlkEP+@^}7Ff)*YD@ zd<9&2V+6z3kXymWe{X(&=+)w`SkkU^v*3N^6%Ji%=~?ZUQcVlfpT67bJ(hgicF@Ht zqhQYzNO#>FPUYgocfnH=aFA3e83F+fot_*D0J&4~RlC8r@JoMM*Xx$Md%3?YIMWby zCUqTLm|F+b2nNk_=xiRJ_eG$cXypi7wGTg-3^JgGe@&lOo3NKYxWQ5VHUu8_{mIt? zwQ}3HOjdwf#`a$Rt(0<|{hscyIH#`A4{vY19Lq(%ug#lGL5`7wX(wG|)?B$-6k|#Q zUrK7qYO8N>@NjSU!-K39T2{4h(;wuk&2!F^>T^x2>hRd^2koZcA~Igc%Q`OHiG6RI z)6h8JEM}Th3VYv)5G#gBxne_=I#;Ne&4fStrSltsj4| zfT+LT%5K?rPe|24&F7Cct{NKznq7pd>OdTWdv=KC`jR425^Q+-K~yD?#Z-?8Y0cj$xg(87K${(~z3JBni1P&dw2tpwO`4&e95c&Wnz$%Uo6#pQSil$fP1VeS;AP}`9 z+ME$3B!o;eQ{+{=mtB+AiSU9B#`m$gGNcQkIC1f%9gt0d>eigGtCPFOJrRqih507` z4Q{7`ZcV)l9J!wwVVYSmYAu|Kl;pWz^=5i)4z4XOy?Wq&nWx2m6Ng1|&ey`hnC8{H^WM)t_T@E3uWjXjUlN@INjf105GsOa%UF}0cFPrIIT-9>`A5BS(0E-W!Iy|c-8sXFrOt+I_NCPX^kx!j!|v_2O2Yq~k2 zz9u#4%Ih~VHfr@st86)P$@IDJaYp)k`ompBr^{TTv>vZR@^Ouu3coc+)~De9mUkm{ zq13mRimZx2j2=Yrak}3h#pkcf2gV?v+p+2K9@fm)$@p~POm-uk0lc9dGVXgy@Hsg+ zLfheRaW1<>rTBA7qj68p+QNf{*UB?YB&Hzo)jzj^X)~O)hb~hH!6GgUN(PWt_&qwyB5lY7 z!jTAK^FaemKHQNA7w*!29CZ{i8I|QQG`iciH8Zo@vSssN>tV&eui9#Im1=Jo)k#h! z;-xNBe8J2!b9!Qk42S_eK=seP2r%Qga?Zf0FedTvKG{&=7oPq$2>hf$BG)7u|Ax!_ zeVc{VXBD@?R=}<&tWezl-+76rHg}d3d^8K)Alb4~M9L$=u)H*ZAr`EhfEVSx_X1<6 z-#>+iVR7)PxzDolUmt()%dSS5gj;)jzj3y@m>*cML7E+BJ$&zdmj7-jCPJ^ z0tGsx{^aAD#wzpS?{M+9Z>a2*3Zh@ck{Pe4wqyif|tgx-)!2i-Jk8Gx0M;0 z9&G#bV`{HZMq^X@eJqE}tMy+0iMI{v_L|GdYV?JFdNh&FMcbWPs`PQqXbsag9i(<88h# zxE%X#Ox;M0CT4*-ATk3PI~HmjsC7-i1c%H*r{p8tahodKr(&q!c3;k0ehSJk3JSKfR zlTI%*lSJ&L%mHj=`pN2@pB3?cNG^X09gYXB*dOcJgDx-`;#fp#At>-DjzVC;EDCd` zPmdpBgGrfrXE844GcS*F7WSP&Ti?;v7H_n0{T`L{OFZzU8G z^ZEdC*i>mdd3?tz<^6OXpWC}`r!RM8E>N1KLpMG08|~i>&87$sz+#Ji4hh1cD|1T) z^~P_Q*&m48e#k=Aqh#h6A9cI>{LA-IO(%VRZ@jrJsB5V^P+2`V)g_|yYw{cD;jJG7 zvDkbPo8F_Z+t6D3;x+zb72OJttu|6JaybRY!XT4DGjT0}ac6x5)cSc(qOJP%4JKdw zy>S@&s#-d`T4q2oO0SQlN-l}-u7Bn@JO##FgH7|iem}-TFO)Vnp!O*wC2O!rV`1;l zC998Xr&K>>G>@eeLHt=sWj5c(4jNvD9J!3^UqzE3mvI1l0S#b$2|ZVJi>-86^qSH#^qoV{APVr&Jk10uQYwUZvhXC?kBXHg4X zE3Y`Fg8lq?`IeV;4;1B9Tf$eQ%X4)k@7}f?zrcm8ZVQP!oDQWuq5lG`x<9?W(l=^Z z7s!?WehpPV)?o${ZQ>4bXl@DW>SfY$Jzv!r?~Z%pLh>`w2&11Qa@blh-Osn?V6|6M zNhN4FD~EDd%4Q_YZCu)0jHS0{b-A*ql8~8REGx*KE==faLM5#7cDa6cf@k1gl%*%%Z^mCAI`X+EojGu4TQefJI8(T#QQ`i^Sk8xPmlFS*KEhucK! z`qe?W3?41Azr?e=01MzTLZ}J|V8j51@&b|42e?8TBnx~MI*KbB#OQ|>3cgGS8gLLR zxDLSkbo^LtC}afXhz1F}NI(#Vej>Nhw3mP6thU+3s# z`XN4C`%#JW^mXvjt`+Gz@Q7zlrVks!4p+S;cJ23#1PnUv?ULe4ua(rB@TR8kJJ`9| z|Ki*+SKv}NN5s-eQlRFT&WGAyovAPBb!Xe#COeVu<<8HVJ!dG#;s@{YtA{AQy!TwK zA(lx_Y=0d##(JLT07R9Ot#Ck9&aP>{3)QF@5Pmul5vHyYwjjQ;bR_jo_dH=}=ylb<8_74mdD)}^sP*-Ob;;Pxt#m6+n-7HOc(D)Sl@$$x%Yg1vCwh6 z7F*2arH7e_)7WZ1(bPH{(^}Y8+wy5@DXn7Qn{f?T&d@zs@!Y zvcIU=(A>X@%(dEdRwPFA%a%0Vq;U%TZPta|LL(nKSH|)fU@QUhgPnu@=!!rNNHfwi zp6D?lajle`Ox3r@OYdN^P+Y75ag%k$cIV`*Qp2fl+tK4O?yAKFd04LzoQ~L33}lmG zfI?zRp=pCv(0Y#8(nL5J2r6bo#gVve9DucwC9&M2`zQI~WqNzNwN4}r3TyB7KInY;(D{xlR<$*5++#9C-R}2Jl*8GQ!}Kl? zSBo2e>quyWQ96Kc7lj}u4Dv0hzW~Sq-ToDMx-rtW?QQwHQ1oMFuv42R`r)D@f?h}L z%I+i2gTonz%&U#|EDPiPgG%kEo)M>Gg=fYOPh0DISJ+SXrx(O|#Jd%~0A4Li$`^;i zod6r{=aipjDB8UTZuXJ>o-KYW>D~JzK-OJwa*M}?<9FSMaRw1(jc>B-^yX<4yN!Uy zN53LpelHk@C9fPE_zzB&qc$T_LJlrB$>-0M734XJc^xV1a0 z#6SrXAV`8w-X*D@%1<<8NrQNQ0|aUyJOU7C*lQ!ZXBRs9(<^@XX0*M|5bGdPV7@JndzP5AmYd#nead;xHnxT;vq&e=<#b#7%G0Etg8?^6C^ME`ogywl zg8Y|A0TeL49ZnF#qnQAE71E+406h2SRMiCO-0OBRLUgr`&$N-xz0$(F3 zrS+Rr<}2Vgx9{w|{T(A6^1#mF4kag>n+8K!h-awiocAgz9X3vdNG90|N>zvb@>UFf z^RD*5P{U!{dSol_M!#vB@3~Jy)#Nh0m*b;h6a79==6I8)VoE=ENR5D$Ra_rMXZ>zt zp?mA=Vfjkd+_9_Q+^@UVGz=G4RbB--DVhy$$+hiVSU5E`A9iq4Z;r~)sSt7bI{!Jo zZMK{7WyU=z`^I$NC{)t7hY042y_f4E?Wn!UXizQ9VBDn>bHA!7iX#@Xiyd3$uG6ig z!uS-A9z}ZSM%P|CMEBd3B3x#@;vvvVI!4}mV6kW7Cqo=fA$#m`-$ljBO~liEyYA?5 z$olAIJ89Cz&H-7LOvzD}62tT3fxD7lwg5Tu;-T)pEZG7a%5)AtK)rLkuvyjwzM6Nb}Mn57QTi(;bb0 z@nx^=)R&Lux_9??qN*o%rf23JoR9TX@%;EUbJpOkc%W1~b7-@;K&0JXVE|**s+2FKxMa`+m9e@ps~9 z5R9NNH^(qAzHq8Bb)IDi|`*pOFcDzjX7`p6CK zuej&Pp~L~+kxkix{9C3Yl4G%JdwEX(Qj@kH{qwh+pc`2(iQaH@JZ5eYwwstJ@4@&i z5N04vW0MUr6{e>NMJWodu1DUUx0jH%i@DJ0r(UNoE{yA{G>=2ZhD6@h#^qKEsZG*f zOT(mTivju(5@-;24d-Id+q#H9((eqmuv^u?%qzy?Y4H$YJogl9HXh9E!&J>IUh>39 z)}Zeel=%gbZkq=pi8hSn%Slc`VM%oUvh>`JbP#4B)9Q%9)mMO;hYkcn5x29b00|^Q zyqQ7>0OG$|#EwO9<`79Le)RHGOa?QkYh)AnxcZ1J)31MyIYGF~f=sCw|Tk%OWW4_5Aukw2YA?pi5g~VD-3Xc*OC=CPmU^0=R-P z=ENX<^^90q^sOk@vpv&b)O1+THd9rV7HuPJT2OiO)%N{Ts9ko#eKXpK5$gQO*w|RY z|3z`3F?_x82LNSgJ*@LAoQ56b7ymLX{~_ToZX=aiXlH}!nN+}9lG9#(C-s{is5jdy z^YcuHEO0NYG&OB*WH|=f-Oh~hFQBBo5BMhwyq4}O{MxtaEjrZ9WTYbZ>Oxa-NN-pv zES3#9rabRL2l}|}i_YlzihGvAC+swpH(NGpYu*paal)FbKY@NuZoYbDodvS_jGbFb z#4@j*VQ4D8Oaz!jX?q2sKFA8E(K4o^%n(2HywM%N9mTiD&a9v_Nn|wpkg0Lfzf~9d z2ej*JT7~q=aN?zMZFaryfPBQEk$whPynzNK+NS_H3lNF`{1))4a*{yajqxB0;Nx-1 zFNCX$KoS5ml8u660t3i=kThsJ55y=*j^dHkeS|OYD;R*J@qw}6MJh|02-FGq9wi#2 z6FG1*veK-<)`M3lN>gyH|GS|Ye|W{!lk<(_>J57a;4tb;bY~@|dIt+U<`u8x!bUa( z*lb<#{QULxAx+D>>h#EV=J$A)!A8hd;`B=tKC_FCq$iA^hZ<7p81!7PvUb^s^S?28ccH0emBR+)^ydvz}moh zh|OT2Y3$njA4~r7ee2zJkl=7E#nbhQ3Bk{|*)8y%`rD@Y&B>nGzJ=4lSEzR`E}Quc zG(Cq&4@FoRnG#f0IYb#coLQ=Q=h_sS>Mk#YGz4r4+|_OxUHkisMXhkQ+tlSv`W(Yo zY-~vXN9s}7ls!nWxls&+3yl}8z{uau6yY?B1zAYiOE9*I4uwCzHu~CWzJg`56qXTb zBq~4P!f1)*n|!j$CW(GRL!VG+s6|JS(F-0BRLy9nR0_{l&E};sKv9v`k{^X8iwh^f zU}TgZ(?c}#g>*J7ngNDm;Fs+HEDgP(9_qk%vS)>GNfZ$RTrU$ z!qEWt8q7?EfY!)hh!`0c3+!?-W6dB+i-0`__h5WRLb?b)AXv?<>yMsb$V5F90<}_F zk*GqwGl!p&GeP?=&$}fXHT`lbk~t!BlT2a$MJ7nE82>x*_Xv?uHemcEd7bq9E?Xy86 zUL)_I;I_>~hCCMP64jsT(kyVeS9Uz%g?y6xU7J zotPnHD>}yJn$d~X@%r=nz@FUvmlNYJ+rowU`0Q33C#@F9ezDV*KNLP>WRq@_jPX<= z&E%k`>TdT1K?52Kr4K8^pH(D7TGYG#3tER$m=f4cC?z+KX5}zbHwX&xrH2`%x)0#; zBi109Y&9}E%J}!=_h<7OnG2%Q67^=!Ih&u{#-%{Ao&H5LaeEPRq2L@5K5fR!MPQ2# zBA6FVjU1(}7C|7l=|ECFESMv@25l}h2sGaeQH2wQk^%5g7HC4$#o=fQv(F+u)esU7 z!Ggi*?3NuY*-u=}SpTI*6S@+e?PYoVpHeR@`Z7u}$JZmT%QqfXRPD51;yRa``2##? zk$pA594xTkMiXc@>Mwl8l9K%Gi~hM}xK0`h3_8uKA7`~Ez8*70Po^sXjfc@Q22L`k zp4cRyu%h?$uj3B^@9ylRWe;j)&40(-ji`6i+(ab`JtXMxTH?O6`re9|<2e7tA@0>` z)>oAp!^Cl^C*rUqq7G2ZkPmhF6Wp&ksT*hGF=kUe_GV!BMj62@k@B&5NL{5Xns5kT2~}bVM4~T#`|? z({Qrqd0ct)rq_b32R7F(BSr(+Fkx&$Z?e|LOrVodmWdC?{Q}L@BfnfUEWd^W3&=8S z^HcG%On=v~V9yZS7IYwlJGTYhiVA&_ThA>W$xwTgi3$2io*NFHY{zls-MaOw>H)>Vi15P%q*qw!rpZq?^T#&+FI--v zek~w5$`3JDwangV{r+Lq{&s=Edf^oDAdi6dBBV*0fji})=4b|;AlEExP(s^( zg3@uE2h|xH6HC-HKLe_-lCjx$+3PTqacVVx+zGPLUIOSibRy4A#?F@sPwj(GAM+R3 zRUbs)o0AL1T3$_Qs5oB~pup-sjAsmW8)y6GxoK{H4U3=c2Tr5M;t%DA>a;6EU9z4P zpWWBHTG|(X%~^g}th-~LJ}oKzqwR`MSdomjbn=E2C->)DgUaA%haGqD zhI%E3G56$K|6-<42wQMaaI}XbU#ZDf>OdP)Zqd}WefA^CoIb3>*fL5dX@O3xrX#&b z&(DWR1s|{Xl#}hDQ#Wyt&V^{+DHX&3*l45k+{}se^ym&;q#fEaJtc$FgSdDuRA6|f zBL>h5z_J~5Oe*IxC~7B+fH9Tq0AsL1@=91 zP(VsZ)HvL9U$0X!L#w&+C+jK2vuU65@4|sM>-3|!ZrQtjzrRwMLrgQXtye5*}nd4FG-GB8jkdkujQul(dY0h>-M>7@T;IpM{1UgT#35_=*gXj>E zk|)jNU^J(z@C1ib7zA<==uH?GP@2=7KcJuW{qDqD3&rF(a}gK@$b4Al3#+Us)>a0; z@(a#ijE~-ETT&Q*fs8;>04;$)h+L$?d>p}-OLYUO8E-IEk^F}0SvV#L7Y-*3e#7L`+IqBHMDcyC?Y>n5;mkXZ+v|*5GYO6o`wY(CsWTlUr`Ymd zwxCyq%+%jWPudXB(TlrlLeUPOCC{5krsVkWC>i6#{9k0ltUzgen=u=+mWD}Jpe)M^ ze}&y<&YP#Vzii1;W|8T#6w+8;ogC04mdTJKB=y2#(Ihs018o2}1^&cKhpr|YgP@aW z*LZqD+UZk|R1%~AKOAl^JS4j%lUCBV^FjguIOmyxID;V?@wzK{19}yBF$t_x<5UC# zSSz(QWY%hKpBffLI!1(JDU283p^!mPqj$C?8ciq+BYX15)fP^`#PJZ}C1CqL&^Xu=jpSs>m-gS*eD2bPhh3aNz zYs*pFbQtR`2+vaJd!d6peIe(D*oAETOf zw*ar$W1q+Tk_*v=)6QR}b<2Eojj*Z+xRf70fe@ofuVaa}G%HUMv1H@Bte8Y)wPdwq z2?D(=0=qH`Njb7FJEqWpy}w7)7@9s93v`s~D>x9p4<4@6^Dy6y05meH@KP|gN$5IS zo81+S{ttUb$}g1%%E91FmabT8q=r>o4aNUON#0(_qUY!ODay>$#KT5#;`( z(IvsL5TwW(~tC0+#+&3I@z@YHs|9(Q|*+TzABzB zph4LNCSYOPyAF?sm`+Z-j&BXs-IGtrgPGX!pl>gu%#(DNhFxcGRexW~*uDC0eK;;l zek<%KdfM@whLDYOpl`48-zFx*k{xd*&Er@Q`iNFelmr@pMmF^u7?k>3%o2I;_qHN5 zsL`ri>5d3jVZ$_&tK3)$tXd?W8@~#Ia0<}hm!n-1V0~b#vRSRXDQ#UHK#u*}01HR! zf;kBRkq~~Nba_-LHigWZ$Vk&`7v5#A>;N}tFKASHpn=66qpBvJK3~FD4@s9E3^wg9 zBjjq73HjrI|6J?@v?A?CkE*lupe!~i!$~5&c2>8uAM_ct3%V(bW_@9~NM9F8Gh_*x z%VA#4-$;(fi~%Aj9VC#2HiHB%4OB`$k3#*|m-45r)M*1p(hzN+2x#He$q)j_pvVmZ zL@LcoG2TMbl+^F_ z4-{`L{n>!ra(kT*RIkseFdN(W1|s?@v$lWnhMt2&uVfiohU?*KvdeAjED`D&Yh9`_zb{{QBX`lVV@bu9EnI!RcX4PCC{gs>oigMe<*x+Om!hAz)s?gf zgYjq(6CZ`qj6q6WqyvHJ&}OvnV2LkRB{*`D4b#Yd@+>|~f;dJ_AYH6d13avq0Ma5gDIR$^Dt6yy?^i_QyW)xH{2Y=YA(bY4DfB)T#RkrnG5)tLjes zVYA(CHTUcHvMIOY5~r-QGR6Uc;_Ix!O_%36hKGlxgGmS9tFkH397DenArf2(8^xEC zCQE3jj2l|iBHEnqx3kTg5-NutJ5CGAs$5Qi*(E^jg5!!GAIE96O!=(G$o|)jE_2KkQQQXhH5<2o)eAC7LlTYc$$En7DCz; z!Ydq@T~YXNbMQklsP7?z1QExsObs#tygkL`0qb?5HXYa~1`D)TKn7ODOIY@>)RL+-3iY9>gNCajye<1=bi>Ch z(^XS~1O_hcxs>f^l>PP0O?+QmS?j2ar%iSj>^XH+U|7*E@YgW$uxB-9@oriKg9 z8PUop0fvwxGFST+oTR(WUa;@J^|Mp!ua|>yyZE%U&2Rika=D?|qP}zXM5<<%a1m|w z18t&|ak5hWCSY;n(GViJKzm(k9V?Q6^;!!9Xd--oJA(v)&7_32)*_~}Z0{goEkLb{iUK(CI zjSQ946OM#LMnlTzyNcXi4Hyv8x3x-l++7#n@3VbdO~(qM{3L{5@+}{@XD*4K&vKj< z`L58rO$D!oot>SAe*NR7-HyN4m_Ko6?~~KN@ci96UmHsZt?Jj_ni?S0QQP0WQTdiqUN}b2rxIRVJmv^c z*t$oU$awPpqaI*W6q=)}v(uK2M>7&wd_J)(bIYU`o2-yQe6?~9)3r^w3=kd8ggwbdGE_V4VcwOz)8=LVa~Th-rYGd^>#a% z4AnYcI+;3t7<%|UwD5|%mW@qwcHqgC-RS1muUJc%N;ve3jPF#e3IZk_T5~d4eH+iy z(}>Z=y3FQeTw)U1&peE2(JkG56rQMrN~I*#V+w=Qd%;y$@-@H6pxs)~?hI@GPGGkn9D|%I??RO7K8r|!k zaak$36?~sErv?4&yp*i{@TTXlS}SRlOWw*G2YZ~Aw&KpRzxA)i%*V7=B0f!v+pME?cG|3hIM$ev+iyh>Yg01| zVETH7lPe!aZUh{)7)(=Zaa%pBWT{CWD!WGqoh3GvERNRZG1FpfFbIpX^X0ae70rpF zX;)>SsjGW!*gkzMFSMju&!N`btcq|YDg4@j6k^UZE@Kd{JgqL)r&GaQoiZ`VT4wR- zVwr_(e+wD`8Na+73n2(Ikc-$Ev+3ynl28M@r3sh>1tR7a8H~WgM?{&)XnGg0{=aBZ zI64Xf#v-SREeZO|=*El<0Y+=dY`!aQ&%E_upDu>`3gY|rZ~5J+t$1cm=?V2C%Tbx{ zt|wj6n!0)xaPYElR;vB)=G~RF-19ddR;F4Sf}c0!&@kh}@s8`-813c2-B0Jo6Kx0e z@(~GGq{En;E>s6chN00&7qT-Qlv#+J{LOCnWwn-(Cip*Q`R_-BKh9YuVIFzj-gI+S zXq1=`nW#{H81;ndw-q&<;u z=-hV~y`lVTIoErhmuHOTey2&}YThtwHNWfgrxtwij#jJ>xy$|M=6yQrUcpddcM{q@ zKG)DAa`VzF`5-M>#?+m`E;?dly;nGF-7Oo@--7{~3!8dkh!N;083j(m^v%Zn$WZ;e zScWE^EEP1Q!2Gbj@qF__7b0x;mL)?#_a>0=v>=&gD zVwEM`iF_}ih~=qCY>7;R#WQ2@dNYiioec#EfLS|rYPB4RNDuD2TT)nv7;^6WVm3MF zXh`fG-UWCT=ld?M}<<-6`Ghe#nI6P;5WPLw+6zufX zsh%_JL%*Zm2WR*Pxs(1+r_P#?RzolHyov~C8oKu+?XZ;M%8XwvOaF60FkkLRgc4!I zk2Ph&DoIYFOqc{q#P4#=?>gTJ|1lA9%9o)gD6ud+WTbWW^}%6@afFC`g!}Bfjr9WK zCkaDg-Q0G9Jei(mi|U~V(d8orzQMK5TBeod$=`MpbauW<@g(ijEPC~PwmwK~Ho>I& zMwRTXvlto^gNqGjaa(+47DoOya)~@RvZFNZ8lxV^6pN>(W0_A&cwDVC@7*L^&Wm^+ z1QqVpMcX#t#yZ)foIf-s@BU;1(N0(^FAOT&95k*#V#|%?{ofXCuoH5q%;{3K0*1pt zvZezoYfpngUrBf9v=5MyQK}S#5i?kxY8(j|J%_ldRwN^4Jm8h z^%VF-u5W4Pn8g)ux67UxM&AM`gl)hOFge;n#Cf80^ zNEgTDShT1y-%Ko-ECCoZSgu)dKSA40WXS z?03}oa~W-q%QzM?}iErsmO|7s>=Fs^8gK|~ir55*Tl5cqgJS{CQU z?D>r?KlQ_7)zgu%9zjv(yi|h*7vSHS@*%0CnIM8QzuM0>CIj6z9~UL5$&}6fc|X(A z5|Q1W_H8Hd^-5H{kZdaDsom3Z{iM$AmeHw%kH=3MrwLw2;3O;jZO^Nrypb2m)c>#~ zxTbpk?!S5g0q+1S-=5WV{Ta(abs=pyCaO)E_PIiu7M%_(0z%V^N>fd812&d5##)?P zTjw4>?2RYksnHN2fEre&i0QvhFNNkr>6!Mub+W%V#{SR%VJYSLl**Z1^alFv=XP*v z3posanlx|R2j#^v4R?TfSHYjqR*vYxPN_7C*i;@5p(`EZqbiWlhV^r{9VP^P_2R&+ ztT*yDddv5r1}}MZJ=XE>8;Y*ixGcZk;MdkF`jE(&hE$QTm#7!6ZbC5A1O{YbE?}r1 zCXS1oeetr|A`+05zwGEpR?mXYt8BLsVL-|r(2m7YzXS=jrupGP`$sVPWyj*9wpSDY>tze^8E{Ju|Cy~U83pQzS7{+ zPg2zzn{Lm6tSz_M^5|?R;uieuiDnrMxnb&@MVO8Ul=1xDwNRqd-WfrTyI ze)hpWTOD>%UcwI@jW&OZvyfDbd+*)YSQ%{aq3Zq4G0_3MJkG1sQUs@gzY1rk|#lS_#Npc9O*p{MsDh!VeG9@_L-}BUQ zZBlDyODEBMc}fav!SxD{r=7>x@pnBitUk+O^s7W1Tcwa{5(CgDiLl5tvDouz&u?ZY z)LzZ=E_}AJx0}!=o$elIccpPl*6WhoUC=k($r7wnnc4mB=Lw@1?Ohi$mV!#FrBuMd zcX{Dd)@2h{a2E|83n%yRmO@*ERnmmyEnI0*yd&Ia>7zHrwyxvI3_id*UK$-}8K?6| zSasGhrU(6Gv6&Yhb(o%bAN0^%xDnZ05YP2V0MoOUxgik3#JHJaob;_f>1898|Ha>A z66Bk{w42{;)Jlx!ovpy+LaD1~U(4HfeERdxO>-pWPr2q_FIx_MsxgnR0xrsULK6B3 z`Di99CJ?w#c^@;gj8stKY0=DR!!#NuScnLL=8zuPzQRTT1O2PV%xv?48NvJ?RqouO zpn{}|dI(%Dg9MQs*0t>t?-uR3VWz&OUGrm^OPYCFr;{gzwdR`Z^P#VPzhl2uwWKZQ zl=-ivk7*x%D%nkGB1D=!eiJMk(5IB_^S`o(8Aopq!durgs{*FH_;2Cos5U zM|3*DE($!!Zb5-q*31XFXTyd%FeXY?mF2b#yOi3g`On+tYyIMTx#CG&_w;{)tFy9A zo2*)l(^VJz`)qq3IKwN456Ib+vsYGUri=PsW(UdUJaof6TGpQY& zD{jBHjU5!ZNV#)WP9BTfS9kuf!|}lSUIT+k8#~r+Z_LmEPHw%7k zT-{yDJ6}bHp5@*db6d6yuWM2f1MS==_pP<~*mxaxa!i*7FWjnb9XuUkwRx$x_2vgn zv{{Si;F@~2_`zotT8t$Ona#H%x*Fz)@c$|j$r;slB`-_fT9{r&2Lny=bor(z90}u; zyr1Lqu;ZTRa!q^B*c1kosmv|BfQozWMHI*xcO_}b zSg-toTVPWhBE+Hg?iYod?A7K0934_Qp#N5qxe4$_xDwUXf+HK2bJM1H^N`9m(s5(& zV>2+AtF;?59We`NQT*-K>$jhcp9{zBBMLWfJ*T%6Z?XLG>0Z^%=LxD?67K}2FN!d( zuc5vjCvLDBf)fVJB#%DTjG8f`NX<@u7MCsES~iL;79FL1aIqF-H5me94itqG1|fJy zpjaXu3K$?$7%#dxMqT#l#$XDr$D`#tU@wtP&dj1ds+|E~rF&}e1pex^#)-X`{SsD2 zxn`0QBa3UC&6*5gAt2|ihy;h`E*#K8Y@{b#P7Wp4XXd5KuW$Z&5cb$pOYZZI)q|=N z_r{UY05s<}MoF`5n#bO52VerzK=q=?@`FDJw`$v{W{?ewQ^*=8|zzhY|2h#Dk^NTwo3U#cm%$XDwgcT8c5x zk1sNj-<<;iCO13cmE{kN^;7AX7Rg|!C_IWkA0xT>Y4l#%O(MZ@oZ419ZC86$oObQI zAQ>gcxXh3%{$r{C*Y|#H`3OrLFM@p(OFF%^wrjfL!Wby`00U)o`FBb)EG|4m=imXs zFz28rgzzrBPqw@KBye+7kV%UvP6upRWISt< zBEKikLb7^hK^B3*FMycu>x!kV7%zs1jWeKL{B6Ovhj#v8I&men-SmnnMb*&YimVs4 zNo+SM$b=CStVt=i{k@IfT2D>;HX)M~5jP>IxcR63tKjm)^{a&s>OX&VZMkq4;tUZ1 z@D!5yV8r;>v()y3`f7ho*vt{5t_<%yz{Q=;Y7sZY2BpH&WYe$O?r@&u&m8C1P|jr| zZygM8|6nPTs2A!GO%(oTlr(b9HheKzXLw2Nyq|Od;jS-*qgL6_j@06%AXr zYdwvSYsk2JcMs58f(aqO#<96RuAS5?-{iOMp0!bjjxC z+@WUFvMF8P31XG3L?#W2?r2~QJiE#TJR|5}eMN7-{O;Nw;E3NvI(C1f@v<%)MWgSH zg$-_|)wTD0z4RF>!(ENM%pk+youPW;Q^iM{-s=Fm6+mbXjvITegj(Sv2noHowq+)|hK0zO_dCQNB$x>^(zM!Mh@j%l}#{{b*+mpG0DRMM|OMCFTsXaR zC#@DM6eyu128Jh0QFzS3Dj$`09(Td*`RotzIKfh$-^YndlK`bY(5wJI+yR|uUWiQ+ zA%Ig5J1wqx!#!W!%1*1yzePN1-y5NuA1>Vf=^YWOIF@Q!|G4tgy#G^)ve5~XB((6C z@mKMjO<^m`%X`rr1C4F_SO0z>_K>8V2tmX{B3BT29I)e52ODM;tdHXmT;^Vx;p-{+ zs8*i);O;kx`bd}|P>DW^7#$6Nd-j3jug%A3prX$Qe)B62AUMRzdV-oAE2eIU^xu{rKQ`inY z7Ks%?l>HA$*B;OG|NUKtxo=nzGBfuSlib-@j8-lo)Q1QuiIO{EZX=hOOP0IbBlk)UP6mrikF?YHD-rvXXuRR|A+nm?ybKC_P6vA-t(xuCtbI0UL9b$ zn+8|OUz4sGhQfdmPE1^f2|IocJJ}95@((=_Eu*Eqq3um$&nTVXtvkM6W-DvP%b%-M z8+#tIwEybMwO2j29Z# z{!Q;*SMki$PHFg0(0=8WrnC}sy5S;spp)m@f5$V_7PPmIo6nE+C7$0A8)QI|5JAXZ zT#HwR4cC*TlAGH@{nHtK0qP&m??>+WeO~nXyu@$~rdQ|n{C(!V?y>VrE{*Q9UzytK zcD{R`?)_Md6$^xWL*B2I_)Y+R3mg-Mt zhq$A(qA(>e-Rr@=;e4KturJ-+I!wJ<f9j#bGFMH2=|4#@F3kO&N13=}+-Ba}L}m*Sk+;H$qUh;S zddv+pj7nj`^2Ts(vNpw#0%wZirU>e54~O{Cx#6;|4+whSoGz=OcLCh0*O=`JcuWp* zF7if9FpP1a#oZXpWEbe1uxrr<)OD{kFY1-yI-DrqkkRV+wg6& zveTU3HhbE6OBDQ)hIxpIFCr&^c8|09Y_BxqAL(TN+8JVlM;s|iLL#`qJ#2vfhBgU4 zCsi)HgZg*$Ft z8k!$A9bB@KQj85~@i|TSnp8C=EM;Ld0QJs08Arx1x^${BY z0PWvH4Bn0$a$V!L64Q6;0fa_>dZfKtW$`-h-ALP4HN(KKew$n{(8DsWiiInpR4Ia$ z9Z$2oFr8(VTV@i~^VPEqc)b|*ig-CR^OHbyj789OYzxf*&P^7oZzT)Jnn20g>VY6{ zu}8vJY~E#>$P<*D;f=*W+81G63uB%z5)dJ23~ zUpBw!Bc#b=(|oiwM%K`AeZ^T}}YvfRDPo zi>JeZp5wv>TG-{=2lNPwe!us}AJr_t+Sw7({`! z;b^(>3vOBX4Kl_doEN|jLeO-c-)I<-vs%u&V~nWpG`GkhcwW|u$Uh7ZA%dM~tFkFz zPD$oR1O?({=iJ*knXV7w?xbF{os?C!au{Ggiv0a^mt~s~Ng^?WWCLf_l>=!(y@7H) zyu0puKgR-GR!)R8KmYWHLe7&^^G-)!9UFbnIKDntLV`xg@rtbYKUxbGIL!NWKD_qQ zzdzjZV2UdB=1&hv7Ug{vO2DCl@jZbi2ztsU*XhY`WklwWKx6E_}VU=sQzSQFJrRC$R4NGgmV3cZ0{6 zyWtuhs`g2Fdnq2sOKcsRdsONd6~wE&btr7CMy>Pc?)}g4QYT3DGRy05G`l7eU-F_r}Fd7a*3#P3ZNpOV1VeM($MGkPsyJ@ zwKM~TkmH6*k276EB2ebX`dB7YoK2t|uTDp1bzfD+mRTzB;69|V`OPgo>3w{V&w50iqm!Qd)9mWYy zx~Ae7D7`EEkJv~-0Ge;e0x}L{#lhji?PV3ZH$>cI#YGd~qPq@%UU)#^K=dO6%KpFimR*c_``+dQVEomojIzI5+ zU3p)z`DnD1Lj!MphAMvVfRw3lUQ!0z8;eHpzd#COj6E?zy3 z^8K$*ST_XsQ)_2S9_vo~__V)RRjc#t4fZ|oKKVN^N-8%M1_vP!6+uKIx?}t>HTfrf#@dZU3dG* zI~^4yu8IE)uWGyz)S%}jzn>ruttwI>wS%q(Nh!6f`JQ?Le?)IIYt^417w2j1&!yJS zoJLORCtm5{kmriNoBQ?S-h%t5^)AH+5J=`*+ZDR^Xx8gYXvWmf5hYlr`}%ghT%wTzUspYS zIDM(h7EXydmv|U_ghgo-NJ~3Ls06lYm>r`ebKXOJlGm&KN$LHIoU6l^Wu#v<1Wg1c zXeH5Bee=n`#%;;+G!&OFqv|&u$6|SpXo_Mu2)ia97qdcRizLzF0%>gv_+|sQHJA$s zBXnkvyLez|2{hmIK=%=&bV+nWJ&r$>;JW;F3IsHem$%5M3{_efTY-{HU z8wlN*;HrKz-uzg5%FcMAs#q~af$+V40W>`Fgd_LDRSs^KU(K^;i|Knl?dy|+uGJ}@ zekn|QtqHp{?rJqUaMHui zK2g5i>PFC|Tny;WLx-vxMr`_pyEU`wkOX6v1+R`el~#Y7cUPgbMy=e0j^gYanNP1Y z4q1wD;ow|1<)E(JfJ-oof^cKJ+xy$9epW<7JAZxU`ZvzcMcQ#iLmn4*r(i{TnyJH_ zoDtVw)nP2uZ;h*ITcjEOk+XN}bA6fO^NL$iq){9&09Pj;*hvX9J^{lHzFuD+w@$Bo zb`8V&Sw*;-q(b=E!uXvLhlAQZ5Glf1I}WEuaWGpZ9)x&iU)#MAFOdM1l*Y0PV_-u- za^nCT27w66W28}0L{ltFp_YR#7#vuObn99KG9?=)?(qq)O>`VITamdTy7i?e2v-b zKhvu`sT-}R02t-;nITMbfDC9%Mg5WL;Cf1U`y83`XEzNu6o$d}@SsA$Z~|L*V|;ob z9GutpP--Vv(tsp*>`Xk5E4tlZQP$jm;C?2N83@MZ^Qg*nb!T`ly?07K^2}D-!ybMC z7L(te)^_&gvm@TUWh?FO?xyhBiQT&8;E7Fq!u)!3_Fe5|>q-lu7t56USKjQb*mif0 z#`LdAe=HR;R&eyUNf&S9MjEw96{wIoh9p{)BuoLLio!C>5^l2_MS40_a9>=))b6Gb{Xj+(2VJNSkea8!?4x2tiHjvxe=agXz<+dPaq6~ z$4KLD8W{%JL?^`!WCF#e9LOtC0H)l@8BXG(vbXBV;M57IK(Hf8%8<V3)h%_FYHI9no8$v;*;%>ZuRmHL55X?cp%7L^c5YbXvh`fg+7OV*8@^3#( zy-&*PI-PzNT9m405@52WGz;f=caj_^nLscLezrQaQkZgK-&k<1??=AP)_kDl z0t&(!4#)4wuLU*lc!r2~9C}ln7CA*uNR8!_R&g6VB?W=ZL|hU4-9BIC;s*8njLL6t zdjD>^Pj? z7n6AC+KHlWt^`8nPLj15Ll(+`x6GvH6(Y@8sx!pWtawbkYMQbvXvqLpQvs;!p;(+L z$8m(W3gj|&^Xg5kAt4A_WQK8!=DH3N1^piH>-%|C(3!e%Wzcs)51!4S>Ui)_fU zM;8H!T7ZUyjwLSS+CxVBP>GK3#4(8cvud4g1wVv6IJLXbgOfKo+|cVN0F5-6#V)i*uO9mbJhR>(ZL_d3Z`W zTXRE^N972@p7)#aY9lu-cD9k51-f-D&)a9S^V(~!eys5~ zIZxeEIva+U07GL)%|lMFx4)*TCgz`&z^SGSF0uq`w(9)Qdog=mLyzT8BYmS@?U0>veHzI*sFIobgVC&^Vq>&wt>4q| zH~WY_Tyok}lCJ>f4=$NtW(?1F%NEQO5LuAPdEUZe%OEXcDQ(FzDm52Gae(S%+O zWdDSfr;~ey!#%S&_DQ(Ro9Bh#9HT#IDl1tFn=O^2YQ~1|{r#VwlDS+|M<2R8=*c+X zZaT{N+4Fi<0$r4ym^#6G^$K9-t!u&tiG(2tOEJXETvCJdB`_rV6+F7?5$YPCfR?ta z3r*mLXy;^_=}E(39XP0#@cn!lRs?~vb(t2yjfe>RXsBgAK}gc)V??Rc8TO&6L1YP? zF(`K?7@?)pollQL*rP)ZK8a5Q$$~_%VUQyl-U;EivFHJr^k%~YO(cL0gNqObmNhZI zu&Kjkhs-2i0FexHIM4(ZK*POKG~0ej@G(M#h?eXzBn7SnMbJg^gckiCg^eGO(l!Mt z4Y&T4gSfyzj-Vf>Ne2s7&d6XcrMzoaYE;{Ui~AioAWof zxTm+MnvNvHXxt{UL4+^F6WfA6uUH6p;qTwLK<{yW=I>v3Me(TlXzYly`6RqT*>ZX5 zU-H}ZfSA*LmYWa;1#Q4m;(7L(>BfOCWoh5|;-|!-KJMQo)YqOq`-{uxSd;VkS3(^1 zf>^`wz6RbacbfV|?B4AMEDSZiDLvbuo^71dKA-_=B@w8Vgyx@AMMcH6w}DdQud1yo zo6~=n&AeFn9i`3=dz3DA)ie5qJ9ad5P1gnWUxu5%HxM%=5go+=jaqoe|2V(E1g-S< z;K2jy_!^u4biUr8UtaEWoP9O@{-#A9#XB9M(3O>)x4W{NE8MwqE1$+vF(EKl5bgB? zN-alyJ0>Is%C@0xX%$SQA~0eb?8Aho4CGPQ4mF|VqzIHr+yAglz=f>1+2}Dek~b99L#8^i&c*yBJ{9FQ6w!m@ zTpF$gzgqU6OBiMG6sH{`Bm_k_m4l9JjnhB?OjRKp4 zZeBo%eHSOwMkMhGV>E<}HD2$pPUgRznN7_II11K0-?>+G>{2~-t|r z-nc3qw;0`jD;+6x7Cn=Eda_>nrN1GnU84W>+wfD`lEnTU>C2j*XHSmaz4<_Ye7f;N zsA1=8fVcPF%CyV*zRUUW%yHw4liJ>0NwvL>9{gc^z;5hw;_q%;HR&T=`P6?DOVP&o zxe)N(TT7RtU`rp|J_^lfGdptUchfIll)tOu&{tWz;aZ{msJU6pr(*KU1FL7utWWvI z*}|6EdL6zjtjFMWfgWT3SUs1`oUh=9;Xqw7r@o|z=S?7w+}L!+bi?CA2=WU`uqa#wrK2$Y%^riILW#QHN9F|Q}rphWp6Lt zIYTnHOu4?b*P-)onJT`{Q(q4BlUidgDEGmUui%41E}H z&aV7Pek!8Q&qrT0i;hEzybv%A#ZOcihQXx5lH%U!c!>JsFB7r{!ozqv^=YLjgl?^r ziyd^;k?;bEPBu>65s6%g)NfS$yaY3fq=JSbA6E#G%po7_#L5Q}s)6Mj{xQWNTZV}C z(!Cu4M#Eu#I(UpQHx`y{G@zuSK~Mhh#2}9>N#x7DL?F%CF>XX8>Dv(`EOozWIRLjJ zJTs6bSs3c&WPZJjKzhiMn@Kj59?{pKVMcNZDs*-QC9`KEgy96P96n+YFM@@F46@_u z4~dfQQzA=Ym9hy$1O-h}vi#BG(`nPx#d(jYGOfZ;fYfb)Mv(tPuLkll5rkN=Tti{w zbp5cVQ)m@Eo6~eY@v# zt!kb>I?+ ziHNiC59z+4u;j;T45p%bRiZ=uUw#xDJU2Foq0CHkHyS}=U|$1=L*hXUANAfn?OPK?rChYhbH0?0|icR`>4poXs`a~KO3)-7N@if>v;!H`GtFoR9*MV`M+I! z;^N}FxjXx#3NGg>GEHaIrEbTcR8uwFZXJZLEq|Q$UI5k65I7Cbzj}61fAPhUaokX2 zLlPY2Z7n9rEJ2!OW*~?UPsPVqmY$CPI2+3dI8J#?ZE%m>GxwhR>gM-bq|{X{X7@>U zJQlPYYH!4{SK02f$MxgK3|(seYunKLbTCEI`mp=l3qSuBv$;_|aBG8u$@p$q`OEX6 zw$Tx#>&q`|7h6wjZN_)z*Y7`z87l^S#V5Pw2@{uw=@nh3vy|2RHo>E9zb;x*cb6Ve z9WHA$X%&}EJmY<1bdP?rJqR6y^m&x-(BNsDfcjVnD9D|T8f4dNfEr4Mg0wJe@35=R z4qUu9ZOlcoGiN1}rAicwk=1NyOeHtv%4Nr^Jkmt4HlOfCoZ~dqw8GQUt;z(_`n2^v zp#M<`34&okx3u*LEk)!;60*Ry2u_!0$AK$`Um6m}vQg!oCv41Q6%fIj6qWQiK6nI? zD-IcvoMoHItPTWg>WV+k{PvcfD=3r4wCe?yr-$wnBrY?InEla^t(s7Pp@J~_!U`C+ za#~|}+6H%US|&?!R<71MiBSwq$HGF6|o(ZJ@X;kp`qZ-UV4b9RsOKi17JH_vM6Ei}$o4bI~=kGd4U9@02#>=hDsbZ5A~ zP@k4uD;O^_v4EAsM0@ZkS~)F{ahvFUJCEBnc58-S`h*v{bA(g+KL`%1?QH8>iy4Oe-jBY!VZDx#M(C?Pi`bKU{i1j4 z34+B^`t!u>vEX`H?z0#m-IHA!hTRffEeZPDi+lF-p(IgE5>}p5%g8tKS?d-C5d;^p zI3irQS^dxqcT0sCNkDKkUCGpvdp|-e|Gt*-JJg_zK+Mi_)IvPeM^g1W*l3MG7t0=C zcsYm$RjV}4K_(A^(O{^A4iHp^RpE+>g!2G}2)P^Ml3K2;Y~$4FMJehYxuhYt71Mx(x@i)r|e1&4g?VM*2`4x6>W+npbVJ?;w+)<|D z?d!ge-#eUR)gBmOlOhO9kdQitwAz57WW1eoYx0l39QX6ulLqJzq(~$@5?&Ws^mE)) zJ(cF-$;oHYKN(wJ3n>-T#mdUc?e{AtsjL3NK98TI$Mpms$1>@hrnEn;D@M%FTn+q9 zzx~DKb!EWGg3$Sl+q*o=@QWpzf7kgH&nDgsP8~)&`mbEZG9t8cUNlZ>*425PEX&MW z>7PC1z0-8?rPQy}n~Po#7*yg^hIJ18)3RUY`f2Lr+TW=DHFW^?_wdae>3iY-bNcqgIHCjD-(N6lv8(vUt^f=VE-9M${?k6=SQ zlr9E4!g!DIPg4oSv(=sUGLMYmPD|B`c0zDG^{C z6LVzD|7=g08pPx1=!hg{eKOtGKvjH-{JB- z;2!jw?CL;Cgw|7Ge__*1-YOcsHr1Qt&zDx6qlqNSHy#WIgwIkyahGF@ge`x*$72!(aEV zSElV_LGVBVZw_3NC}kRAiGRx*mGP)%dvYk;I?h{pKV;9Lz0OkZHq(B+ptHrw)D!=_ zz4WxJra!RTUrmmKIIaAS0asG!rsn3y=3~$EHEzm!v(Mb$vC^x<4<;*SLFQSDaL&tV z&4=x^P3JSyz9(Nb?Js(ud9Gfo@GqG1*HcZR1 z(a-qq2>*z8t4^s?YR;k2O;ONYgAvKh1cW7U(D~h!vOm3e?T@j`3f)}s>yGAk`p7BY z?%EgZ`PIU5`HN6v?=_(rxG_3-Veei0^-4CUKG~rqJ{f7?%e(Uu)8TYTOBR1+I9MeH zh(DG1!2TaY3Wt}=fP_&fMh7jKM+85GP(uzE!ZjdE(5}+r?wk1%5nf3`qP`B<5)CVX2}+Hp z!Z8d`2uL6Eo#v)|Z_})^jS^}Ebh8CyOhbFOgC#@+y{m)GFTVFvX?_ksY}j#zBGqWT z1aOmPYYS^{*=McSvblp`Jv8L&^o8FZzntj%zJCyNL;PG;@dJ}~ag(9tx$CZd{2jdm zESwui;W5I7n#%bw{#b9B7X;xL_#l!vF)}j+9K`iHLA!Z&`G?YU-@vM^zrei|pnLaS ziplIcE3<$1*mA9EY;|hfKpu*5_?ox<_-8Kz>4dGZZn`J>qE$_NIqJ6Ws=6NSMeQz; zbYI~=;sJ{@0d=vbjXe@TJ+B7*FD6Rwhve2zfAUXhe?32R zsPH4a%cEH&iC*!B$Qo@-jLwobtr3dhg9xTu(wuLl@PReeke_Cu^)O~c#6igQcG4J_ z`XF^?dAd>P#oDKPjyoMss~r?f@;2B$JgT|K$t;7D+F-Ku#7co@F-1d5^spWyoIHzO zi3qYo4Z5MkdYb`6(t={D9*UvK&?_Z2c?Fj`*>u4d5rPm%FdLTjNrWAl7jeg7ei~Ak zP!wnwNyh-ai83<63*ZQFxe%-Z)SykU7x?!rFdB*YiyhG+vl*F#f|091U`IK8Vs_ip zN(Wxuys4PAwOc@xOitUMe<7m|U+OMFd0-+h8M(??#-Y$EGhi2TDUMzKy{^dnU_ybg zRAwXutI~wWUP+D#Q7W)q@(MH}&XYJmSGeU&f2JDU6PZ$7O)Fs7x-(DYF%(wE85Elt z!(0R{^9SW~6-N?oC}R1ahF%HXrSHHWZH!$tXbKpaxYt*AR3snJ#M+!UmOgk@m#=9R zIz}fTtQRSTdD%FCza`>*j|GlqU?ig}vS`Qi*>>7LYN~Fz{r!Eo9N?OI8gSO*HmJTc z&+2*J&e^;>H##&V=W@0gE-=$P=rQra+<{;h@G}_9toig!z`3>GX~y~?U;WlVH{MSp z^6&Ti5IY@oBxI_ysJM!4q3%+s_=r8+WT7?=O^_;Uk~L zC~kM%KTvL-D4)Fj?XgvzJO2J9@sz%yr+~eR$<3#jDzx4}RIc=J>fhy-zQ|8Z_rmSG zUqx5WyEcln1=&pu{%Ss8w0G|-(`diFk}lCETU=%ne86}ElK#qtIOxc?1w!eWCfVl| zXmRY)m}VttJ!W^#($!81#G`w7@Z8c2(hLgpo^&hIK}f)UQ=OR$)+!o^?d*}7#pGlH zW)Kt-{AmGCG0${-99CuZj@LH?32zcg{Ou17Q8c8NFMhUZ(K`)gkYof=;NS?6z(joo zdjY1gHCV})R87B@jrr6u{tu(jwfA?!_TFX1R5_x#gfn5V)=K)wB-#)3#90baQd#wK zOaa^>Gt80}c14Av=b_8>T~HLm29c$`X0FErSfd=!KCUgZv2%U=LVS`f9RjBynL(5w z9&IOD!m~)<*+!!uP7nC`Xm4hkIVM%FoQlBs^_!0s&vwq2KFpjoUOPPvAJjNrY$s)G zIS6Sd83=y%d;N94FOdZJN>W-qD2`JAT;CRJ`Ynz}+ZadQWgZ$=7G#ufLkT0}{53J0wZ2%@od~`U1uJ{Cg2Ps0tTLu8uLTRmIgQ>Bg7W<$c5q&OH%4)s6NoiSua`o3X}yd`xpB-{9b zAx~4;X6RjJ4sBaTT%U`A*aD?z%uZ(~@5l4%k^DjB=9!(pjXx}3-7+lPM8%alN-poT zs+SecwhJd3N1^VGgbcuMZbJX@7NEt5myD(CvY&=iC0$UGPC?i;AgC zhU43UqZbvYQ$lA$oMswbsNvUZiUFtMclO1tA9x%K;>HTMd&Se|cOO4KY+HU!FbG)s z^y4-4Ab{%9e00Ql<5lS_`F3xA?ku#Dz6P0CR6W0Mb3n=zF1*$@M_ zmmZO@h3##Pe!sqk)UwN*IpPtSHx?FHG#}X1hpu2^f)cbGWRD{I2uUoo6innp%ZOK9 zHBjilFyX@c`uIQx%uO5!QiV@=wj1aw&iq{!N=W2;50?rg=ke+}WIiPZQUKq&J9!Hn zSUP3|X* zDlU@_hfsT{r6Qqar)9m>4qF_NtmfLeoN^nO*2XOpiX=UU(@oXf9@hAYGiM;>-nBL4R!JO2$l&NywiKff5T&UrRR zJzd#fSG|SxK2E$h{b?SE%C$C`_^BI`1J*{7c+dMJIJ-0Ior8c_Qp;MDFrznZbV^w@Kwq4>e^-T#$J13(Z+te6h? zMee@q*xi0(romhL`Ak#rlkMFFML8%o+IRV4vX*2Rg_O(nseDhz5F(G_Hr{e1-g&6w zLeD9li<)4}hO4&d3B3GP%ev8g;9nq5!B}{J?^Su+w78R;7n&{&4SEx3+Th~)V8~lc zXZDe{eQZ%hNwmFDmDy&5Arlub(zu+Glnj>Iq)OIF4iLE!Po)+hu^wigw7M+ti) z2l+14mW4Nu#8m)BXhoutd=aX3s=Z8DE)#e{)I-&Dk-4&8*kUcEcBb#K$)+6ww{fcU zM&A|5Eq&ND>CUM2ZL+T?wW5UIj7?JB-BYVX-c}!C=V8q9-cwC*j}RHxIb>XJDJv7D zMsAPLFuaQh;pJE@xI`gk<19#XUMfq$co7^(8wG_Sxy$J^AfSVmA21hbV-|Z<*K^^9 z_{O+6fIdLWi4-)_F8<$Kvm)5qdI8IR1n)-QTnx&$?C(DLZdFg73?E2fBA*Y}td`>s zKg&*Yk8HJ7hhe0^@{lxt=m6*jt^fRjNCZ@MI!*vjuvfV@ou(ePugk_yNl} z&biXiZI@l0sAaqi5CAr5guszcTK!$stmL>8P>-G>F<#_AB$Bo;S<A*vsC8p3{YQ-g^e zeIXW*&+|=ki_BMbA_I9OU^?C1VU{;XXaJ5NtqhR?#Y2l_Xo!0hdLyga&{MUcw9XSn zS5Al&rsQ~{=%pjyAP{z1jOKF17mj?;mWW&+Uy*Emaw_!S)W6o5v%%?q)#t|l@&&0} z;sN&(ACuIivzGmDSz?2U0__f2e?9W8%RCRfPBvLuZtcpT16adgPV>Lh?`Bb!cbv`9s#o@Ac(|cXy#qI z#0+?T+(QjNq+*YHD*0@pz?`{1V_|97`pZBt5&1(Sp{-h1OEKly{BY>RO~UE({QJ~G z$(JvEB9rBgW}p7o3r1nSpm{oYon;ZcSxON^cpkjZ zRC;ps{>TpZpN4RJmq*9@N>%Q>C$Zo<@o7#ZXF(hvmTDNq#(O{iYS?PW3jH?jzkMDB zy4H3mk2E*m*BZgr-~#!gun2OT7qD%DsF@T)z*@q;jA*$`Kv0ZP2N$yEdNEqc?gA>F z*fNaaVMs;NR_l+?Adg|M54E!!>2HX9Mzc_&QO>`9PR-uY6v-El13xPoxEnf`+I)QV z?ZKchLkYuu4}8DY5S>O#Fa($2zP?)`l1oMw7@Cw9AzOKwj%|7e zf2M~*v~P)i*Av1@83$b_g1D=AULp(mal+MFDkznCLH*n@irHf=#R^ZzP5n@|c~`iv zZhoXb$$=OcnK@dPyn-S+8{#oxPpUGm*A51q$ekOk@!o50|B^D3A|E>TS`HC#$q+gY z`s6%19h1=AR=eDzdUJuKP+RpSbX{=OV!G+u`s6auB&%z5S)J@%%(uUnt~!2g@NZIX zg5~h!!)N>&VQ~0wA>qY(;DR6b;#v3TnUG&VdsYtLXrHW>`mZ~=&;~mRMZR2*%dCA$$>Yy3_vnY`7QB1j z{hruXWetJqmprg~Acukm;SCkk$O~i2&(h@RLJs%{mRVyIHt|9bL%1ROT^i;tzQmXIn^x{S{Fg9cv-KzN;Oa}-wRw1V zI_fsKfFP6{CfiWs3qGJVpBxum9~U-GMEqup9!L~r$c>M-4Rgf6OcIRQw?qup8OLGNRVOV zyua(#M*V|S>seu7EcGmRqxf`OM{0?1Fjh+nJ}!2Ud*a*z!R^P0uk# z&F1_2vFpH|FM0^@sr$~YrABb!82j@)($Vp)LN^Iex=6y|S6is$*R2zrzy2%jah#38kz3~G z32%It?FydS8 z;jV`CJdxRen%0=7C=1<@l(l%La8GFC!q?}Y0}kK5UVY1=8Afb);B~{|{+P4t z=b3}kql~x=E_aGXK$1zo3qxG0OX|Q(e!LtOe){Aq=;1w6xC{DM1Vc zXi@sHH>v_xr!o2JtNiPYLf@~>WK9LU^>B~4khNi82;0PPg!5oo`2rOiO8;kez6p{J z#XYXNQx$e_hsrM}AQTLymDd3^x>PyaJ4MS~lDh<0i^m~XgK=Qc|M>hp__laFh6gA9 zE5a3mF@+kTxA4HlEk@FM2X_y@GKg4wGRsyFh;)SYJpJQ4!gEZ`}tTE!my3u=Q zDV|dJ;Mg$1DK@0CvQlPjZ?xuPXE$1{)8Ey{BeJNhMEdPp-K()oM>l#vz;Pb?qP#y_ z$KVX!&_kL7kXWlcL>kFl6i8snY%pNAp_>XGXaQY8i|Ra5Zw@Yk7e(z|KAk?;+`D>u z_TPyYCQW+hj0Z+bwFBmR2no*{{6Y<~rtj4)A~{SLiPW8c)PCFtsLR$uI8iNqbjCzE z-qW6yXbr~qsuThx>#3a>B!r2NxtG58n2SsJ?n?XkSagYV2bj!+rDZTOVXgnQ0KL&bn8*vMr#jerqMhksa4nGk4ih4$CIrjvp%AG(31X$ymbvS@jXf$;nHM ztan7i#g%}Z;#-RopqJLM5G&%#?mEsv*oT>H0@%ZQ5hzZ^@jsG^wj45%i2Xm3&O4gz z|NG;_3Z-^St%%rd5iN?65QLgBs#<$fHA`(VL+#mAQ(MiJ)~ec@Dpj+#s#PQQ9>4eJ z`#UG+$T>O3Kd;w)z3%JY`+Pi6*`I89CaSXcLi?FzwH$!cj1h~2fFb#8#h{Yx(4wCS zRFj>7fHWLV0d%K$bB*z>c2H##o^L*4HSX357lez30M{pd?|BCQ^EH0nVw}G8Tl4ID z=y@5g{>>%t_GrI0Dp-K8iFd;FnW<@GTle(1+3ByQqQH}`xC`sEizg=d?SA)`2x1F!F2*_9HYAER4ubhC6@RVoM+4P?}hn zJ_9*_OZIJMkx;vDIVy^K&5b*CfhYJGp5T+68IAyO*7EQ-@MS1Xu=^ z9-2cHiL>~(m*d@wOLJtmcsw)Q^l8kWEYDc)y+DE+?&r$c{PO>{!u^TUs{X%&zvMov zCIT`{u|NvkQoJd8-`D2(&?pb*i>QYuez6XLro8Vkf*SX4Qx;5o$Ufg+TU%?ZrY~^Y zOmbTp;JQEwGmpYmUm2HwyD*!knyQD_%o702e52?3ez;R)NktqG^-mvRQpurV#YX90 z-S#2M(cm&8G!h!3os7Ft5po@>oeN?8OhWoRTpF+&L&!i?mo=)6j0dTPf7?5N_qJl; zIw&927dXK+E=ICaoB{mZgcKVY;bvchvZ2LWw6xW86h|uQ1MCyt4qeeA0R_q*gh{C) z^)!kM6kR|05z-DJj5FfacqhSmgd`GNHe-j}eHe1a*vlnzBa1*FMx5|OY`+}t;pg)3 zTk+XY0A9`>Mk}6)BuAqG>sL@aVTiC0A3e_iH}yZ&C@Gz-}2oLf?^57^ihzP{sl z|FSar(xmsCN6h5x^7c{&#ntlaPW6wAVA8EUiK9QS#r~z}ObOn^2WgkTOP58`G)hCc zVCu&;wgJtlpLdELCBd+vI6hdfGU%4e)66}_`(=%`U(Z|JuB2-hEx8-5UlxwA-{CO) zRo|T@$2m6F*Z+u7{o{pkU$t7>+FEh{Yr{%T6Bpy^cgqjdRBI$;Y6g}pl`+QY?5xOg zH{x#3lI-^F5G>omfc)cS$6R_fhO!&|BDs-viIqTeibDKX>c8kA^$yIziSTkws9jh0S&dv_u{z91^-6B0e3>7?LQo6*dd#@4`ux z;vrVq2vvqa2M8!{G`2(;>)&?2EZU<`RNEBPHp#Rreh zKw?cr0q74OKGZswUuijSD$2i@3Vt~>DcW*6VXbpMQ(Ar{n0aFYz?v-0WXPA9Q-mRS@3Z z#U%HS3{er^&DoR7Umk0vJ3^_I!u>NuyXiCqQ5@{*5)022nGtni!e4Hc%ZaET(^y0H z;?Y==A=twZ8cY%sW{D~wN})?I~>e_lX(yV&8CQk zeNontqSqh@c~Xd?LYx2+4p*A$5H$vA`Su!n+N|_6RXfcnjZ$(hL={%G1rB{e8tw&Q z#sIziK7eoA5vS*r1)g;NS=TtOF+uX8xPid~ScQmI2t9%=RGub#348-%n07 z?(A9HzGYxNuajs{zOKrXF|)TM?0Xh*)69+AGJ&)kA&GPYEd(;bkQR%o1WF!L-_;Yc zgX87ZP07G1fy4MTjXM$LFHwCrA;A84cyVL?*;(a(8QO>Uej;$6?&67cC+~fWeE;p1 zN!h4UR{f=}-uJ#wDvCe92>Z68CPl9laStf4`F(c#oF(%@G#Bt1Of}JC#qm`(y#@w5 zbxHM<`w=0SoPG<*RIXI!L<(^;42&$wqX=^JsavEE zoMHqK-b1d(MB$VBslz)#X1dUPCdah#wd>6e8Sy1kg(t+FwakHXzfK#DyKnu6?r{NgSCUu<-Ua+k$EtD6#K zN$6_06EcJ|Y~#AM1)eRmFcg6oaf+n*>uvGO6yc0B%GY!NI$=fNz^6Ojh?Q^X4MGufBBzgL5M{QCH&xBaGI-Qfps@Abbs!zn3QhNEBL zcObxY{ys7n(!exh{$RH8+b+&|j30KNnJFS%6VUYXrZ5KKp>1lT<&(3TOZ2C!Pp9Jd z-GKUqh=@4r%xFpQVeyA1H%jn$(Cwv3zJ<}p=5gwOpQQzt{iI~fOb{;}I_p^obA0>z zj$~)H$&yTH`!a15OQ!+ed0{M$Ii_ulY8Wo5CCPaG-tIB?*2-!?=-W_iI>;)U(oR<- zRYN1fBXk1PkD#Cx&n8Kum&u#UGi_4;n33AMeJER1%ljCUiV7iZL{j2-h18W~7Q&ji zTe(UjM`Ijq-`YV5{jg`1Cg>{Gc0f|YCF}B>- zZkC^S=EI_TSpvwxFnU5+Qb7Sg}{^Vlcv)XYy0?@gJKAG3~u`;i=7-G(2Fl*|iOs zmJD@$yZ_kyA-6QQ`%EM{#-D@d$0pS;anh%Jx^hvI9j2SMboln(N zDlkG3RG{grU-w*e>ll{T?X69FNFnm7dj#!w4jCvnHoFhRg}V#_vH%2<+!W z^>Bz|AX#$vi;V}r2RTiY%XKyQhj$$QdM59b=DdAH{@;|6aKsQto*XnAYIBRvGPbqme$DI}9s&fy1bIEA9@76;CVA7v zBao|R{C79U^zD_{&E$RUq%KF(-x-lMA=l=G=!Ib@%G^*bR%VANzS;PnQfLOoX&@e% z&^>JP1YD#|GN?0FRt$FwIo~vNSBwFn1Of3$h)x(FS-|1pXc&d=FDjP20p{P*xm-P( zAbU=+UIz!mG9l>v5BBn;yrjGoC=jKFBP_*~!{Mn)HpzH!D8zx#0#T(($}wp&AG!Kg z+!L($jQH&1Mf3+B^Ty6GQ79M6lWw4cD)}XpKHu4`dEao* zc1=tsL1s;aK}0^0*Cr7L1FOdzme$UCKu*B=eLmedY5%DP7Y@97WvF9sES>1#9^=mR zcdac*j8a@K4OH^DXYL!nJ~SuM;k*7tq%?1SILUh<>tBXTMxC}-N=Xb!>K zFLM(B=BG`C%VY76t2Y&)-K@4jw4$uERcuByj-UbvMwhW8_8ysGGUzLboCTFZHb_#J zWHIo!L-#n_x_`jD!q-5au63Nbl2S}!vV#|Zu|WU=NN^h$efLv7zUcQTnC)9?pGbO> zsl>m5$=N2wFLRHydAO8xv6_~&Y!cgO)sVap%K~T(*;JA@s6J|g!6)feitt0<8iK(mUwlo_l;w5D;Txd|GHeL%E(7U^H5;lXVdvx9N)BmAOl!5Xw8LD&N%0qftd38+)*+S*ntbcaJ& zULaK-O_b%crSiD;idG2ab$6U^<*{hgVxQ$hpkcOf(ZYH3`#Tmw%E!pNmRPzX9!G}RA)g|A5) z0|`N?q5Gb#tz+HmpsY3~=u%prhWp$Yy)rg-0w)Ipxb;kv!LXmUnBq%ldW-E0d>kJ$YYQ210 zZUhY-k_w=OlcCIzUG%er!ng#I-*_Rb>i`mk8b%4VNedDkJ;9wtH19qhFx^eVHJFU& zz2dQmevC=Mw(>NaX_)M5+%IkOH9uvHfH|X7?!f3zbK_@JPGH)Dwq&CW9!0^|T82+%47c zc#`}w|J5W+`pC@r$Ahlb#_IIOuUym-Obo)0VUdMUsMk;7jn$6!r_Y(p`)CcSl(XvF#v z52HVop@Hbw4G&sX9vd3bI9fq2RdNwZJPer(@T_vu+lr9&5KasN3Q|ejvXLzof^SVk zaQXhJetea==X~zZRD0Rncq$Uryt8)vt=m82DC4!<6BFw>??adN8P+X@hAYTXzy(m8 zI-Td6AF}Q<_$-Xo#Duco=sQ{lhhJ_>h)*{UcfT4w^A6s~yyKF~l9$$2@7=Q-#drFl z`JmkOf`x7Rv|6NA!NBX?$oPc_(_Ru?!n>8P6snz(JOiaNi zgRvDX)HzYhJT*G?JRwN+IAl%bUqFJ1{XZL&fkyc}wlKO$5HH%6jmSyn-=6%5-X>=^ z=3fK>7B(UkAWE>?dujke>5ZfM8>F;E`HD2{TmDTTsjjn@(i7mbLf}3?!#H){>Kc{Z z%k>$1nO6;Q>ii5MZDj_WB(?-^18ym9p};sHOy?DX5}qTQqCh6;hSmSNlQuBWYLTRO zyOR71N1@+x*$Nf}R!;@Ylg|8<6=xtsa;#9k93v@rI88DsDL(PK2;@y+^hrgON#Cm4J2xJK_rW-e~lw! z<5if1bqk6tn1s~HSlb&&tiqKAK1Zg}06WD7hVPxV78w$FadEYKzPG1>Iv={HHK_HS zkr%Mz;juyRxSJXTc{K%HMJC{+20Q2+c>*zw?>xMXf^I0+5m8i%#ki&0w&mM^# z1QM1PlQ5zMqg5`E+@h!BsTWEo_xkD1fxhwHtHssQi zjTrKYfzYoVOOTL)=Tx(6F!4pZ=_WozsF%}a-a_vyOpL_^POh8?fRejRwQJ9LxEMij#J@^+H zWY$WxK+r_Q^}`}ldYP${U^?u&y%_auqa2Kkt9_T$#JJ>E7zz^R)I;-`2LF@GhQ+x^ z3V2^v^RjASGkxYd)To$?azAI+Ti6QdvsoCe238zsGi0M&l*%orH6Y<-K*@s%^@lOl zw{4jZAXOh)7Jgy(mvB`Xsylf4UGtlN-5LJ1Tt>62i$dxc4qfBY&c^^*`$4MMymvDA zDF9{MFb;bP$G~*Jsc3O%o?dO>TuK6VF!*vQ%j)#|>|(_3I+;Nqd?K{or`{%tX8TzWsp$|Ez>rHPj43m*#Ukj zQxdl5YND^%>pEG!b)-#GcHpll#;>(A^41U#oog6|WKLYXa$cIG9E-_z!OxwbjpboTdZ+d!gR{|J9^h%S-)Whh5T3r+1ZrW1v`H(b!YW6jD@x$Kdg0>M z25>{lEg)g@avKRfd*A*Zav#$jljy*GZjDbr<=*T4&MY>)KK<}U;jI>$e<9fT{J{7- z3Y~N}%gorRZF0TP3tO=+f}RMOHMwX6voV=yz92ht7bSoPkqVUEcPh6L_)yxt?~<1_ zC`6aWo)%$60q>zL(?qnAVW{nZFTlpcz(~k_Tpj;5CndUD$o`i zmwNbkD)csc<8V|;(k<(<_d8_IgUvC!jFkrxX(p+{U|s|hL{mzZe_-%oMu6t@#^I*! zrQ3esMj*d>8K5g*c~EkAQnfcn4xHEj1!Nk~4e~V{zt8wMW~ls04aN$8tJ7kN3e6(= zCiK7M+w*bIntmre8+iOide$%HC0FViX!M!^(F!Oicj+gI>s;HLb|~OwR(d645yG2Y z6KU9P0GNmx=@BX%Tl-?mc<Ixx})MP<{O zFJ1+GUDlPdi}sJ|>y+2mPJ?->+2vz4t0C!=m7xJ89;Yx@C-5Ji7T zm%fJXt-es3`RKlICkkfn-uf4@2jQ;v;_ZE*|3C$Q@=~~q41SRyO_c45SPM@`(%;Pm zM>Uy+u{$UkB?6n09YWgH_~pIMz6-y4Vm6BdodzG2mvfPtnsiN0B)5usG%^8pbhmwW z-0OJwb>7A6+t95jawzTyx=(W5ZN5MJ)b3F?Xh%aJ$W!NKT8IXs9>00_F_1{J?5 zFc8Xpx(Gg`9Rnk}Z8`ehLepa2;sP<=@p9o>_qQv3x~rA58Z-Cx+GPBa!1pmKr#{A*WiECn|{RiBPE z$~QfbSZ^VY~Z}f8?1gn-n%StFrneG&JNX}M4T(yFTED_g=I0&%I zmpoOa1U5K|t(>iqt(=|jRx&!AHEf|SIX32*%=N8AXd@}}Pb{ey0s$gpUQ`+JoJ zW-!LeTC$PuyR;u+UF}gwVFOWMYCO@}x=iqo8x~ks_B0y(yn&nhoBeBq+MNZ5rf~4D zFw89{x+lDR2=)2>%~(E+1!b{DU&UA=d#v`GZLiy{X9$>+WWl}gg)?@=hmk+qC#TM? zx(6@Mcmjeh!^XGfTux-c=oc5NmxJSGgp$VXqx6DCaj|Zc)dp}$yb2FEejtCPm8^K# z_qgdZQ03_p{bW4i(x-`pN&Y!8$R3^La#zeD9zT2;Ix z^^EZ}%y!|?+@+Kvg>3C5&Ei;8{MB|;m`ejQpPCQPyfRZ@Y(vu%_D0W@I=45szK<9m z#i6{S#{T`|+?}G<=jWn1!7#4H%PKt?I&T z>s0l1C>`(ykusc=*UN~8>lhQV4Rk34D##otMa0R}=1G9E16y-4km-a3WN|i!Tv9&Z zPClca(~jM zwajdogzQ$lZH0k#2_Zj)J){L8kV*;*N7LJ=c0sHWAtNM65Z(sX46RV2DU%ZCqhPV# zAWUe#qEAdRBEgCj7^Joh3I@nocM6r~nuwS7(5$G@udPVFy`Fj!!kd~srkkmIA5IC; z1<>LEM>sNtsVEyG&I-Rq>g6*{fJD?v)A?b9-fS)N!;xyO~sNvF2pE?82La)1uS0~vO z%)jqg^9;b+x)r<>KFL1YcoRD&0&up%yXTDZl%AX^fORRl?|Bq=O0 zWCtssaw+haW{Cn$43oAYwX*>+F=o@kIa#tHh6s5^1}xfuCW*CyBy2`*D`JaajV~&& zVA4QNd83Az_+zc_^~K9hJ|@9reCx+2na2MdZG-=qVOZ#%xqvVC6r#U zmd=fdPaV$WoYs+;?J7F;gKybi%wkWo5X-(r7+&noXZ zHW5*j)!K=w=}N*8iGcA3d-Q+93_XZ)OMPXLj#>7z^jk>UW-8*$N@j5G48uoFhSlt!bF>p^?dzKPH_Xk=%*=N5cDsZ77YJE+4AUT(L5WEx z-&TM}@;@y1;nimFInU)N+m87%TZTlWN#>ifUz6!)=z{$t=$~c^ZqM~kMboE0@jMD6 zKlXfHzw|b@o;or+iVftMRju>paRf<1QeHk4W)%oqwDJ;#xEpmlI<$+Fijta?F|jGk&Vmx; zK_;S0vVw1wq?SoM$9$Zp?~&&#w>cqKhls&+SRrT-91`1q9c}|v&mX9Kna=c1+P>3m zg)H9l-%m1utT(U4qcuWfQ!U|zOz*y7*)0$V+1x}}PY-&V3y!2@(y0wV(?&D~n}3SN{IlcPLuS!xpBo_^M;!LEe<7`T_#q-nz?!xqAHC83fS~iHV@Lc8)vK7C z&}Hh$wkAPLGQLM7(De5a(veU}1XqYWH$dkNNJ69C0IKUX803R-C%H8V6HJI>r0a(I z9SR=o0TRWx2?npugFxtcFwUNwHXGC`*(-T{=gkP^v%GUyFED<{W&;A^E&u}5vSn^A z^$V`VokjKeB{jUHNYKue(;-jv`t*wP_d|S{Bs&X#amRlxl#~)k3l$z~5Yl4PvHI5y z@F8v|C8cK45R?=b%|}RB$U<1*%*9NBe6`n;EFuzhWswM&07Oja!s75Z5WB3~MzaSlfK?l+#W5@jDSQu2Oz_J`j zW8w)H%an6c9%U2q!Ino~T`|E*&oG%wDyd{JW67V{@x%^LPu! zkd*N2STa8*m%3XKtTjs~a*nyJ}u)z>|*?Hm@c+@Z(Tlqlr}QJ8E7IOyLi_l!buzI!C#oQ>Ly21jjHeuV1K9 z9aEy`+!>>%QIH;4tyCN*t1}Uh-OudA`h|UzFO|ZP@(x;39LP~ZsNVp&rY;E?InKO~ z?wrfxoUKzVQ4#eO#N~mY#c>5tC}I`!m*hA%NK+JBbX?uph&yuKZLfW{+#G;Z8SH?X z&FS8I*Ckl@&*$qQ-i6KI!271^5}5`YHB%}Li-*h_SsNB22?0Wv430nuII;pBHYR+G z2Iyoq!i4lME*ci*L?JSwT$7L0Q%f%^If;RB7SU8&Re*6)6!GD7v3vfyl%-2sSt8@S z+#4(^k}sTgm9USkrZiILv5|EQVaH#`8{aQ8gj-He|NQgT}#)gTTzHTS@WVT6hnQz5*pBxQ;k4B&=)FIwSGbe0* zYXuoK{W~jse=aJIXn97U8ZK2dk)yEy%;mUEN96zWj{(mW&n?Cefr`!A0s^zWNsdP1 zk~GkVXrJ40jr1^gdCoc+P0pK(vz;B5#i>vez!7s2<@Z68bHtjqYfWnC`;(*YcK4{K!4W`7b0xqtpG zMXW{LqIH;w@GuI03oBI31k3YbvNQy?E(Hd8{H_Y@crS5Cxo-ZTjirP@X(u|QZ*mcjnHUy z;ysEtR}_d;ep=z%TB!6Y*J~SVpu3Gi7G$rA8>MPPArLZkusC%p1L;cIv&U95+k5w) z+vobA(e#~7KkC0K1g)L^=qsO__|F;6Q}JrX_AXG02C?PQN7BPv`98W|y;kTt3qbkq z({xdFFoQbu?P51*6$&-7n!l)BHn@IE2;}3d0II{Pl1VXhfieT;K}%_mX=;Od>ke}E z{w#KXJxaO%he4uAG2k4^{A28VYbj}E>$XrBCOb-B!}fDqr{%&|xs2&=4%3ZqjMJ4^ z0eny*JBs{Chy(*tV!H{;q|sRhqJ}+qw-vTe!pg*l)wND+Nb~BvM{}C zVK-D(r%1c#PSFn=ydI%d00^#ZKtsXq0Mx%o|KQ1K83twkQl4;clkDD^=jsvdffYYD z^L72yFw@B=t$lqZsq8GC$f4F@TmFWEA&~;GiX}HdlhL4*U{!$qGMLT22N_9rohct^ z8zdehV7>a#M-jeC1%RfH__sdkjQwjKrc%lpM;{x~)D}cnNlG2c85R_%il4X_mTj&G ztzkA2Ex42Ft7`X#e~r+O2!-hhqg4U(iR;3tWPaT+gSb$@S58FeUC;qT@Htxzow;+L5J>QqnA4`^(M~pDTLmg z3(&{gff;nbps@)eg* zZ!FZ}8}At+iQzhU2Sv$~n8aj*pwZxe)VfSj%*#9^=U&e{_*k6siO5LwF9D1a?18Lr zy?{SeX7;`O&~6Jw!p;n`w=Sq(<{>4&6jxEywSNmtybZ)-ZzcA8h^=uZgz|xrf<;Um0u?3aj&fb=*z{qXBQA=qX5?I z`{GhYe79Um$$|oy2R6@Ld9>p>ck)^+_~g5E0C^)IG-hTQCyU0#=W;IzGC@4DcQ@OR#37?z)51c7(-sJgsLEX5~ftiHbI0$e<-_= zA48N2e5fo?AXTASA`YLZ8F0gC`st{`NYN1=n8YQa+1hl}PR>0YBez`MJ9y^xODxrIKW$FAIl zJLC&s-!23pqh)}!w2F*Skng8WOS=iSlTx9WvdPI+vctWKC{>DX6vf?xyqg&Od-9f( z!#MPQnE?hWNzGG{1?V$w6sZYGI#QOzf>q6<0}HbLOC8{?RoRY8DzOqc7ycS9GzL^^ zd`Dc$Lk6x4hqZ*jb1<#sojsae_3)3}Lp%@jf%9`rGFJPA+>j5vrQ%HBV!+qet+#y6 zxH8~mptEytcp=2{n5~WqfA`3Jx1adrVWXh%iI5+)%^Lsj)z!Xtcgam632de23!bu-43XDXp!Xo(<@uUjEg&L@g3(yKH-#i;m1rw+4gv z#Rl6qx(g=EwX03J?Ak*XEqhH;1HJrs*dj^odY_5%=ddRZoOp$SHH1gtdQ+Ok5iAfa zdpPsiz$$?eBH1s3|2eakH_XSLbPR^JV0&OnZ)nx4Oc3H)abqIGBSk5e1wYfHjqHVT ztH_+L6)8Pw0d98V5NZdYVHZLQAHI_9o>cH8_ z(h>r&xkyRJvbm6~aoSt_Ab!T}@r^7>A4TEZ9kKPnSkg!8i<&$7a4ll(nS~GZHJo>@g z^p~xCs^<^-y4O$2a3sujT?ViIt*)6j|L;Vkihar>GIzc#z1aC_=UDMd@akawzsL4@ zqfS6B$NJMf%%uxq>ZvCo>PXv#|AP^=Pg|Jp$2Sm3US=!ZOa~@w8SAIT1(?D}jaCz+ z025hQVv9vA@G8rvE=v?{Baprlc;DKxW-SO`<3=fg40F>X#mkIra%)s8HHuZ8nVE!C z<@525EWK<=aKFh^H}Vd8LXVAp<93 zX^KLaej%x=v4-1Rhmt5Y@?sbtF9n;;wgI%_uW%X&xUqC;grY0kPf%s!iPxDR;@DU> z!EV(J(TVWFZ$;d~wxr}D6ZiPoXMIlVldZE&DorXKDcP;%Bx@uq%k$|C=ztNpv{eoX z$}4(*w{cI$^c#^U_2VXkU*>A-{T0~hYj#MCNZ6m&2&%eUF-4?9pX^erZBu!)3}){> z@|gd!gVfG_gN_Gz$8!B4<-qfHF~>%d3ha%Vni9Kb&z`I|>6^0Jg+jEi&zQb69g3LIfu`cpGrH<-PNYLM^S3ws-7qF>JRVDuSp#8MP}hS;mctBjSoz~iC7sUWX8 zSM742>R<03jeZ_0DLYnuzFB*9INsz<k3!K`0 ztXekIG7>P9atOfH(K`P!(Hl#H@1FXJY1TQWV>dAb-)WpNCo- zf)d_CK346zFDk6F^K$a6m5((3`c!*<4 zox+!Mw~i?%E@d^7t=;`W=261Gg|hh9xSw4EkbKzzgXo}N+O?At6e7d*@+FHuO^TlF zE@(fT{Zc3^FfRI}B<7h%+Pg1)&ZldKYM#|u=Sf*7uC5I`SH*ULJ9dhJFPgu`;q1zV zS3LSzJ-O_^2Kj$Hh&w*XGFK=G!j;t+8W^OO=tc;o#>k#NJ4yR={_KS9DbNtYpB#KY zFz$9w(s*sjF*8;=6!gcVm#4I3neGS2)q-3WPj}6QcE_p5%{k5~qp>oHw^NxZwOP}C z>)xIZ)7K?78`qEt6$i^zE*@s(i$gVZfnJ8O5~y(8&)iZPe6f*9b(t?~9K+ZwegpUVUCdQg6j?H`fF5|A+T|s;`94>Pz*Sw~!x+Aez zELWy@IHJ8J63g{NG&%y6os*a6Tc9f>roupX?X|+HT&c{~8+(zY5}b$#bs4He6?K!b zgkB(_{>2-&De#V zR@EtN$NZu__?TYW)nWFSes)kX(C=BoB7MmZu_OKS*&1R3|MG0z!z2$*C9@+^j zuUAclr)t;lD z%yq-!U0sK>@>Nrto(G0ZXT5b_M~|4x^&Sz3qcf75emA;P>5q0V->>I1=*DRp#W>_D zEn%wrzw31$oKKVnR>ZBNfJ1B)p!+MHc-BfwD*_7EDPXVKau1tCJ-GS3cK7q;fzNWl z7T-onmi${Y=^Ut6I8IlEI(9UxfB%-yI?u|pxyEOos^d!?auQz5)!ZsSEIE|y7F4M8 zSKjMVLk4bj2cKVJ4yN*sf=9cKU%U=7YIr$0{r<&}G20ud@~g72ziZw+)xP z{H&&2Q9rwvz0NwkWgVvqtD(yK99?l^u4~dV3*lT`WOQ^`Y>t<$@=eeW(Q6Trk&*H7 z@rZ2jCxf5fA|3Ky21a1I1uV&$jD%++xzc_gD6DUONSTl-k7$_fT^an_PUy?^2`-yd zokQi|xbKy9o@gsveXO4?23HcQDu-^}!ReMuO40c$csBMZv3ka^Sv9kkS6+X~nM)_( zeQ`)_{l%krQJkod=rbszSn~bJ%-quz|7r{>kHT=oz-*NuVklg`>ih6u@xr~SV<9g1 z)0zGHr>(QYn=jtWJC+l7W%}1gWeeZ!nN$b)>~GQo7sAXM-^SjQut|rO5z0t7H8&?B zQS^_uFWpAo#U6{f;@$CQV!8|4S5l{}Xl_E!Oh5#!H*>ItG-gt3TL6IlzLd#QfLnPyK z5%fdy+pPlHAtjNL)NCczaytByU!z8uk}Jkor5ygaPi38>e5xh|_w(uS(Tw+`%hQ{B zy-OKYkd|!WtupN@RH^vmmyqGUyQF?1%As zt!H`E_oe5y(cmcuRpyZo3Ec--|4sC8n|c1=;9)mE8LJ%_;=a-vJ$+pKzMA`s&tc$1 zw;e}&|Nbl$eMyq2%hRL3_fUm`we26~npk}u?l7R#h4YHP7Qr;Qsg#^E$CO@cr^RS{ znGCDy2kQ4|8rl>p8Op{xatYt`Omk7Bu38mQnv-J8Inr8JHi(gm9geG+`4@Wu=}wOr&Dz1fM%JiRn39=0_rqFn3t=tV|8ebW?9?fJ=%nPs~bSDYKn5!%rXU z?TB^UtqHBMm1o zDG**_b7vvK0c`W5#J*I@nLBHD-*K-iYt26Ru=%_r$a7D6YBg)?OtoNr$;y=e{;>R* zWZuQ8HE}SYK_1l*njyO>Oy8y5E&Y(}?VK**yBE^o`wG zBLO!v{}&rwK$11?v%5g=Y<5{&bn1;;UuATC_=QNucyig z<6!FeIbCRSqG}bYR8=TawK-N7m`jKpqr&V9o*v%VM7

      DGo&R_jJ|Bh+3M=sJMO z-x43OR2T8@>iQPkU=^Y@RKLwK+F-z}GN+TZOzK}e!B+?CW!V$Ddho_tRj=#n6dC=&#t%YQ-!NU1UTq(q zk;`ytj8c3to}!*Qj%aAS_~*_!*Ys}i<+Fhj4Wi{#jg!-0O!L9-+SyZ2Lsm0uAU^mH z)^vgw$nfUT+W=(yU}$=FnyqolZ?CuZ#lvYuzaXENHnKcXnZtHG?YLiZwPzJC#cq2< zZ!1MVqv1Eu&NW~yAIf~EevhX_guMtpj(Jkj)>l>XPVDB&_t67cuDxvfCk01tochi` zaCdc8?1t*y&(ecr9m}2bqG?>@)}(dzE13m>Yu#E|`F~j#9T^!H7NM5S8(4dv=!ckCc2`bf<-N3-IjPG9g_sYPx zNMtgbxR3pL`m=_4sBFD;sf?RvM0P@|iy_U)%;2ue#-su&(l`b9zd3j9eV>iK9cw29 z93z&}Om^#xGt_)JZH{$S8IZJA$GX0}Zwy&V@846D4_r@Kic=g8G`(6Z zKN@sh95K1)tljPY#IQ?d+M7@jR zsMK4RaA1DvNBr{dizhWtiH$zQsyXBCys`Ql3x{^YA}Xv%)-*Hp7{-t#!igXvqN~6% zv>FGN%#ckNvP(<*zEmfxs8Ijx)0n2S_E2Vttiw?567sXs(gIi!W&qCVyz1Z}#KLqf#RT1UEn-*0ub9D1h<`%ox!};- zdb7WK_G1@k|LT6;3gnX#=`mp4*D@$ly6rx__v*vlguhC@#jek#tB`Z-(oWAk?!F8( z$WXig$?2Yix5%5QdNV_3tLCAuY)pBsK)2;Xg`Oz-D)|ogq`^4=tKp=_zoPPf(8-iq z%gzN6lhe-l;1jYn#|#Kj1LKEPeq?||S`Rp^c9~F2{Fp*h_r+*gW`LQ3Y!ZhxG%SZxEp*G!3z+j+ zimj@(BKWnqb|q6mzyUt@M*j8fEYpwk!ND2>j~FVBN!gFFR4BZ`9rL{Svws~4Yr~!c zFFmCNwl%G-L%K^)|!6#;$7^+{Mr$y8BI;qlQZ>zF!ECV)i7&t z5n;e2|3^(Din(5>e~0x)-X-=sc@S+B%bm)HijD-sV;}#LIJANY83sr5_NhA=`z24> z=HzAt3hOwjF;?lA_s~Q|3L@^NPFr(Ysp@iVt{T{_@u~z-ymj5XaqROak^y}^i(wzV zbTlEN%m}K81DfzXqb4=n3MMUfb^B4;CWIl#9sD)n3 zHqEQZsHgd$XJpVb`TrD>>-@Oo6MwXJIkP`fnlv6|1Kmu11XDa5ge*vJ|DRf@?P4KAG_`%(Q z*oJry-Q6A29TW3A4sm1faq4H&jZX#nU#TK~O(V4m%JA%mHfcC?J(Loo+3sYz^1TR@ z(eHjae2L@v;Z3uv1tOXR1-a+q1_B#xFQ=~QGmjwCvVwzhrimAGdhcgLA^ zn`qFr0CMJu$jNE#V5;3iY!pH5mQfkk1nY`}2GG(>4}1#pk9lj9imp-iDV42Y!`II% z%b51e|4{|<;N?T+W?X|0<)AR7`8`EIdCenOI6p5mKg&aC7PaF zozDxXHtcb%)2nj!QFK^y~?ZXyOEWz%b5d0SWI6tC}sqEw0z1jz0aI z|A!^R64E=$ysK?FT%=RZx2>RWXuy`2vd~HP=em4#*BNiE~%{bd1~mNKMY{y^0gEG9h z`Tk6eFdJpmk2a7?+<>)#)0S2H)f>$~L`$Ik+VXV^vhMk>AiQzXutwLOBCeIj4KDqM zqsH;RIX<#wjX@`;f}17!GXX9)k%wV*UPs#eV9-;8#Qj6W)dbeKEH@SE#^EvzD&tZ2wKQ&}M-erxQ#h)UUJ0)s{eO>)F-6T%aCIWUM`D#>3 zl`oT>1CXmObQ$2HnLQl};K;oV(_v&Cd;|C|FgPS9qkl>CZR#JO^i^ zL56(h`f+e~I-&rgK|1>Gx-^?!!JJF3D)S+>a{I^dqoSr*!aKqJQiFiy2xWWaKfkab zN5nQZ=rlMW#@8sysk#NGR2|%PoQX_q98D_p>(4sIwoItFvozSK0fkt+OX0t;G2i-> zA;%!2-LLdy*n`SA!vBM>uI~o z+t2UL-f>xcu_c)$Zlo`h*|NGyLhqr{1V?QOgc0L|r&z!0HjRehR)x<`$ z=J1y~&SF3QTe{)^WI*>YNbB^)eZy$&PW=_TP9)2=lyts^_FMUqR7Hq}eTBIzTZM%> zB0-3m>_={Q%GeGN(DzK))-;%|jKYU_mN%`eb@FrHNwYs3%&KOh-`#dH^LF5sz zb14FzdP5lQjXA{0~K`OV@Z0r7eq3F-+*3C@lq;4Q&(xhdfG>)t9 z2=xoXMzyprV~H^;Clw1-IZ8X{Md>o77^KfU>Az4ei!=!4tpd zZysm5T^g%5$qEjSmjcVp&wrb9GHv2E9mr=4Niux2ZmwpA{KRD_2IyJ%v=#F4QE(l% zhp!d~Cer3d*KQSqc3r^FDZeGy<5|*SnN#f)Dc3J6Yw~V4h`Izj!ZQWLD-PhLsvHvc z4bN8>*y0x%5A6mv;##1QuCc@ZQKf-^Uau_}H(4Qyo1;7bOy;zJEW=8YXiH zD$7PMP?x5M;9^PgvSbj&VFIXL1E{A$Bihx)NyEO)(*E1Netx^=TOUEr+^ij1 z;#y@@4v?{$`?mS1h$CH{oeRVjnNaCojCc&ho;LRq7IYB*w0Fz+KY0%iWBk{f8Xo|6 z)xn**mYa`UJ~$fh_9+`Up_Kwfnn3@~&lR{tgrTq61&)3}l!7*Whnf~@Ux)Ef5gz^g zmfq;ga#*)$nS}^-%4CZXM1HwSA+9nCr*Mdkz7aNtaL?wbCzRI49s8j$XjnbpyX~7e zPzVIvLa!x;C91ZF5?;(HPnz%Hz(OjJtY(-3-P_1dU#Qg3d23MLMpA_|Px0f*e*|cd zi@feW;gr!}R}<}|GF8-;UFM&EUR&T_`a(ajC8)(aC;U+hwN)!Et-Km3D!zHHw2gmn zu(YtfNghwRH&ECinm#}~%qp*JX_^m!?6?ss2;eMfGijIO<5no-OAiVMs*-o%a;?!0 ze+cTiPe+(6HUri0(U@(Fa`nF8yHxpY@P1{y+ta%00C)f(nYwOU|1S@Qz3^-qG3EcRfd&#t2y&1d@Gr0-V z&%2xLg6wroBWHAV`_Se`>bCk;OPAlPUOTOY2k=sgb~V7J2~q)>r>NB{_~v`~D0zrg znwPUBWugsOuqVo^a0AqBn&q)ZSEggqygp*H*}flG^brcRI>Ho`kj+mGQD9AwuH;T* z9CLi7M?fQcyqz-mz2jtf6uWO}7_1$J28p&YW-dqxm*J%;2NkHv@$%xMkYfe^`Am&_ z84H_Jpou78Al(EZnDPf%2Qoami#lq{dRcQic|7gZitUnqoPVZlRI#^6QjFF>^xTSh zJXQmnZRVRleAHa+Dy^RQIWgC!=>wo3#ueN5!ldhApBOW3V$nI~@R}kD!YVYP2w8bL zNf9@xgNXT#v(hbxpv}g-r=BqbRbS`cHsdNG%aTIZrp&I~firSS9#wxPKb^8m1s~>m zNa60epRr=l^T{)EVi(+%*1EI(K%7_%pw|Kh_7(x1sIcO zpElgb!rLEflV5RJmq@Rs?V1~g12yo`bIWX~^3fa6)+s;zJp9(TRIhTYZ4jYPVeKKR z*7yr(!>|{;k;I@*0Z#U9?psu00(3gOuunh-j(Rfj_eRKPsY1s_YVLBngze(G$U53G z4a65?!SUHt1;cU-Lek}N0Cn(1Y7m6+M=+p9Hl>&u2?k|d2xLq|;^X={pPyxP`0G+2 zVg70!B@8IRpMB^jrzSh_8gVj*&lSp>w5Z$gEPR=lata@huUs#>V3yBD&JS<8o-4b2 z?_ozrCPHGZkB7EhSly!4J6w>_!Ho2%o2!-`|Fwb9TW%VVCP9jB$lG!qXUC`9wF_b2 zyGz^ag+NP4NA-w4jM}#)eyo!Z+9#56r73atKIpjLzQkwKe7!VqooJ`D<~B8RKq8l~ zivns?vr8+Zv=gc-M&fkM4i*CP%C@*E1@HVQ8=7!fmW@%hlrgq?IRH*(Skx!646~XO zL6aMsBRk7|B8XinTvhnmA`<51*;9XP3RWo*c>30hbNJf$9CuE<>z&xJikrVn=xeP zb-MWNxQN8V^>W^ClG69BqX_M4dLGtoN;;Frlcz-qze_H+R>J~Ur+um5bB$lzyBK-T z54$7w2ck94t~!tF*|OvV_}#D{0hCSg`~~+T$oZDeQvlQ6;hXL_b5Vc9F!A>6yg#mi zi>(G&j&DBB*ls0f!3lL$ulFH2GvX`R{A_;M1}W9Njj$#z(3`)rzfJ4J&QHXfS38&<1ZTlzD+bKQ-U(lt9rR~(vR(L z)oL=83E}uw%Z=4|jCGo^c#3tFBWae&^_Ed3%#k!gbU(6hpH}?F?>oE3M^i!5DIZ7D zg1_weQ9!2%4?e@EJhw=E6mm_w2fYVtB~6KmW$B{^=|n;t3MLx$uVwN8EUYyk@j7$x zqm`*nh-iEMM9pC5`^;4${2GKD-CKZM9YpE|jppS*1)F&tFMEw2xm25!1GP+F*>IG@8zUIhwY~e75`Zl!>kclewa}LJL zCe|~Oa4Q~xFH-J->=cUZGrm}}KJi#}9MbAH(iI!9vd3HXG z)=WAlHE0cyuQ!(*8^f00RLmZQK|w9LYHB!iJ&v!!l)^t`i@ES6mOy6s6Q%`i za@A!R9_EMPPjK}XGog~zqNCd0cac*4iVMHgWOVE;iT9P+4KhTgn$3g~uap`!^;OGj zxJ^o-;pA3qdvXCj+RX-flS6T}^ND+Ms68zN&S^Og7ReqAA8=D>Xs}++B$RMDRma0j zY>6m+Obt7U8$f^wxwfN0=KW#qaY0Dg!t3lRe8`yDi@6zSn_ItLzyuI+Z;|mKN`}>8DrCacwPF8F_eydhMyLov5 zCR5RJQZ6ymOWg16D9nFw8u6v}0umQfOp_{`4r93wB0ul+aM~?i!7M5Zk~4fgeF^k2 zXNvxSFo$>4l#-bwjf&W39A8Faa$7Q}7cEaansQk3`^B#0(@fR%#W2J!d_Mzc`A)}Hh>z4(D6|dE!t?`7MSMif{S$l(JK|N8P?oaN{y;o zJ9phTfA}G?yIP>M8Lu*P`gG;nK-#ZvV&$=D+f)ugz=`@U=i&n|3+Kf4Zi-UzEN3+H zlOt2)_Y^A~pE@SmO{76iD$DbJm|aVLju%eRoS21IygPysxmN+bYR~7Tz~bEQfC53U z37?Hf#vP-PH9?*rn3}lX+1{Q5kFv|zm)cB0A@OG8SXgB(X@-JJt-b+7=Td8;*|mJy zQ9YuFZV|4HV#*NfYYeZkA098~&}) z|I$+U)9*s-LIES1BDz63x(rmjU?uJ$y5#IS>6Ur*#GkeOuiu9-T_-8kt9xC(ddW5E zHTzya-ge;9j)O~$oy(YkACw=Xgr-1ZX;M}?J73oF(|PzVv#e&&Gv}#dPrwoy{IT`o zhvo60^uhI8yvVq4Cl1a9goTvsVD6p;BWVD2dpiy2jk=wMgr6(fL5+-lpt+algNW)= zz+7vKUJXn_7;C1z|H_WdQf)VFC*q3v@OjP_1hiYvw4*Rvv}+m{%k(^&GCbmL*>eiVkzA|1`SF$}m-8Tx(cN zYfjL?VIT&Q<6*jxOXbLL5F;yB;i6*x+u%27g8xR)g?wUI91L3l>=-Akb5f<;AW@{= z^xp`q-p~}q6asptmzzZ?G|MXGHIcV%oqm$+93z3r{>nS7TUVsIK9Jd-O&C(UD<(a+U`Ez2|(wK?cl^x(TfSpKPH>Z02 zlw`V5z`wUr|mtT$0$b^4QKO z&1``ThuLFMBff|Z`rbyHt>I9a5X3|;U1FLUvq|NfuAgE{lWc5Rpj$65?+N{gPBSEP zVnLsDDCb^}$Kw+o?k@Y*_TTSsB0*G#^qGU?0kWQWi#7&RDxln+gZ6g7d#Bu1*^&H%MlQ-}SRNPP%p>2gY5Jf-` zA1&_Sp@oGKH>c+DACJ{rc`j|Ftdf`;k|o3Cpw524C25K5V{_daYP1nK1^Fg;uP4=PppInY}FM-Ah&&8BqnswYeuL*E`SJP^hDmFH3 zu>s*&cj#GSi`O=i&Jpn?x3Wzn|2W~#-|A1E_3tB(4e<0e{(MV;DDKDAuMV2h8xUmF z^A+R!CPV8Ew>pwXG7cN$XkODt?Trb>9S_2=X5>irVh=6m5HWEK4M{mkvvkROHWo=n z7Sd9EaX+UfkFbb2sixEuL9`#N?o@U@d*Y)6lsH>DSsqVv;l1yb^Sp}-j0uxF*BYJg zd+=x#t^8m-zc_w;sI*KzBCZ*QPZKq$ z!T`naLe-!WHp8(@@bJ_mcr`L?!Wp|3IIt}SBBpTEg6w*T3floD+ z8qPHy)*SG%5RjCVbU|SvIRVywH4~<3)4QzLQhu)IrM7quv9lVAplvU4M6?kqDV%YI`QKe43Rj2YDGWOwu17JEzc*H(Y+PM!Mx9@m zHfNN(^e^2FCY&;1I%prnqLY((E*3MgCni%(aYE32ldz~?B`M*(;u<`xr_X5PRATL? z@k?Etv{hbn-0qlZDXq`t&Kq*j7-lO-aAf$L=e2bFor(K1Oy3zp!^D#gH&~`dBPmCz zypU!HBumSxA4x8Z!s-w~@bH8k62Po`38w5%pb4xo>zD5C4;UW4dwr zbq@m1mIDWUKMv$;Bq=E?Ek%VR$2$uA6DL*>7v^J8%U!wf*7Mc4BKAY~X1v5|_I(^T z&%61}_$u8>WL)K(9Wi4QpJT?=>Q%1B^O(>^SbV^tIb7`EV6M_qM|Y9uzcfQ$KB*je zq$jcde*A14d&eG%RBg%i@bV;s@E7CHa5A-VZIG*eyYqv^6pFD@o4`WvgT-j1~_ROMofVPb$Jitt9fZGe$uE zn5!R~s`GxS!1f}SM~T;B`(O_zUg)8=RjE$EYNQfs(}APx$z4YwJN~Tx!F7_uo`tMb z5_!(ypCq%~(v7*^-Z2Oohsi2Ad9`!3ZmyVKyordgp^=1WcHq(9uMfA1={j9dA+kW* z29lubjnGO}awFoP-P3g=2l;L%uA7iiG(K;W$HzP)d$Iq_3uLt! z4e(=^YG%1A3Q9`Ts|@K5W$7hw&lum0Pa7{`6tyV$MSh|JQWDy?q(DJ~j*1ug3PTuZ zz!67oWHdWGd#noH#sZvZ&te|O@$N@Uo3fRrSD~ALlL0$MF&)&GDW-}QwyYYAaF98* z)PO*3vH=g9l<9mrP9wQ(aVg~T^9R{(_cs7=HGVstuG3x;=$pH1l+$uoPunAiS;7?~ zC4?7P1S6twkN=R6)u+nAKF)`j0O+{%;N~AdOkX+ z=4&rYyz7F&)j@`mMBAe8t3<-B*+cL&A674G*TpH3UJzXpIwplsHQzqq^0qW@D* z@p|tv{J58X_j%*%QIBb&fY+Qpo#X_ITV49~G*Xu)@ff|x`3*jumAl_dz#~b?Qo#u% zBZ(Ed)WaV}7ak7)+Omt;RtcE>{4TQN(NVm1wFIg3TZ~1AMKU@-Qv!)Yc*$ zu6;t&Kx_FLc;UgF#whMO8Rg@iETZNyE9iQwI^M*n<^cO-C#QFAi4#+2^QYp(!rTJ} z5qnh-6^}_o#4!b*u$RftT^|wBpxduvkH{a0xxUaVu)RpezC-SQPHq7vXb-9>oKcXR zgm>LTPHlN&QHbB45SD9Xk=&lqU>+cDnNg0>it|3$efblx5pVjlz1$+4gr)gXJrTWTKlsY!ThEV^_%?zp&*``4SBou2iu>=rb~l~r%qye?yk z-{+H}9FMFfw?{P>>t~^q{(HK)k9QL%tH@4DzuNf9>%UVsm+w^{?sZqLyRiem!k2Dh ze^69(J>Ql}1YNu&5De4R!p1Ij#&TE@h}~G zH?nw-&A!}FM`?^-{Ig$u(GhEdT-pEhA`h9hcdTCMh71Gho2n=5XZUIoW zDS1VsMxCP_Y3Jy{qAHx@1Eal5ciq<+77^%4z%cde+S#m-d|6i~hDG03B?Wp1gCmyptjUQS{K z;(pf!LP~K#?{kw)0%Y07O2CGlMCA6i3u&k8ZLs2ET28Z;M@u1JfUyPsupBeGd*mTc z2=`4hdQnPBj~dARpl7j8)JI#Qy-##Tqz&3_oFs`Vf31J5qPXTjXy)}D-aXO-FH4Wo42Lm(X55V}|D zgFufeS$`i3ZnSvW9I(juUO~C{UJ~Xc`Wrs z!ygP-&!(-iMQd{#VRLD#EDuBU)07;z=rtJ3Qj7LsDK!8C?Dy%_L|4!=u$~HBR zjDl_A)Z6X2Jqms*)yo<1HNd-;M3)t+?U6gD15aGcQ`z2El{otY6Fa_1lvS9Iua8dQ z)>Oz~o}$e%A8Qqe=x}j8->4t$nlo@-hVyf0Y4mUs(r5maT@3gOz{2FAep)Waoi6rm zeDjw61s z3KqF!41gCTQghp4Ebiy%?2j4Ki^w-uv?!Du@(2r)l9K8JX{XmKbzdb!=d#1Y4#UPS zh40?ptDtu<0jEQX@%<(s*xat8#W}!O#b(IY<4)Typ^AR{9{V1Vz*#CQ(@WTsb07T) zplMOzarpH1k=sL~fX2!nm(v50JOkqax*0-Bm7xOw+PBsqEcmnEw05*Xc%r^}&>`w( zRBu@Kxm)4}ZBFc%oOt2VVQR*)d|A=xa%E0g832Dbwo352YG=Xe4LM*U{V3+wn`*Av z!-Zj#A|m$)qLo4*Lyb*NPPSB1opWH!Fhb1jFDjk%kiT4CrqO85$W4i)j-Yv^LFJa% z&Z819)ZTaey$T(l4v$u5+HuTSA*afs(5t?+(u=#8jtI!YSqRyHp;Wy|wZ!Ys5yT3> z!1J0&+wZ4CKMs>y_;TE%&~N{*3Gtm5u6_fB*z>X!K9ii^nfmmYa20T#s_=qAG*Eq- zsoF2yAyHAUc&ykKOuGS@e2OYF;oy3fs1ts=H86Qz8IjxVtOT6vLr++5FE%+DnONg0 z*>@$6mbn9(hDxysa82GCX75#m9P)SRMD9m1vqPZqJkJeJ{OoMR?VLuRHp@6%fFkJO%$VY*ahA@^Sy*rE z<|dEg5lx87ZwTt!Hiw+fUz$;zc+wW{HP}U(4R$_`#2EtxctNMnN+B!`3ccpQW+(S=Gi1h?&I|Q2?)cf;ZPn_&AKYLI zZlYFjD4D8=c^58ON3`zuewFHB3V@Sd5nvGz@URe$F60+1V#K7Ray#vqf&=Jf@|MN- ze0HDyPy~AWT0oaNyxhLlk^s#WW%$Q~>F+6u_h99T@d@#E50l?CZX}k!Sv61OOLC3u zJz70RS|#XcT<7lR!3fpM)g{lC*h3@CP+PJ+IPo?|ld1pn+MPjA~C-DAd9hte5a zF>$~3jYM5%3R#4huPQT$Sph%Mv}QV)TRjAQG0Q=DTqF&UL~{aG4|Sre@Ww+#C#z894IA@Ne$*SZ#mhAyH7L`F;v4rx1|yi#%-u0VSUl@bref z;vp4Ap^an0j`EhnjJK$w2;jtCMdp!bcRZhn-84U6a)@0{sv5c;w}VqLPV|&3y{cgd zTjD4kBE_60*yn;Rq8}R{R3}%^c6RePKHksP%Q`D70DY0YG1_NFVc!5(Pp85dMbhV zhz}DT|2x!OS3VyK?%&2>Z{m1tV$~(ot@eJtUo&}N7CUYpfOebjXQ4cAUIlp(`ncMc zD^`VfTag+bvX1DJhG_Jx%plnUG-(cBO(Rgo5I~et^b7vf9f8HJ!H$={^|dVZYF0Uj z3-bi;vCAQAy+wzp-^BN%eeJ4MC+|KK+gN|C!}AF2ajXpB<3Vk2u3l!10f2<>dxg5u zXq~LVlB-oPGRxZfsPBIjTfpCd6R}|;<6KH&4P5TzRg#_-OzUOlY`^G<$E~1%hvdAw zJ`)4GuGD3?uW@sou->jK#j`HN{EUFOlE)v z-eF;plLtrDw)AYpO%{s)F#WYA-cZyX+AK!R9w%C^Wl{pj`YojEZoFn7s}f~+5Wue)`_RnAJ3az=d>RJ=KO}0 ziYVyk%|ZIg@f2&`Pu#|m!bnG5+Pk%<+q@tv+YwEocRJW(Bl?Nh!~wkB z>+pDqV^*7E>O!}m+#aOS-MBbzwd)+^_WNV+<5u0}4!_ey%hjeV>k*tSLFJvL^86MC zI7|epDWkr35qCHn=3vn>pLqS2g_ID8hpz~N6Iy>2&`zmO;mT>T*<7;-;{}^>1SpbD z!f~Z3891uud!Lbiu??R#d>>ziwg|e+6L@`WH=4D4<9wPR;5+E$M#=>Oh>H;eQvKU_ zmoP|pf{cI{P@haNBpMT~vOw88^t&V8yow|jieB9}3SSPWT7uJRiN{?(|SM;jK@=ow2kpAfs)XzzMx??Ud$ zik~N1R~xnK0CB_1rykeE7eaoB9D6)mpz^jnsR5B zZ|B!JR_eoaT>=WS6S_BHIxq!eSI}kz=8P}*g1>7T?0qLuO;l=&l9TvGBDoq2XTD!lhaZ1V;Tu^n4j#4@y-sb$Z}&0kFI4=2ZQSwo zH|b1eP3PT9v3u-2)W&)9b0Ma7Q#%6O}dhGk_7=Xx^~Zl|Ep` zJYFo#sr{h2vU7h^{^q^h^NZWXyvN14MixrJC9)3t%o1&xvWsyI>kI(iYPk+09u^S4 zjdkWl1}B&1u?v*!?}h0)k%^w4O08ZG(y|+N*pR)}_C0DcpAdh3IGEEl&d9xiuVL-3 zZtR1dx5@GhU>8m;@ai(YJaG=jn(jX2f0plc z{w4@dRrCzx9g3K*`El|W2Eh(iTL(VtxAk_CQ7(kUqu5_Ah=46Ijh}zDT5gW=Hxcy1sM6$T=vzx3}>8;bKj2*g>K( z_Z@{ER#ss{X(zCw;^^{RvzK${>(}f}?@Xr;10VLA4Q8;n+$JE&=}I%g68DQWMaxgC zcXO9WZ`%g>r_QUZJe3x5GQ&{}qFmKA_uKRBhx7EcjN`T)gKERpBqzVax`FeGvCz|<^I5Jf>kmm)P>3Jh}(AE-LT}nJD|S*O>A6t-ADvFXUv;aYCcA{Z;wh!aK{w7 zkR3H^mvpWA`4Dx61Nwt!^&qu%KzF_j-Lrz<%@?r9&4j zt*r%mo+l_CuJk-}yaJ!aC%}GrZU?Gfx&qNO%)=>fhH_o#(ffz#f@9?)sEArKU!V-O z$Rr5hY+=>jrEbv>z0~`}o{cKc@DmhnYM=ean$w)fQjrwPR^6f@8a}7JJG*Fo(geF0 zDX-qiE0L2=v(!tuy(Q~btV})oV|Mm3+>|qycwox=P`bp{e3wy?vdjPXWfjtCO;6NK zpXa2HG+E9^MynT0^&R&N6_gZ28X{FKjLVrWc-E>Bf_Tn?;Deg{TA;=q^umApY)ey9_kB)Nk0%&7JQauHv2nLRorciLj5 z#xE>k;4Es!2kB1_?X)nvU~lpg7nyIWg- zZhp?pXl}k|OIHl7Idkd#cJ}9Tdbo4Asc!=fv-gWAS6pEopYp_l*$aQlq{1>nFdQeT@p~`*x$7Spv zG*qo>5=y&I+fvVgtOH z>7s0Ci~fRvi>FYL1l!yQ16NW?>(z|l`5xR4e8_f~5xRVKy8e3NocU&lQJDl=8am^NQf_n zlKbXQ_a|e|;nw$LC_Hs@$X#Ix2~}(>p0g#9%SS_pu}+>ht6%E$Y@D4{9diBwNMUOA zIzBdEBjaDKZ50Bx(Dxq5-8ZDJfgPY%BGI?H_qQ7(LK8c=hQ;{coP-b=s`ET8R_E%c z0g0*yxcc@Zw9H&eGT9WJy7r@FgZI~r0Rq|$K<&=XqO6s-l=I+Dah0rUmDJ9xWDQ{C zZ6si3N8%Bn&pw~{hD8O*c2p)@(#YQF<*RT2Kz-|Mbgi%PM6IZ?APx` zQxZ-x_>cLBByhSh<^0QDlFoBa{`2CE+$R(=zz$x#L+AX@FaVOMPv3xV2DMkDe}76J zwV$n#udLo$|1j}B=3Sb-z)i>PFFo+d!|!K|{LA=er9EQg10O|`jx9u=Rk`UkUr54# zt70~s;PD^nCvX^2xo0Qb79PdfgjwW*7 zo+MQ4P^OMsFz3zGBynUf{CIcrX#9m(mFwy4AL}1r`;)e_vV3c4sm{~-tUcfwG61*0x;d=7UK#xgFctqF zE@ZLTO8;*w7roqINOCuIt*m`vlR@9{KSENF(7L&6WYwIR01WIWyerh(9z|gj1MSfBYM%9MB^6ASXm1-q6T7yGCnIqLbt640vbVJY#=a`FlNH z%)tOz1@bIgEMmQot)D2;|MZBy2rk&jjYN<6gsS(_>Msd;!l&1GlP_MQP67rc_h~aT z7?qbZnDpmMtif-gbp-#uCQdY(<`r-7Z+tqMHR%AWvowg%a94hAP2bZCcbWO5w;ws^ZyuNUfQRhrkNXDsE|JyWaQtc?v zZq0L$o$&~$ZL+a12LCbZ=Uj{Q4eustB24A3KsX(=23Grdtv0zj zH&$nrpX`zjD1I0a%Nq0K!*pIStDBr!%@quJKkjh$__D6au+#DWz9s@PGviFEN=88u z1u^n`*Pi_>DUr9fM;Vb_$=4)bvrsy*>=Xoooy*__X>&d=}%%1Ln)y$hj4TjS1yi?l` z_R)QZtZVnH$fu8-=ZaJp4P1fMe`bzJ2fmKV!nX&6goLjG&f%+*EQ*Nd}1z~6JYd9X8qB~54P|7-D@tu{Q4zC+um{h z_o7Qy9E`oH$?ogRl%K=>i*{ol3rNgyFczi0;JQW25nhb{;*7TvLK04~(XbWcQufmH z^tVpZDB+f%x4!_nGdQx*j0WT%XjV9`s|R5vnhO6=(jog*6;6HiHxH}lMJ&AF`v*azIl)C{1i#zBqB`5!5YO0-}4?N z^?v<|iSzyBznF=V1U+rSb0+N$n{JJF@hn$sqJB3U2oXOt>`}1=ZY3;EjH%yN)B^Ee zSbt^ojLvfEe`iw?C9LPv-^Kl2pgk(a7RIUhbBLy|;>db&ahY$!-Qg^3L?fU~S^2Y} zCTdp%suG@*{biG@8{uUuZ7C@Fufji4_;#-(q_@a+ba|6%FW479)0zDX~ zxLoE@*C=b%`+nWY6mbl1?@u!h&_*AnVJrBg|{aC+3edc*Bc0%W&$ zxdSEl$&a-wT)AZgp;7>kc9Ypkun39WN7&Nz%KG(@oweh<=d#phB=^<0j;)h}G+lH8 zR}aqEe~v9<=YzTZloVvBdldfB%-jj*0Oe^#ipUJWOg6e(8EsIVqc$=;~%g7(i$ z%oy|rm?f=;C@ix*%A-f)XeU6&gYOPCAUsFBP6LW=ega3J1 zIO)a8{qiHrM?5|ht``K1PT<)fHCTM--CjQQXmM?6xjn%2m|=*leS>Z{DkBCm`h)KM zyoF3Hy}n@5Xb-6D#Iha-X7N4DLS<$}yWOOuo$L^;+F6z=U4c=R_v={evm)KhASnkw=w^v^oSV{KrI2<;n+0%2baC>-lH8<{Ca{5Hfdp)*y z|5w{re?=9wZ3`&fAcAx=64FSDgtXuwIdlv$^pFyYgdp88q@)5vI=~<$>5$Uhr8FW8 zh`@Jvp7&ep{RiIpVa`5lW}SWReeS*Y9oM<9FTf({vVYiKxcU1{ZO+NFvp@L)zGnvl zzP+44M{Pxc?b$z_Oxf)))coEC{pB?yCJ6T1L zh7Gup)-}f$qj{{Bc6R}%t0cD9uvLDxexW#ZR5BaoXI_fZx2-^#lT?=)3UXH&gIb?H z!tB-6nTkWY2b~4UiH5ROt?GXYa>ODf4f7|m{@OhG!$LP|0-GFqBv}z}S5K%cUkByT zE*g~Ovm5B-u^&mUoy2I8XZfNlTrRwTV-b(@3c$w?xI(L>UGH9sGoZv~1m1 zQ-?9=vhY#Uf&g7UK286fs42xz;_sxo48k7D$U-_wEA!4lbvc`@gqk3Xv21Bmv_Kfs z;2mYqsHYP_{rf#{ye(l=l1?bMjX+?BP=Zt8q2b;se;olbUOf7gRvT$8z9sDmlN3|C z^zk|S&ET7;f3p;{-D{UeI=Hw=?Uu=~W!*&7H7iE->Xk5&8BTeB9cUKcEQfo!}}PofNYJRS^x)=8VXA(>#gR!_pT{JDl4A-lxgz6pwZ> ziz&@j6vlPaBWs22>TL*6iI%yAMe(c)mCJqxS}=bepfXM!o!PlW&phl?4%1)q`_fpJ z!b$&P*roOfB?MT$&VCO(9yw^exVkwXz8>Z82n|reA0?(5GAoR3kD9x_asqMYul)a& zwYQH*I0PR*%951Ke?0c7)|M2au&e5*H1LL3Pb%o3Y%RLc8F`cLI5H_(V!`3VY_1ur z&sh?bTC#k7*mix|Xc2rmDJDjfZ=IFTdW?zHwGRr9Pq-jF4qgB1JBTVO;yh;0&B>wH zmBx87yR0Tx3mA`5mRmYid(pSMed`la$?T7wOs9wTvx>P!x^xVs_j*d$y6jWRS#b^>00lx&+vXw7~}QcX!9 z*5@6_-s-oe`C9BaoNmWsA7ca3FeH+@?>C@c2c;w~vtv^SEhOH@u(6jr*jAX>3?xr_ zj;C2SEiC8Oii!Q+>|R*iawFJDoXS4*!wh3CGB&kDya?exN&=$gl9U%tG{ z&K^<4PB`)Kqhj)}+UM$&$)r?D>V!l@%HfpYaI^4;4LVVcDE>Jce*gR$VM0Ykw!6f_ zAjm80&$|*M=3x_q;#n8?LE}E=?57l1_l5HlMydQajT2)a=dYf-XMrW!bb?u^a1;OI zhnP-z8W=}F%n@}ay8PZ&6D<_gznCVcHmvHV@p(kV`JDTBtoOIl17Ul?@W{+d2k;w)$gtId(!w;aCOJB#?;Wl<7q*zagArN2BP5&vE? z-hI|Zk_Ms{7$_tCZgpdjE(WjmbeQo_?h>i?;AZHei{R#4_03iLGBPr0yRGTleV0fa z)uOol<>`+LZ>~YL6>DA3xJI&F4~V*oQfW$$`Y|gQ zO4`m203`LnS3`#i|E}Ca*5BY621VLmZ$-1(|5NEDh#wWXb<}O2-~-hQyVY`Xa%NUn zKb~h^UFjcpd!fJdTvh+NzV*~~_$wZhpV?b6sf9IWmO9w*6@}{aOYT2tr+MKDMG7KO z&QwD-)w6E%wPS-lD&{YQB)+;#+OkL*xg#qS7@ybgIB^!D;;_LE%0hEa7z2AGiK_4@@u`t=7N=A`w6t*n&)JaV7#Lu_!|kznZ&H?@E8)+6}{}$({7- zQ{tn45=DJi-YdQ*tR%DhTA(GoD|2>shB5E!I!#_wwU;*&ett*ZgNTk^UYgr&xd{WU zFa2u9{;qyNOVWO0^yfy)fLn&k!G>o`Q>}%|e%3H}>7*1sq4Y!VOH^Wn(sNGQWK^l9ioQxj2L~<{u>=({sN2vAhdt|I9kfYt zh!(;V1%=}932=;pxM2rQ;VoH1%|0`QJ}(v*%`4$c%mr4!E;%_iK`(r`nWyBFm<1zE zsofO9aOrS~NrP)O=&CRRE{Pwspw#QcSSFALG;C(>v=S%La7%#%`!bBzljuML#Qja7 z23lVGqe60kj0}x;unVqFz zx$2+atzT|=66mpxlnUwa3)vh27{I_8v$L}RhlOl-xnu?f;Fzuc{29f%4eSq*7WaEc zVsO@~hy4+F_|+unVk?8&AH`?sls%xGNomX^pf4?Cr{YC zN_`edMzO(rdwboxF%kZ8$mw^mZ{yt#Gq=S`Ew0*c{=K=X2hRl{?JWlIP5X>V24d9y z4z~w}sW~{@9L-nf|HEQK&JXf&m+`wDUTwL*_eUJlFtD=6uhM(&SJnCik=_Q|H~}MS z@90SHU({hud%}#VP)37s!RqyZZ$51NOWohKu{P4!bw?MS@p_Te(T>meVLDT8i*?y3 zW7uRdN>x0@k?yOkb_XOHtH~yXASxE_oDLoXUi)H?gd;xl7x=j(G5vDZ zMMe_tQT@Ft8DxG%lAIgfM5TtccK)tkKCLkGd5Cr1&8ra>cz*V`M82`Tx08~2tHy~Y zn9=(60~_)-ikIxedKS3y80wfR+nX(x%|*>P&5WK`tnA#yoltMl!c>(EcTbOV0gQ@) zhX`rvl(E}$CfkPgfw(rYn7O#Fi^7A&*^M@LYFz8O-t?xi4sDx(M9kgqdg`Hoov*G= z`6(g`^_KE3B@bfop$fer_q`v&24ST~Y_Gx`6~$D$^mUAB#KA&}B&zy9%*1PUeiTo2 zlt=TZc1;fI<2jQ>D5sdFfMW>J;~QUQyha7&xE{XZsuqN^8% zd13i>-vttjWtdFPnM+U1h{6M0MR4?)k5lpD{5`YK#0gk*?%*rw2yy zg6k+d*K>5dYVGqjxNtrYvZag_n3BkvvkuZ^5Pgf~cQc_@&ovFaYloVmuZzI}H-Aip z+VF3~$-1Po@~X0lmP!u;YbCOW?uu0j(2*RAU~kqEwwsPPX;K> zk+d>%=a_vRaZV~KcE#F`4Rjx&>yih{$jL+IS_vr5zhyt&0C>7to(AnQA1uF7%BNn7{Y`Upi z+@2FhR!)^1+xv$w#UY&#m84qya%xC=yVB48i}S}`N~MgNP*2vfyviL5^3ny@(jE+9J`~d)ykO_3;eyrO1!scTi{5E^ zCZdGqd6XWE*90}3j+Vv^Ie17Ln9l#;ng}1Ka^))QN(gS|0}@v6V+eeXlaFFXkwcB6KvB8 ziXtMEj6imf2^neSp;04nzdU>Sc!9rpo5Fe%Pgte?U}keq1-u$eB^QOOg4(}I)Nv;Y zYx5*3s)q|_jVgVQBLoR6#R=0wr|yPMgiENbyowh7q?n?Lhlp@qAyXBrj6^|y=!pxY zl|KAYj*u#nP@yq=31jC#-^W1U2$gzMrQeDAg3gJ~sF~#Vot%XYM|9Lfokgg`zI4@a zH7gy^=G2@(r3>v4C*WD|UPV}7NA>;c9?A@b3YjBPNVsB$`jKQdfsuOO_QKQqubK!h2H3|?_$dX z&}hoRm-Fj#mM_}#=uYIN!gD;uUY2#fZr$43i)Fn&Vs6p88iY4qF2MgH0e2uqS5C)c zFwH$ACEdXcl%uwHnE!zn&{wi9tA(d)Oqr!=lQQ&> z(~FW|qRdi^mr(h*XsNH&N~&l;cUvu9a$xry2SPO*6>xacu>DckwVW*YG-Tmid{@v& zw{8w9AyD`i82DtV>ZGp`Sr|TU%Bm2}s8+qU#&U%~P=8_-*iBD5IUG-j) zd$A8n(jY_`w5eVg;~sJRsS$c9n!5NY00ouVoAi)elu^UJdwTv$h_gzaJtLdLf*en_0N&}(~z4$G>*Ru+y{=5KM==OKO4 z-mJRzS!|;60A8=H2b}jtgk-34xSAW4dGo+}F>|O_(T)P084CAg5HY zixJEhB@APi5JT8mt0)_G(H1-cg&Vwss&G46FA2QjmXI@em3>d=d5S4s1v$!@*sns= z9qhE^s3=(aGZj&(5><*o1iJE}h*I<+U0rJK2OpvyLa9(##nK5$!3zOI8fB@v&JtK0 z6k93UP$T+XUM-xzgbsrs%%mPQz^s97xBE>-B~=eJc74)Ks3FqRY-I0>5y>q`b!m+J zuN+uN(qqaJ>MHmc6luNE517gRYku+Z-au}P>D%aN(s{JSN^qZ?C+W?q(M?c>U3;X>18VDU zO(4>7l`kEO-zqhZt9OKhZcx(}*%pEO636<|Tic332N}cd=j%_=D2C~gr!8-mqkjzB z5G<(kXg^m8;}IJ0_I7e0u^Ivlje6A}nRR^z$XdS$(}@C(UmSm*N02vHRW$E^;x%Vj z?7cEG6QPV=ny!lhUvlm7erM)b;JqkTR|F5-uQyPxKWCcJ7#=?4wLq} zY?8_T5C6v_Cq2K%31y8e4Tt4l-pmNNWExgV#SL+lON*F+E2!R;Ru2757M0*&`lzT5d2iiomfsyO(=*;}0?U~Km#^o}%V+S!knx`;&(jOzo3*C)ZOJMO@ zKf{SJQTrXvuHdH6!%}6qm;cn}&6^B}6!}k9g%!jd2Ytqb$k2IN_<*^?7nv`Hx{HEb z5l#aFT%8Yf_%^8c5J#m^cO*zEqDt-J3bG0#AgaU#pI15UkAJp?qX&Y+0uZU zO4|SLH|5j3W6oGX&rKi`*JK;U2{f2bKHlczIB%nY$W1RN7af`evaW&Nsn@|fE|c48 zm(=p(U%r4p)Eex%hJAxT&U;i*QE5)qvqD3?awn?HgPfHXMEE*ctIx@$RoUU>uw_ZJ9E zc3$xpPyDK}#D5=Da2PlQ6?@i~CY8BD_WdHprUK9#TLJ`6gT;D(1{T{BgEx(z=i8sw z)s^wz;FO4N%cDG-pgP;@a)7_-{+q)gsrDbL0Aa%7Gn7I9tEe%;@a&@Q_Lsk( z2A*8^5L4z~E?ABi)dkp+SKDJBaB@VbkN_-Al%;x^Sr!esxzZFYM+|th`pdM%D z$}f2Q_UkJX&1W+d^RTam^DBEoE*3EZoqVu-(?djr$zGtju2U?cxF#LEGa*+e2RY}0 zEVftPU>EvgnRE5#k-L6HA<@QXVc`RXbU` z$Bd|F%Z6*a&Gk-xq;Ph(;czhAuQqM$(Tu2;^DnU1gZm39h9Hn%{fdCch|NIjIJBOl zgd_kUHU`O02o}ihb=%?ryB2cR<)hTh>#)AkUnT%bByg=3PlpCl?YT)_bF&%ztpgPb zD>Iwl{_ocH^QZbh+W1+St!Sej{a9V!LCchL+S@#8bMQI%Y4$#sU&m*ZC`!q|ie41* z?BB~~rgjQi1uQy0>=Jv}ep2!{Xf;9UZ}p;!6jy-wAQL596#kl)v<-!@Vi3N)0@ib_ zjQ7bF(K>y6zz4uVo)I~eRoJQp){-#!B|ktZfR#-vjI-)CvyW1cJKcVJ8=8I2Pg`-? zut`c4*64HWBUdxCT;CcwIhWILTpBU?MZ*cf7<@{1yaWJ2{esbL(%QeC*Ln?T$RaPy zx;uxql!6@uCi=+W`^AQSw5D%;6mfk*Uq|>n-+gN_voLsav&^#aZzdqmhw0|J>M|rx z%wZ{cCbyeT-Bk)Xku-BoQ9v@ohC^*dRI8}&Vp4-!+aG!_Es>I%pU%a&@ZY^C9Xj>5 zi4ox&pHvJGsp4)>XW8`t>20}@;N=H>3gfYsJH4*!;%?O=4wZt;j~|%uf?QdHQ+XjY zpT%_3KZG^u5;>9+9({Z*BN*c?{&Zk`oF>h*740lJxrNpgF9>DwD7fFfn4(`&MNmH; zhxua&UxcG<9PjQFgcutuAgq=JY_IM-&+tgpnlwNfn1p?hfNkuc$aKCZnW+kyh%rKv zm@pD_kCO+$<9r#tJaIx8Zn#Ds*j4p+$)M|!ndx~Vm@(mj`c^VJD?k&if$0HdUTi_V znv^v#V&-_B$;k;k+uub7$n^*7vVF+!m+XQ3^5BOV=PY0Jn+I~?RfmBD>r=(>1Atvc zYeUY*@NdX`*|<>uM~>a!-;O82^b1_QpEOJxo85nR@p8SvfwLnA9MnzkBc06m8e=7H6Zz z#Vh0_PlX=D!4QlP@|v-mfLM7&|3Phnb0HY$OM0 zmI&Gp+W>`plBEZMlu{}qKky7M7Lwtw-=)fJS25$sXt<^4+^^rO%w+y*;%f-*&l@uz zk(S+Vu7b0SVOy6G+Btp~#>4uDm#z->GQm9oEBF-iv$LLgHty~t(v9uyhjL*I{9-I8 z%_ndDhjT{0SGSzaO=X@Oo~2+cxqk5{JJDwd&}pSjCC)Yq&MzmRZ?>^O?Ly@Os?IVy zKR@i)=C41jqW_?pPMo|aIkLh<5*fAvEguzbdgp<9ht-O!!1H?i=$OxQ*`?P7zkXNB zCl>{{a?Gr(Bvm5$%ag)6pHk6P7Us_M__d<(83z3Y)|n8Ps%xV&EXNdSu->F@`XhgW zJ=>&yzv?wy$5aFxyYph?nTNl7*aI6Qy~Wx;##lF_pZB^x;dXjtD{uvdJ?Ubl&e*6;@t zFT=sQveewP)B_oQX@_mIX`dx*EBxT*Z4hitj%)if_Nt*?G`DTdBdl*9PVZO z`Q>H5yl=5-3<&{?0g7@C!`lhL$2*hN@i7<6{;>9~vB2ZN0(Fni{NYLWN`qR-fe7-lHT~!9`!Wna(=lD=aq#bNBBtOc_!W)&_VXM z(`>08G%$X(UO_&2nOjN8%%IjBt81p8pIca{+#-A-`eWNC5>y3(_h zeNtmh|HW9K5;ZLz4-foR!VE)xO8ty7?D-IQpC9a1Pa~xja=28S6}$odjoGjmlD^#9 zf@Z%c3$bAs=8TDEhfxXA{N^Yc#WxRQ#$Q04Ko+&2gW8R;WIcE2% zs(I(b_BS&t5HgPf&u=F1&h3cv&{^$i_oWHIttBTXccE|l1m0?X?E^ZukF@^SwtR%y z_)#|gxgERIpj&o$9>*xfMUlw=0~JZx`=tO^n^IhuKN5Iq+4Jdl~* zKiYjS^W;fQeZ7&ZU!(5$ua`2imrABWy86q9r5HP>lGyZ3?vZjLcMb%Gg> zGlqr^rvkA;lVOL~eXQb^*9>V!jN(V50=W&TA%XOfjd!;1aVrtB%W0cYQkXqp+L=Sy z0ufjfkSZWbucw>61H!@?*o?EdrEV9Ev(&mW?`ikt;Uyo4tL=KtY4G9ruL(b2-+x0= zSf(Y94q%)g6W8)IRl%m0YUdmM#)2AkjrI#^i@+P+P6e(?*?Gb?l_LS58a-IJI|6*y-ksYGBWP|r%!jPK zyHm@f_|}+)O|ddsEt)r40>IE0bt9SpENv6p)Ger1h>#Kn0PH0^pZz0^qP!hT5*+d< z34-o>1l_vd7n8cB9ds(C*@T|JaC6FN9xzeDfSwfP`q6A1IG*2daE9Srw|yOn187~? zFv368KN$~b>Ut@@J^2?+GMvyCkZKIi0qSUCt417tZE{_x+w!T{L-VLp6~(T)DkFz711 z#anY2z`5<|6N|ln;yZ;9( ClSXa; literal 0 HcmV?d00001 diff --git a/web/src/assets/images/model/bedrock.svg b/web/src/assets/images/model/bedrock.svg new file mode 100644 index 00000000..6a0235af --- /dev/null +++ b/web/src/assets/images/model/bedrock.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/web/src/assets/images/model/dashscope.png b/web/src/assets/images/model/dashscope.png new file mode 100644 index 0000000000000000000000000000000000000000..c1aff40ee092ae1758ad72d592081cc6b99c5b54 GIT binary patch literal 2835 zcmV+u3+(iXP)+~EOR*HiVTy>BvS}iV2_$(*-n;kw{^#7>_g-G^OCXDkGyNwg_rA0K z-@l#zoD1+J-CJVN3^Z+O=^EfTlU@zR7|nDNA z7NYdq&;ryX0O`Kc=ZMYzA&V{9v?=`pSn!t3xWhcN-w~25V9THdsL?(!a^$5BK5xn) zfo$t9cC=ot>c)=6!~~<{GqU^6(SE==(<^ZS6TL>&dv#|^21uz}n*uj(`~44BK}!G_ zH|p9#lxSn;FG|7 zIyxy63(F6+F{A3Ih9Ch!E!}CRH}HF^elH#`jq(8t<#H*5XDfr(55%@HGhrAp+F3^- z3u_`#Br1`^16tZIlyf^i0ukSdI6om0wAezwAJd_^0E`@V{wWL0XURgA6%=%>rP{G$ zzeq=vi^7c6Fe9UxQxVExSb@MO5Kdck0qhy#w_j)$0Fx$t>OnA<5Jx+^eBZKNlr~)d$6ONbPAFDN`D4 z!x``IVb*)=YT>9zWIL|=d&H?}SS<(V!tO`^UwdUy z%G?G43>|vnK^c^X8R!|VG95C6)|CdqWWk<&nLmvk_x#8&rjSz)a~KzD44E)D7_yov zl!prPPNwU)JuQobTE#pUXh$rYdE^ugC{5`?ebPvp-B&J5E{Y3)Y>++o{SVjD(4Wu* zg;$G`Mw~j;BLw%5CQU+5n+|2aBOrm>>B+-3&S2Hi?X`U1*+GBkj?&CQw3xKa0Ug_x z41jw~8Ta4LAvyPw(la}jc|~CWnVFw@B!s3&azaFtqJyOEAC_ik<)^?sqS^l9c4TaG z$f@&I`MrO0>k|ghw(X5)ENPCDG6a?t0`-(=WtMqUC&Im=P|7F@NXf!7-T=L|)uo6D zSyhWk@`fbo+ZwQ+6xe1D9GKr4{>PM?8|XxmTtu7Z>nm3HF2(B?P9;eCot5`1Qd>OJ zVN=B&)`4MiE>+KpH%WH(abNA4PH-nwP|&F|URwo8Pap=k6zxcoDp{&u15#S^$!sib zI;uEkL?;zX2FDNRyDu}|VrEe6H>-_+3?>9HloViADJ@(cFrYHvVP+sfV8CbaK$6!R zNJ&Tvr23KrY01eSKs~g6C51}{%Zan8FU~7_BwoKT0H0|dpa}FK0U^MavVUztsGl+& z1xaCCVy$4#B9c)21xYN$sH&o-LrLTyN{s( z^84GBBjals;?M;PN}s3~fTBZ3F1{}P0EC%Rpvu+6>Z-0{{SyAD?y_!P<_BX(??p(mbfRRolRXxF!)0I_W$V-+a6-U*CyRl@6Vrgujbt~FjVQ6ijDaQxQ>aW!cKoMgVTS=vVvF%^hgx0V$(@hTR zRQ4J>ZpV&p6^7wiMo5FUBD6o;B6FNwP%y%Z?N8?70};w4#@f}Kz|_C*Oq+4Dj=d*} zR@>-tP8h*lr!MZH{@Ag3WxacCc!wnUCMu0lp->st0!J}$K)+x2r>+NM$6hbFbu1$- zyM-k6an#3>bAQ<07BfFOlr?`7#kvwTN&A8h5a0< z;pziuQJ%t}jXujpyQ#BzMs1>msWo`Se0rT4yWGKp-6|js1xhKuLF{-pmxIL1scO^6 zXHl7{I{?N8My6%@kgd~eWS8i%nlqtt6rF4!2xTXsD&qZ(B^j*lD};Br zYa{AAq4gU#rmecuWY5)!PoJJyY6xQ`F-|*`zUK*$BX2r#YH|@YfwHqdZ^byDN3W@l zX*$x^zQ*AK=xWb3;fx>UE?1WUkl|!6UhT&INdVzwtq!gz5_zyqKz9< zufttXz5bK+^p;*G%8vy1?qS?Gr>@Jm#%$;(H@uXDkrh%gxLr@zX+cA>clZAG=~@Tz zapV4eU$DaSS1t^?o}}?KhN=04eVs^jWb>AcXBfM4=>s(Iec-+h2Tv9k5AxHD{b=I0 z!u~R$@A`EF(x)p4qUKVTw5Fha1r3qm_xtBlkdPgDPH!OJ=~#Z|pE)Vr%S{jT*hS`&9X1{H6x_LizDhs8mzUT23^an8 z9e|>oA!oW!`1*heV+K&}D^L^?eU(){@{TE7=vLgs#BhLT;002ovPDHLkV1h8+JI(+A literal 0 HcmV?d00001 diff --git a/web/src/assets/images/model/gpustack.png b/web/src/assets/images/model/gpustack.png new file mode 100644 index 0000000000000000000000000000000000000000..b154821db91ab02007d7c48700dea34d57871ec6 GIT binary patch literal 57988 zcmV)rK$*XZP)Px#L}ge>W=%~1DgXcg2mk?xX#fNO00031000^Q000001E2u_0{{R30RRC20H6W@ z1ONa40RR92ET97b1ONa40RR92EC2ui0N7xH=Kuge07*naRCodGeF>mmMRot&_g+>A zgd}WX3ybVfmPGatWKmGSrHIzLx3*TLR;=1;)oTB(RjIpG>sE2cifp1RCK7gJRTLB< zECxbYLV)b=oB!|kn{(!y@5_5HZ%tmpo#fs*bM`rR=DfN0yGs@7Qb(XY0%yd|EglcA z>fTi zg7?Z(DM_`!>R2P(xhjO?o|#`Q-l#fSJG;?F0IUAG)DZ|_@`>>vOy)mN35Bz^k{Xj( z2uDqQB)nx~>1OTc#vOrj3~t;#sl6X`1hzUM9^L2;^L3qyS{lVl3M0gm69)@7B{cwC zWf^?`WAm#SAoW_dIV_$H{r`pxw*pMdtB$Z7c2=Oh9QYN0Q(c8;V1%MI~8_HA}4MI*z2gP!@dI=Vi*rQ<55p$Qlt*qV(qpZ2fCL3s5=3y0sD@JME+8_5I!wL{UGFj z3-q0_BmCx}Tf)82LalY#gd>?x;x0+=%PwTcL+QG8CGmD-i1})ouK`u!MY>d zdH)?@0W>(PNz~QG8i6`tZLCLbxcojjj_X(vc3}d@1=BmXZK|u?156pxkc_5f`5pkR zmI}Rj%p7&gcF=?{vtX&h`U~!?U~X z8?OV;e8|mTK0e&A@aAfn@z&KSk3gNUM)?66F88*W6Zh!o45uU6oQBDJ6eetEnRsLe zGq?qZB+!x!*f5+xUkOpJ5q{s7ub!;PLs`kYeiI~)`td*870&3sY9d`F1F7lqY_;e& zKGU@s$S2?=@dO&0uq>?JW8Ziq;ERyNe{%QT)!j*3y&sJcD96KS^sn}5uo3uh?3gq^ z9EM4L8j{KBIMDk@6((tU&@pjWV|jEfuJ;`4$x0am%)0VJW+o=qocP=T`&A*KW4L?q zwbkCB^jfw%EWR7EpKv)cQdY%%vO4<#qUle(X5W2jm(OzKGugG|4WIi+##Xy`qrVq4 zI>QBjyenJ}54bq2%P5IJy}}wL2ckDOCcGr>fS<^Y!3t#tCi}e;R)(MU`Z&xHal)iY z;e-yH{=}=pOEAl~&65}gGkdUd#B!GemnWYlE?Mu3N`E~7C5xO;s3GvWP6RR4m5)TI?cnw*d0 z7rE1)zP!4IZf|~4+y?ZQ06YzeF^$^#N=OhPi-s0=R3z8)=_FTBa!dm4M9B@hw0@#40iKsR^u7$YX@YyFa zm>&5=8R}teoB5iORNIQNOk7!~q?^70|1`EAfcNWn-BVp>G>Z$(d-)>fXHe`VnGr6jZblHrt|}FlHQnX$#^1DAV6U-MY^T0O4FbTc9R>Et=to&&jjk6ArSj2BfrdB7!jW zx>ByD0DIyhP4ZGMl4RrrnJAC%;2djm~URT2$ zfjVIgb3lipZnIP3c9;yOVuJi_dy?|W)iQZPcRnbC=r{=HiyTzB9VPj{x!ls0NbyG#gz|b%EaIkUz(Xf1Q;JGwV5yq zxRlQ^GI|*!31R0pN?L{~N1s@?*{3}1Q=Y|P@e*Npf`mce7SMaUI>JBfzf1fM?7&fXAUO8!!T!zcB8Hm$RIP0nVEnpAT0wpKKDyvpnh0BCN?D!U`m-Es?D7 zN~)oiZI6uGtmzDYgw4&Vsj~_%dUSsE^PaSFCDuI_&+&yYMLt&Hut%<|ZtRJ-#7)O3 zA(BHnB{V{Nzcd)rxvO;}%Z^??hyeoX$!{V3W$Un)~MtB#{`F05M^j5qtUNi9$ zW|1zyUMH;el}N1hy#MVN$jkUqPJE`94e2yOsedgzd8}lelqVzgaU*Q&&Ma?e(E+PO z<-laJs25N;#sVb$s)b-nHglrdj7cg-=l2%HutO=yG{VsbJ`oq|EE z)d`j90B|BC_%YBfZdBny{3Th8Ydz-yy`9lCVT}Ktjx}M{l6gaa-OAKCac|t`^=s(m zAFl>Z0uRMaptsRxZ)Mt1aeC((+$MqRY~Wb~Uv@_#d!RRd;6Zl|D;*X_uEDM`ErN=`H|XM3hg zJTd$p$?u~`T;FMxbPaXAt_ajCEJo-Rac}&!e*+*sSGNlW-B^sc1^7$)Yv8}W=KOHQ zKnECa_{4?q({R1#w9p-n!(bl=-KlU8YX(T`?_ujg@LsU&@NjMTaPzv+ZD*frI{?T(1wM|@r@zq-O% zK;xQMABD*S9Z@Oz&o&>qVKQK0TpVW2RahyPG6DF=YclQf&QCJssfRFSRM7DFvz=k$ z6Gz$?ZR(IcV<`CqgU(X=PovV^Oq_cHckZ7}N33Fe@9huweA=Bf)%%_zP{zF{`lgM0 zLmc1nOgIPQ>TeK;^lY6CJa*g%A_nTMxUTc6mFHJ|J})=+w0Izr)9IK%r$f!5F5jXH zs~D`Dt_RNrxR2+;r!TAST4#{w#I7wj3zMF@qoZw`wi|H%EgghIc-EQ}vnNafE%7YnU5@%nyy;~noki_Q z#m!`YD9;T9(OfrbZ?n$wXb>*-qb1&w&*?7b_O$Y&Kn%G5H~bJ!FTDTeN2@;)RhQl) zz}P+qORTCh!o?Wmr{saeDA*t^2fx#85I%tuwi9uh_dMWvJ?do0OwK`U9psBJC@=2D zO`Iz?$0_A`)$$D;XkL55^|5zhb@wv(y$k&5WS`;xV|Wem?pR$d9f*&(E_Etq+l3ad z`xed!&+}U>+Vv=5`86yOlig;_UBF`K$AEk+WsC)zJjrZ3PW+^qK)&b8^tfFeSMre8 zXlfR)+IFqW)MbBN4>Yv+Ci{lV@-9qU6rc^lVjx2J)$rp>I6fb}^^y83L@6q3?>&@j zQ#NJH8SzXEj_-4za_rv*I z<|~Zh_a*R;_j;*s?4QB+B4Hgo(1Zo)YWrDuOeF+fo8kXIfB08bkN0KmbP!Gnt8_|8 zM%q>}xb(^KUW*reEbXw%iB|AryxZ!DMxh?)w9zilHjCR=O@zz$0%tuXy-kKB7L705 z2#jO&o?k5rUr(4stM@4ax~BPD3kFt6@5_U-7(5&>9DpoLOsWYRk8p&I2fW>2y9z6o zcjM~r^HyF`?YrXQ>YtWhjE7@KQL+k$#xZdD+e}tI!E*oEzYumC(}hQJ23#th&g9cT z3?|&-Kvd6VErQ1p?3qR!HnqJrb(zQbJsA*R9Fuj*i=R48zqeh#>vcL=NyBl@eu}sB z&tcSA&T&)#c&*7sUKHx`F$UBWtO~z6bL%kg(5d(Zw=OLaP~2NEn|kA=aYoe{?(>eb z&I6Roe;{%jH1RcmGP|{M|;So9us-``V&qXZD`~PPUnGVKXI4# zE(@>j`}PE~mxVW-cCOM#A^e4acqMVTvTIFv-~D$~_cf7VN(l9z23;$T3hpWYTB`P!p=$!9vnr;a3H z__qqpk3Bg#{QEuk;EHQq)=v7Hd`vpSzCL){Yo8&+`-Y3sD&A&|zm&gSeR1`zjUidl zc}#?Z)Ct?M4=n0aF#+)$Tzbi0BCm5l<*>K~lGWv4^lM?%IThP0ySu_|JI{(`X!P&5ogEM8=)n83ihg>(%099u_%Y$Bf%IrxclbE;9_ZmwZFYd)0=GSU zb4R>XFzdAV(U+39l$TE~V0?-{VA5R9YuDvE*w)5BG%inlhIcvEEw<2h8rfM$XUTZ3 zz|k)K$TrNDWXx8kQ3-Ia&+$Y%EE{Z^-^3TRJ#iUh?yM>C8%Jyrr%>#{#)0C4P=mzU#4$Lm9Zyj0FtCYDx%FLm2Rkmbe;>b6-i}q`O!HfA5H42{gwe;x z7Ol|xt@`Eaj5)Z6Eri!)y*9zzfNX@YM_3t-CC&$$;ag&=_(I+jf%CZTcD!uRURlh( z%aY{s`S=-+i-{MwlCNE!?VM;#(D_^H!amug#~55*IHO5bn?I5RQ%`(`d&8Qp@ayMq z8DAjz=ituA(Q~KkeEAxBt3kv8!{US1YN4^~gXhKLPl?|jb80*!DHuKbZC(&}$2F^O z;c**YXFC&~r3FgaM|RM+s8e#}k9gHjN4Wg3E#rS28X_N!dQO$+Ucx#L_qfDxJqLkr zI^xCPVNgn%tw6+PgQpV=XLdEh%@bc7FCYKnc!G!|9K7I!~)_}OtOlZ@m!3AwR&906Jq%Ac1PFL%bH zHu;EhxsnC~X)GB;DLo#*;?&pX9c1MgU6P^3`<>-Y zEa^0+tRDbj*U4+9(pTuAi1@9B5q?MeePbqvYmVG9?oRr1r9Af%7LTmGs7|AcBg_XY zKp#9B)TE*0K?nf&2f(9WY{-L+pNzU3Zwb8_+nE7xV@Oi#XFv4?afhk1;}<%+^qDQb zF*LpZJ%m+o@4sVr*aOc8zH{+))jf}1Tm1z(^CArQPs8uI^q-wWee6KJU2qd#4IFX0 zNgXUO%-E@zUeItE3vWR6VtDa2dE^6ezDcLi-AB=gq zeSa==w55+{#GUc8`yVh6#%eGy3K&S*Q2Bt!!67Z-c~jrPIL%=)I17{Ut$1xn`EJjMcU@ft zd458Et+=62pILBQ^-gNp7G7sr&5Z_KCBk8PB zye?NQ84o`9IqN2o@}|E9@A}+kn|$s|A@8=b9m_Bjq4kIO^!^wc;J9~8X?lj$IK-Uk!4>aR8d8w^edFZp;1^m5!7 z+MLvUZOm#tIL%49LmoUyelg@f2iY$y{dvXL7`K$^@Z*`~xX}{N;DK+dh^>Qg5%8aV za(p-^T-v+`hhJ?TyE^8Xam8+>1W4$!CKRmu4f1&@4{f3{v zSa?Hqd@roMqIWzbz6bLE=?v7xZlM0q1GiTnaXb?i2zvNA$H|F_KgXSD>QO=M38Y;P zeNHmuN!(2xj%$+_qtUVr$Y#IWb%NJrWS@pyxAB&?vrgl2amw^2Pc>Q2bPC5E4p2+~ z&3>j+FN)AH+}CJ?m)`K~x<5FQq&B{Nj`hap`d~bcRoBb$j;Tg?g$7UvOE6Hr2TgCp zHeSIf27vL<(5=|cTzPTz2R7C!XT-6aEe)>$o{3~1>Tz9YKZ^nP)iK@Szn;7z-IR%> zvnN>22hYCJ1t#&cVKCd4On--0SDp-?pTcSmcan#9f@hNRkqvz~4?G`!{Hp4AX!pBJ zo)agJUyfB4Zb6Vx18)!g)wqhuFi~eHcb+9Y&Qo>9BbbITYEp|o10$+Yx z19hm+=&xYwxB3hMX@tF>C9edGpLFL*LaniMn(LDuDxe%nnJ7x0Y|rf^_9nl}123Pc zs||J=?n7=`+XFxE|z5!)+1I;wwq7z540ueDQ8#I~|uztWUjmr|b0i zuNYMy)WNO-=2bXl%}=V!CcZpwkJGEOFhFMFLh3%Kw?vY;4})t?fp zi7&!C*+X~_k_$iHpudhBcqnVI`uRS7;rSj0)&D?0KkSeOyGCj;(F!>m#EZZ?4)A1Y z;$TfO7l8GHPtG6wU)1fMA3p|N@73hM-S7|JSiMX(`mx*jpm+vO34drlE}pnN?2b>p z_xOsLUG`6>g!J2NwM_7yCZPDJ9Gzq#sKWjeNO4^Yrkt&cTVyEWa_zc$^@)tM(`Ktb z^76yvd(=;dpjW!QE^v}5?6=z_YbnpB>B2i8!iTO|R(+JXO{wG^1GiS6 z1Jq;L6$uM$>6DNh)<5s7*Cds2j*ZD!qGqclZj|w)&`x7pI$4|D79KW;$L4$v6HT1t zykUH9FY6?oys1ii{_!c=f{o8@kW3v!yDqOqdESBBblriP=1;D>7Q48N7jC5egie13u_s8Wm^N1gUvVc zmIZiB)|*is_1IO_DZ`PhsQ%F#szuQIb=if9&Td`EFD>^7@3bVm1%y2)hyV9ZS{1(B zBV&Qg{Y`yI6B)3I*9CtINEsmfPH;}>tdIDDnRw9**TP3emnCRzV^!56OI~c^BbBQ? z%Ak~b6Q;0ha^!J7)~=_-XMLq!S7^B0H2W@}{U={3pW3gSFfq)ZJt6Mo@*8`-=_jl+ z@aZhPlL`k|7pPIsUp+4T-TFFk&sDAI$SYT!5t1(WDJM4)(#eBuj7ylv!nnu(A_3?64TnfS6yBCGB4EwZE=LiP!l zK9=%;okoy_U68!X5MlO7cU!Fec$awBC;!1jU1c97SLkVr-ytw{)3Foc5!AV{mrXli zogBw?tqQ#OxFZiD{x4WP78l%y;nP?HJ@CuZSD3fp;L->-1wLuz9|tmu4{qnnGF5m% z;uzljBiC08VB>o>F}gZC!$0NXN+>&;2aaQ%@5VYYfzVX;Fut?UC>JToFoii(n{Tesf)T<3LEYp(JjZXH(RB= z!R}vapKyDde8N$VrSL=i+!;o8jkbsFV$}~R1SG0!>#50y3gDqaHWqH%C zxVF`k^4$-N+klVvRHX#a5#9?gttUN6G;x~9Xj0bY9kyen{?iBIb4`B7nQUM596$1c z(PjP4bfA-gLT719en0U zKw%80JJ30xSWt6-X$-dG*mj)pssiqrRpI!NNLswmjtO)j?ZEcXEn>+s?P8) z@Hu^O72hz#h4y~_f-1ea-)r?>fk^RC-;4hH#4hn9Q$)piQMcgO#zuJ-)}^kJ*Aqx$ z#NLvQc&JATsHCugX2z3_l64ZMUE4CB%d-ZOc=ykBt4%pC4x@J7+z;@Sutdo3o6(5} zUnlzY)nms+-X612mNLE@We0k48ovm>1h*Aj09uC*FkbM#=A!D~d*WT6xXCYwPh%Wy zuApN==&vXT7K_!Q0nP`e4X;66;n5&`c?Y*h%)_%Ub0<#@*WXjzT$&UPe81z&_wz-wGY|D9Ai>YUM zP}LvTlUExiPcms5BUZaT;R_l>KKjd5)dz{%m`d))#^iKgzK%UL?$_uD*MVme`Ov}d zuD+n+O`RLGY;jyX48HsfgLE?gdi!A2fC532iym_DVJi=e-~vyu1y_X?&~rU>%)@EV zyv>J@z;-w!{sQWcbAM=u8~tQ)yQuG7f#kL5j_TP&>3hHHzA0gOzuR@Xy_N*iX4CEa zBy8t(-?Ar=C$2Yqi@fpq#dzr`Z2T!mB`q+4X2f{WV}TOUl}($4xgWg`wzh(Rc>Z9*WTCB>|_6Zds&={dx~zv%3wDM zRpF-9OTz3huUgT6CBw_P8J_urPaneodp$;28YjgdKr^c5DqmW?*^jrts^KdFWZ?h-E4gGn=?)HboLvfw&a;Vs;u%h0o@i!EHf^FMUiv_vN}Z&*pH`Q#kuA!h zu#W`WaiVd@LI!N&0a)^i)`lzT87}0(-qc6kDCBb=vy9>0eqw6cZ|8G)=`+2A+kGtM z#2do7(^iK!o)_>N^hQwR|3=_v?=Ip&X52e;1j0Q7Os8V-9>s%)vT!UO8CiB=wNS0~ zy~B-_cq;lGIB)teg1ki@P#nCBI(@W(rGwA|Ogd>s|1rKB_zHCcAC=f zp(BCc%=t@O-HKfid(_!NuQ`3Q@~l?b(@atPic!ZDQL>I>+?9dy#Mioq>Ai8r;j_ACy_Ax z{K-?@;gz_-bhQLV@lJgi#s2l~+R1T?G2_D*aXk6Tlqw&zB(WUr*|<4$$ajMG?m+L9 zEl8_vDk&Xv6ii!(@@ZLf&!iTPf2*L&JHX{|Co5 z!e4xN9e;QvzX5mhcLUlVoN26HI=2dQaiEU&fVXrchG#n0g#B0iwD;Gm49;-hQn;wT z3&zcVf$Qa&GYzCGunIiOA5NZzhoA8sz)Rq>?i434{rA3baI#qY)@n>RW$`?`f2zNQ z39KW`>cr{`F`5H0UN}#0cA!60+-hSvFcaKExiFX)sOcHnFw6M=lQZZp+4W<>A z$*@W##YnJxC{`G>BV0RgtN%?|!)Lj5@Z~;3L$*U1Y`xJ+KEf?A^0vum8W?q4f!FmX z*AM<|3t-jjiHmbi)trSgrI1H+&<15fw|t1hZu)w70S#^T*lTTBS=K``G77t+Ls3AFhpgm*2O zTm5(~i~Gf~Q>OBQr@oOm)qQ7e2jv0O=ioyR|MSpwL;Pwq64>-AJRkbzU;xj->gq_? zndp;AzPVkmONV(nfi^eE2w)>I8UEk}cUHG0%-VY;thQeXOL>P`Qy+`hPHR~`fzSOH zpZOwrIn;wZmAcPu@=7MO34EU1=7Zazc=nHaT$c!KZR#oTYr!U4`YU~u>%srE#xA zd_sIV2EjS7Ged(f8C(uOKEC<#@SnXV8$8PLLHl!=oz%%UKY_@jtIg9tz8#cD3f`}E9C{gO`h!S_Fdj--e!XGK9kmNU$*f^-77-xZ{LvI z2H0(sD=Y;24*YdJLI(iB_lf3#*FdSlZ$=_nO~J+f)oa3saL~`f)};fV2Gj3wapJ6} zeldVMzc;`E@5v+*@8cT(j3pV=NP}9}TZnR+(aD#(on`v`rr{s%3g^t25k8OKGAEB) z6Sl++sgv>M*U1quPEjXA|CZP$<41t$FtrzS?+HJq`pSp)bLqhsr^{dGC7(O#I4WtA zFJ;Ky(tjW*O8ulsU7NgY__d-!UUnqVlqb2w3oc|_zXw;kNjLojvy8(K;xB`g(W=vV z>1UgLHYra&*6NqROk^y)BN%d@IlhWZuk5b=H(^J|expoSI551^S^@V1!kDuVO&o&% zB_W1w7pqAphS%doe|+Xn5B}Q0UW)d=E_o_^ZXln@8aB6vs0gd8=OLbYnV^CPeTpRl z$=WBXoAE)d!S1}Mk!Wsro}TL}E?5wFKOBEN=+_Sjd+!~m#?@gjB$T}{iI~Lpg8jYW z+FsDn^2@M%1~U8G=D+#^I!imRkC{BC5}U?R)>kNSdGZT(ThvWuD5cKK4}6BnU~=Ra z!Et0F&1#Vgm~9o8FL<-RV4%FWZBCdVD#(cDww!{tzAY`wGUi_el(iYDWmnV>~BuKQF1Ny_gzYX}6uT+6@AFB@WU7`0MVUj$eX-^=~UL zsdy25$Yt}BxXP)JuBGdkhl+$?*|aKS79P` zaFDZTAlL-RZ2~>eGcN(q5tvur^V{k?vTum8*Iseb*cIVuyoGf(>>rDb@0d?#4{iR- zcRC5Y+_zeI<8_+YdZ3qbB`+twR1`9vcv|!q_Dgx9rJx6Sr%8|Tl{!nfMW^Xnm%M@j zB}!COz{G_E+YlV?>*%UxUHNqNxb$qmo%*=}?f2jG%i}KiG}ld-WLw(-`Pr&*;U!^l zsK!kWyC8v#S+y|yaac~v@NK~@o(zA7gZB~GN}utl~qBbJov7<_jh3q zxEP^PQTE;;P90N)SHPn);m{oTnG&=2#XR){jrn*4-I#^D(Q!HE{Mo#X5$&@Yc}7aZ$a`3%!Hs4u1WcgJbqN$nKvH zcEI0mJjaLU=01pD|CS6!?}n1u@z8hxCf2`TlKpuaitvQvGJR$~;~nq$+9u#M_-!mxR`MX=lz9G_MtE^=AD*D-hA#VWgJ(Irl%&tZaXkvYr)yvI zpM6TPNN%!kDC#o-rh)0d8pYSh5Z@T1)t{Phd$qL2W$;irzfwtU>I*5 zz-yE@?+$#Ez&|4x6Lo;ofqxM2yPvwE`r&8`(Da${U<@*@g!YqfK0xuI4J7k#Ai?ql zZKrXZabU7fD13v%SJ3>=2XF1=nZ(2wX3U7=tIf44c|B14cH(xBT>{y=a4fFF;QjO; ze^WhZ>>Fiu@V4=I^yweq{3(hP0E<&IVW7i|A4^ZPGKq+XyktZzev{2@G8s)t&3T>A z#N8&@{nW6uFFbbHhRL%nm1ZOBVH<=pVKPY(my;EVMC8?#(V#@UOjz9Txw?+d>P2%O zFFq*j4y}h89**A9d0KoG64dwPBqD|X@4y4i!S!1*+e({9DVT?u=i+b`j{$@p2vqZCjv`>TaFdQm+(B|#~0i*faejD8to6oqdhm{ zZutVW$$&Gb#l0{Q{~jHBJ+|iamh2&7sW>XIMVolw6l>!VpUD(sDJKe&Twj^297nRu zP#-oGvm@PrB}ancWruL$PJ6W5b&62n%Vg!NDk@0{$rPsKq?HvrfooVPxiO)I@MQOx zYUb68s(aO1kGqz>u1CA;G=Yw5bF$xG3;il!s8)kv9_M}Q}e?!}JyAM+P= z{xkhpUj?Q|S;1Ff@XON$Oje8yw)<9g;?rCMNLF~DnD=S@2-jL#l9lvv*IbP}gVuYW zo*WJs?a7Kq32y`49_}kyd0$k6#zD$r6X6eavP$t7=$=2@|LXDT3-|$M4@{1K$2j7< z)l$-cGRE6u?Xi~)=dt5J9O*1dTErX8j*;kD2We(7dA`_i?T$Z)_)GnU{KsRP4 zE8Sa#S~rO@16Z z;eXhNvM+MlaKx)`jqux1-IPx+X5zj!=2 z?tBBt!_o1Pbk zPmZS{_Fuwr;purwMyLogS+wJLd{msq#=2mWS(i??rP}nmE!G@2S}Bo%F#g$H>p2l| zr6i}p>~NMrV#+d72wo(x#0P~&Po3^~+tm;D{hDm57;0fxHh39(dOQFJo9|@W+z$G= zSXsTUzmAPNCGLr94*!7IzaH{><7e(zaqu}%RC>nU$;8za!*?D}pOx>g`C;XoepoyS zzXE;;zVNM}8r(F1{yc&9rzUiT&#qY+I#zXr&ttssRm>BtUw2_Fz2|{js*A{9mywIW z(Obr?)$uetZ~o{0k#jA7^9JO_I4YX<$47ol{zH!~+35E(huDnD|ksIf#sxr=nK?|E&p= z!ubQe7i*g%v| z)u98_@2e;Aq3PGVsF*9PbSnh;lMP#)-8TK+j+rC*$uP!e!AuCXMftq4UTv)dBU@Jur-9UA;;Ep98?H)6Cz0FtQ;v)$VJv(A z_+!i$4Zd{r!)p9dyAWCOeT%QF?kQ!rdwzT|{(5~#KQzE5i^(sB%sDGpgfA?b;Qd>8x`1Cp;@6wf#daRRG=B1FqK^r6p`a=8c)wbYzs5liW3p9uyDhe{`3fi} zB;s|E-B0PHMmL6v7mmoh_{<^Ufc2#3$;mrsyxA7*hVYNq{ju|t-lgHLX>x-V+#LE< z7=N>zh+!Ezd(7&Ks#|(=XinUG!o=`qMC6^I&&ZA|%Hdhyzk$!Te`6c2tj{^8a_sa0!{r#$P_N4|k zSOkuk6km!J^_Ow{w&C&2#~*M@JB?Q#Q<~wVt8lD&gR3jYv$k>QdPOr`nFc_&DuCEb zB55_m(}uKKGDTczF#%CEvnCa?O$UnP#g$c4u9@t#(qaN+G5NHLtClr7JNCQ&u8~>s ztKZ4$`rRX0{XLBPJHXlVGjWl9t#>L-JS|Rd;B(rD16?^<1l zaM*;nBVLO5V;tLAJhq(wYaM?Nl#h$%@lO=CokmO>-EvIkAz9;#027Tjd{smRghc~P zI-E<%CGqq66!Da$nw+?BiYPvpQ9?6b$gq!|7+pTq@zU$=?mC64hP%*}^whfu_XnPKRGf}?NWBIJ^!3oQpL9X67=&wp>&ij2@gUs>il&|#qg&lF%yNw|NCx1MdD)d+9kv1S5jKjuUV#}1K!#c;jhov&S& zkHxe~q7rc6fBq$pg7}lm#KYv0S4%~bk~1cgS7_u{(n~t+8kO`nuV-2RlnkvC{PT|g zxH3J=@9&h$zQ+pib`MDk+nhYrX3Rx z!2y3g66LF*cWbFhJ|!RB*AnNpNF$z|Va;~HtqG0rzOkE!uP&HZ@m=S2*@Pl+%!K%M zbmvppx^&C|Xe{&i_W|43NK9#*w8DkFC#$@&q5*75Osc_*3FD`J=~@(17m6nt$}oAM zl&huYiVO@GyoqKo`Kg~mfXkJZY$?%&N5+lm*k|5t!~6a?x;uOY7tuQx+=a5hCSX~Fl~G|83*Z~;OF7e2xGaQ0)RDK#=Gezel_0rGVO^Qs-;e<>rE>H zv&YBT7#Qba{Oh`w#x;+>j}dg2ppTUv=LzY>uZ3*74B;~IFcp<6D6YC%-e*OA3Im^$ z6X8-0&Xp@DI@?@bnH*%K2fCz(9q`1+nr+bV|0~zs+4VOxJ6whC4^KG_Z+7eq{|AM9 zt7Eg!cW$)Qh6m=K$OzI|I4HUi&)HBv1^%yK`|R>d*M89)<$5am4A<(g2=aW@@0!M# zF#VZJhyNj9oG$(sZ1CQ%RIH}|UP$vnnDGTaj_phSbYpdi^VjvJ7lGLm;!YUDKSs}H zYOMJ22SQ_lZNst6&BqMDL<61kCyysFfmS4*lvMDc8_<+QoCf7=r~9Z(P}ioCyi5>H z51PHopb`%Y=bYYtdhM%!^^lE$6SBg7=FNaSW12s?vS)`W%X#d7(UKy z4>tSh$x1XLEC#6mLojhxz;i#EpIP1&z8WrUyYI-E`l#{EK;BHL_mV^Rodt)pEV)aMqA}h&hvs!**RXw74=z-QqB(EuCzEi-%3szwZ{_$Z^ zLPDLB7w|px-+5W9*vBBeS=V(0>In25fuScX+=qm(_wsJ@ymI2iRVgMklNk$c2jEIJ zHEm+!;`6(f|Fq%-$|04BFO1vc9awzNIm@#XI>S#!>;e&f-Sb53HH|Wf_1S;&kdZ#( zE_i}>3OwC-@7L8%tLr)fbp+NOfg!K3 zF!A>H$?6kTs|UBul>~}tggIH+Fg$5kbsfNZs0C_uIw*he5c5#=^)8P)R2ujeqIZ~m z0<$+I=l!!dkxO1ZG83clnw{gU|7#GJzpGzi zoZ?gO>j(@a0z*z%czovfoQPUkuyWXDv62Xr7HD~58;@iXeCI%%?ui#_{z7;FcumVN zpAN>3ZI6r}K(MYvA9TMM`nep5@lB2KVGrCxraSN1KlL9qlT=RN@QE~RPvgn{mppWH z^M%jIt=0+*Fm5>g32B2ZxU>Cm z*x20?;C)YASzR|+UF~v6RNHKGWVjT4`6y_(tqvuI-{I@XvmUvj`o`jU)idb-CA2O3 zi7r^})M0+={{O)8{0DETeomIU)Dfs7Fz^Tr`5G2YVlsak^OAR^YueUse5XP`a{EthPQ< zpRWH`BtgAi27WKcY2MoxT~~cQvjNa6jbrdt)BO+cf|cHRRd>l#!*vAe2nJ*Uq7xRoR0~9EL~GV;tm&`nMnM?vtQc*awYJuS^l#jBrE9Mq9dFLGuzN0{@*&( zWW`?rySu};q3a}U@*UOh@GRCV7hSvd`>OGZ)3=k48_E~!WR(I@@9PMRLdBQ)hZCTT4+VE?rZZEaXfhMVrd3xAzr~7WHj>r(Vi>{>9nh<3Wv?-`i|tz<8E4}Z z?l$;)QsMu(c%SL4o|4tJ&yTyJKQAUPS73PEGT)$DmpTG<1V$(V!$2z0O)p7y);$pdb6wEyVF|M7f0s`=0pmo z%+k}U%7EKGWAszKoa3thaLhNcrUVfW6Q|I~An^}s6v8VEijXaUPP_}* zXC%}>UDCSn2oCWM97Vae{?$Q%fre;Wd63128%mlkJo)Zbv!DXD082JEQ zNnkF9Q@~ddgtFib`Oq0As(}6--d7=b*5{CwwT;4KLj|45+tMG-)mj-hB-I}b~D~xh?(O?EgD7l^IBA_I*jZfP`uXKpLBFHE4Yg@2KWJt0xa!u@c`6 z^tmDe-?je%w#mRJ&W?N22+pV)P19a)r6x z#N3tU&=VG*<#&QT+`DV+m*}$9*_GBCrtuLl^U`pUjuE`4jRLkyb{!HNY1Zi+nB64E z?+g~hZcTtk!+@H`uGXG7H8A9g(&CDlzkb_Pe*iWh{oo~VH@K8Kk!_Z zIv6|FOi!{e99hYY6~rn-b%21Oly);5=i5Rgb4dX zwEP_}G1AL3YrAlVH+3aC+JG&6&A+kNKfqpxUS;+`9O(CLV^PeYx9axeLY07vq%++T znsU{9Jv4B%+H8eF0)KMQphl4*J zzBrn&7#{rR68y7C6_C$6z3OhX_@-(1GMSUUq=QN5vncFUh;S$B7>#X%ZEO976LbHqD-3o8H-e-ugcg>$s&HT&YF7t)6Xcu-7XklWu^~~iaPSEuQ zOLcbpze7t60}IdrS_^k!e8%L%EHliX9A)FmiNm}y#H%&?RMl;{XJ^2}@Hr)HK?x_X zJev0Jm#YHEOSxZwdD!k#UB73ov(yAyUAE&ADA@&Qt{CwCO}tO1;UkRdkkAb)J>CowTc9tD>DAJ)(8*_W>#JF#uflu{ zV;3gF!|eNhBy%2hoGx{)k5;p`_2_z35I-CG->R8lVAXUo+dD?8;ReA~+qlcwE!Anp z;tu6+UtcIpKI?0E*Tu;B+Ex@4(<~&3EQZdkNQ)>wGLqJRL1L7CrEtM z8nPxm$fy-LxdCQ=QcNIZSM|0J+Gk%ePky?XzGG|y=HS6(B=;^`L>XHA#rg)R8BG8>Ydq!p>PSM`hhGgfg9qRI` zUi1wk4LSpa3|SP(D0mY)bLozE2xBuR*^+xygSRqn`4+i<23Y^y&0$dq*wv58#ze4px%%pUId+;~4N;wqzU=(<_nfy6{#E2&PWZ?H^omp~ndQv6KS}+dsF5QAPJuRRSnrf3}^8r?o2o{1Qv2; zEwG6_CvgUsPMZ|pieQY&zW-EJl=-`*35-1~j-+sd@3bZWABi;veLoeiBlmf)sqjW$ zlBoK~fFVgQf z*H-4>^>6vb{rK#ON~GuQPF}yAq6f8H^hGpN%A|GW;bba~pohpK+AECCfrMNPlh0`Q7Q(}l-Xy#J;92G~Sj`eYb`b@%5&7z#`D}VU zm9p5pV%PZm4?}b?v597xsM$Hmt-;^f|B;vH`}-q;kE|6il(dq9ii9IU^J$ef2X*$p z%PxVp_kXP-qJCamH>2^Fqg2Rtru9BWG;_~_?bJgmBaV>bOyCgGYB|#mY=q2tr{|Ea z5wRN1z zfK%og49)xar>g%;=!#kYcj|BxV^++~&PzWt7!3RR8+mGXKZiGavmc{~hfbNNR{pNV zS*SI7dU!b+3Ot6|q2d3=FZt$0Pvo%T`0gO!NOg7EVJ`(Fz4@vVT;A<@K0c>;D&pDx zzgVEty=UuqIp?eu})wOdA`5XfR`s%}(2v zFCn>~uzJEu%%t$Q3cg;<(6fHUC?2RNh*u^;PW=9Wa#i#lA=(3?XwAxBRrQVM zU-EqW$0y)*Oj$KJEKzT07~DZKK2DA$SAuibTPoHcFF8Wa*L}XzN$M& zDvG3iZCI+|{*Znm9%IwK{u9|WY@h6T zncIfF!jFEl{rvBF7Fojy|zA=|4ok=cK9_-{;t8hNBjYETZDqSK4;O|A7s@r~2a^UqsC1IiRv^sOh| zQ{km#fA+r_eMdZ{rA+CZT?hvyet1JAzni2{rk0{Nj1AC{I%#tiIp3_=M9%;(Nw`iFHZe_1;jhiWFo)JPoOaMmS2 zR|~RrA>0e0yZ4%lJcd{%&pXgOpM+YP5efJ~@gFgYI zm@Dzbg#2@@+Lg&{lp5Ol%R+V88b_{Au~D)>AI<=yhM`K&DI*?V4`$GmV@RZ@-n-;r zplvc>usgcNPrOi|n3dkiS#VNDhJnUT1G;fZScE1Jbg#-^fwYmqKG}Drl=ZXJV&xzT zJ4B(}{Wi8wHh;buxfWu4(MaPe2)vUE5|dTIzu5S;A~&41WOJ~ZXOJp{r$T^l9|%}o)p2R450knB7T31VdQ zAAVc8uqF01Dd=)}4%=aWogF@)7WVl1`(jOe?VIR2pf%eu3HzcBG{~hUs(UGbrBx3` zvn4zD$u#2I0}%X!K19hm?l+26b9sOcK>36PhLKeDkb>khFfhAh9{o^+=d`h%lR*?w zlvroTccAl84Qy7_ms3{kJsHCVAsPmPCj4Pj_ZZI=HIc*bLayOT6EwBn+~p}Xg%_$b zczVuIX(o4}736y~>62xm(7w8X8cd2Smz{p_m9AVoCWh`n8kP#)U*xmQl)<8pGTP;B z#_--{=W)G_KQhgMScy9?bqu{s@`L?_8P4`Z%I!(5cGq=Vahx@Mr)vGizzMvzYw2Ti z#K1fM>$XZ2aUQh+F5qX&uOx+U*$u;HaQ5^xgE@bC1JO<%N@MIi;@_kq&o2rLvhjnd zQ}@udR(v`TzTjquh6lj7O}iDj5G*>FgcfB2NK~n|84s?s4lH1@8Inr6lQ=vz!fqJE z1&Bf9jpKx1Y5FA=2e&^z3%E3}e#X?peBxcnJW50Q15YIvl z&O(VXBINVvYF>C#Htv1YS$^oy!3(fHP}y8nz_V9mbww;?aNV2^S3zbAjqSqkkATxR zAo^2RnrcA>;i_O`#`~>ZIenvyg!!lVZ{Zu?|8)K;2}c;xXb6rP$DxC-E>VpF<`R1< zDX1smhO3gO!2qbfDj4bQRc;QXfBGabJpk8dM~9qINTG%{swpQ@Rh#4eTu}DGE9%|l zU37RFRm07BqFTwu8@FNt=n~8-b(C-=*|}$!Jc8jyf+WDrtAve~6E73o^Z*ixA*TB! z)5nx2YVw}3pyiQ;<7nkJTk*&3rfeb47p5{XTG4RZE@SxDA2b}j=%Uxl5lx~(XY;M} z1_%-Q#2ahC+>6-AQ^Rkk2(fqhPF2-mn%&eWk%aA?V#T1;wD?<9^AKDT&1( zVN(qR*cpW(`JF2EoIcm5=uIEdF1bC=p2mC^vJr1yf^Ed5@JEYVN?}JfmnUNe3@91c zTuTM1ac&mI9pCM;cDG8^2U-Q$X1rs8$biaS>lr76mp26$#(|_5GPjaqpr|hO0lvG6 zCBRL-Ofx25?R~@7@#mV0-x;l?vpgczGlmgkAuD;Xs7jPQyw$jrTMI7pnjuRZpXr}a z*A8E|pQ{lEmNtddJGfX^E%hEQeckH#!jK*ozjC^CELv%d?;Nz$vzs22s*|oK*iOqv zJXD;4W*MTRi3)roN@5UY`EUieUVkc&ma~=pLUnr-LSxKXZ8i* z;6qMRhbkihCdpdE%R&hMz>VF1p>U~Bdnv0-NqFA6o4R{zPdv$R__d*e27b2?zfBui z9i9$?IFEi#R|;%Q90!Tnzsn}ozVL5YaF$~L2H!*vMUIPyq!%|L0|K&$U;URZ>(;)Su%|K`Z->`*5_FnHQAqbl@4p z!<5QrlRqmOFooK)&d5~H3I^00Ck&S-r~>XVP`{JB-y8gYDL1K5xaj@TU?4g9%=}a^ zeyaI1dNQ1 z9h+>TQ8cE~OBY7~jby*@QU+C|c!?A30*1k*@$tWVQH(g@A`TATyl*ZKEHqonRqm%q zE~eWN=CU!vSS0$3b^8MMGvZrQR$DJ4ye3yf%;02sv;uq(9l$_U9m#dzAO;l~1vzS2 z3y{4^3!P7oV=GgV7Okew6N2J%S=KbB>8+Gw5K>~XWVUhg%lQUcLJ8`O!F8Q-Ivd!h z+B&eHVz&F)m=R#y`ED~475-UpyU{f2ohc9*LDmuA5{G=+G&1A8y)>)>DHc(gKltHHkj)2J)uz+zOWr!Y7q#1~Trel}ZE0Ts2;VIygxKvka6JV0WKJna6 zhP$+)p<{VUaR)wN`WIp*|(yI{5=Q5LwozYj|AhaPiN>!{S^R*~B6|LK)NU8$&~zo}4>uv^X` z1(r~|vKSL8z-`j8XfBc*{TeIzniVn zQl`2l+i(%))n;R#)){K5DSF}yjis;gQ9>uTZW9gl4eOhJ-X=@FZ4KTad6A;F1@0)cwpf9f4XF zmWtqT-dgn}dKe^&o}YY(0iisP(K8Wd(_q9l02PE<*j53Kn@P;2VBTP|goomwRP8u; zEIPxXTKt+`xA_OXO$y+ZQ|R&E>(nyS6qYTavJ)@)g7;g|u zKia6F|=MzvUBQs|M_`C=3apcKM4vJ(Ml)aQbGGtFpDMTOeO;DKdhDW6dv^IIg@t?vW36dm58_@x$U;U;DvA_muyym?052#<@gR_P4NPFNESu&}7f0|s0cZMl z+>wP>Fv3B?4A!8)HGD%wtbC}2P2ssJ1!@r39Q`DFv{B~EqD3|ZwATX+>RCh{TV2zb zA1zo0@uSH!`*wMKf~Qr;PBnu1O7Q1Xms}ZCZdig>5=2;kx56@H>0ekxo6RP83`#a4)*=3$qHfrmE$08Wxi*1jmpiluB+10`LI=T_ zhQSJWVAnEq;TLB$tZFB!9%9Afy@83oO@{GhCgMzNFr81Rty>+smD99h#~;-w5vYHF zQycLQ9)uT3nl>-Hwa^NvI9%2ohVi7`aF#9LqTF}VCJ}>?n@N)QRo;y81_>Z`{NtYY zF`uV=7xB_$ zUmMVC8QI$-vjQIxt_Dct^|lLWSnU)(c3CWT#Xt_l7T73DW(O&AazGfvoV?YJ z&)thI-Y5bqP0Ny<#x86_aqW!UDV%3sg^tt={>4sHQKT$zU{S0;<*_pp4i9TGBsR*@ zBXvqdEvD=#>%MT*#=FO+!o+jES$RfO2s(;jFo;O3|2;M$7i^vp;d(Fb8|W{?m_VBK z@=)u+>cBTjsm5?l!PJZ*VJ~@{GTXTt(#U+J=c7na?B~t2A1S0X#J+CkdbW#HQqD z1r;mu!9i*S8JhXF0JkLBD*TeMhW8`b9huzucdz^R{I<-X41kl z(FIQT@Txa8xMdABNgq7vb0esRrR|_z#EW3j`#AacxBIy{_Ad)GxX=l*1mrB@>@rXU z|7=uj9x<(%V1G7lLj_AF(<9UMmc;b$I3$LtbD(9n&{La^mY|MWQ2i8KYH91k!^1p1 zoK11yq4skqnTvZV@aI}W0xeWcYuzPZ;aOub_OB3@XVm&D)$S^q7j`Ta9Hjy!8jz{j zaR$OhaRo}Ft+@Pz@Y68PgG{?GO5?Kp3q_z8Q$^1{d?d^3;Ryg%cJ9;D6pxT%KKo;x zm`@ZgM-S1%)rHu@MGqR-&ymyLJfn(1W?18hDNuH05_QyY$yM;&?O_M3(^=&@w3q@} zP(?ks|A4OgKSb#AA&R5R7IBGh^%eoA$WylP`5`IS6gWVJewFR3mrkQ(a(e%}kL(@@jCghvYtUmmVufsOJd;ixtki0yc3H9`@6< ziTX?rcpwUQw!J@1=JQ`@-I>80ji%ysstP^MQE99O&2vSPA`H;KapHxKbkMVDr-r;;*a=cV?j)$MB@u9Sm ze+>U^zCSpS5QZ!FuGPm@(clCV`KKS-qD^VXwNn0AhJOHrTMSIVeFjqh*>E%X*g!cn z10s8ExXgW;LuzmJH%wHAYXcc2Xo}PCWsULfQ24%u!Pq84*eI*nk$gda%~9Pn^fY7A1MnHe_~)4Ui?BbEuo~zc)ZH*PE|4# z^W9e!B1)_I-xa2bP{TBoH+!jb7n`dtT ziWq~+Q%A~HP3Y<;vPx&TCR?T&q2toGc2x?8dR9^>4SsE(-CI7W8QtYn0fbTp!R4<( zw#xBRMgm2eFyJ>`wEFkPeks{hFz;5ASS-h267_<`xL3-1D#PdokM>Y!BRG)j*OWmY zYylN6GXsUGL%rsMN~0MKnVu{w_#B)(F(-;-`0N7HFq!oJ_Hmij(l4Fb)`Joi{Otm# z;x1SL{lI$gJ{}{!6WID8~*Xq;GwqtA5;q=OPwJ zODal7_k$B_6c>sU(M7B!_y;l+CnT`I^^_*dvv|nu5CNhS1g+jmt= zku$zoYw!Y4m_F*esDenMqbP#|eV_08slUW3%B!$y;H1|~*)FNZ zK;dLiyy>4>*gV>;3YWCvV_%C;1+7PyUbq5KV%410GtjPN%OsV2*jMnDS6SSuUoiZQ zc<3?c=weDn5xXgOd0KmA$Qko01vqW~XMyIK-MGB~7nZaVsnR4%8FGMvoOIJLUqXZ6 zI|HZ6(*!z|(KO@IZ9eDThOIKpOXub=8N~F}{ivs~MglzE!TmFYkBQk{Ye77F;=mug z0V-t^Oc<$Yw-(~}5x&#(h*12A`QR}IP+7nQ?ei}+U)zL0eO16nh6{`Ej$5ItAS3VoE@bVGGL#eMci}L{>s^sBbN&4+ghk5ML?U zUiH~dPpf4M)DN%E)aCIlFUb7-C?YW4^&Lsa>3=?hKvK3Fd-@KoXR{kuX5CMi&yN9L z?_#M={VKn>N$KDHXxn87!~D^PyYXsmz<4f{`OqLfK) zr`CfU072HHhI7TKck6ResFUBnZ!ZJRam%vFqSy-Jt25mIY3*G;<;LZ}JgKP2I zU+uhKt~mX(py!?MJ|XITXnbk8PYl9TjlGMn8u^@^mFn0P8y-WU}5BiJc?hW$`EXcV@xx*|Ttdb0q73?iFthK4>}ylCL4 z!r7yWfS~w6uBnUVdB#3_>d_M8KdDt0(x2Afl4$OMW8T$hgqCcEsy+R82esU>=!kjz zIkY*hvXqpbELS|1R$BKbU}AVP;ci0mZchCS2$6yIX(Ky!@mPm9bZf6>+0mfPmjS%O zCOnNbueX_tSeh|9{*k`gmaTMQo-MuoEMTI9MpWR=+&U>76CI|;>Snz9)b0`|>?hO} zDtZxLa|l+J+(eOqp$Q_#V8D2fK1U4u_rz!A5{8WN;Rbb^1f99Xbx|VvRKnk4QAEtE z*T<0u#JcKDH1E@_=l1QGzm=Y;*3}3H<8}do8<%t&jx1!2y@o;QBIPviBQqA|+KYXr zCGPFz^zbf6Wufy7tPj5ow{gAdTsOn4FZZ`!;^aE`G;_n{DQe_Axf8}zmBFQW+rO$Va5ugB8l zab9xX_#BHnskl3}bG7eN@*l@w*T_p9Pm(e6+*p&+eSC^%sLiX{oj*>kmc$6uIn7nS z-^Lu5;w27l0kJKxZ_u>Z4Zz8N{FvdphGKAUb`Nf3uV{X3GOhYDN2E zy<>LMKs#6icP~BtYYSo40*06Hmrl$oHHM-`r;AqY4uQ=GkMJ#$0c|~INK%kd78`@j zUg^@1U(Mh3sAI)p(--<}SH#G>W8vSnkL&XL6Ey1*qqC4lo|E5im@k1a>Cft77;kP< zK5XQWo|O!1;DBp~XQ{`t>e74TPY)=UuK42q10|BBIL}7YxcwP9#CpcCfjS6W0t4LWuI~Pdy8gB|B&y;l%Cye4!T}_J-s;riJPP>NMp$TN{i2URQ_BH z5u@l(7d}^T*od9ylQu5w+k3IzJSG!XT%lNKtlPd6w$jV zo6=YI^eSxK5zh_fSu>7-W{G?Evql!e^!Q;k>6N3=fGIaC4xx8K5?Ii-bC&+^({7mh z#^2u=Db{}EKdj765E4WSd%J_35>{?`7j<6?2tSF=x$iu95DdM+#`);t;@m!UKm5LF zPg?cu)>dY`%}@kgCEGd(;U?XO1%8acJE{>knrSo~uRlFPe+V2xZKmC;{pX)Cwy3vWldc9XZhrLK02NS|W3kPzx;yD`kIQ158!h8?y zX-ts#!c8^eiU+rixSwK)p>eT5GzU%Aiy+Q%lT1NI{mY_|T4b1-jT$ZrgpwE_{3$aw zkeUPd@u0#D95w`*@xs7Z4mXl_h$ITWeQSQtC71`zW5%!^HIS~&NyHEX7Xy+8BtuBo zKcu|&G^Ew!&dG}`@lX2R#H#FmV@MP^1~xoa`8~ODOOF|xUcaIYzIx^uV;{bBB+C!K z?4kju=9T$$Vj+2~IO;x&DmP$mgWAn6)OCe8sn+Nm0+qlzaI2_IwRza9v5Z-NEYosF_tjNC3VyXes0f|eVTug!h zQ3vao&kxm*u-m4YI~-Bh@@^oGlRVpm_TVj*M3Yf1i|#IRio$hMf-TLxVJ_RC>{T&^ zqcUY5*}w2J0ZnDU!q@AVr~`4xiHh~+@KS486*$_?_Fg`3nrKsJT)hP}H%d`2yh2O@k>=cL)T%uXK`U(^uiu&MP5hH0R zy`%~lyeBzdaels1;m$n zuw<8QWjMagQVNQ?Ucf~L^`L-|DW~$hI^U{Ku$vfEM&#p(W8l)u)+|evgwFO?F2<(D zxhn|-1oy0a5XD)j4c>KJ5E!4`ECk@6P_S;vCpHTA_)A!Z(8NRgal&R0DeSMxOeQs| zA8J+0cZ(5{0y?ihN9X*58H6AgJ4_X61&6TPZLsNdvp4iGuqkKdUBd^Q2z-CkwBC;D zqNaITjrl^UMn7Z_^a`{c`INA`WH45bqAx&s>ecraoaR^{nC2*(#k$KT3LdbmB>+&m zKi*o{wQzOAX!F!Yty}b9yh_Pm{_T>|cxA2~`}8TrT3(j$;8`{*Ke^j!Prv|kY(`$9 zX9H$-JgmO%ng(!G{M~(gTE61PbX*8M%MYvNDEpXt_7r?!7Hun1>gVpoyFX51uBk(d zMiIvx2VoNMP)}D<-d;kH{oh!Kf|Z4uT=8xKNh}i_6Z$oB&7u>)P*_hP*IU^v1sb0f zN8~R)mbFxFzw^$3ucl1Re&1D51v z*z`%$I}DphgbhNJ>*1a&%b1sdN&;aZhrXuO$o~Om-Z$Jf7f51zRsAI;#>2EbWJ|1W zMo~iTl)Nj|wZr-7rkA4s)4V-fREGEITX0Keo6CI|ca5=3NVu_f*K^y6!-zr5t1`xx7na>MtW+h) z1uT!pci?ohtEifbI7AD3lU=qMtU)nM)u8pO%I%Kquzi-(mE_3}fbQX|W^s6`l z2j%e#M*xI`*XeLNu_0ZZ#BQ4;3nytmOKG7dXH3=G>c00)JY9vA4rfJ}6lYR~A``wZ zq=Qd;DD<w4iHQj`R?NCb{(CXC#Y2A$kDe{DPYIkBj#QNuW86(5CicYGIB zaH?y*j$6pWreJt}Bv>72Eu>xVaG}}(>eB9@cTr#s<`-$p4NVJSv*yU19<(I`!DO(9 zR`{(zashC;K-o_{MRJo&lfB=iJszaGW(b9G1XLR72zew)%M#~n#xmlNe8&-`9sq)t z3L;qXt}%UXE11usGBp09P&mt;x?EA4s#zu1a6Ml9 zuS~^(BSty$lM2>l)<1;8RT|l{!t#VQzDB%7kd?VU<^Wz(2_9z2uLKyD((9$gUg7-P ze*byP#|aHBB1}$g+uj+(MYb>hI^$WeAw!`tQjycM=mb&H9(a(jBXa(9Fae|MtzYrX z;-#w%TQp;!;&9MNox4n4yhN&mY9lHlLBx}ebU1KRQU6(*d7}{0@|7w_79$>G`0&Ps z?9J7{Q3%JSZ%I~unVU6KvLiQdM*^3_8Kear-37i9+eI)w)nf#4>vX6Zr4q-)-gpMu z#FlL>+;V^!#3|tO5NjQ*I|SHIrRkri5Nt=RHNEh}yZ86}E$<7kMF(&u5{uYc$@(P@ z!-X8Ij;#Z#(B((M1bP#FIU&FeiwqA05tn}Y5d93reB0Jo=9_ktKY6}f#Gltn zcny0l4XXzv3c2#j20Hkytxso(DNLH6p#9R6QsrIo?Q4T1+}Llvgw?A9eu#02dULw{ zXbjK^P9Tnvbz~PN$iqO6Vu!)vq(iSSYQT*`Wg51Q4-3E(YNTVCKUwK@g{QRHPW`tV z9NkEQXbt8STMPXM8C==Z4?R+&L>{;qCi)JRg%uHC_(AhjN*?J^i#4CXHU#jUJHDSDcG^kxmp^gCfB{b@I3k%G_r6uZ<`vfl74f= zHOk-q%d{i63>=2_GGj%(VUcwfI_2c=?)J=-rQP7??5C#n&Q7i4&`{Z-=UsCgNI#MT zMm%%`mGg(|Jr$>ht10N+7wK81tI=HSMlamTO=HC^)OKtxgAJ$F>;KsSRpJ9DCikx8 zrmcwCQ8`=wMvU7k2Ot;kTzgS1%&(9V{ruoHOGO=6e|l>$M-&2zKGYL2{vi zj*j$Tos?|B_8+(|PNEc;MSU+R>k=Ze2mz>eIs%Ep{8xU3Af2)uqOMmfE51vK(WP)e zEW6tXo*MJ&5WlQ9mYUc{}_v3 zPI1Z2KOt;$4P=$2Y}i^*#2!rpw)lq;Y@q3&1-L{pFeHXEX8I1> z?x0)LxF(yqwhPB;gUy6Fr)zg^(DZJR`^7IcV)NYmRiN>?kr!+kJgKpt!jUQcHM~K9 zKoqis+Gkb{67(-(yrP5LPzQA=#K)FjDeoK7$LqAih_Xjg)f=DHOBi|L$P$|+0j zS-v2i9Pb;paA{4W9&F$ISUka&KVkyw8ezf6(o6|%_#B2+xqho_bc&Jv$i zktS(Cv1H;Co=80Zqf*d(NsquS1OL|8zoGWcD2>ZrokPpn9w?RTjLw`W3z#s6C9k~g z^N;F7sbe-$C^_;0cEwaOZ{re!AZPGHH{7j4fPA36!%KUzY6tN+8Y+ocTM)jS%Vvsi zP_?yyJmDXnA7LiWvj)JsLyG)UwJC%IH^#EZ7pNPca@0bw)JA$>ubuwI!ehlrW$C0! zK*6Es>W6GP!c$+i*tkyCP4O`>V0rA~^0Xybbg(j_84C+f>2Vie3?@Vp$Mwp4p;=sQ z8%)wtP|}Wg(bB``?I=|vihHKKm29_oQT7rB^o=0BjMLAb@mV*=Y<=mPiA@(-D$p{) zZ4;Vkdn0S?KamnVS@C37#@WQK9~i!fXQ0K(e^M3`auU9(ebnS=pc{-5S;A{6LNvJP z&G)>NGry9I@xj6@wUj6eAA!=G`D4;4*S1}DREEBpJW`NvrdKj+q{>o0amsI2EfB9O(JO%?NKFvk zOMw9OA?^VZ~7$aI)@;FgJfc5 zz4Oa;d0-C?aF->)L(!B@p(5m&8<$XzM&G(^dD9da>}FtZ7Q?(>bD$ey;9t@Wp%eY6 z5(PzRS(tE)+mbL|M%uNuU`3dI_Fm0YqWT)1e4R@`w{?;xGg6vac0`=?93S1gwYK;( zJB52gTy*$1&pDrujk3=1H@%!Rfl2Vg_9A~nR?O)9Jg zj)S(*9sVY|8^tEw^$;1Tl91qhEC=aQo9-+L%Hp_<Z6>=wH1K(fn(6iJy)D7l#QfVGS9g#7 z{aD`^ilFOfzwUj_re=4H9#KKqyWwlGlI^46QVKEd6$@8cv1G&WAv)w+dH0I(6TQ&} z!Nc)WT;kOJ6zd8Y1~I4Za(Lv+`w*=LEgoD39Vri%778mu97fz_qG9m`EsDf109jWd zUW?k8jB;iQhL5oRT^5<+U!YRXR;y(e6l>Q3pGK@h0{2W^B; zyi1Sa3FsF$>X8$2Y;6Zk0F29;z<1xfx92i#to4mhux*s=iybWT*kiu_$gdI(iYCC@ zh^gNJ3ucgAU~e*D+#26}Ty6$NxCQbsXERq%iMdIR~Oc8V(SOyuso%~cJ zix~KjxeOl{6Jbt-Cr(dGPXs^-cSVwK&NUc`CqWuyPkb4fv#kF~Q3SBfioERGiX3wS zer1ZDzs|-a`nzmzwSG?dWe~&WJDl5)kKUB#gZNwWhpnYG(<8~dJN;!c_?5MQrAt0K z7T5b8q+;XfSDN29lxV|iXG?aJ>v|H_OK8&Uy@c(z=j;$}jN5D*GCT_NgTHY?x;D|g zhXq2#$?8K@o~j@s$pveoQnqQQ&us>*ueI9#cnvFUHisSu&z_!vj)R{1^4@~R;L5eN zRPTleGw&x$bXXorlCx(LvOQ!PEpb38LtXKuIiau20oZuSa*1St;GGW-_N+httLRkM z|LHQk^HOFq)rmex#eI8(Pn5V3(8Yzcaylk>Ro-~}o|<&?=ebt@?+knD7><9(3QUi7 z?Xggg+>aVn)RPv1o*!?hyTcK<#f7`Q;RCz9Vi@xRsYyy;>6SiuG&%IyCrGKW^{l6~ z@)rWm(%o-AY1m{Xf9~2dAW~E#;?h_SWp0t6r;@H>#CB+CxvvY^L2f7ag7hz0Mjz6C zbWzUX+?#|DY&_~ytBD#t^kZ>85U=(oiwFKX_>17go5@%&;;854cPD{4AK+#GLd-=n zzbB_EiX=z;4RT|g-4uBJ$2Np~1}5!B1+5*9r6}N?2d#<{Fm_H zAZ}aAP$E$fnyoCr?h+as!X%Bj^)H4O7k_KFa)q#xO)T701&kUKSHJU=4b)~1Q`P}t zjBSKb1^#kPXH|rDVCdhN=NLg&%h+M!rujL?Dua~;-GRqq8mF>K&%H_A?#@jT3%1JF7g&tvI{gX;&QVT^)jh?O zhpmXKhLbJCkFo0ru8d>(rdC*;0t#h(3CTc8F9r$G^Hrs;U=YwOHZ+|4-em9e6!MP^ zOKF-4=Jo%)0H9W5JrS&emGP0tMB1`$IXEn&1(>B;e8DYal|wCnvB zgcsUbHasVoYyu(#)J>R&>p$(1n~NBO-ZD2X^dxW83Tbh*#dY>Nm6wTe68sz~SboVj zi(bC#Di!3|VGaIC>(5Fy1<7QyFBEhu>8WHG23yLQS1=NsrWzA;!HRUh=Un>iDATM$ z{H<{slG0$Z9AR9H;N!glJ=QK#soyzw^M4 zym_|V_s8O;U&;Z^-F>j`bqF@Ql(JZqxLC7q@}te)IBb@t1$JBuRPqi(o%t;q+>z7!diX5;wh#4@rrd!ia4G* zeQN%!tx;q(yWnotZ|s6Qd3L0<0D9>z{_=dJGp(AXo7;C!zucPqCcJ5Z$JzqZKCt-+ zB#0vzU=jSR?GM_?nNj2IU#DVJxJW=4W*fUF;W7DeN96Q^6=Pr|8AfAcY*T>IQ>N|0 z&>}Au@**kmn0?qrA`?BULZ0bg%`0(}I!23Jt9HL$kQ8ndBIL);ihZhC1UZ_xS4+3j z6C4F|oy<0Ip>%Ox8sojydj5J!pAmo3<=kcIZuu?IaZ3YrA};j2YfFT_AvfHc-3|H| zz^3;QQ}0rBJKxlrr}^9ZGIie)nxjn%46g<9O6@HS0A4Fn@GE%$P)ZP%pqNfjB5ANj z;q@0L8I7rCKndvApV$UsBp$SF2Zb77Yct8A!{oz1@}_*zS?$t!%Yw-QjwMF&cHBzV zKCX4tRe%)5)hznkY}|2ub~8@X{zl=HPiab?nqi^SpKrS%yS*1?Q}BtOXdrk+FXO#} z*0!h5A(5=4EY#BvnUvntDP}G%fJhT43fGxF0f0uVery_!$U{aP}3b^kcNpkLyoxZI0OCyC=IaPRBP| z(Z`qqf^aekI@D|zV;FaN#*2)=k4eelN@nFgNr>i$@A9tVL;E-sh0|%cV1M&E1oS`2 zJ4z%)z*iz`z9Hb{8u*h-c>UU8kzT(gn9>I|7qjEXc=5l?n{C)sH;rN(!u zKE4dS`kDu_{~0$5i=XU=H(;pQNAH;**Iyn@yru=Vw*@wg-1Kq;JZ=_ckRV#{kCqaN zPQe&JjCdSM9Bh|hfn!Mp13u2=5ecqHD4v{ju0tL;>~t=8*XgB&c3X9M(oy_LXgZnB zkgnvY`@FiH4gyOmv2Li`xhz|OQ@+EH7`_F)AEs59BycRoM}vpowk%uOD?`5ri$+9Q zVO9TLZ(5OlfO#$D+#qMqmW^9JN*QCt4hwqR-9OG-$6AOF1)%GH2?LEc}j`rJ>|6viBm4jaRVy1FUuZSu{?V#lE=$651aRnsJe_UH%hLZlcyhj9$nx7Lp5pygOYhA##Mof!mK;C2%3rKT z!3D{GJ!0p&%)#(&(l#wHlopu$kp01JG!Edd}ML?T1*C}9*y1OQ$1MjeSw zLCQXFJ%E;I^58D3NH*x#`D2`khDQXr+Mtmf=fO5nc&H8iz*E~M)t0CvH_JBcnvGrz z%x(o_gDX2#bSpq7xnO?B&7Y44CvT-dCrem(gv{Unp6=ZW+E&3gt;qPaQ+-N@sgA_s zX!Hv_Bg-=q1Xt6Mczb6~q$VCsUk zdm_84Sc>Pc{SXFuss5$-pOxGxpA3AZ_~3daIb&WnGKGh2)Jg zey3XRA1Y|H`O>t&_O?LY`abMTcGB*Ii3Ej-CU#bKss zB|Hu*Q4xm+ziYy}lwi~cOv#a4gPygwHAXrz6byB@e#oH?_^+g!RzG{x9iZ5vkbLl^ zJuQJQOj)$f@p##SZ&ylFe4KV!q4oe?u$!6+Nx#)@2lJziw5uX7&evhyJ~(xy3-L&p zlLjEF8_89x(r*B`-FI+PkJ&jruNy+M)3m^2V1cd?++F26#VMe6DbT|bh(bcAUP@X> z2KD5H7K0^V2w+PrR7$QFCm5itZ4`u?5^`Hm8I1LQmk;D*Vz3#5S zN%@ZTpo%|qk3UX9EyqurmbUn9Kipa|nZu4VpdHpE{)AVSevJuDB_FhlHwh&u3ZPvi z#AKm41_|!>GxyF9 zmIbxOT5CTyz?QF?;+EjXA<%gd(qdib&Dl9`K5AwlhHTOdi3K{dwlm;} zqw7fm$v_uI>LFmK91zE(f@mi$KM7=F5FKC9R)X{7p`8*nL@WtQ$&WPF*D0ZmXqO~D z7}F<=k^wy7vL=6%UnC^x5^c%WIOU;%Z>T(c^4Lo$zPCbEhNR%j&Wo5pc(W9CQ}cRZ zI+=R$*reYg>;v#n*uKzaH__{3yZ(S{6Q-t%Lo) zE!kj#$2~WHbakM{Y|;(N0^D6875qCxz!C^TKuL&w9CT9FOjJeED8UgBN-BYex}Yz? zc-=}#3oykQ$21N&_)~s-+9$l|OEzgpC@qaH=mmFPfJHLQ^V&~9$6>1q!0Cw1LSZTU}E6dgr zJ~~Vu_IFKIDRBMB@prA?FSsR=UFKMk^{f4Q|y#5%uEt8DHr^y2WtkZ zB~7%81Zwc0Q7mH))t2gyLAKR0WC;uRV?7KhH~0=`w}TF4kr3w$|b9_JHh7{F2Aj%wfUo~gRy6mws8wg zK6}G#_}`l>@h~Y^I526@-4zrO*+mT`1%-`ULb3o-i1=9WED{iTatw(kc2XfHa6QD7 zwBkZ~B(spIK6MEl&}49i?%b^*f!_YmNu#?xoq=fO64NQe2HvjWIu;e;3_9wkg{@sF zo-)dz6$24~pR|8!K?f$3Ls4WHbg()n8ToovP zVhD$cCu2+ny8R>I4b_$ONbnFH*G3>7ATs0)O7vxLVW|u(D6_S;u z-8YR*ukHn{32Itk$Sp9X+WuBtB%Y<+3AvX}1=&^jF-ftbnn6P6=K)x8c;Z2yiAD)3 z+N2|_L|@+_VSsohK}k!pm~@ob$jfk;>?~;#WUv@3^!KDkSs?!*sQWBKny@> z`zdhq;`@Lt*(B_*1*Uy+?H>3{=@W8kCICdQySdd23t^58?0LYX{2QoO<8*fEcnZIkHW>kY)Wi zedJAz(v5P&FZ&iC+bmRfNDK5`B&N``CGN+<4vR^QM^}c#(@@DXe7)wK;9LAa1j#vH zS%Wo-k|g+t9n4ZTR^a+L#-5Hhpv6b;o-aIlxBO{_)!6I8ly!pdSAz<-IJ_s?&AMrU zp|-%Jk8PX>o~J4agp;d=Q=>3}D2ZU#Vu?#V+c^P`NrV@+d(w*vyCQm$Xqs^iBEr5NQduaHR$lg~;?iux0pr8PT^8QtfngT!PR@`MHXi>MfN zK@yoqA(;1M_F;!wWmMlk?M3`3hfm1&pE*JIcYPVK(Rvpka~>?-CU4E1pcr$2syy9+ zuLd0e`DcShw^R@={qppe*}LVJ9JNdSY=9anxB-?AFx)_1sGl`wtnVlr;?hKJTA*(W zOgv}xE{N3cBN1gxI9{1$?JUrz;cHoCZ+9}!Hr5{oA&IW^i!RB+6CT@MUP zViZ zkQV%w;60eb?V4wv_8HNUp02q;3hHfMznX#TUf5MFK@#{pNJe=wSdI1@uDdsTHh(0d z+L7kM9={-8S&^05j51VW3<>U|SKgl$a+f}mJ-YPa?Bg3MdVbdXf&4(=f)_r0Nf+RY z&DYKupT87bw`m{f!S5H!XRrf)?VNGBeg@6eXjV-N49WugeSG5^kpf<#1cBX-9GqUs zguuWjcP0t8=!hCKnEH+4sy*<*%l0W!iFk$=d7|(CcAipx+7K zS?pC_nWoupTA)7`nDWU@2jC0DU*b-LiG;3MBoV;WFA@@QZ~)5WR!$%U3q}gIS(6^L zY7;?;#u6X-`iSsNX3&ur`8Y1xJQcJ=7d*j0yiH%p1b>rjwXU-Pg8Wci2L|>w3nUGF z!L!z13&!V*a}6mdOD&uu$`_a4nVqgy>1F1=`JqTM=Yht)>qmPeoU&!29#u1vcS#{^nf$aCTW=#M4qfVz*t=8=>d>K{JuK(1F%vxefI>o7blA zFHISKZ?$_lesq2c^#8N)6YllO^;Fb-(C4$@@2<6_i*)+fovhhwT41Xdn7$xy+t}8A zDK3T|$z-EbCt}AI1282aZ0jTV<**J{n@IxwGMu?Zuo$C*EBT=&ZonyreUvhfZr8NL zi*!g-yueQU9*1-!jQm)FKVkSmtkt7NPf15@8B-eVNF#&@l%T zOIDl@b~1-f%&$R$dIbmOr@UpxzWGNG?EE#Hqex@(NWbXnNQVq0S&{Fu`?K}9F7*xR zfIgmf`aZ(@Dz~Mq{qje$uXn6Y(}4G8==u-nr2?G@`_725>DIYp^0!S-d7F*c>&5G{ zzrc`8YSe!pK$&De1Nw6T`+223Juq)X{?n624E7N>WvR{f6WIbA+S<>lrE;<|=PGt7 zaL`B+f-z|@c$ks!Z`YC7x5ID%r;{>i_+PY(Q$^p+(YAE|dU6AW(}_Rj<1qeFQxx(t&`6~(=R_-G(mkxHa&3`IG3N`49w?a!d-cxoMbv1afW!mRx(D83)?4!0Rf zg9m|g+I9D2KN+k(NAH0zCF9dU$Q=QmwXRA!@ahM%+xp9QK9XtsIQ%YQuHQpEP&$Ix z=W-7kpUKyy|HgALHXXlH`WVJPT+EMnn4?Om#(dk6NZ+!!Bl{m8ZB|VSY+-@PpV;_X zB!}}Y&@qfep#W#3N77(oVcVUqo;(cSk{8+>$H4@pq*SJlu^5jFACl*MP>EhUC?FD7 ziOSl=81X_M3=wW)UWa_r;}27Lv?XoY_o3tWSPP7P7qT6aGJR71Oc?1`U^arf@euQ< z+35kN;J`ui$M`Au)9~|L4-q~T<*42BZ$l<7TFRDRzkF5p?~U`#E60sV{{{np2+8kI z&CBP4b-N$^8^H5vnuo)?&C>;H+~*I_YZrB7-)~%(CdCuo0+T>~(2FxgQ-7R+JUVv_KlEy-kbKZ(=Cpi5XIHuoTIdrpF}3*^ zZ)tBfpr$eIyROx+^Bb;Pnf-Fmiu7P`>~8tNxc_Q7WO6At;xu#O(zOFkhCm&ZH_Ck)bYDrFC(7hI8C`-x^881 z3|EPdx{x>NsclJSR?RlIbY+ttI(~G+KlK+=1m}U5uCBBI)ZLO*!Ht$>LCplErq>GS zlv+n_NEh13(7s-_D!U0Be@DAP{wUme|Biu6$nquYvR^D&pYiKG&jY_J#MgKiY+aW1 zOLG=lef}e=HQdD7EKLiz1tu-XcdoYNzbldz9SH}6oe97`bwf|P6F3qQM|?V0PAHMA zw95%+Bt7`xnwalyz=w&4ypRFP$LR(oRDgphmBc#^oEh3AFk4sK1bGicgfOe z^YL8x!HT+gL-t#&-)DV?Se{F_IfqRopE{Hdd~*51XQzDJV0CHIKmIK+>zjFNYTNX4 zILLuY9(0t5OgOH>h)IVqcOpwj7C>tIZbu0MI7%SmV@Zk0skqMyaPkaXYr9M(sA$*a zS<>`=p&OH!^P()aNzZX6Lx)M{LjP=YmUny{7y~LHdO*Y^YQ=M}V4OCys@pj%fSNjM z3^!Z^4r>}T#-PL;q76s6a&^X^AiEM5yZNTH$A(AJ>xM*5mYym7!A#D7#jF$&gYa= zz#I0%ePA-}U?oi|uZ#HmoT9Dy#{AWBk%ZyEo8pAQD$%5W99sa%YSKBIzJ{LzegjXH zSmZ7WdUW7ON)-%xIu(7$4et?~)<23!W6u#lu|cO^>~`rFKDE2zf9 zhf58E`_7|tz%%Fv;l2;2pF@EaL`9N3V9w6z`TZd48wHmT#IL+_^7$5oK55NA#vDc* zJYr*un@|hdX5$HOfytlR_*De)+nI3a7R4@v4ih^PMkvRq^y5ajc z$ApihLY|&@f?m{=QIZ#lkg_B<=%pWdP@*L%@)W+)dvsWkGg;veuU~uU$i^?UQ@1E& zKe&?FS6X5YWLDl~LC?aDY!`T?HkvmMPKmy$f0{Bp)dmmTbD6+Cqi>3Sjowtu1?Xz? z1u~d~3t8;f<)UM_gy9YF9|AOAo^%%2?~@z8l4luyQxH>U6ynh79QsND+)?={8=c8^ zJHR78apDs?PY^^el8_P)0D((qV}fFFS(4$jbsb295jn5XE18z~#px&A+US5+oLW*w z{o=lMRg?i(mQ~huwPl}XZ!ksd2EzbfM7|jGnm_GAm1R8EoRwzgb*W9f7=!JXu{)e) z!-(`<$kP)Va6N18+;RD_gOXRn?Gj#!x%8NSOjl^^_;}14iS5Owqvkk2Hg#9NjxooioQcjVXyMLUQWPGL}G**)1xFj!0`N>rQ` z=b+h@U_Hi(FAL=9p$(ns2kH;w3%{-dTk7jNv@;%Ev(2yT^rcR9RQeJ0=TjuQ6t@A&V9cH_H>-0+Y_!@FgUxcQW83L0Is^BZ>fm4Y(sh z5Ov>K5ymkbsFgy5XZT7qbTHBp&JuOd5$5fXudnt)w#l;8LtT2F;_3b@$|XGxJ@k({ ztB)7RDiRm3pK(a@O!tl$z40>?J=nsj54zx2WX^*?JbbCHo4Wx8*rmsTBcBg$I9tAUPAbd!(aCNpCbNF1$t7jdy=m~e5bZY+Kg9X?F zCi!|xmX@8A^5+axpC;XoZ2{i8FzK9)=OLB6OUZ_g!C+@V(-{=l3=RakPWJ#64)N@$ z?}9)>m?taIqt66lNyB*-bj0=WBM#vxOi)xI=!*nKSm03~Art1f+i`fg|25wHLznHg@GIW7}r?9^R#E+4tIU!`s|<3 z;Rn(qPk3q#KQSD5!CLHPtBD2l`NSMqvqa)z+s8Vvt9mi~VGjDVn>h`BGZ6~??rAOg zpH6AZXAr+xo){M36GK-$vgwa-fK%xNwxhwqE3h#~1+_VljX8|CaFil(F|8>+A5MPdTKIWSZxftvPyh z!%tj+CcPGl8?rKULVgyeaTaYSrkqnqKs#)#nsA38 zhJ5ST*p{V<3kUFZ9Xx&E9iabOc`lk#d7jXNuoAorZ=NsyOUn42OO>ak{AF;APXTEX z5%tc-%l4K0JGGX|M;9W=25z%{{8?b)Ih&^7jq!H~=!2Aa&|z?EC&K9y3VSoaGJ2}He+x2k&(}i@=NGH;o-u2|j z@Ns(R!gh?w;Y~i+RgPIRd-RnQKNN-P3|U#WGW$;`J0A=$g}#=oIBOiT2Vp7Qj5eQw z@KtnGunYOh5qsr>zttcZ_EBH94qw;F@pT>QgeRJIbT$5U9XuxHnc(r|5QBC9JW|Z5 zn8z(KZ`DMA9PYBbPuIFL=vZHvvLCnA(m}xa0Fc($I~(2$E0s5c@2&H9%0DzG<)g|p z&GzHN0#nc4a2!JUas=~1Od81UWfzfep?HsIytrT1r-+xYOk zo-ih#*M+B&(#|Lkwywu6D`lIk+xP4IHau)FOG}{hIO(0I3+8vG7caowe+w|Dg{^5e z=6Vp;bSj>P`VbuC7W7{X+&7Uk66FuE!+K?JQtG*e z7p~!RwwP+0Wkr?g|h%pmsW3QY8W3`S`o3B?ZhBctOvW za0vllCKWjZlUQj?G6~}0@_WkCSm2;Gd9hmp9g1`e{dplBe`Wm(9@;sAfb*apc=N5* zvIE!89y6q82vqkmD|05~qdGd$h1gxp@mt_Ap^M2ZLr*!qW&FO^Sq`rUg;~6R67dL|44tcGR!PCP1|CF*7Ta$MR zPFuV3{B!W;=fu`jPT7*K!p+gAFM1^Vo6+W#d3&Z~p#PDzO0Ak#DwphiU%H0(oo$VG z6F$5a;FCf(jA;KPg7$O-t;H7uUP%bu2!1*c+QeW|=(#%)CEp;>EjKi5tf5FHGLEG@-5+pCt`o87Vv_RZlTw!qYLHci7> z#E%fZv$Qk8S}8d&(Qs`j&l6XXplr=-UFldDFOr66>WPADuhgK#<#@nR!C36l3VPzZ zyhvu@EAd7so}vYPNxpOyZ1R zv$P(%9>1_7yS8`yEukl+d^+C1u1B)s&U|Ales1g!Y1>JedOkQly&jJCaX9@`#F>r+ zBhh(vS^$Q0%))U}_Q2*mJLHOcvQ=AZ(u53$1ty=p@io|0d>uEtLJMy=TE{q5)A|vGcs#hqI0_nHbGyh+Xe5o|G|c=#GMxvhJ!wgwXp6Vgvn|~W zSEqb{F)j-IOS`)mFDG4T34A2NjnHQ1npxxf{d8Z__P4m2{Y4)hJpK$_VgnL>+ncSiKk8QmoJJx)(tYZ`gNTX+tSB?qGSck8cw)g*+{aYjEhpX91Hwg zcfwKaZRuTvZ>vzxg^y<6&RVkl;h>)Y{YE+}RW42u-8B}Z*c--VoqA(Qr>J8x7zr>C5Gt@&$! zqvy3#FBt5Mi@LI3_r&W@zf#3_smO#!p8xy&l#l5T9dQ=oXUP_?s(v_c%MOHhJw)d# zJ0RN;0ODWCPVO)7Cf>j-u;0fv%xUl3bPe3+)eL4j2VF__kGEPSD8P^iiuEBLe8ASP zXB!+arUXJi@uZ~WK%Rt=7L`&+w1vwtqn9p@%XX1m9431Dkn61EC|(!=O!TY^9%yjf z`;m+TMt*27U3>dj{+{Tt6{@tu6#U-c@32sB=W56l?mG43o4$oNpe3M~T22PMR%FhrHL&1k zV&GpS2rV6747Z%f$jfj+k3Zs&PG5rodL#_u6|y{$0b2=9e!w>9v^y*0JB)If2t!x; zi||20IC~sZZT--NJejP5mio9Jn`?RY2K@TLuzuhpwW!*TQm%X``)@4L+5V{^lgKBI z*(HDVaOu~YrLSOJ+PN$!FPuNZzOHlfh#ZcXeu**MTPnZ#bLvXlyihJKeZ5rDr_G6o zl)qtKH(88i(o;Tt zg1Ddy-uC8ZW8%Y8MBx&K|8Dxb-lXRrc{7nYPt^cX2H!Kqr;?qye%6>_e~1B9*dfd8 zUGwk360f6jP_hFT<)3}!quGVSq3nqx?bmf^c`N{p&R@eh2y*we)krtBbfh`Bf9lb} z%g?iuN9H%c0S=MVrtDPQ&3=9(d{3Ov`5b(+=y!CLaC#caoD`hAjXrmfpKBjm`OMO# z!}kG3Bf39cai=b5$NhKtH*g%!0M<@OI~TLJf*5@Twu0ZBiEA9YCQELC@5zJ)AwH7= zoy`-2(h1vy`E@Jd5e?wrDLSHQ%}u7s@iui*0%I+1N~lUS>X&F-md4l?Pw;j<%ChRZ z&kQVpiZK&OP?DWo4F|xz1cfTjUoe`(JKOaAJ!KTk*=Xwo}beXb={C1ie-neMg=uiie@-hQB$B$J+@+2%inUwr8eU?3l>#DD#BZGL@3pLzf%iP`bv7Mc1F_b)< zr-&%`;7+NR!Dy3Ui^45!b)IZCNr zKNZJ9Xf1%HBtiRwhK}V&_cq5l0RE_S4GFd6F(I*(

      edBnRuu(@m|j_~Qk1q_=Uy zGo1j^nDI22$rnC(MxrdyYtGaON4NR2pKAZI=7{mw?Qfp~kvn!l(%QSLQ}<|1l(Mid z3)`N=uE!qeb(e0)uF*h)FS>iZBi)Df;c{`M#hcC2d&Z{SJJWdFVz8O;p(rPf%in|N z@q9tTD%lb|uIBiG@|{0DAJe%uwO)Kz_DGq>+ygsKs#Pnz@$?8fWa%{K*LhZb-xb$o ze=p-V+bQ8r^)VaL=64|w-wV6#%A~_&5XnS|fre5Np|ijfBMHzc=`>1I3~;phW+52I zfZ)8i?lCSO5G~!0H01AZGCh(RX_W9v^23m?fY9xDEIK6&NX>0Pgd zGkyZFz2)qjRE5uz$2FPOmXBR=#g;di&OWYVPFI$_1&%!(iS+OK^OX4~;gYuXj|YNMiI=R#@P52uIN`Y<3H zE=W@2NEFE`__JmqoCT+%`&eseUU^?tCCl)ODc~mu&Kv&O?*3_#)h=7!^1TgN^#ELl z_)o~&pdCSm?=@EOFu7XSZL?>0oj!d!zFUAY`}q8TqmS$S3ofKy0w+Bc4t_NB8;>C2 zF$1^q;?%iv?X0J5K7acB{@+K|KV7#!)|B^eoH+Hv8_!<1)20=$$EOh9yD^~4r)F9o zU#D2Fs4a*&XpTr+jd@aGGKl0xkj52tjxSg#DRE2(22d-(5Qm474jTBkB*WUq>N_h8 z6Qv|Sl2uu+&^HoiH#y|V1lCi&WRZD#sI#@>McX@BK`Tvx^$uN*-8ugf62`m2CzwNM zJN5-*r$uam^=x{co`b+pqO}h^ds=`wv_NcVg z?u2vj@iBR&C7W^4KeOe7V!93Dxku!CcV=CmKrp-(Q?1bPwF^l2a&{&AR$i%n070+| z4a{Irkl}>^LDMLhqR!F{d9~%_YyPn9uNZAZt?fI!Ag@emZ+{kk{_XA9Jv|2(n_&m2 zz&Uf4Y%5q%BT1NjFye4I2?JU*#79GI+N^}_^9(eVnC(F=D$%8aH zjF)tBS#herg&fNBomDY6%HkZQzv)`+s(dWw$(nkEE!PMD;LAnr_>wRlHA8*{Pc9U?l0$wNLwhV#Ei8iQ({;CD&y4UVe4U>$_6zC`h_U9?2l8=2P*+hfiZ?rDR38 z1qH@*ssU_z3ytD69QV*X@A&2X`Hzq1(N8^N?Vi&<(0=CB&Fx6ec&J{M{sjSzZ+Bu& zSY!nxgI4py{Q0RF{w=B4+!erl!GCekeo7v66byu&9EW4VC0vKmGa&>I$`BsLB_nWw zBl*I`3*kU8TJ$+y93?p%iqk-}O+JCt0rVw3LI+B+al_YT0N?{^aLBDyv-dw%$%<-F z?H#_rV9Udn<=QN12E|0~9PmV1(rd7W*KjR)1YfV0B3}O#Fzm*r0O{vQS__|^^1}x# z(+TlZ(a&RkUG9XhIC1aXZl@S5<89DibyeFnS6tP?UmJb_64Kw9oWgM}04#v&rwm{S zlyF|w7GT-jM@MzMc^l#$v*ClzT(jH$AJ};6v=40fRo2>iH|F|DOm&i7{KhQM@p+hz zasZo~5(S}fu1Ff1KN10HL`QmBMSKjmyBz_O-GJ4Nk1XgBh_)xk;Nzo^0eY7gd@O0X zd`2*Ifq2O&=p~bSOSO{zfDl&@YfFFXMjBqTV(s+xxd6ACezbnZm|^|u4K?elaJoBu z!4EY27E0QqUua0QP<1=-P5Fjz3y6s?+?cH#-I4ey-3@Uq3(RkZaXvDlE6v3PooB)) z{sl5`kz>dU(mXie(x+APZ{sn?dj+pX>&5qHE8zG)q4OhYwB~tDw^uaEXQ233UDonz zSoGBqeCGU^DGkis2L2T^?6XV9W*8udcq&1A;JVfa1}c9z=%${&Vc-4Vx9Ob+{Kw|M zcGlYNhh@*pv&!?Ke;f7XMRfjxG42 zC@b2QWYC99lZ8RaN8c}O7m3m3($=CwMLxC;PEY1euO#I-Me+(-)ZNz)yCm09>qj2a z$z*VLh2E}*=1@u3t!p3iMv8eX3$%TFl(}Q`bC66v;J!k;>L^X%p-NXeZgG2d>p*98 za!dY4@Wu1?;MBEr#KP45@iDxAYL}G$9SitTIKeLAgm6;)>QDBG5u4Mw1G&C+{GRz? zl}diC92R${cV^kdOIKzO4|E=l(j9et$JwCykUSLzWzfmV>98#BxPs9*3NXOw_;PmP z&sz4IwyTA|onNNCGapf{Zak`1sm@1`zYy9SRp_ta#H2JY*BKF6oaVC4Icbi8XY*&^ z7Kt3=;_3YAiHS78FX)X{K~8$NSGPWs1Zij|BtR8_WJ~f#3H?L9y-}DxI#rL;bjl#D z4!pQ7;zFG$g%hFv?2^vctk5awwaat8s0VFh^~$cidhEu-ce;lx9?POBJ(ddeSN4*% z*_p84n_%0A!oHxzNAuRlYV3}M`#;`S)nA@H@wmIUi~Th$Z6}u( z4m1{y@TpBB^BYeZmA`mv^jm{p{6Kacb+_Mn%SXeX{(Wn>!voKz(|PHfa#wl<4*GdG zI)j3J?o8z93;^~yf532~^C82d+m>9Xzhm7~4ti(%iw=1Arq3Pt|JpC9v~FHgt5hyV zu$~Qljtc!$E9gvD>)V_-87{TW7ZdTe0<<)D2Dj*lK)VoC;|7m}Lb%z_afru*Nq~}~ zU({%j9}8rJE#-cM`(dZ*(z4(Y$x$=~Ka(H3C(|)_gPwTY9R_~LL_c&H>av*SjjDOiv_D{VM!Yfn4ZD)`Zh+M5sb7F~F6~*zv`WJ71K>>}tJLsloyxIDOjDFNsV6m~t_(nLchlc?2tF6!Mgj@_ zH4b_uHjMjRjVHNVLRK<@9PMs`CS=N9OggTMd_gSTC+fNtdU9vQcL3^4nQXF%9P<~l+z|@Zdfb=ecgkns zlf*@s%xLbA@NxBT_uQCX$cNuGGPy`tG)ko(2`hNx)`;g~hxIuG!$B6n^u-J@Ry(iS zS^3a~>$Cd3KVo6fI7V} z$L$snl-n~n(f`-$FBq|vC$<{Jq*K#~ab0O#M_2p!YPB-HR_Pj_S1aQXQsZH+@pw1i z6-MJzGNwVFX$YF#G)E6$1)_o*t+HO_cDG;$b+2PuB^x6@3L#yzK~qoC^)aGpZICcL zppB*^LpZKOIq?Kt*tnoC(}w(Rx^S*Q;+Q8d)f5zR*j57MT!8P%STfnysa|P}SRq$@ zLGOL?)~*ilS|vMm{h?#dC;J_xP|YVqnTtEtu`alPoLaa>nAo=#wP$Z_WOAou`5&>6 zx`u^Q{cY~B%Dj1fYwWo64#+(V?x5>iP=k)Y7T@>##OPgcee1Tq@86ocU+0VPypo@r zW2QUyUQwCWXwog~)K`1f3HehiS?9a)`NiAdpyMS|0pPmb(w2JFUt`k)XdY*pa&+KP%I2L`=%i-l+Ptds>&6o2a zU)V(Rk{mJvJu4je(22f=m-@WL-FK9Q{<14^O=rWQQu8;uE4Q zY0v(EiM-9%maFy_{3O?h8kw9vhBV)3giN@+zO`^o_9gi6{@87O4Zu!+mmZ7jJD=LH zCco)~J^Q}CwdBs~1qiIa%W*1c(tZ1Py{utHwn4x<*=HT!`K>H=pRWS^k82w-|f`(_fHf# z?mCdBF3a_&UX%}c##^?v2^IDS6Y3H4&eMI(d6EvL=ln%y^ps8+vK4X3hdd4E+#NAl zX-a%~qLjs(vhQF4K31B~*(bH;uPycaY{naApGeynyl`!{YT@eYX}G~|CYWJRi;KVk|u?wN|;9WCiLv|zSQFAuWK5A~~&ugjkCg|~I$QU6x z)U1;J1bLU5vqCdEEerMV0OJfI05r~EP=EoB8*1^Q7%{@$UU7ELKs0%3+b&9+$wL8C zoZ7JOIvAeG*DVRu&jXk`7Y8Xgro-E=KjnseX~#&_ZpP$b6ytUJ`lpD1bDlmXKH>>_ z{2Lu<$pF2Qys|w-=esAuBLR}W&_CphM_re)e9qtI2PUppod;Sj-{w-nbvm;HgnCjf z=0*MO_$HFod(hh<3#a#lDvLH{XCczhqwQI6P3zn9Tk=z^w~bd4Y4x{(*N@@FtFoIG zJzRMkVX*G#pSqN&tErV%-eyn!xqfy ziC5<12~+s0^l!0F{7HHG$XdLdfakGa*Vk-u_xc|({tObpM;f|@)fZpt)45me9dNd@ z;CZ{2-o4ksAwM~;rSgUH^{wOf@50V1w?`?b{0`v!6FxQ#pN5Wv6Rw1EOYlzn|6Q@H z^{>6jK5cV1=vMNj=hez(9i8qHPk;AOS=QAVMw%Vi~)>*qgDEh{6R#{xS!dduUV>H9*KIJvvDkI}9tmmUHfQULtlwzfCUXA}sL#dsSHIj@ zt?>HRSKv6E3@kVke>LZmYr68AkDJ{2lHi7CSp5WnjVFi$QpqO4p^rtf+8x0(27!xD zBGVfymCA+39Nzh-*)#LWK{snwzSGf1cfD)F#;)4{`vFe-}ip+f3l>4qKn$} ztIKlw&;h)|R^?=Ab~GJrEnHvF**#O>FCLU*@{42#kPM|7P1*4CauD^8j0E<9V z?KYteiaubRzmi>4Pt=}0cfb7ES+(|gIPt-9uu66U+W2WPekx_o9mz)ksRid|z_8dm=3a2hzbX-s zMq_AHvK!#|pBd9q{o%reEuBWt-`6a^?sw@TVDnHp;>DYf!Z)<9gaii4$pLSu=A?|C zubPtPjTcKw7}r1gm>Y}1(IBZGoFfB2Y{Z7y2{q@BzKxMu`WmflgYlkqL0=jp111hn z9_j)V2`A)R@^bwp+w^3ja_EwQmoUw&6RY% zyIyyWawK!6?eTWALmPOW`=Z-I(FQFx?$ zF&y-{S1zf{yz1hX?=M`4)8ip1*ZjP7Da?0|bj;F0Ghe#tz((0>ZnQB7?YfW8mvdx9 zY0T!X5!it87}zeisEN*`A)XvAGqLIM001G%NklmmEOd?g9d zk~j5mdhj+`=(|kGcYO)d8WGQ=7(Kwn7@Cq#dho8x;v50=!;+lKC%F&V{DDhd$%#8E z|5KarfKdkjP?N5(wcFF=r!OK|;nU*BPqO07HGMXo$mPP$44bX=8v3B-nWvYXQc0iL zV$NJ(UwC`I=@uA64q!sL)BU;+ODNndY47^mIYWlKk7sIG#L zAWR3aup=wW0lFYX->D~>;OY487vUc^U*OPBf}iU^I|J%)r%%~`uAMpVbm*ZW4nJNB zr}d;Lf9=YCisgAPE@)^?QJ;Z}@ZBDuj~~bM!R^0}BdYi=Bp+K0pQ3{+c+`SQdgsJ4 z#h9EPcnYv^NIGL}_u*a4>{sFZ(C`I~$kI7S1drF8r?La;*hOjQ$a!LJHtzEVjDg8O zx1O zvd~OVnt~@Sd=c+j=hDyI5zalOn!o7el<$K_%X}D%jmMJEd2q8*o_=}q=={4Ujm@9G zJq*;@RXah>M&)!54W#-zsXLqLTnat~30IH~Y){BmpFgg4S+$zI^}sR<;hs1lO@Q7zRH-o@jQZ->JaDD|ZI=cQ5aP&QaaKyeWEP|nbS^BN4M<10|v z%b{16Iy#;QDgWIcNw}p#-*=;JE$B$aFpB_0lR-!u@DH`Zkn{B9VFAy8#t7k-q=io$ z5j#D4vWsj3P8uf+f5g-27m+yT$w~=`II+_XUci@L?7K|ukVq{YGz$4j9i|_^(vkE^ zG)8awQxBuFxeyls@K+yQC28s_{bdb8Wha{#x&{%7WCT6+1fB4*?EXrgKYz`PaaV}o zas2Y6OIQnV-ZVeYUsJ6l++~iR9p_Uh=}cVcSV$%fE;t>C&YRM)UAUNi($4t{E?S)} z3bbw2^KmD8N7gw5d68e!nE|Jsc~7Nw2-uH6*g+n77|6Xs`5Y14%>9!WPui$79a*fbKqK zfnBnntOpz=kEqGa3*ju#h51%%4!acz#MhuKUt@xUPtZ$0zPQi8{+2+jUtd3kGy)P_eIJLrD+HiAiEhj13nFW>;9PamOl@er=zL;vL+t)0)^bjU7uiuQ5- z@}x^xT(5;G+jL4Lzne*ei*EtJ1;(;-vCTy=R(C**jK@3gZ%*DhpYhk#S+~WXwRgVL zm};8d)zyWhgzpemvKintvorCU5}xKnmN>$h#1(RiK;d{LdjJ@FhVz-zntTQrOVC9v z`eHZ05UlliC%R!GHXKiz{?dHR@}Fkw`omRvGq~|bgV?UaLak`a zP8!w@BWx!7fv&I__!R9}H`TPW(HAs@Ou(aUHU|LOIDNOB6~^fbh0erbDbY(l=Mrmx zVWVM#$kYYn1fmW<6HkK3OXEN!z8_oDpBfZ)Dz&AVk|2R(>^4a0&#fIDFI{`su8(5m z2~nPu3G3-C`MfUNY^g+m1r7(|VzbSpQ85Ont;IF}PJskLYbgr*u)n%O{)AduPq;+@yy>>I?Pou*z zFeEV?QMKqn=DZ05$o1b?$y$f~C&PJX`}EheUj(UqXyJ%7W;4I*_%j@Rm$LrAlx;`> zxB`^couCw{Ie(3BhuUo@2AH0qa<4*Obf>46vMSWk}5 zL)5Um=0#p8oC|Bx#k|CWeKt(TLcaS)K?4-6rTK~RA(eRW4XG|~3!4GYWLi>3``mS_ z#=I|`G&Hxhko}HOo|Fj-r*~h_lQZ2x>dC3}4$gN(tWY?9Tv>$@t|>o2=GyQ8;-U%C|MoriaR27&P{@pAccLX%_; z?8(Xkh|X*}<1XzlT)8m&XK%cR8ivrh0D;Pc#k=!)TDa|mMI{uVv=xd6W!ZjA5=Ej= zA^@g<9>|oZ`2$Rw7i&bG#3h`##4{RO+qh6q9Z1K@=t*O`7`^6XXyRFx1F+O#EgG)3 zx6NA!ce&CV z<2Y+Os^^bs>v}z8%n^TgesgYmCb;-Pr;lU>7v9i1s%6Xj%K9U805~13 z9lvg=wS1kk$1e3b^B3eT3m0tTom_Kqb5SOfP0R-iLNgY9&5bx7v#z9}-(?jwVM=mQlLkmGN4y|Oe*jA_@z$Jf#wW}#S~p{C-_ItajK^tt(j_dsDf0S~ ztu&n_+{_of2#8ov!s(1oBm6AgRjFp5PkGn%8&k`&D^^xk1^Qs><;%0p_yRMYoXTgr ziXucVzi+dDr9=yv(Jty{i3`rhfMZoV>A^V!g5mj~|Y_~P`p zZu(ty!-&5v`+o1=SLCy*BLOhkGMPwwpNGxIgHy2j8flE{)NQu{B-jE_6-zdC8XF-F zhD*0n`1(;9qvv=W;^|I_PaXZK7E+m3&8&kMv9~&fCU}FGYlWvX}1kG5pCc z;%B;uKi6Bj8IJUY9BB}{dhBvXj%zn_96ATB zP&k@g!(nJMD1-wV)}qPoTC_6z@}hfMFS_EMY)Fz-=(==i_Gq=WatieNCiKE-B^{Sy zDIaa>8ts&BEG4h^ANh>U+vmY+3vhWFKSvg}gI=fM@eP0I7`^$1gWuY5V$7vn<5OAa zZTi!MN&vDa;eIe76=P@+j-q+{onDj)m(4RbbXPsQ!*3)}aggZ>|$ zA)A9at4mk>PreDpJwW?WhWVuM{t;I;Pwi-XTFV=)^n+sd^ohtF3oI|Qo~Ok0+f>{ zx&Hi97Ui&xQn2Uzrq3G>b(%b{&&x{t1nV{00e_AC)&}sSjy6gc_t0;7(<%9+U^)5u z4>qT^vW{jsyIth2#>%#r?j7Fed#LFVNvEktwA3|{b68=NZ1kN%oxpWa1r+Ig$6(%f zEd3&zyV#NNi@6)G^85&ES3R=q-4J*DKBjbM5?#Ec^kYm$>)9w+QNkmzpMT@|{DRJ~ zXvmGFPXUzvm!PiKZQv5-6^uTZg5cS=zhB?t4S@^3c5rr>N3uWX*GI`Z911ZOYooFfF>5?swX}VbNbsDX&_d4xHjY?gc zYjxG|*XW{f4`j--AE_R?=nJ+@vCuCSv7t@ z^RjX0HP7JlSY$h)>CUiHI5j^V9r|SpfFNHEgFKq%p0TLu;i_!Af_Y>u1E9;P9-uDTo-{hOF+7TJ)yj z_Hz4f9nj~`(x|@WWJMl8dv`Y9t?dI zor=RO3xFUTgN`<*zOnNDxIyIy$2BD*PNXyO;AXK{$bAt3__}P3!baQ_c<$nfg~1(d z2i9yjwdyYhtMgM$uwwLi$|bXcmY2-VpH*KvKK4NQ(Sp_4Z)G*XY~{uy{p9n#m;4NN zob>12m*C(|!Ydy{uFE`bf_Bqc^tS%e+o2CKaMG%G+vMYVe!Y5PdpJFTYSG!HoDeiT#oRP42w8|k6RdC3KTE}^csAUQ6>6@8 z{%y{VvvGgmMZ7-qiH_1;EY?2-+4{PU1(^$c2AcT^ZVi8Uh(7i-xW#XNWFi?T7dK?c zZU=xZF4;W4_VjO67V`#BI!|`mRBvaxwzK(^ZLk~8%-FrzNejugCC%UJ{rpr&q z=qIUs7kA>mOpM}sSoUkq&c_Y}*I+{j1wNP*q;`LVuE`yY|!N!>{ymB<^LFfcF3rwEjdT3 zyYMRqE=QvlBEY!H1sVu{VK2anqJ`_WYbQxExJ2Q;L81$y5I!)WH7QH^c(Ik5DoM0fVswCKZd z|Dx^j%~)e*G&=c%0H#goMp(LBJ!u;{CWCTH{OfSw(ZGsMT>f^rap{}!9rb6#=b)2M z3~~{cjt?q6fY4n>hH>&xQf}ThdUW%nW5#sO#Q3V`>enFfC&2y#3^;6Kiw(tj{QOsk zob0&7*NqTl_`@C)g=IWYi8bVdzgyQY|Jl}HC7m_F?_YCb;M3I#c7`j1R^Ve`!Fp$Z z8u$iY1|9++(=R>?e%4VY#o~|CYv)w95-Mo3*)JNL8I{pe{%Da7{EipY1}aV_KUP6` zl23kY;zyR3f-I5^sqX4A+vN4DeY8VmMuWN?@WX9;H})BwffanxD$%uBs__8VtJ-o_ zP`7Lgg1rM0o?JgCS82Wpo$@Ah^D})%_P@@!HeD4&SJ08~TKP)-o9!`a*mVc5!8PhL2tIx71}FPRtA3iFj%b3|<4+gfTAU7G zc}!x$aDHKL-IPs_g$LV`IN%%nE@6Q05<)ctTtV&f2w?#WgBSmO@UzSZx9Xw5z@K!e zp$zd*j^&Z7sjKo6L`YA00u9(fS{pr)mj@saos*5cYoqE(Z-7Mmolen{4x6(@cHywK z{oXdw(Gu~PUEt^P*#8#gN_XuZd0OW$6GZ#EVWP{jbOJmud2$kO+El(84xb75z~nfd zq;Z{|ztqSd!1gpcE|$*Fd7N>xtGht&as*F#t<#c7WxARsiAw$OWz zLtsFn|9IKv{AXZ;a)FTJWxHjk2Se)jdCHPp*yHk&2VT?&a8iDP8Oy>gE z?YTE54fTBs+hKDf0<)Y8>&{sFFX2(|Tn)J6;B2(@UJp)bn-$EE?!nphNCrQFhnHUl z5Dr{YIuK~yr&@09qo1&Wml)&0N}$0;04xkZSnq&J0OQKZgH1rDJTLK3=K+aAX|rgm zvTDy+EpSnnx8K_2bdo>DoMsf7Ff|eO*a&$w^t77 z8~PRao?{~e5pz+Cj|Hrxhds@WEMx5>lA zqG6fkQ-0F0&gqmtnU@_XN2=9nz~*?w2LkSSPQ-%)9a}l|w6NdRd6~3dx=`-zp!=PS zRTX?oV9)5rOER#6Puh{_)+~K&9tbP=B-aZo7Uw4hl^jM|cpDHw<%7Wo5j#2a?)bPK zN?rYv{A`R?=32lYgu>tlLuWKkXm8W;;Kmnx0Ajtu`NztS1rHIdXo1>-tOF0W)xd}u z=Gj(k+=E_$k8)1b^Vrv=fQ~wW3gt5I?zVZKY0o;x&toO8AII{!(<=b=;K#kQLz>A; zS&pZgcQSoBbxEF$uAFwh@3|H=*Y6qIX9O<)o`KzLPL|r?0k}0wUBLq@7KiU65VrYW zZ3PzY!udc+gE|Vv51&wca`@*G!}Wy&P3p#_5p(((CyJo=(4jl5)UfG9ABud03=Bs0XYTRtHZA?t-I*j z((S`0CTAQ=8~D|mhK9OHkgW%15#YVbVdcMv%q(Elhoh(ScQ@nx#0tNkNLyCFeP6GQ zN9Xbs9AQ-fSP0{S+*9bXjy;m?^?)K>b}S&GA6Vu9jDFIlp6u()U*3l)A82f(b{KvX zr64F9=x?*d+oQeL??z~#F#WR0@p%3)c&!puChQs4*tBbW-=|=+-WMlNmX6N@bZwTp zhzFvu@=q{jp8_Wh7C#a|M1;v61W{4IWdL?Qp1!{)h)P?AO)kwFI=S>G7CuLA5MP<8 zRO%)`>aQGVoQA-Dv1!(UbB@eqHF(_W3mz;mgLUkqE3$LFdpD9w0YpL0I==zX2tWb_ zWKfXTn@Q)AV)?O&$R}372Ru;f~;jVSirwX4B;T_>9@%(_2r3J8**! zYqBR>EFgn3%VG4=z*&3om7gOgL>+AtSVVE$KEUD}78Cik#~*@&0D=I(Jb{IPfS7)keMR!g=7!R=GS;5Q+E%=?|MOz6{M^=` z)4FJK>5uRn{j4AtT!$vqr-5sM-Hf1nHi_a#-aIdUF~Bl3t($ql&4t03TKXA4;u8#F z=vtodRri@y_Z_;-@u0sxI4H@5{HhOwcI!L$)}IXy#eQqKhdb_n*GIE4kEQ3Z#N-1X zwra$Ky#X|8Ib>12&l`BC+yMzmC%F;(m zD8uxdOw>EDCY@;UUKf-v#3=U%J5TR3KWGYzsL4_%dVsFaQaASiUv7?br5i9mGZO(c zBn>d0T?Sc_Y{n`JkCvLtbJji4duwfT2j5a0gZfth5WN||22rD=z)2VCc`3+wFMb!Cyz)^T{u;R- zR+8MD-BVzb@;qM7efX{2 z^)L8${s!K1@N)>v{{l~rqa7>|_W?!?ze8~F;GsDj_wXcR7`NyjXl$;(<8Y&~!?{y{ z6#}rBpo6lHJhCzxj&KAp?4@q=#|E6V&+4q5IKY37l4LLWQ7v8vV4`Q9$=PADK`R={ z02dnR+b*6s*aZ$eiV*yC=ii# zk(E)l11G0XAcbPug0u^==2!CJ)4TfhUKorDODfI&d~6_H5pD8{89vFOcaSTF! z76SOs5Qx0AOThpE&u>;Qz#aE_0F-|PIP~>FTniw|*YFuQP)@S&>|Dd=79Higg#GUZ zUA2Xd@XwIGN-pEB{yZ)avtg5V$;eQK4*gx~<%oQ8zF51Z$G7dX`d;HeS{xD?zga zl&{8hyrO$==y)GUwb59_W!ocVcBB?)B`TbvpnV7D9LnKo!P<>5y6dR)TI%U$41l% z9S2sjXRg=XoF-VRfEwyM^Sn4~F{( z{Y?Siwj&g{0g?jU0^S3ajZQ$tjKIN~=K-V?l?S5ah=8Ssz;sCmD)LjXz|Vmm=dGLt zR3QK(2@Y)3#e-^jI?|#Hf)-Ya#f#bN0ZjZx zZhiVX#&`t#*&1CDuxda-BN~AQfr`2uI0$H51@Z(GOj6@P(l%1mv?LH4)lI=(~P!PJ#E9ftg4%MAOoy! za-urZOkCb!NnQghfrTGIrJw=r%HxkObMyigR)r9&EkA#RX5z5{(r6PMVGA4cQS#pAJF?GFogAH^ zYquf;tgd~2PKb4H-`jf$7MWXWli*P4fywXHO8`z9T@k2AgGvFH@^`B&`;&Q~lRc0D zRwp~SyNJge(X9X>e7T3CEP@j(SKb}hK0aV~<@9I+aBIoA2kRf{F73?h?A{*80IPeS zzYeqhop06sGKnfv0YI1lw@X3)?p^DV74GZVlkrioCQh0s9__W94l^j8Gd zEM*>O=>dH7^}L}6uE*`aNvIfw5!LTvzKUPMeh;Aaa*z)n-}v|Sd^j^pnFmg&2mT+k WV5hA3PnA6Y0000 + + + + + + + + + + + + + + diff --git a/web/src/assets/images/model/openai.svg b/web/src/assets/images/model/openai.svg new file mode 100644 index 00000000..70686f9b --- /dev/null +++ b/web/src/assets/images/model/openai.svg @@ -0,0 +1,4 @@ + + + + diff --git a/web/src/assets/images/model/xinference.svg b/web/src/assets/images/model/xinference.svg new file mode 100644 index 00000000..f5c5f75e --- /dev/null +++ b/web/src/assets/images/model/xinference.svg @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/web/src/components/Chat/ChatContent.tsx b/web/src/components/Chat/ChatContent.tsx index c90f9208..a5d02b2b 100644 --- a/web/src/components/Chat/ChatContent.tsx +++ b/web/src/components/Chat/ChatContent.tsx @@ -8,6 +8,7 @@ import { type FC, useRef, useEffect } from 'react' import clsx from 'clsx' import Markdown from '@/components/Markdown' import type { ChatContentProps } from './types' +import { Spin } from 'antd' /** * 聊天内容显示组件 @@ -21,7 +22,8 @@ const ChatContent: FC = ({ empty, labelPosition = 'bottom', labelFormat, - errorDesc + errorDesc, + renderRuntime }) => { // 滚动容器引用，用于控制自动滚动到底部 const scrollContainerRef = useRef<(HTMLDivElement | null)>(null) @@ -45,8 +47,8 @@ const ChatContent: FC = ({ 'rb:left-0 rb:text-left': item.role === 'assistant', // 助手消息左对齐 })}> {/* 流式加载时且内容为空则不显示 */} - {streamLoading && item.content === '' - ? null + {streamLoading && item.content === '' && !renderRuntime + ? : <> {/* 顶部标签（如时间戳、用户名等） */} {labelPosition === 'top' && @@ -55,16 +57,17 @@ const ChatContent: FC = ({ } {/* 消息气泡框 */} -

      + {item.subContent && renderRuntime && renderRuntime(item, index)} {/* 使用Markdown组件渲染消息内容 */} - +
      {/* 底部标签（如时间戳、用户名等） */} {labelPosition === 'bottom' && diff --git a/web/src/components/Chat/types.ts b/web/src/components/Chat/types.ts index 851a8ccc..264ce39c 100644 --- a/web/src/components/Chat/types.ts +++ b/web/src/components/Chat/types.ts @@ -19,7 +19,9 @@ export interface ChatItem { /** 消息内容 */ content?: string | null; /** 创建时间 */ - created_at?: number | string + created_at?: number | string; + status?: string; + subContent?: Record[] } /** @@ -81,4 +83,5 @@ export interface ChatContentProps { /** 标签格式化函数 */ labelFormat: (item: ChatItem) => any; errorDesc?: string; + renderRuntime?: (item: ChatItem, index: number) => ReactNode; } \ No newline at end of file diff --git a/web/src/components/Empty/PageEmpty.tsx b/web/src/components/Empty/PageEmpty.tsx new file mode 100644 index 00000000..17926fde --- /dev/null +++ b/web/src/components/Empty/PageEmpty.tsx @@ -0,0 +1,16 @@ +import { useTranslation } from 'react-i18next' +import pageEmptyIcon from '@/assets/images/empty/pageEmpty.png' +import Empty from './index' +const PageEmpty = ({ size = [240, 210] }: { size?: number | number[] }) => { + const { t } = useTranslation() + return ( + + ) +} +export default PageEmpty; \ No newline at end of file diff --git a/web/src/components/Markdown/CodeBlock.tsx b/web/src/components/Markdown/CodeBlock.tsx index 23d54c34..a125a997 100644 --- a/web/src/components/Markdown/CodeBlock.tsx +++ b/web/src/components/Markdown/CodeBlock.tsx @@ -6,6 +6,9 @@ import CopyBtn from './CopyBtn'; type ICodeBlockProps = { value: string; + needCopy?: boolean; + size?: 'small' | 'default'; + showLineNumbers?: boolean; } // enum languageType { @@ -16,6 +19,9 @@ type ICodeBlockProps = { const CodeBlock: FC = ({ value, + needCopy = true, + size = 'default', + showLineNumbers = false }) => { return ( @@ -23,24 +29,26 @@ const CodeBlock: FC = ({ {value} - + />} ) } diff --git a/web/src/components/PageTabs/index.module.css b/web/src/components/PageTabs/index.module.css new file mode 100644 index 00000000..6eab8a48 --- /dev/null +++ b/web/src/components/PageTabs/index.module.css @@ -0,0 +1,13 @@ +.page-tabs:global(.ant-segmented) { + background-color: rgba(91, 97, 103, 0.08); + padding: 4px; +} +.page-tabs:global(.ant-segmented .ant-segmented-item-label) { + line-height: 24px; + min-height: 24px; + padding: 0 12px; +} + +.page-tabs:global(.ant-segmented .ant-segmented-item-selected) { + box-shadow: 0px 2px 4px 0px rgba(33, 35, 50, 0.16); +} \ No newline at end of file diff --git a/web/src/components/PageTabs/index.tsx b/web/src/components/PageTabs/index.tsx new file mode 100644 index 00000000..33f02097 --- /dev/null +++ b/web/src/components/PageTabs/index.tsx @@ -0,0 +1,18 @@ +import { type FC } from 'react'; +import { Segmented, type SegmentedProps } from 'antd'; +import styles from './index.module.css'; + +const PageTabs: FC = ({ + value, + options, + onChange +}) => { + return ; +}; + +export default PageTabs; diff --git a/web/src/components/RbCard/Card.tsx b/web/src/components/RbCard/Card.tsx index f86b1c60..7ed81160 100644 --- a/web/src/components/RbCard/Card.tsx +++ b/web/src/components/RbCard/Card.tsx @@ -1,5 +1,5 @@ import { type FC, type ReactNode } from 'react' -import { Card } from 'antd'; +import { Card, Tooltip } from 'antd'; import clsx from 'clsx'; interface RbCardProps { @@ -9,7 +9,7 @@ interface RbCardProps { extra?: ReactNode; children?: ReactNode; avatar?: ReactNode; - avatarUrl?: string; + avatarUrl?: string | null; bodyPadding?: string; bodyClassName?: string; headerType?: 'border' | 'borderless' | 'borderBL' | 'borderL'; @@ -50,7 +50,7 @@ const RbCard: FC = ({ +
      {avatarUrl ? : avatar ? avatar : null @@ -59,11 +59,11 @@ const RbCard: FC = ({ clsx( { 'rb:max-w-full': !avatarUrl && !avatar, - 'rb:max-w-[calc(100%-60px)]': avatarUrl || avatar, + 'rb:max-w-[calc(100%-80px)]': avatarUrl || avatar, } ) }> -
      {title}
      +
      {title}
       {subTitle &&
      {subTitle}
      }
      : null diff --git a/web/src/components/Upload/UploadImages.tsx b/web/src/components/Upload/UploadImages.tsx index 2006ea09..0875707a 100644 --- a/web/src/components/Upload/UploadImages.tsx +++ b/web/src/components/Upload/UploadImages.tsx @@ -1,23 +1,23 @@ import { useState, useEffect, forwardRef, useImperativeHandle } from 'react'; -import { Upload, Modal, Image, App } from 'antd'; +import { Upload, Image, App } from 'antd'; import type { GetProp, UploadFile, UploadProps } from 'antd'; // import { UploadOutlined, } from '@ant-design/icons'; import type { UploadProps as RcUploadProps } from 'antd/es/upload/interface'; import { useTranslation } from 'react-i18next'; import PlusIcon from '@/assets/images/plus.svg' import { cookieUtils } from '@/utils/request' +import { fileUploadUrl } from '@/api/fileStorage' +import styles from './index.module.less' -const { confirm } = Modal; - -interface UploadImagesProps extends Omit { +interface UploadImagesProps extends Omit { /** 上传接口地址 */ action?: string; /** 是否支持多选 */ multiple?: boolean; /** 已上传的文件列表 */ - fileList?: UploadFile[]; + fileList?: UploadFile[] | UploadFile; /** 文件列表变化回调 */ - onChange?: (fileList: UploadFile[]) => void; + onChange?: (fileList?: UploadFile[] | UploadFile) => void; /** 禁用上传 */ disabled?: boolean; /** 文件大小限制（MB） */ @@ -28,6 +28,7 @@ interface UploadImagesProps extends Omit { isAutoUpload?: boolean; /** 最大上传文件数 */ maxCount?: number; + className?: string; } const ALL_FILE_TYPE: { [key: string]: string; @@ -59,7 +60,7 @@ const getBase64 = (file: FileType): Promise => { * 支持单文件/多文件上传、拖拽上传、文件验证、预览等功能 */ const UploadImages = forwardRef(({ - action = '/api/upload', + action = fileUploadUrl, multiple = false, fileList: propFileList = [], onChange, @@ -68,27 +69,42 @@ const UploadImages = forwardRef(({ fileType = ['png', 'jpg', 'gif'], isAutoUpload = true, maxCount = 1, + className = 'rb:size-24! rb:leading-1!', ...props }, ref) => { const { t } = useTranslation(); - const { message } = App.useApp() - const [fileList, setFileList] = useState(propFileList); + const { message, modal } = App.useApp() + const [fileList, setFileList] = useState([]); const [accept, setAccept] = useState(); // const [loading, setLoading] = useState(false); const [previewOpen, setPreviewOpen] = useState(false); const [previewImage, setPreviewImage] = useState(''); + useEffect(() => { + if (!Array.isArray(propFileList) && typeof propFileList === 'object') { + setFileList([propFileList]); + } + }, [propFileList]) + + const updateValue = (list: UploadFile[]) => { + if (maxCount === 1) { + onChange?.(list[0]) + } else { + onChange?.(list) + } + } + // 处理文件移除 const handleRemove = (file: UploadFile) => { - confirm({ - title: '确定要删除此文件吗？', - okText: '确定', + modal.confirm({ + title: t('common.confirmRemoveFile'), + okText: `${t('common.confirm')}`, okType: 'danger', - cancelText: '取消', + cancelText: `${t('common.cancel')}`, onOk: () => { const newFileList = fileList.filter((item) => item.uid !== file.uid); setFileList(newFileList); - onChange?.(newFileList); + updateValue(newFileList) }, }); return false; // 阻止默认删除行为，由confirm控制 @@ -100,7 +116,7 @@ const UploadImages = forwardRef(({ if (fileSize && file.size) { const isLtMaxSize = (file.size / 1024 / 1024) < fileSize; if (!isLtMaxSize) { - message.error(`文件大小不能超过 ${fileSize}MB`); + message.error(t('common.fileSizeTip', { size: fileSize })); return Upload.LIST_IGNORE; } } @@ -108,7 +124,7 @@ const UploadImages = forwardRef(({ if (accept && accept.length > 0 && file.type) { const isAccept = accept.includes(file.type); if (!isAccept) { - message.error(`不支持的文件类型: ${file.type}`); + message.error(`${t('common.fileAcceptTip')}${file.type}`); return Upload.LIST_IGNORE; } } @@ -119,7 +135,7 @@ const UploadImages = forwardRef(({ } const newFileList = [...fileList, file]; setFileList(newFileList); - onChange?.(newFileList); + updateValue(newFileList); return Upload.LIST_IGNORE; // 阻止自动上传 } @@ -129,17 +145,13 @@ const UploadImages = forwardRef(({ // 处理上传状态变化 const handleChange: UploadProps['onChange'] = ({ fileList: newFileList }) => { setFileList(newFileList); - if (onChange) { - onChange(newFileList); - } + updateValue(newFileList); }; // 清空已上传文件 const clearFiles = () => { setFileList([]); - if (onChange) { - onChange([]); - } + updateValue([]); } const handlePreview = async (file: UploadFile) => { @@ -167,7 +179,7 @@ const UploadImages = forwardRef(({ fileList, beforeUpload, headers: { - authorization: cookieUtils.get('authToken') || '', + authorization: `Bearer ${cookieUtils.get('authToken') }`, }, onPreview: handlePreview, onRemove: handleRemove, @@ -180,6 +192,7 @@ const UploadImages = forwardRef(({ showRemoveIcon: true, showDownloadIcon: false, }, + className: `${styles.imageUpload} ${className}`, ...props, }; @@ -193,16 +206,9 @@ const UploadImages = forwardRef(({ <> {fileList.length < maxCount && ( -
      - -
      {t('common.clickUploadIcon')}
      -
      + )}       {previewImage && ( diff --git a/web/src/components/Upload/index.module.less b/web/src/components/Upload/index.module.less new file mode 100644 index 00000000..a263d743 --- /dev/null +++ b/web/src/components/Upload/index.module.less @@ -0,0 +1,7 @@ +.image-upload:global(.ant-upload-wrapper.ant-upload-picture-card-wrapper .ant-upload-list.ant-upload-list-picture-card .ant-upload-list-item-container), +.image-upload:global(.ant-upload-wrapper.ant-upload-picture-circle-wrapper .ant-upload-list.ant-upload-list-picture-card .ant-upload-list-item-container), +.image-upload:global(.ant-upload-wrapper.ant-upload-picture-card-wrapper .ant-upload-list.ant-upload-list-picture-circle .ant-upload-list-item-container), +.image-upload:global(.ant-upload-wrapper.ant-upload-picture-circle-wrapper .ant-upload-list.ant-upload-list-picture-circle .ant-upload-list-item-container) { + width: 96px; + height: 96px; +} \ No newline at end of file diff --git a/web/src/i18n/en.ts b/web/src/i18n/en.ts index 1df2eb6d..5bbf20e2 100644 --- a/web/src/i18n/en.ts +++ b/web/src/i18n/en.ts @@ -419,6 +419,9 @@ export const en = { statusEnabled: 'Available', statusDisabled: 'Unavailable', remove: 'Remove', + + fileSizeTip: 'File size cannot exceed {{size}}MB', + fileAcceptTip: 'Unsupported file type:' }, model: { searchPlaceholder: 'search model…', @@ -510,6 +513,64 @@ export const en = { gpustack: "Gpustack", bedrock: "Bedrock" }, + modelNew: { + group: 'Model Group', + list: 'Model List', + square: 'Model Plaza', + createGroupModel: 'Create Model Group', + groupSearchPlaceholder: 'Search model groups', + listSearchPlaceholder: 'Search available models', + squareSearchPlaceholder: 'Search platform models', + status: 'Model Status', + created_at: 'Created At', + configureBtn: 'Click to Configure', + showModel: 'Show Model', + keyConfig: 'Configure KEY', + + modelConfiguration: 'Model Configuration', + logo: 'Model LOGO', + name: 'Model Name', + type: 'Model Type', + modelImplement: 'Model Implementation', + addImplement: 'Add Implementation', + noAuth: 'Unauthorized (Limited to 1 implementation)', + implementConfig: 'Configure Model Implementation', + provider: 'Model Provider', + api_key_ids: 'Select Model', + viewAll: 'More', + modelCount: 'Total {{count}} models', + modelList: 'Model List', + added: ' Added', + addSuccess: 'Added successfully', + model_name: 'Model Name', + tags: 'Tags', + createCustomModel: 'Add Custom Model', + edit: 'Edit', + selectOneTip: 'Model API KEY not configured, please configure it in the model list first', + load_balance_strategy: 'Concurrency Strategy', + round_robin: 'Sequential Execution - Call each model in order', + none: 'None', + + api_key: 'API KEY', + api_base: 'API Base URL', + description: 'Description', + add: 'Add', + item: 'item', + apiKeyNum: ' API Keys', + official: 'Official', + deprecated: 'Deprecated', + + llm: 'LLM', + chat: 'Chat', + embedding: 'Embedding', + rerank: 'Rerank', + openai: "Openai", + dashscope: "Dashscope", + ollama: "Ollama", + xinference: "Xinference", + gpustack: "Gpustack", + bedrock: "Bedrock" + }, knowledgeBase: { pleaseUploadFileFirst: 'Please upload file first', shareSuccess: 'Share successfully', @@ -866,7 +927,7 @@ export const en = { minimumRetention: 'Minimum retention (λ_time)', minimumRetentionDesc: 'Controls the minimum retention threshold of memory retention', - forgettingRate: 'Forgetting rate (λ_mem)', + forgettingRate: 'Forgetting rate (λ_mem)', forgettingRateDesc: 'Control the speed of memory forgetting, the higher the value, the faster the forgetting', offset: 'Offset (offset)', offsetDesc: 'The offset of the minimum preservation degree', @@ -934,7 +995,7 @@ export const en = { number: 'Number', checkbox: 'Checkbox', apiVariable: 'API Variable', - + displayName: 'Display Name', maxLength: 'Max Length', required: 'Required', @@ -1175,6 +1236,12 @@ export const en = { priority: 'Structured Integration', addTool: 'Add Tool', tool: 'Tool', + + statistics: 'Data Statistics', + daily_conversations: 'Daily Conversations', + daily_new_users: 'Daily New Users', + daily_api_calls: 'Daily API Calls', + daily_tokens: 'Token Consumption', }, userMemory: { userMemory: 'User Memory', @@ -1534,7 +1601,9 @@ Memory Bear: After the rebellion, regional warlordism intensified for several re noPermissionDesc: ' Please contact the administrator to grant permission', tableEmpty: 'No data available.', loadingEmpty: 'The content is loading…', - loadingEmptyDesc: 'Your content is on its way by rocket! It will soon land on your screen' + loadingEmptyDesc: 'Your content is on its way by rocket! It will soon land on your screen', + pageEmpty: 'Oops！ No search results available at the moment', + pageEmptyDesc: "Red Bear tilts its head and waits for you to change a new keyword, let's explore together.", }, apiKey: { name: 'Project Name', @@ -1765,7 +1834,7 @@ Memory Bear: After the rebellion, regional warlordism intensified for several re externalInteraction: 'External Interaction', "http-request": 'HTTP Request', tool: 'Tools', - code_execution: 'Code Execution', + code: 'Code Execution', "jinja-render": 'Template Rendering', cognitiveUpgrading: 'Cognitive Upgrading (Innovation)', 'memory-read': 'Memory Retrieval', @@ -1858,6 +1927,7 @@ Memory Bear: After the rebellion, regional warlordism intensified for several re 'array[number]': 'Array[Number]', 'array[boolean]': 'Array[Boolean]', 'array[object]': 'Array[Object]', + 'object': 'Object', addParams: 'Add Extract Variable', promptPlaceholder: 'Write prompts here, type "{" to insert variables, type "insert" to insert', }, @@ -1962,6 +2032,12 @@ Memory Bear: After the rebellion, regional warlordism intensified for several re config_id: 'Memory Configuration', search_switch: 'Search Mode', }, + + 'code': { + input_variables: 'Input Variables', + output_variables: 'Output Variables', + refreshTip: '同步函数签名至代码', + }, name: 'Key', type: 'Type', value: 'Value', @@ -1982,6 +2058,10 @@ Memory Bear: After the rebellion, regional warlordism intensified for several re arrange: 'Arrange', redo: 'Redo', undo: 'Undo', + + input: 'Input', + output: 'Output', + error: 'Error Message', }, emotionEngine: { emotionEngineConfig: 'Emotion Engine Configuration', diff --git a/web/src/i18n/zh.ts b/web/src/i18n/zh.ts index 39908757..70fd8c38 100644 --- a/web/src/i18n/zh.ts +++ b/web/src/i18n/zh.ts @@ -658,7 +658,13 @@ export const zh = { priority: '结构化整合', addTool: '添加工具', tool: '工具', - variableConfig: '配置变量' + variableConfig: '配置变量', + + statistics: '数据统计', + daily_conversations: '消息会话数', + daily_new_users: '新增用户数', + daily_api_calls: '调用次数', + daily_tokens: 'Token消耗', }, role: { roleManagement: '角色管理', @@ -967,6 +973,9 @@ export const zh = { statusEnabled: '可用', statusDisabled: '不可用', remove: '删除', + + fileSizeTip: '文件大小不能超过 {{size}}MB', + fileAcceptTip: '不支持的文件类型:' }, product: { applicationManagement: '应用管理', @@ -1076,6 +1085,64 @@ export const zh = { gpustack: "Gpustack", bedrock: "Bedrock" }, + modelNew: { + group: '模型组合', + list: '模型列表', + square: '模型广场', + createGroupModel: '创建模型组合', + groupSearchPlaceholder: '搜索模型组合', + listSearchPlaceholder: '搜索可用模型', + squareSearchPlaceholder: '搜索平台模型', + status: '模型状态', + created_at: '创建时间', + configureBtn: '点击配置', + showModel: '显示模型', + keyConfig: '配置 KEY', + + modelConfiguration: '模型配置', + logo: '模型LOGO', + name: '模型名称', + type: '模型类型', + modelImplement: '模型实现', + addImplement: '添加实现', + noAuth: '未授权(限1个实现)', + implementConfig: '配置模型实现', + provider: '模型供应商', + api_key_ids: '选择模型', + viewAll: '更多', + modelCount: '共 {{count}} 个模型', + modelList: '模型列表', + added: ' 已添加', + addSuccess: '添加成功', + model_name: '模型名称', + tags: '标签', + createCustomModel: '添加自定义模型', + edit: '编辑', + selectOneTip: '模型未配置API KEY，请先在模型列表配置', + load_balance_strategy: '并发策略', + round_robin: '顺序执行 - 按顺序依次调用每个模型', + none: '无', + + api_key: 'API KEY', + api_base: 'API Base URL', + description: '描述', + add: '添加', + item: '个', + apiKeyNum: '个 API Key', + official: '官方', + deprecated: '已弃用', + + llm: 'LLM', + chat: 'Chat', + embedding: 'Embedding', + rerank: 'Rerank', + openai: "Openai", + dashscope: "Dashscope", + ollama: "Ollama", + xinference: "Xinference", + gpustack: "Gpustack", + bedrock: "Bedrock" + }, timezones: { 'Asia/Shanghai': '中国标准时间 (UTC+8)', 'Asia/Kolkata': '印度标准时间 (UTC+5:30)', @@ -1607,13 +1674,10 @@ export const zh = { noPermissionDesc: '请联系管理员授予权限', tableEmpty: '目前没有数据', loadingEmpty: '内容正在加载中…', - loadingEmptyDesc: '您的内容正在火箭运输中！很快就会降落在您的屏幕上' + loadingEmptyDesc: '您的内容正在火箭运输中！很快就会降落在您的屏幕上', + pageEmpty: '哎呀！暂无搜索结果', + pageEmptyDesc: '红熊歪着头等待您更换新的关键词，让我们一起探索吧。', }, - count: '计数: {{count}}', - increment: '增加', - decrement: '减少', - reset: '重置', - switchLanguage: '切换语言', home: { title: '首页', @@ -1858,7 +1922,7 @@ export const zh = { externalInteraction: '外部交互', "http-request": 'HTTP请求', tool: '工具 (Tool)', - code_execution: '代码执行', + code: '代码执行', "jinja-render": '模板渲染', cognitiveUpgrading: '认知升级（创新）', 'memory-read': '记忆提取', @@ -1952,6 +2016,7 @@ export const zh = { 'array[number]': 'Array[Number]', 'array[boolean]': 'Array[Boolean]', 'array[object]': 'Array[Object]', + 'object': 'Object', addParams: '添加提取变量', promptPlaceholder: '在此处编写提示，输入“{”插入变量，输入“insert”插入', }, @@ -2056,6 +2121,12 @@ export const zh = { config_id: '记忆配置', search_switch: '检索模式', }, + + 'code': { + input_variables: '输入变量', + output_variables: '输出变量', + refreshTip: '同步函数签名至代码', + }, name: '键', type: '类型', value: '值', @@ -2076,6 +2147,10 @@ export const zh = { arrange: '整理', redo: '重做', undo: '撤销', + + input: '输入', + output: '输出', + error: '错误信息', }, emotionEngine: { emotionEngineConfig: '情感引擎配置', diff --git a/web/src/styles/antdThemeConfig.ts b/web/src/styles/antdThemeConfig.ts index db1166fb..1d281730 100644 --- a/web/src/styles/antdThemeConfig.ts +++ b/web/src/styles/antdThemeConfig.ts @@ -22,7 +22,7 @@ export const lightTheme: ThemeConfig = { // colorBgContainer: '#FBFDFF', colorError: '#FF5D34', sizeSM: 12, - fontSizeSM: 12, + fontSizeSM: 12, }, components: { Layout: { @@ -105,6 +105,9 @@ export const lightTheme: ThemeConfig = { }, Select: { lineHeightSM: 26 + }, + Upload: { + pictureCardSize: 96, } } }; \ No newline at end of file diff --git a/web/src/utils/request.ts b/web/src/utils/request.ts index 479fc1f3..e7112ded 100644 --- a/web/src/utils/request.ts +++ b/web/src/utils/request.ts @@ -23,9 +23,10 @@ interface data { } +export const API_PREFIX = '/api' // 创建axios实例 const service = axios.create({ - baseURL: '/api', // 与vite.config.ts中的代理配置对应 + baseURL: API_PREFIX, // 与vite.config.ts中的代理配置对应 // timeout: 10000, // 请求超时时间 withCredentials: false, headers: { @@ -126,7 +127,7 @@ service.interceptors.response.use( if (axios.isCancel(error) || error.name === 'AbortError' || error.code === 'ERR_CANCELED') { return Promise.reject(error); } - + // 处理网络错误、超时等 let msg = error.response?.data?.error || error.response?.error; const status = error?.response ? error.response.status : error; diff --git a/web/src/utils/stream.ts b/web/src/utils/stream.ts index e4179e25..be2220da 100644 --- a/web/src/utils/stream.ts +++ b/web/src/utils/stream.ts @@ -123,6 +123,20 @@ export const handleSSE = async (url: string, data: any, onMessage?: (data: SSEMe let response = await makeSSERequest(url, data, token || '', config); switch (response.status) { + case 500: + case 502: + const errorData = await response.json(); + errorData.error || i18n.t('common.serviceUpgrading'); + message.warning(errorData.error || i18n.t('common.serviceUpgrading')); + break + case 400: + const error = await response.json(); + message.warning(error.error); + throw error || 'Bad Request'; + case 504: + const errorJson = await response.json(); + message.warning(errorJson.error || i18n.t('common.serverError')); + break case 401: if (url?.includes('/public')) { return message.warning(i18n.t('common.publicApiCannotRefreshToken')); diff --git a/web/src/views/ApplicationConfig/Agent.tsx b/web/src/views/ApplicationConfig/Agent.tsx index 77e90440..0e9e8b44 100644 --- a/web/src/views/ApplicationConfig/Agent.tsx +++ b/web/src/views/ApplicationConfig/Agent.tsx @@ -20,7 +20,7 @@ import type { } from './types' import type { Variable } from './components/VariableList/types' import type { KnowledgeConfig } from './components/Knowledge/types' -import type { Model } from '@/views/ModelManagement/types' +import type { ModelListItem } from '@/views/ModelManagement/types' import { getModelList } from '@/api/models'; import { saveAgentConfig } from '@/api/application' import Knowledge from './components/Knowledge/Knowledge' @@ -96,8 +96,8 @@ const Agent = forwardRef((_props, ref) => { const [loading, setLoading] = useState(false) const [data, setData] = useState(null); const modelConfigModalRef = useRef(null) - const [modelList, setModelList] = useState([]) - const [defaultModel, setDefaultModel] = useState(null) + const [modelList, setModelList] = useState([]) + const [defaultModel, setDefaultModel] = useState(null) const [chatList, setChatList] = useState([]) const values = Form.useWatch([], form) const [isSave, setIsSave] = useState(false) @@ -126,12 +126,16 @@ const Agent = forwardRef((_props, ref) => { getApplicationConfig(id as string).then(res => { const response = res as Config let allTools = Array.isArray(response.tools) ? response.tools : [] + const memoryContent = response.memory?.memory_content + const parsedMemoryContent = memoryContent === null || memoryContent === '' + ? undefined + : !isNaN(Number(memoryContent)) ? Number(memoryContent) : memoryContent form.setFieldsValue({ ...response, tools: allTools, memory: { ...response.memory, - memory_content: response.memory?.memory_content ? Number(response.memory?.memory_content) : undefined + memory_content: parsedMemoryContent } }) setData({ @@ -212,7 +216,7 @@ const Agent = forwardRef((_props, ref) => { ...data.knowledge_retrieval, ...knowledgeRest, knowledge_bases: knowledge_bases.map(item => ({ - kb_id: item.id, + kb_id: item.kb_id || item.id, ...(item.config || {}) })) } as KnowledgeConfig : null, @@ -237,9 +241,9 @@ const Agent = forwardRef((_props, ref) => { }) } const getModels = () => { - getModelList({ type: 'llm,chat', pagesize: 100, page: 1 }) + getModelList({ type: 'llm,chat', pagesize: 100, page: 1, is_active: true }) .then(res => { - const response = res as { items: Model[] } + const response = res as { items: ModelListItem[] } setModelList(response.items) }) } @@ -249,7 +253,7 @@ const Agent = forwardRef((_props, ref) => { useEffect(() => { if (values?.default_model_config_id && modelList.length > 0) { const filterValue = modelList.find(item => item.id === values.default_model_config_id) - setDefaultModel(filterValue as Model | null) + setDefaultModel(filterValue as ModelListItem | null) setChatList([{ label: filterValue?.name || '', model_config_id: filterValue?.id || '', diff --git a/web/src/views/ApplicationConfig/Cluster.tsx b/web/src/views/ApplicationConfig/Cluster.tsx index 3081aa04..aa4a5d98 100644 --- a/web/src/views/ApplicationConfig/Cluster.tsx +++ b/web/src/views/ApplicationConfig/Cluster.tsx @@ -225,7 +225,7 @@ const Cluster = forwardRef((_props, ref) => { = { + daily_conversations: 'total_conversations', + daily_new_users: 'total_new_users', + daily_api_calls: 'total_api_calls', + daily_tokens: 'total_tokens', +} +const Statistics: FC<{ application: Application | null }> = ({ application }) => { + const [data, setData] = useState({ + daily_conversations: [], + total_conversations: 0, + daily_new_users: [], + total_new_users: 0, + daily_api_calls: [], + total_api_calls: 0, + daily_tokens: [], + total_tokens: 0 + }) + const [query, setQuery] = useState({ + start_date: dayjs().subtract(6, 'd'), + end_date: dayjs().subtract(0, 'd'), + }) + + useEffect(() => { + getData() + }, [application, query]) + const getData = () => { + if (!application?.id) { + return + } + const params = { + start_date: query.start_date.startOf('d').valueOf(), + end_date: query.end_date.endOf('d').valueOf(), + } + + getAppStatistics(application.id, params) + .then(res => { + setData(res as StatisticsData) + }) + } + const handleChange = (date: [Dayjs | null, Dayjs | null] | null) => { + if (!date || !date[0] || !date[1]) return + setQuery({ + start_date: date[0], + end_date: date[1], + }) + } + return ( +
      + + + + + + + {Object.entries(data).map(([key, value]) => { + if (key.includes('total')) { + return null + } + const totalKey = TotalObj[key]; + return ( + + + + ) + })} + +
      + ); +} +export default Statistics; \ No newline at end of file diff --git a/web/src/views/ApplicationConfig/components/AiPromptModal.tsx b/web/src/views/ApplicationConfig/components/AiPromptModal.tsx index b910e1b0..0c7bf480 100644 --- a/web/src/views/ApplicationConfig/components/AiPromptModal.tsx +++ b/web/src/views/ApplicationConfig/components/AiPromptModal.tsx @@ -181,7 +181,7 @@ const AiPromptModal = forwardRef(({ > = { edit: editIcon, copy: copyIcon, diff --git a/web/src/views/ApplicationConfig/components/Knowledge/KnowledgeConfigModal.tsx b/web/src/views/ApplicationConfig/components/Knowledge/KnowledgeConfigModal.tsx index abf56b18..70b17a11 100644 --- a/web/src/views/ApplicationConfig/components/Knowledge/KnowledgeConfigModal.tsx +++ b/web/src/views/ApplicationConfig/components/Knowledge/KnowledgeConfigModal.tsx @@ -66,7 +66,7 @@ const KnowledgeConfigModal = forwardRef { if (values?.retrieve_type) { const fieldsToReset = Object.keys(values).filter(key => - key !== 'kb_id' && key !== 'retrieve_type' + key !== 'kb_id' && key !== 'retrieve_type' && key !== 'top_k' ) as (keyof KnowledgeConfigForm)[]; form.resetFields(fieldsToReset); } diff --git a/web/src/views/ApplicationConfig/components/Knowledge/KnowledgeGlobalConfigModal.tsx b/web/src/views/ApplicationConfig/components/Knowledge/KnowledgeGlobalConfigModal.tsx index 2f349487..e4204836 100644 --- a/web/src/views/ApplicationConfig/components/Knowledge/KnowledgeGlobalConfigModal.tsx +++ b/web/src/views/ApplicationConfig/components/Knowledge/KnowledgeGlobalConfigModal.tsx @@ -97,7 +97,7 @@ const KnowledgeGlobalConfigModal = forwardRef = { + daily_conversations: '#FFB048', + daily_new_users: '#4DA8FF', + daily_api_calls: '#155EEF', + daily_tokens: '#AD88FF' +} + +const LineCard: FC = ({ chartData, type, total }) => { + const { t } = useTranslation() + const chartRef = useRef(null); + + useEffect(() => { + + }, [chartData]) + + const getSeries = () => { + return [{ + ...SeriesConfig, + name: t(`application.${type}`), + data: chartData.map(vo => vo.count), + areaStyle: { + opacity: 0.8, + color: new echarts.graphic.LinearGradient(0, 0, 0, 1, [ + { offset: 0, color: ColorObj[type] }, + { offset: 1, color: '#FFFFFF' } + ]) + }, + }] + } + + return ( + {t(`application.${type}`)} {total}} + > + {chartData && chartData.length > 0 ? ( + item.date), + boundaryGap: false, + }, + yAxis: { + type: 'value', + axisLabel: { + color: '#A8A9AA', + fontFamily: 'PingFangSC, PingFang SC', + align: 'right', + lineHeight: 17, + }, + axisLine: { + lineStyle: { + color: '#EBEBEB', + } + }, + }, + series: getSeries() + }} + style={{ height: '265px', width: '100%', minWidth: '100%', boxSizing: 'border-box' }} + opts={{ renderer: 'canvas' }} + notMerge={true} + lazyUpdate={true} + /> + ) : } + + ) +} + +export default LineCard diff --git a/web/src/views/ApplicationConfig/index.tsx b/web/src/views/ApplicationConfig/index.tsx index 7d5d5950..4dd9231a 100644 --- a/web/src/views/ApplicationConfig/index.tsx +++ b/web/src/views/ApplicationConfig/index.tsx @@ -9,6 +9,7 @@ import ReleasePage from './ReleasePage' import Cluster from './Cluster' import { getApplication } from '@/api/application' import Workflow from '@/views/Workflow'; +import Statistics from './Statistics' const ApplicationConfig: React.FC = () => { const { id } = useParams(); @@ -68,6 +69,7 @@ const ApplicationConfig: React.FC = () => { {activeTab === 'arrangement' && application?.type === 'workflow' && } {activeTab === 'api' && } {activeTab === 'release' && } + {activeTab === 'statistics' && } ); }; diff --git a/web/src/views/ApplicationConfig/types.ts b/web/src/views/ApplicationConfig/types.ts index 6f641ebb..9df6e04a 100644 --- a/web/src/views/ApplicationConfig/types.ts +++ b/web/src/views/ApplicationConfig/types.ts @@ -150,4 +150,19 @@ export interface AiPromptForm { } export interface ChatVariableConfigModalRef { handleOpen: (values: Variable[]) => void; +} + +export interface StatisticsItem { + count: number; + date: string; +} +export interface StatisticsData { + daily_conversations: StatisticsItem[]; + daily_new_users: StatisticsItem[]; + daily_api_calls: StatisticsItem[]; + daily_tokens: StatisticsItem[]; + total_conversations: number; + total_new_users: number; + total_api_calls: number; + total_tokens: number; } \ No newline at end of file diff --git a/web/src/views/EmotionEngine/index.tsx b/web/src/views/EmotionEngine/index.tsx index 73bfd376..6528bbbe 100644 --- a/web/src/views/EmotionEngine/index.tsx +++ b/web/src/views/EmotionEngine/index.tsx @@ -20,7 +20,7 @@ const configList = [ key: 'emotion_model_id', type: 'customSelect', url: getModelListUrl, - params: { type: 'chat,llm', page: 1, pagesize: 100 }, // chat,llm + params: { type: 'chat,llm', page: 1, pagesize: 100, is_active: true }, // chat,llm }, { key: 'emotion_min_intensity', diff --git a/web/src/views/MemberManagement/index.tsx b/web/src/views/MemberManagement/index.tsx index 8ce2fc62..68c90410 100644 --- a/web/src/views/MemberManagement/index.tsx +++ b/web/src/views/MemberManagement/index.tsx @@ -39,7 +39,7 @@ const MemberManagement: React.FC = () => { onOk: () => { deleteMember(member.id) .then(() => { - message.success(t('member.deleteSuccess')); + message.success(t('common.deleteSuccess')); refreshTable(); }) } @@ -93,7 +93,7 @@ const MemberManagement: React.FC = () => { return ( <> -
      +
      diff --git a/web/src/views/MemoryConversation/index.tsx b/web/src/views/MemoryConversation/index.tsx index 424b9878..66a66779 100644 --- a/web/src/views/MemoryConversation/index.tsx +++ b/web/src/views/MemoryConversation/index.tsx @@ -45,7 +45,7 @@ const searchSwitchList = [ ] export interface TestParams { - group_id: string; + end_user_id: string; message: string; search_switch: string; history: { role: string; content: string }[]; @@ -107,7 +107,7 @@ const MemoryConversation: FC = () => { setLoading(true) readService({ message: msg, - group_id: userId, + end_user_id: userId, search_switch: search_switch, history: [], }) @@ -204,7 +204,7 @@ const MemoryConversation: FC = () => { } )} > -
      {log.title}
      +
      {log.title}
      {log.type === 'problem_split' && Array.isArray(log.data) && log.data.length > 0 ? {log.data.map(vo => ( diff --git a/web/src/views/MemoryExtractionEngine/constant.ts b/web/src/views/MemoryExtractionEngine/constant.ts index d1b7b757..5939a1bc 100644 --- a/web/src/views/MemoryExtractionEngine/constant.ts +++ b/web/src/views/MemoryExtractionEngine/constant.ts @@ -1093,606 +1093,4 @@ export const groupDataByType = (data: any[], groupKey: string) => { }) return grouped -} - -export const mockTestResult = { - "generated_at": "2025-12-12T09:48:43.389893", - "entities": { - "extracted_count": 148 - }, - "dedup": { - "total_merged_count": 39, - "breakdown": { - "exact": 30, - "fuzzy": 0, - "llm": 9 - }, - "impact": [ - { - "name": "记忆熊", - "type": "Person", - "appear_count": 9, - "merge_count": 8 - }, - { - "name": "宋朝", - "type": "Organization", - "appear_count": 5, - "merge_count": 2 - }, - { - "name": "军费", - "type": "EconomicMetric", - "appear_count": 2, - "merge_count": 1 - }, - { - "name": "学生", - "type": "Person", - "appear_count": 6, - "merge_count": 5 - }, - { - "name": "废除丞相制度", - "type": "Event", - "appear_count": 6, - "merge_count": 3 - }, - { - "name": "六部", - "type": "Organization", - "appear_count": 4, - "merge_count": 3 - }, - { - "name": "六部缺乏协调机制", - "type": "Concept", - "appear_count": 2, - "merge_count": 1 - }, - { - "name": "丞相", - "type": "Position", - "appear_count": 4, - "merge_count": 1 - }, - { - "name": "总理", - "type": "Position", - "appear_count": 2, - "merge_count": 1 - }, - { - "name": "各部委", - "type": "Organization", - "appear_count": 2, - "merge_count": 1 - }, - { - "name": "六部直接对皇帝负责", - "type": "AdministrativeStructure", - "appear_count": 2, - "merge_count": 1 - }, - { - "name": "秦国", - "type": "Organization", - "appear_count": 5, - "merge_count": 2 - }, - { - "name": "文官集团", - "type": "Organization", - "appear_count": 2, - "merge_count": 1 - } - ] - }, - "disambiguation": { - "block_count": 1, - "effects": [ - { - "left": { - "name": "节度使", - "type": "Role" - }, - "right": { - "name": "节度使", - "type": "Person" - }, - "result": "成功区分" - } - ] - }, - "memory": { - "chunks": 2 - }, - "triplets": { - "count": 88 - }, - "core_entities": [ - { - "type": "Organization", - "type_cn": "组织", - "count": 16, - "entities": [ - "厂卫机构", - "西厂", - "东厂", - "工部", - "地方军阀" - ] - }, - { - "type": "Event", - "type_cn": "事件", - "count": 12, - "entities": [ - "均田制瓦解", - "无法批阅完所有政务", - "废除丞相制度", - "持续战争", - "政令执行困难" - ] - }, - { - "type": "Condition", - "type_cn": "Condition", - "count": 9, - "entities": [ - "缺乏协作机制", - "作战效率低下", - "厢军装备不足", - "军权分散", - "军事专业化难以提升" - ] - }, - { - "type": "Person", - "type_cn": "人物", - "count": 8, - "entities": [ - "官员", - "宦官", - "节度使", - "皇帝", - "文士" - ] - }, - { - "type": "Concept", - "type_cn": "Concept", - "count": 8, - "entities": [ - "行政紧张", - "军力不足", - "秦国统一六国的原因", - "六部缺乏协调机制", - "专业分工" - ] - }, - { - "type": "Action", - "type_cn": "Action", - "count": 6, - "entities": [ - "再花钱募兵", - "建立军功爵制度", - "裁撤兵员", - "削减装备", - "建立法律制度" - ] - }, - { - "type": "Outcome", - "type_cn": "Outcome", - "count": 5, - "entities": [ - "打仗更吃亏", - "提升国家组织能力", - "降低行政效率", - "士兵效忠个人而非国家", - "政令推行困难" - ] - }, - { - "type": "EconomicMetric", - "type_cn": "EconomicMetric", - "count": 4, - "entities": [ - "财政", - "财政支出", - "支出", - "军费" - ] - }, - { - "type": "Statement", - "type_cn": "Statement", - "count": 3, - "entities": [ - "没有银子", - "禁军由文官控制导致作战效率低下", - "武器没材料" - ] - }, - { - "type": "State", - "type_cn": "State", - "count": 3, - "entities": [ - "军队更弱", - "理解不足", - "不足" - ] - }, - { - "type": "HistoricalPeriod", - "type_cn": "HistoricalPeriod", - "count": 3, - "entities": [ - "春秋战国史", - "唐朝史", - "宋朝" - ] - }, - { - "type": "Attribute", - "type_cn": "Attribute", - "count": 3, - "entities": [ - "资源丰富", - "易守难攻", - "政策连续性强" - ] - }, - { - "type": "Right", - "type_cn": "Right", - "count": 3, - "entities": [ - "军事指挥权", - "财政调度权", - "募兵权" - ] - }, - { - "type": "Policy", - "type_cn": "Policy", - "count": 2, - "entities": [ - "商鞅变法", - "禁军由文官控制" - ] - }, - { - "type": "MilitaryCondition", - "type_cn": "MilitaryCondition", - "count": 2, - "entities": [ - "军力不足", - "缺乏战略纵深" - ] - }, - { - "type": "Role", - "type_cn": "Role", - "count": 2, - "entities": [ - "节度使", - "协调中枢" - ] - }, - { - "type": "Position", - "type_cn": "Position", - "count": 2, - "entities": [ - "总理", - "丞相" - ] - }, - { - "type": "PoliticalCharacteristic", - "type_cn": "PoliticalCharacteristic", - "count": 2, - "entities": [ - "旧贵族势力弱", - "中央集权程度高" - ] - }, - { - "type": "Phenomenon", - "type_cn": "Phenomenon", - "count": 1, - "entities": [ - "宋朝军事弱势" - ] - }, - { - "type": "Factor", - "type_cn": "Factor", - "count": 1, - "entities": [ - "制度性因素" - ] - }, - { - "type": "EconomicFactor", - "type_cn": "EconomicFactor", - "count": 1, - "entities": [ - "财政压力" - ] - }, - { - "type": "EconomicIndicator", - "type_cn": "EconomicIndicator", - "count": 1, - "entities": [ - "财政支出" - ] - }, - { - "type": "MilitaryStrategy", - "type_cn": "MilitaryStrategy", - "count": 1, - "entities": [ - "对外战略被动" - ] - }, - { - "type": "MilitaryCapability", - "type_cn": "MilitaryCapability", - "count": 1, - "entities": [ - "机动能力弱" - ] - }, - { - "type": "PersonGroup", - "type_cn": "PersonGroup", - "count": 1, - "entities": [ - "武将" - ] - }, - { - "type": "EconomicCondition", - "type_cn": "EconomicCondition", - "count": 1, - "entities": [ - "财政压力" - ] - }, - { - "type": "InstitutionalPolicy", - "type_cn": "InstitutionalPolicy", - "count": 1, - "entities": [ - "废除丞相制度" - ] - }, - { - "type": "StateOfAffairs", - "type_cn": "StateOfAffairs", - "count": 1, - "entities": [ - "中央决策高度集中于皇帝" - ] - }, - { - "type": "Institution", - "type_cn": "Institution", - "count": 1, - "entities": [ - "科举" - ] - }, - { - "type": "Function", - "type_cn": "Function", - "count": 1, - "entities": [ - "统筹大事小情" - ] - }, - { - "type": "AdministrativeStructure", - "type_cn": "AdministrativeStructure", - "count": 1, - "entities": [ - "六部直接对皇帝负责" - ] - }, - { - "type": "AdministrativeProblem", - "type_cn": "AdministrativeProblem", - "count": 1, - "entities": [ - "皇帝一人批不完政务" - ] - }, - { - "type": "Behavior", - "type_cn": "Behavior", - "count": 1, - "entities": [ - "互相推诿责任" - ] - }, - { - "type": "Resource", - "type_cn": "Resource", - "count": 1, - "entities": [ - "银子" - ] - }, - { - "type": "Situation", - "type_cn": "Situation", - "count": 1, - "entities": [ - "没人拍板" - ] - }, - { - "type": "HistoricalState", - "type_cn": "HistoricalState", - "count": 1, - "entities": [ - "秦国" - ] - }, - { - "type": "Location", - "type_cn": "地点", - "count": 1, - "entities": [ - "关中" - ] - }, - { - "type": "HistoricalEvent", - "type_cn": "HistoricalEvent", - "count": 1, - "entities": [ - "安史之乱" - ] - }, - { - "type": "PoliticalAction", - "type_cn": "PoliticalAction", - "count": 1, - "entities": [ - "中央整顿" - ] - }, - { - "type": "PoliticalPhenomenon", - "type_cn": "PoliticalPhenomenon", - "count": 1, - "entities": [ - "藩镇割据加剧" - ] - }, - { - "type": "EconomicEntity", - "type_cn": "EconomicEntity", - "count": 1, - "entities": [ - "中央财政" - ] - }, - { - "type": "System", - "type_cn": "System", - "count": 1, - "entities": [ - "募兵制" - ] - }, - { - "type": "WorkRole", - "type_cn": "WorkRole", - "count": 1, - "entities": [ - "掌控禁军" - ] - } - ], - "triplet_samples": [ - { - "subject": "记忆熊", - "predicate": "MENTIONS", - "predicate_cn": "提到", - "object": "宋朝军事弱势" - }, - { - "subject": "宋朝军事弱势", - "predicate": "RESULTED_IN", - "predicate_cn": "resulted in", - "object": "制度性因素" - }, - { - "subject": "记忆熊", - "predicate": "MENTIONS", - "predicate_cn": "提到", - "object": "禁军由文官控制导致作战效率低下" - }, - { - "subject": "禁军由文官控制", - "predicate": "RESULTED_IN", - "predicate_cn": "resulted in", - "object": "作战效率低下" - }, - { - "subject": "记忆熊", - "predicate": "MENTIONS", - "predicate_cn": "提到", - "object": "厢军装备不足" - }, - { - "subject": "记忆熊", - "predicate": "MENTIONS", - "predicate_cn": "提到", - "object": "宋朝" - }, - { - "subject": "记忆熊", - "predicate": "MENTIONS", - "predicate_cn": "提到", - "object": "军费" - } - ], - "self_reflexion": [ - { - "conflict": { - "data": [ - { - "id": "76be6d82d8804beda6baa3d3447d6cbc", - "statement": "学生对\"六部缺乏协调机制\"的具体影响表示理解不足。", - "group_id": "group_123", - "chunk_id": "4a0804127d35456f86d4f06e1fa458f7", - "created_at": "2025-12-12 09:48:00.166068", - "expired_at": null, - "valid_at": null, - "invalid_at": null, - "entity_ids": [] - } - ], - "conflict": true, - "conflict_memory": { - "id": "e268a6fff35543fab471986c188e023e", - "statement": "学生对\"六部缺乏协调机制\"的具体影响表示理解不足。", - "group_id": "group_123", - "chunk_id": "e6cb5f56020e4a8d925d148e1d2fbda0", - "created_at": "2025-12-12 09:48:00.166068", - "expired_at": null, - "valid_at": null, - "invalid_at": null, - "entity_ids": [] - } - }, - "reflexion": { - "reason": "同一学生在不同时间点重复提出对'六部缺乏协调机制'具体影响的理解困难，表明原有解释未能有效解决其认知障碍，存在记忆冗余与教学反馈失效的冲突。", - "solution": "保留后出现的记忆记录（chunk_id为4a0804127d35456f86d4f06e1fa458f7）作为最新学习状态，将其设为有效；将前次相同内容的记忆（id为e268a6fff35543fab471986c188e023e）标记为失效，避免重复干预，并基于后续完整解释优化知识呈现逻辑。" - }, - "resolved": { - "original_memory_id": "e268a6fff35543fab471986c188e023e", - "resolved_memory": { - "id": "e268a6fff35543fab471986c188e023e", - "statement": "学生对\"六部缺乏协调机制\"的具体影响表示理解不足。", - "group_id": "group_123", - "chunk_id": "e6cb5f56020e4a8d925d148e1d2fbda0", - "created_at": "2025-12-12 09:48:00.166068", - "expired_at": null, - "valid_at": null, - "invalid_at": "2025-12-12 09:48:00.166068", - "entity_ids": [] - } - } - } - ] - } \ No newline at end of file +} \ No newline at end of file diff --git a/web/src/views/MemoryExtractionEngine/index.tsx b/web/src/views/MemoryExtractionEngine/index.tsx index 3d67270c..96138a55 100644 --- a/web/src/views/MemoryExtractionEngine/index.tsx +++ b/web/src/views/MemoryExtractionEngine/index.tsx @@ -1,14 +1,14 @@ import { type FC, useState, useEffect } from 'react' import { useTranslation } from 'react-i18next' import { useParams } from 'react-router-dom' -import { Row, Col, Space, Switch, Select, InputNumber, Slider, App, Form } from 'antd' +import { Row, Col, Space, Select, InputNumber, Slider, App, Form } from 'antd' import clsx from 'clsx' import Card from './components/Card' import type { ConfigForm, Variable } from './types' import { getMemoryExtractionConfig, updateMemoryExtractionConfig } from '@/api/memory' import Markdown from '@/components/Markdown' import { getModelList } from '@/api/models'; -import type { Model } from '@/views/ModelManagement/types' +import type { ModelListItem } from '@/views/ModelManagement/types' import { configList } from './constant' import Result from './components/Result' import SwitchFormItem from '@/components/FormItem/SwitchFormItem' @@ -43,7 +43,7 @@ const MemoryExtractionEngine: FC = () => { const values = Form.useWatch([], form) const [loading, setLoading] = useState(false) const [iterationPeriodDisabled, setIterationPeriodDisabled] = useState(false) - const [modelList, setModelList] = useState([]) + const [modelList, setModelList] = useState([]) useEffect(() => { if (values?.reflexion_range === 'database') { @@ -55,9 +55,9 @@ const MemoryExtractionEngine: FC = () => { }, [values]) const getModels = () => { - getModelList({ type: 'llm,chat', pagesize: 100, page: 1 }) + getModelList({ type: 'llm,chat', pagesize: 100, page: 1, is_active: true }) .then(res => { - const response = res as { items: Model[] } + const response = res as { items: ModelListItem[] } setModelList(response.items) }) } diff --git a/web/src/views/MemoryManagement/types.ts b/web/src/views/MemoryManagement/types.ts index f926c6c8..55524462 100644 --- a/web/src/views/MemoryManagement/types.ts +++ b/web/src/views/MemoryManagement/types.ts @@ -23,7 +23,6 @@ export interface Memory { include_dialogue_context: boolean; max_context: string; lambda_mem: string; - lambda_mem: string; offset: string; state: boolean; created_at: string; diff --git a/web/src/views/ModelManagement/Group.tsx b/web/src/views/ModelManagement/Group.tsx new file mode 100644 index 00000000..398bd60b --- /dev/null +++ b/web/src/views/ModelManagement/Group.tsx @@ -0,0 +1,92 @@ +import { useState, useEffect, forwardRef, useImperativeHandle } from 'react'; +import clsx from 'clsx' +import { Button } from 'antd' +import { useTranslation } from 'react-i18next'; + +import type { ProviderModelItem, ModelListItem, DescriptionItem, BaseRef } from './types' +import RbCard from '@/components/RbCard/Card' +import { getModelNewList } from '@/api/models' +import PageEmpty from '@/components/Empty/PageEmpty'; +import { formatDateTime } from '@/utils/format'; + +const Group = forwardRef void; }>(({ query, handleEdit }, ref) => { + const { t } = useTranslation(); + const [list, setList] = useState([]) + useEffect(() => { + getList() + }, [query]) + const getList = () => { + getModelNewList({ + ...query, + is_composite: true, + is_active: true, + }) + .then(res => { + const response = res as ProviderModelItem[] + setList(response[0]?.models || []) + }) + } + const formatData = (data: ModelListItem) => { + return [ + { + key: 'type', + label: t(`modelNew.type`), + children: data.type ? t(`modelNew.${data.type}`) : '-', + }, + { + key: 'is_active', + label: t(`modelNew.status`), + children: data.is_active ? t(`common.statusEnabled`) : t(`common.statusDisabled`), + }, + { + key: 'created_at', + label: t(`modelNew.created_at`), + children: data.created_at ? formatDateTime(data.created_at, 'YYYY-MM-DD HH:mm:ss') : '-', + }, + ] + } + + useImperativeHandle(ref, () => ({ + getList, + })); + + return ( + <> + {list.length === 0 + ? + :( +
      + {list.map(item => ( + + {item.name[0]} +
      + } + > + {formatData(item)?.map((description: DescriptionItem) => ( +
      + {(description.label as string)} + {(description.children as string)} +
      + ))} + + + ))} +
      + ) + } + + ) +}) + +export default Group \ No newline at end of file diff --git a/web/src/views/ModelManagement/List.tsx b/web/src/views/ModelManagement/List.tsx new file mode 100644 index 00000000..bb799752 --- /dev/null +++ b/web/src/views/ModelManagement/List.tsx @@ -0,0 +1,86 @@ +import { useRef, useState, useEffect, type FC } from 'react'; +import { Button, Flex, Row, Col } from 'antd' +import { useTranslation } from 'react-i18next'; + +import type { ProviderModelItem, KeyConfigModalRef, ModelListDetailRef } from './types' +import RbCard from '@/components/RbCard/Card' +import { getModelNewList } from '@/api/models' +import PageEmpty from '@/components/Empty/PageEmpty'; +import Tag from '@/components/Tag'; +import KeyConfigModal from './components/KeyConfigModal' +import ModelListDetail from './components/ModelListDetail' +import { getLogoUrl } from './utils' + +const ModelList: FC<{ query: any }> = ({ query }) => { + const { t } = useTranslation(); + const keyConfigModalRef = useRef(null) + const modelListDetailRef = useRef(null) + const [list, setList] = useState([]) + useEffect(() => { + getList() + }, [query]) + const getList = () => { + getModelNewList({ + ...query, + is_composite: false, + }) + .then(res => { + setList((res || []) as ProviderModelItem[]) + }) + } + + const handleShowModel = (vo: ProviderModelItem) => { + modelListDetailRef.current?.handleOpen(vo) + } + const handleKeyConfig = (vo: ProviderModelItem) => { + keyConfigModalRef.current?.handleOpen(vo) + } + + return ( + <> + {list.length === 0 + ? + :( +
      + {list.map(item => ( + + {item.provider[0].toUpperCase()} +
      + } + bodyClassName="rb:relative rb:pb-[64px]! rb:h-[calc(100%-64px)]!" + > + {item.tags.map(tag => {t(`modelNew.${tag}`)})} +
      + + + + + + + + +
      + + ))} +
      + ) + } + + + + + ) +} + +export default ModelList \ No newline at end of file diff --git a/web/src/views/ModelManagement/Square.tsx b/web/src/views/ModelManagement/Square.tsx new file mode 100644 index 00000000..8eb67eef --- /dev/null +++ b/web/src/views/ModelManagement/Square.tsx @@ -0,0 +1,104 @@ +import { useRef, useState, useEffect, forwardRef, useImperativeHandle } from 'react'; +import { Button, Space, App, Divider, Flex, Tooltip } from 'antd' +import { UsergroupAddOutlined } from '@ant-design/icons'; +import { useTranslation } from 'react-i18next'; + +import type { ModelPlaza, ModelPlazaItem, ModelSquareDetailRef, BaseRef } from './types' +import RbCard from '@/components/RbCard/Card' +import { getModelPlaza, addModelPlaza } from '@/api/models' +import PageEmpty from '@/components/Empty/PageEmpty'; +import Tag from '@/components/Tag'; +import ModelSquareDetail from './components/ModelSquareDetail' +import { getLogoUrl } from './utils' + +const ModelSquare = forwardRef void; }>(({ query, handleEdit }, ref) => { + const { t } = useTranslation(); + const { message } = App.useApp() + const modelSquareDetailRef = useRef(null) + const [list, setList] = useState([]) + useEffect(() => { + getList() + }, [query]) + const getList = () => { + getModelPlaza(query) + .then(res => { + setList((res as ModelPlaza[]) || []) + }) + } + + const handleMore = (vo: ModelPlaza) => { + modelSquareDetailRef.current?.handleOpen(vo) + } + const handleAdd = (item: ModelPlazaItem) => { + addModelPlaza(item.id) + .then(() => { + message.success(`${item.name}${t('modelNew.addSuccess')}`) + getList() + }) + } + + useImperativeHandle(ref, () => ({ + getList, + })); + return ( + <> + {list.length === 0 + ? + : list.map(vo => ( +
      +
      +
      {t(`modelNew.${vo.provider}`)}
      + +
      + +
      + {vo.models.slice(0, 6).map(item => ( + + {t(`modelNew.${item.type}`)} + {item.is_official && {t(`modelNew.official`)}} + } + avatarUrl={getLogoUrl(item.logo)} + avatar={ +
      + {item.name[0]} +
      + } + bodyClassName="rb:relative rb:pb-[80px]! rb:h-[calc(100%-64px)]!" + > + +
      {item.description}
      +       + {item.tags.map((tag, tagIndex) => {tag})} +
      + + + {item.add_count} + + {!item.is_official && } + {item.is_added + ? + : + } + + +
      +       + ))} +
      +
      + )) + } + + + + ) +}) + +export default ModelSquare \ No newline at end of file diff --git a/web/src/views/ModelManagement/components/ConfigModal.tsx b/web/src/views/ModelManagement/components/ConfigModal.tsx deleted file mode 100644 index e4bdf84c..00000000 --- a/web/src/views/ModelManagement/components/ConfigModal.tsx +++ /dev/null @@ -1,171 +0,0 @@ -import { forwardRef, useImperativeHandle, useState } from 'react'; -import { Form, Input, App } from 'antd'; -import { useTranslation } from 'react-i18next'; -import type { ModelFormData, Model, ConfigModalRef, ConfigModalProps } from '../types'; -import RbModal from '@/components/RbModal' -import CustomSelect from '@/components/CustomSelect' -import { updateModel, addModel, modelTypeUrl, modelProviderUrl } from '@/api/models' - -const ConfigModal = forwardRef(({ - refresh -}, ref) => { - const { t } = useTranslation(); - const { message } = App.useApp(); - const [visible, setVisible] = useState(false); - const [model, setModel] = useState({} as Model); - const [isEdit, setIsEdit] = useState(false); - const [form] = Form.useForm(); - const [loading, setLoading] = useState(false) - - const values = Form.useWatch([], form); - - // 封装取消方法，添加关闭弹窗逻辑 - const handleClose = () => { - setModel({} as Model); - form.resetFields(); - setLoading(false) - setVisible(false); - }; - - const handleOpen = (model?: Model) => { - if (model) { - setIsEdit(true); - setModel(model); - // 设置表单值 - const apiKeyInfo = model.api_keys[0] - form.setFieldsValue({ - provider: apiKeyInfo.provider, - model_name: apiKeyInfo.model_name, - api_key: apiKeyInfo.api_key, - api_base: apiKeyInfo.api_base - }); - } else { - setIsEdit(false); - form.resetFields(); - } - setVisible(true); - }; - // 封装保存方法，添加提交逻辑 - const handleSave = () => { - form - .validateFields() - .then(() => { - const data = { - name: values.name, - type: values.type, - api_keys: { - provider: values.provider, - model_name: values.model_name, - api_key: values.api_key, - api_base: values.api_base - }, - } - setLoading(true) - const res = isEdit - ? updateModel(model.api_keys[0].id, { - provider: values.provider, - model_name: values.model_name, - api_key: values.api_key, - api_base: values.api_base - } as ModelFormData) - : addModel(data as ModelFormData) - - res.then(() => { - if (refresh) { - refresh(); - } - handleClose() - message.success(isEdit ? t('common.updateSuccess') : t('common.createSuccess')) - }) - .catch(() => { - setLoading(false) - }); - }) - .catch((err) => { - console.log('err', err) - }); - } - - // 暴露给父组件的方法 - useImperativeHandle(ref, () => ({ - handleOpen, - handleClose - })); - - return ( - -
      - {!isEdit && ( - <> - - - - - items.map((item) => ({ label: t(`model.${item}`), value: item }))} - /> - - - )} - - - - items.map((item) => ({ label: t(`model.${item}`), value: item }))} - /> - - - - - - - - - - - - -
      -       - ); -}); - -export default ConfigModal; \ No newline at end of file diff --git a/web/src/views/ModelManagement/components/CustomModelModal.tsx b/web/src/views/ModelManagement/components/CustomModelModal.tsx new file mode 100644 index 00000000..66c16111 --- /dev/null +++ b/web/src/views/ModelManagement/components/CustomModelModal.tsx @@ -0,0 +1,168 @@ +import { forwardRef, useImperativeHandle, useState } from 'react'; +import { Form, Input, App, Select } from 'antd'; +import { useTranslation } from 'react-i18next'; + +import type { CustomModelForm, ModelPlazaItem, CustomModelModalRef, CustomModelModalProps } from '../types'; +import RbModal from '@/components/RbModal' +import CustomSelect from '@/components/CustomSelect' +import UploadImages from '@/components/Upload/UploadImages' +import { updateCustomModel, addCustomModel, modelTypeUrl, modelProviderUrl } from '@/api/models' +import { getFileLink } from '@/api/fileStorage' + +const CustomModelModal = forwardRef(({ + refresh +}, ref) => { + const { t } = useTranslation(); + const { message } = App.useApp(); + const [visible, setVisible] = useState(false); + const [model, setModel] = useState({} as ModelPlazaItem); + const [isEdit, setIsEdit] = useState(false); + const [form] = Form.useForm(); + const [loading, setLoading] = useState(false) + const formValues = Form.useWatch([], form) + + const handleClose = () => { + setModel({} as ModelPlazaItem); + form.resetFields(); + setLoading(false) + setVisible(false); + }; + + const handleOpen = (model?: ModelPlazaItem) => { + if (model) { + setIsEdit(true); + setModel(model); + form.setFieldsValue({ + ...model, + logo: model.logo ? { url: model.logo, uid: model.logo, status: 'done', name: 'logo' } : undefined + }); + } else { + setIsEdit(false); + form.resetFields(); + } + setVisible(true); + }; + const handleUpdate = (data: CustomModelForm) => { + setLoading(true) + const { type, provider, ...rest} = data + const res = isEdit ? updateCustomModel(model.id, rest) : addCustomModel(data) + + res.then(() => { + refresh && refresh() + handleClose() + message.success(isEdit ? t('common.updateSuccess') : t('common.createSuccess')) + }) + .catch(() => { + setLoading(false) + }); + } + const handleSave = () => { + form + .validateFields() + .then((values) => { + setLoading(true) + const { logo, ...rest } = values; + let formData: CustomModelForm = { + ...rest + } + formData.is_official = false; + + if (typeof logo === 'object' && logo?.response?.data.file_id) { + getFileLink(logo?.response?.data.file_id) + .then(res => { + const logoRes = res as { url: string } + formData.logo = logoRes.url + handleUpdate(formData) + }) + .catch(() => { + handleUpdate(formData) + }) + } else { + formData.logo = typeof logo === 'string' ? logo : logo.url + handleUpdate(formData) + } + }) + .catch((err) => { + console.log('err', err) + }); + } + + useImperativeHandle(ref, () => ({ + handleOpen, + })); + + console.log('formValues', formValues) + + return ( + +
      + + + + + + + + + items.map((item) => ({ label: t(`modelNew.${item}`), value: String(item) }))} + /> + + + + items.map((item) => ({ label: t(`modelNew.${item}`), value: String(item) }))} + /> + + + + + + + + + + + items.map((item) => ({ + label: t(`modelNew.${typeof item === 'object' ? item.value : item}`), + value: typeof item === 'object' ? item.value : item + }))} + disabled={isEdit} + /> + + + + + + + + + +
      +       + ); +}); + +export default KeyConfigModal; \ No newline at end of file diff --git a/web/src/views/ModelManagement/components/ModelImplement/SubModelModal.tsx b/web/src/views/ModelManagement/components/ModelImplement/SubModelModal.tsx new file mode 100644 index 00000000..d5b3ad45 --- /dev/null +++ b/web/src/views/ModelManagement/components/ModelImplement/SubModelModal.tsx @@ -0,0 +1,181 @@ +import { forwardRef, useImperativeHandle, useState, useEffect } from 'react'; +import { Form, Cascader, App, type CascaderProps } from 'antd'; +import { useTranslation } from 'react-i18next'; + +import type { SubModelModalForm, SubModelModalRef, SubModelModalProps } from './types'; +import RbModal from '@/components/RbModal' +import CustomSelect from '@/components/CustomSelect' +import { modelProviderUrl, getModelNewList } from '@/api/models' +import type { ProviderModelItem } from '../../types' + +const { SHOW_CHILD } = Cascader; + +interface Option { + value: string | number; + label: string; + children?: Option[]; + [key: string]: any; +} +const SubModelModal = forwardRef(({ + refresh, + type, + groupedByProvider +}, ref) => { + const { t } = useTranslation(); + const { message } = App.useApp() + const [visible, setVisible] = useState(false); + const [form] = Form.useForm(); + const [selecteds, setSelecteds] = useState([]) + const [modelList, setModelList] = useState([]) + const provider = Form.useWatch(['provider'], form) + + useEffect(() => { + if (provider && groupedByProvider) { + const lastModels = groupedByProvider[provider] || [] + const list = lastModels.map(vo => [{ name: vo.model_name, id: vo.model_config_ids[0], value: vo.model_config_ids[0], provider }, { value: vo.id }]) + setSelecteds(list) + form.setFieldValue('api_key_ids', lastModels.map(vo => [vo.model_config_ids[0], vo.id])) + } + }, [groupedByProvider, provider]) + + // 封装取消方法，添加关闭弹窗逻辑 + const handleClose = () => { + form.resetFields(); + setVisible(false); + setSelecteds([]) + setModelList([]) + }; + + const handleOpen = () => { + form.resetFields() + setVisible(true); + }; + // 封装保存方法，添加提交逻辑 + const handleSave = () => { + form + .validateFields() + .then(() => { + refresh?.(selecteds.map(vo => ({ + ...vo[0], + model_name: vo[0].name, + model_config_ids: [vo[0].id], + id: vo[1].value, + api_key: vo[1].label + }))) + handleClose() + }) + } + const handleChange = (value: (string | number)[][], selectedOptions: Option[][]) => { + const filterList = selectedOptions.filter(vo => vo.length === 1).map(item => item[0]) + const lastFilterLit = value.filter(vo => vo.length !== 1) + if (filterList.length) { + message.warning(`【${filterList.map(vo => vo.label)}】${t('modelNew.selectOneTip')}`) + form.setFieldValue('api_key_ids', lastFilterLit) + } + setSelecteds(selectedOptions) + } + + const handleChangeProvider = (provider: string, api_key_ids?: any[]) => { + form.setFieldValue('api_key_ids', undefined) + if (provider) { + getModelNewList({ + provider: provider, + is_composite: false, + is_active: true, + type + }) + .then(res => { + const response = res as ProviderModelItem[] + const list = response[0]?.models || [] + setModelList(list.map(vo => { + const children = vo.api_keys.map(item => ({ + label: item.api_key, + value: item.id, + })) + return { + ...vo, + label: vo.name, + value: vo.id, + children: children + } + })) + + if (api_key_ids?.length) { + form.setFieldsValue({ + api_key_ids: api_key_ids + }) + } + }) + } else { + setModelList([]) + } + } + const displayRender: CascaderProps