Release/v0.2.2 (#258)
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 读取的接口,去掉全局锁
* 输出数组
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化测试接口
* 反思优化测试接口
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 把group_id替换end_user_id
* 把group_id替换end_user_id_
* 把group_id替换end_user_id_
* config_config替换成memory_config
* config_config替换成memory_config
* [fix]Fix the memory interface to use end_user_id.
* config_config替换成memory_config
* config_config替换成memory_config
* config_config替换成memory_config
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID,与develop校对恢复
* 检查项目,修复group_id的遗留问题
* 检查项目,修复group_id的遗留问题
* Fix/interface home (#182)
* [fix]Fix the interface for statistics of recent activities and applications
* [changes]Modify the code based on the AI review
1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter.
2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls.
* [fix]Fix the interface for statistics of recent activities and applications
* [changes]Modify the code based on the AI review
1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter.
2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls.
* Fix/optimize inerface (#183)
* [changes]Optimize the time consumption of the "/end_users" interface
* [fix]Optimize the time consumption of the "/hot_memory_tags" interface
* [changes]Optimize the time consumption of the "/end_users" interface
* [fix]Optimize the time consumption of the "/hot_memory_tags" interface
* [changes]Improve the code based on AI review
* Fix/memory mcp2 1 (#184)
* 优化快速检索的回复内容
* 优化快速检索的回复内容
* Fix/memory mcp2 1 (#185)
* 优化快速检索的回复内容
* 优化快速检索的回复内容
* 路径的BUG修复
* 路径的BUG修复
* 路径的BUG修复
* 路径的BUG修复
* 路径的BUG修复
* Fix/memory mcp2 1 (#188)
* 优化快速检索的回复内容
* 优化快速检索的回复内容
* 路径的BUG修复
* 路径的BUG修复
* 路径的BUG修复
* 路径的BUG修复
* 路径的BUG修复
* LLM生存缺少config_id认证,修复BUG
* LLM生存缺少config_id认证,修复BUG
* LLM生存缺少config_id认证,修复BUG
* 解决冲突
* 解决冲突
* feat(home page): version description update
* Fix/memory mcp2 1 (#190)
* 优化快速检索的回复内容
* 优化快速检索的回复内容
* 路径的BUG修复
* 路径的BUG修复
* 路径的BUG修复
* 路径的BUG修复
* 路径的BUG修复
* LLM生存缺少config_id认证,修复BUG
* LLM生存缺少config_id认证,修复BUG
* LLM生存缺少config_id认证,修复BUG
* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问
* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问
* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问
* end_user_id清理干净
* end_user_id清理干净
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* feat(web): memory related interface parameter transfer adjustment
* 感知meta_data字段BUG修复
* Fix/memory bug fix (#171)
* feat(sandbox): add Python 3 code execution sandbox support
* feat(workflow): emit SSE events for node exception output
* perf(sandbox): optimize code encryption handling
* perf(workflow): update standard node output structure
* [add] migration script
* [modify] migration script
* feat(web): add workflow runtime info
* fix(web): handleSSE bugfix
* fix(sandbox): prevent imports from being blocked when network is disabled
* user_id->现实为config_id_old
* user_id->显示为config_id_old传输
* Fix/memory bug fix (#199)
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 读取的接口,去掉全局锁
* 输出数组
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化测试接口
* 反思优化测试接口
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 把group_id替换end_user_id
* 把group_id替换end_user_id_
* 把group_id替换end_user_id_
* config_config替换成memory_config
* config_config替换成memory_config
* [fix]Fix the memory interface to use end_user_id.
* config_config替换成memory_config
* config_config替换成memory_config
* config_config替换成memory_config
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID,与develop校对恢复
* 检查项目,修复group_id的遗留问题
* 检查项目,修复group_id的遗留问题
* 解决冲突
* 解决冲突
* end_user_id清理干净
* end_user_id清理干净
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 感知meta_data字段BUG修复
* user_id->现实为config_id_old
* user_id->显示为config_id_old传输
---------
Co-authored-by: lanceyq <1982376970@qq.com>
* user_id->显示为config_id_old传输
* feat(web): update read_all_config select valueKey
* user_id->显示为config_id_old传输
* feat(workflow): Add a new node for executing code
* fix(web): KnowledgeConfigModal bugfix
* fix(web): iteration's variable add parameter-extractor node
* fix(sandbox): treat non-zero exit codes as errors instead of relying only on stderr
* Fix/memory bug fix (#200)
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 读取的接口,去掉全局锁
* 输出数组
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化测试接口
* 反思优化测试接口
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 把group_id替换end_user_id
* 把group_id替换end_user_id_
* 把group_id替换end_user_id_
* config_config替换成memory_config
* config_config替换成memory_config
* [fix]Fix the memory interface to use end_user_id.
* config_config替换成memory_config
* config_config替换成memory_config
* config_config替换成memory_config
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID,与develop校对恢复
* 检查项目,修复group_id的遗留问题
* 检查项目,修复group_id的遗留问题
* 解决冲突
* 解决冲突
* end_user_id清理干净
* end_user_id清理干净
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 感知meta_data字段BUG修复
* user_id->现实为config_id_old
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
---------
Co-authored-by: lanceyq <1982376970@qq.com>
* Refactor/benchmark test (#196)
* [changes]refactor locomo_test
* [fix]Fix the circular import of ModelParameters
* [changes]The benchmark test can run stably.
* [fix]Complete end-to-end LoCoMo repair
* [fix]Complete the end-to-end longmemeval and memsciqa fixes
* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.
* [changes]refactor locomo_test
* [fix]Fix the circular import of ModelParameters
* [changes]The benchmark test can run stably.
* [fix]Complete end-to-end LoCoMo repair
* [fix]Complete the end-to-end longmemeval and memsciqa fixes
* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.
* [changes]Benchmark test adaptation for end_user_id
* [changes]refactor locomo_test
* [fix]Fix the circular import of ModelParameters
* [changes]The benchmark test can run stably.
* [fix]Complete end-to-end LoCoMo repair
* [fix]Complete the end-to-end longmemeval and memsciqa fixes
* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.
* [fix]Complete the end-to-end longmemeval and memsciqa fixes
* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.
* [changes]Benchmark test adaptation for end_user_id
* [modify] migration script
* delete benchmark-test (#204)
* Refactor: Move evaluation folder to redbear-mem-benchmark submodule
* [changes]Restore .gitmodules
* feat(web): workflow add code node
* 检查需要更改的格式问题
* Fix/redbear benchmark (#205)
* Refactor: Move evaluation folder to redbear-mem-benchmark submodule
* [changes]Update submodule reference
* Refactor: Move evaluation folder to redbear-mem-benchmark submodule
* [changes]Update submodule reference
* Remove duplicate evaluation submodule, use redbear-mem-benchmark instead
* Fix/memory bug fix (#207)
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 读取的接口,去掉全局锁
* 输出数组
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化测试接口
* 反思优化测试接口
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 把group_id替换end_user_id
* 把group_id替换end_user_id_
* 把group_id替换end_user_id_
* config_config替换成memory_config
* config_config替换成memory_config
* [fix]Fix the memory interface to use end_user_id.
* config_config替换成memory_config
* config_config替换成memory_config
* config_config替换成memory_config
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID,与develop校对恢复
* 检查项目,修复group_id的遗留问题
* 检查项目,修复group_id的遗留问题
* 解决冲突
* 解决冲突
* end_user_id清理干净
* end_user_id清理干净
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 感知meta_data字段BUG修复
* user_id->现实为config_id_old
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* 检查需要更改的格式问题
---------
Co-authored-by: lanceyq <1982376970@qq.com>
* fix(web): remove URI decode and encode
* [add] plugin system and base sso module
* 修复宿主列表获取memory_config_idBUG
* Fix/memory bug fix (#209)
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 读取的接口,去掉全局锁
* 输出数组
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化测试接口
* 反思优化测试接口
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 把group_id替换end_user_id
* 把group_id替换end_user_id_
* 把group_id替换end_user_id_
* config_config替换成memory_config
* config_config替换成memory_config
* [fix]Fix the memory interface to use end_user_id.
* config_config替换成memory_config
* config_config替换成memory_config
* config_config替换成memory_config
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID,与develop校对恢复
* 检查项目,修复group_id的遗留问题
* 检查项目,修复group_id的遗留问题
* 解决冲突
* 解决冲突
* end_user_id清理干净
* end_user_id清理干净
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 感知meta_data字段BUG修复
* user_id->现实为config_id_old
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* 检查需要更改的格式问题
* 修复宿主列表获取memory_config_idBUG
---------
Co-authored-by: lanceyq <1982376970@qq.com>
* [modify] file local server url
* [add] migration script
* fix(workflow): fix activation and branch control issues in streaming output
* fix(workflow): fix function cache not taking effect and potential list index overflow
* style(workflow): enforce PEP8 style and remove redundant imports
* fix(workflow): fix streaming output error when variable is not a string
* [fix]remove aspose-slides
* perf(workflow): enhance streaming output node activation performance
* feat(workflow): store token usage in message table
* feat(web): add PageEmpty component
* feat(web): add PageTabs component
* perf(workflow): make memory configuration backward compatible
* feat(web): update model management
* config_id做映射
* config_id做映射
* Fix/memory bug fix (#211)
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 读取的接口,去掉全局锁
* 输出数组
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化测试接口
* 反思优化测试接口
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 把group_id替换end_user_id
* 把group_id替换end_user_id_
* 把group_id替换end_user_id_
* config_config替换成memory_config
* config_config替换成memory_config
* [fix]Fix the memory interface to use end_user_id.
* config_config替换成memory_config
* config_config替换成memory_config
* config_config替换成memory_config
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID,与develop校对恢复
* 检查项目,修复group_id的遗留问题
* 检查项目,修复group_id的遗留问题
* 解决冲突
* 解决冲突
* end_user_id清理干净
* end_user_id清理干净
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 感知meta_data字段BUG修复
* user_id->现实为config_id_old
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* 检查需要更改的格式问题
* 修复宿主列表获取memory_config_idBUG
* config_id做映射
* config_id做映射
---------
Co-authored-by: lanceyq <1982376970@qq.com>
* feat(web): getModelListUrl add is_active param
* config_id做映射+1
* config_id做映射+1
* config_id做映射+1
* feat(web): remove file url replace
* Fix/memory bug fix (#212)
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 读取的接口,去掉全局锁
* 输出数组
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化测试接口
* 反思优化测试接口
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 把group_id替换end_user_id
* 把group_id替换end_user_id_
* 把group_id替换end_user_id_
* config_config替换成memory_config
* config_config替换成memory_config
* [fix]Fix the memory interface to use end_user_id.
* config_config替换成memory_config
* config_config替换成memory_config
* config_config替换成memory_config
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID,与develop校对恢复
* 检查项目,修复group_id的遗留问题
* 检查项目,修复group_id的遗留问题
* 解决冲突
* 解决冲突
* end_user_id清理干净
* end_user_id清理干净
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 感知meta_data字段BUG修复
* user_id->现实为config_id_old
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* 检查需要更改的格式问题
* 修复宿主列表获取memory_config_idBUG
* config_id做映射
* config_id做映射
* config_id做映射+1
* config_id做映射+1
* config_id做映射+1
---------
Co-authored-by: lanceyq <1982376970@qq.com>
* feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics
* feat(web): model logo update
* 应用层memory_content->memory_config
* fix(web): correct spelling
* 应用层memory_content->memory_config
* 应用层memory_content->memory_config
* Fix/memory bug fix (#215)
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 读取的接口,去掉全局锁
* 输出数组
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化测试接口
* 反思优化测试接口
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 把group_id替换end_user_id
* 把group_id替换end_user_id_
* 把group_id替换end_user_id_
* config_config替换成memory_config
* config_config替换成memory_config
* [fix]Fix the memory interface to use end_user_id.
* config_config替换成memory_config
* config_config替换成memory_config
* config_config替换成memory_config
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID,与develop校对恢复
* 检查项目,修复group_id的遗留问题
* 检查项目,修复group_id的遗留问题
* 解决冲突
* 解决冲突
* end_user_id清理干净
* end_user_id清理干净
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 感知meta_data字段BUG修复
* user_id->现实为config_id_old
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* 检查需要更改的格式问题
* 修复宿主列表获取memory_config_idBUG
* config_id做映射
* config_id做映射
* config_id做映射+1
* config_id做映射+1
* config_id做映射+1
* 应用层memory_content->memory_config
* 应用层memory_content->memory_config
* 应用层memory_content->memory_config
---------
Co-authored-by: lanceyq <1982376970@qq.com>
* feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics
* fix(web): model loading update
* 统一字段为config_id_old
* 统一字段为config_id_old
* feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics
* 统一字段为config_id_old
* 统一字段为config_id_old
* memory_content暂时不修改
* memory_content暂时不修改
* Fix/memory bug fix (#217)
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 图谱数据量限制数量去掉
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 用户详情优化
* 读取的接口,去掉全局锁
* 输出数组
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化1.0(优化隐私输出、时间检索)
* 反思优化测试接口
* 反思优化测试接口
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 读取接口内层嵌套BUG修复
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
* 把group_id替换end_user_id
* 把group_id替换end_user_id_
* 把group_id替换end_user_id_
* config_config替换成memory_config
* config_config替换成memory_config
* [fix]Fix the memory interface to use end_user_id.
* config_config替换成memory_config
* config_config替换成memory_config
* config_config替换成memory_config
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID
* config_id字段改成UUID,与develop校对恢复
* 检查项目,修复group_id的遗留问题
* 检查项目,修复group_id的遗留问题
* 解决冲突
* 解决冲突
* end_user_id清理干净
* end_user_id清理干净
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 修复遗留合并BUG
* 感知meta_data字段BUG修复
* user_id->现实为config_id_old
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* user_id->显示为config_id_old传输
* 检查需要更改的格式问题
* 修复宿主列表获取memory_config_idBUG
* config_id做映射
* config_id做映射
* config_id做映射+1
* config_id做映射+1
* config_id做映射+1
* 应用层memory_content->memory_config
* 应用层memory_content->memory_config
* 应用层memory_content->memory_config
* 统一字段为config_id_old
* 统一字段为config_id_old
* 统一字段为config_id_old
* 统一字段为config_id_old
* memory_content暂时不修改
* memory_content暂时不修改
---------
Co-authored-by: lanceyq <1982376970@qq.com>
* feat(web): add app statistics
* fix(workflow): fix streaming output issues with multi-output End nodes
End nodes with multiple output segments could cause cursor errors or leave some
segments inactive, resulting in incorrect final outputs.
Unified _emit_active_chunks and _update_scope_activate to ensure all segments
are activated in order and streamed correctly.
* feat(web): add apps statistics api
* fix(web): agent's knowledge_bases bugfix
* Revert "feat(web): update read_all_config select valueKey"
This reverts commit 46f0f3cee9.
* [add] migrations script
* perf(workflow): make memory write node backward-compatible and defer config validation
* 旧数据兼容
* 旧数据兼容
* 旧数据兼容
* 旧数据兼容
* fix(web): model bugfix
* fix(web): model bugfix
* 提交遗漏 (#228)
* [fix] chat api for workflow
* [fix] web search set for v1 api
* fix(web): model bugfix
* fix(web): model list remove is_active
* fix(model): bug fix
* [add]migration script
* [fix] api
* [fix] api
* fix(web): model bugfix
* fix(model): the model type does not allow modification, delete tts and speech2text type
* fix(model): bug fix
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* Add/develop memory (#239)
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* feat(web): model ui update
* feat(web): model ui update
* Add/develop memory (#243)
* 遗漏的历史映射
* 遗漏的历史映射
* fix(model): bug fix
* feat(web): model ui update
* Add/develop memory (#247)
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* [modify] migration script
* [add] migration script
* fix(web): change form message
* fix(web): the memoryContent field is compatible with numbers and strings
* feat(web): code node hidden
* fix(model):
1. create a basic model to check if the name and provider are duplicated.
2. The result shows error models because the provider created API Keys for all matching models.
---------
Co-authored-by: lixinyue <2569494688@qq.com>
Co-authored-by: lanceyq <1982376970@qq.com>
Co-authored-by: yujiangping <yujiangping@taofen8.com>
Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com>
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: Eternity <1533512157@qq.com>
Co-authored-by: lixiangcheng1 <lixiangcheng1@wanda.cn>
This commit is contained in:
@@ -55,8 +55,8 @@ class AgentRegistry:
|
||||
"""
|
||||
# 构建查询
|
||||
stmt = select(AgentConfig).join(App).where(
|
||||
AgentConfig.is_active == True,
|
||||
App.is_active == True
|
||||
AgentConfig.is_active.is_(True),
|
||||
App.is_active.is_(True)
|
||||
)
|
||||
|
||||
# 工作空间过滤(同工作空间或公开)
|
||||
|
||||
@@ -758,7 +758,7 @@ class AppService:
|
||||
)
|
||||
|
||||
# 构建查询条件
|
||||
filters = [App.is_active == True]
|
||||
filters = [App.is_active.is_(True)]
|
||||
if type:
|
||||
filters.append(App.type == type)
|
||||
if visibility:
|
||||
@@ -873,7 +873,7 @@ class AppService:
|
||||
|
||||
self._validate_workspace_access(app, workspace_id)
|
||||
|
||||
stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active == True).order_by(
|
||||
stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active.is_(True)).order_by(
|
||||
AgentConfig.updated_at.desc())
|
||||
agent_cfg: Optional[AgentConfig] = self.db.scalars(stmt).first()
|
||||
now = datetime.datetime.now()
|
||||
@@ -1204,7 +1204,7 @@ class AppService:
|
||||
default_model_config_id = None
|
||||
|
||||
if app.type == AppType.AGENT:
|
||||
stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active == True).order_by(
|
||||
stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active.is_(True)).order_by(
|
||||
AgentConfig.updated_at.desc())
|
||||
agent_cfg = self.db.scalars(stmt).first()
|
||||
if not agent_cfg:
|
||||
@@ -1226,7 +1226,7 @@ class AppService:
|
||||
select(MultiAgentConfig)
|
||||
.where(
|
||||
MultiAgentConfig.app_id == app_id,
|
||||
MultiAgentConfig.is_active == True
|
||||
MultiAgentConfig.is_active.is_(True)
|
||||
)
|
||||
.order_by(MultiAgentConfig.updated_at.desc())
|
||||
)
|
||||
@@ -1380,7 +1380,7 @@ class AppService:
|
||||
|
||||
stmt = (
|
||||
select(AppRelease)
|
||||
.where(AppRelease.app_id == app_id, AppRelease.is_active == True)
|
||||
.where(AppRelease.app_id == app_id, AppRelease.is_active.is_(True))
|
||||
.order_by(AppRelease.version.desc())
|
||||
)
|
||||
return list(self.db.scalars(stmt).all())
|
||||
|
||||
193
api/app/services/app_statistics_service.py
Normal file
193
api/app/services/app_statistics_service.py
Normal file
@@ -0,0 +1,193 @@
|
||||
"""应用统计服务"""
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Dict, Any, List
|
||||
import uuid
|
||||
from sqlalchemy import func, and_, cast, Date
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models.conversation_model import Conversation, Message
|
||||
from app.models.end_user_model import EndUser
|
||||
from app.models.api_key_model import ApiKey, ApiKeyLog
|
||||
from app.core.exceptions import BusinessException
|
||||
from app.core.error_codes import BizCode
|
||||
|
||||
|
||||
class AppStatisticsService:
|
||||
"""应用统计服务"""
|
||||
|
||||
def __init__(self, db: Session):
|
||||
self.db = db
|
||||
|
||||
def get_app_statistics(
|
||||
self,
|
||||
app_id: uuid.UUID,
|
||||
workspace_id: uuid.UUID,
|
||||
start_date: int,
|
||||
end_date: int
|
||||
) -> Dict[str, Any]:
|
||||
"""获取应用统计数据
|
||||
|
||||
Args:
|
||||
app_id: 应用ID
|
||||
workspace_id: 工作空间ID
|
||||
start_date: 开始时间戳(毫秒)
|
||||
end_date: 结束时间戳(毫秒)
|
||||
|
||||
Returns:
|
||||
统计数据字典
|
||||
"""
|
||||
# 将毫秒时间戳转换为 datetime
|
||||
start_dt = datetime.fromtimestamp(start_date / 1000)
|
||||
end_dt = datetime.fromtimestamp(end_date / 1000) + timedelta(days=1)
|
||||
|
||||
# 1. 会话统计
|
||||
conversations_stats = self._get_conversations_statistics(app_id, workspace_id, start_dt, end_dt)
|
||||
|
||||
# 2. 新增用户统计
|
||||
users_stats = self._get_new_users_statistics(app_id, start_dt, end_dt)
|
||||
|
||||
# 3. API调用统计
|
||||
api_stats = self._get_api_calls_statistics(app_id, start_dt, end_dt)
|
||||
|
||||
# 4. Token消耗统计
|
||||
token_stats = self._get_token_statistics(app_id, start_dt, end_dt)
|
||||
|
||||
return {
|
||||
"daily_conversations": conversations_stats["daily"],
|
||||
"total_conversations": conversations_stats["total"],
|
||||
"daily_new_users": users_stats["daily"],
|
||||
"total_new_users": users_stats["total"],
|
||||
"daily_api_calls": api_stats["daily"],
|
||||
"total_api_calls": api_stats["total"],
|
||||
"daily_tokens": token_stats["daily"],
|
||||
"total_tokens": token_stats["total"]
|
||||
}
|
||||
|
||||
def _get_conversations_statistics(
|
||||
self,
|
||||
app_id: uuid.UUID,
|
||||
workspace_id: uuid.UUID,
|
||||
start_dt: datetime,
|
||||
end_dt: datetime
|
||||
) -> Dict[str, Any]:
|
||||
"""获取会话统计"""
|
||||
# 每日会话数
|
||||
daily_query = self.db.query(
|
||||
cast(Conversation.created_at, Date).label('date'),
|
||||
func.count(Conversation.id).label('count')
|
||||
).filter(
|
||||
and_(
|
||||
Conversation.app_id == app_id,
|
||||
Conversation.workspace_id == workspace_id,
|
||||
Conversation.created_at >= start_dt,
|
||||
Conversation.created_at < end_dt
|
||||
)
|
||||
).group_by(cast(Conversation.created_at, Date)).all()
|
||||
|
||||
daily_data = [{"date": str(row.date), "count": row.count} for row in daily_query]
|
||||
total = sum(row["count"] for row in daily_data)
|
||||
|
||||
return {"daily": daily_data, "total": total}
|
||||
|
||||
def _get_new_users_statistics(
|
||||
self,
|
||||
app_id: uuid.UUID,
|
||||
start_dt: datetime,
|
||||
end_dt: datetime
|
||||
) -> Dict[str, Any]:
|
||||
"""获取新增用户统计"""
|
||||
# 每日新增用户数
|
||||
daily_query = self.db.query(
|
||||
cast(EndUser.created_at, Date).label('date'),
|
||||
func.count(EndUser.id).label('count')
|
||||
).filter(
|
||||
and_(
|
||||
EndUser.app_id == app_id,
|
||||
EndUser.created_at >= start_dt,
|
||||
EndUser.created_at < end_dt
|
||||
)
|
||||
).group_by(cast(EndUser.created_at, Date)).all()
|
||||
|
||||
daily_data = [{"date": str(row.date), "count": row.count} for row in daily_query]
|
||||
total = sum(row["count"] for row in daily_data)
|
||||
|
||||
return {"daily": daily_data, "total": total}
|
||||
|
||||
def _get_api_calls_statistics(
|
||||
self,
|
||||
app_id: uuid.UUID,
|
||||
start_dt: datetime,
|
||||
end_dt: datetime
|
||||
) -> Dict[str, Any]:
|
||||
"""获取API调用统计"""
|
||||
# 每日API调用次数
|
||||
daily_query = self.db.query(
|
||||
cast(ApiKeyLog.created_at, Date).label('date'),
|
||||
func.count(ApiKeyLog.id).label('count')
|
||||
).join(
|
||||
ApiKey, ApiKeyLog.api_key_id == ApiKey.id
|
||||
).filter(
|
||||
and_(
|
||||
ApiKey.resource_id == app_id,
|
||||
ApiKeyLog.created_at >= start_dt,
|
||||
ApiKeyLog.created_at < end_dt
|
||||
)
|
||||
).group_by(cast(ApiKeyLog.created_at, Date)).all()
|
||||
|
||||
daily_data = [{"date": str(row.date), "count": row.count} for row in daily_query]
|
||||
total = sum(row["count"] for row in daily_data)
|
||||
|
||||
return {"daily": daily_data, "total": total}
|
||||
|
||||
def _get_token_statistics(
|
||||
self,
|
||||
app_id: uuid.UUID,
|
||||
start_dt: datetime,
|
||||
end_dt: datetime
|
||||
) -> Dict[str, Any]:
|
||||
"""获取Token消耗统计(从Message的meta_data中提取)"""
|
||||
from sqlalchemy import text
|
||||
|
||||
# 查询所有相关消息的token使用情况
|
||||
# meta_data中可能包含: {"usage": {"total_tokens": 100}} 或 {"tokens": 100}
|
||||
daily_query = self.db.query(
|
||||
cast(Message.created_at, Date).label('date'),
|
||||
Message.meta_data
|
||||
).join(
|
||||
Conversation, Message.conversation_id == Conversation.id
|
||||
).filter(
|
||||
and_(
|
||||
Conversation.app_id == app_id,
|
||||
Message.created_at >= start_dt,
|
||||
Message.created_at < end_dt,
|
||||
Message.meta_data.isnot(None)
|
||||
)
|
||||
).all()
|
||||
|
||||
# 按日期聚合token
|
||||
daily_tokens = {}
|
||||
for row in daily_query:
|
||||
date_str = str(row.date)
|
||||
meta = row.meta_data or {}
|
||||
|
||||
# 提取token数量(支持多种格式)
|
||||
tokens = 0
|
||||
if isinstance(meta, dict):
|
||||
# 格式1: {"usage": {"total_tokens": 100}}
|
||||
if "usage" in meta and isinstance(meta["usage"], dict):
|
||||
tokens = meta["usage"].get("total_tokens", 0)
|
||||
# 格式2: {"tokens": 100}
|
||||
elif "tokens" in meta:
|
||||
tokens = meta.get("tokens", 0)
|
||||
# 格式3: {"total_tokens": 100}
|
||||
elif "total_tokens" in meta:
|
||||
tokens = meta.get("total_tokens", 0)
|
||||
|
||||
if date_str not in daily_tokens:
|
||||
daily_tokens[date_str] = 0
|
||||
daily_tokens[date_str] += int(tokens)
|
||||
|
||||
daily_data = [{"date": date, "tokens": tokens} for date, tokens in sorted(daily_tokens.items()) if tokens != 0]
|
||||
total = sum(row["tokens"] for row in daily_data)
|
||||
|
||||
return {"daily": daily_data, "total": total}
|
||||
@@ -16,6 +16,7 @@ from app.core.exceptions import BusinessException
|
||||
from app.core.logging_config import get_business_logger
|
||||
from app.core.rag.nlp.search import knowledge_retrieval
|
||||
from app.models import AgentConfig, ModelApiKey, ModelConfig
|
||||
from app.repositories.model_repository import ModelApiKeyRepository
|
||||
from app.repositories.tool_repository import ToolRepository
|
||||
from app.schemas.prompt_schema import PromptMessageRole, render_prompt_message
|
||||
from app.services import task_service
|
||||
@@ -56,7 +57,7 @@ def create_long_term_memory_tool(memory_config: Dict[str, Any], end_user_id: str
|
||||
长期记忆工具
|
||||
"""
|
||||
# search_switch = memory_config.get("search_switch", "2")
|
||||
config_id= memory_config.get("memory_content",None)
|
||||
config_id= memory_config.get("memory_content") or memory_config.get("memory_config",None)
|
||||
logger.info(f"创建长期记忆工具,配置: end_user_id={end_user_id}, config_id={config_id}, storage_type={storage_type}")
|
||||
@tool(args_schema=LongTermMemoryInput)
|
||||
def long_term_memory(question: str) -> str:
|
||||
@@ -92,7 +93,7 @@ def create_long_term_memory_tool(memory_config: Dict[str, Any], end_user_id: str
|
||||
try:
|
||||
memory_content = asyncio.run(
|
||||
MemoryAgentService().read_memory(
|
||||
group_id=end_user_id,
|
||||
end_user_id=end_user_id,
|
||||
message=question,
|
||||
history=[],
|
||||
search_switch="2",
|
||||
@@ -106,9 +107,9 @@ def create_long_term_memory_tool(memory_config: Dict[str, Any], end_user_id: str
|
||||
"app.core.memory.agent.read_message",
|
||||
args=[end_user_id, question, [], "1", config_id, storage_type, user_rag_memory_id]
|
||||
)
|
||||
# result = task_service.get_task_memory_read_result(task.id)
|
||||
# status = result.get("status")
|
||||
# logger.info(f"读取任务状态:{status}")
|
||||
result = task_service.get_task_memory_read_result(task.id)
|
||||
status = result.get("status")
|
||||
logger.info(f"读取任务状态:{status}")
|
||||
|
||||
finally:
|
||||
db.close()
|
||||
@@ -418,7 +419,7 @@ class DraftRunService:
|
||||
)
|
||||
|
||||
memory_config_= agent_config.memory
|
||||
config_id = memory_config_.get("memory_content")
|
||||
config_id = memory_config_.get("memory_content") or memory_config_.get("memory_config",None)
|
||||
|
||||
# 7. 调用 Agent
|
||||
result = await agent.chat(
|
||||
@@ -644,7 +645,7 @@ class DraftRunService:
|
||||
})
|
||||
|
||||
memory_config_ = agent_config.memory
|
||||
config_id = memory_config_.get("memory_content")
|
||||
config_id = memory_config_.get("memory_content") or memory_config_.get("memory_config",None)
|
||||
|
||||
# 9. 流式调用 Agent
|
||||
full_content = ""
|
||||
@@ -724,17 +725,21 @@ class DraftRunService:
|
||||
Raises:
|
||||
BusinessException: 当没有可用的 API Key 时
|
||||
"""
|
||||
stmt = (
|
||||
select(ModelApiKey)
|
||||
.where(
|
||||
ModelApiKey.model_config_id == model_config_id,
|
||||
ModelApiKey.is_active == True
|
||||
)
|
||||
.order_by(ModelApiKey.priority.desc())
|
||||
.limit(1)
|
||||
)
|
||||
|
||||
api_key = self.db.scalars(stmt).first()
|
||||
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config_id)
|
||||
# stmt = (
|
||||
# select(ModelApiKey).join(
|
||||
# ModelConfig, ModelApiKey.model_configs
|
||||
# )
|
||||
# .where(
|
||||
# ModelConfig.id == model_config_id,
|
||||
# ModelApiKey.is_active.is_(True)
|
||||
# )
|
||||
# .order_by(ModelApiKey.priority.desc())
|
||||
# .limit(1)
|
||||
# )
|
||||
#
|
||||
# api_key = self.db.scalars(stmt).first()
|
||||
api_key = api_keys[0] if api_keys else None
|
||||
|
||||
if not api_key:
|
||||
raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING)
|
||||
|
||||
@@ -75,7 +75,7 @@ class EmotionAnalyticsService:
|
||||
|
||||
# 调用仓储层查询
|
||||
tags = await self.emotion_repo.get_emotion_tags(
|
||||
group_id=end_user_id,
|
||||
end_user_id=end_user_id,
|
||||
emotion_type=emotion_type,
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
@@ -157,7 +157,7 @@ class EmotionAnalyticsService:
|
||||
|
||||
# 调用仓储层查询
|
||||
keywords = await self.emotion_repo.get_emotion_wordcloud(
|
||||
group_id=end_user_id,
|
||||
end_user_id=end_user_id,
|
||||
emotion_type=emotion_type,
|
||||
limit=limit
|
||||
)
|
||||
@@ -339,7 +339,7 @@ class EmotionAnalyticsService:
|
||||
|
||||
# 获取时间范围内的情绪数据
|
||||
emotions = await self.emotion_repo.get_emotions_in_range(
|
||||
group_id=end_user_id,
|
||||
end_user_id=end_user_id,
|
||||
time_range=time_range
|
||||
)
|
||||
|
||||
@@ -505,7 +505,7 @@ class EmotionAnalyticsService:
|
||||
)
|
||||
config_service = MemoryConfigService(db)
|
||||
memory_config = config_service.load_memory_config(
|
||||
config_id=int(config_id),
|
||||
config_id=(config_id),
|
||||
service_name="EmotionAnalyticsService.generate_emotion_suggestions"
|
||||
)
|
||||
from app.core.memory.utils.llm.llm_utils import MemoryClientFactory
|
||||
@@ -519,7 +519,7 @@ class EmotionAnalyticsService:
|
||||
|
||||
# 3. 获取情绪数据用于模式分析
|
||||
emotions = await self.emotion_repo.get_emotions_in_range(
|
||||
group_id=end_user_id,
|
||||
end_user_id=end_user_id,
|
||||
time_range="30d"
|
||||
)
|
||||
|
||||
@@ -598,13 +598,13 @@ class EmotionAnalyticsService:
|
||||
# 查询用户的实体和标签
|
||||
query = """
|
||||
MATCH (e:Entity)
|
||||
WHERE e.group_id = $group_id
|
||||
WHERE e.end_user_id = $end_user_id
|
||||
RETURN e.name as name, e.type as type
|
||||
ORDER BY e.created_at DESC
|
||||
LIMIT 20
|
||||
"""
|
||||
|
||||
entities = await connector.execute_query(query, group_id=end_user_id)
|
||||
entities = await connector.execute_query(query, end_user_id=end_user_id)
|
||||
|
||||
# 提取兴趣标签
|
||||
interests = [e["name"] for e in entities if e.get("type") in ["INTEREST", "HOBBY"]][:5]
|
||||
|
||||
@@ -8,9 +8,11 @@ Classes:
|
||||
"""
|
||||
|
||||
from typing import Dict, Any
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models.data_config_model import DataConfig
|
||||
from app.models.memory_config_model import MemoryConfig
|
||||
from app.core.logging_config import get_business_logger
|
||||
|
||||
logger = get_business_logger()
|
||||
@@ -37,7 +39,7 @@ class EmotionConfigService:
|
||||
self.db = db
|
||||
logger.info("情绪配置服务初始化完成")
|
||||
|
||||
def get_emotion_config(self, config_id: int) -> Dict[str, Any]:
|
||||
def get_emotion_config(self, config_id: UUID) -> Dict[str, Any]:
|
||||
"""获取情绪引擎配置
|
||||
|
||||
查询指定配置ID的情绪相关配置字段。
|
||||
@@ -61,8 +63,8 @@ class EmotionConfigService:
|
||||
logger.info(f"获取情绪配置: config_id={config_id}")
|
||||
|
||||
# 查询配置
|
||||
config = self.db.query(DataConfig).filter(
|
||||
DataConfig.config_id == config_id
|
||||
config = self.db.query(MemoryConfig).filter(
|
||||
MemoryConfig.config_id == config_id
|
||||
).first()
|
||||
|
||||
if not config:
|
||||
@@ -144,7 +146,7 @@ class EmotionConfigService:
|
||||
|
||||
def update_emotion_config(
|
||||
self,
|
||||
config_id: int,
|
||||
config_id: UUID,
|
||||
config_data: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""更新情绪引擎配置
|
||||
@@ -173,8 +175,8 @@ class EmotionConfigService:
|
||||
self.validate_emotion_config(config_data)
|
||||
|
||||
# 查询配置
|
||||
config = self.db.query(DataConfig).filter(
|
||||
DataConfig.config_id == config_id
|
||||
config = self.db.query(MemoryConfig).filter(
|
||||
MemoryConfig.config_id == config_id
|
||||
).first()
|
||||
|
||||
if not config:
|
||||
|
||||
@@ -14,7 +14,7 @@ from app.core.memory.llm_tools.llm_client import LLMClientException
|
||||
from app.core.memory.models.emotion_models import EmotionExtraction
|
||||
from app.core.memory.utils.llm.llm_utils import MemoryClientFactory
|
||||
from app.db import get_db_context
|
||||
from app.models.data_config_model import DataConfig
|
||||
from app.models.memory_config_model import MemoryConfig
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -60,7 +60,7 @@ class EmotionExtractionService:
|
||||
async def extract_emotion(
|
||||
self,
|
||||
statement: str,
|
||||
config: DataConfig
|
||||
config: MemoryConfig
|
||||
) -> Optional[EmotionExtraction]:
|
||||
"""Extract emotion information from a statement.
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ import uuid
|
||||
from typing import Dict, Any, List, Optional, Tuple
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.repositories.model_repository import ModelApiKeyRepository
|
||||
from app.services.conversation_state_manager import ConversationStateManager
|
||||
from app.models import ModelConfig, AgentConfig
|
||||
from app.core.logging_config import get_business_logger
|
||||
@@ -382,11 +383,14 @@ class LLMRouter:
|
||||
from app.core.models.base import RedBearModelConfig
|
||||
from app.models import ModelApiKey, ModelType
|
||||
|
||||
# 获取 API Key 配置
|
||||
api_key_config = self.db.query(ModelApiKey).filter(
|
||||
ModelApiKey.model_config_id == self.routing_model_config.id,
|
||||
ModelApiKey.is_active
|
||||
).first()
|
||||
# 获取 API Key 配置(通过关联关系)
|
||||
# api_key_config = self.db.query(ModelApiKey).join(
|
||||
# ModelConfig, ModelApiKey.model_configs
|
||||
# ).filter(ModelConfig.id == self.routing_model_config.id,
|
||||
# ModelApiKey.is_active == True
|
||||
# ).first()
|
||||
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, self.routing_model_config.id)
|
||||
api_key_config = api_keys[0] if api_keys else None
|
||||
|
||||
if not api_key_config:
|
||||
raise Exception("路由模型没有可用的 API Key")
|
||||
@@ -419,6 +423,9 @@ class LLMRouter:
|
||||
|
||||
# 调用模型
|
||||
response = await llm.ainvoke(prompt)
|
||||
|
||||
from app.services.model_service import ModelApiKeyService
|
||||
ModelApiKeyService.record_api_key_usage(self.db, api_key_config.id)
|
||||
|
||||
# 提取响应内容
|
||||
if hasattr(response, 'content'):
|
||||
|
||||
@@ -5,7 +5,7 @@ import uuid
|
||||
from typing import Dict, Any, List, Optional, Tuple
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.schemas import ModelParameters
|
||||
from app.schemas.app_schema import ModelParameters
|
||||
from app.services.conversation_state_manager import ConversationStateManager
|
||||
from app.models import ModelConfig, AgentConfig
|
||||
from app.core.logging_config import get_business_logger
|
||||
|
||||
@@ -9,6 +9,7 @@ import os
|
||||
import re
|
||||
import time
|
||||
import uuid
|
||||
from uuid import UUID
|
||||
from typing import Any, AsyncGenerator, Dict, List, Optional
|
||||
|
||||
import redis
|
||||
@@ -27,6 +28,7 @@ from app.core.memory.analytics.hot_memory_tags import get_hot_memory_tags
|
||||
from app.core.memory.utils.llm.llm_utils import MemoryClientFactory
|
||||
from app.db import get_db_context
|
||||
from app.models.knowledge_model import Knowledge, KnowledgeType
|
||||
from app.repositories.memory_short_repository import ShortTermMemoryRepository
|
||||
from app.repositories.neo4j.neo4j_connector import Neo4jConnector
|
||||
from app.schemas.memory_agent_schema import Write_UserInput
|
||||
from app.schemas.memory_config_schema import ConfigurationError
|
||||
@@ -35,6 +37,7 @@ from app.services.memory_config_service import MemoryConfigService
|
||||
from app.services.memory_konwledges_server import (
|
||||
write_rag,
|
||||
)
|
||||
from langchain_core.messages import AIMessage
|
||||
from langchain_core.messages import HumanMessage
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy import func
|
||||
@@ -54,25 +57,24 @@ _neo4j_connector = Neo4jConnector()
|
||||
class MemoryAgentService:
|
||||
"""Service for memory agent operations"""
|
||||
|
||||
def writer_messages_deal(self, messages, start_time, group_id, config_id, message, context):
|
||||
def writer_messages_deal(self, messages, start_time, end_user_id, config_id, message, context):
|
||||
duration = time.time() - start_time
|
||||
|
||||
if str(messages) == 'success':
|
||||
logger.info(f"Write operation successful for group {group_id} with config_id {config_id}")
|
||||
logger.info(f"Write operation successful for group {end_user_id} with config_id {config_id}")
|
||||
# 记录成功的操作
|
||||
if audit_logger:
|
||||
audit_logger.log_operation(operation="WRITE", config_id=config_id, group_id=group_id, success=True,
|
||||
audit_logger.log_operation(operation="WRITE", config_id=config_id, end_user_id=end_user_id, success=True,
|
||||
duration=duration, details={"message_length": len(message)})
|
||||
return context
|
||||
else:
|
||||
logger.warning(f"Write operation failed for group {group_id}")
|
||||
logger.warning(f"Write operation failed for group {end_user_id}")
|
||||
|
||||
# 记录失败的操作
|
||||
if audit_logger:
|
||||
audit_logger.log_operation(
|
||||
operation="WRITE",
|
||||
config_id=config_id,
|
||||
group_id=group_id,
|
||||
end_user_id=end_user_id,
|
||||
success=False,
|
||||
duration=duration,
|
||||
error=f"写入失败: {messages[:100]}"
|
||||
@@ -173,10 +175,9 @@ class MemoryAgentService:
|
||||
"""
|
||||
logger.info("Reading log file")
|
||||
|
||||
|
||||
current_file = os.path.abspath(__file__) # app/services/memory_agent_service.py
|
||||
app_dir = os.path.dirname(os.path.dirname(current_file)) # app directory
|
||||
project_root = os.path.dirname(app_dir) # redbear-mem directory
|
||||
# Get log file path - use project root directory
|
||||
from pathlib import Path
|
||||
project_root = str(Path(__file__).resolve().parents[2]) # api directory
|
||||
log_path = os.path.join(project_root, "logs", "agent_service.log")
|
||||
|
||||
summer = ''
|
||||
@@ -215,9 +216,8 @@ class MemoryAgentService:
|
||||
logger.info("Starting log content streaming")
|
||||
|
||||
# Get log file path - use project root directory
|
||||
current_file = os.path.abspath(__file__) # app/services/memory_agent_service.py
|
||||
app_dir = os.path.dirname(os.path.dirname(current_file)) # app directory
|
||||
project_root = os.path.dirname(app_dir) # redbear-mem directory
|
||||
from pathlib import Path
|
||||
project_root = str(Path(__file__).resolve().parents[2]) # api directory
|
||||
log_path = os.path.join(project_root, "logs", "agent_service.log")
|
||||
|
||||
# Check if file exists before starting stream
|
||||
@@ -265,13 +265,13 @@ class MemoryAgentService:
|
||||
logger.info("Log streaming completed, cleaning up resources")
|
||||
# LogStreamer uses context manager for file handling, so cleanup is automatic
|
||||
|
||||
async def write_memory(self, group_id: str, messages: list[dict], config_id: Optional[str], db: Session, storage_type: str, user_rag_memory_id: str) -> str:
|
||||
async def write_memory(self, end_user_id: str, messages: list[dict], config_id: Optional[uuid.UUID]|int, db: Session, storage_type: str, user_rag_memory_id: str) -> str:
|
||||
"""
|
||||
Process write operation with config_id
|
||||
|
||||
Args:
|
||||
group_id: Group identifier (also used as end_user_id)
|
||||
messages: Structured message list [{"role": "user", "content": "..."}, ...]
|
||||
end_user_id: Group identifier (also used as end_user_id)
|
||||
message: Message to write
|
||||
config_id: Configuration ID from database
|
||||
db: SQLAlchemy database session
|
||||
storage_type: Storage type (neo4j or rag)
|
||||
@@ -286,15 +286,15 @@ class MemoryAgentService:
|
||||
# Resolve config_id if None using end_user's connected config
|
||||
if config_id is None:
|
||||
try:
|
||||
connected_config = get_end_user_connected_config(group_id, db)
|
||||
connected_config = get_end_user_connected_config(end_user_id, db)
|
||||
config_id = connected_config.get("memory_config_id")
|
||||
if config_id is None:
|
||||
raise ValueError(f"No memory configuration found for end_user {group_id}. Please ensure the user has a connected memory configuration.")
|
||||
raise ValueError(f"No memory configuration found for end_user {end_user_id}. Please ensure the user has a connected memory configuration.")
|
||||
except Exception as e:
|
||||
if "No memory configuration found" in str(e):
|
||||
raise
|
||||
logger.error(f"Failed to get connected config for end_user {group_id}: {e}")
|
||||
raise ValueError(f"Unable to determine memory configuration for end_user {group_id}: {e}")
|
||||
raise # Re-raise our specific error
|
||||
logger.error(f"Failed to get connected config for end_user {end_user_id}: {e}")
|
||||
raise ValueError(f"Unable to determine memory configuration for end_user {end_user_id}: {e}")
|
||||
|
||||
import time
|
||||
start_time = time.time()
|
||||
@@ -314,7 +314,7 @@ class MemoryAgentService:
|
||||
# Log failed operation
|
||||
if audit_logger:
|
||||
duration = time.time() - start_time
|
||||
audit_logger.log_operation(operation="WRITE", config_id=config_id, group_id=group_id, success=False, duration=duration, error=error_msg)
|
||||
audit_logger.log_operation(operation="WRITE", config_id=config_id, end_user_id=end_user_id, success=False, duration=duration, error=error_msg)
|
||||
|
||||
raise ValueError(error_msg)
|
||||
|
||||
@@ -322,24 +322,25 @@ class MemoryAgentService:
|
||||
if storage_type == "rag":
|
||||
# For RAG storage, convert messages to single string
|
||||
message_text = "\n".join([f"{msg['role']}: {msg['content']}" for msg in messages])
|
||||
result = await write_rag(group_id, message_text, user_rag_memory_id)
|
||||
result = await write_rag(end_user_id, message_text, user_rag_memory_id)
|
||||
return result
|
||||
else:
|
||||
async with make_write_graph() as graph:
|
||||
config = {"configurable": {"thread_id": group_id}}
|
||||
config = {"configurable": {"thread_id": end_user_id}}
|
||||
# Convert structured messages to LangChain messages
|
||||
langchain_messages = []
|
||||
for msg in messages:
|
||||
if msg['role'] == 'user':
|
||||
langchain_messages.append(HumanMessage(content=msg['content']))
|
||||
elif msg['role'] == 'assistant':
|
||||
from langchain_core.messages import AIMessage
|
||||
langchain_messages.append(AIMessage(content=msg['content']))
|
||||
|
||||
print(100*'-')
|
||||
print(langchain_messages)
|
||||
print(100*'-')
|
||||
# 初始状态 - 包含所有必要字段
|
||||
initial_state = {
|
||||
"messages": langchain_messages,
|
||||
"group_id": group_id,
|
||||
"end_user_id": end_user_id,
|
||||
"memory_config": memory_config
|
||||
}
|
||||
|
||||
@@ -356,14 +357,14 @@ class MemoryAgentService:
|
||||
contents = massages.get('write_result')
|
||||
# Convert messages back to string for logging
|
||||
message_text = "\n".join([f"{msg['role']}: {msg['content']}" for msg in messages])
|
||||
return self.writer_messages_deal(massagesstatus, start_time, group_id, config_id, message_text, contents)
|
||||
return self.writer_messages_deal(massagesstatus, start_time, end_user_id, config_id, message_text, contents)
|
||||
except Exception as e:
|
||||
# Ensure proper error handling and logging
|
||||
error_msg = f"Write operation failed: {str(e)}"
|
||||
logger.error(error_msg)
|
||||
if audit_logger:
|
||||
duration = time.time() - start_time
|
||||
audit_logger.log_operation(operation="WRITE", config_id=config_id, group_id=group_id, success=False, duration=duration, error=error_msg)
|
||||
audit_logger.log_operation(operation="WRITE", config_id=config_id, end_user_id=end_user_id, success=False, duration=duration, error=error_msg)
|
||||
raise ValueError(error_msg)
|
||||
|
||||
|
||||
@@ -371,15 +372,14 @@ class MemoryAgentService:
|
||||
|
||||
async def read_memory(
|
||||
self,
|
||||
group_id: str,
|
||||
end_user_id: str,
|
||||
message: str,
|
||||
history: List[Dict],
|
||||
search_switch: str,
|
||||
config_id: Optional[str],
|
||||
config_id: Optional[uuid.UUID]|int,
|
||||
db: Session,
|
||||
storage_type: str,
|
||||
user_rag_memory_id: str
|
||||
) -> Dict:
|
||||
user_rag_memory_id: str) -> Dict:
|
||||
"""
|
||||
Process read operation with config_id
|
||||
|
||||
@@ -389,7 +389,7 @@ class MemoryAgentService:
|
||||
- "2": Direct answer based on context
|
||||
|
||||
Args:
|
||||
group_id: Group identifier (also used as end_user_id)
|
||||
end_user_id: Group identifier (also used as end_user_id)
|
||||
message: User message
|
||||
history: Conversation history
|
||||
search_switch: Search mode switch
|
||||
@@ -407,22 +407,22 @@ class MemoryAgentService:
|
||||
|
||||
import time
|
||||
start_time = time.time()
|
||||
logger.info(f"[PERF] read_memory started for group_id={group_id}, search_switch={search_switch}")
|
||||
ori_message= message
|
||||
|
||||
# Resolve config_id if None using end_user's connected config
|
||||
if config_id is None:
|
||||
try:
|
||||
connected_config = get_end_user_connected_config(group_id, db)
|
||||
connected_config = get_end_user_connected_config(end_user_id, db)
|
||||
config_id = connected_config.get("memory_config_id")
|
||||
if config_id is None:
|
||||
raise ValueError(f"No memory configuration found for end_user {group_id}. Please ensure the user has a connected memory configuration.")
|
||||
raise ValueError(f"No memory configuration found for end_user {end_user_id}. Please ensure the user has a connected memory configuration.")
|
||||
except Exception as e:
|
||||
if "No memory configuration found" in str(e):
|
||||
raise # Re-raise our specific error
|
||||
logger.error(f"Failed to get connected config for end_user {group_id}: {e}")
|
||||
raise ValueError(f"Unable to determine memory configuration for end_user {group_id}: {e}")
|
||||
logger.error(f"Failed to get connected config for end_user {end_user_id}: {e}")
|
||||
raise ValueError(f"Unable to determine memory configuration for end_user {end_user_id}: {e}")
|
||||
|
||||
logger.info(f"Read operation for group {group_id} with config_id {config_id}")
|
||||
logger.info(f"Read operation for group {end_user_id} with config_id {config_id}")
|
||||
|
||||
# 导入审计日志记录器
|
||||
try:
|
||||
@@ -450,7 +450,7 @@ class MemoryAgentService:
|
||||
audit_logger.log_operation(
|
||||
operation="READ",
|
||||
config_id=config_id,
|
||||
group_id=group_id,
|
||||
end_user_id=end_user_id,
|
||||
success=False,
|
||||
duration=duration,
|
||||
error=error_msg
|
||||
@@ -460,16 +460,16 @@ class MemoryAgentService:
|
||||
|
||||
# Step 2: Prepare history
|
||||
history.append({"role": "user", "content": message})
|
||||
logger.debug(f"Group ID:{group_id}, Message:{message}, History:{history}, Config ID:{config_id}")
|
||||
logger.debug(f"Group ID:{end_user_id}, Message:{message}, History:{history}, Config ID:{config_id}")
|
||||
|
||||
# Step 3: Initialize MCP client and execute read workflow
|
||||
graph_exec_start = time.time()
|
||||
try:
|
||||
async with make_read_graph() as graph:
|
||||
config = {"configurable": {"thread_id": group_id}}
|
||||
config = {"configurable": {"thread_id": end_user_id}}
|
||||
# 初始状态 - 包含所有必要字段
|
||||
initial_state = {"messages": [HumanMessage(content=message)], "search_switch": search_switch,
|
||||
"group_id": group_id
|
||||
"end_user_id": end_user_id
|
||||
, "storage_type": storage_type, "user_rag_memory_id": user_rag_memory_id,
|
||||
"memory_config": memory_config}
|
||||
# 获取节点更新信息
|
||||
@@ -544,9 +544,8 @@ class MemoryAgentService:
|
||||
if intermediate_type == "search_result":
|
||||
query = intermediate.get('query', '')
|
||||
raw_results = intermediate.get('raw_results', {})
|
||||
reranked_results = raw_results.get('reranked_results', [])
|
||||
|
||||
try:
|
||||
reranked_results = raw_results.get('reranked_results', [])
|
||||
statements = [statement['statement'] for statement in reranked_results.get('statements', [])]
|
||||
except Exception:
|
||||
statements = []
|
||||
@@ -565,13 +564,13 @@ class MemoryAgentService:
|
||||
if '信息不足,无法回答。' != str(summary) and str(search_switch).strip() != "2":
|
||||
# 使用 upsert 方法
|
||||
repo.upsert(
|
||||
end_user_id=group_id,
|
||||
messages=message,
|
||||
end_user_id=end_user_id,
|
||||
messages=ori_message,
|
||||
aimessages=summary,
|
||||
retrieved_content=retrieved_content,
|
||||
search_switch=str(search_switch)
|
||||
)
|
||||
logger.info(f"成功保存短期记忆: group_id={group_id}, search_switch={search_switch}")
|
||||
logger.info(f"成功保存短期记忆: end_user_id={end_user_id}, search_switch={search_switch}")
|
||||
else:
|
||||
logger.debug(f"跳过保存短期记忆: summary={summary[:50] if summary else 'None'}, search_switch={search_switch}")
|
||||
|
||||
@@ -587,7 +586,7 @@ class MemoryAgentService:
|
||||
audit_logger.log_operation(
|
||||
operation="READ",
|
||||
config_id=config_id,
|
||||
group_id=group_id,
|
||||
end_user_id=end_user_id,
|
||||
success=True,
|
||||
duration=duration
|
||||
)
|
||||
@@ -599,20 +598,20 @@ class MemoryAgentService:
|
||||
except Exception as e:
|
||||
# Ensure proper error handling and logging
|
||||
error_msg = f"Read operation failed: {str(e)}"
|
||||
total_time = time.time() - start_time
|
||||
logger.error(f"[PERF] read_memory failed after {total_time:.4f}s: {error_msg}")
|
||||
logger.error(error_msg)
|
||||
if audit_logger:
|
||||
duration = time.time() - start_time
|
||||
audit_logger.log_operation(
|
||||
operation="READ",
|
||||
config_id=config_id,
|
||||
group_id=group_id,
|
||||
end_user_id=end_user_id,
|
||||
success=False,
|
||||
duration=duration,
|
||||
error=error_msg
|
||||
)
|
||||
raise ValueError(error_msg)
|
||||
|
||||
|
||||
def get_messages_list(self, user_input: Write_UserInput) -> list[dict]:
|
||||
"""
|
||||
Get standardized message list from user input.
|
||||
@@ -657,7 +656,7 @@ class MemoryAgentService:
|
||||
logger.info(f"Validation successful: Structured message list, count: {len(user_input.messages)}")
|
||||
return user_input.messages
|
||||
|
||||
async def classify_message_type(self, message: str, config_id: int, db: Session) -> Dict:
|
||||
async def classify_message_type(self, message: str, config_id: UUID, db: Session) -> Dict:
|
||||
"""
|
||||
Determine the type of user message (read or write)
|
||||
Updated to eliminate global variables in favor of explicit parameters.
|
||||
@@ -672,6 +671,8 @@ class MemoryAgentService:
|
||||
"""
|
||||
logger.info("Classifying message type")
|
||||
|
||||
|
||||
|
||||
# Load configuration to get LLM model ID
|
||||
config_service = MemoryConfigService(db)
|
||||
memory_config = config_service.load_memory_config(
|
||||
@@ -682,9 +683,9 @@ class MemoryAgentService:
|
||||
status = await status_typle(message, memory_config.llm_model_id)
|
||||
logger.debug(f"Message type: {status}")
|
||||
return status
|
||||
|
||||
async def generate_summary_from_retrieve(
|
||||
self,
|
||||
end_user_id: str,
|
||||
retrieve_info: str,
|
||||
history: List[Dict],
|
||||
query: str,
|
||||
@@ -706,6 +707,18 @@ class MemoryAgentService:
|
||||
Returns:
|
||||
生成的答案文本
|
||||
"""
|
||||
if config_id is None:
|
||||
try:
|
||||
config_id = get_end_user_connected_config(end_user_id, db)
|
||||
config_id = config_id.get('memory_config_id')
|
||||
if config_id is None:
|
||||
raise ValueError(
|
||||
f"No memory configuration found for end_user {end_user_id}. Please ensure the user has a connected memory configuration.")
|
||||
except Exception as e:
|
||||
if "No memory configuration found" in str(e):
|
||||
raise # Re-raise our specific error
|
||||
logger.error(f"Failed to get connected config for end_user {end_user_id}: {e}")
|
||||
raise ValueError(f"Unable to determine memory configuration for end_user {end_user_id}: {e}")
|
||||
logger.info(f"Generating summary from retrieve info for query: {query[:50]}...")
|
||||
|
||||
try:
|
||||
@@ -731,7 +744,7 @@ class MemoryAgentService:
|
||||
state=state,
|
||||
history=history,
|
||||
retrieve_info=retrieve_info,
|
||||
template_name='Retrieve_Summary_prompt.jinja2',
|
||||
template_name='direct_summary_prompt.jinja2',
|
||||
operation_name='retrieve_summary',
|
||||
response_model=RetrieveSummaryResponse,
|
||||
search_mode="1"
|
||||
@@ -755,7 +768,7 @@ class MemoryAgentService:
|
||||
"""
|
||||
统计知识库类型分布,包含:
|
||||
1. PostgreSQL 中的知识库类型:General, Web, Third-party, Folder(根据 workspace_id 过滤)
|
||||
2. Neo4j 中的 memory 类型(仅统计 Chunk 数量,根据 end_user_id/group_id 过滤)
|
||||
2. Neo4j 中的 memory 类型(仅统计 Chunk 数量,根据 end_user_id/end_user_id 过滤)
|
||||
3. total: 所有类型的总和
|
||||
|
||||
参数:
|
||||
@@ -841,11 +854,11 @@ class MemoryAgentService:
|
||||
for end_user in end_users:
|
||||
end_user_id_str = str(end_user.id)
|
||||
memory_query = """
|
||||
MATCH (n:Chunk) WHERE n.group_id = $group_id RETURN count(n) AS Count
|
||||
MATCH (n:Chunk) WHERE n.end_user_id = $end_user_id RETURN count(n) AS Count
|
||||
"""
|
||||
neo4j_result = await _neo4j_connector.execute_query(
|
||||
memory_query,
|
||||
group_id=end_user_id_str,
|
||||
end_user_id=end_user_id_str,
|
||||
)
|
||||
chunk_count = neo4j_result[0]["Count"] if neo4j_result else 0
|
||||
total_chunks += chunk_count
|
||||
@@ -885,7 +898,7 @@ class MemoryAgentService:
|
||||
获取指定用户的热门记忆标签
|
||||
|
||||
参数:
|
||||
- end_user_id: 用户ID(可选),对应Neo4j中的group_id字段
|
||||
- end_user_id: 用户ID(可选),对应Neo4j中的end_user_id字段
|
||||
- limit: 返回标签数量限制
|
||||
|
||||
返回格式:
|
||||
@@ -895,7 +908,7 @@ class MemoryAgentService:
|
||||
]
|
||||
"""
|
||||
try:
|
||||
# by_user=False 表示按 group_id 查询(在Neo4j中,group_id就是用户维度)
|
||||
# by_user=False 表示按 end_user_id 查询(在Neo4j中,end_user_id就是用户维度)
|
||||
tags = await get_hot_memory_tags(end_user_id, limit=limit, by_user=False)
|
||||
payload=[]
|
||||
for tag, freq in tags:
|
||||
@@ -970,21 +983,21 @@ class MemoryAgentService:
|
||||
# 查询该用户的语句
|
||||
query = (
|
||||
"MATCH (s:Statement) "
|
||||
"WHERE ($group_id IS NULL OR s.group_id = $group_id) AND s.statement IS NOT NULL "
|
||||
"WHERE ($end_user_id IS NULL OR s.end_user_id = $end_user_id) AND s.statement IS NOT NULL "
|
||||
"RETURN s.statement AS statement "
|
||||
"ORDER BY s.created_at DESC LIMIT 100"
|
||||
)
|
||||
rows = await connector.execute_query(query, group_id=end_user_id)
|
||||
rows = await connector.execute_query(query, end_user_id=end_user_id)
|
||||
statements = [r.get("statement", "") for r in rows if r.get("statement")]
|
||||
|
||||
# 查询该用户的热门实体
|
||||
entity_query = (
|
||||
"MATCH (e:ExtractedEntity) "
|
||||
"WHERE ($group_id IS NULL OR e.group_id = $group_id) AND e.entity_type <> '人物' AND e.name IS NOT NULL "
|
||||
"WHERE ($end_user_id IS NULL OR e.end_user_id = $end_user_id) AND e.entity_type <> '人物' AND e.name IS NOT NULL "
|
||||
"RETURN e.name AS name, count(e) AS frequency "
|
||||
"ORDER BY frequency DESC LIMIT 20"
|
||||
)
|
||||
entity_rows = await connector.execute_query(entity_query, group_id=end_user_id)
|
||||
entity_rows = await connector.execute_query(entity_query, end_user_id=end_user_id)
|
||||
entities = [f"{r['name']} ({r['frequency']})" for r in entity_rows]
|
||||
|
||||
await connector.close()
|
||||
@@ -1037,14 +1050,14 @@ class MemoryAgentService:
|
||||
names_to_exclude = ['AI', 'Caroline', 'Melanie', 'Jon', 'Gina', '用户', 'AI助手', 'John', 'Maria']
|
||||
hot_tag_query = (
|
||||
"MATCH (e:ExtractedEntity) "
|
||||
"WHERE ($group_id IS NULL OR e.group_id = $group_id) AND e.entity_type <> '人物' "
|
||||
"WHERE ($end_user_id IS NULL OR e.end_user_id = $end_user_id) AND e.entity_type <> '人物' "
|
||||
"AND e.name IS NOT NULL AND NOT e.name IN $names_to_exclude "
|
||||
"RETURN e.name AS name, count(e) AS frequency "
|
||||
"ORDER BY frequency DESC LIMIT 4"
|
||||
)
|
||||
hot_tag_rows = await connector.execute_query(
|
||||
hot_tag_query,
|
||||
group_id=end_user_id,
|
||||
end_user_id=end_user_id,
|
||||
names_to_exclude=names_to_exclude
|
||||
)
|
||||
await connector.close()
|
||||
@@ -1079,9 +1092,8 @@ class MemoryAgentService:
|
||||
logger.info("Starting log content streaming")
|
||||
|
||||
# Get log file path - use project root directory
|
||||
current_file = os.path.abspath(__file__) # app/services/memory_agent_service.py
|
||||
app_dir = os.path.dirname(os.path.dirname(current_file)) # app directory
|
||||
project_root = os.path.dirname(app_dir) # redbear-mem directory
|
||||
from pathlib import Path
|
||||
project_root = str(Path(__file__).resolve().parents[2]) # api directory
|
||||
log_path = os.path.join(project_root, "logs", "agent_service.log")
|
||||
|
||||
# Check if file exists before starting stream
|
||||
@@ -1179,6 +1191,16 @@ def get_end_user_connected_config(end_user_id: str, db: Session) -> Dict[str, An
|
||||
|
||||
# 3. 从 config 中提取 memory_config_id
|
||||
config = latest_release.config or {}
|
||||
|
||||
# 如果 config 是字符串,解析为字典
|
||||
if isinstance(config, str):
|
||||
import json
|
||||
try:
|
||||
config = json.loads(config)
|
||||
except json.JSONDecodeError:
|
||||
logger.warning(f"Failed to parse config JSON for release {latest_release.id}")
|
||||
config = {}
|
||||
|
||||
memory_obj = config.get('memory', {})
|
||||
memory_config_id = memory_obj.get('memory_content') if isinstance(memory_obj, dict) else None
|
||||
|
||||
@@ -1217,7 +1239,7 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
|
||||
"""
|
||||
from app.models.app_release_model import AppRelease
|
||||
from app.models.end_user_model import EndUser
|
||||
from app.models.data_config_model import DataConfig
|
||||
from app.models.memory_config_model import MemoryConfig
|
||||
from sqlalchemy import select
|
||||
|
||||
logger.info(f"Batch getting connected configs for {len(end_user_ids)} end_users")
|
||||
@@ -1230,10 +1252,10 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
|
||||
|
||||
# 1. 批量查询所有 end_user 及其 app_id
|
||||
end_users = db.query(EndUser).filter(EndUser.id.in_(end_user_ids)).all()
|
||||
|
||||
|
||||
# 创建 end_user_id -> app_id 的映射
|
||||
user_to_app = {str(eu.id): eu.app_id for eu in end_users}
|
||||
|
||||
|
||||
# 记录未找到的用户
|
||||
found_user_ids = set(user_to_app.keys())
|
||||
missing_user_ids = set(end_user_ids) - found_user_ids
|
||||
@@ -1243,7 +1265,7 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
|
||||
result[user_id] = {"memory_config_id": None, "memory_config_name": None}
|
||||
|
||||
# 2. 批量获取所有相关应用的最新发布版本
|
||||
app_ids = list(user_to_app.values())
|
||||
app_ids = list(set(user_to_app.values()))
|
||||
if not app_ids:
|
||||
return result
|
||||
|
||||
@@ -1263,6 +1285,8 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
|
||||
|
||||
# 3. 收集所有 memory_config_id 并批量查询配置名称
|
||||
memory_config_ids = []
|
||||
old_config_ids = [] # 存储旧的整数ID
|
||||
|
||||
for end_user_id, app_id in user_to_app.items():
|
||||
release = app_to_release.get(app_id)
|
||||
if release:
|
||||
@@ -1270,18 +1294,42 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
|
||||
memory_obj = config.get('memory', {})
|
||||
memory_config_id = memory_obj.get('memory_content') if isinstance(memory_obj, dict) else None
|
||||
if memory_config_id:
|
||||
memory_config_ids.append(memory_config_id)
|
||||
|
||||
# 判断是否为UUID格式
|
||||
if len(str(memory_config_id))>=5:
|
||||
uuid.UUID(str(memory_config_id))
|
||||
memory_config_ids.append(memory_config_id)
|
||||
else:
|
||||
old_config_ids.append(str(memory_config_id))
|
||||
|
||||
# 批量查询 memory_config_name
|
||||
config_id_to_name = {}
|
||||
|
||||
# 记录分类结果
|
||||
if memory_config_ids or old_config_ids:
|
||||
logger.info(f"Collected {len(memory_config_ids)} UUID config_ids and {len(old_config_ids)} old integer config_ids")
|
||||
if old_config_ids:
|
||||
logger.debug(f"Old config IDs: {old_config_ids}")
|
||||
|
||||
# 查询新的UUID格式的config_id
|
||||
if memory_config_ids:
|
||||
memory_configs = db.query(DataConfig).filter(DataConfig.config_id.in_(memory_config_ids)).all()
|
||||
config_id_to_name = {str(mc.config_id): mc.config_name for mc in memory_configs}
|
||||
memory_configs = db.query(MemoryConfig).filter(MemoryConfig.config_id.in_(memory_config_ids)).all()
|
||||
config_id_to_name.update({str(mc.config_id): mc.config_name for mc in memory_configs})
|
||||
|
||||
# 查询旧的整数ID(通过config_id_old字段)
|
||||
if old_config_ids:
|
||||
old_memory_configs = db.query(MemoryConfig).filter(MemoryConfig.config_id_old.in_(old_config_ids)).all()
|
||||
# 使用config_id_old作为key,这样后面查找时能匹配上
|
||||
config_id_to_name.update({str(mc.config_id_old): mc.config_name for mc in old_memory_configs})
|
||||
# 同时也添加config_id作为key,方便后续使用
|
||||
for mc in old_memory_configs:
|
||||
if mc.config_id_old:
|
||||
config_id_to_name[str(mc.config_id)] = mc.config_name
|
||||
logger.info(f"Found {len(old_memory_configs)} configs for old IDs")
|
||||
|
||||
# 4. 构建最终结果
|
||||
for end_user_id, app_id in user_to_app.items():
|
||||
release = app_to_release.get(app_id)
|
||||
|
||||
|
||||
if not release:
|
||||
logger.warning(f"No active release found for app: {app_id} (end_user: {end_user_id})")
|
||||
result[end_user_id] = {"memory_config_id": None, "memory_config_name": None}
|
||||
@@ -1292,7 +1340,7 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
|
||||
memory_obj = config.get('memory', {})
|
||||
memory_config_id = memory_obj.get('memory_content') if isinstance(memory_obj, dict) else None
|
||||
|
||||
# 获取配置名称
|
||||
# 获取配置名称(使用字符串形式的ID进行查找,兼容新旧格式)
|
||||
memory_config_name = config_id_to_name.get(str(memory_config_id)) if memory_config_id else None
|
||||
|
||||
result[end_user_id] = {
|
||||
|
||||
@@ -25,7 +25,7 @@ class MemoryAPIService:
|
||||
|
||||
This service provides a thin layer that:
|
||||
1. Validates end_user exists and belongs to the authorized workspace
|
||||
2. Maps end_user_id to group_id for memory operations
|
||||
2. Maps end_user_id to end_user_id for memory operations
|
||||
3. Delegates to MemoryAgentService for actual memory read/write operations
|
||||
"""
|
||||
|
||||
@@ -68,7 +68,7 @@ class MemoryAPIService:
|
||||
)
|
||||
|
||||
end_user = self.db.query(EndUser).filter(EndUser.id == end_user_uuid).first()
|
||||
|
||||
|
||||
if not end_user:
|
||||
logger.warning(f"End user not found: {end_user_id}")
|
||||
raise ResourceNotFoundException(
|
||||
@@ -77,7 +77,10 @@ class MemoryAPIService:
|
||||
)
|
||||
|
||||
# Verify end_user belongs to the workspace via App relationship
|
||||
app = self.db.query(App).filter(App.id == end_user.app_id).first()
|
||||
app = self.db.query(App).filter(
|
||||
App.id == end_user.app_id,
|
||||
App.is_active.is_(True)
|
||||
).first()
|
||||
|
||||
if not app:
|
||||
logger.warning(f"App not found for end_user: {end_user_id}")
|
||||
@@ -115,7 +118,7 @@ class MemoryAPIService:
|
||||
|
||||
Args:
|
||||
workspace_id: Workspace ID for resource validation
|
||||
end_user_id: End user identifier (used as group_id)
|
||||
end_user_id: End user identifier (used as end_user_id)
|
||||
message: Message content to store
|
||||
config_id: Optional memory configuration ID
|
||||
storage_type: Storage backend (neo4j or rag)
|
||||
@@ -133,14 +136,13 @@ class MemoryAPIService:
|
||||
# Validate end_user exists and belongs to workspace
|
||||
self.validate_end_user(end_user_id, workspace_id)
|
||||
|
||||
# Use end_user_id as group_id for memory operations
|
||||
group_id = end_user_id
|
||||
# Use end_user_id as end_user_id for memory operations
|
||||
|
||||
try:
|
||||
# Delegate to MemoryAgentService
|
||||
result = await MemoryAgentService().write_memory(
|
||||
group_id=group_id,
|
||||
message=message,
|
||||
end_user_id=end_user_id,
|
||||
messages=message,
|
||||
config_id=config_id,
|
||||
db=self.db,
|
||||
storage_type=storage_type,
|
||||
@@ -186,7 +188,7 @@ class MemoryAPIService:
|
||||
|
||||
Args:
|
||||
workspace_id: Workspace ID for resource validation
|
||||
end_user_id: End user identifier (used as group_id)
|
||||
end_user_id: End user identifier (used as end_user_id)
|
||||
message: Query message
|
||||
search_switch: Search mode (0=deep search with verification, 1=deep search, 2=fast search)
|
||||
config_id: Optional memory configuration ID
|
||||
@@ -205,13 +207,13 @@ class MemoryAPIService:
|
||||
# Validate end_user exists and belongs to workspace
|
||||
self.validate_end_user(end_user_id, workspace_id)
|
||||
|
||||
# Use end_user_id as group_id for memory operations
|
||||
group_id = end_user_id
|
||||
# Use end_user_id as end_user_id for memory operations
|
||||
|
||||
|
||||
try:
|
||||
# Delegate to MemoryAgentService
|
||||
result = await MemoryAgentService().read_memory(
|
||||
group_id=group_id,
|
||||
end_user_id=end_user_id,
|
||||
message=message,
|
||||
history=[],
|
||||
search_switch=search_switch,
|
||||
|
||||
@@ -326,7 +326,7 @@ class MemoryBaseService:
|
||||
|
||||
Args:
|
||||
summary_id: Summary节点的ID
|
||||
end_user_id: 终端用户ID (group_id)
|
||||
end_user_id: 终端用户ID (end_user_id)
|
||||
|
||||
Returns:
|
||||
最大emotion_intensity对应的emotion_type,如果没有则返回None
|
||||
@@ -334,7 +334,7 @@ class MemoryBaseService:
|
||||
try:
|
||||
query = """
|
||||
MATCH (s:MemorySummary)
|
||||
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
|
||||
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
|
||||
MATCH (s)-[:DERIVED_FROM_STATEMENT]->(stmt:Statement)
|
||||
WHERE stmt.emotion_type IS NOT NULL
|
||||
AND stmt.emotion_intensity IS NOT NULL
|
||||
@@ -347,7 +347,7 @@ class MemoryBaseService:
|
||||
result = await self.neo4j_connector.execute_query(
|
||||
query,
|
||||
summary_id=summary_id,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
if result and len(result) > 0:
|
||||
@@ -381,10 +381,10 @@ class MemoryBaseService:
|
||||
if end_user_id:
|
||||
query = """
|
||||
MATCH (n:MemorySummary)
|
||||
WHERE n.group_id = $group_id
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
RETURN count(n) as count
|
||||
"""
|
||||
result = await self.neo4j_connector.execute_query(query, group_id=end_user_id)
|
||||
result = await self.neo4j_connector.execute_query(query, end_user_id=end_user_id)
|
||||
else:
|
||||
query = """
|
||||
MATCH (n:MemorySummary)
|
||||
@@ -423,12 +423,12 @@ class MemoryBaseService:
|
||||
if end_user_id:
|
||||
semantic_query = """
|
||||
MATCH (e:ExtractedEntity)
|
||||
WHERE e.group_id = $group_id AND e.is_explicit_memory = true
|
||||
WHERE e.end_user_id = $end_user_id AND e.is_explicit_memory = true
|
||||
RETURN count(e) as count
|
||||
"""
|
||||
semantic_result = await self.neo4j_connector.execute_query(
|
||||
semantic_query,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
else:
|
||||
semantic_query = """
|
||||
@@ -519,7 +519,7 @@ class MemoryBaseService:
|
||||
"""
|
||||
|
||||
if end_user_id:
|
||||
query += " AND n.group_id = $group_id"
|
||||
query += " AND n.end_user_id = $end_user_id"
|
||||
|
||||
query += """
|
||||
RETURN sum(CASE WHEN n.activation_value IS NOT NULL AND n.activation_value < $threshold THEN 1 ELSE 0 END) as low_activation_nodes
|
||||
@@ -528,7 +528,7 @@ class MemoryBaseService:
|
||||
# 设置查询参数
|
||||
params = {'threshold': forgetting_threshold}
|
||||
if end_user_id:
|
||||
params['group_id'] = end_user_id
|
||||
params['end_user_id'] = end_user_id
|
||||
|
||||
# 执行查询
|
||||
result = await self.neo4j_connector.execute_query(query, **params)
|
||||
|
||||
@@ -7,14 +7,15 @@ This service eliminates code duplication between MemoryAgentService and MemorySt
|
||||
|
||||
import time
|
||||
from datetime import datetime
|
||||
|
||||
from app.models.memory_config_model import MemoryConfig as MemoryConfigModel
|
||||
from sqlalchemy import select
|
||||
from app.core.logging_config import get_config_logger, get_logger
|
||||
from app.core.validators.memory_config_validators import (
|
||||
validate_and_resolve_model_id,
|
||||
validate_embedding_model,
|
||||
validate_model_exists_and_active,
|
||||
)
|
||||
from app.repositories.data_config_repository import DataConfigRepository
|
||||
from app.repositories.memory_config_repository import MemoryConfigRepository
|
||||
from app.schemas.memory_config_schema import (
|
||||
ConfigurationError,
|
||||
InvalidConfigError,
|
||||
@@ -23,20 +24,24 @@ from app.schemas.memory_config_schema import (
|
||||
ModelNotFoundError,
|
||||
)
|
||||
from sqlalchemy.orm import Session
|
||||
from uuid import UUID
|
||||
|
||||
logger = get_logger(__name__)
|
||||
config_logger = get_config_logger()
|
||||
import uuid
|
||||
|
||||
|
||||
def _validate_config_id(config_id):
|
||||
"""Validate configuration ID format."""
|
||||
def _validate_config_id(config_id, db: Session = None):
|
||||
"""Validate configuration ID format (supports both UUID and integer)."""
|
||||
if isinstance(config_id, uuid.UUID):
|
||||
return config_id
|
||||
|
||||
if config_id is None:
|
||||
raise InvalidConfigError(
|
||||
"Configuration ID cannot be None",
|
||||
field_name="config_id",
|
||||
invalid_value=config_id,
|
||||
)
|
||||
|
||||
|
||||
if isinstance(config_id, int):
|
||||
if config_id <= 0:
|
||||
raise InvalidConfigError(
|
||||
@@ -44,27 +49,56 @@ def _validate_config_id(config_id):
|
||||
field_name="config_id",
|
||||
invalid_value=config_id,
|
||||
)
|
||||
# 如果提供了数据库会话,尝试通过 user_id 查询 config_id
|
||||
if db is not None:
|
||||
# 查询 user_id 匹配的记录
|
||||
stmt = select(MemoryConfigModel).where(MemoryConfigModel.config_id_old == str(config_id))
|
||||
result = db.execute(stmt).scalars().first()
|
||||
if result:
|
||||
logger.info(f"Found config_id {result.config_id} for user_id {config_id}")
|
||||
return result.config_id
|
||||
|
||||
return config_id
|
||||
|
||||
|
||||
if isinstance(config_id, str):
|
||||
config_id_stripped = config_id.strip()
|
||||
|
||||
# Try parsing as UUID first
|
||||
try:
|
||||
parsed_id = int(config_id.strip())
|
||||
return uuid.UUID(config_id_stripped)
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
# Fall back to integer parsing
|
||||
try:
|
||||
parsed_id = int(config_id_stripped)
|
||||
if parsed_id <= 0:
|
||||
raise InvalidConfigError(
|
||||
f"Configuration ID must be positive: {parsed_id}",
|
||||
field_name="config_id",
|
||||
invalid_value=config_id,
|
||||
)
|
||||
|
||||
# 如果提供了数据库会话,尝试通过 user_id 查询 config_id
|
||||
if db is not None:
|
||||
# 查询 user_id 匹配的记录
|
||||
stmt = select(MemoryConfigModel).where(MemoryConfigModel.user_id == str(parsed_id))
|
||||
result = db.execute(stmt).scalars().first()
|
||||
|
||||
if result:
|
||||
logger.info(f"Found config_id {result.config_id} for user_id {parsed_id}")
|
||||
return result.config_id
|
||||
|
||||
return parsed_id
|
||||
except ValueError:
|
||||
raise InvalidConfigError(
|
||||
f"Invalid configuration ID format: '{config_id}'",
|
||||
f"Invalid configuration ID format: '{config_id}' (must be UUID or positive integer)",
|
||||
field_name="config_id",
|
||||
invalid_value=config_id,
|
||||
)
|
||||
|
||||
|
||||
raise InvalidConfigError(
|
||||
f"Invalid type for configuration ID: expected int or str, got {type(config_id).__name__}",
|
||||
f"Invalid type for configuration ID: expected UUID, int or str, got {type(config_id).__name__}",
|
||||
field_name="config_id",
|
||||
invalid_value=config_id,
|
||||
)
|
||||
@@ -73,61 +107,61 @@ def _validate_config_id(config_id):
|
||||
class MemoryConfigService:
|
||||
"""
|
||||
Centralized service for memory configuration loading and validation.
|
||||
|
||||
|
||||
This class provides a single implementation of configuration loading logic
|
||||
that can be shared across multiple services, eliminating code duplication.
|
||||
|
||||
|
||||
Usage:
|
||||
config_service = MemoryConfigService(db)
|
||||
memory_config = config_service.load_memory_config(config_id)
|
||||
model_config = config_service.get_model_config(model_id)
|
||||
"""
|
||||
|
||||
|
||||
def __init__(self, db: Session):
|
||||
"""Initialize the service with a database session.
|
||||
|
||||
|
||||
Args:
|
||||
db: SQLAlchemy database session
|
||||
"""
|
||||
self.db = db
|
||||
|
||||
|
||||
def load_memory_config(
|
||||
self,
|
||||
config_id: int,
|
||||
config_id: UUID,
|
||||
service_name: str = "MemoryConfigService",
|
||||
) -> MemoryConfig:
|
||||
"""
|
||||
Load memory configuration from database by config_id.
|
||||
|
||||
|
||||
Args:
|
||||
config_id: Configuration ID from database
|
||||
config_id: Configuration ID (UUID) from database
|
||||
service_name: Name of the calling service (for logging purposes)
|
||||
|
||||
|
||||
Returns:
|
||||
MemoryConfig: Immutable configuration object
|
||||
|
||||
|
||||
Raises:
|
||||
ConfigurationError: If validation fails
|
||||
"""
|
||||
start_time = time.time()
|
||||
|
||||
|
||||
config_logger.info(
|
||||
"Starting memory configuration loading",
|
||||
extra={
|
||||
"operation": "load_memory_config",
|
||||
"service": service_name,
|
||||
"config_id": config_id,
|
||||
"config_id": str(config_id),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
logger.info(f"Loading memory configuration from database: config_id={config_id}")
|
||||
|
||||
|
||||
try:
|
||||
validated_config_id = _validate_config_id(config_id)
|
||||
|
||||
validated_config_id = _validate_config_id(config_id, self.db)
|
||||
|
||||
# Step 1: Get config and workspace
|
||||
db_query_start = time.time()
|
||||
result = DataConfigRepository.get_config_with_workspace(self.db, validated_config_id)
|
||||
result = MemoryConfigRepository.get_config_with_workspace(self.db, validated_config_id)
|
||||
db_query_time = time.time() - db_query_start
|
||||
logger.info(f"[PERF] Config+Workspace query: {db_query_time:.4f}s")
|
||||
if not result:
|
||||
@@ -136,18 +170,18 @@ class MemoryConfigService:
|
||||
"Configuration not found in database",
|
||||
extra={
|
||||
"operation": "load_memory_config",
|
||||
"config_id": validated_config_id,
|
||||
"config_id": str(config_id),
|
||||
"load_result": "not_found",
|
||||
"elapsed_ms": elapsed_ms,
|
||||
"service": service_name,
|
||||
},
|
||||
)
|
||||
raise ConfigurationError(
|
||||
f"Configuration {validated_config_id} not found in database"
|
||||
f"Configuration {config_id} not found in database"
|
||||
)
|
||||
|
||||
|
||||
memory_config, workspace = result
|
||||
|
||||
|
||||
# Step 2: Validate embedding model (returns both UUID and name)
|
||||
embed_start = time.time()
|
||||
embedding_uuid, embedding_name = validate_embedding_model(
|
||||
@@ -159,7 +193,7 @@ class MemoryConfigService:
|
||||
)
|
||||
embed_time = time.time() - embed_start
|
||||
logger.info(f"[PERF] Embedding validation: {embed_time:.4f}s")
|
||||
|
||||
|
||||
# Step 3: Resolve LLM model
|
||||
llm_start = time.time()
|
||||
llm_uuid, llm_name = validate_and_resolve_model_id(
|
||||
@@ -173,7 +207,7 @@ class MemoryConfigService:
|
||||
)
|
||||
llm_time = time.time() - llm_start
|
||||
logger.info(f"[PERF] LLM validation: {llm_time:.4f}s")
|
||||
|
||||
|
||||
# Step 4: Resolve optional rerank model
|
||||
rerank_start = time.time()
|
||||
rerank_uuid = None
|
||||
@@ -191,10 +225,10 @@ class MemoryConfigService:
|
||||
rerank_time = time.time() - rerank_start
|
||||
if memory_config.rerank_id:
|
||||
logger.info(f"[PERF] Rerank validation: {rerank_time:.4f}s")
|
||||
|
||||
|
||||
# Note: embedding_name is now returned from validate_embedding_model above
|
||||
# No need for redundant query!
|
||||
|
||||
|
||||
# Create immutable MemoryConfig object
|
||||
config = MemoryConfig(
|
||||
config_id=memory_config.config_id,
|
||||
@@ -235,9 +269,9 @@ class MemoryConfigService:
|
||||
pruning_scene=memory_config.pruning_scene or "education",
|
||||
pruning_threshold=float(memory_config.pruning_threshold) if memory_config.pruning_threshold is not None else 0.5,
|
||||
)
|
||||
|
||||
|
||||
elapsed_ms = (time.time() - start_time) * 1000
|
||||
|
||||
|
||||
config_logger.info(
|
||||
"Memory configuration loaded successfully",
|
||||
extra={
|
||||
@@ -250,13 +284,13 @@ class MemoryConfigService:
|
||||
"elapsed_ms": elapsed_ms,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
logger.info(f"Memory configuration loaded successfully: {config.config_name}")
|
||||
return config
|
||||
|
||||
|
||||
except Exception as e:
|
||||
elapsed_ms = (time.time() - start_time) * 1000
|
||||
|
||||
|
||||
config_logger.error(
|
||||
"Failed to load memory configuration",
|
||||
extra={
|
||||
@@ -270,7 +304,7 @@ class MemoryConfigService:
|
||||
},
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
|
||||
logger.error(f"Failed to load memory configuration {config_id}: {e}")
|
||||
if isinstance(e, (ConfigurationError, ValueError)):
|
||||
raise
|
||||
@@ -304,7 +338,7 @@ class MemoryConfigService:
|
||||
"provider": api_config.provider,
|
||||
"api_key": api_config.api_key,
|
||||
"base_url": api_config.api_base,
|
||||
"model_config_id": api_config.model_config_id,
|
||||
"model_config_id": str(config.id),
|
||||
"type": config.type,
|
||||
"timeout": settings.LLM_TIMEOUT,
|
||||
"max_retries": settings.LLM_MAX_RETRIES,
|
||||
@@ -336,7 +370,7 @@ class MemoryConfigService:
|
||||
"provider": api_config.provider,
|
||||
"api_key": api_config.api_key,
|
||||
"base_url": api_config.api_base,
|
||||
"model_config_id": api_config.model_config_id,
|
||||
"model_config_id": str(config.id),
|
||||
"type": config.type,
|
||||
"timeout": 120.0,
|
||||
"max_retries": 5,
|
||||
|
||||
@@ -53,18 +53,28 @@ def get_workspace_end_users(
|
||||
workspace_id: uuid.UUID,
|
||||
current_user: User
|
||||
) -> List[EndUser]:
|
||||
"""获取工作空间的所有宿主"""
|
||||
"""获取工作空间的所有宿主(优化版本:减少数据库查询次数)"""
|
||||
business_logger.info(f"获取工作空间宿主列表: workspace_id={workspace_id}, 操作者: {current_user.username}")
|
||||
|
||||
try:
|
||||
# 查询应用(ORM)并转换为 Pydantic 模型
|
||||
# 查询应用(ORM)
|
||||
apps_orm = app_repository.get_apps_by_workspace_id(db, workspace_id)
|
||||
apps = [AppSchema.model_validate(h) for h in apps_orm]
|
||||
app_ids = [app.id for app in apps]
|
||||
end_users = []
|
||||
for app_id in app_ids:
|
||||
end_user_orm_list = end_user_repository.get_end_users_by_app_id(db, app_id)
|
||||
end_users.extend([EndUserSchema.model_validate(h) for h in end_user_orm_list])
|
||||
|
||||
if not apps_orm:
|
||||
business_logger.info("工作空间下没有应用")
|
||||
return []
|
||||
|
||||
# 提取所有 app_id
|
||||
app_ids = [app.id for app in apps_orm]
|
||||
|
||||
# 批量查询所有 end_users(一次查询而非循环查询)
|
||||
from app.models.end_user_model import EndUser as EndUserModel
|
||||
end_users_orm = db.query(EndUserModel).filter(
|
||||
EndUserModel.app_id.in_(app_ids)
|
||||
).all()
|
||||
|
||||
# 转换为 Pydantic 模型(只在需要时转换)
|
||||
end_users = [EndUserSchema.model_validate(eu) for eu in end_users_orm]
|
||||
|
||||
business_logger.info(f"成功获取 {len(end_users)} 个宿主记录")
|
||||
return end_users
|
||||
@@ -414,6 +424,67 @@ def get_current_user_total_chunk(
|
||||
business_logger.error(f"获取用户总chunk数失败: end_user_id={end_user_id} - {str(e)}")
|
||||
raise
|
||||
|
||||
|
||||
def get_users_total_chunk_batch(
|
||||
end_user_ids: List[str],
|
||||
db: Session,
|
||||
current_user: User
|
||||
) -> dict:
|
||||
"""
|
||||
批量获取多个用户的总chunk数(性能优化版本)
|
||||
|
||||
Args:
|
||||
end_user_ids: 用户ID列表
|
||||
db: 数据库会话
|
||||
current_user: 当前用户
|
||||
|
||||
Returns:
|
||||
字典,key为end_user_id,value为chunk总数
|
||||
格式: {"user_id_1": 100, "user_id_2": 50, ...}
|
||||
"""
|
||||
business_logger.info(f"批量获取 {len(end_user_ids)} 个用户的总chunk数, 操作者: {current_user.username}")
|
||||
|
||||
try:
|
||||
from app.models.document_model import Document
|
||||
from sqlalchemy import func, case
|
||||
|
||||
if not end_user_ids:
|
||||
return {}
|
||||
|
||||
# 构造所有文件名
|
||||
file_names = [f"{user_id}.txt" for user_id in end_user_ids]
|
||||
|
||||
# 一次查询获取所有用户的chunk总数
|
||||
# 使用 GROUP BY file_name 来分组统计
|
||||
results = db.query(
|
||||
Document.file_name,
|
||||
func.sum(Document.chunk_num).label('total_chunk')
|
||||
).filter(
|
||||
Document.file_name.in_(file_names)
|
||||
).group_by(
|
||||
Document.file_name
|
||||
).all()
|
||||
|
||||
# 构建结果字典
|
||||
chunk_map = {}
|
||||
for file_name, total_chunk in results:
|
||||
# 从文件名中提取 end_user_id (去掉 .txt 后缀)
|
||||
user_id = file_name.replace('.txt', '')
|
||||
chunk_map[user_id] = int(total_chunk or 0)
|
||||
|
||||
# 对于没有记录的用户,设置为0
|
||||
for user_id in end_user_ids:
|
||||
if user_id not in chunk_map:
|
||||
chunk_map[user_id] = 0
|
||||
|
||||
business_logger.info(f"成功批量获取 {len(chunk_map)} 个用户的总chunk数")
|
||||
return chunk_map
|
||||
|
||||
except Exception as e:
|
||||
business_logger.error(f"批量获取用户总chunk数失败: {str(e)}")
|
||||
raise
|
||||
|
||||
|
||||
def get_rag_content(
|
||||
end_user_id: str,
|
||||
limit: int,
|
||||
|
||||
@@ -717,8 +717,8 @@ class MemoryInteraction:
|
||||
ori_data= await self.connector.execute_query(Memory_Space_Entity, id=self.id)
|
||||
if ori_data!=[]:
|
||||
# name = ori_data[0]['name']
|
||||
group_id = [i['group_id'] for i in ori_data][0]
|
||||
Space_User = await self.connector.execute_query(Memory_Space_User, group_id=group_id)
|
||||
end_user_id = [i['end_user_id'] for i in ori_data][0]
|
||||
Space_User = await self.connector.execute_query(Memory_Space_User, end_user_id=end_user_id)
|
||||
if not Space_User:
|
||||
return []
|
||||
user_id=Space_User[0]['id']
|
||||
|
||||
@@ -34,7 +34,7 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
|
||||
Args:
|
||||
summary_id: Summary节点的ID
|
||||
end_user_id: 终端用户ID (group_id)
|
||||
end_user_id: 终端用户ID (end_user_id)
|
||||
|
||||
Returns:
|
||||
(标题, 类型)元组,如果不存在则返回默认值
|
||||
@@ -43,14 +43,14 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
# 查询Summary节点的name(作为title)和memory_type(作为type)
|
||||
query = """
|
||||
MATCH (s:MemorySummary)
|
||||
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
|
||||
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
|
||||
RETURN s.name AS title, s.memory_type AS type
|
||||
"""
|
||||
|
||||
result = await self.neo4j_connector.execute_query(
|
||||
query,
|
||||
summary_id=summary_id,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
if not result or len(result) == 0:
|
||||
@@ -77,7 +77,7 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
|
||||
Args:
|
||||
summary_id: Summary节点的ID
|
||||
end_user_id: 终端用户ID (group_id)
|
||||
end_user_id: 终端用户ID (end_user_id)
|
||||
|
||||
Returns:
|
||||
前3个实体的name属性列表
|
||||
@@ -87,7 +87,7 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
# 按activation_value降序排序,返回前3个
|
||||
query = """
|
||||
MATCH (s:MemorySummary)
|
||||
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
|
||||
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
|
||||
MATCH (s)-[:DERIVED_FROM_STATEMENT]->(stmt:Statement)
|
||||
MATCH (stmt)-[:REFERENCES_ENTITY]->(entity:ExtractedEntity)
|
||||
WHERE entity.activation_value IS NOT NULL
|
||||
@@ -99,7 +99,7 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
result = await self.neo4j_connector.execute_query(
|
||||
query,
|
||||
summary_id=summary_id,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
# 提取实体名称
|
||||
@@ -123,7 +123,7 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
|
||||
Args:
|
||||
summary_id: Summary节点的ID
|
||||
end_user_id: 终端用户ID (group_id)
|
||||
end_user_id: 终端用户ID (end_user_id)
|
||||
|
||||
Returns:
|
||||
所有Statement节点的statement属性内容列表
|
||||
@@ -132,7 +132,7 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
# 查询Summary节点指向的所有Statement节点
|
||||
query = """
|
||||
MATCH (s:MemorySummary)
|
||||
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
|
||||
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
|
||||
MATCH (s)-[:DERIVED_FROM_STATEMENT]->(stmt:Statement)
|
||||
WHERE stmt.statement IS NOT NULL AND stmt.statement <> ''
|
||||
RETURN stmt.statement AS statement
|
||||
@@ -141,7 +141,7 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
result = await self.neo4j_connector.execute_query(
|
||||
query,
|
||||
summary_id=summary_id,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
# 提取statement内容
|
||||
@@ -214,12 +214,12 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
# 1. 先查询所有情景记忆的总数(不受筛选条件限制)
|
||||
total_all_query = """
|
||||
MATCH (s:MemorySummary)
|
||||
WHERE s.group_id = $group_id
|
||||
WHERE s.end_user_id = $end_user_id
|
||||
RETURN count(s) AS total_all
|
||||
"""
|
||||
total_all_result = await self.neo4j_connector.execute_query(
|
||||
total_all_query,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
total_all = total_all_result[0]["total_all"] if total_all_result else 0
|
||||
|
||||
@@ -229,7 +229,7 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
# 3. 构建Cypher查询
|
||||
query = """
|
||||
MATCH (s:MemorySummary)
|
||||
WHERE s.group_id = $group_id
|
||||
WHERE s.end_user_id = $end_user_id
|
||||
"""
|
||||
|
||||
# 添加时间范围过滤
|
||||
@@ -248,7 +248,7 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
ORDER BY s.created_at DESC
|
||||
"""
|
||||
|
||||
params = {"group_id": end_user_id}
|
||||
params = {"end_user_id": end_user_id}
|
||||
if time_filter:
|
||||
params["time_filter"] = time_filter
|
||||
if title_keyword:
|
||||
@@ -333,14 +333,14 @@ class MemoryEpisodicService(MemoryBaseService):
|
||||
# 1. 查询指定的MemorySummary节点
|
||||
query = """
|
||||
MATCH (s:MemorySummary)
|
||||
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
|
||||
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
|
||||
RETURN elementId(s) AS id, s.created_at AS created_at
|
||||
"""
|
||||
|
||||
result = await self.neo4j_connector.execute_query(
|
||||
query,
|
||||
summary_id=summary_id,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
# 2. 如果节点不存在,返回错误
|
||||
|
||||
@@ -60,7 +60,7 @@ class MemoryExplicitService(MemoryBaseService):
|
||||
# ========== 1. 查询情景记忆(MemorySummary节点) ==========
|
||||
episodic_query = """
|
||||
MATCH (s:MemorySummary)
|
||||
WHERE s.group_id = $group_id
|
||||
WHERE s.end_user_id = $end_user_id
|
||||
RETURN elementId(s) AS id,
|
||||
s.name AS title,
|
||||
s.content AS content,
|
||||
@@ -70,7 +70,7 @@ class MemoryExplicitService(MemoryBaseService):
|
||||
|
||||
episodic_result = await self.neo4j_connector.execute_query(
|
||||
episodic_query,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
# 处理情景记忆数据
|
||||
@@ -96,7 +96,7 @@ class MemoryExplicitService(MemoryBaseService):
|
||||
# ========== 2. 查询语义记忆(ExtractedEntity节点) ==========
|
||||
semantic_query = """
|
||||
MATCH (e:ExtractedEntity)
|
||||
WHERE e.group_id = $group_id
|
||||
WHERE e.end_user_id = $end_user_id
|
||||
AND e.is_explicit_memory = true
|
||||
RETURN elementId(e) AS id,
|
||||
e.name AS name,
|
||||
@@ -107,7 +107,7 @@ class MemoryExplicitService(MemoryBaseService):
|
||||
|
||||
semantic_result = await self.neo4j_connector.execute_query(
|
||||
semantic_query,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
# 处理语义记忆数据
|
||||
@@ -189,7 +189,7 @@ class MemoryExplicitService(MemoryBaseService):
|
||||
# ========== 1. 先尝试查询情景记忆 ==========
|
||||
episodic_query = """
|
||||
MATCH (s:MemorySummary)
|
||||
WHERE elementId(s) = $memory_id AND s.group_id = $group_id
|
||||
WHERE elementId(s) = $memory_id AND s.end_user_id = $end_user_id
|
||||
RETURN s.name AS title,
|
||||
s.content AS content,
|
||||
s.created_at AS created_at
|
||||
@@ -198,7 +198,7 @@ class MemoryExplicitService(MemoryBaseService):
|
||||
episodic_result = await self.neo4j_connector.execute_query(
|
||||
episodic_query,
|
||||
memory_id=memory_id,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
if episodic_result and len(episodic_result) > 0:
|
||||
@@ -229,7 +229,7 @@ class MemoryExplicitService(MemoryBaseService):
|
||||
semantic_query = """
|
||||
MATCH (e:ExtractedEntity)
|
||||
WHERE elementId(e) = $memory_id
|
||||
AND e.group_id = $group_id
|
||||
AND e.end_user_id = $end_user_id
|
||||
AND e.is_explicit_memory = true
|
||||
RETURN e.name AS name,
|
||||
e.description AS core_definition,
|
||||
@@ -240,7 +240,7 @@ class MemoryExplicitService(MemoryBaseService):
|
||||
semantic_result = await self.neo4j_connector.execute_query(
|
||||
semantic_query,
|
||||
memory_id=memory_id,
|
||||
group_id=end_user_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
if semantic_result and len(semantic_result) > 0:
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
|
||||
from typing import Optional, Dict, Any, Tuple
|
||||
from datetime import datetime, timezone
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
@@ -23,7 +24,7 @@ from app.core.memory.storage_services.forgetting_engine.config_utils import (
|
||||
load_actr_config_from_db,
|
||||
)
|
||||
from app.repositories.neo4j.neo4j_connector import Neo4jConnector
|
||||
from app.repositories.data_config_repository import DataConfigRepository
|
||||
from app.repositories.memory_config_repository import MemoryConfigRepository
|
||||
from app.repositories.forgetting_cycle_history_repository import ForgettingCycleHistoryRepository
|
||||
|
||||
|
||||
@@ -70,7 +71,7 @@ class MemoryForgetService:
|
||||
|
||||
def __init__(self):
|
||||
"""初始化服务"""
|
||||
self.config_repository = DataConfigRepository()
|
||||
self.config_repository = MemoryConfigRepository()
|
||||
self.history_repository = ForgettingCycleHistoryRepository()
|
||||
|
||||
def _get_neo4j_connector(self) -> Neo4jConnector:
|
||||
@@ -87,7 +88,7 @@ class MemoryForgetService:
|
||||
async def _get_forgetting_components(
|
||||
self,
|
||||
db: Session,
|
||||
config_id: Optional[int] = None
|
||||
config_id: Optional[UUID] = None
|
||||
) -> Tuple[ACTRCalculator, ForgettingStrategy, ForgettingScheduler, Dict[str, Any]]:
|
||||
"""
|
||||
获取遗忘引擎组件(计算器、策略、调度器)
|
||||
@@ -132,7 +133,7 @@ class MemoryForgetService:
|
||||
async def _get_knowledge_stats(
|
||||
self,
|
||||
connector: Neo4jConnector,
|
||||
group_id: Optional[str] = None,
|
||||
end_user_id: Optional[str] = None,
|
||||
forgetting_threshold: float = 0.3
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
@@ -140,7 +141,7 @@ class MemoryForgetService:
|
||||
|
||||
Args:
|
||||
connector: Neo4j 连接器
|
||||
group_id: 组ID(可选)
|
||||
end_user_id: 组ID(可选)
|
||||
forgetting_threshold: 遗忘阈值
|
||||
|
||||
Returns:
|
||||
@@ -152,8 +153,8 @@ class MemoryForgetService:
|
||||
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary)
|
||||
"""
|
||||
|
||||
if group_id:
|
||||
query += " AND n.group_id = $group_id"
|
||||
if end_user_id:
|
||||
query += " AND n.end_user_id = $end_user_id"
|
||||
|
||||
query += """
|
||||
WITH n,
|
||||
@@ -172,8 +173,8 @@ class MemoryForgetService:
|
||||
"""
|
||||
|
||||
params = {'threshold': forgetting_threshold}
|
||||
if group_id:
|
||||
params['group_id'] = group_id
|
||||
if end_user_id:
|
||||
params['end_user_id'] = end_user_id
|
||||
|
||||
results = await connector.execute_query(query, **params)
|
||||
|
||||
@@ -200,7 +201,7 @@ class MemoryForgetService:
|
||||
async def _get_pending_forgetting_nodes(
|
||||
self,
|
||||
connector: Neo4jConnector,
|
||||
group_id: str,
|
||||
end_user_id: str,
|
||||
forgetting_threshold: float,
|
||||
min_days_since_access: int,
|
||||
limit: int = 20
|
||||
@@ -212,7 +213,7 @@ class MemoryForgetService:
|
||||
|
||||
Args:
|
||||
connector: Neo4j 连接器
|
||||
group_id: 组ID
|
||||
end_user_id: 组ID
|
||||
forgetting_threshold: 遗忘阈值
|
||||
min_days_since_access: 最小未访问天数
|
||||
limit: 返回节点数量限制
|
||||
@@ -229,7 +230,7 @@ class MemoryForgetService:
|
||||
query = """
|
||||
MATCH (n)
|
||||
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary)
|
||||
AND n.group_id = $group_id
|
||||
AND n.end_user_id = $end_user_id
|
||||
AND n.activation_value IS NOT NULL
|
||||
AND n.activation_value < $threshold
|
||||
AND n.last_access_time IS NOT NULL
|
||||
@@ -250,7 +251,7 @@ class MemoryForgetService:
|
||||
"""
|
||||
|
||||
params = {
|
||||
'group_id': group_id,
|
||||
'end_user_id': end_user_id,
|
||||
'threshold': forgetting_threshold,
|
||||
'min_access_time_str': min_access_time_str,
|
||||
'limit': limit
|
||||
@@ -291,10 +292,10 @@ class MemoryForgetService:
|
||||
async def trigger_forgetting_cycle(
|
||||
self,
|
||||
db: Session,
|
||||
group_id: str,
|
||||
end_user_id: str,
|
||||
max_merge_batch_size: Optional[int] = None,
|
||||
min_days_since_access: Optional[int] = None,
|
||||
config_id: Optional[int] = None
|
||||
config_id: Optional[UUID] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
手动触发遗忘周期
|
||||
@@ -303,10 +304,10 @@ class MemoryForgetService:
|
||||
|
||||
Args:
|
||||
db: 数据库会话
|
||||
group_id: 组ID(即终端用户ID,必填)
|
||||
end_user_id: 组ID(即终端用户ID,必填)
|
||||
max_merge_batch_size: 最大融合批次大小(可选)
|
||||
min_days_since_access: 最小未访问天数(可选)
|
||||
config_id: 配置ID(必填,由控制器层通过 group_id 获取)
|
||||
config_id: 配置ID(必填,由控制器层通过 end_user_id 获取)
|
||||
|
||||
Returns:
|
||||
dict: 遗忘报告
|
||||
@@ -319,7 +320,7 @@ class MemoryForgetService:
|
||||
|
||||
# 运行遗忘周期(LLM 客户端将在需要时由 forgetting_strategy 内部获取)
|
||||
report = await forgetting_scheduler.run_forgetting_cycle(
|
||||
group_id=group_id,
|
||||
end_user_id=end_user_id,
|
||||
max_merge_batch_size=max_merge_batch_size,
|
||||
min_days_since_access=min_days_since_access,
|
||||
config_id=config_id,
|
||||
@@ -338,7 +339,7 @@ class MemoryForgetService:
|
||||
stats_query = """
|
||||
MATCH (n)
|
||||
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary OR n:Chunk)
|
||||
AND n.group_id = $group_id
|
||||
AND n.end_user_id = $end_user_id
|
||||
RETURN
|
||||
count(n) as total_nodes,
|
||||
avg(n.activation_value) as average_activation,
|
||||
@@ -347,7 +348,7 @@ class MemoryForgetService:
|
||||
|
||||
stats_results = await connector.execute_query(
|
||||
stats_query,
|
||||
group_id=group_id,
|
||||
end_user_id=end_user_id,
|
||||
threshold=config['forgetting_threshold']
|
||||
)
|
||||
|
||||
@@ -364,7 +365,7 @@ class MemoryForgetService:
|
||||
# 保存历史记录到数据库
|
||||
self.history_repository.create(
|
||||
db=db,
|
||||
end_user_id=group_id,
|
||||
end_user_id=end_user_id,
|
||||
execution_time=execution_time,
|
||||
merged_count=report['merged_count'],
|
||||
failed_count=report['failed_count'],
|
||||
@@ -376,7 +377,7 @@ class MemoryForgetService:
|
||||
)
|
||||
|
||||
api_logger.info(
|
||||
f"已保存遗忘周期历史记录: end_user_id={group_id}, "
|
||||
f"已保存遗忘周期历史记录: end_user_id={end_user_id}, "
|
||||
f"merged_count={report['merged_count']}"
|
||||
)
|
||||
|
||||
@@ -389,7 +390,7 @@ class MemoryForgetService:
|
||||
def read_forgetting_config(
|
||||
self,
|
||||
db: Session,
|
||||
config_id: int
|
||||
config_id: UUID
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
获取遗忘引擎配置
|
||||
@@ -416,7 +417,7 @@ class MemoryForgetService:
|
||||
def update_forgetting_config(
|
||||
self,
|
||||
db: Session,
|
||||
config_id: int,
|
||||
config_id: UUID,
|
||||
update_fields: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
@@ -465,8 +466,8 @@ class MemoryForgetService:
|
||||
async def get_forgetting_stats(
|
||||
self,
|
||||
db: Session,
|
||||
group_id: Optional[str] = None,
|
||||
config_id: Optional[int] = None
|
||||
end_user_id: Optional[str] = None,
|
||||
config_id: Optional[UUID] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
获取遗忘引擎统计信息
|
||||
@@ -475,7 +476,7 @@ class MemoryForgetService:
|
||||
|
||||
Args:
|
||||
db: 数据库会话
|
||||
group_id: 组ID(可选)
|
||||
end_user_id: 组ID(可选)
|
||||
config_id: 配置ID(可选,用于获取遗忘阈值)
|
||||
|
||||
Returns:
|
||||
@@ -493,8 +494,8 @@ class MemoryForgetService:
|
||||
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary OR n:Chunk)
|
||||
"""
|
||||
|
||||
if group_id:
|
||||
activation_query += " AND n.group_id = $group_id"
|
||||
if end_user_id:
|
||||
activation_query += " AND n.end_user_id = $end_user_id"
|
||||
|
||||
activation_query += """
|
||||
RETURN
|
||||
@@ -506,8 +507,8 @@ class MemoryForgetService:
|
||||
"""
|
||||
|
||||
params = {'threshold': forgetting_threshold}
|
||||
if group_id:
|
||||
params['group_id'] = group_id
|
||||
if end_user_id:
|
||||
params['end_user_id'] = end_user_id
|
||||
|
||||
activation_results = await connector.execute_query(activation_query, **params)
|
||||
|
||||
@@ -539,8 +540,8 @@ class MemoryForgetService:
|
||||
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary OR n:Chunk)
|
||||
"""
|
||||
|
||||
if group_id:
|
||||
distribution_query += " AND n.group_id = $group_id"
|
||||
if end_user_id:
|
||||
distribution_query += " AND n.end_user_id = $end_user_id"
|
||||
|
||||
distribution_query += """
|
||||
WITH n,
|
||||
@@ -558,8 +559,8 @@ class MemoryForgetService:
|
||||
"""
|
||||
|
||||
dist_params = {}
|
||||
if group_id:
|
||||
dist_params['group_id'] = group_id
|
||||
if end_user_id:
|
||||
dist_params['end_user_id'] = end_user_id
|
||||
|
||||
distribution_results = await connector.execute_query(distribution_query, **dist_params)
|
||||
|
||||
@@ -582,11 +583,11 @@ class MemoryForgetService:
|
||||
# 获取最近7个日期的历史趋势数据(每天取最后一次执行)
|
||||
recent_trends = []
|
||||
try:
|
||||
if group_id:
|
||||
if end_user_id:
|
||||
# 查询所有历史记录
|
||||
history_records = self.history_repository.get_recent_by_end_user(
|
||||
db=db,
|
||||
end_user_id=group_id
|
||||
end_user_id=end_user_id
|
||||
)
|
||||
|
||||
# 按日期分组(一天可能有多次执行,取最后一次)
|
||||
@@ -632,7 +633,7 @@ class MemoryForgetService:
|
||||
# 获取待遗忘节点列表(前20个满足遗忘条件的节点)
|
||||
pending_nodes = []
|
||||
try:
|
||||
if group_id:
|
||||
if end_user_id:
|
||||
# 验证 min_days_since_access 配置值
|
||||
min_days = config.get('min_days_since_access')
|
||||
if min_days is None or not isinstance(min_days, (int, float)) or min_days < 0:
|
||||
@@ -643,7 +644,7 @@ class MemoryForgetService:
|
||||
|
||||
pending_nodes = await self._get_pending_forgetting_nodes(
|
||||
connector=connector,
|
||||
group_id=group_id,
|
||||
end_user_id=end_user_id,
|
||||
forgetting_threshold=forgetting_threshold,
|
||||
min_days_since_access=int(min_days),
|
||||
limit=20
|
||||
@@ -677,7 +678,7 @@ class MemoryForgetService:
|
||||
db: Session,
|
||||
importance_score: float,
|
||||
days: int,
|
||||
config_id: Optional[int] = None
|
||||
config_id: Optional[UUID] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
获取遗忘曲线数据
|
||||
|
||||
@@ -450,12 +450,12 @@ async def create_document_chunk(
|
||||
|
||||
return success(data=chunk, msg="文档块创建成功")
|
||||
|
||||
async def write_rag(group_id, message, user_rag_memory_id):
|
||||
async def write_rag(end_user_id, message, user_rag_memory_id):
|
||||
"""
|
||||
将消息写入 RAG 知识库
|
||||
|
||||
Args:
|
||||
group_id: 组ID,用作文件标题
|
||||
end_user_id: 组ID,用作文件标题
|
||||
message: 消息内容
|
||||
user_rag_memory_id: 知识库ID(必须是有效的UUID)
|
||||
|
||||
@@ -487,10 +487,10 @@ async def write_rag(group_id, message, user_rag_memory_id):
|
||||
db = next(db_gen)
|
||||
|
||||
try:
|
||||
create_data = CustomTextFileCreate(title=group_id, content=message)
|
||||
create_data = CustomTextFileCreate(title=end_user_id, content=message)
|
||||
current_user = SimpleUser(user_rag_memory_id)
|
||||
# 检查文档是否已存在
|
||||
document = find_document_id_by_kb_and_filename(db=db, kb_id=user_rag_memory_id, file_name=f"{group_id}.txt")
|
||||
document = find_document_id_by_kb_and_filename(db=db, kb_id=user_rag_memory_id, file_name=f"{end_user_id}.txt")
|
||||
print('======',document)
|
||||
api_logger.info(f"查找文档结果: document_id={document}")
|
||||
if document is not None:
|
||||
@@ -508,7 +508,7 @@ async def write_rag(group_id, message, user_rag_memory_id):
|
||||
return result
|
||||
else:
|
||||
# 文档不存在,创建新文档
|
||||
api_logger.info(f"文档不存在,创建新文档: group_id={group_id}")
|
||||
api_logger.info(f"文档不存在,创建新文档: end_user_id={end_user_id}")
|
||||
result = await memory_konwledges_up(
|
||||
kb_id=user_rag_memory_id,
|
||||
parent_id=user_rag_memory_id,
|
||||
@@ -520,13 +520,13 @@ async def write_rag(group_id, message, user_rag_memory_id):
|
||||
new_document_id = find_document_id_by_kb_and_filename(
|
||||
db=db,
|
||||
kb_id=user_rag_memory_id,
|
||||
file_name=f"{group_id}.txt"
|
||||
file_name=f"{end_user_id}.txt"
|
||||
)
|
||||
|
||||
if new_document_id:
|
||||
await parse_document_by_id(new_document_id, db=db, current_user=current_user)
|
||||
else:
|
||||
api_logger.error(f"创建文档后无法找到文档ID: group_id={group_id}")
|
||||
api_logger.error(f"创建文档后无法找到文档ID: end_user_id={end_user_id}")
|
||||
return result
|
||||
finally:
|
||||
# 确保数据库会话被关闭
|
||||
|
||||
@@ -6,7 +6,7 @@ from sqlalchemy.orm import Session
|
||||
from app.core.error_codes import BizCode
|
||||
from app.core.exceptions import BusinessException
|
||||
from app.core.logging_config import get_business_logger
|
||||
from app.models.memory_perceptual_model import PerceptualType, FileStorageType
|
||||
from app.models.memory_perceptual_model import PerceptualType, FileStorageService
|
||||
from app.repositories.memory_perceptual_repository import MemoryPerceptualRepository
|
||||
from app.schemas.memory_perceptual_schema import (
|
||||
PerceptualQuerySchema,
|
||||
@@ -137,8 +137,19 @@ class MemoryPerceptualService:
|
||||
memory_items = []
|
||||
for memory in memories:
|
||||
meta_data = memory.meta_data or {}
|
||||
content = meta_data.get("content")
|
||||
content = Content(**content)
|
||||
content = meta_data.get("content", {})
|
||||
|
||||
# 安全地提取 content 字段,提供默认值
|
||||
if content:
|
||||
content_obj = Content(**content)
|
||||
topic = content_obj.topic
|
||||
domain = content_obj.domain
|
||||
keywords = content_obj.keywords
|
||||
else:
|
||||
topic = "Unknown"
|
||||
domain = "Unknown"
|
||||
keywords = []
|
||||
|
||||
memory_item = PerceptualMemoryItem(
|
||||
id=memory.id,
|
||||
perceptual_type=PerceptualType(memory.perceptual_type),
|
||||
@@ -146,11 +157,12 @@ class MemoryPerceptualService:
|
||||
file_name=memory.file_name,
|
||||
file_ext=memory.file_ext,
|
||||
summary=memory.summary,
|
||||
topic=content.topic,
|
||||
domain=content.domain,
|
||||
keywords=content.keywords,
|
||||
meta_data=meta_data,
|
||||
topic=topic,
|
||||
domain=domain,
|
||||
keywords=keywords,
|
||||
created_time=int(memory.created_time.timestamp()*1000),
|
||||
storage_type=FileStorageType(memory.storage_service),
|
||||
storage_service=FileStorageService(memory.storage_service),
|
||||
)
|
||||
memory_items.append(memory_item)
|
||||
|
||||
|
||||
@@ -13,11 +13,12 @@ from app.db import get_db
|
||||
from app.core.logging_config import get_api_logger
|
||||
from app.core.memory.storage_services.reflection_engine import ReflectionConfig, ReflectionEngine
|
||||
from app.core.memory.storage_services.reflection_engine.self_reflexion import ReflectionRange, ReflectionBaseline
|
||||
from app.repositories.data_config_repository import DataConfigRepository
|
||||
from app.repositories.memory_config_repository import MemoryConfigRepository
|
||||
from app.repositories.neo4j.neo4j_connector import Neo4jConnector
|
||||
from app.models.app_model import App
|
||||
from app.models.app_release_model import AppRelease
|
||||
from app.models.end_user_model import EndUser
|
||||
from app.utils.config_utils import resolve_config_id
|
||||
|
||||
api_logger = get_api_logger()
|
||||
|
||||
@@ -38,7 +39,10 @@ class WorkspaceAppService:
|
||||
Returns:
|
||||
Dictionary containing detailed application information
|
||||
"""
|
||||
apps = self.db.query(App).filter(App.workspace_id == workspace_id).all()
|
||||
apps = self.db.query(App).filter(
|
||||
App.workspace_id == workspace_id,
|
||||
App.is_active.is_(True)
|
||||
).all()
|
||||
app_ids = [str(app.id) for app in apps]
|
||||
|
||||
apps_detailed_info = []
|
||||
@@ -70,7 +74,7 @@ class WorkspaceAppService:
|
||||
"created_at": app.created_at.isoformat() if app.created_at else None,
|
||||
"updated_at": app.updated_at.isoformat() if app.updated_at else None,
|
||||
"releases": [],
|
||||
"data_configs": [],
|
||||
"memory_configs": [],
|
||||
"end_users": []
|
||||
}
|
||||
|
||||
@@ -85,76 +89,76 @@ class WorkspaceAppService:
|
||||
|
||||
for release in app_releases:
|
||||
memory_content = self._extract_memory_content(release.config)
|
||||
|
||||
|
||||
memory_content=resolve_config_id(memory_content, self.db)
|
||||
if memory_content and memory_content in processed_configs:
|
||||
continue
|
||||
|
||||
|
||||
release_info = {
|
||||
"app_id": str(release.app_id),
|
||||
"config": memory_content
|
||||
}
|
||||
|
||||
|
||||
|
||||
if memory_content:
|
||||
processed_configs.add(memory_content)
|
||||
data_config_info = self._get_data_config(memory_content)
|
||||
|
||||
if data_config_info:
|
||||
if not any(dc["config_id"] == data_config_info["config_id"] for dc in app_info["data_configs"]):
|
||||
app_info["data_configs"].append(data_config_info)
|
||||
|
||||
memory_config_info = self._get_memory_config(memory_content)
|
||||
if memory_config_info:
|
||||
if not any(dc["config_id"] == memory_config_info["config_id"] for dc in app_info["memory_configs"]):
|
||||
app_info["memory_configs"].append(memory_config_info)
|
||||
|
||||
app_info["releases"].append(release_info)
|
||||
|
||||
|
||||
def _extract_memory_content(self, config: Any) -> str:
|
||||
"""Extract memory_comtent from config"""
|
||||
if not config or not isinstance(config, dict):
|
||||
return None
|
||||
|
||||
|
||||
memory_obj = config.get('memory')
|
||||
if memory_obj and isinstance(memory_obj, dict):
|
||||
return memory_obj.get('memory_content')
|
||||
|
||||
return None
|
||||
|
||||
def _get_data_config(self, memory_content: str) -> Dict[str, Any]:
|
||||
"""Retrieve data_comfig information based on memory_comtent"""
|
||||
try:
|
||||
data_config_result = DataConfigRepository.query_reflection_config_by_id(self.db, int(memory_content))
|
||||
|
||||
# data_config_query, data_config_params = DataConfigRepository.build_select_reflection(memory_content)
|
||||
# data_config_result = self.db.execute(text(data_config_query), data_config_params).fetchone()
|
||||
# if data_config_result is None:
|
||||
return None
|
||||
|
||||
def _get_memory_config(self, memory_content: str) -> Dict[str, Any]:
|
||||
"""Retrieve memory_config information based on memory_content"""
|
||||
try:
|
||||
memory_config_result = MemoryConfigRepository.query_reflection_config_by_id(self.db, int(memory_content))
|
||||
|
||||
# memory_config_query, memory_config_params = MemoryConfigRepository.build_select_reflection(memory_content)
|
||||
# memory_config_result = self.db.execute(text(memory_config_query), memory_config_params).fetchone()
|
||||
# if memory_config_result is None:
|
||||
# return None
|
||||
|
||||
if data_config_result:
|
||||
|
||||
if memory_config_result:
|
||||
return {
|
||||
"config_id": data_config_result.config_id,
|
||||
"enable_self_reflexion": data_config_result.enable_self_reflexion,
|
||||
"iteration_period": data_config_result.iteration_period,
|
||||
"reflexion_range": data_config_result.reflexion_range,
|
||||
"baseline": data_config_result.baseline,
|
||||
"reflection_model_id": data_config_result.reflection_model_id,
|
||||
"memory_verify": data_config_result.memory_verify,
|
||||
"quality_assessment": data_config_result.quality_assessment,
|
||||
"user_id": data_config_result.user_id
|
||||
"config_id": memory_config_result.config_id,
|
||||
"enable_self_reflexion": memory_config_result.enable_self_reflexion,
|
||||
"iteration_period": memory_config_result.iteration_period,
|
||||
"reflexion_range": memory_config_result.reflexion_range,
|
||||
"baseline": memory_config_result.baseline,
|
||||
"reflection_model_id": memory_config_result.reflection_model_id,
|
||||
"memory_verify": memory_config_result.memory_verify,
|
||||
"quality_assessment": memory_config_result.quality_assessment,
|
||||
"user_id": memory_config_result.user_id
|
||||
}
|
||||
except Exception as e:
|
||||
api_logger.warning(f"查询data_config失败,memory_content: {memory_content}, 错误: {str(e)}")
|
||||
|
||||
api_logger.warning(f"查询memory_config失败,memory_content: {memory_content}, 错误: {str(e)}")
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _process_end_users(self, app: App, app_info: Dict[str, Any]) -> None:
|
||||
"""Processing end-user information for applications"""
|
||||
end_users = self.db.query(EndUser).filter(EndUser.app_id == app.id).all()
|
||||
|
||||
|
||||
for end_user in end_users:
|
||||
end_user_info = {
|
||||
"id": str(end_user.id),
|
||||
"app_id": str(end_user.app_id)
|
||||
}
|
||||
app_info["end_users"].append(end_user_info)
|
||||
|
||||
print(100*'-')
|
||||
print(app_info)
|
||||
|
||||
def get_end_user_reflection_time(self, end_user_id: str) -> Optional[Any]:
|
||||
"""
|
||||
Read the reflection time of end users
|
||||
@@ -173,7 +177,7 @@ class WorkspaceAppService:
|
||||
except Exception as e:
|
||||
api_logger.error(f"读取用户反思时间失败,end_user_id: {end_user_id}, 错误: {str(e)}")
|
||||
return None
|
||||
|
||||
|
||||
def update_end_user_reflection_time(self, end_user_id: str) -> bool:
|
||||
"""
|
||||
Update the reflection time of end users to the current time
|
||||
@@ -186,7 +190,7 @@ class WorkspaceAppService:
|
||||
"""
|
||||
try:
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
end_user = self.db.query(EndUser).filter(EndUser.id == end_user_id).first()
|
||||
if end_user:
|
||||
end_user.reflection_time = datetime.now()
|
||||
@@ -204,7 +208,7 @@ class WorkspaceAppService:
|
||||
|
||||
class MemoryReflectionService:
|
||||
"""Memory reflection service category"""
|
||||
|
||||
|
||||
def __init__(self,db: Session = Depends(get_db)):
|
||||
self.db=db
|
||||
|
||||
@@ -223,7 +227,7 @@ class MemoryReflectionService:
|
||||
}
|
||||
|
||||
config_data_id = config_data['config_id']
|
||||
reflection_config = WorkspaceAppService(self.db)._get_data_config(config_data_id)
|
||||
reflection_config = WorkspaceAppService(self.db)._get_memory_config(config_data_id)
|
||||
if reflection_config is not None and reflection_config['enable_self_reflexion']:
|
||||
reflection_config = self._create_reflection_config_from_data(reflection_config)
|
||||
# 3. 执行反思引擎
|
||||
@@ -249,22 +253,22 @@ class MemoryReflectionService:
|
||||
"end_user_id": end_user_id,
|
||||
"config_data": config_data
|
||||
}
|
||||
|
||||
|
||||
async def start_reflection_from_data(self, config_data: Dict[str, Any], end_user_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Starting Reflection from Configuration Data
|
||||
|
||||
|
||||
Args:
|
||||
config_data: Configure data dictionary, including reflective configuration information
|
||||
end_user_id: end_user_id
|
||||
|
||||
|
||||
Returns:
|
||||
Reflect on the execution results
|
||||
"""
|
||||
try:
|
||||
config_id = config_data.get("config_id")
|
||||
api_logger.info(f"从配置数据启动反思,config_id: {config_id}, end_user_id: {end_user_id}")
|
||||
|
||||
|
||||
|
||||
if not config_data.get("enable_self_reflexion", False):
|
||||
return {
|
||||
@@ -274,10 +278,10 @@ class MemoryReflectionService:
|
||||
"end_user_id": end_user_id,
|
||||
"config_data": config_data
|
||||
}
|
||||
|
||||
|
||||
|
||||
config_data_id=config_data['config_id']
|
||||
reflection_config=WorkspaceAppService(self.db)._get_data_config(config_data_id)
|
||||
reflection_config=WorkspaceAppService(self.db)._get_memory_config(config_data_id)
|
||||
if reflection_config is not None and reflection_config['enable_self_reflexion']:
|
||||
reflection_config= self._create_reflection_config_from_data(reflection_config)
|
||||
iteration_period = int(reflection_config.iteration_period)
|
||||
|
||||
@@ -12,10 +12,14 @@ from datetime import datetime
|
||||
from typing import Any, AsyncGenerator, Dict, List, Optional
|
||||
|
||||
from app.core.logging_config import get_config_logger, get_logger
|
||||
from app.core.memory.analytics.hot_memory_tags import get_hot_memory_tags
|
||||
from app.core.memory.analytics.hot_memory_tags import (
|
||||
get_hot_memory_tags,
|
||||
get_raw_tags_from_db,
|
||||
filter_tags_with_llm,
|
||||
)
|
||||
from app.core.memory.analytics.recent_activity_stats import get_recent_activity_stats
|
||||
from app.models.user_model import User
|
||||
from app.repositories.data_config_repository import DataConfigRepository
|
||||
from app.repositories.memory_config_repository import MemoryConfigRepository
|
||||
from app.repositories.neo4j.neo4j_connector import Neo4jConnector
|
||||
from app.schemas.memory_config_schema import ConfigurationError
|
||||
from app.schemas.memory_storage_schema import (
|
||||
@@ -125,7 +129,7 @@ class DataConfigService: # 数据配置服务类(PostgreSQL)
|
||||
if not params.rerank_id:
|
||||
params.rerank_id = configs.get('rerank')
|
||||
|
||||
config = DataConfigRepository.create(self.db, params)
|
||||
config = MemoryConfigRepository.create(self.db, params)
|
||||
self.db.commit()
|
||||
return {"affected": 1, "config_id": config.config_id}
|
||||
|
||||
@@ -142,20 +146,20 @@ class DataConfigService: # 数据配置服务类(PostgreSQL)
|
||||
|
||||
# --- Delete ---
|
||||
def delete(self, key: ConfigParamsDelete) -> Dict[str, Any]: # 删除配置参数(按配置ID)
|
||||
success = DataConfigRepository.delete(self.db, key.config_id)
|
||||
success = MemoryConfigRepository.delete(self.db, key.config_id)
|
||||
if not success:
|
||||
raise ValueError("未找到配置")
|
||||
return {"affected": 1}
|
||||
|
||||
# --- Update ---
|
||||
def update(self, update: ConfigUpdate) -> Dict[str, Any]: # 部分更新配置参数
|
||||
config = DataConfigRepository.update(self.db, update)
|
||||
config = MemoryConfigRepository.update(self.db, update)
|
||||
if not config:
|
||||
raise ValueError("未找到配置")
|
||||
return {"affected": 1}
|
||||
|
||||
def update_extracted(self, update: ConfigUpdateExtracted) -> Dict[str, Any]: # 更新记忆萃取引擎配置参数
|
||||
config = DataConfigRepository.update_extracted(self.db, update)
|
||||
config = MemoryConfigRepository.update_extracted(self.db, update)
|
||||
if not config:
|
||||
raise ValueError("未找到配置")
|
||||
return {"affected": 1}
|
||||
@@ -166,25 +170,38 @@ class DataConfigService: # 数据配置服务类(PostgreSQL)
|
||||
|
||||
# --- Read ---
|
||||
def get_extracted(self, key: ConfigKey) -> Dict[str, Any]: # 获取萃取配置参数
|
||||
result = DataConfigRepository.get_extracted_config(self.db, key.config_id)
|
||||
result = MemoryConfigRepository.get_extracted_config(self.db, key.config_id)
|
||||
if not result:
|
||||
raise ValueError("未找到配置")
|
||||
return result
|
||||
|
||||
# --- Read All ---
|
||||
def get_all(self, workspace_id = None) -> List[Dict[str, Any]]: # 获取所有配置参数
|
||||
configs = DataConfigRepository.get_all(self.db, workspace_id)
|
||||
configs = MemoryConfigRepository.get_all(self.db, workspace_id)
|
||||
|
||||
# 将 ORM 对象转换为字典列表
|
||||
data_list = []
|
||||
for config in configs:
|
||||
# 安全地转换 user_id 为 int
|
||||
config_id_old = None
|
||||
if config.config_id_old:
|
||||
try:
|
||||
config_id_old = int(config.config_id_old)
|
||||
except (ValueError, TypeError):
|
||||
config_id_old = None
|
||||
|
||||
|
||||
if config_id_old:
|
||||
memory_config=config_id_old
|
||||
else:
|
||||
memory_config=config.config_id
|
||||
config_dict = {
|
||||
"config_id": config.config_id,
|
||||
"config_id": memory_config,
|
||||
"config_name": config.config_name,
|
||||
"config_desc": config.config_desc,
|
||||
"workspace_id": str(config.workspace_id) if config.workspace_id else None,
|
||||
"group_id": config.group_id,
|
||||
"user_id": config.user_id,
|
||||
"end_user_id": config.end_user_id,
|
||||
"config_id_old": config_id_old,
|
||||
"apply_id": config.apply_id,
|
||||
"llm_id": config.llm_id,
|
||||
"embedding_id": config.embedding_id,
|
||||
@@ -237,7 +254,8 @@ class DataConfigService: # 数据配置服务类(PostgreSQL)
|
||||
ValueError: 当配置无效或参数缺失时
|
||||
RuntimeError: 当管线执行失败时
|
||||
"""
|
||||
project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
from pathlib import Path
|
||||
project_root = str(Path(__file__).resolve().parents[2])
|
||||
|
||||
try:
|
||||
# 发出初始进度事件
|
||||
@@ -263,7 +281,7 @@ class DataConfigService: # 数据配置服务类(PostgreSQL)
|
||||
try:
|
||||
config_service = MemoryConfigService(self.db)
|
||||
memory_config = config_service.load_memory_config(
|
||||
config_id=int(cid),
|
||||
config_id=str(cid),
|
||||
service_name="MemoryStorageService.pilot_run_stream"
|
||||
)
|
||||
logger.info(f"Configuration loaded successfully: {memory_config.config_name}")
|
||||
@@ -390,8 +408,8 @@ _neo4j_connector = Neo4jConnector()
|
||||
|
||||
async def search_dialogue(end_user_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
result = await _neo4j_connector.execute_query(
|
||||
DataConfigRepository.SEARCH_FOR_DIALOGUE,
|
||||
group_id=end_user_id,
|
||||
MemoryConfigRepository.SEARCH_FOR_DIALOGUE,
|
||||
end_user_id=end_user_id,
|
||||
)
|
||||
data = {"search_for": "dialogue", "num": result[0]["num"]}
|
||||
return data
|
||||
@@ -399,8 +417,8 @@ async def search_dialogue(end_user_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
|
||||
async def search_chunk(end_user_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
result = await _neo4j_connector.execute_query(
|
||||
DataConfigRepository.SEARCH_FOR_CHUNK,
|
||||
group_id=end_user_id,
|
||||
MemoryConfigRepository.SEARCH_FOR_CHUNK,
|
||||
end_user_id=end_user_id,
|
||||
)
|
||||
data = {"search_for": "chunk", "num": result[0]["num"]}
|
||||
return data
|
||||
@@ -408,8 +426,8 @@ async def search_chunk(end_user_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
|
||||
async def search_statement(end_user_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
result = await _neo4j_connector.execute_query(
|
||||
DataConfigRepository.SEARCH_FOR_STATEMENT,
|
||||
group_id=end_user_id,
|
||||
MemoryConfigRepository.SEARCH_FOR_STATEMENT,
|
||||
end_user_id=end_user_id,
|
||||
)
|
||||
data = {"search_for": "statement", "num": result[0]["num"]}
|
||||
return data
|
||||
@@ -417,8 +435,8 @@ async def search_statement(end_user_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
|
||||
async def search_entity(end_user_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
result = await _neo4j_connector.execute_query(
|
||||
DataConfigRepository.SEARCH_FOR_ENTITY,
|
||||
group_id=end_user_id,
|
||||
MemoryConfigRepository.SEARCH_FOR_ENTITY,
|
||||
end_user_id=end_user_id,
|
||||
)
|
||||
data = {"search_for": "entity", "num": result[0]["num"]}
|
||||
return data
|
||||
@@ -426,8 +444,8 @@ async def search_entity(end_user_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
|
||||
async def search_all(end_user_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
result = await _neo4j_connector.execute_query(
|
||||
DataConfigRepository.SEARCH_FOR_ALL,
|
||||
group_id=end_user_id,
|
||||
MemoryConfigRepository.SEARCH_FOR_ALL,
|
||||
end_user_id=end_user_id,
|
||||
)
|
||||
|
||||
# 检查结果是否为空或长度不足
|
||||
@@ -461,8 +479,8 @@ async def kb_type_distribution(end_user_id: Optional[str] = None) -> Dict[str, A
|
||||
聚合 dialogue/chunk/statement/entity 四类计数,返回统一的分布结构,便于前端一次性消费。
|
||||
"""
|
||||
result = await _neo4j_connector.execute_query(
|
||||
DataConfigRepository.SEARCH_FOR_ALL,
|
||||
group_id=end_user_id,
|
||||
MemoryConfigRepository.SEARCH_FOR_ALL,
|
||||
end_user_id=end_user_id,
|
||||
)
|
||||
|
||||
# 检查结果是否为空或长度不足
|
||||
@@ -492,21 +510,19 @@ async def kb_type_distribution(end_user_id: Optional[str] = None) -> Dict[str, A
|
||||
|
||||
async def search_detials(end_user_id: Optional[str] = None) -> List[Dict[str, Any]]:
|
||||
result = await _neo4j_connector.execute_query(
|
||||
DataConfigRepository.SEARCH_FOR_DETIALS,
|
||||
group_id=end_user_id,
|
||||
MemoryConfigRepository.SEARCH_FOR_DETIALS,
|
||||
end_user_id=end_user_id,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
async def search_edges(end_user_id: Optional[str] = None) -> List[Dict[str, Any]]:
|
||||
result = await _neo4j_connector.execute_query(
|
||||
DataConfigRepository.SEARCH_FOR_EDGES,
|
||||
group_id=end_user_id,
|
||||
MemoryConfigRepository.SEARCH_FOR_EDGES,
|
||||
end_user_id=end_user_id,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
|
||||
async def analytics_hot_memory_tags(
|
||||
db: Session,
|
||||
current_user: User,
|
||||
@@ -514,27 +530,79 @@ async def analytics_hot_memory_tags(
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
获取热门记忆标签,按数量排序并返回前N个
|
||||
|
||||
优化策略:
|
||||
1. 先从所有用户收集原始标签(不调用LLM)
|
||||
2. 聚合并合并相同标签的频率
|
||||
3. 排序后取前N个
|
||||
4. 只调用一次LLM进行筛选
|
||||
"""
|
||||
workspace_id = current_user.current_workspace_id
|
||||
# 获取更多标签供LLM筛选(获取limit*4个标签)
|
||||
raw_limit = limit * 4
|
||||
from app.services.memory_dashboard_service import get_workspace_end_users
|
||||
end_users = get_workspace_end_users(db, workspace_id, current_user)
|
||||
# 使用 asyncio.to_thread 避免阻塞事件循环
|
||||
end_users = await asyncio.to_thread(get_workspace_end_users, db, workspace_id, current_user)
|
||||
|
||||
tags = []
|
||||
for end_user in end_users:
|
||||
tag = await get_hot_memory_tags(str(end_user.id), limit=raw_limit)
|
||||
if tag:
|
||||
# 将每个用户的标签列表展平到总列表中
|
||||
tags.extend(tag)
|
||||
|
||||
# 按频率降序排序(虽然数据库已经排序,但为了确保正确性再次排序)
|
||||
sorted_tags = sorted(tags, key=lambda x: x[1], reverse=True)
|
||||
if not end_users:
|
||||
return []
|
||||
|
||||
# 只返回前limit个
|
||||
top_tags = sorted_tags[:limit]
|
||||
|
||||
return [{"name": t, "frequency": f} for t, f in top_tags]
|
||||
# 步骤1: 收集所有用户的原始标签(不调用LLM)
|
||||
connector = Neo4jConnector()
|
||||
try:
|
||||
all_raw_tags = []
|
||||
for end_user in end_users:
|
||||
raw_tags = await get_raw_tags_from_db(
|
||||
connector,
|
||||
str(end_user.id),
|
||||
limit=raw_limit,
|
||||
by_user=False
|
||||
)
|
||||
if raw_tags:
|
||||
all_raw_tags.extend(raw_tags)
|
||||
|
||||
if not all_raw_tags:
|
||||
return []
|
||||
|
||||
# 步骤2: 聚合相同标签的频率
|
||||
tag_frequency_map = {}
|
||||
for tag_name, frequency in all_raw_tags:
|
||||
if tag_name in tag_frequency_map:
|
||||
tag_frequency_map[tag_name] += frequency
|
||||
else:
|
||||
tag_frequency_map[tag_name] = frequency
|
||||
|
||||
# 步骤3: 按频率降序排序,取前raw_limit个
|
||||
sorted_tags = sorted(
|
||||
tag_frequency_map.items(),
|
||||
key=lambda x: x[1],
|
||||
reverse=True
|
||||
)[:raw_limit]
|
||||
|
||||
if not sorted_tags:
|
||||
return []
|
||||
|
||||
# 步骤4: 只调用一次LLM进行筛选
|
||||
tag_names = [tag for tag, _ in sorted_tags]
|
||||
|
||||
# 使用第一个用户的end_user_id来获取LLM配置
|
||||
# 因为同一工作空间下的用户应该使用相同的配置
|
||||
first_end_user_id = str(end_users[0].id)
|
||||
filtered_tag_names = await filter_tags_with_llm(tag_names, first_end_user_id)
|
||||
|
||||
# 步骤5: 根据LLM筛选结果构建最终列表(保留频率)
|
||||
final_tags = []
|
||||
for tag, freq in sorted_tags:
|
||||
if tag in filtered_tag_names:
|
||||
final_tags.append((tag, freq))
|
||||
|
||||
# 步骤6: 只返回前limit个
|
||||
top_tags = final_tags[:limit]
|
||||
|
||||
return [{"name": t, "frequency": f} for t, f in top_tags]
|
||||
|
||||
finally:
|
||||
await connector.close()
|
||||
|
||||
|
||||
async def analytics_recent_activity_stats() -> Dict[str, Any]:
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
from datetime import datetime
|
||||
from sqlalchemy.orm import Session
|
||||
from typing import List, Optional, Dict, Any
|
||||
import uuid
|
||||
@@ -6,11 +7,11 @@ import time
|
||||
import asyncio
|
||||
|
||||
from app.models.models_model import ModelConfig, ModelApiKey, ModelType
|
||||
from app.repositories.model_repository import ModelConfigRepository, ModelApiKeyRepository
|
||||
from app.repositories.model_repository import ModelConfigRepository, ModelApiKeyRepository, ModelBaseRepository
|
||||
from app.schemas import model_schema
|
||||
from app.schemas.model_schema import (
|
||||
ModelConfigCreate, ModelConfigUpdate, ModelApiKeyCreate, ModelApiKeyUpdate,
|
||||
ModelConfigQuery, ModelStats
|
||||
ModelConfigQuery, ModelStats, ModelConfigQueryNew
|
||||
)
|
||||
from app.core.logging_config import get_business_logger
|
||||
from app.schemas.response_schema import PageData, PageMeta
|
||||
@@ -47,6 +48,26 @@ class ModelConfigService:
|
||||
items=[model_schema.ModelConfig.model_validate(model) for model in models]
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def get_model_list_new(db: Session, query: ModelConfigQueryNew, tenant_id: uuid.UUID | None = None) -> List[dict]:
|
||||
"""获取模型配置列表"""
|
||||
provider_groups, total = ModelConfigRepository.get_list_new(db, query, tenant_id=tenant_id)
|
||||
|
||||
items = []
|
||||
for provider, models in provider_groups.items():
|
||||
# 验证每个模型并封装分组信息
|
||||
validated_models = [model_schema.ModelConfig.model_validate(model) for model in models]
|
||||
tags = list({model.type for model in validated_models})
|
||||
group_item = {
|
||||
"provider": provider, # 服务商名称
|
||||
"logo": validated_models[0].logo,
|
||||
"tags": tags,
|
||||
"models": validated_models # 该服务商下的所有模型
|
||||
}
|
||||
items.append(group_item)
|
||||
|
||||
return items
|
||||
|
||||
@staticmethod
|
||||
def get_model_by_name(db: Session, name: str, tenant_id: uuid.UUID | None = None) -> ModelConfig:
|
||||
"""根据名称获取模型配置"""
|
||||
@@ -228,37 +249,39 @@ class ModelConfigService:
|
||||
|
||||
# 验证配置
|
||||
if not model_data.skip_validation and model_data.api_keys:
|
||||
api_key_data = model_data.api_keys
|
||||
validation_result = await ModelConfigService.validate_model_config(
|
||||
db=db,
|
||||
model_name=api_key_data.model_name,
|
||||
provider=api_key_data.provider,
|
||||
api_key=api_key_data.api_key,
|
||||
api_base=api_key_data.api_base,
|
||||
model_type=model_data.type, # 传递模型类型
|
||||
test_message="Hello"
|
||||
)
|
||||
if not validation_result["valid"]:
|
||||
raise BusinessException(
|
||||
f"模型配置验证失败: {validation_result['error']}",
|
||||
BizCode.INVALID_PARAMETER
|
||||
api_key_data_list = model_data.api_keys
|
||||
for api_key_data in api_key_data_list:
|
||||
validation_result = await ModelConfigService.validate_model_config(
|
||||
db=db,
|
||||
model_name=api_key_data.model_name,
|
||||
provider=api_key_data.provider,
|
||||
api_key=api_key_data.api_key,
|
||||
api_base=api_key_data.api_base,
|
||||
model_type=model_data.type, # 传递模型类型
|
||||
test_message="Hello"
|
||||
)
|
||||
if not validation_result["valid"]:
|
||||
raise BusinessException(
|
||||
f"模型配置验证失败: {validation_result['error']}",
|
||||
BizCode.INVALID_PARAMETER
|
||||
)
|
||||
|
||||
# 事务处理
|
||||
api_key_data = model_data.api_keys
|
||||
model_config_data = model_data.dict(exclude={"api_keys", "skip_validation"})
|
||||
api_key_datas = model_data.api_keys
|
||||
model_config_data = model_data.model_dump(exclude={"api_keys", "skip_validation"})
|
||||
# 添加租户ID
|
||||
model_config_data["tenant_id"] = tenant_id
|
||||
|
||||
model = ModelConfigRepository.create(db, model_config_data)
|
||||
db.flush() # 获取生成的 ID
|
||||
|
||||
if api_key_data:
|
||||
api_key_create_schema = ModelApiKeyCreate(
|
||||
model_config_id=model.id,
|
||||
**api_key_data.dict()
|
||||
)
|
||||
ModelApiKeyRepository.create(db, api_key_create_schema)
|
||||
if api_key_datas:
|
||||
for api_key_data in api_key_datas:
|
||||
api_key_create_schema = ModelApiKeyCreate(
|
||||
model_config_ids=[model.id],
|
||||
**api_key_data.model_dump()
|
||||
)
|
||||
ModelApiKeyRepository.create(db, api_key_create_schema)
|
||||
|
||||
db.commit()
|
||||
db.refresh(model)
|
||||
@@ -280,6 +303,116 @@ class ModelConfigService:
|
||||
db.refresh(model)
|
||||
return model
|
||||
|
||||
@staticmethod
|
||||
async def create_composite_model(db: Session, model_data: model_schema.CompositeModelCreate, tenant_id: uuid.UUID) -> ModelConfig:
|
||||
"""创建组合模型"""
|
||||
if ModelConfigRepository.get_by_name(db, model_data.name, tenant_id=tenant_id):
|
||||
raise BusinessException("模型名称已存在", BizCode.DUPLICATE_NAME)
|
||||
|
||||
# 验证所有 API Key 存在且类型匹配
|
||||
for api_key_id in model_data.api_key_ids:
|
||||
api_key = ModelApiKeyRepository.get_by_id(db, api_key_id)
|
||||
if not api_key:
|
||||
raise BusinessException(f"API Key {api_key_id} 不存在", BizCode.NOT_FOUND)
|
||||
|
||||
# 检查 API Key 关联的模型配置类型
|
||||
for model_config in api_key.model_configs:
|
||||
# chat 和 llm 类型可以兼容
|
||||
compatible_types = {ModelType.LLM, ModelType.CHAT}
|
||||
config_type = model_config.type
|
||||
request_type = model_data.type
|
||||
|
||||
if not (config_type == request_type or
|
||||
(config_type in compatible_types and request_type in compatible_types)):
|
||||
raise BusinessException(
|
||||
f"API Key {api_key_id} 关联的模型类型 ({model_config.type}) 与组合模型类型 ({model_data.type}) 不匹配",
|
||||
BizCode.INVALID_PARAMETER
|
||||
)
|
||||
# if model_config.is_composite:
|
||||
# raise BusinessException(
|
||||
# f"API Key {api_key_id} 关联的模型是组合模型,不能用于创建新的组合模型",
|
||||
# BizCode.INVALID_PARAMETER
|
||||
# )
|
||||
|
||||
# 创建组合模型
|
||||
model_config_data = {
|
||||
"tenant_id": tenant_id,
|
||||
"name": model_data.name,
|
||||
"type": model_data.type,
|
||||
"logo": model_data.logo,
|
||||
"description": model_data.description,
|
||||
"provider": "composite",
|
||||
"config": model_data.config,
|
||||
"is_active": model_data.is_active,
|
||||
"is_public": model_data.is_public,
|
||||
"is_composite": True
|
||||
}
|
||||
if "load_balance_strategy" in model_data.model_fields_set:
|
||||
model_config_data["load_balance_strategy"] = model_data.load_balance_strategy
|
||||
|
||||
model = ModelConfigRepository.create(db, model_config_data)
|
||||
db.flush()
|
||||
|
||||
# 关联 API Keys
|
||||
for api_key_id in model_data.api_key_ids:
|
||||
api_key = ModelApiKeyRepository.get_by_id(db, api_key_id)
|
||||
if api_key:
|
||||
model.api_keys.append(api_key)
|
||||
|
||||
db.commit()
|
||||
db.refresh(model)
|
||||
return model
|
||||
|
||||
@staticmethod
|
||||
async def update_composite_model(db: Session, model_id: uuid.UUID, model_data: model_schema.CompositeModelCreate, tenant_id: uuid.UUID) -> ModelConfig:
|
||||
"""更新组合模型"""
|
||||
existing_model = ModelConfigRepository.get_by_id(db, model_id, tenant_id=tenant_id)
|
||||
if not existing_model:
|
||||
raise BusinessException("模型配置不存在", BizCode.MODEL_NOT_FOUND)
|
||||
|
||||
if not existing_model.is_composite:
|
||||
raise BusinessException("该模型不是组合模型", BizCode.INVALID_PARAMETER)
|
||||
|
||||
# 验证所有 API Key 存在且类型匹配
|
||||
for api_key_id in model_data.api_key_ids:
|
||||
api_key = ModelApiKeyRepository.get_by_id(db, api_key_id)
|
||||
if not api_key:
|
||||
raise BusinessException(f"API Key {api_key_id} 不存在", BizCode.NOT_FOUND)
|
||||
|
||||
for model_config in api_key.model_configs:
|
||||
compatible_types = {ModelType.LLM, ModelType.CHAT}
|
||||
config_type = model_config.type
|
||||
request_type = existing_model.type
|
||||
|
||||
if not (config_type == request_type or
|
||||
(config_type in compatible_types and request_type in compatible_types)):
|
||||
raise BusinessException(
|
||||
f"API Key {api_key_id} 关联的模型类型 ({model_config.type}) 与组合模型类型 ({model_data.type}) 不匹配",
|
||||
BizCode.INVALID_PARAMETER
|
||||
)
|
||||
|
||||
# 更新基本信息
|
||||
existing_model.name = model_data.name
|
||||
# existing_model.type = model_data.type
|
||||
existing_model.logo = model_data.logo
|
||||
existing_model.description = model_data.description
|
||||
existing_model.config = model_data.config
|
||||
existing_model.is_active = model_data.is_active
|
||||
existing_model.is_public = model_data.is_public
|
||||
if "load_balance_strategy" in model_data.model_fields_set:
|
||||
existing_model.load_balance_strategy = model_data.load_balance_strategy
|
||||
|
||||
# 更新 API Keys 关联
|
||||
existing_model.api_keys.clear()
|
||||
for api_key_id in model_data.api_key_ids:
|
||||
api_key = ModelApiKeyRepository.get_by_id(db, api_key_id)
|
||||
if api_key:
|
||||
existing_model.api_keys.append(api_key)
|
||||
|
||||
db.commit()
|
||||
db.refresh(existing_model)
|
||||
return existing_model
|
||||
|
||||
@staticmethod
|
||||
def delete_model(db: Session, model_id: uuid.UUID, tenant_id: uuid.UUID | None = None) -> bool:
|
||||
"""删除模型配置"""
|
||||
@@ -324,27 +457,133 @@ class ModelApiKeyService:
|
||||
return ModelApiKeyRepository.get_by_model_config(db, model_config_id, is_active)
|
||||
|
||||
@staticmethod
|
||||
async def create_api_key(db: Session, api_key_data: ModelApiKeyCreate) -> ModelApiKey:
|
||||
"""创建API Key"""
|
||||
model_config = ModelConfigRepository.get_by_id(db, api_key_data.model_config_id)
|
||||
if not model_config:
|
||||
raise BusinessException("模型配置不存在", BizCode.MODEL_NOT_FOUND)
|
||||
|
||||
validation_result = await ModelConfigService.validate_model_config(
|
||||
async def create_api_key_by_provider(db: Session, data: model_schema.ModelApiKeyCreateByProvider) -> tuple[
|
||||
list[Any], list[Any]]:
|
||||
"""根据provider为多个ModelConfig创建API Key"""
|
||||
created_keys = []
|
||||
failed_models = [] # 记录验证失败的模型
|
||||
|
||||
for model_config_id in data.model_config_ids:
|
||||
model_config = ModelConfigRepository.get_by_id(db, model_config_id)
|
||||
if not model_config:
|
||||
continue
|
||||
|
||||
# 从ModelBase获取model_name
|
||||
model_name = model_config.model_base.name if model_config.model_base else model_config.name
|
||||
|
||||
# 检查是否存在API Key(包括软删除)
|
||||
existing_key = db.query(ModelApiKey).filter(
|
||||
ModelApiKey.api_key == data.api_key,
|
||||
ModelApiKey.provider == data.provider,
|
||||
ModelApiKey.model_name == model_name
|
||||
).first()
|
||||
|
||||
if existing_key:
|
||||
# 如果已存在,重新激活并更新
|
||||
if existing_key.is_active:
|
||||
continue
|
||||
existing_key.is_active = True
|
||||
existing_key.api_base = data.api_base
|
||||
existing_key.description = data.description
|
||||
existing_key.config = data.config
|
||||
existing_key.priority = data.priority
|
||||
existing_key.model_name = model_name
|
||||
|
||||
# 检查是否已关联该模型配置
|
||||
if model_config not in existing_key.model_configs:
|
||||
existing_key.model_configs.append(model_config)
|
||||
|
||||
created_keys.append(existing_key)
|
||||
continue
|
||||
|
||||
# 验证配置
|
||||
validation_result = await ModelConfigService.validate_model_config(
|
||||
db=db,
|
||||
model_name=api_key_data.model_name,
|
||||
provider=api_key_data.provider,
|
||||
api_key=api_key_data.api_key,
|
||||
api_base=api_key_data.api_base,
|
||||
model_type=model_config.type, # 传递模型类型
|
||||
model_name=model_name,
|
||||
provider=data.provider,
|
||||
api_key=data.api_key,
|
||||
api_base=data.api_base,
|
||||
model_type=model_config.type,
|
||||
test_message="Hello"
|
||||
)
|
||||
print(validation_result)
|
||||
if not validation_result["valid"]:
|
||||
raise BusinessException(
|
||||
f"模型配置验证失败: {validation_result['error']}",
|
||||
BizCode.INVALID_PARAMETER
|
||||
if not validation_result["valid"]:
|
||||
# 记录验证失败的模型,但不抛出异常
|
||||
failed_models.append(model_name)
|
||||
continue
|
||||
|
||||
# 创建API Key
|
||||
api_key_data = ModelApiKeyCreate(
|
||||
model_config_ids=[model_config_id],
|
||||
model_name=model_name,
|
||||
description=data.description,
|
||||
provider=data.provider,
|
||||
api_key=data.api_key,
|
||||
api_base=data.api_base,
|
||||
config=data.config,
|
||||
is_active=data.is_active,
|
||||
priority=data.priority
|
||||
)
|
||||
api_key_obj = ModelApiKeyRepository.create(db, api_key_data)
|
||||
created_keys.append(api_key_obj)
|
||||
|
||||
if created_keys:
|
||||
db.commit()
|
||||
for key in created_keys:
|
||||
db.refresh(key)
|
||||
|
||||
return created_keys, failed_models
|
||||
|
||||
@staticmethod
|
||||
async def create_api_key(db: Session, api_key_data: ModelApiKeyCreate) -> ModelApiKey:
|
||||
# 验证所有关联的模型配置是否存在
|
||||
if api_key_data.model_config_ids:
|
||||
for model_config_id in api_key_data.model_config_ids:
|
||||
model_config = ModelConfigRepository.get_by_id(db, model_config_id)
|
||||
if not model_config:
|
||||
raise BusinessException("模型配置不存在", BizCode.MODEL_NOT_FOUND)
|
||||
|
||||
# 检查API Key是否已存在(包括软删除)
|
||||
existing_key = db.query(ModelApiKey).filter(
|
||||
ModelApiKey.api_key == api_key_data.api_key,
|
||||
ModelApiKey.provider == api_key_data.provider,
|
||||
ModelApiKey.model_name == api_key_data.model_name
|
||||
).first()
|
||||
|
||||
if existing_key:
|
||||
if existing_key.is_active:
|
||||
# 如果已激活,跳过
|
||||
raise BusinessException("该API Key已存在", BizCode.DUPLICATE_NAME)
|
||||
# 如果已存在,重新激活并更新
|
||||
existing_key.is_active = True
|
||||
existing_key.api_base = api_key_data.api_base
|
||||
existing_key.description = api_key_data.description
|
||||
existing_key.config = api_key_data.config
|
||||
existing_key.priority = api_key_data.priority
|
||||
existing_key.model_name = api_key_data.model_name
|
||||
|
||||
# 检查是否已关联该模型配置
|
||||
if model_config not in existing_key.model_configs:
|
||||
existing_key.model_configs.append(model_config)
|
||||
|
||||
db.commit()
|
||||
db.refresh(existing_key)
|
||||
return existing_key
|
||||
|
||||
# 验证配置
|
||||
validation_result = await ModelConfigService.validate_model_config(
|
||||
db=db,
|
||||
model_name=api_key_data.model_name,
|
||||
provider=api_key_data.provider,
|
||||
api_key=api_key_data.api_key,
|
||||
api_base=api_key_data.api_base,
|
||||
model_type=model_config.type,
|
||||
test_message="Hello"
|
||||
)
|
||||
if not validation_result["valid"]:
|
||||
raise BusinessException(
|
||||
f"模型配置验证失败: {validation_result['error']}",
|
||||
BizCode.INVALID_PARAMETER
|
||||
)
|
||||
|
||||
api_key = ModelApiKeyRepository.create(db, api_key_data)
|
||||
db.commit()
|
||||
@@ -359,21 +598,19 @@ class ModelApiKeyService:
|
||||
raise BusinessException("API Key不存在", BizCode.NOT_FOUND)
|
||||
|
||||
# 获取关联的模型配置以获取模型类型
|
||||
model_config = ModelConfigRepository.get_by_id(db, existing_api_key.model_config_id)
|
||||
if not model_config:
|
||||
raise BusinessException("关联的模型配置不存在", BizCode.MODEL_NOT_FOUND)
|
||||
|
||||
validation_result = await ModelConfigService.validate_model_config(
|
||||
if existing_api_key.model_configs:
|
||||
model_config = existing_api_key.model_configs[0]
|
||||
|
||||
validation_result = await ModelConfigService.validate_model_config(
|
||||
db=db,
|
||||
model_name=api_key_data.model_name,
|
||||
provider=api_key_data.provider,
|
||||
api_key=api_key_data.api_key,
|
||||
api_base=api_key_data.api_base,
|
||||
model_type=model_config.type, # 传递模型类型
|
||||
model_name=api_key_data.model_name or existing_api_key.model_name,
|
||||
provider=api_key_data.provider or existing_api_key.provider,
|
||||
api_key=api_key_data.api_key or existing_api_key.api_key,
|
||||
api_base=api_key_data.api_base or existing_api_key.api_base,
|
||||
model_type=model_config.type,
|
||||
test_message="Hello"
|
||||
)
|
||||
print(validation_result)
|
||||
if not validation_result["valid"]:
|
||||
if not validation_result["valid"]:
|
||||
raise BusinessException(
|
||||
f"模型配置验证失败: {validation_result['error']}",
|
||||
BizCode.INVALID_PARAMETER
|
||||
@@ -417,3 +654,87 @@ class ModelApiKeyService:
|
||||
if api_kes and len(api_kes) > 0:
|
||||
return api_kes[0]
|
||||
raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING)
|
||||
|
||||
|
||||
|
||||
class ModelBaseService:
|
||||
"""基础模型服务"""
|
||||
|
||||
@staticmethod
|
||||
def get_model_base_list(db: Session, query: model_schema.ModelBaseQuery, tenant_id: uuid.UUID = None) -> List:
|
||||
models = ModelBaseRepository.get_list(db, query)
|
||||
|
||||
provider_groups = {}
|
||||
for m in models:
|
||||
model_dict = model_schema.ModelBase.model_validate(m).model_dump()
|
||||
if tenant_id:
|
||||
model_dict['is_added'] = ModelBaseRepository.check_added_by_tenant(db, m.id, tenant_id)
|
||||
|
||||
provider = m.provider
|
||||
if provider not in provider_groups:
|
||||
provider_groups[provider] = {
|
||||
"provider": provider,
|
||||
"models": []
|
||||
}
|
||||
provider_groups[provider]["models"].append(model_dict)
|
||||
|
||||
return list(provider_groups.values())
|
||||
|
||||
@staticmethod
|
||||
def get_model_base_by_id(db: Session, model_base_id: uuid.UUID):
|
||||
model = ModelBaseRepository.get_by_id(db, model_base_id)
|
||||
if not model:
|
||||
raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND)
|
||||
return model
|
||||
|
||||
@staticmethod
|
||||
def create_model_base(db: Session, data: model_schema.ModelBaseCreate):
|
||||
existing = ModelBaseRepository.get_by_name_and_provider(db, data.name, data.provider)
|
||||
if existing:
|
||||
raise BusinessException("模型已存在", BizCode.DUPLICATE_NAME)
|
||||
model_base = ModelBaseRepository.create(db, data.model_dump())
|
||||
db.commit()
|
||||
db.refresh(model_base)
|
||||
return model_base
|
||||
|
||||
@staticmethod
|
||||
def update_model_base(db: Session, model_base_id: uuid.UUID, data: model_schema.ModelBaseUpdate):
|
||||
model_base = ModelBaseRepository.update(db, model_base_id, data.model_dump(exclude_unset=True))
|
||||
if not model_base:
|
||||
raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND)
|
||||
db.commit()
|
||||
db.refresh(model_base)
|
||||
return model_base
|
||||
|
||||
@staticmethod
|
||||
def delete_model_base(db: Session, model_base_id: uuid.UUID) -> bool:
|
||||
success = ModelBaseRepository.delete(db, model_base_id)
|
||||
if not success:
|
||||
raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND)
|
||||
db.commit()
|
||||
return success
|
||||
|
||||
@staticmethod
|
||||
def add_model_from_plaza(db: Session, model_base_id: uuid.UUID, tenant_id: uuid.UUID) -> ModelConfig:
|
||||
model_base = ModelBaseRepository.get_by_id(db, model_base_id)
|
||||
if not model_base:
|
||||
raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND)
|
||||
|
||||
if ModelBaseRepository.check_added_by_tenant(db, model_base_id, tenant_id):
|
||||
raise BusinessException("模型已添加", BizCode.DUPLICATE_NAME)
|
||||
|
||||
model_config_data = {
|
||||
"model_id": model_base_id,
|
||||
"tenant_id": tenant_id,
|
||||
"name": model_base.name,
|
||||
"provider": model_base.provider,
|
||||
"type": model_base.type,
|
||||
"logo": model_base.logo,
|
||||
"description": model_base.description,
|
||||
"is_composite": False
|
||||
}
|
||||
model_config = ModelConfigRepository.create(db, model_config_data)
|
||||
ModelBaseRepository.increment_add_count(db, model_base_id)
|
||||
db.commit()
|
||||
db.refresh(model_config)
|
||||
return model_config
|
||||
|
||||
@@ -7,6 +7,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import MultiAgentConfig, AgentConfig, ModelConfig
|
||||
from app.models.multi_agent_model import AggregationStrategy, OrchestrationMode
|
||||
from app.repositories.model_repository import ModelApiKeyRepository
|
||||
from app.services.agent_registry import AgentRegistry
|
||||
from app.services.master_agent_router import MasterAgentRouter
|
||||
from app.services.conversation_state_manager import ConversationStateManager
|
||||
@@ -2546,10 +2547,14 @@ class MultiAgentOrchestrator:
|
||||
return self._smart_merge_results(results, strategy)
|
||||
|
||||
# 获取 API Key 配置
|
||||
api_key_config = self.db.query(ModelApiKey).filter(
|
||||
ModelApiKey.model_config_id == default_model_config_id,
|
||||
ModelApiKey.is_active == True
|
||||
).first()
|
||||
# api_key_config = self.db.query(ModelApiKey).join(
|
||||
# ModelConfig, ModelApiKey.model_configs
|
||||
# ).filter(
|
||||
# ModelConfig.id == default_model_config_id,
|
||||
# ModelApiKey.is_active.is_(True)
|
||||
# ).first()
|
||||
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, default_model_config_id)
|
||||
api_key_config = api_keys[0] if api_keys else None
|
||||
|
||||
if not api_key_config:
|
||||
logger.warning("Master Agent 没有可用的 API Key,使用简单整合")
|
||||
@@ -2703,10 +2708,14 @@ class MultiAgentOrchestrator:
|
||||
return
|
||||
|
||||
# 获取 API Key 配置
|
||||
api_key_config = self.db.query(ModelApiKey).filter(
|
||||
ModelApiKey.model_config_id == default_model_config_id,
|
||||
ModelApiKey.is_active == True
|
||||
).first()
|
||||
# api_key_config = self.db.query(ModelApiKey).join(
|
||||
# ModelConfig, ModelApiKey.model_configs
|
||||
# ).filter(
|
||||
# ModelConfig.id == default_model_config_id,
|
||||
# ModelApiKey.is_active.is_(True)
|
||||
# ).first()
|
||||
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, default_model_config_id)
|
||||
api_key_config = api_keys[0] if api_keys else None
|
||||
|
||||
if not api_key_config:
|
||||
logger.warning("Master Agent 没有可用的 API Key,使用简单整合")
|
||||
|
||||
@@ -74,7 +74,7 @@ class MultiAgentService:
|
||||
select(MultiAgentConfig)
|
||||
.where(
|
||||
MultiAgentConfig.app_id == app_id,
|
||||
MultiAgentConfig.is_active == True
|
||||
MultiAgentConfig.is_active.is_(True)
|
||||
)
|
||||
.order_by(MultiAgentConfig.updated_at.desc())
|
||||
).first()
|
||||
@@ -144,7 +144,7 @@ class MultiAgentService:
|
||||
select(MultiAgentConfig)
|
||||
.where(
|
||||
MultiAgentConfig.app_id == app_id,
|
||||
MultiAgentConfig.is_active == True
|
||||
MultiAgentConfig.is_active.is_(True)
|
||||
)
|
||||
.order_by(MultiAgentConfig.updated_at.desc())
|
||||
).first()
|
||||
|
||||
@@ -91,7 +91,7 @@ async def run_pilot_extraction(
|
||||
dialog = DialogData(
|
||||
context=context,
|
||||
ref_id="pilot_dialog_1",
|
||||
group_id=str(memory_config.workspace_id),
|
||||
end_user_id=str(memory_config.workspace_id),
|
||||
user_id=str(memory_config.tenant_id),
|
||||
apply_id=str(memory_config.config_id),
|
||||
metadata={"source": "pilot_run", "input_type": "frontend_text"},
|
||||
|
||||
@@ -16,7 +16,7 @@ from app.models.prompt_optimizer_model import (
|
||||
PromptOptimizerSession,
|
||||
RoleType
|
||||
)
|
||||
from app.repositories.model_repository import ModelConfigRepository
|
||||
from app.repositories.model_repository import ModelConfigRepository, ModelApiKeyRepository
|
||||
from app.repositories.prompt_optimizer_repository import (
|
||||
PromptOptimizerSessionRepository
|
||||
)
|
||||
@@ -168,7 +168,8 @@ class PromptOptimizerService:
|
||||
logger.info(f"Prompt optimization started, user_id={user_id}, session_id={session_id}")
|
||||
|
||||
# Create LLM instance
|
||||
api_config: ModelApiKey = model_config.api_keys[0]
|
||||
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config.id)
|
||||
api_config: ModelApiKey = api_keys[0] if api_keys else None
|
||||
llm = RedBearLLM(RedBearModelConfig(
|
||||
model_name=api_config.model_name,
|
||||
provider=api_config.provider,
|
||||
|
||||
@@ -4,6 +4,8 @@ import time
|
||||
import asyncio
|
||||
from typing import Optional, Dict, Any, AsyncGenerator
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.repositories.model_repository import ModelApiKeyRepository
|
||||
from app.services.memory_konwledges_server import write_rag
|
||||
from app.models import ReleaseShare, AppRelease, Conversation
|
||||
from app.services.conversation_service import ConversationService
|
||||
@@ -164,16 +166,20 @@ class SharedChatService:
|
||||
raise ResourceNotFoundException("模型配置", str(model_config_id))
|
||||
|
||||
# 获取 API Key
|
||||
stmt = (
|
||||
select(ModelApiKey)
|
||||
.where(
|
||||
ModelApiKey.model_config_id == model_config_id,
|
||||
ModelApiKey.is_active == True
|
||||
)
|
||||
.order_by(ModelApiKey.priority.desc())
|
||||
.limit(1)
|
||||
)
|
||||
api_key_obj = self.db.scalars(stmt).first()
|
||||
# stmt = (
|
||||
# select(ModelApiKey).join(
|
||||
# ModelConfig, ModelApiKey.model_configs
|
||||
# )
|
||||
# .where(
|
||||
# ModelConfig.id == model_config_id,
|
||||
# ModelApiKey.is_active.is_(True)
|
||||
# )
|
||||
# .order_by(ModelApiKey.priority.desc())
|
||||
# .limit(1)
|
||||
# )
|
||||
# api_key_obj = self.db.scalars(stmt).first()
|
||||
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config_id)
|
||||
api_key_obj = api_keys[0] if api_keys else None
|
||||
if not api_key_obj:
|
||||
raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING)
|
||||
|
||||
@@ -358,16 +364,20 @@ class SharedChatService:
|
||||
raise ResourceNotFoundException("模型配置", str(model_config_id))
|
||||
|
||||
# 获取 API Key
|
||||
stmt = (
|
||||
select(ModelApiKey)
|
||||
.where(
|
||||
ModelApiKey.model_config_id == model_config_id,
|
||||
ModelApiKey.is_active == True
|
||||
)
|
||||
.order_by(ModelApiKey.priority.desc())
|
||||
.limit(1)
|
||||
)
|
||||
api_key_obj = self.db.scalars(stmt).first()
|
||||
# stmt = (
|
||||
# select(ModelApiKey).join(
|
||||
# ModelConfig, ModelApiKey.model_configs
|
||||
# )
|
||||
# .where(
|
||||
# ModelConfig.id == model_config_id,
|
||||
# ModelApiKey.is_active.is_(True)
|
||||
# )
|
||||
# .order_by(ModelApiKey.priority.desc())
|
||||
# .limit(1)
|
||||
# )
|
||||
# api_key_obj = self.db.scalars(stmt).first()
|
||||
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config_id)
|
||||
api_key_obj = api_keys[0] if api_keys else None
|
||||
if not api_key_obj:
|
||||
raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING)
|
||||
|
||||
@@ -598,7 +608,7 @@ class SharedChatService:
|
||||
# 获取多 Agent 配置
|
||||
multi_agent_config = self.db.query(MultiAgentConfig).filter(
|
||||
MultiAgentConfig.app_id == release.app_id,
|
||||
MultiAgentConfig.is_active == True
|
||||
MultiAgentConfig.is_active.is_(True)
|
||||
).first()
|
||||
|
||||
if not multi_agent_config:
|
||||
@@ -695,7 +705,7 @@ class SharedChatService:
|
||||
# 获取多 Agent 配置
|
||||
multi_agent_config = self.db.query(MultiAgentConfig).filter(
|
||||
MultiAgentConfig.app_id == release.app_id,
|
||||
MultiAgentConfig.is_active == True
|
||||
MultiAgentConfig.is_active.is_(True)
|
||||
).first()
|
||||
|
||||
if not multi_agent_config:
|
||||
|
||||
@@ -155,10 +155,10 @@ class MemoryInsightHelper:
|
||||
"""
|
||||
query = """
|
||||
MATCH (d:Dialogue)
|
||||
WHERE d.group_id = $group_id AND d.created_at IS NOT NULL AND d.created_at <> ''
|
||||
WHERE d.end_user_id = $end_user_id AND d.created_at IS NOT NULL AND d.created_at <> ''
|
||||
RETURN d.created_at AS creation_time
|
||||
"""
|
||||
records = await self.neo4j_connector.execute_query(query, group_id=self.user_id)
|
||||
records = await self.neo4j_connector.execute_query(query, end_user_id=self.user_id)
|
||||
|
||||
if not records:
|
||||
return []
|
||||
@@ -211,17 +211,17 @@ class MemoryInsightHelper:
|
||||
async def get_social_connections(self) -> dict | None:
|
||||
"""Find the user with whom the most memories are shared."""
|
||||
query = """
|
||||
MATCH (c1:Chunk {group_id: $group_id})
|
||||
MATCH (c1:Chunk {end_user_id: $end_user_id})
|
||||
OPTIONAL MATCH (c1)-[:CONTAINS]->(s:Statement)
|
||||
OPTIONAL MATCH (s)<-[:CONTAINS]-(c2:Chunk)
|
||||
WHERE c1.group_id <> c2.group_id AND s IS NOT NULL AND c2 IS NOT NULL
|
||||
WITH c2.group_id AS other_user_id, COUNT(DISTINCT s) AS common_statements
|
||||
WHERE c1.end_user_id <> c2.end_user_id AND s IS NOT NULL AND c2 IS NOT NULL
|
||||
WITH c2.end_user_id AS other_user_id, COUNT(DISTINCT s) AS common_statements
|
||||
WHERE common_statements > 0
|
||||
RETURN other_user_id, common_statements
|
||||
ORDER BY common_statements DESC
|
||||
LIMIT 1
|
||||
"""
|
||||
records = await self.neo4j_connector.execute_query(query, group_id=self.user_id)
|
||||
records = await self.neo4j_connector.execute_query(query, end_user_id=self.user_id)
|
||||
if not records or not records[0].get("other_user_id"):
|
||||
return None
|
||||
|
||||
@@ -230,7 +230,7 @@ class MemoryInsightHelper:
|
||||
|
||||
time_range_query = """
|
||||
MATCH (c:Chunk)
|
||||
WHERE c.group_id IN [$user_id, $other_user_id]
|
||||
WHERE c.end_user_id IN [$user_id, $other_user_id]
|
||||
RETURN min(c.created_at) AS start_time, max(c.created_at) AS end_time
|
||||
"""
|
||||
time_records = await self.neo4j_connector.execute_query(
|
||||
@@ -294,11 +294,11 @@ class UserSummaryHelper:
|
||||
"""Fetch recent statements authored by the user/group for context."""
|
||||
query = (
|
||||
"MATCH (s:Statement) "
|
||||
"WHERE s.group_id = $group_id AND s.statement IS NOT NULL "
|
||||
"WHERE s.end_user_id = $end_user_id AND s.statement IS NOT NULL "
|
||||
"RETURN s.statement AS statement, s.created_at AS created_at "
|
||||
"ORDER BY created_at DESC LIMIT $limit"
|
||||
)
|
||||
rows = await self.connector.execute_query(query, group_id=self.user_id, limit=limit)
|
||||
rows = await self.connector.execute_query(query, end_user_id=self.user_id, limit=limit)
|
||||
records = []
|
||||
for r in rows:
|
||||
try:
|
||||
@@ -1152,7 +1152,7 @@ async def analytics_user_summary(end_user_id: Optional[str] = None) -> Dict[str,
|
||||
import re
|
||||
|
||||
# 创建 UserSummaryHelper 实例
|
||||
user_summary_tool = UserSummaryHelper(end_user_id or os.getenv("SELECTED_GROUP_ID", "group_123"))
|
||||
user_summary_tool = UserSummaryHelper(end_user_id or os.getenv("SELECTED_end_user_id", "group_123"))
|
||||
|
||||
try:
|
||||
# 1) 收集上下文数据
|
||||
@@ -1273,10 +1273,10 @@ async def analytics_node_statistics(
|
||||
if end_user_id:
|
||||
query = f"""
|
||||
MATCH (n:{node_type})
|
||||
WHERE n.group_id = $group_id
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
RETURN count(n) as count
|
||||
"""
|
||||
result = await _neo4j_connector.execute_query(query, group_id=end_user_id)
|
||||
result = await _neo4j_connector.execute_query(query, end_user_id=end_user_id)
|
||||
else:
|
||||
query = f"""
|
||||
MATCH (n:{node_type})
|
||||
@@ -1387,10 +1387,10 @@ async def analytics_memory_types(
|
||||
# 查询 Statement 节点数量
|
||||
query = """
|
||||
MATCH (n:Statement)
|
||||
WHERE n.group_id = $group_id
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
RETURN count(n) as count
|
||||
"""
|
||||
result = await _neo4j_connector.execute_query(query, group_id=end_user_id)
|
||||
result = await _neo4j_connector.execute_query(query, end_user_id=end_user_id)
|
||||
statement_count = result[0]["count"] if result and len(result) > 0 else 0
|
||||
# 取三分之一作为隐性记忆数量
|
||||
implicit_count = round(statement_count / 3)
|
||||
@@ -1504,7 +1504,7 @@ async def analytics_graph_data(
|
||||
包含节点、边和统计信息的字典
|
||||
"""
|
||||
try:
|
||||
# 1. 获取 group_id
|
||||
# 1. 获取 end_user_id
|
||||
user_uuid = uuid.UUID(end_user_id)
|
||||
repo = EndUserRepository(db)
|
||||
end_user = repo.get_by_id(user_uuid)
|
||||
@@ -1528,7 +1528,7 @@ async def analytics_graph_data(
|
||||
# 基于中心节点的扩展查询
|
||||
node_query = f"""
|
||||
MATCH path = (center)-[*1..{depth}]-(connected)
|
||||
WHERE center.group_id = $group_id
|
||||
WHERE center.end_user_id = $end_user_id
|
||||
AND elementId(center) = $center_node_id
|
||||
WITH collect(DISTINCT center) + collect(DISTINCT connected) as all_nodes
|
||||
UNWIND all_nodes as n
|
||||
@@ -1539,7 +1539,7 @@ async def analytics_graph_data(
|
||||
LIMIT $limit
|
||||
"""
|
||||
node_params = {
|
||||
"group_id": end_user_id,
|
||||
"end_user_id": end_user_id,
|
||||
"center_node_id": center_node_id,
|
||||
"limit": limit
|
||||
}
|
||||
@@ -1547,7 +1547,7 @@ async def analytics_graph_data(
|
||||
# 按节点类型过滤查询
|
||||
node_query = """
|
||||
MATCH (n)
|
||||
WHERE n.group_id = $group_id
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
AND labels(n)[0] IN $node_types
|
||||
RETURN
|
||||
elementId(n) as id,
|
||||
@@ -1556,7 +1556,7 @@ async def analytics_graph_data(
|
||||
LIMIT $limit
|
||||
"""
|
||||
node_params = {
|
||||
"group_id": end_user_id,
|
||||
"end_user_id": end_user_id,
|
||||
"node_types": node_types,
|
||||
"limit": limit
|
||||
}
|
||||
@@ -1564,7 +1564,7 @@ async def analytics_graph_data(
|
||||
# 查询所有节点
|
||||
node_query = """
|
||||
MATCH (n)
|
||||
WHERE n.group_id = $group_id
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
RETURN
|
||||
elementId(n) as id,
|
||||
labels(n)[0] as label,
|
||||
@@ -1572,7 +1572,7 @@ async def analytics_graph_data(
|
||||
LIMIT $limit
|
||||
"""
|
||||
node_params = {
|
||||
"group_id": end_user_id,
|
||||
"end_user_id": end_user_id,
|
||||
"limit": limit
|
||||
}
|
||||
|
||||
|
||||
@@ -528,7 +528,8 @@ class WorkflowService:
|
||||
self.conversation_service.add_message(
|
||||
conversation_id=conversation_id_uuid,
|
||||
role=message["role"],
|
||||
content=message["content"]
|
||||
content=message["content"],
|
||||
meta_data=None if message["role"] == "user" else {"usage": token_usage}
|
||||
)
|
||||
logger.info(f"Workflow Run Success, "
|
||||
f"execution_id: {execution.execution_id}, message count: {len(final_messages)}")
|
||||
@@ -678,7 +679,8 @@ class WorkflowService:
|
||||
self.conversation_service.add_message(
|
||||
conversation_id=conversation_id_uuid,
|
||||
role=message["role"],
|
||||
content=message["content"]
|
||||
content=message["content"],
|
||||
meta_data=None if message["role"] == "user" else {"usage": token_usage}
|
||||
)
|
||||
logger.info(f"Workflow Run Success, "
|
||||
f"execution_id: {execution.execution_id}, message count: {len(final_messages)}")
|
||||
@@ -761,7 +763,10 @@ class WorkflowService:
|
||||
|
||||
# 4. 获取工作空间 ID(从 app 获取)
|
||||
from app.models import App
|
||||
app = self.db.query(App).filter(App.id == app_id).first()
|
||||
app = self.db.query(App).filter(
|
||||
App.id == app_id,
|
||||
App.is_active.is_(True)
|
||||
).first()
|
||||
if not app:
|
||||
raise BusinessException(
|
||||
code=BizCode.NOT_FOUND,
|
||||
|
||||
Reference in New Issue
Block a user