Release/v0.2.2 (#258)

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* Fix/interface home (#182)

* [fix]Fix the interface for statistics of recent activities and applications

* [changes]Modify the code based on the AI review
1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter.
2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls.

* [fix]Fix the interface for statistics of recent activities and applications

* [changes]Modify the code based on the AI review
1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter.
2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls.

* Fix/optimize inerface (#183)

* [changes]Optimize the time consumption of the "/end_users" interface

* [fix]Optimize the time consumption of the "/hot_memory_tags" interface

* [changes]Optimize the time consumption of the "/end_users" interface

* [fix]Optimize the time consumption of the "/hot_memory_tags" interface

* [changes]Improve the code based on AI review

* Fix/memory mcp2 1 (#184)

* 优化快速检索的回复内容

* 优化快速检索的回复内容

* Fix/memory mcp2 1 (#185)

* 优化快速检索的回复内容

* 优化快速检索的回复内容

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* Fix/memory mcp2 1 (#188)

* 优化快速检索的回复内容

* 优化快速检索的回复内容

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* 解决冲突

* 解决冲突

* feat(home page): version description update

* Fix/memory mcp2 1 (#190)

* 优化快速检索的回复内容

* 优化快速检索的回复内容

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问

* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问

* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* feat(web): memory related interface parameter transfer adjustment

* 感知meta_data字段BUG修复

* Fix/memory bug fix (#171)

* feat(sandbox): add Python 3 code execution sandbox support

* feat(workflow): emit SSE events for node exception output

* perf(sandbox): optimize code encryption handling

* perf(workflow): update standard node output structure

* [add] migration script

* [modify] migration script

* feat(web): add workflow runtime info

* fix(web):  handleSSE bugfix

* fix(sandbox): prevent imports from being blocked when network is disabled

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* Fix/memory bug fix (#199)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* user_id->显示为config_id_old传输

* feat(web): update read_all_config select valueKey

* user_id->显示为config_id_old传输

* feat(workflow): Add a new node for executing code

* fix(web): KnowledgeConfigModal bugfix

* fix(web): iteration's variable add parameter-extractor  node

* fix(sandbox): treat non-zero exit codes as errors instead of relying only on stderr

* Fix/memory bug fix (#200)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* Refactor/benchmark test (#196)

* [changes]refactor locomo_test

* [fix]Fix the circular import of ModelParameters

* [changes]The benchmark test can run stably.

* [fix]Complete end-to-end LoCoMo repair

* [fix]Complete the end-to-end longmemeval and memsciqa fixes

* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.

* [changes]refactor locomo_test

* [fix]Fix the circular import of ModelParameters

* [changes]The benchmark test can run stably.

* [fix]Complete end-to-end LoCoMo repair

* [fix]Complete the end-to-end longmemeval and memsciqa fixes

* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.

* [changes]Benchmark test adaptation for end_user_id

* [changes]refactor locomo_test

* [fix]Fix the circular import of ModelParameters

* [changes]The benchmark test can run stably.

* [fix]Complete end-to-end LoCoMo repair

* [fix]Complete the end-to-end longmemeval and memsciqa fixes

* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.

* [fix]Complete the end-to-end longmemeval and memsciqa fixes

* [changes]Complete the benchmark test description document to ensure that the configuration parameters take effect.

* [changes]Benchmark test adaptation for end_user_id

* [modify] migration script

* delete benchmark-test (#204)

* Refactor: Move evaluation folder to redbear-mem-benchmark submodule

* [changes]Restore .gitmodules

* feat(web): workflow add code node

* 检查需要更改的格式问题

* Fix/redbear benchmark (#205)

* Refactor: Move evaluation folder to redbear-mem-benchmark submodule

* [changes]Update submodule reference

* Refactor: Move evaluation folder to redbear-mem-benchmark submodule

* [changes]Update submodule reference

* Remove duplicate evaluation submodule, use redbear-mem-benchmark instead

* Fix/memory bug fix (#207)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* fix(web): remove URI decode and encode

* [add] plugin system and base sso module

* 修复宿主列表获取memory_config_idBUG

* Fix/memory bug fix (#209)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* [modify] file local server url

* [add] migration script

* fix(workflow): fix activation and branch control issues in streaming output

* fix(workflow): fix function cache not taking effect and potential list index overflow

* style(workflow): enforce PEP8 style and remove redundant imports

* fix(workflow): fix streaming output error when variable is not a string

* [fix]remove aspose-slides

* perf(workflow): enhance streaming output node activation performance

* feat(workflow): store token usage in message table

* feat(web): add PageEmpty component

* feat(web): add PageTabs component

* perf(workflow): make memory configuration backward compatible

* feat(web): update model management

* config_id做映射

* config_id做映射

* Fix/memory bug fix (#211)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

* config_id做映射

* config_id做映射

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* feat(web): getModelListUrl add is_active param

* config_id做映射+1

* config_id做映射+1

* config_id做映射+1

* feat(web): remove file url replace

* Fix/memory bug fix (#212)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

* config_id做映射

* config_id做映射

* config_id做映射+1

* config_id做映射+1

* config_id做映射+1

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics

* feat(web): model logo update

* 应用层memory_content->memory_config

* fix(web): correct spelling

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

* Fix/memory bug fix (#215)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

* config_id做映射

* config_id做映射

* config_id做映射+1

* config_id做映射+1

* config_id做映射+1

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics

* fix(web): model loading update

* 统一字段为config_id_old

* 统一字段为config_id_old

* feat(model and app statistic): 1. Optimize the model list; 2. Increase the model combination; 3. Add a model square; 4. Add application management statistics

* 统一字段为config_id_old

* 统一字段为config_id_old

* memory_content暂时不修改

* memory_content暂时不修改

* Fix/memory bug fix (#217)

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 把group_id替换end_user_id

* 把group_id替换end_user_id_

* 把group_id替换end_user_id_

* config_config替换成memory_config

* config_config替换成memory_config

* [fix]Fix the memory interface to use end_user_id.

* config_config替换成memory_config

* config_config替换成memory_config

* config_config替换成memory_config

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID

* config_id字段改成UUID,与develop校对恢复

* 检查项目,修复group_id的遗留问题

* 检查项目,修复group_id的遗留问题

* 解决冲突

* 解决冲突

* end_user_id清理干净

* end_user_id清理干净

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 修复遗留合并BUG

* 感知meta_data字段BUG修复

* user_id->现实为config_id_old

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* user_id->显示为config_id_old传输

* 检查需要更改的格式问题

* 修复宿主列表获取memory_config_idBUG

* config_id做映射

* config_id做映射

* config_id做映射+1

* config_id做映射+1

* config_id做映射+1

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

* 应用层memory_content->memory_config

* 统一字段为config_id_old

* 统一字段为config_id_old

* 统一字段为config_id_old

* 统一字段为config_id_old

* memory_content暂时不修改

* memory_content暂时不修改

---------

Co-authored-by: lanceyq <1982376970@qq.com>

* feat(web): add app statistics

* fix(workflow): fix streaming output issues with multi-output End nodes

End nodes with multiple output segments could cause cursor errors or leave some
segments inactive, resulting in incorrect final outputs.
Unified _emit_active_chunks and _update_scope_activate to ensure all segments
are activated in order and streamed correctly.

* feat(web): add apps statistics api

* fix(web): agent's knowledge_bases bugfix

* Revert "feat(web): update read_all_config select valueKey"

This reverts commit 46f0f3cee9.

* [add] migrations script

* perf(workflow): make memory write node backward-compatible and defer config validation

* 旧数据兼容

* 旧数据兼容

* 旧数据兼容

* 旧数据兼容

* fix(web): model bugfix

* fix(web): model bugfix

* 提交遗漏 (#228)

* [fix] chat api for workflow

* [fix] web search set for v1 api

* fix(web): model bugfix

* fix(web): model list remove is_active

* fix(model): bug fix

* [add]migration script

* [fix] api

* [fix] api

* fix(web): model bugfix

* fix(model): the model type does not allow modification,  delete tts and speech2text type

* fix(model): bug fix

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* Add/develop memory (#239)

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* feat(web): model ui update

* feat(web): model ui update

* Add/develop memory (#243)

* 遗漏的历史映射

* 遗漏的历史映射

* fix(model): bug fix

* feat(web): model ui update

* Add/develop memory (#247)

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* 遗漏的历史映射

* [modify] migration script

* [add] migration script

* fix(web): change form message

* fix(web): the memoryContent field is compatible with numbers and strings

* feat(web): code node hidden

* fix(model):
1. create a basic model to check if the name and provider are duplicated.
2. The result shows error models because the provider created API Keys for all matching models.

---------

Co-authored-by: lixinyue <2569494688@qq.com>
Co-authored-by: lanceyq <1982376970@qq.com>
Co-authored-by: yujiangping <yujiangping@taofen8.com>
Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com>
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: Eternity <1533512157@qq.com>
Co-authored-by: lixiangcheng1 <lixiangcheng1@wanda.cn>
This commit is contained in:
Ke Sun
2026-01-30 14:51:34 +08:00
committed by GitHub
parent 988a41f5e4
commit 0159fdf149
320 changed files with 11769 additions and 11942 deletions

View File

@@ -55,8 +55,8 @@ class AgentRegistry:
"""
# 构建查询
stmt = select(AgentConfig).join(App).where(
AgentConfig.is_active == True,
App.is_active == True
AgentConfig.is_active.is_(True),
App.is_active.is_(True)
)
# 工作空间过滤(同工作空间或公开)

View File

@@ -758,7 +758,7 @@ class AppService:
)
# 构建查询条件
filters = [App.is_active == True]
filters = [App.is_active.is_(True)]
if type:
filters.append(App.type == type)
if visibility:
@@ -873,7 +873,7 @@ class AppService:
self._validate_workspace_access(app, workspace_id)
stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active == True).order_by(
stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active.is_(True)).order_by(
AgentConfig.updated_at.desc())
agent_cfg: Optional[AgentConfig] = self.db.scalars(stmt).first()
now = datetime.datetime.now()
@@ -1204,7 +1204,7 @@ class AppService:
default_model_config_id = None
if app.type == AppType.AGENT:
stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active == True).order_by(
stmt = select(AgentConfig).where(AgentConfig.app_id == app_id, AgentConfig.is_active.is_(True)).order_by(
AgentConfig.updated_at.desc())
agent_cfg = self.db.scalars(stmt).first()
if not agent_cfg:
@@ -1226,7 +1226,7 @@ class AppService:
select(MultiAgentConfig)
.where(
MultiAgentConfig.app_id == app_id,
MultiAgentConfig.is_active == True
MultiAgentConfig.is_active.is_(True)
)
.order_by(MultiAgentConfig.updated_at.desc())
)
@@ -1380,7 +1380,7 @@ class AppService:
stmt = (
select(AppRelease)
.where(AppRelease.app_id == app_id, AppRelease.is_active == True)
.where(AppRelease.app_id == app_id, AppRelease.is_active.is_(True))
.order_by(AppRelease.version.desc())
)
return list(self.db.scalars(stmt).all())

View File

@@ -0,0 +1,193 @@
"""应用统计服务"""
from datetime import datetime, timedelta
from typing import Dict, Any, List
import uuid
from sqlalchemy import func, and_, cast, Date
from sqlalchemy.orm import Session
from app.models.conversation_model import Conversation, Message
from app.models.end_user_model import EndUser
from app.models.api_key_model import ApiKey, ApiKeyLog
from app.core.exceptions import BusinessException
from app.core.error_codes import BizCode
class AppStatisticsService:
"""应用统计服务"""
def __init__(self, db: Session):
self.db = db
def get_app_statistics(
self,
app_id: uuid.UUID,
workspace_id: uuid.UUID,
start_date: int,
end_date: int
) -> Dict[str, Any]:
"""获取应用统计数据
Args:
app_id: 应用ID
workspace_id: 工作空间ID
start_date: 开始时间戳(毫秒)
end_date: 结束时间戳(毫秒)
Returns:
统计数据字典
"""
# 将毫秒时间戳转换为 datetime
start_dt = datetime.fromtimestamp(start_date / 1000)
end_dt = datetime.fromtimestamp(end_date / 1000) + timedelta(days=1)
# 1. 会话统计
conversations_stats = self._get_conversations_statistics(app_id, workspace_id, start_dt, end_dt)
# 2. 新增用户统计
users_stats = self._get_new_users_statistics(app_id, start_dt, end_dt)
# 3. API调用统计
api_stats = self._get_api_calls_statistics(app_id, start_dt, end_dt)
# 4. Token消耗统计
token_stats = self._get_token_statistics(app_id, start_dt, end_dt)
return {
"daily_conversations": conversations_stats["daily"],
"total_conversations": conversations_stats["total"],
"daily_new_users": users_stats["daily"],
"total_new_users": users_stats["total"],
"daily_api_calls": api_stats["daily"],
"total_api_calls": api_stats["total"],
"daily_tokens": token_stats["daily"],
"total_tokens": token_stats["total"]
}
def _get_conversations_statistics(
self,
app_id: uuid.UUID,
workspace_id: uuid.UUID,
start_dt: datetime,
end_dt: datetime
) -> Dict[str, Any]:
"""获取会话统计"""
# 每日会话数
daily_query = self.db.query(
cast(Conversation.created_at, Date).label('date'),
func.count(Conversation.id).label('count')
).filter(
and_(
Conversation.app_id == app_id,
Conversation.workspace_id == workspace_id,
Conversation.created_at >= start_dt,
Conversation.created_at < end_dt
)
).group_by(cast(Conversation.created_at, Date)).all()
daily_data = [{"date": str(row.date), "count": row.count} for row in daily_query]
total = sum(row["count"] for row in daily_data)
return {"daily": daily_data, "total": total}
def _get_new_users_statistics(
self,
app_id: uuid.UUID,
start_dt: datetime,
end_dt: datetime
) -> Dict[str, Any]:
"""获取新增用户统计"""
# 每日新增用户数
daily_query = self.db.query(
cast(EndUser.created_at, Date).label('date'),
func.count(EndUser.id).label('count')
).filter(
and_(
EndUser.app_id == app_id,
EndUser.created_at >= start_dt,
EndUser.created_at < end_dt
)
).group_by(cast(EndUser.created_at, Date)).all()
daily_data = [{"date": str(row.date), "count": row.count} for row in daily_query]
total = sum(row["count"] for row in daily_data)
return {"daily": daily_data, "total": total}
def _get_api_calls_statistics(
self,
app_id: uuid.UUID,
start_dt: datetime,
end_dt: datetime
) -> Dict[str, Any]:
"""获取API调用统计"""
# 每日API调用次数
daily_query = self.db.query(
cast(ApiKeyLog.created_at, Date).label('date'),
func.count(ApiKeyLog.id).label('count')
).join(
ApiKey, ApiKeyLog.api_key_id == ApiKey.id
).filter(
and_(
ApiKey.resource_id == app_id,
ApiKeyLog.created_at >= start_dt,
ApiKeyLog.created_at < end_dt
)
).group_by(cast(ApiKeyLog.created_at, Date)).all()
daily_data = [{"date": str(row.date), "count": row.count} for row in daily_query]
total = sum(row["count"] for row in daily_data)
return {"daily": daily_data, "total": total}
def _get_token_statistics(
self,
app_id: uuid.UUID,
start_dt: datetime,
end_dt: datetime
) -> Dict[str, Any]:
"""获取Token消耗统计从Message的meta_data中提取"""
from sqlalchemy import text
# 查询所有相关消息的token使用情况
# meta_data中可能包含: {"usage": {"total_tokens": 100}} 或 {"tokens": 100}
daily_query = self.db.query(
cast(Message.created_at, Date).label('date'),
Message.meta_data
).join(
Conversation, Message.conversation_id == Conversation.id
).filter(
and_(
Conversation.app_id == app_id,
Message.created_at >= start_dt,
Message.created_at < end_dt,
Message.meta_data.isnot(None)
)
).all()
# 按日期聚合token
daily_tokens = {}
for row in daily_query:
date_str = str(row.date)
meta = row.meta_data or {}
# 提取token数量支持多种格式
tokens = 0
if isinstance(meta, dict):
# 格式1: {"usage": {"total_tokens": 100}}
if "usage" in meta and isinstance(meta["usage"], dict):
tokens = meta["usage"].get("total_tokens", 0)
# 格式2: {"tokens": 100}
elif "tokens" in meta:
tokens = meta.get("tokens", 0)
# 格式3: {"total_tokens": 100}
elif "total_tokens" in meta:
tokens = meta.get("total_tokens", 0)
if date_str not in daily_tokens:
daily_tokens[date_str] = 0
daily_tokens[date_str] += int(tokens)
daily_data = [{"date": date, "tokens": tokens} for date, tokens in sorted(daily_tokens.items()) if tokens != 0]
total = sum(row["tokens"] for row in daily_data)
return {"daily": daily_data, "total": total}

View File

@@ -16,6 +16,7 @@ from app.core.exceptions import BusinessException
from app.core.logging_config import get_business_logger
from app.core.rag.nlp.search import knowledge_retrieval
from app.models import AgentConfig, ModelApiKey, ModelConfig
from app.repositories.model_repository import ModelApiKeyRepository
from app.repositories.tool_repository import ToolRepository
from app.schemas.prompt_schema import PromptMessageRole, render_prompt_message
from app.services import task_service
@@ -56,7 +57,7 @@ def create_long_term_memory_tool(memory_config: Dict[str, Any], end_user_id: str
长期记忆工具
"""
# search_switch = memory_config.get("search_switch", "2")
config_id= memory_config.get("memory_content",None)
config_id= memory_config.get("memory_content") or memory_config.get("memory_config",None)
logger.info(f"创建长期记忆工具,配置: end_user_id={end_user_id}, config_id={config_id}, storage_type={storage_type}")
@tool(args_schema=LongTermMemoryInput)
def long_term_memory(question: str) -> str:
@@ -92,7 +93,7 @@ def create_long_term_memory_tool(memory_config: Dict[str, Any], end_user_id: str
try:
memory_content = asyncio.run(
MemoryAgentService().read_memory(
group_id=end_user_id,
end_user_id=end_user_id,
message=question,
history=[],
search_switch="2",
@@ -106,9 +107,9 @@ def create_long_term_memory_tool(memory_config: Dict[str, Any], end_user_id: str
"app.core.memory.agent.read_message",
args=[end_user_id, question, [], "1", config_id, storage_type, user_rag_memory_id]
)
# result = task_service.get_task_memory_read_result(task.id)
# status = result.get("status")
# logger.info(f"读取任务状态:{status}")
result = task_service.get_task_memory_read_result(task.id)
status = result.get("status")
logger.info(f"读取任务状态:{status}")
finally:
db.close()
@@ -418,7 +419,7 @@ class DraftRunService:
)
memory_config_= agent_config.memory
config_id = memory_config_.get("memory_content")
config_id = memory_config_.get("memory_content") or memory_config_.get("memory_config",None)
# 7. 调用 Agent
result = await agent.chat(
@@ -644,7 +645,7 @@ class DraftRunService:
})
memory_config_ = agent_config.memory
config_id = memory_config_.get("memory_content")
config_id = memory_config_.get("memory_content") or memory_config_.get("memory_config",None)
# 9. 流式调用 Agent
full_content = ""
@@ -724,17 +725,21 @@ class DraftRunService:
Raises:
BusinessException: 当没有可用的 API Key 时
"""
stmt = (
select(ModelApiKey)
.where(
ModelApiKey.model_config_id == model_config_id,
ModelApiKey.is_active == True
)
.order_by(ModelApiKey.priority.desc())
.limit(1)
)
api_key = self.db.scalars(stmt).first()
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config_id)
# stmt = (
# select(ModelApiKey).join(
# ModelConfig, ModelApiKey.model_configs
# )
# .where(
# ModelConfig.id == model_config_id,
# ModelApiKey.is_active.is_(True)
# )
# .order_by(ModelApiKey.priority.desc())
# .limit(1)
# )
#
# api_key = self.db.scalars(stmt).first()
api_key = api_keys[0] if api_keys else None
if not api_key:
raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING)

View File

@@ -75,7 +75,7 @@ class EmotionAnalyticsService:
# 调用仓储层查询
tags = await self.emotion_repo.get_emotion_tags(
group_id=end_user_id,
end_user_id=end_user_id,
emotion_type=emotion_type,
start_date=start_date,
end_date=end_date,
@@ -157,7 +157,7 @@ class EmotionAnalyticsService:
# 调用仓储层查询
keywords = await self.emotion_repo.get_emotion_wordcloud(
group_id=end_user_id,
end_user_id=end_user_id,
emotion_type=emotion_type,
limit=limit
)
@@ -339,7 +339,7 @@ class EmotionAnalyticsService:
# 获取时间范围内的情绪数据
emotions = await self.emotion_repo.get_emotions_in_range(
group_id=end_user_id,
end_user_id=end_user_id,
time_range=time_range
)
@@ -505,7 +505,7 @@ class EmotionAnalyticsService:
)
config_service = MemoryConfigService(db)
memory_config = config_service.load_memory_config(
config_id=int(config_id),
config_id=(config_id),
service_name="EmotionAnalyticsService.generate_emotion_suggestions"
)
from app.core.memory.utils.llm.llm_utils import MemoryClientFactory
@@ -519,7 +519,7 @@ class EmotionAnalyticsService:
# 3. 获取情绪数据用于模式分析
emotions = await self.emotion_repo.get_emotions_in_range(
group_id=end_user_id,
end_user_id=end_user_id,
time_range="30d"
)
@@ -598,13 +598,13 @@ class EmotionAnalyticsService:
# 查询用户的实体和标签
query = """
MATCH (e:Entity)
WHERE e.group_id = $group_id
WHERE e.end_user_id = $end_user_id
RETURN e.name as name, e.type as type
ORDER BY e.created_at DESC
LIMIT 20
"""
entities = await connector.execute_query(query, group_id=end_user_id)
entities = await connector.execute_query(query, end_user_id=end_user_id)
# 提取兴趣标签
interests = [e["name"] for e in entities if e.get("type") in ["INTEREST", "HOBBY"]][:5]

View File

@@ -8,9 +8,11 @@ Classes:
"""
from typing import Dict, Any
from uuid import UUID
from sqlalchemy.orm import Session
from app.models.data_config_model import DataConfig
from app.models.memory_config_model import MemoryConfig
from app.core.logging_config import get_business_logger
logger = get_business_logger()
@@ -37,7 +39,7 @@ class EmotionConfigService:
self.db = db
logger.info("情绪配置服务初始化完成")
def get_emotion_config(self, config_id: int) -> Dict[str, Any]:
def get_emotion_config(self, config_id: UUID) -> Dict[str, Any]:
"""获取情绪引擎配置
查询指定配置ID的情绪相关配置字段。
@@ -61,8 +63,8 @@ class EmotionConfigService:
logger.info(f"获取情绪配置: config_id={config_id}")
# 查询配置
config = self.db.query(DataConfig).filter(
DataConfig.config_id == config_id
config = self.db.query(MemoryConfig).filter(
MemoryConfig.config_id == config_id
).first()
if not config:
@@ -144,7 +146,7 @@ class EmotionConfigService:
def update_emotion_config(
self,
config_id: int,
config_id: UUID,
config_data: Dict[str, Any]
) -> Dict[str, Any]:
"""更新情绪引擎配置
@@ -173,8 +175,8 @@ class EmotionConfigService:
self.validate_emotion_config(config_data)
# 查询配置
config = self.db.query(DataConfig).filter(
DataConfig.config_id == config_id
config = self.db.query(MemoryConfig).filter(
MemoryConfig.config_id == config_id
).first()
if not config:

View File

@@ -14,7 +14,7 @@ from app.core.memory.llm_tools.llm_client import LLMClientException
from app.core.memory.models.emotion_models import EmotionExtraction
from app.core.memory.utils.llm.llm_utils import MemoryClientFactory
from app.db import get_db_context
from app.models.data_config_model import DataConfig
from app.models.memory_config_model import MemoryConfig
logger = logging.getLogger(__name__)
@@ -60,7 +60,7 @@ class EmotionExtractionService:
async def extract_emotion(
self,
statement: str,
config: DataConfig
config: MemoryConfig
) -> Optional[EmotionExtraction]:
"""Extract emotion information from a statement.

View File

@@ -5,6 +5,7 @@ import uuid
from typing import Dict, Any, List, Optional, Tuple
from sqlalchemy.orm import Session
from app.repositories.model_repository import ModelApiKeyRepository
from app.services.conversation_state_manager import ConversationStateManager
from app.models import ModelConfig, AgentConfig
from app.core.logging_config import get_business_logger
@@ -382,11 +383,14 @@ class LLMRouter:
from app.core.models.base import RedBearModelConfig
from app.models import ModelApiKey, ModelType
# 获取 API Key 配置
api_key_config = self.db.query(ModelApiKey).filter(
ModelApiKey.model_config_id == self.routing_model_config.id,
ModelApiKey.is_active
).first()
# 获取 API Key 配置(通过关联关系)
# api_key_config = self.db.query(ModelApiKey).join(
# ModelConfig, ModelApiKey.model_configs
# ).filter(ModelConfig.id == self.routing_model_config.id,
# ModelApiKey.is_active == True
# ).first()
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, self.routing_model_config.id)
api_key_config = api_keys[0] if api_keys else None
if not api_key_config:
raise Exception("路由模型没有可用的 API Key")
@@ -419,6 +423,9 @@ class LLMRouter:
# 调用模型
response = await llm.ainvoke(prompt)
from app.services.model_service import ModelApiKeyService
ModelApiKeyService.record_api_key_usage(self.db, api_key_config.id)
# 提取响应内容
if hasattr(response, 'content'):

View File

@@ -5,7 +5,7 @@ import uuid
from typing import Dict, Any, List, Optional, Tuple
from sqlalchemy.orm import Session
from app.schemas import ModelParameters
from app.schemas.app_schema import ModelParameters
from app.services.conversation_state_manager import ConversationStateManager
from app.models import ModelConfig, AgentConfig
from app.core.logging_config import get_business_logger

View File

@@ -9,6 +9,7 @@ import os
import re
import time
import uuid
from uuid import UUID
from typing import Any, AsyncGenerator, Dict, List, Optional
import redis
@@ -27,6 +28,7 @@ from app.core.memory.analytics.hot_memory_tags import get_hot_memory_tags
from app.core.memory.utils.llm.llm_utils import MemoryClientFactory
from app.db import get_db_context
from app.models.knowledge_model import Knowledge, KnowledgeType
from app.repositories.memory_short_repository import ShortTermMemoryRepository
from app.repositories.neo4j.neo4j_connector import Neo4jConnector
from app.schemas.memory_agent_schema import Write_UserInput
from app.schemas.memory_config_schema import ConfigurationError
@@ -35,6 +37,7 @@ from app.services.memory_config_service import MemoryConfigService
from app.services.memory_konwledges_server import (
write_rag,
)
from langchain_core.messages import AIMessage
from langchain_core.messages import HumanMessage
from pydantic import BaseModel, Field
from sqlalchemy import func
@@ -54,25 +57,24 @@ _neo4j_connector = Neo4jConnector()
class MemoryAgentService:
"""Service for memory agent operations"""
def writer_messages_deal(self, messages, start_time, group_id, config_id, message, context):
def writer_messages_deal(self, messages, start_time, end_user_id, config_id, message, context):
duration = time.time() - start_time
if str(messages) == 'success':
logger.info(f"Write operation successful for group {group_id} with config_id {config_id}")
logger.info(f"Write operation successful for group {end_user_id} with config_id {config_id}")
# 记录成功的操作
if audit_logger:
audit_logger.log_operation(operation="WRITE", config_id=config_id, group_id=group_id, success=True,
audit_logger.log_operation(operation="WRITE", config_id=config_id, end_user_id=end_user_id, success=True,
duration=duration, details={"message_length": len(message)})
return context
else:
logger.warning(f"Write operation failed for group {group_id}")
logger.warning(f"Write operation failed for group {end_user_id}")
# 记录失败的操作
if audit_logger:
audit_logger.log_operation(
operation="WRITE",
config_id=config_id,
group_id=group_id,
end_user_id=end_user_id,
success=False,
duration=duration,
error=f"写入失败: {messages[:100]}"
@@ -173,10 +175,9 @@ class MemoryAgentService:
"""
logger.info("Reading log file")
current_file = os.path.abspath(__file__) # app/services/memory_agent_service.py
app_dir = os.path.dirname(os.path.dirname(current_file)) # app directory
project_root = os.path.dirname(app_dir) # redbear-mem directory
# Get log file path - use project root directory
from pathlib import Path
project_root = str(Path(__file__).resolve().parents[2]) # api directory
log_path = os.path.join(project_root, "logs", "agent_service.log")
summer = ''
@@ -215,9 +216,8 @@ class MemoryAgentService:
logger.info("Starting log content streaming")
# Get log file path - use project root directory
current_file = os.path.abspath(__file__) # app/services/memory_agent_service.py
app_dir = os.path.dirname(os.path.dirname(current_file)) # app directory
project_root = os.path.dirname(app_dir) # redbear-mem directory
from pathlib import Path
project_root = str(Path(__file__).resolve().parents[2]) # api directory
log_path = os.path.join(project_root, "logs", "agent_service.log")
# Check if file exists before starting stream
@@ -265,13 +265,13 @@ class MemoryAgentService:
logger.info("Log streaming completed, cleaning up resources")
# LogStreamer uses context manager for file handling, so cleanup is automatic
async def write_memory(self, group_id: str, messages: list[dict], config_id: Optional[str], db: Session, storage_type: str, user_rag_memory_id: str) -> str:
async def write_memory(self, end_user_id: str, messages: list[dict], config_id: Optional[uuid.UUID]|int, db: Session, storage_type: str, user_rag_memory_id: str) -> str:
"""
Process write operation with config_id
Args:
group_id: Group identifier (also used as end_user_id)
messages: Structured message list [{"role": "user", "content": "..."}, ...]
end_user_id: Group identifier (also used as end_user_id)
message: Message to write
config_id: Configuration ID from database
db: SQLAlchemy database session
storage_type: Storage type (neo4j or rag)
@@ -286,15 +286,15 @@ class MemoryAgentService:
# Resolve config_id if None using end_user's connected config
if config_id is None:
try:
connected_config = get_end_user_connected_config(group_id, db)
connected_config = get_end_user_connected_config(end_user_id, db)
config_id = connected_config.get("memory_config_id")
if config_id is None:
raise ValueError(f"No memory configuration found for end_user {group_id}. Please ensure the user has a connected memory configuration.")
raise ValueError(f"No memory configuration found for end_user {end_user_id}. Please ensure the user has a connected memory configuration.")
except Exception as e:
if "No memory configuration found" in str(e):
raise
logger.error(f"Failed to get connected config for end_user {group_id}: {e}")
raise ValueError(f"Unable to determine memory configuration for end_user {group_id}: {e}")
raise # Re-raise our specific error
logger.error(f"Failed to get connected config for end_user {end_user_id}: {e}")
raise ValueError(f"Unable to determine memory configuration for end_user {end_user_id}: {e}")
import time
start_time = time.time()
@@ -314,7 +314,7 @@ class MemoryAgentService:
# Log failed operation
if audit_logger:
duration = time.time() - start_time
audit_logger.log_operation(operation="WRITE", config_id=config_id, group_id=group_id, success=False, duration=duration, error=error_msg)
audit_logger.log_operation(operation="WRITE", config_id=config_id, end_user_id=end_user_id, success=False, duration=duration, error=error_msg)
raise ValueError(error_msg)
@@ -322,24 +322,25 @@ class MemoryAgentService:
if storage_type == "rag":
# For RAG storage, convert messages to single string
message_text = "\n".join([f"{msg['role']}: {msg['content']}" for msg in messages])
result = await write_rag(group_id, message_text, user_rag_memory_id)
result = await write_rag(end_user_id, message_text, user_rag_memory_id)
return result
else:
async with make_write_graph() as graph:
config = {"configurable": {"thread_id": group_id}}
config = {"configurable": {"thread_id": end_user_id}}
# Convert structured messages to LangChain messages
langchain_messages = []
for msg in messages:
if msg['role'] == 'user':
langchain_messages.append(HumanMessage(content=msg['content']))
elif msg['role'] == 'assistant':
from langchain_core.messages import AIMessage
langchain_messages.append(AIMessage(content=msg['content']))
print(100*'-')
print(langchain_messages)
print(100*'-')
# 初始状态 - 包含所有必要字段
initial_state = {
"messages": langchain_messages,
"group_id": group_id,
"end_user_id": end_user_id,
"memory_config": memory_config
}
@@ -356,14 +357,14 @@ class MemoryAgentService:
contents = massages.get('write_result')
# Convert messages back to string for logging
message_text = "\n".join([f"{msg['role']}: {msg['content']}" for msg in messages])
return self.writer_messages_deal(massagesstatus, start_time, group_id, config_id, message_text, contents)
return self.writer_messages_deal(massagesstatus, start_time, end_user_id, config_id, message_text, contents)
except Exception as e:
# Ensure proper error handling and logging
error_msg = f"Write operation failed: {str(e)}"
logger.error(error_msg)
if audit_logger:
duration = time.time() - start_time
audit_logger.log_operation(operation="WRITE", config_id=config_id, group_id=group_id, success=False, duration=duration, error=error_msg)
audit_logger.log_operation(operation="WRITE", config_id=config_id, end_user_id=end_user_id, success=False, duration=duration, error=error_msg)
raise ValueError(error_msg)
@@ -371,15 +372,14 @@ class MemoryAgentService:
async def read_memory(
self,
group_id: str,
end_user_id: str,
message: str,
history: List[Dict],
search_switch: str,
config_id: Optional[str],
config_id: Optional[uuid.UUID]|int,
db: Session,
storage_type: str,
user_rag_memory_id: str
) -> Dict:
user_rag_memory_id: str) -> Dict:
"""
Process read operation with config_id
@@ -389,7 +389,7 @@ class MemoryAgentService:
- "2": Direct answer based on context
Args:
group_id: Group identifier (also used as end_user_id)
end_user_id: Group identifier (also used as end_user_id)
message: User message
history: Conversation history
search_switch: Search mode switch
@@ -407,22 +407,22 @@ class MemoryAgentService:
import time
start_time = time.time()
logger.info(f"[PERF] read_memory started for group_id={group_id}, search_switch={search_switch}")
ori_message= message
# Resolve config_id if None using end_user's connected config
if config_id is None:
try:
connected_config = get_end_user_connected_config(group_id, db)
connected_config = get_end_user_connected_config(end_user_id, db)
config_id = connected_config.get("memory_config_id")
if config_id is None:
raise ValueError(f"No memory configuration found for end_user {group_id}. Please ensure the user has a connected memory configuration.")
raise ValueError(f"No memory configuration found for end_user {end_user_id}. Please ensure the user has a connected memory configuration.")
except Exception as e:
if "No memory configuration found" in str(e):
raise # Re-raise our specific error
logger.error(f"Failed to get connected config for end_user {group_id}: {e}")
raise ValueError(f"Unable to determine memory configuration for end_user {group_id}: {e}")
logger.error(f"Failed to get connected config for end_user {end_user_id}: {e}")
raise ValueError(f"Unable to determine memory configuration for end_user {end_user_id}: {e}")
logger.info(f"Read operation for group {group_id} with config_id {config_id}")
logger.info(f"Read operation for group {end_user_id} with config_id {config_id}")
# 导入审计日志记录器
try:
@@ -450,7 +450,7 @@ class MemoryAgentService:
audit_logger.log_operation(
operation="READ",
config_id=config_id,
group_id=group_id,
end_user_id=end_user_id,
success=False,
duration=duration,
error=error_msg
@@ -460,16 +460,16 @@ class MemoryAgentService:
# Step 2: Prepare history
history.append({"role": "user", "content": message})
logger.debug(f"Group ID:{group_id}, Message:{message}, History:{history}, Config ID:{config_id}")
logger.debug(f"Group ID:{end_user_id}, Message:{message}, History:{history}, Config ID:{config_id}")
# Step 3: Initialize MCP client and execute read workflow
graph_exec_start = time.time()
try:
async with make_read_graph() as graph:
config = {"configurable": {"thread_id": group_id}}
config = {"configurable": {"thread_id": end_user_id}}
# 初始状态 - 包含所有必要字段
initial_state = {"messages": [HumanMessage(content=message)], "search_switch": search_switch,
"group_id": group_id
"end_user_id": end_user_id
, "storage_type": storage_type, "user_rag_memory_id": user_rag_memory_id,
"memory_config": memory_config}
# 获取节点更新信息
@@ -544,9 +544,8 @@ class MemoryAgentService:
if intermediate_type == "search_result":
query = intermediate.get('query', '')
raw_results = intermediate.get('raw_results', {})
reranked_results = raw_results.get('reranked_results', [])
try:
reranked_results = raw_results.get('reranked_results', [])
statements = [statement['statement'] for statement in reranked_results.get('statements', [])]
except Exception:
statements = []
@@ -565,13 +564,13 @@ class MemoryAgentService:
if '信息不足,无法回答。' != str(summary) and str(search_switch).strip() != "2":
# 使用 upsert 方法
repo.upsert(
end_user_id=group_id,
messages=message,
end_user_id=end_user_id,
messages=ori_message,
aimessages=summary,
retrieved_content=retrieved_content,
search_switch=str(search_switch)
)
logger.info(f"成功保存短期记忆: group_id={group_id}, search_switch={search_switch}")
logger.info(f"成功保存短期记忆: end_user_id={end_user_id}, search_switch={search_switch}")
else:
logger.debug(f"跳过保存短期记忆: summary={summary[:50] if summary else 'None'}, search_switch={search_switch}")
@@ -587,7 +586,7 @@ class MemoryAgentService:
audit_logger.log_operation(
operation="READ",
config_id=config_id,
group_id=group_id,
end_user_id=end_user_id,
success=True,
duration=duration
)
@@ -599,20 +598,20 @@ class MemoryAgentService:
except Exception as e:
# Ensure proper error handling and logging
error_msg = f"Read operation failed: {str(e)}"
total_time = time.time() - start_time
logger.error(f"[PERF] read_memory failed after {total_time:.4f}s: {error_msg}")
logger.error(error_msg)
if audit_logger:
duration = time.time() - start_time
audit_logger.log_operation(
operation="READ",
config_id=config_id,
group_id=group_id,
end_user_id=end_user_id,
success=False,
duration=duration,
error=error_msg
)
raise ValueError(error_msg)
def get_messages_list(self, user_input: Write_UserInput) -> list[dict]:
"""
Get standardized message list from user input.
@@ -657,7 +656,7 @@ class MemoryAgentService:
logger.info(f"Validation successful: Structured message list, count: {len(user_input.messages)}")
return user_input.messages
async def classify_message_type(self, message: str, config_id: int, db: Session) -> Dict:
async def classify_message_type(self, message: str, config_id: UUID, db: Session) -> Dict:
"""
Determine the type of user message (read or write)
Updated to eliminate global variables in favor of explicit parameters.
@@ -672,6 +671,8 @@ class MemoryAgentService:
"""
logger.info("Classifying message type")
# Load configuration to get LLM model ID
config_service = MemoryConfigService(db)
memory_config = config_service.load_memory_config(
@@ -682,9 +683,9 @@ class MemoryAgentService:
status = await status_typle(message, memory_config.llm_model_id)
logger.debug(f"Message type: {status}")
return status
async def generate_summary_from_retrieve(
self,
end_user_id: str,
retrieve_info: str,
history: List[Dict],
query: str,
@@ -706,6 +707,18 @@ class MemoryAgentService:
Returns:
生成的答案文本
"""
if config_id is None:
try:
config_id = get_end_user_connected_config(end_user_id, db)
config_id = config_id.get('memory_config_id')
if config_id is None:
raise ValueError(
f"No memory configuration found for end_user {end_user_id}. Please ensure the user has a connected memory configuration.")
except Exception as e:
if "No memory configuration found" in str(e):
raise # Re-raise our specific error
logger.error(f"Failed to get connected config for end_user {end_user_id}: {e}")
raise ValueError(f"Unable to determine memory configuration for end_user {end_user_id}: {e}")
logger.info(f"Generating summary from retrieve info for query: {query[:50]}...")
try:
@@ -731,7 +744,7 @@ class MemoryAgentService:
state=state,
history=history,
retrieve_info=retrieve_info,
template_name='Retrieve_Summary_prompt.jinja2',
template_name='direct_summary_prompt.jinja2',
operation_name='retrieve_summary',
response_model=RetrieveSummaryResponse,
search_mode="1"
@@ -755,7 +768,7 @@ class MemoryAgentService:
"""
统计知识库类型分布,包含:
1. PostgreSQL 中的知识库类型General, Web, Third-party, Folder根据 workspace_id 过滤)
2. Neo4j 中的 memory 类型(仅统计 Chunk 数量,根据 end_user_id/group_id 过滤)
2. Neo4j 中的 memory 类型(仅统计 Chunk 数量,根据 end_user_id/end_user_id 过滤)
3. total: 所有类型的总和
参数:
@@ -841,11 +854,11 @@ class MemoryAgentService:
for end_user in end_users:
end_user_id_str = str(end_user.id)
memory_query = """
MATCH (n:Chunk) WHERE n.group_id = $group_id RETURN count(n) AS Count
MATCH (n:Chunk) WHERE n.end_user_id = $end_user_id RETURN count(n) AS Count
"""
neo4j_result = await _neo4j_connector.execute_query(
memory_query,
group_id=end_user_id_str,
end_user_id=end_user_id_str,
)
chunk_count = neo4j_result[0]["Count"] if neo4j_result else 0
total_chunks += chunk_count
@@ -885,7 +898,7 @@ class MemoryAgentService:
获取指定用户的热门记忆标签
参数:
- end_user_id: 用户ID可选对应Neo4j中的group_id字段
- end_user_id: 用户ID可选对应Neo4j中的end_user_id字段
- limit: 返回标签数量限制
返回格式:
@@ -895,7 +908,7 @@ class MemoryAgentService:
]
"""
try:
# by_user=False 表示按 group_id 查询在Neo4j中group_id就是用户维度
# by_user=False 表示按 end_user_id 查询在Neo4j中end_user_id就是用户维度
tags = await get_hot_memory_tags(end_user_id, limit=limit, by_user=False)
payload=[]
for tag, freq in tags:
@@ -970,21 +983,21 @@ class MemoryAgentService:
# 查询该用户的语句
query = (
"MATCH (s:Statement) "
"WHERE ($group_id IS NULL OR s.group_id = $group_id) AND s.statement IS NOT NULL "
"WHERE ($end_user_id IS NULL OR s.end_user_id = $end_user_id) AND s.statement IS NOT NULL "
"RETURN s.statement AS statement "
"ORDER BY s.created_at DESC LIMIT 100"
)
rows = await connector.execute_query(query, group_id=end_user_id)
rows = await connector.execute_query(query, end_user_id=end_user_id)
statements = [r.get("statement", "") for r in rows if r.get("statement")]
# 查询该用户的热门实体
entity_query = (
"MATCH (e:ExtractedEntity) "
"WHERE ($group_id IS NULL OR e.group_id = $group_id) AND e.entity_type <> '人物' AND e.name IS NOT NULL "
"WHERE ($end_user_id IS NULL OR e.end_user_id = $end_user_id) AND e.entity_type <> '人物' AND e.name IS NOT NULL "
"RETURN e.name AS name, count(e) AS frequency "
"ORDER BY frequency DESC LIMIT 20"
)
entity_rows = await connector.execute_query(entity_query, group_id=end_user_id)
entity_rows = await connector.execute_query(entity_query, end_user_id=end_user_id)
entities = [f"{r['name']} ({r['frequency']})" for r in entity_rows]
await connector.close()
@@ -1037,14 +1050,14 @@ class MemoryAgentService:
names_to_exclude = ['AI', 'Caroline', 'Melanie', 'Jon', 'Gina', '用户', 'AI助手', 'John', 'Maria']
hot_tag_query = (
"MATCH (e:ExtractedEntity) "
"WHERE ($group_id IS NULL OR e.group_id = $group_id) AND e.entity_type <> '人物' "
"WHERE ($end_user_id IS NULL OR e.end_user_id = $end_user_id) AND e.entity_type <> '人物' "
"AND e.name IS NOT NULL AND NOT e.name IN $names_to_exclude "
"RETURN e.name AS name, count(e) AS frequency "
"ORDER BY frequency DESC LIMIT 4"
)
hot_tag_rows = await connector.execute_query(
hot_tag_query,
group_id=end_user_id,
end_user_id=end_user_id,
names_to_exclude=names_to_exclude
)
await connector.close()
@@ -1079,9 +1092,8 @@ class MemoryAgentService:
logger.info("Starting log content streaming")
# Get log file path - use project root directory
current_file = os.path.abspath(__file__) # app/services/memory_agent_service.py
app_dir = os.path.dirname(os.path.dirname(current_file)) # app directory
project_root = os.path.dirname(app_dir) # redbear-mem directory
from pathlib import Path
project_root = str(Path(__file__).resolve().parents[2]) # api directory
log_path = os.path.join(project_root, "logs", "agent_service.log")
# Check if file exists before starting stream
@@ -1179,6 +1191,16 @@ def get_end_user_connected_config(end_user_id: str, db: Session) -> Dict[str, An
# 3. 从 config 中提取 memory_config_id
config = latest_release.config or {}
# 如果 config 是字符串,解析为字典
if isinstance(config, str):
import json
try:
config = json.loads(config)
except json.JSONDecodeError:
logger.warning(f"Failed to parse config JSON for release {latest_release.id}")
config = {}
memory_obj = config.get('memory', {})
memory_config_id = memory_obj.get('memory_content') if isinstance(memory_obj, dict) else None
@@ -1217,7 +1239,7 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
"""
from app.models.app_release_model import AppRelease
from app.models.end_user_model import EndUser
from app.models.data_config_model import DataConfig
from app.models.memory_config_model import MemoryConfig
from sqlalchemy import select
logger.info(f"Batch getting connected configs for {len(end_user_ids)} end_users")
@@ -1230,10 +1252,10 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
# 1. 批量查询所有 end_user 及其 app_id
end_users = db.query(EndUser).filter(EndUser.id.in_(end_user_ids)).all()
# 创建 end_user_id -> app_id 的映射
user_to_app = {str(eu.id): eu.app_id for eu in end_users}
# 记录未找到的用户
found_user_ids = set(user_to_app.keys())
missing_user_ids = set(end_user_ids) - found_user_ids
@@ -1243,7 +1265,7 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
result[user_id] = {"memory_config_id": None, "memory_config_name": None}
# 2. 批量获取所有相关应用的最新发布版本
app_ids = list(user_to_app.values())
app_ids = list(set(user_to_app.values()))
if not app_ids:
return result
@@ -1263,6 +1285,8 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
# 3. 收集所有 memory_config_id 并批量查询配置名称
memory_config_ids = []
old_config_ids = [] # 存储旧的整数ID
for end_user_id, app_id in user_to_app.items():
release = app_to_release.get(app_id)
if release:
@@ -1270,18 +1294,42 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
memory_obj = config.get('memory', {})
memory_config_id = memory_obj.get('memory_content') if isinstance(memory_obj, dict) else None
if memory_config_id:
memory_config_ids.append(memory_config_id)
# 判断是否为UUID格式
if len(str(memory_config_id))>=5:
uuid.UUID(str(memory_config_id))
memory_config_ids.append(memory_config_id)
else:
old_config_ids.append(str(memory_config_id))
# 批量查询 memory_config_name
config_id_to_name = {}
# 记录分类结果
if memory_config_ids or old_config_ids:
logger.info(f"Collected {len(memory_config_ids)} UUID config_ids and {len(old_config_ids)} old integer config_ids")
if old_config_ids:
logger.debug(f"Old config IDs: {old_config_ids}")
# 查询新的UUID格式的config_id
if memory_config_ids:
memory_configs = db.query(DataConfig).filter(DataConfig.config_id.in_(memory_config_ids)).all()
config_id_to_name = {str(mc.config_id): mc.config_name for mc in memory_configs}
memory_configs = db.query(MemoryConfig).filter(MemoryConfig.config_id.in_(memory_config_ids)).all()
config_id_to_name.update({str(mc.config_id): mc.config_name for mc in memory_configs})
# 查询旧的整数ID通过config_id_old字段
if old_config_ids:
old_memory_configs = db.query(MemoryConfig).filter(MemoryConfig.config_id_old.in_(old_config_ids)).all()
# 使用config_id_old作为key这样后面查找时能匹配上
config_id_to_name.update({str(mc.config_id_old): mc.config_name for mc in old_memory_configs})
# 同时也添加config_id作为key方便后续使用
for mc in old_memory_configs:
if mc.config_id_old:
config_id_to_name[str(mc.config_id)] = mc.config_name
logger.info(f"Found {len(old_memory_configs)} configs for old IDs")
# 4. 构建最终结果
for end_user_id, app_id in user_to_app.items():
release = app_to_release.get(app_id)
if not release:
logger.warning(f"No active release found for app: {app_id} (end_user: {end_user_id})")
result[end_user_id] = {"memory_config_id": None, "memory_config_name": None}
@@ -1292,7 +1340,7 @@ def get_end_users_connected_configs_batch(end_user_ids: List[str], db: Session)
memory_obj = config.get('memory', {})
memory_config_id = memory_obj.get('memory_content') if isinstance(memory_obj, dict) else None
# 获取配置名称
# 获取配置名称使用字符串形式的ID进行查找兼容新旧格式
memory_config_name = config_id_to_name.get(str(memory_config_id)) if memory_config_id else None
result[end_user_id] = {

View File

@@ -25,7 +25,7 @@ class MemoryAPIService:
This service provides a thin layer that:
1. Validates end_user exists and belongs to the authorized workspace
2. Maps end_user_id to group_id for memory operations
2. Maps end_user_id to end_user_id for memory operations
3. Delegates to MemoryAgentService for actual memory read/write operations
"""
@@ -68,7 +68,7 @@ class MemoryAPIService:
)
end_user = self.db.query(EndUser).filter(EndUser.id == end_user_uuid).first()
if not end_user:
logger.warning(f"End user not found: {end_user_id}")
raise ResourceNotFoundException(
@@ -77,7 +77,10 @@ class MemoryAPIService:
)
# Verify end_user belongs to the workspace via App relationship
app = self.db.query(App).filter(App.id == end_user.app_id).first()
app = self.db.query(App).filter(
App.id == end_user.app_id,
App.is_active.is_(True)
).first()
if not app:
logger.warning(f"App not found for end_user: {end_user_id}")
@@ -115,7 +118,7 @@ class MemoryAPIService:
Args:
workspace_id: Workspace ID for resource validation
end_user_id: End user identifier (used as group_id)
end_user_id: End user identifier (used as end_user_id)
message: Message content to store
config_id: Optional memory configuration ID
storage_type: Storage backend (neo4j or rag)
@@ -133,14 +136,13 @@ class MemoryAPIService:
# Validate end_user exists and belongs to workspace
self.validate_end_user(end_user_id, workspace_id)
# Use end_user_id as group_id for memory operations
group_id = end_user_id
# Use end_user_id as end_user_id for memory operations
try:
# Delegate to MemoryAgentService
result = await MemoryAgentService().write_memory(
group_id=group_id,
message=message,
end_user_id=end_user_id,
messages=message,
config_id=config_id,
db=self.db,
storage_type=storage_type,
@@ -186,7 +188,7 @@ class MemoryAPIService:
Args:
workspace_id: Workspace ID for resource validation
end_user_id: End user identifier (used as group_id)
end_user_id: End user identifier (used as end_user_id)
message: Query message
search_switch: Search mode (0=deep search with verification, 1=deep search, 2=fast search)
config_id: Optional memory configuration ID
@@ -205,13 +207,13 @@ class MemoryAPIService:
# Validate end_user exists and belongs to workspace
self.validate_end_user(end_user_id, workspace_id)
# Use end_user_id as group_id for memory operations
group_id = end_user_id
# Use end_user_id as end_user_id for memory operations
try:
# Delegate to MemoryAgentService
result = await MemoryAgentService().read_memory(
group_id=group_id,
end_user_id=end_user_id,
message=message,
history=[],
search_switch=search_switch,

View File

@@ -326,7 +326,7 @@ class MemoryBaseService:
Args:
summary_id: Summary节点的ID
end_user_id: 终端用户ID (group_id)
end_user_id: 终端用户ID (end_user_id)
Returns:
最大emotion_intensity对应的emotion_type如果没有则返回None
@@ -334,7 +334,7 @@ class MemoryBaseService:
try:
query = """
MATCH (s:MemorySummary)
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
MATCH (s)-[:DERIVED_FROM_STATEMENT]->(stmt:Statement)
WHERE stmt.emotion_type IS NOT NULL
AND stmt.emotion_intensity IS NOT NULL
@@ -347,7 +347,7 @@ class MemoryBaseService:
result = await self.neo4j_connector.execute_query(
query,
summary_id=summary_id,
group_id=end_user_id
end_user_id=end_user_id
)
if result and len(result) > 0:
@@ -381,10 +381,10 @@ class MemoryBaseService:
if end_user_id:
query = """
MATCH (n:MemorySummary)
WHERE n.group_id = $group_id
WHERE n.end_user_id = $end_user_id
RETURN count(n) as count
"""
result = await self.neo4j_connector.execute_query(query, group_id=end_user_id)
result = await self.neo4j_connector.execute_query(query, end_user_id=end_user_id)
else:
query = """
MATCH (n:MemorySummary)
@@ -423,12 +423,12 @@ class MemoryBaseService:
if end_user_id:
semantic_query = """
MATCH (e:ExtractedEntity)
WHERE e.group_id = $group_id AND e.is_explicit_memory = true
WHERE e.end_user_id = $end_user_id AND e.is_explicit_memory = true
RETURN count(e) as count
"""
semantic_result = await self.neo4j_connector.execute_query(
semantic_query,
group_id=end_user_id
end_user_id=end_user_id
)
else:
semantic_query = """
@@ -519,7 +519,7 @@ class MemoryBaseService:
"""
if end_user_id:
query += " AND n.group_id = $group_id"
query += " AND n.end_user_id = $end_user_id"
query += """
RETURN sum(CASE WHEN n.activation_value IS NOT NULL AND n.activation_value < $threshold THEN 1 ELSE 0 END) as low_activation_nodes
@@ -528,7 +528,7 @@ class MemoryBaseService:
# 设置查询参数
params = {'threshold': forgetting_threshold}
if end_user_id:
params['group_id'] = end_user_id
params['end_user_id'] = end_user_id
# 执行查询
result = await self.neo4j_connector.execute_query(query, **params)

View File

@@ -7,14 +7,15 @@ This service eliminates code duplication between MemoryAgentService and MemorySt
import time
from datetime import datetime
from app.models.memory_config_model import MemoryConfig as MemoryConfigModel
from sqlalchemy import select
from app.core.logging_config import get_config_logger, get_logger
from app.core.validators.memory_config_validators import (
validate_and_resolve_model_id,
validate_embedding_model,
validate_model_exists_and_active,
)
from app.repositories.data_config_repository import DataConfigRepository
from app.repositories.memory_config_repository import MemoryConfigRepository
from app.schemas.memory_config_schema import (
ConfigurationError,
InvalidConfigError,
@@ -23,20 +24,24 @@ from app.schemas.memory_config_schema import (
ModelNotFoundError,
)
from sqlalchemy.orm import Session
from uuid import UUID
logger = get_logger(__name__)
config_logger = get_config_logger()
import uuid
def _validate_config_id(config_id):
"""Validate configuration ID format."""
def _validate_config_id(config_id, db: Session = None):
"""Validate configuration ID format (supports both UUID and integer)."""
if isinstance(config_id, uuid.UUID):
return config_id
if config_id is None:
raise InvalidConfigError(
"Configuration ID cannot be None",
field_name="config_id",
invalid_value=config_id,
)
if isinstance(config_id, int):
if config_id <= 0:
raise InvalidConfigError(
@@ -44,27 +49,56 @@ def _validate_config_id(config_id):
field_name="config_id",
invalid_value=config_id,
)
# 如果提供了数据库会话,尝试通过 user_id 查询 config_id
if db is not None:
# 查询 user_id 匹配的记录
stmt = select(MemoryConfigModel).where(MemoryConfigModel.config_id_old == str(config_id))
result = db.execute(stmt).scalars().first()
if result:
logger.info(f"Found config_id {result.config_id} for user_id {config_id}")
return result.config_id
return config_id
if isinstance(config_id, str):
config_id_stripped = config_id.strip()
# Try parsing as UUID first
try:
parsed_id = int(config_id.strip())
return uuid.UUID(config_id_stripped)
except ValueError:
pass
# Fall back to integer parsing
try:
parsed_id = int(config_id_stripped)
if parsed_id <= 0:
raise InvalidConfigError(
f"Configuration ID must be positive: {parsed_id}",
field_name="config_id",
invalid_value=config_id,
)
# 如果提供了数据库会话,尝试通过 user_id 查询 config_id
if db is not None:
# 查询 user_id 匹配的记录
stmt = select(MemoryConfigModel).where(MemoryConfigModel.user_id == str(parsed_id))
result = db.execute(stmt).scalars().first()
if result:
logger.info(f"Found config_id {result.config_id} for user_id {parsed_id}")
return result.config_id
return parsed_id
except ValueError:
raise InvalidConfigError(
f"Invalid configuration ID format: '{config_id}'",
f"Invalid configuration ID format: '{config_id}' (must be UUID or positive integer)",
field_name="config_id",
invalid_value=config_id,
)
raise InvalidConfigError(
f"Invalid type for configuration ID: expected int or str, got {type(config_id).__name__}",
f"Invalid type for configuration ID: expected UUID, int or str, got {type(config_id).__name__}",
field_name="config_id",
invalid_value=config_id,
)
@@ -73,61 +107,61 @@ def _validate_config_id(config_id):
class MemoryConfigService:
"""
Centralized service for memory configuration loading and validation.
This class provides a single implementation of configuration loading logic
that can be shared across multiple services, eliminating code duplication.
Usage:
config_service = MemoryConfigService(db)
memory_config = config_service.load_memory_config(config_id)
model_config = config_service.get_model_config(model_id)
"""
def __init__(self, db: Session):
"""Initialize the service with a database session.
Args:
db: SQLAlchemy database session
"""
self.db = db
def load_memory_config(
self,
config_id: int,
config_id: UUID,
service_name: str = "MemoryConfigService",
) -> MemoryConfig:
"""
Load memory configuration from database by config_id.
Args:
config_id: Configuration ID from database
config_id: Configuration ID (UUID) from database
service_name: Name of the calling service (for logging purposes)
Returns:
MemoryConfig: Immutable configuration object
Raises:
ConfigurationError: If validation fails
"""
start_time = time.time()
config_logger.info(
"Starting memory configuration loading",
extra={
"operation": "load_memory_config",
"service": service_name,
"config_id": config_id,
"config_id": str(config_id),
},
)
logger.info(f"Loading memory configuration from database: config_id={config_id}")
try:
validated_config_id = _validate_config_id(config_id)
validated_config_id = _validate_config_id(config_id, self.db)
# Step 1: Get config and workspace
db_query_start = time.time()
result = DataConfigRepository.get_config_with_workspace(self.db, validated_config_id)
result = MemoryConfigRepository.get_config_with_workspace(self.db, validated_config_id)
db_query_time = time.time() - db_query_start
logger.info(f"[PERF] Config+Workspace query: {db_query_time:.4f}s")
if not result:
@@ -136,18 +170,18 @@ class MemoryConfigService:
"Configuration not found in database",
extra={
"operation": "load_memory_config",
"config_id": validated_config_id,
"config_id": str(config_id),
"load_result": "not_found",
"elapsed_ms": elapsed_ms,
"service": service_name,
},
)
raise ConfigurationError(
f"Configuration {validated_config_id} not found in database"
f"Configuration {config_id} not found in database"
)
memory_config, workspace = result
# Step 2: Validate embedding model (returns both UUID and name)
embed_start = time.time()
embedding_uuid, embedding_name = validate_embedding_model(
@@ -159,7 +193,7 @@ class MemoryConfigService:
)
embed_time = time.time() - embed_start
logger.info(f"[PERF] Embedding validation: {embed_time:.4f}s")
# Step 3: Resolve LLM model
llm_start = time.time()
llm_uuid, llm_name = validate_and_resolve_model_id(
@@ -173,7 +207,7 @@ class MemoryConfigService:
)
llm_time = time.time() - llm_start
logger.info(f"[PERF] LLM validation: {llm_time:.4f}s")
# Step 4: Resolve optional rerank model
rerank_start = time.time()
rerank_uuid = None
@@ -191,10 +225,10 @@ class MemoryConfigService:
rerank_time = time.time() - rerank_start
if memory_config.rerank_id:
logger.info(f"[PERF] Rerank validation: {rerank_time:.4f}s")
# Note: embedding_name is now returned from validate_embedding_model above
# No need for redundant query!
# Create immutable MemoryConfig object
config = MemoryConfig(
config_id=memory_config.config_id,
@@ -235,9 +269,9 @@ class MemoryConfigService:
pruning_scene=memory_config.pruning_scene or "education",
pruning_threshold=float(memory_config.pruning_threshold) if memory_config.pruning_threshold is not None else 0.5,
)
elapsed_ms = (time.time() - start_time) * 1000
config_logger.info(
"Memory configuration loaded successfully",
extra={
@@ -250,13 +284,13 @@ class MemoryConfigService:
"elapsed_ms": elapsed_ms,
},
)
logger.info(f"Memory configuration loaded successfully: {config.config_name}")
return config
except Exception as e:
elapsed_ms = (time.time() - start_time) * 1000
config_logger.error(
"Failed to load memory configuration",
extra={
@@ -270,7 +304,7 @@ class MemoryConfigService:
},
exc_info=True,
)
logger.error(f"Failed to load memory configuration {config_id}: {e}")
if isinstance(e, (ConfigurationError, ValueError)):
raise
@@ -304,7 +338,7 @@ class MemoryConfigService:
"provider": api_config.provider,
"api_key": api_config.api_key,
"base_url": api_config.api_base,
"model_config_id": api_config.model_config_id,
"model_config_id": str(config.id),
"type": config.type,
"timeout": settings.LLM_TIMEOUT,
"max_retries": settings.LLM_MAX_RETRIES,
@@ -336,7 +370,7 @@ class MemoryConfigService:
"provider": api_config.provider,
"api_key": api_config.api_key,
"base_url": api_config.api_base,
"model_config_id": api_config.model_config_id,
"model_config_id": str(config.id),
"type": config.type,
"timeout": 120.0,
"max_retries": 5,

View File

@@ -53,18 +53,28 @@ def get_workspace_end_users(
workspace_id: uuid.UUID,
current_user: User
) -> List[EndUser]:
"""获取工作空间的所有宿主"""
"""获取工作空间的所有宿主(优化版本:减少数据库查询次数)"""
business_logger.info(f"获取工作空间宿主列表: workspace_id={workspace_id}, 操作者: {current_user.username}")
try:
# 查询应用ORM并转换为 Pydantic 模型
# 查询应用ORM
apps_orm = app_repository.get_apps_by_workspace_id(db, workspace_id)
apps = [AppSchema.model_validate(h) for h in apps_orm]
app_ids = [app.id for app in apps]
end_users = []
for app_id in app_ids:
end_user_orm_list = end_user_repository.get_end_users_by_app_id(db, app_id)
end_users.extend([EndUserSchema.model_validate(h) for h in end_user_orm_list])
if not apps_orm:
business_logger.info("工作空间下没有应用")
return []
# 提取所有 app_id
app_ids = [app.id for app in apps_orm]
# 批量查询所有 end_users一次查询而非循环查询
from app.models.end_user_model import EndUser as EndUserModel
end_users_orm = db.query(EndUserModel).filter(
EndUserModel.app_id.in_(app_ids)
).all()
# 转换为 Pydantic 模型(只在需要时转换)
end_users = [EndUserSchema.model_validate(eu) for eu in end_users_orm]
business_logger.info(f"成功获取 {len(end_users)} 个宿主记录")
return end_users
@@ -414,6 +424,67 @@ def get_current_user_total_chunk(
business_logger.error(f"获取用户总chunk数失败: end_user_id={end_user_id} - {str(e)}")
raise
def get_users_total_chunk_batch(
end_user_ids: List[str],
db: Session,
current_user: User
) -> dict:
"""
批量获取多个用户的总chunk数性能优化版本
Args:
end_user_ids: 用户ID列表
db: 数据库会话
current_user: 当前用户
Returns:
字典key为end_user_idvalue为chunk总数
格式: {"user_id_1": 100, "user_id_2": 50, ...}
"""
business_logger.info(f"批量获取 {len(end_user_ids)} 个用户的总chunk数, 操作者: {current_user.username}")
try:
from app.models.document_model import Document
from sqlalchemy import func, case
if not end_user_ids:
return {}
# 构造所有文件名
file_names = [f"{user_id}.txt" for user_id in end_user_ids]
# 一次查询获取所有用户的chunk总数
# 使用 GROUP BY file_name 来分组统计
results = db.query(
Document.file_name,
func.sum(Document.chunk_num).label('total_chunk')
).filter(
Document.file_name.in_(file_names)
).group_by(
Document.file_name
).all()
# 构建结果字典
chunk_map = {}
for file_name, total_chunk in results:
# 从文件名中提取 end_user_id (去掉 .txt 后缀)
user_id = file_name.replace('.txt', '')
chunk_map[user_id] = int(total_chunk or 0)
# 对于没有记录的用户设置为0
for user_id in end_user_ids:
if user_id not in chunk_map:
chunk_map[user_id] = 0
business_logger.info(f"成功批量获取 {len(chunk_map)} 个用户的总chunk数")
return chunk_map
except Exception as e:
business_logger.error(f"批量获取用户总chunk数失败: {str(e)}")
raise
def get_rag_content(
end_user_id: str,
limit: int,

View File

@@ -717,8 +717,8 @@ class MemoryInteraction:
ori_data= await self.connector.execute_query(Memory_Space_Entity, id=self.id)
if ori_data!=[]:
# name = ori_data[0]['name']
group_id = [i['group_id'] for i in ori_data][0]
Space_User = await self.connector.execute_query(Memory_Space_User, group_id=group_id)
end_user_id = [i['end_user_id'] for i in ori_data][0]
Space_User = await self.connector.execute_query(Memory_Space_User, end_user_id=end_user_id)
if not Space_User:
return []
user_id=Space_User[0]['id']

View File

@@ -34,7 +34,7 @@ class MemoryEpisodicService(MemoryBaseService):
Args:
summary_id: Summary节点的ID
end_user_id: 终端用户ID (group_id)
end_user_id: 终端用户ID (end_user_id)
Returns:
(标题, 类型)元组,如果不存在则返回默认值
@@ -43,14 +43,14 @@ class MemoryEpisodicService(MemoryBaseService):
# 查询Summary节点的name(作为title)和memory_type(作为type)
query = """
MATCH (s:MemorySummary)
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
RETURN s.name AS title, s.memory_type AS type
"""
result = await self.neo4j_connector.execute_query(
query,
summary_id=summary_id,
group_id=end_user_id
end_user_id=end_user_id
)
if not result or len(result) == 0:
@@ -77,7 +77,7 @@ class MemoryEpisodicService(MemoryBaseService):
Args:
summary_id: Summary节点的ID
end_user_id: 终端用户ID (group_id)
end_user_id: 终端用户ID (end_user_id)
Returns:
前3个实体的name属性列表
@@ -87,7 +87,7 @@ class MemoryEpisodicService(MemoryBaseService):
# 按activation_value降序排序,返回前3个
query = """
MATCH (s:MemorySummary)
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
MATCH (s)-[:DERIVED_FROM_STATEMENT]->(stmt:Statement)
MATCH (stmt)-[:REFERENCES_ENTITY]->(entity:ExtractedEntity)
WHERE entity.activation_value IS NOT NULL
@@ -99,7 +99,7 @@ class MemoryEpisodicService(MemoryBaseService):
result = await self.neo4j_connector.execute_query(
query,
summary_id=summary_id,
group_id=end_user_id
end_user_id=end_user_id
)
# 提取实体名称
@@ -123,7 +123,7 @@ class MemoryEpisodicService(MemoryBaseService):
Args:
summary_id: Summary节点的ID
end_user_id: 终端用户ID (group_id)
end_user_id: 终端用户ID (end_user_id)
Returns:
所有Statement节点的statement属性内容列表
@@ -132,7 +132,7 @@ class MemoryEpisodicService(MemoryBaseService):
# 查询Summary节点指向的所有Statement节点
query = """
MATCH (s:MemorySummary)
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
MATCH (s)-[:DERIVED_FROM_STATEMENT]->(stmt:Statement)
WHERE stmt.statement IS NOT NULL AND stmt.statement <> ''
RETURN stmt.statement AS statement
@@ -141,7 +141,7 @@ class MemoryEpisodicService(MemoryBaseService):
result = await self.neo4j_connector.execute_query(
query,
summary_id=summary_id,
group_id=end_user_id
end_user_id=end_user_id
)
# 提取statement内容
@@ -214,12 +214,12 @@ class MemoryEpisodicService(MemoryBaseService):
# 1. 先查询所有情景记忆的总数(不受筛选条件限制)
total_all_query = """
MATCH (s:MemorySummary)
WHERE s.group_id = $group_id
WHERE s.end_user_id = $end_user_id
RETURN count(s) AS total_all
"""
total_all_result = await self.neo4j_connector.execute_query(
total_all_query,
group_id=end_user_id
end_user_id=end_user_id
)
total_all = total_all_result[0]["total_all"] if total_all_result else 0
@@ -229,7 +229,7 @@ class MemoryEpisodicService(MemoryBaseService):
# 3. 构建Cypher查询
query = """
MATCH (s:MemorySummary)
WHERE s.group_id = $group_id
WHERE s.end_user_id = $end_user_id
"""
# 添加时间范围过滤
@@ -248,7 +248,7 @@ class MemoryEpisodicService(MemoryBaseService):
ORDER BY s.created_at DESC
"""
params = {"group_id": end_user_id}
params = {"end_user_id": end_user_id}
if time_filter:
params["time_filter"] = time_filter
if title_keyword:
@@ -333,14 +333,14 @@ class MemoryEpisodicService(MemoryBaseService):
# 1. 查询指定的MemorySummary节点
query = """
MATCH (s:MemorySummary)
WHERE elementId(s) = $summary_id AND s.group_id = $group_id
WHERE elementId(s) = $summary_id AND s.end_user_id = $end_user_id
RETURN elementId(s) AS id, s.created_at AS created_at
"""
result = await self.neo4j_connector.execute_query(
query,
summary_id=summary_id,
group_id=end_user_id
end_user_id=end_user_id
)
# 2. 如果节点不存在,返回错误

View File

@@ -60,7 +60,7 @@ class MemoryExplicitService(MemoryBaseService):
# ========== 1. 查询情景记忆MemorySummary节点 ==========
episodic_query = """
MATCH (s:MemorySummary)
WHERE s.group_id = $group_id
WHERE s.end_user_id = $end_user_id
RETURN elementId(s) AS id,
s.name AS title,
s.content AS content,
@@ -70,7 +70,7 @@ class MemoryExplicitService(MemoryBaseService):
episodic_result = await self.neo4j_connector.execute_query(
episodic_query,
group_id=end_user_id
end_user_id=end_user_id
)
# 处理情景记忆数据
@@ -96,7 +96,7 @@ class MemoryExplicitService(MemoryBaseService):
# ========== 2. 查询语义记忆ExtractedEntity节点 ==========
semantic_query = """
MATCH (e:ExtractedEntity)
WHERE e.group_id = $group_id
WHERE e.end_user_id = $end_user_id
AND e.is_explicit_memory = true
RETURN elementId(e) AS id,
e.name AS name,
@@ -107,7 +107,7 @@ class MemoryExplicitService(MemoryBaseService):
semantic_result = await self.neo4j_connector.execute_query(
semantic_query,
group_id=end_user_id
end_user_id=end_user_id
)
# 处理语义记忆数据
@@ -189,7 +189,7 @@ class MemoryExplicitService(MemoryBaseService):
# ========== 1. 先尝试查询情景记忆 ==========
episodic_query = """
MATCH (s:MemorySummary)
WHERE elementId(s) = $memory_id AND s.group_id = $group_id
WHERE elementId(s) = $memory_id AND s.end_user_id = $end_user_id
RETURN s.name AS title,
s.content AS content,
s.created_at AS created_at
@@ -198,7 +198,7 @@ class MemoryExplicitService(MemoryBaseService):
episodic_result = await self.neo4j_connector.execute_query(
episodic_query,
memory_id=memory_id,
group_id=end_user_id
end_user_id=end_user_id
)
if episodic_result and len(episodic_result) > 0:
@@ -229,7 +229,7 @@ class MemoryExplicitService(MemoryBaseService):
semantic_query = """
MATCH (e:ExtractedEntity)
WHERE elementId(e) = $memory_id
AND e.group_id = $group_id
AND e.end_user_id = $end_user_id
AND e.is_explicit_memory = true
RETURN e.name AS name,
e.description AS core_definition,
@@ -240,7 +240,7 @@ class MemoryExplicitService(MemoryBaseService):
semantic_result = await self.neo4j_connector.execute_query(
semantic_query,
memory_id=memory_id,
group_id=end_user_id
end_user_id=end_user_id
)
if semantic_result and len(semantic_result) > 0:

View File

@@ -12,6 +12,7 @@
from typing import Optional, Dict, Any, Tuple
from datetime import datetime, timezone
from uuid import UUID
from sqlalchemy.orm import Session
@@ -23,7 +24,7 @@ from app.core.memory.storage_services.forgetting_engine.config_utils import (
load_actr_config_from_db,
)
from app.repositories.neo4j.neo4j_connector import Neo4jConnector
from app.repositories.data_config_repository import DataConfigRepository
from app.repositories.memory_config_repository import MemoryConfigRepository
from app.repositories.forgetting_cycle_history_repository import ForgettingCycleHistoryRepository
@@ -70,7 +71,7 @@ class MemoryForgetService:
def __init__(self):
"""初始化服务"""
self.config_repository = DataConfigRepository()
self.config_repository = MemoryConfigRepository()
self.history_repository = ForgettingCycleHistoryRepository()
def _get_neo4j_connector(self) -> Neo4jConnector:
@@ -87,7 +88,7 @@ class MemoryForgetService:
async def _get_forgetting_components(
self,
db: Session,
config_id: Optional[int] = None
config_id: Optional[UUID] = None
) -> Tuple[ACTRCalculator, ForgettingStrategy, ForgettingScheduler, Dict[str, Any]]:
"""
获取遗忘引擎组件(计算器、策略、调度器)
@@ -132,7 +133,7 @@ class MemoryForgetService:
async def _get_knowledge_stats(
self,
connector: Neo4jConnector,
group_id: Optional[str] = None,
end_user_id: Optional[str] = None,
forgetting_threshold: float = 0.3
) -> Dict[str, Any]:
"""
@@ -140,7 +141,7 @@ class MemoryForgetService:
Args:
connector: Neo4j 连接器
group_id: 组ID可选
end_user_id: 组ID可选
forgetting_threshold: 遗忘阈值
Returns:
@@ -152,8 +153,8 @@ class MemoryForgetService:
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary)
"""
if group_id:
query += " AND n.group_id = $group_id"
if end_user_id:
query += " AND n.end_user_id = $end_user_id"
query += """
WITH n,
@@ -172,8 +173,8 @@ class MemoryForgetService:
"""
params = {'threshold': forgetting_threshold}
if group_id:
params['group_id'] = group_id
if end_user_id:
params['end_user_id'] = end_user_id
results = await connector.execute_query(query, **params)
@@ -200,7 +201,7 @@ class MemoryForgetService:
async def _get_pending_forgetting_nodes(
self,
connector: Neo4jConnector,
group_id: str,
end_user_id: str,
forgetting_threshold: float,
min_days_since_access: int,
limit: int = 20
@@ -212,7 +213,7 @@ class MemoryForgetService:
Args:
connector: Neo4j 连接器
group_id: 组ID
end_user_id: 组ID
forgetting_threshold: 遗忘阈值
min_days_since_access: 最小未访问天数
limit: 返回节点数量限制
@@ -229,7 +230,7 @@ class MemoryForgetService:
query = """
MATCH (n)
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary)
AND n.group_id = $group_id
AND n.end_user_id = $end_user_id
AND n.activation_value IS NOT NULL
AND n.activation_value < $threshold
AND n.last_access_time IS NOT NULL
@@ -250,7 +251,7 @@ class MemoryForgetService:
"""
params = {
'group_id': group_id,
'end_user_id': end_user_id,
'threshold': forgetting_threshold,
'min_access_time_str': min_access_time_str,
'limit': limit
@@ -291,10 +292,10 @@ class MemoryForgetService:
async def trigger_forgetting_cycle(
self,
db: Session,
group_id: str,
end_user_id: str,
max_merge_batch_size: Optional[int] = None,
min_days_since_access: Optional[int] = None,
config_id: Optional[int] = None
config_id: Optional[UUID] = None
) -> Dict[str, Any]:
"""
手动触发遗忘周期
@@ -303,10 +304,10 @@ class MemoryForgetService:
Args:
db: 数据库会话
group_id: 组ID即终端用户ID必填
end_user_id: 组ID即终端用户ID必填
max_merge_batch_size: 最大融合批次大小(可选)
min_days_since_access: 最小未访问天数(可选)
config_id: 配置ID必填由控制器层通过 group_id 获取)
config_id: 配置ID必填由控制器层通过 end_user_id 获取)
Returns:
dict: 遗忘报告
@@ -319,7 +320,7 @@ class MemoryForgetService:
# 运行遗忘周期LLM 客户端将在需要时由 forgetting_strategy 内部获取)
report = await forgetting_scheduler.run_forgetting_cycle(
group_id=group_id,
end_user_id=end_user_id,
max_merge_batch_size=max_merge_batch_size,
min_days_since_access=min_days_since_access,
config_id=config_id,
@@ -338,7 +339,7 @@ class MemoryForgetService:
stats_query = """
MATCH (n)
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary OR n:Chunk)
AND n.group_id = $group_id
AND n.end_user_id = $end_user_id
RETURN
count(n) as total_nodes,
avg(n.activation_value) as average_activation,
@@ -347,7 +348,7 @@ class MemoryForgetService:
stats_results = await connector.execute_query(
stats_query,
group_id=group_id,
end_user_id=end_user_id,
threshold=config['forgetting_threshold']
)
@@ -364,7 +365,7 @@ class MemoryForgetService:
# 保存历史记录到数据库
self.history_repository.create(
db=db,
end_user_id=group_id,
end_user_id=end_user_id,
execution_time=execution_time,
merged_count=report['merged_count'],
failed_count=report['failed_count'],
@@ -376,7 +377,7 @@ class MemoryForgetService:
)
api_logger.info(
f"已保存遗忘周期历史记录: end_user_id={group_id}, "
f"已保存遗忘周期历史记录: end_user_id={end_user_id}, "
f"merged_count={report['merged_count']}"
)
@@ -389,7 +390,7 @@ class MemoryForgetService:
def read_forgetting_config(
self,
db: Session,
config_id: int
config_id: UUID
) -> Dict[str, Any]:
"""
获取遗忘引擎配置
@@ -416,7 +417,7 @@ class MemoryForgetService:
def update_forgetting_config(
self,
db: Session,
config_id: int,
config_id: UUID,
update_fields: Dict[str, Any]
) -> Dict[str, Any]:
"""
@@ -465,8 +466,8 @@ class MemoryForgetService:
async def get_forgetting_stats(
self,
db: Session,
group_id: Optional[str] = None,
config_id: Optional[int] = None
end_user_id: Optional[str] = None,
config_id: Optional[UUID] = None
) -> Dict[str, Any]:
"""
获取遗忘引擎统计信息
@@ -475,7 +476,7 @@ class MemoryForgetService:
Args:
db: 数据库会话
group_id: 组ID可选
end_user_id: 组ID可选
config_id: 配置ID可选用于获取遗忘阈值
Returns:
@@ -493,8 +494,8 @@ class MemoryForgetService:
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary OR n:Chunk)
"""
if group_id:
activation_query += " AND n.group_id = $group_id"
if end_user_id:
activation_query += " AND n.end_user_id = $end_user_id"
activation_query += """
RETURN
@@ -506,8 +507,8 @@ class MemoryForgetService:
"""
params = {'threshold': forgetting_threshold}
if group_id:
params['group_id'] = group_id
if end_user_id:
params['end_user_id'] = end_user_id
activation_results = await connector.execute_query(activation_query, **params)
@@ -539,8 +540,8 @@ class MemoryForgetService:
WHERE (n:Statement OR n:ExtractedEntity OR n:MemorySummary OR n:Chunk)
"""
if group_id:
distribution_query += " AND n.group_id = $group_id"
if end_user_id:
distribution_query += " AND n.end_user_id = $end_user_id"
distribution_query += """
WITH n,
@@ -558,8 +559,8 @@ class MemoryForgetService:
"""
dist_params = {}
if group_id:
dist_params['group_id'] = group_id
if end_user_id:
dist_params['end_user_id'] = end_user_id
distribution_results = await connector.execute_query(distribution_query, **dist_params)
@@ -582,11 +583,11 @@ class MemoryForgetService:
# 获取最近7个日期的历史趋势数据每天取最后一次执行
recent_trends = []
try:
if group_id:
if end_user_id:
# 查询所有历史记录
history_records = self.history_repository.get_recent_by_end_user(
db=db,
end_user_id=group_id
end_user_id=end_user_id
)
# 按日期分组(一天可能有多次执行,取最后一次)
@@ -632,7 +633,7 @@ class MemoryForgetService:
# 获取待遗忘节点列表前20个满足遗忘条件的节点
pending_nodes = []
try:
if group_id:
if end_user_id:
# 验证 min_days_since_access 配置值
min_days = config.get('min_days_since_access')
if min_days is None or not isinstance(min_days, (int, float)) or min_days < 0:
@@ -643,7 +644,7 @@ class MemoryForgetService:
pending_nodes = await self._get_pending_forgetting_nodes(
connector=connector,
group_id=group_id,
end_user_id=end_user_id,
forgetting_threshold=forgetting_threshold,
min_days_since_access=int(min_days),
limit=20
@@ -677,7 +678,7 @@ class MemoryForgetService:
db: Session,
importance_score: float,
days: int,
config_id: Optional[int] = None
config_id: Optional[UUID] = None
) -> Dict[str, Any]:
"""
获取遗忘曲线数据

View File

@@ -450,12 +450,12 @@ async def create_document_chunk(
return success(data=chunk, msg="文档块创建成功")
async def write_rag(group_id, message, user_rag_memory_id):
async def write_rag(end_user_id, message, user_rag_memory_id):
"""
将消息写入 RAG 知识库
Args:
group_id: 组ID用作文件标题
end_user_id: 组ID用作文件标题
message: 消息内容
user_rag_memory_id: 知识库ID必须是有效的UUID
@@ -487,10 +487,10 @@ async def write_rag(group_id, message, user_rag_memory_id):
db = next(db_gen)
try:
create_data = CustomTextFileCreate(title=group_id, content=message)
create_data = CustomTextFileCreate(title=end_user_id, content=message)
current_user = SimpleUser(user_rag_memory_id)
# 检查文档是否已存在
document = find_document_id_by_kb_and_filename(db=db, kb_id=user_rag_memory_id, file_name=f"{group_id}.txt")
document = find_document_id_by_kb_and_filename(db=db, kb_id=user_rag_memory_id, file_name=f"{end_user_id}.txt")
print('======',document)
api_logger.info(f"查找文档结果: document_id={document}")
if document is not None:
@@ -508,7 +508,7 @@ async def write_rag(group_id, message, user_rag_memory_id):
return result
else:
# 文档不存在,创建新文档
api_logger.info(f"文档不存在,创建新文档: group_id={group_id}")
api_logger.info(f"文档不存在,创建新文档: end_user_id={end_user_id}")
result = await memory_konwledges_up(
kb_id=user_rag_memory_id,
parent_id=user_rag_memory_id,
@@ -520,13 +520,13 @@ async def write_rag(group_id, message, user_rag_memory_id):
new_document_id = find_document_id_by_kb_and_filename(
db=db,
kb_id=user_rag_memory_id,
file_name=f"{group_id}.txt"
file_name=f"{end_user_id}.txt"
)
if new_document_id:
await parse_document_by_id(new_document_id, db=db, current_user=current_user)
else:
api_logger.error(f"创建文档后无法找到文档ID: group_id={group_id}")
api_logger.error(f"创建文档后无法找到文档ID: end_user_id={end_user_id}")
return result
finally:
# 确保数据库会话被关闭

View File

@@ -6,7 +6,7 @@ from sqlalchemy.orm import Session
from app.core.error_codes import BizCode
from app.core.exceptions import BusinessException
from app.core.logging_config import get_business_logger
from app.models.memory_perceptual_model import PerceptualType, FileStorageType
from app.models.memory_perceptual_model import PerceptualType, FileStorageService
from app.repositories.memory_perceptual_repository import MemoryPerceptualRepository
from app.schemas.memory_perceptual_schema import (
PerceptualQuerySchema,
@@ -137,8 +137,19 @@ class MemoryPerceptualService:
memory_items = []
for memory in memories:
meta_data = memory.meta_data or {}
content = meta_data.get("content")
content = Content(**content)
content = meta_data.get("content", {})
# 安全地提取 content 字段,提供默认值
if content:
content_obj = Content(**content)
topic = content_obj.topic
domain = content_obj.domain
keywords = content_obj.keywords
else:
topic = "Unknown"
domain = "Unknown"
keywords = []
memory_item = PerceptualMemoryItem(
id=memory.id,
perceptual_type=PerceptualType(memory.perceptual_type),
@@ -146,11 +157,12 @@ class MemoryPerceptualService:
file_name=memory.file_name,
file_ext=memory.file_ext,
summary=memory.summary,
topic=content.topic,
domain=content.domain,
keywords=content.keywords,
meta_data=meta_data,
topic=topic,
domain=domain,
keywords=keywords,
created_time=int(memory.created_time.timestamp()*1000),
storage_type=FileStorageType(memory.storage_service),
storage_service=FileStorageService(memory.storage_service),
)
memory_items.append(memory_item)

View File

@@ -13,11 +13,12 @@ from app.db import get_db
from app.core.logging_config import get_api_logger
from app.core.memory.storage_services.reflection_engine import ReflectionConfig, ReflectionEngine
from app.core.memory.storage_services.reflection_engine.self_reflexion import ReflectionRange, ReflectionBaseline
from app.repositories.data_config_repository import DataConfigRepository
from app.repositories.memory_config_repository import MemoryConfigRepository
from app.repositories.neo4j.neo4j_connector import Neo4jConnector
from app.models.app_model import App
from app.models.app_release_model import AppRelease
from app.models.end_user_model import EndUser
from app.utils.config_utils import resolve_config_id
api_logger = get_api_logger()
@@ -38,7 +39,10 @@ class WorkspaceAppService:
Returns:
Dictionary containing detailed application information
"""
apps = self.db.query(App).filter(App.workspace_id == workspace_id).all()
apps = self.db.query(App).filter(
App.workspace_id == workspace_id,
App.is_active.is_(True)
).all()
app_ids = [str(app.id) for app in apps]
apps_detailed_info = []
@@ -70,7 +74,7 @@ class WorkspaceAppService:
"created_at": app.created_at.isoformat() if app.created_at else None,
"updated_at": app.updated_at.isoformat() if app.updated_at else None,
"releases": [],
"data_configs": [],
"memory_configs": [],
"end_users": []
}
@@ -85,76 +89,76 @@ class WorkspaceAppService:
for release in app_releases:
memory_content = self._extract_memory_content(release.config)
memory_content=resolve_config_id(memory_content, self.db)
if memory_content and memory_content in processed_configs:
continue
release_info = {
"app_id": str(release.app_id),
"config": memory_content
}
if memory_content:
processed_configs.add(memory_content)
data_config_info = self._get_data_config(memory_content)
if data_config_info:
if not any(dc["config_id"] == data_config_info["config_id"] for dc in app_info["data_configs"]):
app_info["data_configs"].append(data_config_info)
memory_config_info = self._get_memory_config(memory_content)
if memory_config_info:
if not any(dc["config_id"] == memory_config_info["config_id"] for dc in app_info["memory_configs"]):
app_info["memory_configs"].append(memory_config_info)
app_info["releases"].append(release_info)
def _extract_memory_content(self, config: Any) -> str:
"""Extract memory_comtent from config"""
if not config or not isinstance(config, dict):
return None
memory_obj = config.get('memory')
if memory_obj and isinstance(memory_obj, dict):
return memory_obj.get('memory_content')
return None
def _get_data_config(self, memory_content: str) -> Dict[str, Any]:
"""Retrieve data_comfig information based on memory_comtent"""
try:
data_config_result = DataConfigRepository.query_reflection_config_by_id(self.db, int(memory_content))
# data_config_query, data_config_params = DataConfigRepository.build_select_reflection(memory_content)
# data_config_result = self.db.execute(text(data_config_query), data_config_params).fetchone()
# if data_config_result is None:
return None
def _get_memory_config(self, memory_content: str) -> Dict[str, Any]:
"""Retrieve memory_config information based on memory_content"""
try:
memory_config_result = MemoryConfigRepository.query_reflection_config_by_id(self.db, int(memory_content))
# memory_config_query, memory_config_params = MemoryConfigRepository.build_select_reflection(memory_content)
# memory_config_result = self.db.execute(text(memory_config_query), memory_config_params).fetchone()
# if memory_config_result is None:
# return None
if data_config_result:
if memory_config_result:
return {
"config_id": data_config_result.config_id,
"enable_self_reflexion": data_config_result.enable_self_reflexion,
"iteration_period": data_config_result.iteration_period,
"reflexion_range": data_config_result.reflexion_range,
"baseline": data_config_result.baseline,
"reflection_model_id": data_config_result.reflection_model_id,
"memory_verify": data_config_result.memory_verify,
"quality_assessment": data_config_result.quality_assessment,
"user_id": data_config_result.user_id
"config_id": memory_config_result.config_id,
"enable_self_reflexion": memory_config_result.enable_self_reflexion,
"iteration_period": memory_config_result.iteration_period,
"reflexion_range": memory_config_result.reflexion_range,
"baseline": memory_config_result.baseline,
"reflection_model_id": memory_config_result.reflection_model_id,
"memory_verify": memory_config_result.memory_verify,
"quality_assessment": memory_config_result.quality_assessment,
"user_id": memory_config_result.user_id
}
except Exception as e:
api_logger.warning(f"查询data_config失败memory_content: {memory_content}, 错误: {str(e)}")
api_logger.warning(f"查询memory_config失败memory_content: {memory_content}, 错误: {str(e)}")
return None
def _process_end_users(self, app: App, app_info: Dict[str, Any]) -> None:
"""Processing end-user information for applications"""
end_users = self.db.query(EndUser).filter(EndUser.app_id == app.id).all()
for end_user in end_users:
end_user_info = {
"id": str(end_user.id),
"app_id": str(end_user.app_id)
}
app_info["end_users"].append(end_user_info)
print(100*'-')
print(app_info)
def get_end_user_reflection_time(self, end_user_id: str) -> Optional[Any]:
"""
Read the reflection time of end users
@@ -173,7 +177,7 @@ class WorkspaceAppService:
except Exception as e:
api_logger.error(f"读取用户反思时间失败end_user_id: {end_user_id}, 错误: {str(e)}")
return None
def update_end_user_reflection_time(self, end_user_id: str) -> bool:
"""
Update the reflection time of end users to the current time
@@ -186,7 +190,7 @@ class WorkspaceAppService:
"""
try:
from datetime import datetime
end_user = self.db.query(EndUser).filter(EndUser.id == end_user_id).first()
if end_user:
end_user.reflection_time = datetime.now()
@@ -204,7 +208,7 @@ class WorkspaceAppService:
class MemoryReflectionService:
"""Memory reflection service category"""
def __init__(self,db: Session = Depends(get_db)):
self.db=db
@@ -223,7 +227,7 @@ class MemoryReflectionService:
}
config_data_id = config_data['config_id']
reflection_config = WorkspaceAppService(self.db)._get_data_config(config_data_id)
reflection_config = WorkspaceAppService(self.db)._get_memory_config(config_data_id)
if reflection_config is not None and reflection_config['enable_self_reflexion']:
reflection_config = self._create_reflection_config_from_data(reflection_config)
# 3. 执行反思引擎
@@ -249,22 +253,22 @@ class MemoryReflectionService:
"end_user_id": end_user_id,
"config_data": config_data
}
async def start_reflection_from_data(self, config_data: Dict[str, Any], end_user_id: str) -> Dict[str, Any]:
"""
Starting Reflection from Configuration Data
Args:
config_data: Configure data dictionary, including reflective configuration information
end_user_id: end_user_id
Returns:
Reflect on the execution results
"""
try:
config_id = config_data.get("config_id")
api_logger.info(f"从配置数据启动反思config_id: {config_id}, end_user_id: {end_user_id}")
if not config_data.get("enable_self_reflexion", False):
return {
@@ -274,10 +278,10 @@ class MemoryReflectionService:
"end_user_id": end_user_id,
"config_data": config_data
}
config_data_id=config_data['config_id']
reflection_config=WorkspaceAppService(self.db)._get_data_config(config_data_id)
reflection_config=WorkspaceAppService(self.db)._get_memory_config(config_data_id)
if reflection_config is not None and reflection_config['enable_self_reflexion']:
reflection_config= self._create_reflection_config_from_data(reflection_config)
iteration_period = int(reflection_config.iteration_period)

View File

@@ -12,10 +12,14 @@ from datetime import datetime
from typing import Any, AsyncGenerator, Dict, List, Optional
from app.core.logging_config import get_config_logger, get_logger
from app.core.memory.analytics.hot_memory_tags import get_hot_memory_tags
from app.core.memory.analytics.hot_memory_tags import (
get_hot_memory_tags,
get_raw_tags_from_db,
filter_tags_with_llm,
)
from app.core.memory.analytics.recent_activity_stats import get_recent_activity_stats
from app.models.user_model import User
from app.repositories.data_config_repository import DataConfigRepository
from app.repositories.memory_config_repository import MemoryConfigRepository
from app.repositories.neo4j.neo4j_connector import Neo4jConnector
from app.schemas.memory_config_schema import ConfigurationError
from app.schemas.memory_storage_schema import (
@@ -125,7 +129,7 @@ class DataConfigService: # 数据配置服务类PostgreSQL
if not params.rerank_id:
params.rerank_id = configs.get('rerank')
config = DataConfigRepository.create(self.db, params)
config = MemoryConfigRepository.create(self.db, params)
self.db.commit()
return {"affected": 1, "config_id": config.config_id}
@@ -142,20 +146,20 @@ class DataConfigService: # 数据配置服务类PostgreSQL
# --- Delete ---
def delete(self, key: ConfigParamsDelete) -> Dict[str, Any]: # 删除配置参数按配置ID
success = DataConfigRepository.delete(self.db, key.config_id)
success = MemoryConfigRepository.delete(self.db, key.config_id)
if not success:
raise ValueError("未找到配置")
return {"affected": 1}
# --- Update ---
def update(self, update: ConfigUpdate) -> Dict[str, Any]: # 部分更新配置参数
config = DataConfigRepository.update(self.db, update)
config = MemoryConfigRepository.update(self.db, update)
if not config:
raise ValueError("未找到配置")
return {"affected": 1}
def update_extracted(self, update: ConfigUpdateExtracted) -> Dict[str, Any]: # 更新记忆萃取引擎配置参数
config = DataConfigRepository.update_extracted(self.db, update)
config = MemoryConfigRepository.update_extracted(self.db, update)
if not config:
raise ValueError("未找到配置")
return {"affected": 1}
@@ -166,25 +170,38 @@ class DataConfigService: # 数据配置服务类PostgreSQL
# --- Read ---
def get_extracted(self, key: ConfigKey) -> Dict[str, Any]: # 获取萃取配置参数
result = DataConfigRepository.get_extracted_config(self.db, key.config_id)
result = MemoryConfigRepository.get_extracted_config(self.db, key.config_id)
if not result:
raise ValueError("未找到配置")
return result
# --- Read All ---
def get_all(self, workspace_id = None) -> List[Dict[str, Any]]: # 获取所有配置参数
configs = DataConfigRepository.get_all(self.db, workspace_id)
configs = MemoryConfigRepository.get_all(self.db, workspace_id)
# 将 ORM 对象转换为字典列表
data_list = []
for config in configs:
# 安全地转换 user_id 为 int
config_id_old = None
if config.config_id_old:
try:
config_id_old = int(config.config_id_old)
except (ValueError, TypeError):
config_id_old = None
if config_id_old:
memory_config=config_id_old
else:
memory_config=config.config_id
config_dict = {
"config_id": config.config_id,
"config_id": memory_config,
"config_name": config.config_name,
"config_desc": config.config_desc,
"workspace_id": str(config.workspace_id) if config.workspace_id else None,
"group_id": config.group_id,
"user_id": config.user_id,
"end_user_id": config.end_user_id,
"config_id_old": config_id_old,
"apply_id": config.apply_id,
"llm_id": config.llm_id,
"embedding_id": config.embedding_id,
@@ -237,7 +254,8 @@ class DataConfigService: # 数据配置服务类PostgreSQL
ValueError: 当配置无效或参数缺失时
RuntimeError: 当管线执行失败时
"""
project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from pathlib import Path
project_root = str(Path(__file__).resolve().parents[2])
try:
# 发出初始进度事件
@@ -263,7 +281,7 @@ class DataConfigService: # 数据配置服务类PostgreSQL
try:
config_service = MemoryConfigService(self.db)
memory_config = config_service.load_memory_config(
config_id=int(cid),
config_id=str(cid),
service_name="MemoryStorageService.pilot_run_stream"
)
logger.info(f"Configuration loaded successfully: {memory_config.config_name}")
@@ -390,8 +408,8 @@ _neo4j_connector = Neo4jConnector()
async def search_dialogue(end_user_id: Optional[str] = None) -> Dict[str, Any]:
result = await _neo4j_connector.execute_query(
DataConfigRepository.SEARCH_FOR_DIALOGUE,
group_id=end_user_id,
MemoryConfigRepository.SEARCH_FOR_DIALOGUE,
end_user_id=end_user_id,
)
data = {"search_for": "dialogue", "num": result[0]["num"]}
return data
@@ -399,8 +417,8 @@ async def search_dialogue(end_user_id: Optional[str] = None) -> Dict[str, Any]:
async def search_chunk(end_user_id: Optional[str] = None) -> Dict[str, Any]:
result = await _neo4j_connector.execute_query(
DataConfigRepository.SEARCH_FOR_CHUNK,
group_id=end_user_id,
MemoryConfigRepository.SEARCH_FOR_CHUNK,
end_user_id=end_user_id,
)
data = {"search_for": "chunk", "num": result[0]["num"]}
return data
@@ -408,8 +426,8 @@ async def search_chunk(end_user_id: Optional[str] = None) -> Dict[str, Any]:
async def search_statement(end_user_id: Optional[str] = None) -> Dict[str, Any]:
result = await _neo4j_connector.execute_query(
DataConfigRepository.SEARCH_FOR_STATEMENT,
group_id=end_user_id,
MemoryConfigRepository.SEARCH_FOR_STATEMENT,
end_user_id=end_user_id,
)
data = {"search_for": "statement", "num": result[0]["num"]}
return data
@@ -417,8 +435,8 @@ async def search_statement(end_user_id: Optional[str] = None) -> Dict[str, Any]:
async def search_entity(end_user_id: Optional[str] = None) -> Dict[str, Any]:
result = await _neo4j_connector.execute_query(
DataConfigRepository.SEARCH_FOR_ENTITY,
group_id=end_user_id,
MemoryConfigRepository.SEARCH_FOR_ENTITY,
end_user_id=end_user_id,
)
data = {"search_for": "entity", "num": result[0]["num"]}
return data
@@ -426,8 +444,8 @@ async def search_entity(end_user_id: Optional[str] = None) -> Dict[str, Any]:
async def search_all(end_user_id: Optional[str] = None) -> Dict[str, Any]:
result = await _neo4j_connector.execute_query(
DataConfigRepository.SEARCH_FOR_ALL,
group_id=end_user_id,
MemoryConfigRepository.SEARCH_FOR_ALL,
end_user_id=end_user_id,
)
# 检查结果是否为空或长度不足
@@ -461,8 +479,8 @@ async def kb_type_distribution(end_user_id: Optional[str] = None) -> Dict[str, A
聚合 dialogue/chunk/statement/entity 四类计数,返回统一的分布结构,便于前端一次性消费。
"""
result = await _neo4j_connector.execute_query(
DataConfigRepository.SEARCH_FOR_ALL,
group_id=end_user_id,
MemoryConfigRepository.SEARCH_FOR_ALL,
end_user_id=end_user_id,
)
# 检查结果是否为空或长度不足
@@ -492,21 +510,19 @@ async def kb_type_distribution(end_user_id: Optional[str] = None) -> Dict[str, A
async def search_detials(end_user_id: Optional[str] = None) -> List[Dict[str, Any]]:
result = await _neo4j_connector.execute_query(
DataConfigRepository.SEARCH_FOR_DETIALS,
group_id=end_user_id,
MemoryConfigRepository.SEARCH_FOR_DETIALS,
end_user_id=end_user_id,
)
return result
async def search_edges(end_user_id: Optional[str] = None) -> List[Dict[str, Any]]:
result = await _neo4j_connector.execute_query(
DataConfigRepository.SEARCH_FOR_EDGES,
group_id=end_user_id,
MemoryConfigRepository.SEARCH_FOR_EDGES,
end_user_id=end_user_id,
)
return result
async def analytics_hot_memory_tags(
db: Session,
current_user: User,
@@ -514,27 +530,79 @@ async def analytics_hot_memory_tags(
) -> List[Dict[str, Any]]:
"""
获取热门记忆标签按数量排序并返回前N个
优化策略:
1. 先从所有用户收集原始标签不调用LLM
2. 聚合并合并相同标签的频率
3. 排序后取前N个
4. 只调用一次LLM进行筛选
"""
workspace_id = current_user.current_workspace_id
# 获取更多标签供LLM筛选获取limit*4个标签
raw_limit = limit * 4
from app.services.memory_dashboard_service import get_workspace_end_users
end_users = get_workspace_end_users(db, workspace_id, current_user)
# 使用 asyncio.to_thread 避免阻塞事件循环
end_users = await asyncio.to_thread(get_workspace_end_users, db, workspace_id, current_user)
tags = []
for end_user in end_users:
tag = await get_hot_memory_tags(str(end_user.id), limit=raw_limit)
if tag:
# 将每个用户的标签列表展平到总列表中
tags.extend(tag)
# 按频率降序排序(虽然数据库已经排序,但为了确保正确性再次排序)
sorted_tags = sorted(tags, key=lambda x: x[1], reverse=True)
if not end_users:
return []
# 只返回前limit个
top_tags = sorted_tags[:limit]
return [{"name": t, "frequency": f} for t, f in top_tags]
# 步骤1: 收集所有用户的原始标签不调用LLM
connector = Neo4jConnector()
try:
all_raw_tags = []
for end_user in end_users:
raw_tags = await get_raw_tags_from_db(
connector,
str(end_user.id),
limit=raw_limit,
by_user=False
)
if raw_tags:
all_raw_tags.extend(raw_tags)
if not all_raw_tags:
return []
# 步骤2: 聚合相同标签的频率
tag_frequency_map = {}
for tag_name, frequency in all_raw_tags:
if tag_name in tag_frequency_map:
tag_frequency_map[tag_name] += frequency
else:
tag_frequency_map[tag_name] = frequency
# 步骤3: 按频率降序排序取前raw_limit个
sorted_tags = sorted(
tag_frequency_map.items(),
key=lambda x: x[1],
reverse=True
)[:raw_limit]
if not sorted_tags:
return []
# 步骤4: 只调用一次LLM进行筛选
tag_names = [tag for tag, _ in sorted_tags]
# 使用第一个用户的end_user_id来获取LLM配置
# 因为同一工作空间下的用户应该使用相同的配置
first_end_user_id = str(end_users[0].id)
filtered_tag_names = await filter_tags_with_llm(tag_names, first_end_user_id)
# 步骤5: 根据LLM筛选结果构建最终列表保留频率
final_tags = []
for tag, freq in sorted_tags:
if tag in filtered_tag_names:
final_tags.append((tag, freq))
# 步骤6: 只返回前limit个
top_tags = final_tags[:limit]
return [{"name": t, "frequency": f} for t, f in top_tags]
finally:
await connector.close()
async def analytics_recent_activity_stats() -> Dict[str, Any]:

View File

@@ -1,3 +1,4 @@
from datetime import datetime
from sqlalchemy.orm import Session
from typing import List, Optional, Dict, Any
import uuid
@@ -6,11 +7,11 @@ import time
import asyncio
from app.models.models_model import ModelConfig, ModelApiKey, ModelType
from app.repositories.model_repository import ModelConfigRepository, ModelApiKeyRepository
from app.repositories.model_repository import ModelConfigRepository, ModelApiKeyRepository, ModelBaseRepository
from app.schemas import model_schema
from app.schemas.model_schema import (
ModelConfigCreate, ModelConfigUpdate, ModelApiKeyCreate, ModelApiKeyUpdate,
ModelConfigQuery, ModelStats
ModelConfigQuery, ModelStats, ModelConfigQueryNew
)
from app.core.logging_config import get_business_logger
from app.schemas.response_schema import PageData, PageMeta
@@ -47,6 +48,26 @@ class ModelConfigService:
items=[model_schema.ModelConfig.model_validate(model) for model in models]
)
@staticmethod
def get_model_list_new(db: Session, query: ModelConfigQueryNew, tenant_id: uuid.UUID | None = None) -> List[dict]:
"""获取模型配置列表"""
provider_groups, total = ModelConfigRepository.get_list_new(db, query, tenant_id=tenant_id)
items = []
for provider, models in provider_groups.items():
# 验证每个模型并封装分组信息
validated_models = [model_schema.ModelConfig.model_validate(model) for model in models]
tags = list({model.type for model in validated_models})
group_item = {
"provider": provider, # 服务商名称
"logo": validated_models[0].logo,
"tags": tags,
"models": validated_models # 该服务商下的所有模型
}
items.append(group_item)
return items
@staticmethod
def get_model_by_name(db: Session, name: str, tenant_id: uuid.UUID | None = None) -> ModelConfig:
"""根据名称获取模型配置"""
@@ -228,37 +249,39 @@ class ModelConfigService:
# 验证配置
if not model_data.skip_validation and model_data.api_keys:
api_key_data = model_data.api_keys
validation_result = await ModelConfigService.validate_model_config(
db=db,
model_name=api_key_data.model_name,
provider=api_key_data.provider,
api_key=api_key_data.api_key,
api_base=api_key_data.api_base,
model_type=model_data.type, # 传递模型类型
test_message="Hello"
)
if not validation_result["valid"]:
raise BusinessException(
f"模型配置验证失败: {validation_result['error']}",
BizCode.INVALID_PARAMETER
api_key_data_list = model_data.api_keys
for api_key_data in api_key_data_list:
validation_result = await ModelConfigService.validate_model_config(
db=db,
model_name=api_key_data.model_name,
provider=api_key_data.provider,
api_key=api_key_data.api_key,
api_base=api_key_data.api_base,
model_type=model_data.type, # 传递模型类型
test_message="Hello"
)
if not validation_result["valid"]:
raise BusinessException(
f"模型配置验证失败: {validation_result['error']}",
BizCode.INVALID_PARAMETER
)
# 事务处理
api_key_data = model_data.api_keys
model_config_data = model_data.dict(exclude={"api_keys", "skip_validation"})
api_key_datas = model_data.api_keys
model_config_data = model_data.model_dump(exclude={"api_keys", "skip_validation"})
# 添加租户ID
model_config_data["tenant_id"] = tenant_id
model = ModelConfigRepository.create(db, model_config_data)
db.flush() # 获取生成的 ID
if api_key_data:
api_key_create_schema = ModelApiKeyCreate(
model_config_id=model.id,
**api_key_data.dict()
)
ModelApiKeyRepository.create(db, api_key_create_schema)
if api_key_datas:
for api_key_data in api_key_datas:
api_key_create_schema = ModelApiKeyCreate(
model_config_ids=[model.id],
**api_key_data.model_dump()
)
ModelApiKeyRepository.create(db, api_key_create_schema)
db.commit()
db.refresh(model)
@@ -280,6 +303,116 @@ class ModelConfigService:
db.refresh(model)
return model
@staticmethod
async def create_composite_model(db: Session, model_data: model_schema.CompositeModelCreate, tenant_id: uuid.UUID) -> ModelConfig:
"""创建组合模型"""
if ModelConfigRepository.get_by_name(db, model_data.name, tenant_id=tenant_id):
raise BusinessException("模型名称已存在", BizCode.DUPLICATE_NAME)
# 验证所有 API Key 存在且类型匹配
for api_key_id in model_data.api_key_ids:
api_key = ModelApiKeyRepository.get_by_id(db, api_key_id)
if not api_key:
raise BusinessException(f"API Key {api_key_id} 不存在", BizCode.NOT_FOUND)
# 检查 API Key 关联的模型配置类型
for model_config in api_key.model_configs:
# chat 和 llm 类型可以兼容
compatible_types = {ModelType.LLM, ModelType.CHAT}
config_type = model_config.type
request_type = model_data.type
if not (config_type == request_type or
(config_type in compatible_types and request_type in compatible_types)):
raise BusinessException(
f"API Key {api_key_id} 关联的模型类型 ({model_config.type}) 与组合模型类型 ({model_data.type}) 不匹配",
BizCode.INVALID_PARAMETER
)
# if model_config.is_composite:
# raise BusinessException(
# f"API Key {api_key_id} 关联的模型是组合模型,不能用于创建新的组合模型",
# BizCode.INVALID_PARAMETER
# )
# 创建组合模型
model_config_data = {
"tenant_id": tenant_id,
"name": model_data.name,
"type": model_data.type,
"logo": model_data.logo,
"description": model_data.description,
"provider": "composite",
"config": model_data.config,
"is_active": model_data.is_active,
"is_public": model_data.is_public,
"is_composite": True
}
if "load_balance_strategy" in model_data.model_fields_set:
model_config_data["load_balance_strategy"] = model_data.load_balance_strategy
model = ModelConfigRepository.create(db, model_config_data)
db.flush()
# 关联 API Keys
for api_key_id in model_data.api_key_ids:
api_key = ModelApiKeyRepository.get_by_id(db, api_key_id)
if api_key:
model.api_keys.append(api_key)
db.commit()
db.refresh(model)
return model
@staticmethod
async def update_composite_model(db: Session, model_id: uuid.UUID, model_data: model_schema.CompositeModelCreate, tenant_id: uuid.UUID) -> ModelConfig:
"""更新组合模型"""
existing_model = ModelConfigRepository.get_by_id(db, model_id, tenant_id=tenant_id)
if not existing_model:
raise BusinessException("模型配置不存在", BizCode.MODEL_NOT_FOUND)
if not existing_model.is_composite:
raise BusinessException("该模型不是组合模型", BizCode.INVALID_PARAMETER)
# 验证所有 API Key 存在且类型匹配
for api_key_id in model_data.api_key_ids:
api_key = ModelApiKeyRepository.get_by_id(db, api_key_id)
if not api_key:
raise BusinessException(f"API Key {api_key_id} 不存在", BizCode.NOT_FOUND)
for model_config in api_key.model_configs:
compatible_types = {ModelType.LLM, ModelType.CHAT}
config_type = model_config.type
request_type = existing_model.type
if not (config_type == request_type or
(config_type in compatible_types and request_type in compatible_types)):
raise BusinessException(
f"API Key {api_key_id} 关联的模型类型 ({model_config.type}) 与组合模型类型 ({model_data.type}) 不匹配",
BizCode.INVALID_PARAMETER
)
# 更新基本信息
existing_model.name = model_data.name
# existing_model.type = model_data.type
existing_model.logo = model_data.logo
existing_model.description = model_data.description
existing_model.config = model_data.config
existing_model.is_active = model_data.is_active
existing_model.is_public = model_data.is_public
if "load_balance_strategy" in model_data.model_fields_set:
existing_model.load_balance_strategy = model_data.load_balance_strategy
# 更新 API Keys 关联
existing_model.api_keys.clear()
for api_key_id in model_data.api_key_ids:
api_key = ModelApiKeyRepository.get_by_id(db, api_key_id)
if api_key:
existing_model.api_keys.append(api_key)
db.commit()
db.refresh(existing_model)
return existing_model
@staticmethod
def delete_model(db: Session, model_id: uuid.UUID, tenant_id: uuid.UUID | None = None) -> bool:
"""删除模型配置"""
@@ -324,27 +457,133 @@ class ModelApiKeyService:
return ModelApiKeyRepository.get_by_model_config(db, model_config_id, is_active)
@staticmethod
async def create_api_key(db: Session, api_key_data: ModelApiKeyCreate) -> ModelApiKey:
"""创建API Key"""
model_config = ModelConfigRepository.get_by_id(db, api_key_data.model_config_id)
if not model_config:
raise BusinessException("模型配置不存在", BizCode.MODEL_NOT_FOUND)
validation_result = await ModelConfigService.validate_model_config(
async def create_api_key_by_provider(db: Session, data: model_schema.ModelApiKeyCreateByProvider) -> tuple[
list[Any], list[Any]]:
"""根据provider为多个ModelConfig创建API Key"""
created_keys = []
failed_models = [] # 记录验证失败的模型
for model_config_id in data.model_config_ids:
model_config = ModelConfigRepository.get_by_id(db, model_config_id)
if not model_config:
continue
# 从ModelBase获取model_name
model_name = model_config.model_base.name if model_config.model_base else model_config.name
# 检查是否存在API Key包括软删除
existing_key = db.query(ModelApiKey).filter(
ModelApiKey.api_key == data.api_key,
ModelApiKey.provider == data.provider,
ModelApiKey.model_name == model_name
).first()
if existing_key:
# 如果已存在,重新激活并更新
if existing_key.is_active:
continue
existing_key.is_active = True
existing_key.api_base = data.api_base
existing_key.description = data.description
existing_key.config = data.config
existing_key.priority = data.priority
existing_key.model_name = model_name
# 检查是否已关联该模型配置
if model_config not in existing_key.model_configs:
existing_key.model_configs.append(model_config)
created_keys.append(existing_key)
continue
# 验证配置
validation_result = await ModelConfigService.validate_model_config(
db=db,
model_name=api_key_data.model_name,
provider=api_key_data.provider,
api_key=api_key_data.api_key,
api_base=api_key_data.api_base,
model_type=model_config.type, # 传递模型类型
model_name=model_name,
provider=data.provider,
api_key=data.api_key,
api_base=data.api_base,
model_type=model_config.type,
test_message="Hello"
)
print(validation_result)
if not validation_result["valid"]:
raise BusinessException(
f"模型配置验证失败: {validation_result['error']}",
BizCode.INVALID_PARAMETER
if not validation_result["valid"]:
# 记录验证失败的模型,但不抛出异常
failed_models.append(model_name)
continue
# 创建API Key
api_key_data = ModelApiKeyCreate(
model_config_ids=[model_config_id],
model_name=model_name,
description=data.description,
provider=data.provider,
api_key=data.api_key,
api_base=data.api_base,
config=data.config,
is_active=data.is_active,
priority=data.priority
)
api_key_obj = ModelApiKeyRepository.create(db, api_key_data)
created_keys.append(api_key_obj)
if created_keys:
db.commit()
for key in created_keys:
db.refresh(key)
return created_keys, failed_models
@staticmethod
async def create_api_key(db: Session, api_key_data: ModelApiKeyCreate) -> ModelApiKey:
# 验证所有关联的模型配置是否存在
if api_key_data.model_config_ids:
for model_config_id in api_key_data.model_config_ids:
model_config = ModelConfigRepository.get_by_id(db, model_config_id)
if not model_config:
raise BusinessException("模型配置不存在", BizCode.MODEL_NOT_FOUND)
# 检查API Key是否已存在(包括软删除)
existing_key = db.query(ModelApiKey).filter(
ModelApiKey.api_key == api_key_data.api_key,
ModelApiKey.provider == api_key_data.provider,
ModelApiKey.model_name == api_key_data.model_name
).first()
if existing_key:
if existing_key.is_active:
# 如果已激活,跳过
raise BusinessException("该API Key已存在", BizCode.DUPLICATE_NAME)
# 如果已存在,重新激活并更新
existing_key.is_active = True
existing_key.api_base = api_key_data.api_base
existing_key.description = api_key_data.description
existing_key.config = api_key_data.config
existing_key.priority = api_key_data.priority
existing_key.model_name = api_key_data.model_name
# 检查是否已关联该模型配置
if model_config not in existing_key.model_configs:
existing_key.model_configs.append(model_config)
db.commit()
db.refresh(existing_key)
return existing_key
# 验证配置
validation_result = await ModelConfigService.validate_model_config(
db=db,
model_name=api_key_data.model_name,
provider=api_key_data.provider,
api_key=api_key_data.api_key,
api_base=api_key_data.api_base,
model_type=model_config.type,
test_message="Hello"
)
if not validation_result["valid"]:
raise BusinessException(
f"模型配置验证失败: {validation_result['error']}",
BizCode.INVALID_PARAMETER
)
api_key = ModelApiKeyRepository.create(db, api_key_data)
db.commit()
@@ -359,21 +598,19 @@ class ModelApiKeyService:
raise BusinessException("API Key不存在", BizCode.NOT_FOUND)
# 获取关联的模型配置以获取模型类型
model_config = ModelConfigRepository.get_by_id(db, existing_api_key.model_config_id)
if not model_config:
raise BusinessException("关联的模型配置不存在", BizCode.MODEL_NOT_FOUND)
validation_result = await ModelConfigService.validate_model_config(
if existing_api_key.model_configs:
model_config = existing_api_key.model_configs[0]
validation_result = await ModelConfigService.validate_model_config(
db=db,
model_name=api_key_data.model_name,
provider=api_key_data.provider,
api_key=api_key_data.api_key,
api_base=api_key_data.api_base,
model_type=model_config.type, # 传递模型类型
model_name=api_key_data.model_name or existing_api_key.model_name,
provider=api_key_data.provider or existing_api_key.provider,
api_key=api_key_data.api_key or existing_api_key.api_key,
api_base=api_key_data.api_base or existing_api_key.api_base,
model_type=model_config.type,
test_message="Hello"
)
print(validation_result)
if not validation_result["valid"]:
if not validation_result["valid"]:
raise BusinessException(
f"模型配置验证失败: {validation_result['error']}",
BizCode.INVALID_PARAMETER
@@ -417,3 +654,87 @@ class ModelApiKeyService:
if api_kes and len(api_kes) > 0:
return api_kes[0]
raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING)
class ModelBaseService:
"""基础模型服务"""
@staticmethod
def get_model_base_list(db: Session, query: model_schema.ModelBaseQuery, tenant_id: uuid.UUID = None) -> List:
models = ModelBaseRepository.get_list(db, query)
provider_groups = {}
for m in models:
model_dict = model_schema.ModelBase.model_validate(m).model_dump()
if tenant_id:
model_dict['is_added'] = ModelBaseRepository.check_added_by_tenant(db, m.id, tenant_id)
provider = m.provider
if provider not in provider_groups:
provider_groups[provider] = {
"provider": provider,
"models": []
}
provider_groups[provider]["models"].append(model_dict)
return list(provider_groups.values())
@staticmethod
def get_model_base_by_id(db: Session, model_base_id: uuid.UUID):
model = ModelBaseRepository.get_by_id(db, model_base_id)
if not model:
raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND)
return model
@staticmethod
def create_model_base(db: Session, data: model_schema.ModelBaseCreate):
existing = ModelBaseRepository.get_by_name_and_provider(db, data.name, data.provider)
if existing:
raise BusinessException("模型已存在", BizCode.DUPLICATE_NAME)
model_base = ModelBaseRepository.create(db, data.model_dump())
db.commit()
db.refresh(model_base)
return model_base
@staticmethod
def update_model_base(db: Session, model_base_id: uuid.UUID, data: model_schema.ModelBaseUpdate):
model_base = ModelBaseRepository.update(db, model_base_id, data.model_dump(exclude_unset=True))
if not model_base:
raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND)
db.commit()
db.refresh(model_base)
return model_base
@staticmethod
def delete_model_base(db: Session, model_base_id: uuid.UUID) -> bool:
success = ModelBaseRepository.delete(db, model_base_id)
if not success:
raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND)
db.commit()
return success
@staticmethod
def add_model_from_plaza(db: Session, model_base_id: uuid.UUID, tenant_id: uuid.UUID) -> ModelConfig:
model_base = ModelBaseRepository.get_by_id(db, model_base_id)
if not model_base:
raise BusinessException("基础模型不存在", BizCode.MODEL_NOT_FOUND)
if ModelBaseRepository.check_added_by_tenant(db, model_base_id, tenant_id):
raise BusinessException("模型已添加", BizCode.DUPLICATE_NAME)
model_config_data = {
"model_id": model_base_id,
"tenant_id": tenant_id,
"name": model_base.name,
"provider": model_base.provider,
"type": model_base.type,
"logo": model_base.logo,
"description": model_base.description,
"is_composite": False
}
model_config = ModelConfigRepository.create(db, model_config_data)
ModelBaseRepository.increment_add_count(db, model_base_id)
db.commit()
db.refresh(model_config)
return model_config

View File

@@ -7,6 +7,7 @@ from sqlalchemy.orm import Session
from app.models import MultiAgentConfig, AgentConfig, ModelConfig
from app.models.multi_agent_model import AggregationStrategy, OrchestrationMode
from app.repositories.model_repository import ModelApiKeyRepository
from app.services.agent_registry import AgentRegistry
from app.services.master_agent_router import MasterAgentRouter
from app.services.conversation_state_manager import ConversationStateManager
@@ -2546,10 +2547,14 @@ class MultiAgentOrchestrator:
return self._smart_merge_results(results, strategy)
# 获取 API Key 配置
api_key_config = self.db.query(ModelApiKey).filter(
ModelApiKey.model_config_id == default_model_config_id,
ModelApiKey.is_active == True
).first()
# api_key_config = self.db.query(ModelApiKey).join(
# ModelConfig, ModelApiKey.model_configs
# ).filter(
# ModelConfig.id == default_model_config_id,
# ModelApiKey.is_active.is_(True)
# ).first()
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, default_model_config_id)
api_key_config = api_keys[0] if api_keys else None
if not api_key_config:
logger.warning("Master Agent 没有可用的 API Key使用简单整合")
@@ -2703,10 +2708,14 @@ class MultiAgentOrchestrator:
return
# 获取 API Key 配置
api_key_config = self.db.query(ModelApiKey).filter(
ModelApiKey.model_config_id == default_model_config_id,
ModelApiKey.is_active == True
).first()
# api_key_config = self.db.query(ModelApiKey).join(
# ModelConfig, ModelApiKey.model_configs
# ).filter(
# ModelConfig.id == default_model_config_id,
# ModelApiKey.is_active.is_(True)
# ).first()
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, default_model_config_id)
api_key_config = api_keys[0] if api_keys else None
if not api_key_config:
logger.warning("Master Agent 没有可用的 API Key使用简单整合")

View File

@@ -74,7 +74,7 @@ class MultiAgentService:
select(MultiAgentConfig)
.where(
MultiAgentConfig.app_id == app_id,
MultiAgentConfig.is_active == True
MultiAgentConfig.is_active.is_(True)
)
.order_by(MultiAgentConfig.updated_at.desc())
).first()
@@ -144,7 +144,7 @@ class MultiAgentService:
select(MultiAgentConfig)
.where(
MultiAgentConfig.app_id == app_id,
MultiAgentConfig.is_active == True
MultiAgentConfig.is_active.is_(True)
)
.order_by(MultiAgentConfig.updated_at.desc())
).first()

View File

@@ -91,7 +91,7 @@ async def run_pilot_extraction(
dialog = DialogData(
context=context,
ref_id="pilot_dialog_1",
group_id=str(memory_config.workspace_id),
end_user_id=str(memory_config.workspace_id),
user_id=str(memory_config.tenant_id),
apply_id=str(memory_config.config_id),
metadata={"source": "pilot_run", "input_type": "frontend_text"},

View File

@@ -16,7 +16,7 @@ from app.models.prompt_optimizer_model import (
PromptOptimizerSession,
RoleType
)
from app.repositories.model_repository import ModelConfigRepository
from app.repositories.model_repository import ModelConfigRepository, ModelApiKeyRepository
from app.repositories.prompt_optimizer_repository import (
PromptOptimizerSessionRepository
)
@@ -168,7 +168,8 @@ class PromptOptimizerService:
logger.info(f"Prompt optimization started, user_id={user_id}, session_id={session_id}")
# Create LLM instance
api_config: ModelApiKey = model_config.api_keys[0]
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config.id)
api_config: ModelApiKey = api_keys[0] if api_keys else None
llm = RedBearLLM(RedBearModelConfig(
model_name=api_config.model_name,
provider=api_config.provider,

View File

@@ -4,6 +4,8 @@ import time
import asyncio
from typing import Optional, Dict, Any, AsyncGenerator
from sqlalchemy.orm import Session
from app.repositories.model_repository import ModelApiKeyRepository
from app.services.memory_konwledges_server import write_rag
from app.models import ReleaseShare, AppRelease, Conversation
from app.services.conversation_service import ConversationService
@@ -164,16 +166,20 @@ class SharedChatService:
raise ResourceNotFoundException("模型配置", str(model_config_id))
# 获取 API Key
stmt = (
select(ModelApiKey)
.where(
ModelApiKey.model_config_id == model_config_id,
ModelApiKey.is_active == True
)
.order_by(ModelApiKey.priority.desc())
.limit(1)
)
api_key_obj = self.db.scalars(stmt).first()
# stmt = (
# select(ModelApiKey).join(
# ModelConfig, ModelApiKey.model_configs
# )
# .where(
# ModelConfig.id == model_config_id,
# ModelApiKey.is_active.is_(True)
# )
# .order_by(ModelApiKey.priority.desc())
# .limit(1)
# )
# api_key_obj = self.db.scalars(stmt).first()
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config_id)
api_key_obj = api_keys[0] if api_keys else None
if not api_key_obj:
raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING)
@@ -358,16 +364,20 @@ class SharedChatService:
raise ResourceNotFoundException("模型配置", str(model_config_id))
# 获取 API Key
stmt = (
select(ModelApiKey)
.where(
ModelApiKey.model_config_id == model_config_id,
ModelApiKey.is_active == True
)
.order_by(ModelApiKey.priority.desc())
.limit(1)
)
api_key_obj = self.db.scalars(stmt).first()
# stmt = (
# select(ModelApiKey).join(
# ModelConfig, ModelApiKey.model_configs
# )
# .where(
# ModelConfig.id == model_config_id,
# ModelApiKey.is_active.is_(True)
# )
# .order_by(ModelApiKey.priority.desc())
# .limit(1)
# )
# api_key_obj = self.db.scalars(stmt).first()
api_keys = ModelApiKeyRepository.get_by_model_config(self.db, model_config_id)
api_key_obj = api_keys[0] if api_keys else None
if not api_key_obj:
raise BusinessException("没有可用的 API Key", BizCode.AGENT_CONFIG_MISSING)
@@ -598,7 +608,7 @@ class SharedChatService:
# 获取多 Agent 配置
multi_agent_config = self.db.query(MultiAgentConfig).filter(
MultiAgentConfig.app_id == release.app_id,
MultiAgentConfig.is_active == True
MultiAgentConfig.is_active.is_(True)
).first()
if not multi_agent_config:
@@ -695,7 +705,7 @@ class SharedChatService:
# 获取多 Agent 配置
multi_agent_config = self.db.query(MultiAgentConfig).filter(
MultiAgentConfig.app_id == release.app_id,
MultiAgentConfig.is_active == True
MultiAgentConfig.is_active.is_(True)
).first()
if not multi_agent_config:

View File

@@ -155,10 +155,10 @@ class MemoryInsightHelper:
"""
query = """
MATCH (d:Dialogue)
WHERE d.group_id = $group_id AND d.created_at IS NOT NULL AND d.created_at <> ''
WHERE d.end_user_id = $end_user_id AND d.created_at IS NOT NULL AND d.created_at <> ''
RETURN d.created_at AS creation_time
"""
records = await self.neo4j_connector.execute_query(query, group_id=self.user_id)
records = await self.neo4j_connector.execute_query(query, end_user_id=self.user_id)
if not records:
return []
@@ -211,17 +211,17 @@ class MemoryInsightHelper:
async def get_social_connections(self) -> dict | None:
"""Find the user with whom the most memories are shared."""
query = """
MATCH (c1:Chunk {group_id: $group_id})
MATCH (c1:Chunk {end_user_id: $end_user_id})
OPTIONAL MATCH (c1)-[:CONTAINS]->(s:Statement)
OPTIONAL MATCH (s)<-[:CONTAINS]-(c2:Chunk)
WHERE c1.group_id <> c2.group_id AND s IS NOT NULL AND c2 IS NOT NULL
WITH c2.group_id AS other_user_id, COUNT(DISTINCT s) AS common_statements
WHERE c1.end_user_id <> c2.end_user_id AND s IS NOT NULL AND c2 IS NOT NULL
WITH c2.end_user_id AS other_user_id, COUNT(DISTINCT s) AS common_statements
WHERE common_statements > 0
RETURN other_user_id, common_statements
ORDER BY common_statements DESC
LIMIT 1
"""
records = await self.neo4j_connector.execute_query(query, group_id=self.user_id)
records = await self.neo4j_connector.execute_query(query, end_user_id=self.user_id)
if not records or not records[0].get("other_user_id"):
return None
@@ -230,7 +230,7 @@ class MemoryInsightHelper:
time_range_query = """
MATCH (c:Chunk)
WHERE c.group_id IN [$user_id, $other_user_id]
WHERE c.end_user_id IN [$user_id, $other_user_id]
RETURN min(c.created_at) AS start_time, max(c.created_at) AS end_time
"""
time_records = await self.neo4j_connector.execute_query(
@@ -294,11 +294,11 @@ class UserSummaryHelper:
"""Fetch recent statements authored by the user/group for context."""
query = (
"MATCH (s:Statement) "
"WHERE s.group_id = $group_id AND s.statement IS NOT NULL "
"WHERE s.end_user_id = $end_user_id AND s.statement IS NOT NULL "
"RETURN s.statement AS statement, s.created_at AS created_at "
"ORDER BY created_at DESC LIMIT $limit"
)
rows = await self.connector.execute_query(query, group_id=self.user_id, limit=limit)
rows = await self.connector.execute_query(query, end_user_id=self.user_id, limit=limit)
records = []
for r in rows:
try:
@@ -1152,7 +1152,7 @@ async def analytics_user_summary(end_user_id: Optional[str] = None) -> Dict[str,
import re
# 创建 UserSummaryHelper 实例
user_summary_tool = UserSummaryHelper(end_user_id or os.getenv("SELECTED_GROUP_ID", "group_123"))
user_summary_tool = UserSummaryHelper(end_user_id or os.getenv("SELECTED_end_user_id", "group_123"))
try:
# 1) 收集上下文数据
@@ -1273,10 +1273,10 @@ async def analytics_node_statistics(
if end_user_id:
query = f"""
MATCH (n:{node_type})
WHERE n.group_id = $group_id
WHERE n.end_user_id = $end_user_id
RETURN count(n) as count
"""
result = await _neo4j_connector.execute_query(query, group_id=end_user_id)
result = await _neo4j_connector.execute_query(query, end_user_id=end_user_id)
else:
query = f"""
MATCH (n:{node_type})
@@ -1387,10 +1387,10 @@ async def analytics_memory_types(
# 查询 Statement 节点数量
query = """
MATCH (n:Statement)
WHERE n.group_id = $group_id
WHERE n.end_user_id = $end_user_id
RETURN count(n) as count
"""
result = await _neo4j_connector.execute_query(query, group_id=end_user_id)
result = await _neo4j_connector.execute_query(query, end_user_id=end_user_id)
statement_count = result[0]["count"] if result and len(result) > 0 else 0
# 取三分之一作为隐性记忆数量
implicit_count = round(statement_count / 3)
@@ -1504,7 +1504,7 @@ async def analytics_graph_data(
包含节点、边和统计信息的字典
"""
try:
# 1. 获取 group_id
# 1. 获取 end_user_id
user_uuid = uuid.UUID(end_user_id)
repo = EndUserRepository(db)
end_user = repo.get_by_id(user_uuid)
@@ -1528,7 +1528,7 @@ async def analytics_graph_data(
# 基于中心节点的扩展查询
node_query = f"""
MATCH path = (center)-[*1..{depth}]-(connected)
WHERE center.group_id = $group_id
WHERE center.end_user_id = $end_user_id
AND elementId(center) = $center_node_id
WITH collect(DISTINCT center) + collect(DISTINCT connected) as all_nodes
UNWIND all_nodes as n
@@ -1539,7 +1539,7 @@ async def analytics_graph_data(
LIMIT $limit
"""
node_params = {
"group_id": end_user_id,
"end_user_id": end_user_id,
"center_node_id": center_node_id,
"limit": limit
}
@@ -1547,7 +1547,7 @@ async def analytics_graph_data(
# 按节点类型过滤查询
node_query = """
MATCH (n)
WHERE n.group_id = $group_id
WHERE n.end_user_id = $end_user_id
AND labels(n)[0] IN $node_types
RETURN
elementId(n) as id,
@@ -1556,7 +1556,7 @@ async def analytics_graph_data(
LIMIT $limit
"""
node_params = {
"group_id": end_user_id,
"end_user_id": end_user_id,
"node_types": node_types,
"limit": limit
}
@@ -1564,7 +1564,7 @@ async def analytics_graph_data(
# 查询所有节点
node_query = """
MATCH (n)
WHERE n.group_id = $group_id
WHERE n.end_user_id = $end_user_id
RETURN
elementId(n) as id,
labels(n)[0] as label,
@@ -1572,7 +1572,7 @@ async def analytics_graph_data(
LIMIT $limit
"""
node_params = {
"group_id": end_user_id,
"end_user_id": end_user_id,
"limit": limit
}

View File

@@ -528,7 +528,8 @@ class WorkflowService:
self.conversation_service.add_message(
conversation_id=conversation_id_uuid,
role=message["role"],
content=message["content"]
content=message["content"],
meta_data=None if message["role"] == "user" else {"usage": token_usage}
)
logger.info(f"Workflow Run Success, "
f"execution_id: {execution.execution_id}, message count: {len(final_messages)}")
@@ -678,7 +679,8 @@ class WorkflowService:
self.conversation_service.add_message(
conversation_id=conversation_id_uuid,
role=message["role"],
content=message["content"]
content=message["content"],
meta_data=None if message["role"] == "user" else {"usage": token_usage}
)
logger.info(f"Workflow Run Success, "
f"execution_id: {execution.execution_id}, message count: {len(final_messages)}")
@@ -761,7 +763,10 @@ class WorkflowService:
# 4. 获取工作空间 ID从 app 获取)
from app.models import App
app = self.db.query(App).filter(App.id == app_id).first()
app = self.db.query(App).filter(
App.id == app_id,
App.is_active.is_(True)
).first()
if not app:
raise BusinessException(
code=BizCode.NOT_FOUND,