Commit Graph

51 Commits

Author SHA1 Message Date
lixinyue11
09afec17f9 Fix/develop memory bug (#354)
* 遗漏的历史映射

* 遗漏的历史映射

* fix_timeline_memories

* fix_timeline_memories

* write_gragp/bug_fix

* write_gragp/bug_fix

* write_gragp/bug_fix

* write_gragp/bug_fix

* Multiple independent transactions - single transaction

* memory_content ->memory_config_id

* memory_content ->memory_config_id

* memory_content ->memory_config_id

* memory_content ->memory_config_id

* memory_content ->memory_config_id

* memory_content ->memory_config_id

* memory_content ->memory_config_id

* tasks/bug_fix/long

* tasks_reflection/bug/fix

* tasks_reflection/bug/fix

* tasks_reflection/bug/fix

* tasks_reflection/bug/fix

* change/get_db_context/way

* change/get_db_context/way
2026-02-06 18:45:47 +08:00
Ke Sun
8a37869489 feat(memory): refactor config resolution to always retrieve workspace_id fallback 2026-02-06 15:14:08 +08:00
Ke Sun
5c10f11681 feat(memory): add workspace_id fallback support for memory config resolution
- Add workspace_id fallback parameter to memory config loading across all services
- Update hot_memory_tags.py to pass workspace_id when resolving memory configuration
- Enhance emotion_analytics_service.py to support workspace_id as fallback for config resolution
- Improve implicit_memory_service.py with workspace_id fallback in config loading
- Update memory_agent_service.py to handle workspace_id resolution and add refactoring TODO
- Enhance preference_analysis.jinja2 prompt with critical guidance on supporting_evidence extraction
- Add validation to check both config_id and workspace_id before raising configuration errors
- Improve error handling and logging for memory configuration resolution across services
- This enables more flexible memory configuration resolution when config_id is unavailable
2026-02-06 14:48:58 +08:00
Ke Sun
7a78f15a90 Merge branch 'develop' into fix/memory-enduser-config 2026-02-06 11:56:21 +08:00
lixinyue11
c1941809e9 Fix/develop memory bug (#336)
* 遗漏的历史映射

* 遗漏的历史映射

* fix_timeline_memories

* fix_timeline_memories

* write_gragp/bug_fix

* write_gragp/bug_fix

* write_gragp/bug_fix

* write_gragp/bug_fix

* Multiple independent transactions - single transaction

* memory_content ->memory_config_id

* memory_content ->memory_config_id
2026-02-06 11:42:02 +08:00
Ke Sun
a3cf773e75 fix(agent): add memory config validation and fix config id reference
- Add null check for actual_config_id before calling term_memory_save in langchain_agent.py to prevent errors when memory config is unavailable
- Add warning log when skipping term_memory_save due to missing memory config
- Fix incorrect attribute reference from memory_config.id to memory_config.config_id in memory_agent_service.py
- Fix method call from private _get_workspace_default_config to public get_workspace_default_config in memory_config_service.py
- Ensures graceful handling of missing memory configurations and prevents runtime errors
2026-02-05 10:19:43 +08:00
Ke Sun
1e9c32a102 Merge branch 'develop' into fix/memory-enduser-config 2026-02-03 19:40:08 +08:00
乐力齐
8670aaba1e Fix/language unification (#283)
* [changes]add user_summary language unification

* [add]Entity extraction, user memory, emotion suggestions, unified language type for writing

* [add]Complete the switch between Chinese and English for the emotion labels and emotion suggestions fields.

* [changes]add user_summary language unification

* [add]Entity extraction, user memory, emotion suggestions, unified language type for writing

* [add]Complete the switch between Chinese and English for the emotion labels and emotion suggestions fields.

* [changes]Modify the code based on the AI review
2026-02-03 16:03:08 +08:00
Ke Sun
48f95e0ea4 refactor(memory): simplify config retrieval and remove redundant functions
- Remove get_memory_config_id function from end_user_repository.py as it's no longer needed
- Remove get_end_user_memory_config_id function from memory_agent_service.py to reduce duplication
- Simplify get_end_user_connected_config to use MemoryConfigService.get_config_with_fallback
- Update get_config_with_fallback signature to accept memory_config_id directly instead of end_user_id
- Remove unnecessary AppRelease query and config parsing logic from get_end_user_connected_config
- Streamline memory config retrieval flow to use service layer abstraction
- Improves code maintainability by centralizing config fallback logic in MemoryConfigService
2026-02-02 14:38:17 +08:00
Ke Sun
42b59a644d feat(memory): add protected memory config deletion with end-user safeguards
- Add force parameter to delete_config endpoint for controlled deletion of in-use configs
- Implement MemoryConfigService.delete_config with protection against deleting default configs
- Add validation to prevent deletion of configs with connected end-users unless force=True
- Reorganize controller imports to remove duplicates and improve maintainability
- Clean up unused database connection management code from memory_storage_controller
- Add detailed docstring to delete_config endpoint explaining protection mechanisms
- Update error handling with specific BizCode.RESOURCE_IN_USE for configs in active use
- Add comprehensive logging for deletion attempts, warnings, and affected users
- Refactor ConfigParamsDelete schema usage to use MemoryConfigService directly
- Improve API response structure with affected_users count and force_required flag
2026-01-29 12:05:50 +08:00
Ke Sun
d9fa9039bb feat(memory): add memory config caching to end_user model
- Add memory_config_id field to EndUser model for lazy caching of memory configuration
- Create get_end_user_memory_config_id() function for fast retrieval of cached config ID
- Implement lazy update mechanism in get_end_user_connected_config() to cache memory_config_id
- Optimize memory config lookup by storing config ID directly on end_user record
- Improve import organization and formatting in memory_agent_service.py
- Add indexed foreign key relationship to data_config table for efficient queries
2026-01-29 12:05:50 +08:00
lixinyue
00c4a524b7 旧数据兼容 2026-01-28 16:04:38 +08:00
lixinyue
2a23082203 config_id做映射+1 2026-01-27 21:15:38 +08:00
lixinyue
d51355a0ad config_id做映射+1 2026-01-27 21:09:06 +08:00
lixinyue
f821893653 config_id做映射 2026-01-27 20:22:14 +08:00
lixinyue
a047cf2e91 修复宿主列表获取memory_config_idBUG 2026-01-27 14:32:48 +08:00
lixinyue
af6e1e2b99 end_user_id清理干净 2026-01-23 17:20:07 +08:00
lixinyue
4ad51c1b24 Merge branch 'refs/heads/develop' into fix/memory_bug_fix 2026-01-23 17:15:22 +08:00
lixinyue11
1919580759 Fix/memory mcp2 1 (#190)
* 优化快速检索的回复内容

* 优化快速检索的回复内容

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问

* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问

* 深度检索优化,搜索不到数据/提问的概念过于蘑菇,以引导的方式继续提问
2026-01-23 17:12:21 +08:00
lixinyue
1aabaff1f2 解决冲突 2026-01-23 15:00:09 +08:00
lixinyue
21c0383efb Merge branch 'refs/heads/develop' into fix/memory_bug_fix
# Conflicts:
#	api/app/services/memory_agent_service.py
2026-01-23 14:57:25 +08:00
lixinyue11
313f19eba4 Fix/memory mcp2 1 (#188)
* 优化快速检索的回复内容

* 优化快速检索的回复内容

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* 路径的BUG修复

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG

* LLM生存缺少config_id认证,修复BUG
2026-01-23 14:49:44 +08:00
lixinyue11
15f9c49418 Fix/memory mcp2 1 (#184)
* 优化快速检索的回复内容

* 优化快速检索的回复内容
2026-01-23 12:21:54 +08:00
乐力齐
7870c6c33f Fix/interface home (#182)
* [fix]Fix the interface for statistics of recent activities and applications

* [changes]Modify the code based on the AI review
1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter.
2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls.

* [fix]Fix the interface for statistics of recent activities and applications

* [changes]Modify the code based on the AI review
1.Use the boolean auxiliary methods provided by SQLAlchemy instead of using == True in the is_active filter.
2.The calculation of the "PROJECT_ROOT" has now been hardcoded with five levels of nested os.path.dirname calls.
2026-01-23 10:50:24 +08:00
lixinyue
9e828b1750 config_id字段改成UUID,与develop校对恢复 2026-01-22 21:53:15 +08:00
lixinyue
8db4f914d8 config_config替换成memory_config 2026-01-22 18:43:22 +08:00
lixinyue
a2a69840f7 config_config替换成memory_config 2026-01-22 16:38:24 +08:00
lixinyue
1c7fe6d134 config_config替换成memory_config 2026-01-22 14:59:01 +08:00
lixinyue
4aeec8afbf 把group_id替换end_user_id_ 2026-01-21 20:37:39 +08:00
lixinyue
f0efed8aa1 把group_id替换end_user_id 2026-01-21 20:33:22 +08:00
lixinyue
4a4931bee2 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段 2026-01-21 19:37:03 +08:00
lixinyue11
fb25495f1b Fix/memory mcp2 1 (#170)
* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* feat(celery): add comprehensive logging to worker and write task

- Initialize logging system in Celery worker entry point with LoggingConfig
- Add logger instance and startup message to celery_worker.py
- Reorganize imports in tasks.py for better readability and consistency
- Add detailed logging to write_message_task for debugging and monitoring
- Log task start with group_id, config_id, and storage_type parameters
- Log service execution and completion status with results
- Add exception handling with error logging and stack trace capture
- Log task completion time and Celery task ID for performance tracking
- Improves observability and troubleshooting of async task execution

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 快速检索,需要在接口部分添加LLM整合

* 快速检索,需要在接口部分添加LLM整合

---------

Co-authored-by: Ke Sun <kesun5@illinois.edu>
2026-01-21 18:21:51 +08:00
乐力齐
b6e6dbf27f Fix/memory interface (#169)
* [changes]《Modify the interface》
1.Remove the "/search/entity_graph" interface
2.Reconstruct the "/updated_end_user/profile" interface
3.Remove the "Update Username" interface
4.Fix the batch query of user association memory configuration

* [changes]《Modify the interface》
1.Remove the "/search/entity_graph" interface
2.Reconstruct the "/updated_end_user/profile" interface
3.Remove the "Update Username" interface
4.Fix the batch query of user association memory configuration

* [fix]Fix the error response type
2026-01-21 18:20:28 +08:00
Ke Sun
c24fb73147 Fix/memory celery fix (#168)
* refactor(celery): optimize task routing and worker configuration

- Simplify Celery queue configuration with single default 'io_tasks' queue
- Implement task routing strategy separating IO-bound and CPU-bound tasks
- Add Flower monitoring support with task event tracking enabled
- Add summary node search optimization to only retrieve summary nodes
- Clean up unused imports and reorganize import statements for consistency
- Update docker-compose configuration to support multi-queue worker setup

* chore(celery): simplify flower configuration and add gevent dependency

* chore(dependencies): add gevent dependency to requirements

- Add gevent==24.11.1 to api/requirements.txt
- Gevent is required for async worker support in Celery
- Complements existing flower and celery configuration

* refactor(celery): simplify async event loop handling and reorganize task queues

- Replace complex nest_asyncio and manual event loop management with asyncio.run() in read_message_task, write_message_task, regenerate_memory_cache, and workspace_reflection_task
- Rename task queues from io_tasks/cpu_tasks to memory_tasks/document_tasks for better semantic clarity
- Update task routing configuration to reflect new queue names for memory agent tasks and document processing tasks
- Remove redundant exception handling comments and simplify error handling logic
- Update README with improved community support section including GitHub Issues, Pull Requests, Discussions, and WeChat community links
- Simplifies event loop management by leveraging asyncio.run() which handles loop creation and cleanup automatically, reducing code complexity and potential race conditions
2026-01-21 17:58:46 +08:00
乐力齐
37ef497f4c Feature/distinction role (#167)
* [feature]A set of information for role recognition writing

* [feature]A set of information for role recognition writing

* [fix]Fix the code after rebasing.

* [feature]A set of information for role recognition writing

* [fix]Fix the code after rebasing.

* [fix]Based on the AI review to fix the code

* [changes]Disable the function of batch writing multiple groups of conversations in a cumulative manner

* [fix]Addressing vulnerability risks

* [fix]Fixing short-term memory writing

* [feature]A set of information for role recognition writing

* [fix]Fix the code after rebasing.

* [feature]A set of information for role recognition writing

* [fix]Fix the code after rebasing.

* [fix]Based on the AI review to fix the code

* [fix]Fixing short-term memory writing
2026-01-21 16:04:16 +08:00
乐力齐
2e504f9c48 Feature/distinction role (#165)
* [feature]A set of information for role recognition writing

* [feature]A set of information for role recognition writing

* [fix]Fix the code after rebasing.

* [feature]A set of information for role recognition writing

* [fix]Fix the code after rebasing.

* [fix]Based on the AI review to fix the code

* [changes]Disable the function of batch writing multiple groups of conversations in a cumulative manner

* [fix]Addressing vulnerability risks
2026-01-21 13:55:32 +08:00
lixinyue11
6920deef63 Fix/memory bug fix (#162)
* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁

* 输出数组

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化1.0(优化隐私输出、时间检索)

* 反思优化测试接口

* 反思优化测试接口

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 读取接口内层嵌套BUG修复

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段

* 新增中翻英功能(记忆时间线)(用户摘要)(兴趣分布接口)(查询核心档案)(记忆洞察)-接口添加翻译字段
2026-01-21 11:33:52 +08:00
lixinyue11
642587fc97 Fix/memory mcp2 1 (#145)
* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* 去掉MCP框架,重构

* feat(celery): add comprehensive logging to worker and write task

- Initialize logging system in Celery worker entry point with LoggingConfig
- Add logger instance and startup message to celery_worker.py
- Reorganize imports in tasks.py for better readability and consistency
- Add detailed logging to write_message_task for debugging and monitoring
- Log task start with group_id, config_id, and storage_type parameters
- Log service execution and completion status with results
- Add exception handling with error logging and stack trace capture
- Log task completion time and Celery task ID for performance tracking
- Improves observability and troubleshooting of async task execution

* 去掉MCP框架,重构

* 去掉MCP框架,重构

---------

Co-authored-by: Ke Sun <kesun5@illinois.edu>
2026-01-20 10:36:30 +08:00
lixinyue11
000fbf6e98 Fix/memory bug fix (#128)
* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 读取的接口,去掉全局锁
2026-01-15 16:54:09 +08:00
lixinyue11
8262045b1e Fix/memory bug fix (#118)
* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 图谱数据量限制数量去掉

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化

* 用户详情优化
2026-01-14 18:36:24 +08:00
Ke Sun
a6e1898e1b perf(memory): add detailed performance logging and optimize batch access recording
- Add [PERF] prefixed logging throughout hybrid search pipeline for better performance visibility
- Break down latency metrics with separate timing for config loading, embedder initialization, and rerank computation
- Format latency breakdown as JSON in performance summary logs
- Optimize batch_record_access to process node access records in parallel using asyncio.gather instead of sequential processing
- Add performance timing instrumentation for forgetting config loading and rerank computation stages
- Reorganize imports in access_history_manager for consistency
- Improve observability of search performance bottlenecks through structured logging
2026-01-14 12:02:10 +08:00
乐力齐
0a73b18823 Feature/return memoryconfig (#89)
* [add]Newly added: Memory configuration for returning results

* [add]Newly added: Memory configuration for returning results

* [changes]Based on the improvement of AI review
2026-01-13 14:55:12 +08:00
lixinyue11
17318d8205 Flx/develop config (#77)
* 修改BUG,读取接口的end_user_id获取不准确修复,statements检索内容为空需要给空列表

* 反思的默认部分检索替换为partial

* 反思的默认部分检索替换为partial

* 反思的默认部分检索替换为partial
2026-01-12 18:46:07 +08:00
lixinyue11
5b05009123 修改BUG,读取接口的end_user_id获取不准确修复,statements检索内容为空需要给空列表 (#75) 2026-01-12 17:40:49 +08:00
lixinyue11
bcb3d587a1 dev新增短期记忆功能 (#47)
* dev新增短期记忆功能

* dev新增短期记忆功能

* dev新增短期记忆功能

* dev新增短期记忆功能

* dev新增短期记忆功能

* dev新增短期记忆功能

* dev新增短期记忆功能
2026-01-07 16:36:11 +08:00
Ke Sun
0fff23d2a2 feat(memory): make config_id optional and improve configuration validation
- Make config_id optional in UserInput and Write_UserInput schemas
- Update write_memory and read_memory method signatures to accept Optional[str] for config_id
- Implement automatic config retrieval from end_user connection when config_id is not provided
- Add explicit error handling for missing memory configurations with descriptive error messages
- Improve emotion_controller to validate config_id using MemoryConfigService when provided
- Fallback to get_end_user_connected_config when config_id is None
- Distinguish between specific "no configuration found" errors and other exceptions for better debugging
- Ensures users can operate without explicitly providing config_id if they have a connected configuration
2025-12-24 16:06:54 +08:00
Ke Sun
283c64a358 refactor(memory): restructure memory system and improve configuration management
- Remove deprecated main.py entry point from memory module
- Reorganize imports across controllers and services for consistency
- Update emotion controller to pass db session instead of config_id to services
- Enhance memory agent controller with db session parameter for status_type and user_profile endpoints
- Refactor memory agent service to accept db parameter in classify_message_type method
- Improve configuration handling in celery_app by removing automatic database reload
- Update all memory-related services to use centralized config management
- Standardize import ordering and remove unused imports across 50+ files
- Add pilot_run_service for new pilot execution workflow
- Refactor extraction engine, reflection engine, and search services for better modularity
- Update LLM utilities and embedder configuration for improved flexibility
- Enhance type classifier and verification tools with better error handling
- Improve memory evaluation modules (LOCOMO, LongMemEval, MemSciQA) with consistent patterns
2025-12-23 17:17:04 +08:00
Ke Sun
1e3ba39150 refactor(memory): restructure memory agent and config management
- Reorganize imports and remove unused dependencies across memory agent controllers
- Extract config validation logic into dedicated validators module
- Create new memory_config_model and memory_config_schema for configuration management
- Implement memory_config_service for centralized config handling
- Add embedder_utils module for embedding model utilities
- Refactor memory agent service to use new config validation framework
- Clean up configuration files (remove config.json, testdata.json, dbrun.json)
- Remove deprecated hybrid_chatbot.py and config overrides
- Update logging configuration and error handling across memory modules
- Consolidate LLM and embedding model validation into validators
- Improve code organization and reduce duplication in memory storage services
- Enhance type classification and verification tools with better error handling
2025-12-21 20:32:41 +08:00
Mark
a4e276ab27 [MODIFY] Code optimization 2025-12-15 14:09:43 +08:00
lanceyq
0117194a67 [fix]修复服务启动 (#3) 2025-12-04 18:50:09 +08:00