* feat(web): add PageEmpty component
* feat(web): add PageTabs component
* feat(web): add PageEmpty component
* feat(web): add PageTabs component
* feat(prompt): add history tracking for prompt releases
* feat(web): add prompt menu
* refactor: The PageScrollList component supports two generic parameters
* feat(web): BodyWrapper compoent update PageLoading
* feat(web): add Ontology menu
* feat(web): memory management add scene
* feat(tasks): add celery task configuration for periodic jobs
- Add ignore_result=True to prevent storing results for periodic tasks
- Set max_retries=0 to skip failed periodic tasks without retry attempts
- Configure acks_late=False for immediate acknowledgment in beat tasks
- Add time_limit and soft_time_limit to regenerate_memory_cache task (3600s/3300s)
- Add time_limit and soft_time_limit to workspace_reflection_task (300s/240s)
- Add time_limit and soft_time_limit to run_forgetting_cycle_task (7200s/7000s)
- Improve task reliability and resource management for scheduled jobs
* feat(sandbox): add Node.js code execution support to sandbox
* Release/v0.2.2 (#260)
* [modify] migration script
* [add] migration script
* fix(web): change form message
* fix(web): the memoryContent field is compatible with numbers and strings
* feat(web): code node hidden
* fix(model):
1. create a basic model to check if the name and provider are duplicated.
2. The result shows error models because the provider created API Keys for all matching models.
---------
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
* Feature/ontology class clean (#249)
* [add] Complete ontology engineering feature implementation
* [add] Add ontology feature integration and validation utilities
* [add] Add OWL validator and validation utilities
* [fix] Add missing render_ontology_extraction_prompt function
* [fix]Add dependencies, fix functionality
* [add] migration script
* feat(celery): add dedicated periodic tasks worker and queue (#261)
* fix(web): conflict resolve
* Fix/v022 bug (#263)
* [fix]Fix the issue of inconsistent language in explicit and episodic memory.
* [fix]Fix the issue of inconsistent language in explicit and episodic memory.
* [add]Add scene_id
* [fix]Based on the AI review to fix the code
* Fix/develop memory reflex (#265)
* 遗漏的历史映射
* 遗漏的历史映射
* 反思后台报错处理
* [add] migration script
* fix: chat conversation_id add node_start
* feat(web): show code node
* fix(web): Restructure the CustomSelect component, repair the interface that is called multiple times when the form is updated
* feat(web): RadioGroupCard support block mode
* feat(web): create space add icon
* feat(app and model): token consumption statistics
* Add/develop memory (#264)
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 新增长期记忆功能
* 新增长期记忆功能
* 新增长期记忆功能
* 知识库检索多余字段
* 长期
* feat(app and model): token consumption statistics of the cluster
* memory_BUG_fix
* fix(web): prompt history remove pageLoading
* fix(prompt): remove hard-coded import of prompt file paths (#279)
* Fix/develop memory bug (#274)
* 遗漏的历史映射
* 遗漏的历史映射
* fix_timeline_memories
* fix(web): update retrieve_type key
* Fix/develop memory bug (#276)
* 遗漏的历史映射
* 遗漏的历史映射
* fix_timeline_memories
* fix_timeline_memories
* write_gragp/bug_fix
* write_gragp/bug_fix
* write_gragp/bug_fix
* chore(celery): disable periodic task scheduling
* fix(prompt): remove hard-coded import of prompt file paths
---------
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Ke Sun <kesun5@illinois.edu>
* fix(web): remove delete confirm content
* refactor(workflow): relocate template directory into workflow
* feat(memory): add long-term storage task routing and batching
* fix(web): PageScrollList loading update
* fix(web): PageScrollList loading update
* Ontology v1 bug (#291)
* [changes]Add 'id' as the secondary sorting key, and 'scene_id' now returns a UUID object
* [fix]Fix the "end_user" return to be sorted by update time.
* [fix]Set the default values of the memory configuration model based on the spatial model.
* [fix]Remove the entity extraction check combination model, read the configuration list, and add the return of scene_id
* [fix]Fix the "end_user" return to be sorted by update time.
* [fix]
* fix(memory): add Redis session validation
- Add macOS fork() safety configuration in celery_app.py to prevent initialization issues
- Add null/False checks for Redis session queries in term_memory_save to handle missing sessions gracefully
- Add null/False checks in memory_long_term_storage to prevent processing empty Redis results
- Add null/False checks in aggregate_judgment before format_parsing to avoid errors on missing data
- Initialize redis_messages variable in window_dialogue for consistency
- Add debug logging when no existing session found in Redis for better troubleshooting
- Add TODO comments for magic numbers (scope=6, time=5) to be extracted as constants
- Improve error handling when Redis returns False or empty results instead of crashing
* fix(web): PageScrollList style update
* fix(workflow): fix argument passing in code execution nodes
* fix(web): prompt add disabled
* fix(web): space icon required
* feat(app): modify the key of the token
* fix(fix the key of the app's token):
* fix(workflow): switch code input encoding to base64+URL encoding
* [add]The main project adds multi-API Key load balancing.
* [changes]Attribute security access, secure numerical conversion, unified use of local variables
* fix(web): save add session update
* fix(web): language editor support paste
* [changes]Active status filtering logic, API Key selection strategy
* memory_BUG
* memory_BUG_long_term
* [changes]
* memory_BUG_long_term
* memory_BUG_long_term
* Fix/release memory bug (#306)
* memory_BUG_fix
* memory_BUG
* memory_BUG_long_term
* memory_BUG_long_term
* memory_BUG_long_term
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* [fix]1.The "read_all_config" interface returns "scene_name";2.Memory configuration for lightweight query ontology scenarios
* fix(web): replace code editor
* [changes]Modify the description of the time for the recent event
* [changes]Modify the code based on the AI review
* feat(web): update memory config ontology api
* fix(web): ui update
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* feat(workflow): add token usage statistics for question classifier and parameter extraction
* feat(web): move prompt menu
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Write Missing None (#321)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/release memory bug (#324)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/writer memory bug (#326)
* [fix]Fix the bug
* [fix]Fix the bug
* [fix]Correct the direction indication.
* fix(web): markdown table ui update
* Fix/release memory bug (#332)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
* writer_dup_bug/fix
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/fact summary (#333)
* [fix]Disable the contents related to fact_summary
* [fix]Disable the contents related to fact_summary
* [fix]Modify the code based on the AI review
* Fix/release memory bug (#335)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
* writer_dup_bug/fix
* writer_graph_bug/fix
* writer_graph_bug/fix
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Revert "feat(web): move prompt menu"
This reverts commit 9e6e8f50f8.
* fix(web): ui update
* fix(web): update text
* fix(web): ui update
* fix(model): change the "vl" model type of dashscope to "chat"
* fix(model): change the "vl" model type of dashscope to "chat"
---------
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: Eternity <1533512157@qq.com>
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com>
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: lixinyue <2569494688@qq.com>
Co-authored-by: Eternity <61316157+myhMARS@users.noreply.github.com>
Co-authored-by: lanceyq <1982376970@qq.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
268 lines
10 KiB
Python
268 lines
10 KiB
Python
import asyncio
|
||
import time
|
||
import uuid
|
||
from uuid import UUID
|
||
|
||
from app.core.logging_config import get_api_logger
|
||
from app.core.memory.storage_services.reflection_engine.self_reflexion import (
|
||
ReflectionConfig,
|
||
ReflectionEngine, ReflectionRange, ReflectionBaseline,
|
||
)
|
||
from app.core.response_utils import success
|
||
from app.db import get_db
|
||
from app.dependencies import get_current_user
|
||
from app.models.user_model import User
|
||
from app.repositories.memory_config_repository import MemoryConfigRepository
|
||
from app.repositories.neo4j.neo4j_connector import Neo4jConnector
|
||
from app.schemas.memory_reflection_schemas import Memory_Reflection
|
||
from app.services.memory_reflection_service import (
|
||
MemoryReflectionService,
|
||
WorkspaceAppService,
|
||
)
|
||
from app.services.model_service import ModelConfigService
|
||
from dotenv import load_dotenv
|
||
from fastapi import APIRouter, Depends, HTTPException, status,Header
|
||
from sqlalchemy import text
|
||
from sqlalchemy.orm import Session
|
||
|
||
from app.utils.config_utils import resolve_config_id
|
||
|
||
load_dotenv()
|
||
api_logger = get_api_logger()
|
||
|
||
router = APIRouter(
|
||
prefix="/memory",
|
||
tags=["Memory"],
|
||
)
|
||
|
||
|
||
@router.post("/reflection/save")
|
||
async def save_reflection_config(
|
||
request: Memory_Reflection,
|
||
current_user: User = Depends(get_current_user),
|
||
db: Session = Depends(get_db),
|
||
) -> dict:
|
||
"""Save reflection configuration to data_comfig table"""
|
||
try:
|
||
config_id = request.config_id
|
||
config_id = resolve_config_id(config_id, db)
|
||
if not config_id:
|
||
raise HTTPException(
|
||
status_code=status.HTTP_400_BAD_REQUEST,
|
||
detail="缺少必需参数: config_id"
|
||
)
|
||
api_logger.info(f"用户 {current_user.username} 保存反思配置,config_id: {config_id}")
|
||
|
||
memory_config = MemoryConfigRepository.update_reflection_config(
|
||
db,
|
||
config_id=config_id,
|
||
enable_self_reflexion=request.reflection_enabled,
|
||
iteration_period=request.reflection_period_in_hours,
|
||
reflexion_range=request.reflexion_range,
|
||
baseline=request.baseline,
|
||
reflection_model_id=request.reflection_model_id,
|
||
memory_verify=request.memory_verify,
|
||
quality_assessment=request.quality_assessment
|
||
)
|
||
|
||
db.commit()
|
||
db.refresh(memory_config)
|
||
|
||
reflection_result={
|
||
"config_id": memory_config.config_id,
|
||
"enable_self_reflexion": memory_config.enable_self_reflexion,
|
||
"iteration_period": memory_config.iteration_period,
|
||
"reflexion_range": memory_config.reflexion_range,
|
||
"baseline": memory_config.baseline,
|
||
"reflection_model_id": memory_config.reflection_model_id,
|
||
"memory_verify": memory_config.memory_verify,
|
||
"quality_assessment": memory_config.quality_assessment}
|
||
|
||
return success(data=reflection_result, msg="反思配置成功")
|
||
|
||
|
||
|
||
except ValueError as ve:
|
||
api_logger.error(f"参数错误: {str(ve)}")
|
||
raise HTTPException(
|
||
status_code=status.HTTP_400_BAD_REQUEST,
|
||
detail=f"参数错误: {str(ve)}"
|
||
)
|
||
except Exception as e:
|
||
api_logger.error(f"反思配置保存失败: {str(e)}")
|
||
raise HTTPException(
|
||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||
detail=f"反思配置保存失败: {str(e)}"
|
||
)
|
||
|
||
|
||
@router.get("/reflection")
|
||
async def start_workspace_reflection(
|
||
current_user: User = Depends(get_current_user),
|
||
db: Session = Depends(get_db),
|
||
) -> dict:
|
||
"""启动工作空间中所有匹配应用的反思功能"""
|
||
workspace_id = current_user.current_workspace_id
|
||
reflection_service = MemoryReflectionService(db)
|
||
|
||
try:
|
||
api_logger.info(f"用户 {current_user.username} 启动workspace反思,workspace_id: {workspace_id}")
|
||
|
||
service = WorkspaceAppService(db)
|
||
result = service.get_workspace_apps_detailed(workspace_id)
|
||
reflection_results = []
|
||
for data in result['apps_detailed_info']:
|
||
# 跳过没有配置的应用
|
||
if not data['memory_configs']:
|
||
api_logger.debug(f"应用 {data['id']} 没有memory_configs,跳过")
|
||
continue
|
||
|
||
releases = data['releases']
|
||
memory_configs = data['memory_configs']
|
||
end_users = data['end_users']
|
||
|
||
# 为每个配置和用户组合执行反思
|
||
for config in memory_configs:
|
||
config_id_str = str(config['config_id'])
|
||
|
||
# 找到匹配此配置的所有release
|
||
matching_releases = [r for r in releases if str(r['config']) == config_id_str]
|
||
|
||
if not matching_releases:
|
||
api_logger.debug(f"配置 {config_id_str} 没有匹配的release")
|
||
continue
|
||
|
||
# 为每个用户执行反思
|
||
for user in end_users:
|
||
api_logger.info(f"为用户 {user['id']} 启动反思,config_id: {config_id_str}")
|
||
|
||
try:
|
||
reflection_result = await reflection_service.start_text_reflection(
|
||
config_data=config,
|
||
end_user_id=user['id']
|
||
)
|
||
|
||
reflection_results.append({
|
||
"app_id": data['id'],
|
||
"config_id": config_id_str,
|
||
"end_user_id": user['id'],
|
||
"reflection_result": reflection_result
|
||
})
|
||
except Exception as e:
|
||
api_logger.error(f"用户 {user['id']} 反思失败: {str(e)}")
|
||
reflection_results.append({
|
||
"app_id": data['id'],
|
||
"config_id": config_id_str,
|
||
"end_user_id": user['id'],
|
||
"reflection_result": {
|
||
"status": "错误",
|
||
"message": f"反思失败: {str(e)}"
|
||
}
|
||
})
|
||
|
||
return success(data=reflection_results, msg="反思配置成功")
|
||
|
||
except Exception as e:
|
||
api_logger.error(f"启动workspace反思失败: {str(e)}")
|
||
raise HTTPException(
|
||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||
detail=f"启动workspace反思失败: {str(e)}"
|
||
)
|
||
|
||
|
||
@router.get("/reflection/configs")
|
||
async def start_reflection_configs(
|
||
config_id: uuid.UUID|int,
|
||
current_user: User = Depends(get_current_user),
|
||
db: Session = Depends(get_db),
|
||
) -> dict:
|
||
"""通过config_id查询memory_config表中的反思配置信息"""
|
||
config_id = resolve_config_id(config_id, db)
|
||
try:
|
||
config_id=resolve_config_id(config_id,db)
|
||
api_logger.info(f"用户 {current_user.username} 查询反思配置,config_id: {config_id}")
|
||
result = MemoryConfigRepository.query_reflection_config_by_id(db, config_id)
|
||
memory_config_id = resolve_config_id(result.config_id, db)
|
||
# 构建返回数据
|
||
reflection_config = {
|
||
"config_id": memory_config_id,
|
||
"reflection_enabled": result.enable_self_reflexion,
|
||
"reflection_period_in_hours": result.iteration_period,
|
||
"reflexion_range": result.reflexion_range,
|
||
"baseline": result.baseline,
|
||
"reflection_model_id": result.reflection_model_id,
|
||
"memory_verify": result.memory_verify,
|
||
"quality_assessment": result.quality_assessment
|
||
}
|
||
api_logger.info(f"成功查询反思配置,config_id: {config_id}")
|
||
return success(data=reflection_config, msg="反思配置查询成功")
|
||
|
||
|
||
except HTTPException:
|
||
# 重新抛出HTTP异常
|
||
raise
|
||
except Exception as e:
|
||
api_logger.error(f"查询反思配置失败: {str(e)}")
|
||
raise HTTPException(
|
||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||
detail=f"查询反思配置失败: {str(e)}"
|
||
)
|
||
|
||
@router.get("/reflection/run")
|
||
async def reflection_run(
|
||
config_id: UUID|int,
|
||
language_type: str = Header(default="zh", alias="X-Language-Type"),
|
||
current_user: User = Depends(get_current_user),
|
||
db: Session = Depends(get_db),
|
||
) -> dict:
|
||
"""Activate the reflection function for all matching applications in the workspace"""
|
||
|
||
api_logger.info(f"用户 {current_user.username} 查询反思配置,config_id: {config_id}")
|
||
config_id = resolve_config_id(config_id, db)
|
||
# 使用MemoryConfigRepository查询反思配置
|
||
result = MemoryConfigRepository.query_reflection_config_by_id(db, config_id)
|
||
if not result:
|
||
raise HTTPException(
|
||
status_code=status.HTTP_404_NOT_FOUND,
|
||
detail=f"未找到config_id为 {config_id} 的配置"
|
||
)
|
||
|
||
api_logger.info(f"成功查询反思配置,config_id: {config_id}")
|
||
|
||
# 验证模型ID是否存在
|
||
model_id = result.reflection_model_id
|
||
if model_id:
|
||
try:
|
||
ModelConfigService.get_model_by_id(db=db, model_id=uuid.UUID(model_id))
|
||
api_logger.info(f"模型ID验证成功: {model_id}")
|
||
except Exception as e:
|
||
api_logger.warning(f"模型ID '{model_id}' 不存在,将使用默认模型: {str(e)}")
|
||
# 可以设置为None,让反思引擎使用默认模型
|
||
model_id = None
|
||
|
||
config = ReflectionConfig(
|
||
enabled=result.enable_self_reflexion,
|
||
iteration_period=result.iteration_period,
|
||
reflexion_range=ReflectionRange(result.reflexion_range),
|
||
baseline=ReflectionBaseline(result.baseline),
|
||
output_example='',
|
||
memory_verify=result.memory_verify,
|
||
quality_assessment=result.quality_assessment,
|
||
violation_handling_strategy="block",
|
||
model_id=model_id,
|
||
language_type=language_type
|
||
)
|
||
connector = Neo4jConnector()
|
||
engine = ReflectionEngine(
|
||
config=config,
|
||
neo4j_connector=connector,
|
||
llm_client=model_id # 传入验证后的 model_id
|
||
)
|
||
|
||
result=await (engine.reflection_run())
|
||
return success(data=result, msg="反思试运行")
|
||
|
||
|
||
|
||
|