* feat(web): add PageEmpty component
* feat(web): add PageTabs component
* feat(web): add PageEmpty component
* feat(web): add PageTabs component
* feat(prompt): add history tracking for prompt releases
* feat(web): add prompt menu
* refactor: The PageScrollList component supports two generic parameters
* feat(web): BodyWrapper compoent update PageLoading
* feat(web): add Ontology menu
* feat(web): memory management add scene
* feat(tasks): add celery task configuration for periodic jobs
- Add ignore_result=True to prevent storing results for periodic tasks
- Set max_retries=0 to skip failed periodic tasks without retry attempts
- Configure acks_late=False for immediate acknowledgment in beat tasks
- Add time_limit and soft_time_limit to regenerate_memory_cache task (3600s/3300s)
- Add time_limit and soft_time_limit to workspace_reflection_task (300s/240s)
- Add time_limit and soft_time_limit to run_forgetting_cycle_task (7200s/7000s)
- Improve task reliability and resource management for scheduled jobs
* feat(sandbox): add Node.js code execution support to sandbox
* Release/v0.2.2 (#260)
* [modify] migration script
* [add] migration script
* fix(web): change form message
* fix(web): the memoryContent field is compatible with numbers and strings
* feat(web): code node hidden
* fix(model):
1. create a basic model to check if the name and provider are duplicated.
2. The result shows error models because the provider created API Keys for all matching models.
---------
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
* Feature/ontology class clean (#249)
* [add] Complete ontology engineering feature implementation
* [add] Add ontology feature integration and validation utilities
* [add] Add OWL validator and validation utilities
* [fix] Add missing render_ontology_extraction_prompt function
* [fix]Add dependencies, fix functionality
* [add] migration script
* feat(celery): add dedicated periodic tasks worker and queue (#261)
* fix(web): conflict resolve
* Fix/v022 bug (#263)
* [fix]Fix the issue of inconsistent language in explicit and episodic memory.
* [fix]Fix the issue of inconsistent language in explicit and episodic memory.
* [add]Add scene_id
* [fix]Based on the AI review to fix the code
* Fix/develop memory reflex (#265)
* 遗漏的历史映射
* 遗漏的历史映射
* 反思后台报错处理
* [add] migration script
* fix: chat conversation_id add node_start
* feat(web): show code node
* fix(web): Restructure the CustomSelect component, repair the interface that is called multiple times when the form is updated
* feat(web): RadioGroupCard support block mode
* feat(web): create space add icon
* feat(app and model): token consumption statistics
* Add/develop memory (#264)
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 新增长期记忆功能
* 新增长期记忆功能
* 新增长期记忆功能
* 知识库检索多余字段
* 长期
* feat(app and model): token consumption statistics of the cluster
* memory_BUG_fix
* fix(web): prompt history remove pageLoading
* fix(prompt): remove hard-coded import of prompt file paths (#279)
* Fix/develop memory bug (#274)
* 遗漏的历史映射
* 遗漏的历史映射
* fix_timeline_memories
* fix(web): update retrieve_type key
* Fix/develop memory bug (#276)
* 遗漏的历史映射
* 遗漏的历史映射
* fix_timeline_memories
* fix_timeline_memories
* write_gragp/bug_fix
* write_gragp/bug_fix
* write_gragp/bug_fix
* chore(celery): disable periodic task scheduling
* fix(prompt): remove hard-coded import of prompt file paths
---------
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Ke Sun <kesun5@illinois.edu>
* fix(web): remove delete confirm content
* refactor(workflow): relocate template directory into workflow
* feat(memory): add long-term storage task routing and batching
* fix(web): PageScrollList loading update
* fix(web): PageScrollList loading update
* Ontology v1 bug (#291)
* [changes]Add 'id' as the secondary sorting key, and 'scene_id' now returns a UUID object
* [fix]Fix the "end_user" return to be sorted by update time.
* [fix]Set the default values of the memory configuration model based on the spatial model.
* [fix]Remove the entity extraction check combination model, read the configuration list, and add the return of scene_id
* [fix]Fix the "end_user" return to be sorted by update time.
* [fix]
* fix(memory): add Redis session validation
- Add macOS fork() safety configuration in celery_app.py to prevent initialization issues
- Add null/False checks for Redis session queries in term_memory_save to handle missing sessions gracefully
- Add null/False checks in memory_long_term_storage to prevent processing empty Redis results
- Add null/False checks in aggregate_judgment before format_parsing to avoid errors on missing data
- Initialize redis_messages variable in window_dialogue for consistency
- Add debug logging when no existing session found in Redis for better troubleshooting
- Add TODO comments for magic numbers (scope=6, time=5) to be extracted as constants
- Improve error handling when Redis returns False or empty results instead of crashing
* fix(web): PageScrollList style update
* fix(workflow): fix argument passing in code execution nodes
* fix(web): prompt add disabled
* fix(web): space icon required
* feat(app): modify the key of the token
* fix(fix the key of the app's token):
* fix(workflow): switch code input encoding to base64+URL encoding
* [add]The main project adds multi-API Key load balancing.
* [changes]Attribute security access, secure numerical conversion, unified use of local variables
* fix(web): save add session update
* fix(web): language editor support paste
* [changes]Active status filtering logic, API Key selection strategy
* memory_BUG
* memory_BUG_long_term
* [changes]
* memory_BUG_long_term
* memory_BUG_long_term
* Fix/release memory bug (#306)
* memory_BUG_fix
* memory_BUG
* memory_BUG_long_term
* memory_BUG_long_term
* memory_BUG_long_term
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* [fix]1.The "read_all_config" interface returns "scene_name";2.Memory configuration for lightweight query ontology scenarios
* fix(web): replace code editor
* [changes]Modify the description of the time for the recent event
* [changes]Modify the code based on the AI review
* feat(web): update memory config ontology api
* fix(web): ui update
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* feat(workflow): add token usage statistics for question classifier and parameter extraction
* feat(web): move prompt menu
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Write Missing None (#321)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/release memory bug (#324)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/writer memory bug (#326)
* [fix]Fix the bug
* [fix]Fix the bug
* [fix]Correct the direction indication.
* fix(web): markdown table ui update
* Fix/release memory bug (#332)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
* writer_dup_bug/fix
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/fact summary (#333)
* [fix]Disable the contents related to fact_summary
* [fix]Disable the contents related to fact_summary
* [fix]Modify the code based on the AI review
* Fix/release memory bug (#335)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
* writer_dup_bug/fix
* writer_graph_bug/fix
* writer_graph_bug/fix
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Revert "feat(web): move prompt menu"
This reverts commit 9e6e8f50f8.
* fix(web): ui update
* fix(web): update text
* fix(web): ui update
* fix(model): change the "vl" model type of dashscope to "chat"
* fix(model): change the "vl" model type of dashscope to "chat"
---------
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: Eternity <1533512157@qq.com>
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com>
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: lixinyue <2569494688@qq.com>
Co-authored-by: Eternity <61316157+myhMARS@users.noreply.github.com>
Co-authored-by: lanceyq <1982376970@qq.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
405 lines
12 KiB
Python
405 lines
12 KiB
Python
# -*- coding: utf-8 -*-
|
||
"""本体类型Repository层
|
||
|
||
本模块提供本体类型的数据访问层实现。
|
||
|
||
Classes:
|
||
OntologyClassRepository: 本体类型数据访问类
|
||
"""
|
||
|
||
import logging
|
||
from typing import List, Optional
|
||
from uuid import UUID
|
||
|
||
from sqlalchemy.orm import Session, joinedload
|
||
|
||
from app.core.logging_config import get_db_logger
|
||
from app.models.ontology_class import OntologyClass
|
||
from app.models.ontology_scene import OntologyScene
|
||
|
||
|
||
logger = get_db_logger()
|
||
|
||
|
||
class OntologyClassRepository:
|
||
"""本体类型Repository
|
||
|
||
提供本体类型的CRUD操作和权限检查。
|
||
|
||
Attributes:
|
||
db: SQLAlchemy数据库会话
|
||
"""
|
||
|
||
def __init__(self, db: Session):
|
||
"""初始化Repository
|
||
|
||
Args:
|
||
db: SQLAlchemy数据库会话
|
||
"""
|
||
self.db = db
|
||
|
||
def create(self, class_data: dict, scene_id: UUID) -> OntologyClass:
|
||
"""创建本体类型
|
||
|
||
Args:
|
||
class_data: 类型数据字典,包含class_name和class_description
|
||
scene_id: 所属场景ID
|
||
|
||
Returns:
|
||
OntologyClass: 创建的类型对象
|
||
|
||
Raises:
|
||
Exception: 数据库操作失败
|
||
|
||
Examples:
|
||
>>> repo = OntologyClassRepository(db)
|
||
>>> ontology_class = repo.create(
|
||
... {"class_name": "患者", "class_description": "描述"},
|
||
... scene_id
|
||
... )
|
||
"""
|
||
try:
|
||
logger.info(
|
||
f"Creating ontology class - "
|
||
f"name={class_data.get('class_name')}, "
|
||
f"scene_id={scene_id}"
|
||
)
|
||
|
||
ontology_class = OntologyClass(
|
||
class_name=class_data.get("class_name"),
|
||
class_description=class_data.get("class_description"),
|
||
scene_id=scene_id
|
||
)
|
||
|
||
self.db.add(ontology_class)
|
||
self.db.flush() # 获取ID但不提交
|
||
|
||
logger.info(
|
||
f"Ontology class created successfully - "
|
||
f"class_id={ontology_class.class_id}"
|
||
)
|
||
|
||
return ontology_class
|
||
|
||
except Exception as e:
|
||
logger.error(
|
||
f"Failed to create ontology class: {str(e)}",
|
||
exc_info=True
|
||
)
|
||
raise
|
||
|
||
def get_by_id(self, class_id: UUID) -> Optional[OntologyClass]:
|
||
"""根据ID获取类型
|
||
|
||
Args:
|
||
class_id: 类型ID
|
||
|
||
Returns:
|
||
Optional[OntologyClass]: 类型对象,不存在则返回None
|
||
|
||
Examples:
|
||
>>> repo = OntologyClassRepository(db)
|
||
>>> ontology_class = repo.get_by_id(class_id)
|
||
"""
|
||
try:
|
||
logger.debug(f"Getting ontology class by ID: {class_id}")
|
||
|
||
ontology_class = self.db.query(OntologyClass).filter(
|
||
OntologyClass.class_id == class_id
|
||
).first()
|
||
|
||
if ontology_class:
|
||
logger.debug(f"Ontology class found: {class_id}")
|
||
else:
|
||
logger.debug(f"Ontology class not found: {class_id}")
|
||
|
||
return ontology_class
|
||
|
||
except Exception as e:
|
||
logger.error(
|
||
f"Failed to get ontology class by ID: {str(e)}",
|
||
exc_info=True
|
||
)
|
||
raise
|
||
|
||
def get_by_name(self, class_name: str, scene_id: UUID) -> Optional[OntologyClass]:
|
||
"""根据类型名称和场景ID获取类型(精确匹配)
|
||
|
||
Args:
|
||
class_name: 类型名称
|
||
scene_id: 场景ID
|
||
|
||
Returns:
|
||
Optional[OntologyClass]: 类型对象,不存在则返回None
|
||
|
||
Examples:
|
||
>>> repo = OntologyClassRepository(db)
|
||
>>> ontology_class = repo.get_by_name("患者", scene_id)
|
||
"""
|
||
try:
|
||
logger.debug(f"Getting ontology class by name: {class_name}, scene_id: {scene_id}")
|
||
|
||
ontology_class = self.db.query(OntologyClass).filter(
|
||
OntologyClass.class_name == class_name,
|
||
OntologyClass.scene_id == scene_id
|
||
).first()
|
||
|
||
if ontology_class:
|
||
logger.debug(f"Ontology class found: {class_name}")
|
||
else:
|
||
logger.debug(f"Ontology class not found: {class_name}")
|
||
|
||
return ontology_class
|
||
|
||
except Exception as e:
|
||
logger.error(
|
||
f"Failed to get ontology class by name: {str(e)}",
|
||
exc_info=True
|
||
)
|
||
raise
|
||
|
||
def search_by_name(self, keyword: str, scene_id: UUID) -> List[OntologyClass]:
|
||
"""根据关键词模糊搜索类型
|
||
|
||
使用 LIKE 进行模糊匹配,支持中文和英文。
|
||
|
||
Args:
|
||
keyword: 搜索关键词
|
||
scene_id: 场景ID
|
||
|
||
Returns:
|
||
List[OntologyClass]: 匹配的类型列表
|
||
|
||
Examples:
|
||
>>> repo = OntologyClassRepository(db)
|
||
>>> classes = repo.search_by_name("患者", scene_id)
|
||
"""
|
||
try:
|
||
logger.debug(
|
||
f"Searching ontology classes by keyword - "
|
||
f"keyword={keyword}, scene_id={scene_id}"
|
||
)
|
||
|
||
# 使用 ilike 进行不区分大小写的模糊匹配
|
||
classes = self.db.query(OntologyClass).filter(
|
||
OntologyClass.class_name.ilike(f"%{keyword}%"),
|
||
OntologyClass.scene_id == scene_id
|
||
).order_by(
|
||
OntologyClass.created_at.desc()
|
||
).all()
|
||
|
||
logger.info(
|
||
f"Found {len(classes)} ontology classes matching keyword '{keyword}' "
|
||
f"in scene {scene_id}"
|
||
)
|
||
|
||
return classes
|
||
|
||
except Exception as e:
|
||
logger.error(
|
||
f"Failed to search ontology classes by keyword: {str(e)}",
|
||
exc_info=True
|
||
)
|
||
raise
|
||
|
||
def get_by_scene(self, scene_id: UUID) -> List[OntologyClass]:
|
||
"""获取场景下的所有类型
|
||
|
||
按创建时间倒序排列。
|
||
|
||
Args:
|
||
scene_id: 场景ID
|
||
|
||
Returns:
|
||
List[OntologyClass]: 类型列表
|
||
|
||
Examples:
|
||
>>> repo = OntologyClassRepository(db)
|
||
>>> classes = repo.get_by_scene(scene_id)
|
||
"""
|
||
try:
|
||
logger.debug(f"Getting ontology classes by scene: {scene_id}")
|
||
|
||
classes = self.db.query(OntologyClass).filter(
|
||
OntologyClass.scene_id == scene_id
|
||
).order_by(
|
||
OntologyClass.created_at.desc()
|
||
).all()
|
||
|
||
logger.info(
|
||
f"Found {len(classes)} ontology classes in scene {scene_id}"
|
||
)
|
||
|
||
return classes
|
||
|
||
except Exception as e:
|
||
logger.error(
|
||
f"Failed to get ontology classes by scene: {str(e)}",
|
||
exc_info=True
|
||
)
|
||
raise
|
||
|
||
def update(self, class_id: UUID, update_data: dict) -> Optional[OntologyClass]:
|
||
"""更新类型信息
|
||
|
||
Args:
|
||
class_id: 类型ID
|
||
update_data: 更新数据字典
|
||
|
||
Returns:
|
||
Optional[OntologyClass]: 更新后的类型对象,不存在则返回None
|
||
|
||
Raises:
|
||
Exception: 数据库操作失败
|
||
|
||
Examples:
|
||
>>> repo = OntologyClassRepository(db)
|
||
>>> ontology_class = repo.update(
|
||
... class_id,
|
||
... {"class_name": "新名称"}
|
||
... )
|
||
"""
|
||
try:
|
||
logger.info(f"Updating ontology class: {class_id}")
|
||
|
||
ontology_class = self.get_by_id(class_id)
|
||
if not ontology_class:
|
||
logger.warning(f"Ontology class not found for update: {class_id}")
|
||
return None
|
||
|
||
# 更新字段
|
||
if "class_name" in update_data and update_data["class_name"] is not None:
|
||
ontology_class.class_name = update_data["class_name"]
|
||
|
||
if "class_description" in update_data:
|
||
ontology_class.class_description = update_data["class_description"]
|
||
|
||
self.db.flush()
|
||
|
||
logger.info(f"Ontology class updated successfully: {class_id}")
|
||
|
||
return ontology_class
|
||
|
||
except Exception as e:
|
||
logger.error(
|
||
f"Failed to update ontology class: {str(e)}",
|
||
exc_info=True
|
||
)
|
||
raise
|
||
|
||
def delete(self, class_id: UUID) -> bool:
|
||
"""删除类型
|
||
|
||
Args:
|
||
class_id: 类型ID
|
||
|
||
Returns:
|
||
bool: 删除成功返回True,类型不存在返回False
|
||
|
||
Raises:
|
||
Exception: 数据库操作失败
|
||
|
||
Examples:
|
||
>>> repo = OntologyClassRepository(db)
|
||
>>> success = repo.delete(class_id)
|
||
"""
|
||
try:
|
||
logger.info(f"Deleting ontology class: {class_id}")
|
||
|
||
ontology_class = self.get_by_id(class_id)
|
||
if not ontology_class:
|
||
logger.warning(f"Ontology class not found for delete: {class_id}")
|
||
return False
|
||
|
||
self.db.delete(ontology_class)
|
||
self.db.flush()
|
||
|
||
logger.info(f"Ontology class deleted successfully: {class_id}")
|
||
|
||
return True
|
||
|
||
except Exception as e:
|
||
logger.error(
|
||
f"Failed to delete ontology class: {str(e)}",
|
||
exc_info=True
|
||
)
|
||
raise
|
||
|
||
def check_ownership(self, class_id: UUID, workspace_id: UUID) -> bool:
|
||
"""检查类型是否属于指定工作空间(通过场景关联)
|
||
|
||
Args:
|
||
class_id: 类型ID
|
||
workspace_id: 工作空间ID
|
||
|
||
Returns:
|
||
bool: 属于返回True,否则返回False
|
||
|
||
Examples:
|
||
>>> repo = OntologyClassRepository(db)
|
||
>>> is_owner = repo.check_ownership(class_id, workspace_id)
|
||
"""
|
||
try:
|
||
logger.debug(
|
||
f"Checking class ownership - "
|
||
f"class_id={class_id}, workspace_id={workspace_id}"
|
||
)
|
||
|
||
count = self.db.query(OntologyClass).join(
|
||
OntologyScene,
|
||
OntologyClass.scene_id == OntologyScene.scene_id
|
||
).filter(
|
||
OntologyClass.class_id == class_id,
|
||
OntologyScene.workspace_id == workspace_id
|
||
).count()
|
||
|
||
is_owner = count > 0
|
||
|
||
logger.debug(
|
||
f"Class ownership check result: {is_owner} - "
|
||
f"class_id={class_id}"
|
||
)
|
||
|
||
return is_owner
|
||
|
||
except Exception as e:
|
||
logger.error(
|
||
f"Failed to check class ownership: {str(e)}",
|
||
exc_info=True
|
||
)
|
||
raise
|
||
|
||
def get_scene_id_by_class(self, class_id: UUID) -> Optional[UUID]:
|
||
"""根据类型ID获取所属场景ID
|
||
|
||
Args:
|
||
class_id: 类型ID
|
||
|
||
Returns:
|
||
Optional[UUID]: 场景ID,类型不存在则返回None
|
||
|
||
Examples:
|
||
>>> repo = OntologyClassRepository(db)
|
||
>>> scene_id = repo.get_scene_id_by_class(class_id)
|
||
"""
|
||
try:
|
||
logger.debug(f"Getting scene ID by class: {class_id}")
|
||
|
||
ontology_class = self.get_by_id(class_id)
|
||
if not ontology_class:
|
||
logger.debug(f"Class not found: {class_id}")
|
||
return None
|
||
|
||
logger.debug(
|
||
f"Found scene ID: {ontology_class.scene_id} for class: {class_id}"
|
||
)
|
||
|
||
return ontology_class.scene_id
|
||
|
||
except Exception as e:
|
||
logger.error(
|
||
f"Failed to get scene ID by class: {str(e)}",
|
||
exc_info=True
|
||
)
|
||
raise
|