Release/v0.2.3 (#355)
* feat(web): add PageEmpty component
* feat(web): add PageTabs component
* feat(web): add PageEmpty component
* feat(web): add PageTabs component
* feat(prompt): add history tracking for prompt releases
* feat(web): add prompt menu
* refactor: The PageScrollList component supports two generic parameters
* feat(web): BodyWrapper compoent update PageLoading
* feat(web): add Ontology menu
* feat(web): memory management add scene
* feat(tasks): add celery task configuration for periodic jobs
- Add ignore_result=True to prevent storing results for periodic tasks
- Set max_retries=0 to skip failed periodic tasks without retry attempts
- Configure acks_late=False for immediate acknowledgment in beat tasks
- Add time_limit and soft_time_limit to regenerate_memory_cache task (3600s/3300s)
- Add time_limit and soft_time_limit to workspace_reflection_task (300s/240s)
- Add time_limit and soft_time_limit to run_forgetting_cycle_task (7200s/7000s)
- Improve task reliability and resource management for scheduled jobs
* feat(sandbox): add Node.js code execution support to sandbox
* Release/v0.2.2 (#260)
* [modify] migration script
* [add] migration script
* fix(web): change form message
* fix(web): the memoryContent field is compatible with numbers and strings
* feat(web): code node hidden
* fix(model):
1. create a basic model to check if the name and provider are duplicated.
2. The result shows error models because the provider created API Keys for all matching models.
---------
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
* Feature/ontology class clean (#249)
* [add] Complete ontology engineering feature implementation
* [add] Add ontology feature integration and validation utilities
* [add] Add OWL validator and validation utilities
* [fix] Add missing render_ontology_extraction_prompt function
* [fix]Add dependencies, fix functionality
* [add] migration script
* feat(celery): add dedicated periodic tasks worker and queue (#261)
* fix(web): conflict resolve
* Fix/v022 bug (#263)
* [fix]Fix the issue of inconsistent language in explicit and episodic memory.
* [fix]Fix the issue of inconsistent language in explicit and episodic memory.
* [add]Add scene_id
* [fix]Based on the AI review to fix the code
* Fix/develop memory reflex (#265)
* 遗漏的历史映射
* 遗漏的历史映射
* 反思后台报错处理
* [add] migration script
* fix: chat conversation_id add node_start
* feat(web): show code node
* fix(web): Restructure the CustomSelect component, repair the interface that is called multiple times when the form is updated
* feat(web): RadioGroupCard support block mode
* feat(web): create space add icon
* feat(app and model): token consumption statistics
* Add/develop memory (#264)
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 新增长期记忆功能
* 新增长期记忆功能
* 新增长期记忆功能
* 知识库检索多余字段
* 长期
* feat(app and model): token consumption statistics of the cluster
* memory_BUG_fix
* fix(web): prompt history remove pageLoading
* fix(prompt): remove hard-coded import of prompt file paths (#279)
* Fix/develop memory bug (#274)
* 遗漏的历史映射
* 遗漏的历史映射
* fix_timeline_memories
* fix(web): update retrieve_type key
* Fix/develop memory bug (#276)
* 遗漏的历史映射
* 遗漏的历史映射
* fix_timeline_memories
* fix_timeline_memories
* write_gragp/bug_fix
* write_gragp/bug_fix
* write_gragp/bug_fix
* chore(celery): disable periodic task scheduling
* fix(prompt): remove hard-coded import of prompt file paths
---------
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Ke Sun <kesun5@illinois.edu>
* fix(web): remove delete confirm content
* refactor(workflow): relocate template directory into workflow
* feat(memory): add long-term storage task routing and batching
* fix(web): PageScrollList loading update
* fix(web): PageScrollList loading update
* Ontology v1 bug (#291)
* [changes]Add 'id' as the secondary sorting key, and 'scene_id' now returns a UUID object
* [fix]Fix the "end_user" return to be sorted by update time.
* [fix]Set the default values of the memory configuration model based on the spatial model.
* [fix]Remove the entity extraction check combination model, read the configuration list, and add the return of scene_id
* [fix]Fix the "end_user" return to be sorted by update time.
* [fix]
* fix(memory): add Redis session validation
- Add macOS fork() safety configuration in celery_app.py to prevent initialization issues
- Add null/False checks for Redis session queries in term_memory_save to handle missing sessions gracefully
- Add null/False checks in memory_long_term_storage to prevent processing empty Redis results
- Add null/False checks in aggregate_judgment before format_parsing to avoid errors on missing data
- Initialize redis_messages variable in window_dialogue for consistency
- Add debug logging when no existing session found in Redis for better troubleshooting
- Add TODO comments for magic numbers (scope=6, time=5) to be extracted as constants
- Improve error handling when Redis returns False or empty results instead of crashing
* fix(web): PageScrollList style update
* fix(workflow): fix argument passing in code execution nodes
* fix(web): prompt add disabled
* fix(web): space icon required
* feat(app): modify the key of the token
* fix(fix the key of the app's token):
* fix(workflow): switch code input encoding to base64+URL encoding
* [add]The main project adds multi-API Key load balancing.
* [changes]Attribute security access, secure numerical conversion, unified use of local variables
* fix(web): save add session update
* fix(web): language editor support paste
* [changes]Active status filtering logic, API Key selection strategy
* memory_BUG
* memory_BUG_long_term
* [changes]
* memory_BUG_long_term
* memory_BUG_long_term
* Fix/release memory bug (#306)
* memory_BUG_fix
* memory_BUG
* memory_BUG_long_term
* memory_BUG_long_term
* memory_BUG_long_term
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* [fix]1.The "read_all_config" interface returns "scene_name";2.Memory configuration for lightweight query ontology scenarios
* fix(web): replace code editor
* [changes]Modify the description of the time for the recent event
* [changes]Modify the code based on the AI review
* feat(web): update memory config ontology api
* fix(web): ui update
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* feat(workflow): add token usage statistics for question classifier and parameter extraction
* feat(web): move prompt menu
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Write Missing None (#321)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/release memory bug (#324)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/writer memory bug (#326)
* [fix]Fix the bug
* [fix]Fix the bug
* [fix]Correct the direction indication.
* fix(web): markdown table ui update
* Fix/release memory bug (#332)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
* writer_dup_bug/fix
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/fact summary (#333)
* [fix]Disable the contents related to fact_summary
* [fix]Disable the contents related to fact_summary
* [fix]Modify the code based on the AI review
* Fix/release memory bug (#335)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
* writer_dup_bug/fix
* writer_graph_bug/fix
* writer_graph_bug/fix
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Revert "feat(web): move prompt menu"
This reverts commit 9e6e8f50f8.
* fix(web): ui update
* fix(web): update text
* fix(web): ui update
* fix(model): change the "vl" model type of dashscope to "chat"
* fix(model): change the "vl" model type of dashscope to "chat"
---------
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: Eternity <1533512157@qq.com>
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com>
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: lixinyue <2569494688@qq.com>
Co-authored-by: Eternity <61316157+myhMARS@users.noreply.github.com>
Co-authored-by: lanceyq <1982376970@qq.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
This commit is contained in:
@@ -86,7 +86,8 @@ class MemoryConfigRepository:
|
||||
n.description AS description,
|
||||
n.entity_type AS entity_type,
|
||||
n.name AS name,
|
||||
COALESCE(n.fact_summary, '') AS fact_summary,
|
||||
// TODO: fact_summary 功能暂时禁用,待后续开发完善后启用
|
||||
// COALESCE(n.fact_summary, '') AS fact_summary,
|
||||
n.end_user_id AS end_user_id,
|
||||
n.apply_id AS apply_id,
|
||||
n.user_id AS user_id,
|
||||
@@ -156,7 +157,7 @@ class MemoryConfigRepository:
|
||||
return memory_config_obj
|
||||
|
||||
@staticmethod
|
||||
def query_reflection_config_by_id(db: Session, config_id: uuid.UUID) -> MemoryConfig:
|
||||
def query_reflection_config_by_id(db: Session, config_id: uuid.UUID|int|str) -> MemoryConfig:
|
||||
"""构建反思配置查询语句,通过config_id查询反思配置(SQLAlchemy text() 命名参数)
|
||||
|
||||
Args:
|
||||
@@ -230,9 +231,12 @@ class MemoryConfigRepository:
|
||||
config_name=params.config_name,
|
||||
config_desc=params.config_desc,
|
||||
workspace_id=params.workspace_id,
|
||||
scene_id=params.scene_id,
|
||||
llm_id=params.llm_id,
|
||||
embedding_id=params.embedding_id,
|
||||
rerank_id=params.rerank_id,
|
||||
reflection_model_id=params.reflection_model_id,
|
||||
emotion_model_id=params.emotion_model_id,
|
||||
)
|
||||
db.add(db_config)
|
||||
db.flush() # 获取自增ID但不提交事务
|
||||
@@ -275,6 +279,9 @@ class MemoryConfigRepository:
|
||||
if update.config_desc is not None:
|
||||
db_config.config_desc = update.config_desc
|
||||
has_update = True
|
||||
if update.scene_id is not None:
|
||||
db_config.scene_id = update.scene_id
|
||||
has_update = True
|
||||
|
||||
if not has_update:
|
||||
raise ValueError("No fields to update")
|
||||
@@ -643,28 +650,32 @@ class MemoryConfigRepository:
|
||||
raise
|
||||
|
||||
@staticmethod
|
||||
def get_all(db: Session, workspace_id: Optional[uuid.UUID] = None) -> List[MemoryConfig]:
|
||||
"""获取所有配置参数
|
||||
def get_all(db: Session, workspace_id: Optional[uuid.UUID] = None) -> List[Tuple[MemoryConfig, Optional[str]]]:
|
||||
"""获取所有配置参数,包含关联的场景名称
|
||||
|
||||
Args:
|
||||
db: 数据库会话
|
||||
workspace_id: 工作空间ID,用于过滤查询结果
|
||||
|
||||
Returns:
|
||||
List[MemoryConfig]: 配置列表
|
||||
List[Tuple[MemoryConfig, Optional[str]]]: 配置列表,每项为 (配置对象, 场景名称)
|
||||
"""
|
||||
from app.models.ontology_scene import OntologyScene
|
||||
|
||||
db_logger.debug(f"查询所有配置: workspace_id={workspace_id}")
|
||||
|
||||
try:
|
||||
query = db.query(MemoryConfig)
|
||||
query = db.query(MemoryConfig, OntologyScene.scene_name).outerjoin(
|
||||
OntologyScene, MemoryConfig.scene_id == OntologyScene.scene_id
|
||||
)
|
||||
|
||||
if workspace_id:
|
||||
query = query.filter(MemoryConfig.workspace_id == workspace_id)
|
||||
|
||||
configs = query.order_by(desc(MemoryConfig.updated_at)).all()
|
||||
results = query.order_by(desc(MemoryConfig.updated_at)).all()
|
||||
|
||||
db_logger.debug(f"配置列表查询成功: 数量={len(configs)}")
|
||||
return configs
|
||||
db_logger.debug(f"配置列表查询成功: 数量={len(results)}")
|
||||
return results
|
||||
|
||||
except Exception as e:
|
||||
db_logger.error(f"查询所有配置失败: workspace_id={workspace_id} - {str(e)}")
|
||||
|
||||
@@ -79,7 +79,8 @@ async def add_memory_summary_statement_edges(summaries: List[MemorySummaryNode],
|
||||
try:
|
||||
edges: List[dict] = []
|
||||
for s in summaries:
|
||||
for chunk_id in getattr(s, "chunk_ids", []) or []:
|
||||
chunk_ids = getattr(s, "chunk_ids", []) or []
|
||||
for chunk_id in chunk_ids:
|
||||
edges.append({
|
||||
"summary_id": s.id,
|
||||
"chunk_id": chunk_id,
|
||||
@@ -91,12 +92,11 @@ async def add_memory_summary_statement_edges(summaries: List[MemorySummaryNode],
|
||||
|
||||
if not edges:
|
||||
return []
|
||||
|
||||
result = await connector.execute_query(
|
||||
MEMORY_SUMMARY_STATEMENT_EDGE_SAVE,
|
||||
edges=edges
|
||||
)
|
||||
created = [record.get("uuid") for record in result] if result else []
|
||||
return created
|
||||
except Exception:
|
||||
except Exception as e:
|
||||
return None
|
||||
|
||||
@@ -217,8 +217,10 @@ async def add_memory_summary_nodes(summaries: List[MemorySummaryNode], connector
|
||||
summaries=flattened
|
||||
)
|
||||
created_ids = [record.get("uuid") for record in result]
|
||||
print(f"Successfully saved {len(created_ids)} MemorySummary nodes to Neo4j")
|
||||
return created_ids
|
||||
except Exception:
|
||||
except Exception as e:
|
||||
print(f"Failed to save MemorySummary nodes to Neo4j: {e}")
|
||||
return None
|
||||
|
||||
|
||||
|
||||
@@ -101,10 +101,11 @@ SET e.name = CASE WHEN entity.name IS NOT NULL AND entity.name <> '' THEN entity
|
||||
e.name_embedding = CASE
|
||||
WHEN entity.name_embedding IS NOT NULL AND size(entity.name_embedding) > 0 THEN entity.name_embedding
|
||||
ELSE e.name_embedding END,
|
||||
e.fact_summary = CASE
|
||||
WHEN entity.fact_summary IS NOT NULL AND entity.fact_summary <> ''
|
||||
AND (e.fact_summary IS NULL OR size(e.fact_summary) = 0 OR size(entity.fact_summary) > size(e.fact_summary))
|
||||
THEN entity.fact_summary ELSE e.fact_summary END,
|
||||
// TODO: fact_summary 功能暂时禁用,待后续开发完善后启用
|
||||
// e.fact_summary = CASE
|
||||
// WHEN entity.fact_summary IS NOT NULL AND entity.fact_summary <> ''
|
||||
// AND (e.fact_summary IS NULL OR size(e.fact_summary) = 0 OR size(entity.fact_summary) > size(e.fact_summary))
|
||||
// THEN entity.fact_summary ELSE e.fact_summary END,
|
||||
e.connect_strength = CASE
|
||||
WHEN entity.connect_strength IS NULL OR entity.connect_strength = '' THEN e.connect_strength
|
||||
ELSE CASE
|
||||
@@ -321,7 +322,8 @@ RETURN e.id AS id,
|
||||
e.description AS description,
|
||||
e.aliases AS aliases,
|
||||
e.name_embedding AS name_embedding,
|
||||
COALESCE(e.fact_summary, '') AS fact_summary,
|
||||
// TODO: fact_summary 功能暂时禁用,待后续开发完善后启用
|
||||
// COALESCE(e.fact_summary, '') AS fact_summary,
|
||||
e.connect_strength AS connect_strength,
|
||||
collect(DISTINCT s.id) AS statement_ids,
|
||||
collect(DISTINCT c.id) AS chunk_ids,
|
||||
@@ -877,7 +879,8 @@ RETURN
|
||||
CASE
|
||||
WHEN ms:ExtractedEntity THEN {
|
||||
text: ms.name,
|
||||
created_at: ms.created_at
|
||||
created_at: ms.created_at,
|
||||
type: "情景记忆"
|
||||
}
|
||||
END
|
||||
) AS ExtractedEntity,
|
||||
@@ -887,7 +890,8 @@ RETURN
|
||||
CASE
|
||||
WHEN n:MemorySummary THEN {
|
||||
text: n.content,
|
||||
created_at: n.created_at
|
||||
created_at: n.created_at,
|
||||
type: "长期沉淀"
|
||||
}
|
||||
END
|
||||
) AS MemorySummary,
|
||||
@@ -895,7 +899,8 @@ RETURN
|
||||
collect(
|
||||
DISTINCT {
|
||||
text: e.statement,
|
||||
created_at: e.created_at
|
||||
created_at: e.created_at,
|
||||
type: "情绪记忆"
|
||||
}
|
||||
) AS statement;
|
||||
"""
|
||||
@@ -999,3 +1004,58 @@ RETURN DISTINCT
|
||||
x.statement as statement,x.created_at as created_at
|
||||
"""
|
||||
|
||||
Graph_Node_query = """
|
||||
MATCH (n:MemorySummary)
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
RETURN
|
||||
elementId(n) AS id,
|
||||
labels(n) AS labels,
|
||||
properties(n) AS properties,
|
||||
0 AS priority
|
||||
LIMIT $limit
|
||||
|
||||
UNION ALL
|
||||
|
||||
MATCH (n:Dialogue)
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
RETURN
|
||||
elementId(n) AS id,
|
||||
labels(n) AS labels,
|
||||
properties(n) AS properties,
|
||||
1 AS priority
|
||||
LIMIT 1
|
||||
|
||||
UNION ALL
|
||||
|
||||
MATCH (n:Statement)
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
RETURN
|
||||
elementId(n) AS id,
|
||||
labels(n) AS labels,
|
||||
properties(n) AS properties,
|
||||
1 AS priority
|
||||
LIMIT $limit
|
||||
|
||||
UNION ALL
|
||||
|
||||
MATCH (n:ExtractedEntity)
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
RETURN
|
||||
elementId(n) AS id,
|
||||
labels(n) AS labels,
|
||||
properties(n) AS properties,
|
||||
2 AS priority
|
||||
LIMIT $limit
|
||||
|
||||
UNION ALL
|
||||
|
||||
MATCH (n:Chunk)
|
||||
WHERE n.end_user_id = $end_user_id
|
||||
RETURN
|
||||
elementId(n) AS id,
|
||||
labels(n) AS labels,
|
||||
properties(n) AS properties,
|
||||
3 AS priority
|
||||
LIMIT $limit
|
||||
|
||||
"""
|
||||
@@ -21,7 +21,8 @@ from app.core.memory.models.graph_models import (
|
||||
ExtractedEntityNode,
|
||||
EntityEntityEdge,
|
||||
)
|
||||
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
async def save_entities_and_relationships(
|
||||
entity_nodes: List[ExtractedEntityNode],
|
||||
entity_entity_edges: List[EntityEntityEdge],
|
||||
@@ -41,8 +42,8 @@ async def save_entities_and_relationships(
|
||||
'statement': edge.statement,
|
||||
'valid_at': edge.valid_at.isoformat() if edge.valid_at else None,
|
||||
'invalid_at': edge.invalid_at.isoformat() if edge.invalid_at else None,
|
||||
'created_at': edge.created_at.isoformat(),
|
||||
'expired_at': edge.expired_at.isoformat(),
|
||||
'created_at': edge.created_at.isoformat() if edge.created_at else None,
|
||||
'expired_at': edge.expired_at.isoformat() if edge.expired_at else None,
|
||||
'run_id': edge.run_id,
|
||||
'end_user_id': edge.end_user_id,
|
||||
}
|
||||
@@ -147,14 +148,14 @@ async def save_statement_entity_edges(
|
||||
|
||||
|
||||
async def save_dialog_and_statements_to_neo4j(
|
||||
dialogue_nodes: List[DialogueNode],
|
||||
chunk_nodes: List[ChunkNode],
|
||||
statement_nodes: List[StatementNode],
|
||||
entity_nodes: List[ExtractedEntityNode],
|
||||
entity_edges: List[EntityEntityEdge],
|
||||
statement_chunk_edges: List[StatementChunkEdge],
|
||||
statement_entity_edges: List[StatementEntityEdge],
|
||||
connector: Neo4jConnector
|
||||
dialogue_nodes: List[DialogueNode],
|
||||
chunk_nodes: List[ChunkNode],
|
||||
statement_nodes: List[StatementNode],
|
||||
entity_nodes: List[ExtractedEntityNode],
|
||||
entity_edges: List[EntityEntityEdge],
|
||||
statement_chunk_edges: List[StatementChunkEdge],
|
||||
statement_entity_edges: List[StatementEntityEdge],
|
||||
connector: Neo4jConnector
|
||||
) -> bool:
|
||||
"""Save dialogue nodes, chunk nodes, statement nodes, entities, and all relationships to Neo4j using graph models.
|
||||
|
||||
@@ -171,40 +172,127 @@ async def save_dialog_and_statements_to_neo4j(
|
||||
Returns:
|
||||
bool: True if successful, False otherwise
|
||||
"""
|
||||
try:
|
||||
# Save all dialogue nodes in batch
|
||||
dialogue_uuids = await add_dialogue_nodes(dialogue_nodes, connector)
|
||||
if dialogue_uuids:
|
||||
|
||||
# 定义事务函数,将所有写操作放在一个事务中
|
||||
async def _save_all_in_transaction(tx):
|
||||
"""在单个事务中执行所有保存操作,避免死锁"""
|
||||
results = {}
|
||||
|
||||
# 1. Save all dialogue nodes in batch
|
||||
if dialogue_nodes:
|
||||
from app.repositories.neo4j.cypher_queries import DIALOGUE_NODE_SAVE
|
||||
dialogue_data = [node.model_dump() for node in dialogue_nodes]
|
||||
result = await tx.run(DIALOGUE_NODE_SAVE, dialogues=dialogue_data)
|
||||
dialogue_uuids = [record["uuid"] async for record in result]
|
||||
results['dialogues'] = dialogue_uuids
|
||||
print(f"Dialogues saved to Neo4j with UUIDs: {dialogue_uuids}")
|
||||
else:
|
||||
print("Failed to save dialogues to Neo4j")
|
||||
return False
|
||||
|
||||
# Save all chunk nodes in batch
|
||||
await save_chunk_nodes(chunk_nodes, connector)
|
||||
# 2. Save all chunk nodes in batch
|
||||
if chunk_nodes:
|
||||
from app.repositories.neo4j.cypher_queries import CHUNK_NODE_SAVE
|
||||
chunk_data = [node.model_dump() for node in chunk_nodes]
|
||||
result = await tx.run(CHUNK_NODE_SAVE, chunks=chunk_data)
|
||||
chunk_uuids = [record["uuid"] async for record in result]
|
||||
results['chunks'] = chunk_uuids
|
||||
logger.info(f"Successfully saved {len(chunk_uuids)} chunk nodes to Neo4j")
|
||||
|
||||
# Save all statement nodes in batch
|
||||
# 3. Save all statement nodes in batch
|
||||
if statement_nodes:
|
||||
statement_uuids = await add_statement_nodes(statement_nodes, connector)
|
||||
if statement_uuids:
|
||||
print(f"Successfully saved {len(statement_uuids)} statement nodes to Neo4j")
|
||||
else:
|
||||
print("Failed to save statement nodes to Neo4j")
|
||||
return False
|
||||
else:
|
||||
print("No statement nodes to save")
|
||||
from app.repositories.neo4j.cypher_queries import STATEMENT_NODE_SAVE
|
||||
statement_data = [node.model_dump() for node in statement_nodes]
|
||||
result = await tx.run(STATEMENT_NODE_SAVE, statements=statement_data)
|
||||
statement_uuids = [record["uuid"] async for record in result]
|
||||
results['statements'] = statement_uuids
|
||||
logger.info(f"Successfully saved {len(statement_uuids)} statement nodes to Neo4j")
|
||||
|
||||
# Save entities and relationships
|
||||
await save_entities_and_relationships(entity_nodes, entity_edges, connector)
|
||||
print("Successfully saved entities and relationships to Neo4j")
|
||||
# 4. Save entities
|
||||
if entity_nodes:
|
||||
from app.repositories.neo4j.cypher_queries import EXTRACTED_ENTITY_NODE_SAVE
|
||||
entity_data = [entity.model_dump() for entity in entity_nodes]
|
||||
result = await tx.run(EXTRACTED_ENTITY_NODE_SAVE, entities=entity_data)
|
||||
entity_uuids = [record["uuid"] async for record in result]
|
||||
results['entities'] = entity_uuids
|
||||
logger.info(f"Successfully saved {len(entity_uuids)} entity nodes to Neo4j")
|
||||
|
||||
# Save new edges
|
||||
await save_statement_chunk_edges(statement_chunk_edges, connector)
|
||||
await save_statement_entity_edges(statement_entity_edges, connector)
|
||||
# 5. Create entity relationships
|
||||
if entity_edges:
|
||||
from app.repositories.neo4j.cypher_queries import ENTITY_RELATIONSHIP_SAVE
|
||||
relationship_data = []
|
||||
for edge in entity_edges:
|
||||
relationship_data.append({
|
||||
'source_id': edge.source,
|
||||
'target_id': edge.target,
|
||||
'predicate': edge.relation_type,
|
||||
'statement_id': edge.source_statement_id,
|
||||
'value': edge.relation_value,
|
||||
'statement': edge.statement,
|
||||
'valid_at': edge.valid_at.isoformat() if edge.valid_at else None,
|
||||
'invalid_at': edge.invalid_at.isoformat() if edge.invalid_at else None,
|
||||
'created_at': edge.created_at.isoformat() if edge.created_at else None,
|
||||
'expired_at': edge.expired_at.isoformat() if edge.expired_at else None,
|
||||
'run_id': edge.run_id,
|
||||
'end_user_id': edge.end_user_id,
|
||||
})
|
||||
result = await tx.run(ENTITY_RELATIONSHIP_SAVE, relationships=relationship_data)
|
||||
rel_uuids = [record["uuid"] async for record in result]
|
||||
results['entity_relationships'] = rel_uuids
|
||||
logger.info(f"Successfully saved {len(rel_uuids)} entity relationships to Neo4j")
|
||||
|
||||
# 6. Save statement-chunk edges
|
||||
if statement_chunk_edges:
|
||||
from app.repositories.neo4j.cypher_queries import CHUNK_STATEMENT_EDGE_SAVE
|
||||
sc_edge_data = []
|
||||
for edge in statement_chunk_edges:
|
||||
sc_edge_data.append({
|
||||
"id": edge.id,
|
||||
"source": edge.source,
|
||||
"target": edge.target,
|
||||
"created_at": edge.created_at.isoformat() if edge.created_at else None,
|
||||
"expired_at": edge.expired_at.isoformat() if edge.expired_at else None,
|
||||
"run_id": edge.run_id,
|
||||
"end_user_id": edge.end_user_id,
|
||||
})
|
||||
result = await tx.run(CHUNK_STATEMENT_EDGE_SAVE, chunk_statement_edges=sc_edge_data)
|
||||
sc_uuids = [record["uuid"] async for record in result]
|
||||
results['statement_chunk_edges'] = sc_uuids
|
||||
logger.info(f"Successfully saved {len(sc_uuids)} statement-chunk edges to Neo4j")
|
||||
|
||||
# 7. Save statement-entity edges
|
||||
if statement_entity_edges:
|
||||
from app.repositories.neo4j.cypher_queries import STATEMENT_ENTITY_EDGE_SAVE
|
||||
se_edge_data = []
|
||||
for edge in statement_entity_edges:
|
||||
se_edge_data.append({
|
||||
"source": edge.source,
|
||||
"target": edge.target,
|
||||
"created_at": edge.created_at.isoformat() if edge.created_at else None,
|
||||
"expired_at": edge.expired_at.isoformat() if edge.expired_at else None,
|
||||
"run_id": edge.run_id,
|
||||
"end_user_id": edge.end_user_id,
|
||||
"connect_strength": getattr(edge, "connect_strength", "strong"),
|
||||
})
|
||||
result = await tx.run(STATEMENT_ENTITY_EDGE_SAVE, relationships=se_edge_data)
|
||||
se_uuids = [record["uuid"] async for record in result]
|
||||
results['statement_entity_edges'] = se_uuids
|
||||
logger.info(f"Successfully saved {len(se_uuids)} statement-entity edges to Neo4j")
|
||||
|
||||
return results
|
||||
|
||||
try:
|
||||
# 使用显式写事务执行所有操作,避免死锁
|
||||
results = await connector.execute_write_transaction(_save_all_in_transaction)
|
||||
summary = {
|
||||
key: len(value)
|
||||
for key, value in results.items()
|
||||
if isinstance(value, (list, tuple, set))
|
||||
}
|
||||
logger.info("Transaction completed. Summary: %s", summary)
|
||||
logger.debug("Full transaction results: %r", results)
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Neo4j integration error: {e}", exc_info=True)
|
||||
print(f"Neo4j integration error: {e}")
|
||||
print("Continuing without database storage...")
|
||||
return False
|
||||
|
||||
|
||||
404
api/app/repositories/ontology_class_repository.py
Normal file
404
api/app/repositories/ontology_class_repository.py
Normal file
@@ -0,0 +1,404 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""本体类型Repository层
|
||||
|
||||
本模块提供本体类型的数据访问层实现。
|
||||
|
||||
Classes:
|
||||
OntologyClassRepository: 本体类型数据访问类
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import List, Optional
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy.orm import Session, joinedload
|
||||
|
||||
from app.core.logging_config import get_db_logger
|
||||
from app.models.ontology_class import OntologyClass
|
||||
from app.models.ontology_scene import OntologyScene
|
||||
|
||||
|
||||
logger = get_db_logger()
|
||||
|
||||
|
||||
class OntologyClassRepository:
|
||||
"""本体类型Repository
|
||||
|
||||
提供本体类型的CRUD操作和权限检查。
|
||||
|
||||
Attributes:
|
||||
db: SQLAlchemy数据库会话
|
||||
"""
|
||||
|
||||
def __init__(self, db: Session):
|
||||
"""初始化Repository
|
||||
|
||||
Args:
|
||||
db: SQLAlchemy数据库会话
|
||||
"""
|
||||
self.db = db
|
||||
|
||||
def create(self, class_data: dict, scene_id: UUID) -> OntologyClass:
|
||||
"""创建本体类型
|
||||
|
||||
Args:
|
||||
class_data: 类型数据字典,包含class_name和class_description
|
||||
scene_id: 所属场景ID
|
||||
|
||||
Returns:
|
||||
OntologyClass: 创建的类型对象
|
||||
|
||||
Raises:
|
||||
Exception: 数据库操作失败
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologyClassRepository(db)
|
||||
>>> ontology_class = repo.create(
|
||||
... {"class_name": "患者", "class_description": "描述"},
|
||||
... scene_id
|
||||
... )
|
||||
"""
|
||||
try:
|
||||
logger.info(
|
||||
f"Creating ontology class - "
|
||||
f"name={class_data.get('class_name')}, "
|
||||
f"scene_id={scene_id}"
|
||||
)
|
||||
|
||||
ontology_class = OntologyClass(
|
||||
class_name=class_data.get("class_name"),
|
||||
class_description=class_data.get("class_description"),
|
||||
scene_id=scene_id
|
||||
)
|
||||
|
||||
self.db.add(ontology_class)
|
||||
self.db.flush() # 获取ID但不提交
|
||||
|
||||
logger.info(
|
||||
f"Ontology class created successfully - "
|
||||
f"class_id={ontology_class.class_id}"
|
||||
)
|
||||
|
||||
return ontology_class
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to create ontology class: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def get_by_id(self, class_id: UUID) -> Optional[OntologyClass]:
|
||||
"""根据ID获取类型
|
||||
|
||||
Args:
|
||||
class_id: 类型ID
|
||||
|
||||
Returns:
|
||||
Optional[OntologyClass]: 类型对象,不存在则返回None
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologyClassRepository(db)
|
||||
>>> ontology_class = repo.get_by_id(class_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(f"Getting ontology class by ID: {class_id}")
|
||||
|
||||
ontology_class = self.db.query(OntologyClass).filter(
|
||||
OntologyClass.class_id == class_id
|
||||
).first()
|
||||
|
||||
if ontology_class:
|
||||
logger.debug(f"Ontology class found: {class_id}")
|
||||
else:
|
||||
logger.debug(f"Ontology class not found: {class_id}")
|
||||
|
||||
return ontology_class
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to get ontology class by ID: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def get_by_name(self, class_name: str, scene_id: UUID) -> Optional[OntologyClass]:
|
||||
"""根据类型名称和场景ID获取类型(精确匹配)
|
||||
|
||||
Args:
|
||||
class_name: 类型名称
|
||||
scene_id: 场景ID
|
||||
|
||||
Returns:
|
||||
Optional[OntologyClass]: 类型对象,不存在则返回None
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologyClassRepository(db)
|
||||
>>> ontology_class = repo.get_by_name("患者", scene_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(f"Getting ontology class by name: {class_name}, scene_id: {scene_id}")
|
||||
|
||||
ontology_class = self.db.query(OntologyClass).filter(
|
||||
OntologyClass.class_name == class_name,
|
||||
OntologyClass.scene_id == scene_id
|
||||
).first()
|
||||
|
||||
if ontology_class:
|
||||
logger.debug(f"Ontology class found: {class_name}")
|
||||
else:
|
||||
logger.debug(f"Ontology class not found: {class_name}")
|
||||
|
||||
return ontology_class
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to get ontology class by name: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def search_by_name(self, keyword: str, scene_id: UUID) -> List[OntologyClass]:
|
||||
"""根据关键词模糊搜索类型
|
||||
|
||||
使用 LIKE 进行模糊匹配,支持中文和英文。
|
||||
|
||||
Args:
|
||||
keyword: 搜索关键词
|
||||
scene_id: 场景ID
|
||||
|
||||
Returns:
|
||||
List[OntologyClass]: 匹配的类型列表
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologyClassRepository(db)
|
||||
>>> classes = repo.search_by_name("患者", scene_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(
|
||||
f"Searching ontology classes by keyword - "
|
||||
f"keyword={keyword}, scene_id={scene_id}"
|
||||
)
|
||||
|
||||
# 使用 ilike 进行不区分大小写的模糊匹配
|
||||
classes = self.db.query(OntologyClass).filter(
|
||||
OntologyClass.class_name.ilike(f"%{keyword}%"),
|
||||
OntologyClass.scene_id == scene_id
|
||||
).order_by(
|
||||
OntologyClass.created_at.desc()
|
||||
).all()
|
||||
|
||||
logger.info(
|
||||
f"Found {len(classes)} ontology classes matching keyword '{keyword}' "
|
||||
f"in scene {scene_id}"
|
||||
)
|
||||
|
||||
return classes
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to search ontology classes by keyword: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def get_by_scene(self, scene_id: UUID) -> List[OntologyClass]:
|
||||
"""获取场景下的所有类型
|
||||
|
||||
按创建时间倒序排列。
|
||||
|
||||
Args:
|
||||
scene_id: 场景ID
|
||||
|
||||
Returns:
|
||||
List[OntologyClass]: 类型列表
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologyClassRepository(db)
|
||||
>>> classes = repo.get_by_scene(scene_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(f"Getting ontology classes by scene: {scene_id}")
|
||||
|
||||
classes = self.db.query(OntologyClass).filter(
|
||||
OntologyClass.scene_id == scene_id
|
||||
).order_by(
|
||||
OntologyClass.created_at.desc()
|
||||
).all()
|
||||
|
||||
logger.info(
|
||||
f"Found {len(classes)} ontology classes in scene {scene_id}"
|
||||
)
|
||||
|
||||
return classes
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to get ontology classes by scene: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def update(self, class_id: UUID, update_data: dict) -> Optional[OntologyClass]:
|
||||
"""更新类型信息
|
||||
|
||||
Args:
|
||||
class_id: 类型ID
|
||||
update_data: 更新数据字典
|
||||
|
||||
Returns:
|
||||
Optional[OntologyClass]: 更新后的类型对象,不存在则返回None
|
||||
|
||||
Raises:
|
||||
Exception: 数据库操作失败
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologyClassRepository(db)
|
||||
>>> ontology_class = repo.update(
|
||||
... class_id,
|
||||
... {"class_name": "新名称"}
|
||||
... )
|
||||
"""
|
||||
try:
|
||||
logger.info(f"Updating ontology class: {class_id}")
|
||||
|
||||
ontology_class = self.get_by_id(class_id)
|
||||
if not ontology_class:
|
||||
logger.warning(f"Ontology class not found for update: {class_id}")
|
||||
return None
|
||||
|
||||
# 更新字段
|
||||
if "class_name" in update_data and update_data["class_name"] is not None:
|
||||
ontology_class.class_name = update_data["class_name"]
|
||||
|
||||
if "class_description" in update_data:
|
||||
ontology_class.class_description = update_data["class_description"]
|
||||
|
||||
self.db.flush()
|
||||
|
||||
logger.info(f"Ontology class updated successfully: {class_id}")
|
||||
|
||||
return ontology_class
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to update ontology class: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def delete(self, class_id: UUID) -> bool:
|
||||
"""删除类型
|
||||
|
||||
Args:
|
||||
class_id: 类型ID
|
||||
|
||||
Returns:
|
||||
bool: 删除成功返回True,类型不存在返回False
|
||||
|
||||
Raises:
|
||||
Exception: 数据库操作失败
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologyClassRepository(db)
|
||||
>>> success = repo.delete(class_id)
|
||||
"""
|
||||
try:
|
||||
logger.info(f"Deleting ontology class: {class_id}")
|
||||
|
||||
ontology_class = self.get_by_id(class_id)
|
||||
if not ontology_class:
|
||||
logger.warning(f"Ontology class not found for delete: {class_id}")
|
||||
return False
|
||||
|
||||
self.db.delete(ontology_class)
|
||||
self.db.flush()
|
||||
|
||||
logger.info(f"Ontology class deleted successfully: {class_id}")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to delete ontology class: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def check_ownership(self, class_id: UUID, workspace_id: UUID) -> bool:
|
||||
"""检查类型是否属于指定工作空间(通过场景关联)
|
||||
|
||||
Args:
|
||||
class_id: 类型ID
|
||||
workspace_id: 工作空间ID
|
||||
|
||||
Returns:
|
||||
bool: 属于返回True,否则返回False
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologyClassRepository(db)
|
||||
>>> is_owner = repo.check_ownership(class_id, workspace_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(
|
||||
f"Checking class ownership - "
|
||||
f"class_id={class_id}, workspace_id={workspace_id}"
|
||||
)
|
||||
|
||||
count = self.db.query(OntologyClass).join(
|
||||
OntologyScene,
|
||||
OntologyClass.scene_id == OntologyScene.scene_id
|
||||
).filter(
|
||||
OntologyClass.class_id == class_id,
|
||||
OntologyScene.workspace_id == workspace_id
|
||||
).count()
|
||||
|
||||
is_owner = count > 0
|
||||
|
||||
logger.debug(
|
||||
f"Class ownership check result: {is_owner} - "
|
||||
f"class_id={class_id}"
|
||||
)
|
||||
|
||||
return is_owner
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to check class ownership: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def get_scene_id_by_class(self, class_id: UUID) -> Optional[UUID]:
|
||||
"""根据类型ID获取所属场景ID
|
||||
|
||||
Args:
|
||||
class_id: 类型ID
|
||||
|
||||
Returns:
|
||||
Optional[UUID]: 场景ID,类型不存在则返回None
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologyClassRepository(db)
|
||||
>>> scene_id = repo.get_scene_id_by_class(class_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(f"Getting scene ID by class: {class_id}")
|
||||
|
||||
ontology_class = self.get_by_id(class_id)
|
||||
if not ontology_class:
|
||||
logger.debug(f"Class not found: {class_id}")
|
||||
return None
|
||||
|
||||
logger.debug(
|
||||
f"Found scene ID: {ontology_class.scene_id} for class: {class_id}"
|
||||
)
|
||||
|
||||
return ontology_class.scene_id
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to get scene ID by class: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
439
api/app/repositories/ontology_scene_repository.py
Normal file
439
api/app/repositories/ontology_scene_repository.py
Normal file
@@ -0,0 +1,439 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""本体场景Repository层
|
||||
|
||||
本模块提供本体场景的数据访问层实现。
|
||||
|
||||
Classes:
|
||||
OntologySceneRepository: 本体场景数据访问类
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import List, Optional
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy.orm import Session, joinedload
|
||||
|
||||
from app.core.logging_config import get_db_logger
|
||||
from app.models.ontology_scene import OntologyScene
|
||||
|
||||
|
||||
logger = get_db_logger()
|
||||
|
||||
|
||||
class OntologySceneRepository:
|
||||
"""本体场景Repository
|
||||
|
||||
提供本体场景的CRUD操作和权限检查。
|
||||
|
||||
Attributes:
|
||||
db: SQLAlchemy数据库会话
|
||||
"""
|
||||
|
||||
def __init__(self, db: Session):
|
||||
"""初始化Repository
|
||||
|
||||
Args:
|
||||
db: SQLAlchemy数据库会话
|
||||
"""
|
||||
self.db = db
|
||||
|
||||
def create(self, scene_data: dict, workspace_id: UUID) -> OntologyScene:
|
||||
"""创建本体场景
|
||||
|
||||
Args:
|
||||
scene_data: 场景数据字典,包含scene_name和scene_description
|
||||
workspace_id: 所属工作空间ID
|
||||
|
||||
Returns:
|
||||
OntologyScene: 创建的场景对象
|
||||
|
||||
Raises:
|
||||
Exception: 数据库操作失败
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologySceneRepository(db)
|
||||
>>> scene = repo.create(
|
||||
... {"scene_name": "医疗场景", "scene_description": "描述"},
|
||||
... workspace_id
|
||||
... )
|
||||
"""
|
||||
try:
|
||||
logger.info(
|
||||
f"Creating ontology scene - "
|
||||
f"name={scene_data.get('scene_name')}, "
|
||||
f"workspace_id={workspace_id}"
|
||||
)
|
||||
|
||||
scene = OntologyScene(
|
||||
scene_name=scene_data.get("scene_name"),
|
||||
scene_description=scene_data.get("scene_description"),
|
||||
workspace_id=workspace_id
|
||||
)
|
||||
|
||||
self.db.add(scene)
|
||||
self.db.flush() # 获取ID但不提交
|
||||
|
||||
logger.info(
|
||||
f"Ontology scene created successfully - "
|
||||
f"scene_id={scene.scene_id}"
|
||||
)
|
||||
|
||||
return scene
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to create ontology scene: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def get_by_id(self, scene_id: UUID) -> Optional[OntologyScene]:
|
||||
"""根据ID获取场景
|
||||
|
||||
Args:
|
||||
scene_id: 场景ID
|
||||
|
||||
Returns:
|
||||
Optional[OntologyScene]: 场景对象,不存在则返回None
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologySceneRepository(db)
|
||||
>>> scene = repo.get_by_id(scene_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(f"Getting ontology scene by ID: {scene_id}")
|
||||
|
||||
scene = self.db.query(OntologyScene).filter(
|
||||
OntologyScene.scene_id == scene_id
|
||||
).first()
|
||||
|
||||
if scene:
|
||||
logger.debug(f"Ontology scene found: {scene_id}")
|
||||
else:
|
||||
logger.debug(f"Ontology scene not found: {scene_id}")
|
||||
|
||||
return scene
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to get ontology scene by ID: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def get_by_name(self, scene_name: str, workspace_id: UUID) -> Optional[OntologyScene]:
|
||||
"""根据场景名称和工作空间ID获取场景(精确匹配)
|
||||
|
||||
Args:
|
||||
scene_name: 场景名称
|
||||
workspace_id: 工作空间ID
|
||||
|
||||
Returns:
|
||||
Optional[OntologyScene]: 场景对象,不存在则返回None
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologySceneRepository(db)
|
||||
>>> scene = repo.get_by_name("医疗场景", workspace_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(
|
||||
f"Getting ontology scene by name - "
|
||||
f"scene_name={scene_name}, workspace_id={workspace_id}"
|
||||
)
|
||||
|
||||
scene = self.db.query(OntologyScene).options(
|
||||
joinedload(OntologyScene.classes)
|
||||
).filter(
|
||||
OntologyScene.scene_name == scene_name,
|
||||
OntologyScene.workspace_id == workspace_id
|
||||
).first()
|
||||
|
||||
if scene:
|
||||
logger.debug(f"Ontology scene found: {scene_name}")
|
||||
else:
|
||||
logger.debug(f"Ontology scene not found: {scene_name}")
|
||||
|
||||
return scene
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to get ontology scene by name: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def search_by_name(self, keyword: str, workspace_id: UUID) -> List[OntologyScene]:
|
||||
"""根据关键词模糊搜索场景
|
||||
|
||||
使用 LIKE 进行模糊匹配,支持中文和英文。
|
||||
|
||||
Args:
|
||||
keyword: 搜索关键词
|
||||
workspace_id: 工作空间ID
|
||||
|
||||
Returns:
|
||||
List[OntologyScene]: 匹配的场景列表
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologySceneRepository(db)
|
||||
>>> scenes = repo.search_by_name("医疗", workspace_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(
|
||||
f"Searching ontology scenes by keyword - "
|
||||
f"keyword={keyword}, workspace_id={workspace_id}"
|
||||
)
|
||||
|
||||
# 使用 ilike 进行不区分大小写的模糊匹配
|
||||
scenes = self.db.query(OntologyScene).options(
|
||||
joinedload(OntologyScene.classes)
|
||||
).filter(
|
||||
OntologyScene.scene_name.ilike(f"%{keyword}%"),
|
||||
OntologyScene.workspace_id == workspace_id
|
||||
).order_by(
|
||||
OntologyScene.updated_at.desc()
|
||||
).all()
|
||||
|
||||
logger.info(
|
||||
f"Found {len(scenes)} ontology scenes matching keyword '{keyword}' "
|
||||
f"in workspace {workspace_id}"
|
||||
)
|
||||
|
||||
return scenes
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to search ontology scenes by keyword: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def get_by_workspace(self, workspace_id: UUID, page: Optional[int] = None, page_size: Optional[int] = None) -> tuple:
|
||||
"""获取工作空间下的所有场景(支持分页)
|
||||
|
||||
使用joinedload预加载classes关系以统计数量。
|
||||
|
||||
Args:
|
||||
workspace_id: 工作空间ID
|
||||
page: 页码(可选,从1开始)
|
||||
page_size: 每页数量(可选)
|
||||
|
||||
Returns:
|
||||
tuple: (场景列表, 总数量)
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologySceneRepository(db)
|
||||
>>> scenes, total = repo.get_by_workspace(workspace_id)
|
||||
>>> scenes, total = repo.get_by_workspace(workspace_id, page=1, page_size=10)
|
||||
"""
|
||||
try:
|
||||
logger.debug(f"Getting ontology scenes by workspace: {workspace_id}, page={page}, page_size={page_size}")
|
||||
|
||||
# 构建基础查询
|
||||
query = self.db.query(OntologyScene).options(
|
||||
joinedload(OntologyScene.classes)
|
||||
).filter(
|
||||
OntologyScene.workspace_id == workspace_id
|
||||
).order_by(
|
||||
OntologyScene.updated_at.desc()
|
||||
)
|
||||
|
||||
# 获取总数
|
||||
total = query.count()
|
||||
|
||||
# 如果提供了分页参数,应用分页
|
||||
if page is not None and page_size is not None:
|
||||
offset = (page - 1) * page_size
|
||||
query = query.offset(offset).limit(page_size)
|
||||
logger.debug(f"Applying pagination: offset={offset}, limit={page_size}")
|
||||
|
||||
scenes = query.all()
|
||||
|
||||
logger.info(
|
||||
f"Found {len(scenes)} ontology scenes (total: {total}) in workspace {workspace_id}"
|
||||
)
|
||||
|
||||
return scenes, total
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to get ontology scenes by workspace: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def update(self, scene_id: UUID, update_data: dict) -> Optional[OntologyScene]:
|
||||
"""更新场景信息
|
||||
|
||||
Args:
|
||||
scene_id: 场景ID
|
||||
update_data: 更新数据字典
|
||||
|
||||
Returns:
|
||||
Optional[OntologyScene]: 更新后的场景对象,不存在则返回None
|
||||
|
||||
Raises:
|
||||
Exception: 数据库操作失败
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologySceneRepository(db)
|
||||
>>> scene = repo.update(
|
||||
... scene_id,
|
||||
... {"scene_name": "新名称"}
|
||||
... )
|
||||
"""
|
||||
try:
|
||||
logger.info(f"Updating ontology scene: {scene_id}")
|
||||
|
||||
scene = self.get_by_id(scene_id)
|
||||
if not scene:
|
||||
logger.warning(f"Ontology scene not found for update: {scene_id}")
|
||||
return None
|
||||
|
||||
# 更新字段
|
||||
if "scene_name" in update_data and update_data["scene_name"] is not None:
|
||||
scene.scene_name = update_data["scene_name"]
|
||||
|
||||
if "scene_description" in update_data:
|
||||
scene.scene_description = update_data["scene_description"]
|
||||
|
||||
self.db.flush()
|
||||
|
||||
logger.info(f"Ontology scene updated successfully: {scene_id}")
|
||||
|
||||
return scene
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to update ontology scene: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def delete(self, scene_id: UUID) -> bool:
|
||||
"""删除场景(级联删除类型)
|
||||
|
||||
依赖数据库级联删除配置(ondelete="CASCADE")。
|
||||
|
||||
Args:
|
||||
scene_id: 场景ID
|
||||
|
||||
Returns:
|
||||
bool: 删除成功返回True,场景不存在返回False
|
||||
|
||||
Raises:
|
||||
Exception: 数据库操作失败
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologySceneRepository(db)
|
||||
>>> success = repo.delete(scene_id)
|
||||
"""
|
||||
try:
|
||||
logger.info(f"Deleting ontology scene: {scene_id}")
|
||||
|
||||
scene = self.get_by_id(scene_id)
|
||||
if not scene:
|
||||
logger.warning(f"Ontology scene not found for delete: {scene_id}")
|
||||
return False
|
||||
|
||||
self.db.delete(scene)
|
||||
self.db.flush()
|
||||
|
||||
logger.info(
|
||||
f"Ontology scene deleted successfully (cascade): {scene_id}"
|
||||
)
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to delete ontology scene: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def check_ownership(self, scene_id: UUID, workspace_id: UUID) -> bool:
|
||||
"""检查场景是否属于指定工作空间
|
||||
|
||||
Args:
|
||||
scene_id: 场景ID
|
||||
workspace_id: 工作空间ID
|
||||
|
||||
Returns:
|
||||
bool: 属于返回True,否则返回False
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologySceneRepository(db)
|
||||
>>> is_owner = repo.check_ownership(scene_id, workspace_id)
|
||||
"""
|
||||
try:
|
||||
logger.debug(
|
||||
f"Checking scene ownership - "
|
||||
f"scene_id={scene_id}, workspace_id={workspace_id}"
|
||||
)
|
||||
|
||||
count = self.db.query(OntologyScene).filter(
|
||||
OntologyScene.scene_id == scene_id,
|
||||
OntologyScene.workspace_id == workspace_id
|
||||
).count()
|
||||
|
||||
is_owner = count > 0
|
||||
|
||||
logger.debug(
|
||||
f"Scene ownership check result: {is_owner} - "
|
||||
f"scene_id={scene_id}"
|
||||
)
|
||||
|
||||
return is_owner
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to check scene ownership: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
|
||||
def get_simple_list(self, workspace_id: UUID) -> List[dict]:
|
||||
"""获取场景简单列表(仅包含scene_id和scene_name,用于下拉选择)
|
||||
|
||||
这是一个轻量级查询,不加载关联的classes,响应速度快。
|
||||
|
||||
Args:
|
||||
workspace_id: 工作空间ID
|
||||
|
||||
Returns:
|
||||
List[dict]: 场景简单列表,每项包含scene_id和scene_name
|
||||
|
||||
Examples:
|
||||
>>> repo = OntologySceneRepository(db)
|
||||
>>> scenes = repo.get_simple_list(workspace_id)
|
||||
>>> # [{"scene_id": "xxx", "scene_name": "场景1"}, ...]
|
||||
"""
|
||||
try:
|
||||
logger.debug(f"Getting simple scene list for workspace: {workspace_id}")
|
||||
|
||||
# 只查询需要的字段,不加载关联数据
|
||||
results = self.db.query(
|
||||
OntologyScene.scene_id,
|
||||
OntologyScene.scene_name
|
||||
).filter(
|
||||
OntologyScene.workspace_id == workspace_id
|
||||
).order_by(
|
||||
OntologyScene.updated_at.desc()
|
||||
).all()
|
||||
|
||||
scenes = [
|
||||
{"scene_id": str(r.scene_id), "scene_name": r.scene_name}
|
||||
for r in results
|
||||
]
|
||||
|
||||
logger.info(f"Found {len(scenes)} scenes (simple list) in workspace {workspace_id}")
|
||||
|
||||
return scenes
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to get simple scene list: {str(e)}",
|
||||
exc_info=True
|
||||
)
|
||||
raise
|
||||
@@ -4,7 +4,10 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app.core.logging_config import get_db_logger
|
||||
from app.models.prompt_optimizer_model import (
|
||||
PromptOptimizerSession, PromptOptimizerSessionHistory, RoleType
|
||||
PromptOptimizerSession,
|
||||
PromptOptimizerSessionHistory,
|
||||
RoleType,
|
||||
PromptHistory
|
||||
)
|
||||
|
||||
db_logger = get_db_logger()
|
||||
@@ -16,6 +19,12 @@ class PromptOptimizerSessionRepository:
|
||||
def __init__(self, db: Session):
|
||||
self.db = db
|
||||
|
||||
def get_session_by_id(self, session_id: uuid.UUID) -> PromptOptimizerSession | None:
|
||||
session = self.db.query(PromptOptimizerSession).filter(
|
||||
PromptOptimizerSession.id == session_id,
|
||||
).first()
|
||||
return session
|
||||
|
||||
def create_session(
|
||||
self,
|
||||
tenant_id: uuid.UUID,
|
||||
@@ -38,12 +47,9 @@ class PromptOptimizerSessionRepository:
|
||||
user_id=user_id,
|
||||
)
|
||||
self.db.add(session)
|
||||
self.db.commit()
|
||||
self.db.refresh(session)
|
||||
db_logger.debug(f"Prompt optimization session created: ID:{session.id}")
|
||||
return session
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error creating prompt optimization session: user_id={user_id} - {str(e)}")
|
||||
db_logger.error(f"Error creating prompt optimization session: - {str(e)}")
|
||||
raise
|
||||
|
||||
def get_session_history(
|
||||
@@ -71,10 +77,10 @@ class PromptOptimizerSessionRepository:
|
||||
PromptOptimizerSession.id == session_id,
|
||||
PromptOptimizerSession.user_id == user_id
|
||||
).first()
|
||||
|
||||
|
||||
if not session:
|
||||
return []
|
||||
|
||||
|
||||
history = self.db.query(PromptOptimizerSessionHistory).filter(
|
||||
PromptOptimizerSessionHistory.session_id == session.id,
|
||||
PromptOptimizerSessionHistory.user_id == user_id
|
||||
@@ -104,11 +110,11 @@ class PromptOptimizerSessionRepository:
|
||||
PromptOptimizerSession.user_id == user_id,
|
||||
PromptOptimizerSession.tenant_id == tenant_id
|
||||
).first()
|
||||
|
||||
|
||||
if not session:
|
||||
db_logger.error(f"Session {session_id} not found for user {user_id}")
|
||||
raise ValueError(f"Session {session_id} not found for user {user_id}")
|
||||
|
||||
|
||||
message = PromptOptimizerSessionHistory(
|
||||
tenant_id=tenant_id,
|
||||
session_id=session.id,
|
||||
@@ -117,8 +123,199 @@ class PromptOptimizerSessionRepository:
|
||||
content=content,
|
||||
)
|
||||
self.db.add(message)
|
||||
self.db.commit()
|
||||
|
||||
return message
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error creating prompt optimization session history: session_id={session_id} - {str(e)}")
|
||||
raise
|
||||
|
||||
def get_first_user_message(self, session_id: uuid.UUID) -> str | None:
|
||||
"""
|
||||
Get the first user message from a session.
|
||||
|
||||
Args:
|
||||
session_id (uuid.UUID): The session ID.
|
||||
|
||||
Returns:
|
||||
str | None: The content of the first user message, or None if not found.
|
||||
"""
|
||||
try:
|
||||
message = self.db.query(PromptOptimizerSessionHistory).filter(
|
||||
PromptOptimizerSessionHistory.session_id == session_id,
|
||||
PromptOptimizerSessionHistory.role == RoleType.USER.value
|
||||
).order_by(
|
||||
PromptOptimizerSessionHistory.created_at.asc()
|
||||
).first()
|
||||
|
||||
return message.content if message else None
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error getting first user message: session_id={session_id} - {str(e)}")
|
||||
raise
|
||||
|
||||
|
||||
class PromptReleaseRepository:
|
||||
def __init__(self, db: Session):
|
||||
self.db = db
|
||||
|
||||
def get_prompt_by_session_id(self, session_id: uuid.UUID) -> PromptHistory | None:
|
||||
prompt_obj = self.db.query(PromptHistory).filter(
|
||||
PromptHistory.session_id == session_id,
|
||||
PromptHistory.is_delete.is_(False)
|
||||
).first()
|
||||
return prompt_obj
|
||||
|
||||
def create_prompt_release(
|
||||
self,
|
||||
tenant_id: uuid.UUID,
|
||||
title: str,
|
||||
session_id: uuid.UUID,
|
||||
prompt: str,
|
||||
) -> PromptHistory:
|
||||
try:
|
||||
prompt_obj = PromptHistory(
|
||||
tenant_id=tenant_id,
|
||||
title=title,
|
||||
session_id=session_id,
|
||||
prompt=prompt,
|
||||
)
|
||||
self.db.add(prompt_obj)
|
||||
return prompt_obj
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error creating prompt release: session_id={session_id} - {str(e)}")
|
||||
raise
|
||||
|
||||
def soft_delete_prompt(self, prompt_obj: PromptHistory) -> None:
|
||||
"""
|
||||
Soft delete a prompt release by setting is_delete flag to True.
|
||||
|
||||
Args:
|
||||
prompt_obj (PromptHistory): The prompt release object to delete.
|
||||
"""
|
||||
try:
|
||||
prompt_obj.is_delete = True
|
||||
db_logger.debug(f"Soft deleted prompt release: id={prompt_obj.id}, session_id={prompt_obj.session_id}")
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error soft deleting prompt release: id={prompt_obj.id} - {str(e)}")
|
||||
raise
|
||||
|
||||
def get_prompt_by_id(self, prompt_id: uuid.UUID) -> PromptHistory | None:
|
||||
"""
|
||||
Get a prompt release by its ID.
|
||||
|
||||
Args:
|
||||
prompt_id (uuid.UUID): The prompt release ID.
|
||||
|
||||
Returns:
|
||||
PromptHistory | None: The prompt release object or None if not found.
|
||||
"""
|
||||
try:
|
||||
prompt_obj = self.db.query(PromptHistory).filter(
|
||||
PromptHistory.id == prompt_id
|
||||
).first()
|
||||
return prompt_obj
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error getting prompt release by id: id={prompt_id} - {str(e)}")
|
||||
raise
|
||||
|
||||
def count_prompts(self, tenant_id: uuid.UUID) -> int:
|
||||
"""
|
||||
Count total number of non-deleted prompts for a tenant.
|
||||
|
||||
Args:
|
||||
tenant_id (uuid.UUID): The tenant ID.
|
||||
|
||||
Returns:
|
||||
int: Total count of prompts.
|
||||
"""
|
||||
try:
|
||||
count = self.db.query(PromptHistory).filter(
|
||||
PromptHistory.tenant_id == tenant_id,
|
||||
PromptHistory.is_delete.is_(False)
|
||||
).count()
|
||||
return count
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error counting prompts: tenant_id={tenant_id} - {str(e)}")
|
||||
raise
|
||||
|
||||
def get_prompts_paginated(
|
||||
self,
|
||||
tenant_id: uuid.UUID,
|
||||
offset: int,
|
||||
limit: int
|
||||
) -> list[PromptHistory]:
|
||||
"""
|
||||
Get paginated list of prompt releases for a tenant.
|
||||
|
||||
Args:
|
||||
tenant_id (uuid.UUID): The tenant ID.
|
||||
offset (int): Number of records to skip.
|
||||
limit (int): Maximum number of records to return.
|
||||
|
||||
Returns:
|
||||
list[PromptHistory]: List of prompt releases.
|
||||
"""
|
||||
try:
|
||||
prompts = self.db.query(PromptHistory).filter(
|
||||
PromptHistory.tenant_id == tenant_id,
|
||||
PromptHistory.is_delete.is_(False)
|
||||
).order_by(
|
||||
PromptHistory.created_at.desc()
|
||||
).offset(offset).limit(limit).all()
|
||||
return prompts
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error getting paginated prompts: tenant_id={tenant_id} - {str(e)}")
|
||||
raise
|
||||
|
||||
def count_prompts_by_keyword(self, tenant_id: uuid.UUID, keyword: str) -> int:
|
||||
"""
|
||||
Count total number of non-deleted prompts matching keyword for a tenant.
|
||||
|
||||
Args:
|
||||
tenant_id (uuid.UUID): The tenant ID.
|
||||
keyword (str): Search keyword for title.
|
||||
|
||||
Returns:
|
||||
int: Total count of matching prompts.
|
||||
"""
|
||||
try:
|
||||
count = self.db.query(PromptHistory).filter(
|
||||
PromptHistory.tenant_id == tenant_id,
|
||||
PromptHistory.is_delete.is_(False),
|
||||
PromptHistory.title.ilike(f"%{keyword}%")
|
||||
).count()
|
||||
return count
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error counting prompts by keyword: tenant_id={tenant_id}, keyword={keyword} - {str(e)}")
|
||||
raise
|
||||
|
||||
def search_prompts_paginated(
|
||||
self,
|
||||
tenant_id: uuid.UUID,
|
||||
keyword: str,
|
||||
offset: int,
|
||||
limit: int
|
||||
) -> list[PromptHistory]:
|
||||
"""
|
||||
Search prompt releases by keyword in title with pagination.
|
||||
|
||||
Args:
|
||||
tenant_id (uuid.UUID): The tenant ID.
|
||||
keyword (str): Search keyword for title.
|
||||
offset (int): Number of records to skip.
|
||||
limit (int): Maximum number of records to return.
|
||||
|
||||
Returns:
|
||||
list[PromptHistory]: List of matching prompt releases.
|
||||
"""
|
||||
try:
|
||||
prompts = self.db.query(PromptHistory).filter(
|
||||
PromptHistory.tenant_id == tenant_id,
|
||||
PromptHistory.is_delete.is_(False),
|
||||
PromptHistory.title.ilike(f"%{keyword}%")
|
||||
).order_by(
|
||||
PromptHistory.created_at.desc()
|
||||
).offset(offset).limit(limit).all()
|
||||
return prompts
|
||||
except Exception as e:
|
||||
db_logger.error(f"Error searching prompts: tenant_id={tenant_id}, keyword={keyword} - {str(e)}")
|
||||
raise
|
||||
|
||||
Reference in New Issue
Block a user