Commit Graph

1527 Commits

Author SHA1 Message Date
Mark
3b359df02f [modify] fix 2026-04-14 17:02:11 +08:00
Mark
fcf3071cb0 [modify] Optimize ES connections and add rerank security checks 2026-04-14 16:46:57 +08:00
Timebomb2018
e3265e4ba3 fix(http-request,embedding,naive): tighten form-data validation, reduce truncation length to 8000, and disable chunking for Excel
The form-data validation now ensures all items in the list are of type HttpFormData. Truncation length for embedding inputs is reduced from 8191 to 8000 to accommodate tokenizer differences and avoid overflow. Excel parsing now disables chunking by setting chunk_token_num to 0, aligning with intended behavior for structured file ingestion.
2026-04-14 16:14:01 +08:00
Timebomb2018
0965008210 fix(http-request): support array and file variables in form-data files upload
- Updated form-data handling to accept both single FileVariable and ArrayVariable containing FileVariable for file uploads
- Fixed HTTP client redirect handling by enabling follow_redirects=True when downloading remote files
- Adjusted config validation to correctly require list type for form-data fields instead of HttpFormData class
2026-04-14 15:53:16 +08:00
Timebomb2018
2450fe3afe refactor(workflow): move _merge_conv_vars call inside iteration loop for consistent state updates 2026-04-13 19:00:36 +08:00
Timebomb2018
7ca80b5d01 perf(app): optimize FileMetadata queries by batching lookups
Multiple services were performing individual database queries for FileMetadata when resolving missing file names/sizes. This change batches the queries using `in_()` to reduce database round trips and improve performance.
2026-04-13 18:52:43 +08:00
lanceyq
9470dd2f1e refactor(memory): extract shared MemorySummary count query and replace magic number
- Move duplicated Neo4j MemorySummary count query into
  MemoryBaseService.get_valid_memory_summary_count()
- Introduce MIN_MEMORY_SUMMARY_COUNT constant to replace hardcoded 5
- Fix import ordering in implicit_emotions_storage_repository
- Use UTC consistently for date calculations (remove CST offset,
  datetime.now → datetime.utcnow)
2026-04-13 18:47:56 +08:00
Timebomb2018
10f1089198 feat(workflow): refactor iteration runtime to support independent subgraph per task
feat(app): support file metadata in chat messages and DSL app overwrite
- Extended chat message file objects with `name`, `size`, and `file_type` fields across app_chat_service and workflow_service
- Added ability to overwrite existing app configurations via DSL import in app_dsl_service, including type validation and config update logic for AgentConfig, MultiAgentConfig, and WorkflowConfig
2026-04-13 18:38:12 +08:00
lanceyq
ef8c7093b5 refactor(memory): use MemorySummary node count for implicit memory metrics
- Replace Statement-based implicit memory count (count/3) with actual
  MemorySummary node count filtered by DERIVED_FROM_STATEMENT relationship
- Add minimum threshold of 5 MemorySummary nodes before reporting data
- Add _build_empty_profile() to return structured empty profile when
  insufficient data exists, skipping unnecessary LLM calls
2026-04-13 18:32:43 +08:00
Timebomb2018
5bb9ce9018 fix(user): add user retrieval regardless of active status and update DSL config enrichment
Added `get_user_by_id_regardless_active` in user repository to support activation/deactivation workflows, updated `user_service` to use it, and refactored `_enrich_release_config` in `app_dsl_service` to accept `default_model_config_id` as a parameter instead of reading from config dict.
2026-04-13 14:40:57 +08:00
Eternity
dca3173ed9 refactor(memory): restructure memory search architecture
- Replace storage_services/search with new read_services/memory_search structure
- Implement content_search and perceptual_search strategies
- Add query_preprocessor for search optimization
- Create memory_service as unified interface
- Update celery_app and graph_search for new architecture
- Add enums for memory operations
- Implement base_pipeline and memory_read pipeline patterns
2026-04-13 14:03:47 +08:00
Ke Sun
5eaedaad77 Merge pull request #862 from SuanmoSuanyangTechnology/feat/metadata-show
refactor(memory): flatten meta_data fields in update_end_user_info re…
2026-04-13 13:54:41 +08:00
lanceyq
7a2a941ac4 refactor(neo4j): rename execute_query parameter from query to cypher
Improves readability by making the parameter name explicitly reflect
that it expects a Cypher query string rather than a generic query.
2026-04-13 13:47:59 +08:00
Mark
cba24e58db Merge branch 'feature/rag2' into develop
* feature/rag2:
  [modify] parse document workflow, add graph queue hand build graph
  [modify] mineru
  [modify] 优化tasks ,拆分graphirag 队列

# Conflicts:
#	api/app/tasks.py
2026-04-13 13:46:19 +08:00
wxy
72be9f75f9 feat: Add quota check decorator and implement tenant-level API rate limiting
- Add quota check decorator module quota_stub.py, providing community edition stub implementation
- Add quota check decorators to multiple controllers
- Implement tenant-level API call rate limiting
- Remove redundant plan fields from tenant_model.py
- Optimize user permission check logic with added error handling
2026-04-13 11:58:14 +08:00
Mark
a96f20ee05 [modify] parse document workflow, add graph queue hand build graph 2026-04-13 10:40:58 +08:00
Ke Sun
b8507a1df6 Merge pull request #843 from SuanmoSuanyangTechnology/feature/openclaw_lm
Feature/openclaw lm
2026-04-10 18:54:09 +08:00
miao
0f28d54c43 fix(tools): add get_required_config_parameters to OpenClawTool
Without this method, the tool status would show as available even when
server_url and api_key are not configured.
2026-04-10 18:47:31 +08:00
lanceyq
0afc38e7ef refactor(memory): flatten meta_data fields in update_end_user_info response
Align update response with get_end_user_info by extracting profile,
knowledge_tags, and behavioral_hints to top-level keys instead of
returning raw meta_data dict.
2026-04-10 18:45:35 +08:00
Timebomb2018
7cfb6ace22 Merge branch 'refs/heads/develop' into feature/agent-tool_xjn 2026-04-10 18:33:39 +08:00
山程漫悟
91cc20d589 Merge pull request #857 from wanxunyang/feature/switch-app-version-for-shared-api-key-apps
feat: add versioned app chat API and fix release isolation bug
2026-04-10 18:33:03 +08:00
Timebomb2018
f01ca51896 Merge branch 'refs/heads/develop' into feature/agent-tool_xjn 2026-04-10 18:30:46 +08:00
Timebomb2018
f4a63f7d55 feat(workflow): support Dify features conversion and file variable migration 2026-04-10 18:30:12 +08:00
Ke Sun
0019f3acfd Merge pull request #860 from SuanmoSuanyangTechnology/hotfix/v0.2.10
Hotfix/v0.2.10
2026-04-10 18:29:38 +08:00
Ke Sun
58d18b476c Merge pull request #851 from SuanmoSuanyangTechnology/feat/extract-metadata
Feat/extract metadata
2026-04-10 18:11:04 +08:00
Ke Sun
5e5927a0b9 Merge pull request #852 from SuanmoSuanyangTechnology/fix/rag-num
fix:Remove "total"
2026-04-10 18:06:50 +08:00
wxy
7869121382 feat: add versioned app chat API and fix release isolation bug 2026-04-10 17:53:24 +08:00
wxy
af83980f99 feat: add versioned app chat API and fix release isolation bug 2026-04-10 17:22:11 +08:00
山程漫悟
cf0d11208c Merge pull request #855 from wanxunyang/feature/switch-app-version-for-shared-api-key-apps
Feature/switch app version for shared api key apps
2026-04-10 16:36:06 +08:00
Timebomb2018
e5e6699168 feat(workflow): support nested variable access and DashScope rerank provider 2026-04-10 16:21:49 +08:00
Timebomb2018
068e2bfb7e fix(workflow): update output pattern to handle standalone curly braces 2026-04-10 15:24:18 +08:00
Timebomb2018
4ce6fede67 fix(workflow): update cycle graph node output type validation 2026-04-10 14:08:51 +08:00
miao
8497c955f9 fix(tools): make image_understand image_url optional and remove unused operation variable
Change image_url from required to optional in both operation_tool.py and
tool_service.py for image_understand operation, avoiding parameter validation
conflict with uploaded_files priority logic.
Remove unused operation variable from OpenClawTool.execute().
2026-04-10 13:31:09 +08:00
wxy
72fe3962cf feat(api): Support specifying app version for chat 2026-04-10 12:18:11 +08:00
wxy
c253968aa8 feat(api): Support specifying app version for chat 2026-04-10 12:10:24 +08:00
wxy
412183c359 feat(api): Support specifying app version for chat 2026-04-10 11:44:50 +08:00
wxy
90e8e90528 feat(api): Support specifying app version for chat 2026-04-10 11:11:39 +08:00
wxy
fd05c000f6 feat(api): Support specifying app version for chat 2026-04-10 11:04:59 +08:00
lanceyq
627d6a0381 fix : add comments 2026-04-10 10:43:43 +08:00
Ke Sun
807dee8460 Merge branch 'hotfix/v0.2.10' into develop 2026-04-10 10:16:39 +08:00
lanceyq
cd018814fe fix(memory): improve metadata language detection and clean_metadata logic
- Make MetadataExtractor language param optional (default None) to
  support auto-detection fallback when no language is explicitly set
- Refactor clean_metadata from walrus-operator dict comprehension to
  explicit loop for correctness and readability
2026-04-10 00:42:11 +08:00
lanceyq
e0b7e95af6 refactor(memory): remove first-person pronoun replacement and inline metadata utils
- Remove _replace_first_person_with_user from StatementExtractor to preserve
  original user text for downstream metadata/alias extraction
- Delete metadata_utils.py module, inline clean_metadata into Celery task
- Remove unused imports and commented-out collect_user_raw_messages method
- Apply formatting cleanup across metadata models and extraction orchestrator
2026-04-10 00:29:18 +08:00
lanceyq
fc5ce63e44 fix:Remove "total" 2026-04-09 21:57:17 +08:00
lanceyq
15a863b41a feat(memory): unify alias extraction into metadata pipeline and deduplicate user entity nodes
- Merge alias add/remove into MetadataExtractionResponse and Celery metadata task,
  removing the separate sync step from extraction_orchestrator
- Replace first-person pronouns ("我") with "用户" in statement extraction to
  preserve identity semantics for downstream metadata/alias extraction
- Update extract_statement.jinja2 prompt to enforce "用户" as subject for user
  statements instead of resolving to real names
- Add alias change instructions (aliases_to_add/aliases_to_remove) to
  extract_user_metadata.jinja2 with incremental merge logic
- Deduplicate special entities ("用户", "AI助手") in graph_saver by reusing
  existing Neo4j node IDs per end_user_id
- Sync final aliases from PgSQL to Neo4j user entity nodes after metadata write
2026-04-09 21:55:59 +08:00
miao
7842435321 fix(tools): forward set_runtime_context through OperationTool to base_tool
OperationTool wraps builtin tools for multi-operation support but did not
forward set_runtime_context, causing OpenClawTool to miss uploaded_files
and conversation_id when used with operation routing.
2026-04-09 20:01:07 +08:00
Timebomb2018
ca4f7aa65d refactor(rag/nlp): refactor reranking logic to apply post-deduplication and remove debug log 2026-04-09 19:35:43 +08:00
miao
b875626f18 fix(tools): revert CustomTool __init__ to upstream, remove redundant schema parsing
The _parse_openapi_schema() method already handles string-to-dict conversion internally, so the duplicate json.loads in __init__ was unnecessary.
2026-04-09 19:33:27 +08:00
Timebomb2018
130684cac0 refactor(rag/nlp): standardize knowledge graph retrieval to use DocumentChunk and add debug logging
The knowledge graph retrieval logic in `search.py` was updated to consistently return `DocumentChunk` instances instead of raw dictionaries, improving type safety and alignment with the RAG pipeline's expected data structure. Additionally, debug logging was enhanced in `draft_run_service.py` to log the full `retrieve_chunks_result` before extracting page content, aiding troubleshooting.
2026-04-09 19:07:53 +08:00
Timebomb2018
62e0b2730b refactor(workflow/knowledge): update pattern matching to support multiple retrieve types 2026-04-09 18:29:08 +08:00
miao
55b2e05ba8 feat(tools): refactor migrate OpenClaw from custom tool to builtin tool
Create OpenClawTool class inheriting BuiltinTool with dedicated config
Remove all x-openclaw special handling from CustomTool (~270 lines)
Add multi-operation support (print_task, device_query, image_understand, general)
Change ensure_builtin_tools_initialized to incremental mode for auto-provisioning
Fix OperationTool and LangchainAdapter to support OpenClaw operation routing
2026-04-09 18:14:31 +08:00