Commit Graph

606 Commits

Author SHA1 Message Date
Timebomb2018
0965008210 fix(http-request): support array and file variables in form-data files upload
- Updated form-data handling to accept both single FileVariable and ArrayVariable containing FileVariable for file uploads
- Fixed HTTP client redirect handling by enabling follow_redirects=True when downloading remote files
- Adjusted config validation to correctly require list type for form-data fields instead of HttpFormData class
2026-04-14 15:53:16 +08:00
Ke Sun
b8507a1df6 Merge pull request #843 from SuanmoSuanyangTechnology/feature/openclaw_lm
Feature/openclaw lm
2026-04-10 18:54:09 +08:00
miao
0f28d54c43 fix(tools): add get_required_config_parameters to OpenClawTool
Without this method, the tool status would show as available even when
server_url and api_key are not configured.
2026-04-10 18:47:31 +08:00
Timebomb2018
f01ca51896 Merge branch 'refs/heads/develop' into feature/agent-tool_xjn 2026-04-10 18:30:46 +08:00
Timebomb2018
f4a63f7d55 feat(workflow): support Dify features conversion and file variable migration 2026-04-10 18:30:12 +08:00
Ke Sun
0019f3acfd Merge pull request #860 from SuanmoSuanyangTechnology/hotfix/v0.2.10
Hotfix/v0.2.10
2026-04-10 18:29:38 +08:00
Ke Sun
58d18b476c Merge pull request #851 from SuanmoSuanyangTechnology/feat/extract-metadata
Feat/extract metadata
2026-04-10 18:11:04 +08:00
Timebomb2018
e5e6699168 feat(workflow): support nested variable access and DashScope rerank provider 2026-04-10 16:21:49 +08:00
Timebomb2018
068e2bfb7e fix(workflow): update output pattern to handle standalone curly braces 2026-04-10 15:24:18 +08:00
Timebomb2018
4ce6fede67 fix(workflow): update cycle graph node output type validation 2026-04-10 14:08:51 +08:00
miao
8497c955f9 fix(tools): make image_understand image_url optional and remove unused operation variable
Change image_url from required to optional in both operation_tool.py and
tool_service.py for image_understand operation, avoiding parameter validation
conflict with uploaded_files priority logic.
Remove unused operation variable from OpenClawTool.execute().
2026-04-10 13:31:09 +08:00
Ke Sun
807dee8460 Merge branch 'hotfix/v0.2.10' into develop 2026-04-10 10:16:39 +08:00
lanceyq
cd018814fe fix(memory): improve metadata language detection and clean_metadata logic
- Make MetadataExtractor language param optional (default None) to
  support auto-detection fallback when no language is explicitly set
- Refactor clean_metadata from walrus-operator dict comprehension to
  explicit loop for correctness and readability
2026-04-10 00:42:11 +08:00
lanceyq
e0b7e95af6 refactor(memory): remove first-person pronoun replacement and inline metadata utils
- Remove _replace_first_person_with_user from StatementExtractor to preserve
  original user text for downstream metadata/alias extraction
- Delete metadata_utils.py module, inline clean_metadata into Celery task
- Remove unused imports and commented-out collect_user_raw_messages method
- Apply formatting cleanup across metadata models and extraction orchestrator
2026-04-10 00:29:18 +08:00
lanceyq
15a863b41a feat(memory): unify alias extraction into metadata pipeline and deduplicate user entity nodes
- Merge alias add/remove into MetadataExtractionResponse and Celery metadata task,
  removing the separate sync step from extraction_orchestrator
- Replace first-person pronouns ("我") with "用户" in statement extraction to
  preserve identity semantics for downstream metadata/alias extraction
- Update extract_statement.jinja2 prompt to enforce "用户" as subject for user
  statements instead of resolving to real names
- Add alias change instructions (aliases_to_add/aliases_to_remove) to
  extract_user_metadata.jinja2 with incremental merge logic
- Deduplicate special entities ("用户", "AI助手") in graph_saver by reusing
  existing Neo4j node IDs per end_user_id
- Sync final aliases from PgSQL to Neo4j user entity nodes after metadata write
2026-04-09 21:55:59 +08:00
miao
7842435321 fix(tools): forward set_runtime_context through OperationTool to base_tool
OperationTool wraps builtin tools for multi-operation support but did not
forward set_runtime_context, causing OpenClawTool to miss uploaded_files
and conversation_id when used with operation routing.
2026-04-09 20:01:07 +08:00
Timebomb2018
ca4f7aa65d refactor(rag/nlp): refactor reranking logic to apply post-deduplication and remove debug log 2026-04-09 19:35:43 +08:00
miao
b875626f18 fix(tools): revert CustomTool __init__ to upstream, remove redundant schema parsing
The _parse_openapi_schema() method already handles string-to-dict conversion internally, so the duplicate json.loads in __init__ was unnecessary.
2026-04-09 19:33:27 +08:00
Timebomb2018
130684cac0 refactor(rag/nlp): standardize knowledge graph retrieval to use DocumentChunk and add debug logging
The knowledge graph retrieval logic in `search.py` was updated to consistently return `DocumentChunk` instances instead of raw dictionaries, improving type safety and alignment with the RAG pipeline's expected data structure. Additionally, debug logging was enhanced in `draft_run_service.py` to log the full `retrieve_chunks_result` before extracting page content, aiding troubleshooting.
2026-04-09 19:07:53 +08:00
Timebomb2018
62e0b2730b refactor(workflow/knowledge): update pattern matching to support multiple retrieve types 2026-04-09 18:29:08 +08:00
miao
55b2e05ba8 feat(tools): refactor migrate OpenClaw from custom tool to builtin tool
Create OpenClawTool class inheriting BuiltinTool with dedicated config
Remove all x-openclaw special handling from CustomTool (~270 lines)
Add multi-operation support (print_task, device_query, image_understand, general)
Change ensure_builtin_tools_initialized to incremental mode for auto-provisioning
Fix OperationTool and LangchainAdapter to support OpenClaw operation routing
2026-04-09 18:14:31 +08:00
miao
562ca6c1f1 fix(tools): fix OpenClaw connection test and multimodal format compatibility
- Use safe .get() for server URL to avoid KeyError
- Support both api_key and token in connection test auth
- Add OpenAI/Volcano image format (image_url) support
- Add aiohttp import in _test_openclaw_connection
2026-04-09 18:14:30 +08:00
miao
e298b38de9 feat(tools): add OpenClaw remote agent tool integration
- Detect x-openclaw flag in OpenAPI schema and init dedicated config
- Implement multimodal input/output (image download, compress, base64)
- Add OpenClaw connection test and status validation in tool service
- Fix auth_config token check to support both api_key and bearer_token
- Inject runtime context (user_id, conversation_id, files) in chat services
2026-04-09 18:14:29 +08:00
Timebomb2018
a7b8ba0c66 fix(rag): fix pdfplumber concurrency issue and add debug logging
The pdfplumber parser now uses a global lock to prevent concurrent access issues during PDF image rendering. Additionally, added a warning log to trace knowledge retrieval results for debugging purposes. The syntax fix in knowledge node's match case ensures correct pattern matching behavior.

BREAKING CHANGE: The pdfplumber parser now requires LOCK_KEY_pdfplumber to be defined in sys.modules for thread safety.

Closes #841
2026-04-09 17:48:16 +08:00
Timebomb2018
ea2f5e61c9 fix(tool): strip input_value in datetime_to_timestamp to prevent whitespace-related parsing errors 2026-04-09 15:18:39 +08:00
Timebomb2018
5975d70bf9 feat(tool): add datetime_to_timestamp operation with timezone support 2026-04-09 15:14:15 +08:00
lanceyq
e0546e01ef refactor(memory): delegate metadata merging to LLM instead of code-based merge
- Remove merge_metadata and its helper functions from metadata_utils.py
- Pass existing_metadata to MetadataExtractor.extract_metadata() as LLM context
- Add merge instructions to extract_user_metadata.jinja2 prompt (zh/en)
- Update Celery task to read existing metadata before extraction and overwrite
- Simplify field descriptions in UserMetadataProfile model
- Add _update_timestamps helper to track changed fields
2026-04-09 15:10:29 +08:00
Timebomb2018
70aab94fc3 feat(knowledge): support graph retrieval type with dynamic API key selection 2026-04-09 15:00:49 +08:00
lanceyq
f2d7479229 feat(memory): add async user metadata extraction pipeline
- Add MetadataExtractor to collect user-related statements post-dedup
  and extract profile/behavioral metadata via independent LLM call
- Add Celery task (extract_user_metadata) routed to memory_tasks queue
- Add metadata models (UserMetadata, UserMetadataProfile, etc.)
- Add metadata utility functions (clean, validate, merge with _op support)
- Add Jinja2 prompt template for metadata extraction (zh/en)
- Fix Lucene query parameter naming: rename `q` to `query` across all
  Cypher queries, graph_search functions, and callers
- Escape `/` in Lucene queries to prevent TokenMgrError
- Add `speaker` field to ChunkNode and persist it in Neo4j
- Remove unused imports (argparse, os, UUID) in search.py
- Fix unnecessary db context nesting in interest distribution task
2026-04-09 11:01:56 +08:00
Ke Sun
ae1909b7e9 Merge pull request #833 from SuanmoSuanyangTechnology/release/v0.2.10
Release/v0.2.10
2026-04-08 21:45:35 +08:00
Timebomb2018
e3d50c5c55 fix(workflow): unify token usage metadata handling across LLM-related nodes 2026-04-08 20:44:02 +08:00
山程漫悟
6475387af8 Merge pull request #825 from SuanmoSuanyangTechnology/fix/parameter_extractor_nonevalue
fix(parameter_extractor): add _extract_output method for handling default values
2026-04-08 17:32:56 +08:00
Eternity
b330bdba29 fix(parameter_extractor): add _extract_output method for handling default values 2026-04-08 17:09:57 +08:00
Timebomb2018
931b800bb6 fix(workflow): List operation node, exception handling for variables after importing the dify file 2026-04-08 15:07:57 +08:00
Timebomb2018
4eed393db5 fix(app):
1. List operation node sub-variable comparison;
2. Non-Dashscope Omni model processing;
3.Handling the issue of disappearing iterative nodes
2026-04-08 11:11:57 +08:00
Timebomb2018
ca1a2c7b9e fix(workflow):
Sorting of list operation nodes
2026-04-07 23:01:27 +08:00
Timebomb2018
b9439b337a fix(workflow): 1. List operation node;2.Add space error message;3.File session variable handling 2026-04-07 21:33:11 +08:00
Timebomb2018
9a931389ea fix(app):
1. Import issue handling;
2. embedding model checkout;
3. omni model removes thinking
2026-04-07 17:15:32 +08:00
Ke Sun
cfbf83f71e Merge pull request #787 from SuanmoSuanyangTechnology/fix/atomic-update
fix(memory): improve optimistic lock resilience in access history man…
2026-04-07 10:57:20 +08:00
Timebomb2018
38f3455bab feat(workflow):
1. add list operator node for filtering, sorting, limiting, and extracting list items;
2. Increase the session variable to the "file" type
2026-04-03 18:57:28 +08:00
lanceyq
99862db7a0 refactor(forgetting-engine): replace optimistic locking with APOC atomic operations in access history manager
- Replace version-based optimistic locking and retry loop with apoc.atomic.add/insert for concurrent safety
- Merge duplicate accesses within a batch before updating (access_count_delta)
- Simplify _calculate_update to only compute on new timestamps instead of full history rebuild
- Remove max_retries instance variable (kept as param for backward compat)
- Trim verbose docstrings and inline comments
2026-04-03 18:40:03 +08:00
lanceyq
00a8099857 changes:(api) Change the "jitter" to "tremble". 2026-04-03 16:55:53 +08:00
lanceyq
117e29fbe3 fix(memory): improve optimistic lock resilience in access history manager
- Increase max_retries from 3 to 5 for concurrent conflict recovery
- Add randomized exponential backoff between retries to reduce contention
- Merge duplicate node accesses in batch operations to avoid self-conflicts
- Support access_times parameter for merged batch access counting
- Add Community node label support in atomic update content field map
2026-04-03 16:46:09 +08:00
Ke Sun
bc5ea2d421 Merge pull request #784 from SuanmoSuanyangTechnology/fix/aliases-extract
feat(memory): prevent cross-role alias contamination between user and…
2026-04-03 15:26:31 +08:00
lanceyq
c4ff1a325b refactor(memory): harden alias extraction and sync PgSQL with Neo4j deduped aliases
- Strengthen anti-hallucination rules in extract_triplet prompt to
  enforce verbatim-only alias extraction, removing suggestive examples
- Add _extract_deduped_entity_aliases to sync historical aliases from
  Neo4j two-stage dedup into PgSQL end_user_info
- Remove unused _fetch_neo4j_user_aliases; reuse injected connector
  instead of instantiating new Neo4jConnector
- Simplify _would_merge_cross_role and reuse clean_cross_role_aliases
  in _normalize_special_entity_names
- Reuse _USER_PLACEHOLDER_NAMES from dedup module to avoid duplication
2026-04-03 14:38:55 +08:00
Mark
a711635694 Merge pull request #785 from wanxunyang/feat/app-log-wxy
feat(workflow): add opening statement and citation support
2026-04-03 13:41:08 +08:00
lanceyq
15b3ce3dd5 refactor(memory): deduplicate assistant alias query and fix case-sensitive placeholder matching
- Extract fetch_neo4j_assistant_aliases() into deduped_and_disamb.py as
  single source of truth, replacing inline Cypher in write_tools and
  extraction_orchestrator
- Normalize USER_PLACEHOLDER_NAMES to lowercase and apply .lower() on
  all comparisons to prevent case-variant names leaking into aliases
2026-04-03 13:15:57 +08:00
lanceyq
9cc19047b4 fix(memory): prevent cross-role alias contamination in entity dedup
- Extract user aliases from raw dialog statements instead of post-dedup
  entities to bypass merge pollution
- Add alias cross-cleaning step in _normalize_special_entity_names to
  strip AI assistant aliases from user entities before dedup
- Call clean_cross_role_aliases after second-layer dedup to handle
  historical dirty data merged from Neo4j
- Fix syntax error in prompt_utils.py (ontology_types variable assignment)
2026-04-03 12:34:04 +08:00
wxy
2e8e63878e feat(workflow): add opening statement and citation support
- Trigger opening statement on new conversation in run/run_stream
- Fix /opening endpoint to support workflow app type
- Fix features field missing in workflow config release snapshot
- Knowledge node returns citations alongside chunks
- Aggregate citations from all knowledge nodes in result builder
- Filter citations based on features.citation.enabled switch
- Fix WorkflowConfigCreate circular import in app_schema
2026-04-03 11:44:07 +08:00
Ke Sun
38955d7d45 Merge pull request #783 from SuanmoSuanyangTechnology/fix/code-none-value
fix(code-node): prevent null errors by adding default value handling
2026-04-03 11:19:58 +08:00