Release/v0.2.3 (#355)
* feat(web): add PageEmpty component
* feat(web): add PageTabs component
* feat(web): add PageEmpty component
* feat(web): add PageTabs component
* feat(prompt): add history tracking for prompt releases
* feat(web): add prompt menu
* refactor: The PageScrollList component supports two generic parameters
* feat(web): BodyWrapper compoent update PageLoading
* feat(web): add Ontology menu
* feat(web): memory management add scene
* feat(tasks): add celery task configuration for periodic jobs
- Add ignore_result=True to prevent storing results for periodic tasks
- Set max_retries=0 to skip failed periodic tasks without retry attempts
- Configure acks_late=False for immediate acknowledgment in beat tasks
- Add time_limit and soft_time_limit to regenerate_memory_cache task (3600s/3300s)
- Add time_limit and soft_time_limit to workspace_reflection_task (300s/240s)
- Add time_limit and soft_time_limit to run_forgetting_cycle_task (7200s/7000s)
- Improve task reliability and resource management for scheduled jobs
* feat(sandbox): add Node.js code execution support to sandbox
* Release/v0.2.2 (#260)
* [modify] migration script
* [add] migration script
* fix(web): change form message
* fix(web): the memoryContent field is compatible with numbers and strings
* feat(web): code node hidden
* fix(model):
1. create a basic model to check if the name and provider are duplicated.
2. The result shows error models because the provider created API Keys for all matching models.
---------
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
* Feature/ontology class clean (#249)
* [add] Complete ontology engineering feature implementation
* [add] Add ontology feature integration and validation utilities
* [add] Add OWL validator and validation utilities
* [fix] Add missing render_ontology_extraction_prompt function
* [fix]Add dependencies, fix functionality
* [add] migration script
* feat(celery): add dedicated periodic tasks worker and queue (#261)
* fix(web): conflict resolve
* Fix/v022 bug (#263)
* [fix]Fix the issue of inconsistent language in explicit and episodic memory.
* [fix]Fix the issue of inconsistent language in explicit and episodic memory.
* [add]Add scene_id
* [fix]Based on the AI review to fix the code
* Fix/develop memory reflex (#265)
* 遗漏的历史映射
* 遗漏的历史映射
* 反思后台报错处理
* [add] migration script
* fix: chat conversation_id add node_start
* feat(web): show code node
* fix(web): Restructure the CustomSelect component, repair the interface that is called multiple times when the form is updated
* feat(web): RadioGroupCard support block mode
* feat(web): create space add icon
* feat(app and model): token consumption statistics
* Add/develop memory (#264)
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 遗漏的历史映射
* 新增长期记忆功能
* 新增长期记忆功能
* 新增长期记忆功能
* 知识库检索多余字段
* 长期
* feat(app and model): token consumption statistics of the cluster
* memory_BUG_fix
* fix(web): prompt history remove pageLoading
* fix(prompt): remove hard-coded import of prompt file paths (#279)
* Fix/develop memory bug (#274)
* 遗漏的历史映射
* 遗漏的历史映射
* fix_timeline_memories
* fix(web): update retrieve_type key
* Fix/develop memory bug (#276)
* 遗漏的历史映射
* 遗漏的历史映射
* fix_timeline_memories
* fix_timeline_memories
* write_gragp/bug_fix
* write_gragp/bug_fix
* write_gragp/bug_fix
* chore(celery): disable periodic task scheduling
* fix(prompt): remove hard-coded import of prompt file paths
---------
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Ke Sun <kesun5@illinois.edu>
* fix(web): remove delete confirm content
* refactor(workflow): relocate template directory into workflow
* feat(memory): add long-term storage task routing and batching
* fix(web): PageScrollList loading update
* fix(web): PageScrollList loading update
* Ontology v1 bug (#291)
* [changes]Add 'id' as the secondary sorting key, and 'scene_id' now returns a UUID object
* [fix]Fix the "end_user" return to be sorted by update time.
* [fix]Set the default values of the memory configuration model based on the spatial model.
* [fix]Remove the entity extraction check combination model, read the configuration list, and add the return of scene_id
* [fix]Fix the "end_user" return to be sorted by update time.
* [fix]
* fix(memory): add Redis session validation
- Add macOS fork() safety configuration in celery_app.py to prevent initialization issues
- Add null/False checks for Redis session queries in term_memory_save to handle missing sessions gracefully
- Add null/False checks in memory_long_term_storage to prevent processing empty Redis results
- Add null/False checks in aggregate_judgment before format_parsing to avoid errors on missing data
- Initialize redis_messages variable in window_dialogue for consistency
- Add debug logging when no existing session found in Redis for better troubleshooting
- Add TODO comments for magic numbers (scope=6, time=5) to be extracted as constants
- Improve error handling when Redis returns False or empty results instead of crashing
* fix(web): PageScrollList style update
* fix(workflow): fix argument passing in code execution nodes
* fix(web): prompt add disabled
* fix(web): space icon required
* feat(app): modify the key of the token
* fix(fix the key of the app's token):
* fix(workflow): switch code input encoding to base64+URL encoding
* [add]The main project adds multi-API Key load balancing.
* [changes]Attribute security access, secure numerical conversion, unified use of local variables
* fix(web): save add session update
* fix(web): language editor support paste
* [changes]Active status filtering logic, API Key selection strategy
* memory_BUG
* memory_BUG_long_term
* [changes]
* memory_BUG_long_term
* memory_BUG_long_term
* Fix/release memory bug (#306)
* memory_BUG_fix
* memory_BUG
* memory_BUG_long_term
* memory_BUG_long_term
* memory_BUG_long_term
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* [fix]1.The "read_all_config" interface returns "scene_name";2.Memory configuration for lightweight query ontology scenarios
* fix(web): replace code editor
* [changes]Modify the description of the time for the recent event
* [changes]Modify the code based on the AI review
* feat(web): update memory config ontology api
* fix(web): ui update
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* knowledge_retrieval/bug/fix
* feat(workflow): add token usage statistics for question classifier and parameter extraction
* feat(web): move prompt menu
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Multiple independent transactions - single transaction
* Write Missing None (#321)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/release memory bug (#324)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/writer memory bug (#326)
* [fix]Fix the bug
* [fix]Fix the bug
* [fix]Correct the direction indication.
* fix(web): markdown table ui update
* Fix/release memory bug (#332)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
* writer_dup_bug/fix
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Fix/fact summary (#333)
* [fix]Disable the contents related to fact_summary
* [fix]Disable the contents related to fact_summary
* [fix]Modify the code based on the AI review
* Fix/release memory bug (#335)
* Write Missing None
* Write Missing None
* Write Missing None
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Write Missing None
* redis update
* redis update
* redis update
* redis update
* writer_dup_bug/fix
* writer_graph_bug/fix
* writer_graph_bug/fix
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Revert "feat(web): move prompt menu"
This reverts commit 9e6e8f50f8.
* fix(web): ui update
* fix(web): update text
* fix(web): ui update
* fix(model): change the "vl" model type of dashscope to "chat"
* fix(model): change the "vl" model type of dashscope to "chat"
---------
Co-authored-by: zhaoying <yzhao96@best-inc.com>
Co-authored-by: Eternity <1533512157@qq.com>
Co-authored-by: Mark <zhuwenhui5566@163.com>
Co-authored-by: yingzhao <zhaoyingyz@126.com>
Co-authored-by: Timebomb2018 <18868801967@163.com>
Co-authored-by: 乐力齐 <162269739+lanceyq@users.noreply.github.com>
Co-authored-by: lixinyue11 <94037597+lixinyue11@users.noreply.github.com>
Co-authored-by: lixinyue <2569494688@qq.com>
Co-authored-by: Eternity <61316157+myhMARS@users.noreply.github.com>
Co-authored-by: lanceyq <1982376970@qq.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
This commit is contained in:
@@ -1 +1,40 @@
|
||||
"""Code runners package"""
|
||||
import pwd
|
||||
import subprocess
|
||||
|
||||
from app.config import get_config
|
||||
from app.logger import get_logger
|
||||
|
||||
logger = get_logger()
|
||||
|
||||
|
||||
def init_sandbox_user():
|
||||
config = get_config()
|
||||
sandbox_user = config.sandbox_user
|
||||
sandbox_uid = config.sandbox_uid
|
||||
try:
|
||||
pwd.getpwnam(sandbox_user)
|
||||
logger.info(f"User '{sandbox_user}' already exists")
|
||||
except KeyError:
|
||||
try:
|
||||
subprocess.run(
|
||||
["useradd", "-u", str(sandbox_uid), sandbox_user],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True
|
||||
)
|
||||
logger.info(f"Created user '{sandbox_user}' with UID {sandbox_uid}")
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.error(f"Failed to create user: {e.stderr}")
|
||||
raise RuntimeError(f"Failed to create user '{sandbox_user}': {e.stderr}") from e
|
||||
|
||||
try:
|
||||
user_info = pwd.getpwnam(sandbox_user)
|
||||
config.set_sandbox_gid(user_info.pw_gid)
|
||||
logger.info(f"Sandbox user GID: {config.sandbox_gid}")
|
||||
except KeyError as e:
|
||||
logger.error(f"Failed to get GID for user '{sandbox_user}'")
|
||||
raise RuntimeError(f"Failed to get GID for user '{sandbox_user}'") from e
|
||||
|
||||
|
||||
|
||||
|
||||
3
sandbox/app/core/runners/nodejs/__init__.py
Normal file
3
sandbox/app/core/runners/nodejs/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
from app.core.runners.nodejs.env import release_lib_binary
|
||||
|
||||
release_lib_binary(True)
|
||||
124
sandbox/app/core/runners/nodejs/env.py
Normal file
124
sandbox/app/core/runners/nodejs/env.py
Normal file
@@ -0,0 +1,124 @@
|
||||
import asyncio
|
||||
import ctypes
|
||||
import os
|
||||
import shutil
|
||||
import stat
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from app.logger import get_logger
|
||||
from app.config import get_config
|
||||
|
||||
logger = get_logger()
|
||||
|
||||
RELEASE_LIB_PATH = "./lib/seccomp_redbear/target/release/libnodejs.so"
|
||||
LIB_PATH = "/var/sandbox/sandbox-nodejs"
|
||||
LIB_NAME = "libnodejs.so"
|
||||
|
||||
lib = ctypes.CDLL(RELEASE_LIB_PATH)
|
||||
lib.get_lib_version_static.restype = ctypes.c_char_p
|
||||
lib.get_lib_feature_static.restype = ctypes.c_char_p
|
||||
logger.info(f"Seccomp Env: nodejs, "
|
||||
f"Seccomp Feature: {lib.get_lib_feature_static().decode('utf-8')}, "
|
||||
f"Seccomp Version: {lib.get_lib_version_static().decode('utf-8')}")
|
||||
|
||||
try:
|
||||
with open(RELEASE_LIB_PATH, "rb") as f:
|
||||
_NODEJS_LIB = f.read()
|
||||
except:
|
||||
logger.critical("failed to load nodejs lib")
|
||||
raise
|
||||
|
||||
|
||||
def check_lib_avaiable():
|
||||
return os.path.exists(os.path.join(LIB_PATH, LIB_NAME))
|
||||
|
||||
|
||||
def release_lib_binary(force_remove: bool):
|
||||
logger.info("init runtime enviroment")
|
||||
|
||||
lib_file = os.path.join(LIB_PATH, LIB_NAME)
|
||||
if os.path.exists(lib_file):
|
||||
if force_remove:
|
||||
try:
|
||||
os.remove(lib_file)
|
||||
except OSError:
|
||||
logger.critical(f"failed to remove {os.path.join(LIB_PATH, LIB_NAME)}")
|
||||
raise
|
||||
|
||||
try:
|
||||
os.makedirs(LIB_PATH, mode=0o755, exist_ok=True)
|
||||
except OSError:
|
||||
logger.critical(f"failed to create {LIB_PATH}")
|
||||
raise
|
||||
|
||||
try:
|
||||
with open(lib_file, "wb") as f:
|
||||
f.write(_NODEJS_LIB)
|
||||
os.chmod(lib_file, 0o755)
|
||||
except OSError:
|
||||
logger.critical(f"failed to write {lib_file}")
|
||||
raise
|
||||
else:
|
||||
try:
|
||||
os.makedirs(LIB_PATH, mode=0o755, exist_ok=True)
|
||||
except OSError:
|
||||
logger.critical(f"failed to create {LIB_PATH}")
|
||||
raise
|
||||
|
||||
try:
|
||||
with open(lib_file, "wb") as f:
|
||||
f.write(_NODEJS_LIB)
|
||||
os.chmod(lib_file, 0o755)
|
||||
except OSError:
|
||||
logger.critical(f"failed to write {lib_file}")
|
||||
raise
|
||||
|
||||
logger.info("nodejs runner environment initialized")
|
||||
|
||||
|
||||
async def prepare_nodejs_dependencies_env():
|
||||
config = get_config()
|
||||
|
||||
with tempfile.TemporaryDirectory(dir="/") as root_path:
|
||||
root = Path(root_path)
|
||||
|
||||
env_sh = root / "env.sh"
|
||||
with open("script/env.sh") as f:
|
||||
env_sh.write_text(f.read())
|
||||
env_sh.chmod(env_sh.stat().st_mode | stat.S_IXUSR)
|
||||
|
||||
shutil.copytree("dependencies/nodejs", os.path.join(LIB_PATH, "node_temp"), dirs_exist_ok=True)
|
||||
for root, dirs, files in os.walk(os.path.join(LIB_PATH, "node_temp")):
|
||||
for d in dirs:
|
||||
os.chmod(os.path.join(root, d), 0o755)
|
||||
for f in files:
|
||||
os.chmod(os.path.join(root, f), 0o444)
|
||||
|
||||
for lib_path in config.nodejs_lib_paths:
|
||||
lib_path = Path(lib_path)
|
||||
|
||||
if not lib_path.exists():
|
||||
logger.warning("nodejs lib path %s is not available", lib_path)
|
||||
continue
|
||||
|
||||
cmd = [
|
||||
"bash",
|
||||
str(env_sh),
|
||||
str(lib_path),
|
||||
str(LIB_PATH),
|
||||
]
|
||||
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE
|
||||
)
|
||||
|
||||
stdout, stderr = await process.communicate()
|
||||
retcode = process.returncode
|
||||
|
||||
if retcode != 0:
|
||||
logger.error(
|
||||
f"create env error for file {lib_path}: retcode={retcode}, stderr={stderr.decode()}"
|
||||
)
|
||||
138
sandbox/app/core/runners/nodejs/nodejs_runner.py
Normal file
138
sandbox/app/core/runners/nodejs/nodejs_runner.py
Normal file
@@ -0,0 +1,138 @@
|
||||
"""Nodejs code runner"""
|
||||
import asyncio
|
||||
import os
|
||||
import uuid
|
||||
from typing import Optional
|
||||
|
||||
from app.core.executor import CodeExecutor, ExecutionResult
|
||||
from app.core.runners.nodejs.env import check_lib_avaiable, release_lib_binary, LIB_PATH
|
||||
from app.logger import get_logger
|
||||
from app.models import RunnerOptions
|
||||
|
||||
# Nodejs sandbox prescript template
|
||||
with open("app/core/runners/nodejs/prescript.js") as f:
|
||||
NODEJS_PRESCRIPT = f.read()
|
||||
|
||||
logger = get_logger()
|
||||
|
||||
|
||||
class NodejsRunner(CodeExecutor):
|
||||
"""Node.js code runner with security isolation"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
|
||||
@staticmethod
|
||||
def init_environment(code: str, preload: str) -> str:
|
||||
if not check_lib_avaiable():
|
||||
release_lib_binary(False)
|
||||
code_file_name = uuid.uuid4().hex.replace("-", "_")
|
||||
|
||||
script = NODEJS_PRESCRIPT.replace("{{preload}}", preload, 1)
|
||||
|
||||
eval_code = f"eval(Buffer.from('{code}', 'base64').toString('utf-8'))"
|
||||
script = script.replace("{{code}}", eval_code, 1)
|
||||
|
||||
code_path = f"{LIB_PATH}/node_temp/tmp/{code_file_name}.js"
|
||||
try:
|
||||
os.makedirs(os.path.dirname(code_path), mode=0o755, exist_ok=True)
|
||||
with open(code_path, "w", encoding="utf-8") as f:
|
||||
f.write(script)
|
||||
os.chmod(code_path, 0o755)
|
||||
|
||||
except OSError as e:
|
||||
raise RuntimeError(f"Failed to write {code_path}") from e
|
||||
|
||||
return code_path
|
||||
|
||||
async def run(
|
||||
self,
|
||||
code: str,
|
||||
options: RunnerOptions,
|
||||
preload: str = "",
|
||||
timeout: Optional[int] = None
|
||||
) -> ExecutionResult:
|
||||
"""Run Python code in sandbox
|
||||
|
||||
Args:
|
||||
options:
|
||||
code: Base64 encoded encrypted code
|
||||
preload: Preload code to execute before main code
|
||||
timeout: Execution timeout in seconds
|
||||
|
||||
Returns:
|
||||
ExecutionResult with stdout, stderr, and exit code
|
||||
"""
|
||||
config = self.config
|
||||
|
||||
if timeout is None:
|
||||
timeout = config.worker_timeout
|
||||
|
||||
# Check if preload is allowed
|
||||
if not preload or not config.enable_preload:
|
||||
preload = ""
|
||||
script_path = self.init_environment(code, preload)
|
||||
|
||||
try:
|
||||
# Setup environment
|
||||
env = {
|
||||
"UV_USE_IO_URING": "0"
|
||||
}
|
||||
|
||||
# Add proxy settings if configured
|
||||
if config.proxy.socks5:
|
||||
env["HTTPS_PROXY"] = config.proxy.socks5
|
||||
env["HTTP_PROXY"] = config.proxy.socks5
|
||||
elif config.proxy.https or config.proxy.http:
|
||||
if config.proxy.https:
|
||||
env["HTTPS_PROXY"] = config.proxy.https
|
||||
if config.proxy.http:
|
||||
env["HTTP_PROXY"] = config.proxy.http
|
||||
|
||||
# Add allowed syscalls if configured
|
||||
if config.allowed_syscalls:
|
||||
env["ALLOWED_SYSCALLS"] = ",".join(map(str, config.allowed_syscalls))
|
||||
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
config.nodejs_path,
|
||||
script_path,
|
||||
LIB_PATH,
|
||||
str(config.sandbox_uid),
|
||||
str(config.sandbox_gid),
|
||||
options.model_dump_json(),
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
env=env,
|
||||
cwd=LIB_PATH
|
||||
)
|
||||
|
||||
# Wait for completion with timeout
|
||||
try:
|
||||
stdout, stderr = await asyncio.wait_for(
|
||||
process.communicate(),
|
||||
timeout=timeout
|
||||
)
|
||||
|
||||
return ExecutionResult(
|
||||
stdout=stdout.decode('utf-8', errors='replace'),
|
||||
stderr=stderr.decode('utf-8', errors='replace'),
|
||||
exit_code=process.returncode
|
||||
)
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
# Kill process on timeout
|
||||
try:
|
||||
process.kill()
|
||||
await process.wait()
|
||||
except:
|
||||
pass
|
||||
|
||||
return ExecutionResult(
|
||||
stdout="",
|
||||
stderr="Execution timeout",
|
||||
exit_code=-1,
|
||||
)
|
||||
|
||||
finally:
|
||||
# Cleanup temporary file
|
||||
self.cleanup_temp_file(script_path)
|
||||
31
sandbox/app/core/runners/nodejs/prescript.js
Normal file
31
sandbox/app/core/runners/nodejs/prescript.js
Normal file
@@ -0,0 +1,31 @@
|
||||
let argv = process.argv
|
||||
|
||||
let koffi = require('koffi')
|
||||
|
||||
process.chdir(argv[2])
|
||||
|
||||
let lib = koffi.load("./libnodejs.so")
|
||||
/** @type {(uid: number, gid: number, enableNetwork: boolean) => number} */
|
||||
let initSeccomp = lib.func('int init_seccomp(int, int, bool)')
|
||||
|
||||
let uid = parseInt(argv[3])
|
||||
let gid = parseInt(argv[4])
|
||||
|
||||
let options = JSON.parse(argv[5])
|
||||
|
||||
let seccomp_init = initSeccomp(uid, gid, options['enable_network'])
|
||||
if (seccomp_init !== 0) {
|
||||
throw `code executor err - ${seccomp_init}`
|
||||
}
|
||||
|
||||
delete process.argv
|
||||
argv = undefined
|
||||
koffi = undefined
|
||||
lib = undefined
|
||||
initSeccomp = undefined
|
||||
uid = undefined
|
||||
gid = undefined
|
||||
options = undefined
|
||||
seccomp_init = undefined
|
||||
|
||||
{{code}}
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: UTF-8 -*-
|
||||
# Author: Eternity
|
||||
# @Email: 1533512157@qq.com
|
||||
# @Time : 2026/1/23 11:27
|
||||
from app.core.runners.python.env import release_lib_binary
|
||||
|
||||
release_lib_binary(True)
|
||||
|
||||
@@ -1,14 +1,80 @@
|
||||
import asyncio
|
||||
import tempfile
|
||||
import ctypes
|
||||
import os
|
||||
import stat
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from app.config import get_config
|
||||
from app.core.runners.python.settings import LIB_PATH
|
||||
from app.logger import get_logger
|
||||
|
||||
logger = get_logger()
|
||||
|
||||
RELEASE_LIB_PATH = "./lib/seccomp_redbear/target/release/libpython.so"
|
||||
LIB_PATH = "/var/sandbox/sandbox-python"
|
||||
LIB_NAME = "libpython.so"
|
||||
|
||||
lib = ctypes.CDLL(RELEASE_LIB_PATH)
|
||||
lib.get_lib_version_static.restype = ctypes.c_char_p
|
||||
lib.get_lib_feature_static.restype = ctypes.c_char_p
|
||||
logger.info(f"Seccomp Env: python3, "
|
||||
f"Seccomp Feature: {lib.get_lib_feature_static().decode('utf-8')}, "
|
||||
f"Seccomp Version: {lib.get_lib_version_static().decode('utf-8')}")
|
||||
|
||||
try:
|
||||
with open(RELEASE_LIB_PATH, "rb") as f:
|
||||
_PYTHON_LIB = f.read()
|
||||
except:
|
||||
logger.critical("failed to load python lib")
|
||||
raise
|
||||
|
||||
|
||||
def check_lib_avaiable():
|
||||
return os.path.exists(os.path.join(LIB_PATH, LIB_NAME))
|
||||
|
||||
|
||||
def release_lib_binary(force_remove: bool):
|
||||
logger.info("init runtime enviroment")
|
||||
|
||||
lib_file = os.path.join(LIB_PATH, LIB_NAME)
|
||||
if os.path.exists(lib_file):
|
||||
if force_remove:
|
||||
try:
|
||||
os.remove(lib_file)
|
||||
except OSError:
|
||||
logger.critical(f"failed to remove {os.path.join(LIB_PATH, LIB_NAME)}")
|
||||
raise
|
||||
|
||||
try:
|
||||
os.makedirs(LIB_PATH, mode=0o755, exist_ok=True)
|
||||
except OSError:
|
||||
logger.critical(f"failed to create {LIB_PATH}")
|
||||
raise
|
||||
|
||||
try:
|
||||
with open(lib_file, "wb") as f:
|
||||
f.write(_PYTHON_LIB)
|
||||
os.chmod(lib_file, 0o755)
|
||||
except OSError:
|
||||
logger.critical(f"failed to write {lib_file}")
|
||||
raise
|
||||
else:
|
||||
try:
|
||||
os.makedirs(LIB_PATH, mode=0o755, exist_ok=True)
|
||||
except OSError:
|
||||
logger.critical(f"failed to create {LIB_PATH}")
|
||||
raise
|
||||
|
||||
try:
|
||||
with open(lib_file, "wb") as f:
|
||||
f.write(_PYTHON_LIB)
|
||||
os.chmod(lib_file, 0o755)
|
||||
except OSError:
|
||||
logger.critical(f"failed to write {lib_file}")
|
||||
raise
|
||||
|
||||
logger.info("python runner environment initialized")
|
||||
|
||||
|
||||
async def prepare_python_dependencies_env():
|
||||
config = get_config()
|
||||
|
||||
@@ -17,7 +17,7 @@ sys.excepthook = excepthook
|
||||
# Load security library if available
|
||||
lib = ctypes.CDLL("./libpython.so")
|
||||
lib.init_seccomp.argtypes = [ctypes.c_uint32, ctypes.c_uint32, ctypes.c_bool]
|
||||
lib.init_seccomp.restype = None # TODO: raise error info
|
||||
lib.init_seccomp.restype = ctypes.c_int
|
||||
|
||||
# Get running path
|
||||
running_path = sys.argv[1]
|
||||
@@ -37,7 +37,10 @@ os.chdir(running_path)
|
||||
{{preload}}
|
||||
|
||||
# Apply security if library is available
|
||||
lib.init_seccomp({{uid}}, {{gid}}, {{enable_network}})
|
||||
init_status = lib.init_seccomp({{uid}}, {{gid}}, {{enable_network}})
|
||||
if init_status != 0:
|
||||
raise Exception(f"code executor err - {str(init_status)}")
|
||||
del lib
|
||||
|
||||
# Decrypt and execute code
|
||||
code = b64decode("{{code}}")
|
||||
|
||||
@@ -5,10 +5,10 @@ import os
|
||||
import uuid
|
||||
from typing import Optional
|
||||
|
||||
from app.config import SANDBOX_USER_ID, SANDBOX_GROUP_ID, get_config
|
||||
from app.config import get_config
|
||||
from app.core.encryption import generate_key, encrypt_code
|
||||
from app.core.executor import CodeExecutor, ExecutionResult
|
||||
from app.core.runners.python.settings import check_lib_avaiable, release_lib_binary, LIB_PATH
|
||||
from app.core.runners.python.env import check_lib_avaiable, release_lib_binary, LIB_PATH
|
||||
from app.logger import get_logger
|
||||
from app.models import RunnerOptions
|
||||
|
||||
@@ -32,8 +32,8 @@ class PythonRunner(CodeExecutor):
|
||||
config = get_config()
|
||||
code_file_name = uuid.uuid4().hex.replace("-", "_")
|
||||
|
||||
script = PYTHON_PRESCRIPT.replace("{{uid}}", str(SANDBOX_USER_ID), 1)
|
||||
script = script.replace("{{gid}}", str(SANDBOX_GROUP_ID), 1)
|
||||
script = PYTHON_PRESCRIPT.replace("{{uid}}", str(config.sandbox_uid), 1)
|
||||
script = script.replace("{{gid}}", str(config.sandbox_gid), 1)
|
||||
script = script.replace(
|
||||
"{{enable_network}}",
|
||||
str(int(options.enable_network and config.enable_network)
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
import os
|
||||
|
||||
from app.logger import get_logger
|
||||
|
||||
logger = get_logger()
|
||||
|
||||
RELEASE_LIB_PATH = "./lib/seccomp_python/target/release/libpython.so"
|
||||
LIB_PATH = "/var/sandbox/sandbox-python"
|
||||
LIB_NAME = "libpython.so"
|
||||
|
||||
try:
|
||||
with open(RELEASE_LIB_PATH, "rb") as f:
|
||||
_PYTHON_LIB = f.read()
|
||||
except:
|
||||
logger.critical("failed to load python lib")
|
||||
raise
|
||||
|
||||
|
||||
def check_lib_avaiable():
|
||||
return os.path.exists(os.path.join(LIB_PATH, LIB_NAME))
|
||||
|
||||
|
||||
def release_lib_binary(force_remove: bool):
|
||||
logger.info("init runtime enviroment")
|
||||
lib_file = os.path.join(LIB_PATH, LIB_NAME)
|
||||
if os.path.exists(lib_file):
|
||||
if force_remove:
|
||||
try:
|
||||
os.remove(lib_file)
|
||||
except OSError:
|
||||
logger.critical(f"failed to remove {os.path.join(LIB_PATH, LIB_NAME)}")
|
||||
raise
|
||||
|
||||
try:
|
||||
os.makedirs(LIB_PATH, mode=0o755, exist_ok=True)
|
||||
except OSError:
|
||||
logger.critical(f"failed to create {LIB_PATH}")
|
||||
raise
|
||||
|
||||
try:
|
||||
with open(lib_file, "wb") as f:
|
||||
f.write(_PYTHON_LIB)
|
||||
os.chmod(lib_file, 0o755)
|
||||
except OSError:
|
||||
logger.critical(f"failed to write {lib_file}")
|
||||
raise
|
||||
else:
|
||||
try:
|
||||
os.makedirs(LIB_PATH, mode=0o755, exist_ok=True)
|
||||
except OSError:
|
||||
logger.critical(f"failed to create {LIB_PATH}")
|
||||
raise
|
||||
|
||||
try:
|
||||
with open(lib_file, "wb") as f:
|
||||
f.write(_PYTHON_LIB)
|
||||
os.chmod(lib_file, 0o755)
|
||||
except OSError:
|
||||
logger.critical(f"failed to write {lib_file}")
|
||||
raise
|
||||
|
||||
logger.info("python runner environment initialized")
|
||||
Reference in New Issue
Block a user