Commit 846639535 for llama.cpp
commit 84663953535d4323e92b4bc90971aa022c305d06
Author: Max Krasnyansky <maxk@qti.qualcomm.com>
Date: Mon Oct 19 23:49:59 2026 -0700
hexagon: update to snapdragon-toolchain v0.8 and enable hexagon v85 (#30854)
* hex-build: bump toolchain, enable v85
* hex-build: bump toolchain to v0.8 in the CI and build scripts
* hex-build: add a little python script for swiv needed on some iot devices
* hex-build: fix flake8 warnings
* hex-arch: update version cap to v85
* hex-inspect: flexible TOOLS_ROOT
* hex-build: update to snake_case naming
* hex-build: update leftover old toolchain refs
* hex-target: update build and run scripts to support ubuntu with adb transport
diff --git a/.github/workflows/build-and-test-snapdragon.yml b/.github/workflows/build-and-test-snapdragon.yml
index 29be95c8b..f7f94d7ee 100644
--- a/.github/workflows/build-and-test-snapdragon.yml
+++ b/.github/workflows/build-and-test-snapdragon.yml
@@ -35,7 +35,7 @@ jobs:
android-ndk-snapdragon:
runs-on: ubuntu-24.04 # previously ubuntu-latest
container:
- image: 'ghcr.io/snapdragon-toolchain/arm64-android:v0.7'
+ image: 'ghcr.io/snapdragon-toolchain/arm64-android:v0.8'
defaults:
run:
shell: bash
@@ -69,7 +69,7 @@ jobs:
linux-iot-snapdragon:
runs-on: ubuntu-24.04 # previously ubuntu-latest
container:
- image: 'ghcr.io/snapdragon-toolchain/arm64-linux:v0.7'
+ image: 'ghcr.io/snapdragon-toolchain/arm64-linux:v0.8'
defaults:
run:
shell: bash
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index 6f828b94b..c43c609ad 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -547,7 +547,7 @@ jobs:
if: ${{ needs.check-release.outputs.should_release == 'true' }}
runs-on: ubuntu-latest
- container: 'ghcr.io/snapdragon-toolchain/arm64-android:v0.7'
+ container: 'ghcr.io/snapdragon-toolchain/arm64-android:v0.8'
defaults:
run:
@@ -600,7 +600,7 @@ jobs:
if: ${{ needs.check-release.outputs.should_release == 'true' }}
runs-on: ubuntu-latest
- container: 'ghcr.io/snapdragon-toolchain/arm64-linux:v0.7'
+ container: 'ghcr.io/snapdragon-toolchain/arm64-linux:v0.8'
defaults:
run:
diff --git a/docs/backend/snapdragon/README.md b/docs/backend/snapdragon/README.md
index b8347a224..f0682ce7a 100644
--- a/docs/backend/snapdragon/README.md
+++ b/docs/backend/snapdragon/README.md
@@ -6,8 +6,8 @@ The cross-compilation toolchain images are provided by the
[Qualcomm Snapdragon Toolchain registry](https://github.com/snapdragon-toolchain).
These Docker images include the Android NDK, OpenCL SDK, Hexagon SDK, CMake, and the necessary cross-compilers:
-* **Android toolchain**: `ghcr.io/snapdragon-toolchain/arm64-android:v0.7`
-* **Linux toolchain**: `ghcr.io/snapdragon-toolchain/arm64-linux:v0.7`
+* **Android toolchain**: `ghcr.io/snapdragon-toolchain/arm64-android:v0.8`
+* **Linux toolchain**: `ghcr.io/snapdragon-toolchain/arm64-linux:v0.8`
The unified build utility (`scripts/snapdragon/build.py`) automatically pulls
and orchestrates these containers to perform target compilation.
@@ -40,7 +40,7 @@ Alternatively, you can build llama.cpp manually by entering the cross-compilatio
```bash
# Start the cross-compilation container manually:
-~/src/llama.cpp$ docker run -it --rm -u $(id -u):$(id -g) --volume $(pwd):/workspace --platform linux/amd64 ghcr.io/snapdragon-toolchain/arm64-android:v0.7
+~/src/llama.cpp$ docker run -it --rm -u $(id -u):$(id -g) --volume $(pwd):/workspace --platform linux/amd64 ghcr.io/snapdragon-toolchain/arm64-android:v0.8
# Inside the container, build the project using presets:
[d]/workspace> cp docs/backend/snapdragon/CMakeUserPresets.json .
@@ -53,7 +53,7 @@ Preset CMake variables:
GGML_HEXAGON="ON"
GGML_OPENCL="ON"
GGML_OPENMP="OFF"
- HEXAGON_SDK_ROOT="/opt/hexagon/6.6.0.0"
+ HEXAGON_SDK_ROOT="/opt/hexagon/6.7.0.0"
...
-- Including OpenCL backend
-- Including Hexagon backend
diff --git a/docs/backend/snapdragon/linux.md b/docs/backend/snapdragon/linux.md
index d4ecd9b1b..20cc4817a 100644
--- a/docs/backend/snapdragon/linux.md
+++ b/docs/backend/snapdragon/linux.md
@@ -3,7 +3,7 @@
The cross-compilation is performed using the Snapdragon Linux Docker toolchain image (see
[github.com/snapdragon-toolchain](https://github.com/snapdragon-toolchain)):
-* **Linux toolchain**: `ghcr.io/snapdragon-toolchain/arm64-linux:v0.7`
+* **Linux toolchain**: `ghcr.io/snapdragon-toolchain/arm64-linux:v0.8`
The unified build utility (`scripts/snapdragon/build.py`) automatically pulls
and orchestrates this container to perform target compilation. You only need to
@@ -29,7 +29,7 @@ Alternatively, you can build llama.cpp manually by entering the cross-compilatio
```bash
# Start the cross-compilation container manually:
-~/src/llama.cpp$ docker run -it --rm -u $(id -u):$(id -g) --volume $(pwd):/workspace --platform linux/amd64 ghcr.io/snapdragon-toolchain/arm64-linux:v0.7
+~/src/llama.cpp$ docker run -it --rm -u $(id -u):$(id -g) --volume $(pwd):/workspace --platform linux/amd64 ghcr.io/snapdragon-toolchain/arm64-linux:v0.8
# Inside the container, build the project using presets:
[d]/workspace> cp docs/backend/snapdragon/CMakeUserPresets.json .
diff --git a/docs/backend/snapdragon/windows.md b/docs/backend/snapdragon/windows.md
index 886cfda3f..8e768491a 100644
--- a/docs/backend/snapdragon/windows.md
+++ b/docs/backend/snapdragon/windows.md
@@ -8,7 +8,7 @@ Native Windows 11 arm64 builds have the following tool dependencies:
- UCRT and Driver Kit
- LLVM core libraries and Clang compiler (winget)
- CMake, Git, Python (winget)
-- Hexagon SDK Community Edition 6.6 or later (see below)
+- Hexagon SDK Community Edition 6.7 or later (see below)
- OpenCL SDK 2.3 or later (see below)
Note: The rest of the **Windows** build process assumes that you're running natively in Powershell.
@@ -54,15 +54,15 @@ c:\Qualcomm\OpenCL_SDK\2.3.2
Either use the trimmed down version (optimized for CI) from
- https://github.com/snapdragon-toolchain/hexagon-sdk/releases/download/v6.6.0.0/hexagon-sdk-v6.6.0.0-arm64-wos.tar.xz
+ https://github.com/snapdragon-toolchain/hexagon-sdk/releases/download/v6.7.0.0/hexagon-sdk-v6.7.0.0-arm64-wos.tar.xz
Or download the complete official version from
- https://softwarecenter.qualcomm.com/catalog/item/Hexagon_SDK?version=6.6.0.0
+ https://softwarecenter.qualcomm.com/catalog/item/Hexagon_SDK?version=6.7.0.0
Unzip/untar the archive into
```
-c:\Qualcomm\Hexagon_SDK\6.6.0.0
+c:\Qualcomm\Hexagon_SDK\6.7.0.0
```
## Install the latest Adreno GPU driver
@@ -150,8 +150,8 @@ The overall Hexagon backend build procedure for Windows on Snapdragon is the sam
However, additional settings are required for generating and signing HTP Ops libraries.
```
> $env:OPENCL_SDK_ROOT="C:\Qualcomm\OpenCL_SDK\2.3.2"
-> $env:HEXAGON_SDK_ROOT="C:\Qualcomm\Hexagon_SDK\6.6.0.0"
-> $env:HEXAGON_TOOLS_ROOT="C:\Qualcomm\Hexagon_SDK\6.6.0.0\tools\HEXAGON_Tools\19.0.07"
+> $env:HEXAGON_SDK_ROOT="C:\Qualcomm\Hexagon_SDK\6.7.0.0"
+> $env:HEXAGON_TOOLS_ROOT="C:\Qualcomm\Hexagon_SDK\6.7.0.0\tools\HEXAGON_Tools\21.0.04"
> $env:HEXAGON_HTP_CERT="c:\Users\MyUsers\Certs\ggml-htp-v1.pfx"
> $env:WINDOWS_SDK_BIN="C:\Program Files (x86)\Windows Kits\10\bin\10.0.26100.0"
diff --git a/ggml/src/ggml-hexagon/CMakeLists.txt b/ggml/src/ggml-hexagon/CMakeLists.txt
index 277a4c863..6ab520175 100644
--- a/ggml/src/ggml-hexagon/CMakeLists.txt
+++ b/ggml/src/ggml-hexagon/CMakeLists.txt
@@ -21,6 +21,7 @@ message(STATUS "hexagon: using ${HEXAGON_SDK_ROOT} and ${HEXAGON_TOOLS_ROOT} for
include(${HEXAGON_SDK_ROOT}/build/cmake/hexagon_fun.cmake)
include(ExternalProject)
+find_package(Python3 REQUIRED COMPONENTS Interpreter)
option(GGML_HEXAGON_HTP_DEBUG "ggml-hexagon: enable HTP debug output" OFF)
set(GGML_HEXAGON_HTP_BUILD_TYPE "Release" CACHE STRING "ggml-hexagon: HTP skel build type (Release, RelWithDebInfo, Debug)")
@@ -74,7 +75,8 @@ function(build_htp_skel V)
-DHEXAGON_TOOLS_ROOT=${HEXAGON_TOOLS_ROOT}
-DHEXAGON_HTP_DEBUG=${GGML_HEXAGON_HTP_DEBUG}
-DDSP_VERSION=${V}
- -DPREBUILT_LIB_DIR="toolv19_${V}")
+ -DPython3_EXECUTABLE=${Python3_EXECUTABLE}
+ -DPREBUILT_LIB_DIR="toolv21_${V}")
list(APPEND HTP_SKELS ${CMAKE_CURRENT_BINARY_DIR}/libggml-htp-${V}.so)
list(APPEND HTP_PROJECTS htp-${V})
set(HTP_SKELS ${HTP_SKELS} PARENT_SCOPE)
@@ -85,6 +87,7 @@ build_htp_skel(v73)
build_htp_skel(v75)
build_htp_skel(v79)
build_htp_skel(v81)
+build_htp_skel(v85)
# Install Hexagon skels required at runtime
install(FILES ${HTP_SKELS} TYPE LIB)
diff --git a/ggml/src/ggml-hexagon/ggml-hexagon.cpp b/ggml/src/ggml-hexagon/ggml-hexagon.cpp
index 2d69ac47e..da6dba85e 100644
--- a/ggml/src/ggml-hexagon/ggml-hexagon.cpp
+++ b/ggml/src/ggml-hexagon/ggml-hexagon.cpp
@@ -8399,9 +8399,9 @@ static void ggml_hexagon_init(ggml_backend_reg * reg) {
if (opt_arch < 73) {
GGML_LOG_WARN("ggml-hex: Hexagon arch v%d is under supported range, capping at v73\n", opt_arch);
opt_arch = 73;
- } else if (opt_arch > 81) {
- GGML_LOG_WARN("ggml-hex: Hexagon arch v%d is over supported range, capping at v81\n", opt_arch);
- opt_arch = 81;
+ } else if (opt_arch > 85) {
+ GGML_LOG_WARN("ggml-hex: Hexagon arch v%d is over supported range, capping at v85\n", opt_arch);
+ opt_arch = 85;
}
}
} else {
diff --git a/ggml/src/ggml-hexagon/htp/CMakeLists.txt b/ggml/src/ggml-hexagon/htp/CMakeLists.txt
index b1131317f..28fae4fe8 100644
--- a/ggml/src/ggml-hexagon/htp/CMakeLists.txt
+++ b/ggml/src/ggml-hexagon/htp/CMakeLists.txt
@@ -53,9 +53,17 @@ target_compile_definitions(${HTP_LIB} PRIVATE
$<IF:$<BOOL:${HEXAGON_HTP_DEBUG}>,HTP_DEBUG=1,NDEBUG=1>
$<IF:$<BOOL:${HEXAGON_HTP_DEBUG}>,FARF_HIGH=1,>)
-
build_idl(htp_iface.idl ${HTP_LIB})
set_target_properties(${HTP_LIB} PROPERTIES EXPORT_COMPILE_COMMANDS ON)
+if (NOT Python3_EXECUTABLE)
+ find_package(Python3 REQUIRED COMPONENTS Interpreter)
+endif()
+
+add_custom_command(TARGET ${HTP_LIB} POST_BUILD
+ COMMAND ${CMAKE_COMMAND} -E echo "Adding SWIV section $<TARGET_FILE_NAME:${HTP_LIB}>"
+ COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/htp-swiv.py -i $<TARGET_FILE:${HTP_LIB}> -o $<TARGET_FILE:${HTP_LIB}>
+ VERBATIM)
+
install(TARGETS ${HTP_LIB})
diff --git a/ggml/src/ggml-hexagon/htp/htp-swiv.py b/ggml/src/ggml-hexagon/htp/htp-swiv.py
new file mode 100644
index 000000000..15d7bef27
--- /dev/null
+++ b/ggml/src/ggml-hexagon/htp/htp-swiv.py
@@ -0,0 +1,512 @@
+#!/usr/bin/env python3
+
+import argparse
+import logging
+import os
+import shutil
+import struct
+import sys
+import tempfile
+import zlib
+
+ELF_MAGIC = b'\x7fELF'
+ELF_CLASS_INDEX = 4
+ELFCLASS32 = 1
+ELFCLASS64 = 2
+ELF_VERSION_INDEX = 6
+EV_CURRENT = 1
+MAX_PHDR_COUNT = 100
+
+LOAD_TYPE = 0x1
+
+SHT_NOBITS = 8
+
+SWIV_MAGIC = b'SWIV'
+SWIV_SEG_SIZE_MIN = 16
+# SHT_LOUSER plus ASCII "SWIV".
+SWIV_SECTION_TYPE = 0xD3574956
+
+logger = logging.getLogger("htp-swiv")
+
+
+class Elf_Ehdr_common:
+ s = struct.Struct('16sHHI')
+
+ def __init__(self, data):
+ unpacked_data = (Elf_Ehdr_common.s).unpack(data)
+ self.e_ident = unpacked_data[0]
+ self.e_type = unpacked_data[1]
+ self.e_machine = unpacked_data[2]
+ self.e_version = unpacked_data[3]
+
+
+class Elf32_Ehdr:
+ s = struct.Struct('16sHHIIIIIHHHHHH')
+
+ def __init__(self, data):
+ unpacked_data = (Elf32_Ehdr.s).unpack(data)
+ self.e_ident = unpacked_data[0]
+ self.e_type = unpacked_data[1]
+ self.e_machine = unpacked_data[2]
+ self.e_version = unpacked_data[3]
+ self.e_entry = unpacked_data[4]
+ self.e_phoff = unpacked_data[5]
+ self.e_shoff = unpacked_data[6]
+ self.e_flags = unpacked_data[7]
+ self.e_ehsize = unpacked_data[8]
+ self.e_phentsize = unpacked_data[9]
+ self.e_phnum = unpacked_data[10]
+ self.e_shentsize = unpacked_data[11]
+ self.e_shnum = unpacked_data[12]
+ self.e_shstrndx = unpacked_data[13]
+
+ def get_packed_data(self):
+ values = [self.e_ident,
+ self.e_type,
+ self.e_machine,
+ self.e_version,
+ self.e_entry,
+ self.e_phoff,
+ self.e_shoff,
+ self.e_flags,
+ self.e_ehsize,
+ self.e_phentsize,
+ self.e_phnum,
+ self.e_shentsize,
+ self.e_shnum,
+ self.e_shstrndx
+ ]
+
+ return (Elf32_Ehdr.s).pack(*values)
+
+
+class Elf32_Phdr:
+ s = struct.Struct('I' * 8)
+
+ def __init__(self, data):
+ unpacked_data = (Elf32_Phdr.s).unpack(data)
+ self.p_type = unpacked_data[0]
+ self.p_offset = unpacked_data[1]
+ self.p_vaddr = unpacked_data[2]
+ self.p_paddr = unpacked_data[3]
+ self.p_filesz = unpacked_data[4]
+ self.p_memsz = unpacked_data[5]
+ self.p_flags = unpacked_data[6]
+ self.p_align = unpacked_data[7]
+
+ def get_packed_data(self):
+ values = [self.p_type,
+ self.p_offset,
+ self.p_vaddr,
+ self.p_paddr,
+ self.p_filesz,
+ self.p_memsz,
+ self.p_flags,
+ self.p_align
+ ]
+
+ return (Elf32_Phdr.s).pack(*values)
+
+
+class Elf32_Shdr:
+ s = struct.Struct('I' * 10)
+
+ def __init__(self, data):
+ unpacked_data = (Elf32_Shdr.s).unpack(data)
+ self.sh_name = unpacked_data[0]
+ self.sh_type = unpacked_data[1]
+ self.sh_flags = unpacked_data[2]
+ self.sh_addr = unpacked_data[3]
+ self.sh_offset = unpacked_data[4]
+ self.sh_size = unpacked_data[5]
+ self.sh_link = unpacked_data[6]
+ self.sh_info = unpacked_data[7]
+ self.sh_addralign = unpacked_data[8]
+ self.sh_entsize = unpacked_data[9]
+ self.sh_name_str = ""
+
+ def get_packed_data(self):
+ values = [self.sh_name,
+ self.sh_type,
+ self.sh_flags,
+ self.sh_addr,
+ self.sh_offset,
+ self.sh_size,
+ self.sh_link,
+ self.sh_info,
+ self.sh_addralign,
+ self.sh_entsize]
+
+ return (Elf32_Shdr.s).pack(*values)
+
+
+class Elf64_Ehdr:
+ s = struct.Struct('16sHHIQQQIHHHHHH')
+
+ def __init__(self, data):
+ unpacked_data = (Elf64_Ehdr.s).unpack(data)
+ self.e_ident = unpacked_data[0]
+ self.e_type = unpacked_data[1]
+ self.e_machine = unpacked_data[2]
+ self.e_version = unpacked_data[3]
+ self.e_entry = unpacked_data[4]
+ self.e_phoff = unpacked_data[5]
+ self.e_shoff = unpacked_data[6]
+ self.e_flags = unpacked_data[7]
+ self.e_ehsize = unpacked_data[8]
+ self.e_phentsize = unpacked_data[9]
+ self.e_phnum = unpacked_data[10]
+ self.e_shentsize = unpacked_data[11]
+ self.e_shnum = unpacked_data[12]
+ self.e_shstrndx = unpacked_data[13]
+
+ def get_packed_data(self):
+ values = [self.e_ident,
+ self.e_type,
+ self.e_machine,
+ self.e_version,
+ self.e_entry,
+ self.e_phoff,
+ self.e_shoff,
+ self.e_flags,
+ self.e_ehsize,
+ self.e_phentsize,
+ self.e_phnum,
+ self.e_shentsize,
+ self.e_shnum,
+ self.e_shstrndx
+ ]
+
+ return (Elf64_Ehdr.s).pack(*values)
+
+
+class Elf64_Phdr:
+ s = struct.Struct('IIQQQQQQ')
+
+ def __init__(self, data):
+ unpacked_data = (Elf64_Phdr.s).unpack(data)
+ self.p_type = unpacked_data[0]
+ self.p_flags = unpacked_data[1]
+ self.p_offset = unpacked_data[2]
+ self.p_vaddr = unpacked_data[3]
+ self.p_paddr = unpacked_data[4]
+ self.p_filesz = unpacked_data[5]
+ self.p_memsz = unpacked_data[6]
+ self.p_align = unpacked_data[7]
+
+ def get_packed_data(self):
+ values = [self.p_type,
+ self.p_flags,
+ self.p_offset,
+ self.p_vaddr,
+ self.p_paddr,
+ self.p_filesz,
+ self.p_memsz,
+ self.p_align
+ ]
+
+ return (Elf64_Phdr.s).pack(*values)
+
+
+class Elf64_Shdr:
+ s = struct.Struct('IIQQQQIIQQ')
+
+ def __init__(self, data):
+ unpacked_data = (Elf64_Shdr.s).unpack(data)
+ self.sh_name = unpacked_data[0]
+ self.sh_type = unpacked_data[1]
+ self.sh_flags = unpacked_data[2]
+ self.sh_addr = unpacked_data[3]
+ self.sh_offset = unpacked_data[4]
+ self.sh_size = unpacked_data[5]
+ self.sh_link = unpacked_data[6]
+ self.sh_info = unpacked_data[7]
+ self.sh_addralign = unpacked_data[8]
+ self.sh_entsize = unpacked_data[9]
+ self.sh_name_str = ""
+
+ def get_packed_data(self):
+ values = [self.sh_name,
+ self.sh_type,
+ self.sh_flags,
+ self.sh_addr,
+ self.sh_offset,
+ self.sh_size,
+ self.sh_link,
+ self.sh_info,
+ self.sh_addralign,
+ self.sh_entsize]
+
+ return (Elf64_Shdr.s).pack(*values)
+
+
+class SWIV_Seg:
+ s = struct.Struct('4sIQ')
+
+ def __init__(self):
+ self.magic = SWIV_MAGIC
+ self.crc = 0
+ self.padding = 0
+
+ def get_packed_data(self):
+ values = [self.magic,
+ self.crc,
+ self.padding
+ ]
+
+ return (SWIV_Seg.s).pack(*values)
+
+
+def is_elf64(elf_header):
+ return elf_header.e_ident[ELF_CLASS_INDEX] == ELFCLASS64
+
+
+def verify_elf_header(elf_header):
+ ident = elf_header.e_ident
+ return (ident.startswith(ELF_MAGIC)
+ and ident[ELF_CLASS_INDEX] in (ELFCLASS32, ELFCLASS64)
+ and ident[ELF_VERSION_INDEX] == EV_CURRENT)
+
+
+def file_copy_offset(in_fp, in_off, out_fp, out_off, num_bytes):
+ in_fp.seek(in_off)
+ read_in = in_fp.read(num_bytes)
+ out_fp.seek(out_off)
+ out_fp.write(read_in)
+
+ return num_bytes
+
+
+def preprocess_elf_file(elf_file_name):
+ with open(elf_file_name, 'rb') as elf_fp:
+ elf_header = Elf_Ehdr_common(elf_fp.read(Elf_Ehdr_common.s.size))
+
+ if verify_elf_header(elf_header) is False:
+ raise RuntimeError("Input ELF verification failed : " + elf_file_name)
+
+ elf_fp.seek(0)
+
+ if is_elf64(elf_header):
+ elf_header = Elf64_Ehdr(elf_fp.read(Elf64_Ehdr.s.size))
+ else:
+ elf_header = Elf32_Ehdr(elf_fp.read(Elf32_Ehdr.s.size))
+
+ phdr_table = []
+
+ phdr_size = elf_header.e_phentsize
+
+ file_offset = elf_header.e_phoff
+ elf_fp.seek(file_offset)
+
+ for _ in range(elf_header.e_phnum):
+ if is_elf64(elf_header):
+ phdr_table.append(Elf64_Phdr(elf_fp.read(phdr_size)))
+ else:
+ phdr_table.append(Elf32_Phdr(elf_fp.read(phdr_size)))
+
+ return [elf_header, phdr_table]
+
+
+def process_section_header(elf_file_name):
+ with open(elf_file_name, 'rb') as elf_fp:
+ elf_header = Elf_Ehdr_common(elf_fp.read(Elf_Ehdr_common.s.size))
+
+ if verify_elf_header(elf_header) is False:
+ raise RuntimeError("Input ELF verification failed : " + elf_file_name)
+
+ elf_fp.seek(0)
+
+ if is_elf64(elf_header):
+ elf_header = Elf64_Ehdr(elf_fp.read(Elf64_Ehdr.s.size))
+ else:
+ elf_header = Elf32_Ehdr(elf_fp.read(Elf32_Ehdr.s.size))
+
+ shdr_table = []
+
+ shdr_size = elf_header.e_shentsize
+ elf_fp.seek(elf_header.e_shoff)
+
+ for _ in range(elf_header.e_shnum):
+ if is_elf64(elf_header):
+ shdr_table.append(Elf64_Shdr(elf_fp.read(shdr_size)))
+ else:
+ shdr_table.append(Elf32_Shdr(elf_fp.read(shdr_size)))
+
+ return shdr_table
+
+
+def crc32_update_file(elf_fp, offset, size, crc):
+ elf_fp.seek(offset)
+ remaining = size
+ while remaining > 0:
+ chunk = elf_fp.read(min(1024 * 1024, remaining))
+ if not chunk:
+ raise RuntimeError("Unexpected EOF while calculating CRC")
+ crc = zlib.crc32(chunk, crc)
+ remaining -= len(chunk)
+ return crc
+
+
+def crc32_update_zeros(size, crc):
+ zero_chunk = b'\x00' * min(1024 * 1024, size)
+ remaining = size
+ while remaining > 0:
+ chunk = zero_chunk[:min(len(zero_chunk), remaining)]
+ crc = zlib.crc32(chunk, crc)
+ remaining -= len(chunk)
+ return crc
+
+
+def crc32_calculate(elf_fp, phdr_table):
+ crc = 0
+
+ # SWIV validates loadable segment contents, including BSS as zeros.
+ for curr_phdr in phdr_table:
+ if curr_phdr.p_type != LOAD_TYPE:
+ continue
+
+ crc = crc32_update_file(elf_fp, curr_phdr.p_offset, curr_phdr.p_filesz, crc)
+
+ zero_size = curr_phdr.p_memsz - curr_phdr.p_filesz
+ if zero_size > 0:
+ crc = crc32_update_zeros(zero_size, crc)
+
+ crc &= 0xffffffff
+ logger.info("calculated CRC: %#x", crc)
+
+ return crc
+
+
+def verify_swiv_seg(elf_in_fp, offset):
+
+ elf_in_fp.seek(offset)
+ swiv_seg = SWIV_Seg()
+ unpacked_data = (SWIV_Seg.s).unpack(elf_in_fp.read(SWIV_SEG_SIZE_MIN))
+ swiv_seg.magic = unpacked_data[0]
+
+ return swiv_seg.magic == SWIV_MAGIC
+
+
+def same_output_file(input_elf, output_elf):
+ try:
+ return os.path.exists(output_elf) and os.path.samefile(input_elf, output_elf)
+ except FileNotFoundError:
+ return False
+
+
+def make_output_target(input_elf, output_elf):
+ if not same_output_file(input_elf, output_elf):
+ return output_elf, None
+
+ output_dir = os.path.dirname(os.path.abspath(output_elf))
+ output_base = os.path.basename(output_elf)
+ fd, temp_output = tempfile.mkstemp(prefix=output_base + ".", suffix=".tmp", dir=output_dir)
+ os.close(fd)
+ shutil.copymode(input_elf, temp_output)
+ return temp_output, temp_output
+
+
+def swiv_generate(input_elf, output_elf):
+ output_target, temp_output = make_output_target(input_elf, output_elf)
+
+ try:
+ with open(input_elf, "rb") as elf_in_fp, open(output_target, "wb+") as elf_out_fp:
+ [elf_header, phdr_table] = preprocess_elf_file(input_elf)
+
+ num_phdrs = elf_header.e_phnum
+ num_shdrs = elf_header.e_shnum
+ phdr_size = elf_header.e_phentsize
+
+ if num_phdrs == 0 or num_phdrs > MAX_PHDR_COUNT:
+ raise RuntimeError("Input ELF has invalid number of program headers.")
+
+ if elf_header.e_shoff == 0 or num_shdrs == 0:
+ raise RuntimeError("Input ELF does not contain a section header table.")
+
+ shdr_table_input = process_section_header(input_elf)
+ shdr_table_output = process_section_header(input_elf)
+
+ last_phdr = phdr_table[num_phdrs - 1]
+ if last_phdr.p_memsz == SWIV_SEG_SIZE_MIN and verify_swiv_seg(elf_in_fp, last_phdr.p_offset):
+ raise RuntimeError("A SWIV segment is detected in the input ELF.")
+
+ last_shdr = shdr_table_input[elf_header.e_shnum - 1]
+ if last_shdr.sh_size == SWIV_SEG_SIZE_MIN and verify_swiv_seg(elf_in_fp, last_shdr.sh_offset):
+ raise RuntimeError("A SWIV section is detected in the input ELF.")
+
+ if is_elf64(elf_header):
+ swiv_shdr = Elf64_Shdr(b'\0' * Elf64_Shdr.s.size)
+ else:
+ swiv_shdr = Elf32_Shdr(b'\0' * Elf32_Shdr.s.size)
+
+ swiv_shdr.sh_name = 0
+ swiv_shdr.sh_type = SWIV_SECTION_TYPE
+ swiv_shdr.sh_flags = 0
+ swiv_shdr.sh_addr = 0
+ swiv_shdr.sh_size = SWIV_SEG_SIZE_MIN
+ swiv_shdr.sh_offset = elf_header.e_shoff + (elf_header.e_shnum + 1) * elf_header.e_shentsize
+ shdr_table_output.append(swiv_shdr)
+ elf_header.e_shnum += 1
+
+ if elf_header.e_phoff != 0:
+ for phdr in phdr_table:
+ file_copy_offset(elf_in_fp, phdr.p_offset, elf_out_fp, phdr.p_offset, phdr.p_filesz)
+
+ elf_out_fp.seek(0)
+ elf_out_fp.write(elf_header.get_packed_data())
+
+ phdr_start = elf_header.e_phoff
+ for phdr in phdr_table:
+ elf_out_fp.seek(phdr_start)
+ elf_out_fp.write(phdr.get_packed_data())
+ phdr_start += phdr_size
+
+ for shdr in shdr_table_input[:num_shdrs]:
+ if shdr.sh_type == SHT_NOBITS or shdr.sh_size == 0:
+ continue
+ file_copy_offset(elf_in_fp, shdr.sh_offset, elf_out_fp, shdr.sh_offset, shdr.sh_size)
+
+ shdr_start = elf_header.e_shoff
+ for shdr in shdr_table_output:
+ elf_out_fp.seek(shdr_start)
+ elf_out_fp.write(shdr.get_packed_data())
+ shdr_start += elf_header.e_shentsize
+
+ elf_out_fp.flush()
+
+ swiv_seg = SWIV_Seg()
+ swiv_seg.crc = crc32_calculate(elf_out_fp, phdr_table)
+ elf_out_fp.seek(swiv_shdr.sh_offset)
+ elf_out_fp.write(swiv_seg.get_packed_data())
+
+ if temp_output is not None:
+ os.replace(temp_output, output_elf)
+ temp_output = None
+
+ return 0
+ finally:
+ if temp_output is not None and os.path.exists(temp_output):
+ os.unlink(temp_output)
+
+
+def main(argv):
+ parser = argparse.ArgumentParser(description="Add a Software Image Validation (SWIV) section to an HTP ELF file.")
+ parser.add_argument("-i", "--input", dest="input_file", metavar="INPUT", required=True)
+ parser.add_argument("-o", "--output", dest="output_file", metavar="OUTPUT", required=True)
+ parser.add_argument("-v", "--verbose", action="store_true", help="show calculated CRC")
+ args = parser.parse_args(argv)
+
+ logging.basicConfig(
+ level=logging.INFO if args.verbose else logging.WARNING,
+ format="%(message)s",
+ )
+
+ try:
+ return swiv_generate(args.input_file, args.output_file)
+ except RuntimeError as exc:
+ logger.error("ERROR: %s", exc)
+ return 2
+
+
+if __name__ == "__main__":
+ sys.exit(main(sys.argv[1:]))
diff --git a/ggml/src/ggml-hexagon/libggml-htp.inf b/ggml/src/ggml-hexagon/libggml-htp.inf
index 874dde1b8..ff6ac27cd 100644
--- a/ggml/src/ggml-hexagon/libggml-htp.inf
+++ b/ggml/src/ggml-hexagon/libggml-htp.inf
@@ -18,6 +18,7 @@ libggml-htp-v73.so = 1
libggml-htp-v75.so = 1
libggml-htp-v79.so = 1
libggml-htp-v81.so = 1
+libggml-htp-v85.so = 1
[ControlFlags]
ExcludeFromSelect = *
@@ -30,6 +31,7 @@ libggml-htp-v73.so,,,0x10 ;COPYFLG_NO_OVERWRITE
libggml-htp-v75.so,,,0x10 ;COPYFLG_NO_OVERWRITE
libggml-htp-v79.so,,,0x10 ;COPYFLG_NO_OVERWRITE
libggml-htp-v81.so,,,0x10 ;COPYFLG_NO_OVERWRITE
+libggml-htp-v85.so,,,0x10 ;COPYFLG_NO_OVERWRITE
[Strings]
GGML = 'GGML'
diff --git a/scripts/snapdragon/build.py b/scripts/snapdragon/build.py
index 62a114e04..05e515167 100755
--- a/scripts/snapdragon/build.py
+++ b/scripts/snapdragon/build.py
@@ -11,6 +11,9 @@ import platform
import shutil
import logging
import glob
+import shlex
+import tarfile
+import tempfile
from sdk import validate_windows_sdks
@@ -19,19 +22,70 @@ logger = logging.getLogger("build")
def parse_target(target_str):
if not target_str:
- return None, None
- if target_str.startswith("adb") or target_str.startswith("android"):
- parts = target_str.split(":", 1)
- serial = parts[1] if len(parts) > 1 else None
- return "android", serial
- elif target_str.startswith("lnx") or target_str.startswith("linux") or target_str.startswith("ubuntu"):
- parts = target_str.split(":", 1)
- host = parts[1] if len(parts) > 1 else None
- return "linux", host
+ return None, None, None
+
+ parts = target_str.split(":")
+ target_name = parts[0]
+
+ if target_name in ("adb", "android"):
+ serial = ":".join(parts[1:]) if len(parts) > 1 else None
+ return "android", serial, "adb"
+ elif target_name in ("lnx", "linux", "ubuntu"):
+ if len(parts) > 1 and parts[1] == "adb":
+ serial = ":".join(parts[2:]) if len(parts) > 2 else None
+ return "linux", serial, "adb"
+ host = ":".join(parts[1:]) if len(parts) > 1 else None
+ return "linux", host, "ssh"
elif target_str in ("wos", "windows"):
- return "windows", None
+ return "windows", None, None
else:
- return None, None
+ return None, None, None
+
+
+def default_target_dir(target_type, target_transport, home_dir):
+ if target_type == "android":
+ return "/data/local/tmp/llama.cpp"
+ if target_type == "linux":
+ if home_dir:
+ return home_dir.rstrip("/") + "/llama.cpp"
+ if target_transport == "adb":
+ return "/home/llama.cpp"
+ return "~/llama.cpp"
+ return "~/llama.cpp"
+
+
+def normalize_target_dir(target_dir):
+ target_dir = target_dir.rstrip("/")
+ if not target_dir or target_dir == "/":
+ raise RuntimeError("target directory must not be the filesystem root")
+ return target_dir
+
+
+def adb_root(adb_cmd):
+ root_cmd = adb_cmd + ["root"]
+ logger.info(f"+ {' '.join(root_cmd)}")
+ res = subprocess.run(root_cmd)
+ if res.returncode != 0:
+ logger.error("ADB root failed.")
+ sys.exit(res.returncode)
+
+ wait_cmd = adb_cmd + ["wait-for-device"]
+ logger.info(f"+ {' '.join(wait_cmd)}")
+ res = subprocess.run(wait_cmd)
+ if res.returncode != 0:
+ logger.error("ADB wait-for-device failed.")
+ sys.exit(res.returncode)
+
+
+def make_package_archive(src_path):
+ fd, archive_path = tempfile.mkstemp(prefix="llama-cpp-", suffix=".tar.gz")
+ os.close(fd)
+ with tarfile.open(archive_path, "w:gz", dereference=False) as tar:
+ for item in os.listdir(src_path):
+ if item.startswith("."):
+ continue
+ tar.add(os.path.join(src_path, item), arcname=item, recursive=True)
+ return archive_path
def get_uid_gid():
@@ -45,7 +99,7 @@ def main():
parser = argparse.ArgumentParser(
description="Build llama.cpp for Snapdragon using cross-compilation docker containers or natively."
)
- parser.add_argument("--target", default="android", help="Compilation target and deployment definition (e.g. android[:serial]/adb[:serial], linux:[user@]host/lnx:[user@]host/ubuntu:[user@]host, windows/wos) (default: android)")
+ parser.add_argument("--target", default="android", help="Compilation target and deployment definition (e.g. android[:serial]/adb[:serial], ubuntu:adb[:serial], linux:[user@]host/lnx:[user@]host/ubuntu:[user@]host, windows/wos) (default: android)")
parser.add_argument("--build-dir", help="Build directory name (defaults to build-TARGET[-dbg], e.g. build-android)")
parser.add_argument("--install-dir", help="Install directory name (defaults to pkg-TARGET[-dbg], e.g. pkg-android)")
parser.add_argument("--jobs", "-j", type=int, help="Number of build jobs (defaults to CPU thread count)")
@@ -53,21 +107,22 @@ def main():
parser.add_argument("--preset", help="Override the CMake preset to use")
parser.add_argument("--debug", action="store_true", help="Build in debug mode (uses -debug presets instead of -release)")
parser.add_argument("--hex-only", "--hexagon-only", action="store_true", help="Only build and push ggml-hexagon and libggml-htp libraries")
- parser.add_argument("--hex-arch", choices=["v73", "v75", "v79", "v81", "73", "75", "79", "81"], help="Specific Hexagon DSP architecture to build with --hex-only (default: all)")
+ parser.add_argument("--hex-arch", choices=["v73", "v75", "v79", "v81", "v85", "73", "75", "79", "81", "85"], help="Specific Hexagon DSP architecture to build with --hex-only (default: all)")
# Push options
parser.add_argument("--push", action="store_true", help="Push built package to the target device via ADB or SSH/SCP")
- parser.add_argument("--target-dir", help="Target directory on the device (default: /data/local/tmp/llama.cpp for Android, ~/llama.cpp for Linux)")
+ parser.add_argument("--target-dir", help="Target directory on the device (default: /data/local/tmp/llama.cpp for Android, ~/llama.cpp for Linux, /home/llama.cpp for Linux via ADB)")
+ parser.add_argument("--home-dir", help="Home directory for Linux targets when deriving the default target directory")
# Toolchain options
- parser.add_argument("--toolchain-version", default="v0.7", help="Docker toolchain image version/tag (default: v0.7)")
+ parser.add_argument("--toolchain-version", default="v0.8", help="Docker toolchain image version/tag (default: v0.8)")
parser.add_argument("--toolchain-url", default="ghcr.io/snapdragon-toolchain", help="Docker toolchain registry URL/namespace (default: ghcr.io/snapdragon-toolchain)")
args = parser.parse_args()
- target_type, target_val = parse_target(args.target)
+ target_type, target_val, target_transport = parse_target(args.target)
if not target_type:
- logger.error(f"Error: Invalid target format '{args.target}'. Must be android[:serial]/adb[:serial], linux:[user@]host/lnx:[user@]host/ubuntu:[user@]host, or windows/wos.")
+ logger.error(f"Error: Invalid target format '{args.target}'. Must be android[:serial]/adb[:serial], ubuntu:adb[:serial], linux:[user@]host/lnx:[user@]host/ubuntu:[user@]host, or windows/wos.")
sys.exit(1)
if target_type == "windows":
@@ -140,7 +195,7 @@ def main():
if hex_arch:
hex_targets = ["ggml-hexagon", f"htp-{hex_arch}"]
else:
- hex_targets = ["ggml-hexagon", "htp-v73", "htp-v75", "htp-v79", "htp-v81"]
+ hex_targets = ["ggml-hexagon", "htp-v73", "htp-v75", "htp-v79", "htp-v81", "htp-v85"]
if args.no_docker:
# Native/local host build
@@ -218,8 +273,8 @@ def main():
# Resolve target directory on device
target_dir = args.target_dir
if not target_dir:
- target_dir = "/data/local/tmp/llama.cpp" if target_type == "android" else "~/llama.cpp"
- target_dir = target_dir.rstrip("/")
+ target_dir = default_target_dir(target_type, target_transport, args.home_dir)
+ target_dir = normalize_target_dir(target_dir)
sub_items = [item for item in os.listdir(src_path) if not item.startswith(".")]
@@ -235,14 +290,23 @@ def main():
logger.error(f"Error: no hexagon libraries found in {lib_dir} to push.")
sys.exit(1)
- if target_type == "android":
+ if target_transport == "adb":
adb_cmd = ["adb"]
if target_val: # serial
adb_cmd += ["-s", target_val]
+ if target_type == "linux":
+ adb_root(adb_cmd)
+
if args.hex_only:
- logger.info("\nPushing Hexagon libraries to Android device via ADB...")
+ logger.info("\nPushing Hexagon libraries to target device via ADB...")
target_lib_dir = f"{target_dir}/lib"
+ mkdir_cmd = adb_cmd + ["shell", f"mkdir -p {shlex.quote(target_lib_dir)}"]
+ logger.info(f"+ {' '.join(mkdir_cmd)}")
+ res = subprocess.run(mkdir_cmd)
+ if res.returncode != 0:
+ logger.error("ADB mkdir failed.")
+ sys.exit(res.returncode)
for lib in hex_libs:
push_cmd = adb_cmd + ["push", lib, f"{target_lib_dir}/"]
logger.info(f"+ {' '.join(push_cmd)}")
@@ -251,6 +315,44 @@ def main():
logger.error("ADB push failed.")
sys.exit(res.returncode)
logger.info("Hexagon libraries deployed successfully!")
+ elif target_type == "linux":
+ logger.info("\nDeploying Linux package to target device via ADB tar...")
+ remote_archive = f"/tmp/llama.cpp-package-{os.getpid()}.tar.gz"
+ archive_path = make_package_archive(src_path)
+ try:
+ mkdir_cmd = adb_cmd + ["shell", f"mkdir -p {shlex.quote(target_dir)}"]
+ logger.info(f"+ {' '.join(mkdir_cmd)}")
+ res = subprocess.run(mkdir_cmd)
+ if res.returncode != 0:
+ logger.error("ADB mkdir failed.")
+ sys.exit(res.returncode)
+
+ if sub_items:
+ clean_paths = " ".join(shlex.quote(f"{target_dir}/{item}") for item in sub_items)
+ clean_cmd = adb_cmd + ["shell", f"rm -rf {clean_paths}"]
+ logger.info(f"+ {' '.join(clean_cmd)}")
+ subprocess.run(clean_cmd)
+
+ push_cmd = adb_cmd + ["push", archive_path, remote_archive]
+ logger.info(f"+ {' '.join(push_cmd)}")
+ res = subprocess.run(push_cmd)
+ if res.returncode != 0:
+ logger.error("ADB push failed.")
+ sys.exit(res.returncode)
+
+ extract_cmd = adb_cmd + ["shell", f"tar -xzf {shlex.quote(remote_archive)} -C {shlex.quote(target_dir)} && rm -f {shlex.quote(remote_archive)}"]
+ logger.info(f"+ {' '.join(extract_cmd)}")
+ res = subprocess.run(extract_cmd)
+ if res.returncode != 0:
+ logger.error("ADB untar failed.")
+ sys.exit(res.returncode)
+
+ chmod_cmd = adb_cmd + ["shell", f"chmod -R 755 {shlex.quote(target_dir)}/bin 2>/dev/null || true"]
+ logger.info(f"+ {' '.join(chmod_cmd)}")
+ subprocess.run(chmod_cmd)
+ finally:
+ os.unlink(archive_path)
+ logger.info("ADB tar deploy completed successfully!")
else:
# Clean stale package files on device
if sub_items:
diff --git a/scripts/snapdragon/ggml-hexagon-inspect.py b/scripts/snapdragon/ggml-hexagon-inspect.py
index 7b0e2aefa..a3572b3e8 100755
--- a/scripts/snapdragon/ggml-hexagon-inspect.py
+++ b/scripts/snapdragon/ggml-hexagon-inspect.py
@@ -255,7 +255,7 @@ class HexagonToolchain:
use_docker: bool = False,
image_url: str = "ghcr.io/snapdragon-toolchain",
image_name: str = "arm64-android",
- image_ver: str = "v0.7",
+ image_ver: str = "v0.8",
):
self.repo_root = repo_root
self.image = f"{image_url}/{image_name}:{image_ver}"
@@ -322,15 +322,14 @@ class HexagonToolchain:
if not self.docker_bin:
raise RuntimeError("Docker is required but not installed or found on PATH.")
- container_tools_dir = "/opt/hexagon/6.6.0.0/tools/HEXAGON_Tools/19.0.07/Tools/bin"
if "objdump" in tool_name:
- container_tool = f"{container_tools_dir}/hexagon-llvm-objdump"
+ tool_bin = "hexagon-llvm-objdump"
elif "addr2line" in tool_name:
- container_tool = f"{container_tools_dir}/hexagon-addr2line"
+ tool_bin = "hexagon-addr2line"
elif "nm" in tool_name:
- container_tool = f"{container_tools_dir}/llvm-nm"
+ tool_bin = "llvm-nm"
else:
- container_tool = f"{container_tools_dir}/{tool_name}"
+ tool_bin = tool_name
# Translate file paths from host to /workspace
translated_args = [translate_container_arg(arg, self.repo_root) for arg in args]
@@ -350,7 +349,9 @@ class HexagonToolchain:
if platform.system() != "Windows":
docker_cmd += ["-u", f"{os.getuid()}:{os.getgid()}"]
- docker_cmd += [self.image, container_tool] + translated_args
+ default_tools_root = "/opt/hexagon/6.7.0.0/tools/HEXAGON_Tools/21.0.04"
+ sh_cmd = f'exec "${{HEXAGON_TOOLS_ROOT:-{default_tools_root}}}/Tools/bin/{tool_bin}" "$@"'
+ docker_cmd += [self.image, "sh", "-c", sh_cmd, "--"] + translated_args
res = subprocess.run(docker_cmd, capture_output=True, text=True, input=stdin_data)
if res.returncode != 0:
@@ -1457,8 +1458,8 @@ def main():
)
parser.add_argument(
"--toolchain-version",
- default="v0.7",
- help="Docker toolchain tag (default: v0.7).",
+ default="v0.8",
+ help="Docker toolchain tag (default: v0.8).",
)
parser.add_argument(
"--toolchain-url",
diff --git a/scripts/snapdragon/run.py b/scripts/snapdragon/run.py
index 7e11d90de..99b43bb2b 100755
--- a/scripts/snapdragon/run.py
+++ b/scripts/snapdragon/run.py
@@ -49,27 +49,46 @@ MANAGED_ENV_NAMES = (
"GGML_OPENCL_ADRENO_XMEM_GEMM",
"GGML_OPENCL_ADRENO_USE_LARGE_BUFFER",
"GGML_SCHED_DEBUG",
+ "ADSP_LIBRARY_PATH",
+ "CDSP_LIBRARY_PATH",
+ "CDSP1_LIBRARY_PATH",
+ "CDSP2_LIBRARY_PATH",
+ "CDSP3_LIBRARY_PATH",
"MTMD_BACKEND_DEVICE",
"D",
"DEVICE",
)
+DSP_LIBRARY_ENV_NAMES = (
+ "ADSP_LIBRARY_PATH",
+ "CDSP_LIBRARY_PATH",
+ "CDSP1_LIBRARY_PATH",
+ "CDSP2_LIBRARY_PATH",
+ "CDSP3_LIBRARY_PATH",
+)
+
+
def parse_target(target_str):
if not target_str:
- return None, None
- if target_str.startswith("adb") or target_str.startswith("android"):
- parts = target_str.split(":", 1)
- serial = parts[1] if len(parts) > 1 else None
- return "android", serial
- elif target_str.startswith("lnx") or target_str.startswith("linux") or target_str.startswith("ubuntu"):
- parts = target_str.split(":", 1)
- host = parts[1] if len(parts) > 1 else None
- return "linux", host
+ return None, None, None
+
+ parts = target_str.split(":")
+ target_name = parts[0]
+
+ if target_name in ("adb", "android"):
+ serial = ":".join(parts[1:]) if len(parts) > 1 else None
+ return "android", serial, "adb"
+ elif target_name in ("lnx", "linux", "ubuntu"):
+ if len(parts) > 1 and parts[1] == "adb":
+ serial = ":".join(parts[2:]) if len(parts) > 2 else None
+ return "linux", serial, "adb"
+ host = ":".join(parts[1:]) if len(parts) > 1 else None
+ return "linux", host, "ssh"
elif target_str in ("wos", "windows"):
- return "windows", None
+ return "windows", None, None
else:
- return None, None
+ return None, None, None
def shlex_join(args_list):
@@ -79,6 +98,41 @@ def shlex_join(args_list):
return " ".join(pipes.quote(x) for x in args_list)
+def default_target_dir(target_type, target_transport, home_dir):
+ if target_type == "android":
+ return "/data/local/tmp/llama.cpp"
+ if target_type == "linux":
+ if home_dir:
+ return home_dir.rstrip("/") + "/llama.cpp"
+ if target_transport == "adb":
+ return "/home/llama.cpp"
+ return "~/llama.cpp"
+ return "~/llama.cpp"
+
+
+def library_env_parts(lib_dir):
+ env_parts = [f"LD_LIBRARY_PATH={lib_dir}"]
+ for name in DSP_LIBRARY_ENV_NAMES:
+ env_parts.append(f"{name}={lib_dir}")
+ return env_parts
+
+
+def adb_root(adb_base):
+ root_cmd = adb_base + ["root"]
+ logger.info(f"+ {' '.join(root_cmd)}")
+ res = subprocess.run(root_cmd)
+ if res.returncode != 0:
+ logger.error("ADB root failed.")
+ sys.exit(res.returncode)
+
+ wait_cmd = adb_base + ["wait-for-device"]
+ logger.info(f"+ {' '.join(wait_cmd)}")
+ res = subprocess.run(wait_cmd)
+ if res.returncode != 0:
+ logger.error("ADB wait-for-device failed.")
+ sys.exit(res.returncode)
+
+
def split_device_list(devices):
parts = []
curr = []
@@ -144,8 +198,9 @@ def main():
parser = argparse.ArgumentParser(
description="Unified runner for llama.cpp tools on Snapdragon (natively, via ADB, or via SSH)."
)
- parser.add_argument("--target", help="Execution target (e.g. android[:serial]/adb[:serial], linux:[user@]host/lnx:[user@]host/ubuntu:[user@]host, windows/wos) (default: local run)")
- parser.add_argument("--target-dir", help="Target directory on the device (default: /data/local/tmp/llama.cpp for Android, ~/llama.cpp for Linux)")
+ parser.add_argument("--target", help="Execution target (e.g. android[:serial]/adb[:serial], ubuntu:adb[:serial], linux:[user@]host/lnx:[user@]host/ubuntu:[user@]host, windows/wos) (default: local run)")
+ parser.add_argument("--target-dir", help="Target directory on the device (default: /data/local/tmp/llama.cpp for Android, ~/llama.cpp for Linux, /home/llama.cpp for Linux via ADB)")
+ parser.add_argument("--home-dir", help="Home directory for Linux targets when deriving the default target directory")
parser.add_argument("--install-dir", help="Install directory name (defaults to pkg-TARGET or pkg-TARGET-dbg prefix based on target)")
parser.add_argument("--debug", action="store_true", help="Use debug build (defaults to pkg-TARGET-dbg folder)")
parser.add_argument("--devices", "--device", "-d", help="Select execution devices (split into NPU and OpenCL GPUs automatically, default: HTP0)")
@@ -201,11 +256,12 @@ def main():
target_type = None
target_val = None
+ target_transport = None
target_prefix = None
if args.target:
- target_type, target_val = parse_target(args.target)
+ target_type, target_val, target_transport = parse_target(args.target)
if not target_type:
- logger.error(f"Error: Invalid target format '{args.target}'. Must be android[:serial]/adb[:serial], linux:[user@]host/lnx:[user@]host/ubuntu:[user@]host, or windows/wos.")
+ logger.error(f"Error: Invalid target format '{args.target}'. Must be android[:serial]/adb[:serial], ubuntu:adb[:serial], linux:[user@]host/lnx:[user@]host/ubuntu:[user@]host, or windows/wos.")
sys.exit(1)
target_prefix = args.target.split(":", 1)[0]
@@ -435,18 +491,19 @@ def main():
# Resolve target directory on device
target_dir = args.target_dir
if not target_dir:
- target_dir = "/data/local/tmp/llama.cpp" if target_type == "android" else "~/llama.cpp"
+ target_dir = default_target_dir(target_type, target_transport, args.home_dir)
- if target_type == "android":
- # Run via ADB
+ if target_transport == "adb":
adb_base = ["adb"]
if target_val: # serial
adb_base += ["-s", target_val]
- env_parts = [
- "LD_LIBRARY_PATH=./lib",
- "ADSP_LIBRARY_PATH=./lib"
- ]
+ if target_type == "linux":
+ adb_root(adb_base)
+
+ env_parts = library_env_parts("./lib")
+ if target_type == "linux":
+ env_parts.insert(0, f"HOME={args.home_dir or '/home'}")
for k, v in env_vars.items():
env_parts.append(f"{k}={v}")
env_str = " ".join(env_parts)
@@ -466,10 +523,7 @@ def main():
sys.exit(1)
# Linux remote run via SSH
- env_parts = [
- "LD_LIBRARY_PATH=./lib",
- "ADSP_LIBRARY_PATH=./lib"
- ]
+ env_parts = library_env_parts("./lib")
for k, v in env_vars.items():
env_parts.append(f"{k}={v}")
env_str = " ".join(env_parts)
@@ -490,7 +544,6 @@ def main():
# Run locally
local_env = os.environ.copy()
lib_dir = os.path.normpath(os.path.join(package_path, "lib"))
- local_env["ADSP_LIBRARY_PATH"] = lib_dir
if platform.system() == "Windows":
local_env["PATH"] = lib_dir + os.path.pathsep + local_env.get("PATH", "")
else:
@@ -498,6 +551,8 @@ def main():
for k in MANAGED_ENV_NAMES:
local_env.pop(k, None)
+ for name in DSP_LIBRARY_ENV_NAMES:
+ local_env[name] = lib_dir
for k, v in env_vars.items():
local_env[k] = v
diff --git a/scripts/snapdragon/sdk.py b/scripts/snapdragon/sdk.py
index bb3cb77b2..72b019346 100644
--- a/scripts/snapdragon/sdk.py
+++ b/scripts/snapdragon/sdk.py
@@ -6,7 +6,7 @@ SDK_CONFIGS = (
{
"name": "Hexagon SDK",
"repo": "snapdragon-toolchain/hexagon-sdk",
- "default_version": "6.6.0.0",
+ "default_version": "6.7.0.0",
"parent_dir": "Hexagon_SDK",
"archive_prefix": "hexagon-sdk-v",
"markers": ("hexagon_sdk.json",),