Commit 845646b for stable-diffusion.cpp
commit 845646b034a12ed844966c84bd7585c8fcb37b7f
Author: losewayy <1134285612@qq.com>
Date: Wed Oct 21 23:11:36 2026 +0800
feat(server): add ADetailer support to image generation (#2164)
diff --git a/docs/adetailer.md b/docs/adetailer.md
index 2bc64df..f428318 100644
--- a/docs/adetailer.md
+++ b/docs/adetailer.md
@@ -1,9 +1,10 @@
# ADetailer
`sd-cli` can run a YOLOv8 object detector on an existing or newly generated
-image and perform a cropped inpaint pass for every detected object. The first
-implementation supports YOLOv8 detection checkpoints. YOLOv8 segmentation and
-MediaPipe models are not supported yet.
+image, and `sd-server` can run it after image generation. Both perform a cropped
+inpaint pass for every detected object. The implementation supports YOLOv8
+detection checkpoints. YOLOv8 segmentation and MediaPipe models are not
+supported yet.
## Convert a detector
@@ -68,6 +69,36 @@ ADetailer can also run automatically after normal image generation:
--extra-ad-args "confidence=0.3,denoising_strength=0.4,inpaint_width=512,inpaint_height=512"
```
+The same options can be supplied when starting `sd-server`. When `--ad-model` is set, its detector runs after OpenAI-compatible and native asynchronous image generation. SDAPI requests enable ADetailer through `alwayson_scripts`, as described below. If no object is detected, the original image is returned unchanged. RGBA output keeps its original alpha channel, including when detected regions are repaired. Detector initialization or detail-pass failures are reported as request failures.
+
+Server image-generation requests limit base, hires, and detail-pass dimensions to 4096 pixels per side. ADetailer step overrides are limited to 1–100; `steps=0` in `extra_ad_args` keeps inheriting the main generation steps. These limits also apply to inherited dimensions and `extra_ad_args` overrides. Requests exceeding the limits return a parameter error before generation. CLI limits are unchanged.
+
+### SDAPI requests
+
+`/sdapi/v1/txt2img` and `/sdapi/v1/img2img` accept the WebUI ADetailer argument format:
+
+```json
+{
+ "prompt": "portrait photo",
+ "alwayson_scripts": {
+ "ADetailer": {
+ "args": [true, false, {
+ "ad_model": "face_yolov8n.pt",
+ "ad_prompt": "[PROMPT], detailed face",
+ "ad_use_steps": true,
+ "ad_steps": 20
+ }]
+ }
+ }
+}
+```
+
+The detector must be configured at server startup with `--ad-model`. `ad_model` must match that file's basename; the original `.pt` basename is also accepted for a converted detector. Requests cannot load arbitrary detector paths. Omitting the script, setting its first argument to `false`, selecting `"None"`, or setting `ad_tab_enable` to `false` disables detailing. Dictionary-only argument arrays are also accepted. Multiple enabled dictionaries run in order using the configured detector.
+
+Supported settings include prompts, confidence, mask filtering by area or confidence, mask count and area ratios, dilation/erosion, offsets, mask merging/inversion, blur, denoising strength, masked or full-image inpainting, padding, and optional overrides for inpaint dimensions, steps, CFG, sampler/scheduler, and CLIP skip. The corresponding `ad_use_*` flags control whether overrides apply. Otherwise dimensions, steps, CFG, sampler, scheduler, and CLIP skip inherit the main request. Denoising strength defaults to `0.4`. Pixel mask settings are limited to the larger output dimension, with a minimum limit of 32 and a maximum of 16384 pixels.
+
+Model/VAE switching, non-default noise multipliers, face restoration, ControlNet, and skipping the initial img2img pass are not supported; active requests for these features return an error. Unknown settings and invalid active values also return errors. During detailing, `/sdapi/v1/progress` remains active with an `adetailer` job; its percentage and preview stay fixed until detailing finishes.
+
An empty ADetailer prompt inherits the main prompt. `[PROMPT]` inserts the main
prompt, `[SEP]` assigns different prompts to consecutive masks, and `[SKIP]`
skips the corresponding mask.
@@ -82,6 +113,7 @@ All settings other than the detector path and prompts are passed through
| `nms` | `0.45` | NMS IoU threshold |
| `max_detections` | `100` | Maximum detections retained after NMS |
| `mask_k_largest` | `0` | Keep only the largest K masks; zero keeps all |
+| `mask_filter_method` | `area` | Rank retained masks by `area` or `confidence` |
| `mask_min_ratio` | `0` | Minimum bbox area relative to the image |
| `mask_max_ratio` | `1` | Maximum bbox area relative to the image |
| `dilate_erode` | `4` | Positive values dilate; negative values erode |
@@ -90,6 +122,7 @@ All settings other than the detector path and prompts are passed through
| `merge_masks`, `invert_mask` | `false` | Boolean alternatives to `mask_mode` |
| `mask_blur` | `4` | Final composite feather radius |
| `inpaint_padding` | `32` | Padding around the detected region |
+| `inpaint_only_masked` | `true` | Crop to the detected region; `false` uses the full image with the detection mask |
| `inpaint_width`, `inpaint_height` | mode-specific | `512x512` after generation; input/output size in `adetailer` mode |
| `denoising_strength` | mode-specific | `0.4` after generation; inherits `--strength` in `adetailer` mode |
| `steps` | `0` | Detail steps; zero inherits the main generation |
diff --git a/examples/common/common.cpp b/examples/common/common.cpp
index 9e7851f..8a90673 100644
--- a/examples/common/common.cpp
+++ b/examples/common/common.cpp
@@ -2219,7 +2219,7 @@ bool SDGenerationParams::from_json_str(
if (target_custom_sigmas != nullptr &&
sample_json.contains("custom_sigmas") &&
sample_json["custom_sigmas"].is_array() &&
- !sample_json["custom_sigmas"].empty()) { // empty [] = "not specified", keep server/CLI default
+ !sample_json["custom_sigmas"].empty()) { // empty [] = "not specified", keep server/CLI default
*target_custom_sigmas = sample_json["custom_sigmas"].get<std::vector<float>>();
}
if (sample_json.contains("sample_method") && sample_json["sample_method"].is_string()) {
diff --git a/examples/server/async_jobs.cpp b/examples/server/async_jobs.cpp
index 7f3c8a1..edd9359 100644
--- a/examples/server/async_jobs.cpp
+++ b/examples/server/async_jobs.cpp
@@ -244,6 +244,10 @@ bool execute_img_gen_job(ServerRuntime& runtime,
}
clear_preview_callback();
results.adopt(raw_results, num_results);
+ if (!results.empty() && !apply_server_adetailer(runtime, job.img_gen, params, results)) {
+ error_message = "ADetailer failed";
+ return false;
+ }
}
const int num_results = results.count();
diff --git a/examples/server/routes_openai.cpp b/examples/server/routes_openai.cpp
index 29dfe31..27a7e18 100644
--- a/examples/server/routes_openai.cpp
+++ b/examples/server/routes_openai.cpp
@@ -75,7 +75,7 @@ static bool build_openai_generation_request(const httplib::Request& req,
error_message = "invalid params";
return false;
}
- return true;
+ return validate_server_img_gen_request(runtime, request, error_message);
}
static bool build_openai_edit_request(const httplib::Request& req,
@@ -158,22 +158,22 @@ static bool build_openai_edit_request(const httplib::Request& req,
request.gen_params.height = height;
request.gen_params.batch_count = n;
- std::string sd_cpp_extra_args_str = extract_and_remove_sd_cpp_extra_args(request.gen_params.prompt);
- const json extra_args = sd_cpp_extra_args_str.empty()
- ? json::object()
- : json::parse(sd_cpp_extra_args_str, nullptr, false);
- const bool has_extra_init_image_override = extra_args.is_object() && extra_args.contains("init_image");
- const bool has_extra_ref_images_override = extra_args.is_object() && extra_args.contains("ref_images");
- const bool has_extra_strength_override = extra_args.is_object() &&
- extra_args.contains("strength") &&
- extra_args["strength"].is_number();
- const bool has_extra_mask_image = extra_args.is_object() &&
- extra_args.contains("mask_image") &&
- !extra_args["mask_image"].is_null();
+ std::string sd_cpp_extra_args_str = extract_and_remove_sd_cpp_extra_args(request.gen_params.prompt);
+ const json extra_args = sd_cpp_extra_args_str.empty()
+ ? json::object()
+ : json::parse(sd_cpp_extra_args_str, nullptr, false);
+ const bool has_extra_init_image_override = extra_args.is_object() && extra_args.contains("init_image");
+ const bool has_extra_ref_images_override = extra_args.is_object() && extra_args.contains("ref_images");
+ const bool has_extra_strength_override = extra_args.is_object() &&
+ extra_args.contains("strength") &&
+ extra_args["strength"].is_number();
+ const bool has_extra_mask_image = extra_args.is_object() &&
+ extra_args.contains("mask_image") &&
+ !extra_args["mask_image"].is_null();
const bool supports_ref_image_conditioning = sd_ctx_supports_ref_image_conditioning(runtime.sd_ctx);
const bool may_need_uploaded_init_image = !has_extra_init_image_override &&
- (!supports_ref_image_conditioning || !mask_bytes.empty() ||
- has_extra_mask_image || has_extra_strength_override || has_extra_ref_images_override);
+ (!supports_ref_image_conditioning || !mask_bytes.empty() ||
+ has_extra_mask_image || has_extra_strength_override || has_extra_ref_images_override);
SDImageOwner uploaded_image_for_init;
bool uploaded_image_was_first_reference = false;
for (auto& bytes : images_bytes) {
@@ -252,7 +252,7 @@ static bool build_openai_edit_request(const httplib::Request& req,
return false;
}
- return true;
+ return validate_server_img_gen_request(runtime, request, error_message);
}
static bool execute_sync_img_gen_request(ServerRuntime& runtime,
@@ -270,6 +270,10 @@ static bool execute_sync_img_gen_request(ServerRuntime& runtime,
num_results = 0;
}
results.adopt(raw_results, num_results);
+ if (!results.empty() && !apply_server_adetailer(runtime, request, img_gen_params, results)) {
+ error_message = "ADetailer failed";
+ return false;
+ }
}
if (results.empty()) {
diff --git a/examples/server/routes_sdapi.cpp b/examples/server/routes_sdapi.cpp
index 4b8e7a5..014b0b5 100644
--- a/examples/server/routes_sdapi.cpp
+++ b/examples/server/routes_sdapi.cpp
@@ -3,9 +3,12 @@
#include <algorithm>
#include <cctype>
#include <cstring>
+#include <limits>
#include <regex>
+#include <stdexcept>
#include <string_view>
#include <unordered_map>
+#include <unordered_set>
#include "common/common.h"
#include "common/media_io.h"
@@ -81,6 +84,202 @@ static enum sample_method_t get_sdapi_sample_method(std::string name) {
return it != hardcoded.end() ? it->second : SAMPLE_METHOD_COUNT;
}
+static bool parse_sdapi_adetailer(const json& body,
+ ServerRuntime& runtime,
+ bool img2img,
+ ImgGenJobRequest& request,
+ std::string& error_message) {
+ request.adetailer_enabled = false;
+ request.adetailer_passes.clear();
+ if (!body.contains("alwayson_scripts")) {
+ return true;
+ }
+ try {
+ const auto& scripts = body.at("alwayson_scripts");
+ if (!scripts.is_object()) {
+ throw std::invalid_argument("alwayson_scripts must be an object");
+ }
+ if (!scripts.contains("ADetailer")) {
+ return true;
+ }
+ const auto& args = scripts.at("ADetailer").at("args");
+ if (!args.is_array()) {
+ throw std::invalid_argument("ADetailer.args must be an array");
+ }
+ if (args.empty() || (args[0].is_boolean() && !args[0].get<bool>())) {
+ return true;
+ }
+ static const std::unordered_set<std::string> keys = {
+ "ad_model", "ad_model_classes", "ad_tab_enable", "ad_prompt", "ad_negative_prompt",
+ "ad_confidence", "ad_mask_filter_method", "ad_mask_k", "ad_mask_min_ratio", "ad_mask_max_ratio",
+ "ad_dilate_erode", "ad_x_offset", "ad_y_offset", "ad_mask_merge_invert", "ad_mask_blur",
+ "ad_denoising_strength", "ad_inpaint_only_masked", "ad_inpaint_only_masked_padding",
+ "ad_use_inpaint_width_height", "ad_inpaint_width", "ad_inpaint_height", "ad_use_steps", "ad_steps",
+ "ad_use_cfg_scale", "ad_cfg_scale", "ad_use_sampler", "ad_sampler", "ad_scheduler",
+ "ad_use_checkpoint", "ad_checkpoint", "ad_use_vae", "ad_vae", "ad_use_noise_multiplier",
+ "ad_noise_multiplier", "ad_use_clip_skip", "ad_clip_skip", "ad_restore_face", "ad_controlnet_model",
+ "ad_controlnet_module", "ad_controlnet_weight", "ad_controlnet_guidance_start", "ad_controlnet_guidance_end", "is_api"};
+ for (const auto& config : args) {
+ if (!config.is_object()) {
+ continue;
+ }
+ auto boolean = [&](const char* key, bool fallback) {
+ if (!config.contains(key))
+ return fallback;
+ if (!config.at(key).is_boolean())
+ throw std::invalid_argument(std::string(key) + " must be boolean");
+ return config.at(key).get<bool>();
+ };
+ auto text = [&](const char* key, const char* fallback) {
+ if (!config.contains(key))
+ return std::string(fallback);
+ if (!config.at(key).is_string())
+ throw std::invalid_argument(std::string(key) + " must be a string");
+ return config.at(key).get<std::string>();
+ };
+ auto number = [&](const char* key, double fallback, double minimum, double maximum) {
+ double value = fallback;
+ if (config.contains(key)) {
+ if (!config.at(key).is_number())
+ throw std::invalid_argument(std::string(key) + " must be numeric");
+ value = config.at(key).get<double>();
+ }
+ if (!std::isfinite(value) || value < minimum || value > maximum)
+ throw std::invalid_argument(std::string(key) + " is out of range");
+ return value;
+ };
+ auto integer = [&](const char* key, int fallback, int minimum, int maximum = std::numeric_limits<int>::max()) {
+ if (config.contains(key) && !config.at(key).is_number_integer())
+ throw std::invalid_argument(std::string(key) + " must be an integer");
+ return static_cast<int>(number(key, fallback, minimum, maximum));
+ };
+ const std::string model = text("ad_model", "None");
+ if (model == "None" || !boolean("ad_tab_enable", true)) {
+ continue;
+ }
+ for (const auto& item : config.items()) {
+ if (keys.count(item.key()) == 0)
+ throw std::invalid_argument("unsupported ADetailer setting: " + item.key());
+ }
+ const auto& configured = runtime.default_gen_params->ad_model_path;
+ if (configured.empty())
+ throw std::invalid_argument("ADetailer requires a server-configured --ad-model");
+ const auto path = fs::u8path(configured);
+ if (model.find_first_of("/\\") != std::string::npos ||
+ (lower_ascii(model) != lower_ascii(path.filename().u8string()) && lower_ascii(model) != lower_ascii(path.stem().u8string() + ".pt"))) {
+ throw std::invalid_argument("ADetailer model must match the server-configured detector");
+ }
+ if (!text("ad_model_classes", "").empty())
+ throw std::invalid_argument("ad_model_classes is not supported");
+ if (boolean("ad_use_checkpoint", false)) {
+ const auto checkpoint = config.contains("ad_checkpoint") && config.at("ad_checkpoint").is_null() ? "" : text("ad_checkpoint", "Use same checkpoint");
+ if (!checkpoint.empty() && checkpoint != "None" && checkpoint != "Use same checkpoint")
+ throw std::invalid_argument("ADetailer checkpoint switching is not supported");
+ }
+ if (boolean("ad_use_vae", false)) {
+ const auto vae = config.contains("ad_vae") && config.at("ad_vae").is_null() ? "" : text("ad_vae", "Use same VAE");
+ if (!vae.empty() && vae != "None" && vae != "Use same VAE")
+ throw std::invalid_argument("ADetailer VAE switching is not supported");
+ }
+ if (boolean("ad_use_noise_multiplier", false) && number("ad_noise_multiplier", 1, 0.5, 1.5) != 1)
+ throw std::invalid_argument("ADetailer noise multiplier is not supported");
+ if (boolean("ad_restore_face", false))
+ throw std::invalid_argument("ADetailer face restoration is not supported");
+ if (text("ad_controlnet_model", "None") != "None")
+ throw std::invalid_argument("ADetailer ControlNet is not supported");
+
+ ServerADetailerPass pass;
+ pass.prompt = text("ad_prompt", "");
+ pass.negative_prompt = text("ad_negative_prompt", "");
+ pass.width = request.gen_params.get_resolved_width();
+ pass.height = request.gen_params.get_resolved_height();
+ pass.strength = static_cast<float>(number("ad_denoising_strength", 0.4, 0, 1));
+ auto append = [&](const char* key, const std::string& value) {
+ if (!pass.extra_args.empty())
+ pass.extra_args += ",";
+ pass.extra_args += std::string(key) + "=" + value;
+ };
+ auto append_number = [&](const char* target, const char* key, double fallback, double minimum, double maximum) {
+ append(target, json(number(key, fallback, minimum, maximum)).dump());
+ };
+ auto append_integer = [&](const char* target, const char* key, int fallback, int minimum, int maximum = std::numeric_limits<int>::max()) {
+ append(target, std::to_string(integer(key, fallback, minimum, maximum)));
+ };
+ append_number("confidence", "ad_confidence", 0.3, 0, 1);
+ append_integer("mask_k_largest", "ad_mask_k", 0, 0);
+ const auto filter = text("ad_mask_filter_method", "Area");
+ if (filter != "Area" && filter != "Confidence")
+ throw std::invalid_argument("invalid ad_mask_filter_method");
+ append("mask_filter_method", lower_ascii(filter));
+ const auto min_ratio = number("ad_mask_min_ratio", 0, 0, 1);
+ const auto max_ratio = number("ad_mask_max_ratio", 1, 0, 1);
+ if (min_ratio > max_ratio)
+ throw std::invalid_argument("ad_mask_min_ratio exceeds ad_mask_max_ratio");
+ append("mask_min_ratio", json(min_ratio).dump());
+ append("mask_max_ratio", json(max_ratio).dump());
+ // Bound pixel operations before converting them to the native mask parameters.
+ const int mask_limit = std::clamp(std::max(pass.width, pass.height), 32, 16384);
+ append_integer("dilate_erode", "ad_dilate_erode", 4, -mask_limit, mask_limit);
+ append_integer("x_offset", "ad_x_offset", 0, -mask_limit, mask_limit);
+ append_integer("y_offset", "ad_y_offset", 0, -mask_limit, mask_limit);
+ append_integer("mask_blur", "ad_mask_blur", 4, 0, mask_limit);
+ append_integer("inpaint_padding", "ad_inpaint_only_masked_padding", 32, 0, mask_limit);
+ append("inpaint_only_masked", boolean("ad_inpaint_only_masked", true) ? "true" : "false");
+ const auto merge = text("ad_mask_merge_invert", "None");
+ if (merge != "None" && merge != "Merge" && merge != "Merge and Invert")
+ throw std::invalid_argument("invalid ad_mask_merge_invert");
+ append("mask_mode", merge == "None" ? "none" : merge == "Merge" ? "merge"
+ : "merge_invert");
+ if (boolean("ad_use_inpaint_width_height", false)) {
+ pass.width = integer("ad_inpaint_width", 512, 1, k_max_generation_dimension);
+ pass.height = integer("ad_inpaint_height", 512, 1, k_max_generation_dimension);
+ }
+ if (boolean("ad_use_steps", false))
+ append_integer("steps", "ad_steps", 28, 1, k_max_adetailer_steps);
+ if (boolean("ad_use_cfg_scale", false))
+ append_number("cfg_scale", "ad_cfg_scale", 7, 0, std::numeric_limits<float>::max());
+ if (boolean("ad_use_clip_skip", false)) {
+ pass.clip_skip = integer("ad_clip_skip", 1, 1);
+ if (pass.clip_skip > 12)
+ throw std::invalid_argument("ad_clip_skip is out of range");
+ }
+ if (boolean("ad_use_sampler", false)) {
+ std::string sampler = text("ad_sampler", "DPM++ 2M Karras");
+ if (sampler != "Use same sampler") {
+ const std::string suffix = " karras";
+ const bool legacy_karras = sampler.size() > suffix.size() && lower_ascii(sampler.substr(sampler.size() - suffix.size())) == suffix;
+ if (legacy_karras)
+ sampler.resize(sampler.size() - suffix.size());
+ const auto method = get_sdapi_sample_method(sampler);
+ if (method == SAMPLE_METHOD_COUNT)
+ throw std::invalid_argument("unsupported ADetailer sampler");
+ append("sample_method", sd_sample_method_name(method));
+ if (legacy_karras && text("ad_scheduler", "Use same scheduler") == "Use same scheduler")
+ append("scheduler", sd_scheduler_name(KARRAS_SCHEDULER));
+ }
+ const auto scheduler = text("ad_scheduler", "Use same scheduler");
+ if (scheduler == "Automatic") {
+ pass.automatic_scheduler = true;
+ } else if (scheduler != "Use same scheduler") {
+ const auto value = str_to_scheduler(scheduler.c_str());
+ if (value == SCHEDULER_COUNT)
+ throw std::invalid_argument("unsupported ADetailer scheduler");
+ append("scheduler", sd_scheduler_name(value));
+ }
+ }
+ request.adetailer_passes.push_back(std::move(pass));
+ }
+ if (!request.adetailer_passes.empty() && img2img && args.size() > 1 && args[1].is_boolean() && args[1].get<bool>()) {
+ throw std::invalid_argument("ADetailer skip img2img is not supported");
+ }
+ request.adetailer_enabled = !request.adetailer_passes.empty();
+ return true;
+ } catch (const std::exception& error) {
+ error_message = std::string("invalid ADetailer args: ") + error.what();
+ return false;
+ }
+}
+
static bool build_sdapi_img_gen_request(const json& j,
ServerRuntime& runtime,
bool img2img,
@@ -259,7 +458,8 @@ static bool build_sdapi_img_gen_request(const json& j,
return false;
}
- return true;
+ return parse_sdapi_adetailer(j, runtime, img2img, request, error_message) &&
+ validate_server_img_gen_request(runtime, request, error_message);
}
static nlohmann::json prepare_info_field(const SDContextParams& ctx_params,
@@ -349,7 +549,7 @@ static void preview_callback(int step, int frame_count, sd_image_t* frames, bool
}
struct SDAPIPreviewGuard {
- SDAPIPreviewGuard(const std::string& id_task, const sd_img_gen_params_t& params, const std::string& job_type) {
+ SDAPIPreviewGuard(const std::string& id_task, const sd_img_gen_params_t& params, const std::string& job_type, bool detailing = false) {
std::lock_guard<std::mutex> lock(g_preview.mutex);
g_preview.id_task = id_task;
g_preview.total = 0;
@@ -358,11 +558,20 @@ struct SDAPIPreviewGuard {
g_preview.step = 0;
g_preview.active = true;
// Each batch item has a base pass and, when enabled, a hires pass.
- g_preview.job_count = static_cast<int64_t>(std::max(1, params.batch_count)) * (params.hires.enabled ? 2 : 1);
+ g_preview.job_count = static_cast<int64_t>(std::max(1, params.batch_count)) * (params.hires.enabled ? 2 : 1) + (detailing ? 1 : 0);
g_preview.job_no = 0;
sd_set_preview_callback(preview_callback, PREVIEW_PROJ, 1, true, false, nullptr);
}
+ void begin_detailing() {
+ sd_set_preview_callback(nullptr, PREVIEW_NONE, 1, false, false, nullptr);
+ std::lock_guard<std::mutex> lock(g_preview.mutex);
+ g_preview.job_type = "adetailer";
+ g_preview.job_no = static_cast<int>(g_preview.job_count - 1);
+ g_preview.step = 0;
+ g_preview.total = 0;
+ }
+
~SDAPIPreviewGuard() {
sd_set_preview_callback(nullptr, PREVIEW_NONE, 1, false, false, nullptr);
std::lock_guard<std::mutex> lock(g_preview.mutex);
@@ -410,13 +619,21 @@ void register_sdapi_endpoints(httplib::Server& svr, ServerRuntime& rt) {
{
std::lock_guard<std::mutex> lock(*runtime->sd_ctx_mutex);
- SDAPIPreviewGuard preview_guard(id_task, img_gen_params, img2img ? "img2img" : "txt2img");
+ const bool detailing = request.adetailer_enabled && !runtime->default_gen_params->ad_model_path.empty();
+ SDAPIPreviewGuard preview_guard(id_task, img_gen_params, img2img ? "img2img" : "txt2img", detailing);
sd_image_t* raw_results = nullptr;
if (!generate_image(runtime->sd_ctx, &img_gen_params, &raw_results, &num_results)) {
raw_results = nullptr;
num_results = 0;
}
results.adopt(raw_results, num_results);
+ if (detailing && !results.empty())
+ preview_guard.begin_detailing();
+ if (!results.empty() && !apply_server_adetailer(*runtime, request, img_gen_params, results)) {
+ res.status = 500;
+ res.set_content(R"({"error":"ADetailer failed"})", "application/json");
+ return;
+ }
}
if (results.empty()) {
diff --git a/examples/server/routes_sdcpp.cpp b/examples/server/routes_sdcpp.cpp
index 7e93e5e..8eb9c9a 100644
--- a/examples/server/routes_sdcpp.cpp
+++ b/examples/server/routes_sdcpp.cpp
@@ -410,9 +410,9 @@ static json make_capabilities_json(ServerRuntime& runtime) {
result["defaults_by_mode"] = defaults_by_mode;
result["limits"] = {
{"min_width", 64},
- {"max_width", 4096},
+ {"max_width", k_max_generation_dimension},
{"min_height", 64},
- {"max_height", 4096},
+ {"max_height", k_max_generation_dimension},
{"max_batch_count", 8},
{"max_queue_size", manager.max_pending_jobs},
{"max_upscale_width", k_max_upscale_dimension},
@@ -457,7 +457,7 @@ static bool parse_img_gen_request(const json& body,
error_message = "invalid generation parameters";
return false;
}
- return true;
+ return validate_server_img_gen_request(runtime, request, error_message);
}
static bool parse_vid_gen_request(const json& body,
diff --git a/examples/server/runtime.cpp b/examples/server/runtime.cpp
index 34140ea..5b33b33 100644
--- a/examples/server/runtime.cpp
+++ b/examples/server/runtime.cpp
@@ -3,6 +3,7 @@
#include <algorithm>
#include <cctype>
#include <chrono>
+#include <cmath>
#include <cstdlib>
#include <filesystem>
#include <mutex>
@@ -18,6 +19,157 @@
namespace fs = std::filesystem;
+bool validate_server_img_gen_request(const ServerRuntime& runtime,
+ const ImgGenJobRequest& request,
+ std::string& error_message) {
+ auto dimensions_valid = [&](double width, double height) {
+ if (!std::isfinite(width) || !std::isfinite(height) ||
+ width < 1 || height < 1 || width > k_max_generation_dimension || height > k_max_generation_dimension) {
+ error_message = "generation width and height must be between 1 and " + std::to_string(k_max_generation_dimension);
+ return false;
+ }
+ return true;
+ };
+ const auto& params = request.gen_params;
+ const int width = params.get_resolved_width();
+ const int height = params.get_resolved_height();
+ if (!dimensions_valid(width, height)) {
+ return false;
+ }
+ if (params.hires_enabled) {
+ const double hires_width = params.hires_width > 0 ? params.hires_width : params.hires_height > 0 ? params.hires_height
+ : std::round(width * params.hires_scale);
+ const double hires_height = params.hires_height > 0 ? params.hires_height : params.hires_width > 0 ? params.hires_width
+ : std::round(height * params.hires_scale);
+ if (!dimensions_valid(hires_width, hires_height)) {
+ return false;
+ }
+ }
+ if (!request.adetailer_enabled || runtime.default_gen_params->ad_model_path.empty()) {
+ return true;
+ }
+ auto pass_valid = [&](const ServerADetailerPass& pass) {
+ if (!dimensions_valid(pass.width, pass.height)) {
+ return false;
+ }
+ // Match the native key normalization, but leave full option parsing to ADetailer.
+ auto trim = [](const std::string& text) {
+ const auto first = text.find_first_not_of(" \t\r\n");
+ return first == std::string::npos ? std::string() : text.substr(first, text.find_last_not_of(" \t\r\n") - first + 1);
+ };
+ std::stringstream stream(pass.extra_args);
+ std::string item;
+ while (std::getline(stream, item, ',')) {
+ const auto equals = item.find('=');
+ if (equals == std::string::npos) {
+ continue;
+ }
+ auto key = trim(item.substr(0, equals));
+ std::transform(key.begin(), key.end(), key.begin(), [](unsigned char c) { return static_cast<char>(std::tolower(c)); });
+ if (key != "inpaint_width" && key != "inpaint_height" && key != "steps") {
+ continue;
+ }
+ const auto value = trim(item.substr(equals + 1));
+ const int minimum = key == "steps" ? 0 : 1;
+ const int maximum = key == "steps" ? k_max_adetailer_steps : k_max_generation_dimension;
+ try {
+ size_t used = 0;
+ const int parsed = std::stoi(value, &used);
+ if (used == value.size() && parsed >= minimum && parsed <= maximum) {
+ continue;
+ }
+ } catch (const std::exception&) {
+ }
+ error_message = "ADetailer " + key + " must be between " + std::to_string(minimum) + " and " + std::to_string(maximum);
+ return false;
+ }
+ return true;
+ };
+ if (request.adetailer_passes.empty()) {
+ ServerADetailerPass pass;
+ pass.extra_args = params.extra_ad_args;
+ return pass_valid(pass);
+ }
+ return std::all_of(request.adetailer_passes.begin(), request.adetailer_passes.end(), pass_valid);
+}
+
+bool apply_server_adetailer(ServerRuntime& runtime,
+ const ImgGenJobRequest& request,
+ const sd_img_gen_params_t& params,
+ SDImageVec& results) {
+ const auto& gen_params = request.gen_params;
+ const auto& ad_model_path = runtime.default_gen_params->ad_model_path;
+ if (!request.adetailer_enabled || ad_model_path.empty()) {
+ return true;
+ }
+ std::string error_message;
+ if (!validate_server_img_gen_request(runtime, request, error_message)) {
+ LOG_ERROR("%s", error_message.c_str());
+ return false;
+ }
+
+ std::vector<ServerADetailerPass> passes = request.adetailer_passes;
+ if (passes.empty()) {
+ ServerADetailerPass pass;
+ pass.prompt = gen_params.ad_prompt;
+ pass.negative_prompt = gen_params.ad_negative_prompt;
+ pass.extra_args = gen_params.extra_ad_args;
+ passes.push_back(std::move(pass));
+ }
+
+ ADetailerCtxPtr ad_ctx(new_adetailer_ctx(ad_model_path.c_str(),
+ runtime.ctx_params->n_threads,
+ runtime.ctx_params->effective_backend.c_str(),
+ runtime.ctx_params->effective_params_backend.c_str()));
+ if (ad_ctx == nullptr) {
+ LOG_ERROR("new_adetailer_ctx failed");
+ return false;
+ }
+
+ for (const auto& pass : passes) {
+ sd_adetailer_params_t ad_params{};
+ ad_params.prompt = pass.prompt.empty() ? nullptr : pass.prompt.c_str();
+ ad_params.negative_prompt = pass.negative_prompt.empty() ? nullptr : pass.negative_prompt.c_str();
+ ad_params.extra_ad_args = pass.extra_args.c_str();
+ for (int i = 0; i < results.count(); ++i) {
+ if (results[i].data == nullptr) {
+ continue;
+ }
+ sd_img_gen_params_t ad_generation_params = params;
+ ad_generation_params.seed = params.seed + i;
+ ad_generation_params.width = pass.width;
+ ad_generation_params.height = pass.height;
+ ad_generation_params.strength = pass.strength;
+ if (pass.clip_skip > 0) {
+ ad_generation_params.clip_skip = pass.clip_skip;
+ }
+ if (pass.automatic_scheduler) {
+ ad_generation_params.sample_params.scheduler = SCHEDULER_COUNT;
+ }
+
+ sd_image_t* detailed_images = nullptr;
+ int detailed_count = 0;
+ if (!adetail_image(ad_ctx.get(),
+ runtime.sd_ctx,
+ results[i],
+ &ad_params,
+ &ad_generation_params,
+ &detailed_images,
+ &detailed_count) ||
+ detailed_count <= 0 || detailed_images == nullptr || detailed_images[0].data == nullptr) {
+ free_sd_images(detailed_images, detailed_count);
+ LOG_ERROR("ADetailer failed for image %d", i + 1);
+ return false;
+ }
+ free(results[i].data);
+ results[i] = detailed_images[0];
+ detailed_images[0] = {0, 0, 0, nullptr};
+ free_sd_images(detailed_images, detailed_count);
+ }
+ }
+ return true;
+}
+
static std::string lower_ascii(std::string value) {
std::transform(value.begin(), value.end(), value.begin(), [](unsigned char c) {
return static_cast<char>(std::tolower(c));
diff --git a/examples/server/runtime.h b/examples/server/runtime.h
index d9272f6..8986f38 100644
--- a/examples/server/runtime.h
+++ b/examples/server/runtime.h
@@ -67,18 +67,38 @@ struct ServerRuntime {
ServerModels* models;
};
+struct ServerADetailerPass {
+ std::string prompt;
+ std::string negative_prompt;
+ std::string extra_args;
+ int width = 512;
+ int height = 512;
+ float strength = 0.4f;
+ int clip_skip = -1;
+ bool automatic_scheduler = false;
+};
+
struct ImgGenJobRequest {
SDGenerationParams gen_params;
std::string output_format = "png";
int output_compression = 100;
std::string preview_mode = "none";
int preview_interval = 1;
+ bool adetailer_enabled = true;
+ std::vector<ServerADetailerPass> adetailer_passes;
sd_img_gen_params_t to_sd_img_gen_params_t() {
return gen_params.to_sd_img_gen_params_t();
}
};
+constexpr int k_max_generation_dimension = 4096;
+constexpr int k_max_adetailer_steps = 100;
+
+bool validate_server_img_gen_request(const ServerRuntime& runtime,
+ const ImgGenJobRequest& request,
+ std::string& error_message);
+
struct VidGenJobRequest {
SDGenerationParams gen_params;
std::string output_format = "webm";
@@ -111,3 +131,7 @@ void refresh_lora_cache(ServerRuntime& rt);
std::string get_lora_full_path(ServerRuntime& rt, const std::string& path);
void refresh_upscaler_cache(ServerRuntime& rt);
int64_t unix_timestamp_now_ms();
+bool apply_server_adetailer(ServerRuntime& runtime,
+ const ImgGenJobRequest& request,
+ const sd_img_gen_params_t& params,
+ SDImageVec& results);
diff --git a/src/detailer.cpp b/src/detailer.cpp
index 72d01dc..ec4d606 100644
--- a/src/detailer.cpp
+++ b/src/detailer.cpp
@@ -291,7 +291,9 @@ static void filter_and_sort_detections(std::vector<ADetailerDetection>* detectio
std::partial_sort(detections->begin(),
detections->begin() + params.mask_k_largest,
detections->end(),
- [](const auto& a, const auto& b) { return detection_area(a) > detection_area(b); });
+ [&](const auto& a, const auto& b) {
+ return params.mask_filter_by_confidence ? a.confidence > b.confidence : detection_area(a) > detection_area(b);
+ });
detections->resize(params.mask_k_largest);
}
@@ -545,7 +547,16 @@ static sd_image_t resize_crop_mask(const Mask& source, const CropRegion& crop, i
return output;
}
-static sd_image_t copy_as_rgb(const sd_image_t& source) {
+static sd_image_t copy_for_detailing(const sd_image_t& source) {
+ if (source.channel == 4) {
+ sd_image_t output = source;
+ size_t bytes = static_cast<size_t>(source.width) * source.height * source.channel;
+ output.data = static_cast<uint8_t*>(malloc(bytes));
+ if (output.data != nullptr) {
+ memcpy(output.data, source.data, bytes);
+ }
+ return output;
+ }
CropRegion full{0, 0, static_cast<int>(source.width), static_cast<int>(source.height)};
return resize_crop_image(source, full, source.width, source.height);
}
@@ -566,7 +577,7 @@ static void composite_crop(sd_image_t* destination,
continue;
}
float source_x = (x - crop.x1 + 0.5f) * scale_x - 0.5f;
- size_t destination_index = (static_cast<size_t>(y) * destination->width + x) * 3;
+ size_t destination_index = (static_cast<size_t>(y) * destination->width + x) * destination->channel;
for (int c = 0; c < 3; ++c) {
float generated_value = bilinear_channel(generated, source_x, source_y, c) * 255.f;
float original_value = destination->data[destination_index + c];
@@ -787,6 +798,10 @@ static bool parse_adetailer_params(const sd_adetailer_params_t& public_params,
ok = parse_int(value, ¶ms->max_detections);
} else if (key == "mask_k_largest") {
ok = parse_int(value, ¶ms->mask_k_largest);
+ } else if (key == "mask_filter_method") {
+ const std::string method = lower_copy(value);
+ ok = method == "area" || method == "confidence";
+ params->mask_filter_by_confidence = method == "confidence";
} else if (key == "mask_min_ratio") {
ok = parse_float(value, ¶ms->mask_min_ratio);
} else if (key == "mask_max_ratio") {
@@ -805,6 +820,8 @@ static bool parse_adetailer_params(const sd_adetailer_params_t& public_params,
ok = parse_int(value, ¶ms->mask_blur);
} else if (key == "inpaint_padding") {
ok = parse_int(value, ¶ms->inpaint_padding);
+ } else if (key == "inpaint_only_masked") {
+ ok = parse_bool(value, ¶ms->inpaint_only_masked);
} else if (key == "inpaint_width") {
ok = parse_int(value, ¶ms->inpaint_width);
} else if (key == "inpaint_height") {
@@ -920,7 +937,7 @@ bool adetail_image(adetailer_ctx_t* context,
filter_and_sort_detections(&detections, input_image.width, input_image.height, params);
auto masks = make_masks(detections, input_image.width, input_image.height, params);
- sd_image_t current = copy_as_rgb(input_image);
+ sd_image_t current = copy_for_detailing(input_image);
if (current.data == nullptr) {
return false;
}
@@ -939,12 +956,13 @@ bool adetail_image(adetailer_ctx_t* context,
if (!mask_bbox(masks[i], &crop)) {
continue;
}
- crop = expand_crop(crop,
- current.width,
- current.height,
- params.inpaint_padding,
- params.inpaint_width,
- params.inpaint_height);
+ crop = params.inpaint_only_masked ? expand_crop(crop,
+ current.width,
+ current.height,
+ params.inpaint_padding,
+ params.inpaint_width,
+ params.inpaint_height)
+ : CropRegion{0, 0, static_cast<int>(current.width), static_cast<int>(current.height)};
sd_image_t local_image = resize_crop_image(current, crop, params.inpaint_width, params.inpaint_height);
sd_image_t local_mask = resize_crop_mask(masks[i], crop, params.inpaint_width, params.inpaint_height);
if (local_image.data == nullptr || local_mask.data == nullptr) {
diff --git a/src/detailer.h b/src/detailer.h
index 17a81a1..0e3ae50 100644
--- a/src/detailer.h
+++ b/src/detailer.h
@@ -27,30 +27,32 @@ enum ADetailerSort {
};
struct ADetailerParams {
- const char* prompt = nullptr;
- const char* negative_prompt = nullptr;
- int input_size = 640;
- float confidence = 0.3f;
- float nms_threshold = 0.45f;
- int max_detections = 100;
- int mask_k_largest = 0;
- float mask_min_ratio = 0.f;
- float mask_max_ratio = 1.f;
- int dilate_erode = 4;
- int x_offset = 0;
- int y_offset = 0;
- bool merge_masks = false;
- bool invert_mask = false;
- int mask_blur = 4;
- int inpaint_padding = 32;
- int inpaint_width = 512;
- int inpaint_height = 512;
- float denoising_strength = 0.4f;
- int steps = 0;
- float cfg_scale = -1.f;
- sample_method_t sample_method = SAMPLE_METHOD_COUNT;
- scheduler_t scheduler = SCHEDULER_COUNT;
- ADetailerSort sort_by = ADETAILER_SORT_NONE;
+ const char* prompt = nullptr;
+ const char* negative_prompt = nullptr;
+ int input_size = 640;
+ float confidence = 0.3f;
+ float nms_threshold = 0.45f;
+ int max_detections = 100;
+ int mask_k_largest = 0;
+ bool mask_filter_by_confidence = false;
+ float mask_min_ratio = 0.f;
+ float mask_max_ratio = 1.f;
+ int dilate_erode = 4;
+ int x_offset = 0;
+ int y_offset = 0;
+ bool merge_masks = false;
+ bool invert_mask = false;
+ int mask_blur = 4;
+ int inpaint_padding = 32;
+ bool inpaint_only_masked = true;
+ int inpaint_width = 512;
+ int inpaint_height = 512;
+ float denoising_strength = 0.4f;
+ int steps = 0;
+ float cfg_scale = -1.f;
+ sample_method_t sample_method = SAMPLE_METHOD_COUNT;
+ scheduler_t scheduler = SCHEDULER_COUNT;
+ ADetailerSort sort_by = ADETAILER_SORT_NONE;
};
struct ADetailerGGML {