HaRP in docker, behind HAproxy

[stefangweichinger]

I am starting fresh with HaRP, so maybe I have multiple mistakes in my approach.

docker-image nextcloud:32.0.3 in docker-compose stack, behind a HAproxy setup on a pfSense-Plus-25.11

The docker-host runs on 192.168.220.222
It's a debian-13.3 host

# docker info
Client:
 Version:    26.1.5+dfsg1
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc.)
    Version:  2.26.1-4
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

 Server Version: 26.1.5+dfsg1

The PROXY-IP in that subnet is 192.168.220.254

I have set up HAproxy to forward "https://nctest.my.tld" to 192.168.220.222:8780

The docker-compose.yml for harp:

# cat docker-compose.yml 
services:
  appapi-harp:
    image: ghcr.io/nextcloud/nextcloud-appapi-harp:release
    restart: unless-stopped
    environment:
      - HP_SHARED_KEY=xxxx
      - NC_INSTANCE_URL="https://nctest.my.tld"
      #- HP_EXAPPS_ADDRESS="192.168.220.222:8780"
      - HP_TRUSTED_PROXY_IPS="192.168.220.254"
      - HP_LOG_LEVEL=info
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./harp-certs:/certs
    ports:
      - "8780:8780"
      - "8782:8782"

I am not sure if I get the picture right, if external URL matches the internal config, etc

I also see an issue #69 mentioning debian-13, not sure if I simply should wait for fixes coming ;-)

nextcloud

I can add that proxy in nextcloud, the connection test works.
Somehow the password isn't stored consistently (that might be a nc-issue, sure).

See my logs:

appapi-harp-1  | INFO: Creating /haproxy.cfg from haproxy.cfg.template...
appapi-harp-1  | INFO: No /certs/cert.pem found, disabling HTTPS frontends...
appapi-harp-1  | INFO: Final /haproxy.cfg:
appapi-harp-1  | # SPDX-FileCopyrightText: 2025 Nextcloud GmbH and Nextcloud contributors
appapi-harp-1  | # SPDX-License-Identifier: AGPL-3.0-or-later
appapi-harp-1  | 
appapi-harp-1  | ###############################################################################
appapi-harp-1  | # haproxy.cfg.template
appapi-harp-1  | #
appapi-harp-1  | # This template is processed by envsubst in start.sh to replace variables:
appapi-harp-1  | #   HP_EXAPPS_ADDRESS,
appapi-harp-1  | #   HP_EXAPPS_HTTPS_ADDRESS,
appapi-harp-1  | #   HP_SPOA_ADDRESS,
appapi-harp-1  | #   HP_TIMEOUT_CONNECT,
appapi-harp-1  | #   HP_TIMEOUT_CLIENT,
appapi-harp-1  | #   HP_TIMEOUT_SERVER,
appapi-harp-1  | #
appapi-harp-1  | ## If /certs/cert.pem is not found, lines containing "_HTTPS_FRONTEND_" are
appapi-harp-1  | # commented out automatically in start.sh.
appapi-harp-1  | ###############################################################################
appapi-harp-1  | 
appapi-harp-1  | global
appapi-harp-1  |     log stdout local0 info
appapi-harp-1  |     maxconn 8192
appapi-harp-1  |     ca-base /etc/ssl/certs
appapi-harp-1  | 
appapi-harp-1  | defaults
appapi-harp-1  |     log global
appapi-harp-1  |     option httplog
appapi-harp-1  |     option dontlognull
appapi-harp-1  |     timeout connect 30s
appapi-harp-1  |     timeout client 30s
appapi-harp-1  |     timeout server 1800s
appapi-harp-1  | 
appapi-harp-1  | 
appapi-harp-1  | ###############################################################################
appapi-harp-1  | # FRONTEND: ex_apps (HTTP)
appapi-harp-1  | ###############################################################################
appapi-harp-1  | frontend ex_apps
appapi-harp-1  |     mode http
appapi-harp-1  |     bind 0.0.0.0:8780
appapi-harp-1  | 
appapi-harp-1  |     filter spoe engine exapps-spoe config /etc/haproxy/spoe-agent.conf
appapi-harp-1  |     http-request silent-drop if { var(txn.exapps.bad_request) -m int eq 1 }
appapi-harp-1  |     http-request return status 401 content-type text/plain string "401 Unauthorized" if { var(txn.exapps.unauthorized) -m int eq 1 }
appapi-harp-1  |     http-request return status 403 content-type text/plain string "403 Forbidden" if { var(txn.exapps.forbidden) -m int eq 1 }
appapi-harp-1  |     http-request return status 404 content-type text/plain string "404 Not Found" if { var(txn.exapps.not_found) -m int eq 1 }
appapi-harp-1  |     use_backend %[var(txn.exapps.backend)]
appapi-harp-1  | 
appapi-harp-1  | ###############################################################################
appapi-harp-1  | # FRONTEND: ex_apps_https (only enabled if /certs/cert.pem exists)
appapi-harp-1  | ###############################################################################
appapi-harp-1  | #_HTTPS_FRONTEND_ frontend ex_apps_https
appapi-harp-1  | #_HTTPS_FRONTEND_     mode http
appapi-harp-1  | #_HTTPS_FRONTEND_     bind 0.0.0.0:8781 ssl crt /certs/cert.pem
appapi-harp-1  | 
appapi-harp-1  | #_HTTPS_FRONTEND_     filter spoe engine exapps-spoe config /etc/haproxy/spoe-agent.conf
appapi-harp-1  | #_HTTPS_FRONTEND_     http-request silent-drop if { var(txn.exapps.bad_request) -m int eq 1 }
appapi-harp-1  | #_HTTPS_FRONTEND_     http-request return status 401 content-type text/plain string "401 Unauthorized" if { var(txn.exapps.unauthorized) -m int eq 1 }
appapi-harp-1  | #_HTTPS_FRONTEND_     http-request return status 403 content-type text/plain string "403 Forbidden" if { var(txn.exapps.forbidden) -m int eq 1 }
appapi-harp-1  | #_HTTPS_FRONTEND_     http-request return status 404 content-type text/plain string "404 Not Found" if { var(txn.exapps.not_found) -m int eq 1 }
appapi-harp-1  | #_HTTPS_FRONTEND_     use_backend %[var(txn.exapps.backend)]
appapi-harp-1  | 
appapi-harp-1  | ###############################################################################
appapi-harp-1  | # BACKENDS: ex_apps & ex_apps_backend_w_bruteforce
appapi-harp-1  | ###############################################################################
appapi-harp-1  | backend ex_apps_backend
appapi-harp-1  |     mode http
appapi-harp-1  |     server frp_server 0.0.0.0
appapi-harp-1  |     http-request set-path %[var(txn.exapps.target_path)]
appapi-harp-1  |     http-request set-dst var(txn.exapps.target_ip)
appapi-harp-1  |     http-request set-dst-port var(txn.exapps.target_port)
appapi-harp-1  |     http-request set-header EX-APP-ID %[var(txn.exapps.exapp_id)]
appapi-harp-1  |     http-request set-header EX-APP-VERSION %[var(txn.exapps.exapp_version)]
appapi-harp-1  |     http-request set-header AUTHORIZATION-APP-API %[var(txn.exapps.exapp_token)]
appapi-harp-1  |     http-request set-header AA-VERSION "32"  # TO-DO: temporary, remove it after we update all ExApps.
appapi-harp-1  | 
appapi-harp-1  | backend ex_apps_backend_w_bruteforce
appapi-harp-1  |     mode http
appapi-harp-1  |     server frp_server 0.0.0.0
appapi-harp-1  |     http-request set-path %[var(txn.exapps.target_path)]
appapi-harp-1  |     http-request set-dst var(txn.exapps.target_ip)
appapi-harp-1  |     http-request set-dst-port var(txn.exapps.target_port)
appapi-harp-1  |     http-request set-header EX-APP-ID %[var(txn.exapps.exapp_id)]
appapi-harp-1  |     http-request set-header EX-APP-VERSION %[var(txn.exapps.exapp_version)]
appapi-harp-1  |     http-request set-header AUTHORIZATION-APP-API %[var(txn.exapps.exapp_token)]
appapi-harp-1  |     http-request set-header AA-VERSION "32"  # TO-DO: temporary, remove it after we update all ExApps.
appapi-harp-1  |     filter spoe engine exapps-bruteforce-protection-spoe config /etc/haproxy/spoe-agent.conf
appapi-harp-1  | 
appapi-harp-1  | ###############################################################################
appapi-harp-1  | # BACKEND: nextcloud_control (HTTP)
appapi-harp-1  | ###############################################################################
appapi-harp-1  | backend nextcloud_control_backend
appapi-harp-1  |     mode http
appapi-harp-1  |     server nextcloud_control 127.0.0.1:8200
appapi-harp-1  |     http-request set-path %[var(txn.exapps.target_path)]
appapi-harp-1  | 
appapi-harp-1  | ###############################################################################
appapi-harp-1  | # BACKEND: docker_engine (HTTP)
appapi-harp-1  | ###############################################################################
appapi-harp-1  | backend docker_engine_backend
appapi-harp-1  |     mode http
appapi-harp-1  |     server frp_server 127.0.0.1
appapi-harp-1  |     http-request set-dst-port var(txn.exapps.target_port)
appapi-harp-1  |     http-request set-path %[var(txn.exapps.target_path)]
appapi-harp-1  | 
appapi-harp-1  |     # docker system _ping
appapi-harp-1  |     http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/_ping$ } METH_GET
appapi-harp-1  |     # docker inspect image
appapi-harp-1  |     http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/images/.*/json } METH_GET
appapi-harp-1  |     # container inspect: GET containers/%s/json
appapi-harp-1  |     http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/json } METH_GET
appapi-harp-1  |     # container inspect: GET containers/%s/logs
appapi-harp-1  |     http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/logs } METH_GET
appapi-harp-1  | 
appapi-harp-1  |     # image pull: POST images/create?fromImage=%s
appapi-harp-1  |     http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/images/create } METH_POST
appapi-harp-1  |     http-request deny
appapi-harp-1  | 
appapi-harp-1  | 
appapi-harp-1  | backend agents
appapi-harp-1  |     mode tcp
appapi-harp-1  |     timeout connect 5s
appapi-harp-1  |     timeout server  3m
appapi-harp-1  |     option spop-check
appapi-harp-1  |     server agent1 127.0.0.1:9600 check
appapi-harp-1  | INFO: FRP server configuration generated at /frps.toml.
appapi-harp-1  | INFO: Detected /var/run/docker.sock, generating /frpc-docker.toml configuration file...
appapi-harp-1  | INFO: Starting Python HaProxy Agent on 127.0.0.1:8200 and 127.0.0.1:9600...
appapi-harp-1  | INFO: Waiting for HaRP Agent HTTP (GET http://127.0.0.1:8200/info) to be ready...
appapi-harp-1  | [2026-01-14T12:13:08+0000] [ERROR] Invalid value for HP_TRUSTED_PROXY_IPS: '"192.168.220.254"' does not appear to be an IPv4 or IPv6 network. Client IP detection from headers is disabled. The X-Forwarded-For and X-Real-IP headers will not be respected. This can lead to the outer proxy's IP being blocked during a bruteforce attempt instead of the actual client's IP.
appapi-harp-1  | [2026-01-14T12:13:08+0000] [INFO] Starting both servers: SPOA on 127.0.0.1:9600, HTTP on 127.0.0.1:8200
appapi-harp-1  | [2026-01-14T12:13:08+0000] [INFO] HTTP server listening at 127.0.0.1:8200
appapi-harp-1  | [2026-01-14T12:13:08+0000] [INFO] HAProxy SPO Agent listening at 127.0.0.1:9600
appapi-harp-1  | [2026-01-14T12:13:08+0000] [INFO] 127.0.0.1 [14/Jan/2026:12:13:08 +0000] "GET /info HTTP/1.1" 200 175 "-" "curl/8.14.1"
appapi-harp-1  | INFO: Waiting for SPOA port 127.0.0.1:9600...
appapi-harp-1  | INFO: Starting FRP server on 0.0.0.0:8782...
appapi-harp-1  | INFO: Waiting for FRP server port 127.0.0.1:8782...
appapi-harp-1  | INFO: Starting FRP client for Docker Engine...
appapi-harp-1  | INFO: Starting HAProxy...
appapi-harp-1  | 2026-01-14 12:13:09.495 [I] [sub/root.go:142] start frpc service for config file [/frpc-docker.toml]
appapi-harp-1  | 2026-01-14 12:13:09.495 [I] [client/service.go:295] try to connect to server...
appapi-harp-1  | [NOTICE]   (1) : Initializing new worker (52)
appapi-harp-1  | [2026-01-14T12:13:09+0000] [INFO] 127.0.0.1 [14/Jan/2026:12:13:09 +0000] "POST /frp_handler?op=Login&version=0.1.0 HTTP/1.1" 200 194 "-" "Go-http-client/1.1"
appapi-harp-1  | 2026-01-14 12:13:09.501 [I] [client/service.go:287] [a3c00f7d24f46f36] login to server success, get run id [a3c00f7d24f46f36]
appapi-harp-1  | 2026-01-14 12:13:09.501 [I] [proxy/proxy_manager.go:173] [a3c00f7d24f46f36] proxy added: [bundled-deploy-daemon]
appapi-harp-1  | 2026-01-14 12:13:09.501 [I] [client/control.go:168] [a3c00f7d24f46f36] [bundled-deploy-daemon] start proxy success
appapi-harp-1  | [NOTICE]   (1) : Loading success.
appapi-harp-1  | [2026-01-14T12:13:09+0000] [INFO] [a8d550f8] Received request on key 'exapps_msg'
appapi-harp-1  | [2026-01-14T12:13:09+0000] [INFO] [a8d550f8] Found 1 matching handlers, awaiting response...
appapi-harp-1  | [2026-01-14T12:13:09+0000] [ERROR] Invalid request path, cannot find AppID: /index.php/login
appapi-harp-1  | [2026-01-14T12:13:09+0000] [WARNING] Recorded failure for IP 192.168.220.254. Failures in window: 1
appapi-harp-1  | [2026-01-14T12:13:09+0000] [INFO] [a8d550f8] Responding with combined payload of 50 bytes
appapi-harp-1  | <134>Jan 14 12:13:09 haproxy[52]: 192.168.220.254:17620 [14/Jan/2026:12:13:09.677] ex_apps ex_apps/<NOSRV> 1/-1/-1/-1/1 404 92 - - LR-- 1/1/0/0/0 0/0 "GET /index.php/login HTTP/1.1"
appapi-harp-1  | [2026-01-14T12:13:10+0000] [INFO] [a8d550f8] Received request on key 'exapps_msg'
appapi-harp-1  | [2026-01-14T12:13:10+0000] [INFO] [a8d550f8] Found 1 matching handlers, awaiting response...
appapi-harp-1  | [2026-01-14T12:13:10+0000] [ERROR] Invalid request path, cannot find AppID: /index.php/login
appapi-harp-1  | [2026-01-14T12:13:10+0000] [WARNING] Recorded failure for IP 192.168.220.254. Failures in window: 2
appapi-harp-1  | [2026-01-14T12:13:10+0000] [INFO] [a8d550f8] Responding with combined payload of 50 bytes
appapi-harp-1  | <134>Jan 14 12:13:10 haproxy[52]: 192.168.220.254:18736 [14/Jan/2026:12:13:10.680] ex_apps ex_apps/<NOSRV> 1/-1/-1/-1/1 404 92 - - LR-- 1/1/0/0/0 0/0 "GET /index.php/login HTTP/1.1"
appapi-harp-1  | [2026-01-14T12:13:11+0000] [INFO] [a8d550f8] Received request on key 'exapps_msg'
appapi-harp-1  | [2026-01-14T12:13:11+0000] [INFO] [a8d550f8] Found 1 matching handlers, awaiting response...

I see this:

[ERROR] Invalid value for HP_TRUSTED_PROXY_IPS: '"192.168.220.254"' does not appear to be an IPv4 or IPv6 network.

Seems as if I have to allow a full subnet here? Although that's a bit generous, right?

I will try to fix that one issue.

Maybe someone can tell me where I have a mistake or if there's something upstream.

[stefangweichinger]

progress:

the connection seems to be OK now, but I see:

appapi-harp-1  | [NOTICE]   (1) : Loading success.
appapi-harp-1  | [2026-01-14T13:53:27+0000] [ERROR] Failed to start container 'nc_app_test-deploy' for cert install (status 404): {"message":"network nctest_internal not found"}
appapi-harp-1  | 
appapi-harp-1  | [2026-01-14T13:53:27+0000] [ERROR] Error during certificate installation for 'nc_app_test-deploy': Service Unavailable
appapi-harp-1  | [2026-01-14T13:53:27+0000] [WARNING] Container 'nc_app_test-deploy' not found, cannot start.

The network "nc_internal" definitely exists and I start the container like:

services:
  appapi-harp:
    image: ghcr.io/nextcloud/nextcloud-appapi-harp:release
    restart: unless-stopped
    # network_mode: host
    networks:
      - default
      - nctest_internal
    environment:
      - HP_SHARED_KEY=xxx
      - NC_INSTANCE_URL=https://nctest.my.tld
      - HP_TRUSTED_PROXY_IPS=192.168.220.254
      - HP_BLACKLIST_COUNT=300
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./harp-certs:/certs
    ports:
      - "8780:8780"
      - "8782:8782"

I use "nc_internal" for the internal connections between nc, redis and postgres, for example.

[stefangweichinger]

I tried to run the harp-container in that one network only, didn't help:

networks:
  - nctest_internal