Compare commits

...

23 Commits

Author SHA1 Message Date
Antonio Pauletich
c83dbd7baf test(chromium): cover header and footer image deduplication 2026-10-05 13:32:59 +02:00
dependabot[bot]
cc9cd5d82d chore(deps): bump github.com/shirou/gopsutil/v4 from 4.26.8 to 4.26.9
Bumps [github.com/shirou/gopsutil/v4](https://github.com/shirou/gopsutil) from 4.26.8 to 4.26.9.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v4.26.8...v4.26.9)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil/v4
  dependency-version: 4.26.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-02 18:59:45 +02:00
dependabot[bot]
768f87bb1c chore(deps): bump github.com/labstack/echo/v5 from 5.3.1 to 5.4.0
Bumps [github.com/labstack/echo/v5](https://github.com/labstack/echo) from 5.3.1 to 5.4.0.
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v5.3.1...v5.4.0)

---
updated-dependencies:
- dependency-name: github.com/labstack/echo/v5
  dependency-version: 5.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-02 18:59:37 +02:00
dependabot[bot]
a5bc10b9d8 chore(deps-dev): bump prettier-plugin-sh from 0.19.0 to 0.20.2
Bumps [prettier-plugin-sh](https://github.com/un-ts/prettier) from 0.19.0 to 0.20.2.
- [Release notes](https://github.com/un-ts/prettier/releases)
- [Changelog](https://github.com/un-ts/prettier/blob/master/CHANGELOG.md)
- [Commits](https://github.com/un-ts/prettier/compare/prettier-plugin-sh@0.19.0...prettier-plugin-sh@0.20.2)

---
updated-dependencies:
- dependency-name: prettier-plugin-sh
  dependency-version: 0.20.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-02 18:59:20 +02:00
dependabot[bot]
ffc3cb98af chore(deps-dev): bump prettier from 3.9.8 to 3.9.9
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.8 to 3.9.9.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.8...3.9.9)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.9
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-02 18:59:08 +02:00
dependabot[bot]
8b46090eac chore(deps-dev): bump prettier from 3.9.6 to 3.9.8
Bumps [prettier](https://github.com/prettier/prettier) from 3.9.6 to 3.9.8.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.9.6...3.9.8)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.9.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-25 06:46:33 +02:00
dependabot[bot]
39e43205b1 chore(deps): bump golang.org/x/net from 0.58.0 to 0.59.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.58.0 to 0.59.0.
- [Commits](https://github.com/golang/net/compare/v0.58.0...v0.59.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.59.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-18 09:27:35 +02:00
dependabot[bot]
c4c9ea12e5 chore(deps): bump golang.org/x/text from 0.41.0 to 0.42.0
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.41.0 to 0.42.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.41.0...v0.42.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-18 07:02:04 +02:00
Julien Neuhart
ff817f6cf8 chore(deps)!: migrate from regexp2 v1 to regexp2 v2 2026-09-17 15:33:55 +02:00
Julien Neuhart
ab18384a51 build(dockerfile): update AWS Lambda Web Adapter to 1.0.1 2026-09-17 15:33:55 +02:00
Julien Neuhart
70783a01c6 fix(gotenberg): retry pattern matches aborted by a process pause 2026-09-16 21:23:43 +02:00
Julien Neuhart
2f9020455a chore(deps)!: migrate from echo v4 to echo v5 2026-09-16 19:38:45 +02:00
Julien Neuhart
8e850b1c93 fix(outbound): ignore IPv6 zone identifiers when classifying addresses 2026-09-13 10:15:07 +02:00
Julien Neuhart
430ed38cc5 build(makefile): re-enable the embedlit go fix modernizer 2026-09-11 16:13:06 +02:00
Julien Neuhart
623e93bf3e chore(deps): update Go to 1.27.1 2026-09-11 16:13:06 +02:00
dependabot[bot]
fe4fb9416d chore(deps): bump golang.org/x/sync from 0.22.0 to 0.23.0 (#1658)
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/sync/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-11 07:20:22 +02:00
Julien Neuhart
fb7536a526 fix(chromium): disable the WebUI omnibox popup preloaded at browser start 2026-09-09 19:38:40 +02:00
Julien Neuhart
b16ce08da7 perf(chromium): skip formatting a discarded debug message per response 2026-09-08 18:35:03 +02:00
Julien Neuhart
fcfd590169 perf(api): build the access log with a single record 2026-09-08 18:35:03 +02:00
Julien Neuhart
b39b8c76aa perf(gotenberg): match allow and deny patterns without recompiling them 2026-09-08 18:34:58 +02:00
Julien Neuhart
06ed58b6e7 ci(build): reuse the Docker build cache across runs 2026-09-08 18:34:54 +02:00
Julien Neuhart
34b7b4845e perf(pdfengines): process a request's files concurrently behind an opt-in ceiling 2026-09-08 18:34:51 +02:00
Julien Neuhart
ab3832d9e5 test(integration): drop the per-scenario Docker network 2026-09-08 18:34:43 +02:00
57 changed files with 1920 additions and 601 deletions

View File

@@ -83,12 +83,16 @@ runs:
INPUT_PLATFORM: ${{ inputs.platform }}
INPUT_ALTERNATE_REPOSITORY: ${{ inputs.alternate_repository }}
INPUT_DRY_RUN: ${{ inputs.dry_run }}
# Exporting the build cache needs a registry login. Forks run without
# credentials, so they import the cache but never export it.
INPUT_CACHE_WRITABLE: ${{ inputs.docker_hub_username != '' }}
run: |
.github/actions/build-test-push/build.sh \
--version "$INPUT_VERSION" \
--platform "$INPUT_PLATFORM" \
--alternate-repository "$INPUT_ALTERNATE_REPOSITORY" \
--dry-run "$INPUT_DRY_RUN"
--dry-run "$INPUT_DRY_RUN" \
--cache-writable "$INPUT_CACHE_WRITABLE"
- name: Run integration tests
if: inputs.skip_integrations_tests != 'true'

View File

@@ -12,6 +12,7 @@ version=""
platform=""
alternate_repository=""
dry_run=""
cache_writable=""
while [[ $# -gt 0 ]]; do
case $1 in
@@ -31,6 +32,10 @@ while [[ $# -gt 0 ]]; do
dry_run="$2"
shift 2
;;
--cache-writable)
cache_writable="$2"
shift 2
;;
*)
echo "Unknown option $1"
exit 1
@@ -44,11 +49,41 @@ echo
echo "Gotenberg version: $version"
echo "Target platform: $platform"
# The build cache lives under the canonical repository, captured before the
# alternate-repository override below. Pull requests build into "snapshot", so
# deriving the cache ref after the override would give them a cache namespace
# of their own and they would never import what main published, which is the
# population that benefits most.
cache_image="$DOCKER_REGISTRY/$DOCKER_REPOSITORY"
# Layers are per-architecture, so each platform keeps its own cache manifest.
cache_platform="${platform//\//-}"
# Layers running "apt-get upgrade" install whatever versions are current at
# build time, and the packages are deliberately not pinned. A persistent cache
# would turn those into hits and freeze security patches into a published
# image until debian:13-slim itself changes digest. Keying them on the ISO week
# bounds that staleness to seven days while leaving every build within a week
# free to reuse the cache.
apt_snapshot="$(date -u +%G-W%V)"
# Only a build that is not redirected to an alternate repository writes the
# cache, so a pull request cannot make its own state the baseline for main.
# Reading stays enabled everywhere, including forks, since the cache ref is
# public and needs no credentials.
cache_to_enabled="false"
if [ "$cache_writable" = "true" ] && [ -z "$alternate_repository" ]; then
cache_to_enabled="true"
fi
if [ -n "$alternate_repository" ]; then
DOCKER_REPOSITORY=$alternate_repository
echo "⚠️ Using $alternate_repository for DOCKER_REPOSITORY"
fi
echo "Build cache: $cache_image:buildcache-<target>-$cache_platform (write: $cache_to_enabled)"
echo "APT snapshot: $apt_snapshot"
if [ "$dry_run" = "true" ]; then
echo "🚧 Dry run"
fi
@@ -189,12 +224,36 @@ join() {
echo "$*"
}
# cache_flags echoes the buildx cache arguments for a build target. Each target
# keeps its own manifest so that the Chromium and LibreOffice variants, which
# branch from common-stage rather than from each other, do not overwrite one
# another's entry.
#
# mode=max exports intermediate stages too, not just the final layers, which is
# what makes the expensive apt and jlink stages reusable. type=registry, not
# type=gha: the GitHub Actions cache is capped at 10 GB per repository and is
# already carrying the Go and golangci-lint caches that the lint and test jobs
# depend on. Multi-GB image layers across five platforms would evict them.
cache_flags() {
local target="$1"
local ref="$cache_image:buildcache-$target-$cache_platform"
local flags="--cache-from type=registry,ref=$ref"
if [ "$cache_to_enabled" = "true" ]; then
flags="$flags --cache-to type=registry,ref=$ref,mode=max"
fi
echo "$flags"
}
no_arch_tag="$DOCKER_REGISTRY/$DOCKER_REPOSITORY:$version"
# Full variant.
cmd="docker buildx build \
--target gotenberg \
--build-arg GOTENBERG_VERSION=$version \
--build-arg APT_SNAPSHOT=$apt_snapshot \
$(cache_flags gotenberg) \
--platform $platform \
--load \
${tags_flags[*]} \
@@ -207,6 +266,8 @@ run_cmd "$cmd"
cmd="docker buildx build \
--target gotenberg-chromium \
--build-arg GOTENBERG_VERSION=$version \
--build-arg APT_SNAPSHOT=$apt_snapshot \
$(cache_flags gotenberg-chromium) \
--platform $platform \
--load \
${tags_chromium_flags[*]} \
@@ -218,6 +279,8 @@ run_cmd "$cmd"
cmd="docker buildx build \
--target gotenberg-libreoffice \
--build-arg GOTENBERG_VERSION=$version \
--build-arg APT_SNAPSHOT=$apt_snapshot \
$(cache_flags gotenberg-libreoffice) \
--platform $platform \
--load \
${tags_libreoffice_flags[*]} \
@@ -230,6 +293,8 @@ if [ "$platform" = "linux/amd64" ]; then
cmd="docker buildx build \
--target gotenberg-cloudrun \
--build-arg GOTENBERG_VERSION=$version \
--build-arg APT_SNAPSHOT=$apt_snapshot \
$(cache_flags gotenberg-cloudrun) \
--platform $platform \
--load \
${tags_cloud_run_flags[*]} \
@@ -240,6 +305,8 @@ if [ "$platform" = "linux/amd64" ]; then
cmd="docker buildx build \
--target gotenberg-cloudrun-chromium \
--build-arg GOTENBERG_VERSION=$version \
--build-arg APT_SNAPSHOT=$apt_snapshot \
$(cache_flags gotenberg-cloudrun-chromium) \
--platform $platform \
--load \
${tags_cloud_run_chromium_flags[*]} \
@@ -250,6 +317,8 @@ if [ "$platform" = "linux/amd64" ]; then
cmd="docker buildx build \
--target gotenberg-cloudrun-libreoffice \
--build-arg GOTENBERG_VERSION=$version \
--build-arg APT_SNAPSHOT=$apt_snapshot \
$(cache_flags gotenberg-cloudrun-libreoffice) \
--platform $platform \
--load \
${tags_cloud_run_libreoffice_flags[*]} \
@@ -263,6 +332,8 @@ if [ "$platform" = "linux/amd64" ] || [ "$platform" = "linux/arm64" ]; then
cmd="docker buildx build \
--target gotenberg-aws-lambda \
--build-arg GOTENBERG_VERSION=$version \
--build-arg APT_SNAPSHOT=$apt_snapshot \
$(cache_flags gotenberg-aws-lambda) \
--platform $platform \
--load \
${tags_aws_lambda_flags[*]} \
@@ -273,6 +344,8 @@ if [ "$platform" = "linux/amd64" ] || [ "$platform" = "linux/arm64" ]; then
cmd="docker buildx build \
--target gotenberg-aws-lambda-chromium \
--build-arg GOTENBERG_VERSION=$version \
--build-arg APT_SNAPSHOT=$apt_snapshot \
$(cache_flags gotenberg-aws-lambda-chromium) \
--platform $platform \
--load \
${tags_aws_lambda_chromium_flags[*]} \
@@ -283,6 +356,8 @@ if [ "$platform" = "linux/amd64" ] || [ "$platform" = "linux/arm64" ]; then
cmd="docker buildx build \
--target gotenberg-aws-lambda-libreoffice \
--build-arg GOTENBERG_VERSION=$version \
--build-arg APT_SNAPSHOT=$apt_snapshot \
$(cache_flags gotenberg-aws-lambda-libreoffice) \
--platform $platform \
--load \
${tags_aws_lambda_libreoffice_flags[*]} \

View File

@@ -87,6 +87,7 @@ LOG_STD_FORMAT=auto
LOG_STD_ENABLE_GCP_FIELDS=false
LOG_STD_LEVEL_CASE=lower
PDFENGINES_DISABLE_ROUTES=false
PDFENGINES_MAX_CONCURRENCY=1
PDFENGINES_MERGE_ENGINES=qpdf,pdfcpu,pdftk
PDFENGINES_SPLIT_ENGINES=pdfcpu,qpdf,pdftk
PDFENGINES_FLATTEN_ENGINES=qpdf
@@ -153,6 +154,7 @@ NO_CONCURRENCY=false
# chromium
# chromium-concurrent
# chromium-convert-html
# chromium-image-deduplication
# chromium-convert-markdown
# chromium-convert-url
# chromium-screenshot-html
@@ -218,20 +220,17 @@ lint-prettier: ## Lint non-Golang codebase
lint-todo: ## Find TODOs in Golang codebase
golangci-lint run --no-config --default=none --enable godox
# TODO: restore a plain "go fix ./..." once both modernizers stop rewriting this
# codebase into code that does not compile. Re-check by dropping a flag and
# running "make fmt && make lint". Removing an analyzer upstream makes go fix
# fail with "flag provided but not defined", so this cannot rot silently.
# embedlit folds a post-literal field assignment into a literal that already
# sets that key, producing a duplicate field name (cmd/gotenberg.go). Fixed in
# Go 1.27.1, so drop that flag once the toolchain moves past 1.27.0.
# See https://github.com/golang/go/issues/81101.
# TODO: restore a plain "go fix ./..." once the errorsastype modernizer stops
# rewriting this codebase into code that does not compile. Re-check by dropping
# the flag and running "make fmt && make lint". Removing the analyzer upstream
# makes go fix fail with "flag provided but not defined", so this cannot rot
# silently.
# errorsastype rewrites errors.As to errors.AsType[T] without checking that T
# satisfies error, which breaks on api.HttpError since it does not embed
# error. Still broken as of Go 1.27.1.
.PHONY: fmt
fmt: ## Format Golang codebase and "optimize" the dependencies
go fix -embedlit=false -errorsastype=false ./...
go fix -errorsastype=false ./...
golangci-lint fmt
go mod tidy

View File

@@ -1,7 +1,7 @@
# ARG instructions do not create additional layers. Instead, next layers will
# concatenate them. Also, we have to repeat ARG instructions in each build
# stage that uses them.
ARG GOLANG_VERSION=1.27.0
ARG GOLANG_VERSION=1.27.1
# ----------------------------------------------
# pdfcpu binary build stage
@@ -59,7 +59,14 @@ RUN go build -o gotenberg -ldflags "-s -w -X 'github.com/gotenberg/gotenberg/v8/
# ----------------------------------------------
FROM debian:13-slim AS custom-jre-stage
RUN apt-get update -qq \
# APT_SNAPSHOT busts every layer below it when CI rotates the value, weekly.
# Without it a persistent build cache turns the unpinned "apt-get upgrade" into
# a cache hit and the published image keeps shipping the package versions that
# were current when the cache was first populated.
ARG APT_SNAPSHOT=""
RUN echo "apt snapshot: $APT_SNAPSHOT" \
&& apt-get update -qq \
&& apt-get upgrade -yqq \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends default-jdk-headless binutils
@@ -114,9 +121,15 @@ FROM base-image-stage AS common-stage
ARG GOTENBERG_USER_GID=1001
ARG GOTENBERG_USER_UID=1001
# See the note on APT_SNAPSHOT in custom-jre-stage. Declaring it here covers
# every "apt-get upgrade" in the gotenberg, gotenberg-chromium and
# gotenberg-libreoffice targets too, since all three branch from this stage.
ARG APT_SNAPSHOT=""
# Create a non-root user.
# All processes in the Docker container will run with this dedicated user.
RUN groupadd --gid "$GOTENBERG_USER_GID" gotenberg \
RUN echo "apt snapshot: $APT_SNAPSHOT" \
&& groupadd --gid "$GOTENBERG_USER_GID" gotenberg \
&& useradd --uid "$GOTENBERG_USER_UID" --gid gotenberg --shell /bin/bash --home /home/gotenberg --no-create-home gotenberg \
&& mkdir /home/gotenberg \
&& chown gotenberg: /home/gotenberg
@@ -485,7 +498,7 @@ FROM gotenberg AS gotenberg-aws-lambda
USER root
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.9.1 /lambda-adapter /opt/extensions/lambda-adapter
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:1.0.1 /lambda-adapter /opt/extensions/lambda-adapter
ENV AWS_LWA_PORT=3000
ENV AWS_LWA_READINESS_CHECK_PATH=/health
@@ -503,7 +516,7 @@ FROM gotenberg-chromium AS gotenberg-aws-lambda-chromium
USER root
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.9.1 /lambda-adapter /opt/extensions/lambda-adapter
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:1.0.1 /lambda-adapter /opt/extensions/lambda-adapter
ENV AWS_LWA_PORT=3000
ENV AWS_LWA_READINESS_CHECK_PATH=/health
@@ -521,7 +534,7 @@ FROM gotenberg-libreoffice AS gotenberg-aws-lambda-libreoffice
USER root
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.9.1 /lambda-adapter /opt/extensions/lambda-adapter
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:1.0.1 /lambda-adapter /opt/extensions/lambda-adapter
ENV AWS_LWA_PORT=3000
ENV AWS_LWA_READINESS_CHECK_PATH=/health

View File

@@ -96,6 +96,7 @@ services:
- "--pdfengines-embed-engines=${PDFENGINES_EMBED_ENGINES}"
- "--pdfengines-embed-metadata-engines=${PDFENGINES_EMBED_METADATA_ENGINES}"
- "--pdfengines-factur-x-engines=${PDFENGINES_FACTUR_X_ENGINES}"
- "--pdfengines-max-concurrency=${PDFENGINES_MAX_CONCURRENCY}"
- "--pdfengines-disable-routes=${PDFENGINES_DISABLE_ROUTES}"
- "--prometheus-namespace=${PROMETHEUS_NAMESPACE}"
- "--prometheus-collect-interval=${PROMETHEUS_COLLECT_INTERVAL}"

24
go.mod
View File

@@ -1,6 +1,6 @@
module github.com/gotenberg/gotenberg/v8
go 1.27.0
go 1.27.1
require (
github.com/alexliesenfeld/health v0.8.1
@@ -8,18 +8,18 @@ require (
github.com/chromedp/chromedp v0.14.2 // pinned: v0.15.x breaks the headless print-mode paint pipeline (rAF / ResizeObserver / IntersectionObserver stop firing, blank charts). See https://github.com/gotenberg/gotenberg/issues/1535.
github.com/coreos/go-oidc/v3 v3.21.0
github.com/cucumber/godog v0.16.0
github.com/dlclark/regexp2 v1.12.0
github.com/dlclark/regexp2/v2 v2.8.0
github.com/gomarkdown/markdown v0.0.0-20260824154242-13c5cf49db8d
github.com/google/uuid v1.6.0
github.com/hashicorp/go-retryablehttp v0.7.8
github.com/labstack/echo/v4 v4.15.4
github.com/labstack/echo/v5 v5.4.0
github.com/labstack/gommon v0.5.0
github.com/mholt/archives v0.1.5
github.com/microcosm-cc/bluemonday v1.0.27
github.com/moby/moby/api v1.56.0
github.com/moby/moby/client v0.6.0
github.com/prometheus/client_golang v1.24.1
github.com/shirou/gopsutil/v4 v4.26.8
github.com/shirou/gopsutil/v4 v4.26.9
github.com/spf13/pflag v1.0.10
github.com/stretchr/testify v1.12.1
github.com/testcontainers/testcontainers-go v0.44.0
@@ -33,10 +33,10 @@ require (
go.opentelemetry.io/otel/sdk/log v0.22.0
go.opentelemetry.io/otel/sdk/metric v1.46.0
go.opentelemetry.io/otel/trace v1.46.0
golang.org/x/net v0.58.0
golang.org/x/sync v0.22.0
golang.org/x/term v0.45.0
golang.org/x/text v0.41.0
golang.org/x/net v0.59.0
golang.org/x/sync v0.23.0
golang.org/x/term v0.46.0
golang.org/x/text v0.42.0
)
require (
@@ -65,7 +65,7 @@ require (
github.com/docker/go-connections v0.8.1 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/ebitengine/purego v0.11.0 // indirect
github.com/ebitengine/purego v0.11.1 // indirect
github.com/felixge/httpsnoop v1.1.0 // indirect
github.com/go-jose/go-jose/v4 v4.1.5 // indirect
github.com/go-json-experiment/json v0.0.0-20260820222146-c27c302e5fc3 // indirect
@@ -114,8 +114,6 @@ require (
github.com/tklauser/go-sysconf v0.4.0 // indirect
github.com/tklauser/numcpus v0.12.0 // indirect
github.com/ulikunitz/xz v0.5.16 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasttemplate v1.2.2 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/bridges/prometheus v0.71.0 // indirect
@@ -133,9 +131,9 @@ require (
go.opentelemetry.io/proto/otlp v1.11.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
go4.org v0.0.0-20260112195520-a5071408f32f // indirect
golang.org/x/crypto v0.56.0 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/oauth2 v0.36.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.15.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260831171406-18b4a7587f8a // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260831171406-18b4a7587f8a // indirect

44
go.sum
View File

@@ -58,8 +58,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/dlclark/regexp2 v1.12.0 h1:0j4c5qQmnC6XOWNjP3PIXURXN2gWx76rd3KvgdPkCz8=
github.com/dlclark/regexp2 v1.12.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dlclark/regexp2/v2 v2.8.0 h1:CekDhPLGfm+GAJmPVFIG+5dqMIQPkyHJll7BbdneDfw=
github.com/dlclark/regexp2/v2 v2.8.0/go.mod h1:avUrQvPaLz2DrFNHJF0taWAFFX2C1GMSSoeiqFjcBmU=
github.com/docker/go-connections v0.8.1 h1:JibmG5hULs5qXSr/cp/w3Pw5fZuStt4MOHMUExb29/M=
github.com/docker/go-connections v0.8.1/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
@@ -67,8 +67,8 @@ github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDD
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 h1:2tV76y6Q9BB+NEBasnqvs7e49aEBFI8ejC89PSnWH+4=
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707/go.mod h1:qssHWj60/X5sZFNxpG4HBPDHVqxNm4DfnCKgrbZOT+s=
github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY=
github.com/ebitengine/purego v0.11.0 h1:jhp/D+Nyv7UUW8HAcmcjt2N2rYrYi9m3SL21k0Ua/NI=
github.com/ebitengine/purego v0.11.0/go.mod h1:DCHPP08djqhNSoTfImcnHYQRZmd0qhakvrozqaEYhGQ=
github.com/ebitengine/purego v0.11.1 h1:2zpWRSQNVKN4eKsKO9eM1ILDgWfYMY9GwqRmK6XeQ/0=
github.com/ebitengine/purego v0.11.1/go.mod h1:DCHPP08djqhNSoTfImcnHYQRZmd0qhakvrozqaEYhGQ=
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
github.com/felixge/httpsnoop v1.1.0 h1:3YtUj32ZZkqZtt3sZZsClsymw/QDuVfpNhoA31zeORc=
@@ -131,8 +131,8 @@ github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU
github.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/labstack/echo/v4 v4.15.4 h1:DL45vVYa+BWE+XuW+zZNd9H0YEdZ80UAWJGcTVW4EVs=
github.com/labstack/echo/v4 v4.15.4/go.mod h1:CuMetKIRwsuO/qlAgMq+KTAalwGoB/h4tC+yPdrTj1g=
github.com/labstack/echo/v5 v5.4.0 h1:iY674460IvSmUcj7MziL3YrgyTClghz5BpdZoKiGxR4=
github.com/labstack/echo/v5 v5.4.0/go.mod h1:4iEGNQiPPZnkfYpNR/L6fINd3NLiGWUD5+eBotFALas=
github.com/labstack/gommon v0.5.0 h1:6VSQ2NOzsnEJ5W6+84E0RbcaDDmgB6NIAzWCczTEe6c=
github.com/labstack/gommon v0.5.0/go.mod h1:Rzlg7HHy1maLfzBYGg9NZcVuz1sA68HHhLjhcEllYE0=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80 h1:6Yzfa6GP0rIo/kULo2bwGEkFvCePZ3qHDDTC3/J9Swo=
@@ -200,8 +200,8 @@ github.com/prometheus/otlptranslator v1.0.0 h1:s0LJW/iN9dkIH+EnhiD3BlkkP5QVIUVEo
github.com/prometheus/otlptranslator v1.0.0/go.mod h1:vRYWnXvI6aWGpsdY/mOT/cbeVRBlPWtBNDb7kGR3uKM=
github.com/prometheus/procfs v0.22.0 h1:6q9+/JL9IKAPbCmBrv9n5O5Ty3NKnciV5X7YGw0oics=
github.com/prometheus/procfs v0.22.0/go.mod h1:CvmFr/GVhIjIvWJZW3tgkODBQMRIf0EyWMQLHCHab58=
github.com/shirou/gopsutil/v4 v4.26.8 h1:YQMTF/1J50B5+Y0vlo1eDRf5DoR7Gk69hY+8wjYkQeo=
github.com/shirou/gopsutil/v4 v4.26.8/go.mod h1:5O9FjBiXoTDFatIWjZZosqj4pV0DRtLx598xGbBehzM=
github.com/shirou/gopsutil/v4 v4.26.9 h1:CaBo/hBFqvlJoLvAQEKdHdf5GD97/MN2ACfyv63+2Ig=
github.com/shirou/gopsutil/v4 v4.26.9/go.mod h1:nKH+8wX2zxr/mDbsR+AAy6Qd2z4ZOQ7wfQaEp+yrYmg=
github.com/sirupsen/logrus v1.10.2 h1:G2SED73/qrAu6YwbdxOD6peLkCBI3z7L+ykJFTXJBBo=
github.com/sirupsen/logrus v1.10.2/go.mod h1:SLEg8TqYulVKKfIGHldVp2K2aYz2DKSVBq4g/H5bR7Q=
github.com/sorairolake/lzip-go v0.3.8 h1:j5Q2313INdTA80ureWYRhX+1K78mUXfMoPZCw/ivWik=
@@ -231,10 +231,6 @@ github.com/tklauser/numcpus v0.12.0/go.mod h1:ABHeXzJnr/qqwguhClkZKT1/8VABcYrsyU
github.com/ulikunitz/xz v0.5.8/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/ulikunitz/xz v0.5.16 h1:ld6NyySjx5lowVKwJvMRLnW5nxKX/xnpSiFYZ/Lxur0=
github.com/ulikunitz/xz v0.5.16/go.mod h1:H9Rt/W6/Qj27PGauhQc6nfCDy7vHpzsOThBSaYDoEhw=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasttemplate v1.2.2 h1:lxLXG0uE3Qnshl9QyaK6XJxMXlQZELvChBOCmQD0Loo=
github.com/valyala/fasttemplate v1.2.2/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
@@ -299,25 +295,25 @@ go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
go4.org v0.0.0-20260112195520-a5071408f32f h1:ziUVAjmTPwQMBmYR1tbdRFJPtTcQUI12fH9QQjfb0Sw=
go4.org v0.0.0-20260112195520-a5071408f32f/go.mod h1:ZRJnO5ZI4zAwMFp+dS1+V6J6MSyAowhRqAE+DPa1Xp0=
golang.org/x/crypto v0.56.0 h1:GUh5Ii4J5jtcseSMiRqr1jXCNHoxjeV9Fmekc2oLy6Y=
golang.org/x/crypto v0.56.0/go.mod h1:OMW5y6CY9l38uPLmxU6l6pwcXp1obtLo3e6gT7gQR2I=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues=
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=
golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE=
golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc=
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

24
package-lock.json generated
View File

@@ -5,9 +5,9 @@
"packages": {
"": {
"devDependencies": {
"prettier": "3.9.6",
"prettier": "3.9.9",
"prettier-plugin-gherkin": "^4.0.0",
"prettier-plugin-sh": "^0.19.0"
"prettier-plugin-sh": "^0.20.2"
}
},
"node_modules/@cucumber/gherkin": {
@@ -114,9 +114,9 @@
"license": "MIT"
},
"node_modules/prettier": {
"version": "3.9.6",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.6.tgz",
"integrity": "sha512-OpN0zzVdiaiAhxpuuj5efpIS4sY9j7bY6uR5mnj5yPzGkdkjNKSJeUThPb60Jw29QuAZgA4o+/iB49kFiaBX6g==",
"version": "3.9.9",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.9.tgz",
"integrity": "sha512-Z/CJHIkdujO/OtN7nXUii0Rf3VT5SRuhjBA82Xvu2XhBUgX3nhP67T0LHceBdQLex7OOFGTox+Q5Yg8Jk2Qivg==",
"dev": true,
"license": "MIT",
"bin": {
@@ -142,14 +142,14 @@
}
},
"node_modules/prettier-plugin-sh": {
"version": "0.19.0",
"resolved": "https://registry.npmjs.org/prettier-plugin-sh/-/prettier-plugin-sh-0.19.0.tgz",
"integrity": "sha512-39VXFZH/cOGtcuu8aeSvqp/hhwomOR4QroZUj+jBz2cNb3os9s0sqFZSNlYts6jdtLLDU7D2YT3Z1+abtb7adQ==",
"version": "0.20.2",
"resolved": "https://registry.npmjs.org/prettier-plugin-sh/-/prettier-plugin-sh-0.20.2.tgz",
"integrity": "sha512-4QVyY7Vav0f5i6AVa80SFwUJXaWsAv9t0yUZIivwI0ucxTei6Jwhso/khWqaKDj4QGdwjP2py1nxQrDUeZ0jyQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@reteps/dockerfmt": "^0.5.4",
"sh-syntax": "^0.6.0"
"sh-syntax": "^0.7.0"
},
"engines": {
"node": ">=16.0.0"
@@ -169,9 +169,9 @@
"license": "Apache-2.0"
},
"node_modules/sh-syntax": {
"version": "0.6.0",
"resolved": "https://registry.npmjs.org/sh-syntax/-/sh-syntax-0.6.0.tgz",
"integrity": "sha512-52VK6z/cdZHv7UURjIcwfBUQZrAhIEEe0bY4lrkfypjnFIKsDZdD3Uaz/dBiw/sF8BeX0Mssv140s8EnrsJ9dQ==",
"version": "0.7.0",
"resolved": "https://registry.npmjs.org/sh-syntax/-/sh-syntax-0.7.0.tgz",
"integrity": "sha512-sgMyAxpHiGgAaNg+gD3ad41vha24CzIjjghD4mdiON1lUafZPLJNYknyjcXmU4geilPstd69zCKkpccY0YvajA==",
"dev": true,
"license": "MIT",
"engines": {

View File

@@ -1,7 +1,7 @@
{
"devDependencies": {
"prettier": "3.9.6",
"prettier": "3.9.9",
"prettier-plugin-gherkin": "^4.0.0",
"prettier-plugin-sh": "^0.19.0"
"prettier-plugin-sh": "^0.20.2"
}
}

View File

@@ -3,7 +3,7 @@ package gotenberg
import (
"testing"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
)
func TestAuditAllowList(t *testing.T) {
@@ -93,7 +93,7 @@ func TestAuditAllowList_FlaggedPatternsAreActuallyExploitable(t *testing.T) {
t.Fatalf("pattern %q was not flagged", tc.pattern)
}
ok, err := regexp2.MustCompile(tc.pattern, 0).MatchString(tc.attack)
ok, err := regexp2.MustCompile(tc.pattern, regexp2.None).MatchString(tc.attack)
if err != nil {
t.Fatalf("match %q: %v", tc.attack, err)
}
@@ -129,7 +129,7 @@ func TestAuditAllowList_SafePatternsRejectTheAttacks(t *testing.T) {
t.Fatalf("safe pattern %q was flagged as %q", pattern, findings[0].Risk)
}
re := regexp2.MustCompile(pattern, 0)
re := regexp2.MustCompile(pattern, regexp2.None)
for _, attack := range attacks {
ok, err := re.MatchString(attack)
if err != nil {

View File

@@ -1,70 +0,0 @@
package gotenberg
import (
"context"
"errors"
"fmt"
"time"
"github.com/dlclark/regexp2"
)
// ErrFiltered happens if a value is filtered by the [FilterDeadline] function.
var ErrFiltered = errors.New("value filtered")
// FilterDeadline checks if the given value is allowed and not denied according
// to regex patterns. The allowed list uses OR semantics (value must match at
// least one pattern). The denied list uses OR semantics (value is denied if it
// matches any pattern). It returns a [context.DeadlineExceeded] if it takes
// too long to process.
func FilterDeadline(allowed, denied []*regexp2.Regexp, s string, deadline time.Time) error {
if len(allowed) > 0 {
matched := false
for _, pattern := range allowed {
// FIXME: not ideal to compile everytime, but is there another way to create a clone?
clone := regexp2.MustCompile(pattern.String(), 0)
clone.MatchTimeout = time.Until(deadline)
ok, err := clone.MatchString(s)
if err != nil {
if time.Now().After(deadline) {
return context.DeadlineExceeded
}
return fmt.Errorf("'%s' cannot handle '%s': %w", clone.String(), s, err)
}
if ok {
matched = true
break
}
}
if !matched {
return fmt.Errorf("'%s' does not match any expression from the allowed list: %w", s, ErrFiltered)
}
}
if len(denied) > 0 {
for _, pattern := range denied {
clone := regexp2.MustCompile(pattern.String(), 0)
clone.MatchTimeout = time.Until(deadline)
ok, err := clone.MatchString(s)
if err != nil {
if time.Now().After(deadline) {
return context.DeadlineExceeded
}
return fmt.Errorf("'%s' cannot handle '%s': %w", clone.String(), s, err)
}
if ok {
return fmt.Errorf("'%s' matches the expression from the denied list: %w", s, ErrFiltered)
}
}
}
return nil
}

View File

@@ -1,117 +0,0 @@
package gotenberg
import (
"context"
"errors"
"testing"
"time"
"github.com/dlclark/regexp2"
)
func TestFilterDeadline(t *testing.T) {
for _, tc := range []struct {
scenario string
allowed []*regexp2.Regexp
denied []*regexp2.Regexp
s string
deadline time.Time
expectError bool
expectedError error
}{
{
scenario: "DeadlineExceeded (allowed)",
allowed: []*regexp2.Regexp{regexp2.MustCompile("foo", 0)},
denied: nil,
s: "foo",
deadline: time.Now().Add(time.Duration(-1) * time.Hour),
expectError: true,
expectedError: context.DeadlineExceeded,
},
{
scenario: "ErrFiltered (allowed, no match)",
allowed: []*regexp2.Regexp{regexp2.MustCompile("foo", 0)},
denied: nil,
s: "bar",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: true,
expectedError: ErrFiltered,
},
{
scenario: "DeadlineExceeded (denied)",
allowed: nil,
denied: []*regexp2.Regexp{regexp2.MustCompile("foo", 0)},
s: "foo",
deadline: time.Now().Add(time.Duration(-1) * time.Hour),
expectError: true,
expectedError: context.DeadlineExceeded,
},
{
scenario: "ErrFiltered (denied)",
allowed: nil,
denied: []*regexp2.Regexp{regexp2.MustCompile("foo", 0)},
s: "foo",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: true,
expectedError: ErrFiltered,
},
{
scenario: "success (empty lists)",
allowed: nil,
denied: nil,
s: "foo",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: false,
},
{
scenario: "multi-pattern allow list, second matches",
allowed: []*regexp2.Regexp{regexp2.MustCompile("^https://", 0), regexp2.MustCompile("^file:///tmp/", 0)},
denied: nil,
s: "file:///tmp/abc/index.html",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: false,
},
{
scenario: "multi-pattern allow list, none matches",
allowed: []*regexp2.Regexp{regexp2.MustCompile("^https://", 0), regexp2.MustCompile("^ftp://", 0)},
denied: nil,
s: "file:///tmp/abc/index.html",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: true,
expectedError: ErrFiltered,
},
{
scenario: "multi-pattern deny list, second matches",
allowed: nil,
denied: []*regexp2.Regexp{regexp2.MustCompile("^ftp://", 0), regexp2.MustCompile("^file:.*", 0)},
s: "file:///etc/passwd",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: true,
expectedError: ErrFiltered,
},
{
scenario: "https URL passes deny list targeting file://",
allowed: nil,
denied: []*regexp2.Regexp{regexp2.MustCompile("^file:.*", 0)},
s: "https://example.com",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: false,
},
} {
t.Run(tc.scenario, func(t *testing.T) {
err := FilterDeadline(tc.allowed, tc.denied, tc.s, tc.deadline)
if tc.expectError && err == nil {
t.Fatal("expected an error but got none")
}
if !tc.expectError && err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if tc.expectedError != nil && !errors.Is(err, tc.expectedError) {
t.Fatalf("expected error %v but got: %v", tc.expectedError, err)
}
})
}
}

View File

@@ -7,7 +7,7 @@ import (
"strings"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"github.com/labstack/gommon/bytes"
flag "github.com/spf13/pflag"
@@ -207,15 +207,42 @@ func (f *ParsedFlags) MustDeprecatedHumanReadableBytes(deprecated string, newNam
return f.MustHumanReadableBytes(newName)
}
// PatternMatchTimeout bounds a single match against an operator-supplied
// allow-list or deny-list pattern.
//
// regexp2 backtracks, and the strings matched against these patterns are
// client-controlled: a request URL, a CONNECT host. A pattern that backtracks
// catastrophically would otherwise burn a core for as long as the caller's
// deadline allows, which is --api-timeout (env API_TIMEOUT), 30 seconds by
// default. The ceiling mirrors the one the Chromium module already applies to
// the per-request extraHttpHeaders scope pattern.
//
// [ParsedFlags.MustRegexp] and [ParsedFlags.MustRegexpSlice] stamp this onto
// every pattern they compile, which is how all four production lists are
// built. Patterns compiled any other way keep regexp2's default of
// math.MaxInt64, which it treats as no timeout at all, so a hand-built slice
// must set this itself before reaching [DecideOutbound].
//
// The ceiling is wall-clock. Match through [MatchPattern] rather than calling
// regexp2 directly: a match that never approaches the ceiling still aborts if
// the process loses the CPU at the wrong moment.
const PatternMatchTimeout = 250 * time.Millisecond
// MustRegexp returns the regular expression of a flag given by name.
// It panics if an error occurs.
//
// The returned expression carries [PatternMatchTimeout] and is safe to match
// on concurrently: callers must not compile a private copy per match.
func (f *ParsedFlags) MustRegexp(name string) *regexp2.Regexp {
val, err := f.GetString(name)
if err != nil {
panic(err)
}
return regexp2.MustCompile(val, 0)
re := regexp2.MustCompile(val, regexp2.None)
re.MatchTimeout = PatternMatchTimeout
return re
}
// MustDeprecatedRegexp returns the regular expression of a deprecated flag if
@@ -235,6 +262,9 @@ func (f *ParsedFlags) MustDeprecatedRegexp(deprecated string, newName string) *r
//
// Every allow-list and deny-list in Gotenberg is read through this method, so
// it is also where allow-list patterns are audited. See [AuditAllowList].
//
// The returned expressions carry [PatternMatchTimeout] and are safe to match
// on concurrently: callers must not compile a private copy per match.
func (f *ParsedFlags) MustRegexpSlice(name string) []*regexp2.Regexp {
vals := f.MustStringSlice(name)
@@ -246,7 +276,10 @@ func (f *ParsedFlags) MustRegexpSlice(name string) []*regexp2.Regexp {
continue
}
regexps = append(regexps, regexp2.MustCompile(val, 0))
re := regexp2.MustCompile(val, regexp2.None)
re.MatchTimeout = PatternMatchTimeout
regexps = append(regexps, re)
}
return regexps

View File

@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
flag "github.com/spf13/pflag"
)
@@ -1054,3 +1054,36 @@ func TestEnvVarName(t *testing.T) {
})
}
}
func TestParsedFlags_RegexpMatchTimeout(t *testing.T) {
// [DecideOutbound] matches on these patterns directly instead of compiling
// a private copy per call, so the bound has to come from here. regexp2's
// own default is math.MaxInt64, which it treats as no
// timeout at all, so a pattern built without this stamp runs unbounded
// against a client-controlled string.
fs := flag.NewFlagSet("tests", flag.ContinueOnError)
fs.StringSlice("some-deny-list", []string{`^file:`, `^https?://`}, "")
fs.String("some-pattern", `^file:`, "")
err := fs.Parse(nil)
if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
parsedFlags := ParsedFlags{FlagSet: fs}
regexps := parsedFlags.MustRegexpSlice("some-deny-list")
if len(regexps) != 2 {
t.Fatalf("expected 2 patterns but got %d", len(regexps))
}
for _, re := range regexps {
if re.MatchTimeout != PatternMatchTimeout {
t.Fatalf("pattern '%s' has MatchTimeout %s, expected %s", re.String(), re.MatchTimeout, PatternMatchTimeout)
}
}
if got := parsedFlags.MustRegexp("some-pattern").MatchTimeout; got != PatternMatchTimeout {
t.Fatalf("expected MustRegexp MatchTimeout %s but got %s", PatternMatchTimeout, got)
}
}

View File

@@ -15,7 +15,7 @@ import (
"strings"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"github.com/hashicorp/go-retryablehttp"
"golang.org/x/net/http/httpproxy"
)
@@ -27,6 +27,12 @@ import (
// example [::ffff:127.0.0.1]).
var ErrNonPublicIP = errors.New("non-public IP")
// ErrFiltered happens when a value is rejected by an allow-list or a
// deny-list, or when it cannot be validated and [DecideOutbound] fails closed.
// Callers map it to a generic 403: the specific reason stays in the operator
// logs so a client cannot probe the lists.
var ErrFiltered = errors.New("value filtered")
// ErrPublicIP indicates that an outbound URL targets an IP address that is
// reachable on the public internet. It is returned when a caller opts
// into denying public destinations via [WithDenyPublicIPs]; typical use
@@ -103,11 +109,18 @@ var nonPublicIPv4Prefixes = []netip.Prefix{
// the prefixes themselves are deprecated or translation-only. See
// [nonPublicIPv6Prefixes] and [nonPublicIPv4Prefixes] for the full lists
// and rationale.
//
// An IPv6 zone identifier is ignored, so [::%1] classifies the same as [::].
func IsPublicIP(addr netip.Addr) bool {
if !addr.IsValid() {
return false
}
addr = addr.Unmap()
// A zone does not change where a non-link-local address routes, but
// [netip.Prefix.Contains] never matches a zoned address and
// [netip.Addr.IsUnspecified] compares the zone too. Keeping it would let
// http://[::%251]/ or any zoned address in the prefixes below pass as
// public.
addr = addr.WithZone("").Unmap()
switch {
case addr.IsLoopback(),
addr.IsPrivate(),
@@ -291,6 +304,15 @@ func DecideOutbound(ctx context.Context, rawURL string, allowList, denyList []*r
opt(&cfg)
}
// Each match is bounded by [PatternMatchTimeout] rather than by the
// remaining budget, so an already-spent deadline no longer surfaces from
// the match itself. Schemes that resolve a host still learn about it from
// resolveHost, but a non-matching file:// or data: URL returns before that
// point, so check it here to keep failing closed on every path.
if !time.Now().Before(deadline) {
return OutboundDecision{}, context.DeadlineExceeded
}
parsed, err := url.Parse(rawURL)
if err != nil {
return OutboundDecision{}, fmt.Errorf("parse URL %q: %w", rawURL, ErrFiltered)
@@ -314,15 +336,16 @@ func DecideOutbound(ctx context.Context, rawURL string, allowList, denyList []*r
allowMatched := false
if len(allowList) > 0 {
for _, pattern := range allowList {
clone := regexp2.MustCompile(pattern.String(), 0)
clone.MatchTimeout = time.Until(deadline)
ok, err := clone.MatchString(normalized)
ok, err := MatchPattern(pattern, normalized)
if err != nil {
if time.Now().After(deadline) {
return OutboundDecision{}, context.DeadlineExceeded
}
return OutboundDecision{}, fmt.Errorf("'%s' cannot handle '%s': %w", clone.String(), normalized, err)
// The pattern could not be evaluated, so the URL cannot be
// cleared for the IP-check bypass an allow-list match grants.
// Fail closed like an unresolvable host does below, so the
// client gets a generic 403 rather than a 500.
return OutboundDecision{}, fmt.Errorf("'%s' cannot handle '%s': %v: %w", pattern.String(), normalized, err, ErrFiltered)
}
if ok {
@@ -337,15 +360,15 @@ func DecideOutbound(ctx context.Context, rawURL string, allowList, denyList []*r
}
for _, pattern := range denyList {
clone := regexp2.MustCompile(pattern.String(), 0)
clone.MatchTimeout = time.Until(deadline)
ok, err := clone.MatchString(normalized)
ok, err := MatchPattern(pattern, normalized)
if err != nil {
if time.Now().After(deadline) {
return OutboundDecision{}, context.DeadlineExceeded
}
return OutboundDecision{}, fmt.Errorf("'%s' cannot handle '%s': %w", clone.String(), normalized, err)
// The pattern could not be evaluated, so the URL cannot be proven
// to fall outside the deny-list. Fail closed rather than letting a
// deny-list that never ran pass the request through.
return OutboundDecision{}, fmt.Errorf("'%s' cannot handle '%s': %v: %w", pattern.String(), normalized, err, ErrFiltered)
}
if ok {
@@ -392,9 +415,8 @@ func DecideOutbound(ctx context.Context, rawURL string, allowList, denyList []*r
}
// FilterOutboundURL validates that rawURL is acceptable for an outbound
// request from Gotenberg. It is the URL-aware replacement for
// [FilterDeadline] and should be preferred for any new code that filters
// a URL before issuing or instructing an outbound request.
// request from Gotenberg. Prefer it for any new code that filters a URL
// before issuing or instructing an outbound request.
//
// The default behavior is permissive: the URL passes as long as it clears
// the regex allow-list and deny-list. Callers that need IP-class checks

View File

@@ -9,7 +9,7 @@ import (
"testing"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
)
func TestIsPublicIP(t *testing.T) {
@@ -103,6 +103,23 @@ func TestIsPublicIP(t *testing.T) {
// Discard prefix (RFC 6666).
{"100::1", false},
// A zone identifier must not change the classification.
{"::%1", false},
{"::%lo", false},
{"::1%1", false},
{"fe80::1%eth0", false},
{"fc00::1%1", false},
{"::ffff:127.0.0.1%1", false},
{"fec0::1%eth0", false},
{"2002:a9fe:a9fe::%1", false},
{"2001:0:abcd:ef12:3456:7890:a9fe:a9fe%1", false},
{"64:ff9b::a9fe:a9fe%1", false},
{"64:ff9b:1::a9fe:a9fe%1", false},
{"::a9fe:a9fe%1", false},
{"2001:db8::1%1", false},
{"100::1%1", false},
{"2606:4700:4700::1111%1", true},
} {
t.Run(tc.addr, func(t *testing.T) {
addr, err := netip.ParseAddr(tc.addr)
@@ -147,10 +164,10 @@ func mustAddrs(t *testing.T, ss ...string) []netip.Addr {
func TestFilterOutboundURL(t *testing.T) {
defaultDeny := []*regexp2.Regexp{
regexp2.MustCompile(`^https?://(10\.|172\.(1[6-9]|2[0-9]|3[01])\.|192\.168\.|169\.254\.|0\.0\.0\.0|127\.|localhost|\[::1\]|\[fd)`, 0),
regexp2.MustCompile(`^https?://(10\.|172\.(1[6-9]|2[0-9]|3[01])\.|192\.168\.|169\.254\.|0\.0\.0\.0|127\.|localhost|\[::1\]|\[fd)`, regexp2.None),
}
chromiumDeny := []*regexp2.Regexp{
regexp2.MustCompile(`^file:(?!//\/tmp/).*`, 0),
regexp2.MustCompile(`^file:(?!//\/tmp/).*`, regexp2.None),
}
for _, tc := range []struct {
@@ -229,7 +246,7 @@ func TestFilterOutboundURL(t *testing.T) {
{
scenario: "allow-list match bypasses IP check",
rawURL: "http://internal.service/api",
allow: []*regexp2.Regexp{regexp2.MustCompile(`^http://internal\.service`, 0)},
allow: []*regexp2.Regexp{regexp2.MustCompile(`^http://internal\.service`, regexp2.None)},
deny: defaultDeny,
opts: []DecideOption{WithDenyPrivateIPs(true)},
expectErr: false,
@@ -237,15 +254,15 @@ func TestFilterOutboundURL(t *testing.T) {
{
scenario: "deny-list still wins over allow-list match",
rawURL: "http://internal.service/api",
allow: []*regexp2.Regexp{regexp2.MustCompile(`^http://internal`, 0)},
deny: []*regexp2.Regexp{regexp2.MustCompile(`/api$`, 0)},
allow: []*regexp2.Regexp{regexp2.MustCompile(`^http://internal`, regexp2.None)},
deny: []*regexp2.Regexp{regexp2.MustCompile(`/api$`, regexp2.None)},
expectErr: true,
expectIs: ErrFiltered,
},
{
scenario: "allow-list non-empty and no match rejects",
rawURL: "https://other.example/",
allow: []*regexp2.Regexp{regexp2.MustCompile(`^https://allowed\.example`, 0)},
allow: []*regexp2.Regexp{regexp2.MustCompile(`^https://allowed\.example`, regexp2.None)},
expectErr: true,
expectIs: ErrFiltered,
},
@@ -304,6 +321,22 @@ func TestFilterOutboundURL(t *testing.T) {
expectErr: true,
expectIs: ErrFiltered,
},
{
scenario: "zoned unspecified address blocked with deny-private-ips",
rawURL: "http://[::%251]:9999/",
deny: defaultDeny,
opts: []DecideOption{WithDenyPrivateIPs(true)},
expectErr: true,
expectIs: ErrFiltered,
},
{
scenario: "zoned site-local address blocked with deny-private-ips",
rawURL: "http://[fec0:1234::3%25eth0]:8080/",
deny: defaultDeny,
opts: []DecideOption{WithDenyPrivateIPs(true)},
expectErr: true,
expectIs: ErrFiltered,
},
} {
t.Run(tc.scenario, func(t *testing.T) {
if tc.stub != nil {
@@ -506,7 +539,7 @@ func TestDecideOutbound_DenyLists_WinOverDenyPrivateIPs(t *testing.T) {
// The regex deny-list fires before any resolution; verifies that
// operator-supplied deny patterns remain effective regardless of
// IP-class options.
deny := []*regexp2.Regexp{regexp2.MustCompile(`^http://evil\.`, 0)}
deny := []*regexp2.Regexp{regexp2.MustCompile(`^http://evil\.`, regexp2.None)}
_, err := DecideOutbound(
context.Background(),
@@ -572,7 +605,7 @@ func TestDecideOutbound_UserinfoDoesNotEvadeDenyList(t *testing.T) {
context.Background(),
rawURL,
nil,
[]*regexp2.Regexp{regexp2.MustCompile(privateIPsDenyList, 0)},
[]*regexp2.Regexp{regexp2.MustCompile(privateIPsDenyList, regexp2.None)},
time.Now().Add(5*time.Second),
)
if !errors.Is(err, ErrFiltered) {
@@ -584,7 +617,7 @@ func TestDecideOutbound_UserinfoDoesNotEvadeDenyList(t *testing.T) {
func TestDecideOutbound_UserinfoDoesNotSatisfyAllowList(t *testing.T) {
// A host-terminated allow-list, the shape the documentation recommends.
allowList := []*regexp2.Regexp{regexp2.MustCompile(`^https://trusted\.example\.com(:[0-9]+)?(/|$)`, 0)}
allowList := []*regexp2.Regexp{regexp2.MustCompile(`^https://trusted\.example\.com(:[0-9]+)?(/|$)`, regexp2.None)}
for _, rawURL := range []string{
"https://trusted.example.com@169.254.169.254/latest/meta-data/",
@@ -623,7 +656,7 @@ func TestDecideOutbound_UserinfoKeptOutOfErrorMessages(t *testing.T) {
context.Background(),
"http://alice:hunter2@127.0.0.1:9999/",
nil,
[]*regexp2.Regexp{regexp2.MustCompile(privateIPsDenyList, 0)},
[]*regexp2.Regexp{regexp2.MustCompile(privateIPsDenyList, regexp2.None)},
time.Now().Add(5*time.Second),
)
if err == nil {
@@ -647,7 +680,7 @@ func TestDecideOutbound_LegitimateCredentialsStillReachTheHost(t *testing.T) {
decision, err := DecideOutbound(
context.Background(),
"https://alice:hunter2@example.com/report.pdf",
[]*regexp2.Regexp{regexp2.MustCompile(`^https://example\.com(:[0-9]+)?(/|$)`, 0)},
[]*regexp2.Regexp{regexp2.MustCompile(`^https://example\.com(:[0-9]+)?(/|$)`, regexp2.None)},
nil,
time.Now().Add(5*time.Second),
WithDenyPrivateIPs(true),
@@ -715,3 +748,61 @@ func TestNewOutboundHttpClient_NonPositiveTimeout(t *testing.T) {
t.Fatalf("timeout for a negative budget = %s, want a positive value so the client fails closed", got)
}
}
func TestDecideOutboundExpiredDeadline(t *testing.T) {
// Patterns are matched under the fixed PatternMatchTimeout rather than
// under the caller's remaining budget, so an expired deadline no longer
// surfaces from the match itself. Every scheme must still fail closed,
// including the ones that return before a host is resolved.
expired := time.Now().Add(-time.Second)
for _, rawURL := range []string{
"https://example.com/",
"file:///tmp/foo.html",
"data:text/html,hello",
} {
_, err := DecideOutbound(context.Background(), rawURL, nil, nil, expired)
if !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("DecideOutbound(%q) with an expired deadline = %v, want context.DeadlineExceeded", rawURL, err)
}
}
}
func TestDecideOutboundBoundsCatastrophicPatterns(t *testing.T) {
// A deny-list pattern that backtracks catastrophically, matched against a
// client-controlled URL. Before PatternMatchTimeout the ceiling was the
// caller's whole budget, so a 30s API_TIMEOUT bought a 30s CPU burn.
// The trailing "!" makes the match fail only after the nested quantifier
// has explored every way to split the run of "a"s.
pattern := regexp2.MustCompile(`^https://example\.com/(a+)+$`, regexp2.None)
pattern.MatchTimeout = PatternMatchTimeout
rawURL := "https://example.com/" + strings.Repeat("a", 40) + "!"
start := time.Now()
_, err := DecideOutbound(
context.Background(),
rawURL,
nil,
[]*regexp2.Regexp{pattern},
time.Now().Add(30*time.Second),
)
elapsed := time.Since(start)
if err == nil {
t.Fatal("expected an error from a catastrophic deny-list pattern")
}
// A deny-list that could not be evaluated cannot clear the URL, so the
// decision fails closed and the client gets a generic 403 rather than a
// 500 naming the pattern.
if !errors.Is(err, ErrFiltered) {
t.Fatalf("expected ErrFiltered from an unevaluable deny-list pattern but got: %v", err)
}
// Generous headroom over the 250ms ceiling, still far below the 30s
// deadline the match would otherwise have been allowed to consume.
if elapsed > 5*time.Second {
t.Fatalf("match took %s, want it aborted near PatternMatchTimeout (%s)", elapsed, PatternMatchTimeout)
}
}

55
pkg/gotenberg/pattern.go Normal file
View File

@@ -0,0 +1,55 @@
package gotenberg
import (
"github.com/dlclark/regexp2/v2"
)
// patternMatchAttempts caps how many times [MatchPattern] runs one pattern
// against one string. It is what keeps a pattern that is genuinely out of
// budget from retrying forever: three attempts bound its cost at three
// [PatternMatchTimeout], which is still two orders of magnitude below the
// --api-timeout (env API_TIMEOUT) the ceiling exists to protect.
const patternMatchAttempts = 3
// MatchPattern reports whether s matches pattern. It bounds the match by the
// pattern's MatchTimeout without the false timeouts that the bound alone
// produces.
//
// regexp2 does not time a match against [time.Now]. It derives the deadline
// from a process-global clock that a background goroutine advances every
// 100ms, and it tests that deadline on the very first step of the match.
// Anything that stops the whole process, a cgroup CPU-quota throttle or a long
// stop-the-world pause, also stops that goroutine, which then advances the
// clock by the full pause in a single write. A match holding a deadline from
// before that jump aborts whatever work it had done: a 366ns match against a
// short URL reports "match timeout after 250ms". The abort lands on whichever
// match straddles the jump rather than on a match that was slow, which is why
// it fires on an idle instance and against Gotenberg's own file:///tmp/ URLs.
// See https://github.com/gotenberg/gotenberg/issues/1659.
//
// Retrying separates the two cases. Catastrophic backtracking is
// deterministic: the same pattern against the same string exhausts the same
// budget on every attempt, so a genuine runaway still aborts, and costs at
// most patternMatchAttempts ceilings to prove it. A clock-induced abort needs
// the process to lose the CPU inside one specific match, which the next
// attempt does not reproduce.
//
// Elapsed time cannot make that call instead. A match frozen mid-flight
// reports the freeze as its own cost, 806ms against a 250ms ceiling in one
// measured run, so it is indistinguishable by wall clock from a match that
// really did spend its budget. Go exposes no per-goroutine CPU time, and
// process CPU time counts every other request in flight.
func MatchPattern(pattern *regexp2.Regexp, s string) (bool, error) {
var err error
for range patternMatchAttempts {
var ok bool
ok, err = pattern.MatchString(s)
if err == nil {
return ok, nil
}
}
return false, err
}

View File

@@ -0,0 +1,99 @@
package gotenberg
import (
"strings"
"testing"
"time"
"github.com/dlclark/regexp2/v2"
)
// mustPattern compiles a pattern the way the production lists are built.
func mustPattern(t *testing.T, expr string) *regexp2.Regexp {
t.Helper()
re := regexp2.MustCompile(expr, regexp2.None)
re.MatchTimeout = PatternMatchTimeout
return re
}
func TestMatchPattern(t *testing.T) {
// The abort [MatchPattern] absorbs cannot be staged here: it needs the
// whole process to lose the CPU around one specific match, which no test
// can schedule. What is testable is the other half of the contract, that
// retrying never turns a genuine runaway into a pass.
// See https://github.com/gotenberg/gotenberg/issues/1659.
for _, tc := range []struct {
scenario string
pattern *regexp2.Regexp
s string
expectMatch bool
expectError bool
}{
{
scenario: "deny-list match",
pattern: mustPattern(t, `^file:(?!//\/tmp/).*`),
s: "file:///etc/passwd",
expectMatch: true,
},
{
scenario: "no match against Gotenberg's own working directory",
pattern: mustPattern(t, `^file:(?!//\/tmp/).*`),
s: "file:///tmp/1a2b3c4d/5e6f7a8b/9c0d1e2f.html",
expectMatch: false,
},
{
scenario: "no match",
pattern: mustPattern(t, `^https://example\.com/`),
s: "https://example.org/",
expectMatch: false,
},
{
scenario: "catastrophic backtracking still aborts",
pattern: mustPattern(t, `^https://example\.com/(a+)+$`),
s: "https://example.com/" + strings.Repeat("a", 40) + "!",
expectError: true,
},
} {
t.Run(tc.scenario, func(t *testing.T) {
ok, err := MatchPattern(tc.pattern, tc.s)
if tc.expectError && err == nil {
t.Fatal("expected an error but got none")
}
if !tc.expectError && err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if ok != tc.expectMatch {
t.Fatalf("expected match %t but got %t", tc.expectMatch, ok)
}
})
}
}
func TestMatchPatternBoundsCatastrophicPattern(t *testing.T) {
// Proving a runaway is genuine costs one ceiling per attempt, so the
// worst case is patternMatchAttempts of them plus regexp2's clock period
// on each, roughly a second. The bound that matters is the one this
// replaced: before the ceiling existed, the same match was allowed to
// burn a core for the caller's whole 30s budget.
pattern := mustPattern(t, `^https://example\.com/(a+)+$`)
s := "https://example.com/" + strings.Repeat("a", 40) + "!"
start := time.Now()
_, err := MatchPattern(pattern, s)
elapsed := time.Since(start)
if err == nil {
t.Fatal("expected an error from a catastrophic pattern")
}
// Generous headroom over the expected second keeps this stable on a
// loaded CI box while still failing if the bound is gone.
if elapsed > 5*time.Second {
t.Fatalf("match took %s, want at most %d ceilings of %s", elapsed, patternMatchAttempts, PatternMatchTimeout)
}
}

View File

@@ -201,17 +201,13 @@ func TestNewServerRecordMetrics(t *testing.T) {
server.RecordMetrics(t.Context(), semconv.ServerMetricData{
ServerName: "stuff",
ResponseSize: 200,
MetricAttributes: semconv.MetricAttributes{
Req: req,
StatusCode: 301,
AdditionalAttributes: []attribute.KeyValue{
attribute.String("key", "value"),
},
},
MetricData: semconv.MetricData{
RequestSize: 100,
ElapsedTime: 300,
Req: req,
StatusCode: 301,
AdditionalAttributes: []attribute.KeyValue{
attribute.String("key", "value"),
},
RequestSize: 100,
ElapsedTime: 300,
})
rm := metricdata.ResourceMetrics{}

View File

@@ -12,10 +12,9 @@ import (
"time"
"github.com/alexliesenfeld/health"
"github.com/dlclark/regexp2"
"github.com/labstack/echo/v4"
"github.com/dlclark/regexp2/v2"
"github.com/labstack/echo/v5"
flag "github.com/spf13/pflag"
"golang.org/x/net/http2"
"golang.org/x/sync/errgroup"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
@@ -58,6 +57,12 @@ type Api struct {
fs *gotenberg.FileSystem
logger *slog.Logger
srv *echo.Echo
// shutdownCancel triggers the graceful shutdown of the server started by
// [echo.StartConfig]. Echo v5 drives shutdown from a context instead of an
// Echo.Shutdown method. serveDone closes once that shutdown completed.
shutdownCancel context.CancelFunc
serveDone chan struct{}
}
type downloadFromConfig struct {
@@ -134,7 +139,7 @@ const (
// middleware := Middleware{
// Handler: func() echo.MiddlewareFunc {
// return func(next echo.HandlerFunc) echo.HandlerFunc {
// return func(c echo.Context) error {
// return func(c *echo.Context) error {
// rootPath := c.Get("rootPath").(string)
// healthURI := fmt.Sprintf("%shealth", rootPath)
//
@@ -536,16 +541,29 @@ func (a *Api) Validate() error {
return nil
}
// newEchoServer builds the [echo.Echo] instance with the settings Gotenberg
// relies on, independently of the routes and middlewares added by [Api.Start].
func newEchoServer() *echo.Echo {
srv := echo.New()
srv.HTTPErrorHandler = httpErrorHandler()
// Echo v5 serves files through Echo.Filesystem, an [fs.FS] rooted at the
// working directory, and [fs.FS] rejects absolute names. Every output file
// lives under the request's temporary directory, so Context.Attachment gets
// an absolute path and the default filesystem answers 404 for every
// conversion. Rooting at "/" restores the v4 behavior, where the path was
// opened as-is.
srv.Filesystem = echo.NewDefaultFS("/")
// Echo v5 dropped the X-Forwarded-For and X-Real-IP fallbacks from
// Context.RealIP. Keep the previous behavior so that the access log still
// reports the client IP when Gotenberg sits behind a reverse proxy.
srv.IPExtractor = echo.LegacyIPExtractor()
return srv
}
// Start starts the HTTP server.
func (a *Api) Start() error {
a.srv = echo.New()
a.srv.HideBanner = true
a.srv.HidePort = true
a.srv.Server.ReadTimeout = a.timeout
a.srv.Server.IdleTimeout = a.timeout
// See https://github.com/gotenberg/gotenberg/issues/396.
a.srv.Server.WriteTimeout = a.timeout + a.timeout
a.srv.HTTPErrorHandler = httpErrorHandler()
a.srv = newEchoServer()
// Let's prepare the modules' routes.
var disableTelemetryForPaths []string
@@ -609,7 +627,7 @@ func (a *Api) Start() error {
securityMiddleware = oidcAuthMiddleware(verifier)
default:
securityMiddleware = func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return func(c *echo.Context) error {
return next(c)
}
}
@@ -641,7 +659,7 @@ func (a *Api) Start() error {
// Root route.
a.srv.GET(
a.rootPath,
func(c echo.Context) error {
func(c *echo.Context) error {
return c.HTML(http.StatusOK, `Hey, Gotenberg has no UI, it's an API. Head to the <a href="https://gotenberg.dev">documentation</a> to learn how to interact with it 🚀`)
},
securityMiddleware,
@@ -650,7 +668,7 @@ func (a *Api) Start() error {
// Favicon route.
a.srv.GET(
fmt.Sprintf("%s%s", a.rootPath, "favicon.ico"),
func(c echo.Context) error {
func(c *echo.Context) error {
return c.NoContent(http.StatusNoContent)
},
securityMiddleware,
@@ -681,7 +699,7 @@ func (a *Api) Start() error {
// ...the version route.
a.srv.GET(
fmt.Sprintf("%s%s", a.rootPath, "version"),
func(c echo.Context) error {
func(c *echo.Context) error {
return c.String(http.StatusOK, gotenberg.Version)
},
securityMiddleware,
@@ -691,7 +709,7 @@ func (a *Api) Start() error {
if a.enableDebugRoute {
a.srv.GET(
fmt.Sprintf("%s%s", a.rootPath, "debug"),
func(c echo.Context) error {
func(c *echo.Context) error {
return c.JSONPretty(http.StatusOK, gotenberg.Debug(), " ")
},
securityMiddleware,
@@ -712,18 +730,50 @@ func (a *Api) Start() error {
return fmt.Errorf("waiting for modules readiness: %w", err)
}
startConfig := echo.StartConfig{
Address: fmt.Sprintf("%s:%d", a.bindIp, a.port),
HideBanner: true,
HidePort: true,
BeforeServeFunc: func(s *http.Server) error {
s.ReadTimeout = a.timeout
s.IdleTimeout = a.timeout
// See https://github.com/gotenberg/gotenberg/issues/396.
s.WriteTimeout = a.timeout + a.timeout
if a.tlsCertFile == "" || a.tlsKeyFile == "" {
// Serve HTTP/2 Cleartext (h2c). Echo v5 dropped
// StartH2CServer and golang.org/x/net/http2/h2c is deprecated,
// so the standard library serves h2c through Server.Protocols
// instead.
protocols := new(http.Protocols)
protocols.SetHTTP1(true)
protocols.SetUnencryptedHTTP2(true)
s.Protocols = protocols
}
return nil
},
}
// Not named cancel: that would reassign the readiness timeout's cancel from
// above, whose deferred call is already bound to the old value.
serveCtx, serveCancel := context.WithCancel(context.Background())
a.shutdownCancel = serveCancel
a.serveDone = make(chan struct{})
// As the following code is blocking, run it in a goroutine.
go func() {
defer close(a.serveDone)
var err error
if a.tlsCertFile != "" && a.tlsKeyFile != "" {
// Start an HTTPS server (supports HTTP/2).
err = a.srv.StartTLS(fmt.Sprintf("%s:%d", a.bindIp, a.port), a.tlsCertFile, a.tlsKeyFile)
err = startConfig.StartTLS(serveCtx, a.srv, a.tlsCertFile, a.tlsKeyFile)
} else {
// Start an HTTP/2 Cleartext (non-HTTPS) server.
server := &http2.Server{}
err = a.srv.StartH2CServer(fmt.Sprintf("%s:%d", a.bindIp, a.port), server)
err = startConfig.Start(serveCtx, a.srv)
}
if !errors.Is(err, http.ErrServerClosed) {
if err != nil && !errors.Is(err, http.ErrServerClosed) {
a.logger.ErrorContext(context.Background(), err.Error())
}
}()
@@ -740,6 +790,26 @@ func (a *Api) StartupMessage() string {
return fmt.Sprintf("server started on %s:%d", ip, a.port)
}
// shutdown triggers the server's graceful shutdown and waits for it to
// complete, or for ctx to be done. Cancelling the serve context is what Echo v5
// exposes in place of Echo.Shutdown: [echo.StartConfig] shuts the server down
// on its own deadline, so the shutdown proceeds even when ctx is already done.
func (a *Api) shutdown(ctx context.Context) error {
if a.shutdownCancel == nil {
// Start never ran, so there is nothing to shut down.
return nil
}
a.shutdownCancel()
select {
case <-a.serveDone:
return nil
case <-ctx.Done():
return ctx.Err()
}
}
// Stop stops the HTTP server.
func (a *Api) Stop(ctx context.Context) error {
for {
@@ -749,7 +819,7 @@ func (a *Api) Stop(ctx context.Context) error {
}
select {
case <-ctx.Done():
return a.srv.Shutdown(ctx)
return a.shutdown(ctx)
default:
a.logger.DebugContext(ctx, fmt.Sprintf("%d asynchronous requests", count))
if count > 0 {
@@ -757,7 +827,7 @@ func (a *Api) Stop(ctx context.Context) error {
continue
}
a.logger.DebugContext(ctx, "no more asynchronous requests, continue with shutdown")
err := a.srv.Shutdown(ctx)
err := a.shutdown(ctx)
if err != nil {
return fmt.Errorf("shutdown: %w", err)
}

View File

@@ -18,7 +18,7 @@ import (
"github.com/google/uuid"
"github.com/hashicorp/go-retryablehttp"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"github.com/mholt/archives"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/codes"
@@ -70,7 +70,7 @@ type Context struct {
outputFilename string
logger *slog.Logger
echoCtx echo.Context
echoCtx *echo.Context
mkdirAll gotenberg.MkdirAll
pathRename gotenberg.PathRename
context.Context
@@ -158,7 +158,7 @@ type downloadFrom struct {
}
// newContext returns a [Context] by parsing a "multipart/form-data" request.
func newContext(echoCtx echo.Context, logger *slog.Logger, fs *gotenberg.FileSystem, timeout time.Duration, bodyLimit int64, downloadFromCfg downloadFromConfig) (*Context, context.CancelFunc, error) {
func newContext(echoCtx *echo.Context, logger *slog.Logger, fs *gotenberg.FileSystem, timeout time.Duration, bodyLimit int64, downloadFromCfg downloadFromConfig) (*Context, context.CancelFunc, error) {
processCtx, processCancel := context.WithTimeout(echoCtx.Request().Context(), timeout)
// We want to make sure the multipart/form-data does not exceed a given

View File

@@ -19,8 +19,8 @@ import (
"testing"
"time"
"github.com/dlclark/regexp2"
"github.com/labstack/echo/v4"
"github.com/dlclark/regexp2/v2"
"github.com/labstack/echo/v5"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
)
@@ -875,7 +875,7 @@ func TestNewContext_DownloadFromRedirectVerdictStaysGeneric(t *testing.T) {
fs := gotenberg.NewFileSystem(new(gotenberg.OsMkdirAll))
// The first hop is allowed, the redirect target is denied by the deny-list.
denyList := []*regexp2.Regexp{regexp2.MustCompile("^"+regexp.QuoteMeta(private.URL), 0)}
denyList := []*regexp2.Regexp{regexp2.MustCompile("^"+regexp.QuoteMeta(private.URL), regexp2.None)}
_, cancel, err := newContext(echoCtx, logger, fs, 10*time.Second, 0, downloadFromConfig{
denyList: denyList,

View File

@@ -12,8 +12,8 @@ import (
"github.com/coreos/go-oidc/v3/oidc"
"github.com/google/uuid"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v4/middleware"
"github.com/labstack/echo/v5"
"github.com/labstack/echo/v5/middleware"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
@@ -37,10 +37,12 @@ var (
// ParseError parses an error and returns the corresponding HTTP status and
// HTTP message.
func ParseError(err error) (int, string) {
var echoErr *echo.HTTPError
ok := errors.As(err, &echoErr)
if ok {
return echoErr.Code, http.StatusText(echoErr.Code)
// [echo.StatusCode] also matches the router's ErrNotFound and
// ErrMethodNotAllowed sentinels, which Echo v5 no longer models as
// [echo.HTTPError]. Matching that type alone would let every unrouted
// request fall through to a 500.
if code := echo.StatusCode(err); code != 0 {
return code, http.StatusText(code)
}
if errors.Is(err, context.DeadlineExceeded) {
@@ -100,14 +102,14 @@ const statusClientClosedRequest = 499
// A server-side timeout is [context.DeadlineExceeded], mapped to 503 by
// [ParseError], and is deliberately not treated as a client abort.
// See https://github.com/gotenberg/gotenberg/issues/1627.
func requestCanceled(c echo.Context, err error) bool {
func requestCanceled(c *echo.Context, err error) bool {
return errors.Is(err, context.Canceled) && errors.Is(c.Request().Context().Err(), context.Canceled)
}
// httpErrorHandler is the centralized HTTP error handler. It parses the error,
// returns a response as "text/plain; charset=UTF-8".
func httpErrorHandler() echo.HTTPErrorHandler {
return func(err error, c echo.Context) {
return func(c *echo.Context, err error) {
logger := c.Get("logger").(*slog.Logger)
if requestCanceled(c, err) {
@@ -134,7 +136,7 @@ func httpErrorHandler() echo.HTTPErrorHandler {
// startTime := c.Get("startTime").(time.Time)
func latencyMiddleware() echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return func(c *echo.Context) error {
// First piece for calculating the latency.
startTime := time.Now()
c.Set("startTime", startTime)
@@ -159,7 +161,7 @@ func latencyMiddleware() echo.MiddlewareFunc {
// }
func rootPathMiddleware(rootPath string) echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return func(c *echo.Context) error {
c.Set("rootPath", rootPath)
// Call the next middleware in the chain.
return next(c)
@@ -173,7 +175,7 @@ func rootPathMiddleware(rootPath string) echo.MiddlewareFunc {
// outputFilename := c.Get("outputFilename").(string)
func outputFilenameMiddleware() echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return func(c *echo.Context) error {
filename := c.Request().Header.Get("Gotenberg-Output-Filename")
// Keep only the last path segment, so that a caller cannot name an
// output file after a path.
@@ -203,10 +205,18 @@ func telemetryMiddleware(logger *slog.Logger, serverName, correlationIdHeader st
semconvSrv := semconvutil.NewHTTPServer(meter)
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return func(c *echo.Context) error {
startTime := c.Get("startTime").(time.Time)
rootPath := c.Get("rootPath").(string)
// Echo v5 returns the bare [http.ResponseWriter] from
// Context.Response, so unwrap it to read the recorded status and
// size below.
response, errUnwrap := echo.UnwrapResponse(c.Response())
if errUnwrap != nil {
return fmt.Errorf("unwrap response: %w", errUnwrap)
}
request := c.Request()
savedCtx := request.Context()
defer func() {
@@ -239,7 +249,7 @@ func telemetryMiddleware(logger *slog.Logger, serverName, correlationIdHeader st
err := next(c)
if err != nil {
c.Error(err)
c.Echo().HTTPErrorHandler(c, err)
}
return nil
}
@@ -288,7 +298,7 @@ func telemetryMiddleware(logger *slog.Logger, serverName, correlationIdHeader st
err := next(c)
finishTime := time.Now()
status := c.Response().Status
status := response.Status
canceled := false
if err != nil {
canceled = requestCanceled(c, err)
@@ -300,58 +310,68 @@ func telemetryMiddleware(logger *slog.Logger, serverName, correlationIdHeader st
}
span.SetAttributes(attribute.String("error", err.Error()))
c.Error(err)
c.Echo().HTTPErrorHandler(c, err)
}
span.SetStatus(semconvSrv.Status(status))
span.SetAttributes(semconvSrv.ResponseTraceAttrs(semconvutil.ResponseTelemetry{
StatusCode: status,
WriteBytes: c.Response().Size,
WriteBytes: response.Size,
})...)
accessLogger := logger.
With(slog.String("log_type", "access")).
With(slog.String("correlation_id", correlationId)).
With(slog.String("remote_ip", c.RealIP())).
With(slog.String("host", c.Request().Host)).
With(slog.String("uri", c.Request().RequestURI)).
With(slog.String("method", c.Request().Method)).
With(slog.String("path", routePath)).
With(slog.String("referer", c.Request().Referer())).
With(slog.String("user_agent", c.Request().UserAgent())).
With(slog.Int("status", c.Response().Status)).
With(slog.Int64("latency", int64(finishTime.Sub(startTime)))).
With(slog.String("latency_human", finishTime.Sub(startTime).String())).
With(slog.Int64("bytes_in", c.Request().ContentLength)).
With(slog.Int64("bytes_out", c.Response().Size))
// Pick the level and message before building the record: err.Error
// walks a joined error chain, and the nil-error branch has no use
// for it.
level := slog.LevelInfo
msg := "request handled"
switch {
case err == nil:
accessLogger.InfoContext(ctx, "request handled")
case canceled:
// A client abort is expected, not a server failure; keep it
// visible but out of the error stream.
accessLogger.InfoContext(ctx, err.Error())
msg = err.Error()
default:
accessLogger.ErrorContext(ctx, err.Error())
level = slog.LevelError
msg = err.Error()
}
// One record rather than a chain of With calls. Each With clones
// the whole handler chain, and this logger fans out to a JSON
// handler and an OpenTelemetry bridge that is wired in even when no
// exporter is configured, so a 14-deep chain clones both sub-chains
// 14 times to emit a single line.
latency := finishTime.Sub(startTime)
logger.LogAttrs(ctx, level, msg,
slog.String("log_type", "access"),
slog.String("correlation_id", correlationId),
slog.String("remote_ip", c.RealIP()),
slog.String("host", c.Request().Host),
slog.String("uri", c.Request().RequestURI),
slog.String("method", c.Request().Method),
slog.String("path", routePath),
slog.String("referer", c.Request().Referer()),
slog.String("user_agent", c.Request().UserAgent()),
slog.Int("status", response.Status),
slog.Int64("latency", int64(latency)),
slog.String("latency_human", latency.String()),
slog.Int64("bytes_in", c.Request().ContentLength),
slog.Int64("bytes_out", response.Size),
)
additionalAttributes := []attribute.KeyValue{
semconvSrv.Route(routePath),
}
semconvSrv.RecordMetrics(ctx, semconvutil.ServerMetricData{
ServerName: serverName,
ResponseSize: c.Response().Size,
MetricAttributes: semconvutil.MetricAttributes{
Req: request,
StatusCode: status,
AdditionalAttributes: additionalAttributes,
},
MetricData: semconvutil.MetricData{
RequestSize: request.ContentLength,
ElapsedTime: float64(time.Since(startTime)) / float64(time.Millisecond),
},
ServerName: serverName,
ResponseSize: response.Size,
Req: request,
StatusCode: status,
AdditionalAttributes: additionalAttributes,
RequestSize: request.ContentLength,
ElapsedTime: float64(time.Since(startTime)) / float64(time.Millisecond),
})
return nil
@@ -361,7 +381,7 @@ func telemetryMiddleware(logger *slog.Logger, serverName, correlationIdHeader st
// basicAuthMiddleware manages basic authentication.
func basicAuthMiddleware(username, password string) echo.MiddlewareFunc {
return middleware.BasicAuth(func(u string, p string, e echo.Context) (bool, error) {
return middleware.BasicAuth(func(c *echo.Context, u string, p string) (bool, error) {
if subtle.ConstantTimeCompare([]byte(u), []byte(username)) == 1 &&
subtle.ConstantTimeCompare([]byte(p), []byte(password)) == 1 {
return true, nil
@@ -407,7 +427,7 @@ func (a *Api) buildOidcVerifier() (*oidc.IDTokenVerifier, error) {
// it to the client.
func oidcAuthMiddleware(verifier *oidc.IDTokenVerifier) echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return func(c *echo.Context) error {
rawToken, ok := strings.CutPrefix(c.Request().Header.Get("Authorization"), "Bearer ")
if !ok || rawToken == "" {
return echo.NewHTTPError(http.StatusUnauthorized, "a Bearer token is required in the Authorization header")
@@ -436,7 +456,7 @@ func oidcAuthMiddleware(verifier *oidc.IDTokenVerifier) echo.MiddlewareFunc {
// cancel := c.Get("cancel").(context.CancelFunc)
func contextMiddleware(fs *gotenberg.FileSystem, timeout time.Duration, bodyLimit int64, downloadFromCfg downloadFromConfig) echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return func(c *echo.Context) error {
logger, _ := c.Get("logger").(*slog.Logger)
if logger == nil {
return errors.New("no logger in context (possible pool reuse)")
@@ -497,7 +517,7 @@ func contextMiddleware(fs *gotenberg.FileSystem, timeout time.Duration, bodyLimi
// handler fails to timeout as expected.
func hardTimeoutMiddleware(hardTimeout time.Duration) echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return func(c *echo.Context) error {
// Guard the type assertion so a pooled [echo.Context] whose
// store has been recycled under us does not crash the process.
// See the webhook async handler for the race this protects

View File

@@ -1,6 +1,7 @@
package api
import (
"bytes"
"context"
"crypto/rand"
"crypto/rsa"
@@ -9,13 +10,15 @@ import (
"log/slog"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/coreos/go-oidc/v3/oidc"
"github.com/coreos/go-oidc/v3/oidc/oidctest"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
)
// TestRequestCanceled pins the client-abort discriminator: only a
@@ -73,7 +76,7 @@ func TestHttpErrorHandler_ClientClosedRequest(t *testing.T) {
c := echo.New().NewContext(req, rec)
c.Set("logger", slog.New(slog.DiscardHandler))
httpErrorHandler()(tc.err, c)
httpErrorHandler()(c, tc.err)
if rec.Code != tc.wantStatus {
t.Fatalf("status = %d, want %d", rec.Code, tc.wantStatus)
@@ -105,7 +108,7 @@ func TestOutputFilenameMiddleware(t *testing.T) {
{"control characters", "fo\x01o\x7f", "foo"},
} {
t.Run(tc.name, func(t *testing.T) {
handler := outputFilenameMiddleware()(func(c echo.Context) error { return nil })
handler := outputFilenameMiddleware()(func(c *echo.Context) error { return nil })
req := httptest.NewRequest(http.MethodPost, "/", nil)
if tc.header != "" {
@@ -131,7 +134,7 @@ func TestOutputFilenameMiddleware(t *testing.T) {
func TestHardTimeoutMiddleware_MissingLoggerReturnsErrorInsteadOfPanicking(t *testing.T) {
mw := hardTimeoutMiddleware(100 * time.Millisecond)
handler := mw(func(c echo.Context) error { return nil })
handler := mw(func(c *echo.Context) error { return nil })
e := echo.New()
req := httptest.NewRequest(http.MethodGet, "/", nil)
@@ -220,7 +223,7 @@ func TestOidcAuthMiddleware(t *testing.T) {
}
c := echo.New().NewContext(req, httptest.NewRecorder())
handler := oidcAuthMiddleware(verifier)(func(c echo.Context) error {
handler := oidcAuthMiddleware(verifier)(func(c *echo.Context) error {
return c.NoContent(http.StatusOK)
})
@@ -243,3 +246,114 @@ func TestOidcAuthMiddleware(t *testing.T) {
})
}
}
// TestParseError_StatusMapping pins the statuses [ParseError] derives from the
// errors Echo and Gotenberg produce.
//
// Echo v5 models the router's ErrNotFound and ErrMethodNotAllowed as an
// unexported type rather than [echo.HTTPError], so matching that type alone
// would turn every unrouted request into a 500. It also guards the ordering:
// Gotenberg's own [SentinelHttpError] carries a client-facing message and must
// not be shadowed by the generic status lookup.
func TestParseError_StatusMapping(t *testing.T) {
for _, tc := range []struct {
scenario string
err error
wantStatus int
wantMessage string
}{
{"router not found", echo.ErrNotFound, http.StatusNotFound, http.StatusText(http.StatusNotFound)},
{"router method not allowed", echo.ErrMethodNotAllowed, http.StatusMethodNotAllowed, http.StatusText(http.StatusMethodNotAllowed)},
{"explicit HTTP error", echo.NewHTTPError(http.StatusUnauthorized, "nope"), http.StatusUnauthorized, http.StatusText(http.StatusUnauthorized)},
{"wrapped HTTP error", fmt.Errorf("authenticate request: %w", echo.NewHTTPError(http.StatusUnauthorized, "nope")), http.StatusUnauthorized, http.StatusText(http.StatusUnauthorized)},
{"sentinel keeps its message", NewSentinelHttpError(http.StatusBadRequest, "Invalid 'foo' form field value"), http.StatusBadRequest, "Invalid 'foo' form field value"},
{"unknown error", errors.New("boom"), http.StatusInternalServerError, http.StatusText(http.StatusInternalServerError)},
} {
t.Run(tc.scenario, func(t *testing.T) {
status, message := ParseError(tc.err)
if status != tc.wantStatus {
t.Fatalf("status = %d, want %d", status, tc.wantStatus)
}
if message != tc.wantMessage {
t.Fatalf("message = %q, want %q", message, tc.wantMessage)
}
})
}
}
// TestNewEchoServer_RealIP pins the client IP extraction that the access log's
// "remote_ip" field depends on.
//
// Echo v5.1.0 dropped the X-Forwarded-For and X-Real-IP fallbacks from
// Context.RealIP, so without an explicit extractor a Gotenberg behind a reverse
// proxy would log the proxy's address for every request. [newEchoServer]
// restores the previous behavior.
func TestNewEchoServer_RealIP(t *testing.T) {
srv := newEchoServer()
if srv.IPExtractor == nil {
t.Fatal("no IPExtractor configured: remote_ip would report the proxy address")
}
for _, tc := range []struct {
scenario string
headers map[string]string
want string
}{
{"x-forwarded-for keeps the client, not the proxy", map[string]string{"X-Forwarded-For": "203.0.113.7, 70.41.3.18"}, "203.0.113.7"},
{"single x-forwarded-for", map[string]string{"X-Forwarded-For": "203.0.113.7"}, "203.0.113.7"},
{"bracketed IPv6 is unwrapped", map[string]string{"X-Forwarded-For": "[2001:db8::1], 70.41.3.18"}, "2001:db8::1"},
{"x-real-ip when no x-forwarded-for", map[string]string{"X-Real-IP": "203.0.113.9"}, "203.0.113.9"},
{"no headers falls back to the remote address", nil, "192.0.2.1"},
} {
t.Run(tc.scenario, func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.RemoteAddr = "192.0.2.1:1234"
for key, value := range tc.headers {
req.Header.Set(key, value)
}
c := srv.NewContext(req, httptest.NewRecorder())
if got := c.RealIP(); got != tc.want {
t.Fatalf("RealIP = %q, want %q", got, tc.want)
}
})
}
}
// TestNewEchoServer_AttachmentServesAbsolutePath pins the filesystem that every
// conversion response is sent through.
//
// Echo v5 serves files through Echo.Filesystem, an [fs.FS] rooted at the working
// directory, and [fs.FS] rejects absolute names. Gotenberg builds every output
// file under the request's temporary directory and hands Context.Attachment an
// absolute path, so with the default filesystem every conversion route answers
// 404 while still reading the whole upload.
func TestNewEchoServer_AttachmentServesAbsolutePath(t *testing.T) {
path := filepath.Join(t.TempDir(), "output.pdf")
want := []byte("%PDF-1.7 not really a PDF")
err := os.WriteFile(path, want, 0o600)
if err != nil {
t.Fatalf("write fixture: %v", err)
}
srv := newEchoServer()
rec := httptest.NewRecorder()
c := srv.NewContext(httptest.NewRequest(http.MethodPost, "/", nil), rec)
err = c.Attachment(path, "output.pdf")
if err != nil {
t.Fatalf("Attachment(%q) = %v, want nil", path, err)
}
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
}
if got := rec.Body.Bytes(); !bytes.Equal(got, want) {
t.Fatalf("body = %q, want %q", got, want)
}
if got := rec.Header().Get("Content-Disposition"); !strings.Contains(got, `filename="output.pdf"`) {
t.Fatalf("Content-Disposition = %q, want it to carry filename=\"output.pdf\"", got)
}
}

View File

@@ -4,7 +4,7 @@ import (
"log/slog"
"github.com/alexliesenfeld/health"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
)
@@ -83,7 +83,7 @@ func (ctx *ContextMock) SetLogger(logger *slog.Logger) {
//
// ctx := &api.ContextMock{Context: &api.Context{}}
// ctx.setEchoContext(c)
func (ctx *ContextMock) SetEchoContext(c echo.Context) {
func (ctx *ContextMock) SetEchoContext(c *echo.Context) {
ctx.echoCtx = c
}

View File

@@ -17,12 +17,31 @@ import (
"github.com/chromedp/cdproto/page"
"github.com/chromedp/cdproto/runtime"
"github.com/chromedp/chromedp"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"github.com/shirou/gopsutil/v4/process"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
)
// chromiumDisableFeatures is the value of Chromium's --disable-features
// switch.
//
// It restates the "site-per-process,Translate,BlinkGenPropertyTrees" default
// from chromedp.DefaultExecAllocatorOptions (chromedp v0.14.2) on purpose:
// chromedp.Flag keys its flags by switch name, so a second --disable-features
// replaces chromedp's value instead of merging with it. Revisit this list when
// bumping chromedp.
//
// WebUIOmniboxPopup and WebUIOmniboxAimPopup became enabled by default in
// Chromium 151.0.7922.132. Their presenters build the address-bar popup WebUI
// at browser start, headless included, which leaves a renderer process holding
// ~85 MB of anonymous memory for a UI a PDF service can never show. Chromium
// silently ignores feature names it does not know, so both stay harmless on
// older builds (they exist but default to disabled on the Chromium pinned for
// ppc64el) and once upstream eventually removes them.
// See https://github.com/gotenberg/gotenberg/issues/1656.
const chromiumDisableFeatures = "site-per-process,Translate,BlinkGenPropertyTrees,WebUIOmniboxPopup,WebUIOmniboxAimPopup"
type browser interface {
gotenberg.Process
pdf(ctx context.Context, logger *slog.Logger, url, outputPath string, options PdfOptions, aggregate *networkAggregate) error
@@ -133,6 +152,8 @@ func (b *chromiumBrowser) Start(logger *slog.Logger) error {
chromedp.Flag("disable-dev-shm-usage", true),
// See https://github.com/gotenberg/gotenberg/issues/1293.
chromedp.Flag("disable-component-update", false),
// See https://github.com/gotenberg/gotenberg/issues/1656.
chromedp.Flag("disable-features", chromiumDisableFeatures),
)
if b.arguments.allowInsecureLocalhost {

View File

@@ -3,6 +3,7 @@ package chromium
import (
"context"
"log/slog"
"slices"
"strings"
"testing"
)
@@ -37,3 +38,24 @@ func TestChromiumBrowser_Start_rejectsOverlappingStart(t *testing.T) {
t.Fatal("expected the browser to stay not started")
}
}
// TestChromiumDisableFeatures guards the override described in
// https://github.com/gotenberg/gotenberg/issues/1656. Gotenberg replaces
// chromedp's --disable-features value rather than extending it, as
// chromedp.Flag keys its flags by switch name. Dropping one of chromedp's own
// entries while editing this list would silently re-enable it.
func TestChromiumDisableFeatures(t *testing.T) {
for _, feature := range []string{
// chromedp.DefaultExecAllocatorOptions.
"site-per-process",
"Translate",
"BlinkGenPropertyTrees",
// The address-bar popup WebUI, built even in headless.
"WebUIOmniboxPopup",
"WebUIOmniboxAimPopup",
} {
if !slices.Contains(strings.Split(chromiumDisableFeatures, ","), feature) {
t.Errorf("expected %q to be disabled, got %q", feature, chromiumDisableFeatures)
}
}
}

View File

@@ -14,7 +14,7 @@ import (
"github.com/alexliesenfeld/health"
"github.com/chromedp/cdproto/network"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
flag "github.com/spf13/pflag"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"

View File

@@ -19,7 +19,7 @@ import (
"github.com/chromedp/cdproto/page"
"github.com/chromedp/cdproto/runtime"
"github.com/chromedp/chromedp"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"golang.org/x/sync/errgroup"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
@@ -206,7 +206,7 @@ func listenForEventRequestPaused(ctx context.Context, logger *slog.Logger, optio
}
matchStart := time.Now()
ok, err := header.Scope.MatchString(e.Request.URL)
ok, err := gotenberg.MatchPattern(header.Scope, e.Request.URL)
budget.consume(time.Since(matchStart))
switch {
@@ -323,7 +323,15 @@ func listenForEventResponseReceived(
return
}
logger.DebugContext(ctx, fmt.Sprintf("event EventResponseReceived fired for a resource: %+v", ev.Response))
// Formatting the whole response is the most expensive thing this
// listener does, and it runs per sub-resource on chromedp's single
// per-target event goroutine while that goroutine holds the mutex
// it also takes to dispatch command responses. At the default log
// level the result is discarded, so gate it on the level rather
// than let slog drop it after the fact.
if logger.Enabled(ctx, slog.LevelDebug) {
logger.DebugContext(ctx, fmt.Sprintf("event EventResponseReceived fired for a resource: %+v", ev.Response))
}
if slices.Contains(options.failOnResourceOnHttpStatusCode, ev.Response.Status) {
if !shouldCheckResourceHttpStatusCode(ev.Response.URL, normalizedIgnoreDomains) {

View File

@@ -14,7 +14,7 @@ import (
"sync/atomic"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"golang.org/x/net/http/httpproxy"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"

View File

@@ -18,7 +18,7 @@ import (
"testing"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
)

View File

@@ -14,7 +14,7 @@ import (
"testing"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
)

View File

@@ -14,9 +14,9 @@ import (
"strings"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"github.com/gomarkdown/markdown"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"github.com/microcosm-cc/bluemonday"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
@@ -466,7 +466,7 @@ func convertUrlRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/chromium/convert/url",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumPdfOptions(ctx)
mode := pdfengines.FormDataPdfSplitMode(form, false)
@@ -528,7 +528,7 @@ func screenshotUrlRoute(chromium Api) api.Route {
Method: http.MethodPost,
Path: "/forms/chromium/screenshot/url",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumScreenshotOptions(ctx)
@@ -562,7 +562,7 @@ func convertHtmlRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/chromium/convert/html",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumPdfOptions(ctx)
mode := pdfengines.FormDataPdfSplitMode(form, false)
@@ -621,7 +621,7 @@ func screenshotHtmlRoute(chromium Api) api.Route {
Method: http.MethodPost,
Path: "/forms/chromium/screenshot/html",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumScreenshotOptions(ctx)
@@ -652,7 +652,7 @@ func convertMarkdownRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/chromium/convert/markdown",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumPdfOptions(ctx)
mode := pdfengines.FormDataPdfSplitMode(form, false)
@@ -720,7 +720,7 @@ func screenshotMarkdownRoute(chromium Api) api.Route {
Method: http.MethodPost,
Path: "/forms/chromium/screenshot/markdown",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumScreenshotOptions(ctx)

View File

@@ -6,7 +6,9 @@ import (
"testing"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
)
func TestScopeMatchBudget(t *testing.T) {
@@ -90,7 +92,7 @@ func TestScopeMatchBudget_BoundsCatastrophicBacktracking(t *testing.T) {
break
}
matchStart := time.Now()
_, _ = pattern.MatchString(url)
_, _ = gotenberg.MatchPattern(pattern, url)
b.consume(time.Since(matchStart))
matched++
}
@@ -100,10 +102,13 @@ func TestScopeMatchBudget_BoundsCatastrophicBacktracking(t *testing.T) {
t.Errorf("all %d headers were matched, want the budget to stop matching early", headers)
}
// Each match is separately capped at extraHttpHeaderScopeMatchTimeout, so
// the worst case is the budget plus one final match that started with the
// last of the credit. Generous slack keeps this stable on a loaded CI box.
ceiling := budget + extraHttpHeaderScopeMatchTimeout + time.Second
// A match that is genuinely out of budget costs a few
// extraHttpHeaderScopeMatchTimeout rather than one:
// [gotenberg.MatchPattern] retries an abort to tell a real runaway from
// one caused by the process losing the CPU. The worst case is the budget
// plus one final match that started with the last of the credit. Generous
// slack keeps this stable on a loaded CI box.
ceiling := budget + 4*extraHttpHeaderScopeMatchTimeout + time.Second
if elapsed > ceiling {
t.Errorf("matching took %s, want at most %s", elapsed, ceiling)
}

View File

@@ -14,7 +14,7 @@ import (
"sync"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
"golang.org/x/net/http/httpproxy"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"

View File

@@ -17,14 +17,14 @@ import (
"testing"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
)
func compileRegexes(t *testing.T, patterns ...string) []*regexp2.Regexp {
t.Helper()
out := make([]*regexp2.Regexp, 0, len(patterns))
for _, p := range patterns {
r, err := regexp2.Compile(p, 0)
r, err := regexp2.Compile(p, regexp2.None)
if err != nil {
t.Fatalf("compile %q: %v", p, err)
}

View File

@@ -7,7 +7,7 @@ import (
"slices"
"strconv"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v8/pkg/modules/api"
@@ -27,7 +27,7 @@ func convertRoute(libreOffice libreofficeapi.Uno, engine gotenberg.PdfEngine) ap
Method: http.MethodPost,
Path: "/forms/libreoffice/convert",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
defaultOptions := libreofficeapi.DefaultOptions()

View File

@@ -12,7 +12,7 @@ import (
"path/filepath"
"testing"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v8/pkg/modules/api"

View File

@@ -0,0 +1,172 @@
package pdfengines
import (
"sync"
"github.com/gotenberg/gotenberg/v8/pkg/modules/api"
)
// defaultMaxConcurrency is the number of PDF files a single stub processes at
// once when --pdfengines-max-concurrency (env PDFENGINES_MAX_CONCURRENCY) is
// not set. Each unit of work forks an external binary (qpdf, pdfcpu, pdftk or
// exiftool), so the ceiling trades wall clock against process count and RSS.
//
// It defaults to one, which processes files exactly as the sequential loops
// this package used to run did. Raising it only ever affects a request that
// carries several files, or one that splits into several outputs: a
// single-file request never reaches the concurrent path at all. Operators who
// send multi-file batches and have the memory headroom opt in.
//
// This never covers LibreOffice. libreoffice-pdfengine implements Convert and
// nothing else, every other [gotenberg.PdfEngine] method on it returns
// [gotenberg.ErrPdfEngineMethodNotSupported], and [ConvertStub] deliberately
// does not use this package's helpers. A soffice instance costs too much
// memory to run several of per container, so LibreOffice throughput is scaled
// by adding Gotenberg containers, not by raising this number.
const defaultMaxConcurrency = 1
// maxFileConcurrency is how many files one request may have in flight at once.
// It is replaced during [PdfEngines.Provision].
var maxFileConcurrency = defaultMaxConcurrency
// engineExtraSlots bounds the concurrency this package ADDS, across the whole
// process rather than per request, and holds one fewer slot than
// [maxFileConcurrency] because every request already owns one unit of its own.
//
// Bounding the added concurrency rather than the total is what keeps the
// ceiling from becoming a throughput regression. The sequential loops this
// helper replaced had no ceiling at all: X concurrent requests ran X engine
// binaries, one apiece. A pool covering the total would cut those X requests
// down to the ceiling, so an operator raising the flag to speed up a single
// multi-file request would slow the server down under real load. Reserving
// each request the unit it always had makes the worst case "what happened
// before, plus at most maxFileConcurrency-1".
//
// A per-request limit would have the opposite failure: X simultaneous requests
// forking X times the limit, trading the timeouts this exists to prevent for
// memory exhaustion.
var engineExtraSlots = make(chan struct{}, defaultMaxConcurrency-1)
// acquireEngineSlot waits for the first unit of capacity to become available,
// either this request's reserved unit or a slot from the shared pool, and
// returns the function that gives it back.
//
// Waiting on both at once is the whole point. Committing to one source and
// blocking on it strands the other: a goroutine parked on an exhausted pool
// cannot pick up its own request's reserved unit when the file before it
// finishes, so the reserved units sit idle while every file queues on the
// pool, which is slower than having no pool at all.
func acquireEngineSlot(ctx *api.Context, reserved chan struct{}) (func(), error) {
// An [api.Context] carries a request context in production, but one built
// as a literal, which the unit tests do, embeds a nil [context.Context]
// and would panic on Done. A nil channel never fires, which correctly
// leaves the two capacity sources as the only things to wait on.
var done <-chan struct{}
if ctx != nil && ctx.Context != nil {
done = ctx.Done()
}
// A select whose cancellation and capacity cases are both ready picks
// between them at random, so an already-dead request would start more
// files on a coin flip. Check first and stop taking on work.
if done != nil {
select {
case <-done:
return nil, ctx.Err()
default:
}
}
select {
case <-reserved:
return func() { reserved <- struct{}{} }, nil
case engineExtraSlots <- struct{}{}:
return func() { <-engineExtraSlots }, nil
case <-done:
return nil, ctx.Err()
}
}
// forEachInputPath runs fn against every input path, up to
// --pdfengines-max-concurrency (env PDFENGINES_MAX_CONCURRENCY) at a time.
//
// The stubs mutate each PDF in place, so distinct input paths never touch the
// same file and may run together. Callers that layer operations on one file,
// like [WatermarkStub] applying several watermarks in order, must keep that
// outer sequence and parallelize only the file dimension.
//
// Every path is attempted even after one fails, and the error returned is the
// first in input order rather than the first to arrive. That keeps the failing
// filename in the error message identical to what the sequential form
// reported, which the integration scenarios assert on.
func forEachInputPath(ctx *api.Context, inputPaths []string, fn func(inputPath string) error) error {
return forEachInputPathIndexed(ctx, inputPaths, func(_ int, inputPath string) error {
return fn(inputPath)
})
}
// forEachInputPathIndexed is [forEachInputPath] with the input path's index,
// for callers collecting a result per file. Writing into a preallocated slice
// at the given index needs no further synchronization; writing into a shared
// map does and must not be done from fn.
func forEachInputPathIndexed(ctx *api.Context, inputPaths []string, fn func(i int, inputPath string) error) error {
if len(inputPaths) == 0 {
return nil
}
// The common case is a single file. Skip the goroutine and the slot: the
// caller is already inside whatever bound its own route applies.
if len(inputPaths) == 1 {
return fn(0, inputPaths[0])
}
// At the default ceiling of one, run the plain sequential loop this helper
// replaced. Racing goroutines for a single slot would serialize the work
// just the same, but the order files are picked up in would be down to the
// scheduler, and every file would be attempted even once one has failed.
// Taking the old path keeps the default a genuine no-op: same order, same
// early return, no goroutines.
if maxFileConcurrency < 2 {
for i, inputPath := range inputPaths {
err := fn(i, inputPath)
if err != nil {
return err
}
}
return nil
}
// The unit this request would have had all to itself before any of this
// existed. Whichever file claims it runs without touching the shared pool,
// so concurrent requests can never throttle each other below the
// one-binary-apiece they already got. See [engineExtraSlots].
reserved := make(chan struct{}, 1)
reserved <- struct{}{}
errs := make([]error, len(inputPaths))
var wg sync.WaitGroup
for i, inputPath := range inputPaths {
wg.Go(func() {
release, err := acquireEngineSlot(ctx, reserved)
if err != nil {
errs[i] = err
return
}
defer release()
errs[i] = fn(i, inputPath)
})
}
wg.Wait()
for _, err := range errs {
if err != nil {
return err
}
}
return nil
}

View File

@@ -0,0 +1,341 @@
package pdfengines
import (
"context"
"errors"
"fmt"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/gotenberg/gotenberg/v8/pkg/modules/api"
)
func TestForEachInputPath(t *testing.T) {
for _, tc := range []struct {
scenario string
inputPaths []string
fn func(inputPath string) error
expectErr string
}{
{
scenario: "no input path",
inputPaths: nil,
fn: func(string) error { return errors.New("must not run") },
},
{
scenario: "single input path",
inputPaths: []string{"a.pdf"},
fn: func(string) error { return nil },
},
{
scenario: "single input path with error",
inputPaths: []string{"a.pdf"},
fn: func(p string) error { return fmt.Errorf("boom %s", p) },
expectErr: "boom a.pdf",
},
{
scenario: "many input paths",
inputPaths: []string{"a.pdf", "b.pdf", "c.pdf", "d.pdf", "e.pdf"},
fn: func(string) error { return nil },
},
{
scenario: "error is the first in input order, not the first to arrive",
inputPaths: []string{"a.pdf", "b.pdf", "c.pdf"},
fn: func(p string) error {
// "c.pdf" fails without delay so that it lands well before
// "b.pdf"; the reported error must still be "b.pdf".
if p == "b.pdf" {
var counter int
for i := range 5_000_000 {
counter += i
}
return fmt.Errorf("slow failure %s (%d)", p, counter%1)
}
if p == "c.pdf" {
return fmt.Errorf("fast failure %s", p)
}
return nil
},
expectErr: "slow failure b.pdf (0)",
},
} {
t.Run(tc.scenario, func(t *testing.T) {
err := forEachInputPath(new(api.Context), tc.inputPaths, tc.fn)
if tc.expectErr == "" {
if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
return
}
if err == nil {
t.Fatalf("expected error %q but got none", tc.expectErr)
}
if err.Error() != tc.expectErr {
t.Fatalf("expected error %q but got %q", tc.expectErr, err.Error())
}
})
}
}
func TestForEachInputPathRunsEveryPath(t *testing.T) {
inputPaths := make([]string, 50)
for i := range inputPaths {
inputPaths[i] = fmt.Sprintf("%d.pdf", i)
}
var (
mu sync.Mutex
seen = make(map[string]int)
)
err := forEachInputPath(new(api.Context), inputPaths, func(inputPath string) error {
mu.Lock()
defer mu.Unlock()
seen[inputPath]++
return nil
})
if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if len(seen) != len(inputPaths) {
t.Fatalf("expected %d distinct paths but got %d", len(inputPaths), len(seen))
}
for path, count := range seen {
if count != 1 {
t.Fatalf("expected '%s' to run once but it ran %d times", path, count)
}
}
}
func TestForEachInputPathRespectsTheSlotCeiling(t *testing.T) {
previousSlots, previousMax := engineExtraSlots, maxFileConcurrency
defer func() { engineExtraSlots, maxFileConcurrency = previousSlots, previousMax }()
// One request may run its reserved unit plus ceiling-1 borrowed ones.
const ceiling = 3
maxFileConcurrency = ceiling
engineExtraSlots = make(chan struct{}, ceiling-1)
inputPaths := make([]string, 40)
for i := range inputPaths {
inputPaths[i] = fmt.Sprintf("%d.pdf", i)
}
var inFlight, peak atomic.Int64
err := forEachInputPath(new(api.Context), inputPaths, func(string) error {
current := inFlight.Add(1)
defer inFlight.Add(-1)
for {
observed := peak.Load()
if current <= observed || peak.CompareAndSwap(observed, current) {
break
}
}
// Hold the slot long enough that the ceiling would be exceeded if it
// were not enforced.
var counter int
for i := range 200_000 {
counter += i
}
_ = counter
return nil
})
if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if peak.Load() > ceiling {
t.Fatalf("expected at most %d concurrent runs but observed %d", ceiling, peak.Load())
}
}
func TestForEachInputPathHonorsCancellation(t *testing.T) {
previousSlots, previousMax := engineExtraSlots, maxFileConcurrency
defer func() { engineExtraSlots, maxFileConcurrency = previousSlots, previousMax }()
// Concurrent path, with the shared pool exhausted by another request, so
// only this request's reserved unit is available.
maxFileConcurrency = 3
engineExtraSlots = make(chan struct{}, 2)
engineExtraSlots <- struct{}{}
engineExtraSlots <- struct{}{}
cancelledCtx, cancel := context.WithCancel(context.Background())
cancel()
// Neither source of capacity is available: the pool is exhausted by other
// requests and this request's reserved unit is already in use by one of its
// own files. A waiter must observe the cancelled context rather than block
// forever. Driving acquireEngineSlot directly keeps that deterministic:
// through forEachInputPath the reserved unit is reusable, so whether a
// given file waits at all depends on how fast the file before it finishes.
inUse := make(chan struct{}, 1)
release, err := acquireEngineSlot(&api.Context{Context: cancelledCtx}, inUse)
if !errors.Is(err, context.Canceled) {
t.Fatalf("expected context.Canceled but got: %v", err)
}
if release != nil {
t.Fatal("expected no release function when acquisition fails")
}
// A cancelled request stops taking on work even when capacity is free,
// rather than deciding on the coin flip a ready select would give.
inUse <- struct{}{}
_, err = acquireEngineSlot(&api.Context{Context: cancelledCtx}, inUse)
if !errors.Is(err, context.Canceled) {
t.Fatalf("expected context.Canceled with the reserved unit free but got: %v", err)
}
// Live request, free reserved unit: acquired and handed back.
release, err = acquireEngineSlot(&api.Context{Context: context.Background()}, inUse)
if err != nil {
t.Fatalf("expected the reserved unit to be acquired but got: %v", err)
}
release()
if len(inUse) != 1 {
t.Fatalf("expected the reserved unit to be returned but the channel holds %d", len(inUse))
}
}
func TestForEachInputPathCompletesWithACancelledContext(t *testing.T) {
previousSlots, previousMax := engineExtraSlots, maxFileConcurrency
defer func() { engineExtraSlots, maxFileConcurrency = previousSlots, previousMax }()
maxFileConcurrency = 3
engineExtraSlots = make(chan struct{}, 2)
engineExtraSlots <- struct{}{}
engineExtraSlots <- struct{}{}
cancelledCtx, cancel := context.WithCancel(context.Background())
cancel()
done := make(chan struct{})
go func() {
defer close(done)
_ = forEachInputPath(&api.Context{Context: cancelledCtx}, []string{"a.pdf", "b.pdf", "c.pdf"}, func(string) error {
return nil
})
}()
select {
case <-done:
case <-time.After(10 * time.Second):
t.Fatal("forEachInputPath hung on a cancelled context with the shared pool exhausted")
}
}
func TestForEachInputPathNeverThrottlesBelowOnePerRequest(t *testing.T) {
previousSlots, previousMax := engineExtraSlots, maxFileConcurrency
defer func() { engineExtraSlots, maxFileConcurrency = previousSlots, previousMax }()
// A small ceiling against far more concurrent requests than it covers.
// Before the shared pool existed each of these ran a binary of its own, so
// the pool must not drop aggregate concurrency below one per request.
const (
ceiling = 2
requests = 8
)
maxFileConcurrency = ceiling
engineExtraSlots = make(chan struct{}, ceiling-1)
// Every runner announces itself and then blocks, so the count of arrivals
// is the true simultaneous concurrency rather than whatever the scheduler
// happened to overlap.
arrived := make(chan struct{}, requests*3)
release := make(chan struct{})
var wg sync.WaitGroup
for range requests {
wg.Go(func() {
_ = forEachInputPath(new(api.Context), []string{"a.pdf", "b.pdf", "c.pdf"}, func(string) error {
arrived <- struct{}{}
<-release
return nil
})
})
}
// One runner per request must be able to start without waiting on the
// shared pool. If the pool governed the total instead of the surplus, only
// `ceiling` runners would ever arrive and this would time out.
for i := range requests {
select {
case <-arrived:
case <-time.After(10 * time.Second):
t.Fatalf("only %d runners started concurrently, expected at least one per request (%d); the shared pool is throttling requests against each other", i, requests)
}
}
close(release)
wg.Wait()
}
func TestForEachInputPathIsSequentialAtTheDefaultCeiling(t *testing.T) {
if defaultMaxConcurrency != 1 {
t.Fatalf("this test pins the default as a no-op, but defaultMaxConcurrency is %d", defaultMaxConcurrency)
}
previousSlots, previousMax := engineExtraSlots, maxFileConcurrency
defer func() { engineExtraSlots, maxFileConcurrency = previousSlots, previousMax }()
maxFileConcurrency = defaultMaxConcurrency
engineExtraSlots = make(chan struct{}, defaultMaxConcurrency-1)
// At the default the helper must behave exactly like the sequential loops
// it replaced: files in input order, and no file attempted once one has
// failed.
var order []string
err := forEachInputPath(new(api.Context), []string{"a.pdf", "b.pdf", "c.pdf", "d.pdf"}, func(inputPath string) error {
order = append(order, inputPath)
if inputPath == "b.pdf" {
return errors.New("boom")
}
return nil
})
if err == nil || err.Error() != "boom" {
t.Fatalf("expected error \"boom\" but got: %v", err)
}
if len(order) != 2 || order[0] != "a.pdf" || order[1] != "b.pdf" {
t.Fatalf("expected the run to stop after b.pdf in input order but got %v", order)
}
}
func TestForEachInputPathIndexed(t *testing.T) {
inputPaths := []string{"a.pdf", "b.pdf", "c.pdf", "d.pdf"}
collected := make([]string, len(inputPaths))
err := forEachInputPathIndexed(new(api.Context), inputPaths, func(i int, inputPath string) error {
collected[i] = inputPath
return nil
})
if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
for i, inputPath := range inputPaths {
if collected[i] != inputPath {
t.Fatalf("expected index %d to hold '%s' but got '%s'", i, inputPath, collected[i])
}
}
}

View File

@@ -45,6 +45,7 @@ type PdfEngines struct {
rotateNames []string
facturXNames []string
engines []gotenberg.PdfEngine
maxConcurrency int
disableRoutes bool
}
@@ -70,6 +71,7 @@ func (mod *PdfEngines) Descriptor() gotenberg.ModuleDescriptor {
fs.StringSlice("pdfengines-stamp-engines", []string{"pdfcpu", "pdftk"}, "Set the PDF engines and their order for the stamp feature - empty means all")
fs.StringSlice("pdfengines-rotate-engines", []string{"pdfcpu", "pdftk"}, "Set the PDF engines and their order for the rotate feature - empty means all")
fs.StringSlice("pdfengines-factur-x-engines", []string{"qpdf"}, "Set the PDF engines and their order for the Factur-X XMP feature - empty means all")
fs.Int("pdfengines-max-concurrency", defaultMaxConcurrency, "Set the maximum number of PDF files a feature processes concurrently, across all requests - bounds how many qpdf, pdfcpu, pdftk and exiftool processes run at once, so raising it trades memory for speed. Does not apply to LibreOffice: scale Gotenberg containers instead")
fs.Bool("pdfengines-disable-routes", false, "Disable the routes")
// Deprecated flags.
@@ -105,8 +107,16 @@ func (mod *PdfEngines) Provision(ctx *gotenberg.Context) error {
stampNames := flags.MustStringSlice("pdfengines-stamp-engines")
rotateNames := flags.MustStringSlice("pdfengines-rotate-engines")
facturXNames := flags.MustStringSlice("pdfengines-factur-x-engines")
mod.maxConcurrency = flags.MustInt("pdfengines-max-concurrency")
mod.disableRoutes = flags.MustBool("pdfengines-disable-routes")
if mod.maxConcurrency > 0 {
maxFileConcurrency = mod.maxConcurrency
// One fewer than the ceiling: each request already reserves a unit of
// its own. See [engineExtraSlots].
engineExtraSlots = make(chan struct{}, mod.maxConcurrency-1)
}
engines, err := ctx.Modules(new(gotenberg.PdfEngine))
if err != nil {
return fmt.Errorf("get PDF engines: %w", err)
@@ -222,6 +232,10 @@ func (mod *PdfEngines) Validate() error {
return errors.New("no PDF engine is available; enable at least one engine module (e.g. qpdf, pdfcpu, pdftk, libreoffice-pdfengine, exiftool)")
}
if mod.maxConcurrency < 1 {
return fmt.Errorf("PDF engines max concurrency must be at least 1, got %d; set --pdfengines-max-concurrency (env PDFENGINES_MAX_CONCURRENCY) to a positive value", mod.maxConcurrency)
}
availableEngines := make([]string, len(mod.engines))
for i, engine := range mod.engines {
@@ -296,6 +310,7 @@ func (mod *PdfEngines) SystemMessages() []string {
fmt.Sprintf("stamp engines - %s", strings.Join(mod.stampNames, " ")),
fmt.Sprintf("rotate engines - %s", strings.Join(mod.rotateNames, " ")),
fmt.Sprintf("factur-x engines - %s", strings.Join(mod.facturXNames, " ")),
fmt.Sprintf("max concurrency - %d", mod.maxConcurrency),
}
}

View File

@@ -11,7 +11,7 @@ import (
"strings"
"github.com/google/uuid"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v8/pkg/modules/api"
@@ -208,14 +208,14 @@ func RotateStub(ctx *api.Context, engine gotenberg.PdfEngine, angle int, pages s
return nil
}
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
err := engine.Rotate(ctx, ctx.Log(), inputPath, angle, pages)
if err != nil {
return fmt.Errorf("rotate '%s': %w", inputPath, err)
}
}
return nil
return nil
})
}
// ValidatePdfFormatsCompat checks for incompatible combinations of PDF formats
@@ -334,14 +334,14 @@ func SplitPdfStub(ctx *api.Context, engine gotenberg.PdfEngine, mode gotenberg.S
// FlattenStub merges annotation appearances with page content for each given
// PDF, effectively deleting the original annotations.
func FlattenStub(ctx *api.Context, engine gotenberg.PdfEngine, inputPaths []string) error {
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
err := engine.Flatten(ctx, ctx.Log(), inputPath)
if err != nil {
return fmt.Errorf("flatten '%s': %w", inputPath, err)
}
}
return nil
return nil
})
}
// defaultImageQuality is the JPEG quality applied by the image optimization
@@ -393,19 +393,27 @@ func OptimizeStub(ctx *api.Context, engine gotenberg.PdfEngine, optimizeImages b
return nil
}
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
err := engine.OptimizeImages(ctx, ctx.Log(), imageQuality, inputPath)
if err != nil {
return fmt.Errorf("optimize images of '%s': %w", inputPath, err)
}
}
return nil
return nil
})
}
// ConvertStub transforms a given PDF to the specified formats defined in
// [gotenberg.PdfFormats]. If no format, it does nothing and returns the input
// paths.
//
// This loop stays sequential on purpose. Convert is the one PDF engine method
// LibreOffice implements, and libreoffice-pdfengine is the default and only
// convert engine, so every iteration here drives the single soffice daemon. A
// LibreOffice instance is far too memory-hungry to run several of per
// container: the way to convert more documents at once is to scale Gotenberg
// containers, not to widen this loop. Do not route it through
// [forEachInputPath].
func ConvertStub(ctx *api.Context, engine gotenberg.PdfEngine, formats gotenberg.PdfFormats, inputPaths []string) ([]string, error) {
zeroValued := gotenberg.PdfFormats{}
if formats == zeroValued {
@@ -432,14 +440,14 @@ func WriteMetadataStub(ctx *api.Context, engine gotenberg.PdfEngine, metadata ma
return nil
}
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
err := engine.WriteMetadata(ctx, ctx.Log(), metadata, inputPath)
if err != nil {
return fmt.Errorf("write metadata into '%s': %w", inputPath, err)
}
}
return nil
return nil
})
}
// documentTitle returns the input PDF's Title metadata entry, falling back to
@@ -491,28 +499,33 @@ func WriteBookmarksStub(ctx *api.Context, engine gotenberg.PdfEngine, bookmarks
return nil
}
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
err := engine.WriteBookmarks(ctx, ctx.Log(), inputPath, b)
if err != nil {
return fmt.Errorf("write bookmarks into '%s': %w", inputPath, err)
}
}
return nil
})
case map[string][]gotenberg.Bookmark:
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
filename := ctx.OriginalFilename(inputPath)
if specificBookmarks, ok := b[filename]; ok {
err := engine.WriteBookmarks(ctx, ctx.Log(), inputPath, specificBookmarks)
if err != nil {
return fmt.Errorf("write bookmarks into '%s': %w", inputPath, err)
}
specificBookmarks, ok := b[filename]
if !ok {
return nil
}
}
err := engine.WriteBookmarks(ctx, ctx.Log(), inputPath, specificBookmarks)
if err != nil {
return fmt.Errorf("write bookmarks into '%s': %w", inputPath, err)
}
return nil
})
default:
// Should not happen.
return fmt.Errorf("bookmarks type '%T' not supported", bookmarks)
}
return nil
}
// FormDataPdfEmbeds extracts embedded file paths from form data.
@@ -537,14 +550,14 @@ func EmbedFilesMetadataStub(ctx *api.Context, engine gotenberg.PdfEngine, metada
return nil
}
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
err := engine.EmbedFilesMetadata(ctx, ctx.Log(), metadata, inputPath)
if err != nil {
return fmt.Errorf("set embeds metadata on PDF '%s': %w", inputPath, err)
}
}
return nil
return nil
})
}
// FormDataPdfFacturX extracts the Factur-X parameters and the invoice XML path
@@ -737,14 +750,14 @@ func InjectFacturXXMPStub(ctx *api.Context, engine gotenberg.PdfEngine, facturX
return nil
}
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
err := engine.InjectFacturXXMP(ctx, ctx.Log(), facturX, inputPath)
if err != nil {
return fmt.Errorf("inject Factur-X XMP into PDF '%s': %w", inputPath, err)
}
}
return nil
return nil
})
}
// FormDataPdfEncrypt extracts the encryption parameters and permissions from
@@ -783,14 +796,14 @@ func EncryptPdfStub(ctx *api.Context, engine gotenberg.PdfEngine, opts gotenberg
return nil
}
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
err := engine.Encrypt(ctx, ctx.Log(), inputPath, opts)
if err != nil {
return fmt.Errorf("encrypt PDF '%s': %w", inputPath, err)
}
}
return nil
return nil
})
}
// EmbedFilesStub embeds files into PDF files.
@@ -819,14 +832,14 @@ func EmbedFilesStub(ctx *api.Context, engine gotenberg.PdfEngine, embedPaths []s
resolvedPaths[i] = resolvedPath
}
for _, inputPath := range inputPaths {
return forEachInputPath(ctx, inputPaths, func(inputPath string) error {
err := engine.EmbedFiles(ctx, ctx.Log(), resolvedPaths, inputPath)
if err != nil {
return fmt.Errorf("embed files into PDF '%s': %w", inputPath, err)
}
}
return nil
return nil
})
}
// FormDataPdfStamps builds the ordered list of stamps from the repeated stamp
@@ -943,16 +956,23 @@ func bindStampOrWatermarkFiles(stamps []gotenberg.Stamp, files []string, kind st
// WatermarkStub applies each watermark to a list of PDF files, in order.
// Entries with no source are skipped, so an empty list does nothing.
func WatermarkStub(ctx *api.Context, engine gotenberg.PdfEngine, watermarks []gotenberg.Stamp, inputPaths []string) error {
// Watermarks stack on the same file, so the outer loop stays sequential;
// only the file dimension is parallel.
for _, watermark := range watermarks {
if watermark.Source == "" {
continue
}
for _, inputPath := range inputPaths {
err := engine.Watermark(ctx, ctx.Log(), inputPath, watermark)
if err != nil {
return fmt.Errorf("watermark '%s': %w", inputPath, err)
err := forEachInputPath(ctx, inputPaths, func(inputPath string) error {
errWatermark := engine.Watermark(ctx, ctx.Log(), inputPath, watermark)
if errWatermark != nil {
return fmt.Errorf("watermark '%s': %w", inputPath, errWatermark)
}
return nil
})
if err != nil {
return err
}
}
@@ -962,16 +982,23 @@ func WatermarkStub(ctx *api.Context, engine gotenberg.PdfEngine, watermarks []go
// StampStub applies each stamp to a list of PDF files, in order. Entries with
// no source are skipped, so an empty list does nothing.
func StampStub(ctx *api.Context, engine gotenberg.PdfEngine, stamps []gotenberg.Stamp, inputPaths []string) error {
// Stamps stack on the same file, so the outer loop stays sequential; only
// the file dimension is parallel.
for _, stamp := range stamps {
if stamp.Source == "" {
continue
}
for _, inputPath := range inputPaths {
err := engine.Stamp(ctx, ctx.Log(), inputPath, stamp)
if err != nil {
return fmt.Errorf("stamp '%s': %w", inputPath, err)
err := forEachInputPath(ctx, inputPaths, func(inputPath string) error {
errStamp := engine.Stamp(ctx, ctx.Log(), inputPath, stamp)
if errStamp != nil {
return fmt.Errorf("stamp '%s': %w", inputPath, errStamp)
}
return nil
})
if err != nil {
return err
}
}
@@ -984,7 +1011,7 @@ func mergeRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/merge",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1185,7 +1212,7 @@ func splitRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/split",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1337,7 +1364,7 @@ func flattenRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/flatten",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1372,7 +1399,7 @@ func optimizeRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/optimize",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1408,7 +1435,7 @@ func convertRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/convert",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1465,7 +1492,7 @@ func readMetadataRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/metadata/read",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
var inputPaths []string
@@ -1476,14 +1503,25 @@ func readMetadataRoute(engine gotenberg.PdfEngine) api.Route {
return fmt.Errorf("validate form data: %w", err)
}
res := make(map[string]map[string]any, len(inputPaths))
for _, inputPath := range inputPaths {
metadata, err := engine.ReadMetadata(ctx, ctx.Log(), inputPath)
if err != nil {
return fmt.Errorf("read metadata: %w", err)
// Collected per index, then folded into the map on this
// goroutine: a shared map cannot be written concurrently.
collected := make([]map[string]any, len(inputPaths))
err = forEachInputPathIndexed(ctx, inputPaths, func(i int, inputPath string) error {
metadata, errRead := engine.ReadMetadata(ctx, ctx.Log(), inputPath)
if errRead != nil {
return fmt.Errorf("read metadata: %w", errRead)
}
res[ctx.OriginalFilename(inputPath)] = metadata
collected[i] = metadata
return nil
})
if err != nil {
return err
}
res := make(map[string]map[string]any, len(inputPaths))
for i, inputPath := range inputPaths {
res[ctx.OriginalFilename(inputPath)] = collected[i]
}
err = c.JSON(http.StatusOK, res)
@@ -1508,7 +1546,7 @@ func writeMetadataRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/metadata/write",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1543,7 +1581,7 @@ func readBookmarksRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/bookmarks/read",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
var inputPaths []string
@@ -1554,14 +1592,25 @@ func readBookmarksRoute(engine gotenberg.PdfEngine) api.Route {
return fmt.Errorf("validate form data: %w", err)
}
res := make(map[string][]gotenberg.Bookmark, len(inputPaths))
for _, inputPath := range inputPaths {
bookmarks, err := engine.ReadBookmarks(ctx, ctx.Log(), inputPath)
if err != nil {
return fmt.Errorf("read bookmarks: %w", err)
// Collected per index, then folded into the map on this
// goroutine: a shared map cannot be written concurrently.
collected := make([][]gotenberg.Bookmark, len(inputPaths))
err = forEachInputPathIndexed(ctx, inputPaths, func(i int, inputPath string) error {
bookmarks, errRead := engine.ReadBookmarks(ctx, ctx.Log(), inputPath)
if errRead != nil {
return fmt.Errorf("read bookmarks: %w", errRead)
}
res[ctx.OriginalFilename(inputPath)] = bookmarks
collected[i] = bookmarks
return nil
})
if err != nil {
return err
}
res := make(map[string][]gotenberg.Bookmark, len(inputPaths))
for i, inputPath := range inputPaths {
res[ctx.OriginalFilename(inputPath)] = collected[i]
}
err = c.JSON(http.StatusOK, res)
@@ -1585,7 +1634,7 @@ func writeBookmarksRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/bookmarks/write",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1620,7 +1669,7 @@ func encryptRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/encrypt",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1665,7 +1714,7 @@ func embedRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/embed",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1727,7 +1776,7 @@ func watermarkRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/watermark",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1784,7 +1833,7 @@ func stampRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/stamp",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1843,7 +1892,7 @@ func rotateRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/rotate",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()
@@ -1880,7 +1929,7 @@ func facturXRoute(engine gotenberg.PdfEngine) api.Route {
Method: http.MethodPost,
Path: "/forms/pdfengines/factur-x",
IsMultipart: true,
Handler: func(c echo.Context) error {
Handler: func(c *echo.Context) error {
ctx := c.Get("context").(*api.Context)
form := ctx.FormData()

View File

@@ -7,7 +7,7 @@ import (
"net/http"
"time"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
flag "github.com/spf13/pflag"

View File

@@ -11,7 +11,7 @@ import (
"time"
"github.com/hashicorp/go-retryablehttp"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/codes"
"go.opentelemetry.io/otel/propagation"

View File

@@ -14,7 +14,7 @@ import (
"time"
"github.com/hashicorp/go-retryablehttp"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v8/pkg/modules/api"
@@ -95,7 +95,7 @@ func webhookMiddleware(w *Webhook) api.Middleware {
})
}
return func(c echo.Context) error {
return func(c *echo.Context) error {
webhookUrl := c.Request().Header.Get("Gotenberg-Webhook-Url")
if webhookUrl == "" {
// No webhook URL, call the next middleware in the chain.
@@ -208,7 +208,7 @@ func webhookMiddleware(w *Webhook) api.Middleware {
}
}
// Retrieve values from echo.Context before it gets recycled.
// Retrieve values from [echo.Context] before it gets recycled.
// See https://github.com/gotenberg/gotenberg/issues/1000.
startTime := c.Get("startTime").(time.Time)
correlationIdHeader := c.Get("correlationIdHeader").(string)
@@ -323,16 +323,16 @@ func webhookMiddleware(w *Webhook) api.Middleware {
// As a webhook URL has been given, we handle the request in a
// goroutine and return immediately.
//
// Echo returns the echo.Context back to its sync.Pool as
// Echo returns the [echo.Context] back to its sync.Pool as
// soon as this synchronous handler returns ErrAsyncProcess.
// A concurrent request can then claim the recycled context
// and c.Reset() wipes the shared store, which would cause
// any c.Get("...").(T) assertion downstream of the webhook
// goroutine to panic on a nil value and crash the process.
// Snapshot the keys downstream reads onto a detached
// wrapper before spawning the goroutine so pool reuse
// context before spawning the goroutine so pool reuse
// cannot reach into our async work.
detached := newPoolSafeContext(c, "logger", "context", "correlationId", "correlationIdHeader", "startTime")
detached := newDetachedContext(c, "logger", "context", "correlationId", "correlationIdHeader", "startTime")
w.asyncCount.Add(1)
go func() {

View File

@@ -1,58 +1,34 @@
package webhook
import (
"sync"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
)
// poolSafeContext wraps an [echo.Context] and keeps a private snapshot of
// the values that downstream middleware and route handlers read from the
// store. Echo returns an [echo.Context] to its sync.Pool as soon as the
// synchronous handler returns, including when the webhook middleware
// returns [api.ErrAsyncProcess]. A concurrent request can then claim the
// recycled context and c.Reset() wipes the shared store out from under
// the webhook goroutine, which causes any
// `c.Get("logger").(*slog.Logger)`-style assertion further down the
// chain to panic on a nil value.
// newDetachedContext returns an [echo.Context] carrying a snapshot of the given
// keys, detached from Echo's context pool.
//
// Wrapping c before handing it to the goroutine insulates the async work
// from pool reuse: Get/Set read and write the private store while every
// other [echo.Context] method delegates to the embedded context for
// anything the downstream might still need.
type poolSafeContext struct {
echo.Context
mu sync.RWMutex
store map[string]any
}
// newPoolSafeContext snapshots the given keys from c into a detached
// store and returns a wrapper whose Get/Set operate on that store
// exclusively. Keys absent from c are omitted; the wrapper still
// Echo returns an [echo.Context] to its sync.Pool as soon as the synchronous
// handler returns, including when the webhook middleware returns
// [api.ErrAsyncProcess]. A concurrent request can then claim the recycled
// context and c.Reset() wipes the shared store out from under the webhook
// goroutine, which causes any `c.Get("logger").(*slog.Logger)`-style assertion
// further down the chain to panic on a nil value.
//
// [echo.NewContext] allocates outside the pool, so recycling cannot reach the
// returned context. Keys absent from c are omitted; the returned context still
// returns nil for them, matching [echo.Context.Get] behavior.
func newPoolSafeContext(c echo.Context, keys ...string) *poolSafeContext {
store := make(map[string]any, len(keys))
//
// Only the asynchronous path uses this. Nothing downstream of the webhook
// middleware writes to the response: contextMiddleware sits upstream and has
// already answered 204 by the time the goroutine runs.
func newDetachedContext(c *echo.Context, keys ...string) *echo.Context {
detached := echo.NewContext(c.Request(), c.Response(), c.Echo())
for _, key := range keys {
if v := c.Get(key); v != nil {
store[key] = v
detached.Set(key, v)
}
}
return &poolSafeContext{Context: c, store: store}
}
// Get returns the value stored in the detached store, not the embedded
// context's pooled store.
func (p *poolSafeContext) Get(key string) any {
p.mu.RLock()
defer p.mu.RUnlock()
return p.store[key]
}
// Set writes to the detached store, not the embedded context's pooled
// store. This prevents downstream middleware writes from leaking into a
// later request that claims the same pooled context.
func (p *poolSafeContext) Set(key string, val any) {
p.mu.Lock()
defer p.mu.Unlock()
p.store[key] = val
return detached
}

View File

@@ -6,10 +6,10 @@ import (
"net/http/httptest"
"testing"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
)
func TestPoolSafeContext_SurvivesUnderlyingReset(t *testing.T) {
func TestNewDetachedContext_SurvivesUnderlyingReset(t *testing.T) {
e := echo.New()
req := httptest.NewRequest(http.MethodPost, "/", nil)
rec := httptest.NewRecorder()
@@ -19,10 +19,10 @@ func TestPoolSafeContext_SurvivesUnderlyingReset(t *testing.T) {
c.Set("logger", logger)
c.Set("correlationId", "abc-123")
detached := newPoolSafeContext(c, "logger", "correlationId", "missing")
detached := newDetachedContext(c, "logger", "correlationId", "missing")
// Simulate Echo recycling c for a concurrent request. Reset wipes the
// shared store, which is exactly the crash scenario the wrapper
// Simulate Echo recycling c for a concurrent request. Reset clears the
// pooled store, which is exactly the crash scenario the detached context
// guards against.
c.Reset(httptest.NewRequest(http.MethodGet, "/", nil), httptest.NewRecorder())
@@ -38,17 +38,17 @@ func TestPoolSafeContext_SurvivesUnderlyingReset(t *testing.T) {
// Underlying c must remain clean.
if c.Get("logger") != nil {
t.Fatalf("underlying c.Get(\"logger\") leaked wrapper state after reset")
t.Fatalf("underlying c.Get(\"logger\") leaked detached state after reset")
}
}
func TestPoolSafeContext_SetDoesNotTouchUnderlying(t *testing.T) {
func TestNewDetachedContext_SetDoesNotTouchUnderlying(t *testing.T) {
e := echo.New()
req := httptest.NewRequest(http.MethodPost, "/", nil)
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
detached := newPoolSafeContext(c)
detached := newDetachedContext(c)
detached.Set("foo", "bar")
if got, _ := detached.Get("foo").(string); got != "bar" {

View File

@@ -5,7 +5,7 @@ import (
"sync/atomic"
"time"
"github.com/dlclark/regexp2"
"github.com/dlclark/regexp2/v2"
flag "github.com/spf13/pflag"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"

View File

@@ -25,7 +25,7 @@ Available tags:
| Group | Tags |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Chromium | `chromium`, `chromium-concurrent`, `chromium-convert-html`, `chromium-convert-markdown`, `chromium-convert-url`, `chromium-screenshot-html`, `chromium-screenshot-markdown`, `chromium-screenshot-url`, `chromium-ssrf` |
| Chromium | `chromium`, `chromium-concurrent`, `chromium-convert-html`, `chromium-image-deduplication`, `chromium-convert-markdown`, `chromium-convert-url`, `chromium-screenshot-html`, `chromium-screenshot-markdown`, `chromium-screenshot-url`, `chromium-ssrf` |
| LibreOffice | `libreoffice`, `libreoffice-convert`, `libreoffice-ssrf` |
| PDF Engines | `pdfengines`, `pdfengines-convert`, `pdfengines-merge`, `merge`, `pdfengines-split`, `split`, `pdfengines-flatten`, `flatten`, `pdfengines-optimize`, `optimize`, `pdfengines-rotate`, `rotate`, `pdfengines-embed`, `embed`, `pdfengines-encrypt`, `encrypt`, `pdfengines-watermark`, `watermark`, `pdfengines-stamp`, `stamp`, `pdfengines-metadata`, `metadata`, `pdfengines-bookmarks`, `bookmarks` |
| Infra | `health`, `debug`, `root`, `version`, `output-filename`, `prometheus-metrics`, `webhook`, `download-from` |
@@ -75,6 +75,7 @@ Available tags:
- `the (response|webhook request) PDF(s) (should|should NOT) allow "<action>"` (actions: `printing`, `copying`, `modifying`, `annotating`)
- `the (response|webhook request) PDF(s) (should|should NOT) have the "<filename>" file embedded`
- `the "<name>" PDF should have <N> image(s)`
- `the "<name>" PDF should reuse <N> image XObject(s) across <P> page(s)` (checks image appearances on every page and distinct PDF object IDs)
- `the Gotenberg container (should|should NOT) log the following entries:` (table of log substrings)
- `all concurrent response status codes should be <code>`
- `all concurrent responses should have <N> PDF(s)`

View File

@@ -0,0 +1,57 @@
@chromium
@chromium-convert-html
@chromium-image-deduplication
Feature: Chromium header and footer image deduplication
# See: https://github.com/gotenberg/gotenberg/issues/1077.
# See: https://github.com/gotenberg/gotenberg/issues/1669.
# The PNG must appear on every page, but its image XObject must be shared.
Scenario: POST /forms/chromium/convert/html (Header image reused across pages)
Given I have a default Gotenberg container
When I make a "POST" request to Gotenberg at the "/forms/chromium/convert/html" endpoint with the following form data and header(s):
| files | testdata/page-1-html/index.html | file |
| files | testdata/header-footer-image/header.html | file |
| marginTop | 1 | field |
| marginBottom | 1 | field |
| Gotenberg-Output-Filename | header | header |
Then the response status code should be 200
Then the response header "Content-Type" should be "application/pdf"
Then there should be 1 PDF(s) in the response
Then the "header.pdf" PDF should have 1 page(s)
Then the "header.pdf" PDF should reuse 1 image XObject(s) across 1 page(s)
When I make a "POST" request to Gotenberg at the "/forms/chromium/convert/html" endpoint with the following form data and header(s):
| files | testdata/pages-3-html/index.html | file |
| files | testdata/header-footer-image/header.html | file |
| marginTop | 1 | field |
| marginBottom | 1 | field |
| Gotenberg-Output-Filename | header | header |
Then the response status code should be 200
Then the response header "Content-Type" should be "application/pdf"
Then there should be 1 PDF(s) in the response
Then the "header.pdf" PDF should have 3 page(s)
Then the "header.pdf" PDF should reuse 1 image XObject(s) across 3 page(s)
Scenario: POST /forms/chromium/convert/html (Footer image reused across pages)
Given I have a default Gotenberg container
When I make a "POST" request to Gotenberg at the "/forms/chromium/convert/html" endpoint with the following form data and header(s):
| files | testdata/page-1-html/index.html | file |
| files | testdata/header-footer-image/footer.html | file |
| marginTop | 1 | field |
| marginBottom | 1 | field |
| Gotenberg-Output-Filename | footer | header |
Then the response status code should be 200
Then the response header "Content-Type" should be "application/pdf"
Then there should be 1 PDF(s) in the response
Then the "footer.pdf" PDF should have 1 page(s)
Then the "footer.pdf" PDF should reuse 1 image XObject(s) across 1 page(s)
When I make a "POST" request to Gotenberg at the "/forms/chromium/convert/html" endpoint with the following form data and header(s):
| files | testdata/pages-3-html/index.html | file |
| files | testdata/header-footer-image/footer.html | file |
| marginTop | 1 | field |
| marginBottom | 1 | field |
| Gotenberg-Output-Filename | footer | header |
Then the response status code should be 200
Then the response header "Content-Type" should be "application/pdf"
Then there should be 1 PDF(s) in the response
Then the "footer.pdf" PDF should have 3 page(s)
Then the "footer.pdf" PDF should reuse 1 image XObject(s) across 3 page(s)

View File

@@ -11,7 +11,6 @@ import (
"github.com/moby/moby/client"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/exec"
"github.com/testcontainers/testcontainers-go/network"
"github.com/testcontainers/testcontainers-go/wait"
)
@@ -21,11 +20,13 @@ import (
const testcontainersLabel = "org.testcontainers"
// PruneOrphanedNetworks removes dangling networks created by the test suite.
// Each scenario spins a dedicated network, and a failed container start can
// leak one before teardown records it. Leaked networks consume Docker's
// predefined address pools until none remain and every later scenario fails
// with "all predefined address pools have been fully subnetted". Call this
// before a run and between retries to reclaim the subnets.
// Scenarios no longer create one: the Gotenberg container is reached over its
// mapped port and the host-side helper over host.docker.internal, so the
// default bridge suffices. This stays as cheap insurance against networks
// leaked by an older suite version or an interrupted run, which consume
// Docker's predefined address pools until none remain and every later
// scenario fails with "all predefined address pools have been fully
// subnetted".
//
// Only unused networks bearing the testcontainers label are removed, so
// running containers and operator networks are never affected.
@@ -99,17 +100,18 @@ func applyDefaultEnv(env map[string]string) map[string]string {
return env
}
func startGotenbergContainer(ctx context.Context, env map[string]string) (*testcontainers.DockerNetwork, testcontainers.Container, error) {
// startGotenbergContainer starts a Gotenberg container on Docker's default
// bridge. No dedicated network is created: the suite addresses the container
// through container.Host plus its mapped port, and the container reaches the
// host-side webhook and static file server through the host.docker.internal
// alias below, so a per-scenario network would carry no traffic while still
// consuming one of Docker's predefined subnets.
func startGotenbergContainer(ctx context.Context, env map[string]string) (testcontainers.Container, error) {
ctx, cancel := context.WithTimeout(ctx, 2*time.Minute)
defer cancel()
env = applyDefaultEnv(env)
n, err := network.New(ctx)
if err != nil {
return nil, nil, fmt.Errorf("create Gotenberg container network: %w", err)
}
healthPath := "/health"
if env["API_ROOT_PATH"] != "" {
healthPath = fmt.Sprintf("%shealth", env["API_ROOT_PATH"])
@@ -122,7 +124,6 @@ func startGotenbergContainer(ctx context.Context, env map[string]string) (*testc
HostConfigModifier: func(hostConfig *container.HostConfig) {
hostConfig.ExtraHosts = []string{"host.docker.internal:host-gateway"}
},
Networks: []string{n.Name},
WaitingFor: wait.ForHTTP(healthPath),
Env: env,
}
@@ -148,19 +149,10 @@ func startGotenbergContainer(ctx context.Context, env map[string]string) (*testc
}
}
// The network is already created. The scenario teardown only
// removes networks it knows about, and the caller discards n on
// error, so remove it here to avoid leaking a subnet on every
// failed start. Leaked networks accumulate until Docker's address
// pools are fully subnetted and all later scenarios fail.
if errRemove := n.Remove(ctx); errRemove != nil {
err = fmt.Errorf("%w (also failed to remove network: %v)", err, errRemove)
}
return nil, nil, err
return nil, err
}
return n, c, nil
return c, nil
}
func execCommandInIntegrationToolsContainer(ctx context.Context, cmd []string, path string) (string, error) {

View File

@@ -0,0 +1,72 @@
package scenario
import (
"context"
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
)
func (s *scenario) thePdfShouldReuseImageXObjects(ctx context.Context, name string, images, pages int) error {
path := filepath.Join(s.workdir, s.resp.Header().Get("Gotenberg-Trace"), name)
if _, err := os.Stat(path); err != nil {
return fmt.Errorf("stat PDF %q: %w", path, err)
}
cmd := []string{"pdfimages", "-list", filepath.Base(path)}
output, err := execCommandInIntegrationToolsContainer(ctx, cmd, path)
if err != nil {
return fmt.Errorf("exec %q: %w", cmd, err)
}
return checkImageXObjectReuse(output, images, pages)
}
func checkImageXObjectReuse(output string, images, pages int) error {
lines := strings.Split(strings.TrimSpace(output), "\n")
if len(lines) < 2 || !strings.HasPrefix(strings.TrimSpace(lines[0]), "page") {
return fmt.Errorf("invalid pdfimages -list output: %q", output)
}
objects := make(map[[2]int]struct{})
imagesPerPage := make(map[int]int)
for _, line := range lines[2:] {
fields := strings.Fields(line)
if len(fields) < 12 {
return fmt.Errorf("invalid pdfimages -list row: %q", line)
}
page, err := strconv.Atoi(fields[0])
if err != nil {
return fmt.Errorf("parse image page %q: %w", fields[0], err)
}
if page < 1 || page > pages {
return fmt.Errorf("image on page %d outside expected range 1-%d", page, pages)
}
// pdfimages lists appearances, including repeated references to the
// same image. Columns 11 and 12 identify the underlying PDF object.
var ref [2]int
for i := range ref {
ref[i], err = strconv.Atoi(fields[10+i])
if err != nil {
return fmt.Errorf("parse image object ID in %q: %w", line, err)
}
}
objects[ref] = struct{}{}
imagesPerPage[page]++
}
for page := 1; page <= pages; page++ {
if imagesPerPage[page] != images {
return fmt.Errorf("expected %d image appearance(s) on page %d, but actual is %d", images, page, imagesPerPage[page])
}
}
if len(objects) != images {
return fmt.Errorf("expected %d unique image XObject(s) shared across %d page(s), but actual is %d", images, pages, len(objects))
}
return nil
}

View File

@@ -0,0 +1,106 @@
package scenario
import (
"fmt"
"strings"
"testing"
)
func TestCheckImageXObjectReuse(t *testing.T) {
t.Parallel()
const header = `page num type width height color comp bpc enc interp object ID x-ppi y-ppi size ratio
--------------------------------------------------------------------------------------------
`
imageRow := func(page, object, generation int) string {
return fmt.Sprintf("%d %d image 32 32 rgb 3 8 image no %d %d 96 96 3104B 101%%\n", page, page-1, object, generation)
}
tests := []struct {
name string
output string
pages int
wantErr string
}{
{
name: "one page baseline",
output: header + imageRow(1, 7, 0),
pages: 1,
},
{
name: "three pages share one image object",
output: header + imageRow(1, 7, 0) + imageRow(2, 7, 0) + imageRow(3, 7, 0),
pages: 3,
},
{
name: "regression embeds a different object per page",
output: header + imageRow(1, 7, 0) + imageRow(2, 8, 0) + imageRow(3, 9, 0),
pages: 3,
wantErr: "expected 1 unique image XObject(s) shared across 3 page(s), but actual is 3",
},
{
name: "object generation is part of its identity",
output: header + imageRow(1, 7, 0) + imageRow(2, 7, 1) + imageRow(3, 7, 0),
pages: 3,
wantErr: "expected 1 unique image XObject(s) shared across 3 page(s), but actual is 2",
},
{
name: "image missing from a page",
output: header + imageRow(1, 7, 0) + imageRow(3, 7, 0),
pages: 3,
wantErr: "expected 1 image appearance(s) on page 2, but actual is 0",
},
{
name: "no image cannot pass",
output: header,
pages: 1,
wantErr: "expected 1 image appearance(s) on page 1, but actual is 0",
},
{
name: "repeated image only on first page cannot pass",
output: header + imageRow(1, 7, 0) + imageRow(1, 7, 0) + imageRow(1, 7, 0),
pages: 3,
wantErr: "expected 1 image appearance(s) on page 1, but actual is 3",
},
{
name: "image on unexpected page",
output: header + imageRow(1, 7, 0) + imageRow(2, 7, 0),
pages: 1,
wantErr: "image on page 2 outside expected range 1-1",
},
{
name: "tool error cannot pass",
output: "Syntax Error: Couldn't find trailer dictionary",
pages: 1,
wantErr: "invalid pdfimages -list output:",
},
{
name: "malformed image row",
output: header + "1 0 image\n",
pages: 1,
wantErr: "invalid pdfimages -list row:",
},
{
name: "inline image has no reusable object ID",
output: header + "1 0 image 32 32 rgb 3 8 image no [inline] - 96 96 3104B 101%\n",
pages: 1,
wantErr: "parse image object ID",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
err := checkImageXObjectReuse(tt.output, 1, tt.pages)
if tt.wantErr == "" {
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
return
}
if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
t.Fatalf("expected error containing %q, got %v", tt.wantErr, err)
}
})
}
}

View File

@@ -86,16 +86,15 @@ func findScenarioLine(filePath, name string) int {
}
type scenario struct {
resp *httptest.ResponseRecorder
concurrentResps []*httptest.ResponseRecorder
probeResps []*httptest.ResponseRecorder
sequentialResps []*httptest.ResponseRecorder
workdir string
teststoreDir string
gotenbergContainer testcontainers.Container
gotenbergContainerNetwork *testcontainers.DockerNetwork
server *server
hostPort int
resp *httptest.ResponseRecorder
concurrentResps []*httptest.ResponseRecorder
probeResps []*httptest.ResponseRecorder
sequentialResps []*httptest.ResponseRecorder
workdir string
teststoreDir string
gotenbergContainer testcontainers.Container
server *server
hostPort int
}
func (s *scenario) reset(ctx context.Context) error {
@@ -123,11 +122,10 @@ func (s *scenario) reset(ctx context.Context) error {
}
func (s *scenario) iHaveADefaultGotenbergContainer(ctx context.Context) error {
n, c, err := startGotenbergContainer(ctx, nil)
c, err := startGotenbergContainer(ctx, nil)
if err != nil {
return fmt.Errorf("create Gotenberg container: %s", err)
}
s.gotenbergContainerNetwork = n
s.gotenbergContainer = c
return nil
}
@@ -137,11 +135,10 @@ func (s *scenario) iHaveAGotenbergContainerWithTheFollowingEnvironmentVariables(
for _, row := range envTable.Rows {
env[row.Cells[0].Value] = row.Cells[1].Value
}
n, c, err := startGotenbergContainer(ctx, env)
c, err := startGotenbergContainer(ctx, env)
if err != nil {
return fmt.Errorf("create Gotenberg container: %s", err)
}
s.gotenbergContainerNetwork = n
s.gotenbergContainer = c
return nil
}
@@ -1802,6 +1799,7 @@ func InitializeScenario(ctx *godog.ScenarioContext) {
ctx.Then(`^the "([^"]*)" PDF (should|should NOT) have the following content at page (\d+):$`, s.thePdfShouldHaveTheFollowingContentAtPage)
ctx.Then(`^the "([^"]*)" PDF (should|should NOT) have content matching "([^"]*)" at page (\d+)$`, s.thePdfShouldHaveContentMatchingAtPage)
ctx.Then(`^the "([^"]*)" PDF should have (\d+) image\(s\)$`, s.thePdfShouldHaveImages)
ctx.Then(`^the "([^"]*)" PDF should reuse (\d+) image XObject\(s\) across (\d+) page\(s\)$`, s.thePdfShouldReuseImageXObjects)
ctx.Then(`^the "([^"]*)" image should be (\d+)x(\d+) pixels$`, s.theImageShouldBePixels)
ctx.Then(`^the "([^"]*)" image pixel at (\d+),(\d+) should be "([^"]*)"$`, s.theImagePixelShouldBe)
ctx.Then(`^the "([^"]*)" file size should be (less|greater) than (\d+) KB$`, s.theFileSizeShouldBe)
@@ -1812,12 +1810,6 @@ func InitializeScenario(ctx *godog.ScenarioContext) {
return ctx, fmt.Errorf("terminate Gotenberg container: %w", errTerminate)
}
}
if s.gotenbergContainerNetwork != nil {
errRemove := s.gotenbergContainerNetwork.Remove(ctx)
if errRemove != nil {
return ctx, fmt.Errorf("remove Gotenberg container network: %w", errRemove)
}
}
return ctx, nil
})
ctx.After(func(ctx context.Context, sc *godog.Scenario, err error) (context.Context, error) {

View File

@@ -15,7 +15,7 @@ import (
"github.com/cucumber/godog"
"github.com/google/uuid"
"github.com/labstack/echo/v4"
"github.com/labstack/echo/v5"
"github.com/mholt/archives"
)
@@ -26,12 +26,17 @@ type server struct {
errChan chan error
eventBody []byte
eventMu sync.Mutex
shutdown context.CancelFunc
done chan struct{}
}
func newServer(ctx context.Context, workdir string) (*server, error) {
srv := echo.New()
srv.HideBanner = true
srv.HidePort = true
// The static file handlers below serve absolute paths, which Echo v5's
// default working-directory filesystem rejects. See newEchoServer in
// pkg/modules/api.
srv.Filesystem = echo.NewDefaultFS("/")
s := &server{
srv: srv,
errChan: make(chan error, 1),
@@ -47,7 +52,7 @@ func newServer(ctx context.Context, workdir string) (*server, error) {
return err
}
webhookHandler := func(c echo.Context) error {
webhookHandler := func(c *echo.Context) error {
s.req = c.Request()
body, err := io.ReadAll(s.req.Body)
@@ -131,7 +136,7 @@ func newServer(ctx context.Context, workdir string) (*server, error) {
return webhookErr(c.String(http.StatusOK, http.StatusText(http.StatusOK)))
}
webhookErrorHandler := func(c echo.Context) error {
webhookErrorHandler := func(c *echo.Context) error {
s.req = c.Request()
body, err := io.ReadAll(s.req.Body)
if err != nil {
@@ -148,7 +153,7 @@ func newServer(ctx context.Context, workdir string) (*server, error) {
srv.PATCH("/webhook/error", webhookErrorHandler)
srv.PUT("/webhook/error", webhookErrorHandler)
webhookEventsHandler := func(c echo.Context) error {
webhookEventsHandler := func(c *echo.Context) error {
body, err := io.ReadAll(c.Request().Body)
if err != nil {
return c.String(http.StatusInternalServerError, err.Error())
@@ -159,7 +164,7 @@ func newServer(ctx context.Context, workdir string) (*server, error) {
return c.String(http.StatusOK, http.StatusText(http.StatusOK))
}
srv.POST("/webhook/events", webhookEventsHandler)
srv.GET("/static/:path", func(c echo.Context) error {
srv.GET("/static/:path", func(c *echo.Context) error {
s.req = c.Request()
path := c.Param("path")
if strings.Contains(path, "teststore") {
@@ -167,7 +172,7 @@ func newServer(ctx context.Context, workdir string) (*server, error) {
}
return c.Attachment(fmt.Sprintf("%s/%s", wd, path), filepath.Base(path))
})
srv.GET("/html/:path", func(c echo.Context) error {
srv.GET("/html/:path", func(c *echo.Context) error {
s.req = c.Request()
path := fmt.Sprintf("%s/%s", wd, c.Param("path"))
f, err := os.Open(path)
@@ -181,7 +186,7 @@ func newServer(ctx context.Context, workdir string) (*server, error) {
}
return c.HTML(http.StatusOK, string(b))
})
srv.GET("/redirect-to-private", func(c echo.Context) error {
srv.GET("/redirect-to-private", func(c *echo.Context) error {
s.req = c.Request()
// Redirect the browser to a non-public address so the outbound filter
// is exercised on the redirected request rather than on this URL.
@@ -206,9 +211,20 @@ func (s *server) start(ctx context.Context) (int, error) {
port := ln.Addr().(*net.TCPAddr).Port
startConfig := echo.StartConfig{
Listener: ln,
HideBanner: true,
HidePort: true,
}
serveCtx, cancel := context.WithCancel(context.Background())
s.shutdown = cancel
s.done = make(chan struct{})
go func() {
s.srv.Listener = ln
err = s.srv.Start("")
defer close(s.done)
err := startConfig.Start(serveCtx, s.srv)
if err != nil && !errors.Is(err, http.ErrServerClosed) {
godog.Log(ctx, err.Error())
}
@@ -219,5 +235,17 @@ func (s *server) start(ctx context.Context) (int, error) {
func (s *server) stop(ctx context.Context) error {
close(s.errChan)
return s.srv.Shutdown(ctx)
if s.shutdown == nil {
return nil
}
s.shutdown()
select {
case <-s.done:
return nil
case <-ctx.Done():
return ctx.Err()
}
}

View File

@@ -0,0 +1,15 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
</head>
<body style="margin: 0; width: 100%">
<!-- An opaque RGB PNG avoids a separate alpha-mask image XObject. -->
<img
alt="Regression test logo"
width="32"
height="32"
src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAMK0lEQVR4nAEgDN/zAHFeV9yZjwo6SF080rU79JVR0qeuIuYqIIkxwf4rwT7KASBVtxNyEiAxJ4grbwObwdwYZ6P0T7ZyaHzTFJ005RAb+dDFcre2+xqwdWMu6K2qYgD5LRL11Gbts9TyrnGOXwDlHG5y4w4MwrW50xeDinDuoJTl8/S8A8ZLkQMA1u8ezIEjb2uuymxnltx/FBavvETNXZziNgILdKHlL8eh4arzO57gogg1pmz87SbQ68LuEWliRgfzVbw5zKP0DV+iMwMA/Uo3EaY4a9y1Gb5+wwZvfwZoLxNRqlG/ihDl7ZuePDIBwSIq9uDtvTmlFmt51h+uwN/P0/JA8TD79MeLDcLyfWrOiHYWgmZfIIRQ55No5JDAhmwwdhdSUULvyfYHfRYUAA87FNMkQJMhwONa9zyeAZpvUbc3YYjl7PXdmWt9TjIOR9KLlr/oYX4v0p2gwqBEpFuU8rztON0/kzogZ6AyV/Ii59Y1U1XmBQOZc1ZuB0GZdNFLd7YEl5xZ0RoQBng9MACeb3oQG5av5e+OeVd0EBSwTVjj2LIeVDPjC6K3pzm71SWZlQnzGva6q8yRXNgnwCWqhw7il9UPqutAY92KZbAAWXRPSghQMcfJXKhMhTk9IKc00BUIwWMrAM/Iiag0BUsAJ28WinyHUJ+Z7i6kY3mRkpTElQVtxb9PkzzJEkiDNBM2zvI213tqQbms1p3uDAJl7ZhfxXdzN0GUN35tW41YJHqKAx+goQH2joWt3KjyZ1EU0MjktAvsQPLI4aMGNLLLAHjgm2bT2agbLgwuHy/s8zu4MsitoBUFrkFaapQimZrqQe7rk2mE1CE+m7nR/T0TIqzLZ8FfY/kw08XIks4pw63W2xPi039Ixpm4GWd9Dw/XvkgqeKvWtIm+SrK+/hVfEwAwAx4vZY4b9ludgZxTSo3GCP+eJ56dnvDxstPd/iRjbWOLlzMO9RIyHQsQiR6GyAHaYMd10nVDTURBLBRh/vOIVgqW/y5sFLWAT/JwsYFrKSqq+aFD5Qi1vzVrNPbFSRYAeUpOpmKQMUpXRGFn2121iPDhkQtxMuhgIV3GLdj7N2D8jvFitz1ArnJAmbaLXAhUsqvaFVW8uQIl9/RnuS5/nTNZVyzGPpn6/DdR+ilEVioeoJbnMc2dsaHMV3dMX6o6AMtgzHv6tqdLzj1e1JNd6NO7kLp3hJgMZdA6QCjZZTta7F6kUkn+50IrgZKvYSV/OAvJlp5R0hYnIZxYT/GjnG1NX46skoyZYLVEaAbta6Ilt0hT2tJm/k+OW20J7nHXbgCgdIDlXjHrwu0j7Lhb9YlCw3+FY4FZax6fq8ito7hwB1P1HRTXppGkpRcKrNf/RG/98IblVELbH5AvahR5o+fsAVkAMVF2ikK3Jb9QcWdyZuz9+adExsJl2ThBU3fMf5AAQnkPbPrOi7S/aiHfuFP/elFTtZCb9d0XGvEe2KW608dbRIitQrF7CG75lg7kTtuo5MKyBjkdG4ZhtXvAVCbP46MkxLTp/IJacHf9n876CzrC5n09UzQ/WiLD0KmtWyHYAJZArRFStklX9WzAcLi416r/QDof5aShd4zM0K0WwmQFr35OgAmN78LDxglaRbQLAbG8YWb45XwrVfXF/8jCS/wCfaID5TutbSLRVUFPchzq+NeQP0RXVHipz7Bl4b59xwAXI/gQ3FN+tlw/+vQpGgfeMX0QBbI3Q6mu8GaiDZyBoSlDa9jNKmdjM+XcpJAnNLIHXWZ2ZvHZkbOHWw4rer7JWffwPYhKlbbLiCpEBTRWquJRfFL2OHYtKYYpEdd0mJgAkwRzNP3YQg99kB1QNJnQfaejmXSrokCxVMbnbmdW4T2gCN8JjGxUO+ZT1MaSNTpzviDfWBvr44z5bBL5PFPsjNx8AEEDYq91v5qew+43n8sat3dEytOM6aZmQnxjBmYtAEqN9xFdkB1/z0qCCA3fuCN44ERMy7Jow6p0wotUsMJg96xE6bult59nhteUqGUEgtj0X1z/xhEx9WII6tuUJIW6BeTT3FfEegZbXzKBjkZi9BqeCmVCOIzTfTrCJxDLHQBEMri8mSZWdzrQoZTLSjEtRu4WJpIFcMnd/acdues9mbUB7qcfmZURJiPP6SD9byHKkfM+Kp6jCbicn0+Pmyzd9Rpzui25NaoJ9o7hetV4wOB803SYynbu1R8TjRovHNMAqfNrqBjFCS2++XVHB367TUdL1rnyL1Gl3LIZ7ypYhsESIPv/eHkDoi47rtoCuohtsy2QHcmmGqB2GaJYhyyPdVUWYDJvOAH59qRjNMPMkUCGsTBurwug/aTzaF5AmdVmAH3e7o+C5ANUCCxTpHMory8yI8BtJoeKbZbwsYF4RuhzHnlFrRiixiCoGp0HpWwaTkasHjJT2JfVFLtEOqauCQlKYWZgxN+o5lyG7GPt6B1sRfK9BcBesrMPbMuBFjGgbADdq1xYYWJre/Iku1F3q7S3X9CIbwQ5HndUQcTZqotqll9orYPbstFytlAHbl+hCAyVvmtbFHi5HCkKSOtdy3I2we8ziuW+w2keW0cxb3hTvW3LDl6+KrxYfvfp1a6rr3oAZhd+aTUupdRJnkMTiLM0YARjYKkxsvIeJ+rXscGEa9+uKzUjHfCM8MCRqJuDMLJOYQZJMOy8/gESh/r2df6zjwu0rOPK2g3mQN24HQpf4NHA3Js0eJgW8x6H21nn3xbWACB/7AHIBaIXx29qFx9Vc+2Mn1EjMpupjzL5gdKvFACuszoKYQFGLonvisV1+rSGhvCBsWZjL+zNfEbhSmD5RgBdVMsAzZUGCm+Hc2jHk0sasbmh9jb4YX9Od4iy7BjjQwCXObO/HyL9I3eokGoAXQL7Cv5oflCMnzOw0kg0JhPvZdAfAqUNGba4sTAWQ79aNhmXYON7wb6yr+47G9gR6pLgdeWwLZO0fuFUy8G33hlagsilZKpgZZSs2ZKcozp0UC8AeMqdXC2TLaMHmdqzCZ/677p9TlCINu86p7LB8cmH8jzFEnirZmg/t8GClPin9DkttqoLW1mb8xsBy5/Mkv5+E06p+YUsrpoFU971XdOr6hlft10OqQmZcHpGBUcLZqjdAJR4uCC1SWcHO5/yT00LTyKtib1NDjIw3TgVlTt48gkn/Ssl7ScPCch3Wd4JYVIfy8FBCVPAklz6FZqdn7WbsVsgRIm0h2V6952GTBWtSOFwBsCoikWEO7uqRc2czrF0KQAKtLj6OhcofSTdbVMjSc7+H1OoZYl0Qaf4qFc1ElGZTjFS3JQ9dHuTc927mzDXYbCFmE5e8iMQ30pDlO2xUKAvXw3XceMY6e6708a/Amumxlppr1OMH2nQrLTPV1RuQ3YAtV7Nz6wAZf53b3umPpauivw+BRxmrGdw4xxx8IFZ2Xsg0bjjOq4uF2ce3qRhaEEibWNJlq8LiykF7/jMfNbC75T20pEBauCau7zJ0yLZABmlLDoLEznqnNtdUa4c8/VEANZCgXnsKm56WwPCP2QnMyKSSzgNg8wFISFXn5I3aUEs8Qgzo8lwzeHGwL3NDaX4yhbqHjHk4TRZWReQqpOFZ8xEtkNVYyuqfWwXmyoBO0rSKNU/36G+/LL7vkdRETQjfgBD64rnfYLXwSR0MEEJL9WQQImW58JbcDFpoiZ2TTCpzj3iX9p2X9fH2a5gHqjIyuVuHu+BXRU3bDZGdBzEBlMHk9n7zG+95PNXWO23ri3d0rwmL1IWrGfJBkNOIs7dkwsAjgR1a7R+EpPEPm0lTkcKEzLYCpHRN3jKkNJrqahi0nM7nOXLy+IKIp6lywIbwmP4vgOYS/kkEDlT6Rf3AN77C8B7RubVMiHcS54ZlAgVxLN9GCsFuUjX1ixxO3SqhEE5AKTJGK+E6+eWiOunUeL+xbQH+/ai6TEuNd0/CF80seyg32ozQvPiVytrPFPwWW5Zm0BejcYhdrD5LP/VfZoxnpTSsYd2JE84ox8M8XjXhhn1ijeGXTCIX9aG7IMfWfZQgQAbd1xNL6atgyHE6lwQwB2rZ42OEk7tkkRjr9DiWKMQmDOKb2WYoW9vi6AEenhJXcgT6Lr/h6PFXyMtF/aCej/FJq4i9/U+F8W5CVfmBAUdOk91ozn4HQFhLrSAGZrcaa5BZe5R1Of8LAAAAABJRU5ErkJggg=="
/>
</body>
</html>

View File

@@ -0,0 +1,15 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
</head>
<body style="margin: 0; width: 100%">
<!-- An opaque RGB PNG avoids a separate alpha-mask image XObject. -->
<img
alt="Regression test logo"
width="32"
height="32"
src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAMK0lEQVR4nAEgDN/zAHFeV9yZjwo6SF080rU79JVR0qeuIuYqIIkxwf4rwT7KASBVtxNyEiAxJ4grbwObwdwYZ6P0T7ZyaHzTFJ005RAb+dDFcre2+xqwdWMu6K2qYgD5LRL11Gbts9TyrnGOXwDlHG5y4w4MwrW50xeDinDuoJTl8/S8A8ZLkQMA1u8ezIEjb2uuymxnltx/FBavvETNXZziNgILdKHlL8eh4arzO57gogg1pmz87SbQ68LuEWliRgfzVbw5zKP0DV+iMwMA/Uo3EaY4a9y1Gb5+wwZvfwZoLxNRqlG/ihDl7ZuePDIBwSIq9uDtvTmlFmt51h+uwN/P0/JA8TD79MeLDcLyfWrOiHYWgmZfIIRQ55No5JDAhmwwdhdSUULvyfYHfRYUAA87FNMkQJMhwONa9zyeAZpvUbc3YYjl7PXdmWt9TjIOR9KLlr/oYX4v0p2gwqBEpFuU8rztON0/kzogZ6AyV/Ii59Y1U1XmBQOZc1ZuB0GZdNFLd7YEl5xZ0RoQBng9MACeb3oQG5av5e+OeVd0EBSwTVjj2LIeVDPjC6K3pzm71SWZlQnzGva6q8yRXNgnwCWqhw7il9UPqutAY92KZbAAWXRPSghQMcfJXKhMhTk9IKc00BUIwWMrAM/Iiag0BUsAJ28WinyHUJ+Z7i6kY3mRkpTElQVtxb9PkzzJEkiDNBM2zvI213tqQbms1p3uDAJl7ZhfxXdzN0GUN35tW41YJHqKAx+goQH2joWt3KjyZ1EU0MjktAvsQPLI4aMGNLLLAHjgm2bT2agbLgwuHy/s8zu4MsitoBUFrkFaapQimZrqQe7rk2mE1CE+m7nR/T0TIqzLZ8FfY/kw08XIks4pw63W2xPi039Ixpm4GWd9Dw/XvkgqeKvWtIm+SrK+/hVfEwAwAx4vZY4b9ludgZxTSo3GCP+eJ56dnvDxstPd/iRjbWOLlzMO9RIyHQsQiR6GyAHaYMd10nVDTURBLBRh/vOIVgqW/y5sFLWAT/JwsYFrKSqq+aFD5Qi1vzVrNPbFSRYAeUpOpmKQMUpXRGFn2121iPDhkQtxMuhgIV3GLdj7N2D8jvFitz1ArnJAmbaLXAhUsqvaFVW8uQIl9/RnuS5/nTNZVyzGPpn6/DdR+ilEVioeoJbnMc2dsaHMV3dMX6o6AMtgzHv6tqdLzj1e1JNd6NO7kLp3hJgMZdA6QCjZZTta7F6kUkn+50IrgZKvYSV/OAvJlp5R0hYnIZxYT/GjnG1NX46skoyZYLVEaAbta6Ilt0hT2tJm/k+OW20J7nHXbgCgdIDlXjHrwu0j7Lhb9YlCw3+FY4FZax6fq8ito7hwB1P1HRTXppGkpRcKrNf/RG/98IblVELbH5AvahR5o+fsAVkAMVF2ikK3Jb9QcWdyZuz9+adExsJl2ThBU3fMf5AAQnkPbPrOi7S/aiHfuFP/elFTtZCb9d0XGvEe2KW608dbRIitQrF7CG75lg7kTtuo5MKyBjkdG4ZhtXvAVCbP46MkxLTp/IJacHf9n876CzrC5n09UzQ/WiLD0KmtWyHYAJZArRFStklX9WzAcLi416r/QDof5aShd4zM0K0WwmQFr35OgAmN78LDxglaRbQLAbG8YWb45XwrVfXF/8jCS/wCfaID5TutbSLRVUFPchzq+NeQP0RXVHipz7Bl4b59xwAXI/gQ3FN+tlw/+vQpGgfeMX0QBbI3Q6mu8GaiDZyBoSlDa9jNKmdjM+XcpJAnNLIHXWZ2ZvHZkbOHWw4rer7JWffwPYhKlbbLiCpEBTRWquJRfFL2OHYtKYYpEdd0mJgAkwRzNP3YQg99kB1QNJnQfaejmXSrokCxVMbnbmdW4T2gCN8JjGxUO+ZT1MaSNTpzviDfWBvr44z5bBL5PFPsjNx8AEEDYq91v5qew+43n8sat3dEytOM6aZmQnxjBmYtAEqN9xFdkB1/z0qCCA3fuCN44ERMy7Jow6p0wotUsMJg96xE6bult59nhteUqGUEgtj0X1z/xhEx9WII6tuUJIW6BeTT3FfEegZbXzKBjkZi9BqeCmVCOIzTfTrCJxDLHQBEMri8mSZWdzrQoZTLSjEtRu4WJpIFcMnd/acdues9mbUB7qcfmZURJiPP6SD9byHKkfM+Kp6jCbicn0+Pmyzd9Rpzui25NaoJ9o7hetV4wOB803SYynbu1R8TjRovHNMAqfNrqBjFCS2++XVHB367TUdL1rnyL1Gl3LIZ7ypYhsESIPv/eHkDoi47rtoCuohtsy2QHcmmGqB2GaJYhyyPdVUWYDJvOAH59qRjNMPMkUCGsTBurwug/aTzaF5AmdVmAH3e7o+C5ANUCCxTpHMory8yI8BtJoeKbZbwsYF4RuhzHnlFrRiixiCoGp0HpWwaTkasHjJT2JfVFLtEOqauCQlKYWZgxN+o5lyG7GPt6B1sRfK9BcBesrMPbMuBFjGgbADdq1xYYWJre/Iku1F3q7S3X9CIbwQ5HndUQcTZqotqll9orYPbstFytlAHbl+hCAyVvmtbFHi5HCkKSOtdy3I2we8ziuW+w2keW0cxb3hTvW3LDl6+KrxYfvfp1a6rr3oAZhd+aTUupdRJnkMTiLM0YARjYKkxsvIeJ+rXscGEa9+uKzUjHfCM8MCRqJuDMLJOYQZJMOy8/gESh/r2df6zjwu0rOPK2g3mQN24HQpf4NHA3Js0eJgW8x6H21nn3xbWACB/7AHIBaIXx29qFx9Vc+2Mn1EjMpupjzL5gdKvFACuszoKYQFGLonvisV1+rSGhvCBsWZjL+zNfEbhSmD5RgBdVMsAzZUGCm+Hc2jHk0sasbmh9jb4YX9Od4iy7BjjQwCXObO/HyL9I3eokGoAXQL7Cv5oflCMnzOw0kg0JhPvZdAfAqUNGba4sTAWQ79aNhmXYON7wb6yr+47G9gR6pLgdeWwLZO0fuFUy8G33hlagsilZKpgZZSs2ZKcozp0UC8AeMqdXC2TLaMHmdqzCZ/677p9TlCINu86p7LB8cmH8jzFEnirZmg/t8GClPin9DkttqoLW1mb8xsBy5/Mkv5+E06p+YUsrpoFU971XdOr6hlft10OqQmZcHpGBUcLZqjdAJR4uCC1SWcHO5/yT00LTyKtib1NDjIw3TgVlTt48gkn/Ssl7ScPCch3Wd4JYVIfy8FBCVPAklz6FZqdn7WbsVsgRIm0h2V6952GTBWtSOFwBsCoikWEO7uqRc2czrF0KQAKtLj6OhcofSTdbVMjSc7+H1OoZYl0Qaf4qFc1ElGZTjFS3JQ9dHuTc927mzDXYbCFmE5e8iMQ30pDlO2xUKAvXw3XceMY6e6708a/Amumxlppr1OMH2nQrLTPV1RuQ3YAtV7Nz6wAZf53b3umPpauivw+BRxmrGdw4xxx8IFZ2Xsg0bjjOq4uF2ce3qRhaEEibWNJlq8LiykF7/jMfNbC75T20pEBauCau7zJ0yLZABmlLDoLEznqnNtdUa4c8/VEANZCgXnsKm56WwPCP2QnMyKSSzgNg8wFISFXn5I3aUEs8Qgzo8lwzeHGwL3NDaX4yhbqHjHk4TRZWReQqpOFZ8xEtkNVYyuqfWwXmyoBO0rSKNU/36G+/LL7vkdRETQjfgBD64rnfYLXwSR0MEEJL9WQQImW58JbcDFpoiZ2TTCpzj3iX9p2X9fH2a5gHqjIyuVuHu+BXRU3bDZGdBzEBlMHk9n7zG+95PNXWO23ri3d0rwmL1IWrGfJBkNOIs7dkwsAjgR1a7R+EpPEPm0lTkcKEzLYCpHRN3jKkNJrqahi0nM7nOXLy+IKIp6lywIbwmP4vgOYS/kkEDlT6Rf3AN77C8B7RubVMiHcS54ZlAgVxLN9GCsFuUjX1ixxO3SqhEE5AKTJGK+E6+eWiOunUeL+xbQH+/ai6TEuNd0/CF80seyg32ozQvPiVytrPFPwWW5Zm0BejcYhdrD5LP/VfZoxnpTSsYd2JE84ox8M8XjXhhn1ijeGXTCIX9aG7IMfWfZQgQAbd1xNL6atgyHE6lwQwB2rZ42OEk7tkkRjr9DiWKMQmDOKb2WYoW9vi6AEenhJXcgT6Lr/h6PFXyMtF/aCej/FJq4i9/U+F8W5CVfmBAUdOk91ozn4HQFhLrSAGZrcaa5BZe5R1Of8LAAAAABJRU5ErkJggg=="
/>
</body>
</html>