Compare commits

...

59 Commits

Author SHA1 Message Date
Julien Neuhart
a780373cfc chore(go): update to 1.22 2024-03-15 16:23:27 +01:00
Julien Neuhart
5ad90ccdb8 fix: file rename now correctly works for zip archive 2024-03-15 16:12:18 +01:00
Julien Neuhart
ef97d81ca8 fix(libreoffice): wrong number of file extensions 2024-03-15 11:37:34 +01:00
Julien Neuhart
c9f7b642c5 fix(api): wrong expected message in tests 2024-03-15 11:25:55 +01:00
Julien Neuhart
770962be3f chore(deps): update golangci-lint 2024-03-15 11:22:32 +01:00
Julien Neuhart
d6c857f4a8 chore(libreoffice): disable PDF tagging by default - for consistency with Chromium 2024-03-15 11:11:51 +01:00
Julien Neuhart
7b7ffb427d fix(chromium): disable PDF tagging 2024-03-15 10:56:35 +01:00
Julien Neuhart
b80318d1b6 chore(deps): update Go dependencies 2024-03-15 09:50:19 +01:00
dependabot[bot]
3997d69e49 chore(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0
Bumps google.golang.org/protobuf from 1.32.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-14 07:10:17 +01:00
Julien Neuhart
4af41b33c0 chore: update Go dependencies 2024-03-04 11:33:10 +01:00
Julien Neuhart
0f8bbead66 fix(api): do not catch gotenberg.ErrPdfEngineMethodNotSupported as it may hides other errors 2024-03-04 11:17:36 +01:00
Julien Neuhart
804fcff5b6 fix(chromium): remove tagged PDF by default 2024-03-04 10:28:54 +01:00
Julien Neuhart
d18e6920fc chore(libreoffice): order ext list alphabetically 2024-03-04 10:09:42 +01:00
eMerzh
c95174ff11 feat(libreoffice): add macOS *.key support 2024-02-26 16:47:52 +01:00
Julien Neuhart
c70329c22b chore(api): improve gotenberg.ErrPdfFormatNotSupported HTTP message 2024-02-24 14:55:48 +01:00
Julien Neuhart
34f69263a2 feat(libreoffice): explicity set tagged PDF as default 2024-02-24 13:45:01 +01:00
Julien Neuhart
bf5056f8fe feat(chromium): single page PDF 2024-02-23 21:27:18 +01:00
Julien Neuhart
c47f5b06ba feat(libreoffice): add macOS *.pages support 2024-02-23 18:16:42 +01:00
Julien Neuhart
63055e3df2 feat: better file sorting 2024-02-23 17:13:37 +01:00
Julien Neuhart
8ee555140b chore: rename regexp file to filter 2024-02-23 17:13:37 +01:00
Julien Neuhart
859f78e986 chore: update Go dependencies 2024-02-23 17:13:37 +01:00
Julien Neuhart
366eebb68c chore: using error middleware to handle common errors and lighten codebase 2024-02-23 14:52:43 +01:00
Julien Neuhart
25ce0b9aee fix(pdfengines): only preserve filenames in zip archive 2024-02-18 18:27:34 +01:00
Julien Neuhart
376a7db545 feat(pdfengines): preserve filenames in convert route 2024-02-18 18:17:12 +01:00
Julien Neuhart
f906ed741d fix(flags): typos 2024-02-18 17:52:51 +01:00
Julien Neuhart
ad152e62e5 fix(chromium): better default deny list regexp 2024-02-18 17:18:40 +01:00
Julien Neuhart
7f3a61ef43 chore: update dependencies 2024-02-18 17:18:08 +01:00
Julien Neuhart
a7ba8c8213 chore(testdata): add more data 2024-02-18 13:13:01 +01:00
Julien Neuhart
b1f3d6d4c0 feat: handling more ErrMaximumQueueSizeExceeded scenarios 2024-02-18 13:05:48 +01:00
Julien Neuhart
5707ae027f fix(libreoffice): expect output paths tests 2024-02-18 12:00:37 +01:00
timgrohmann
770208024d feat: enable max queue size for chromium and libreoffice
fix gotenberg/gotenberg#463
2024-02-18 11:35:49 +01:00
Julien Neuhart
975a9f5344 fix(libreoffice): wrong archive filename 2024-02-16 19:50:17 +01:00
Julien Neuhart
a45008411e chore: update dependencies 2024-02-16 19:50:17 +01:00
Julien Neuhart
9091ffd1b3 fix(libreoffice): keep original extension all the time for preserved filenames in zipped output 2024-02-16 19:50:17 +01:00
Chris Hughes
089f161d1e feat(libreoffice): preserve filenames for zipped files instead of using randomly generated UUID filenames (#793)
* feat(libreoffice): preserve filenames for zipped files instead of using generated UUID filenames

* chore: move github.com/google/uuid back to its original line

* fix: add libreoffice/routes.go to PR
2024-02-16 19:50:17 +01:00
dependabot[bot]
e6d7131701 chore(deps): bump golangci/golangci-lint-action from 3 to 4
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3 to 4.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-12 08:12:05 +01:00
Julien Neuhart
ed82170c2c chore(testdata): switch to px instead of rem in header.html & footer.html 2024-02-11 11:38:56 +01:00
Julien Neuhart
3472d42cdd chore: update dependencies 2024-02-07 10:44:30 +01:00
Julien Neuhart
b8926350b9 fix(libreoffice): wrong HTTP status codes if invalid PDF formats 2024-01-18 12:15:39 +01:00
Julien Neuhart
e5194b313d chore: update Go dependencies 2024-01-18 12:15:39 +01:00
Julien Neuhart
94228375ad chore: update Go dependencies 2024-01-10 11:55:05 +01:00
Julien Neuhart
0e8385a5a7 fix(chromium): missing webp validation 2024-01-08 11:44:56 +01:00
Julien Neuhart
afd3763d96 chore: update LICENSE year 2024-01-04 19:39:35 +01:00
Julien Neuhart
1122f09093 chore: remove OpenAPI documentation 2024-01-04 19:30:40 +01:00
Julien Neuhart
1123af15c9 chore: upgrade to v8 (#767) 2024-01-04 19:30:03 +01:00
Julien Neuhart
b2f6e865a7 feat(chromium): capture screenshot (#765) 2024-01-04 19:14:38 +01:00
Julien Neuhart
93b9ac2d13 feat(chromium): flags for clearing cache/cookies between each conversion (#756) 2023-12-14 22:06:13 +01:00
Julien Neuhart
64af65b0e7 feat(chromium): return 409 Conflict if HTTP status code from main page is not acceptable (#755) 2023-12-14 21:39:36 +01:00
Julien Neuhart
390c8437c5 fix(supervisor): if process is already restarting, requeue a task (#754) 2023-12-14 20:02:49 +01:00
Julien Neuhart
cc65ca9b6a fix(api): wait for modules readiness before starting server (#752) 2023-12-13 14:37:35 +01:00
Julien Neuhart
4bd7cba247 fix(libreoffice): accept PDF/A-1b instead of PDF/A-1a (#751) 2023-12-13 12:40:09 +01:00
Julien Neuhart
92e2f7f8b7 feat(chromium): skip networkIdle event 2023-12-12 18:40:55 +01:00
Julien Neuhart
98054ddff6 feat: upgrade pdfcpu 2023-12-12 18:40:55 +01:00
Julien Neuhart
4e5c1aba2e fix(armhf): now download latest working version from snapshots 2023-12-11 11:47:08 +01:00
Julien Neuhart
5eb44320ae fix(typo): Pdf -> PDF 2023-12-11 10:23:09 +01:00
dependabot[bot]
f751269289 chore(deps): bump actions/setup-go from 4 to 5 (#744)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4 to 5.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-11 10:15:20 +01:00
Julien Neuhart
0e836622ec chore(chromium): typo in a comment 2023-12-08 09:38:17 +01:00
Julien Neuhart
a6ac0a813d fix: special characters issues with filenames (#736) 2023-12-05 16:34:35 +01:00
Julien Neuhart
21c47f60f2 feat: remove deprecated stuff (#726) 2023-12-01 09:45:07 +01:00
78 changed files with 4605 additions and 2578 deletions

View File

@@ -15,16 +15,16 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Setup Go - name: Setup Go
uses: actions/setup-go@v4 uses: actions/setup-go@v5
with: with:
go-version: '1.21' go-version: '1.22'
cache: false cache: false
- name: Checkout source code - name: Checkout source code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Run linters - name: Run linters
uses: golangci/golangci-lint-action@v3 uses: golangci/golangci-lint-action@v4
with: with:
version: v1.54.2 version: v1.56.2
tests: tests:
needs: needs:

View File

@@ -3,9 +3,8 @@ linters-settings:
sections: sections:
- standard - standard
- default - default
- prefix(github.com/gotenberg/gotenberg/v7) - prefix(github.com/gotenberg/gotenberg/v8)
skip-generated: true skip-generated: true
skip-vendor: true
custom-order: true custom-order: true
linters: linters:
@@ -27,7 +26,7 @@ linters:
- unused - unused
run: run:
deadline: 5m timeout: 5m
issues-exit-code: 1 issues-exit-code: 1
tests: false tests: false

View File

@@ -1,6 +1,6 @@
MIT License MIT License
Copyright (c) 2023 Julien Neuhart Copyright (c) 2024 Julien Neuhart
Permission is hereby granted, free of charge, to any person obtaining a copy Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal of this software and associated documentation files (the "Software"), to deal

View File

@@ -5,14 +5,14 @@ help: ## Show the help
.PHONY: it .PHONY: it
it: build build-tests ## Initialize the development environment it: build build-tests ## Initialize the development environment
GOLANG_VERSION=1.21 GOLANG_VERSION=1.22
DOCKER_REPOSITORY=gotenberg DOCKER_REPOSITORY=gotenberg
GOTENBERG_VERSION=snapshot GOTENBERG_VERSION=snapshot
GOTENBERG_USER_GID=1001 GOTENBERG_USER_GID=1001
GOTENBERG_USER_UID=1001 GOTENBERG_USER_UID=1001
NOTO_COLOR_EMOJI_VERSION=v2.040 # See https://github.com/googlefonts/noto-emoji/releases. NOTO_COLOR_EMOJI_VERSION=v2.042 # See https://github.com/googlefonts/noto-emoji/releases.
PDFTK_VERSION=v3.3.3 # See https://gitlab.com/pdftk-java/pdftk/-/releases - Binary package. PDFTK_VERSION=v3.3.3 # See https://gitlab.com/pdftk-java/pdftk/-/releases - Binary package.
GOLANGCI_LINT_VERSION=v1.54.2 # See https://github.com/golangci/golangci-lint/releases. GOLANGCI_LINT_VERSION=v1.56.2 # See https://github.com/golangci/golangci-lint/releases.
.PHONY: build .PHONY: build
build: ## Build the Gotenberg's Docker image build: ## Build the Gotenberg's Docker image
@@ -29,13 +29,15 @@ build: ## Build the Gotenberg's Docker image
GOTENBERG_GRACEFUL_SHUTDOWN_DURATION=30s GOTENBERG_GRACEFUL_SHUTDOWN_DURATION=30s
API_PORT=3000 API_PORT=3000
API_PORT_FROM_ENV= API_PORT_FROM_ENV=
API_START_TIMEOUT=30s
API_TIMEOUT=30s API_TIMEOUT=30s
API_ROOT_PATH=/ API_ROOT_PATH=/
API_TRACE_HEADER=Gotenberg-Trace API_TRACE_HEADER=Gotenberg-Trace
API_DISABLE_HEALTH_CHECK_LOGGING=false API_DISABLE_HEALTH_CHECK_LOGGING=false
CHROMIUM_RESTART_AFTER=0 CHROMIUM_RESTART_AFTER=0
CHROMIUM_MAX_QUEUE_SIZE=0
CHROMIUM_AUTO_START=false CHROMIUM_AUTO_START=false
CHROMIUM_START_TIMEOUT=10s CHROMIUM_START_TIMEOUT=20s
CHROMIUM_INCOGNITO=false CHROMIUM_INCOGNITO=false
CHROMIUM_ALLOW_INSECURE_LOCALHOST=false CHROMIUM_ALLOW_INSECURE_LOCALHOST=false
CHROMIUM_IGNORE_CERTIFICATE_ERRORS=false CHROMIUM_IGNORE_CERTIFICATE_ERRORS=false
@@ -44,12 +46,15 @@ CHROMIUM_ALLOW_FILE_ACCESS_FROM_FILES=false
CHROMIUM_HOST_RESOLVER_RULES= CHROMIUM_HOST_RESOLVER_RULES=
CHROMIUM_PROXY_SERVER= CHROMIUM_PROXY_SERVER=
CHROMIUM_ALLOW_LIST= CHROMIUM_ALLOW_LIST=
CHROMIUM_DENY_LIST="^file:///[^tmp].*" CHROMIUM_DENY_LIST=^file:(?!//\/tmp/).*
CHROMIUM_CLEAR_CACHE=false
CHROMIUM_CLEAR_COOKIES=false
CHROMIUM_DISABLE_JAVASCRIPT=false CHROMIUM_DISABLE_JAVASCRIPT=false
CHROMIUM_DISABLE_ROUTES=false CHROMIUM_DISABLE_ROUTES=false
LIBREOFFICE_RESTART_AFTER=10 LIBREOFFICE_RESTART_AFTER=10
LIBREOFFICE_MAX_QUEUE_SIZE=0
LIBREOFFICE_AUTO_START=false LIBREOFFICE_AUTO_START=false
LIBREOFFICE_START_TIMEOUT=10s LIBREOFFICE_START_TIMEOUT=20s
LIBREOFFICE_DISABLE_ROUTES=false LIBREOFFICE_DISABLE_ROUTES=false
LOG_LEVEL=info LOG_LEVEL=info
LOG_FORMAT=auto LOG_FORMAT=auto
@@ -79,12 +84,14 @@ run: ## Start a Gotenberg container
--gotenberg-graceful-shutdown-duration=$(GOTENBERG_GRACEFUL_SHUTDOWN_DURATION) \ --gotenberg-graceful-shutdown-duration=$(GOTENBERG_GRACEFUL_SHUTDOWN_DURATION) \
--api-port=$(API_PORT) \ --api-port=$(API_PORT) \
--api-port-from-env=$(API_PORT_FROM_ENV) \ --api-port-from-env=$(API_PORT_FROM_ENV) \
--api-start-timeout=$(API_START_TIMEOUT) \
--api-timeout=$(API_TIMEOUT) \ --api-timeout=$(API_TIMEOUT) \
--api-root-path=$(API_ROOT_PATH) \ --api-root-path=$(API_ROOT_PATH) \
--api-trace-header=$(API_TRACE_HEADER) \ --api-trace-header=$(API_TRACE_HEADER) \
--api-disable-health-check-logging=$(API_DISABLE_HEALTH_CHECK_LOGGING) \ --api-disable-health-check-logging=$(API_DISABLE_HEALTH_CHECK_LOGGING) \
--chromium-restart-after=$(CHROMIUM_RESTART_AFTER) \ --chromium-restart-after=$(CHROMIUM_RESTART_AFTER) \
--chromium-auto-start=$(CHROMIUM_AUTO_START) \ --chromium-auto-start=$(CHROMIUM_AUTO_START) \
--chromium-max-queue-size=$(CHROMIUM_MAX_QUEUE_SIZE) \
--chromium-start-timeout=$(CHROMIUM_START_TIMEOUT) \ --chromium-start-timeout=$(CHROMIUM_START_TIMEOUT) \
--chromium-incognito=$(CHROMIUM_INCOGNITO) \ --chromium-incognito=$(CHROMIUM_INCOGNITO) \
--chromium-allow-insecure-localhost=$(CHROMIUM_ALLOW_INSECURE_LOCALHOST) \ --chromium-allow-insecure-localhost=$(CHROMIUM_ALLOW_INSECURE_LOCALHOST) \
@@ -93,11 +100,14 @@ run: ## Start a Gotenberg container
--chromium-allow-file-access-from-files=$(CHROMIUM_ALLOW_FILE_ACCESS_FROM_FILES) \ --chromium-allow-file-access-from-files=$(CHROMIUM_ALLOW_FILE_ACCESS_FROM_FILES) \
--chromium-host-resolver-rules=$(CHROMIUM_HOST_RESOLVER_RULES) \ --chromium-host-resolver-rules=$(CHROMIUM_HOST_RESOLVER_RULES) \
--chromium-proxy-server=$(CHROMIUM_PROXY_SERVER) \ --chromium-proxy-server=$(CHROMIUM_PROXY_SERVER) \
--chromium-allow-list=$(CHROMIUM_ALLOW_LIST) \ --chromium-allow-list="$(CHROMIUM_ALLOW_LIST)" \
--chromium-deny-list=$(CHROMIUM_DENY_LIST) \ --chromium-deny-list="$(CHROMIUM_DENY_LIST)" \
--chromium-clear-cache=$(CHROMIUM_CLEAR_CACHE) \
--chromium-clear-cookies=$(CHROMIUM_CLEAR_COOKIES) \
--chromium-disable-javascript=$(CHROMIUM_DISABLE_JAVASCRIPT) \ --chromium-disable-javascript=$(CHROMIUM_DISABLE_JAVASCRIPT) \
--chromium-disable-routes=$(CHROMIUM_DISABLE_ROUTES) \ --chromium-disable-routes=$(CHROMIUM_DISABLE_ROUTES) \
--libreoffice-restart-after=$(LIBREOFFICE_RESTART_AFTER) \ --libreoffice-restart-after=$(LIBREOFFICE_RESTART_AFTER) \
--libreoffice-max-queue-size=$(LIBREOFFICE_MAX_QUEUE_SIZE) \
--libreoffice-auto-start=$(LIBREOFFICE_AUTO_START) \ --libreoffice-auto-start=$(LIBREOFFICE_AUTO_START) \
--libreoffice-start-timeout=$(LIBREOFFICE_START_TIMEOUT) \ --libreoffice-start-timeout=$(LIBREOFFICE_START_TIMEOUT) \
--libreoffice-disable-routes=$(LIBREOFFICE_DISABLE_ROUTES) \ --libreoffice-disable-routes=$(LIBREOFFICE_DISABLE_ROUTES) \
@@ -110,8 +120,8 @@ run: ## Start a Gotenberg container
--prometheus-collect-interval=$(PROMETHEUS_COLLECT_INTERVAL) \ --prometheus-collect-interval=$(PROMETHEUS_COLLECT_INTERVAL) \
--prometheus-disable-route-logging=$(PROMETHEUS_DISABLE_ROUTE_LOGGING) \ --prometheus-disable-route-logging=$(PROMETHEUS_DISABLE_ROUTE_LOGGING) \
--prometheus-disable-collect=$(PROMETHEUS_DISABLE_COLLECT) \ --prometheus-disable-collect=$(PROMETHEUS_DISABLE_COLLECT) \
--webhook-allow-list=$(WEBHOOK_ALLOW_LIST) \ --webhook-allow-list="$(WEBHOOK_ALLOW_LIST)" \
--webhook-deny-list=$(WEBHOOK_DENY_LIST) \ --webhook-deny-list="$(WEBHOOK_DENY_LIST)" \
--webhook-error-allow-list=$(WEBHOOK_ERROR_ALLOW_LIST) \ --webhook-error-allow-list=$(WEBHOOK_ERROR_ALLOW_LIST) \
--webhook-error-deny-list=$(WEBHOOK_ERROR_DENY_LIST) \ --webhook-error-deny-list=$(WEBHOOK_ERROR_DENY_LIST) \
--webhook-max-retry=$(WEBHOOK_MAX_RETRY) \ --webhook-max-retry=$(WEBHOOK_MAX_RETRY) \
@@ -149,13 +159,13 @@ tests-once: ## Run the tests once (prefer the "tests" command while developing)
.PHONY: fmt .PHONY: fmt
fmt: ## Format the code and "optimize" the dependencies fmt: ## Format the code and "optimize" the dependencies
gofumpt -l -w . gofumpt -l -w .
gci write -s standard -s default -s "prefix(github.com/gotenberg/gotenberg/v7)" --skip-generated --skip-vendor --custom-order . gci write -s standard -s default -s "prefix(github.com/gotenberg/gotenberg/v8)" --skip-generated --skip-vendor --custom-order .
go mod tidy go mod tidy
# go install golang.org/x/tools/cmd/godoc@latest # go install golang.org/x/tools/cmd/godoc@latest
.PHONY: godoc .PHONY: godoc
godoc: ## Run a webserver with Gotenberg godoc godoc: ## Run a webserver with Gotenberg godoc
$(info http://localhost:6060/pkg/github.com/gotenberg/gotenberg/v7) $(info http://localhost:6060/pkg/github.com/gotenberg/gotenberg/v8)
godoc -http=:6060 godoc -http=:6060
.PHONY: release .PHONY: release

View File

@@ -15,13 +15,13 @@ numerous document formats (HTML, Markdown, Word, Excel, etc.) into PDF files, an
Open a terminal and run the following command: Open a terminal and run the following command:
``` ```
docker run --rm -p 3000:3000 gotenberg/gotenberg:7 docker run --rm -p 3000:3000 gotenberg/gotenberg:8
``` ```
Alternatively, using the historic Docker repository from our sponsor [TheCodingMachine](https://www.thecodingmachine.com): Alternatively, using the historic Docker repository from our sponsor [TheCodingMachine](https://www.thecodingmachine.com):
``` ```
docker run --rm -p 3000:3000 thecodingmachine/gotenberg:7 docker run --rm -p 3000:3000 thecodingmachine/gotenberg:8
``` ```
The API is now available on your host at http://localhost:3000. The API is now available on your host at http://localhost:3000.
@@ -41,5 +41,5 @@ Head to the [documentation](https://gotenberg.dev/docs/getting-started/introduct
[![Docker pulls](https://img.shields.io/docker/pulls/gotenberg/gotenberg)](https://hub.docker.com/r/gotenberg/gotenberg) [![Docker pulls](https://img.shields.io/docker/pulls/gotenberg/gotenberg)](https://hub.docker.com/r/gotenberg/gotenberg)
[![Docker pulls](https://img.shields.io/docker/pulls/thecodingmachine/gotenberg)](https://hub.docker.com/r/thecodingmachine/gotenberg) [![Docker pulls](https://img.shields.io/docker/pulls/thecodingmachine/gotenberg)](https://hub.docker.com/r/thecodingmachine/gotenberg)
[![Continuous Integration](https://github.com/gotenberg/gotenberg/actions/workflows/continuous_integration.yml/badge.svg)](https://github.com/gotenberg/gotenberg/actions/workflows/continuous_integration.yml) [![Continuous Integration](https://github.com/gotenberg/gotenberg/actions/workflows/continuous_integration.yml/badge.svg)](https://github.com/gotenberg/gotenberg/actions/workflows/continuous_integration.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/gotenberg/gotenberg.svg)](https://pkg.go.dev/github.com/gotenberg/gotenberg/v7) [![Go Reference](https://pkg.go.dev/badge/github.com/gotenberg/gotenberg.svg)](https://pkg.go.dev/github.com/gotenberg/gotenberg/v8)
[![Codecov](https://codecov.io/gh/gotenberg/gotenberg/branch/main/graph/badge.svg)](https://codecov.io/gh/gotenberg/gotenberg) [![Codecov](https://codecov.io/gh/gotenberg/gotenberg/branch/main/graph/badge.svg)](https://codecov.io/gh/gotenberg/gotenberg)

View File

@@ -2,7 +2,6 @@
# concatenate them. Also, we have to repeat ARG instructions in each build # concatenate them. Also, we have to repeat ARG instructions in each build
# stage that uses them. # stage that uses them.
ARG GOLANG_VERSION ARG GOLANG_VERSION
ARG GOTENBERG_VERSION
# ---------------------------------------------- # ----------------------------------------------
# Gotenberg binary build stage # Gotenberg binary build stage
@@ -25,7 +24,7 @@ RUN go mod download &&\
COPY cmd ./cmd COPY cmd ./cmd
COPY pkg ./pkg COPY pkg ./pkg
RUN go build -o gotenberg -ldflags "-X 'github.com/gotenberg/gotenberg/v7/cmd.Version=$GOTENBERG_VERSION'" cmd/gotenberg/main.go RUN go build -o gotenberg -ldflags "-X 'github.com/gotenberg/gotenberg/v8/cmd.Version=$GOTENBERG_VERSION'" cmd/gotenberg/main.go
# ---------------------------------------------- # ----------------------------------------------
# Final stage # Final stage
@@ -134,7 +133,12 @@ RUN \
mv /usr/bin/google-chrome-stable /usr/bin/chromium; \ mv /usr/bin/google-chrome-stable /usr/bin/chromium; \
elif [[ "$(dpkg --print-architecture)" == "armhf" ]]; then \ elif [[ "$(dpkg --print-architecture)" == "armhf" ]]; then \
apt-get update -qq &&\ apt-get update -qq &&\
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends chromium-common="$TMP_CHOMIUM_VERSION_ARMHF" chromium="$TMP_CHOMIUM_VERSION_ARMHF"; \ DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends devscripts &&\
debsnap chromium-common "$TMP_CHOMIUM_VERSION_ARMHF" -v --force --binary --architecture armhf &&\
debsnap chromium "$TMP_CHOMIUM_VERSION_ARMHF" -v --force --binary --architecture armhf &&\
DEBIAN_FRONTEND=noninteractive apt-get install --fix-broken -y -qq --no-install-recommends "./binary-chromium-common/chromium-common_${TMP_CHOMIUM_VERSION_ARMHF}_armhf.deb" "./binary-chromium/chromium_${TMP_CHOMIUM_VERSION_ARMHF}_armhf.deb" &&\
DEBIAN_FRONTEND=noninteractive apt-get purge -y -qq devscripts &&\
rm -rf ./binary-chromium-common/* ./binary-chromium/*; \
else \ else \
apt-get update -qq &&\ apt-get update -qq &&\
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends chromium; \ DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends chromium; \

View File

@@ -11,7 +11,7 @@ import (
flag "github.com/spf13/pflag" flag "github.com/spf13/pflag"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
// See https://patorjk.com/software/taag/#p=display&f=Small%20Slant&t=Gotenberg. // See https://patorjk.com/software/taag/#p=display&f=Small%20Slant&t=Gotenberg.
@@ -75,7 +75,6 @@ func Run() {
go func(app gotenberg.App) { go func(app gotenberg.App) {
id := app.(gotenberg.Module).Descriptor().ID id := app.(gotenberg.Module).Descriptor().ID
err = app.Start() err = app.Start()
if err != nil { if err != nil {
fmt.Printf("[FATAL] starting %s: %s\n", id, err) fmt.Printf("[FATAL] starting %s: %s\n", id, err)
os.Exit(1) os.Exit(1)
@@ -84,7 +83,6 @@ func Run() {
startupMessage := app.StartupMessage() startupMessage := app.StartupMessage()
if startupMessage == "" { if startupMessage == "" {
fmt.Printf("[SYSTEM] %s: application started\n", id) fmt.Printf("[SYSTEM] %s: application started\n", id)
return return
} }
@@ -144,7 +142,6 @@ func Run() {
} }
fmt.Printf("[SYSTEM] %s: application stopped\n", id) fmt.Printf("[SYSTEM] %s: application stopped\n", id)
return nil return nil
} }
}(a.(gotenberg.App))) }(a.(gotenberg.App)))

View File

@@ -1,9 +1,9 @@
package main package main
import ( import (
gotenbergcmd "github.com/gotenberg/gotenberg/v7/cmd" gotenbergcmd "github.com/gotenberg/gotenberg/v8/cmd"
// Gotenberg modules. // Gotenberg modules.
_ "github.com/gotenberg/gotenberg/v7/pkg/standard" _ "github.com/gotenberg/gotenberg/v8/pkg/standard"
) )
func main() { func main() {

View File

@@ -1,744 +0,0 @@
openapi: 3.0.3
info:
title: Gotenberg
version: 7.x
license:
name: MIT
url: 'https://github.com/gotenberg/gotenberg/blob/main/LICENSE'
contact:
url: 'https://github.com/gotenberg/gotenberg'
description: >-
A Docker-powered stateless API for PDF files.
externalDocs:
url: https://gotenberg.dev
servers:
- url: 'http://localhost:3000'
description: Local server with the default Docker image and port
tags:
- name: chromium
description: Operations of the Chromium module
externalDocs:
url: https://gotenberg.dev/docs/modules/chromium
- name: libreoffice
description: Operations of the Libreoffice module
externalDocs:
url: https://gotenberg.dev/docs/modules/libreoffice
- name: pdfengines
description: Operations of the PDF Engines module
externalDocs:
url: https://gotenberg.dev/docs/modules/pdf-engines
paths:
/forms/chromium/convert/url:
post:
tags:
- chromium
summary: Convert the contents of a given URL to PDF
externalDocs:
url: https://gotenberg.dev/docs/modules/chromium
description: >-
Send a URL in your API request via the `url` form field
Send a remote URL in your API request via the `remoteURL` parameter, and
get the resulting PDF file. The API will fetch the given URL and render
the page to PDF using the underlying headless Chrome instance.
You can optionally include `header.html` and `footer.html` files as part of the request as well.
See externalDocs for more details.
parameters:
- in: header
name: Gotenberg-Output-Filename
description: >-
By default, the API generates a UUID filename.
However, you may also specify the filename per request,
thanks to the Gotenberg-Output-Filename header.
Caution! The API adds the file extension automatically; you don't have to set it.
schema:
type: string
required: false
- in: header
name: Gotenberg-Trace
description: >-
The trace, or request ID, identifies a request in the logs.
By default, the API generates a UUID trace for each request.
However, you may also specify the trace per request, thanks to the Gotenberg-Trace header.
schema:
type: string
required: false
requestBody:
required: true
description: >-
The request must be `multipart/form-data` that includes a `url` form field.
The API uses a headless Chrome instance to do the conversion, therefore print
parameter such as margins and paper size are also accepted as optional parameters.
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/URLConvertRequestBody'
examples: { }
responses:
'200':
$ref: '#/components/responses/SuccessfulPDF'
'400':
description: Bad Request
'503':
$ref: '#/components/responses/Timeout'
/forms/chromium/convert/html:
post:
tags:
- chromium
summary: Convert a given HTML file to PDF
externalDocs:
url: https://gotenberg.dev/docs/modules/chromium
description: >-
Send an HTML file called `index.html` as a multipart form request, and
get the resulting PDF file. You can optionally include `header.html` and
`footer.html` files as part of the request as well.
See externalDocs for more details.
parameters:
- in: header
name: Gotenberg-Output-Filename
description: >-
By default, the API generates a UUID filename.
However, you may also specify the filename per request,
thanks to the Gotenberg-Output-Filename header.
Caution! The API adds the file extension automatically; you don't have to set it.
schema:
type: string
required: false
- in: header
name: Gotenberg-Trace
description: >-
The trace, or request ID, identifies a request in the logs.
By default, the API generates a UUID trace for each request.
However, you may also specify the trace per request, thanks to the Gotenberg-Trace header.
schema:
type: string
required: false
requestBody:
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/HTMLConvertRequestBody'
description: >-
The request body must have an `index.html` file in the `files` array,
as well as all the referred resources on the same level as the
`index.html` file. The request can also include `header.html` and
`footer.html`, given the limitations in the API description above.
responses:
'200':
$ref: '#/components/responses/SuccessfulPDF'
'400':
description: Bad Request
'503':
$ref: '#/components/responses/Timeout'
/forms/chromium/convert/markdown:
post:
tags:
- chromium
summary: Convert a Markdown file to PDF
externalDocs:
url: https://gotenberg.dev/docs/modules/chromium
description: >-
Accepts an HTML file called `index.html` plus markdown files as a multipart
form request and embeds the markdown files into the HTML file using the Golang template
function `toHTML`.
The API will convert the markdown to HTML and embed it into your `index.html` file,
then render the resulting page. You can include your own styling and more in your HTML file.
Refer to the HTML conversion page for all the options you can use when converting
Markdown documents as well. You can optionally include `header.html` and
`footer.html` files as part of the request as well.
See externalDocs for more details.
parameters:
- in: header
name: Gotenberg-Output-Filename
description: >-
By default, the API generates a UUID filename.
However, you may also specify the filename per request,
thanks to the Gotenberg-Output-Filename header.
Caution! The API adds the file extension automatically; you don't have to set it.
schema:
type: string
required: false
- in: header
name: Gotenberg-Trace
description: >-
The trace, or request ID, identifies a request in the logs.
By default, the API generates a UUID trace for each request.
However, you may also specify the trace per request, thanks to the Gotenberg-Trace header.
schema:
type: string
required: false
requestBody:
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/MarkdownConvertRequestBody'
description: >-
The request body must have an `index.html` file in the `files` array,
as well as all the referred markdown resources on the same level as
the `index.html` file.
responses:
'200':
$ref: '#/components/responses/SuccessfulPDF'
'400':
description: Bad Request
'503':
$ref: '#/components/responses/Timeout'
/forms/libreoffice/convert:
post:
tags:
- libreoffice
summary: Convert an Office document to PDF
externalDocs:
url: https://gotenberg.dev/docs/modules/libreoffice
description: >-
This route accepts multipart/form-data requests and files with the following extensions:
.bib .doc .xml .docx .fodt .html .ltx .txt .odt .ott .pdb .pdf .psw .rtf
.sdw .stw .sxw .uot .vor .wps .epub .png .bmp .emf .eps .fodg .gif .jpg
.met .odd .otg .pbm .pct .pgm .ppm .ras .std .svg .svm .swf .sxd .sxw
.tiff .xhtml .xpm .fodp .potm .pot .pptx .pps .ppt .pwp .sda .sdd .sti
.sxi .uop .wmf .csv .dbf .dif .fods .ods .ots .pxl .sdc .slk .stc .sxc
.uos .xls .xlt .xlsx .tif .jpeg .odp
By default, if you send more than one file to convert, the route returns a ZIP archive of the
resulting PDF files. However, you may prefer to merge all the PDF files into an individual PDF file.
> **Attention:** The files will be merged alphabetically for the
resulting PDF.
You may also specify the page ranges to convert from the incoming Office
documents. The expected format is the same as the one from the print
options of LibreOffice, e.g. `1-1` or `1-4`.
> **Attention:** if more than one document, the page ranges will be
applied for each document.
See externalDocs for more details.
parameters:
- in: header
name: Gotenberg-Output-Filename
description: >-
By default, the API generates a UUID filename.
However, you may also specify the filename per request,
thanks to the Gotenberg-Output-Filename header.
Caution! The API adds the file extension automatically; you don't have to set it.
schema:
type: string
required: false
- in: header
name: Gotenberg-Trace
description: >-
The trace, or request ID, identifies a request in the logs.
By default, the API generates a UUID trace for each request.
However, you may also specify the trace per request, thanks to the Gotenberg-Trace header.
schema:
type: string
required: false
requestBody:
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/OfficeConvertRequestBody'
responses:
'200':
$ref: '#/components/responses/SuccessfulPDF'
'400':
description: Bad Request, e.g. Both 'pdfFormat' and 'nativePdfA1aFormat' form values are provided
'503':
$ref: '#/components/responses/Timeout'
/forms/pdfengines/merge:
post:
tags:
- pdfengines
summary: Merge multiple PDFs into a single PDF
externalDocs:
url: https://gotenberg.dev/docs/modules/pdf-engines
description: >-
You can send multiple PDF files to this endpoint, the API will merge
them into a single PDF and return the resulting PDF file.
> **Attention:** The PDF files will be merged alphabetically.
parameters:
- in: header
name: Gotenberg-Output-Filename
description: >-
By default, the API generates a UUID filename.
However, you may also specify the filename per request,
thanks to the Gotenberg-Output-Filename header.
Caution! The API adds the file extension automatically; you don't have to set it.
schema:
type: string
required: false
- in: header
name: Gotenberg-Trace
description: >-
The trace, or request ID, identifies a request in the logs.
By default, the API generates a UUID trace for each request.
However, you may also specify the trace per request, thanks to the Gotenberg-Trace header.
schema:
type: string
required: false
requestBody:
content:
multipart/form-data:
schema:
type: object
properties:
files:
type: array
items:
type: string
format: binary
pdfFormat:
type: string
description: The PDF format of the resulting PDF
example: PDF/A-1a
required:
- files
responses:
'200':
$ref: '#/components/responses/SuccessfulPDF'
'400':
description: Bad Request
'503':
$ref: '#/components/responses/Timeout'
/forms/pdfengines/convert:
post:
tags:
- pdfengines
summary: Convert PDFs into the given formats
externalDocs:
url: https://gotenberg.dev/docs/modules/pdf-engines
description: >-
This route accepts PDF files and a form field pdfFormat for converting them into the specified format.
parameters:
- in: header
name: Gotenberg-Output-Filename
description: >-
By default, the API generates a UUID filename.
However, you may also specify the filename per request,
thanks to the Gotenberg-Output-Filename header.
Caution! The API adds the file extension automatically; you don't have to set it.
schema:
type: string
required: false
- in: header
name: Gotenberg-Trace
description: >-
The trace, or request ID, identifies a request in the logs.
By default, the API generates a UUID trace for each request.
However, you may also specify the trace per request, thanks to the Gotenberg-Trace header.
schema:
type: string
required: false
requestBody:
content:
multipart/form-data:
schema:
type: object
properties:
files:
type: array
items:
type: string
format: binary
pdfFormat:
type: string
description: The PDF format of the resulting PDF
example: PDF/A-1a
required:
- files
- pdfFormat
responses:
'200':
$ref: '#/components/responses/SuccessfulPDF'
'400':
description: >-
Bad Request, e.g. Invalid form data: no form file found for extensions: [.pdf]; form value 'pdfFormat' is required
'503':
$ref: '#/components/responses/Timeout'
components:
schemas:
HTMLConvertRequestBody:
title: HTML Conversion Request Body
type: object
properties:
files:
type: array
description: >-
List of HTML files to be converted to PDF. An `index.html` file is
required, and any other resources that are referenced through the
HTML file must be included as well. All the referenced files must be
on the same level as the `index.html` file.
items:
type: string
format: binary
marginTop:
type: number
example: 0
default: 1
description: Top margin for the page in inches.
marginBottom:
type: number
example: 0
default: 1
description: Bottom margin for the page in inches.
marginLeft:
type: number
example: 0
default: 1
description: Left margin for the page in inches.
marginRight:
type: number
example: 0
default: 1
description: Right margin for the page in inches.
paperWidth:
type: number
example: 8.27
description: >-
Paper width to be used while rendering the PDF. The default page
size is A4.
paperHeight:
type: number
example: 11.69
description: >-
Paper height to be used while rendering the PDF. The default page
size is A4.
preferCssPageSize:
type: boolean
description: >-
Define whether to prefer page size as defined by CSS (default false)
default: false
printBackground:
type: boolean
description: >-
Print the background graphics (default false)
default: false
landscape:
type: boolean
example: true
default: false
description: >-
The default orientation for rendering the page is "portrait" mode.
By sending "landscape" parameter, you can ask the output to be
landscape.
scale:
type: number
minimum: 0.1
maximum: 2.0
example: 1.5
description: >-
The scale of the page rendering
default: 1.0
waitDelay:
type: string
example: 5s
description: >-
When the page relies on JavaScript for rendering, and you don't have access to the page's code,
you may want to wait a certain amount of time to make sure Chromium has fully rendered the page
you're trying to generate.
waitWindowStatus:
type: string
example: done
description: >-
If you have access to the page's code, you may set the window status and tell Gotenberg to wait for a specific value.
For instance
await promises()
window.status = 'ready'
Prefer this option over waitDelay.
extraHttpHeaders:
type: string
description: HTTP headers to send by Chromium while loading the HTML document (JSON format)
nativePageRanges:
type: string
example: 1-4
description: >-
The page ranges to be converted to PDF for the incoming Office
documents.
pdfFormat:
type: string
description: >-
The PDF format of the resulting PDF.
Caution! You cannot use both nativePdfA1aFormat and pdfFormat form fields.
example: PDF/A-1a
required:
- files
MarkdownConvertRequestBody:
title: Markdown Conversion Request Body
type: object
properties:
files:
type: array
items:
type: string
format: binary
marginTop:
type: number
example: 0
default: 1
description: Top margin for the page in inches.
marginBottom:
type: number
example: 0
default: 1
description: Bottom margin for the page in inches.
marginLeft:
type: number
example: 0
default: 1
description: Left margin for the page in inches.
marginRight:
type: number
example: 0
default: 1
description: Right margin for the page in inches.
paperWidth:
type: number
example: 8.27
description: >-
Paper width to be used while rendering the PDF. The default page
size is A4.
paperHeight:
type: number
example: 11.69
description: >-
Paper height to be used while rendering the PDF. The default page
size is A4.
preferCssPageSize:
type: boolean
description: >-
Define whether to prefer page size as defined by CSS (default false)
default: false
printBackground:
type: boolean
description: >-
Print the background graphics (default false)
default: false
landscape:
type: boolean
example: true
default: false
description: >-
The default orientation for rendering the page is "portrait" mode.
By sending "landscape" parameter, you can ask the output to be
landscape.
scale:
type: number
minimum: 0.1
maximum: 2.0
example: 1.5
description: >-
The scale of the page rendering
default: 1.0
waitDelay:
type: string
example: 5s
description: >-
When the page relies on JavaScript for rendering, and you don't have access to the page's code,
you may want to wait a certain amount of time to make sure Chromium has fully rendered the page
you're trying to generate.
waitWindowStatus:
type: string
example: done
description: >-
If you have access to the page's code, you may set the window status and tell Gotenberg to wait for a specific value.
For instance
await promises()
window.status = 'ready'
Prefer this option over waitDelay.
extraHttpHeaders:
type: string
description: HTTP headers to send by Chromium while loading the HTML document (JSON format)
nativePageRanges:
type: string
example: 1-4
description: >-
The page ranges to be converted to PDF for the incoming Office
documents.
pdfFormat:
type: string
description: >-
The PDF format of the resulting PDF.
Caution! You cannot use both nativePdfA1aFormat and pdfFormat form fields.
example: PDF/A-1a
required:
- files
URLConvertRequestBody:
title: URL Conversion Request Body
type: object
properties:
url:
type: string
example: 'https://google.com'
files:
description: Optional files named header.html and footer.html
type: array
items:
type: string
format: binary
marginTop:
type: number
example: 0
default: 1
description: Top margin for the page in inches.
marginBottom:
type: number
example: 0
default: 1
description: Bottom margin for the page in inches.
marginLeft:
type: number
example: 0
default: 1
description: Left margin for the page in inches.
marginRight:
type: number
example: 0
default: 1
description: Right margin for the page in inches.
paperWidth:
type: number
example: 8.27
description: >-
Paper width to be used while rendering the PDF. The default page
size is A4.
paperHeight:
type: number
example: 11.69
description: >-
Paper height to be used while rendering the PDF. The default page
size is A4.
preferCssPageSize:
type: boolean
description: >-
Define whether to prefer page size as defined by CSS (default false)
default: false
printBackground:
type: boolean
description: >-
Print the background graphics (default false)
default: false
landscape:
type: boolean
example: true
default: false
description: >-
The default orientation for rendering the page is "portrait" mode.
By sending "landscape" parameter, you can ask the output to be
landscape.
scale:
type: number
minimum: 0.1
maximum: 2.0
example: 1.5
description: >-
The scale of the page rendering
default: 1.0
waitDelay:
type: string
example: 5s
description: >-
When the page relies on JavaScript for rendering, and you don't have access to the page's code,
you may want to wait a certain amount of time to make sure Chromium has fully rendered the page
you're trying to generate.
waitWindowStatus:
type: string
example: done
description: >-
If you have access to the page's code, you may set the window status and tell Gotenberg to wait for a specific value.
For instance
await promises()
window.status = 'ready'
Prefer this option over waitDelay.
extraHttpHeaders:
type: string
description: HTTP headers to send by Chromium while loading the HTML document (JSON format)
nativePageRanges:
type: string
example: 1-4
description: >-
The page ranges to be converted to PDF for the incoming Office
documents.
pdfFormat:
type: string
description: >-
The PDF format of the resulting PDF.
Caution! You cannot use both nativePdfA1aFormat and pdfFormat form fields.
example: PDF/A-1a
required:
- url
OfficeConvertRequestBody:
title: Office Conversion Request Body
type: object
properties:
files:
type: array
items:
type: string
format: binary
nativePageRanges:
type: string
example: 1-4
description: >-
The page ranges to be converted to PDF for the incoming Office
documents. **If there are multiple files sent to the API, this page
range will apply to all of the documents**. Empty means all pages.
nativePdfA1aFormat:
type: boolean
description: >-
Use unoconv to convert the resulting PDF to the 'PDF/A-1a' format.
Caution! You cannot use both nativePdfA1aFormat and pdfFormat form fields.
pdfFormat:
type: string
description: >-
The PDF format of the resulting PDF.
Caution! You cannot use both nativePdfA1aFormat and pdfFormat form fields.
example: PDF/A-1a
landscape:
type: boolean
example: true
default: false
description: >-
The default orientation for rendering the page is "portrait" mode.
By sending "landscape" parameter, you can ask the output to be
landscape.
merge:
type: boolean
description: >-
Merge all PDF files into an individual PDF file.
required:
- files
MergeFilesRequestBody:
title: Merge Files Request Body
type: object
properties:
files:
type: array
items:
type: string
format: binary
required:
- files
securitySchemes: { }
responses:
SuccessfulPDF:
description: Resulting PDF file from the conversion.
content:
application/pdf:
schema:
type: string
format: binary
Timeout:
description: Service Unavailable

53
go.mod
View File

@@ -1,41 +1,43 @@
module github.com/gotenberg/gotenberg/v7 module github.com/gotenberg/gotenberg/v8
go 1.21 go 1.22
require ( require (
github.com/alexliesenfeld/health v0.8.0 github.com/alexliesenfeld/health v0.8.0
github.com/andybalholm/brotli v1.0.6 // indirect github.com/andybalholm/brotli v1.1.0 // indirect
github.com/chromedp/cdproto v0.0.0-20231114014204-3e458d5176f9 github.com/chromedp/cdproto v0.0.0-20240312231614-1e5096e63154
github.com/chromedp/chromedp v0.9.3 github.com/chromedp/chromedp v0.9.5
github.com/golang/snappy v0.0.4 // indirect github.com/golang/snappy v0.0.4 // indirect
github.com/google/uuid v1.4.0 github.com/google/uuid v1.6.0
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-retryablehttp v0.7.5 github.com/hashicorp/go-retryablehttp v0.7.5
github.com/klauspost/compress v1.17.3 // indirect github.com/klauspost/compress v1.17.7 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect github.com/klauspost/pgzip v1.2.6 // indirect
github.com/labstack/echo/v4 v4.11.3 github.com/labstack/echo/v4 v4.11.4
github.com/labstack/gommon v0.4.1 github.com/labstack/gommon v0.4.2
github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mholt/archiver/v3 v3.5.1 github.com/mholt/archiver/v3 v3.5.1
github.com/microcosm-cc/bluemonday v1.0.26 github.com/microcosm-cc/bluemonday v1.0.26
github.com/nwaples/rardecode v1.1.3 // indirect github.com/nwaples/rardecode v1.1.3 // indirect
github.com/pdfcpu/pdfcpu v0.5.0 github.com/pdfcpu/pdfcpu v0.7.0
github.com/pierrec/lz4/v4 v4.1.18 // indirect github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/prometheus/client_golang v1.17.0 github.com/prometheus/client_golang v1.19.0
github.com/russross/blackfriday/v2 v2.1.0 github.com/russross/blackfriday/v2 v2.1.0
github.com/spf13/pflag v1.0.5 github.com/spf13/pflag v1.0.5
github.com/ulikunitz/xz v0.5.11 // indirect github.com/ulikunitz/xz v0.5.11 // indirect
go.uber.org/multierr v1.11.0 go.uber.org/multierr v1.11.0
go.uber.org/zap v1.26.0 go.uber.org/zap v1.27.0
golang.org/x/crypto v0.15.0 // indirect golang.org/x/crypto v0.21.0 // indirect
golang.org/x/image v0.14.0 // indirect golang.org/x/image v0.15.0 // indirect
golang.org/x/net v0.18.0 golang.org/x/net v0.22.0
golang.org/x/sync v0.5.0 golang.org/x/sync v0.6.0
golang.org/x/sys v0.14.0 // indirect golang.org/x/sys v0.18.0 // indirect
golang.org/x/term v0.14.0 golang.org/x/term v0.18.0
golang.org/x/text v0.14.0 golang.org/x/text v0.14.0
) )
require github.com/dlclark/regexp2 v1.11.0
require ( require (
github.com/aymerick/douceur v0.2.0 // indirect github.com/aymerick/douceur v0.2.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect
@@ -44,7 +46,7 @@ require (
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/gobwas/httphead v0.1.0 // indirect github.com/gobwas/httphead v0.1.0 // indirect
github.com/gobwas/pool v0.2.1 // indirect github.com/gobwas/pool v0.2.1 // indirect
github.com/gobwas/ws v1.3.1 // indirect github.com/gobwas/ws v1.3.2 // indirect
github.com/gorilla/css v1.0.1 // indirect github.com/gorilla/css v1.0.1 // indirect
github.com/hhrutter/lzw v1.0.0 // indirect github.com/hhrutter/lzw v1.0.0 // indirect
github.com/hhrutter/tiff v1.0.1 // indirect github.com/hhrutter/tiff v1.0.1 // indirect
@@ -52,15 +54,14 @@ require (
github.com/mailru/easyjson v0.7.7 // indirect github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
github.com/pkg/errors v0.9.1 // indirect github.com/pkg/errors v0.9.1 // indirect
github.com/prometheus/client_model v0.5.0 // indirect github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/common v0.45.0 // indirect github.com/prometheus/common v0.50.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect github.com/prometheus/procfs v0.13.0 // indirect
github.com/rivo/uniseg v0.4.4 // indirect github.com/rivo/uniseg v0.4.7 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasttemplate v1.2.2 // indirect github.com/valyala/fasttemplate v1.2.2 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
google.golang.org/protobuf v1.31.0 // indirect google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect
) )

110
go.sum
View File

@@ -1,23 +1,25 @@
github.com/alexliesenfeld/health v0.8.0 h1:lCV0i+ZJPTbqP7LfKG7p3qZBl5VhelwUFCIVWl77fgk= github.com/alexliesenfeld/health v0.8.0 h1:lCV0i+ZJPTbqP7LfKG7p3qZBl5VhelwUFCIVWl77fgk=
github.com/alexliesenfeld/health v0.8.0/go.mod h1:TfNP0f+9WQVWMQRzvMUjlws4ceXKEL3WR+6Hp95HUFc= github.com/alexliesenfeld/health v0.8.0/go.mod h1:TfNP0f+9WQVWMQRzvMUjlws4ceXKEL3WR+6Hp95HUFc=
github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y= github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/andybalholm/brotli v1.0.6 h1:Yf9fFpf49Zrxb9NlQaluyE92/+X7UVHlhMNJN2sxfOI= github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1U3M=
github.com/andybalholm/brotli v1.0.6/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig= github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk= github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4= github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chromedp/cdproto v0.0.0-20231011050154-1d073bb38998/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs= github.com/chromedp/cdproto v0.0.0-20240202021202-6d0b6a386732/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs=
github.com/chromedp/cdproto v0.0.0-20231114014204-3e458d5176f9 h1:e3tMnG8i9SfKOilykpprojNk3a49O4dn+wqZsam1qYQ= github.com/chromedp/cdproto v0.0.0-20240312231614-1e5096e63154 h1:jeAmkzyOAQBPRmZMhX+i/CJv0VViLkHk1nF0qx8s0Mk=
github.com/chromedp/cdproto v0.0.0-20231114014204-3e458d5176f9/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs= github.com/chromedp/cdproto v0.0.0-20240312231614-1e5096e63154/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs=
github.com/chromedp/chromedp v0.9.3 h1:Wq58e0dZOdHsxaj9Owmfcf+ibtpYN1N0FWVbaxa/esg= github.com/chromedp/chromedp v0.9.5 h1:viASzruPJOiThk7c5bueOUY91jGLJVximoEMGoH93rg=
github.com/chromedp/chromedp v0.9.3/go.mod h1:NipeUkUcuzIdFbBP8eNNvl9upcceOfWzoJn6cRe4ksA= github.com/chromedp/chromedp v0.9.5/go.mod h1:D4I2qONslauw/C7INoCir1BJkSwBYMyZgx8X276z3+Y=
github.com/chromedp/sysutil v1.0.0 h1:+ZxhTpfpZlmchB58ih/LBHX52ky7w2VhQVKQMucy3Ic= github.com/chromedp/sysutil v1.0.0 h1:+ZxhTpfpZlmchB58ih/LBHX52ky7w2VhQVKQMucy3Ic=
github.com/chromedp/sysutil v1.0.0/go.mod h1:kgWmDdq8fTzXYcKIBqIYvRRTnYb9aNS9moAV0xufSww= github.com/chromedp/sysutil v1.0.0/go.mod h1:kgWmDdq8fTzXYcKIBqIYvRRTnYb9aNS9moAV0xufSww=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 h1:iFaUwBSo5Svw6L7HYpRu/0lE3e0BaElwnNO1qkNQxBY= github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 h1:iFaUwBSo5Svw6L7HYpRu/0lE3e0BaElwnNO1qkNQxBY=
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5/go.mod h1:qssHWj60/X5sZFNxpG4HBPDHVqxNm4DfnCKgrbZOT+s= github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5/go.mod h1:qssHWj60/X5sZFNxpG4HBPDHVqxNm4DfnCKgrbZOT+s=
github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY= github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY=
@@ -25,18 +27,16 @@ github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU
github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM= github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og= github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og=
github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw= github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw=
github.com/gobwas/ws v1.3.0/go.mod h1:hRKAFb8wOxFROYNsT1bqfWnhX+b5MFeJM9r2ZSwg/KY= github.com/gobwas/ws v1.3.2 h1:zlnbNHxumkRvfPWgfXu8RBwyNR1x8wh9cf5PTOCqs9Q=
github.com/gobwas/ws v1.3.1 h1:Qi34dfLMWJbiKaNbDVzM9x27nZBjmkaW6i4+Ku+pGVU= github.com/gobwas/ws v1.3.2/go.mod h1:hRKAFb8wOxFROYNsT1bqfWnhX+b5MFeJM9r2ZSwg/KY=
github.com/gobwas/ws v1.3.1/go.mod h1:hRKAFb8wOxFROYNsT1bqfWnhX+b5MFeJM9r2ZSwg/KY=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/snappy v0.0.2/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/golang/snappy v0.0.2/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM= github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=
github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8= github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8=
github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0= github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0=
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ= github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
@@ -53,8 +53,8 @@ github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8Hm
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A= github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.11.4/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= github.com/klauspost/compress v1.11.4/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.17.3 h1:qkRjuerhUU1EmXLYGkSH6EZL+vPSxIrYjLNAK4slzwA= github.com/klauspost/compress v1.17.7 h1:ehO88t2UGzQK66LMdE8tibEd1ErmzZjNEqWkjLAKQQg=
github.com/klauspost/compress v1.17.3/go.mod h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM= github.com/klauspost/compress v1.17.7/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw=
github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek= github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/klauspost/pgzip v1.2.5/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs= github.com/klauspost/pgzip v1.2.5/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU= github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU=
@@ -63,10 +63,10 @@ github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/labstack/echo/v4 v4.11.3 h1:Upyu3olaqSHkCjs1EJJwQ3WId8b8b1hxbogyommKktM= github.com/labstack/echo/v4 v4.11.4 h1:vDZmA+qNeh1pd/cCkEicDMrjtrnMGQ1QFI9gWN1zGq8=
github.com/labstack/echo/v4 v4.11.3/go.mod h1:UcGuQ8V6ZNRmSweBIJkPvGfwCMIlFmiqrPqiEBfPYws= github.com/labstack/echo/v4 v4.11.4/go.mod h1:noh7EvLwqDsmh/X/HWKPUl1AjzJrhyptRyEbQJfxen8=
github.com/labstack/gommon v0.4.1 h1:gqEff0p/hTENGMABzezPoPSRtIh1Cvw0ueMOe0/dfOk= github.com/labstack/gommon v0.4.2 h1:F8qTUNXgG1+6WQmqoUWnz8WiEU60mXVVw0P4ht1WRA0=
github.com/labstack/gommon v0.4.1/go.mod h1:TyTrpPqxR5KMk8LKVtLmfMjeQ5FEkBYdxLYPw/WfrOM= github.com/labstack/gommon v0.4.2/go.mod h1:QlUFxVM+SNXhDL/Z7YhocGIBYOiwB0mXm1+1bAPHPyU=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80 h1:6Yzfa6GP0rIo/kULo2bwGEkFvCePZ3qHDDTC3/J9Swo= github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80 h1:6Yzfa6GP0rIo/kULo2bwGEkFvCePZ3qHDDTC3/J9Swo=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs= github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs=
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
@@ -78,8 +78,6 @@ github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWE
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U= github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U=
github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 h1:jWpvCLoY8Z/e3VKvlsiIGKtc+UG6U5vzxaoagmhXfyg=
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0/go.mod h1:QUyp042oQthUoa9bqDv0ER0wrtXnBruoNd7aNjkbP+k=
github.com/mholt/archiver/v3 v3.5.1 h1:rDjOBX9JSF5BvoJGvjqK479aL70qh9DIpZCl+k7Clwo= github.com/mholt/archiver/v3 v3.5.1 h1:rDjOBX9JSF5BvoJGvjqK479aL70qh9DIpZCl+k7Clwo=
github.com/mholt/archiver/v3 v3.5.1/go.mod h1:e3dqJ7H78uzsRSEACH1joayhuSyhnonssnDhppzS1L4= github.com/mholt/archiver/v3 v3.5.1/go.mod h1:e3dqJ7H78uzsRSEACH1joayhuSyhnonssnDhppzS1L4=
github.com/microcosm-cc/bluemonday v1.0.26 h1:xbqSvqzQMeEHCqMi64VAs4d8uy6Mequs3rQ0k/Khz58= github.com/microcosm-cc/bluemonday v1.0.26 h1:xbqSvqzQMeEHCqMi64VAs4d8uy6Mequs3rQ0k/Khz58=
@@ -89,26 +87,26 @@ github.com/nwaples/rardecode v1.1.3 h1:cWCaZwfM5H7nAD6PyEdcVnczzV8i/JtotnyW/dD9l
github.com/nwaples/rardecode v1.1.3/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0= github.com/nwaples/rardecode v1.1.3/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw= github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0= github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
github.com/pdfcpu/pdfcpu v0.5.0 h1:F3wC4bwPbaJM+RPgm1D0Q4SAUwxElw7BhwNvL3iPgDo= github.com/pdfcpu/pdfcpu v0.7.0 h1:cd7/z7hAyyDuzdciKfNZyQ3TYreJza2DsuPdIHYURcA=
github.com/pdfcpu/pdfcpu v0.5.0/go.mod h1:UPcHdWcMw1V6Bo5tcWHd3jZfkG8cwUwrJkQOlB6o+7g= github.com/pdfcpu/pdfcpu v0.7.0/go.mod h1:kmpD0rk8YnZj0l3qSeGBlAB+XszHUgNv//ORH/E7EYo=
github.com/pierrec/lz4/v4 v4.1.2/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4= github.com/pierrec/lz4/v4 v4.1.2/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/pierrec/lz4/v4 v4.1.18 h1:xaKrnTkyoqfh1YItXl56+6KJNVYWlEEPuAQW9xsplYQ= github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ=
github.com/pierrec/lz4/v4 v4.1.18/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4= github.com/pierrec/lz4/v4 v4.1.21/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.17.0 h1:rl2sfwZMtSthVU752MqfjQozy7blglC+1SOtjMAMh+Q= github.com/prometheus/client_golang v1.19.0 h1:ygXvpU1AoN1MhdzckN+PyD9QJOSD4x7kmXYlnfbA6JU=
github.com/prometheus/client_golang v1.17.0/go.mod h1:VeL+gMmOAxkS2IqfCq0ZmHSL+LjWfWDUmp1mBz9JgUY= github.com/prometheus/client_golang v1.19.0/go.mod h1:ZRM9uEAypZakd+q/x7+gmsvXdURP+DABIEIjnmDdp+k=
github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw= github.com/prometheus/client_model v0.6.0 h1:k1v3CzpSRUTrKMppY35TLwPvxHqBu0bYgxZzqGIgaos=
github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI= github.com/prometheus/client_model v0.6.0/go.mod h1:NTQHnmxFpouOD0DpvP4XujX3CdOAGQPoaGhyTchlyt8=
github.com/prometheus/common v0.45.0 h1:2BGz0eBc2hdMDLnO/8n0jeB3oPrt2D08CekT0lneoxM= github.com/prometheus/common v0.50.0 h1:YSZE6aa9+luNa2da6/Tik0q0A5AbR+U003TItK57CPQ=
github.com/prometheus/common v0.45.0/go.mod h1:YJmSTw9BoKxJplESWWxlbyttQR4uaEcGyv9MZjVOJsY= github.com/prometheus/common v0.50.0/go.mod h1:wHFBCEVWVmHMUpg7pYcOm2QUR/ocQdYSJVQJKnHc3xQ=
github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo= github.com/prometheus/procfs v0.13.0 h1:GqzLlQyfsPbaEHaQkO7tbDlriv/4o5Hudv6OXHGKX7o=
github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo= github.com/prometheus/procfs v0.13.0/go.mod h1:cd4PFCR54QLnGKPaKGA6l+cfuNXtht43ZKY6tow0Y1g=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis= github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
@@ -130,32 +128,32 @@ github.com/valyala/fasttemplate v1.2.2 h1:lxLXG0uE3Qnshl9QyaK6XJxMXlQZELvChBOCmQ
github.com/valyala/fasttemplate v1.2.2/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ= github.com/valyala/fasttemplate v1.2.2/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 h1:nIPpBwaJSVYIxUFsDv3M8ofmx9yWTog9BfvIu0q41lo= github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 h1:nIPpBwaJSVYIxUFsDv3M8ofmx9yWTog9BfvIu0q41lo=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos= github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos=
go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.2.0/go.mod h1:XJYK+MuIchqpmGmUSAzotztawfKvYLUIgg7guXrwVUo= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo= go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so= go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
golang.org/x/crypto v0.15.0 h1:frVn1TEaCEaZcn3Tmd7Y2b5KKPaZ+I32Q2OA3kYp5TA= golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
golang.org/x/crypto v0.15.0/go.mod h1:4ChreQoLWfG3xLDer1WdlH5NdlQ3+mwnQq1YTKY+72g= golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
golang.org/x/image v0.14.0 h1:tNgSxAFe3jC4uYqvZdTr84SZoM1KfwdC9SKIFrLjFn4= golang.org/x/image v0.15.0 h1:kOELfmgrmJlw4Cdb7g/QGuB3CvDrXbqEIww/pNtNBm8=
golang.org/x/image v0.14.0/go.mod h1:HUYqC05R2ZcZ3ejNQsIHQDQiwWM4JBqmm6MKANTp4LE= golang.org/x/image v0.15.0/go.mod h1:HUYqC05R2ZcZ3ejNQsIHQDQiwWM4JBqmm6MKANTp4LE=
golang.org/x/net v0.18.0 h1:mIYleuAkSbHh0tCv7RvjL3F6ZVbLjq4+R7zbOn3Kokg= golang.org/x/net v0.22.0 h1:9sGLhx7iRIHEiX0oAJ3MRZMUCElJgy7Br1nO+AMN3Tc=
golang.org/x/net v0.18.0/go.mod h1:/czyP5RqHAH4odGYxBJ1qz0+CE5WZ+2j1YgoEo8F2jQ= golang.org/x/net v0.22.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE= golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.14.0 h1:Vz7Qs629MkJkGyHxUlRHizWJRG2j8fbQKjELVSNhy7Q= golang.org/x/sys v0.16.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/term v0.14.0 h1:LGK9IlZ8T9jvdy6cTdfKUCltatMFOehAQo9SRC46UQ8= golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.14.0/go.mod h1:TySc+nGkYR6qt8km8wUhuFRTVSMIX3XPR58y2lC8vww= golang.org/x/term v0.18.0 h1:FcHjZXDMxI8mM3nwhX9HlKop4C0YQvCVCdwYl2wOtE8=
golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=

53
pkg/gotenberg/filter.go Normal file
View File

@@ -0,0 +1,53 @@
package gotenberg
import (
"context"
"errors"
"fmt"
"time"
"github.com/dlclark/regexp2"
)
// ErrFiltered happens if a value is filtered by the [FilterDeadline] function.
var ErrFiltered = errors.New("value filtered")
// FilterDeadline checks if given value is allowed and not denied according to
// regex patterns. It returns a [context.DeadlineExceeded] if it takes too long
// to process.
func FilterDeadline(allowed, denied *regexp2.Regexp, s string, deadline time.Time) error {
// FIXME: not ideal to compile everytime, but is there another way to create a clone?
if allowed.String() != "" {
allow := regexp2.MustCompile(allowed.String(), 0)
allow.MatchTimeout = time.Until(deadline)
ok, err := allow.MatchString(s)
if err != nil {
if time.Now().After(deadline) {
return context.DeadlineExceeded
}
return fmt.Errorf("'%s' cannot handle '%s': %w", allow.String(), s, err)
}
if !ok {
return fmt.Errorf("'%s' does not match the expression from the allowed list: %w", s, ErrFiltered)
}
}
if denied.String() != "" {
deny := regexp2.MustCompile(denied.String(), 0)
deny.MatchTimeout = time.Until(deadline)
ok, err := deny.MatchString(s)
if err != nil {
if time.Now().After(deadline) {
return context.DeadlineExceeded
}
return fmt.Errorf("'%s' cannot handle '%s': %w", deny.String(), s, err)
}
if ok {
return fmt.Errorf("'%s' matches the expression from the denied list: %w", s, ErrFiltered)
}
}
return nil
}

View File

@@ -0,0 +1,83 @@
package gotenberg
import (
"context"
"errors"
"testing"
"time"
"github.com/dlclark/regexp2"
)
func TestFilterDeadline(t *testing.T) {
for _, tc := range []struct {
scenario string
allowed *regexp2.Regexp
denied *regexp2.Regexp
s string
deadline time.Time
expectError bool
expectedError error
}{
{
scenario: "DeadlineExceeded (allowed)",
allowed: regexp2.MustCompile("foo", 0),
denied: regexp2.MustCompile("", 0),
s: "foo",
deadline: time.Now().Add(time.Duration(-1) * time.Hour),
expectError: true,
expectedError: context.DeadlineExceeded,
},
{
scenario: "ErrFiltered (allowed)",
allowed: regexp2.MustCompile("foo", 0),
denied: regexp2.MustCompile("", 0),
s: "bar",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: true,
expectedError: ErrFiltered,
},
{
scenario: "DeadlineExceeded (denied)",
allowed: regexp2.MustCompile("", 0),
denied: regexp2.MustCompile("foo", 0),
s: "foo",
deadline: time.Now().Add(time.Duration(-1) * time.Hour),
expectError: true,
expectedError: context.DeadlineExceeded,
},
{
scenario: "ErrFiltered (denied)",
allowed: regexp2.MustCompile("", 0),
denied: regexp2.MustCompile("foo", 0),
s: "foo",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: true,
expectedError: ErrFiltered,
},
{
scenario: "success",
allowed: regexp2.MustCompile("", 0),
denied: regexp2.MustCompile("", 0),
s: "foo",
deadline: time.Now().Add(time.Duration(5) * time.Second),
expectError: false,
},
} {
t.Run(tc.scenario, func(t *testing.T) {
err := FilterDeadline(tc.allowed, tc.denied, tc.s, tc.deadline)
if tc.expectError && err == nil {
t.Fatal("expected an error but got none")
}
if !tc.expectError && err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if tc.expectedError != nil && !errors.Is(err, tc.expectedError) {
t.Fatalf("expected error %v but got: %v", tc.expectedError, err)
}
})
}
}

View File

@@ -1,9 +1,9 @@
package gotenberg package gotenberg
import ( import (
"regexp"
"time" "time"
"github.com/dlclark/regexp2"
"github.com/labstack/gommon/bytes" "github.com/labstack/gommon/bytes"
flag "github.com/spf13/pflag" flag "github.com/spf13/pflag"
) )
@@ -199,19 +199,19 @@ func (f *ParsedFlags) MustDeprecatedHumanReadableBytesString(deprecated string,
// MustRegexp returns the regular expression of a flag given by name. // MustRegexp returns the regular expression of a flag given by name.
// It panics if an error occurs. // It panics if an error occurs.
func (f *ParsedFlags) MustRegexp(name string) *regexp.Regexp { func (f *ParsedFlags) MustRegexp(name string) *regexp2.Regexp {
val, err := f.GetString(name) val, err := f.GetString(name)
if err != nil { if err != nil {
panic(err) panic(err)
} }
return regexp.MustCompile(val) return regexp2.MustCompile(val, 0)
} }
// MustDeprecatedRegexp returns the regular expression of a deprecated flag if // MustDeprecatedRegexp returns the regular expression of a deprecated flag if
// it was explicitly set or the regular expression of the new flag. // it was explicitly set or the regular expression of the new flag.
// It panics if an error occurs. // It panics if an error occurs.
func (f *ParsedFlags) MustDeprecatedRegexp(deprecated string, newName string) *regexp.Regexp { func (f *ParsedFlags) MustDeprecatedRegexp(deprecated string, newName string) *regexp2.Regexp {
if f.Changed(deprecated) { if f.Changed(deprecated) {
return f.MustRegexp(deprecated) return f.MustRegexp(deprecated)
} }

View File

@@ -51,3 +51,11 @@ func (fs *FileSystem) MkdirAll() (string, error) {
return path, nil return path, nil
} }
// PathRename defines the method signature for renaming files. Implement this
// interface if you don't want to rely on [os.Rename], notably for testing
// purpose.
type PathRename interface {
// Rename uses the same signature as [os.Rename].
Rename(oldpath, newpath string) error
}

View File

@@ -127,6 +127,15 @@ func (provider *MetricsProviderMock) Metrics() ([]Metric, error) {
return provider.MetricsMock() return provider.MetricsMock()
} }
// PathRenameMock is a mock for the [PathRename] interface.
type PathRenameMock struct {
RenameMock func(oldpath, newpath string) error
}
func (rename *PathRenameMock) Rename(oldpath, newpath string) error {
return rename.RenameMock(oldpath, newpath)
}
// Interface guards. // Interface guards.
var ( var (
_ Module = (*ModuleMock)(nil) _ Module = (*ModuleMock)(nil)

View File

@@ -188,3 +188,16 @@ func TestMetricsProviderMock(t *testing.T) {
t.Errorf("expected no error from MetricsProviderMock.Metrics, but got: %v", err) t.Errorf("expected no error from MetricsProviderMock.Metrics, but got: %v", err)
} }
} }
func TestPathRenameMock(t *testing.T) {
mock := &PathRenameMock{
RenameMock: func(oldpath, newpath string) error {
return nil
},
}
err := mock.Rename("", "")
if err != nil {
t.Errorf("expected no error from PathRenameMock.Rename, but got: %v", err)
}
}

View File

@@ -83,7 +83,7 @@ type SystemLogger interface {
// gotenberg.MustRegisterModule(YourModule{}) // gotenberg.MustRegisterModule(YourModule{})
// } // }
// //
// Then, in the main command (github.com/gotenberg/gotenberg/v7/cmd/gotenberg), // Then, in the main command (github.com/gotenberg/gotenberg/v8/cmd/gotenberg),
// import the module: // import the module:
// //
// imports ( // imports (

69
pkg/gotenberg/sort.go Normal file
View File

@@ -0,0 +1,69 @@
package gotenberg
import (
"regexp"
"sort"
"strconv"
)
// AlphanumericSort implements sort.Interface and helps to sort strings
// alphanumerically.
//
// See https://github.com/gotenberg/gotenberg/issues/805.
type AlphanumericSort []string
func (s AlphanumericSort) Len() int {
return len(s)
}
func (s AlphanumericSort) Swap(i, j int) {
s[i], s[j] = s[j], s[i]
}
func (s AlphanumericSort) Less(i, j int) bool {
numI, restI := extractPrefix(s[i])
numJ, restJ := extractPrefix(s[j])
// Compares numerical prefixes if they exist.
if numI != -1 && numJ != -1 {
if numI != numJ {
return numI < numJ
}
// If numbers are equal, falls back to string comparison of the rest.
return restI < restJ
}
// If one has a numerical prefix and the other doesn't, the one with the
// number comes first.
if numI != -1 {
return true
}
if numJ != -1 {
return false
}
// If neither has a numerical prefix, compare as strings
return s[i] < s[j]
}
// extractPrefix attempts to extract a numerical prefix and the rest of the filename
func extractPrefix(filename string) (int, string) {
matches := numPrefixRegexp.FindStringSubmatch(filename)
if len(matches) > 2 {
prefix, err := strconv.Atoi(matches[1])
if err == nil {
return prefix, matches[2]
}
}
// Returns -1 if no numerical prefix is found, indicating to just compare
// as strings.
return -1, filename
}
var numPrefixRegexp = regexp.MustCompile(`^(\d+)(.*)$`)
// Interface guard.
var (
_ sort.Interface = (*AlphanumericSort)(nil)
)

View File

@@ -0,0 +1,34 @@
package gotenberg
import (
"reflect"
"sort"
"testing"
)
func TestAlphanumericSort(t *testing.T) {
for _, tc := range []struct {
scenario string
values []string
expectedSort []string
}{
{
scenario: "numeric and letters",
values: []string{"10qux.pdf", "2_baz.txt", "2_aza.txt", "1bar.pdf", "Afoo.txt", "Bbar.docx", "25zeta.txt", "3.pdf", "4_foo.pdf"},
expectedSort: []string{"1bar.pdf", "2_aza.txt", "2_baz.txt", "3.pdf", "4_foo.pdf", "10qux.pdf", "25zeta.txt", "Afoo.txt", "Bbar.docx"},
},
{
scenario: "hrtime (PHP library)",
values: []string{"245654773395259", "245654773395039", "245654773395149", "245654773394919", "245654773394369"},
expectedSort: []string{"245654773394369", "245654773394919", "245654773395039", "245654773395149", "245654773395259"},
},
} {
t.Run(tc.scenario, func(t *testing.T) {
sort.Sort(AlphanumericSort(tc.values))
if !reflect.DeepEqual(tc.values, tc.expectedSort) {
t.Fatalf("expected %+v but got: %+v", tc.expectedSort, tc.values)
}
})
}
}

View File

@@ -2,12 +2,21 @@ package gotenberg
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"sync/atomic" "sync/atomic"
"go.uber.org/zap" "go.uber.org/zap"
) )
// ErrProcessAlreadyRestarting happens if the [ProcessSupervisor] is trying
// to restart an already restarting [Process].
var ErrProcessAlreadyRestarting = errors.New("process already restarting")
// ErrMaximumQueueSizeExceeded happens if Run() is called but the maximum queue
// size is already used.
var ErrMaximumQueueSizeExceeded = errors.New("maximum queue size exceeded")
// Process is an interface that represents an abstract process // Process is an interface that represents an abstract process
// and provides methods for starting, stopping, and checking the health of the // and provides methods for starting, stopping, and checking the health of the
// process. // process.
@@ -69,6 +78,7 @@ type processSupervisor struct {
logger *zap.Logger logger *zap.Logger
process Process process Process
maxReqLimit int64 maxReqLimit int64
maxQueueSize int64
mutexChan chan struct{} mutexChan chan struct{}
firstStart atomic.Bool firstStart atomic.Bool
reqCounter atomic.Int64 reqCounter atomic.Int64
@@ -78,12 +88,13 @@ type processSupervisor struct {
} }
// NewProcessSupervisor initializes a new [ProcessSupervisor]. // NewProcessSupervisor initializes a new [ProcessSupervisor].
func NewProcessSupervisor(logger *zap.Logger, process Process, maxReqLimit int64) ProcessSupervisor { func NewProcessSupervisor(logger *zap.Logger, process Process, maxReqLimit, maxQueueSize int64) ProcessSupervisor {
b := &processSupervisor{ b := &processSupervisor{
logger: logger, logger: logger,
process: process, process: process,
mutexChan: make(chan struct{}, 1), mutexChan: make(chan struct{}, 1),
maxReqLimit: maxReqLimit, maxReqLimit: maxReqLimit,
maxQueueSize: maxQueueSize,
} }
b.reqCounter.Store(0) b.reqCounter.Store(0)
b.reqQueueSize.Store(0) b.reqQueueSize.Store(0)
@@ -122,7 +133,7 @@ func (s *processSupervisor) restart() error {
if s.isRestarting.Load() { if s.isRestarting.Load() {
s.logger.Debug("process already restarting, skip restart") s.logger.Debug("process already restarting, skip restart")
return nil return ErrProcessAlreadyRestarting
} }
s.logger.Debug("restart process") s.logger.Debug("restart process")
@@ -162,55 +173,73 @@ func (s *processSupervisor) Healthy() bool {
} }
func (s *processSupervisor) Run(ctx context.Context, logger *zap.Logger, task func() error) error { func (s *processSupervisor) Run(ctx context.Context, logger *zap.Logger, task func() error) error {
currentQueueSize := s.reqQueueSize.Load()
if s.maxQueueSize > 0 && currentQueueSize >= s.maxQueueSize {
return ErrMaximumQueueSizeExceeded
}
s.reqQueueSize.Add(1) s.reqQueueSize.Add(1)
select { for {
case s.mutexChan <- struct{}{}: err := func() error {
logger.Debug("process lock acquired") select {
s.reqQueueSize.Add(-1) case s.mutexChan <- struct{}{}:
s.reqCounter.Add(1) logger.Debug("process lock acquired")
s.reqQueueSize.Add(-1)
s.reqCounter.Add(1)
defer func() { defer func() {
logger.Debug("process lock released") logger.Debug("process lock released")
<-s.mutexChan <-s.mutexChan
}()
if !s.firstStart.Load() {
err := s.runWithDeadline(ctx, func() error {
return s.Launch()
})
if err != nil {
return fmt.Errorf("process first start: %w", err)
}
}
if !s.Healthy() {
s.logger.Debug("process is unhealthy, cannot handle task, restarting...")
err := s.runWithDeadline(ctx, func() error {
return s.restart()
})
if err != nil {
return fmt.Errorf("process restart before task: %w", err)
}
}
if s.maxReqLimit > 0 && s.reqCounter.Load() >= s.maxReqLimit {
s.logger.Debug("max request limit reached, restarting...")
err := s.runWithDeadline(ctx, func() error {
return s.restart()
})
if err != nil {
return fmt.Errorf("process restart before task: %w", err)
}
}
// Note: no error wrapping because it leaks on Chromium console exceptions output.
return s.runWithDeadline(ctx, task)
case <-ctx.Done():
logger.Debug("failed to acquire process lock before deadline")
s.reqQueueSize.Add(-1)
return fmt.Errorf("acquire process lock: %w", ctx.Err())
}
}() }()
if !s.firstStart.Load() { if errors.Is(err, ErrProcessAlreadyRestarting) {
err := s.runWithDeadline(ctx, func() error { logger.Debug("process is already restarting, trying to acquire process lock again...")
return s.Launch() s.reqQueueSize.Add(1)
}) continue
if err != nil {
return fmt.Errorf("process first start: %w", err)
}
} }
if !s.Healthy() { // Note: no error wrapping because it leaks on Chromium console exceptions output.
s.logger.Debug("process is unhealthy, cannot handle task, restarting...") return err
err := s.runWithDeadline(ctx, func() error {
return s.restart()
})
if err != nil {
return fmt.Errorf("process restart before task: %w", err)
}
}
if s.maxReqLimit > 0 && s.reqCounter.Load() >= s.maxReqLimit {
s.logger.Debug("max request limit reached, restarting...")
err := s.runWithDeadline(ctx, func() error {
return s.restart()
})
if err != nil {
return fmt.Errorf("process restart before task: %w", err)
}
}
// FIXME: no error wrapping because it leaks on Chromium console exceptions output.
return s.runWithDeadline(ctx, task)
case <-ctx.Done():
logger.Debug("failed to acquire process lock before deadline")
s.reqQueueSize.Add(-1)
return fmt.Errorf("acquire process lock: %w", ctx.Err())
} }
} }

View File

@@ -46,7 +46,7 @@ func TestProcessSupervisor_Launch(t *testing.T) {
}, },
} }
ps := NewProcessSupervisor(logger, process, 5).(*processSupervisor) ps := NewProcessSupervisor(logger, process, 5, 0).(*processSupervisor)
if tc.firstStartSet { if tc.firstStartSet {
ps.firstStart.Store(true) ps.firstStart.Store(true)
} }
@@ -94,7 +94,7 @@ func TestProcessSupervisor_Shutdown(t *testing.T) {
}, },
} }
ps := NewProcessSupervisor(logger, process, 5) ps := NewProcessSupervisor(logger, process, 5, 0)
err := ps.Shutdown() err := ps.Shutdown()
if !tc.expectError && err != nil { if !tc.expectError && err != nil {
@@ -115,11 +115,13 @@ func TestProcessSupervisor_restart(t *testing.T) {
startError error startError error
stopError error stopError error
expectError bool expectError bool
expectedError error
}{ }{
{ {
scenario: "already restarting", scenario: "already restarting",
initiallyRestarting: true, initiallyRestarting: true,
expectError: false, expectError: true,
expectedError: ErrProcessAlreadyRestarting,
}, },
{ {
scenario: "successful restart", scenario: "successful restart",
@@ -152,7 +154,7 @@ func TestProcessSupervisor_restart(t *testing.T) {
}, },
} }
ps := NewProcessSupervisor(logger, process, 5).(*processSupervisor) ps := NewProcessSupervisor(logger, process, 5, 0).(*processSupervisor)
if tc.initiallyRestarting { if tc.initiallyRestarting {
ps.isRestarting.Store(true) ps.isRestarting.Store(true)
} }
@@ -166,6 +168,10 @@ func TestProcessSupervisor_restart(t *testing.T) {
if tc.expectError && err == nil { if tc.expectError && err == nil {
t.Fatal("expected error but got none") t.Fatal("expected error but got none")
} }
if tc.expectedError != nil && !errors.Is(err, tc.expectedError) {
t.Fatalf("expected error %v but got: %v", tc.expectedError, err)
}
}) })
} }
} }
@@ -211,7 +217,7 @@ func TestProcessSupervisor_Healthy(t *testing.T) {
}, },
} }
ps := NewProcessSupervisor(logger, process, 5).(*processSupervisor) ps := NewProcessSupervisor(logger, process, 5, 0).(*processSupervisor)
if tc.initiallyStarted { if tc.initiallyStarted {
ps.firstStart.Store(true) ps.firstStart.Store(true)
} }
@@ -232,19 +238,24 @@ func TestProcessSupervisor_Run(t *testing.T) {
for _, tc := range []struct { for _, tc := range []struct {
scenario string scenario string
initiallyStarted bool initiallyStarted bool
isRestarting bool
startError error startError error
processHealthy bool processHealthy bool
maxReqLimit int64 maxReqLimit int64
tasksToRun int tasksToRun int
taskError error taskError error
expectError bool expectError bool
skipCallsCheck bool
expectedStartCalls int64 expectedStartCalls int64
expectedHealthyCalls int64 expectedHealthyCalls int64
expectedStopCalls int64 expectedStopCalls int64
currentQueueSize int64
maxQueueSize int64
}{ }{
{ {
scenario: "successfully run task on non-started process", scenario: "successfully run task on non-started process",
initiallyStarted: false, initiallyStarted: false,
isRestarting: false,
processHealthy: true, processHealthy: true,
maxReqLimit: 2, maxReqLimit: 2,
tasksToRun: 1, tasksToRun: 1,
@@ -256,6 +267,7 @@ func TestProcessSupervisor_Run(t *testing.T) {
{ {
scenario: "cannot launch non-started process", scenario: "cannot launch non-started process",
initiallyStarted: false, initiallyStarted: false,
isRestarting: false,
startError: errors.New("launch error"), startError: errors.New("launch error"),
processHealthy: true, processHealthy: true,
maxReqLimit: 2, maxReqLimit: 2,
@@ -268,6 +280,7 @@ func TestProcessSupervisor_Run(t *testing.T) {
{ {
scenario: "run task with unhealthy process causing restart", scenario: "run task with unhealthy process causing restart",
initiallyStarted: true, initiallyStarted: true,
isRestarting: false,
processHealthy: false, processHealthy: false,
maxReqLimit: 2, maxReqLimit: 2,
tasksToRun: 1, tasksToRun: 1,
@@ -280,6 +293,7 @@ func TestProcessSupervisor_Run(t *testing.T) {
scenario: "cannot restart unhealthy process", scenario: "cannot restart unhealthy process",
startError: errors.New("start error"), startError: errors.New("start error"),
initiallyStarted: true, initiallyStarted: true,
isRestarting: false,
processHealthy: false, processHealthy: false,
maxReqLimit: 2, maxReqLimit: 2,
tasksToRun: 1, tasksToRun: 1,
@@ -288,9 +302,20 @@ func TestProcessSupervisor_Run(t *testing.T) {
expectedHealthyCalls: 1, expectedHealthyCalls: 1,
expectedStopCalls: 1, expectedStopCalls: 1,
}, },
{
scenario: "ErrProcessAlreadyRestarting",
initiallyStarted: true,
isRestarting: true,
processHealthy: false,
maxReqLimit: 1,
tasksToRun: 1,
expectError: true,
skipCallsCheck: true,
},
{ {
scenario: "run tasks reaching max request limit causing restart", scenario: "run tasks reaching max request limit causing restart",
initiallyStarted: true, initiallyStarted: true,
isRestarting: false,
processHealthy: true, processHealthy: true,
maxReqLimit: 2, maxReqLimit: 2,
tasksToRun: 3, tasksToRun: 3,
@@ -303,6 +328,7 @@ func TestProcessSupervisor_Run(t *testing.T) {
scenario: "cannot restart after reaching max request limit", scenario: "cannot restart after reaching max request limit",
startError: errors.New("start error"), startError: errors.New("start error"),
initiallyStarted: true, initiallyStarted: true,
isRestarting: false,
processHealthy: true, processHealthy: true,
maxReqLimit: 2, maxReqLimit: 2,
tasksToRun: 2, tasksToRun: 2,
@@ -314,6 +340,7 @@ func TestProcessSupervisor_Run(t *testing.T) {
{ {
scenario: "task error", scenario: "task error",
initiallyStarted: true, initiallyStarted: true,
isRestarting: false,
processHealthy: true, processHealthy: true,
maxReqLimit: 0, maxReqLimit: 0,
tasksToRun: 1, tasksToRun: 1,
@@ -323,6 +350,34 @@ func TestProcessSupervisor_Run(t *testing.T) {
expectedHealthyCalls: 1, expectedHealthyCalls: 1,
expectedStopCalls: 0, expectedStopCalls: 0,
}, },
{
scenario: "queue size exceeded",
initiallyStarted: false,
isRestarting: false,
processHealthy: true,
maxReqLimit: 2,
tasksToRun: 1,
expectError: true,
expectedStartCalls: 0,
expectedHealthyCalls: 0,
expectedStopCalls: 0,
currentQueueSize: 1,
maxQueueSize: 1,
},
{
scenario: "queue size not exceeded",
initiallyStarted: false,
isRestarting: false,
processHealthy: true,
maxReqLimit: 2,
tasksToRun: 1,
expectError: true,
expectedStartCalls: 1,
expectedHealthyCalls: 1,
expectedStopCalls: 0,
currentQueueSize: 1,
maxQueueSize: 2,
},
} { } {
t.Run(tc.scenario, func(t *testing.T) { t.Run(tc.scenario, func(t *testing.T) {
logger := zap.NewNop() logger := zap.NewNop()
@@ -347,10 +402,16 @@ func TestProcessSupervisor_Run(t *testing.T) {
}, },
} }
ps := NewProcessSupervisor(logger, process, tc.maxReqLimit).(*processSupervisor) ps := NewProcessSupervisor(logger, process, tc.maxReqLimit, tc.maxQueueSize).(*processSupervisor)
if tc.initiallyStarted { if tc.initiallyStarted {
ps.firstStart.Store(true) ps.firstStart.Store(true)
} }
if tc.isRestarting {
ps.isRestarting.Store(true)
}
if tc.currentQueueSize > 0 {
ps.reqQueueSize.Store(tc.currentQueueSize)
}
task := func() error { task := func() error {
return tc.taskError return tc.taskError
@@ -386,6 +447,10 @@ func TestProcessSupervisor_Run(t *testing.T) {
} }
} }
if tc.skipCallsCheck {
return
}
if startCalls.Load() != tc.expectedStartCalls { if startCalls.Load() != tc.expectedStartCalls {
t.Errorf("expected %d process.Start calls, got %d", tc.expectedStartCalls, startCalls.Load()) t.Errorf("expected %d process.Start calls, got %d", tc.expectedStartCalls, startCalls.Load())
} }
@@ -419,13 +484,13 @@ func TestProcessSupervisor_runWithDeadline(t *testing.T) {
}, },
} { } {
t.Run(tc.scenario, func(t *testing.T) { t.Run(tc.scenario, func(t *testing.T) {
ps := NewProcessSupervisor(zap.NewNop(), new(ProcessMock), 0).(*processSupervisor) ps := NewProcessSupervisor(zap.NewNop(), new(ProcessMock), 0, 0).(*processSupervisor)
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second) ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()
if tc.ctxDone { if tc.ctxDone {
cancel() cancel()
} else {
defer cancel()
} }
err := ps.runWithDeadline(ctx, func() error { err := ps.runWithDeadline(ctx, func() error {
@@ -453,7 +518,7 @@ func TestProcessSupervisor_ReqQueueSize(t *testing.T) {
return true return true
}, },
} }
ps := NewProcessSupervisor(logger, process, 0).(*processSupervisor) ps := NewProcessSupervisor(logger, process, 0, 0).(*processSupervisor)
// Simulating a lock. // Simulating a lock.
ps.mutexChan <- struct{}{} ps.mutexChan <- struct{}{}
@@ -554,7 +619,7 @@ func TestProcessSupervisor_RestartsCount(t *testing.T) {
}, },
} }
ps := NewProcessSupervisor(logger, process, 0).(*processSupervisor) ps := NewProcessSupervisor(logger, process, 0, 0).(*processSupervisor)
ps.restartsCounter.Store(tc.initialRestartsCount) ps.restartsCounter.Store(tc.initialRestartsCount)
for i := 0; i < tc.restartAttempts; i++ { for i := 0; i < tc.restartAttempts; i++ {

View File

@@ -17,8 +17,9 @@ import (
"go.uber.org/multierr" "go.uber.org/multierr"
"go.uber.org/zap" "go.uber.org/zap"
"golang.org/x/net/http2" "golang.org/x/net/http2"
"golang.org/x/sync/errgroup"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func init() { func init() {
@@ -29,8 +30,7 @@ func init() {
// middlewares or health checks. // middlewares or health checks.
type Api struct { type Api struct {
port int port int
readTimeout time.Duration startTimeout time.Duration
writeTimeout time.Duration
timeout time.Duration timeout time.Duration
rootPath string rootPath string
traceHeader string traceHeader string
@@ -39,6 +39,7 @@ type Api struct {
routes []Route routes []Route
externalMiddlewares []Middleware externalMiddlewares []Middleware
healthChecks []health.CheckerOption healthChecks []health.CheckerOption
readyFn []func() error
fs *gotenberg.FileSystem fs *gotenberg.FileSystem
logger *zap.Logger logger *zap.Logger
srv *echo.Echo srv *echo.Echo
@@ -147,6 +148,7 @@ type Middleware struct {
// See https://github.com/alexliesenfeld/health for more details. // See https://github.com/alexliesenfeld/health for more details.
type HealthChecker interface { type HealthChecker interface {
Checks() ([]health.CheckerOption, error) Checks() ([]health.CheckerOption, error)
Ready() error
} }
// Descriptor returns an [Api]'s module descriptor. // Descriptor returns an [Api]'s module descriptor.
@@ -157,23 +159,12 @@ func (a *Api) Descriptor() gotenberg.ModuleDescriptor {
fs := flag.NewFlagSet("api", flag.ExitOnError) fs := flag.NewFlagSet("api", flag.ExitOnError)
fs.Int("api-port", 3000, "Set the port on which the API should listen") fs.Int("api-port", 3000, "Set the port on which the API should listen")
fs.String("api-port-from-env", "", "Set the environment variable with the port on which the API should listen - override the default port") fs.String("api-port-from-env", "", "Set the environment variable with the port on which the API should listen - override the default port")
fs.Duration("api-read-timeout", time.Duration(30)*time.Second, "Set the maximum duration allowed to read a complete request, including the body") fs.Duration("api-start-timeout", time.Duration(30)*time.Second, "Set the time limit for the API to start")
fs.Duration("api-process-timeout", time.Duration(30)*time.Second, "Set the maximum duration allowed to process a request")
fs.Duration("api-write-timeout", time.Duration(30)*time.Second, "Set the maximum duration before timing out writes of the response")
fs.Duration("api-timeout", time.Duration(30)*time.Second, "Set the time limit for requests") fs.Duration("api-timeout", time.Duration(30)*time.Second, "Set the time limit for requests")
fs.String("api-root-path", "/", "Set the root path of the API - for service discovery via URL paths") fs.String("api-root-path", "/", "Set the root path of the API - for service discovery via URL paths")
fs.String("api-trace-header", "Gotenberg-Trace", "Set the header name to use for identifying requests") fs.String("api-trace-header", "Gotenberg-Trace", "Set the header name to use for identifying requests")
fs.Bool("api-disable-health-check-logging", false, "Disable health check logging") fs.Bool("api-disable-health-check-logging", false, "Disable health check logging")
var err error
err = multierr.Append(err, fs.MarkDeprecated("api-read-timeout", "use api-timeout instead"))
err = multierr.Append(err, fs.MarkDeprecated("api-process-timeout", "use api-timeout instead"))
err = multierr.Append(err, fs.MarkDeprecated("api-write-timeout", "use api-timeout instead"))
if err != nil {
panic(fmt.Errorf("create deprecated flags for the api module: %v", err))
}
return fs return fs
}(), }(),
New: func() gotenberg.Module { return new(Api) }, New: func() gotenberg.Module { return new(Api) },
@@ -184,9 +175,8 @@ func (a *Api) Descriptor() gotenberg.ModuleDescriptor {
func (a *Api) Provision(ctx *gotenberg.Context) error { func (a *Api) Provision(ctx *gotenberg.Context) error {
flags := ctx.ParsedFlags() flags := ctx.ParsedFlags()
a.port = flags.MustInt("api-port") a.port = flags.MustInt("api-port")
a.readTimeout = flags.MustDeprecatedDuration("api-read-timeout", "api-timeout") a.startTimeout = flags.MustDuration("api-start-timeout")
a.writeTimeout = flags.MustDeprecatedDuration("api-write-timeout", "api-timeout") a.timeout = flags.MustDuration("api-timeout")
a.timeout = flags.MustDeprecatedDuration("api-process-timeout", "api-timeout")
a.rootPath = flags.MustString("api-root-path") a.rootPath = flags.MustString("api-root-path")
a.traceHeader = flags.MustString("api-trace-header") a.traceHeader = flags.MustString("api-trace-header")
a.disableHealthCheckLogging = flags.MustBool("api-disable-health-check-logging") a.disableHealthCheckLogging = flags.MustBool("api-disable-health-check-logging")
@@ -275,6 +265,7 @@ func (a *Api) Provision(ctx *gotenberg.Context) error {
} }
a.healthChecks = append(a.healthChecks, checks...) a.healthChecks = append(a.healthChecks, checks...)
a.readyFn = append(a.readyFn, healthChecker.Ready)
} }
// Logger. // Logger.
@@ -373,10 +364,10 @@ func (a *Api) Start() error {
a.srv = echo.New() a.srv = echo.New()
a.srv.HideBanner = true a.srv.HideBanner = true
a.srv.HidePort = true a.srv.HidePort = true
a.srv.Server.ReadTimeout = a.readTimeout a.srv.Server.ReadTimeout = a.timeout
a.srv.Server.IdleTimeout = a.timeout a.srv.Server.IdleTimeout = a.timeout
// See https://github.com/gotenberg/gotenberg/issues/396. // See https://github.com/gotenberg/gotenberg/issues/396.
a.srv.Server.WriteTimeout = a.writeTimeout + a.writeTimeout a.srv.Server.WriteTimeout = a.timeout + a.timeout
a.srv.HTTPErrorHandler = httpErrorHandler() a.srv.HTTPErrorHandler = httpErrorHandler()
// Let's prepare the modules' routes. // Let's prepare the modules' routes.
@@ -446,12 +437,25 @@ func (a *Api) Start() error {
func() echo.HandlerFunc { func() echo.HandlerFunc {
checks := append(a.healthChecks, health.WithTimeout(a.timeout)) checks := append(a.healthChecks, health.WithTimeout(a.timeout))
checker := health.NewChecker(checks...) checker := health.NewChecker(checks...)
return echo.WrapHandler(health.NewHandler(checker)) return echo.WrapHandler(health.NewHandler(checker))
}(), }(),
hardTimeoutMiddleware(hardTimeout), hardTimeoutMiddleware(hardTimeout),
) )
// Wait for all modules to be ready.
ctx, cancel := context.WithTimeout(context.Background(), a.startTimeout)
defer cancel()
eg, _ := errgroup.WithContext(ctx)
for _, f := range a.readyFn {
eg.Go(f)
}
err := eg.Wait()
if err != nil {
return fmt.Errorf("waiting for modules readiness: %w", err)
}
// As the following code is blocking, run it in a goroutine. // As the following code is blocking, run it in a goroutine.
go func() { go func() {
server := &http2.Server{} server := &http2.Server{}

View File

@@ -10,12 +10,13 @@ import (
"os" "os"
"reflect" "reflect"
"testing" "testing"
"time"
"github.com/alexliesenfeld/health" "github.com/alexliesenfeld/health"
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestApi_Descriptor(t *testing.T) { func TestApi_Descriptor(t *testing.T) {
@@ -222,9 +223,6 @@ func TestApi_Provision(t *testing.T) {
mod.ValidateMock = func() error { mod.ValidateMock = func() error {
return errors.New("foo") return errors.New("foo")
} }
mod.ChecksMock = func() ([]health.CheckerOption, error) {
return nil, nil
}
return gotenberg.NewContext( return gotenberg.NewContext(
gotenberg.ParsedFlags{ gotenberg.ParsedFlags{
FlagSet: new(Api).Descriptor().FlagSet, FlagSet: new(Api).Descriptor().FlagSet,
@@ -347,6 +345,9 @@ func TestApi_Provision(t *testing.T) {
mod3.ChecksMock = func() ([]health.CheckerOption, error) { mod3.ChecksMock = func() ([]health.CheckerOption, error) {
return []health.CheckerOption{health.WithDisabledAutostart()}, nil return []health.CheckerOption{health.WithDisabledAutostart()}, nil
} }
mod3.ReadyMock = func() error {
return nil
}
mod4 := &struct { mod4 := &struct {
gotenberg.ModuleMock gotenberg.ModuleMock
@@ -643,141 +644,176 @@ func TestApi_Validate(t *testing.T) {
} }
func TestApi_Start(t *testing.T) { func TestApi_Start(t *testing.T) {
mod := new(Api) for _, tc := range []struct {
mod.port = 3000 scenario string
mod.rootPath = "/" readyFn []func() error
mod.disableHealthCheckLogging = true expectError bool
mod.routes = []Route{ }{
{ {
Method: http.MethodPost, scenario: "at least one module not ready",
Path: "/forms/foo", readyFn: []func() error{
IsMultipart: true, func() error { return nil },
DisableLogging: true, func() error { return errors.New("not ready") },
Handler: func(c echo.Context) error {
ctx := c.Get("context").(*Context)
ctx.outputPaths = []string{
"/tests/test/testdata/api/sample1.txt",
}
return nil
}, },
expectError: true,
}, },
{ {
Method: http.MethodPost, scenario: "success",
Path: "/forms/bar", readyFn: []func() error{
IsMultipart: true, func() error { return nil },
Handler: func(_ echo.Context) error { return errors.New("foo") }, func() error { return nil },
},
expectError: false,
}, },
} } {
mod.externalMiddlewares = []Middleware{ t.Run(tc.scenario, func(t *testing.T) {
{ mod := new(Api)
Stack: PreRouterStack, mod.port = 3000
Handler: func() echo.MiddlewareFunc { mod.startTimeout = time.Duration(30) * time.Second
return func(next echo.HandlerFunc) echo.HandlerFunc { mod.rootPath = "/"
return func(c echo.Context) error { mod.disableHealthCheckLogging = true
return next(c) mod.routes = []Route{
} {
} Method: http.MethodPost,
}(), Path: "/forms/foo",
}, IsMultipart: true,
{ DisableLogging: true,
Stack: MultipartStack, Handler: func(c echo.Context) error {
Handler: func() echo.MiddlewareFunc { ctx := c.Get("context").(*Context)
return func(next echo.HandlerFunc) echo.HandlerFunc { ctx.outputPaths = []string{
return func(c echo.Context) error { "/tests/test/testdata/api/sample1.txt",
return next(c) }
}
}
}(),
},
{
Stack: DefaultStack,
Handler: func() echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return next(c)
}
}
}(),
},
{
Handler: func() echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return next(c)
}
}
}(),
},
}
mod.fs = gotenberg.NewFileSystem()
mod.logger = zap.NewNop()
err := mod.Start() return nil
if err != nil { },
t.Fatalf("expected no error but got: %v", err) },
} {
Method: http.MethodPost,
Path: "/forms/bar",
IsMultipart: true,
Handler: func(_ echo.Context) error { return errors.New("foo") },
},
}
mod.externalMiddlewares = []Middleware{
{
Stack: PreRouterStack,
Handler: func() echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return next(c)
}
}
}(),
},
{
Stack: MultipartStack,
Handler: func() echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return next(c)
}
}
}(),
},
{
Stack: DefaultStack,
Handler: func() echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return next(c)
}
}
}(),
},
{
Handler: func() echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
return next(c)
}
}
}(),
},
}
mod.readyFn = tc.readyFn
mod.fs = gotenberg.NewFileSystem()
mod.logger = zap.NewNop()
// health request. err := mod.Start()
recorder := httptest.NewRecorder() if !tc.expectError && err != nil {
healthRequest := httptest.NewRequest(http.MethodGet, "/health", nil)
mod.srv.ServeHTTP(recorder, healthRequest)
if recorder.Code != http.StatusOK {
t.Errorf("expected %d status code but got %d", http.StatusOK, recorder.Code)
}
// "multipart/form-data" request.
multipartRequest := func(url string) *http.Request {
body := &bytes.Buffer{}
writer := multipart.NewWriter(body)
defer func() {
err := writer.Close()
if err != nil {
t.Fatalf("expected no error but got: %v", err) t.Fatalf("expected no error but got: %v", err)
} }
}()
err := writer.WriteField("foo", "foo") if tc.expectError && err == nil {
if err != nil { t.Fatal("expected error but got none")
t.Fatalf("expected no error but got: %v", err) }
}
part, err := writer.CreateFormFile("foo.txt", "foo.txt") if tc.expectError {
if err != nil { return
t.Fatalf("expected no error but got: %v", err) }
}
_, err = part.Write([]byte("foo")) // health request.
if err != nil { recorder := httptest.NewRecorder()
t.Fatalf("expected no error but got: %v", err) healthRequest := httptest.NewRequest(http.MethodGet, "/health", nil)
}
req := httptest.NewRequest(http.MethodPost, url, body) mod.srv.ServeHTTP(recorder, healthRequest)
req.Header.Set(echo.HeaderContentType, writer.FormDataContentType()) if recorder.Code != http.StatusOK {
t.Errorf("expected %d status code but got %d", http.StatusOK, recorder.Code)
}
return req // "multipart/form-data" request.
} multipartRequest := func(url string) *http.Request {
body := &bytes.Buffer{}
recorder = httptest.NewRecorder() writer := multipart.NewWriter(body)
mod.srv.ServeHTTP(recorder, multipartRequest("/forms/foo"))
if recorder.Code != http.StatusOK { defer func() {
t.Errorf("expected %d status code but got %d", http.StatusOK, recorder.Code) err := writer.Close()
} if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
}()
recorder = httptest.NewRecorder() err := writer.WriteField("foo", "foo")
mod.srv.ServeHTTP(recorder, multipartRequest("/forms/bar")) if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if recorder.Code != http.StatusInternalServerError { part, err := writer.CreateFormFile("foo.txt", "foo.txt")
t.Errorf("expected %d status code but got %d", http.StatusInternalServerError, recorder.Code) if err != nil {
} t.Fatalf("expected no error but got: %v", err)
}
err = mod.srv.Shutdown(context.TODO()) _, err = part.Write([]byte("foo"))
if err != nil { if err != nil {
t.Errorf("expected no error but got: %v", err) t.Fatalf("expected no error but got: %v", err)
}
req := httptest.NewRequest(http.MethodPost, url, body)
req.Header.Set(echo.HeaderContentType, writer.FormDataContentType())
return req
}
recorder = httptest.NewRecorder()
mod.srv.ServeHTTP(recorder, multipartRequest("/forms/foo"))
if recorder.Code != http.StatusOK {
t.Errorf("expected %d status code but got %d", http.StatusOK, recorder.Code)
}
recorder = httptest.NewRecorder()
mod.srv.ServeHTTP(recorder, multipartRequest("/forms/bar"))
if recorder.Code != http.StatusInternalServerError {
t.Errorf("expected %d status code but got %d", http.StatusInternalServerError, recorder.Code)
}
err = mod.srv.Shutdown(context.TODO())
if err != nil {
t.Errorf("expected no error but got: %v", err)
}
})
} }
} }

View File

@@ -12,17 +12,14 @@ import (
"path/filepath" "path/filepath"
"strings" "strings"
"time" "time"
"unicode"
"github.com/google/uuid" "github.com/google/uuid"
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"github.com/mholt/archiver/v3" "github.com/mholt/archiver/v3"
"go.uber.org/zap" "go.uber.org/zap"
"golang.org/x/text/runes"
"golang.org/x/text/transform"
"golang.org/x/text/unicode/norm" "golang.org/x/text/unicode/norm"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
var ( var (
@@ -37,18 +34,24 @@ var (
// Context is the request context for a "multipart/form-data" requests. // Context is the request context for a "multipart/form-data" requests.
type Context struct { type Context struct {
dirPath string dirPath string
values map[string][]string values map[string][]string
files map[string]string files map[string]string
outputPaths []string outputPaths []string
cancelled bool
cancelled bool logger *zap.Logger
logger *zap.Logger echoCtx echo.Context
echoCtx echo.Context pathRename gotenberg.PathRename
context.Context context.Context
} }
type osPathRename struct{}
func (o *osPathRename) Rename(oldpath, newpath string) error {
return os.Rename(oldpath, newpath)
}
// newContext returns a [Context] by parsing a "multipart/form-data" request. // newContext returns a [Context] by parsing a "multipart/form-data" request.
func newContext(echoCtx echo.Context, logger *zap.Logger, fs *gotenberg.FileSystem, timeout time.Duration) (*Context, context.CancelFunc, error) { func newContext(echoCtx echo.Context, logger *zap.Logger, fs *gotenberg.FileSystem, timeout time.Duration) (*Context, context.CancelFunc, error) {
processCtx, processCancel := context.WithTimeout(context.Background(), timeout) processCtx, processCancel := context.WithTimeout(context.Background(), timeout)
@@ -58,6 +61,7 @@ func newContext(echoCtx echo.Context, logger *zap.Logger, fs *gotenberg.FileSyst
cancelled: false, cancelled: false,
logger: logger, logger: logger,
echoCtx: echoCtx, echoCtx: echoCtx,
pathRename: new(osPathRename),
Context: processCtx, Context: processCtx,
} }
@@ -124,15 +128,6 @@ func newContext(echoCtx echo.Context, logger *zap.Logger, fs *gotenberg.FileSyst
ctx.files = make(map[string]string) ctx.files = make(map[string]string)
copyToDisk := func(fh *multipart.FileHeader) error { copyToDisk := func(fh *multipart.FileHeader) error {
// Avoid directory traversal and normalize filename.
// See https://github.com/gotenberg/gotenberg/issues/104.
t := transform.Chain(norm.NFD, runes.Remove(runes.In(unicode.Mn)), norm.NFC)
filename, _, err := transform.String(t, filepath.Base(fh.Filename))
if err != nil {
return fmt.Errorf("transform filename: %w", err)
}
in, err := fh.Open() in, err := fh.Open()
if err != nil { if err != nil {
return fmt.Errorf("open multipart file: %w", err) return fmt.Errorf("open multipart file: %w", err)
@@ -145,6 +140,10 @@ func newContext(echoCtx echo.Context, logger *zap.Logger, fs *gotenberg.FileSyst
} }
}() }()
// Avoid directory traversal and make sure filename characters are
// normalized.
// See: https://github.com/gotenberg/gotenberg/issues/662.
filename := norm.NFC.String(filepath.Base(fh.Filename))
path := fmt.Sprintf("%s/%s", ctx.dirPath, filename) path := fmt.Sprintf("%s/%s", ctx.dirPath, filename)
out, err := os.Create(path) out, err := os.Create(path)
@@ -172,7 +171,6 @@ func newContext(echoCtx echo.Context, logger *zap.Logger, fs *gotenberg.FileSyst
for _, files := range form.File { for _, files := range form.File {
for _, fh := range files { for _, fh := range files {
err = copyToDisk(fh) err = copyToDisk(fh)
if err != nil { if err != nil {
return ctx, cancel, fmt.Errorf("copy to disk: %w", err) return ctx, cancel, fmt.Errorf("copy to disk: %w", err)
} }
@@ -199,10 +197,20 @@ func (ctx *Context) FormData() *FormData {
} }
} }
// GeneratePath generates a path within the context's working directory. It // GeneratePath generates a path within the context's working directory.
// does not create a file. // It generates a new UUID-based filename. It does not create a file.
func (ctx *Context) GeneratePath(extension string) string { func (ctx *Context) GeneratePath(extension string) string {
return fmt.Sprintf("%s/%s%s", ctx.dirPath, uuid.New(), extension) return fmt.Sprintf("%s/%s%s", ctx.dirPath, uuid.New().String(), extension)
}
// Rename is just a wrapper around [os.Rename], as we need to mock this
// behavior in our tests.
func (ctx *Context) Rename(oldpath, newpath string) error {
err := ctx.pathRename.Rename(oldpath, newpath)
if err != nil {
return fmt.Errorf("rename path: %w", err)
}
return nil
} }
// AddOutputPaths adds the given paths. Those paths will be used later to build // AddOutputPaths adds the given paths. Those paths will be used later to build
@@ -277,3 +285,8 @@ func (ctx *Context) OutputFilename(outputPath string) string {
return fmt.Sprintf("%s%s", filename, filepath.Ext(outputPath)) return fmt.Sprintf("%s%s", filename, filepath.Ext(outputPath))
} }
// Interface guard.
var (
_ gotenberg.PathRename = (*osPathRename)(nil)
)

View File

@@ -3,21 +3,77 @@ package api
import ( import (
"bytes" "bytes"
"errors" "errors"
"fmt"
"io"
"mime/multipart" "mime/multipart"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os" "os"
"path/filepath"
"reflect" "reflect"
"strings" "strings"
"testing" "testing"
"time" "time"
"github.com/google/uuid"
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestOsPathRename_Rename(t *testing.T) {
dirPath, err := gotenberg.NewFileSystem().MkdirAll()
if err != nil {
t.Fatalf("create working directory: %v", err)
}
path := "/tests/test/testdata/api/sample1.txt"
copyPath := filepath.Join(dirPath, fmt.Sprintf("%s.txt", uuid.NewString()))
in, err := os.Open(path)
if err != nil {
t.Fatalf("open file: %v", err)
}
defer func() {
err := in.Close()
if err != nil {
t.Fatalf("close file: %v", err)
}
}()
out, err := os.Create(copyPath)
if err != nil {
t.Fatalf("create new file: %v", err)
}
defer func() {
err := out.Close()
if err != nil {
t.Fatalf("close new file: %v", err)
}
}()
_, err = io.Copy(out, in)
if err != nil {
t.Fatalf("copy file to new file: %v", err)
}
rename := new(osPathRename)
newPath := filepath.Join(dirPath, fmt.Sprintf("%s.txt", uuid.NewString()))
err = rename.Rename(copyPath, newPath)
if err != nil {
t.Errorf("expected no error but got: %v", err)
}
err = os.RemoveAll(dirPath)
if err != nil {
t.Fatalf("remove working directory: %v", err)
}
}
func TestNewContext(t *testing.T) { func TestNewContext(t *testing.T) {
for _, tc := range []struct { for _, tc := range []struct {
scenario string scenario string
@@ -191,12 +247,46 @@ func TestContext_GeneratePath(t *testing.T) {
} }
path := ctx.GeneratePath(".pdf") path := ctx.GeneratePath(".pdf")
if !strings.HasPrefix(path, ctx.dirPath) { if !strings.HasPrefix(path, ctx.dirPath) {
t.Errorf("expected '%s' to start with '%s'", path, ctx.dirPath) t.Errorf("expected '%s' to start with '%s'", path, ctx.dirPath)
} }
} }
func TestContext_Rename(t *testing.T) {
for _, tc := range []struct {
scenario string
ctx *Context
expectError bool
}{
{
scenario: "failure",
ctx: &Context{pathRename: &gotenberg.PathRenameMock{RenameMock: func(oldpath, newpath string) error {
return errors.New("cannot rename")
}}},
expectError: true,
},
{
scenario: "success",
ctx: &Context{pathRename: &gotenberg.PathRenameMock{RenameMock: func(oldpath, newpath string) error {
return nil
}}},
expectError: false,
},
} {
t.Run(tc.scenario, func(t *testing.T) {
err := tc.ctx.Rename("", "")
if tc.expectError && err == nil {
t.Fatal("expected error but got none", err)
}
if !tc.expectError && err != nil {
t.Fatalf("expected no error but got: %v", err)
}
})
}
}
func TestContext_AddOutputPaths(t *testing.T) { func TestContext_AddOutputPaths(t *testing.T) {
for _, tc := range []struct { for _, tc := range []struct {
scenario string scenario string

View File

@@ -11,6 +11,8 @@ import (
"time" "time"
"go.uber.org/multierr" "go.uber.org/multierr"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
// FormData is a helper for validating and hydrating values from a // FormData is a helper for validating and hydrating values from a
@@ -304,7 +306,7 @@ func (form *FormData) paths(extensions []string, target *[]string) *FormData {
} }
// See https://github.com/gotenberg/gotenberg/issues/139. // See https://github.com/gotenberg/gotenberg/issues/139.
sort.Strings(*target) sort.Sort(gotenberg.AlphanumericSort(*target))
return form return form
} }

View File

@@ -12,7 +12,7 @@ import (
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
// ErrAsyncProcess happens when a handler or middleware handles a request in an // ErrAsyncProcess happens when a handler or middleware handles a request in an
@@ -32,6 +32,18 @@ func ParseError(err error) (int, string) {
return http.StatusServiceUnavailable, http.StatusText(http.StatusServiceUnavailable) return http.StatusServiceUnavailable, http.StatusText(http.StatusServiceUnavailable)
} }
if errors.Is(err, gotenberg.ErrFiltered) {
return http.StatusForbidden, http.StatusText(http.StatusForbidden)
}
if errors.Is(err, gotenberg.ErrMaximumQueueSizeExceeded) {
return http.StatusTooManyRequests, http.StatusText(http.StatusTooManyRequests)
}
if errors.Is(err, gotenberg.ErrPdfFormatNotSupported) {
return http.StatusBadRequest, "At least one PDF engine cannot process the requested PDF format, while others may have failed to convert due to different issues"
}
var httpErr HttpError var httpErr HttpError
if errors.As(err, &httpErr) { if errors.As(err, &httpErr) {
return httpErr.HttpError() return httpErr.HttpError()

View File

@@ -14,7 +14,7 @@ import (
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestParseError(t *testing.T) { func TestParseError(t *testing.T) {
@@ -29,9 +29,19 @@ func TestParseError(t *testing.T) {
expectMessage: http.StatusText(http.StatusInternalServerError), expectMessage: http.StatusText(http.StatusInternalServerError),
}, },
{ {
err: context.DeadlineExceeded, err: gotenberg.ErrFiltered,
expectStatus: http.StatusServiceUnavailable, expectStatus: http.StatusForbidden,
expectMessage: http.StatusText(http.StatusServiceUnavailable), expectMessage: http.StatusText(http.StatusForbidden),
},
{
err: gotenberg.ErrMaximumQueueSizeExceeded,
expectStatus: http.StatusTooManyRequests,
expectMessage: http.StatusText(http.StatusTooManyRequests),
},
{
err: gotenberg.ErrPdfFormatNotSupported,
expectStatus: http.StatusBadRequest,
expectMessage: "At least one PDF engine cannot process the requested PDF format, while others may have failed to convert due to different issues",
}, },
{ {
err: WrapError( err: WrapError(

View File

@@ -4,6 +4,8 @@ import (
"github.com/alexliesenfeld/health" "github.com/alexliesenfeld/health"
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
// ContextMock is a helper for tests. // ContextMock is a helper for tests.
@@ -76,7 +78,7 @@ func (ctx *ContextMock) SetLogger(logger *zap.Logger) {
ctx.logger = logger ctx.logger = logger
} }
// SetEchoContext sets the echo.Context. // SetEchoContext sets the [echo.Context].
// //
// ctx := &api.ContextMock{Context: &api.Context{}} // ctx := &api.ContextMock{Context: &api.Context{}}
// ctx.setEchoContext(c) // ctx.setEchoContext(c)
@@ -84,6 +86,14 @@ func (ctx *ContextMock) SetEchoContext(c echo.Context) {
ctx.Context.echoCtx = c ctx.Context.echoCtx = c
} }
// SetPathRename sets the [gotenberg.PathRename].
//
// ctx := &api.ContextMock{Context: &api.Context{}}
// ctx.setPathRename(rename)
func (ctx *ContextMock) SetPathRename(rename gotenberg.PathRename) {
ctx.Context.pathRename = rename
}
// RouterMock is a mock for the [Router] interface. // RouterMock is a mock for the [Router] interface.
type RouterMock struct { type RouterMock struct {
RoutesMock func() ([]Route, error) RoutesMock func() ([]Route, error)
@@ -105,12 +115,17 @@ func (provider *MiddlewareProviderMock) Middlewares() ([]Middleware, error) {
// HealthCheckerMock is mock for the [HealthChecker] interface. // HealthCheckerMock is mock for the [HealthChecker] interface.
type HealthCheckerMock struct { type HealthCheckerMock struct {
ChecksMock func() ([]health.CheckerOption, error) ChecksMock func() ([]health.CheckerOption, error)
ReadyMock func() error
} }
func (mod *HealthCheckerMock) Checks() ([]health.CheckerOption, error) { func (mod *HealthCheckerMock) Checks() ([]health.CheckerOption, error) {
return mod.ChecksMock() return mod.ChecksMock()
} }
func (mod *HealthCheckerMock) Ready() error {
return mod.ReadyMock()
}
// Interface guards. // Interface guards.
var ( var (
_ Router = (*RouterMock)(nil) _ Router = (*RouterMock)(nil)

View File

@@ -117,6 +117,19 @@ func TestContextMock_SetEchoContext(t *testing.T) {
} }
} }
func TestContextMock_SetPathRename(t *testing.T) {
mock := ContextMock{&Context{}}
expect := new(osPathRename)
mock.SetPathRename(expect)
actual := mock.pathRename
if actual != expect {
t.Errorf("expected %v but got %v", expect, actual)
}
}
func TestRouterMock(t *testing.T) { func TestRouterMock(t *testing.T) {
mock := &RouterMock{ mock := &RouterMock{
RoutesMock: func() ([]Route, error) { RoutesMock: func() ([]Route, error) {
@@ -148,10 +161,18 @@ func TestHealthCheckerMock(t *testing.T) {
ChecksMock: func() ([]health.CheckerOption, error) { ChecksMock: func() ([]health.CheckerOption, error) {
return nil, nil return nil, nil
}, },
ReadyMock: func() error {
return nil
},
} }
_, err := mock.Checks() _, err := mock.Checks()
if err != nil { if err != nil {
t.Errorf("expected no error from HealthCheckerMock.Checks, but got: %v", err) t.Errorf("expected no error from HealthCheckerMock.Checks, but got: %v", err)
} }
err = mock.Ready()
if err != nil {
t.Errorf("expected no error from HealthCheckerMock.Ready, but got: %v", err)
}
} }

View File

@@ -5,7 +5,6 @@ import (
"errors" "errors"
"fmt" "fmt"
"os" "os"
"regexp"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -15,14 +14,16 @@ import (
"github.com/chromedp/cdproto/network" "github.com/chromedp/cdproto/network"
"github.com/chromedp/cdproto/runtime" "github.com/chromedp/cdproto/runtime"
"github.com/chromedp/chromedp" "github.com/chromedp/chromedp"
"github.com/dlclark/regexp2"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
type browser interface { type browser interface {
gotenberg.Process gotenberg.Process
pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error
screenshot(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error
} }
type browserArguments struct { type browserArguments struct {
@@ -38,8 +39,10 @@ type browserArguments struct {
wsUrlReadTimeout time.Duration wsUrlReadTimeout time.Duration
// Tasks specific. // Tasks specific.
allowList *regexp.Regexp allowList *regexp2.Regexp
denyList *regexp.Regexp denyList *regexp2.Regexp
clearCache bool
clearCookies bool
disableJavaScript bool disableJavaScript bool
} }
@@ -87,6 +90,8 @@ func (b *chromiumBrowser) Start(logger *zap.Logger) error {
// https://github.com/puppeteer/puppeteer/issues/2410 // https://github.com/puppeteer/puppeteer/issues/2410
chromedp.Flag("font-render-hinting", "none"), chromedp.Flag("font-render-hinting", "none"),
chromedp.UserDataDir(b.userProfileDirPath), chromedp.UserDataDir(b.userProfileDirPath),
// See https://github.com/gotenberg/gotenberg/issues/831.
chromedp.Flag("disable-pdf-tagging", true),
) )
if b.arguments.incognito { if b.arguments.incognito {
@@ -137,7 +142,7 @@ func (b *chromiumBrowser) Start(logger *zap.Logger) error {
b.ctxMu.Lock() b.ctxMu.Lock()
defer b.ctxMu.Unlock() defer b.ctxMu.Unlock()
// We have to keep the context around, as we need it to create a new tabs // We have to keep the context around, as we need it to create new tabs
// later. // later.
b.ctx = ctx b.ctx = ctx
b.cancelFunc = func() { b.cancelFunc = func() {
@@ -161,7 +166,7 @@ func (b *chromiumBrowser) Stop(logger *zap.Logger) error {
go func() { go func() {
// FIXME: Chromium seems to recreate the user profile directory // FIXME: Chromium seems to recreate the user profile directory
// right after its deletion if we do not wait a certain amount // right after its deletion if we do not wait a certain amount
// of time before re-deleting it. // of time before deleting it.
<-time.After(10 * time.Second) <-time.After(10 * time.Second)
err := os.RemoveAll(userProfileDirPath) err := os.RemoveAll(userProfileDirPath)
@@ -211,18 +216,53 @@ func (b *chromiumBrowser) Healthy(logger *zap.Logger) bool {
return true return true
} }
func (b *chromiumBrowser) pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error { func (b *chromiumBrowser) pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error {
// Note: no error wrapping because it leaks on errors we want to display to
// the end user.
return b.do(ctx, logger, url, options.Options, chromedp.Tasks{
network.Enable(),
fetch.Enable(),
runtime.Enable(),
clearCacheActionFunc(logger, b.arguments.clearCache),
clearCookiesActionFunc(logger, b.arguments.clearCookies),
disableJavaScriptActionFunc(logger, b.arguments.disableJavaScript),
extraHttpHeadersActionFunc(logger, options.ExtraHttpHeaders),
navigateActionFunc(logger, url, options.SkipNetworkIdleEvent),
hideDefaultWhiteBackgroundActionFunc(logger, options.OmitBackground, options.PrintBackground),
forceExactColorsActionFunc(),
emulateMediaTypeActionFunc(logger, options.EmulatedMediaType),
waitDelayBeforePrintActionFunc(logger, b.arguments.disableJavaScript, options.WaitDelay),
waitForExpressionBeforePrintActionFunc(logger, b.arguments.disableJavaScript, options.WaitForExpression),
// PDF specific.
printToPdfActionFunc(logger, outputPath, options),
})
}
func (b *chromiumBrowser) screenshot(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error {
// Note: no error wrapping because it leaks on errors we want to display to
// the end user.
return b.do(ctx, logger, url, options.Options, chromedp.Tasks{
network.Enable(),
fetch.Enable(),
runtime.Enable(),
clearCacheActionFunc(logger, b.arguments.clearCache),
clearCookiesActionFunc(logger, b.arguments.clearCookies),
disableJavaScriptActionFunc(logger, b.arguments.disableJavaScript),
extraHttpHeadersActionFunc(logger, options.ExtraHttpHeaders),
navigateActionFunc(logger, url, options.SkipNetworkIdleEvent),
hideDefaultWhiteBackgroundActionFunc(logger, options.OmitBackground, true),
forceExactColorsActionFunc(),
emulateMediaTypeActionFunc(logger, options.EmulatedMediaType),
waitDelayBeforePrintActionFunc(logger, b.arguments.disableJavaScript, options.WaitDelay),
waitForExpressionBeforePrintActionFunc(logger, b.arguments.disableJavaScript, options.WaitForExpression),
// Screenshot specific.
captureScreenshotActionFunc(logger, outputPath, options),
})
}
func (b *chromiumBrowser) do(ctx context.Context, logger *zap.Logger, url string, options Options, tasks chromedp.Tasks) error {
if !b.isStarted.Load() { if !b.isStarted.Load() {
return errors.New("browser not started, cannot handle PDF conversion") return errors.New("browser not started, cannot handle tasks")
}
// We validate the "main" URL against our allow / deny lists.
if !b.arguments.allowList.MatchString(url) {
return fmt.Errorf("'%s' does not match the expression from the allowed list: %w", url, ErrUrlNotAuthorized)
}
if b.arguments.denyList.String() != "" && b.arguments.denyList.MatchString(url) {
return fmt.Errorf("'%s' matches the expression from the denied list: %w", url, ErrUrlNotAuthorized)
} }
deadline, ok := ctx.Deadline() deadline, ok := ctx.Deadline()
@@ -230,6 +270,12 @@ func (b *chromiumBrowser) pdf(ctx context.Context, logger *zap.Logger, url, outp
return errors.New("context has no deadline") return errors.New("context has no deadline")
} }
// We validate the "main" URL against our allow / deny lists.
err := gotenberg.FilterDeadline(b.arguments.allowList, b.arguments.denyList, url, deadline)
if err != nil {
return fmt.Errorf("filter URL: %w", err)
}
b.ctxMu.RLock() b.ctxMu.RLock()
defer b.ctxMu.RUnlock() defer b.ctxMu.RUnlock()
@@ -243,6 +289,16 @@ func (b *chromiumBrowser) pdf(ctx context.Context, logger *zap.Logger, url, outp
// If a request does not pass the validation, we make it fail. // If a request does not pass the validation, we make it fail.
listenForEventRequestPaused(taskCtx, logger, b.arguments.allowList, b.arguments.denyList) listenForEventRequestPaused(taskCtx, logger, b.arguments.allowList, b.arguments.denyList)
var (
invalidHttpStatusCode error
invalidHttpStatusCodeMu sync.RWMutex
)
// See https://github.com/gotenberg/gotenberg/issues/613.
if len(options.FailOnHttpStatusCodes) != 0 {
listenForEventResponseReceived(taskCtx, logger, url, options.FailOnHttpStatusCodes, &invalidHttpStatusCode, &invalidHttpStatusCodeMu)
}
var ( var (
consoleExceptions error consoleExceptions error
consoleExceptionsMu sync.RWMutex consoleExceptionsMu sync.RWMutex
@@ -253,22 +309,7 @@ func (b *chromiumBrowser) pdf(ctx context.Context, logger *zap.Logger, url, outp
listenForEventExceptionThrown(taskCtx, logger, &consoleExceptions, &consoleExceptionsMu) listenForEventExceptionThrown(taskCtx, logger, &consoleExceptions, &consoleExceptionsMu)
} }
tasks := chromedp.Tasks{ err = chromedp.Run(taskCtx, tasks...)
network.Enable(),
fetch.Enable(),
runtime.Enable(),
disableJavaScriptActionFunc(logger, b.arguments.disableJavaScript),
extraHttpHeadersActionFunc(logger, options.ExtraHttpHeaders),
navigateActionFunc(logger, url),
hideDefaultWhiteBackgroundActionFunc(logger, options.OmitBackground, options.PrintBackground),
forceExactColorsActionFunc(),
emulateMediaTypeActionFunc(logger, options.EmulatedMediaType),
waitDelayBeforePrintActionFunc(logger, b.arguments.disableJavaScript, options.WaitDelay),
waitForExpressionBeforePrintActionFunc(logger, b.arguments.disableJavaScript, options.WaitForExpression),
printToPdfActionFunc(logger, outputPath, options),
}
err := chromedp.Run(taskCtx, tasks...)
if err != nil { if err != nil {
errMessage := err.Error() errMessage := err.Error()
@@ -284,7 +325,15 @@ func (b *chromiumBrowser) pdf(ctx context.Context, logger *zap.Logger, url, outp
return ErrRpccMessageTooLarge return ErrRpccMessageTooLarge
} }
return fmt.Errorf("print to PDF: %w", err) return fmt.Errorf("handle tasks: %w", err)
}
// See https://github.com/gotenberg/gotenberg/issues/613.
invalidHttpStatusCodeMu.RLock()
defer invalidHttpStatusCodeMu.RUnlock()
if invalidHttpStatusCode != nil {
return fmt.Errorf("%v: %w", invalidHttpStatusCode, ErrInvalidHttpStatusCode)
} }
// See https://github.com/gotenberg/gotenberg/issues/262. // See https://github.com/gotenberg/gotenberg/issues/262.

File diff suppressed because it is too large Load Diff

View File

@@ -9,11 +9,10 @@ import (
"github.com/alexliesenfeld/health" "github.com/alexliesenfeld/health"
flag "github.com/spf13/pflag" flag "github.com/spf13/pflag"
"go.uber.org/multierr"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
func init() { func init() {
@@ -21,14 +20,6 @@ func init() {
} }
var ( var (
// ErrUrlNotAuthorized happens if a URL is not acceptable according to the
// allowed/denied lists.
ErrUrlNotAuthorized = errors.New("URL not authorized")
// ErrOmitBackgroundWithoutPrintBackground happens if
// Options.OmitBackground is set to true but not Options.PrintBackground.
ErrOmitBackgroundWithoutPrintBackground = errors.New("omit background without print background")
// ErrInvalidEmulatedMediaType happens if the emulated media type is not // ErrInvalidEmulatedMediaType happens if the emulated media type is not
// "screen" nor "print". Empty value are allowed though. // "screen" nor "print". Empty value are allowed though.
ErrInvalidEmulatedMediaType = errors.New("invalid emulated media type") ErrInvalidEmulatedMediaType = errors.New("invalid emulated media type")
@@ -37,22 +28,32 @@ var (
// returns an exception or undefined. // returns an exception or undefined.
ErrInvalidEvaluationExpression = errors.New("invalid evaluation expression") ErrInvalidEvaluationExpression = errors.New("invalid evaluation expression")
// ErrInvalidPrinterSettings happens if the Options have one or more
// aberrant values.
ErrInvalidPrinterSettings = errors.New("invalid printer settings")
// ErrPageRangesSyntaxError happens if the Options have an invalid page
// ranges.
ErrPageRangesSyntaxError = errors.New("page ranges syntax error")
// ErrRpccMessageTooLarge happens when the messages received by // ErrRpccMessageTooLarge happens when the messages received by
// ChromeDevTools are larger than 100 MB. // ChromeDevTools are larger than 100 MB.
ErrRpccMessageTooLarge = errors.New("rpcc message too large") ErrRpccMessageTooLarge = errors.New("rpcc message too large")
// ErrInvalidHttpStatusCode happens when the status code from the main page
// matches with one of the entry in [Options.FailOnHttpStatusCodes].
ErrInvalidHttpStatusCode = errors.New("invalid HTTP status code")
// ErrConsoleExceptions happens when there are exceptions in the Chromium // ErrConsoleExceptions happens when there are exceptions in the Chromium
// console. It also happens only if the [Options.FailOnConsoleExceptions] // console. It also happens only if the [Options.FailOnConsoleExceptions]
// is set to true. // is set to true.
ErrConsoleExceptions = errors.New("console exceptions") ErrConsoleExceptions = errors.New("console exceptions")
// PDF specific.
// ErrOmitBackgroundWithoutPrintBackground happens if
// PdfOptions.OmitBackground is set to true but not PdfOptions.PrintBackground.
ErrOmitBackgroundWithoutPrintBackground = errors.New("omit background without print background")
// ErrInvalidPrinterSettings happens if the PdfOptions have one or more
// aberrant values.
ErrInvalidPrinterSettings = errors.New("invalid printer settings")
// ErrPageRangesSyntaxError happens if the PdfOptions have an invalid page
// ranges.
ErrPageRangesSyntaxError = errors.New("page ranges syntax error")
) )
// Chromium is a module which provides both an [Api] and routes for converting // Chromium is a module which provides both an [Api] and routes for converting
@@ -68,25 +69,37 @@ type Chromium struct {
engine gotenberg.PdfEngine engine gotenberg.PdfEngine
} }
// Options are the available expectedOptions for converting HTML document to PDF. // Options are the common options for all conversions.
type Options struct { type Options struct {
// SkipNetworkIdleEvent set if the conversion should wait for the
// "networkIdle" event, drastically improving the conversion speed. It may
// not be suitable for all HTML documents, as some may not be fully
// rendered until this event is fired.
// Optional.
SkipNetworkIdleEvent bool
// FailOnHttpStatusCodes sets if the conversion should fail if the status
// code from the main page matches with one of its entries.
// Optional.
FailOnHttpStatusCodes []int64
// FailOnConsoleExceptions sets if the conversion should fail if there are // FailOnConsoleExceptions sets if the conversion should fail if there are
// exceptions in the Chromium console. // exceptions in the Chromium console.
// Optional. // Optional.
FailOnConsoleExceptions bool FailOnConsoleExceptions bool
// WaitDelay is the duration to wait when loading an HTML document before // WaitDelay is the duration to wait when loading an HTML document before
// converting it to PDF. // converting it.
// Optional. // Optional.
WaitDelay time.Duration WaitDelay time.Duration
// WaitWindowStatus is the window.status value to wait for before // WaitWindowStatus is the window.status value to wait for before
// converting an HTML document to PDF. // converting an HTML document.
// Optional. // Optional.
WaitWindowStatus string WaitWindowStatus string
// WaitForExpression is the custom JavaScript expression to wait before // WaitForExpression is the custom JavaScript expression to wait before
// converting an HTML document to PDF until it returns true // converting an HTML document until it returns true
// Optional. // Optional.
WaitForExpression string WaitForExpression string
@@ -100,6 +113,31 @@ type Options struct {
// Optional. // Optional.
EmulatedMediaType string EmulatedMediaType string
// OmitBackground hides default white background and allows generating PDFs
// with transparency.
// Optional.
OmitBackground bool
}
// DefaultOptions returns the default values for Options.
func DefaultOptions() Options {
return Options{
SkipNetworkIdleEvent: false,
FailOnHttpStatusCodes: []int64{499, 599},
FailOnConsoleExceptions: false,
WaitDelay: 0,
WaitWindowStatus: "",
WaitForExpression: "",
ExtraHttpHeaders: nil,
EmulatedMediaType: "",
OmitBackground: false,
}
}
// PdfOptions are the available options for converting an HTML document to PDF.
type PdfOptions struct {
Options
// Landscape sets the paper orientation. // Landscape sets the paper orientation.
// Optional. // Optional.
Landscape bool Landscape bool
@@ -108,15 +146,15 @@ type Options struct {
// Optional. // Optional.
PrintBackground bool PrintBackground bool
// OmitBackground hides default white background and allows generating PDFs
// with transparency.
// Optional.
OmitBackground bool
// Scale is the scale of the page rendering. // Scale is the scale of the page rendering.
// Optional. // Optional.
Scale float64 Scale float64
// SinglePage defines whether to print the entire content in one single
// page.
// Optional.
SinglePage bool
// PaperWidth is the paper width, in inches. // PaperWidth is the paper width, in inches.
// Optional. // Optional.
PaperWidth float64 PaperWidth float64
@@ -169,35 +207,61 @@ type Options struct {
PreferCssPageSize bool PreferCssPageSize bool
} }
// DefaultOptions returns the default values for Options. // DefaultPdfOptions returns the default values for PdfOptions.
func DefaultOptions() Options { func DefaultPdfOptions() PdfOptions {
return Options{ return PdfOptions{
FailOnConsoleExceptions: false, Options: DefaultOptions(),
WaitDelay: 0, Landscape: false,
WaitWindowStatus: "", PrintBackground: false,
WaitForExpression: "", Scale: 1.0,
ExtraHttpHeaders: nil, SinglePage: false,
EmulatedMediaType: "", PaperWidth: 8.5,
Landscape: false, PaperHeight: 11,
PrintBackground: false, MarginTop: 0.39,
OmitBackground: false, MarginBottom: 0.39,
Scale: 1.0, MarginLeft: 0.39,
PaperWidth: 8.5, MarginRight: 0.39,
PaperHeight: 11, PageRanges: "",
MarginTop: 0.39, HeaderTemplate: "<html><head></head><body></body></html>",
MarginBottom: 0.39, FooterTemplate: "<html><head></head><body></body></html>",
MarginLeft: 0.39, PreferCssPageSize: false,
MarginRight: 0.39, }
PageRanges: "", }
HeaderTemplate: "<html><head></head><body></body></html>",
FooterTemplate: "<html><head></head><body></body></html>", // ScreenshotOptions are the available options for capturing a screenshot from
PreferCssPageSize: false, // an HTML document.
type ScreenshotOptions struct {
Options
// Format is the image compression format, either "png" or "jpeg" or
// "webp".
// Optional.
Format string
// Quality is the compression quality from range [0..100] (jpeg only).
// Optional.
Quality int
// OptimizeForSpeed defines whether to optimize image encoding for speed,
// not for resulting size.
// Optional.
OptimizeForSpeed bool
}
// DefaultScreenshotOptions returns the default values for ScreenshotOptions.
func DefaultScreenshotOptions() ScreenshotOptions {
return ScreenshotOptions{
Options: DefaultOptions(),
Format: "png",
Quality: 100,
OptimizeForSpeed: false,
} }
} }
// Api helps to interact with Chromium for converting HTML documents to PDF. // Api helps to interact with Chromium for converting HTML documents to PDF.
type Api interface { type Api interface {
Pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error Pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error
Screenshot(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error
} }
// Provider is a module interface which exposes a method for creating an [Api] // Provider is a module interface which exposes a method for creating an [Api]
@@ -217,21 +281,10 @@ func (mod *Chromium) Descriptor() gotenberg.ModuleDescriptor {
ID: "chromium", ID: "chromium",
FlagSet: func() *flag.FlagSet { FlagSet: func() *flag.FlagSet {
fs := flag.NewFlagSet("chromium", flag.ExitOnError) fs := flag.NewFlagSet("chromium", flag.ExitOnError)
// Deprecated flags.
fs.String("chromium-user-agent", "", "Override the default User-Agent header")
fs.Int("chromium-failed-starts-threshold", 5, "Set the number of consecutive failed starts after which the module is considered unhealthy - 0 means ignore")
var err error
err = multierr.Append(err, fs.MarkDeprecated("chromium-user-agent", "use the extraHttpHeaders form field instead"))
err = multierr.Append(err, fs.MarkDeprecated("chromium-failed-starts-threshold", "use the chromium-restart-after property instead"))
if err != nil {
panic(fmt.Errorf("create deprecated flags for the Chromium module: %v", err))
}
fs.Int64("chromium-restart-after", 0, "Number of conversions after which Chromium will automatically restart. Set to 0 to disable this feature") fs.Int64("chromium-restart-after", 0, "Number of conversions after which Chromium will automatically restart. Set to 0 to disable this feature")
fs.Int64("chromium-max-queue-size", 0, "Maximum request queue size for Chromium. Set to 0 to disable this feature")
fs.Bool("chromium-auto-start", false, "Automatically launch Chromium upon initialization if set to true; otherwise, Chromium will start at the time of the first conversion") fs.Bool("chromium-auto-start", false, "Automatically launch Chromium upon initialization if set to true; otherwise, Chromium will start at the time of the first conversion")
fs.Duration("chromium-start-timeout", time.Duration(10)*time.Second, "Maximum duration to wait for Chromium to start or restart") fs.Duration("chromium-start-timeout", time.Duration(20)*time.Second, "Maximum duration to wait for Chromium to start or restart")
fs.Bool("chromium-incognito", false, "Start Chromium with incognito mode") fs.Bool("chromium-incognito", false, "Start Chromium with incognito mode")
fs.Bool("chromium-allow-insecure-localhost", false, "Ignore TLS/SSL errors on localhost") fs.Bool("chromium-allow-insecure-localhost", false, "Ignore TLS/SSL errors on localhost")
fs.Bool("chromium-ignore-certificate-errors", false, "Ignore the certificate errors") fs.Bool("chromium-ignore-certificate-errors", false, "Ignore the certificate errors")
@@ -240,7 +293,9 @@ func (mod *Chromium) Descriptor() gotenberg.ModuleDescriptor {
fs.String("chromium-host-resolver-rules", "", "Set custom mappings to the host resolver") fs.String("chromium-host-resolver-rules", "", "Set custom mappings to the host resolver")
fs.String("chromium-proxy-server", "", "Set the outbound proxy server; this switch only affects HTTP and HTTPS requests") fs.String("chromium-proxy-server", "", "Set the outbound proxy server; this switch only affects HTTP and HTTPS requests")
fs.String("chromium-allow-list", "", "Set the allowed URLs for Chromium using a regular expression") fs.String("chromium-allow-list", "", "Set the allowed URLs for Chromium using a regular expression")
fs.String("chromium-deny-list", "^file:///[^tmp].*", "Set the denied URLs for Chromium using a regular expression") fs.String("chromium-deny-list", `^file:(?!//\/tmp/).*`, "Set the denied URLs for Chromium using a regular expression")
fs.Bool("chromium-clear-cache", false, "Clear Chromium cache between each conversion")
fs.Bool("chromium-clear-cookies", false, "Clear Chromium cookies between each conversion")
fs.Bool("chromium-disable-javascript", false, "Disable JavaScript") fs.Bool("chromium-disable-javascript", false, "Disable JavaScript")
fs.Bool("chromium-disable-routes", false, "Disable the routes") fs.Bool("chromium-disable-routes", false, "Disable the routes")
@@ -274,6 +329,8 @@ func (mod *Chromium) Provision(ctx *gotenberg.Context) error {
allowList: flags.MustRegexp("chromium-allow-list"), allowList: flags.MustRegexp("chromium-allow-list"),
denyList: flags.MustRegexp("chromium-deny-list"), denyList: flags.MustRegexp("chromium-deny-list"),
clearCache: flags.MustBool("chromium-clear-cache"),
clearCookies: flags.MustBool("chromium-clear-cookies"),
disableJavaScript: flags.MustBool("chromium-disable-javascript"), disableJavaScript: flags.MustBool("chromium-disable-javascript"),
} }
@@ -290,7 +347,7 @@ func (mod *Chromium) Provision(ctx *gotenberg.Context) error {
// Process. // Process.
mod.browser = newChromiumBrowser(mod.args) mod.browser = newChromiumBrowser(mod.args)
mod.supervisor = gotenberg.NewProcessSupervisor(mod.logger, mod.browser, flags.MustInt64("chromium-restart-after")) mod.supervisor = gotenberg.NewProcessSupervisor(mod.logger, mod.browser, flags.MustInt64("chromium-restart-after"), flags.MustInt64("chromium-max-queue-size"))
// PDF Engine. // PDF Engine.
provider, err := ctx.Module(new(gotenberg.PdfEngineProvider)) provider, err := ctx.Module(new(gotenberg.PdfEngineProvider))
@@ -359,22 +416,6 @@ func (mod *Chromium) Stop(ctx context.Context) error {
// Metrics returns the metrics. // Metrics returns the metrics.
func (mod *Chromium) Metrics() ([]gotenberg.Metric, error) { func (mod *Chromium) Metrics() ([]gotenberg.Metric, error) {
return []gotenberg.Metric{ return []gotenberg.Metric{
{
// TODO: remove deprecated.
Name: "chromium_active_instances_count",
Description: "Current number of active Chromium instances - deprecated.",
Read: func() float64 {
return 1
},
},
{
// TODO: remove deprecated.
Name: "chromium_failed_starts_count",
Description: "Current number of Chromium consecutive starting failures - deprecated.",
Read: func() float64 {
return 0
},
},
{ {
Name: "chromium_requests_queue_size", Name: "chromium_requests_queue_size",
Description: "Current number of Chromium conversion requests waiting to be treated.", Description: "Current number of Chromium conversion requests waiting to be treated.",
@@ -408,6 +449,34 @@ func (mod *Chromium) Checks() ([]health.CheckerOption, error) {
}, nil }, nil
} }
// Ready returns no error if the module is ready.
func (mod *Chromium) Ready() error {
if !mod.autoStart {
return nil
}
ctx, cancel := context.WithTimeout(context.Background(), mod.args.wsUrlReadTimeout)
defer cancel()
ticker := time.NewTicker(time.Duration(100) * time.Millisecond)
for {
select {
case <-ctx.Done():
ticker.Stop()
return fmt.Errorf("context done while waiting for Chromium to be ready: %w", ctx.Err())
case <-ticker.C:
ok := mod.browser.Healthy(mod.logger)
if ok {
ticker.Stop()
return nil
}
continue
}
}
}
// Chromium returns an [Api] for interacting with Chromium for converting HTML // Chromium returns an [Api] for interacting with Chromium for converting HTML
// documents to PDF. // documents to PDF.
func (mod *Chromium) Chromium() (Api, error) { func (mod *Chromium) Chromium() (Api, error) {
@@ -422,19 +491,31 @@ func (mod *Chromium) Routes() ([]api.Route, error) {
return []api.Route{ return []api.Route{
convertUrlRoute(mod, mod.engine), convertUrlRoute(mod, mod.engine),
screenshotUrlRoute(mod),
convertHtmlRoute(mod, mod.engine), convertHtmlRoute(mod, mod.engine),
screenshotHtmlRoute(mod),
convertMarkdownRoute(mod, mod.engine), convertMarkdownRoute(mod, mod.engine),
screenshotMarkdownRoute(mod),
}, nil }, nil
} }
// Pdf converts a URL to PDF. // Pdf converts a URL to PDF.
func (mod *Chromium) Pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error { func (mod *Chromium) Pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error {
// FIXME: no error wrapping because it leaks on console exceptions output. // Note: no error wrapping because it leaks on errors we want to display to
// the end user.
return mod.supervisor.Run(ctx, logger, func() error { return mod.supervisor.Run(ctx, logger, func() error {
return mod.browser.pdf(ctx, logger, url, outputPath, options) return mod.browser.pdf(ctx, logger, url, outputPath, options)
}) })
} }
func (mod *Chromium) Screenshot(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error {
// Note: no error wrapping because it leaks on errors we want to display to
// the end user.
return mod.supervisor.Run(ctx, logger, func() error {
return mod.browser.screenshot(ctx, logger, url, outputPath, options)
})
}
// Interface guards. // Interface guards.
var ( var (
_ gotenberg.Module = (*Chromium)(nil) _ gotenberg.Module = (*Chromium)(nil)

View File

@@ -11,12 +11,12 @@ import (
"github.com/alexliesenfeld/health" "github.com/alexliesenfeld/health"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestDefaultOptions(t *testing.T) { func TestDefaultOptions(t *testing.T) {
actual := DefaultOptions() actual := DefaultPdfOptions()
notExpect := Options{} notExpect := PdfOptions{}
if reflect.DeepEqual(actual, notExpect) { if reflect.DeepEqual(actual, notExpect) {
t.Errorf("expected %v and got identical %v", actual, notExpect) t.Errorf("expected %v and got identical %v", actual, notExpect)
@@ -334,26 +334,16 @@ func TestChromium_Metrics(t *testing.T) {
t.Fatalf("expected no error but got: %v", err) t.Fatalf("expected no error but got: %v", err)
} }
if len(metrics) != 4 { if len(metrics) != 2 {
t.Fatalf("expected %d metrics, but got %d", 4, len(metrics)) t.Fatalf("expected %d metrics, but got %d", 2, len(metrics))
} }
actual := metrics[0].Read() actual := metrics[0].Read()
if actual != float64(1) {
t.Errorf("expected %f for chromium_active_instances_count, but got %f", float64(1), actual)
}
actual = metrics[1].Read()
if actual != float64(0) {
t.Errorf("expected %f for chromium_failed_starts_count, but got %f", float64(0), actual)
}
actual = metrics[2].Read()
if actual != float64(10) { if actual != float64(10) {
t.Errorf("expected %f for chromium_requests_queue_size, but got %f", float64(10), actual) t.Errorf("expected %f for chromium_requests_queue_size, but got %f", float64(10), actual)
} }
actual = metrics[3].Read() actual = metrics[1].Read()
if actual != float64(0) { if actual != float64(0) {
t.Errorf("expected %f for chromium_restarts_count, but got %f", float64(0), actual) t.Errorf("expected %f for chromium_restarts_count, but got %f", float64(0), actual)
} }
@@ -399,6 +389,61 @@ func TestChromium_Checks(t *testing.T) {
} }
} }
func TestChromium_Ready(t *testing.T) {
for _, tc := range []struct {
scenario string
autoStart bool
startTimeout time.Duration
browser browser
expectError bool
}{
{
scenario: "no auto-start",
autoStart: false,
startTimeout: time.Duration(30) * time.Second,
browser: &browserMock{ProcessMock: gotenberg.ProcessMock{HealthyMock: func(logger *zap.Logger) bool {
return false
}}},
expectError: false,
},
{
scenario: "auto-start: context done",
autoStart: true,
startTimeout: time.Duration(200) * time.Millisecond,
browser: &browserMock{ProcessMock: gotenberg.ProcessMock{HealthyMock: func(logger *zap.Logger) bool {
return false
}}},
expectError: true,
},
{
scenario: "auto-start success",
autoStart: true,
startTimeout: time.Duration(30) * time.Second,
browser: &browserMock{ProcessMock: gotenberg.ProcessMock{HealthyMock: func(logger *zap.Logger) bool {
return true
}}},
expectError: false,
},
} {
t.Run(tc.scenario, func(t *testing.T) {
mod := new(Chromium)
mod.autoStart = tc.autoStart
mod.args = browserArguments{wsUrlReadTimeout: tc.startTimeout}
mod.browser = tc.browser
err := mod.Ready()
if !tc.expectError && err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if tc.expectError && err == nil {
t.Fatal("expected error but got none")
}
})
}
}
func TestChromium_Chromium(t *testing.T) { func TestChromium_Chromium(t *testing.T) {
mod := new(Chromium) mod := new(Chromium)
@@ -416,7 +461,7 @@ func TestChromium_Routes(t *testing.T) {
}{ }{
{ {
scenario: "routes not disabled", scenario: "routes not disabled",
expectRoutes: 3, expectRoutes: 6,
disableRoutes: false, disableRoutes: false,
}, },
{ {
@@ -450,14 +495,14 @@ func TestChromium_Pdf(t *testing.T) {
}{ }{
{ {
scenario: "PDF task success", scenario: "PDF task success",
browser: &browserMock{pdfMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error { browser: &browserMock{pdfMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error {
return nil return nil
}}, }},
expectError: false, expectError: false,
}, },
{ {
scenario: "PDF task error", scenario: "PDF task error",
browser: &browserMock{pdfMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error { browser: &browserMock{pdfMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error {
return errors.New("PDF task error") return errors.New("PDF task error")
}}, }},
expectError: true, expectError: true,
@@ -470,7 +515,49 @@ func TestChromium_Pdf(t *testing.T) {
}} }}
mod.browser = tc.browser mod.browser = tc.browser
err := mod.Pdf(context.Background(), zap.NewNop(), "", "", Options{}) err := mod.Pdf(context.Background(), zap.NewNop(), "", "", PdfOptions{})
if !tc.expectError && err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if tc.expectError && err == nil {
t.Fatal("expected error but got none")
}
})
}
}
func TestChromium_Screenshot(t *testing.T) {
for _, tc := range []struct {
scenario string
supervisor gotenberg.ProcessSupervisor
browser browser
expectError bool
}{
{
scenario: "Screenshot task success",
browser: &browserMock{screenshotMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error {
return nil
}},
expectError: false,
},
{
scenario: "Screenshot task error",
browser: &browserMock{screenshotMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error {
return errors.New("screenshot task error")
}},
expectError: true,
},
} {
t.Run(tc.scenario, func(t *testing.T) {
mod := new(Chromium)
mod.supervisor = &gotenberg.ProcessSupervisorMock{RunMock: func(ctx context.Context, logger *zap.Logger, task func() error) error {
return task()
}}
mod.browser = tc.browser
err := mod.Screenshot(context.Background(), zap.NewNop(), "", "", ScreenshotOptions{})
if !tc.expectError && err != nil { if !tc.expectError && err != nil {
t.Fatalf("expected no error but got: %v", err) t.Fatalf("expected no error but got: %v", err)

View File

@@ -3,7 +3,7 @@ package chromium
import ( import (
"context" "context"
"fmt" "fmt"
"regexp" "slices"
"sync" "sync"
"github.com/chromedp/cdproto/cdp" "github.com/chromedp/cdproto/cdp"
@@ -12,14 +12,17 @@ import (
"github.com/chromedp/cdproto/page" "github.com/chromedp/cdproto/page"
"github.com/chromedp/cdproto/runtime" "github.com/chromedp/cdproto/runtime"
"github.com/chromedp/chromedp" "github.com/chromedp/chromedp"
"github.com/dlclark/regexp2"
"go.uber.org/multierr" "go.uber.org/multierr"
"go.uber.org/zap" "go.uber.org/zap"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
"github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
// listenForEventRequestPaused listens for requests to check if they are // listenForEventRequestPaused listens for requests to check if they are
// allowed or not. // allowed or not.
func listenForEventRequestPaused(ctx context.Context, logger *zap.Logger, allowList *regexp.Regexp, denyList *regexp.Regexp) { func listenForEventRequestPaused(ctx context.Context, logger *zap.Logger, allowList *regexp2.Regexp, denyList *regexp2.Regexp) {
chromedp.ListenTarget(ctx, func(ev interface{}) { chromedp.ListenTarget(ctx, func(ev interface{}) {
switch e := ev.(type) { switch e := ev.(type) {
case *fetch.EventRequestPaused: case *fetch.EventRequestPaused:
@@ -27,13 +30,15 @@ func listenForEventRequestPaused(ctx context.Context, logger *zap.Logger, allowL
logger.Debug(fmt.Sprintf("event EventRequestPaused fired for '%s'", e.Request.URL)) logger.Debug(fmt.Sprintf("event EventRequestPaused fired for '%s'", e.Request.URL))
allow := true allow := true
if !allowList.MatchString(e.Request.URL) { deadline, ok := ctx.Deadline()
logger.Warn(fmt.Sprintf("'%s' does not match the expression from the allowed list", e.Request.URL)) if !ok {
allow = false logger.Error("context has no deadline, cannot filter URL")
return
} }
if denyList.String() != "" && denyList.MatchString(e.Request.URL) { err := gotenberg.FilterDeadline(allowList, denyList, e.Request.URL, deadline)
logger.Warn(fmt.Sprintf("'%s' matches the expression from the denied list", e.Request.URL)) if err != nil {
logger.Warn(err.Error())
allow = false allow = false
} }
@@ -42,16 +47,15 @@ func listenForEventRequestPaused(ctx context.Context, logger *zap.Logger, allowL
if allow { if allow {
req := fetch.ContinueRequest(e.RequestID) req := fetch.ContinueRequest(e.RequestID)
err := req.Do(executorCtx) err = req.Do(executorCtx)
if err != nil { if err != nil {
logger.Error(fmt.Sprintf("continue request: %s", err)) logger.Error(fmt.Sprintf("continue request: %s", err))
} }
return return
} }
req := fetch.FailRequest(e.RequestID, network.ErrorReasonAccessDenied) req := fetch.FailRequest(e.RequestID, network.ErrorReasonAccessDenied)
err := req.Do(executorCtx) err = req.Do(executorCtx)
if err != nil { if err != nil {
logger.Error(fmt.Sprintf("fail request: %s", err)) logger.Error(fmt.Sprintf("fail request: %s", err))
} }
@@ -60,6 +64,37 @@ func listenForEventRequestPaused(ctx context.Context, logger *zap.Logger, allowL
}) })
} }
// listenForEventResponseReceived listens for an invalid HTTP status code is
// returned by the main page.
// See https://github.com/gotenberg/gotenberg/issues/613.
func listenForEventResponseReceived(ctx context.Context, logger *zap.Logger, url string, failOnHttpStatusCodes []int64, invalidHttpStatusCode *error, invalidHttpStatusCodeMu *sync.RWMutex) {
for _, code := range []int64{199, 299, 399, 499, 599} {
if slices.Contains(failOnHttpStatusCodes, code) {
for i := code - 99; i <= code; i++ {
failOnHttpStatusCodes = append(failOnHttpStatusCodes, i)
}
}
}
chromedp.ListenTarget(ctx, func(ev interface{}) {
switch ev := ev.(type) {
case *network.EventResponseReceived:
if ev.Response.URL != url {
return
}
logger.Debug(fmt.Sprintf("event EventResponseReceived fired for main page: %+v", ev.Response))
if slices.Contains(failOnHttpStatusCodes, ev.Response.Status) {
invalidHttpStatusCodeMu.Lock()
defer invalidHttpStatusCodeMu.Unlock()
*invalidHttpStatusCode = fmt.Errorf("%d: %s", ev.Response.Status, ev.Response.StatusText)
}
}
})
}
// listenForEventExceptionThrown listens for exceptions in the console and // listenForEventExceptionThrown listens for exceptions in the console and
// appends those exceptions to the given error pointer. // appends those exceptions to the given error pointer.
// See https://github.com/gotenberg/gotenberg/issues/262. // See https://github.com/gotenberg/gotenberg/issues/262.
@@ -179,7 +214,6 @@ func waitForEventLoadingFinished(ctx context.Context, logger *zap.Logger) func()
// completed or an error is encountered. // completed or an error is encountered.
func runBatch(ctx context.Context, fn ...func() error) error { func runBatch(ctx context.Context, fn ...func() error) error {
eg, _ := errgroup.WithContext(ctx) eg, _ := errgroup.WithContext(ctx)
for _, f := range fn { for _, f := range fn {
eg.Go(f) eg.Go(f)
} }

View File

@@ -5,28 +5,38 @@ import (
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
// ApiMock is a mock for the [Api] interface. // ApiMock is a mock for the [Api] interface.
type ApiMock struct { type ApiMock struct {
PdfMock func(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error PdfMock func(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error
ScreenshotMock func(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error
} }
func (api *ApiMock) Pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error { func (api *ApiMock) Pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error {
return api.PdfMock(ctx, logger, url, outputPath, options) return api.PdfMock(ctx, logger, url, outputPath, options)
} }
func (api *ApiMock) Screenshot(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error {
return api.ScreenshotMock(ctx, logger, url, outputPath, options)
}
// browserMock is a mock for the [browser] interface. // browserMock is a mock for the [browser] interface.
type browserMock struct { type browserMock struct {
gotenberg.ProcessMock gotenberg.ProcessMock
pdfMock func(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error pdfMock func(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error
screenshotMock func(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error
} }
func (b *browserMock) pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error { func (b *browserMock) pdf(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error {
return b.pdfMock(ctx, logger, url, outputPath, options) return b.pdfMock(ctx, logger, url, outputPath, options)
} }
func (b *browserMock) screenshot(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error {
return b.screenshotMock(ctx, logger, url, outputPath, options)
}
// Interface guards. // Interface guards.
var ( var (
_ Api = (*ApiMock)(nil) _ Api = (*ApiMock)(nil)

View File

@@ -9,26 +9,42 @@ import (
func TestApiMock(t *testing.T) { func TestApiMock(t *testing.T) {
mock := &ApiMock{ mock := &ApiMock{
PdfMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error { PdfMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error {
return nil
},
ScreenshotMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error {
return nil return nil
}, },
} }
err := mock.Pdf(context.Background(), zap.NewNop(), "", "", Options{}) err := mock.Pdf(context.Background(), zap.NewNop(), "", "", PdfOptions{})
if err != nil { if err != nil {
t.Errorf("expected no error from ApiMock.Pdf, but got: %v", err) t.Errorf("expected no error from ApiMock.Pdf, but got: %v", err)
} }
err = mock.Screenshot(context.Background(), zap.NewNop(), "", "", ScreenshotOptions{})
if err != nil {
t.Errorf("expected no error from ApiMock.Screenshot, but got: %v", err)
}
} }
func TestBrowserMock(t *testing.T) { func TestBrowserMock(t *testing.T) {
mock := &browserMock{ mock := &browserMock{
pdfMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options Options) error { pdfMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options PdfOptions) error {
return nil
},
screenshotMock: func(ctx context.Context, logger *zap.Logger, url, outputPath string, options ScreenshotOptions) error {
return nil return nil
}, },
} }
err := mock.pdf(context.Background(), zap.NewNop(), "", "", Options{}) err := mock.pdf(context.Background(), zap.NewNop(), "", "", PdfOptions{})
if err != nil { if err != nil {
t.Errorf("expected no error from browserMock.pdf, but got: %v", err) t.Errorf("expected no error from browserMock.pdf, but got: %v", err)
} }
err = mock.screenshot(context.Background(), zap.NewNop(), "", "", ScreenshotOptions{})
if err != nil {
t.Errorf("expected no error from browserMock.screenshot, but got: %v", err)
}
} }

View File

@@ -9,6 +9,7 @@ import (
"net/http" "net/http"
"os" "os"
"path/filepath" "path/filepath"
"strconv"
"strings" "strings"
"time" "time"
@@ -17,47 +18,55 @@ import (
"github.com/russross/blackfriday/v2" "github.com/russross/blackfriday/v2"
"go.uber.org/multierr" "go.uber.org/multierr"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
// FormDataChromiumPdfOptions creates [Options] from the form data. Fallback to // FormDataChromiumOptions creates [Options] from the form data. Fallback to
// default value if the considered key is not present. // default value if the considered key is not present.
func FormDataChromiumPdfOptions(ctx *api.Context) (*api.FormData, Options) { func FormDataChromiumOptions(ctx *api.Context) (*api.FormData, Options) {
defaultOptions := DefaultOptions() defaultOptions := DefaultOptions()
var ( var (
failOnConsoleExceptions bool skipNetworkIdleEvent bool
waitDelay time.Duration failOnHttpStatusCodes []int64
waitWindowStatus string failOnConsoleExceptions bool
waitForExpression string waitDelay time.Duration
userAgent string waitWindowStatus string
extraHttpHeaders map[string]string waitForExpression string
emulatedMediaType string extraHttpHeaders map[string]string
landscape, printBackground, omitBackground bool emulatedMediaType string
scale, paperWidth, paperHeight float64 omitBackground bool
marginTop, marginBottom, marginLeft, marginRight float64
pageRanges string
headerTemplate, footerTemplate string
preferCssPageSize bool
) )
form := ctx.FormData(). form := ctx.FormData().
Bool("skipNetworkIdleEvent", &skipNetworkIdleEvent, defaultOptions.SkipNetworkIdleEvent).
Custom("failOnHttpStatusCodes", func(value string) error {
if value == "" {
failOnHttpStatusCodes = defaultOptions.FailOnHttpStatusCodes
return nil
}
err := json.Unmarshal([]byte(value), &failOnHttpStatusCodes)
if err != nil {
return fmt.Errorf("unmarshal failOnHttpStatusCodes: %w", err)
}
return nil
}).
Bool("failOnConsoleExceptions", &failOnConsoleExceptions, defaultOptions.FailOnConsoleExceptions). Bool("failOnConsoleExceptions", &failOnConsoleExceptions, defaultOptions.FailOnConsoleExceptions).
Duration("waitDelay", &waitDelay, defaultOptions.WaitDelay). Duration("waitDelay", &waitDelay, defaultOptions.WaitDelay).
String("waitWindowStatus", &waitWindowStatus, defaultOptions.WaitWindowStatus). String("waitWindowStatus", &waitWindowStatus, defaultOptions.WaitWindowStatus).
String("waitForExpression", &waitForExpression, defaultOptions.WaitForExpression). String("waitForExpression", &waitForExpression, defaultOptions.WaitForExpression).
String("userAgent", &userAgent, ""). // FIXME: deprecated.
Custom("extraHttpHeaders", func(value string) error { Custom("extraHttpHeaders", func(value string) error {
if value == "" { if value == "" {
extraHttpHeaders = defaultOptions.ExtraHttpHeaders extraHttpHeaders = defaultOptions.ExtraHttpHeaders
return nil return nil
} }
err := json.Unmarshal([]byte(value), &extraHttpHeaders) err := json.Unmarshal([]byte(value), &extraHttpHeaders)
if err != nil { if err != nil {
return fmt.Errorf("unmarshal extra HTTP headers: %w", err) return fmt.Errorf("unmarshal extraHttpHeaders: %w", err)
} }
return nil return nil
@@ -65,7 +74,6 @@ func FormDataChromiumPdfOptions(ctx *api.Context) (*api.FormData, Options) {
Custom("emulatedMediaType", func(value string) error { Custom("emulatedMediaType", func(value string) error {
if value == "" { if value == "" {
emulatedMediaType = defaultOptions.EmulatedMediaType emulatedMediaType = defaultOptions.EmulatedMediaType
return nil return nil
} }
@@ -77,87 +85,150 @@ func FormDataChromiumPdfOptions(ctx *api.Context) (*api.FormData, Options) {
return nil return nil
}). }).
Bool("landscape", &landscape, defaultOptions.Landscape). Bool("omitBackground", &omitBackground, defaultOptions.OmitBackground)
Bool("printBackground", &printBackground, defaultOptions.PrintBackground).
Bool("omitBackground", &omitBackground, defaultOptions.OmitBackground).
Float64("scale", &scale, defaultOptions.Scale).
Float64("paperWidth", &paperWidth, defaultOptions.PaperWidth).
Float64("paperHeight", &paperHeight, defaultOptions.PaperHeight).
Float64("marginTop", &marginTop, defaultOptions.MarginTop).
Float64("marginBottom", &marginBottom, defaultOptions.MarginBottom).
Float64("marginLeft", &marginLeft, defaultOptions.MarginLeft).
Float64("marginRight", &marginRight, defaultOptions.MarginRight).
String("nativePageRanges", &pageRanges, defaultOptions.PageRanges).
Content("header.html", &headerTemplate, defaultOptions.HeaderTemplate).
Content("footer.html", &footerTemplate, defaultOptions.FooterTemplate).
Bool("preferCssPageSize", &preferCssPageSize, defaultOptions.PreferCssPageSize)
// FIXME: deprecated.
if userAgent != "" {
ctx.Log().Warn("'userAgent' is deprecated; prefer the 'extraHttpHeaders' form field instead")
if extraHttpHeaders == nil {
extraHttpHeaders = make(map[string]string)
}
extraHttpHeaders["User-Agent"] = userAgent
}
options := Options{ options := Options{
SkipNetworkIdleEvent: skipNetworkIdleEvent,
FailOnHttpStatusCodes: failOnHttpStatusCodes,
FailOnConsoleExceptions: failOnConsoleExceptions, FailOnConsoleExceptions: failOnConsoleExceptions,
WaitDelay: waitDelay, WaitDelay: waitDelay,
WaitWindowStatus: waitWindowStatus, WaitWindowStatus: waitWindowStatus,
WaitForExpression: waitForExpression, WaitForExpression: waitForExpression,
ExtraHttpHeaders: extraHttpHeaders, ExtraHttpHeaders: extraHttpHeaders,
EmulatedMediaType: emulatedMediaType, EmulatedMediaType: emulatedMediaType,
Landscape: landscape,
PrintBackground: printBackground,
OmitBackground: omitBackground, OmitBackground: omitBackground,
Scale: scale,
PaperWidth: paperWidth,
PaperHeight: paperHeight,
MarginTop: marginTop,
MarginBottom: marginBottom,
MarginLeft: marginLeft,
MarginRight: marginRight,
PageRanges: pageRanges,
HeaderTemplate: headerTemplate,
FooterTemplate: footerTemplate,
PreferCssPageSize: preferCssPageSize,
} }
return form, options return form, options
} }
// FormDataChromiumPdfFormats creates [gotenberg.PdfFormats] from the form // FormDataChromiumPdfOptions creates [PdfOptions] from the form data. Fallback to
// data. Fallback to default value if the considered key is not present. // default value if the considered key is not present.
func FormDataChromiumPdfFormats(ctx *api.Context) gotenberg.PdfFormats { func FormDataChromiumPdfOptions(ctx *api.Context) (*api.FormData, PdfOptions) {
form, options := FormDataChromiumOptions(ctx)
defaultPdfOptions := DefaultPdfOptions()
var ( var (
pdfFormat string landscape, printBackground, singlePage bool
pdfa string scale, paperWidth, paperHeight float64
pdfua bool marginTop, marginBottom, marginLeft, marginRight float64
pageRanges string
headerTemplate, footerTemplate string
preferCssPageSize bool
) )
ctx.FormData(). form.
String("pdfFormat", &pdfFormat, ""). Bool("landscape", &landscape, defaultPdfOptions.Landscape).
Bool("printBackground", &printBackground, defaultPdfOptions.PrintBackground).
Float64("scale", &scale, defaultPdfOptions.Scale).
Bool("singlePage", &singlePage, defaultPdfOptions.SinglePage).
Float64("paperWidth", &paperWidth, defaultPdfOptions.PaperWidth).
Float64("paperHeight", &paperHeight, defaultPdfOptions.PaperHeight).
Float64("marginTop", &marginTop, defaultPdfOptions.MarginTop).
Float64("marginBottom", &marginBottom, defaultPdfOptions.MarginBottom).
Float64("marginLeft", &marginLeft, defaultPdfOptions.MarginLeft).
Float64("marginRight", &marginRight, defaultPdfOptions.MarginRight).
String("nativePageRanges", &pageRanges, defaultPdfOptions.PageRanges).
Content("header.html", &headerTemplate, defaultPdfOptions.HeaderTemplate).
Content("footer.html", &footerTemplate, defaultPdfOptions.FooterTemplate).
Bool("preferCssPageSize", &preferCssPageSize, defaultPdfOptions.PreferCssPageSize)
pdfOptions := PdfOptions{
Options: options,
Landscape: landscape,
PrintBackground: printBackground,
Scale: scale,
SinglePage: singlePage,
PaperWidth: paperWidth,
PaperHeight: paperHeight,
MarginTop: marginTop,
MarginBottom: marginBottom,
MarginLeft: marginLeft,
MarginRight: marginRight,
PageRanges: pageRanges,
HeaderTemplate: headerTemplate,
FooterTemplate: footerTemplate,
PreferCssPageSize: preferCssPageSize,
}
return form, pdfOptions
}
// FormDataChromiumScreenshotOptions creates [ScreenshotOptions] from the form
// data. Fallback to default value if the considered key is not present.
func FormDataChromiumScreenshotOptions(ctx *api.Context) (*api.FormData, ScreenshotOptions) {
form, options := FormDataChromiumOptions(ctx)
defaultScreenshotOptions := DefaultScreenshotOptions()
var (
format string
quality int
optimizeForSpeed bool
)
form.
Custom("format", func(value string) error {
if value == "" {
format = defaultScreenshotOptions.Format
return nil
}
if value != "png" && value != "jpeg" && value != "webp" {
return fmt.Errorf("wrong value, expected either 'png', 'jpeg' or 'webp'")
}
format = value
return nil
}).
Custom("quality", func(value string) error {
if value == "" {
quality = defaultScreenshotOptions.Quality
return nil
}
intValue, err := strconv.Atoi(value)
if err != nil {
return err
}
if intValue < 0 {
return errors.New("value is negative")
}
if intValue > 100 {
return errors.New("value is superior to 100")
}
quality = intValue
return nil
}).
Bool("optimizeForSpeed", &optimizeForSpeed, defaultScreenshotOptions.OptimizeForSpeed)
screenshotOptions := ScreenshotOptions{
Options: options,
Format: format,
Quality: quality,
OptimizeForSpeed: optimizeForSpeed,
}
return form, screenshotOptions
}
// FormDataChromiumPdfFormats creates [gotenberg.PdfFormats] from the form
// data. Fallback to default value if the considered key is not present.
func FormDataChromiumPdfFormats(form *api.FormData) gotenberg.PdfFormats {
var (
pdfa string
pdfua bool
)
form.
String("pdfa", &pdfa, ""). String("pdfa", &pdfa, "").
Bool("pdfua", &pdfua, false) Bool("pdfua", &pdfua, false)
// FIXME: deprecated.
// pdfa > pdfFormat.
var actualPdfArchive string
if pdfFormat != "" {
ctx.Log().Warn("'pdfFormat' is deprecated; prefer the 'pdfa' form field instead")
actualPdfArchive = pdfFormat
}
if pdfa != "" {
actualPdfArchive = pdfa
}
return gotenberg.PdfFormats{ return gotenberg.PdfFormats{
PdfA: actualPdfArchive, PdfA: pdfa,
PdfUa: pdfua, PdfUa: pdfua,
} }
} }
@@ -171,7 +242,7 @@ func convertUrlRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
Handler: func(c echo.Context) error { Handler: func(c echo.Context) error {
ctx := c.Get("context").(*api.Context) ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumPdfOptions(ctx) form, options := FormDataChromiumPdfOptions(ctx)
pdfFormats := FormDataChromiumPdfFormats(ctx) pdfFormats := FormDataChromiumPdfFormats(form)
var url string var url string
err := form. err := form.
@@ -191,6 +262,35 @@ func convertUrlRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
} }
} }
// screenshotUrlRoute returns an [api.Route] which can take a screenshot from a
// URL.
func screenshotUrlRoute(chromium Api) api.Route {
return api.Route{
Method: http.MethodPost,
Path: "/forms/chromium/screenshot/url",
IsMultipart: true,
Handler: func(c echo.Context) error {
ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumScreenshotOptions(ctx)
var url string
err := form.
MandatoryString("url", &url).
Validate()
if err != nil {
return fmt.Errorf("validate form data: %w", err)
}
err = screenshotUrl(ctx, chromium, url, options)
if err != nil {
return fmt.Errorf("URL screenshot: %w", err)
}
return nil
},
}
}
// convertHtmlRoute returns an [api.Route] which can convert an HTML file to // convertHtmlRoute returns an [api.Route] which can convert an HTML file to
// PDF. // PDF.
func convertHtmlRoute(chromium Api, engine gotenberg.PdfEngine) api.Route { func convertHtmlRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
@@ -201,7 +301,7 @@ func convertHtmlRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
Handler: func(c echo.Context) error { Handler: func(c echo.Context) error {
ctx := c.Get("context").(*api.Context) ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumPdfOptions(ctx) form, options := FormDataChromiumPdfOptions(ctx)
pdfFormats := FormDataChromiumPdfFormats(ctx) pdfFormats := FormDataChromiumPdfFormats(form)
var inputPath string var inputPath string
err := form. err := form.
@@ -222,6 +322,36 @@ func convertHtmlRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
} }
} }
// screenshotHtmlRoute returns an [api.Route] which can take a screenshot from
// an HTML file.
func screenshotHtmlRoute(chromium Api) api.Route {
return api.Route{
Method: http.MethodPost,
Path: "/forms/chromium/screenshot/html",
IsMultipart: true,
Handler: func(c echo.Context) error {
ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumScreenshotOptions(ctx)
var inputPath string
err := form.
MandatoryPath("index.html", &inputPath).
Validate()
if err != nil {
return fmt.Errorf("validate form data: %w", err)
}
url := fmt.Sprintf("file://%s", inputPath)
err = screenshotUrl(ctx, chromium, url, options)
if err != nil {
return fmt.Errorf("HTML screenshot: %w", err)
}
return nil
},
}
}
// convertMarkdownRoute returns an [api.Route] which can convert markdown files // convertMarkdownRoute returns an [api.Route] which can convert markdown files
// to PDF. // to PDF.
func convertMarkdownRoute(chromium Api, engine gotenberg.PdfEngine) api.Route { func convertMarkdownRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
@@ -232,7 +362,7 @@ func convertMarkdownRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
Handler: func(c echo.Context) error { Handler: func(c echo.Context) error {
ctx := c.Get("context").(*api.Context) ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumPdfOptions(ctx) form, options := FormDataChromiumPdfOptions(ctx)
pdfFormats := FormDataChromiumPdfFormats(ctx) pdfFormats := FormDataChromiumPdfFormats(form)
var ( var (
inputPath string inputPath string
@@ -247,79 +377,11 @@ func convertMarkdownRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
return fmt.Errorf("validate form data: %w", err) return fmt.Errorf("validate form data: %w", err)
} }
// We have to convert each markdown file referenced in the HTML url, err := markdownToHtml(ctx, inputPath, markdownPaths)
// file to... HTML. Thanks to the "html/template" package, we are
// able to provide the "toHTML" function which the user may call
// directly inside the HTML file.
var markdownFilesNotFoundErr error
tmpl, err := template.
New(filepath.Base(inputPath)).
Funcs(template.FuncMap{
"toHTML": func(filename string) (template.HTML, error) {
var path string
for _, markdownPath := range markdownPaths {
markdownFilename := filepath.Base(markdownPath)
if filename == markdownFilename {
path = markdownPath
break
}
}
if path == "" {
markdownFilesNotFoundErr = multierr.Append(
markdownFilesNotFoundErr,
fmt.Errorf("'%s'", filename),
)
return "", nil
}
b, err := os.ReadFile(path)
if err != nil {
return "", fmt.Errorf("read markdown file '%s': %w", filename, err)
}
unsafe := blackfriday.Run(b)
sanitized := bluemonday.UGCPolicy().SanitizeBytes(unsafe)
// #nosec
return template.HTML(sanitized), nil
},
}).ParseFiles(inputPath)
if err != nil { if err != nil {
return fmt.Errorf("parse template file: %w", err) return fmt.Errorf("transform markdown file(s) to HTML: %w", err)
} }
var buffer bytes.Buffer
err = tmpl.Execute(&buffer, &struct{}{})
if err != nil {
return fmt.Errorf("execute template: %w", err)
}
if markdownFilesNotFoundErr != nil {
return api.WrapError(
fmt.Errorf("markdown files not found: %w", markdownFilesNotFoundErr),
api.NewSentinelHttpError(
http.StatusBadRequest,
fmt.Sprintf("Markdown file(s) not found: %s", markdownFilesNotFoundErr),
),
)
}
inputPath = ctx.GeneratePath(".html")
err = os.WriteFile(inputPath, buffer.Bytes(), 0o600)
if err != nil {
return fmt.Errorf("write template result: %w", err)
}
url := fmt.Sprintf("file://%s", inputPath)
err = convertUrl(ctx, chromium, engine, url, pdfFormats, options) err = convertUrl(ctx, chromium, engine, url, pdfFormats, options)
if err != nil { if err != nil {
return fmt.Errorf("convert markdown to PDF: %w", err) return fmt.Errorf("convert markdown to PDF: %w", err)
@@ -330,23 +392,126 @@ func convertMarkdownRoute(chromium Api, engine gotenberg.PdfEngine) api.Route {
} }
} }
// convertUrl is a stub which is called by the other methods of this file. // screenshotMarkdownRoute returns an [api.Route] which can take a screenshot
func convertUrl(ctx *api.Context, chromium Api, engine gotenberg.PdfEngine, url string, pdfFormats gotenberg.PdfFormats, options Options) error { // from markdown files.
func screenshotMarkdownRoute(chromium Api) api.Route {
return api.Route{
Method: http.MethodPost,
Path: "/forms/chromium/screenshot/markdown",
IsMultipart: true,
Handler: func(c echo.Context) error {
ctx := c.Get("context").(*api.Context)
form, options := FormDataChromiumScreenshotOptions(ctx)
var (
inputPath string
markdownPaths []string
)
err := form.
MandatoryPath("index.html", &inputPath).
MandatoryPaths([]string{".md"}, &markdownPaths).
Validate()
if err != nil {
return fmt.Errorf("validate form data: %w", err)
}
url, err := markdownToHtml(ctx, inputPath, markdownPaths)
if err != nil {
return fmt.Errorf("transform markdown file(s) to HTML: %w", err)
}
err = screenshotUrl(ctx, chromium, url, options)
if err != nil {
return fmt.Errorf("markdown screenshot: %w", err)
}
return nil
},
}
}
func markdownToHtml(ctx *api.Context, inputPath string, markdownPaths []string) (string, error) {
// We have to convert each markdown file referenced in the HTML
// file to... HTML. Thanks to the "html/template" package, we are
// able to provide the "toHTML" function which the user may call
// directly inside the HTML file.
var markdownFilesNotFoundErr error
tmpl, err := template.
New(filepath.Base(inputPath)).
Funcs(template.FuncMap{
"toHTML": func(filename string) (template.HTML, error) {
var path string
for _, markdownPath := range markdownPaths {
markdownFilename := filepath.Base(markdownPath)
if filename == markdownFilename {
path = markdownPath
break
}
}
if path == "" {
markdownFilesNotFoundErr = multierr.Append(
markdownFilesNotFoundErr,
fmt.Errorf("'%s'", filename),
)
return "", nil
}
b, err := os.ReadFile(path)
if err != nil {
return "", fmt.Errorf("read markdown file '%s': %w", filename, err)
}
unsafe := blackfriday.Run(b)
sanitized := bluemonday.UGCPolicy().SanitizeBytes(unsafe)
// #nosec
return template.HTML(sanitized), nil
},
}).ParseFiles(inputPath)
if err != nil {
return "", fmt.Errorf("parse template file: %w", err)
}
var buffer bytes.Buffer
err = tmpl.Execute(&buffer, &struct{}{})
if err != nil {
return "", fmt.Errorf("execute template: %w", err)
}
if markdownFilesNotFoundErr != nil {
return "", api.WrapError(
fmt.Errorf("markdown files not found: %w", markdownFilesNotFoundErr),
api.NewSentinelHttpError(
http.StatusBadRequest,
fmt.Sprintf("Markdown file(s) not found: %s", markdownFilesNotFoundErr),
),
)
}
inputPath = ctx.GeneratePath(".html")
err = os.WriteFile(inputPath, buffer.Bytes(), 0o600)
if err != nil {
return "", fmt.Errorf("write template result: %w", err)
}
return fmt.Sprintf("file://%s", inputPath), nil
}
func convertUrl(ctx *api.Context, chromium Api, engine gotenberg.PdfEngine, url string, pdfFormats gotenberg.PdfFormats, options PdfOptions) error {
outputPath := ctx.GeneratePath(".pdf") outputPath := ctx.GeneratePath(".pdf")
err := chromium.Pdf(ctx, ctx.Log(), url, outputPath, options) err := chromium.Pdf(ctx, ctx.Log(), url, outputPath, options)
err = handleChromiumError(err, options.Options)
if err != nil { if err != nil {
if errors.Is(err, ErrUrlNotAuthorized) {
return api.WrapError(
fmt.Errorf("convert to PDF: %w", err),
api.NewSentinelHttpError(
http.StatusForbidden,
fmt.Sprintf("'%s' does not match the authorized URLs", url),
),
)
}
if errors.Is(err, ErrOmitBackgroundWithoutPrintBackground) { if errors.Is(err, ErrOmitBackgroundWithoutPrintBackground) {
return api.WrapError( return api.WrapError(
fmt.Errorf("convert to PDF: %w", err), fmt.Errorf("convert to PDF: %w", err),
@@ -357,23 +522,6 @@ func convertUrl(ctx *api.Context, chromium Api, engine gotenberg.PdfEngine, url
) )
} }
if errors.Is(err, ErrInvalidEvaluationExpression) {
if options.WaitForExpression == "" {
// We do not expect the 'waitWindowStatus' form field to return
// an ErrInvalidEvaluationExpression error. In such a scenario,
// we return a 500.
return fmt.Errorf("convert to PDF: %w", err)
}
return api.WrapError(
fmt.Errorf("convert to PDF: %w", err),
api.NewSentinelHttpError(
http.StatusBadRequest,
fmt.Sprintf("The expression '%s' (waitForExpression) returned an exception or undefined", options.WaitForExpression),
),
)
}
if errors.Is(err, ErrInvalidPrinterSettings) { if errors.Is(err, ErrInvalidPrinterSettings) {
return api.WrapError( return api.WrapError(
fmt.Errorf("convert to PDF: %w", err), fmt.Errorf("convert to PDF: %w", err),
@@ -394,16 +542,6 @@ func convertUrl(ctx *api.Context, chromium Api, engine gotenberg.PdfEngine, url
) )
} }
if errors.Is(err, ErrConsoleExceptions) {
return api.WrapError(
fmt.Errorf("convert to PDF: %w", err),
api.NewSentinelHttpError(
http.StatusConflict,
fmt.Sprintf("Chromium console exceptions:\n %s", strings.ReplaceAll(err.Error(), ErrConsoleExceptions.Error(), "")),
),
)
}
return fmt.Errorf("convert to PDF: %w", err) return fmt.Errorf("convert to PDF: %w", err)
} }
@@ -416,18 +554,7 @@ func convertUrl(ctx *api.Context, chromium Api, engine gotenberg.PdfEngine, url
convertOutputPath := ctx.GeneratePath(".pdf") convertOutputPath := ctx.GeneratePath(".pdf")
err = engine.Convert(ctx, ctx.Log(), pdfFormats, convertInputPath, convertOutputPath) err = engine.Convert(ctx, ctx.Log(), pdfFormats, convertInputPath, convertOutputPath)
if err != nil { if err != nil {
if errors.Is(err, gotenberg.ErrPdfFormatNotSupported) {
return api.WrapError(
fmt.Errorf("convert PDF: %w", err),
api.NewSentinelHttpError(
http.StatusBadRequest,
fmt.Sprintf("At least one PDF engine does not handle one of the PDF format in '%+v', while other have failed to convert for other reasons", pdfFormats),
),
)
}
return fmt.Errorf("convert PDF: %w", err) return fmt.Errorf("convert PDF: %w", err)
} }
@@ -442,3 +569,66 @@ func convertUrl(ctx *api.Context, chromium Api, engine gotenberg.PdfEngine, url
return nil return nil
} }
func screenshotUrl(ctx *api.Context, chromium Api, url string, options ScreenshotOptions) error {
ext := fmt.Sprintf(".%s", options.Format)
outputPath := ctx.GeneratePath(ext)
err := chromium.Screenshot(ctx, ctx.Log(), url, outputPath, options)
err = handleChromiumError(err, options.Options)
if err != nil {
return fmt.Errorf("screenshot: %w", err)
}
err = ctx.AddOutputPaths(outputPath)
if err != nil {
return fmt.Errorf("add output path: %w", err)
}
return nil
}
func handleChromiumError(err error, options Options) error {
if err == nil {
return nil
}
if errors.Is(err, ErrInvalidEvaluationExpression) {
if options.WaitForExpression == "" {
// We do not expect the 'waitWindowStatus' form field to return
// an ErrInvalidEvaluationExpression error. In such a scenario,
// we return a 500.
return err
}
return api.WrapError(
err,
api.NewSentinelHttpError(
http.StatusBadRequest,
fmt.Sprintf("The expression '%s' (waitForExpression) returned an exception or undefined", options.WaitForExpression),
),
)
}
if errors.Is(err, ErrInvalidHttpStatusCode) {
return api.WrapError(
err,
api.NewSentinelHttpError(
http.StatusConflict,
fmt.Sprintf("Invalid HTTP status code from the main page: %s", strings.ReplaceAll(err.Error(), fmt.Sprintf(": %s", ErrInvalidHttpStatusCode.Error()), "")),
),
)
}
if errors.Is(err, ErrConsoleExceptions) {
return api.WrapError(
err,
api.NewSentinelHttpError(
http.StatusConflict,
fmt.Sprintf("Chromium console exceptions:\n %s", strings.ReplaceAll(err.Error(), ErrConsoleExceptions.Error(), "")),
),
)
}
return err
}

File diff suppressed because it is too large Load Diff

View File

@@ -15,24 +15,44 @@ import (
"go.uber.org/zap" "go.uber.org/zap"
) )
func printToPdfActionFunc(logger *zap.Logger, outputPath string, options Options) chromedp.ActionFunc { func printToPdfActionFunc(logger *zap.Logger, outputPath string, options PdfOptions) chromedp.ActionFunc {
return func(ctx context.Context) error { return func(ctx context.Context) error {
paperHeight := options.PaperHeight
pageRanges := options.PageRanges
if options.SinglePage {
logger.Debug("single page PDF")
_, _, _, _, _, cssContentSize, err := page.GetLayoutMetrics().Do(ctx)
if err != nil {
return fmt.Errorf("get layout metrics: %w", err)
}
// There are 96 CSS pixels per inch.
// See https://issues.chromium.org/issues/40267771#comment14.
paperHeight = cssContentSize.Height / 96
pageRanges = "1" // little dirty hack to avoid leftovers.
}
printToPdf := page.PrintToPDF(). printToPdf := page.PrintToPDF().
WithTransferMode(page.PrintToPDFTransferModeReturnAsStream). WithTransferMode(page.PrintToPDFTransferModeReturnAsStream).
WithLandscape(options.Landscape). WithLandscape(options.Landscape).
WithPrintBackground(options.PrintBackground). WithPrintBackground(options.PrintBackground).
WithScale(options.Scale). WithScale(options.Scale).
WithPaperWidth(options.PaperWidth). WithPaperWidth(options.PaperWidth).
WithPaperHeight(options.PaperHeight). WithPaperHeight(paperHeight).
WithMarginTop(options.MarginTop). WithMarginTop(options.MarginTop).
WithMarginBottom(options.MarginBottom). WithMarginBottom(options.MarginBottom).
WithMarginLeft(options.MarginLeft). WithMarginLeft(options.MarginLeft).
WithMarginRight(options.MarginRight). WithMarginRight(options.MarginRight).
WithPageRanges(options.PageRanges). WithPageRanges(pageRanges).
WithPreferCSSPageSize(options.PreferCssPageSize) WithPreferCSSPageSize(options.PreferCssPageSize).
// Does not seem to work.
// See https://github.com/gotenberg/gotenberg/issues/831.
WithGenerateTaggedPDF(false)
hasCustomHeaderFooter := options.HeaderTemplate != DefaultOptions().HeaderTemplate || hasCustomHeaderFooter := options.HeaderTemplate != DefaultPdfOptions().HeaderTemplate ||
options.FooterTemplate != DefaultOptions().FooterTemplate options.FooterTemplate != DefaultPdfOptions().FooterTemplate
if !hasCustomHeaderFooter { if !hasCustomHeaderFooter {
logger.Debug("no custom header nor footer") logger.Debug("no custom header nor footer")
@@ -47,11 +67,11 @@ func printToPdfActionFunc(logger *zap.Logger, outputPath string, options Options
WithFooterTemplate(options.FooterTemplate) WithFooterTemplate(options.FooterTemplate)
} }
logger.Debug(fmt.Sprintf("print to Pdf with: %+v", printToPdf)) logger.Debug(fmt.Sprintf("print to PDF with: %+v", printToPdf))
_, stream, err := printToPdf.Do(ctx) _, stream, err := printToPdf.Do(ctx)
if err != nil { if err != nil {
return fmt.Errorf("print to Pdf: %w", err) return fmt.Errorf("print to PDF: %w", err)
} }
reader := &streamReader{ reader := &streamReader{
@@ -63,7 +83,7 @@ func printToPdfActionFunc(logger *zap.Logger, outputPath string, options Options
} }
defer func() { defer func() {
err := reader.Close() err = reader.Close()
if err != nil { if err != nil {
logger.Error(fmt.Sprintf("close reader: %s", err)) logger.Error(fmt.Sprintf("close reader: %s", err))
} }
@@ -75,7 +95,7 @@ func printToPdfActionFunc(logger *zap.Logger, outputPath string, options Options
} }
defer func() { defer func() {
err := file.Close() err = file.Close()
if err != nil { if err != nil {
logger.Error(fmt.Sprintf("close output path: %s", err)) logger.Error(fmt.Sprintf("close output path: %s", err))
} }
@@ -92,12 +112,90 @@ func printToPdfActionFunc(logger *zap.Logger, outputPath string, options Options
} }
} }
func captureScreenshotActionFunc(logger *zap.Logger, outputPath string, options ScreenshotOptions) chromedp.ActionFunc {
return func(ctx context.Context) error {
captureScreenshot := page.CaptureScreenshot().
WithCaptureBeyondViewport(true).
WithFromSurface(true).
WithOptimizeForSpeed(options.OptimizeForSpeed).
WithFormat(page.CaptureScreenshotFormat(options.Format))
if options.Format == "jpeg" {
captureScreenshot = captureScreenshot.
WithQuality(int64(options.Quality))
}
logger.Debug(fmt.Sprintf("capture screenshot with: %+v", captureScreenshot))
buffer, err := captureScreenshot.Do(ctx)
if err != nil {
return fmt.Errorf("capture screenshot: %w", err)
}
file, err := os.OpenFile(outputPath, os.O_CREATE|os.O_WRONLY, 0o600)
if err != nil {
return fmt.Errorf("open output path: %w", err)
}
defer func() {
err = file.Close()
if err != nil {
logger.Error(fmt.Sprintf("close output path: %s", err))
}
}()
_, err = file.Write(buffer)
if err != nil {
return fmt.Errorf("write result to output path: %w", err)
}
return nil
}
}
func clearCacheActionFunc(logger *zap.Logger, clear bool) chromedp.ActionFunc {
return func(ctx context.Context) error {
// See https://github.com/gotenberg/gotenberg/issues/753.
if !clear {
logger.Debug("cache not cleared")
return nil
}
logger.Debug("clear cache")
err := network.ClearBrowserCache().Do(ctx)
if err == nil {
return nil
}
return fmt.Errorf("clear cache: %w", err)
}
}
func clearCookiesActionFunc(logger *zap.Logger, clear bool) chromedp.ActionFunc {
return func(ctx context.Context) error {
// See https://github.com/gotenberg/gotenberg/issues/753.
if !clear {
logger.Debug("cookies not cleared")
return nil
}
logger.Debug("clear cookies")
err := network.ClearBrowserCookies().Do(ctx)
if err == nil {
return nil
}
return fmt.Errorf("clear cookies: %w", err)
}
}
func disableJavaScriptActionFunc(logger *zap.Logger, disable bool) chromedp.ActionFunc { func disableJavaScriptActionFunc(logger *zap.Logger, disable bool) chromedp.ActionFunc {
return func(ctx context.Context) error { return func(ctx context.Context) error {
// See https://github.com/gotenberg/gotenberg/issues/175. // See https://github.com/gotenberg/gotenberg/issues/175.
if !disable { if !disable {
logger.Debug("JavaScript not disabled") logger.Debug("JavaScript not disabled")
return nil return nil
} }
@@ -116,7 +214,6 @@ func extraHttpHeadersActionFunc(logger *zap.Logger, extraHttpHeaders map[string]
return func(ctx context.Context) error { return func(ctx context.Context) error {
if len(extraHttpHeaders) == 0 { if len(extraHttpHeaders) == 0 {
logger.Debug("no extra HTTP headers") logger.Debug("no extra HTTP headers")
return nil return nil
} }
@@ -136,7 +233,7 @@ func extraHttpHeadersActionFunc(logger *zap.Logger, extraHttpHeaders map[string]
} }
} }
func navigateActionFunc(logger *zap.Logger, url string) chromedp.ActionFunc { func navigateActionFunc(logger *zap.Logger, url string, skipNetworkIdleEvent bool) chromedp.ActionFunc {
return func(ctx context.Context) error { return func(ctx context.Context) error {
logger.Debug(fmt.Sprintf("navigate to '%s'", url)) logger.Debug(fmt.Sprintf("navigate to '%s'", url))
@@ -145,12 +242,21 @@ func navigateActionFunc(logger *zap.Logger, url string) chromedp.ActionFunc {
return fmt.Errorf("navigate to '%s': %w", url, err) return fmt.Errorf("navigate to '%s': %w", url, err)
} }
err = runBatch( waitFunc := []func() error{
ctx,
waitForEventDomContentEventFired(ctx, logger), waitForEventDomContentEventFired(ctx, logger),
waitForEventLoadEventFired(ctx, logger), waitForEventLoadEventFired(ctx, logger),
waitForEventNetworkIdle(ctx, logger),
waitForEventLoadingFinished(ctx, logger), waitForEventLoadingFinished(ctx, logger),
}
if !skipNetworkIdleEvent {
waitFunc = append(waitFunc, waitForEventNetworkIdle(ctx, logger))
} else {
logger.Debug("skipping network idle event")
}
err = runBatch(
ctx,
waitFunc...,
) )
if err == nil { if err == nil {
@@ -166,7 +272,6 @@ func hideDefaultWhiteBackgroundActionFunc(logger *zap.Logger, omitBackground, pr
// See https://github.com/gotenberg/gotenberg/issues/226. // See https://github.com/gotenberg/gotenberg/issues/226.
if !omitBackground { if !omitBackground {
logger.Debug("default white background not hidden") logger.Debug("default white background not hidden")
return nil return nil
} }
@@ -225,7 +330,6 @@ func emulateMediaTypeActionFunc(logger *zap.Logger, mediaType string) chromedp.A
return func(ctx context.Context) error { return func(ctx context.Context) error {
if mediaType == "" { if mediaType == "" {
logger.Debug("no emulated media type") logger.Debug("no emulated media type")
return nil return nil
} }
@@ -249,13 +353,11 @@ func waitDelayBeforePrintActionFunc(logger *zap.Logger, disableJavaScript bool,
return func(ctx context.Context) error { return func(ctx context.Context) error {
if disableJavaScript { if disableJavaScript {
logger.Debug("JavaScript disabled, skipping wait delay") logger.Debug("JavaScript disabled, skipping wait delay")
return nil return nil
} }
if delay <= 0 { if delay <= 0 {
logger.Debug("no wait delay") logger.Debug("no wait delay")
return nil return nil
} }
@@ -276,13 +378,11 @@ func waitForExpressionBeforePrintActionFunc(logger *zap.Logger, disableJavaScrip
return func(ctx context.Context) error { return func(ctx context.Context) error {
if disableJavaScript { if disableJavaScript {
logger.Debug("JavaScript disabled, skipping wait expression") logger.Debug("JavaScript disabled, skipping wait expression")
return nil return nil
} }
if expression == "" { if expression == "" {
logger.Debug("no wait expression") logger.Debug("no wait expression")
return nil return nil
} }
@@ -295,12 +395,11 @@ func waitForExpressionBeforePrintActionFunc(logger *zap.Logger, disableJavaScrip
select { select {
case <-ctx.Done(): case <-ctx.Done():
ticker.Stop() ticker.Stop()
return fmt.Errorf("context done while evaluating '%s': %w", expression, ctx.Err()) return fmt.Errorf("context done while evaluating '%s': %w", expression, ctx.Err())
case <-ticker.C: case <-ticker.C:
var ok bool var ok bool
evaluate := chromedp.Evaluate(expression, &ok) evaluate := chromedp.Evaluate(expression, &ok)
err := evaluate.Do(ctx) err := evaluate.Do(ctx)
if err != nil { if err != nil {
return fmt.Errorf("evaluate: %v: %w", err, ErrInvalidEvaluationExpression) return fmt.Errorf("evaluate: %v: %w", err, ErrInvalidEvaluationExpression)
@@ -308,7 +407,6 @@ func waitForExpressionBeforePrintActionFunc(logger *zap.Logger, disableJavaScrip
if ok { if ok {
ticker.Stop() ticker.Stop()
return nil return nil
} }

View File

@@ -12,8 +12,8 @@ import (
"go.uber.org/multierr" "go.uber.org/multierr"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
func init() { func init() {
@@ -21,9 +21,9 @@ func init() {
} }
var ( var (
// ErrInvalidPdfFormat happens if the PDF format option cannot be handled // ErrInvalidPdfFormats happens if the PDF formats option cannot be handled
// by LibreOffice. // by LibreOffice.
ErrInvalidPdfFormat = errors.New("invalid PDF format") ErrInvalidPdfFormats = errors.New("invalid PDF formats")
// ErrMalformedPageRanges happens if the page ranges option cannot be // ErrMalformedPageRanges happens if the page ranges option cannot be
// interpreted by LibreOffice. // interpreted by LibreOffice.
@@ -47,11 +47,10 @@ type Options struct {
Landscape bool Landscape bool
// PageRanges allows to select the pages to convert. // PageRanges allows to select the pages to convert.
// TODO: should prefer a method form PdfEngine.
// Optional. // Optional.
PageRanges string PageRanges string
// PdfFormats allows to convert the resulting PDF to PDF/A-1a, PDF/A-2b, // PdfFormats allows to convert the resulting PDF to PDF/A-1b, PDF/A-2b,
// PDF/A-3b and PDF/UA. // PDF/A-3b and PDF/UA.
// Optional. // Optional.
PdfFormats gotenberg.PdfFormats PdfFormats gotenberg.PdfFormats
@@ -80,24 +79,10 @@ func (a *Api) Descriptor() gotenberg.ModuleDescriptor {
ID: "libreoffice-api", ID: "libreoffice-api",
FlagSet: func() *flag.FlagSet { FlagSet: func() *flag.FlagSet {
fs := flag.NewFlagSet("api", flag.ExitOnError) fs := flag.NewFlagSet("api", flag.ExitOnError)
// Deprecated flags.
fs.Duration("uno-listener-start-timeout", time.Duration(10)*time.Second, "Time limit for restarting the LibreOffice")
fs.Int64("uno-listener-restart-threshold", 10, "Conversions limit after which the LibreOffice listener is restarted - 0 means no restart")
fs.Bool("unoconv-disable-listener", false, "Do not start a long-running listener - save resources in detriment of unitary performance")
var err error
err = multierr.Append(err, fs.MarkDeprecated("uno-listener-start-timeout", "use the libreOffice-start-timeout property instead"))
err = multierr.Append(err, fs.MarkDeprecated("uno-listener-restart-threshold", "use the libreOffice-restart-after property instead"))
err = multierr.Append(err, fs.MarkDeprecated("unoconv-disable-listener", "use the libreOffice-auto-start property instead"))
if err != nil {
panic(fmt.Errorf("create deprecated flags for the LibreOffice module: %v", err))
}
fs.Int64("libreoffice-restart-after", 10, "Number of conversions after which LibreOffice will automatically restart. Set to 0 to disable this feature") fs.Int64("libreoffice-restart-after", 10, "Number of conversions after which LibreOffice will automatically restart. Set to 0 to disable this feature")
fs.Int64("libreoffice-max-queue-size", 0, "Maximum request queue size for LibreOffice. Set to 0 to disable this feature")
fs.Bool("libreoffice-auto-start", false, "Automatically launch LibreOffice upon initialization if set to true; otherwise, LibreOffice will start at the time of the first conversion") fs.Bool("libreoffice-auto-start", false, "Automatically launch LibreOffice upon initialization if set to true; otherwise, LibreOffice will start at the time of the first conversion")
fs.Duration("libreoffice-start-timeout", time.Duration(10)*time.Second, "Maximum duration to wait for LibreOffice to start or restart") fs.Duration("libreoffice-start-timeout", time.Duration(20)*time.Second, "Maximum duration to wait for LibreOffice to start or restart")
return fs return fs
}(), }(),
@@ -123,7 +108,7 @@ func (a *Api) Provision(ctx *gotenberg.Context) error {
a.args = libreOfficeArguments{ a.args = libreOfficeArguments{
binPath: libreOfficeBinPath, binPath: libreOfficeBinPath,
unoBinPath: unoBinPath, unoBinPath: unoBinPath,
startTimeout: flags.MustDeprecatedDuration("uno-listener-start-timeout", "libreoffice-start-timeout"), startTimeout: flags.MustDuration("libreoffice-start-timeout"),
} }
// Logger. // Logger.
@@ -139,7 +124,7 @@ func (a *Api) Provision(ctx *gotenberg.Context) error {
// Process. // Process.
a.libreOffice = newLibreOfficeProcess(a.args) a.libreOffice = newLibreOfficeProcess(a.args)
a.supervisor = gotenberg.NewProcessSupervisor(a.logger, a.libreOffice, flags.MustDeprecatedInt64("uno-listener-restart-threshold", "libreoffice-restart-after")) a.supervisor = gotenberg.NewProcessSupervisor(a.logger, a.libreOffice, flags.MustInt64("libreoffice-restart-after"), flags.MustInt64("libreoffice-max-queue-size"))
return nil return nil
} }
@@ -204,46 +189,6 @@ func (a *Api) Stop(ctx context.Context) error {
// Metrics returns the metrics. // Metrics returns the metrics.
func (a *Api) Metrics() ([]gotenberg.Metric, error) { func (a *Api) Metrics() ([]gotenberg.Metric, error) {
return []gotenberg.Metric{ return []gotenberg.Metric{
// TODO: remove deprecated.
{
Name: "unoconv_active_instances_count",
Description: "Current number of active unoconv instances - deprecated.",
Read: func() float64 {
return 1
},
},
// TODO: remove deprecated.
{
Name: "libreoffice_listener_active_instances_count",
Description: "Current number of active LibreOffice listener instances - deprecated.",
Read: func() float64 {
return 1
},
},
// TODO: remove deprecated.
{
Name: "unoconv_listener_active_instances_count",
Description: "Current number of active unoconv listener instances- deprecated.",
Read: func() float64 {
return 1
},
},
// TODO: remove deprecated.
{
Name: "libreoffice_listener_queue_length",
Description: "Current number of processes in the LibreOffice listener queue - deprecated, prefer libreoffice_requests_queue_size.",
Read: func() float64 {
return float64(a.supervisor.ReqQueueSize())
},
},
// TODO: remove deprecated.
{
Name: "unoconv_listener_queue_length",
Description: "Current number of processes in the queue - deprecated, prefer libreoffice_requests_queue_size.",
Read: func() float64 {
return float64(a.supervisor.ReqQueueSize())
},
},
{ {
Name: "libreoffice_requests_queue_size", Name: "libreoffice_requests_queue_size",
Description: "Current number of LibreOffice conversion requests waiting to be treated.", Description: "Current number of LibreOffice conversion requests waiting to be treated.",
@@ -277,6 +222,34 @@ func (a *Api) Checks() ([]health.CheckerOption, error) {
}, nil }, nil
} }
// Ready returns no error if the module is ready.
func (a *Api) Ready() error {
if !a.autoStart {
return nil
}
ctx, cancel := context.WithTimeout(context.Background(), a.args.startTimeout)
defer cancel()
ticker := time.NewTicker(time.Duration(100) * time.Millisecond)
for {
select {
case <-ctx.Done():
ticker.Stop()
return fmt.Errorf("context done while waiting for LibreOffice to be ready: %w", ctx.Err())
case <-ticker.C:
ok := a.libreOffice.Healthy(a.logger)
if ok {
ticker.Stop()
return nil
}
continue
}
}
}
// LibreOffice returns a [Uno] for interacting with LibreOffice. // LibreOffice returns a [Uno] for interacting with LibreOffice.
func (a *Api) LibreOffice() (Uno, error) { func (a *Api) LibreOffice() (Uno, error) {
return a, nil return a, nil
@@ -294,84 +267,86 @@ func (a *Api) Pdf(ctx context.Context, logger *zap.Logger, inputPath, outputPath
func (a *Api) Extensions() []string { func (a *Api) Extensions() []string {
return []string{ return []string{
".bib", ".bib",
".doc",
".xml",
".docx",
".fodt",
".html",
".ltx",
".txt",
".odt",
".ott",
".pdb",
".pdf",
".psw",
".rtf",
".sdw",
".stw",
".sxw",
".uot",
".vor",
".wps",
".epub",
".png",
".bmp", ".bmp",
".emf",
".eps",
".fodg",
".gif",
".jpg",
".jpeg",
".met",
".odd",
".otg",
".pbm",
".pct",
".pgm",
".ppm",
".ras",
".std",
".svg",
".svm",
".swf",
".sxd",
".sxw",
".tif",
".tiff",
".xhtml",
".xpm",
".odp",
".fodp",
".potm",
".pot",
".pptx",
".pps",
".ppt",
".pwp",
".sda",
".sdd",
".sti",
".sxi",
".uop",
".wmf",
".csv", ".csv",
".dbf", ".dbf",
".dif", ".dif",
".doc",
".docx",
".dotx",
".emf",
".eps",
".epub",
".fodg",
".fodp",
".fods", ".fods",
".fodt",
".gif",
".html",
".jpeg",
".jpg",
".key",
".ltx",
".met",
".odd",
".odg",
".odp",
".ods", ".ods",
".odt",
".otg",
".ots", ".ots",
".ott",
".pages",
".pbm",
".pct",
".pdb",
".pdf",
".pgm",
".png",
".pot",
".potm",
".ppm",
".pps",
".ppt",
".pptx",
".psw",
".pwp",
".pxl", ".pxl",
".ras",
".rtf",
".sda",
".sdc", ".sdc",
".sdd",
".sdw",
".slk", ".slk",
".stc", ".stc",
".std",
".sti",
".stw",
".svg",
".svm",
".swf",
".sxc", ".sxc",
".sxd",
".sxi",
".sxw",
".sxw",
".tif",
".tiff",
".txt",
".uop",
".uos", ".uos",
".uot",
".vor",
".wmf",
".wps",
".xhtml",
".xls", ".xls",
".xlt",
".xlsx", ".xlsx",
".odg", ".xlt",
".dotx",
".xltx", ".xltx",
".xml",
".xpm",
} }
} }

View File

@@ -11,7 +11,7 @@ import (
"github.com/alexliesenfeld/health" "github.com/alexliesenfeld/health"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestApi_Descriptor(t *testing.T) { func TestApi_Descriptor(t *testing.T) {
@@ -284,41 +284,16 @@ func TestApi_Metrics(t *testing.T) {
t.Fatalf("expected no error but got: %v", err) t.Fatalf("expected no error but got: %v", err)
} }
if len(metrics) != 7 { if len(metrics) != 2 {
t.Fatalf("expected %d metrics, but got %d", 7, len(metrics)) t.Fatalf("expected %d metrics, but got %d", 2, len(metrics))
} }
actual := metrics[0].Read() actual := metrics[0].Read()
if actual != float64(1) {
t.Errorf("expected %f for unoconv_active_instances_count, but got %f", float64(1), actual)
}
actual = metrics[1].Read()
if actual != float64(1) {
t.Errorf("expected %f for libreoffice_listener_active_instances_count, but got %f", float64(1), actual)
}
actual = metrics[2].Read()
if actual != float64(1) {
t.Errorf("expected %f for unoconv_listener_active_instances_count, but got %f", float64(1), actual)
}
actual = metrics[3].Read()
if actual != float64(10) {
t.Errorf("expected %f for libreoffice_listener_queue_length, but got %f", float64(10), actual)
}
actual = metrics[4].Read()
if actual != float64(10) {
t.Errorf("expected %f for unoconv_listener_queue_length, but got %f", float64(10), actual)
}
actual = metrics[5].Read()
if actual != float64(10) { if actual != float64(10) {
t.Errorf("expected %f for libreoffice_requests_queue_size, but got %f", float64(10), actual) t.Errorf("expected %f for libreoffice_requests_queue_size, but got %f", float64(10), actual)
} }
actual = metrics[6].Read() actual = metrics[1].Read()
if actual != float64(0) { if actual != float64(0) {
t.Errorf("expected %f for libreoffice_restarts_count, but got %f", float64(0), actual) t.Errorf("expected %f for libreoffice_restarts_count, but got %f", float64(0), actual)
} }
@@ -364,6 +339,61 @@ func TestApi_Checks(t *testing.T) {
} }
} }
func TestChromium_Ready(t *testing.T) {
for _, tc := range []struct {
scenario string
autoStart bool
startTimeout time.Duration
libreOffice libreOffice
expectError bool
}{
{
scenario: "no auto-start",
autoStart: false,
startTimeout: time.Duration(30) * time.Second,
libreOffice: &libreOfficeMock{ProcessMock: gotenberg.ProcessMock{HealthyMock: func(logger *zap.Logger) bool {
return false
}}},
expectError: false,
},
{
scenario: "auto-start: context done",
autoStart: true,
startTimeout: time.Duration(200) * time.Millisecond,
libreOffice: &libreOfficeMock{ProcessMock: gotenberg.ProcessMock{HealthyMock: func(logger *zap.Logger) bool {
return false
}}},
expectError: true,
},
{
scenario: "auto-start success",
autoStart: true,
startTimeout: time.Duration(30) * time.Second,
libreOffice: &libreOfficeMock{ProcessMock: gotenberg.ProcessMock{HealthyMock: func(logger *zap.Logger) bool {
return true
}}},
expectError: false,
},
} {
t.Run(tc.scenario, func(t *testing.T) {
a := new(Api)
a.autoStart = tc.autoStart
a.args = libreOfficeArguments{startTimeout: tc.startTimeout}
a.libreOffice = tc.libreOffice
err := a.Ready()
if !tc.expectError && err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if tc.expectError && err == nil {
t.Fatal("expected error but got none")
}
})
}
}
func TestApi_LibreOffice(t *testing.T) { func TestApi_LibreOffice(t *testing.T) {
a := new(Api) a := new(Api)
@@ -420,7 +450,7 @@ func TestApi_Extensions(t *testing.T) {
extensions := a.Extensions() extensions := a.Extensions()
actual := len(extensions) actual := len(extensions)
expect := 79 expect := 81
if actual != expect { if actual != expect {
t.Errorf("expected %d extensions, but got %d", expect, actual) t.Errorf("expected %d extensions, but got %d", expect, actual)

View File

@@ -4,15 +4,18 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"io"
"net" "net"
"os" "os"
"path/filepath"
"sync" "sync"
"sync/atomic" "sync/atomic"
"time" "time"
"github.com/google/uuid"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
type libreOffice interface { type libreOffice interface {
@@ -272,22 +275,33 @@ func (p *libreOfficeProcess) pdf(ctx context.Context, logger *zap.Logger, inputP
switch options.PdfFormats.PdfA { switch options.PdfFormats.PdfA {
case "": case "":
case gotenberg.PdfA1a: case gotenberg.PdfA1b:
args = append(args, "--export", "SelectPdfVersion=1") args = append(args, "--export", "SelectPdfVersion=1")
case gotenberg.PdfA2b: case gotenberg.PdfA2b:
args = append(args, "--export", "SelectPdfVersion=2") args = append(args, "--export", "SelectPdfVersion=2")
case gotenberg.PdfA3b: case gotenberg.PdfA3b:
args = append(args, "--export", "SelectPdfVersion=3") args = append(args, "--export", "SelectPdfVersion=3")
default: default:
return ErrInvalidPdfFormat return ErrInvalidPdfFormats
} }
if options.PdfFormats.PdfUa { if options.PdfFormats.PdfUa {
args = append( args = append(
args, args,
"--export", "EnableTextAccessForAccessibilityTools=true",
"--export", "UseTaggedPDF=true", "--export", "UseTaggedPDF=true",
"--export", "EnableTextAccessForAccessibilityTools=true",
) )
} else {
args = append(
args,
"--export", "UseTaggedPDF=false",
"--export", "EnableTextAccessForAccessibilityTools=false",
)
}
inputPath, err := nonBasicLatinCharactersGuard(logger, inputPath)
if err != nil {
return fmt.Errorf("non-basic latin characters guard: %w", err)
} }
args = append(args, "--output", outputPath, inputPath) args = append(args, "--output", outputPath, inputPath)
@@ -321,6 +335,65 @@ func (p *libreOfficeProcess) pdf(ctx context.Context, logger *zap.Logger, inputP
return fmt.Errorf("convert to PDF: %w", err) return fmt.Errorf("convert to PDF: %w", err)
} }
// LibreOffice cannot convert a file with a name containing non-basic Latin
// characters.
// See:
// https://github.com/gotenberg/gotenberg/issues/104
// https://github.com/gotenberg/gotenberg/issues/730
func nonBasicLatinCharactersGuard(logger *zap.Logger, inputPath string) (string, error) {
hasNonBasicLatinChars := func(str string) bool {
for _, r := range str {
// Check if the character is outside basic Latin.
if r != '.' && (r < ' ' || r > '~') {
return true
}
}
return false
}
filename := filepath.Base(inputPath)
if !hasNonBasicLatinChars(filename) {
logger.Debug("no non-basic latin characters in filename, skip copy")
return inputPath, nil
}
logger.Warn("non-basic latin characters in filename, copy to a file with a valid filename")
basePath := filepath.Dir(inputPath)
ext := filepath.Ext(inputPath)
newInputPath := filepath.Join(basePath, fmt.Sprintf("%s%s", uuid.NewString(), ext))
in, err := os.Open(inputPath)
if err != nil {
return "", fmt.Errorf("open file: %w", err)
}
defer func() {
err := in.Close()
if err != nil {
logger.Error(fmt.Sprintf("close file: %s", err))
}
}()
out, err := os.Create(newInputPath)
if err != nil {
return "", fmt.Errorf("create new file: %w", err)
}
defer func() {
err := out.Close()
if err != nil {
logger.Error(fmt.Sprintf("close new file: %s", err))
}
}()
_, err = io.Copy(out, in)
if err != nil {
return "", fmt.Errorf("copy file to new file: %w", err)
}
return newInputPath, nil
}
// Interface guards. // Interface guards.
var ( var (
_ gotenberg.Process = (*libreOfficeProcess)(nil) _ gotenberg.Process = (*libreOfficeProcess)(nil)

View File

@@ -11,7 +11,7 @@ import (
"github.com/google/uuid" "github.com/google/uuid"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestLibreOfficeProcess_Start(t *testing.T) { func TestLibreOfficeProcess_Start(t *testing.T) {
@@ -235,7 +235,7 @@ func TestLibreOfficeProcess_pdf(t *testing.T) {
expectError: true, expectError: true,
}, },
{ {
scenario: "ErrInvalidPdfFormat", scenario: "ErrInvalidPdfFormats",
libreOffice: func() libreOffice { libreOffice: func() libreOffice {
p := new(libreOfficeProcess) p := new(libreOfficeProcess)
p.socketPort = 12345 p.socketPort = 12345
@@ -247,7 +247,7 @@ func TestLibreOfficeProcess_pdf(t *testing.T) {
cancelledCtx: false, cancelledCtx: false,
start: false, start: false,
expectError: true, expectError: true,
expectedError: ErrInvalidPdfFormat, expectedError: ErrInvalidPdfFormats,
}, },
{ {
scenario: "ErrMalformedPageRanges", scenario: "ErrMalformedPageRanges",
@@ -394,7 +394,7 @@ func TestLibreOfficeProcess_pdf(t *testing.T) {
expectError: false, expectError: false,
}, },
{ {
scenario: "success (PDF/A-1a)", scenario: "success (PDF/A-1b)",
libreOffice: newLibreOfficeProcess( libreOffice: newLibreOfficeProcess(
libreOfficeArguments{ libreOfficeArguments{
binPath: os.Getenv("LIBREOFFICE_BIN_PATH"), binPath: os.Getenv("LIBREOFFICE_BIN_PATH"),
@@ -417,7 +417,7 @@ func TestLibreOfficeProcess_pdf(t *testing.T) {
return fs return fs
}(), }(),
options: Options{PdfFormats: gotenberg.PdfFormats{PdfA: gotenberg.PdfA1a}}, options: Options{PdfFormats: gotenberg.PdfFormats{PdfA: gotenberg.PdfA1b}},
cancelledCtx: false, cancelledCtx: false,
start: true, start: true,
expectError: false, expectError: false,
@@ -545,7 +545,7 @@ func TestLibreOfficeProcess_pdf(t *testing.T) {
err := tc.libreOffice.pdf( err := tc.libreOffice.pdf(
ctx, ctx,
logger, logger,
fmt.Sprintf("file://%s/document.txt", tc.fs.WorkingDirPath()), fmt.Sprintf("%s/document.txt", tc.fs.WorkingDirPath()),
fmt.Sprintf("%s/%s.pdf", tc.fs.WorkingDirPath(), uuid.NewString()), fmt.Sprintf("%s/%s.pdf", tc.fs.WorkingDirPath(), uuid.NewString()),
tc.options, tc.options,
) )
@@ -564,3 +564,87 @@ func TestLibreOfficeProcess_pdf(t *testing.T) {
}) })
} }
} }
func TestNonBasicLatinCharactersGuard(t *testing.T) {
for _, tc := range []struct {
scenario string
fs *gotenberg.FileSystem
filename string
expectSameInputPath bool
expectError bool
}{
{
scenario: "basic latin characters",
fs: func() *gotenberg.FileSystem {
fs := gotenberg.NewFileSystem()
err := os.MkdirAll(fs.WorkingDirPath(), 0o755)
if err != nil {
t.Fatalf(fmt.Sprintf("expected no error but got: %v", err))
}
err = os.WriteFile(fmt.Sprintf("%s/document.txt", fs.WorkingDirPath()), []byte("Basic latin characters"), 0o755)
if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
return fs
}(),
filename: "document.txt",
expectSameInputPath: true,
expectError: false,
},
{
scenario: "non-basic latin characters",
fs: func() *gotenberg.FileSystem {
fs := gotenberg.NewFileSystem()
err := os.MkdirAll(fs.WorkingDirPath(), 0o755)
if err != nil {
t.Fatalf(fmt.Sprintf("expected no error but got: %v", err))
}
err = os.WriteFile(fmt.Sprintf("%s/éèßàùä.txt", fs.WorkingDirPath()), []byte("Non-basic latin characters"), 0o755)
if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
return fs
}(),
filename: "éèßàùä.txt",
expectSameInputPath: false,
expectError: false,
},
} {
t.Run(tc.scenario, func(t *testing.T) {
defer func() {
err := os.RemoveAll(tc.fs.WorkingDirPath())
if err != nil {
t.Fatalf("expected no error while cleaning up, but got: %v", err)
}
}()
inputPath := fmt.Sprintf("%s/%s", tc.fs.WorkingDirPath(), tc.filename)
newInputPath, err := nonBasicLatinCharactersGuard(
zap.NewNop(),
inputPath,
)
if !tc.expectError && err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if tc.expectError && err == nil {
t.Fatal("expected error but got none")
}
if tc.expectSameInputPath && newInputPath != inputPath {
t.Fatalf("expected same input path, but got '%s'", newInputPath)
}
if !tc.expectSameInputPath && newInputPath == inputPath {
t.Fatalf("expected different input path, but got same '%s'", newInputPath)
}
})
}
}

View File

@@ -5,7 +5,7 @@ import (
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
// ApiMock is a mock for the [Uno] interface. // ApiMock is a mock for the [Uno] interface.

View File

@@ -5,9 +5,9 @@ import (
flag "github.com/spf13/pflag" flag "github.com/spf13/pflag"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
libeofficeapi "github.com/gotenberg/gotenberg/v7/pkg/modules/libreoffice/api" libeofficeapi "github.com/gotenberg/gotenberg/v8/pkg/modules/libreoffice/api"
) )
func init() { func init() {

View File

@@ -5,8 +5,8 @@ import (
"reflect" "reflect"
"testing" "testing"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
libreofficeapi "github.com/gotenberg/gotenberg/v7/pkg/modules/libreoffice/api" libreofficeapi "github.com/gotenberg/gotenberg/v8/pkg/modules/libreoffice/api"
) )
func TestLibreOffice_Descriptor(t *testing.T) { func TestLibreOffice_Descriptor(t *testing.T) {

View File

@@ -7,8 +7,8 @@ import (
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/libreoffice/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/libreoffice/api"
) )
func init() { func init() {
@@ -18,7 +18,7 @@ func init() {
// LibreOfficePdfEngine interacts with the LibreOffice (Universal Network Objects) API // LibreOfficePdfEngine interacts with the LibreOffice (Universal Network Objects) API
// and implements the [gotenberg.PdfEngine] interface. // and implements the [gotenberg.PdfEngine] interface.
type LibreOfficePdfEngine struct { type LibreOfficePdfEngine struct {
unoAPI api.Uno unoApi api.Uno
} }
// Descriptor returns a [LibreOfficePdfEngine]'s module descriptor. // Descriptor returns a [LibreOfficePdfEngine]'s module descriptor.
@@ -36,12 +36,12 @@ func (engine *LibreOfficePdfEngine) Provision(ctx *gotenberg.Context) error {
return fmt.Errorf("get LibreOffice Uno provider: %w", err) return fmt.Errorf("get LibreOffice Uno provider: %w", err)
} }
unoAPI, err := provider.(api.Provider).LibreOffice() unoApi, err := provider.(api.Provider).LibreOffice()
if err != nil { if err != nil {
return fmt.Errorf("get LibreOffice Uno: %w", err) return fmt.Errorf("get LibreOffice Uno: %w", err)
} }
engine.unoAPI = unoAPI engine.unoApi = unoApi
return nil return nil
} }
@@ -52,11 +52,11 @@ func (engine *LibreOfficePdfEngine) Merge(ctx context.Context, logger *zap.Logge
} }
// Convert converts the given PDF to a specific PDF format. Currently, only the // Convert converts the given PDF to a specific PDF format. Currently, only the
// PDF/A-1a, PDF/A-2b, PDF/A-3b and PDF/UA formats are available. If another // PDF/A-1b, PDF/A-2b, PDF/A-3b and PDF/UA formats are available. If another
// PDF format is requested, it returns a [gotenberg.ErrPdfFormatNotSupported] // PDF format is requested, it returns a [gotenberg.ErrPdfFormatNotSupported]
// error. // error.
func (engine *LibreOfficePdfEngine) Convert(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error { func (engine *LibreOfficePdfEngine) Convert(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error {
err := engine.unoAPI.Pdf(ctx, logger, inputPath, outputPath, api.Options{ err := engine.unoApi.Pdf(ctx, logger, inputPath, outputPath, api.Options{
PdfFormats: formats, PdfFormats: formats,
}) })
@@ -64,7 +64,7 @@ func (engine *LibreOfficePdfEngine) Convert(ctx context.Context, logger *zap.Log
return nil return nil
} }
if errors.Is(err, api.ErrInvalidPdfFormat) { if errors.Is(err, api.ErrInvalidPdfFormats) {
return fmt.Errorf("convert PDF to '%+v' with LibreOffice: %w", formats, gotenberg.ErrPdfFormatNotSupported) return fmt.Errorf("convert PDF to '%+v' with LibreOffice: %w", formats, gotenberg.ErrPdfFormatNotSupported)
} }

View File

@@ -8,8 +8,8 @@ import (
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/libreoffice/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/libreoffice/api"
) )
func TestLibreOfficePdfEngine_Descriptor(t *testing.T) { func TestLibreOfficePdfEngine_Descriptor(t *testing.T) {
@@ -137,7 +137,7 @@ func TestLibreOfficePdfEngine_Convert(t *testing.T) {
scenario: "invalid PDF format", scenario: "invalid PDF format",
api: &api.ApiMock{ api: &api.ApiMock{
PdfMock: func(ctx context.Context, logger *zap.Logger, inputPath, outputPath string, options api.Options) error { PdfMock: func(ctx context.Context, logger *zap.Logger, inputPath, outputPath string, options api.Options) error {
return api.ErrInvalidPdfFormat return api.ErrInvalidPdfFormats
}, },
}, },
expectError: true, expectError: true,
@@ -153,7 +153,7 @@ func TestLibreOfficePdfEngine_Convert(t *testing.T) {
}, },
} { } {
t.Run(tc.scenario, func(t *testing.T) { t.Run(tc.scenario, func(t *testing.T) {
engine := &LibreOfficePdfEngine{unoAPI: tc.api} engine := &LibreOfficePdfEngine{unoApi: tc.api}
err := engine.Convert(context.Background(), zap.NewNop(), gotenberg.PdfFormats{}, "", "") err := engine.Convert(context.Background(), zap.NewNop(), gotenberg.PdfFormats{}, "", "")
if !tc.expectError && err != nil { if !tc.expectError && err != nil {

View File

@@ -7,9 +7,9 @@ import (
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
libreofficeapi "github.com/gotenberg/gotenberg/v7/pkg/modules/libreoffice/api" libreofficeapi "github.com/gotenberg/gotenberg/v8/pkg/modules/libreoffice/api"
) )
// convertRoute returns an [api.Route] which can convert LibreOffice documents // convertRoute returns an [api.Route] which can convert LibreOffice documents
@@ -24,25 +24,19 @@ func convertRoute(libreOffice libreofficeapi.Uno, engine gotenberg.PdfEngine) ap
// Let's get the data from the form and validate them. // Let's get the data from the form and validate them.
var ( var (
inputPaths []string inputPaths []string
landscape bool landscape bool
nativePageRanges string nativePageRanges string
nativePdfA1aFormat bool pdfa string
nativePdfFormat string pdfua bool
pdfFormat string nativePdfFormats bool
pdfa string merge bool
pdfua bool
nativePdfFormats bool
merge bool
) )
err := ctx.FormData(). err := ctx.FormData().
MandatoryPaths(libreOffice.Extensions(), &inputPaths). MandatoryPaths(libreOffice.Extensions(), &inputPaths).
Bool("landscape", &landscape, false). Bool("landscape", &landscape, false).
String("nativePageRanges", &nativePageRanges, ""). String("nativePageRanges", &nativePageRanges, "").
Bool("nativePdfA1aFormat", &nativePdfA1aFormat, false).
String("nativePdfFormat", &nativePdfFormat, "").
String("pdfFormat", &pdfFormat, "").
String("pdfa", &pdfa, ""). String("pdfa", &pdfa, "").
Bool("pdfua", &pdfua, false). Bool("pdfua", &pdfua, false).
Bool("nativePdfFormats", &nativePdfFormats, true). Bool("nativePdfFormats", &nativePdfFormats, true).
@@ -52,42 +46,8 @@ func convertRoute(libreOffice libreofficeapi.Uno, engine gotenberg.PdfEngine) ap
return fmt.Errorf("validate form data: %w", err) return fmt.Errorf("validate form data: %w", err)
} }
// FIXME: deprecated.
// pdfa > nativePdfFormat > pdfFormat > nativePdfA1aFormat.
var (
actualPdfArchive string
nativeFormats bool
)
if nativePdfA1aFormat {
ctx.Log().Warn("'nativePdfA1aFormat' is deprecated; prefer the 'pdfa' form field instead")
actualPdfArchive = gotenberg.PdfA1a
nativeFormats = true
}
if pdfFormat != "" {
ctx.Log().Warn("'pdfFormat' is deprecated; prefer the 'pdfa' form field instead")
actualPdfArchive = pdfFormat
nativeFormats = false
}
if nativePdfFormat != "" {
ctx.Log().Warn("'nativePdfFormat' is deprecated; prefer the 'pdfa' form field instead")
actualPdfArchive = nativePdfFormat
nativeFormats = true
}
if pdfa != "" {
actualPdfArchive = pdfa
nativeFormats = nativePdfFormats
}
if pdfua {
nativeFormats = nativePdfFormats
}
pdfFormats := gotenberg.PdfFormats{ pdfFormats := gotenberg.PdfFormats{
PdfA: actualPdfArchive, PdfA: pdfa,
PdfUa: pdfua, PdfUa: pdfua,
} }
@@ -95,18 +55,27 @@ func convertRoute(libreOffice libreofficeapi.Uno, engine gotenberg.PdfEngine) ap
outputPaths := make([]string, len(inputPaths)) outputPaths := make([]string, len(inputPaths))
for i, inputPath := range inputPaths { for i, inputPath := range inputPaths {
outputPaths[i] = ctx.GeneratePath(".pdf") outputPaths[i] = ctx.GeneratePath(".pdf")
options := libreofficeapi.Options{ options := libreofficeapi.Options{
Landscape: landscape, Landscape: landscape,
PageRanges: nativePageRanges, PageRanges: nativePageRanges,
} }
if nativeFormats { if nativePdfFormats {
options.PdfFormats = pdfFormats options.PdfFormats = pdfFormats
} }
err = libreOffice.Pdf(ctx, ctx.Log(), inputPath, outputPaths[i], options) err = libreOffice.Pdf(ctx, ctx.Log(), inputPath, outputPaths[i], options)
if err != nil { if err != nil {
if errors.Is(err, libreofficeapi.ErrInvalidPdfFormats) {
return api.WrapError(
fmt.Errorf("convert to PDF: %w", err),
api.NewSentinelHttpError(
http.StatusBadRequest,
fmt.Sprintf("A PDF format in '%+v' is not supported", pdfFormats),
),
)
}
if errors.Is(err, libreofficeapi.ErrMalformedPageRanges) { if errors.Is(err, libreofficeapi.ErrMalformedPageRanges) {
return api.WrapError( return api.WrapError(
fmt.Errorf("convert to PDF: %w", err), fmt.Errorf("convert to PDF: %w", err),
@@ -120,7 +89,6 @@ func convertRoute(libreOffice libreofficeapi.Uno, engine gotenberg.PdfEngine) ap
// So far so good, let's check if we have to merge the PDFs. Quick // So far so good, let's check if we have to merge the PDFs. Quick
// win: if there is only one PDF, skip this step. // win: if there is only one PDF, skip this step.
if len(outputPaths) > 1 && merge { if len(outputPaths) > 1 && merge {
outputPath := ctx.GeneratePath(".pdf") outputPath := ctx.GeneratePath(".pdf")
@@ -129,25 +97,15 @@ func convertRoute(libreOffice libreofficeapi.Uno, engine gotenberg.PdfEngine) ap
return fmt.Errorf("merge PDFs: %w", err) return fmt.Errorf("merge PDFs: %w", err)
} }
// Now, let's check if the client want to convert this result // Now, let's check if the client want to convert this
// PDF to specific PDF formats. // resulting PDF to specific PDF formats.
zeroValued := gotenberg.PdfFormats{} zeroValued := gotenberg.PdfFormats{}
if !nativeFormats && pdfFormats != zeroValued { if !nativePdfFormats && pdfFormats != zeroValued {
convertInputPath := outputPath convertInputPath := outputPath
convertOutputPath := ctx.GeneratePath(".pdf") convertOutputPath := ctx.GeneratePath(".pdf")
err = engine.Convert(ctx, ctx.Log(), pdfFormats, convertInputPath, convertOutputPath) err = engine.Convert(ctx, ctx.Log(), pdfFormats, convertInputPath, convertOutputPath)
if err != nil { if err != nil {
if errors.Is(err, gotenberg.ErrPdfFormatNotSupported) {
return api.WrapError(
fmt.Errorf("convert PDF: %w", err),
api.NewSentinelHttpError(
http.StatusBadRequest,
fmt.Sprintf("At least one PDF engine does not handle one of the PDF format in '%+v', while other have failed to convert for other reasons", pdfFormats),
),
)
}
return fmt.Errorf("convert PDF: %w", err) return fmt.Errorf("convert PDF: %w", err)
} }
@@ -156,8 +114,7 @@ func convertRoute(libreOffice libreofficeapi.Uno, engine gotenberg.PdfEngine) ap
} }
// Last but not least, add the output path to the context so that // Last but not least, add the output path to the context so that
// the Uno is able to send it as a response to the client. // the API is able to send it as a response to the client.
err = ctx.AddOutputPaths(outputPath) err = ctx.AddOutputPaths(outputPath)
if err != nil { if err != nil {
return fmt.Errorf("add output path: %w", err) return fmt.Errorf("add output path: %w", err)
@@ -169,7 +126,7 @@ func convertRoute(libreOffice libreofficeapi.Uno, engine gotenberg.PdfEngine) ap
// Ok, we don't have to merge the PDFs. Let's check if the client // Ok, we don't have to merge the PDFs. Let's check if the client
// want to convert each PDF to a specific PDF format. // want to convert each PDF to a specific PDF format.
zeroValued := gotenberg.PdfFormats{} zeroValued := gotenberg.PdfFormats{}
if !nativeFormats && pdfFormats != zeroValued { if !nativePdfFormats && pdfFormats != zeroValued {
convertOutputPaths := make([]string, len(outputPaths)) convertOutputPaths := make([]string, len(outputPaths))
for i, outputPath := range outputPaths { for i, outputPath := range outputPaths {
@@ -178,28 +135,30 @@ func convertRoute(libreOffice libreofficeapi.Uno, engine gotenberg.PdfEngine) ap
err = engine.Convert(ctx, ctx.Log(), pdfFormats, convertInputPath, convertOutputPaths[i]) err = engine.Convert(ctx, ctx.Log(), pdfFormats, convertInputPath, convertOutputPaths[i])
if err != nil { if err != nil {
if errors.Is(err, gotenberg.ErrPdfFormatNotSupported) {
return api.WrapError(
fmt.Errorf("convert PDF: %w", err),
api.NewSentinelHttpError(
http.StatusBadRequest,
fmt.Sprintf("At least one PDF engine does not handle one of the PDF format in '%+v', while other have failed to convert for other reasons", pdfFormats),
),
)
}
return fmt.Errorf("convert PDF: %w", err) return fmt.Errorf("convert PDF: %w", err)
} }
} }
// Important: the output paths are now the converted files. // Important: the output paths are now the converted files.
outputPaths = convertOutputPaths outputPaths = convertOutputPaths
} }
// Last but not least, add the output paths to the context so that if len(outputPaths) > 1 {
// the Uno is able to send them as a response to the client. // If .zip archive, document.docx -> document.docx.pdf.
for i, inputPath := range inputPaths {
outputPath := fmt.Sprintf("%s.pdf", inputPath)
err = ctx.Rename(outputPaths[i], outputPath)
if err != nil {
return fmt.Errorf("rename output path: %w", err)
}
outputPaths[i] = outputPath
}
}
// Last but not least, add the output paths to the context so that
// the API is able to send them as a response to the client.
err = ctx.AddOutputPaths(outputPaths...) err = ctx.AddOutputPaths(outputPaths...)
if err != nil { if err != nil {
return fmt.Errorf("add output paths: %w", err) return fmt.Errorf("add output paths: %w", err)

View File

@@ -4,14 +4,15 @@ import (
"context" "context"
"errors" "errors"
"net/http" "net/http"
"slices"
"testing" "testing"
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
libreofficeapi "github.com/gotenberg/gotenberg/v7/pkg/modules/libreoffice/api" libreofficeapi "github.com/gotenberg/gotenberg/v8/pkg/modules/libreoffice/api"
) )
func TestConvertRoute(t *testing.T) { func TestConvertRoute(t *testing.T) {
@@ -25,6 +26,7 @@ func TestConvertRoute(t *testing.T) {
expectHttpError bool expectHttpError bool
expectHttpStatus int expectHttpStatus int
expectOutputPathsCount int expectOutputPathsCount int
expectOutputPaths []string
}{ }{
{ {
scenario: "missing at least one mandatory file", scenario: "missing at least one mandatory file",
@@ -37,6 +39,28 @@ func TestConvertRoute(t *testing.T) {
expectHttpStatus: http.StatusBadRequest, expectHttpStatus: http.StatusBadRequest,
expectOutputPathsCount: 0, expectOutputPathsCount: 0,
}, },
{
scenario: "ErrPdfFormatNotSupported (nativePdfFormats)",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"document.docx": "/document.docx",
})
return ctx
}(),
libreOffice: &libreofficeapi.ApiMock{
PdfMock: func(ctx context.Context, logger *zap.Logger, inputPath, outputPath string, options libreofficeapi.Options) error {
return libreofficeapi.ErrInvalidPdfFormats
},
ExtensionsMock: func() []string {
return []string{".docx"}
},
},
expectError: true,
expectHttpError: true,
expectHttpStatus: http.StatusBadRequest,
expectOutputPathsCount: 0,
},
{ {
scenario: "ErrMalformedPageRanges", scenario: "ErrMalformedPageRanges",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
@@ -44,6 +68,11 @@ func TestConvertRoute(t *testing.T) {
ctx.SetFiles(map[string]string{ ctx.SetFiles(map[string]string{
"document.docx": "/document.docx", "document.docx": "/document.docx",
}) })
ctx.SetValues(map[string][]string{
"pdfa": {
"foo",
},
})
return ctx return ctx
}(), }(),
libreOffice: &libreofficeapi.ApiMock{ libreOffice: &libreofficeapi.ApiMock{
@@ -80,38 +109,6 @@ func TestConvertRoute(t *testing.T) {
expectHttpError: false, expectHttpError: false,
expectOutputPathsCount: 0, expectOutputPathsCount: 0,
}, },
{
scenario: "ErrPdfFormatNotSupported (single file)",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"document.docx": "/document.docx",
})
ctx.SetValues(map[string][]string{
"pdfFormat": {
"foo",
},
})
return ctx
}(),
libreOffice: &libreofficeapi.ApiMock{
PdfMock: func(ctx context.Context, logger *zap.Logger, inputPath, outputPath string, options libreofficeapi.Options) error {
return nil
},
ExtensionsMock: func() []string {
return []string{".docx"}
},
},
engine: &gotenberg.PdfEngineMock{
ConvertMock: func(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error {
return gotenberg.ErrPdfFormatNotSupported
},
},
expectError: true,
expectHttpError: true,
expectHttpStatus: http.StatusBadRequest,
expectOutputPathsCount: 0,
},
{ {
scenario: "PDF engine convert error (single file)", scenario: "PDF engine convert error (single file)",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
@@ -120,8 +117,11 @@ func TestConvertRoute(t *testing.T) {
"document.docx": "/document.docx", "document.docx": "/document.docx",
}) })
ctx.SetValues(map[string][]string{ ctx.SetValues(map[string][]string{
"pdfFormat": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
},
"nativePdfFormats": {
"false",
}, },
}) })
return ctx return ctx
@@ -186,6 +186,32 @@ func TestConvertRoute(t *testing.T) {
expectHttpError: false, expectHttpError: false,
expectOutputPathsCount: 1, expectOutputPathsCount: 1,
}, },
{
scenario: "cannot rename many files",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"document.docx": "/document.docx",
"document2.docx": "/document2.docx",
"document2.doc": "/document2.doc",
})
ctx.SetPathRename(&gotenberg.PathRenameMock{RenameMock: func(oldpath, newpath string) error {
return errors.New("cannot rename")
}})
return ctx
}(),
libreOffice: &libreofficeapi.ApiMock{
PdfMock: func(ctx context.Context, logger *zap.Logger, inputPath, outputPath string, options libreofficeapi.Options) error {
return nil
},
ExtensionsMock: func() []string {
return []string{".docx", ".doc"}
},
},
expectError: true,
expectHttpError: false,
expectOutputPathsCount: 0,
},
{ {
scenario: "success (many files)", scenario: "success (many files)",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
@@ -193,7 +219,11 @@ func TestConvertRoute(t *testing.T) {
ctx.SetFiles(map[string]string{ ctx.SetFiles(map[string]string{
"document.docx": "/document.docx", "document.docx": "/document.docx",
"document2.docx": "/document2.docx", "document2.docx": "/document2.docx",
"document2.doc": "/document2.doc",
}) })
ctx.SetPathRename(&gotenberg.PathRenameMock{RenameMock: func(oldpath, newpath string) error {
return nil
}})
return ctx return ctx
}(), }(),
libreOffice: &libreofficeapi.ApiMock{ libreOffice: &libreofficeapi.ApiMock{
@@ -201,66 +231,25 @@ func TestConvertRoute(t *testing.T) {
return nil return nil
}, },
ExtensionsMock: func() []string { ExtensionsMock: func() []string {
return []string{".docx"} return []string{".docx", ".doc"}
}, },
}, },
expectError: false, expectError: false,
expectHttpError: false, expectHttpError: false,
expectOutputPathsCount: 2, expectOutputPathsCount: 3,
expectOutputPaths: []string{"/document.docx.pdf", "/document2.docx.pdf", "/document2.doc.pdf"},
}, },
{ {
scenario: "success with non-native PDF/A (single file)", scenario: "success with non-native PDF/A & PDF/UA (many files)",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)} ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{ ctx.SetFiles(map[string]string{
"document.docx": "/document.docx", "document.docx": "/document.docx",
"document2.docx": "/document2.docx",
}) })
ctx.SetValues(map[string][]string{ ctx.SetValues(map[string][]string{
"pdfa": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
},
"nativePdfFormats": {
"false",
},
})
return ctx
}(),
libreOffice: &libreofficeapi.ApiMock{
PdfMock: func(ctx context.Context, logger *zap.Logger, inputPath, outputPath string, options libreofficeapi.Options) error {
return nil
},
ExtensionsMock: func() []string {
return []string{".docx"}
},
},
engine: &gotenberg.PdfEngineMock{
ConvertMock: func(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error {
return nil
},
},
expectError: false,
expectHttpError: false,
expectOutputPathsCount: 1,
},
{
scenario: "success with every non-native PDF/A & PDF/UA form fields (single file)",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"document.docx": "/document.docx",
})
ctx.SetValues(map[string][]string{
"nativePdfA1aFormat": {
"true",
},
"pdfFormat": {
gotenberg.PdfA1a,
},
"nativePdfFormat": {
gotenberg.PdfA1a,
},
"pdfa": {
gotenberg.PdfA1a,
}, },
"pdfua": { "pdfua": {
"true", "true",
@@ -269,6 +258,9 @@ func TestConvertRoute(t *testing.T) {
"false", "false",
}, },
}) })
ctx.SetPathRename(&gotenberg.PathRenameMock{RenameMock: func(oldpath, newpath string) error {
return nil
}})
return ctx return ctx
}(), }(),
libreOffice: &libreofficeapi.ApiMock{ libreOffice: &libreofficeapi.ApiMock{
@@ -286,7 +278,47 @@ func TestConvertRoute(t *testing.T) {
}, },
expectError: false, expectError: false,
expectHttpError: false, expectHttpError: false,
expectOutputPathsCount: 1, expectOutputPathsCount: 2,
expectOutputPaths: []string{"/document.docx.pdf", "/document2.docx.pdf"},
},
{
scenario: "success with native PDF/A & PDF/UA (many files)",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"document.docx": "/document.docx",
"document2.docx": "/document2.docx",
})
ctx.SetValues(map[string][]string{
"pdfa": {
gotenberg.PdfA1b,
},
"pdfua": {
"true",
},
})
ctx.SetPathRename(&gotenberg.PathRenameMock{RenameMock: func(oldpath, newpath string) error {
return nil
}})
return ctx
}(),
libreOffice: &libreofficeapi.ApiMock{
PdfMock: func(ctx context.Context, logger *zap.Logger, inputPath, outputPath string, options libreofficeapi.Options) error {
return nil
},
ExtensionsMock: func() []string {
return []string{".docx"}
},
},
engine: &gotenberg.PdfEngineMock{
ConvertMock: func(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error {
return nil
},
},
expectError: false,
expectHttpError: false,
expectOutputPathsCount: 2,
expectOutputPaths: []string{"/document.docx.pdf", "/document2.docx.pdf"},
}, },
{ {
scenario: "merge error", scenario: "merge error",
@@ -320,48 +352,6 @@ func TestConvertRoute(t *testing.T) {
expectHttpError: false, expectHttpError: false,
expectOutputPathsCount: 0, expectOutputPathsCount: 0,
}, },
{
scenario: "ErrPdfFormatNotSupported (merge)",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"document.docx": "/document.docx",
"document2.docx": "/document2.docx",
})
ctx.SetValues(map[string][]string{
"merge": {
"true",
},
"pdfa": {
"foo",
},
"nativePdfFormats": {
"false",
},
})
return ctx
}(),
libreOffice: &libreofficeapi.ApiMock{
PdfMock: func(ctx context.Context, logger *zap.Logger, inputPath, outputPath string, options libreofficeapi.Options) error {
return nil
},
ExtensionsMock: func() []string {
return []string{".docx"}
},
},
engine: &gotenberg.PdfEngineMock{
MergeMock: func(ctx context.Context, logger *zap.Logger, inputPaths []string, outputPath string) error {
return nil
},
ConvertMock: func(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error {
return gotenberg.ErrPdfFormatNotSupported
},
},
expectError: true,
expectHttpError: true,
expectHttpStatus: http.StatusBadRequest,
expectOutputPathsCount: 0,
},
{ {
scenario: "PDF engine convert error (merge)", scenario: "PDF engine convert error (merge)",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
@@ -375,7 +365,7 @@ func TestConvertRoute(t *testing.T) {
"true", "true",
}, },
"pdfa": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
}, },
"nativePdfFormats": { "nativePdfFormats": {
"false", "false",
@@ -469,7 +459,7 @@ func TestConvertRoute(t *testing.T) {
expectOutputPathsCount: 1, expectOutputPathsCount: 1,
}, },
{ {
scenario: "success with non-native PDF/A (merge)", scenario: "success with non-native PDF/A & PDF/UA (merge)",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)} ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{ ctx.SetFiles(map[string]string{
@@ -481,7 +471,54 @@ func TestConvertRoute(t *testing.T) {
"true", "true",
}, },
"pdfa": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
},
"pdfua": {
"true",
},
"nativePdfFormats": {
"false",
},
})
return ctx
}(),
libreOffice: &libreofficeapi.ApiMock{
PdfMock: func(ctx context.Context, logger *zap.Logger, inputPath, outputPath string, options libreofficeapi.Options) error {
return nil
},
ExtensionsMock: func() []string {
return []string{".docx"}
},
},
engine: &gotenberg.PdfEngineMock{
MergeMock: func(ctx context.Context, logger *zap.Logger, inputPaths []string, outputPath string) error {
return nil
},
ConvertMock: func(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error {
return nil
},
},
expectError: false,
expectHttpError: false,
expectOutputPathsCount: 1,
},
{
scenario: "success with non-native PDF/A & PDF/UA (merge)",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"document.docx": "/document.docx",
"document2.docx": "/document2.docx",
})
ctx.SetValues(map[string][]string{
"merge": {
"true",
},
"pdfa": {
gotenberg.PdfA1b,
},
"pdfua": {
"true",
}, },
"nativePdfFormats": { "nativePdfFormats": {
"false", "false",
@@ -546,6 +583,12 @@ func TestConvertRoute(t *testing.T) {
if tc.expectOutputPathsCount != len(tc.ctx.OutputPaths()) { if tc.expectOutputPathsCount != len(tc.ctx.OutputPaths()) {
t.Errorf("expected %d output paths but got %d", tc.expectOutputPathsCount, len(tc.ctx.OutputPaths())) t.Errorf("expected %d output paths but got %d", tc.expectOutputPathsCount, len(tc.ctx.OutputPaths()))
} }
for _, path := range tc.expectOutputPaths {
if !slices.Contains(tc.ctx.OutputPaths(), path) {
t.Errorf("expected '%s' in output paths %v", path, tc.ctx.OutputPaths())
}
}
}) })
} }
} }

View File

@@ -11,7 +11,7 @@ import (
"go.uber.org/zap/zapcore" "go.uber.org/zap/zapcore"
"golang.org/x/term" "golang.org/x/term"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func init() { func init() {

View File

@@ -9,7 +9,7 @@ import (
"go.uber.org/zap/zapcore" "go.uber.org/zap/zapcore"
"go.uber.org/zap/zaptest/observer" "go.uber.org/zap/zaptest/observer"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestLogging_Descriptor(t *testing.T) { func TestLogging_Descriptor(t *testing.T) {

View File

@@ -9,7 +9,7 @@ import (
pdfcpuConfig "github.com/pdfcpu/pdfcpu/pkg/pdfcpu/model" pdfcpuConfig "github.com/pdfcpu/pdfcpu/pkg/pdfcpu/model"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func init() { func init() {
@@ -41,7 +41,7 @@ func (engine *PdfCpu) Provision(ctx *gotenberg.Context) error {
// Merge combines multiple PDFs into a single PDF. // Merge combines multiple PDFs into a single PDF.
func (engine *PdfCpu) Merge(ctx context.Context, logger *zap.Logger, inputPaths []string, outputPath string) error { func (engine *PdfCpu) Merge(ctx context.Context, logger *zap.Logger, inputPaths []string, outputPath string) error {
err := pdfcpuAPI.MergeCreateFile(inputPaths, outputPath, engine.conf) err := pdfcpuAPI.MergeCreateFile(inputPaths, outputPath, false, engine.conf)
if err == nil { if err == nil {
return nil return nil
} }

View File

@@ -9,7 +9,7 @@ import (
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestPdfCpu_Descriptor(t *testing.T) { func TestPdfCpu_Descriptor(t *testing.T) {

View File

@@ -7,7 +7,7 @@ import (
"go.uber.org/multierr" "go.uber.org/multierr"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
type multiPdfEngines struct { type multiPdfEngines struct {

View File

@@ -7,7 +7,7 @@ import (
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestMultiPdfEngines_Merge(t *testing.T) { func TestMultiPdfEngines_Merge(t *testing.T) {

View File

@@ -7,8 +7,8 @@ import (
flag "github.com/spf13/pflag" flag "github.com/spf13/pflag"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
func init() { func init() {
@@ -54,18 +54,6 @@ func (mod *PdfEngines) Provision(ctx *gotenberg.Context) error {
names := flags.MustStringSlice("pdfengines-engines") names := flags.MustStringSlice("pdfengines-engines")
mod.disableRoutes = flags.MustBool("pdfengines-disable-routes") mod.disableRoutes = flags.MustBool("pdfengines-disable-routes")
loggerProvider, err := ctx.Module(new(gotenberg.LoggerProvider))
if err != nil {
return fmt.Errorf("get logger provider: %w", err)
}
logger, err := loggerProvider.(gotenberg.LoggerProvider).Logger(mod)
if err != nil {
return fmt.Errorf("get logger: %w", err)
}
logger = logger.Named("pdfengines")
engines, err := ctx.Modules(new(gotenberg.PdfEngine)) engines, err := ctx.Modules(new(gotenberg.PdfEngine))
if err != nil { if err != nil {
return fmt.Errorf("get PDF engines: %w", err) return fmt.Errorf("get PDF engines: %w", err)
@@ -81,13 +69,13 @@ func (mod *PdfEngines) Provision(ctx *gotenberg.Context) error {
// Selection from user. // Selection from user.
mod.names = names mod.names = names
for i, name := range names { // Example in case of deprecated module name.
// FIXME: deprecated. //for i, name := range names {
if name == "unoconv-pdfengine" || name == "uno-pdfengine" { // if name == "unoconv-pdfengine" || name == "uno-pdfengine" {
logger.Warn(fmt.Sprintf("%s is deprecated; prefer libreoffice-pdfengine instead", name)) // logger.Warn(fmt.Sprintf("%s is deprecated; prefer libreoffice-pdfengine instead", name))
mod.names[i] = "libreoffice-pdfengine" // mod.names[i] = "libreoffice-pdfengine"
} // }
} //}
return nil return nil
} }

View File

@@ -6,9 +6,7 @@ import (
"strings" "strings"
"testing" "testing"
"go.uber.org/zap" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg"
) )
func TestPdfEngines_Descriptor(t *testing.T) { func TestPdfEngines_Descriptor(t *testing.T) {
@@ -34,16 +32,12 @@ func TestPdfEngines_Provision(t *testing.T) {
ctx: func() *gotenberg.Context { ctx: func() *gotenberg.Context {
provider := &struct { provider := &struct {
gotenberg.ModuleMock gotenberg.ModuleMock
gotenberg.LoggerProviderMock
}{} }{}
provider.DescriptorMock = func() gotenberg.ModuleDescriptor { provider.DescriptorMock = func() gotenberg.ModuleDescriptor {
return gotenberg.ModuleDescriptor{ID: "foo", New: func() gotenberg.Module { return gotenberg.ModuleDescriptor{ID: "foo", New: func() gotenberg.Module {
return provider return provider
}} }}
} }
provider.LoggerMock = func(mod gotenberg.Module) (*zap.Logger, error) {
return zap.NewNop(), nil
}
engine := &struct { engine := &struct {
gotenberg.ModuleMock gotenberg.ModuleMock
@@ -75,17 +69,12 @@ func TestPdfEngines_Provision(t *testing.T) {
ctx: func() *gotenberg.Context { ctx: func() *gotenberg.Context {
provider := &struct { provider := &struct {
gotenberg.ModuleMock gotenberg.ModuleMock
gotenberg.LoggerProviderMock
}{} }{}
provider.DescriptorMock = func() gotenberg.ModuleDescriptor { provider.DescriptorMock = func() gotenberg.ModuleDescriptor {
return gotenberg.ModuleDescriptor{ID: "foo", New: func() gotenberg.Module { return gotenberg.ModuleDescriptor{ID: "foo", New: func() gotenberg.Module {
return provider return provider
}} }}
} }
provider.LoggerMock = func(mod gotenberg.Module) (*zap.Logger, error) {
return zap.NewNop(), nil
}
engine1 := &struct { engine1 := &struct {
gotenberg.ModuleMock gotenberg.ModuleMock
gotenberg.ValidatorMock gotenberg.ValidatorMock
@@ -130,108 +119,17 @@ func TestPdfEngines_Provision(t *testing.T) {
expectedPdfEngines: []string{"b", "a"}, expectedPdfEngines: []string{"b", "a"},
expectError: false, expectError: false,
}, },
{
scenario: "user select deprecated unoconv-pdfengine",
ctx: func() *gotenberg.Context {
provider := &struct {
gotenberg.ModuleMock
gotenberg.LoggerProviderMock
}{}
provider.DescriptorMock = func() gotenberg.ModuleDescriptor {
return gotenberg.ModuleDescriptor{ID: "foo", New: func() gotenberg.Module {
return provider
}}
}
provider.LoggerMock = func(mod gotenberg.Module) (*zap.Logger, error) {
return zap.NewNop(), nil
}
engine := &struct {
gotenberg.ModuleMock
gotenberg.ValidatorMock
gotenberg.PdfEngineMock
}{}
engine.DescriptorMock = func() gotenberg.ModuleDescriptor {
return gotenberg.ModuleDescriptor{ID: "libreoffice-pdfengine", New: func() gotenberg.Module { return engine }}
}
engine.ValidateMock = func() error {
return nil
}
fs := new(PdfEngines).Descriptor().FlagSet
err := fs.Parse([]string{"--pdfengines-engines=unoconv-pdfengine"})
if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
return gotenberg.NewContext(
gotenberg.ParsedFlags{
FlagSet: fs,
},
[]gotenberg.ModuleDescriptor{
provider.Descriptor(),
engine.Descriptor(),
},
)
}(),
expectedPdfEngines: []string{"libreoffice-pdfengine"},
expectError: false,
},
{
scenario: "no logger provider",
ctx: func() *gotenberg.Context {
return gotenberg.NewContext(
gotenberg.ParsedFlags{
FlagSet: new(PdfEngines).Descriptor().FlagSet,
},
[]gotenberg.ModuleDescriptor{},
)
}(),
expectError: true,
},
{
scenario: "no logger from logger provider",
ctx: func() *gotenberg.Context {
provider := &struct {
gotenberg.ModuleMock
gotenberg.LoggerProviderMock
}{}
provider.DescriptorMock = func() gotenberg.ModuleDescriptor {
return gotenberg.ModuleDescriptor{ID: "foo", New: func() gotenberg.Module {
return provider
}}
}
provider.LoggerMock = func(mod gotenberg.Module) (*zap.Logger, error) {
return nil, errors.New("foo")
}
return gotenberg.NewContext(
gotenberg.ParsedFlags{
FlagSet: new(PdfEngines).Descriptor().FlagSet,
},
[]gotenberg.ModuleDescriptor{
provider.Descriptor(),
},
)
}(),
expectError: true,
},
{ {
scenario: "no valid PDF engine", scenario: "no valid PDF engine",
ctx: func() *gotenberg.Context { ctx: func() *gotenberg.Context {
provider := &struct { provider := &struct {
gotenberg.ModuleMock gotenberg.ModuleMock
gotenberg.LoggerProviderMock
}{} }{}
provider.DescriptorMock = func() gotenberg.ModuleDescriptor { provider.DescriptorMock = func() gotenberg.ModuleDescriptor {
return gotenberg.ModuleDescriptor{ID: "foo", New: func() gotenberg.Module { return gotenberg.ModuleDescriptor{ID: "foo", New: func() gotenberg.Module {
return provider return provider
}} }}
} }
provider.LoggerMock = func(mod gotenberg.Module) (*zap.Logger, error) {
return zap.NewNop(), nil
}
engine := &struct { engine := &struct {
gotenberg.ModuleMock gotenberg.ModuleMock
gotenberg.ValidatorMock gotenberg.ValidatorMock

View File

@@ -7,8 +7,8 @@ import (
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
// mergeRoute returns an [api.Route] which can merge PDFs. // mergeRoute returns an [api.Route] which can merge PDFs.
@@ -23,14 +23,12 @@ func mergeRoute(engine gotenberg.PdfEngine) api.Route {
// Let's get the data from the form and validate them. // Let's get the data from the form and validate them.
var ( var (
inputPaths []string inputPaths []string
pdfFormat string
pdfa string pdfa string
pdfua bool pdfua bool
) )
err := ctx.FormData(). err := ctx.FormData().
MandatoryPaths([]string{".pdf"}, &inputPaths). MandatoryPaths([]string{".pdf"}, &inputPaths).
String("pdfFormat", &pdfFormat, "").
String("pdfa", &pdfa, ""). String("pdfa", &pdfa, "").
Bool("pdfua", &pdfua, false). Bool("pdfua", &pdfua, false).
Validate() Validate()
@@ -38,25 +36,12 @@ func mergeRoute(engine gotenberg.PdfEngine) api.Route {
return fmt.Errorf("validate form data: %w", err) return fmt.Errorf("validate form data: %w", err)
} }
var actualPdfArchive string
if pdfFormat != "" {
// FIXME: deprecated
ctx.Log().Warn("'pdfFormat' is deprecated; prefer the 'pdfa' form field instead")
actualPdfArchive = pdfFormat
}
if pdfa != "" {
actualPdfArchive = pdfa
}
pdfFormats := gotenberg.PdfFormats{ pdfFormats := gotenberg.PdfFormats{
PdfA: actualPdfArchive, PdfA: pdfa,
PdfUa: pdfua, PdfUa: pdfua,
} }
// Alright, let's merge the PDFs. // Alright, let's merge the PDFs.
outputPath := ctx.GeneratePath(".pdf") outputPath := ctx.GeneratePath(".pdf")
err = engine.Merge(ctx, ctx.Log(), inputPaths, outputPath) err = engine.Merge(ctx, ctx.Log(), inputPaths, outputPath)
@@ -73,18 +58,7 @@ func mergeRoute(engine gotenberg.PdfEngine) api.Route {
convertOutputPath := ctx.GeneratePath(".pdf") convertOutputPath := ctx.GeneratePath(".pdf")
err = engine.Convert(ctx, ctx.Log(), pdfFormats, convertInputPath, convertOutputPath) err = engine.Convert(ctx, ctx.Log(), pdfFormats, convertInputPath, convertOutputPath)
if err != nil { if err != nil {
if errors.Is(err, gotenberg.ErrPdfFormatNotSupported) {
return api.WrapError(
fmt.Errorf("convert PDF: %w", err),
api.NewSentinelHttpError(
http.StatusBadRequest,
fmt.Sprintf("At least one PDF engine does not handle one of the PDF format in '%+v', while other have failed to convert for other reasons", pdfFormats),
),
)
}
return fmt.Errorf("convert PDF: %w", err) return fmt.Errorf("convert PDF: %w", err)
} }
@@ -94,7 +68,6 @@ func mergeRoute(engine gotenberg.PdfEngine) api.Route {
// Last but not least, add the output path to the context so that // Last but not least, add the output path to the context so that
// the API is able to send it as a response to the client. // the API is able to send it as a response to the client.
err = ctx.AddOutputPaths(outputPath) err = ctx.AddOutputPaths(outputPath)
if err != nil { if err != nil {
return fmt.Errorf("add output path: %w", err) return fmt.Errorf("add output path: %w", err)
@@ -118,14 +91,12 @@ func convertRoute(engine gotenberg.PdfEngine) api.Route {
// Let's get the data from the form and validate them. // Let's get the data from the form and validate them.
var ( var (
inputPaths []string inputPaths []string
pdfFormat string
pdfa string pdfa string
pdfua bool pdfua bool
) )
err := ctx.FormData(). err := ctx.FormData().
MandatoryPaths([]string{".pdf"}, &inputPaths). MandatoryPaths([]string{".pdf"}, &inputPaths).
String("pdfFormat", &pdfFormat, "").
String("pdfa", &pdfa, ""). String("pdfa", &pdfa, "").
Bool("pdfua", &pdfua, false). Bool("pdfua", &pdfua, false).
Validate() Validate()
@@ -133,20 +104,8 @@ func convertRoute(engine gotenberg.PdfEngine) api.Route {
return fmt.Errorf("validate form data: %w", err) return fmt.Errorf("validate form data: %w", err)
} }
var actualPdfArchive string
if pdfFormat != "" {
// FIXME: deprecated.
ctx.Log().Warn("'pdfFormat' is deprecated; prefer the 'pdfa' form field instead")
actualPdfArchive = pdfFormat
}
if pdfa != "" {
actualPdfArchive = pdfa
}
pdfFormats := gotenberg.PdfFormats{ pdfFormats := gotenberg.PdfFormats{
PdfA: actualPdfArchive, PdfA: pdfa,
PdfUa: pdfua, PdfUa: pdfua,
} }
@@ -163,30 +122,27 @@ func convertRoute(engine gotenberg.PdfEngine) api.Route {
// Alright, let's convert the PDFs. // Alright, let's convert the PDFs.
outputPaths := make([]string, len(inputPaths)) outputPaths := make([]string, len(inputPaths))
for i, inputPath := range inputPaths { for i, inputPath := range inputPaths {
outputPaths[i] = ctx.GeneratePath(".pdf") outputPaths[i] = ctx.GeneratePath(".pdf")
err = engine.Convert(ctx, ctx.Log(), pdfFormats, inputPath, outputPaths[i]) err = engine.Convert(ctx, ctx.Log(), pdfFormats, inputPath, outputPaths[i])
if err != nil { if err != nil {
if errors.Is(err, gotenberg.ErrPdfFormatNotSupported) { return fmt.Errorf("convert PDF: %w", err)
return api.WrapError( }
fmt.Errorf("convert PDF: %w", err),
api.NewSentinelHttpError( if len(outputPaths) > 1 {
http.StatusBadRequest, // If .zip archive, keep the original filename.
fmt.Sprintf("At least one PDF engine does not handle one of the PDF format in '%+v', while other have failed to convert for other reasons", pdfFormats), err = ctx.Rename(outputPaths[i], inputPath)
), if err != nil {
) return fmt.Errorf("rename output path: %w", err)
} }
return fmt.Errorf("convert PDF: %w", err) outputPaths[i] = inputPath
} }
} }
// Last but not least, add the output paths to the context so that // Last but not least, add the output paths to the context so that
// the API is able to send them as a response to the client. // the API is able to send them as a response to the client.
err = ctx.AddOutputPaths(outputPaths...) err = ctx.AddOutputPaths(outputPaths...)
if err != nil { if err != nil {
return fmt.Errorf("add output paths: %w", err) return fmt.Errorf("add output paths: %w", err)

View File

@@ -4,13 +4,14 @@ import (
"context" "context"
"errors" "errors"
"net/http" "net/http"
"slices"
"testing" "testing"
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
func TestMergeHandler(t *testing.T) { func TestMergeHandler(t *testing.T) {
@@ -89,34 +90,6 @@ func TestMergeHandler(t *testing.T) {
expectHttpError: false, expectHttpError: false,
expectOutputPathsCount: 1, expectOutputPathsCount: 1,
}, },
{
scenario: "ErrPdfFormatNotSupported",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"file.pdf": "/file.pdf",
"file2.pdf": "/file2.pdf",
})
ctx.SetValues(map[string][]string{
"pdfa": {
gotenberg.PdfA1a,
},
})
return ctx
}(),
engine: &gotenberg.PdfEngineMock{
MergeMock: func(ctx context.Context, logger *zap.Logger, inputPaths []string, outputPath string) error {
return nil
},
ConvertMock: func(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error {
return gotenberg.ErrPdfFormatNotSupported
},
},
expectError: true,
expectHttpError: true,
expectHttpStatus: http.StatusBadRequest,
expectOutputPathsCount: 0,
},
{ {
scenario: "error from PDF engine (convert)", scenario: "error from PDF engine (convert)",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
@@ -127,7 +100,7 @@ func TestMergeHandler(t *testing.T) {
}) })
ctx.SetValues(map[string][]string{ ctx.SetValues(map[string][]string{
"pdfa": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
}, },
}) })
return ctx return ctx
@@ -145,7 +118,7 @@ func TestMergeHandler(t *testing.T) {
expectOutputPathsCount: 0, expectOutputPathsCount: 0,
}, },
{ {
scenario: "success with every PDF/A & PDF/UA form fields", scenario: "success with PDF/A & PDF/UA form fields",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)} ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{ ctx.SetFiles(map[string]string{
@@ -153,11 +126,8 @@ func TestMergeHandler(t *testing.T) {
"file2.pdf": "/file2.pdf", "file2.pdf": "/file2.pdf",
}) })
ctx.SetValues(map[string][]string{ ctx.SetValues(map[string][]string{
"pdfFormat": {
gotenberg.PdfA1a,
},
"pdfa": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
}, },
"pdfua": { "pdfua": {
"true", "true",
@@ -227,6 +197,7 @@ func TestConvertHandler(t *testing.T) {
expectHttpError bool expectHttpError bool
expectHttpStatus int expectHttpStatus int
expectOutputPathsCount int expectOutputPathsCount int
expectOutputPaths []string
}{ }{
{ {
scenario: "missing at least one mandatory file", scenario: "missing at least one mandatory file",
@@ -250,30 +221,6 @@ func TestConvertHandler(t *testing.T) {
expectHttpStatus: http.StatusBadRequest, expectHttpStatus: http.StatusBadRequest,
expectOutputPathsCount: 0, expectOutputPathsCount: 0,
}, },
{
scenario: "ErrPdfFormatNotSupported",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"file.pdf": "/file.pdf",
})
ctx.SetValues(map[string][]string{
"pdfa": {
gotenberg.PdfA1a,
},
})
return ctx
}(),
engine: &gotenberg.PdfEngineMock{
ConvertMock: func(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error {
return gotenberg.ErrPdfFormatNotSupported
},
},
expectError: true,
expectHttpError: true,
expectHttpStatus: http.StatusBadRequest,
expectOutputPathsCount: 0,
},
{ {
scenario: "error from PDF engine", scenario: "error from PDF engine",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
@@ -283,7 +230,7 @@ func TestConvertHandler(t *testing.T) {
}) })
ctx.SetValues(map[string][]string{ ctx.SetValues(map[string][]string{
"pdfa": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
}, },
}) })
return ctx return ctx
@@ -306,7 +253,7 @@ func TestConvertHandler(t *testing.T) {
}) })
ctx.SetValues(map[string][]string{ ctx.SetValues(map[string][]string{
"pdfa": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
}, },
}) })
ctx.SetCancelled(true) ctx.SetCancelled(true)
@@ -322,18 +269,15 @@ func TestConvertHandler(t *testing.T) {
expectOutputPathsCount: 0, expectOutputPathsCount: 0,
}, },
{ {
scenario: "success with every PDF/A & PDF/UA form fields (single file)", scenario: "success with PDF/A & PDF/UA form fields (single file)",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)} ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{ ctx.SetFiles(map[string]string{
"file.pdf": "/file.pdf", "file.pdf": "/file.pdf",
}) })
ctx.SetValues(map[string][]string{ ctx.SetValues(map[string][]string{
"pdfFormat": {
gotenberg.PdfA1a,
},
"pdfa": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
}, },
"pdfua": { "pdfua": {
"true", "true",
@@ -351,7 +295,7 @@ func TestConvertHandler(t *testing.T) {
expectOutputPathsCount: 1, expectOutputPathsCount: 1,
}, },
{ {
scenario: "success with every PDF/A & PDF/UA form fields (many files)", scenario: "cannot rename many files",
ctx: func() *api.ContextMock { ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)} ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{ ctx.SetFiles(map[string]string{
@@ -359,16 +303,46 @@ func TestConvertHandler(t *testing.T) {
"file2.pdf": "/file2.pdf", "file2.pdf": "/file2.pdf",
}) })
ctx.SetValues(map[string][]string{ ctx.SetValues(map[string][]string{
"pdfFormat": {
gotenberg.PdfA1a,
},
"pdfa": { "pdfa": {
gotenberg.PdfA1a, gotenberg.PdfA1b,
}, },
"pdfua": { "pdfua": {
"true", "true",
}, },
}) })
ctx.SetPathRename(&gotenberg.PathRenameMock{RenameMock: func(oldpath, newpath string) error {
return errors.New("cannot rename")
}})
return ctx
}(),
engine: &gotenberg.PdfEngineMock{
ConvertMock: func(ctx context.Context, logger *zap.Logger, formats gotenberg.PdfFormats, inputPath, outputPath string) error {
return nil
},
},
expectError: true,
expectHttpError: false,
expectOutputPathsCount: 0,
},
{
scenario: "success with PDF/A & PDF/UA form fields (many files)",
ctx: func() *api.ContextMock {
ctx := &api.ContextMock{Context: new(api.Context)}
ctx.SetFiles(map[string]string{
"file.pdf": "/file.pdf",
"file2.pdf": "/file2.pdf",
})
ctx.SetValues(map[string][]string{
"pdfa": {
gotenberg.PdfA1b,
},
"pdfua": {
"true",
},
})
ctx.SetPathRename(&gotenberg.PathRenameMock{RenameMock: func(oldpath, newpath string) error {
return nil
}})
return ctx return ctx
}(), }(),
engine: &gotenberg.PdfEngineMock{ engine: &gotenberg.PdfEngineMock{
@@ -379,6 +353,7 @@ func TestConvertHandler(t *testing.T) {
expectError: false, expectError: false,
expectHttpError: false, expectHttpError: false,
expectOutputPathsCount: 2, expectOutputPathsCount: 2,
expectOutputPaths: []string{"/file.pdf", "/file2.pdf"},
}, },
} { } {
t.Run(tc.scenario, func(t *testing.T) { t.Run(tc.scenario, func(t *testing.T) {
@@ -417,6 +392,12 @@ func TestConvertHandler(t *testing.T) {
if tc.expectOutputPathsCount != len(tc.ctx.OutputPaths()) { if tc.expectOutputPathsCount != len(tc.ctx.OutputPaths()) {
t.Errorf("expected %d output paths but got %d", tc.expectOutputPathsCount, len(tc.ctx.OutputPaths())) t.Errorf("expected %d output paths but got %d", tc.expectOutputPathsCount, len(tc.ctx.OutputPaths()))
} }
for _, path := range tc.expectOutputPaths {
if !slices.Contains(tc.ctx.OutputPaths(), path) {
t.Errorf("expected '%s' in output paths %v", path, tc.ctx.OutputPaths())
}
}
}) })
} }
} }

View File

@@ -5,11 +5,10 @@ import (
"errors" "errors"
"fmt" "fmt"
"os" "os"
"sync"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func init() { func init() {
@@ -52,23 +51,6 @@ func (engine *PdfTk) Validate() error {
return nil return nil
} }
// Metrics returns the metrics.
func (engine *PdfTk) Metrics() ([]gotenberg.Metric, error) {
// TODO: remove deprecated.
return []gotenberg.Metric{
{
Name: "pdftk_active_instances_count",
Description: "Current number of active PDFtk instances - deprecated.",
Read: func() float64 {
activeInstancesCountMu.RLock()
defer activeInstancesCountMu.RUnlock()
return activeInstancesCount
},
},
}, nil
}
// Merge combines multiple PDFs into a single PDF. // Merge combines multiple PDFs into a single PDF.
func (engine *PdfTk) Merge(ctx context.Context, logger *zap.Logger, inputPaths []string, outputPath string) error { func (engine *PdfTk) Merge(ctx context.Context, logger *zap.Logger, inputPaths []string, outputPath string) error {
var args []string var args []string
@@ -80,16 +62,7 @@ func (engine *PdfTk) Merge(ctx context.Context, logger *zap.Logger, inputPaths [
return fmt.Errorf("create command: %w", err) return fmt.Errorf("create command: %w", err)
} }
activeInstancesCountMu.Lock()
activeInstancesCount += 1
activeInstancesCountMu.Unlock()
_, err = cmd.Exec() _, err = cmd.Exec()
activeInstancesCountMu.Lock()
activeInstancesCount -= 1
activeInstancesCountMu.Unlock()
if err == nil { if err == nil {
return nil return nil
} }
@@ -102,16 +75,10 @@ func (engine *PdfTk) Convert(ctx context.Context, logger *zap.Logger, formats go
return fmt.Errorf("convert PDF to '%+v' with PDFtk: %w", formats, gotenberg.ErrPdfEngineMethodNotSupported) return fmt.Errorf("convert PDF to '%+v' with PDFtk: %w", formats, gotenberg.ErrPdfEngineMethodNotSupported)
} }
var (
activeInstancesCount float64
activeInstancesCountMu sync.RWMutex
)
// Interface guards. // Interface guards.
var ( var (
_ gotenberg.Module = (*PdfTk)(nil) _ gotenberg.Module = (*PdfTk)(nil)
_ gotenberg.Provisioner = (*PdfTk)(nil) _ gotenberg.Provisioner = (*PdfTk)(nil)
_ gotenberg.Validator = (*PdfTk)(nil) _ gotenberg.Validator = (*PdfTk)(nil)
_ gotenberg.MetricsProvider = (*PdfTk)(nil) _ gotenberg.PdfEngine = (*PdfTk)(nil)
_ gotenberg.PdfEngine = (*PdfTk)(nil)
) )

View File

@@ -9,7 +9,7 @@ import (
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestPdfTk_Descriptor(t *testing.T) { func TestPdfTk_Descriptor(t *testing.T) {
@@ -71,22 +71,6 @@ func TestPdfTk_Validate(t *testing.T) {
} }
} }
func TestPdfTk_Metrics(t *testing.T) {
metrics, err := new(PdfTk).Metrics()
if err != nil {
t.Fatalf("expected no error but got: %v", err)
}
if len(metrics) != 1 {
t.Fatalf("expected %d metrics, but got %d", 1, len(metrics))
}
actual := metrics[0].Read()
if actual != 0 {
t.Errorf("expected %d PDFtk instances, but got %f", 0, actual)
}
}
func TestPdfTk_Merge(t *testing.T) { func TestPdfTk_Merge(t *testing.T) {
for _, tc := range []struct { for _, tc := range []struct {
scenario string scenario string

View File

@@ -12,8 +12,8 @@ import (
"github.com/prometheus/client_golang/prometheus/promhttp" "github.com/prometheus/client_golang/prometheus/promhttp"
flag "github.com/spf13/pflag" flag "github.com/spf13/pflag"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
func init() { func init() {

View File

@@ -8,7 +8,7 @@ import (
"github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestPrometheus_Descriptor(t *testing.T) { func TestPrometheus_Descriptor(t *testing.T) {

View File

@@ -5,11 +5,10 @@ import (
"errors" "errors"
"fmt" "fmt"
"os" "os"
"sync"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func init() { func init() {
@@ -52,23 +51,6 @@ func (engine *QPdf) Validate() error {
return nil return nil
} }
// Metrics returns the metrics.
func (engine *QPdf) Metrics() ([]gotenberg.Metric, error) {
// TODO: remove deprecated.
return []gotenberg.Metric{
{
Name: "qpdf_active_instances_count",
Description: "Current number of active QPDF instances - deprecated.",
Read: func() float64 {
activeInstancesCountMu.RLock()
defer activeInstancesCountMu.RUnlock()
return activeInstancesCount
},
},
}, nil
}
// Merge combines multiple PDFs into a single PDF. // Merge combines multiple PDFs into a single PDF.
func (engine *QPdf) Merge(ctx context.Context, logger *zap.Logger, inputPaths []string, outputPath string) error { func (engine *QPdf) Merge(ctx context.Context, logger *zap.Logger, inputPaths []string, outputPath string) error {
var args []string var args []string
@@ -82,16 +64,7 @@ func (engine *QPdf) Merge(ctx context.Context, logger *zap.Logger, inputPaths []
return fmt.Errorf("create command: %w", err) return fmt.Errorf("create command: %w", err)
} }
activeInstancesCountMu.Lock()
activeInstancesCount += 1
activeInstancesCountMu.Unlock()
_, err = cmd.Exec() _, err = cmd.Exec()
activeInstancesCountMu.Lock()
activeInstancesCount -= 1
activeInstancesCountMu.Unlock()
if err == nil { if err == nil {
return nil return nil
} }
@@ -105,14 +78,8 @@ func (engine *QPdf) Convert(ctx context.Context, logger *zap.Logger, formats got
} }
var ( var (
activeInstancesCount float64 _ gotenberg.Module = (*QPdf)(nil)
activeInstancesCountMu sync.RWMutex _ gotenberg.Provisioner = (*QPdf)(nil)
) _ gotenberg.Validator = (*QPdf)(nil)
_ gotenberg.PdfEngine = (*QPdf)(nil)
var (
_ gotenberg.Module = (*QPdf)(nil)
_ gotenberg.Provisioner = (*QPdf)(nil)
_ gotenberg.Validator = (*QPdf)(nil)
_ gotenberg.MetricsProvider = (*QPdf)(nil)
_ gotenberg.PdfEngine = (*QPdf)(nil)
) )

View File

@@ -9,7 +9,7 @@ import (
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestQPdf_Descriptor(t *testing.T) { func TestQPdf_Descriptor(t *testing.T) {
@@ -71,22 +71,6 @@ func TestQPdf_Validate(t *testing.T) {
} }
} }
func TestQPdf_Metrics(t *testing.T) {
metrics, err := new(QPdf).Metrics()
if err != nil {
t.Errorf("expected no error but got: %v", err)
}
if len(metrics) != 1 {
t.Errorf("expected %d metrics, but got %d", 1, len(metrics))
}
actual := metrics[0].Read()
if actual != 0 {
t.Errorf("expected %d QPdf instances, but got %f", 0, actual)
}
}
func TestQPdf_Merge(t *testing.T) { func TestQPdf_Merge(t *testing.T) {
for _, tc := range []struct { for _, tc := range []struct {
scenario string scenario string

View File

@@ -9,7 +9,6 @@ import (
"fmt" "fmt"
"net/http" "net/http"
"os" "os"
"regexp"
"strconv" "strconv"
"strings" "strings"
"time" "time"
@@ -17,7 +16,8 @@ import (
"github.com/hashicorp/go-retryablehttp" "github.com/hashicorp/go-retryablehttp"
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
func webhookMiddleware(w *Webhook) api.Middleware { func webhookMiddleware(w *Webhook) api.Middleware {
@@ -44,38 +44,19 @@ func webhookMiddleware(w *Webhook) api.Middleware {
) )
} }
// Let's check if the webhook URLs are acceptable according to our deadline, ok := ctx.Deadline()
// allowed/denied lists. if !ok {
filter := func(URL, header string, allowList, denyList *regexp.Regexp) error { return errors.New("context has no deadline")
if !allowList.MatchString(URL) {
return api.WrapError(
fmt.Errorf("'%s' does not match the expression from the allowed list", URL),
api.NewSentinelHttpError(
http.StatusForbidden,
fmt.Sprintf("Invalid '%s' header value: '%s' does not match the authorized URLs", header, URL),
),
)
}
if denyList.String() != "" && denyList.MatchString(URL) {
return api.WrapError(
fmt.Errorf("'%s' matches the expression from the denied list", URL),
api.NewSentinelHttpError(
http.StatusForbidden,
fmt.Sprintf("Invalid '%s' header value: '%s' does not match the authorized URLs", header, URL),
),
)
}
return nil
} }
err := filter(webhookUrl, "Gotenberg-Webhook-Url", w.allowList, w.denyList) // Let's check if the webhook URLs are acceptable according to our
// allowed/denied lists.
err := gotenberg.FilterDeadline(w.allowList, w.denyList, webhookUrl, deadline)
if err != nil { if err != nil {
return fmt.Errorf("filter webhook URL: %w", err) return fmt.Errorf("filter webhook URL: %w", err)
} }
err = filter(webhookErrorUrl, "Gotenberg-Webhook-Error-Url", w.errorAllowList, w.errorDenyList) err = gotenberg.FilterDeadline(w.errorAllowList, w.errorDenyList, webhookErrorUrl, deadline)
if err != nil { if err != nil {
return fmt.Errorf("filter webhook error URL: %w", err) return fmt.Errorf("filter webhook error URL: %w", err)
} }

View File

@@ -11,15 +11,15 @@ import (
"mime/multipart" "mime/multipart"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"regexp"
"strings" "strings"
"testing" "testing"
"time" "time"
"github.com/dlclark/regexp2"
"github.com/labstack/echo/v4" "github.com/labstack/echo/v4"
"go.uber.org/zap" "go.uber.org/zap"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
func TestWebhookMiddlewareGuards(t *testing.T) { func TestWebhookMiddlewareGuards(t *testing.T) {
@@ -47,10 +47,10 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
buildWebhookModule := func() *Webhook { buildWebhookModule := func() *Webhook {
return &Webhook{ return &Webhook{
allowList: regexp.MustCompile(""), allowList: regexp2.MustCompile("", 0),
denyList: regexp.MustCompile(""), denyList: regexp2.MustCompile("", 0),
errorAllowList: regexp.MustCompile(""), errorAllowList: regexp2.MustCompile("", 0),
errorDenyList: regexp.MustCompile(""), errorDenyList: regexp2.MustCompile("", 0),
maxRetry: 0, maxRetry: 0,
retryMinWait: 0, retryMinWait: 0,
retryMaxWait: 0, retryMaxWait: 0,
@@ -63,6 +63,7 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
request *http.Request request *http.Request
mod *Webhook mod *Webhook
next echo.HandlerFunc next echo.HandlerFunc
noDeadline bool
expectError bool expectError bool
expectHttpError bool expectHttpError bool
expectHttpStatus int expectHttpStatus int
@@ -76,6 +77,7 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
return nil return nil
} }
}(), }(),
noDeadline: false,
expectError: false, expectError: false,
expectHttpError: false, expectHttpError: false,
}, },
@@ -87,10 +89,23 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
return req return req
}(), }(),
mod: buildWebhookModule(), mod: buildWebhookModule(),
noDeadline: false,
expectError: true, expectError: true,
expectHttpError: true, expectHttpError: true,
expectHttpStatus: http.StatusBadRequest, expectHttpStatus: http.StatusBadRequest,
}, },
{
scenario: "context has no deadline",
request: func() *http.Request {
req := buildMultipartFormDataRequest()
req.Header.Set("Gotenberg-Webhook-Url", "foo")
req.Header.Set("Gotenberg-Webhook-Error-Url", "bar")
return req
}(),
mod: buildWebhookModule(),
noDeadline: true,
expectError: true,
},
{ {
scenario: "webhook URL is not allowed", scenario: "webhook URL is not allowed",
request: func() *http.Request { request: func() *http.Request {
@@ -101,12 +116,11 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
}(), }(),
mod: func() *Webhook { mod: func() *Webhook {
mod := buildWebhookModule() mod := buildWebhookModule()
mod.allowList = regexp.MustCompile("bar") mod.allowList = regexp2.MustCompile("bar", 0)
return mod return mod
}(), }(),
expectError: true, noDeadline: false,
expectHttpError: true, expectError: true,
expectHttpStatus: http.StatusForbidden,
}, },
{ {
scenario: "webhook URL is denied", scenario: "webhook URL is denied",
@@ -118,12 +132,11 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
}(), }(),
mod: func() *Webhook { mod: func() *Webhook {
mod := buildWebhookModule() mod := buildWebhookModule()
mod.denyList = regexp.MustCompile("foo") mod.denyList = regexp2.MustCompile("foo", 0)
return mod return mod
}(), }(),
expectError: true, noDeadline: false,
expectHttpError: true, expectError: true,
expectHttpStatus: http.StatusForbidden,
}, },
{ {
scenario: "webhook error URL is not allowed", scenario: "webhook error URL is not allowed",
@@ -135,12 +148,11 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
}(), }(),
mod: func() *Webhook { mod: func() *Webhook {
mod := buildWebhookModule() mod := buildWebhookModule()
mod.errorAllowList = regexp.MustCompile("foo") mod.errorAllowList = regexp2.MustCompile("foo", 0)
return mod return mod
}(), }(),
expectError: true, noDeadline: false,
expectHttpError: true, expectError: true,
expectHttpStatus: http.StatusForbidden,
}, },
{ {
scenario: "webhook error URL is denied", scenario: "webhook error URL is denied",
@@ -152,12 +164,11 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
}(), }(),
mod: func() *Webhook { mod: func() *Webhook {
mod := buildWebhookModule() mod := buildWebhookModule()
mod.errorDenyList = regexp.MustCompile("bar") mod.errorDenyList = regexp2.MustCompile("bar", 0)
return mod return mod
}(), }(),
expectError: true, noDeadline: false,
expectHttpError: true, expectError: true,
expectHttpStatus: http.StatusForbidden,
}, },
{ {
scenario: "invalid webhook method (GET)", scenario: "invalid webhook method (GET)",
@@ -169,6 +180,7 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
return req return req
}(), }(),
mod: buildWebhookModule(), mod: buildWebhookModule(),
noDeadline: false,
expectError: true, expectError: true,
expectHttpError: true, expectHttpError: true,
expectHttpStatus: http.StatusBadRequest, expectHttpStatus: http.StatusBadRequest,
@@ -183,6 +195,7 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
return req return req
}(), }(),
mod: buildWebhookModule(), mod: buildWebhookModule(),
noDeadline: false,
expectError: true, expectError: true,
expectHttpError: true, expectHttpError: true,
expectHttpStatus: http.StatusBadRequest, expectHttpStatus: http.StatusBadRequest,
@@ -198,6 +211,7 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
return req return req
}(), }(),
mod: buildWebhookModule(), mod: buildWebhookModule(),
noDeadline: false,
expectError: true, expectError: true,
expectHttpError: true, expectHttpError: true,
expectHttpStatus: http.StatusBadRequest, expectHttpStatus: http.StatusBadRequest,
@@ -213,6 +227,7 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
return req return req
}(), }(),
mod: buildWebhookModule(), mod: buildWebhookModule(),
noDeadline: false,
expectError: true, expectError: true,
expectHttpError: true, expectHttpError: true,
expectHttpStatus: http.StatusBadRequest, expectHttpStatus: http.StatusBadRequest,
@@ -228,6 +243,7 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
return req return req
}(), }(),
mod: buildWebhookModule(), mod: buildWebhookModule(),
noDeadline: false,
expectError: true, expectError: true,
expectHttpError: true, expectHttpError: true,
expectHttpStatus: http.StatusBadRequest, expectHttpStatus: http.StatusBadRequest,
@@ -242,6 +258,7 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
return req return req
}(), }(),
mod: buildWebhookModule(), mod: buildWebhookModule(),
noDeadline: false,
expectError: true, expectError: true,
expectHttpError: true, expectHttpError: true,
expectHttpStatus: http.StatusBadRequest, expectHttpStatus: http.StatusBadRequest,
@@ -254,15 +271,20 @@ func TestWebhookMiddlewareGuards(t *testing.T) {
c := srv.NewContext(tc.request, httptest.NewRecorder()) c := srv.NewContext(tc.request, httptest.NewRecorder())
ctx := &api.ContextMock{Context: &api.Context{}} if tc.noDeadline {
ctx.SetEchoContext(c) ctx := &api.ContextMock{Context: &api.Context{Context: context.Background()}}
ctx.SetEchoContext(c)
c.Set("context", ctx.Context) c.Set("context", ctx.Context)
c.Set("cancel", func() context.CancelFunc { c.Set("cancel", func() context.CancelFunc {
return func() { return nil
return }())
} } else {
}()) timeoutCtx, cancel := context.WithTimeout(context.Background(), time.Duration(10)*time.Second)
ctx := &api.ContextMock{Context: &api.Context{Context: timeoutCtx}}
ctx.SetEchoContext(c)
c.Set("context", ctx.Context)
c.Set("cancel", cancel)
}
err := webhookMiddleware(tc.mod).Handler(tc.next)(c) err := webhookMiddleware(tc.mod).Handler(tc.next)(c)
@@ -320,10 +342,10 @@ func TestWebhookMiddlewareAsynchronousProcess(t *testing.T) {
buildWebhookModule := func() *Webhook { buildWebhookModule := func() *Webhook {
return &Webhook{ return &Webhook{
allowList: regexp.MustCompile(""), allowList: regexp2.MustCompile("", 0),
denyList: regexp.MustCompile(""), denyList: regexp2.MustCompile("", 0),
errorAllowList: regexp.MustCompile(""), errorAllowList: regexp2.MustCompile("", 0),
errorDenyList: regexp.MustCompile(""), errorDenyList: regexp2.MustCompile("", 0),
maxRetry: 0, maxRetry: 0,
retryMinWait: 0, retryMinWait: 0,
retryMaxWait: 0, retryMaxWait: 0,
@@ -426,22 +448,17 @@ func TestWebhookMiddlewareAsynchronousProcess(t *testing.T) {
c.Set("trace", "foo") c.Set("trace", "foo")
c.Set("startTime", time.Now()) c.Set("startTime", time.Now())
ctx := &api.ContextMock{Context: &api.Context{}} timeoutCtx, cancel := context.WithTimeout(context.Background(), time.Duration(10)*time.Second)
ctx := &api.ContextMock{Context: &api.Context{Context: timeoutCtx}}
ctx.SetLogger(zap.NewNop()) ctx.SetLogger(zap.NewNop())
ctx.SetEchoContext(c) ctx.SetEchoContext(c)
c.Set("context", ctx.Context) c.Set("context", ctx.Context)
c.Set("cancel", func() context.CancelFunc { c.Set("cancel", cancel)
return func() {
return
}
}())
webhook := echo.New() webhook := echo.New()
webhook.HideBanner = true webhook.HideBanner = true
webhook.HidePort = true webhook.HidePort = true
rand.Seed(time.Now().UnixNano())
webhookPort := rand.Intn(65535-1025+1) + 1025 webhookPort := rand.Intn(65535-1025+1) + 1025
c.Request().Header.Set("Gotenberg-Webhook-Url", fmt.Sprintf("http://localhost:%d/", webhookPort)) c.Request().Header.Set("Gotenberg-Webhook-Url", fmt.Sprintf("http://localhost:%d/", webhookPort))

View File

@@ -1,15 +1,13 @@
package webhook package webhook
import ( import (
"fmt"
"regexp"
"time" "time"
"github.com/dlclark/regexp2"
flag "github.com/spf13/pflag" flag "github.com/spf13/pflag"
"go.uber.org/multierr"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
"github.com/gotenberg/gotenberg/v7/pkg/modules/api" "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
) )
func init() { func init() {
@@ -19,10 +17,10 @@ func init() {
// Webhook is a module which provides a middleware for uploading output files // Webhook is a module which provides a middleware for uploading output files
// to any destinations in an asynchronous fashion. // to any destinations in an asynchronous fashion.
type Webhook struct { type Webhook struct {
allowList *regexp.Regexp allowList *regexp2.Regexp
denyList *regexp.Regexp denyList *regexp2.Regexp
errorAllowList *regexp.Regexp errorAllowList *regexp2.Regexp
errorDenyList *regexp.Regexp errorDenyList *regexp2.Regexp
maxRetry int maxRetry int
retryMinWait time.Duration retryMinWait time.Duration
retryMaxWait time.Duration retryMaxWait time.Duration
@@ -36,31 +34,6 @@ func (w *Webhook) Descriptor() gotenberg.ModuleDescriptor {
ID: "webhook", ID: "webhook",
FlagSet: func() *flag.FlagSet { FlagSet: func() *flag.FlagSet {
fs := flag.NewFlagSet("webhook", flag.ExitOnError) fs := flag.NewFlagSet("webhook", flag.ExitOnError)
// Deprecated flags.
fs.String("api-webhook-allow-list", "", "Set the allowed URLs for the webhook feature using a regular expression")
fs.String("api-webhook-deny-list", "", "Set the denied URLs for the webhook feature using a regular expression")
fs.String("api-webhook-error-allow-list", "", "Set the allowed URLs in case of an error for the webhook feature using a regular expression")
fs.String("api-webhook-error-deny-list", "", "Set the denied URLs in case of an error for the webhook feature using a regular expression")
fs.Int("api-webhook-max-retry", 4, "Set the maximum number of retries for the webhook feature")
fs.Duration("api-webhook-retry-min-wait", time.Duration(1)*time.Second, "Set the minimum duration to wait before trying to call the webhook again")
fs.Duration("api-webhook-retry-max-wait", time.Duration(30)*time.Second, "Set the maximum duration to wait before trying to call the webhook again")
fs.Bool("api-disable-webhook", false, "Disable the webhook feature")
var err error
err = multierr.Append(err, fs.MarkDeprecated("api-webhook-allow-list", "use webhook-allow-list instead"))
err = multierr.Append(err, fs.MarkDeprecated("api-webhook-deny-list", "use webhook-deny-list instead"))
err = multierr.Append(err, fs.MarkDeprecated("api-webhook-error-allow-list", "use webhook-error-allow-list instead"))
err = multierr.Append(err, fs.MarkDeprecated("api-webhook-error-deny-list", "use webhook-error-deny-list instead"))
err = multierr.Append(err, fs.MarkDeprecated("api-webhook-max-retry", "use webhook-max-retry instead"))
err = multierr.Append(err, fs.MarkDeprecated("api-webhook-retry-min-wait", "use webhook-retry-min-wait instead"))
err = multierr.Append(err, fs.MarkDeprecated("api-webhook-retry-max-wait", "use webhook-retry-max-wait instead"))
err = multierr.Append(err, fs.MarkDeprecated("api-disable-webhook", "use webhook-disable instead"))
if err != nil {
panic(fmt.Errorf("create deprecated flags for the webhook module: %v", err))
}
// New flags.
fs.String("webhook-allow-list", "", "Set the allowed URLs for the webhook feature using a regular expression") fs.String("webhook-allow-list", "", "Set the allowed URLs for the webhook feature using a regular expression")
fs.String("webhook-deny-list", "", "Set the denied URLs for the webhook feature using a regular expression") fs.String("webhook-deny-list", "", "Set the denied URLs for the webhook feature using a regular expression")
fs.String("webhook-error-allow-list", "", "Set the allowed URLs in case of an error for the webhook feature using a regular expression") fs.String("webhook-error-allow-list", "", "Set the allowed URLs in case of an error for the webhook feature using a regular expression")
@@ -80,15 +53,15 @@ func (w *Webhook) Descriptor() gotenberg.ModuleDescriptor {
// Provision sets the module properties. // Provision sets the module properties.
func (w *Webhook) Provision(ctx *gotenberg.Context) error { func (w *Webhook) Provision(ctx *gotenberg.Context) error {
flags := ctx.ParsedFlags() flags := ctx.ParsedFlags()
w.allowList = flags.MustDeprecatedRegexp("api-webhook-allow-list", "webhook-allow-list") w.allowList = flags.MustRegexp("webhook-allow-list")
w.denyList = flags.MustDeprecatedRegexp("api-webhook-deny-list", "webhook-deny-list") w.denyList = flags.MustRegexp("webhook-deny-list")
w.errorAllowList = flags.MustDeprecatedRegexp("api-webhook-error-allow-list", "webhook-error-allow-list") w.errorAllowList = flags.MustRegexp("webhook-error-allow-list")
w.errorDenyList = flags.MustDeprecatedRegexp("api-webhook-error-deny-list", "webhook-error-deny-list") w.errorDenyList = flags.MustRegexp("webhook-error-deny-list")
w.maxRetry = flags.MustDeprecatedInt("api-webhook-max-retry", "webhook-max-retry") w.maxRetry = flags.MustInt("webhook-max-retry")
w.retryMinWait = flags.MustDeprecatedDuration("api-webhook-retry-min-wait", "webhook-retry-min-wait") w.retryMinWait = flags.MustDuration("webhook-retry-min-wait")
w.retryMaxWait = flags.MustDeprecatedDuration("api-webhook-retry-min-wait", "webhook-retry-max-wait") w.retryMaxWait = flags.MustDuration("webhook-retry-max-wait")
w.clientTimeout = flags.MustDuration("webhook-client-timeout") w.clientTimeout = flags.MustDuration("webhook-client-timeout")
w.disable = flags.MustDeprecatedBool("api-disable-webhook", "webhook-disable") w.disable = flags.MustBool("webhook-disable")
return nil return nil
} }

View File

@@ -4,7 +4,7 @@ import (
"reflect" "reflect"
"testing" "testing"
"github.com/gotenberg/gotenberg/v7/pkg/gotenberg" "github.com/gotenberg/gotenberg/v8/pkg/gotenberg"
) )
func TestWebhook_Descriptor(t *testing.T) { func TestWebhook_Descriptor(t *testing.T) {

View File

@@ -2,16 +2,16 @@ package standard
import ( import (
// Standard Gotenberg modules. // Standard Gotenberg modules.
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/api" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/api"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/chromium" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/chromium"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/libreoffice" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/libreoffice"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/libreoffice/api" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/libreoffice/api"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/libreoffice/pdfengine" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/libreoffice/pdfengine"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/logging" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/logging"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/pdfcpu" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/pdfcpu"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/pdfengines" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/pdfengines"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/pdftk" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/pdftk"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/prometheus" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/prometheus"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/qpdf" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/qpdf"
_ "github.com/gotenberg/gotenberg/v7/pkg/modules/webhook" _ "github.com/gotenberg/gotenberg/v8/pkg/modules/webhook"
) )

View File

@@ -2,8 +2,8 @@
<head> <head>
<style> <style>
body { body {
font-size: 30rem; font-size: 12px;
margin: 4rem auto; margin: auto 20px;
} }
</style> </style>
</head> </head>

View File

@@ -2,8 +2,8 @@
<head> <head>
<style> <style>
body { body {
font-size: 8rem; font-size: 12px;
margin: 4rem auto; margin: auto 20px;
} }
</style> </style>
</head> </head>

View File

@@ -59,6 +59,14 @@
<p id="javascript" style="display: none">Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p> <p id="javascript" style="display: none">Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
</div> </div>
<div class="page-break-after">
<h2>/etc/passwd</h2>
<iframe src="/etc/passwd"></iframe>
<h2>\\localhost/etc/passwd</h2>
<iframe src="\\localhost/etc/passwd"></iframe>
</div>
<script type="application/javascript"> <script type="application/javascript">
var globalVar = 'notReady' var globalVar = 'notReady'

View File

@@ -2,8 +2,8 @@
<head> <head>
<style> <style>
body { body {
font-size: 8rem; font-size: 12px;
margin: 4rem auto; margin: auto 20px;
} }
</style> </style>
</head> </head>

View File

@@ -2,8 +2,8 @@
<head> <head>
<style> <style>
body { body {
font-size: 8rem; font-size: 12px;
margin: 4rem auto; margin: auto 20px;
} }
</style> </style>
</head> </head>