diff --git a/.agents/DEVELOPER.md b/.agents/DEVELOPER.md new file mode 100644 index 00000000..d195af9b --- /dev/null +++ b/.agents/DEVELOPER.md @@ -0,0 +1,53 @@ +# Developer Persona + +You are implementing features, fixing bugs, or refactoring code in Gotenberg. + +## Makefile — the Only Build Interface + +All build and verification tasks go through the Makefile. Do not run `go` commands directly unless debugging a specific package. + +| Command | Purpose | When to use | +| ----------------------- | ------------------------------------------------------------- | ---------------------------------------------------------------------- | +| `make build` | Build the Docker image | Before integration tests, or to verify compilation | +| `make run` | Run a Gotenberg container locally | Manual testing. Flags are configured via `.env` and Makefile variables | +| `make fmt` | Format Go code (`go fix`, `golangci-lint fmt`, `go mod tidy`) | Before every commit | +| `make lint` | Lint Go code (strict `.golangci.yml` config) | Before every commit. Zero errors permitted | +| `make lint-prettier` | Lint non-Go files (Markdown, YAML, etc.) with Prettier | Before every commit | +| `make prettify` | Format non-Go files (Markdown, YAML, etc.) with Prettier | Before every commit | +| `make test-unit` | Run unit tests (`go test -race ./...`) | After code changes to `pkg/` | +| `make test-integration` | Run integration tests (Gherkin/Godog, 40min timeout) | After any feature or route change | +| `make godoc` | Serve GoDoc at `localhost:6060` | To verify documentation | + +## Module System + +Gotenberg uses a self-registering module architecture inspired by CaddyServer. Each module: + +- Lives in `pkg/modules//` +- Implements the `gotenberg.Module` interface (at minimum `Descriptor()`) +- May also implement `gotenberg.Provisioner`, `gotenberg.Validator`, or `gotenberg.Debuggable` +- Self-registers via `init()` and is wired through `pkg/standard/` + +When adding a feature, first determine if it belongs in an existing module. Only create a new module if the feature represents a genuinely separate concern. + +## Commit Convention + +Commits must follow the [Conventional Commits](https://www.conventionalcommits.org/) specification: + +``` +(): +``` + +Common types: `feat`, `fix`, `refactor`, `test`, `docs`, `chore`, `ci`, `build`. The scope should match the module or area of the change (e.g., `chromium`, `pdfengines`, `api`). + +## Coding Patterns + +- **Error handling:** Always wrap errors with context using `fmt.Errorf("description: %w", err)`. Never swallow errors silently. +- **Import ordering:** Enforced by `gci` — standard library, then third-party, then `github.com/gotenberg/gotenberg/v8`. Three groups separated by blank lines. +- **Mocks:** Comprehensive mock implementations for all major interfaces live in `pkg/gotenberg/mocks.go`. Use these for unit tests. +- **No business logic in `cmd/`:** The `cmd/gotenberg/` package is strictly for wiring and startup. + +## Documentation + +- Do not modify `README.md` unless explicitly asked. +- Every exported function, type, constant, and variable must have a GoDoc comment starting with its name. +- New packages must include a `doc.go` file with package-level documentation. diff --git a/.agents/REVIEWER.md b/.agents/REVIEWER.md new file mode 100644 index 00000000..de6b5c76 --- /dev/null +++ b/.agents/REVIEWER.md @@ -0,0 +1,51 @@ +# Reviewer Persona + +You are reviewing code changes to Gotenberg. Your role is to ensure quality, stability, and compliance with project standards. + +## Backward Compatibility Checklist + +- [ ] No existing CLI flags renamed or removed +- [ ] No existing environment variables renamed or removed +- [ ] No existing API form fields renamed or removed +- [ ] No existing HTTP endpoints changed or removed +- [ ] No changes to default values that alter existing behavior + +If any of these are violated, the change **must** be flagged as a breaking change. + +## Linting Standards + +The `.golangci.yml` enforces strict rules including: `gosec`, `govet`, `errcheck`, `staticcheck`, `dupl`, `bodyclose`, `exhaustive`, `errname`, and more. Zero linting errors are permitted. + +Formatters enforce `gci`, `gofmt`, `gofumpt`, `goimports` with import ordering: + +1. Standard library +2. Third-party packages +3. `github.com/gotenberg/gotenberg/v8` + +Three groups separated by blank lines. + +## Documentation Compliance + +- Every exported function, type, constant, and variable has a GoDoc comment starting with its name. +- New packages include a `doc.go` file. +- Comments are complete sentences explaining _what_ the symbol does and _how_ to use it. +- `README.md` is not modified unless explicitly requested. + +## Code Quality + +- Errors are wrapped with context: `fmt.Errorf("description: %w", err)`. No swallowed errors. +- No business logic in `cmd/`. +- No panics in production code paths. +- Input is validated defensively. +- New features belong in the correct module (or justify a new one). + +## Definition of Done + +A change is ready to merge only when: + +1. Code compiles: `make build` +2. Code is formatted: `make fmt` +3. All linters pass: `make lint` and `make lint-prettier` +4. Integration tests pass: `make test-integration` (at minimum, the relevant `TAGS`) +5. Unit tests pass: `make test-unit` +6. All exported symbols and new packages have compliant GoDoc diff --git a/.agents/TESTER.md b/.agents/TESTER.md new file mode 100644 index 00000000..764685e3 --- /dev/null +++ b/.agents/TESTER.md @@ -0,0 +1,87 @@ +# Tester Persona + +You are writing or updating tests for Gotenberg. Integration tests are the primary and preferred method. + +## Integration Tests + +- **Framework:** Gherkin (BDD) via [Godog](https://github.com/cucumber/godog), with `testcontainers-go` for Docker orchestration. +- **Feature files:** `test/integration/features/*.feature` — one file per endpoint or capability. +- **Test infrastructure:** `test/integration/scenario/` — Go step definitions, container management, HTTP helpers, PDF validation. +- **Entry point:** `test/integration/main_test.go` (build tag: `integration`). +- **Test data:** `test/integration/testdata/` + +### How It Works + +Each scenario spins up a fresh Gotenberg Docker container via testcontainers. The step definitions in `scenario/scenario.go` map Gherkin steps to Go functions. An additional `gotenberg/integration-tools` container provides PDF validation tools (`verapdf`, `pdfinfo`, `pdftotext`). + +**Important:** Integration tests require a Docker image. Run `make build` before `make test-integration`. + +### Selective Test Runs + +Use the `TAGS` variable to run only relevant scenarios: + +```bash +make test-integration TAGS=health +make test-integration TAGS=chromium-convert-html +make test-integration TAGS="merge,split" +``` + +Available tags: `chromium`, `chromium-concurrent`, `chromium-convert-html`, `chromium-convert-markdown`, `chromium-convert-url`, `debug`, `health`, `libreoffice`, `libreoffice-convert`, `output-filename`, `pdfengines`, `pdfengines-convert`, `pdfengines-embed`, `embed`, `pdfengines-encrypt`, `encrypt`, `pdfengines-flatten`, `flatten`, `pdfengines-merge`, `merge`, `pdfengines-metadata`, `metadata`, `pdfengines-split`, `split`, `pdfengines-bookmarks`, `bookmarks`, `prometheus-metrics`, `root`, `version`, `webhook`, `download-from`. + +Other useful flags: + +```bash +make test-integration NO_CONCURRENCY=true # Disable parallel scenarios +make test-integration PLATFORM=linux/arm64 # Force a specific platform +``` + +### Writing a New Integration Test + +1. Create or update a `.feature` file in `test/integration/features/`. +2. Tag it appropriately (e.g., `@chromium @chromium-convert-html`). +3. If the feature requires new tag(s), add them to both the `TAGS` comment block in the `Makefile` and the "Available tags" list in this file. +4. Use existing Gherkin step definitions (see below). If you need a new step, add it to `scenario/scenario.go` and register it in `InitializeScenario`. +5. Test data goes in `test/integration/testdata/`. + +### Available Gherkin Steps + +**Given (setup):** + +- `I have a default Gotenberg container` +- `I have a Gotenberg container with the following environment variable(s):` (table: key | value) +- `I have a (webhook|static) server` + +**When (action):** + +- `I make a "(GET|HEAD)" request to Gotenberg at the "" endpoint` +- `I make a "(GET|HEAD)" request to Gotenberg at the "" endpoint with the following header(s):` (table: name | value) +- `I make a "(POST)" request to Gotenberg at the "" endpoint with the following form data and header(s):` (table: name | value | kind — where kind is `file`, `field`, or `header`) +- `I make concurrent "(POST)" requests to Gotenberg at the "" endpoint with the following form data and header(s):` (same table format) +- `I wait for the asynchronous request to the webhook` + +**Then (assertions):** + +- `the response status code should be ` +- `the (response|webhook request) header "" should be ""` +- `the (response|webhook request) cookie "" should be ""` +- `the (response|webhook request) body should match string:` (docstring) +- `the (response|webhook request) body should contain string:` (docstring) +- `the (response|webhook request) body should match JSON:` (docstring — use `"ignore"` for dynamic values like timestamps) +- `there should be PDF(s) in the (response|webhook request)` +- `there should be the following file(s) in the (response|webhook request):` (table of filenames) +- `the "" PDF should have page(s)` +- `the "" PDF (should|should NOT) be set to landscape orientation` +- `the "" PDF (should|should NOT) have the following content at page :` (docstring) +- `the (response|webhook request) PDF(s) should be valid "" with a tolerance of failed rule(s)` (standards: `PDF/A-1b`, `PDF/A-2b`, `PDF/A-3b`, `PDF/UA-1`, `PDF/UA-2`) +- `the (response|webhook request) PDF(s) (should|should NOT) be flatten` +- `the (response|webhook request) PDF(s) (should|should NOT) be encrypted` +- `the (response|webhook request) PDF(s) (should|should NOT) have the "" file embedded` +- `the Gotenberg container (should|should NOT) log the following entries:` (table of log substrings) +- `all concurrent response status codes should be ` +- `all concurrent responses should have PDF(s)` + +## Unit Tests + +- Use **table-driven tests** for pure logic in `pkg/`. +- Mock external dependencies using the comprehensive mocks in `pkg/gotenberg/mocks.go`. +- Run with `make test-unit`. diff --git a/AGENTS.md b/AGENTS.md index 294530dc..e8a45fbe 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,66 +1,31 @@ # Operational Guidelines for Gotenberg -As an AI agent working on the Gotenberg repository, you are expected to act with the diligence and architectural foresight of a Senior Go Engineer. Gotenberg is a widely used production dependency; stability and backward compatibility are paramount. +You are working on **Gotenberg**, a Docker-based API for converting documents to PDF. It is a widely used production dependency. Stability and backward compatibility are paramount. -## 1. Core Philosophy & Stability +## Core Principles -- **Backward Compatibility is Law:** This project creates a public API. Never modify existing flags, configuration environment variables, or API form fields unless explicitly instructed to perform a breaking change. If a change is breaking, it must be flagged immediately in the plan. -- **Defensive Programming:** Assume input data is malformed. Handle errors explicitly. Do not panic. -- **Atomic Commits:** Isolate refactoring from feature additions. A Pull Request should do one thing well. +- **Backward compatibility is law.** Never modify existing CLI flags, environment variables, or API form fields unless explicitly instructed to perform a breaking change. Flag any breaking change immediately. +- **Defensive programming.** Assume input is malformed. Handle errors explicitly. Never panic. +- **Atomic commits.** One feature or fix per PR. Isolate refactoring from feature work. +- **Idiomatic Go.** Follow "Effective Go" principles. All exported symbols must have GoDoc comments starting with their name. -## 2. Development Workflow & Tooling +## Project Layout -You must rely strictly on the project's Makefile for build and verification tasks. Do not run `go` commands directly unless debugging a specific package requires it. +``` +cmd/gotenberg/ → Entry point only (wiring/startup). No business logic. +pkg/gotenberg/ → Core module system, interfaces, utilities, mocks. +pkg/modules/ → Feature modules (api, chromium, libreoffice, pdfengines, etc.). +pkg/standard/ → Wires all standard modules together via imports. +test/integration/ → Gherkin feature files + Go test infrastructure. +build/ → Dockerfile, fonts, Chromium config. +``` -- **Formatting:** Run `make fmt` to format Go code before committing. -- **Linting:** - - Run `make lint` to ensure Go code strictly adheres to the `.golangci.yml` configuration. - - Run `make lint-prettier` to verify formatting for non-Go files (Markdown, YAML, etc.). - - Zero linting errors are permitted. -- **Building:** Run `make build` to verify compilation and Docker image construction. +Key interfaces live in `pkg/gotenberg/` — `Module`, `Provisioner`, `Validator`, `Debuggable`. Every module implements `Descriptor()` and self-registers. When adding features, determine if they belong in an existing module or require a new one. -## 3. Architecture & Code Structure +## Personas -- **Idiomatic Go:** Follow "Effective Go" principles. -- **Directory Separation:** - - `cmd/`: Application entry points only. Contains wiring and startup logic. **No business logic is permitted here.** - - `pkg/`: Core library code and modules. All business logic resides here. -- **Module System:** Gotenberg is modular (e.g., Chromium, LibreOffice). When adding features, determine if they belong to an existing module or require a new strict isolation. +Depending on the task at hand, load the relevant persona for additional context: -## 4. Testing Standards - -Gotenberg utilizes a split testing strategy. **Integration tests are the primary and preferred method for verifying features.** - -- **Integration Tests (`make test-integration`):** - - **First Priority:** Always start here when adding features or routes. - - Gotenberg uses **Gherkin (Godog)** for end-to-end verification. - - You **must** create or update the corresponding `.feature` file in `test/integration`. - - These tests run within the Docker context; ensure environment consistency. -- **Unit Tests (`make test-unit`):** - - Use table-driven tests for pure logic within `pkg/`. - - Mock external dependencies (filesystem, network) where appropriate. - -## 5. Documentation Requirements - -- **No README Updates:** Do not modify the root `README.md` unless explicitly asked. -- **GoDoc is Mandatory:** - - **New Packages:** If creating a new package, you must include a `doc.go` file containing the package-level documentation. - - **Exported Symbols:** Every exported function, type, constant, and variable must have a proper GoDoc comment starting with its name. - - **Quality:** Comments must be complete sentences explaining _what_ the symbol does and _how_ to use it. - - **Example:** - ```go - // Convert transforms the input document to PDF using the Chromium engine. - // It returns an error if the connection to the browser instance fails. - func Convert(...) error - ``` - -## 6. Definition of Done - -A task is considered complete only when: - -1. The code compiles via `make build`. -2. The code is formatted via `make fmt`. -3. All linters pass via `make lint` and `make lint-prettier`. -4. Integration scenarios pass via `make test-integration`. -5. Unit tests pass via `make test-unit`. -6. All exported symbols and new packages have compliant GoDoc. +- **[DEVELOPER](.agents/DEVELOPER.md)** — Writing code: architecture, module system, Makefile workflow, coding patterns. +- **[TESTER](.agents/TESTER.md)** — Writing tests: Gherkin/Godog integration tests, unit tests, available step definitions, selective test runs. +- **[REVIEWER](.agents/REVIEWER.md)** — Reviewing code: linting rules, backward compatibility checks, documentation compliance, Definition of Done. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..92adfd63 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,20 @@ +# Claude Code — Gotenberg + +Read [AGENTS.md](AGENTS.md) first. It contains the core principles, project layout, and links to specialized personas you must load depending on the task. + +## Quick Reference + +- Format before committing: `make fmt` (Go) and `make prettify` (non-Go) +- Lint before committing: `make lint && make lint-prettier` +- Commits must follow [Conventional Commits](https://www.conventionalcommits.org/) (e.g., `feat(chromium): add screenshot endpoint`) +- Run unit tests: `make test-unit` +- Run integration tests: `make build && make test-integration TAGS=` +- Never run `go` commands directly — use the Makefile. + +## Claude-Specific Guidance + +- When exploring the codebase, start with `pkg/gotenberg/` for core interfaces and `pkg/modules/` for feature implementations. +- The integration test infrastructure in `test/integration/scenario/` is well-structured — read `scenario.go` and `containers.go` to understand the Gherkin step definitions before writing new tests. +- Mocks for all major interfaces are in `pkg/gotenberg/mocks.go` — use them for unit tests rather than creating new ones. +- Import ordering is enforced: standard library, third-party, then `github.com/gotenberg/gotenberg/v8` — separated by blank lines. +- When making changes, run only the relevant integration test tag rather than the full suite (40min timeout). diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..86361b76 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,69 @@ +# Contributing to Gotenberg + +Thank you for your interest in contributing to Gotenberg! This guide will help you get started. + +## Before You Start + +Please read the [AGENTS.md](AGENTS.md) file — it describes the core principles, project layout, and development standards that all contributions must follow. Even though it is written for AI agents, the same rules apply to human contributors. + +For deeper context on specific areas, see the personas in `.agents/`: + +- **[DEVELOPER](.agents/DEVELOPER.md)** — Makefile workflow, module system, coding patterns. +- **[TESTER](.agents/TESTER.md)** — How to write integration and unit tests. +- **[REVIEWER](.agents/REVIEWER.md)** — What reviewers look for (useful to check before submitting). + +## Getting Started + +### Prerequisites + +- Go (see version in `go.mod`) +- Docker +- Node.js (see version in `.node-version`) — for Prettier linting +- [golangci-lint](https://golangci-lint.run/) v2+ + +### Build and Run + +```bash +make build # Build the Docker image +make run # Run a local Gotenberg container +``` + +### Development Loop + +```bash +# Write your code, then: +make fmt # Format Go code +make prettify # Format non-Go files (Markdown, YAML, etc.) +make lint # Lint Go code (zero errors permitted) +make lint-prettier # Lint non-Go files +make test-unit # Run unit tests +make build # Build the Docker image (required before integration tests) +make test-integration # Run all integration tests +``` + +To run only the integration tests relevant to your change: + +```bash +make test-integration TAGS=health +make test-integration TAGS=chromium-convert-html +make test-integration TAGS="merge,split" +``` + +## Submitting a Pull Request + +Before opening a PR, verify: + +1. Code compiles: `make build` +2. Code is formatted: `make fmt` and `make prettify` +3. All linters pass: `make lint` and `make lint-prettier` +4. Integration tests pass: `make test-integration` (at minimum, the relevant tags) +5. Unit tests pass: `make test-unit` +6. All exported symbols and new packages have GoDoc comments + +### Guidelines + +- **Conventional Commits.** Commit messages must follow the [Conventional Commits](https://www.conventionalcommits.org/) specification (e.g., `feat(chromium): add screenshot endpoint`, `fix(api): handle empty body`). +- **One thing per PR.** Keep features, bug fixes, and refactoring in separate PRs. +- **Backward compatibility matters.** Do not rename or remove existing CLI flags, environment variables, or API form fields without discussion. +- **Integration tests first.** When adding a feature or route, start by writing the Gherkin scenario in `test/integration/features/`. +- **No business logic in `cmd/`.** All logic belongs in `pkg/`. diff --git a/GEMINI.md b/GEMINI.md new file mode 100644 index 00000000..4c22c1dc --- /dev/null +++ b/GEMINI.md @@ -0,0 +1,21 @@ +# Gemini — Gotenberg + +Read [AGENTS.md](AGENTS.md) first. It contains the core principles, project layout, and links to specialized personas you must load depending on the task. + +## Quick Reference + +- Format before committing: `make fmt` (Go) and `make prettify` (non-Go) +- Lint before committing: `make lint && make lint-prettier` +- Commits must follow [Conventional Commits](https://www.conventionalcommits.org/) (e.g., `feat(chromium): add screenshot endpoint`) +- Run unit tests: `make test-unit` +- Run integration tests: `make build && make test-integration TAGS=` +- Never run `go` commands directly — use the Makefile. + +## Gemini-Specific Guidance + +- When exploring the codebase, start with `pkg/gotenberg/` for core interfaces and `pkg/modules/` for feature implementations. +- The integration test infrastructure in `test/integration/scenario/` is well-structured — read `scenario.go` and `containers.go` to understand the Gherkin step definitions before writing new tests. +- Mocks for all major interfaces are in `pkg/gotenberg/mocks.go` — use them for unit tests rather than creating new ones. +- Import ordering is enforced: standard library, third-party, then `github.com/gotenberg/gotenberg/v8` — separated by blank lines. +- When making changes, run only the relevant integration test tag rather than the full suite (40min timeout). +- This project values stability over velocity. When in doubt about whether a change is breaking, flag it rather than assuming it's safe.