-
-
-
A stateless API for converting Markdown files, HTML files and Office documents to PDF
- - ---- - -At TheCodingMachine, we build a lot of web applications (intranets, extranets and so on) which require to generate PDF from -various sources. Each time, we ended up using some well known libraries like **wkhtmltopdf** or **unoconv** and kind of lost time by -reimplementing a solution from a project to another project. Meh. - -# Menu - -* [Usage](#usage) -* [Security](#security) -* [Scalability](#scalability) -* [Custom implementation](#custom-implementation) -* [Clients](#clients) - -## Usage - -Let's say you're starting the API using this simple command: - -```sh -$ docker run --rm -p 3000:3000 thecodingmachine/gotenberg:{{ .Orbit.Latest }} -``` - -The API is now available on your host under `http://127.0.0.1:3000`. - -It accepts `POST` requests with a `multipart/form-data` Content-Type. Your form data should provide one or more files to convert. -It currently accepts the following: - -* Markdown files -* HTML files -* Office documents (.docx, .doc, .odt, .pptx, .ppt, .odp and so on) -* PDF files (if more than one file to convert) - -**Heads up:** the API relies on the file extension to determine which library to use for conversion. - -There are two use cases: - -* If you send one file, it will convert it and return the resulting PDF -* If many files, it will convert them to PDF, merge the resulting PDFs into a single PDF and return it - -### Examples: - -* One file - -```sh -$ curl --request POST \ - --url http://127.0.0.1:3000 \ - --header 'Content-Type: multipart/form-data' \ - --form files=@file.docx \ - > result.pdf -``` - -* Many files - -```sh -$ curl --request POST \ - --url http://127.0.0.1:3000 \ - --header 'Content-Type: multipart/form-data' \ - --form files=@file.md \ - --form files=@file.html \ - --form files=@file.pdf \ - --form files=@file.docx \ - > result.pdf -``` - -## Security - -The API does not provide any authentication mechanisms. Make sure to not put it on a public facing port and your client(s) should always -controls what is sent to the API. - -## Scalability - -Some libraries like **unoconv** cannot perform concurrent conversions. That's why the API does only one conversion at a time. -If your API is under heavy load, a request will take time to be processed. - -Fortunately, you may pass through this limitation by scaling the API. - -In the following example, I'll demonstrate how to do some vertical scaling (= on the same machine) with Docker Compose, but of course horizontal scaling works too! - -```yaml -version: '3' - -services: - - # your others services - - gotenberg: - image: gotenberg:1.0.0 -``` - -You may now launch your services using: - -```bash -docker-compose up --scale gotenberg=your_number_of_instances -``` - -When requesting the Gotenberg service with your client(s), Docker will automatically redirect a request to a Gotenberg container -according to the round-robin strategy. - -## Custom implementation - -The API relies on a simple YAML configuration file called `gotenberg.yml`. It allows you to tweak some values and even provides you -a way to change the commands called for each kind of conversion. The configuration file should be located under `/gotenberg` in your container. - -The default configuration is located here: [.ci/gotenberg.yml](.ci/gotenberg.yml) - -## Clients - -* https://github.com/thecodingmachine/gotenberg-php-client (PHP client) -* Add your own client by submitting a [pull request](../../pulls)! - ---- - -Would you like to update this documentation ? Feel free to open an [issue](../../issues). \ No newline at end of file diff --git a/.ci/gotenberg.yml b/.ci/gotenberg.yml index ba2422e5..eccdf32b 100644 --- a/.ci/gotenberg.yml +++ b/.ci/gotenberg.yml @@ -4,30 +4,46 @@ port: 3000 logs: # Accepted values, in order of severity: DEBUG, INFO, WARN, ERROR, FATAL, PANIC. # Messages at and above the selected level will be logged. - level: "INFO" + level: "DEBUG" # Accepted values: text, json. # When a TTY is not attached, the output will be in the defined format. - format: "text" + formatter: "text" -# You don't like a library which is used for a conversion? You may provide here your own implementation. +# You don't like a library which is used for a conversion? You want to handle a new file type? +# You may provide here your own implementation! commands: - markdown: - # Duration in seconds after which the command will be killed if it has not finished. - timeout: 30 - # The command template: you have access to FilePath and ResultFilePath variables. - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - - office: - timeout: 30 - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" - + # Unlike others commands' templates, you have access to FilesPaths instead of FilePath: it gathers all PDF files which should be merged. merge: + template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" timeout: 30 - # Unlike others commands' templates, you have access to FilesPaths instead of FilePath: it gathers all PDF files which should be merged. - template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" \ No newline at end of file + + conversions: + + # The command template: you have access to FilePath and ResultFilePath variables. + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + # Duration in seconds after which the command will be killed if it has not finished. + timeout: 30 + # Files with the following extensions will be converted by the current command. + extensions: + - ".md" + + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".html" + - ".htm" + + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 30 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" \ No newline at end of file diff --git a/README.md b/README.md index 0e806bc2..ab69f6ef 100644 --- a/README.md +++ b/README.md @@ -107,7 +107,7 @@ services: # your others services gotenberg: - image: thecodingmachine/gotenberg:1.0.0 + image: gotenberg:1.0.0 ``` You may now launch your services using: @@ -124,7 +124,7 @@ according to the round-robin strategy. The API relies on a simple YAML configuration file called `gotenberg.yml`. It allows you to tweak some values and even provides you a way to change the commands called for each kind of conversion. The configuration file should be located under `/gotenberg` in your container. -The default configuration is located here: [.ci/gotenberg.yml](.ci/gotenberg.yml) +The default configuration is located here: [.ci/gotenberg.yml](https://github.com/thecodingmachine/gotenberg/blob/1.0.0/.ci/gotenberg.yml) ## Clients @@ -133,4 +133,4 @@ The default configuration is located here: [.ci/gotenberg.yml](.ci/gotenberg.yml --- -Would you like to update this documentation ? Feel free to open an [issue](../../issues). +Would you like to update this documentation ? Feel free to open an [issue](../../issues). \ No newline at end of file diff --git a/_tests/configurations/broken-gotenberg.yml b/_tests/configurations/broken-gotenberg.yml index d593a215..77201b79 100644 --- a/_tests/configurations/broken-gotenberg.yml +++ b/_tests/configurations/broken-gotenberg.yml @@ -1,20 +1,34 @@ port: 3000 logs: level: "DEBUG" - format: { + formatter: { value: [ ... } commands: - markdown: - timeout: 30 - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - office: - timeout: 30 - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" merge: - timeout: 30 template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + timeout: 30 + conversions: + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".md" + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".html" + - ".htm" + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 30 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" + diff --git a/_tests/configurations/duplicate-command-gotenberg.yml b/_tests/configurations/duplicate-command-gotenberg.yml new file mode 100644 index 00000000..ed0f2a65 --- /dev/null +++ b/_tests/configurations/duplicate-command-gotenberg.yml @@ -0,0 +1,34 @@ +port: 3000 +logs: + level: "DEBUG" + formatter: "text" +commands: + merge: + template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + timeout: 30 + conversions: + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".md" + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".md" + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".html" + - ".htm" + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 30 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" diff --git a/_tests/configurations/gotenberg.yml b/_tests/configurations/gotenberg.yml index d0fe6da2..37d416ac 100644 --- a/_tests/configurations/gotenberg.yml +++ b/_tests/configurations/gotenberg.yml @@ -1,17 +1,30 @@ port: 3000 logs: level: "DEBUG" - format: "text" + formatter: "text" commands: - markdown: - timeout: 30 - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - office: - timeout: 30 - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" merge: - timeout: 30 template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + timeout: 30 + conversions: + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".md" + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".html" + - ".htm" + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 30 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" diff --git a/_tests/configurations/merge-timeout-gotenberg.yml b/_tests/configurations/merge-timeout-gotenberg.yml index 41fd7f9f..86204982 100644 --- a/_tests/configurations/merge-timeout-gotenberg.yml +++ b/_tests/configurations/merge-timeout-gotenberg.yml @@ -1,17 +1,30 @@ port: 3000 logs: level: "DEBUG" - format: "text" + formatter: "text" commands: - markdown: - timeout: 30 - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - office: - timeout: 30 - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" merge: - timeout: 0 template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + timeout: 0 + conversions: + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".md" + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".html" + - ".htm" + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 30 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" diff --git a/_tests/configurations/timeout-gotenberg.yml b/_tests/configurations/timeout-gotenberg.yml index ffd402e5..dfc7e81a 100644 --- a/_tests/configurations/timeout-gotenberg.yml +++ b/_tests/configurations/timeout-gotenberg.yml @@ -1,17 +1,30 @@ port: 3000 logs: level: "DEBUG" - format: "text" + formatter: "text" commands: - markdown: - timeout: 0 - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 0 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - office: - timeout: 0 - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" merge: - timeout: 0 template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + timeout: 0 + conversions: + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + timeout: 0 + extensions: + - ".md" + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 0 + extensions: + - ".html" + - ".htm" + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 0 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" diff --git a/_tests/configurations/wrong-command-template-gotenberg.yml b/_tests/configurations/wrong-command-template-gotenberg.yml new file mode 100644 index 00000000..383d4507 --- /dev/null +++ b/_tests/configurations/wrong-command-template-gotenberg.yml @@ -0,0 +1,30 @@ +port: 3000 +logs: + level: "DEBUG" + formatter: "text" +commands: + merge: + template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + timeout: 30 + conversions: + - template: "markdown-pdf {{ FilePath }} -o {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".md" + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".html" + - ".htm" + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 30 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" diff --git a/_tests/configurations/wrong-html-command-template-gotenberg.yml b/_tests/configurations/wrong-html-command-template-gotenberg.yml deleted file mode 100644 index 4c8ea014..00000000 --- a/_tests/configurations/wrong-html-command-template-gotenberg.yml +++ /dev/null @@ -1,17 +0,0 @@ -port: 3000 -logs: - level: "DEBUG" - format: "text" -commands: - markdown: - timeout: 30 - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ FilePath }} {{ .ResultFilePath }}" - office: - timeout: 30 - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" - merge: - timeout: 30 - template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" diff --git a/_tests/configurations/wrong-logging-format-gotenberg.yml b/_tests/configurations/wrong-logging-format-gotenberg.yml deleted file mode 100644 index 118135e2..00000000 --- a/_tests/configurations/wrong-logging-format-gotenberg.yml +++ /dev/null @@ -1,17 +0,0 @@ -port: 3000 -logs: - level: "DEBUG" - format: "DEBUG" -commands: - markdown: - timeout: 30 - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - office: - timeout: 30 - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" - merge: - timeout: 30 - template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" diff --git a/_tests/configurations/wrong-logging-formatter-gotenberg.yml b/_tests/configurations/wrong-logging-formatter-gotenberg.yml new file mode 100644 index 00000000..2a09f9d5 --- /dev/null +++ b/_tests/configurations/wrong-logging-formatter-gotenberg.yml @@ -0,0 +1,30 @@ +port: 3000 +logs: + level: "DEBUG" + formatter: "DEBUG" +commands: + merge: + template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + timeout: 30 + conversions: + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".md" + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".html" + - ".htm" + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 30 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" diff --git a/_tests/configurations/wrong-logging-level-gotenberg.yml b/_tests/configurations/wrong-logging-level-gotenberg.yml index acd3a0a3..1372cd53 100644 --- a/_tests/configurations/wrong-logging-level-gotenberg.yml +++ b/_tests/configurations/wrong-logging-level-gotenberg.yml @@ -1,17 +1,30 @@ port: 3000 logs: level: "text" - format: "text" + formatter: "text" commands: - markdown: - timeout: 30 - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - office: - timeout: 30 - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" merge: - timeout: 30 template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + timeout: 30 + conversions: + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".md" + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".html" + - ".htm" + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 30 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" diff --git a/_tests/configurations/wrong-markdown-command-template-gotenberg.yml b/_tests/configurations/wrong-markdown-command-template-gotenberg.yml deleted file mode 100644 index d9cd3647..00000000 --- a/_tests/configurations/wrong-markdown-command-template-gotenberg.yml +++ /dev/null @@ -1,17 +0,0 @@ -port: 3000 -logs: - level: "DEBUG" - format: "text" -commands: - markdown: - timeout: 30 - template: "markdown-pdf {{ FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - office: - timeout: 30 - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" - merge: - timeout: 30 - template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" diff --git a/_tests/configurations/wrong-merge-command-template-gotenberg.yml b/_tests/configurations/wrong-merge-command-template-gotenberg.yml index 64813c4e..987ab88f 100644 --- a/_tests/configurations/wrong-merge-command-template-gotenberg.yml +++ b/_tests/configurations/wrong-merge-command-template-gotenberg.yml @@ -1,17 +1,30 @@ port: 3000 logs: level: "DEBUG" - format: "text" + formatter: "text" commands: - markdown: - timeout: 30 - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - office: - timeout: 30 - template: "unoconv --format pdf --output {{ .ResultFilePath }}\" \"{{ .FilePath }}\"" merge: + template: "pdftk {{ range $filePath := FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" timeout: 30 - template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} cat output {{ .ResultFilePath }}" + conversions: + - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".md" + - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" + timeout: 30 + extensions: + - ".html" + - ".htm" + - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\"" + timeout: 30 + extensions: + - ".doc" + - ".docx" + - ".odt" + - ".xls" + - ".xlsx" + - ".ods" + - ".ppt" + - ".pptx" + - ".odp" diff --git a/_tests/configurations/wrong-office-command-template-gotenberg.yml b/_tests/configurations/wrong-office-command-template-gotenberg.yml deleted file mode 100644 index 3738d05d..00000000 --- a/_tests/configurations/wrong-office-command-template-gotenberg.yml +++ /dev/null @@ -1,17 +0,0 @@ -port: 3000 -logs: - level: "DEBUG" - format: "text" -commands: - markdown: - timeout: 30 - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}" - html: - timeout: 30 - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}" - office: - timeout: 30 - template: "unoconv --format pdf --output \"{{ ResultFilePath }}\" \"{{ .FilePath }}\"" - merge: - timeout: 30 - template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" diff --git a/app/config/config.go b/app/config/config.go index bff203ac..c82455ba 100644 --- a/app/config/config.go +++ b/app/config/config.go @@ -7,139 +7,50 @@ It should be located where the user starts the application from the CLI. package config import ( - "io/ioutil" + "fmt" "text/template" "github.com/sirupsen/logrus" - "gopkg.in/yaml.v2" ) type ( - // AppConfig gathers all data required to instantiate the application. - AppConfig struct { - // Port is the port which the application will listen to. - Port string - // Logs contains the logging configuration. - Logs struct { - // Level is the level of messages which will be logged. - Level logrus.Level - // Formatter defines the logging format when a TTY is not attached. - Formatter logrus.Formatter - } - // CommandsConfig is... an instance of CommandsConfig. - CommandsConfig *CommandsConfig + // appConfig gathers all configuration data. + appConfig struct { + port string + logsLevel logrus.Level + logsFormatter logrus.Formatter + // commands associates a file extension with a Command instance. + // Particular case: ".pdf" extension is used for the merge command. + commands map[string]*Command } - // CommandsConfig gathers all commands' configurations as defined - // by the user in the gotenberg.yml file. - CommandsConfig struct { - // Markdown is the command's configuration for converting - // an Markdown file to PDF. - Markdown *CommandConfig - // HTML is the command's configuration for converting - // an HTML file to PDF. - HTML *CommandConfig - // Office is the command's configuration for converting - // an Office document to PDF. - Office *CommandConfig - // Merge is the command's configuration for merging - // multiple PDF files into one PDF file. - Merge *CommandConfig - } - - // CommandConfig is a command's configuration. - CommandConfig struct { + // Command gathers information on how to launch an external binary used for converting + // a file to PDF. + Command struct { + // Template is the data-driven template of the command. + Template *template.Template // Timeout is the duration in seconds after which the command's process will be killed // if it does not finish before. Timeout int - // Template is the data-driven template of the command. - Template *template.Template } ) -// NewAppConfig instantiates the application's configuration. -// If something bad happens here, the application should not start. -func NewAppConfig(configurationFilePath string) (*AppConfig, error) { - fileConfig, err := loadFileConfig(configurationFilePath) - if err != nil { - return nil, err - } +// our default instance of appConfig. +var config = &appConfig{} - c := &AppConfig{} - c.Port = fileConfig.Port - - if err := makeLogs(c, fileConfig); err != nil { - return nil, err - } - - if err := makeCommandsConfig(c, fileConfig); err != nil { - return nil, err - } - - return c, nil +// Reset reinitializes our configuration. +func Reset() { + config = &appConfig{} } -// fileConfig gathers all data coming from the configuration file gotenberg.yml. -type fileConfig struct { - Port string `yaml:"port"` - Logs struct { - Level string `yaml:"level"` - Format string `yaml:"format"` - } `yaml:"logs"` - Commands struct { - Markdown struct { - Timeout int `yaml:"timeout"` - Template string `yaml:"template"` - } `yaml:"markdown"` - HTML struct { - Timeout int - Template string - } `yaml:"html"` - Office struct { - Timeout int `yaml:"timeout"` - Template string `yaml:"template"` - } `yaml:"office"` - Merge struct { - Timeout int `yaml:"timeout"` - Template string `yaml:"template"` - } `yaml:"merge"` - } `yaml:"commands"` +// WithPort sets the port which will be used by the application. +func WithPort(port string) { + config.port = port } -// loadFileConfig instantiates a fileConfig instance by loading -// the configuration file gotenberg.yml. -func loadFileConfig(configurationFilePath string) (*fileConfig, error) { - c := &fileConfig{} - - data, err := ioutil.ReadFile(configurationFilePath) - if err != nil { - return nil, err - } - - if err := yaml.Unmarshal(data, &c); err != nil { - return nil, err - } - - return c, nil -} - -// makeLogs is a simple wrapper which populates all data related -// to application's logging. -func makeLogs(appConfig *AppConfig, fileConfig *fileConfig) error { - lvl, err := getLoggingLevelFromFileConfig(fileConfig) - if err != nil { - return err - } - - formatter, err := getLoggingFormatterFromFileConfig(fileConfig) - if err != nil { - return err - } - - appConfig.Logs.Level = lvl - appConfig.Logs.Formatter = formatter - - return nil +// GetPort returns the current port. +func GetPort() string { + return config.port } // levels associates logging levels as defined in the configuration file gotenberg.yml @@ -153,102 +64,122 @@ var levels = map[string]logrus.Level{ "PANIC": logrus.PanicLevel, } -type wrongLoggingLevelError struct{} +type wrongLogsLevelError struct{} -const wrongLoggingLevelErrorMessage = "Accepted values for logging level: DEBUG, INFO, WARN, ERROR, FATAL, PANIC" +const wrongLogsLevelErrorMessage = "accepted values for logs level: DEBUG, INFO, WARN, ERROR, FATAL, PANIC" -func (e *wrongLoggingLevelError) Error() string { - return wrongLoggingLevelErrorMessage +func (e *wrongLogsLevelError) Error() string { + return wrongLogsLevelErrorMessage } -// getLoggingLevelFromFileConfig returns a logrus level if a matching was found -// with the one defined by the user. -// If no match, throws an error. -func getLoggingLevelFromFileConfig(c *fileConfig) (logrus.Level, error) { - l, ok := levels[c.Logs.Level] +// WithLogsLevel sets the logs level. +// If the given string does not match with a logrus level, +// throws an error. +func WithLogsLevel(level string) error { + l, ok := levels[level] if !ok { - return 999, &wrongLoggingLevelError{} + return &wrongLogsLevelError{} } - return l, nil + config.logsLevel = l + return nil } -// levels associates logging formats as defined in the configuration file gotenberg.yml +// GetLogsLevel returns the current logs level. +func GetLogsLevel() logrus.Level { + return config.logsLevel +} + +// formatters associates logging formatter as defined in the configuration file gotenberg.yml // with its counterpart from the logrus library. var formatters = map[string]logrus.Formatter{ "text": &logrus.TextFormatter{}, "json": &logrus.JSONFormatter{}, } -type wrongLoggingFormatError struct{} +type wrongLogsFormatterError struct{} -const wrongLoggingFormatErrorMessage = "Accepted value for logging format: text, json" +const wrongLogsFormatterErrorMessage = "accepted value for logs formatter: text, json" -func (e *wrongLoggingFormatError) Error() string { - return wrongLoggingFormatErrorMessage +func (e *wrongLogsFormatterError) Error() string { + return wrongLogsFormatterErrorMessage } -// getLoggingLevelFromFileConfig returns a logrus Formatter if a matching was found -// with the format defined by the user. -// If no match, throws an error. -func getLoggingFormatterFromFileConfig(c *fileConfig) (logrus.Formatter, error) { - f, ok := formatters[c.Logs.Format] +// WithLogsFormatter sets the logs formatter. +// If the given string does not match with a logrus formatter, +// throws an error. +func WithLogsFormatter(formatter string) error { + f, ok := formatters[formatter] if !ok { - return nil, &wrongLoggingFormatError{} + return &wrongLogsFormatterError{} } - return f, nil -} - -// makeCommandsConfigs is a simple wrapper which populates all data related -// to commands' configurations. -func makeCommandsConfig(appConfig *AppConfig, fileConfig *fileConfig) error { - appConfig.CommandsConfig = &CommandsConfig{} - appConfig.CommandsConfig.Markdown = &CommandConfig{} - appConfig.CommandsConfig.HTML = &CommandConfig{} - appConfig.CommandsConfig.Office = &CommandConfig{} - appConfig.CommandsConfig.Merge = &CommandConfig{} - - appConfig.CommandsConfig.Markdown.Timeout = fileConfig.Commands.Markdown.Timeout - appConfig.CommandsConfig.HTML.Timeout = fileConfig.Commands.HTML.Timeout - appConfig.CommandsConfig.Office.Timeout = fileConfig.Commands.Office.Timeout - appConfig.CommandsConfig.Merge.Timeout = fileConfig.Commands.Merge.Timeout - - tmplMarkdown, err := getCommandTemplate(fileConfig.Commands.Markdown.Template, "Markdown") - if err != nil { - return err - } - - tmplHTML, err := getCommandTemplate(fileConfig.Commands.HTML.Template, "HTML") - if err != nil { - return err - } - - tmplOffice, err := getCommandTemplate(fileConfig.Commands.Office.Template, "Office") - if err != nil { - return err - } - - tmplMerge, err := getCommandTemplate(fileConfig.Commands.Merge.Template, "Merge") - if err != nil { - return err - } - - appConfig.CommandsConfig.Markdown.Template = tmplMarkdown - appConfig.CommandsConfig.HTML.Template = tmplHTML - appConfig.CommandsConfig.Office.Template = tmplOffice - appConfig.CommandsConfig.Merge.Template = tmplMerge - + config.logsFormatter = f return nil } -// getCommandTemplate is a simple helper for parsing a command template as defined by the user. -// If the user gives us a wrong template, throws an error. -func getCommandTemplate(command string, commandName string) (*template.Template, error) { - t, err := template.New(commandName).Parse(command) +// GetLogsFormatter returns the current logs formatter. +func GetLogsFormatter() logrus.Formatter { + return config.logsFormatter +} + +// NewCommand instantiates a Command. If the given command string +// is not a valid template, throws an error. +func NewCommand(command string, timeout int) (*Command, error) { + t, err := template.New(command).Parse(command) if err != nil { return nil, err } - return t, nil + return &Command{t, timeout}, nil +} + +type fileExtensionAlreadyUsedError struct { + extension string + command *Command + existingCommand *Command +} + +const fileExtensionAlreadyUsedErrorMessage = "file extension '%s' from command '%s' is already used by command '%s'" + +func (e *fileExtensionAlreadyUsedError) Error() string { + return fmt.Sprintf(fileExtensionAlreadyUsedErrorMessage, e.extension, e.command.Template.Name(), e.existingCommand.Template.Name()) +} + +// WithCommand adds a Command instance and associates it with the given +// file extension. If the file extension is already used by another Command +// instance, throws an error. +func WithCommand(extension string, command *Command) error { + if config.commands == nil { + config.commands = make(map[string]*Command) + } + + existingCommand, ok := config.commands[extension] + if ok { + return &fileExtensionAlreadyUsedError{extension, command, existingCommand} + } + + config.commands[extension] = command + return nil +} + +type noCommandFoundForFileExtensionError struct { + extension string +} + +const noCommandFoundForFileExtensionErrorMessage = "no command found for file extension '%s'" + +func (e *noCommandFoundForFileExtensionError) Error() string { + return fmt.Sprintf(noCommandFoundForFileExtensionErrorMessage, e.extension) +} + +// GetCommand returns the Command instance associated with the given +// file extension. If no Command instance found, throws an error. +func GetCommand(extension string) (*Command, error) { + c, ok := config.commands[extension] + if !ok { + return nil, &noCommandFoundForFileExtensionError{extension} + } + + return c, nil } diff --git a/app/config/config_test.go b/app/config/config_test.go index a30df9b1..24e822cc 100644 --- a/app/config/config_test.go +++ b/app/config/config_test.go @@ -1,77 +1,171 @@ package config import ( - "path/filepath" + "fmt" "testing" + + "github.com/sirupsen/logrus" ) -func TestNewAppConfig(t *testing.T) { - var path string +func TestReset(t *testing.T) { + c := &appConfig{} + config.port = "3000" + Reset() - // case 1: uses an empty configuration file path. - if _, err := NewAppConfig(""); err == nil { - t.Error("AppConfig should not have been instantiated by using an empty configuration file path") - } - - // case 2: uses a broken configuration file. - path, _ = filepath.Abs("../../_tests/configurations/broken-gotenberg.yml") - if _, err := NewAppConfig(path); err == nil { - t.Errorf("AppConfig should not have been instantiated with '%s'", path) - } - - // case 3: uses a configuration file with a wrong logging level. - path, _ = filepath.Abs("../../_tests/configurations/wrong-logging-level-gotenberg.yml") - if _, err := NewAppConfig(path); err == nil { - t.Errorf("AppConfig should not have been instantiated with '%s'", path) - } - - // case 4: uses a configuration file with a wrong logging format. - path, _ = filepath.Abs("../../_tests/configurations/wrong-logging-format-gotenberg.yml") - if _, err := NewAppConfig(path); err == nil { - t.Errorf("AppConfig should not have been instantiated with '%s'", path) - } - - // case 5: uses a configuration file with a wrong markdown command template. - path, _ = filepath.Abs("../../_tests/configurations/wrong-markdown-command-template-gotenberg.yml") - if _, err := NewAppConfig(path); err == nil { - t.Errorf("AppConfig should not have been instantiated with '%s'", path) - } - - // case 6: uses a configuration file with a wrong HTML command template. - path, _ = filepath.Abs("../../_tests/configurations/wrong-html-command-template-gotenberg.yml") - if _, err := NewAppConfig(path); err == nil { - t.Errorf("AppConfig should not have been instantiated with '%s'", path) - } - - // case 7: uses a configuration file with a wrong Office command template. - path, _ = filepath.Abs("../../_tests/configurations/wrong-office-command-template-gotenberg.yml") - if _, err := NewAppConfig(path); err == nil { - t.Errorf("AppConfig should not have been instantiated with '%s'", path) - } - - // case 8: uses a configuration file with a wrong merge command template. - path, _ = filepath.Abs("../../_tests/configurations/wrong-merge-command-template-gotenberg.yml") - if _, err := NewAppConfig(path); err == nil { - t.Errorf("AppConfig should not have been instantiated with '%s'", path) - } - - // case 9: uses a correct configuration file. - path, _ = filepath.Abs("../../_tests/configurations/gotenberg.yml") - if _, err := NewAppConfig(path); err != nil { - t.Errorf("AppConfig should have been instantiated with '%s'", path) + if c.port != config.port { + t.Error("Configuration should have been reset") } } -func TestWrongLoggingLevelError(t *testing.T) { - err := &wrongLoggingLevelError{} - if err.Error() != wrongLoggingLevelErrorMessage { - t.Errorf("Error returned a wrong message: got '%s' want '%s'", err.Error(), wrongLoggingLevelErrorMessage) +func TestWithPort(t *testing.T) { + port := "3000" + WithPort(port) + + if config.port != port { + t.Errorf("Configuration populated with a wrong port: got '%s' want '%s'", config.port, port) } } -func TestWrongLoggingFormatError(t *testing.T) { - err := &wrongLoggingFormatError{} - if err.Error() != wrongLoggingFormatErrorMessage { - t.Errorf("Error returned a wrong message: got '%s' want '%s'", err.Error(), wrongLoggingFormatErrorMessage) +func TestGetPort(t *testing.T) { + port := "3000" + config.port = port + + if GetPort() != port { + t.Errorf("Configuration returned a wrong port: got '%s' want '%s'", GetPort(), port) + } +} + +func TestWrongLogsLevelError(t *testing.T) { + err := &wrongLogsLevelError{} + if err.Error() != wrongLogsLevelErrorMessage { + t.Errorf("Error returned a wrong message: got '%s' want '%s'", err.Error(), wrongLogsLevelErrorMessage) + } +} + +func TestWithLogsLevel(t *testing.T) { + var lvl string + + // case 1: uses a wrong logs level. + lvl = "text" + if err := WithLogsLevel(lvl); err == nil { + t.Errorf("Configuration should not have been populated by using '%s' as logs level", lvl) + } + + // case 2: uses a correct logs level. + lvl = "DEBUG" + if err := WithLogsLevel(lvl); err != nil { + t.Errorf("Configuration should have been populated by using '%s' as logs level", lvl) + } +} + +func TestGetLogsLevel(t *testing.T) { + lvl := logrus.DebugLevel + config.logsLevel = lvl + + if GetLogsLevel() != lvl { + t.Errorf("Configuration returned a wrong logs level: got '%s' want '%s'", GetLogsLevel(), lvl) + } +} + +func TestWrongLogsFormatterError(t *testing.T) { + err := &wrongLogsFormatterError{} + if err.Error() != wrongLogsFormatterErrorMessage { + t.Errorf("Error returned a wrong message: got '%s' want '%s'", err.Error(), wrongLogsFormatterErrorMessage) + } +} + +func TestWithLogsFormatter(t *testing.T) { + var formatter string + + // case 1: uses a wrong logs formatter. + formatter = "DEBUG" + if err := WithLogsFormatter(formatter); err == nil { + t.Errorf("Configuration should not have been populated by using '%s' as logs formatter", formatter) + } + + // case 2: uses a correct logs formatter. + formatter = "text" + if err := WithLogsFormatter(formatter); err != nil { + t.Errorf("Configuration should have been populated by using '%s' as logs formatter", formatter) + } +} + +func TestGetLogsFormatter(t *testing.T) { + formatter := &logrus.TextFormatter{} + config.logsFormatter = formatter + + if GetLogsFormatter() != formatter { + t.Errorf("Configuration returned a wrong logs formatter: got '%v' want '%v'", GetLogsFormatter(), formatter) + } +} + +func TestNewCommand(t *testing.T) { + var cmd string + + // case 1: uses a wrong command template. + cmd = "pdftk {{ range $filePath := FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + if _, err := NewCommand(cmd, 0); err == nil { + t.Errorf("Command should not have been instantiated by using '%s' as command template", cmd) + } + + // case 2: uses a correct command template. + cmd = "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}" + if _, err := NewCommand(cmd, 0); err != nil { + t.Errorf("Command should have been instantiated by using '%s' as command template", cmd) + } +} + +func TestFileExtensionAlreadyUsedError(t *testing.T) { + ext := ".pdf" + cmd1, _ := NewCommand("echo", 0) + cmd2, _ := NewCommand("echo", 0) + expected := fmt.Sprintf(fileExtensionAlreadyUsedErrorMessage, ext, cmd1.Template.Name(), cmd2.Template.Name()) + + err := &fileExtensionAlreadyUsedError{ext, cmd1, cmd2} + if err.Error() != expected { + t.Errorf("Error returned a wrong message: got '%s' want '%s'", err.Error(), expected) + } +} + +func TestWithCommand(t *testing.T) { + ext := ".pdf" + cmd, _ := NewCommand("echo", 0) + + // case 1: uses a command with a file extension not already referenced. + if err := WithCommand(ext, cmd); err != nil { + t.Errorf("Configuration should have been populated by using a command with the file extension '%s'", ext) + } + + // case 2: uses a command with a file extension already referenced. + if err := WithCommand(ext, cmd); err == nil { + t.Errorf("Configuration should not have been populated by using a command with the file extension '%s'", ext) + } +} + +func TestNoCommandFoundForFileExtensionError(t *testing.T) { + ext := ".pdf" + expected := fmt.Sprintf(noCommandFoundForFileExtensionErrorMessage, ext) + + err := &noCommandFoundForFileExtensionError{ext} + if err.Error() != expected { + t.Errorf("Error returned a wrong message: got '%s' want '%s'", err.Error(), expected) + } +} + +func TestGetCommand(t *testing.T) { + Reset() + ext := ".pdf" + cmd, _ := NewCommand("echo", 0) + WithCommand(ext, cmd) + + // case 1: uses a file extension which has a command associated. + if _, err := GetCommand(ext); err != nil { + t.Errorf("Configuration should have been able to return a command by using the file extension '%s'", ext) + } + + // case 2: uses a file extension which has no command associated. + ext = ".docx" + if _, err := GetCommand(ext); err == nil { + t.Errorf("Configuration should not have been able to return a command by using the file extension '%s'", ext) } } diff --git a/app/config/parser.go b/app/config/parser.go new file mode 100644 index 00000000..82089a41 --- /dev/null +++ b/app/config/parser.go @@ -0,0 +1,94 @@ +package config + +import ( + "io/ioutil" + + "gopkg.in/yaml.v2" +) + +// ParseFile instantiates the application's configuration using the given YAML file. +func ParseFile(configurationFilePath string) error { + fileConfig, err := readFile(configurationFilePath) + if err != nil { + return err + } + + WithPort(fileConfig.Port) + + if err := WithLogsLevel(fileConfig.Logs.Level); err != nil { + return err + } + + if err := WithLogsFormatter(fileConfig.Logs.Formatter); err != nil { + return err + } + + // handles merge command first... + cmd, err := NewCommand(fileConfig.Commands.Merge.Template, fileConfig.Commands.Merge.Timeout) + if err != nil { + return err + } + + WithCommand(".pdf", cmd) + + // ...then conversion commands! + for _, command := range fileConfig.Commands.Conversions { + cmd, err := NewCommand(command.Template, command.Timeout) + if err != nil { + return err + } + + for _, ext := range command.Extensions { + if err := WithCommand(ext, cmd); err != nil { + return err + } + } + } + + return nil +} + +type ( + // fileConfig gathers all data coming from the configuration file gotenberg.yml. + fileConfig struct { + Port string `yaml:"port"` + Logs struct { + Level string `yaml:"level"` + Formatter string `yaml:"formatter"` + } `yaml:"logs"` + Commands struct { + Merge *mergeCommand `yaml:"merge"` + Conversions []*conversionCommand `yaml:"conversions,omitempty"` + } `yaml:"commands"` + } + + // mergeCommand gathers all data regarding the... merge command. + mergeCommand struct { + Template string `yaml:"template"` + Timeout int `yaml:"timeout"` + } + + // conversionCommand gathers all data regarding a conversion command. + conversionCommand struct { + Template string `yaml:"template"` + Timeout int `yaml:"timeout"` + Extensions []string `yaml:"extensions"` + } +) + +// readFile instantiates a fileConfig instance by reading +// the given YAML file. +func readFile(configurationFilePath string) (*fileConfig, error) { + c := &fileConfig{} + + data, err := ioutil.ReadFile(configurationFilePath) + if err != nil { + return nil, err + } + + if err := yaml.Unmarshal(data, &c); err != nil { + return nil, err + } + + return c, nil +} diff --git a/app/config/parser_test.go b/app/config/parser_test.go new file mode 100644 index 00000000..ebebbb14 --- /dev/null +++ b/app/config/parser_test.go @@ -0,0 +1,62 @@ +package config + +import ( + "path/filepath" + "testing" +) + +func load(configurationFilePath string) error { + Reset() + return ParseFile(configurationFilePath) +} + +func TestParseFile(t *testing.T) { + var path string + + // case 1: uses an empty configuration file path. + if err := load(""); err == nil { + t.Error("Configuration should not have been populated by using an empty configuration file path") + } + + // case 2: uses a broken configuration file. + path, _ = filepath.Abs("../../_tests/configurations/broken-gotenberg.yml") + if err := load(path); err == nil { + t.Errorf("Configuration should not have been populated with '%s'", path) + } + + // case 3: uses a configuration file with a wrong logging level. + path, _ = filepath.Abs("../../_tests/configurations/wrong-logging-level-gotenberg.yml") + if err := load(path); err == nil { + t.Errorf("Configuration should not have been populated with '%s'", path) + } + + // case 4: uses a configuration file with a wrong logging formatter. + path, _ = filepath.Abs("../../_tests/configurations/wrong-logging-formatter-gotenberg.yml") + if err := load(path); err == nil { + t.Errorf("Configuration should not have been populated with '%s'", path) + } + + // case 5: uses a configuration file with a wrong merge command template. + path, _ = filepath.Abs("../../_tests/configurations/wrong-merge-command-template-gotenberg.yml") + if err := load(path); err == nil { + t.Errorf("Configuration should not have been populated with '%s'", path) + } + + // case 6: uses a configuration file with a wrong command template. + path, _ = filepath.Abs("../../_tests/configurations/wrong-command-template-gotenberg.yml") + if err := load(path); err == nil { + t.Errorf("Configuration should not have been populated with '%s'", path) + } + + // case 7: uses a configuration file with a duplicate command. + path, _ = filepath.Abs("../../_tests/configurations/duplicate-command-gotenberg.yml") + if err := load(path); err == nil { + t.Errorf("Configuration should not have been populated with '%s'", path) + } + + // case 8: uses a correct configuration file. + path, _ = filepath.Abs("../../_tests/configurations/gotenberg.yml") + if err := load(path); err != nil { + t.Errorf("Configuration should have been populated with '%s'", path) + } +} diff --git a/app/converter/converter.go b/app/converter/converter.go index 232229c9..90b23f86 100644 --- a/app/converter/converter.go +++ b/app/converter/converter.go @@ -75,7 +75,7 @@ func NewConverter(r *http.Request) (*Converter, error) { func (c *Converter) Convert() (string, error) { var filesPaths []string for _, f := range c.files { - if f.Type != gfile.PDFType { + if f.Extension != ".pdf" { path, err := process.Unconv(c.workingDir, f) if err != nil { return "", err diff --git a/app/converter/converter_test.go b/app/converter/converter_test.go index 719a6d13..5edf63e0 100644 --- a/app/converter/converter_test.go +++ b/app/converter/converter_test.go @@ -10,7 +10,6 @@ import ( "testing" "github.com/thecodingmachine/gotenberg/app/config" - "github.com/thecodingmachine/gotenberg/app/converter/process" ) func makeRequest(filesPaths ...string) *http.Request { @@ -43,10 +42,10 @@ func makeRequest(filesPaths ...string) *http.Request { return req } -func loadCommandConfigs(configurationFilePath string) { +func load(configurationFilePath string) { + config.Reset() path, _ := filepath.Abs(configurationFilePath) - c, _ := config.NewAppConfig(path) - process.Load(c.CommandsConfig) + config.ParseFile(path) } func TestNewConverter(t *testing.T) { @@ -55,6 +54,8 @@ func TestNewConverter(t *testing.T) { oPath string ) + load("../../_tests/configurations/gotenberg.yml") + // case 1: uses a request with a single file. path, _ = filepath.Abs("../../_tests/file.docx") if _, err := NewConverter(makeRequest(path)); err != nil { @@ -94,7 +95,7 @@ func TestConvert(t *testing.T) { c *Converter ) - loadCommandConfigs("../../_tests/configurations/gotenberg.yml") + load("../../_tests/configurations/gotenberg.yml") // case 1: uses a request with a single file. path, _ = filepath.Abs("../../_tests/file.docx") @@ -111,7 +112,7 @@ func TestConvert(t *testing.T) { t.Errorf("Converter should have been able to convert '%s' and '%s' to PDF", path, oPath) } - loadCommandConfigs("../../_tests/configurations/timeout-gotenberg.yml") + load("../../_tests/configurations/timeout-gotenberg.yml") // case 3: uses a request with a single file and a configuration with an unsuitable timeout for the conversion commands. path, _ = filepath.Abs("../../_tests/file.docx") @@ -120,7 +121,7 @@ func TestConvert(t *testing.T) { t.Errorf("Converter should not have been able to convert '%s' to PDF", path) } - loadCommandConfigs("../../_tests/configurations/merge-timeout-gotenberg.yml") + load("../../_tests/configurations/merge-timeout-gotenberg.yml") // case 4: uses a request with two files and a configuration with an unsuitable timeout for the merge command. path, _ = filepath.Abs("../../_tests/file.pdf") @@ -132,6 +133,8 @@ func TestConvert(t *testing.T) { } func TestClear(t *testing.T) { + load("../../_tests/configurations/gotenberg.yml") + path, _ := filepath.Abs("../../_tests/file.docx") c, _ := NewConverter(makeRequest(path)) if err := c.Clear(); err != nil { diff --git a/app/converter/file/file.go b/app/converter/file/file.go index 2fae7b42..089c50ef 100644 --- a/app/converter/file/file.go +++ b/app/converter/file/file.go @@ -7,70 +7,32 @@ import ( "os" "path/filepath" + "github.com/thecodingmachine/gotenberg/app/config" + "github.com/satori/go.uuid" ) // File represents a file which has been created // from a request. type File struct { - // Type is the kind of file. - Type Type + // Extension is the extension of the file. + Extension string // Path is the file path. Path string } -// Type represents what kind of file we're dealing with. -type Type uint32 - -const ( - // PDFType represents a... PDF file. - PDFType Type = iota - // MarkdownType represents a... Markdown file. - MarkdownType - // HTMLType represents an... HTML file. - HTMLType - // OfficeType represents an... Office document. - OfficeType -) - -// filesTypes associates a file extension with its file kind counterpart. -var filesTypes = map[string]Type{ - ".pdf": PDFType, - ".md": MarkdownType, - ".htm": HTMLType, - ".html": HTMLType, - ".doc": OfficeType, - ".docx": OfficeType, - ".odt": OfficeType, - ".xls": OfficeType, - ".xlsx": OfficeType, - ".ods": OfficeType, - ".ppt": OfficeType, - ".pptx": OfficeType, - ".odp": OfficeType, -} - -type fileTypeNotFoundError struct { - fileName string -} - -func (e *fileTypeNotFoundError) Error() string { - return fmt.Sprintf("File type was not found for '%s'", e.fileName) -} - // NewFile creates a file in the considered directory. // Returns a *File instance or an error if something bad happened. func NewFile(workingDir string, r io.Reader, fileName string) (*File, error) { ext := filepath.Ext(fileName) - t, ok := filesTypes[ext] - if !ok { - return nil, &fileTypeNotFoundError{fileName: fileName} + if _, err := config.GetCommand(ext); err != nil { + return nil, err } f := &File{ - Path: MakeFilePath(workingDir, ext), - Type: t, + Extension: ext, + Path: MakeFilePath(workingDir, ext), } file, err := os.Create(f.Path) diff --git a/app/converter/file/file_test.go b/app/converter/file/file_test.go index 5323c4e2..b1140dda 100644 --- a/app/converter/file/file_test.go +++ b/app/converter/file/file_test.go @@ -2,13 +2,22 @@ package file import ( "bytes" - "fmt" "os" "path/filepath" "testing" + + "github.com/thecodingmachine/gotenberg/app/config" ) +func load(configurationFilePath string) { + config.Reset() + path, _ := filepath.Abs(configurationFilePath) + config.ParseFile(path) +} + func TestNewFile(t *testing.T) { + load("../../../_tests/configurations/gotenberg.yml") + workingDir := "test" os.Mkdir(workingDir, 0666) @@ -17,21 +26,13 @@ func TestNewFile(t *testing.T) { t.Error("File should not have been instantiated with an empty buffer") } - // case 2: uses a reader from a correct file type. + // case 2: uses a file name. filePath, _ := filepath.Abs("../../../_tests/file.pdf") r, _ := os.Open(filePath) defer r.Close() if _, err := NewFile(workingDir, r, "file.pdf"); err != nil { - t.Errorf("File should have been instantiated using a reader from '%s'", filePath) + t.Errorf("File should have been instantiated using a reader of '%s'", filePath) } os.RemoveAll(workingDir) } -func TestFileTypeNotFoundError(t *testing.T) { - fileName := "file.wp" - err := &fileTypeNotFoundError{fileName: fileName} - expected := fmt.Sprintf("File type was not found for '%s'", fileName) - if err.Error() != expected { - t.Errorf("Error returned a wrong message: got '%s' want '%s'", err.Error(), expected) - } -} diff --git a/app/converter/process/process.go b/app/converter/process/process.go index 4507ed41..a0da53ad 100644 --- a/app/converter/process/process.go +++ b/app/converter/process/process.go @@ -6,7 +6,6 @@ import ( "fmt" "os/exec" "sync" - "text/template" "time" "github.com/thecodingmachine/gotenberg/app/config" @@ -14,8 +13,7 @@ import ( ) type runner struct { - mu sync.Mutex - commandsConfig *config.CommandsConfig + mu sync.Mutex } var forest = &runner{} @@ -64,25 +62,12 @@ func (r *runner) run(command string, timeout int) error { } } -// Load loads the commands configuration coming from the application configuration. -func Load(config *config.CommandsConfig) { - forest.commandsConfig = config -} - // conversionData will be applied to the data-driven templates of conversions commands. type conversionData struct { FilePath string ResultFilePath string } -type impossibleConversionError struct{} - -const impossibleConversionErrorMessage = "Impossible conversion" - -func (e *impossibleConversionError) Error() string { - return impossibleConversionErrorMessage -} - // Unconv converts a file to PDF and returns the new file path. func Unconv(workingDir string, file *gfile.File) (string, error) { cmdData := &conversionData{ @@ -90,34 +75,17 @@ func Unconv(workingDir string, file *gfile.File) (string, error) { ResultFilePath: gfile.MakeFilePath(workingDir, ".pdf"), } - var ( - cmdTimeout int - cmdTemplate *template.Template - ) - - switch file.Type { - case gfile.MarkdownType: - cmdTimeout = forest.commandsConfig.Markdown.Timeout - cmdTemplate = forest.commandsConfig.Markdown.Template - break - case gfile.HTMLType: - cmdTimeout = forest.commandsConfig.HTML.Timeout - cmdTemplate = forest.commandsConfig.HTML.Template - break - case gfile.OfficeType: - cmdTimeout = forest.commandsConfig.Office.Timeout - cmdTemplate = forest.commandsConfig.Office.Template - break - default: - return "", &impossibleConversionError{} - } - - var data bytes.Buffer - if err := cmdTemplate.Execute(&data, cmdData); err != nil { + cmd, err := config.GetCommand(file.Extension) + if err != nil { return "", err } - err := forest.run(data.String(), cmdTimeout) + var data bytes.Buffer + if err := cmd.Template.Execute(&data, cmdData); err != nil { + return "", err + } + + err = forest.run(data.String(), cmd.Timeout) if err != nil { return "", err } @@ -138,15 +106,17 @@ func Merge(workingDir string, filesPaths []string) (string, error) { ResultFilePath: gfile.MakeFilePath(workingDir, ".pdf"), } - cmdTimeout := forest.commandsConfig.Merge.Timeout - cmdTemplate := forest.commandsConfig.Merge.Template - - var data bytes.Buffer - if err := cmdTemplate.Execute(&data, cmdData); err != nil { + cmd, err := config.GetCommand(".pdf") + if err != nil { return "", err } - err := forest.run(data.String(), cmdTimeout) + var data bytes.Buffer + if err := cmd.Template.Execute(&data, cmdData); err != nil { + return "", err + } + + err = forest.run(data.String(), cmd.Timeout) if err != nil { return "", err } diff --git a/app/converter/process/process_test.go b/app/converter/process/process_test.go index a78eba22..fb87ed86 100644 --- a/app/converter/process/process_test.go +++ b/app/converter/process/process_test.go @@ -22,20 +22,10 @@ func makeFile(workingDir string, fileName string) *gfile.File { return f } -func loadCommandConfigs(configurationFilePath string) { +func load(configurationFilePath string) { + config.Reset() path, _ := filepath.Abs(configurationFilePath) - c, _ := config.NewAppConfig(path) - Load(c.CommandsConfig) -} - -func TestLoad(t *testing.T) { - path, _ := filepath.Abs("../../../_tests/configurations/gotenberg.yml") - c, _ := config.NewAppConfig(path) - Load(c.CommandsConfig) - - if c.CommandsConfig != forest.commandsConfig { - t.Error("Commands configuration should have been loaded correctly") - } + config.ParseFile(path) } func TestRun(t *testing.T) { @@ -66,7 +56,7 @@ func TestUnconv(t *testing.T) { workingDir := "test" os.Mkdir(workingDir, 0666) - loadCommandConfigs("../../../_tests/configurations/gotenberg.yml") + load("../../../_tests/configurations/gotenberg.yml") // case 1: uses an Markdown file type. file = makeFile(workingDir, "file.md") @@ -92,7 +82,7 @@ func TestUnconv(t *testing.T) { t.Errorf("Converting '%s' to PDF should not have worked", file.Path) } - loadCommandConfigs("../../../_tests/configurations/timeout-gotenberg.yml") + load("../../../_tests/configurations/timeout-gotenberg.yml") // case 5: uses a command with an unsuitable timeout. file = makeFile(workingDir, "file.docx") @@ -107,7 +97,7 @@ func TestMerge(t *testing.T) { workingDir := "test" os.Mkdir(workingDir, 0666) - loadCommandConfigs("../../../_tests/configurations/gotenberg.yml") + load("../../../_tests/configurations/gotenberg.yml") var filesPaths []string path, _ := filepath.Abs("../../../_tests/file.pdf") @@ -119,7 +109,7 @@ func TestMerge(t *testing.T) { t.Error("Merge should have worked") } - loadCommandConfigs("../../../_tests/configurations/timeout-gotenberg.yml") + load("../../../_tests/configurations/timeout-gotenberg.yml") // case 2: uses a command with an unsuitable timeout. if _, err := Merge(workingDir, filesPaths); err == nil { @@ -129,13 +119,6 @@ func TestMerge(t *testing.T) { os.RemoveAll(workingDir) } -func TestImpossibleConversionError(t *testing.T) { - err := &impossibleConversionError{} - if err.Error() != impossibleConversionErrorMessage { - t.Errorf("Error returned a wrong message: got '%s' want '%s'", err.Error(), impossibleConversionErrorMessage) - } -} - func TestCommandTimeoutError(t *testing.T) { err := &commandTimeoutError{ command: "echo hello", diff --git a/app/handlers_test.go b/app/handlers_test.go index b947fd9a..18ac7e5b 100644 --- a/app/handlers_test.go +++ b/app/handlers_test.go @@ -11,7 +11,6 @@ import ( "github.com/thecodingmachine/gotenberg/app/config" "github.com/thecodingmachine/gotenberg/app/context" - "github.com/thecodingmachine/gotenberg/app/converter/process" "github.com/justinas/alice" ) @@ -46,10 +45,10 @@ func makeRequest(filesPaths ...string) *http.Request { return req } -func loadCommandConfigs(configurationFilePath string) { +func load(configurationFilePath string) { + config.Reset() path, _ := filepath.Abs(configurationFilePath) - c, _ := config.NewAppConfig(path) - process.Load(c.CommandsConfig) + config.ParseFile(path) } func fakeSuccessHandler(w http.ResponseWriter, r *http.Request) { @@ -140,7 +139,7 @@ func TestConvertHandler(t *testing.T) { t.Errorf("Handler returned a wrong status code: got '%v' want '%v'", status, http.StatusBadRequest) } - loadCommandConfigs("../_tests/configurations/merge-timeout-gotenberg.yml") + load("../_tests/configurations/merge-timeout-gotenberg.yml") // case 3: sends a request with two files and using an unsuitable timeout for merge commande. path, _ = filepath.Abs("../_tests/file.pdf") @@ -152,7 +151,7 @@ func TestConvertHandler(t *testing.T) { t.Errorf("Handler returned a wrong status code: got '%v' want '%v'", status, http.StatusInternalServerError) } - loadCommandConfigs("../_tests/configurations/gotenberg.yml") + load("../_tests/configurations/gotenberg.yml") // case 4: sends a request with two files. path, _ = filepath.Abs("../_tests/file.pdf") diff --git a/app/http/http.go b/app/http/http.go index 31a4a2ba..45938e37 100644 --- a/app/http/http.go +++ b/app/http/http.go @@ -1,4 +1,4 @@ -// Package http provides functions for detecting a request or a file content type. +// Package http provides functions for detecting a request content type. package http import ( diff --git a/main.go b/main.go index 2a7371a1..25a60e5a 100644 --- a/main.go +++ b/main.go @@ -17,7 +17,6 @@ import ( "github.com/thecodingmachine/gotenberg/app" "github.com/thecodingmachine/gotenberg/app/config" - "github.com/thecodingmachine/gotenberg/app/converter/process" "github.com/thecodingmachine/gotenberg/app/logger" "github.com/gorilla/mux" @@ -33,7 +32,7 @@ const defaultConfigurationFilePath = "gotenberg.yml" // main initializes the application, starts it, and handles // graceful shutdown. func main() { - c, err := config.NewAppConfig(defaultConfigurationFilePath) + err := config.ParseFile(defaultConfigurationFilePath) if err != nil { logger.SetLevel(logrus.InfoLevel) logger.Fatal(err) @@ -41,8 +40,8 @@ func main() { } // defines our application logging. - logger.SetLevel(c.Logs.Level) - logger.SetFormatter(c.Logs.Formatter) + logger.SetLevel(config.GetLogsLevel()) + logger.SetFormatter(config.GetLogsFormatter()) // defines our application router. r := mux.NewRouter() @@ -50,13 +49,12 @@ func main() { // defines our server. s := &http.Server{ - Addr: fmt.Sprintf(":%s", c.Port), + Addr: fmt.Sprintf(":%s", config.GetPort()), Handler: r, } - process.Load(c.CommandsConfig) logger.Infof("Starting Gotenberg version %s", version) - logger.Infof("Listening on port %s", c.Port) + logger.Infof("Listening on port %s", config.GetPort()) // runs our server in a goroutine so that it doesn't block. go func() { diff --git a/orbit-payload.yml b/orbit-payload.yml index 27cb56af..30873531 100644 --- a/orbit-payload.yml +++ b/orbit-payload.yml @@ -1,7 +1,4 @@ payload: - key: Version - value: snapshot - - - key: Latest - value: 1.0.0 \ No newline at end of file + value: snapshot \ No newline at end of file diff --git a/orbit.yml b/orbit.yml index c325eb60..217743d9 100644 --- a/orbit.yml +++ b/orbit.yml @@ -8,7 +8,6 @@ tasks: - use: generate short: Generates all files from blueprints run: - - BRANCH="$(git symbolic-ref --short HEAD)"; orbit generate -f .blueprints/README.blueprint.md -o README.md -p "Branch,${BRANCH}" - orbit generate -f .blueprints/Dockerfile.blueprint -o Dockerfile.ci -p "Image,CI" - orbit generate -f .blueprints/Dockerfile.blueprint -o Dockerfile -p "Image,MAIN"
- 
- 
- 
-