pax_global_header00006660000000000000000000000064151471043420014513gustar00rootroot0000000000000052 comment=072881778e4ea38074ad780e7388a0ce41f0bed6 golang-github-caddyserver-certmagic-0.25.2/000077500000000000000000000000001514710434200205755ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.25.2/.github/000077500000000000000000000000001514710434200221355ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.25.2/.github/CONTRIBUTING.md000066400000000000000000000232101514710434200243640ustar00rootroot00000000000000Contributing to CertMagic ========================= ## Common Tasks - [Contributing code](#contributing-code) - [Reporting a bug](#reporting-bugs) - [Suggesting an enhancement or a new feature](#suggesting-features) - [Improving documentation](#improving-documentation) Other menu items: - [Values](#values) - [Thank You](#thank-you) ### Contributing code You can have a direct impact on the project by helping with its code. To contribute code to CertMagic, open a [pull request](https://github.com/caddyserver/certmagic/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.** You can get familiar with CertMagic's code base by using [code search at Sourcegraph](https://sourcegraph.com/github.com/caddyserver/certmagic). We hold contributions to a high standard for quality :bowtie:, so don't be surprised if we ask for revisions—even if it seems small or insignificant. Please don't take it personally. :wink: If your change is on the right track, we can guide you to make it mergable. Here are some of the expectations we have of contributors: - If your change is more than just a minor alteration, **open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that changes are in-line with the project's goals and the best interests of its users. If there's already an issue about it, comment on the existing issue to claim it. - **Keep pull requests small.** Smaller PRs are more likely to be merged because they are easier to review! We might ask you to break up large PRs into smaller ones. [An example of what we DON'T do.](https://twitter.com/iamdevloper/status/397664295875805184) - [**Don't "push" your pull requests.**](https://www.igvita.com/2011/12/19/dont-push-your-pull-requests/) Basically, work with—not against—the maintainers -- theirs is not a glorious job. In fact, consider becoming a CertMagic maintainer yourself! - **Keep related commits together in a PR.** We do want pull requests to be small, but you should also keep multiple related commits in the same PR if they rely on each other. - **Write tests.** Tests are essential! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass. - **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven with benchmarks or profiling. - **[Squash](https://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance. Interactive rebase can do this, or a simpler way is `git reset --soft ` then `git commit -s`. - **Maintain your contributions.** Please help maintain your change after it is merged. - **Use comments properly.** We expect good godoc comments for package-level functions, types, and values. Comments are also useful whenever the purpose for a line of code is not obvious, and comments should not state the obvious. We often grant [collaborator status](#collaborator-instructions) to contributors who author one or more significant, high-quality PRs that are merged into the code base! ### HOW TO MAKE A PULL REQUEST TO CERTMAGIC Contributing to Go projects on GitHub is fun and easy. We recommend the following workflow: 1. [Fork this repo](https://github.com/caddyserver/certmagic). This makes a copy of the code you can write to. 2. If you don't already have this repo (caddyserver/certmagic.git) repo on your computer, get it with `go get github.com/caddyserver/certmagic`. 3. Tell git that it can push the caddyserver/certmagic.git repo to your fork by adding a remote: `git remote add myfork https://github.com/you/certmagic.git` 4. Make your changes in the caddyserver/certmagic.git repo on your computer. 5. Push your changes to your fork: `git push myfork` 6. [Create a pull request](https://github.com/caddyserver/certmagic/pull/new/master) to merge your changes into caddyserver/certmagic @ master. (Click "compare across forks" and change the head fork.) This workflow is nice because you don't have to change import paths. You can get fancier by using different branches if you want. ### Reporting bugs Like every software, CertMagic has its flaws. If you find one, [search the issues](https://github.com/caddyserver/certmagic/issues) to see if it has already been reported. If not, [open a new issue](https://github.com/caddyserver/certmagic/issues/new) and describe the bug clearly. **You can help stop bugs in their tracks!** Speed up the patching process by identifying the bug in the code. This can sometimes be done by adding `fmt.Println()` statements (or similar) in relevant code paths to narrow down where the problem may be. It's a good way to [introduce yourself to the Go language](https://tour.golang.org), too. Please follow the issue template so we have all the needed information. Unredacted—yes, actual values matter. We need to be able to repeat the bug using your instructions. Please simplify the issue as much as possible. The burden is on you to convince us that it is actually a bug in CertMagic. This is easiest to do when you write clear, concise instructions so we can reproduce the behavior (even if it seems obvious). The more detailed and specific you are, the faster we will be able to help you! Failure to fill out the issue template will probably result in the issue being closed. We suggest reading [How to Report Bugs Effectively](https://www.chiark.greenend.org.uk/~sgtatham/bugs.html). Please be kind. :smile: Remember that CertMagic comes at no cost to you, and you're getting free support when we fix your issues. If we helped you, please consider helping someone else! ### Suggesting features First, [search to see if your feature has already been requested](https://github.com/caddyserver/certmagic/issues). If it has, you can add a :+1: reaction to vote for it. If your feature idea is new, open an issue to request the feature. You don't have to follow the bug template for feature requests. Please describe your idea thoroughly so that we know how to implement it! Really vague requests may not be helpful or actionable and without clarification will have to be closed. **Please do not "bump" issues with comments that ask if there are any updates.** While we really do value your requests and implement many of them, not all features are a good fit for CertMagic. If a feature is not in the best interest of the CertMagic project or its users in general, we may politely decline to implement it. ## Collaborator Instructions Collabators have push rights to the repository. We grant this permission after one or more successful, high-quality PRs are merged! We thank them for their help.The expectations we have of collaborators are: - **Help review pull requests.** Be meticulous, but also kind. We love our contributors, but we critique the contribution to make it better. Multiple, thorough reviews make for the best contributions! Here are some questions to consider: - Can the change be made more elegant? - Is this a maintenance burden? - What assumptions does the code make? - Is it well-tested? - Is the change a good fit for the project? - Does it actually fix the problem or is it creating a special case instead? - Does the change incur any new dependencies? (Avoid these!) - **Answer issues.** If every collaborator helped out with issues, we could count the number of open issues on two hands. This means getting involved in the discussion, investigating the code, and yes, debugging it. It's fun. Really! :smile: Please, please help with open issues. Granted, some issues need to be done before others. And of course some are larger than others: you don't have to do it all yourself. Work with other collaborators as a team! - **Do not merge pull requests until they have been approved by one or two other collaborators.** If a project owner approves the PR, it can be merged (as long as the conversation has finished too). - **Prefer squashed commits over a messy merge.** If there are many little commits, please [squash the commits](https://stackoverflow.com/a/11732910/1048862) so we don't clutter the commit history. - **Don't accept new dependencies lightly.** Dependencies can make the world crash and burn, but they are sometimes necessary. Choose carefully. Extremely small dependencies (a few lines of code) can be inlined. The rest may not be needed. - **Make sure tests test the actual thing.** Double-check that the tests fail without the change, and pass with it. It's important that they assert what they're purported to assert. - **Recommended reading** - [CodeReviewComments](https://github.com/golang/go/wiki/CodeReviewComments) for an idea of what we look for in good, clean Go code - [Linus Torvalds describes a good commit message](https://gist.github.com/matthewhudson/1475276) - [Best Practices for Maintainers](https://opensource.guide/best-practices/) - [Shrinking Code Review](https://alexgaynor.net/2015/dec/29/shrinking-code-review/) ## Values - A person is always more important than code. People don't like being handled "efficiently". But we can still process issues and pull requests efficiently while being kind, patient, and considerate. - The ends justify the means, if the means are good. A good tree won't produce bad fruit. But if we cut corners or are hasty in our process, the end result will not be good. ## Thank you Thanks for your help! CertMagic would not be what it is today without your contributions.golang-github-caddyserver-certmagic-0.25.2/.github/FUNDING.yml000066400000000000000000000013171514710434200237540ustar00rootroot00000000000000# These are supported funding model platforms github: [mholt] # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2] patreon: # Replace with a single Patreon username open_collective: # Replace with a single Open Collective username ko_fi: # Replace with a single Ko-fi username tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry liberapay: # Replace with a single Liberapay username issuehunt: # Replace with a single IssueHunt username otechie: # Replace with a single Otechie username custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2'] golang-github-caddyserver-certmagic-0.25.2/.github/ISSUE_TEMPLATE/000077500000000000000000000000001514710434200243205ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.25.2/.github/ISSUE_TEMPLATE/bug_report.md000066400000000000000000000024451514710434200270170ustar00rootroot00000000000000--- name: Bug report about: For behaviors which violate documentation or cause incorrect results title: '' labels: '' assignees: '' --- ## What version of the package are you using? ## What are you trying to do? ## What steps did you take? ## What did you expect to happen, and what actually happened instead? ## How do you think this should be fixed? ## Please link to any related issues, pull requests, and/or discussion ## Bonus: What do you use CertMagic for, and do you find it useful? golang-github-caddyserver-certmagic-0.25.2/.github/ISSUE_TEMPLATE/feature_request.md000066400000000000000000000020011514710434200300360ustar00rootroot00000000000000--- name: Feature request about: Suggest an idea for this project title: '' labels: feature request assignees: '' --- ## What would you like to have changed? ## Why is this feature a useful, necessary, and/or important addition to this project? ## What alternatives are there, or what are you doing in the meantime to work around the lack of this feature? ## Please link to any relevant issues, pull requests, or other discussions. golang-github-caddyserver-certmagic-0.25.2/.github/ISSUE_TEMPLATE/question.md000066400000000000000000000015231514710434200265120ustar00rootroot00000000000000--- name: Question about: For help or questions about this package title: '' labels: question assignees: '' --- ## What is your question? ## What have you already tried? ## Include any other information or discussion. ## Bonus: What do you use this package for, and does it help you? golang-github-caddyserver-certmagic-0.25.2/.github/workflows/000077500000000000000000000000001514710434200241725ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.25.2/.github/workflows/ci.yml000066400000000000000000000016211514710434200253100ustar00rootroot00000000000000# Used as inspiration: https://github.com/mvdan/github-actions-golang name: Tests on: push: branches: - master pull_request: branches: - master jobs: test: strategy: matrix: os: [ ubuntu-latest, macos-latest, windows-latest ] go: [ '1.25', '1.26' ] runs-on: ${{ matrix.os }} steps: - name: Checkout code uses: actions/checkout@v3 - name: Install Go uses: actions/setup-go@v4 with: go-version: ${{ matrix.go }} - name: Print Go version and environment id: vars run: | printf "Using go at: $(which go)\n" printf "Go version: $(go version)\n" printf "\n\nGo environment:\n\n" go env printf "\n\nSystem environment:\n\n" env - name: Install dependencies run: go mod download - name: Run tests run: go test -v -short -race ./... golang-github-caddyserver-certmagic-0.25.2/.gitignore000066400000000000000000000000141514710434200225600ustar00rootroot00000000000000_gitignore/ golang-github-caddyserver-certmagic-0.25.2/LICENSE.txt000066400000000000000000000261351514710434200224270ustar00rootroot00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "{}" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright {yyyy} {name of copyright owner} Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. golang-github-caddyserver-certmagic-0.25.2/README.md000066400000000000000000000773411514710434200220700ustar00rootroot00000000000000

CertMagic

Easy and Powerful TLS Automation

The same library used by the Caddy Web Server

Caddy's [automagic TLS features](https://caddyserver.com/docs/automatic-https)—now for your own Go programs—in one powerful and easy-to-use library! CertMagic is the most mature, robust, and powerful ACME client integration for Go... and perhaps ever. With CertMagic, you can add one line to your Go application to serve securely over TLS, without ever having to touch certificates. Instead of: ```go // plaintext HTTP, gross 🤢 http.ListenAndServe(":80", mux) ``` Use CertMagic: ```go // encrypted HTTPS with HTTP->HTTPS redirects - yay! 🔒😍 certmagic.HTTPS([]string{"example.com"}, mux) ``` That line of code will serve your HTTP router `mux` over HTTPS, complete with HTTP->HTTPS redirects. It obtains and renews the TLS certificates. It staples OCSP responses for greater privacy and security. As long as your domain name points to your server, CertMagic will keep its connections secure. Compared to other ACME client libraries for Go, only CertMagic supports the full suite of ACME features, and no other library matches CertMagic's maturity and reliability. CertMagic - Automatic HTTPS using Let's Encrypt =============================================== ## Menu - [Features](#features) - [Requirements](#requirements) - [Installation](#installation) - [Usage](#usage) - [Package Overview](#package-overview) - [Certificate authority](#certificate-authority) - [The `Config` type](#the-config-type) - [Defaults](#defaults) - [Providing an email address](#providing-an-email-address) - [Rate limiting](#rate-limiting) - [Development and testing](#development-and-testing) - [Examples](#examples) - [Serving HTTP handlers with HTTPS](#serving-http-handlers-with-https) - [Starting a TLS listener](#starting-a-tls-listener) - [Getting a tls.Config](#getting-a-tlsconfig) - [Advanced use](#advanced-use) - [Wildcard Certificates](#wildcard-certificates) - [Behind a load balancer (or in a cluster)](#behind-a-load-balancer-or-in-a-cluster) - [The ACME Challenges](#the-acme-challenges) - [HTTP Challenge](#http-challenge) - [TLS-ALPN Challenge](#tls-alpn-challenge) - [DNS Challenge](#dns-challenge) - [On-Demand TLS](#on-demand-tls) - [Storage](#storage) - [Cache](#cache) - [Events](#events) - [ZeroSSL](#zerossl) - [FAQ](#faq) - [Contributing](#contributing) - [Project History](#project-history) - [Credits and License](#credits-and-license) ## Features - Fully automated certificate management including issuance and renewal - One-line, fully managed HTTPS servers - Full control over almost every aspect of the system - HTTP->HTTPS redirects - Multiple issuers supported: get certificates from multiple sources/CAs for redundancy and resiliency - Solves all 3 common ACME challenges: HTTP, TLS-ALPN, and DNS (and capable of others) - Most robust error handling of _any_ ACME client - Challenges are randomized to avoid accidental dependence - Challenges are rotated to overcome certain network blockages - Robust retries for up to 30 days - Exponential backoff with carefully-tuned intervals - Retries with optional test/staging CA endpoint instead of production, to avoid rate limits - Written in Go, a language with memory-safety guarantees - Powered by [ACMEz](https://github.com/mholt/acmez/v3), _the_ premier ACME client library for Go - All [libdns](https://github.com/libdns) DNS providers work out-of-the-box - Pluggable storage backends (default: file system) - Pluggable key sources - Wildcard certificates - Automatic OCSP stapling ([done right](https://gist.github.com/sleevi/5efe9ef98961ecfb4da8#gistcomment-2336055)) [keeps your sites online!](https://twitter.com/caddyserver/status/1234874273724084226) - Will [automatically attempt](https://twitter.com/mholt6/status/1235577699541762048) to replace [revoked certificates](https://community.letsencrypt.org/t/2020-02-29-caa-rechecking-bug/114591/3?u=mholt)! - Staples stored to disk in case of responder outages - Distributed solving of all challenges (works behind load balancers) - Highly efficient, coordinated management in a fleet - Active locking - Smart queueing - Supports "on-demand" issuance of certificates (during TLS handshakes!) - Caddy / CertMagic pioneered this technology - Custom decision functions to regulate and throttle on-demand behavior - Optional event hooks for observation - One-time private keys by default (new key for each cert) to discourage pinning and reduce scope of key compromise - Works with any certificate authority (CA) compliant with the ACME specification RFC 8555 - Certificate revocation (please, only if private key is compromised) - Must-Staple (optional; not default) - Cross-platform support! Mac, Windows, Linux, BSD, Android... - Scales to hundreds of thousands of names/certificates per instance - Use in conjunction with your own certificates - Full support for [RFC 9773](https://datatracker.ietf.org/doc/html/rfc9773) (ACME Renewal Information; ARI) extension ## Requirements 0. ACME server (can be a publicly-trusted CA, or your own) 1. Public DNS name(s) you control 2. Server reachable from public Internet - Or use the DNS challenge to waive this requirement 3. Control over port 80 (HTTP) and/or 443 (HTTPS) - Or they can be forwarded to other ports you control - Or use the DNS challenge to waive this requirement - (This is a requirement of the ACME protocol, not a library limitation) 4. Persistent storage - Typically the local file system (default) - Other integrations available/possible 5. Go 1.21 or newer **_Before using this library, your domain names MUST be pointed (A/AAAA records) at your server (unless you use the DNS challenge)!_** ## Installation ```bash $ go get github.com/caddyserver/certmagic ``` ## Usage ### Package Overview #### Certificate authority This library uses Let's Encrypt by default, but you can use any certificate authority that conforms to the ACME specification. Known/common CAs are provided as consts in the package, for example `LetsEncryptStagingCA` and `LetsEncryptProductionCA`. #### The `Config` type The `certmagic.Config` struct is how you can wield the power of this fully armed and operational battle station. However, an empty/uninitialized `Config` is _not_ a valid one! In time, you will learn to use the force of `certmagic.NewDefault()` as I have. #### Defaults The default `Config` value is called `certmagic.Default`. Change its fields to suit your needs, then call `certmagic.NewDefault()` when you need a valid `Config` value. In other words, `certmagic.Default` is a template and is not valid for use directly. You can set the default values easily, for example: `certmagic.Default.Issuer = ...`. Similarly, to configure ACME-specific defaults, use `certmagic.DefaultACME`. The high-level functions in this package (`HTTPS()`, `Listen()`, `ManageSync()`, and `ManageAsync()`) use the default config exclusively. This is how most of you will interact with the package. This is suitable when all your certificates are managed the same way. However, if you need to manage certificates differently depending on their name, you will need to make your own cache and configs (keep reading). #### Providing an email address Although not strictly required, this is highly recommended best practice. It allows you to receive expiration emails if your certificates are expiring for some reason, and also allows the CA's engineers to potentially get in touch with you if something is wrong. I recommend setting `certmagic.DefaultACME.Email` or always setting the `Email` field of a new `Config` struct. #### Rate limiting To avoid firehosing the CA's servers, CertMagic has built-in rate limiting. Currently, its default limit is up to 10 transactions (obtain or renew) every 1 minute (sliding window). This can be changed by setting the `RateLimitEvents` and `RateLimitEventsWindow` variables, if desired. The CA may still enforce their own rate limits, and there's nothing (well, nothing ethical) CertMagic can do to bypass them for you. Additionally, CertMagic will retry failed validations with exponential backoff for up to 30 days, with a reasonable maximum interval between attempts (an "attempt" means trying each enabled challenge type once). ### Development and Testing Note that Let's Encrypt imposes [strict rate limits](https://letsencrypt.org/docs/rate-limits/) at its production endpoint, so using it while developing your application may lock you out for a few days if you aren't careful! While developing your application and testing it, use [their staging endpoint](https://letsencrypt.org/docs/staging-environment/) which has much higher rate limits. Even then, don't hammer it: but it's much safer for when you're testing. When deploying, though, use their production CA because their staging CA doesn't issue trusted certificates. To use staging, set `certmagic.DefaultACME.CA = certmagic.LetsEncryptStagingCA` or set `CA` of every `ACMEIssuer` struct. ### Examples There are many ways to use this library. We'll start with the highest-level (simplest) and work down (more control). All these high-level examples use `certmagic.Default` and `certmagic.DefaultACME` for the config and the default cache and storage for serving up certificates. First, we'll follow best practices and do the following: ```go // read and agree to your CA's legal documents certmagic.DefaultACME.Agreed = true // provide an email address certmagic.DefaultACME.Email = "you@yours.com" // use the staging endpoint while we're developing certmagic.DefaultACME.CA = certmagic.LetsEncryptStagingCA ``` For fully-functional program examples, check out [this X thread](https://x.com/mholt6/status/1073103805112147968) (or read it [unrolled into a single post](https://threadreaderapp.com/thread/1073103805112147968.html)). (Note that the package API has changed slightly since these posts.) #### Serving HTTP handlers with HTTPS ```go err := certmagic.HTTPS([]string{"example.com", "www.example.com"}, mux) if err != nil { return err } ``` This starts HTTP and HTTPS listeners and redirects HTTP to HTTPS! #### Starting a TLS listener ```go ln, err := certmagic.Listen([]string{"example.com"}) if err != nil { return err } ``` #### Getting a tls.Config ```go tlsConfig, err := certmagic.TLS([]string{"example.com"}) if err != nil { return err } // be sure to customize NextProtos if serving a specific // application protocol after the TLS handshake, for example: tlsConfig.NextProtos = append([]string{"h2", "http/1.1"}, tlsConfig.NextProtos...) ``` #### Advanced use For more control (particularly, if you need a different way of managing each certificate), you'll make and use a `Cache` and a `Config` like so: ```go // First make a pointer to a Cache as we need to reference the same Cache in // GetConfigForCert below. var cache *certmagic.Cache cache = certmagic.NewCache(certmagic.CacheOptions{ GetConfigForCert: func(cert certmagic.Certificate) (*certmagic.Config, error) { // Here we use New to get a valid Config associated with the same cache. // The provided Config is used as a template and will be completed with // any defaults that are set in the Default config. return certmagic.New(cache, certmagic.Config{ // ... }), nil }, ... }) magic := certmagic.New(cache, certmagic.Config{ // any customizations you need go here }) myACME := certmagic.NewACMEIssuer(magic, certmagic.ACMEIssuer{ CA: certmagic.LetsEncryptStagingCA, Email: "you@yours.com", Agreed: true, // plus any other customizations you need }) magic.Issuers = []certmagic.Issuer{myACME} // this obtains certificates or renews them if necessary err := magic.ManageSync(context.TODO(), []string{"example.com", "sub.example.com"}) if err != nil { return err } // to use its certificates and solve the TLS-ALPN challenge, // you can get a TLS config to use in a TLS listener! tlsConfig := magic.TLSConfig() // be sure to customize NextProtos if serving a specific // application protocol after the TLS handshake, for example: tlsConfig.NextProtos = append([]string{"h2", "http/1.1"}, tlsConfig.NextProtos...) //// OR //// // if you already have a TLS config you don't want to replace, // we can simply set its GetCertificate field and append the // TLS-ALPN challenge protocol to the NextProtos myTLSConfig.GetCertificate = magic.GetCertificate myTLSConfig.NextProtos = append(myTLSConfig.NextProtos, acmez.ACMETLS1Protocol) // the HTTP challenge has to be handled by your HTTP server; // if you don't have one, you should have disabled it earlier // when you made the certmagic.Config httpMux = myACME.HTTPChallengeHandler(httpMux) ``` Great! This example grants you much more flexibility for advanced programs. However, _the vast majority of you will only use the high-level functions described earlier_, especially since you can still customize them by setting the package-level `Default` config. ### Wildcard certificates At time of writing (December 2018), Let's Encrypt only issues wildcard certificates with the DNS challenge. You can easily enable the DNS challenge with CertMagic for numerous providers (see the relevant section in the docs). ### Behind a load balancer (or in a cluster) CertMagic runs effectively behind load balancers and/or in cluster/fleet environments. In other words, you can have 10 or 1,000 servers all serving the same domain names, all sharing certificates and OCSP staples. To do so, simply ensure that each instance is using the same Storage. That is the sole criteria for determining whether an instance is part of a cluster. The default Storage is implemented using the file system, so mounting the same shared folder is sufficient (see [Storage](#storage) for more on that)! If you need an alternate Storage implementation, feel free to use one, provided that all the instances use the _same_ one. :) See [Storage](#storage) and the associated [pkg.go.dev](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Storage) for more information! ## The ACME Challenges This section describes how to solve the ACME challenges. Challenges are how you demonstrate to the certificate authority some control over your domain name, thus authorizing them to grant you a certificate for that name. [The great innovation of ACME](https://www.youtube.com/watch?v=KdX51QJWQTA) is that verification by CAs can now be automated, rather than having to click links in emails (who ever thought that was a good idea??). If you're using the high-level convenience functions like `HTTPS()`, `Listen()`, or `TLS()`, the HTTP and/or TLS-ALPN challenges are solved for you because they also start listeners. However, if you're making a `Config` and you start your own server manually, you'll need to be sure the ACME challenges can be solved so certificates can be renewed. The HTTP and TLS-ALPN challenges are the defaults because they don't require configuration from you, but they require that your server is accessible from external IPs on low ports. If that is not possible in your situation, you can enable the DNS challenge, which will disable the HTTP and TLS-ALPN challenges and use the DNS challenge exclusively. Technically, only one challenge needs to be enabled for things to work, but using multiple is good for reliability in case a challenge is discontinued by the CA. This happened to the TLS-SNI challenge in early 2018—many popular ACME clients such as Traefik and Autocert broke, resulting in downtime for some sites, until new releases were made and patches deployed, because they used only one challenge; Caddy, however—this library's forerunner—was unaffected because it also used the HTTP challenge. If multiple challenges are enabled, they are chosen randomly to help prevent false reliance on a single challenge type. And if one fails, any remaining enabled challenges are tried before giving up. ### HTTP Challenge Per the ACME spec, the HTTP challenge requires port 80, or at least packet forwarding from port 80. It works by serving a specific HTTP response that only the genuine server would have to a normal HTTP request at a special endpoint. If you are running an HTTP server, solving this challenge is very easy: just wrap your handler in `HTTPChallengeHandler` _or_ call `SolveHTTPChallenge()` inside your own `ServeHTTP()` method. For example, if you're using the standard library: ```go mux := http.NewServeMux() mux.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) { fmt.Fprintf(w, "Lookit my cool website over HTTPS!") }) http.ListenAndServe(":80", myACME.HTTPChallengeHandler(mux)) ``` If wrapping your handler is not a good solution, try this inside your `ServeHTTP()` instead: ```go magic := certmagic.NewDefault() myACME := certmagic.NewACMEIssuer(magic, certmagic.DefaultACME) func ServeHTTP(w http.ResponseWriter, req *http.Request) { if myACME.HandleHTTPChallenge(w, r) { return // challenge handled; nothing else to do } ... } ``` If you are not running an HTTP server, you should disable the HTTP challenge _or_ run an HTTP server whose sole job it is to solve the HTTP challenge. ### TLS-ALPN Challenge Per the ACME spec, the TLS-ALPN challenge requires port 443, or at least packet forwarding from port 443. It works by providing a special certificate using a standard TLS extension, Application Layer Protocol Negotiation (ALPN), having a special value. This is the most convenient challenge type because it usually requires no extra configuration and uses the standard TLS port which is where the certificates are used, also. This challenge is easy to solve: just use the provided `tls.Config` when you make your TLS listener: ```go // use this to configure a TLS listener tlsConfig := magic.TLSConfig() ``` Or make two simple changes to an existing `tls.Config`: ```go myTLSConfig.GetCertificate = magic.GetCertificate myTLSConfig.NextProtos = append(myTLSConfig.NextProtos, acmez.ACMETLS1Protocol} ``` Then just make sure your TLS listener is listening on port 443: ```go ln, err := tls.Listen("tcp", ":443", myTLSConfig) ``` ### DNS Challenge The DNS challenge is perhaps the most useful challenge because it allows you to obtain certificates without your server needing to be publicly accessible on the Internet, and it's the only challenge by which Let's Encrypt will issue wildcard certificates. This challenge works by setting a special record in the domain's zone. To do this automatically, your DNS provider needs to offer an API by which changes can be made to domain names, and the changes need to take effect immediately for best results. CertMagic supports [all DNS providers with `libdns` implementations](https://github.com/libdns)! It always cleans up the temporary record after the challenge completes. To enable it, just set the `DNS01Solver` field on a `certmagic.ACMEIssuer` struct, or set the default `certmagic.ACMEIssuer.DNS01Solver` variable. For example, if my domains' DNS was served by Cloudflare: ```go import "github.com/libdns/cloudflare" certmagic.DefaultACME.DNS01Solver = &certmagic.DNS01Solver{ DNSManager: certmagic.DNSManager{ DNSProvider: &cloudflare.Provider{ APIToken: "topsecret", }, }, } ``` Now the DNS challenge will be used by default, and I can obtain certificates for wildcard domains, too. Enabling the DNS challenge disables the other challenges for that `certmagic.ACMEIssuer` instance. ## On-Demand TLS Normally, certificates are obtained and renewed before a listener starts serving, and then those certificates are maintained throughout the lifetime of the program. In other words, the certificate names are static. But sometimes you don't know all the names ahead of time, or you don't want to manage all the certificates up front. This is where On-Demand TLS shines. Originally invented for use in Caddy (which was the first program to use such technology), On-Demand TLS makes it possible and easy to serve certificates for arbitrary or specific names during the lifetime of the server. When a TLS handshake is received, CertMagic will read the Server Name Indication (SNI) value and either load and present that certificate in the ServerHello, or if one does not exist, it will obtain it from a CA right then-and-there. Of course, this has some obvious security implications. You don't want to DoS a CA or allow arbitrary clients to fill your storage with spammy TLS handshakes. That's why, when you enable On-Demand issuance, you should set limits or policy to allow getting certificates. CertMagic has an implicit whitelist built-in which is sufficient for nearly everyone, but also has a more advanced way to control on-demand issuance. The simplest way to enable on-demand issuance is to set the OnDemand field of a Config (or the default package-level value): ```go certmagic.Default.OnDemand = new(certmagic.OnDemandConfig) ``` By setting this to a non-nil value, on-demand TLS is enabled for that config. For convenient security, CertMagic's high-level abstraction functions such as `HTTPS()`, `TLS()`, `ManageSync()`, `ManageAsync()`, and `Listen()` (which all accept a list of domain names) will whitelist those names automatically so only certificates for those names can be obtained when using the Default config. Usually this is sufficient for most users. However, if you require advanced control over which domains can be issued certificates on-demand (for example, if you do not know which domain names you are managing, or just need to defer their operations until later), you should implement your own DecisionFunc: ```go // if the decision function returns an error, a certificate // may not be obtained for that name at that time certmagic.Default.OnDemand = &certmagic.OnDemandConfig{ DecisionFunc: func(name string) error { if name != "example.com" { return fmt.Errorf("not allowed") } return nil }, } ``` The [pkg.go.dev](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#OnDemandConfig) describes how to use this in full detail, so please check it out! ## Storage CertMagic relies on storage to store certificates and other TLS assets (OCSP staple cache, coordinating locks, etc). Persistent storage is a requirement when using CertMagic: ephemeral storage will likely lead to rate limiting on the CA-side as CertMagic will always have to get new certificates. By default, CertMagic stores assets on the local file system in `$HOME/.local/share/certmagic` (and honors `$XDG_DATA_HOME` if set). CertMagic will create the directory if it does not exist. If writes are denied, things will not be happy, so make sure CertMagic can write to it! The notion of a "cluster" or "fleet" of instances that may be serving the same site and sharing certificates, etc, is tied to storage. Simply, any instances that use the same storage facilities are considered part of the cluster. So if you deploy 100 instances of CertMagic behind a load balancer, they are all part of the same cluster if they share the same storage configuration. Sharing storage could be mounting a shared folder, or implementing some other distributed storage system such as a database server or KV store. The easiest way to change the storage being used is to set `certmagic.Default.Storage` to a value that satisfies the [Storage interface](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Storage). Keep in mind that a valid `Storage` must be able to implement some operations atomically in order to provide locking and synchronization. If you write a Storage implementation, please add it to the [project wiki](https://github.com/caddyserver/certmagic/wiki/Storage-Implementations) so people can find it! ## Cache All of the certificates in use are de-duplicated and cached in memory for optimal performance at handshake-time. This cache must be backed by persistent storage as described above. Most applications will not need to interact with certificate caches directly. Usually, the closest you will come is to set the package-wide `certmagic.Default.Storage` variable (before attempting to create any Configs) which defines how the cache is persisted. However, if your use case requires using different storage facilities for different Configs (that's highly unlikely and NOT recommended! Even Caddy doesn't get that crazy), you will need to call `certmagic.NewCache()` and pass in the storage you want to use, then get new `Config` structs with `certmagic.NewWithCache()` and pass in the cache. Again, if you're needing to do this, you've probably over-complicated your application design. ## Events (Events are new and still experimental, so they may change.) CertMagic emits events when possible things of interest happen. Set the [`OnEvent` field of your `Config`](https://pkg.go.dev/github.com/caddyserver/certmagic#Config.OnEvent) to subscribe to events; ignore the ones you aren't interested in. Here are the events currently emitted along with their metadata you can use: - **`cached_unmanaged_cert`** An unmanaged certificate was cached - `sans`: The subject names on the certificate - **`cert_obtaining`** A certificate is about to be obtained - `renewal`: Whether this is a renewal - `identifier`: The name on the certificate - `forced`: Whether renewal is being forced (if renewal) - `remaining`: Time left on the certificate (if renewal) - `issuer`: The previous or current issuer - **`cert_obtained`** A certificate was successfully obtained - `renewal`: Whether this is a renewal - `identifier`: The name on the certificate - `remaining`: Time left on the certificate (if renewal) - `issuer`: The previous or current issuer - `storage_path`: The path to the folder containing the cert resources within storage - `private_key_path`: The path to the private key file in storage - `certificate_path`: The path to the public key file in storage - `metadata_path`: The path to the metadata file in storage - **`cert_failed`** An attempt to obtain a certificate failed - `renewal`: Whether this is a renewal - `identifier`: The name on the certificate - `remaining`: Time left on the certificate (if renewal) - `issuers`: The issuer(s) tried - `error`: The (final) error message - **`tls_get_certificate`** The GetCertificate phase of a TLS handshake is under way - `client_hello`: The tls.ClientHelloInfo struct - **`cert_ocsp_revoked`** A certificate's OCSP indicates it has been revoked - `subjects`: The subject names on the certificate - `certificate`: The Certificate struct - `reason`: The OCSP revocation reason - `revoked_at`: When the certificate was revoked `OnEvent` can return an error. Some events may be aborted by returning an error. For example, returning an error from `cert_obtained` can cancel obtaining the certificate. Only return an error from `OnEvent` if you want to abort program flow. ## ZeroSSL ZeroSSL has both ACME and HTTP API services for getting certificates. CertMagic works with both of them. To use ZeroSSL's ACME server, configure CertMagic with an [`ACMEIssuer`](https://pkg.go.dev/github.com/caddyserver/certmagic#ACMEIssuer) like you would with any other ACME CA (just adjust the directory URL). External Account Binding (EAB) is required for ZeroSSL. You can use the [ZeroSSL API](https://pkg.go.dev/github.com/caddyserver/zerossl) to generate one, or your account dashboard. To use ZeroSSL's API instead, use the [`ZeroSSLIssuer`](https://pkg.go.dev/github.com/caddyserver/certmagic#ZeroSSLIssuer). Here is a simple example: ```go magic := certmagic.NewDefault() magic.Issuers = []certmagic.Issuer{ certmagic.ZeroSSLIssuer{ APIKey: "", }), } err := magic.ManageSync(ctx, []string{"example.com"}) ``` ## FAQ ### Can I use some of my own certificates while using CertMagic? Yes, just call the relevant method on the `Config` to add your own certificate to the cache: - [`CacheUnmanagedCertificatePEMBytes()`](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Config.CacheUnmanagedCertificatePEMBytes) - [`CacheUnmanagedCertificatePEMFile()`](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Config.CacheUnmanagedCertificatePEMFile) - [`CacheUnmanagedTLSCertificate()`](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Config.CacheUnmanagedTLSCertificate) Keep in mind that unmanaged certificates are (obviously) not renewed for you, so you'll have to replace them when you do. However, OCSP stapling is performed even for unmanaged certificates that qualify. ### Does CertMagic obtain SAN certificates? Technically all certificates these days are SAN certificates because CommonName is deprecated. But if you're asking whether CertMagic issues and manages certificates with multiple SANs, the answer is no. But it does support serving them, if you provide your own. ### How can I listen on ports 80 and 443? Do I have to run as root? On Linux, you can use `setcap` to grant your binary the permission to bind low ports: ```bash $ sudo setcap cap_net_bind_service=+ep /path/to/your/binary ``` and then you will not need to run with root privileges. ## Contributing We welcome your contributions! Please see our **[contributing guidelines](https://github.com/caddyserver/certmagic/blob/master/.github/CONTRIBUTING.md)** for instructions. ## Project History CertMagic is the core of Caddy's advanced TLS automation code, extracted into a library. The underlying ACME client implementation is [ACMEz](https://github.com/mholt/acmez/v3). CertMagic's code was originally a central part of Caddy even before Let's Encrypt entered public beta in 2015. In the years since then, Caddy's TLS automation techniques have been widely adopted, tried and tested in production, and served millions of sites and secured trillions of connections. Now, CertMagic is _the actual library used by Caddy_. It's incredibly powerful and feature-rich, but also easy to use for simple Go programs: one line of code can enable fully-automated HTTPS applications with HTTP->HTTPS redirects. Caddy is known for its robust HTTPS+ACME features. When ACME certificate authorities have had outages, in some cases Caddy was the only major client that didn't experience any downtime. Caddy can weather OCSP outages lasting days, or CA outages lasting weeks, without taking your sites offline. Caddy was also the first to sport "on-demand" issuance technology, which obtains certificates during the first TLS handshake for an allowed SNI name. Consequently, CertMagic brings all these (and more) features and capabilities right into your own Go programs. You can [watch a 2016 dotGo talk](https://youtu.be/KdX51QJWQTA) by the author of this library about using ACME to automate certificate management in Go programs: [![Matthew Holt speaking at dotGo 2016 about ACME in Go](https://user-images.githubusercontent.com/1128849/49921557-2d506780-fe6b-11e8-97bf-6053b6b4eb48.png)](https://youtu.be/KdX51QJWQTA) ## Credits and License CertMagic is a project by [Matthew Holt](https://x.com/mholt6), who is the author; and various contributors, who are credited in the commit history of either CertMagic or Caddy. CertMagic is licensed under Apache 2.0, an open source license. For convenience, its main points are summarized as follows (but this is no replacement for the actual license text): - The author owns the copyright to this code - Use, distribute, and modify the software freely - Private and internal use is allowed - License text and copyright notices must stay intact and be included with distributions - Any and all changes to the code must be documented golang-github-caddyserver-certmagic-0.25.2/account.go000066400000000000000000000373151514710434200225710ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bufio" "bytes" "context" "crypto/ecdsa" "crypto/elliptic" "crypto/rand" "encoding/json" "errors" "fmt" "io" "io/fs" "os" "path" "sort" "strings" "sync" "github.com/mholt/acmez/v3/acme" "go.uber.org/zap" ) // getAccountToUse will either load or create an account based on the configuration of the issuer. // It will try to get one from storage if one exists, and if not, it will create one, all the while // honoring the configured account key PEM (if any) to restrict which account is used. func (iss *ACMEIssuer) getAccountToUse(ctx context.Context, directory string) (acme.Account, error) { var account acme.Account var err error if iss.AccountKeyPEM != "" { iss.Logger.Info("using configured ACME account") account, err = iss.GetAccount(ctx, []byte(iss.AccountKeyPEM)) } else { account, err = iss.loadOrCreateAccount(ctx, directory, iss.getEmail()) } if err != nil { return acme.Account{}, fmt.Errorf("getting ACME account: %v", err) } return account, nil } // loadOrCreateAccount either loads or creates a new account, depending on if // an account can be found in storage for the given CA + email combo. func (am *ACMEIssuer) loadOrCreateAccount(ctx context.Context, ca, email string) (acme.Account, error) { acct, err := am.loadAccount(ctx, ca, email) if errors.Is(err, fs.ErrNotExist) { am.Logger.Info("creating new account because no account for configured email is known to us", zap.String("email", email), zap.String("ca", ca), zap.Error(err)) return am.newAccount(email) } am.Logger.Debug("using existing ACME account because key found in storage associated with email", zap.String("email", email), zap.String("ca", ca)) return acct, err } // loadAccount loads an account from storage, but does not create a new one. func (am *ACMEIssuer) loadAccount(ctx context.Context, ca, email string) (acme.Account, error) { regBytes, err := am.config.Storage.Load(ctx, am.storageKeyUserReg(ca, email)) if err != nil { return acme.Account{}, err } keyBytes, err := am.config.Storage.Load(ctx, am.storageKeyUserPrivateKey(ca, email)) if err != nil { return acme.Account{}, err } var acct acme.Account err = json.Unmarshal(regBytes, &acct) if err != nil { return acct, err } acct.PrivateKey, err = PEMDecodePrivateKey(keyBytes) if err != nil { return acct, fmt.Errorf("could not decode account's private key: %v", err) } return acct, nil } // newAccount generates a new private key for a new ACME account, but // it does not register or save the account. func (*ACMEIssuer) newAccount(email string) (acme.Account, error) { var acct acme.Account if email != "" { acct.Contact = []string{"mailto:" + email} // TODO: should we abstract the contact scheme? } privateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) if err != nil { return acct, fmt.Errorf("generating private key: %v", err) } acct.PrivateKey = privateKey return acct, nil } // GetAccount first tries loading the account with the associated private key from storage. // If it does not exist in storage, it will be retrieved from the ACME server and added to storage. // The account must already exist; it does not create a new account. func (am *ACMEIssuer) GetAccount(ctx context.Context, privateKeyPEM []byte) (acme.Account, error) { email := am.getEmail() if email == "" { if account, err := am.loadAccountByKey(ctx, privateKeyPEM); err == nil { return account, nil } } else { keyBytes, err := am.config.Storage.Load(ctx, am.storageKeyUserPrivateKey(am.CA, email)) if err == nil && bytes.Equal(bytes.TrimSpace(keyBytes), bytes.TrimSpace(privateKeyPEM)) { return am.loadAccount(ctx, am.CA, email) } } return am.lookUpAccount(ctx, privateKeyPEM) } // loadAccountByKey loads the account with the given private key from storage, if it exists. // If it does not exist, an error of type fs.ErrNotExist is returned. This is not very efficient // for lots of accounts. func (am *ACMEIssuer) loadAccountByKey(ctx context.Context, privateKeyPEM []byte) (acme.Account, error) { accountList, err := am.config.Storage.List(ctx, am.storageKeyUsersPrefix(am.CA), false) if err != nil { return acme.Account{}, err } for _, accountFolderKey := range accountList { email := path.Base(accountFolderKey) keyBytes, err := am.config.Storage.Load(ctx, am.storageKeyUserPrivateKey(am.CA, email)) if err != nil { // Try the next account: This one is missing its private key, if it turns out to be the one we're looking // for we will try to save it again after confirming with the ACME server. continue } if bytes.Equal(bytes.TrimSpace(keyBytes), bytes.TrimSpace(privateKeyPEM)) { // Found the account with the correct private key, try loading it. If this fails we we will follow // the same procedure as if the private key was not found and confirm with the ACME server before saving // it again. return am.loadAccount(ctx, am.CA, email) } } return acme.Account{}, fs.ErrNotExist } // lookUpAccount looks up the account associated with privateKeyPEM from the ACME server. // If the account is found by the server, it will be saved to storage and returned. func (am *ACMEIssuer) lookUpAccount(ctx context.Context, privateKeyPEM []byte) (acme.Account, error) { client, err := am.newACMEClient(false) if err != nil { return acme.Account{}, fmt.Errorf("creating ACME client: %v", err) } privateKey, err := PEMDecodePrivateKey([]byte(privateKeyPEM)) if err != nil { return acme.Account{}, fmt.Errorf("decoding private key: %v", err) } // look up the account account := acme.Account{PrivateKey: privateKey} account, err = client.GetAccount(ctx, account) if err != nil { return acme.Account{}, fmt.Errorf("looking up account with server: %v", err) } // save the account details to storage err = am.saveAccount(ctx, client.Directory, account) if err != nil { return account, fmt.Errorf("could not save account to storage: %v", err) } return account, nil } // saveAccount persists an ACME account's info and private key to storage. // It does NOT register the account via ACME or prompt the user. func (am *ACMEIssuer) saveAccount(ctx context.Context, ca string, account acme.Account) error { regBytes, err := json.MarshalIndent(account, "", "\t") if err != nil { return err } keyBytes, err := PEMEncodePrivateKey(account.PrivateKey) if err != nil { return err } // extract primary contact (email), without scheme (e.g. "mailto:") primaryContact := getPrimaryContact(account) all := []keyValue{ { key: am.storageKeyUserReg(ca, primaryContact), value: regBytes, }, { key: am.storageKeyUserPrivateKey(ca, primaryContact), value: keyBytes, }, } return storeTx(ctx, am.config.Storage, all) } // deleteAccountLocally deletes the registration info and private key of the account // for the given CA from storage. func (am *ACMEIssuer) deleteAccountLocally(ctx context.Context, ca string, account acme.Account) error { primaryContact := getPrimaryContact(account) if err := am.config.Storage.Delete(ctx, am.storageKeyUserReg(ca, primaryContact)); err != nil { return err } return am.config.Storage.Delete(ctx, am.storageKeyUserPrivateKey(ca, primaryContact)) } // setEmail does everything it can to obtain an email address // from the user within the scope of memory and storage to use // for ACME TLS. If it cannot get an email address, it does nothing // (If user is prompted, it will warn the user of // the consequences of an empty email.) This function MAY prompt // the user for input. If allowPrompts is false, the user // will NOT be prompted and an empty email may be returned. func (am *ACMEIssuer) setEmail(ctx context.Context, allowPrompts bool) error { leEmail := am.Email // First try package default email, or a discovered email address if leEmail == "" { leEmail = DefaultACME.Email } if leEmail == "" { discoveredEmailMu.Lock() leEmail = discoveredEmail discoveredEmailMu.Unlock() } // Then try to get most recent user email from storage var gotRecentEmail bool if leEmail == "" { leEmail, gotRecentEmail = am.mostRecentAccountEmail(ctx, am.CA) } if !gotRecentEmail && leEmail == "" && allowPrompts { // Looks like there is no email address readily available, // so we will have to ask the user if we can. var err error leEmail, err = am.promptUserForEmail() if err != nil { return err } // User might have just signified their agreement am.mu.Lock() am.agreed = DefaultACME.Agreed am.mu.Unlock() } // Save the email for later and ensure it is consistent // for repeated use; then update cfg with the email leEmail = strings.TrimSpace(strings.ToLower(leEmail)) discoveredEmailMu.Lock() if discoveredEmail == "" { discoveredEmail = leEmail } discoveredEmailMu.Unlock() // The unexported email field is the one we use // because we have thread-safe control over it am.mu.Lock() am.email = leEmail am.mu.Unlock() return nil } // promptUserForEmail prompts the user for an email address // and returns the email address they entered (which could // be the empty string). If no error is returned, then Agreed // will also be set to true, since continuing through the // prompt signifies agreement. func (am *ACMEIssuer) promptUserForEmail() (string, error) { // prompt the user for an email address and terms agreement reader := bufio.NewReader(stdin) am.promptUserAgreement("") fmt.Println("Please enter your email address to signify agreement and to be notified") fmt.Println("in case of issues. You can leave it blank, but we don't recommend it.") fmt.Print(" Email address: ") leEmail, err := reader.ReadString('\n') if err != nil && err != io.EOF { return "", fmt.Errorf("reading email address: %v", err) } leEmail = strings.TrimSpace(leEmail) DefaultACME.Agreed = true return leEmail, nil } // promptUserAgreement simply outputs the standard user // agreement prompt with the given agreement URL. // It outputs a newline after the message. func (am *ACMEIssuer) promptUserAgreement(agreementURL string) { userAgreementPrompt := `Your sites will be served over HTTPS automatically using an automated CA. By continuing, you agree to the CA's terms of service` if agreementURL == "" { fmt.Printf("\n\n%s.\n", userAgreementPrompt) return } fmt.Printf("\n\n%s at:\n %s\n", userAgreementPrompt, agreementURL) } // askUserAgreement prompts the user to agree to the agreement // at the given agreement URL via stdin. It returns whether the // user agreed or not. func (am *ACMEIssuer) askUserAgreement(agreementURL string) bool { am.promptUserAgreement(agreementURL) fmt.Print("Do you agree to the terms? (y/n): ") reader := bufio.NewReader(stdin) answer, err := reader.ReadString('\n') if err != nil { return false } answer = strings.ToLower(strings.TrimSpace(answer)) return answer == "y" || answer == "yes" } func storageKeyACMECAPrefix(issuerKey string) string { return path.Join(prefixACME, StorageKeys.Safe(issuerKey)) } func (am *ACMEIssuer) storageKeyCAPrefix(caURL string) string { return storageKeyACMECAPrefix(am.issuerKey(caURL)) } func (am *ACMEIssuer) storageKeyUsersPrefix(caURL string) string { return path.Join(am.storageKeyCAPrefix(caURL), "users") } func (am *ACMEIssuer) storageKeyUserPrefix(caURL, email string) string { if email == "" { email = emptyEmail } return path.Join(am.storageKeyUsersPrefix(caURL), StorageKeys.Safe(email)) } func (am *ACMEIssuer) storageKeyUserReg(caURL, email string) string { return am.storageSafeUserKey(caURL, email, "registration", ".json") } func (am *ACMEIssuer) storageKeyUserPrivateKey(caURL, email string) string { return am.storageSafeUserKey(caURL, email, "private", ".key") } // storageSafeUserKey returns a key for the given email, with the default // filename, and the filename ending in the given extension. func (am *ACMEIssuer) storageSafeUserKey(ca, email, defaultFilename, extension string) string { if email == "" { email = emptyEmail } email = strings.ToLower(email) filename := am.emailUsername(email) if filename == "" { filename = defaultFilename } filename = StorageKeys.Safe(filename) return path.Join(am.storageKeyUserPrefix(ca, email), filename+extension) } // emailUsername returns the username portion of an email address (part before // '@') or the original input if it can't find the "@" symbol. func (*ACMEIssuer) emailUsername(email string) string { at := strings.Index(email, "@") if at == -1 { return email } else if at == 0 { return email[1:] } return email[:at] } // mostRecentAccountEmail finds the most recently-written account file // in storage. Since this is part of a complex sequence to get a user // account, errors here are discarded to simplify code flow in // the caller, and errors are not important here anyway. func (am *ACMEIssuer) mostRecentAccountEmail(ctx context.Context, caURL string) (string, bool) { accountList, err := am.config.Storage.List(ctx, am.storageKeyUsersPrefix(caURL), false) if err != nil || len(accountList) == 0 { return "", false } // get all the key infos ahead of sorting, because // we might filter some out stats := make(map[string]KeyInfo) for i := 0; i < len(accountList); i++ { u := accountList[i] keyInfo, err := am.config.Storage.Stat(ctx, u) if err != nil { continue } if keyInfo.IsTerminal { // I found a bug when macOS created a .DS_Store file in // the users folder, and CertMagic tried to use that as // the user email because it was newer than the other one // which existed... sure, this isn't a perfect fix but // frankly one's OS shouldn't mess with the data folder // in the first place. accountList = append(accountList[:i], accountList[i+1:]...) i-- continue } stats[u] = keyInfo } sort.Slice(accountList, func(i, j int) bool { iInfo := stats[accountList[i]] jInfo := stats[accountList[j]] return jInfo.Modified.Before(iInfo.Modified) }) if len(accountList) == 0 { return "", false } account, err := am.loadOrCreateAccount(ctx, caURL, path.Base(accountList[0])) if err != nil { return "", false } return getPrimaryContact(account), true } func accountRegLockKey(acc acme.Account) string { key := "register_acme_account" if len(acc.Contact) == 0 { return key } key += "_" + getPrimaryContact(acc) return key } // getPrimaryContact returns the first contact on the account (if any) // without the scheme. (I guess we assume an email address.) func getPrimaryContact(account acme.Account) string { // TODO: should this be abstracted with some lower-level helper? var primaryContact string if len(account.Contact) > 0 { primaryContact = account.Contact[0] if idx := strings.Index(primaryContact, ":"); idx >= 0 { primaryContact = primaryContact[idx+1:] } } return primaryContact } // When an email address is not explicitly specified, we can remember // the last one we discovered to avoid having to ask again later. // (We used to store this in DefaultACME.Email but it was racey; see #127) var ( discoveredEmail string discoveredEmailMu sync.Mutex ) // stdin is used to read the user's input if prompted; // this is changed by tests during tests. var stdin = io.ReadWriter(os.Stdin) // The name of the folder for accounts where the email // address was not provided; default 'username' if you will, // but only for local/storage use, not with the CA. const emptyEmail = "default" golang-github-caddyserver-certmagic-0.25.2/account_test.go000066400000000000000000000344641514710434200236320ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bytes" "context" "io/fs" "os" "path/filepath" "reflect" "strings" "sync" "testing" "time" "go.uber.org/zap" ) // memoryStorage is an in-memory storage implementation with known contents *and* fixed iteration order for List. type memoryStorage struct { contents []memoryStorageItem } type memoryStorageItem struct { key string data []byte } func (m *memoryStorage) lookup(_ context.Context, key string) *memoryStorageItem { for _, item := range m.contents { if item.key == key { return &item } } return nil } func (m *memoryStorage) Delete(ctx context.Context, key string) error { for i, item := range m.contents { if item.key == key { m.contents = append(m.contents[:i], m.contents[i+1:]...) return nil } } return fs.ErrNotExist } func (m *memoryStorage) Store(ctx context.Context, key string, value []byte) error { m.contents = append(m.contents, memoryStorageItem{key: key, data: value}) return nil } func (m *memoryStorage) Exists(ctx context.Context, key string) bool { return m.lookup(ctx, key) != nil } func (m *memoryStorage) List(ctx context.Context, path string, recursive bool) ([]string, error) { if recursive { panic("unimplemented") } result := []string{} nextitem: for _, item := range m.contents { if !strings.HasPrefix(item.key, path+"/") { continue } name := strings.TrimPrefix(item.key, path+"/") if i := strings.Index(name, "/"); i >= 0 { name = name[:i] } for _, existing := range result { if existing == name { continue nextitem } } result = append(result, name) } return result, nil } func (m *memoryStorage) Load(ctx context.Context, key string) ([]byte, error) { if item := m.lookup(ctx, key); item != nil { return item.data, nil } return nil, fs.ErrNotExist } func (m *memoryStorage) Stat(ctx context.Context, key string) (KeyInfo, error) { if item := m.lookup(ctx, key); item != nil { return KeyInfo{Key: key, Size: int64(len(item.data))}, nil } return KeyInfo{}, fs.ErrNotExist } func (m *memoryStorage) Lock(ctx context.Context, name string) error { panic("unimplemented") } func (m *memoryStorage) Unlock(ctx context.Context, name string) error { panic("unimplemented") } var _ Storage = (*memoryStorage)(nil) type recordingStorage struct { Storage calls []recordedCall } func (r *recordingStorage) Delete(ctx context.Context, key string) error { r.record("Delete", key) return r.Storage.Delete(ctx, key) } func (r *recordingStorage) Exists(ctx context.Context, key string) bool { r.record("Exists", key) return r.Storage.Exists(ctx, key) } func (r *recordingStorage) List(ctx context.Context, path string, recursive bool) ([]string, error) { r.record("List", path, recursive) return r.Storage.List(ctx, path, recursive) } func (r *recordingStorage) Load(ctx context.Context, key string) ([]byte, error) { r.record("Load", key) return r.Storage.Load(ctx, key) } func (r *recordingStorage) Lock(ctx context.Context, name string) error { r.record("Lock", name) return r.Storage.Lock(ctx, name) } func (r *recordingStorage) Stat(ctx context.Context, key string) (KeyInfo, error) { r.record("Stat", key) return r.Storage.Stat(ctx, key) } func (r *recordingStorage) Store(ctx context.Context, key string, value []byte) error { r.record("Store", key) return r.Storage.Store(ctx, key, value) } func (r *recordingStorage) Unlock(ctx context.Context, name string) error { r.record("Unlock", name) return r.Storage.Unlock(ctx, name) } type recordedCall struct { name string args []any } func (r *recordingStorage) record(name string, args ...any) { r.calls = append(r.calls, recordedCall{name: name, args: args}) } var _ Storage = (*recordingStorage)(nil) func TestNewAccount(t *testing.T) { am := &ACMEIssuer{CA: dummyCA, Logger: zap.NewNop(), mu: new(sync.Mutex)} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata_tmp"}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig email := "me@foobar.com" account, err := am.newAccount(email) if err != nil { t.Fatalf("Error creating account: %v", err) } if account.PrivateKey == nil { t.Error("Private key is nil") } if account.Contact[0] != "mailto:"+email { t.Errorf("Expected email to be %s, but was %s", email, account.Contact[0]) } if account.Status != "" { t.Error("New account already has a status; it shouldn't") } } func TestSaveAccount(t *testing.T) { ctx := context.Background() am := &ACMEIssuer{CA: dummyCA, Logger: zap.NewNop(), mu: new(sync.Mutex)} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata1_tmp"}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() email := "me@foobar.com" account, err := am.newAccount(email) if err != nil { t.Fatalf("Error creating account: %v", err) } err = am.saveAccount(ctx, am.CA, account) if err != nil { t.Fatalf("Error saving account: %v", err) } _, err = am.loadOrCreateAccount(ctx, am.CA, email) if err != nil { t.Errorf("Cannot access account data, error: %v", err) } } func TestGetAccountDoesNotAlreadyExist(t *testing.T) { ctx := context.Background() am := &ACMEIssuer{CA: dummyCA, Logger: zap.NewNop(), mu: new(sync.Mutex)} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata_tmp"}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig account, err := am.loadOrCreateAccount(ctx, am.CA, "account_does_not_exist@foobar.com") if err != nil { t.Fatalf("Error getting account: %v", err) } if account.PrivateKey == nil { t.Error("Expected account to have a private key, but it was nil") } } func TestGetAccountAlreadyExists(t *testing.T) { ctx := context.Background() am := &ACMEIssuer{CA: dummyCA, Logger: zap.NewNop(), mu: new(sync.Mutex)} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata2_tmp"}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() email := "me@foobar.com" // Set up test account, err := am.newAccount(email) if err != nil { t.Fatalf("Error creating account: %v", err) } err = am.saveAccount(ctx, am.CA, account) if err != nil { t.Fatalf("Error saving account: %v", err) } // Expect to load account from disk loadedAccount, err := am.loadOrCreateAccount(ctx, am.CA, email) if err != nil { t.Fatalf("Error getting account: %v", err) } // Assert keys are the same if !privateKeysSame(account.PrivateKey, loadedAccount.PrivateKey) { t.Error("Expected private key to be the same after loading, but it wasn't") } // Assert emails are the same if !reflect.DeepEqual(account.Contact, loadedAccount.Contact) { t.Errorf("Expected contacts to be equal, but was '%s' before and '%s' after loading", account.Contact, loadedAccount.Contact) } } func TestGetAccountAlreadyExistsSkipsBroken(t *testing.T) { ctx := context.Background() am := &ACMEIssuer{CA: dummyCA, Logger: zap.NewNop(), mu: new(sync.Mutex)} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &memoryStorage{}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig email := "me@foobar.com" // Create a "corrupted" account am.config.Storage.Store(ctx, am.storageKeyUserReg(am.CA, "notmeatall@foobar.com"), []byte("this is not a valid account")) // Create the actual account account, err := am.newAccount(email) if err != nil { t.Fatalf("Error creating account: %v", err) } err = am.saveAccount(ctx, am.CA, account) if err != nil { t.Fatalf("Error saving account: %v", err) } // Expect to load account from disk keyBytes, err := PEMEncodePrivateKey(account.PrivateKey) if err != nil { t.Fatalf("Error encoding private key: %v", err) } loadedAccount, err := am.GetAccount(ctx, keyBytes) if err != nil { t.Fatalf("Error getting account: %v", err) } // Assert keys are the same if !privateKeysSame(account.PrivateKey, loadedAccount.PrivateKey) { t.Error("Expected private key to be the same after loading, but it wasn't") } // Assert emails are the same if !reflect.DeepEqual(account.Contact, loadedAccount.Contact) { t.Errorf("Expected contacts to be equal, but was '%s' before and '%s' after loading", account.Contact, loadedAccount.Contact) } } func TestGetAccountWithEmailAlreadyExists(t *testing.T) { ctx := context.Background() am := &ACMEIssuer{CA: dummyCA, Logger: zap.NewNop(), mu: new(sync.Mutex)} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &recordingStorage{Storage: &memoryStorage{}}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig email := "me@foobar.com" // Set up test account, err := am.newAccount(email) if err != nil { t.Fatalf("Error creating account: %v", err) } err = am.saveAccount(ctx, am.CA, account) if err != nil { t.Fatalf("Error saving account: %v", err) } // Set the expected email: am.Email = email err = am.setEmail(ctx, true) if err != nil { t.Fatalf("setEmail error: %v", err) } // Expect to load account from disk keyBytes, err := PEMEncodePrivateKey(account.PrivateKey) if err != nil { t.Fatalf("Error encoding private key: %v", err) } loadedAccount, err := am.GetAccount(ctx, keyBytes) if err != nil { t.Fatalf("Error getting account: %v", err) } // Assert keys are the same if !privateKeysSame(account.PrivateKey, loadedAccount.PrivateKey) { t.Error("Expected private key to be the same after loading, but it wasn't") } // Assert emails are the same if !reflect.DeepEqual(account.Contact, loadedAccount.Contact) { t.Errorf("Expected contacts to be equal, but was '%s' before and '%s' after loading", account.Contact, loadedAccount.Contact) } // Assert that this was found without listing all accounts rs := testConfig.Storage.(*recordingStorage) for _, call := range rs.calls { if call.name == "List" { t.Error("Unexpected List call") } } } func TestGetEmailFromPackageDefault(t *testing.T) { ctx := context.Background() DefaultACME.Email = "tEsT2@foo.com" defer func() { DefaultACME.Email = "" discoveredEmail = "" }() am := &ACMEIssuer{CA: dummyCA, Logger: zap.NewNop(), mu: new(sync.Mutex)} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata2_tmp"}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig err := am.setEmail(ctx, true) if err != nil { t.Fatalf("getEmail error: %v", err) } lowerEmail := strings.ToLower(DefaultACME.Email) if am.getEmail() != lowerEmail { t.Errorf("Did not get correct email from memory; expected '%s' but got '%s'", lowerEmail, am.Email) } } func TestGetEmailFromUserInput(t *testing.T) { ctx := context.Background() am := &ACMEIssuer{CA: dummyCA, Logger: zap.NewNop(), mu: new(sync.Mutex)} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata3_tmp"}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig // let's not clutter up the output origStdout := os.Stdout os.Stdout = nil agreementTestURL = "(none - testing)" defer func() { os.Stdout = origStdout agreementTestURL = "" }() email := "test3@foo.com" stdin = bytes.NewBufferString(email + "\n") err := am.setEmail(ctx, true) if err != nil { t.Fatalf("getEmail error: %v", err) } if am.email != email { t.Errorf("Did not get correct email from user input prompt; expected '%s' but got '%s'", email, am.Email) } if !am.isAgreed() { t.Error("Expect Config.agreed to be true, but got false") } } func TestGetEmailFromRecent(t *testing.T) { ctx := context.Background() am := &ACMEIssuer{CA: dummyCA, Logger: zap.NewNop(), mu: new(sync.Mutex)} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata4_tmp"}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() DefaultACME.Email = "" discoveredEmail = "" for i, eml := range []string{ "test4-1@foo.com", "test4-2@foo.com", "TEST4-3@foo.com", // test case insensitivity } { account, err := am.newAccount(eml) if err != nil { t.Fatalf("Error creating user %d: %v", i, err) } err = am.saveAccount(ctx, am.CA, account) if err != nil { t.Fatalf("Error saving user %d: %v", i, err) } // Change modified time so they're all different and the test becomes more deterministic fs := testConfig.Storage.(*FileStorage) userFolder := filepath.Join(fs.Path, am.storageKeyUserPrefix(am.CA, eml)) f, err := os.Stat(userFolder) if err != nil { t.Fatalf("Could not access user folder for '%s': %v", eml, err) } chTime := f.ModTime().Add(time.Duration(i) * time.Hour) // 1 second isn't always enough spacing! if err := os.Chtimes(userFolder, chTime, chTime); err != nil { t.Fatalf("Could not change user folder mod time for '%s': %v", eml, err) } } err := am.setEmail(ctx, true) if err != nil { t.Fatalf("getEmail error: %v", err) } if am.getEmail() != "test4-3@foo.com" { t.Errorf("Did not get correct email from storage; expected '%s' but got '%s'", "test4-3@foo.com", am.Email) } } // agreementTestURL is set during tests to skip requiring // setting up an entire ACME CA endpoint. var agreementTestURL string golang-github-caddyserver-certmagic-0.25.2/acmeclient.go000066400000000000000000000334011514710434200232310ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/x509" "fmt" "log/slog" "net" "net/http" "net/url" "strconv" "strings" "sync" "time" "github.com/mholt/acmez/v3" "github.com/mholt/acmez/v3/acme" "go.uber.org/zap" "go.uber.org/zap/exp/zapslog" ) // acmeClient holds state necessary to perform ACME operations // for certificate management with an ACME account. Call // ACMEIssuer.newACMEClientWithAccount() to get a valid one. type acmeClient struct { iss *ACMEIssuer acmeClient *acmez.Client account acme.Account } // newACMEClientWithAccount creates an ACME client ready to use with an account, including // loading one from storage or registering a new account with the CA if necessary. If // useTestCA is true, am.TestCA will be used if set; otherwise, the primary CA will be used. func (iss *ACMEIssuer) newACMEClientWithAccount(ctx context.Context, useTestCA, interactive bool) (*acmeClient, error) { // first, get underlying ACME client client, err := iss.newACMEClient(useTestCA) if err != nil { return nil, err } // we try loading the account from storage before a potential // lock, and after obtaining the lock as well, to ensure we don't // repeat work done by another instance or goroutine account, err := iss.getAccountToUse(ctx, client.Directory) if err != nil { return nil, err } // register account if it is new if account.Status == "" { iss.Logger.Info("ACME account has empty status; registering account with ACME server", zap.Strings("contact", account.Contact), zap.String("location", account.Location)) // synchronize this so the account is only created once acctLockKey := accountRegLockKey(account) err = acquireLock(ctx, iss.config.Storage, acctLockKey) if err != nil { return nil, fmt.Errorf("locking account registration: %v", err) } defer func() { if err := releaseLock(ctx, iss.config.Storage, acctLockKey); err != nil { iss.Logger.Error("failed to unlock account registration lock", zap.Error(err)) } }() // if we're not the only one waiting for this account, then by this point it should already be registered and in storage; reload it account, err = iss.getAccountToUse(ctx, client.Directory) if err != nil { return nil, err } // if we are the only or first one waiting for this account, then proceed to register it while we have the lock if account.Status == "" { if iss.NewAccountFunc != nil { // obtain lock here, since NewAccountFunc calls happen concurrently and they typically read and change the issuer iss.mu.Lock() account, err = iss.NewAccountFunc(ctx, iss, account) iss.mu.Unlock() if err != nil { return nil, fmt.Errorf("account pre-registration callback: %v", err) } } // agree to terms if interactive { if !iss.isAgreed() { var termsURL string dir, err := client.GetDirectory(ctx) if err != nil { return nil, fmt.Errorf("getting directory: %w", err) } if dir.Meta != nil { termsURL = dir.Meta.TermsOfService } if termsURL != "" { agreed := iss.askUserAgreement(termsURL) if !agreed { return nil, fmt.Errorf("user must agree to CA terms") } iss.mu.Lock() iss.agreed = agreed iss.mu.Unlock() } } } else { // can't prompt a user who isn't there; they should // have reviewed the terms beforehand iss.mu.Lock() iss.agreed = true iss.mu.Unlock() } account.TermsOfServiceAgreed = iss.isAgreed() // associate account with external binding, if configured if iss.ExternalAccount != nil { err := account.SetExternalAccountBinding(ctx, client.Client, *iss.ExternalAccount) if err != nil { return nil, err } } // create account account, err = client.NewAccount(ctx, account) if err != nil { return nil, fmt.Errorf("registering account %v with server: %w", account.Contact, err) } iss.Logger.Info("new ACME account registered", zap.Strings("contact", account.Contact), zap.String("status", account.Status)) // persist the account to storage err = iss.saveAccount(ctx, client.Directory, account) if err != nil { return nil, fmt.Errorf("could not save account %v: %v", account.Contact, err) } } else { iss.Logger.Info("account has already been registered; reloaded", zap.Strings("contact", account.Contact), zap.String("status", account.Status), zap.String("location", account.Location)) } } c := &acmeClient{ iss: iss, acmeClient: client, account: account, } return c, nil } // newACMEClient creates a new underlying ACME client using the settings in am, // independent of any particular ACME account. If useTestCA is true, am.TestCA // will be used if it is set; otherwise, the primary CA will be used. func (iss *ACMEIssuer) newACMEClient(useTestCA bool) (*acmez.Client, error) { client, err := iss.newBasicACMEClient() if err != nil { return nil, err } // fill in a little more beyond a basic client if useTestCA && iss.TestCA != "" { client.Client.Directory = iss.TestCA } certObtainTimeout := iss.CertObtainTimeout if certObtainTimeout == 0 { certObtainTimeout = DefaultACME.CertObtainTimeout } client.Client.PollTimeout = certObtainTimeout client.ChallengeSolvers = make(map[string]acmez.Solver) // configure challenges (most of the time, DNS challenge is // exclusive of other ones because it is usually only used // in situations where the default challenges would fail) if iss.DNS01Solver == nil { // enable HTTP-01 challenge if !iss.DisableHTTPChallenge { var solver acmez.Solver = &httpSolver{ handler: iss.HTTPChallengeHandler(http.NewServeMux()), address: net.JoinHostPort(iss.ListenHost, strconv.Itoa(iss.getHTTPPort())), } if !iss.DisableDistributedSolvers { solver = distributedSolver{ storage: iss.config.Storage, storageKeyIssuerPrefix: iss.storageKeyCAPrefix(client.Directory), solver: solver, } } client.ChallengeSolvers[acme.ChallengeTypeHTTP01] = solver } // enable TLS-ALPN-01 challenge if !iss.DisableTLSALPNChallenge { var solver acmez.Solver = &tlsALPNSolver{ config: iss.config, address: net.JoinHostPort(iss.ListenHost, strconv.Itoa(iss.getTLSALPNPort())), } if !iss.DisableDistributedSolvers { solver = distributedSolver{ storage: iss.config.Storage, storageKeyIssuerPrefix: iss.storageKeyCAPrefix(client.Directory), solver: solver, } } client.ChallengeSolvers[acme.ChallengeTypeTLSALPN01] = solver } } else { // use DNS challenge exclusively client.ChallengeSolvers[acme.ChallengeTypeDNS01] = iss.DNS01Solver } // wrap solvers in our wrapper so that we can keep track of challenge // info: this is useful for solving challenges globally as a process; // for example, usually there is only one process that can solve the // HTTP and TLS-ALPN challenges, and only one server in that process // that can bind the necessary port(s), so if a server listening on // a different port needed a certificate, it would have to know about // the other server listening on that port, and somehow convey its // challenge info or share its config, but this isn't always feasible; // what the wrapper does is it accesses a global challenge memory so // that unrelated servers in this process can all solve each others' // challenges without having to know about each other - Caddy's admin // endpoint uses this functionality since it and the HTTP/TLS modules // do not know about each other // (doing this here in a separate loop ensures that even if we expose // solver config to users later, we will even wrap their own solvers) for name, solver := range client.ChallengeSolvers { client.ChallengeSolvers[name] = solverWrapper{solver} } return client, nil } // newBasicACMEClient sets up a basically-functional ACME client that is not capable // of solving challenges but can provide basic interactions with the server. func (iss *ACMEIssuer) newBasicACMEClient() (*acmez.Client, error) { caURL := iss.CA if caURL == "" { caURL = DefaultACME.CA } // ensure endpoint is secure (assume HTTPS if scheme is missing) if !strings.Contains(caURL, "://") { caURL = "https://" + caURL } u, err := url.Parse(caURL) if err != nil { return nil, err } if u.Scheme != "https" && !SubjectIsInternal(u.Host) { return nil, fmt.Errorf("%s: insecure CA URL (HTTPS required for non-internal CA)", caURL) } return &acmez.Client{ Client: &acme.Client{ Directory: caURL, UserAgent: buildUAString(), HTTPClient: iss.httpClient, Logger: slog.New(zapslog.NewHandler(iss.Logger.Named("acme_client").Core())), }, }, nil } // GetRenewalInfo gets the ACME Renewal Information (ARI) for the certificate. func (iss *ACMEIssuer) GetRenewalInfo(ctx context.Context, cert Certificate) (acme.RenewalInfo, error) { acmeClient, err := iss.newBasicACMEClient() if err != nil { return acme.RenewalInfo{}, err } return acmeClient.GetRenewalInfo(ctx, cert.Certificate.Leaf) } func (iss *ACMEIssuer) getHTTPPort() int { useHTTPPort := HTTPChallengePort if HTTPPort > 0 && HTTPPort != HTTPChallengePort { useHTTPPort = HTTPPort } if iss.AltHTTPPort > 0 { useHTTPPort = iss.AltHTTPPort } return useHTTPPort } func (iss *ACMEIssuer) getTLSALPNPort() int { useTLSALPNPort := TLSALPNChallengePort if HTTPSPort > 0 && HTTPSPort != TLSALPNChallengePort { useTLSALPNPort = HTTPSPort } if iss.AltTLSALPNPort > 0 { useTLSALPNPort = iss.AltTLSALPNPort } return useTLSALPNPort } func (c *acmeClient) throttle(ctx context.Context, names []string) error { email := c.iss.getEmail() // throttling is scoped to CA + account email rateLimiterKey := c.acmeClient.Directory + "," + email rateLimitersMu.Lock() rl, ok := rateLimiters[rateLimiterKey] if !ok { rl = NewRateLimiter(RateLimitEvents, RateLimitEventsWindow) rateLimiters[rateLimiterKey] = rl // TODO: stop rate limiter when it is garbage-collected... } rateLimitersMu.Unlock() c.iss.Logger.Info("waiting on internal rate limiter", zap.Strings("identifiers", names), zap.String("ca", c.acmeClient.Directory), zap.String("account", email), ) err := rl.Wait(ctx) if err != nil { return err } c.iss.Logger.Info("done waiting on internal rate limiter", zap.Strings("identifiers", names), zap.String("ca", c.acmeClient.Directory), zap.String("account", email), ) return nil } func (c *acmeClient) usingTestCA() bool { return c.iss.TestCA != "" && c.acmeClient.Directory == c.iss.TestCA } func (c *acmeClient) revoke(ctx context.Context, cert *x509.Certificate, reason int) error { return c.acmeClient.RevokeCertificate(ctx, c.account, cert, c.account.PrivateKey, reason) } func buildUAString() string { ua := "CertMagic" if UserAgent != "" { ua = UserAgent + " " + ua } return ua } // RenewalInfoGetter is a type that can get ACME Renewal Information (ARI). // Users of this package that wrap the ACMEIssuer or use any other issuer // that supports ARI will need to implement this so that CertMagic can // update ARI which happens outside the normal issuance flow and is thus // not required by the Issuer interface (a type assertion is performed). type RenewalInfoGetter interface { GetRenewalInfo(context.Context, Certificate) (acme.RenewalInfo, error) } // These internal rate limits are designed to prevent accidentally // firehosing a CA's ACME endpoints. They are not intended to // replace or replicate the CA's actual rate limits. // // Let's Encrypt's rate limits can be found here: // https://letsencrypt.org/docs/rate-limits/ // // Currently (as of December 2019), Let's Encrypt's most relevant // rate limit for large deployments is 300 new orders per account // per 3 hours (on average, or best case, that's about 1 every 36 // seconds, or 2 every 72 seconds, etc.); but it's not reasonable // to try to assume that our internal state is the same as the CA's // (due to process restarts, config changes, failed validations, // etc.) and ultimately, only the CA's actual rate limiter is the // authority. Thus, our own rate limiters do not attempt to enforce // external rate limits. Doing so causes problems when the domains // are not in our control (i.e. serving customer sites) and/or lots // of domains fail validation: they clog our internal rate limiter // and nearly starve out (or at least slow down) the other domains // that need certificates. Failed transactions are already retried // with exponential backoff, so adding in rate limiting can slow // things down even more. // // Instead, the point of our internal rate limiter is to avoid // hammering the CA's endpoint when there are thousands or even // millions of certificates under management. Our goal is to // allow small bursts in a relatively short timeframe so as to // not block any one domain for too long, without unleashing // thousands of requests to the CA at once. var ( rateLimiters = make(map[string]*RingBufferRateLimiter) rateLimitersMu sync.RWMutex // RateLimitEvents is how many new events can be allowed // in RateLimitEventsWindow. RateLimitEvents = 10 // RateLimitEventsWindow is the size of the sliding // window that throttles events. RateLimitEventsWindow = 10 * time.Second ) // Some default values passed down to the underlying ACME client. var ( UserAgent string HTTPTimeout = 30 * time.Second ) golang-github-caddyserver-certmagic-0.25.2/acmeissuer.go000066400000000000000000000565051514710434200232770ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/tls" "crypto/x509" "errors" "fmt" "net" "net/http" "net/url" "sort" "strings" "sync" "time" "github.com/mholt/acmez/v3" "github.com/mholt/acmez/v3/acme" "go.uber.org/zap" ) // ACMEIssuer gets certificates using ACME. It implements the PreChecker, // Issuer, and Revoker interfaces. // // It is NOT VALID to use an ACMEIssuer without calling NewACMEIssuer(). // It fills in any default values from DefaultACME as well as setting up // internal state that is necessary for valid use. Always call // NewACMEIssuer() to get a valid ACMEIssuer value. type ACMEIssuer struct { // The endpoint of the directory for the ACME // CA we are to use CA string // TestCA is the endpoint of the directory for // an ACME CA to use to test domain validation, // but any certs obtained from this CA are // discarded; it should perform real and valid // ACME verifications, but probably should not // issue real, publicly-trusted certificates TestCA string // The email address to use when creating or // selecting an existing ACME server account Email string // The PEM-encoded private key of the ACME // account to use; only needed if the account // is already created on the server and // can be looked up with the ACME protocol AccountKeyPEM string // Set to true if agreed to the CA's // subscriber agreement Agreed bool // An optional external account to associate // with this ACME account ExternalAccount *acme.EAB // Optionally select an ACME profile offered // by the ACME server. The list of supported // profile names can be obtained from the ACME // server's directory endpoint. For details: // https://datatracker.ietf.org/doc/draft-aaron-acme-profiles/ // // (EXPERIMENTAL: Subject to change.) Profile string // Optionally specify the validity period of // the certificate(s) here as offsets from the // approximate time of certificate issuance, // but note that not all CAs support this // (EXPERIMENTAL: Subject to change) NotBefore, NotAfter time.Duration // Disable all HTTP challenges DisableHTTPChallenge bool // Disable all TLS-ALPN challenges DisableTLSALPNChallenge bool // Disable distributed solving; avoids writing // challenge info to storage backend and will // only use data in memory to solve the HTTP and // TLS-ALPN challenges; will still attempt to // solve distributed HTTP challenges blindly by // using available account and challenge token // as read from request URI DisableDistributedSolvers bool // The host (ONLY the host, not port) to listen // on if necessary to start a listener to solve // an ACME challenge ListenHost string // The alternate port to use for the ACME HTTP // challenge; if non-empty, this port will be // used instead of HTTPChallengePort to spin up // a listener for the HTTP challenge AltHTTPPort int // The alternate port to use for the ACME // TLS-ALPN challenge; the system must forward // TLSALPNChallengePort to this port for // challenge to succeed AltTLSALPNPort int // The solver for the dns-01 challenge; // usually this is a DNS01Solver value // from this package DNS01Solver acmez.Solver // TrustedRoots specifies a pool of root CA // certificates to trust when communicating // over a network to a peer. TrustedRoots *x509.CertPool // The maximum amount of time to allow for // obtaining a certificate. If empty, the // default from the underlying ACME lib is // used. If set, it must not be too low so // as to cancel challenges too early. CertObtainTimeout time.Duration // Address of custom DNS resolver to be used // when communicating with ACME server Resolver string // Callback function that is called before a // new ACME account is registered with the CA; // it allows for last-second config changes // of the ACMEIssuer and the Account. // (TODO: this feature is still EXPERIMENTAL and subject to change) NewAccountFunc func(context.Context, *ACMEIssuer, acme.Account) (acme.Account, error) // Preferences for selecting alternate // certificate chains PreferredChains ChainPreference // Set a logger to configure logging; a default // logger must always be set; if no logging is // desired, set this to zap.NewNop(). Logger *zap.Logger // Set a http proxy to use when issuing a certificate. // Default is http.ProxyFromEnvironment HTTPProxy func(*http.Request) (*url.URL, error) config *Config httpClient *http.Client // Some fields are changed on-the-fly during // certificate management. For example, the // email might be implicitly discovered if not // explicitly configured, and agreement might // happen during the flow. Changing the exported // fields field is racey (issue #195) so we // control unexported fields that we can // synchronize properly. email string agreed bool mu *sync.Mutex // protects the above grouped fields, as well as entire struct during NewAccountFunc calls } // NewACMEIssuer constructs a valid ACMEIssuer based on a template // configuration; any empty values will be filled in by defaults in // DefaultACME, and if any required values are still empty, sensible // defaults will be used. // // Typically, you'll create the Config first with New() or NewDefault(), // then call NewACMEIssuer(), then assign the return value to the Issuers // field of the Config. func NewACMEIssuer(cfg *Config, template ACMEIssuer) *ACMEIssuer { if cfg == nil { panic("cannot make valid ACMEIssuer without an associated CertMagic config") } if template.CA == "" { template.CA = DefaultACME.CA } if template.TestCA == "" && template.CA == DefaultACME.CA { // only use the default test CA if the CA is also // the default CA; no point in testing against // Let's Encrypt's staging server if we are not // using their production server too template.TestCA = DefaultACME.TestCA } if template.Email == "" { template.Email = DefaultACME.Email } if template.AccountKeyPEM == "" { template.AccountKeyPEM = DefaultACME.AccountKeyPEM } if !template.Agreed { template.Agreed = DefaultACME.Agreed } if template.ExternalAccount == nil { template.ExternalAccount = DefaultACME.ExternalAccount } if template.NotBefore == 0 { template.NotBefore = DefaultACME.NotBefore } if template.NotAfter == 0 { template.NotAfter = DefaultACME.NotAfter } if !template.DisableHTTPChallenge { template.DisableHTTPChallenge = DefaultACME.DisableHTTPChallenge } if !template.DisableTLSALPNChallenge { template.DisableTLSALPNChallenge = DefaultACME.DisableTLSALPNChallenge } if template.ListenHost == "" { template.ListenHost = DefaultACME.ListenHost } if template.AltHTTPPort == 0 { template.AltHTTPPort = DefaultACME.AltHTTPPort } if template.AltTLSALPNPort == 0 { template.AltTLSALPNPort = DefaultACME.AltTLSALPNPort } if template.DNS01Solver == nil { template.DNS01Solver = DefaultACME.DNS01Solver } if template.TrustedRoots == nil { template.TrustedRoots = DefaultACME.TrustedRoots } if template.CertObtainTimeout == 0 { template.CertObtainTimeout = DefaultACME.CertObtainTimeout } if template.Resolver == "" { template.Resolver = DefaultACME.Resolver } if template.NewAccountFunc == nil { template.NewAccountFunc = DefaultACME.NewAccountFunc } if template.Logger == nil { template.Logger = DefaultACME.Logger } // absolutely do not allow a nil logger; that would panic if template.Logger == nil { template.Logger = defaultLogger } if template.HTTPProxy == nil { template.HTTPProxy = DefaultACME.HTTPProxy } if template.HTTPProxy == nil { template.HTTPProxy = http.ProxyFromEnvironment } template.config = cfg template.mu = new(sync.Mutex) // set up the dialer and transport / HTTP client dialer := &net.Dialer{ Timeout: 30 * time.Second, KeepAlive: 2 * time.Minute, } if template.Resolver != "" { dialer.Resolver = &net.Resolver{ PreferGo: true, Dial: func(ctx context.Context, network, _ string) (net.Conn, error) { return (&net.Dialer{ Timeout: 15 * time.Second, }).DialContext(ctx, network, template.Resolver) }, } } transport := &http.Transport{ Proxy: template.HTTPProxy, DialContext: dialer.DialContext, TLSHandshakeTimeout: 30 * time.Second, // increase to 30s requested in #175 ResponseHeaderTimeout: 30 * time.Second, // increase to 30s requested in #175 ExpectContinueTimeout: 2 * time.Second, ForceAttemptHTTP2: true, } if template.TrustedRoots != nil { transport.TLSClientConfig = &tls.Config{ RootCAs: template.TrustedRoots, } } template.httpClient = &http.Client{ Transport: transport, Timeout: HTTPTimeout, } return &template } // IssuerKey returns the unique issuer key for the // configured CA endpoint. func (am *ACMEIssuer) IssuerKey() string { return am.issuerKey(am.CA) } func (*ACMEIssuer) issuerKey(ca string) string { key := ca if caURL, err := url.Parse(key); err == nil { key = caURL.Host if caURL.Path != "" { // keep the path, but make sure it's a single // component (i.e. no forward slashes, and for // good measure, no backward slashes either) const hyphen = "-" repl := strings.NewReplacer( "/", hyphen, "\\", hyphen, ) path := strings.Trim(repl.Replace(caURL.Path), hyphen) if path != "" { key += hyphen + path } } } return key } func (iss *ACMEIssuer) getEmail() string { iss.mu.Lock() defer iss.mu.Unlock() return iss.email } func (iss *ACMEIssuer) isAgreed() bool { iss.mu.Lock() defer iss.mu.Unlock() return iss.agreed } // PreCheck performs a few simple checks before obtaining or // renewing a certificate with ACME, and returns whether this // batch is eligible for certificates. It also ensures that an // email address is available if possible. // // IP certificates via ACME are defined in RFC 8738. func (am *ACMEIssuer) PreCheck(ctx context.Context, names []string, interactive bool) error { publicCAsAndIPCerts := map[string]bool{ // map of public CAs to whether they support IP certificates (last updated: Q1 2024) "api.letsencrypt.org": true, // https://letsencrypt.org/2025/07/01/issuing-our-first-ip-address-certificate/ "acme.zerossl.com": false, // only supported via their API, not ACME endpoint "api.pki.goog": true, // https://pki.goog/faq/#faq-IPCerts "api.buypass.com": false, // https://community.buypass.com/t/h7hm76w/buypass-support-for-rfc-8738 "acme.ssl.com": false, } var publicCA, ipCertAllowed bool for caSubstr, ipCert := range publicCAsAndIPCerts { if strings.Contains(am.CA, caSubstr) { publicCA, ipCertAllowed = true, ipCert break } } if publicCA { for _, name := range names { if !SubjectQualifiesForPublicCert(name) { return fmt.Errorf("subject '%s' does not qualify for a public certificate", name) } if !ipCertAllowed && SubjectIsIP(name) { return fmt.Errorf("subject '%s' cannot have public IP certificate from %s (if CA's policy has changed, please notify the developers in an issue)", name, am.CA) } } } return am.setEmail(ctx, interactive) } // Issue implements the Issuer interface. It obtains a certificate for the given csr using // the ACME configuration am. func (am *ACMEIssuer) Issue(ctx context.Context, csr *x509.CertificateRequest) (*IssuedCertificate, error) { if am.config == nil { panic("missing config pointer (must use NewACMEIssuer)") } var attempts int if attemptsPtr, ok := ctx.Value(AttemptsCtxKey).(*int); ok { attempts = *attemptsPtr } isRetry := attempts > 0 cert, usedTestCA, err := am.doIssue(ctx, csr, attempts) if err != nil { return nil, err } // important to note that usedTestCA is not necessarily the same as isRetry // (usedTestCA can be true if the main CA and the test CA happen to be the same) if isRetry && usedTestCA && am.CA != am.TestCA { // succeeded with testing endpoint, so try again with production endpoint // (only if the production endpoint is different from the testing endpoint) // TODO: This logic is imperfect and could benefit from some refinement. // The two CA endpoints likely have different states, which could cause one // to succeed and the other to fail, even if it's not a validation error. // Two common cases would be: // 1) Rate limiter state. This is more likely to cause prod to fail while // staging succeeds, since prod usually has tighter rate limits. Thus, if // initial attempt failed in prod due to rate limit, first retry (on staging) // might succeed, and then trying prod again right way would probably still // fail; normally this would terminate retries but the right thing to do in // this case is to back off and retry again later. We could refine this logic // to stick with the production endpoint on retries unless the error changes. // 2) Cached authorizations state. If a domain validates successfully with // one endpoint, but then the other endpoint is used, it might fail, e.g. if // DNS was just changed or is still propagating. In this case, the second CA // should continue to be retried with backoff, without switching back to the // other endpoint. This is more likely to happen if a user is testing with // the staging CA as the main CA, then changes their configuration once they // think they are ready for the production endpoint. cert, _, err = am.doIssue(ctx, csr, 0) if err != nil { // succeeded with test CA but failed just now with the production CA; // either we are observing differing internal states of each CA that will // work out with time, or there is a bug/misconfiguration somewhere // externally; it is hard to tell which! one easy cue is whether the // error is specifically a 429 (Too Many Requests); if so, we should // probably keep retrying var problem acme.Problem if errors.As(err, &problem) { if problem.Status == http.StatusTooManyRequests { // DON'T abort retries; the test CA succeeded (even // if it's cached, it recently succeeded!) so we just // need to keep trying (with backoff) until this CA's // rate limits expire... // TODO: as mentioned in comment above, we would benefit // by pinning the main CA at this point instead of // needlessly retrying with the test CA first each time return nil, err } } return nil, ErrNoRetry{err} } } return cert, err } func (am *ACMEIssuer) doIssue(ctx context.Context, csr *x509.CertificateRequest, attempts int) (*IssuedCertificate, bool, error) { useTestCA := attempts > 0 client, err := am.newACMEClientWithAccount(ctx, useTestCA, false) if err != nil { return nil, false, err } usingTestCA := client.usingTestCA() nameSet := namesFromCSR(csr) if !useTestCA { if err := client.throttle(ctx, nameSet); err != nil { return nil, usingTestCA, err } } params, err := acmez.OrderParametersFromCSR(client.account, csr) if err != nil { return nil, false, fmt.Errorf("generating order parameters from CSR: %v", err) } if am.NotBefore != 0 { params.NotBefore = time.Now().Add(am.NotBefore) } if am.NotAfter != 0 { params.NotAfter = time.Now().Add(am.NotAfter) } params.Profile = am.Profile // Notify the ACME server we are replacing a certificate (if the caller says we are), // only if the following conditions are met: // - The caller has set a Replaces value in the context, indicating this is a renewal. // - Not using test CA. This should be obvious, but a test CA should be in a separate // environment from production, and thus not have knowledge of the cert being replaced. // - Not a certain attempt number. We skip setting Replaces once early on in the retries // in case the reason the order is failing is only because there is a state inconsistency // between client and server or some sort of bookkeeping error with regards to the certID // and the server is rejecting the ARI certID. In any case, an invalid certID may cause // orders to fail. So try once without setting it. if !am.config.DisableARI && !usingTestCA && attempts != 2 { if replacing, ok := ctx.Value(ctxKeyARIReplaces).(*x509.Certificate); ok { params.Replaces = replacing } } // do this in a loop because there's an error case that may necessitate a retry, but not more than once var certChains []acme.Certificate for range 2 { am.Logger.Info("using ACME account", zap.String("account_id", params.Account.Location), zap.Strings("account_contact", params.Account.Contact)) certChains, err = client.acmeClient.ObtainCertificate(ctx, params) if err != nil { var prob acme.Problem if errors.As(err, &prob) && prob.Type == acme.ProblemTypeAccountDoesNotExist { am.Logger.Warn("ACME account does not exist on server; attempting to recreate", zap.String("account_id", client.account.Location), zap.Strings("account_contact", client.account.Contact), zap.String("key_location", am.storageKeyUserPrivateKey(client.acmeClient.Directory, am.getEmail())), zap.Any("problem", prob)) // the account we have no longer exists on the CA, so we need to create a new one; // we could use the same key pair, but this is a good opportunity to rotate keys // (see https://caddy.community/t/acme-account-is-not-regenerated-when-acme-server-gets-reinstalled/22627) // (basically this happens if the CA gets reset or reinstalled; usually just internal PKI) err := am.deleteAccountLocally(ctx, client.iss.CA, client.account) if err != nil { return nil, usingTestCA, fmt.Errorf("%v ACME account no longer exists on CA, but resetting our local copy of the account info failed: %v", nameSet, err) } // recreate account and try again client, err = am.newACMEClientWithAccount(ctx, useTestCA, false) if err != nil { return nil, false, err } continue } return nil, usingTestCA, fmt.Errorf("%v %w (ca=%s)", nameSet, err, client.acmeClient.Directory) } break } if len(certChains) == 0 { return nil, usingTestCA, fmt.Errorf("no certificate chains") } preferredChain := am.selectPreferredChain(certChains) ic := &IssuedCertificate{ Certificate: preferredChain.ChainPEM, Metadata: preferredChain, } am.Logger.Debug("selected certificate chain", zap.String("url", preferredChain.URL)) return ic, usingTestCA, nil } // selectPreferredChain sorts and then filters the certificate chains to find the optimal // chain preferred by the client. If there's only one chain, that is returned without any // processing. If there are no matches, the first chain is returned. func (am *ACMEIssuer) selectPreferredChain(certChains []acme.Certificate) acme.Certificate { if len(certChains) == 1 { if len(am.PreferredChains.AnyCommonName) > 0 || len(am.PreferredChains.RootCommonName) > 0 { am.Logger.Debug("there is only one chain offered; selecting it regardless of preferences", zap.String("chain_url", certChains[0].URL)) } return certChains[0] } if am.PreferredChains.Smallest != nil { if *am.PreferredChains.Smallest { sort.Slice(certChains, func(i, j int) bool { return len(certChains[i].ChainPEM) < len(certChains[j].ChainPEM) }) } else { sort.Slice(certChains, func(i, j int) bool { return len(certChains[i].ChainPEM) > len(certChains[j].ChainPEM) }) } } if len(am.PreferredChains.AnyCommonName) > 0 || len(am.PreferredChains.RootCommonName) > 0 { // in order to inspect, we need to decode their PEM contents decodedChains := make([][]*x509.Certificate, len(certChains)) for i, chain := range certChains { certs, err := parseCertsFromPEMBundle(chain.ChainPEM) if err != nil { am.Logger.Error("unable to parse PEM certificate chain", zap.Int("chain", i), zap.Error(err)) continue } decodedChains[i] = certs } if len(am.PreferredChains.AnyCommonName) > 0 { for _, prefAnyCN := range am.PreferredChains.AnyCommonName { for i, chain := range decodedChains { for _, cert := range chain { if cert.Issuer.CommonName == prefAnyCN { am.Logger.Debug("found preferred certificate chain by issuer common name", zap.String("preference", prefAnyCN), zap.Int("chain", i)) return certChains[i] } } } } } if len(am.PreferredChains.RootCommonName) > 0 { for _, prefRootCN := range am.PreferredChains.RootCommonName { for i, chain := range decodedChains { if chain[len(chain)-1].Issuer.CommonName == prefRootCN { am.Logger.Debug("found preferred certificate chain by root common name", zap.String("preference", prefRootCN), zap.Int("chain", i)) return certChains[i] } } } } am.Logger.Warn("did not find chain matching preferences; using first") } return certChains[0] } // Revoke implements the Revoker interface. It revokes the given certificate. func (am *ACMEIssuer) Revoke(ctx context.Context, cert CertificateResource, reason int) error { client, err := am.newACMEClientWithAccount(ctx, false, false) if err != nil { return err } certs, err := parseCertsFromPEMBundle(cert.CertificatePEM) if err != nil { return err } return client.revoke(ctx, certs[0], reason) } // ChainPreference describes the client's preferred certificate chain, // useful if the CA offers alternate chains. The first matching chain // will be selected. type ChainPreference struct { // Prefer chains with the fewest number of bytes. Smallest *bool // Select first chain having a root with one of // these common names. RootCommonName []string // Select first chain that has any issuer with one // of these common names. AnyCommonName []string } // DefaultACME specifies default settings to use for ACMEIssuers. // Using this value is optional but can be convenient. var DefaultACME = ACMEIssuer{ CA: LetsEncryptProductionCA, TestCA: LetsEncryptStagingCA, Logger: defaultLogger, HTTPProxy: http.ProxyFromEnvironment, } // Some well-known CA endpoints available to use. See // the documentation for each service; some may require // External Account Binding (EAB) and possibly payment. // COMPATIBILITY NOTICE: These constants refer to external // resources and are thus subject to change or removal // without a major version bump. const ( LetsEncryptStagingCA = "https://acme-staging-v02.api.letsencrypt.org/directory" // https://letsencrypt.org/docs/staging-environment/ LetsEncryptProductionCA = "https://acme-v02.api.letsencrypt.org/directory" // https://letsencrypt.org/getting-started/ ZeroSSLProductionCA = "https://acme.zerossl.com/v2/DV90" // https://zerossl.com/documentation/acme/ GoogleTrustStagingCA = "https://dv.acme-v02.test-api.pki.goog/directory" // https://cloud.google.com/certificate-manager/docs/public-ca-tutorial GoogleTrustProductionCA = "https://dv.acme-v02.api.pki.goog/directory" // https://cloud.google.com/certificate-manager/docs/public-ca-tutorial ) // prefixACME is the storage key prefix used for ACME-specific assets. const prefixACME = "acme" type ctxKey string const ctxKeyARIReplaces = ctxKey("ari_replaces") // Interface guards var ( _ PreChecker = (*ACMEIssuer)(nil) _ Issuer = (*ACMEIssuer)(nil) _ Revoker = (*ACMEIssuer)(nil) ) golang-github-caddyserver-certmagic-0.25.2/acmeissuer_test.go000066400000000000000000000001101514710434200243130ustar00rootroot00000000000000package certmagic const dummyCA = "https://example.com/acme/directory" golang-github-caddyserver-certmagic-0.25.2/async.go000066400000000000000000000120231514710434200222370ustar00rootroot00000000000000package certmagic import ( "context" "errors" "log" "runtime" "sync" "time" "go.uber.org/zap" ) var jm = &jobManager{maxConcurrentJobs: 1000} type jobManager struct { mu sync.Mutex maxConcurrentJobs int activeWorkers int queue []namedJob names map[string]struct{} } type namedJob struct { name string job func() error logger *zap.Logger } // Submit enqueues the given job with the given name. If name is non-empty // and a job with the same name is already enqueued or running, this is a // no-op. If name is empty, no duplicate prevention will occur. The job // manager will then run this job as soon as it is able. func (jm *jobManager) Submit(logger *zap.Logger, name string, job func() error) { jm.mu.Lock() defer jm.mu.Unlock() if jm.names == nil { jm.names = make(map[string]struct{}) } if name != "" { // prevent duplicate jobs if _, ok := jm.names[name]; ok { return } jm.names[name] = struct{}{} } jm.queue = append(jm.queue, namedJob{name, job, logger}) if jm.activeWorkers < jm.maxConcurrentJobs { jm.activeWorkers++ go jm.worker() } } func (jm *jobManager) worker() { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: certificate worker: %v\n%s", err, buf) } }() for { jm.mu.Lock() if len(jm.queue) == 0 { jm.activeWorkers-- jm.mu.Unlock() return } next := jm.queue[0] jm.queue = jm.queue[1:] jm.mu.Unlock() if err := next.job(); err != nil { next.logger.Error("job failed", zap.Error(err)) } if next.name != "" { jm.mu.Lock() delete(jm.names, next.name) jm.mu.Unlock() } } } func doWithRetry(ctx context.Context, log *zap.Logger, f func(context.Context) error) error { var attempts int ctx = context.WithValue(ctx, AttemptsCtxKey, &attempts) // the initial intervalIndex is -1, signaling // that we should not wait for the first attempt start, intervalIndex := time.Now(), -1 var err error for time.Since(start) < maxRetryDuration { var wait time.Duration if intervalIndex >= 0 { wait = retryIntervals[intervalIndex] } timer := time.NewTimer(wait) select { case <-ctx.Done(): timer.Stop() return context.Canceled case <-timer.C: err = f(ctx) attempts++ if err == nil || errors.Is(err, context.Canceled) { return err } var errNoRetry ErrNoRetry if errors.As(err, &errNoRetry) { return err } if intervalIndex < len(retryIntervals)-1 { intervalIndex++ } if time.Since(start) < maxRetryDuration { log.Error("will retry", zap.Error(err), zap.Int("attempt", attempts), zap.Duration("retrying_in", retryIntervals[intervalIndex]), zap.Duration("elapsed", time.Since(start)), zap.Duration("max_duration", maxRetryDuration)) } else { log.Error("final attempt; giving up", zap.Error(err), zap.Int("attempt", attempts), zap.Duration("elapsed", time.Since(start)), zap.Duration("max_duration", maxRetryDuration)) return nil } } } return err } // ErrNoRetry is an error type which signals // to stop retries early. type ErrNoRetry struct{ Err error } // Unwrap makes it so that e wraps e.Err. func (e ErrNoRetry) Unwrap() error { return e.Err } func (e ErrNoRetry) Error() string { return e.Err.Error() } type retryStateCtxKey struct{} // AttemptsCtxKey is the context key for the value // that holds the attempt counter. The value counts // how many times the operation has been attempted. // A value of 0 means first attempt. var AttemptsCtxKey retryStateCtxKey // retryIntervals are based on the idea of exponential // backoff, but weighed a little more heavily to the // front. We figure that intermittent errors would be // resolved after the first retry, but any errors after // that would probably require at least a few minutes // or hours to clear up: either for DNS to propagate, for // the administrator to fix their DNS or network config, // or some other external factor needs to change. We // chose intervals that we think will be most useful // without introducing unnecessary delay. The last // interval in this list will be used until the time // of maxRetryDuration has elapsed. var retryIntervals = []time.Duration{ 1 * time.Minute, 2 * time.Minute, 2 * time.Minute, 5 * time.Minute, // elapsed: 10 min 10 * time.Minute, 10 * time.Minute, 10 * time.Minute, 20 * time.Minute, // elapsed: 1 hr 20 * time.Minute, 20 * time.Minute, 20 * time.Minute, // elapsed: 2 hr 30 * time.Minute, 30 * time.Minute, // elapsed: 3 hr 30 * time.Minute, 30 * time.Minute, // elapsed: 4 hr 30 * time.Minute, 30 * time.Minute, // elapsed: 5 hr 1 * time.Hour, // elapsed: 6 hr 1 * time.Hour, 1 * time.Hour, // elapsed: 8 hr 2 * time.Hour, 2 * time.Hour, // elapsed: 12 hr 3 * time.Hour, 3 * time.Hour, // elapsed: 18 hr 6 * time.Hour, // repeat for up to maxRetryDuration } // maxRetryDuration is the maximum duration to try // doing retries using the above intervals. const maxRetryDuration = 24 * time.Hour * 30 golang-github-caddyserver-certmagic-0.25.2/cache.go000066400000000000000000000342121514710434200221710ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "fmt" weakrand "math/rand/v2" "strings" "sync" "time" "go.uber.org/zap" ) // Cache is a structure that stores certificates in memory. // A Cache indexes certificates by name for quick access // during TLS handshakes, and avoids duplicating certificates // in memory. Generally, there should only be one per process. // However, that is not a strict requirement; but using more // than one is a code smell, and may indicate an // over-engineered design. // // An empty cache is INVALID and must not be used. Be sure // to call NewCache to get a valid value. // // These should be very long-lived values and must not be // copied. Before all references leave scope to be garbage // collected, ensure you call Stop() to stop maintenance on // the certificates stored in this cache and release locks. // // Caches are not usually manipulated directly; create a // Config value with a pointer to a Cache, and then use // the Config to interact with the cache. Caches are // agnostic of any particular storage or ACME config, // since each certificate may be managed and stored // differently. type Cache struct { // User configuration of the cache options CacheOptions optionsMu sync.RWMutex // The cache is keyed by certificate hash cache map[string]Certificate // cacheIndex is a map of SAN to cache key (cert hash) cacheIndex map[string][]string // Protects the cache and cacheIndex maps mu sync.RWMutex // Close this channel to cancel asset maintenance stopChan chan struct{} // Used to signal when stopping is completed doneChan chan struct{} logger *zap.Logger } // NewCache returns a new, valid Cache for efficiently // accessing certificates in memory. It also begins a // maintenance goroutine to tend to the certificates // in the cache. Call Stop() when you are done with the // cache so it can clean up locks and stuff. // // Most users of this package will not need to call this // because a default certificate cache is created for you. // Only advanced use cases require creating a new cache. // // This function panics if opts.GetConfigForCert is not // set. The reason is that a cache absolutely needs to // be able to get a Config with which to manage TLS // assets, and it is not safe to assume that the Default // config is always the correct one, since you have // created the cache yourself. // // See the godoc for Cache to use it properly. When // no longer needed, caches should be stopped with // Stop() to clean up resources even if the process // is being terminated, so that it can clean up // any locks for other processes to unblock! func NewCache(opts CacheOptions) *Cache { c := &Cache{ cache: make(map[string]Certificate), cacheIndex: make(map[string][]string), stopChan: make(chan struct{}), doneChan: make(chan struct{}), logger: opts.Logger, } // absolutely do not allow a nil logger; panics galore if c.logger == nil { c.logger = defaultLogger } c.SetOptions(opts) go c.maintainAssets(0) return c } func (certCache *Cache) SetOptions(opts CacheOptions) { // assume default options if necessary if opts.OCSPCheckInterval <= 0 { opts.OCSPCheckInterval = DefaultOCSPCheckInterval } if opts.RenewCheckInterval <= 0 { opts.RenewCheckInterval = DefaultRenewCheckInterval } if opts.Capacity < 0 { opts.Capacity = 0 } // this must be set, because we cannot not // safely assume that the Default Config // is always the correct one to use if opts.GetConfigForCert == nil { panic("cache must be initialized with a GetConfigForCert callback") } certCache.optionsMu.Lock() certCache.options = opts certCache.optionsMu.Unlock() } // Stop stops the maintenance goroutine for // certificates in certCache. It blocks until // stopping is complete. Once a cache is // stopped, it cannot be reused. func (certCache *Cache) Stop() { close(certCache.stopChan) // signal to stop <-certCache.doneChan // wait for stop to complete } // CacheOptions is used to configure certificate caches. // Once a cache has been created with certain options, // those settings cannot be changed. type CacheOptions struct { // REQUIRED. A function that returns a configuration // used for managing a certificate, or for accessing // that certificate's asset storage (e.g. for // OCSP staples, etc). The returned Config MUST // be associated with the same Cache as the caller, // use New to obtain a valid Config. // // The reason this is a callback function, dynamically // returning a Config (instead of attaching a static // pointer to a Config on each certificate) is because // the config for how to manage a domain's certificate // might change from maintenance to maintenance. The // cache is so long-lived, we cannot assume that the // host's situation will always be the same; e.g. the // certificate might switch DNS providers, so the DNS // challenge (if used) would need to be adjusted from // the last time it was run ~8 weeks ago. GetConfigForCert ConfigGetter // How often to check certificates for renewal; // if unset, DefaultOCSPCheckInterval will be used. OCSPCheckInterval time.Duration // How often to check certificates for renewal; // if unset, DefaultRenewCheckInterval will be used. RenewCheckInterval time.Duration // Maximum number of certificates to allow in the cache. // If reached, certificates will be randomly evicted to // make room for new ones. 0 means unlimited. Capacity int // Set a logger to enable logging Logger *zap.Logger } // ConfigGetter is a function that returns a prepared, // valid config that should be used when managing the // given certificate or its assets. type ConfigGetter func(Certificate) (*Config, error) // cacheCertificate calls unsyncedCacheCertificate with a write lock. // // This function is safe for concurrent use. func (certCache *Cache) cacheCertificate(cert Certificate) { certCache.mu.Lock() certCache.unsyncedCacheCertificate(cert) certCache.mu.Unlock() } // unsyncedCacheCertificate adds cert to the in-memory cache unless // it already exists in the cache (according to cert.Hash). It // updates the name index. // // This function is NOT safe for concurrent use. Callers MUST acquire // a write lock on certCache.mu first. func (certCache *Cache) unsyncedCacheCertificate(cert Certificate) { // if this certificate already exists in the cache, this is basically // a no-op so we reuse existing cert (prevent duplication), but we do // modify the cert to add tags it may be missing (see issue #211) if existingCert, ok := certCache.cache[cert.hash]; ok { logMsg := "certificate already cached" if len(cert.Tags) > 0 { for _, tag := range cert.Tags { if !existingCert.HasTag(tag) { existingCert.Tags = append(existingCert.Tags, tag) } } certCache.cache[cert.hash] = existingCert logMsg += "; appended any missing tags to cert" } certCache.logger.Debug(logMsg, zap.Strings("subjects", cert.Names), zap.Time("expiration", expiresAt(cert.Leaf)), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash), zap.Strings("tags", cert.Tags)) return } // if the cache is at capacity, make room for new cert cacheSize := len(certCache.cache) certCache.optionsMu.RLock() atCapacity := certCache.options.Capacity > 0 && cacheSize >= certCache.options.Capacity certCache.optionsMu.RUnlock() if atCapacity { // Go maps are "nondeterministic" but not actually random, // so although we could just chop off the "front" of the // map with less code, that is a heavily skewed eviction // strategy; generating random numbers is cheap and // ensures a much better distribution. rnd := weakrand.IntN(cacheSize) i := 0 for _, randomCert := range certCache.cache { if i >= rnd && randomCert.managed { // don't evict manually-loaded certs certCache.logger.Debug("cache full; evicting random certificate", zap.Strings("removing_subjects", randomCert.Names), zap.String("removing_hash", randomCert.hash), zap.Strings("inserting_subjects", cert.Names), zap.String("inserting_hash", cert.hash)) certCache.removeCertificate(randomCert) break } i++ } } // store the certificate certCache.cache[cert.hash] = cert // update the index so we can access it by name for _, name := range cert.Names { certCache.cacheIndex[name] = append(certCache.cacheIndex[name], cert.hash) } certCache.optionsMu.RLock() certCache.logger.Debug("added certificate to cache", zap.Strings("subjects", cert.Names), zap.Time("expiration", expiresAt(cert.Leaf)), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash), zap.Int("cache_size", len(certCache.cache)), zap.Int("cache_capacity", certCache.options.Capacity)) certCache.optionsMu.RUnlock() } // removeCertificate removes cert from the cache. // // This function is NOT safe for concurrent use; callers // MUST first acquire a write lock on certCache.mu. func (certCache *Cache) removeCertificate(cert Certificate) { // delete all mentions of this cert from the name index for _, name := range cert.Names { keyList := certCache.cacheIndex[name] for i := 0; i < len(keyList); i++ { if keyList[i] == cert.hash { keyList = append(keyList[:i], keyList[i+1:]...) i-- } } if len(keyList) == 0 { delete(certCache.cacheIndex, name) } else { certCache.cacheIndex[name] = keyList } } // delete the actual cert from the cache delete(certCache.cache, cert.hash) certCache.optionsMu.RLock() certCache.logger.Debug("removed certificate from cache", zap.Strings("subjects", cert.Names), zap.Time("expiration", expiresAt(cert.Leaf)), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash), zap.Int("cache_size", len(certCache.cache)), zap.Int("cache_capacity", certCache.options.Capacity)) certCache.optionsMu.RUnlock() } // replaceCertificate atomically replaces oldCert with newCert in // the cache. // // This method is safe for concurrent use. func (certCache *Cache) replaceCertificate(oldCert, newCert Certificate) { certCache.mu.Lock() certCache.removeCertificate(oldCert) certCache.unsyncedCacheCertificate(newCert) certCache.mu.Unlock() certCache.logger.Info("replaced certificate in cache", zap.Strings("subjects", newCert.Names), zap.Time("new_expiration", expiresAt(newCert.Leaf))) } // getAllMatchingCerts returns all certificates with exactly this subject // (wildcards are NOT expanded). func (certCache *Cache) getAllMatchingCerts(subject string) []Certificate { certCache.mu.RLock() defer certCache.mu.RUnlock() allCertKeys := certCache.cacheIndex[subject] certs := make([]Certificate, len(allCertKeys)) for i := range allCertKeys { certs[i] = certCache.cache[allCertKeys[i]] } return certs } func (certCache *Cache) getAllCerts() []Certificate { certCache.mu.RLock() defer certCache.mu.RUnlock() certs := make([]Certificate, 0, len(certCache.cache)) for _, cert := range certCache.cache { certs = append(certs, cert) } return certs } func (certCache *Cache) getConfig(cert Certificate) (*Config, error) { certCache.optionsMu.RLock() getCert := certCache.options.GetConfigForCert certCache.optionsMu.RUnlock() cfg, err := getCert(cert) if err != nil { return nil, err } if cfg == nil { // this is bad if this happens, probably a programmer error (oops) return nil, fmt.Errorf("no configuration associated with certificate: %v;", cert.Names) } if cfg.certCache == nil { return nil, fmt.Errorf("config returned for certificate %v has nil cache; expected %p (this one)", cert.Names, certCache) } if cfg.certCache != certCache { return nil, fmt.Errorf("config returned for certificate %v is not nil and points to different cache; got %p, expected %p (this one)", cert.Names, cfg.certCache, certCache) } return cfg, nil } // AllMatchingCertificates returns a list of all certificates that could // be used to serve the given SNI name, including exact SAN matches and // wildcard matches. func (certCache *Cache) AllMatchingCertificates(name string) []Certificate { // get exact matches first certs := certCache.getAllMatchingCerts(name) // then look for wildcard matches by replacing each // label of the domain name with wildcards labels := strings.Split(name, ".") for i := range labels { labels[i] = "*" candidate := strings.Join(labels, ".") certs = append(certs, certCache.getAllMatchingCerts(candidate)...) } return certs } // SubjectIssuer pairs a subject name with an issuer ID/key. type SubjectIssuer struct { Subject, IssuerKey string } // RemoveManaged removes managed certificates for the given subjects from the cache. // This effectively stops maintenance of those certificates. If an IssuerKey is // specified alongside the subject, only certificates for that subject from the // specified issuer will be removed. func (certCache *Cache) RemoveManaged(subjects []SubjectIssuer) { deleteQueue := make([]string, 0, len(subjects)) for _, subj := range subjects { certs := certCache.getAllMatchingCerts(subj.Subject) // does NOT expand wildcards; exact matches only for _, cert := range certs { if !cert.managed { continue } if subj.IssuerKey == "" || cert.issuerKey == subj.IssuerKey { deleteQueue = append(deleteQueue, cert.hash) } } } certCache.Remove(deleteQueue) } // Remove removes certificates with the given hashes from the cache. // This is effectively used to unload manually-loaded certificates. func (certCache *Cache) Remove(hashes []string) { certCache.mu.Lock() for _, h := range hashes { cert := certCache.cache[h] certCache.removeCertificate(cert) } certCache.mu.Unlock() } var ( defaultCache *Cache defaultCacheMu sync.Mutex ) golang-github-caddyserver-certmagic-0.25.2/cache_test.go000066400000000000000000000027241514710434200232330ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import "testing" func TestNewCache(t *testing.T) { noop := func(Certificate) (*Config, error) { return new(Config), nil } c := NewCache(CacheOptions{GetConfigForCert: noop}) defer c.Stop() c.optionsMu.RLock() defer c.optionsMu.RUnlock() if c.options.RenewCheckInterval != DefaultRenewCheckInterval { t.Errorf("Expected RenewCheckInterval to be set to default value, but it wasn't: %s", c.options.RenewCheckInterval) } if c.options.OCSPCheckInterval != DefaultOCSPCheckInterval { t.Errorf("Expected OCSPCheckInterval to be set to default value, but it wasn't: %s", c.options.OCSPCheckInterval) } if c.options.GetConfigForCert == nil { t.Error("Expected GetConfigForCert to be set, but it was nil") } if c.cache == nil { t.Error("Expected cache to be set, but it was nil") } if c.stopChan == nil { t.Error("Expected stopChan to be set, but it was nil") } } golang-github-caddyserver-certmagic-0.25.2/certificates.go000066400000000000000000000627611514710434200236050ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/tls" "crypto/x509" "encoding/json" "errors" "fmt" "math/rand/v2" "net" "os" "strings" "time" "github.com/mholt/acmez/v3/acme" "go.uber.org/zap" "golang.org/x/crypto/ocsp" ) // Certificate is a tls.Certificate with associated metadata tacked on. // Even if the metadata can be obtained by parsing the certificate, // we are more efficient by extracting the metadata onto this struct, // but at the cost of slightly higher memory use. type Certificate struct { tls.Certificate // Names is the list of subject names this // certificate is signed for. Names []string // Optional; user-provided, and arbitrary. Tags []string // OCSP contains the certificate's parsed OCSP response. // It is not necessarily the response that is stapled // (e.g. if the status is not Good), it is simply the // most recent OCSP response we have for this certificate. ocsp *ocsp.Response // The hex-encoded hash of this cert's chain's DER bytes. hash string // Whether this certificate is under our management. managed bool // The unique string identifying the issuer of this certificate. issuerKey string // ACME Renewal Information, if available ari acme.RenewalInfo } // Empty returns true if the certificate struct is not filled out; at // least the tls.Certificate.Certificate field is expected to be set. func (cert Certificate) Empty() bool { return len(cert.Certificate.Certificate) == 0 } // Hash returns a checksum of the certificate chain's DER-encoded bytes. func (cert Certificate) Hash() string { return cert.hash } // NeedsRenewal returns true if the certificate is expiring // soon (according to ARI and/or cfg) or has expired. func (cert Certificate) NeedsRenewal(cfg *Config) bool { return cfg.certNeedsRenewal(cert.Leaf, cert.ari, true) } // certNeedsRenewal consults ACME Renewal Info (ARI) and certificate expiration to determine // whether the leaf certificate needs to be renewed yet. If true is returned, the certificate // should be renewed as soon as possible. The reasoning for a true return value is logged // unless emitLogs is false; this can be useful to suppress noisy logs in the case where you // first call this to determine if a cert in memory needs renewal, and then right after you // call it again to see if the cert in storage still needs renewal -- you probably don't want // to log the second time for checking the cert in storage which is mainly for synchronization. func (cfg *Config) certNeedsRenewal(leaf *x509.Certificate, ari acme.RenewalInfo, emitLogs bool) bool { // though this should never happen, safeguard to avoid panics which happened before (since patched; but just in case) if leaf == nil { if emitLogs { cfg.Logger.Error("cannot check if nil leaf cert needs renewal") } return false } expiration := expiresAt(leaf) var logger *zap.Logger if emitLogs { logger = cfg.Logger.With( zap.Strings("subjects", leaf.DNSNames), zap.Time("expiration", expiration), zap.String("ari_cert_id", ari.UniqueIdentifier), zap.Timep("next_ari_update", ari.RetryAfter), zap.Duration("renew_check_interval", cfg.certCache.options.RenewCheckInterval), zap.Time("window_start", ari.SuggestedWindow.Start), zap.Time("window_end", ari.SuggestedWindow.End)) } else { logger = zap.NewNop() } if !cfg.DisableARI { // first check ARI: if it says it's time to renew, it's time to renew // (notice that we don't strictly require an ARI window to also exist; we presume // that if a time has been selected, a window does or did exist, even if it didn't // get stored/encoded for some reason - but also: this allows administrators to // manually or explicitly schedule a renewal time independently of ARI which could // be useful) selectedTime := ari.SelectedTime // if, for some reason a random time in the window hasn't been selected yet, but an ARI // window does exist, we can always improvise one... even if this is called repeatedly, // a random time is a random time, whether you generate it once or more :D // (code borrowed from our acme package) if selectedTime.IsZero() && (!ari.SuggestedWindow.Start.IsZero() && !ari.SuggestedWindow.End.IsZero()) { start, end := ari.SuggestedWindow.Start.Unix()+1, ari.SuggestedWindow.End.Unix() selectedTime = time.Unix(rand.Int64N(end-start)+start, 0).UTC() logger.Warn("no renewal time had been selected with ARI; chose an ephemeral one for now", zap.Time("ephemeral_selected_time", selectedTime)) } // if a renewal time has been selected, start with that if !selectedTime.IsZero() { // ARI spec recommends an algorithm that renews after the randomly-selected // time OR just before it if the next waking time would be after it; this // cutoff can actually be before the start of the renewal window, but the spec // author says that's OK: https://github.com/aarongable/draft-acme-ari/issues/71 cutoff := ari.SelectedTime.Add(-cfg.certCache.options.RenewCheckInterval) if time.Now().After(cutoff) { logger.Info("certificate needs renewal based on ARI window", zap.Time("selected_time", selectedTime), zap.Time("renewal_cutoff", cutoff)) return true } // according to ARI, we are not ready to renew; however, we do not rely solely on // ARI calculations... what if there is a bug in our implementation, or in the // server's, or the stored metadata? for redundancy, give credence to the expiration // date; ignore ARI if we are past a "dangerously close" limit, to avoid any // possibility of a bug in ARI compromising a site's uptime: we should always always // always give heed to actual validity period if currentlyInRenewalWindow(leaf.NotBefore, expiration, 1.0/20.0) { logger.Warn("certificate is in emergency renewal window; superseding ARI", zap.Duration("remaining", time.Until(expiration)), zap.Time("renewal_cutoff", cutoff)) return true } } } // the normal check, in the absence of ARI, is to determine if we're near enough (or past) // the expiration date based on the configured remaining:lifetime ratio if currentlyInRenewalWindow(leaf.NotBefore, expiration, cfg.RenewalWindowRatio) { logger.Info("certificate is in configured renewal window based on expiration date", zap.Duration("remaining", time.Until(expiration))) return true } // finally, if the certificate is expiring imminently, always attempt a renewal; // we check both a (very low) lifetime ratio and also a strict difference between // the time until expiration and the interval at which we run the standard maintenance // routine to check for renewals, to accommodate both exceptionally long and short // cert lifetimes if currentlyInRenewalWindow(leaf.NotBefore, expiration, 1.0/50.0) || time.Until(expiration) < cfg.certCache.options.RenewCheckInterval*5 { logger.Warn("certificate is in emergency renewal window; expiration imminent", zap.Duration("remaining", time.Until(expiration))) return true } return false } // Expired returns true if the certificate has expired. func (cert Certificate) Expired() bool { if cert.Leaf == nil { // ideally cert.Leaf would never be nil, but this can happen for // "synthetic" certs like those made to solve the TLS-ALPN challenge // which adds a special cert directly to the cache, since // tls.X509KeyPair() discards the leaf; oh well return false } return time.Now().After(expiresAt(cert.Leaf)) } // Lifetime returns the duration of the certificate's validity. func (cert Certificate) Lifetime() time.Duration { if cert.Leaf == nil || cert.Leaf.NotAfter.IsZero() { return 0 } return expiresAt(cert.Leaf).Sub(cert.Leaf.NotBefore) } // currentlyInRenewalWindow returns true if the current time is within // (or after) the renewal window, according to the given start/end // dates and the ratio of the renewal window. If true is returned, // the certificate being considered is due for renewal. The ratio // is remaining:total time, i.e. 1/3 = 1/3 of lifetime remaining, // or 9/10 = 9/10 of time lifetime remaining. func currentlyInRenewalWindow(notBefore, notAfter time.Time, renewalWindowRatio float64) bool { if notAfter.IsZero() { return false } lifetime := notAfter.Sub(notBefore) if renewalWindowRatio == 0 { renewalWindowRatio = DefaultRenewalWindowRatio } renewalWindow := time.Duration(float64(lifetime) * renewalWindowRatio) renewalWindowStart := notAfter.Add(-renewalWindow) return time.Now().After(renewalWindowStart) } // HasTag returns true if cert.Tags has tag. func (cert Certificate) HasTag(tag string) bool { for _, t := range cert.Tags { if t == tag { return true } } return false } // expiresAt return the time that a certificate expires. Account for the 1s // resolution of ASN.1 UTCTime/GeneralizedTime by including the extra fraction // of a second of certificate validity beyond the NotAfter value. func expiresAt(cert *x509.Certificate) time.Time { if cert == nil { return time.Time{} } return cert.NotAfter.Truncate(time.Second).Add(1 * time.Second) } // CacheManagedCertificate loads the certificate for domain into the // cache, from the TLS storage for managed certificates. It returns a // copy of the Certificate that was put into the cache. // // This is a lower-level method; normally you'll call Manage() instead. // // This method is safe for concurrent use. func (cfg *Config) CacheManagedCertificate(ctx context.Context, domain string) (Certificate, error) { domain = cfg.transformSubject(ctx, nil, domain) cert, err := cfg.loadManagedCertificate(ctx, domain) if err != nil { return cert, err } cfg.certCache.cacheCertificate(cert) cfg.emit(ctx, "cached_managed_cert", map[string]any{"sans": cert.Names}) return cert, nil } // loadManagedCertificate loads the managed certificate for domain from any // of the configured issuers' storage locations, but it does not add it to // the cache. It just loads from storage and returns it. func (cfg *Config) loadManagedCertificate(ctx context.Context, domain string) (Certificate, error) { certRes, err := cfg.loadCertResourceAnyIssuer(ctx, domain) if err != nil { return Certificate{}, err } cert, err := cfg.makeCertificateWithOCSP(ctx, certRes.CertificatePEM, certRes.PrivateKeyPEM) if err != nil { return cert, err } cert.managed = true cert.issuerKey = certRes.issuerKey if ari, err := certRes.getARI(); err == nil && ari != nil { cert.ari = *ari } return cert, nil } // getARI unpacks ACME Renewal Information from the issuer data, if available. // It is only an error if there is invalid JSON. func (certRes CertificateResource) getARI() (*acme.RenewalInfo, error) { acmeData, err := certRes.getACMEData() if err != nil { return nil, err } return acmeData.RenewalInfo, nil } // getACMEData returns the ACME certificate metadata from the IssuerData, but // note that a non-ACME-issued certificate may return an empty value and nil // since the JSON may still decode successfully but just not match any or all // of the fields. Remember that the IssuerKey is used to store and access the // cert files in the first place (it is part of the path) so in theory if you // load a CertificateResource from an ACME issuer it should work as expected. func (certRes CertificateResource) getACMEData() (acme.Certificate, error) { if len(certRes.IssuerData) == 0 { return acme.Certificate{}, nil } var acmeCert acme.Certificate err := json.Unmarshal(certRes.IssuerData, &acmeCert) return acmeCert, err } // CacheUnmanagedCertificatePEMFile loads a certificate for host using certFile // and keyFile, which must be in PEM format. It stores the certificate in // the in-memory cache and returns the hash, useful for removing from the cache. // // This method is safe for concurrent use. func (cfg *Config) CacheUnmanagedCertificatePEMFile(ctx context.Context, certFile, keyFile string, tags []string) (string, error) { cert, err := cfg.makeCertificateFromDiskWithOCSP(ctx, certFile, keyFile) if err != nil { return "", err } cert.Tags = tags cfg.certCache.cacheCertificate(cert) cfg.emit(ctx, "cached_unmanaged_cert", map[string]any{"sans": cert.Names}) return cert.hash, nil } // CacheUnmanagedTLSCertificate adds tlsCert to the certificate cache // // and returns the hash, useful for removing from the cache. // // It staples OCSP if possible. // // This method is safe for concurrent use. func (cfg *Config) CacheUnmanagedTLSCertificate(ctx context.Context, tlsCert tls.Certificate, tags []string) (string, error) { var cert Certificate err := fillCertFromLeaf(&cert, tlsCert) if err != nil { return "", err } if time.Now().After(cert.Leaf.NotAfter) { cfg.Logger.Warn("unmanaged certificate has expired", zap.Time("not_after", cert.Leaf.NotAfter), zap.Strings("sans", cert.Names)) } else if time.Until(cert.Leaf.NotAfter) < 24*time.Hour { cfg.Logger.Warn("unmanaged certificate expires within 1 day", zap.Time("not_after", cert.Leaf.NotAfter), zap.Strings("sans", cert.Names)) } if !cfg.OCSP.DisableStapling { err = stapleOCSP(ctx, cfg.OCSP, cfg.Storage, &cert, nil) if err != nil { if errors.Is(err, ErrNoOCSPServerSpecified) { cfg.Logger.Debug("stapling OCSP", zap.Error(err)) } else { cfg.Logger.Warn("stapling OCSP", zap.Error(err)) } } } cfg.emit(ctx, "cached_unmanaged_cert", map[string]any{"sans": cert.Names}) cert.Tags = tags cfg.certCache.cacheCertificate(cert) return cert.hash, nil } // CacheUnmanagedCertificatePEMBytes makes a certificate out of the PEM bytes // of the certificate and key, then caches it in memory, and returns the hash, // which is useful for removing from the cache. // // This method is safe for concurrent use. func (cfg *Config) CacheUnmanagedCertificatePEMBytes(ctx context.Context, certBytes, keyBytes []byte, tags []string) (string, error) { cert, err := cfg.makeCertificateWithOCSP(ctx, certBytes, keyBytes) if err != nil { return "", err } cert.Tags = tags cfg.certCache.cacheCertificate(cert) cfg.emit(ctx, "cached_unmanaged_cert", map[string]any{"sans": cert.Names}) return cert.hash, nil } // CacheUnmanagedCertificatePEMBytesAsReplacement is the same as CacheUnmanagedCertificatePEMBytes, // but it also removes any other loaded certificates for the SANs on the certificate being cached. // This has the effect of using this certificate exclusively and immediately for its SANs. The SANs // for which the certificate should apply may optionally be passed in as well. By default, a cert // is used for any of its SANs. // // This method is safe for concurrent use. // // EXPERIMENTAL: Subject to change/removal. func (cfg *Config) CacheUnmanagedCertificatePEMBytesAsReplacement(ctx context.Context, certBytes, keyBytes []byte, tags, sans []string) (string, error) { cert, err := cfg.makeCertificateWithOCSP(ctx, certBytes, keyBytes) if err != nil { return "", err } cert.Tags = tags if len(sans) > 0 { cert.Names = sans } cfg.certCache.mu.Lock() for _, san := range cert.Names { existingCerts := cfg.certCache.getAllMatchingCerts(san) for _, existingCert := range existingCerts { cfg.certCache.removeCertificate(existingCert) } } cfg.certCache.unsyncedCacheCertificate(cert) cfg.certCache.mu.Unlock() cfg.emit(ctx, "cached_unmanaged_cert", map[string]any{"sans": cert.Names, "replacement": true}) return cert.hash, nil } // makeCertificateFromDiskWithOCSP makes a Certificate by loading the // certificate and key files. It fills out all the fields in // the certificate except for the Managed and OnDemand flags. // (It is up to the caller to set those.) It staples OCSP. func (cfg Config) makeCertificateFromDiskWithOCSP(ctx context.Context, certFile, keyFile string) (Certificate, error) { certPEMBlock, err := os.ReadFile(certFile) if err != nil { return Certificate{}, err } keyPEMBlock, err := os.ReadFile(keyFile) if err != nil { return Certificate{}, err } return cfg.makeCertificateWithOCSP(ctx, certPEMBlock, keyPEMBlock) } // makeCertificateWithOCSP is the same as makeCertificate except that it also // staples OCSP to the certificate. func (cfg Config) makeCertificateWithOCSP(ctx context.Context, certPEMBlock, keyPEMBlock []byte) (Certificate, error) { cert, err := makeCertificate(certPEMBlock, keyPEMBlock) if err != nil { return cert, err } if !cfg.OCSP.DisableStapling { err = stapleOCSP(ctx, cfg.OCSP, cfg.Storage, &cert, certPEMBlock) if errors.Is(err, ErrNoOCSPServerSpecified) { cfg.Logger.Debug("stapling OCSP", zap.Error(err), zap.Strings("identifiers", cert.Names)) } else { cfg.Logger.Warn("stapling OCSP", zap.Error(err), zap.Strings("identifiers", cert.Names)) } } return cert, nil } // makeCertificate turns a certificate PEM bundle and a key PEM block into // a Certificate with necessary metadata from parsing its bytes filled into // its struct fields for convenience (except for the OnDemand and Managed // flags; it is up to the caller to set those properties!). This function // does NOT staple OCSP. func makeCertificate(certPEMBlock, keyPEMBlock []byte) (Certificate, error) { var cert Certificate // Convert to a tls.Certificate tlsCert, err := tls.X509KeyPair(certPEMBlock, keyPEMBlock) if err != nil { return cert, err } // Extract necessary metadata err = fillCertFromLeaf(&cert, tlsCert) if err != nil { return cert, err } return cert, nil } // fillCertFromLeaf populates cert from tlsCert. If it succeeds, it // guarantees that cert.Leaf is non-nil. func fillCertFromLeaf(cert *Certificate, tlsCert tls.Certificate) error { if len(tlsCert.Certificate) == 0 { return fmt.Errorf("certificate is empty") } cert.Certificate = tlsCert // the leaf cert should be the one for the site; we must set // the tls.Certificate.Leaf field so that TLS handshakes are // more efficient leaf := cert.Certificate.Leaf if leaf == nil { var err error leaf, err = x509.ParseCertificate(tlsCert.Certificate[0]) if err != nil { return err } cert.Certificate.Leaf = leaf } // for convenience, we do want to assemble all the // subjects on the certificate into one list if leaf.Subject.CommonName != "" { // TODO: CommonName is deprecated cert.Names = []string{strings.ToLower(leaf.Subject.CommonName)} } for _, name := range leaf.DNSNames { if name != leaf.Subject.CommonName { // TODO: CommonName is deprecated cert.Names = append(cert.Names, strings.ToLower(name)) } } for _, ip := range leaf.IPAddresses { if ipStr := ip.String(); ipStr != leaf.Subject.CommonName { // TODO: CommonName is deprecated cert.Names = append(cert.Names, strings.ToLower(ipStr)) } } for _, email := range leaf.EmailAddresses { if email != leaf.Subject.CommonName { // TODO: CommonName is deprecated cert.Names = append(cert.Names, strings.ToLower(email)) } } for _, u := range leaf.URIs { if u.String() != leaf.Subject.CommonName { // TODO: CommonName is deprecated cert.Names = append(cert.Names, u.String()) } } if len(cert.Names) == 0 { return fmt.Errorf("certificate has no names") } cert.hash = hashCertificateChain(cert.Certificate.Certificate) return nil } // managedCertInStorageNeedsRenewal returns true if cert (being a // managed certificate) is expiring soon (according to cfg) or if // ACME Renewal Information (ARI) is available and says that it is // time to renew (it uses existing ARI; it does not update it). // It returns false if there was an error, the cert is not expiring // soon, and ARI window is still future. A certificate that is expiring // soon in our cache but is not expiring soon in storage probably // means that another instance renewed the certificate in the // meantime, and it would be a good idea to simply load the cert // into our cache rather than repeating the renewal process again. func (cfg *Config) managedCertInStorageNeedsRenewal(ctx context.Context, cert Certificate) (bool, error) { certRes, err := cfg.loadCertResourceAnyIssuer(ctx, cert.Names[0]) if err != nil { return false, err } _, _, needsRenew := cfg.managedCertNeedsRenewal(certRes, false) return needsRenew, nil } // reloadManagedCertificate reloads the certificate corresponding to the name(s) // on oldCert into the cache, from storage. This also replaces the old certificate // with the new one, so that all configurations that used the old cert now point // to the new cert. It assumes that the new certificate for oldCert.Names[0] is // already in storage. It returns the newly-loaded certificate if successful. func (cfg *Config) reloadManagedCertificate(ctx context.Context, oldCert Certificate) (Certificate, error) { cfg.Logger.Info("reloading managed certificate", zap.Strings("identifiers", oldCert.Names)) newCert, err := cfg.loadManagedCertificate(ctx, oldCert.Names[0]) if err != nil { return Certificate{}, fmt.Errorf("loading managed certificate for %v from storage: %v", oldCert.Names, err) } cfg.certCache.replaceCertificate(oldCert, newCert) return newCert, nil } // SubjectQualifiesForCert returns true if subj is a name which, // as a quick sanity check, looks like it could be the subject // of a certificate. Requirements are: // - must not be empty // - must not start or end with a dot (RFC 1034; RFC 6066 section 3) // - must not contain common accidental special characters func SubjectQualifiesForCert(subj string) bool { // must not be empty return strings.TrimSpace(subj) != "" && // must not start or end with a dot !strings.HasPrefix(subj, ".") && !strings.HasSuffix(subj, ".") && // if it has a wildcard, must be a left-most label (or exactly "*" // which won't be trusted by browsers but still technically works) (!strings.Contains(subj, "*") || strings.HasPrefix(subj, "*.") || subj == "*") && // must not contain other common special characters !strings.ContainsAny(subj, "()[]{}<> \t\n\"\\!@#$%^&|;'+=") } // SubjectQualifiesForPublicCert returns true if the subject // name appears eligible for automagic TLS with a public // CA such as Let's Encrypt. For example: internal IP addresses // and localhost are not eligible because we cannot obtain certs // for those names with a public CA. Wildcard names are // allowed, as long as they conform to CABF requirements (only // one wildcard label, and it must be the left-most label). func SubjectQualifiesForPublicCert(subj string) bool { // must at least qualify for a certificate return SubjectQualifiesForCert(subj) && // loopback hosts and internal IPs are ineligible !SubjectIsInternal(subj) && // only one wildcard label allowed, and it must be left-most, with 3+ labels (!strings.Contains(subj, "*") || (strings.Count(subj, "*") == 1 && strings.Count(subj, ".") > 1 && len(subj) > 2 && strings.HasPrefix(subj, "*."))) } // SubjectIsIP returns true if subj is an IP address. func SubjectIsIP(subj string) bool { return net.ParseIP(subj) != nil } // SubjectIsInternal returns true if subj is an internal-facing // hostname or address, including localhost/loopback hosts. // Ports are ignored, if present. func SubjectIsInternal(subj string) bool { subj = strings.ToLower(strings.TrimSuffix(hostOnly(subj), ".")) return subj == "localhost" || strings.HasSuffix(subj, ".localhost") || strings.HasSuffix(subj, ".local") || strings.HasSuffix(subj, ".internal") || strings.HasSuffix(subj, ".home.arpa") || isInternalIP(subj) } // isInternalIP returns true if the IP of addr // belongs to a private network IP range. addr // must only be an IP or an IP:port combination. func isInternalIP(addr string) bool { privateNetworks := []string{ "127.0.0.0/8", // IPv4 loopback "0.0.0.0/16", "10.0.0.0/8", // RFC1918 "172.16.0.0/12", // RFC1918 "192.168.0.0/16", // RFC1918 "169.254.0.0/16", // RFC3927 link-local "::1/7", // IPv6 loopback "fe80::/10", // IPv6 link-local "fc00::/7", // IPv6 unique local addr } host := hostOnly(addr) ip := net.ParseIP(host) if ip == nil { return false } for _, privateNetwork := range privateNetworks { _, ipnet, _ := net.ParseCIDR(privateNetwork) if ipnet.Contains(ip) { return true } } return false } // hostOnly returns only the host portion of hostport. // If there is no port or if there is an error splitting // the port off, the whole input string is returned. func hostOnly(hostport string) string { host, _, err := net.SplitHostPort(hostport) if err != nil { return hostport // OK; probably had no port to begin with } return host } // MatchWildcard returns true if subject (a candidate DNS name) // matches wildcard (a reference DNS name), mostly according to // RFC 6125-compliant wildcard rules. See also RFC 2818 which // states that IP addresses must match exactly, but this function // does not attempt to distinguish IP addresses from internal or // external DNS names that happen to look like IP addresses. // It uses DNS wildcard matching logic and is case-insensitive. // https://tools.ietf.org/html/rfc2818#section-3.1 func MatchWildcard(subject, wildcard string) bool { subject, wildcard = strings.ToLower(subject), strings.ToLower(wildcard) if subject == wildcard { return true } if !strings.Contains(wildcard, "*") { return false } labels := strings.Split(subject, ".") for i := range labels { if labels[i] == "" { continue // invalid label } labels[i] = "*" candidate := strings.Join(labels, ".") if candidate == wildcard { return true } } return false } golang-github-caddyserver-certmagic-0.25.2/certificates_test.go000066400000000000000000000175661514710434200246470ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "crypto/tls" "crypto/x509" "reflect" "testing" "time" ) func TestUnexportedGetCertificate(t *testing.T) { certCache := &Cache{cache: make(map[string]Certificate), cacheIndex: make(map[string][]string), logger: defaultTestLogger} cfg := &Config{Logger: defaultTestLogger, certCache: certCache} // When cache is empty if _, matched, defaulted := cfg.getCertificateFromCache(&tls.ClientHelloInfo{ServerName: "example.com"}); matched || defaulted { t.Errorf("Got a certificate when cache was empty; matched=%v, defaulted=%v", matched, defaulted) } // When cache has one certificate in it firstCert := Certificate{Names: []string{"example.com"}} certCache.cache["0xdeadbeef"] = firstCert certCache.cacheIndex["example.com"] = []string{"0xdeadbeef"} if cert, matched, defaulted := cfg.getCertificateFromCache(&tls.ClientHelloInfo{ServerName: "example.com"}); !matched || defaulted || cert.Names[0] != "example.com" { t.Errorf("Didn't get a cert for 'example.com' or got the wrong one: %v, matched=%v, defaulted=%v", cert, matched, defaulted) } // When retrieving wildcard certificate certCache.cache["0xb01dface"] = Certificate{Names: []string{"*.example.com"}} certCache.cacheIndex["*.example.com"] = []string{"0xb01dface"} if cert, matched, defaulted := cfg.getCertificateFromCache(&tls.ClientHelloInfo{ServerName: "sub.example.com"}); !matched || defaulted || cert.Names[0] != "*.example.com" { t.Errorf("Didn't get wildcard cert for 'sub.example.com' or got the wrong one: %v, matched=%v, defaulted=%v", cert, matched, defaulted) } // When no certificate matches and SNI is provided, return no certificate (should be TLS alert) if cert, matched, defaulted := cfg.getCertificateFromCache(&tls.ClientHelloInfo{ServerName: "nomatch"}); matched || defaulted { t.Errorf("Expected matched=false, defaulted=false; but got matched=%v, defaulted=%v (cert: %v)", matched, defaulted, cert) } } func TestCacheCertificate(t *testing.T) { certCache := &Cache{cache: make(map[string]Certificate), cacheIndex: make(map[string][]string), logger: defaultTestLogger} certCache.cacheCertificate(Certificate{Names: []string{"example.com", "sub.example.com"}, hash: "foobar", Certificate: tls.Certificate{Leaf: &x509.Certificate{NotAfter: time.Now()}}}) if len(certCache.cache) != 1 { t.Errorf("Expected length of certificate cache to be 1") } if _, ok := certCache.cache["foobar"]; !ok { t.Error("Expected first cert to be cached by key 'foobar', but it wasn't") } if _, ok := certCache.cacheIndex["example.com"]; !ok { t.Error("Expected first cert to be keyed by 'example.com', but it wasn't") } if _, ok := certCache.cacheIndex["sub.example.com"]; !ok { t.Error("Expected first cert to be keyed by 'sub.example.com', but it wasn't") } // using same cache; and has cert with overlapping name, but different hash certCache.cacheCertificate(Certificate{Names: []string{"example.com"}, hash: "barbaz", Certificate: tls.Certificate{Leaf: &x509.Certificate{NotAfter: time.Now()}}}) if _, ok := certCache.cache["barbaz"]; !ok { t.Error("Expected second cert to be cached by key 'barbaz.com', but it wasn't") } if hashes, ok := certCache.cacheIndex["example.com"]; !ok { t.Error("Expected second cert to be keyed by 'example.com', but it wasn't") } else if !reflect.DeepEqual(hashes, []string{"foobar", "barbaz"}) { t.Errorf("Expected second cert to map to 'barbaz' but it was %v instead", hashes) } } func TestSubjectQualifiesForCert(t *testing.T) { for i, test := range []struct { host string expect bool }{ {"hostname", true}, {"example.com", true}, {"sub.example.com", true}, {"Sub.Example.COM", true}, {"127.0.0.1", true}, {"127.0.1.5", true}, {"69.123.43.94", true}, {"::1", true}, {"::", true}, {"0.0.0.0", true}, {"", false}, {" ", false}, {"*.example.com", true}, {"*.*.example.com", true}, {"sub.*.example.com", false}, {"*sub.example.com", false}, {"**.tld", false}, {"*", true}, {"*.tld", true}, {".tld", false}, {"example.com.", false}, {"localhost", true}, {"foo.localhost", true}, {"local", true}, {"192.168.1.3", true}, {"10.0.2.1", true}, {"169.112.53.4", true}, {"$hostname", false}, {"%HOSTNAME%", false}, {"{hostname}", false}, {"hostname!", false}, {"", false}, {"# hostname", false}, {"// hostname", false}, {"user@hostname", false}, {"hostname;", false}, {`"hostname"`, false}, } { actual := SubjectQualifiesForCert(test.host) if actual != test.expect { t.Errorf("Test %d: Expected SubjectQualifiesForCert(%s)=%v, but got %v", i, test.host, test.expect, actual) } } } func TestSubjectQualifiesForPublicCert(t *testing.T) { for i, test := range []struct { host string expect bool }{ {"hostname", true}, {"example.com", true}, {"sub.example.com", true}, {"Sub.Example.COM", true}, {"127.0.0.1", false}, {"127.0.1.5", false}, {"1.2.3.4", true}, {"69.123.43.94", true}, {"::1", false}, {"::", false}, {"0.0.0.0", false}, {"", false}, {" ", false}, {"*.example.com", true}, {"*.*.example.com", false}, {"sub.*.example.com", false}, {"*sub.example.com", false}, {"*", false}, // won't be trusted by browsers {"*.tld", false}, // won't be trusted by browsers {".tld", false}, {"example.com.", false}, {"localhost", false}, {"foo.localhost", false}, {"local", true}, {"foo.local", false}, {"foo.bar.local", false}, {"foo.internal", false}, {"foo.bar.internal", false}, {"foo.home.arpa", false}, {"foo.bar.home.arpa", false}, {"192.168.1.3", false}, {"10.0.2.1", false}, {"169.112.53.4", true}, {"$hostname", false}, {"%HOSTNAME%", false}, {"{hostname}", false}, {"hostname!", false}, {"", false}, {"# hostname", false}, {"// hostname", false}, {"user@hostname", false}, {"hostname;", false}, {`"hostname"`, false}, } { actual := SubjectQualifiesForPublicCert(test.host) if actual != test.expect { t.Errorf("Test %d: Expected SubjectQualifiesForPublicCert(%s)=%v, but got %v", i, test.host, test.expect, actual) } } } func TestMatchWildcard(t *testing.T) { for i, test := range []struct { subject, wildcard string expect bool }{ {"hostname", "hostname", true}, {"HOSTNAME", "hostname", true}, {"hostname", "HOSTNAME", true}, {"foo.localhost", "foo.localhost", true}, {"foo.localhost", "bar.localhost", false}, {"foo.localhost", "*.localhost", true}, {"bar.localhost", "*.localhost", true}, {"FOO.LocalHost", "*.localhost", true}, {"Bar.localhost", "*.LOCALHOST", true}, {"foo.bar.localhost", "*.localhost", false}, {".localhost", "*.localhost", false}, {"foo.localhost", "foo.*", false}, {"foo.bar.local", "foo.*.local", false}, {"foo.bar.local", "foo.bar.*", false}, {"foo.bar.local", "*.bar.local", true}, {"1.2.3.4.5.6", "*.2.3.4.5.6", true}, {"1.2.3.4.5.6", "*.*.3.4.5.6", true}, {"1.2.3.4.5.6", "*.*.*.4.5.6", true}, {"1.2.3.4.5.6", "*.*.*.*.5.6", true}, {"1.2.3.4.5.6", "*.*.*.*.*.6", true}, {"1.2.3.4.5.6", "*.*.*.*.*.*", true}, {"0.1.2.3.4.5.6", "*.*.*.*.*.*", false}, {"1.2.3.4", "1.2.3.*", false}, // https://tools.ietf.org/html/rfc2818#section-3.1 } { actual := MatchWildcard(test.subject, test.wildcard) if actual != test.expect { t.Errorf("Test %d: Expected MatchWildcard(%s, %s)=%v, but got %v", i, test.subject, test.wildcard, test.expect, actual) } } } golang-github-caddyserver-certmagic-0.25.2/certmagic.go000066400000000000000000000431041514710434200230640ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. // Package certmagic automates the obtaining and renewal of TLS certificates, // including TLS & HTTPS best practices such as robust OCSP stapling, caching, // HTTP->HTTPS redirects, and more. // // Its high-level API serves your HTTP handlers over HTTPS if you simply give // the domain name(s) and the http.Handler; CertMagic will create and run // the HTTPS server for you, fully managing certificates during the lifetime // of the server. Similarly, it can be used to start TLS listeners or return // a ready-to-use tls.Config -- whatever layer you need TLS for, CertMagic // makes it easy. See the HTTPS, Listen, and TLS functions for that. // // If you need more control, create a Cache using NewCache() and then make // a Config using New(). You can then call Manage() on the config. But if // you use this lower-level API, you'll have to be sure to solve the HTTP // and TLS-ALPN challenges yourself (unless you disabled them or use the // DNS challenge) by using the provided Config.GetCertificate function // in your tls.Config and/or Config.HTTPChallengeHandler in your HTTP // handler. // // See the package's README for more instruction. package certmagic import ( "context" "crypto" "crypto/tls" "crypto/x509" "encoding/json" "fmt" "log" "net" "net/http" "os" "sort" "strings" "sync" "time" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) // HTTPS serves mux for all domainNames using the HTTP // and HTTPS ports, redirecting all HTTP requests to HTTPS. // It uses the Default config and a background context. // // This high-level convenience function is opinionated and // applies sane defaults for production use, including // timeouts for HTTP requests and responses. To allow very // long-lived connections, you should make your own // http.Server values and use this package's Listen(), TLS(), // or Config.TLSConfig() functions to customize to your needs. // For example, servers which need to support large uploads or // downloads with slow clients may need to use longer timeouts, // thus this function is not suitable. // // Calling this function signifies your acceptance to // the CA's Subscriber Agreement and/or Terms of Service. func HTTPS(domainNames []string, mux http.Handler) error { ctx := context.Background() if mux == nil { mux = http.DefaultServeMux } DefaultACME.Agreed = true cfg := NewDefault() err := cfg.ManageSync(ctx, domainNames) if err != nil { return err } httpWg.Add(1) defer httpWg.Done() // if we haven't made listeners yet, do so now, // and clean them up when all servers are done lnMu.Lock() if httpLn == nil && httpsLn == nil { httpLn, err = net.Listen("tcp", fmt.Sprintf(":%d", HTTPPort)) if err != nil { lnMu.Unlock() return err } tlsConfig := cfg.TLSConfig() tlsConfig.NextProtos = append([]string{"h2", "http/1.1"}, tlsConfig.NextProtos...) httpsLn, err = tls.Listen("tcp", fmt.Sprintf(":%d", HTTPSPort), tlsConfig) if err != nil { httpLn.Close() httpLn = nil lnMu.Unlock() return err } go func() { httpWg.Wait() lnMu.Lock() httpLn.Close() httpsLn.Close() lnMu.Unlock() }() } hln, hsln := httpLn, httpsLn lnMu.Unlock() // create HTTP/S servers that are configured // with sane default timeouts and appropriate // handlers (the HTTP server solves the HTTP // challenge and issues redirects to HTTPS, // while the HTTPS server simply serves the // user's handler) httpServer := &http.Server{ ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 5 * time.Second, WriteTimeout: 5 * time.Second, IdleTimeout: 5 * time.Second, BaseContext: func(listener net.Listener) context.Context { return ctx }, } if len(cfg.Issuers) > 0 { if am, ok := cfg.Issuers[0].(*ACMEIssuer); ok { httpServer.Handler = am.HTTPChallengeHandler(http.HandlerFunc(httpRedirectHandler)) } } httpsServer := &http.Server{ ReadHeaderTimeout: 10 * time.Second, ReadTimeout: 30 * time.Second, WriteTimeout: 2 * time.Minute, IdleTimeout: 5 * time.Minute, Handler: mux, BaseContext: func(listener net.Listener) context.Context { return ctx }, } log.Printf("%v Serving HTTP->HTTPS on %s and %s", domainNames, hln.Addr(), hsln.Addr()) go httpServer.Serve(hln) return httpsServer.Serve(hsln) } func httpRedirectHandler(w http.ResponseWriter, r *http.Request) { toURL := "https://" // since we redirect to the standard HTTPS port, we // do not need to include it in the redirect URL requestHost := hostOnly(r.Host) toURL += requestHost toURL += r.URL.RequestURI() // get rid of this disgusting unencrypted HTTP connection 🤢 w.Header().Set("Connection", "close") http.Redirect(w, r, toURL, http.StatusMovedPermanently) } // TLS enables management of certificates for domainNames // and returns a valid tls.Config. It uses the Default // config. // // Because this is a convenience function that returns // only a tls.Config, it does not assume HTTP is being // served on the HTTP port, so the HTTP challenge is // disabled (no HTTPChallengeHandler is necessary). The // package variable Default is modified so that the // HTTP challenge is disabled. // // Calling this function signifies your acceptance to // the CA's Subscriber Agreement and/or Terms of Service. func TLS(domainNames []string) (*tls.Config, error) { DefaultACME.Agreed = true DefaultACME.DisableHTTPChallenge = true cfg := NewDefault() return cfg.TLSConfig(), cfg.ManageSync(context.Background(), domainNames) } // Listen manages certificates for domainName and returns a // TLS listener. It uses the Default config. // // Because this convenience function returns only a TLS-enabled // listener and does not presume HTTP is also being served, // the HTTP challenge will be disabled. The package variable // Default is modified so that the HTTP challenge is disabled. // // Calling this function signifies your acceptance to // the CA's Subscriber Agreement and/or Terms of Service. func Listen(domainNames []string) (net.Listener, error) { DefaultACME.Agreed = true DefaultACME.DisableHTTPChallenge = true cfg := NewDefault() err := cfg.ManageSync(context.Background(), domainNames) if err != nil { return nil, err } return tls.Listen("tcp", fmt.Sprintf(":%d", HTTPSPort), cfg.TLSConfig()) } // ManageSync obtains certificates for domainNames and keeps them // renewed using the Default config. // // This is a slightly lower-level function; you will need to // wire up support for the ACME challenges yourself. You can // obtain a Config to help you do that by calling NewDefault(). // // You will need to ensure that you use a TLS config that gets // certificates from this Config and that the HTTP and TLS-ALPN // challenges can be solved. The easiest way to do this is to // use NewDefault().TLSConfig() as your TLS config and to wrap // your HTTP handler with NewDefault().HTTPChallengeHandler(). // If you don't have an HTTP server, you will need to disable // the HTTP challenge. // // If you already have a TLS config you want to use, you can // simply set its GetCertificate field to // NewDefault().GetCertificate. // // Calling this function signifies your acceptance to // the CA's Subscriber Agreement and/or Terms of Service. func ManageSync(ctx context.Context, domainNames []string) error { DefaultACME.Agreed = true return NewDefault().ManageSync(ctx, domainNames) } // ManageAsync is the same as ManageSync, except that // certificates are managed asynchronously. This means // that the function will return before certificates // are ready, and errors that occur during certificate // obtain or renew operations are only logged. It is // vital that you monitor the logs if using this method, // which is only recommended for automated/non-interactive // environments. func ManageAsync(ctx context.Context, domainNames []string) error { DefaultACME.Agreed = true return NewDefault().ManageAsync(ctx, domainNames) } // OnDemandConfig configures on-demand TLS (certificate // operations as-needed, like during TLS handshakes, // rather than immediately). // // When this package's high-level convenience functions // are used (HTTPS, Manage, etc., where the Default // config is used as a template), this struct regulates // certificate operations using an implicit whitelist // containing the names passed into those functions if // no DecisionFunc is set. This ensures some degree of // control by default to avoid certificate operations for // arbitrary domain names. To override this whitelist, // manually specify a DecisionFunc. To impose rate limits, // specify your own DecisionFunc. type OnDemandConfig struct { // If set, this function will be called to determine // whether a certificate can be obtained or renewed // for the given name. If an error is returned, the // request will be denied. IDNs will be given as // punycode. DecisionFunc func(ctx context.Context, name string) error // Sources for getting new, unmanaged certificates. // They will be invoked only during TLS handshakes // before on-demand certificate management occurs, // for certificates that are not already loaded into // the in-memory cache. // // TODO: EXPERIMENTAL: subject to change and/or removal. Managers []Manager // List of allowed hostnames (SNI values) for // deferred (on-demand) obtaining of certificates. // Used only by higher-level functions in this // package to persist the list of hostnames that // the config is supposed to manage. This is done // because it seems reasonable that if you say // "Manage [domain names...]", then only those // domain names should be able to have certs; // we don't NEED this feature, but it makes sense // for higher-level convenience functions to be // able to retain their convenience (alternative // is: the user manually creates a DecisionFunc // that allows the same names it already passed // into Manage) and without letting clients have // their run of any domain names they want. // Only enforced if len > 0. (This is a map to // avoid O(n^2) performance; when it was a slice, // we saw a 30s CPU profile for a config managing // 110K names where 29s was spent checking for // duplicates. Order is not important here.) hostAllowlist map[string]struct{} } // PreChecker is an interface that can be optionally implemented by // Issuers. Pre-checks are performed before each call (or batch of // identical calls) to Issue(), giving the issuer the option to ensure // it has all the necessary information/state. type PreChecker interface { PreCheck(ctx context.Context, names []string, interactive bool) error } // Issuer is a type that can issue certificates. type Issuer interface { // Issue obtains a certificate for the given CSR. It // must honor context cancellation if it is long-running. // It can also use the context to find out if the current // call is part of a retry, via AttemptsCtxKey. Issue(ctx context.Context, request *x509.CertificateRequest) (*IssuedCertificate, error) // IssuerKey must return a string that uniquely identifies // this particular configuration of the Issuer such that // any certificates obtained by this Issuer will be treated // as identical if they have the same SANs. // // Certificates obtained from Issuers with the same IssuerKey // will overwrite others with the same SANs. For example, an // Issuer might be able to obtain certificates from different // CAs, say A and B. It is likely that the CAs have different // use cases and purposes (e.g. testing and production), so // their respective certificates should not overwrite eaach // other. IssuerKey() string } // Revoker can revoke certificates. Reason codes are defined // by RFC 5280 §5.3.1: https://tools.ietf.org/html/rfc5280#section-5.3.1 // and are available as constants in our ACME library. type Revoker interface { Revoke(ctx context.Context, cert CertificateResource, reason int) error } // Manager is a type that manages certificates (keeps them renewed) such // that we can get certificates during TLS handshakes to immediately serve // to clients. // // TODO: This is an EXPERIMENTAL API. It is subject to change/removal. type Manager interface { // GetCertificate returns the certificate to use to complete the handshake. // Since this is called during every TLS handshake, it must be very fast and not block. // Returning any non-nil value indicates that this Manager manages a certificate // for the described handshake. Returning (nil, nil) is valid and is simply treated as // a no-op Return (nil, nil) when the Manager has no certificate for this handshake. // Return an error or a certificate only if the Manager is supposed to get a certificate // for this handshake. Returning (nil, nil) other Managers or Issuers to try to get // a certificate for the handshake. GetCertificate(context.Context, *tls.ClientHelloInfo) (*tls.Certificate, error) } // KeyGenerator can generate a private key. type KeyGenerator interface { // GenerateKey generates a private key. The returned // PrivateKey must be able to expose its associated // public key. GenerateKey() (crypto.PrivateKey, error) } // IssuerPolicy is a type that enumerates how to // choose which issuer to use. EXPERIMENTAL and // subject to change. type IssuerPolicy string // Supported issuer policies. These are subject to change. const ( // UseFirstIssuer uses the first issuer that // successfully returns a certificate. UseFirstIssuer = "first" // UseFirstRandomIssuer shuffles the list of // configured issuers, then uses the first one // that successfully returns a certificate. UseFirstRandomIssuer = "first_random" ) // IssuedCertificate represents a certificate that was just issued. type IssuedCertificate struct { // The PEM-encoding of DER-encoded ASN.1 data. Certificate []byte // Any extra information to serialize alongside the // certificate in storage. It MUST be serializable // as JSON in order to be preserved. Metadata any } // CertificateResource associates a certificate with its private // key and other useful information, for use in maintaining the // certificate. type CertificateResource struct { // The list of names on the certificate; // for convenience only. SANs []string `json:"sans,omitempty"` // The PEM-encoding of DER-encoded ASN.1 data // for the cert or chain. CertificatePEM []byte `json:"-"` // The PEM-encoding of the certificate's private key. PrivateKeyPEM []byte `json:"-"` // Any extra information associated with the certificate, // usually provided by the issuer implementation. IssuerData json.RawMessage `json:"issuer_data,omitempty"` // The unique string identifying the issuer of the // certificate; internally useful for storage access. issuerKey string } // NamesKey returns the list of SANs as a single string, // truncated to some ridiculously long size limit. It // can act as a key for the set of names on the resource. func (cr *CertificateResource) NamesKey() string { sort.Strings(cr.SANs) result := strings.Join(cr.SANs, ",") if len(result) > 1024 { const trunc = "_trunc" result = result[:1024-len(trunc)] + trunc } return result } // Default contains the package defaults for the // various Config fields. This is used as a template // when creating your own Configs with New() or // NewDefault(), and it is also used as the Config // by all the high-level functions in this package // that abstract away most configuration (HTTPS(), // TLS(), Listen(), etc). // // The fields of this value will be used for Config // fields which are unset. Feel free to modify these // defaults, but do not use this Config by itself: it // is only a template. Valid configurations can be // obtained by calling New() (if you have your own // certificate cache) or NewDefault() (if you only // need a single config and want to use the default // cache). // // Even if the Issuers or Storage fields are not set, // defaults will be applied in the call to New(). var Default = Config{ RenewalWindowRatio: DefaultRenewalWindowRatio, Storage: defaultFileStorage, KeySource: DefaultKeyGenerator, Logger: defaultLogger, } // defaultLogger is guaranteed to be a non-nil fallback logger. var defaultLogger = zap.New(zapcore.NewCore( zapcore.NewConsoleEncoder(zap.NewProductionEncoderConfig()), os.Stderr, zap.InfoLevel, )) const ( // HTTPChallengePort is the officially-designated port for // the HTTP challenge according to the ACME spec. HTTPChallengePort = 80 // TLSALPNChallengePort is the officially-designated port for // the TLS-ALPN challenge according to the ACME spec. TLSALPNChallengePort = 443 ) // Port variables must remain their defaults unless you // forward packets from the defaults to whatever these // are set to; otherwise ACME challenges will fail. var ( // HTTPPort is the port on which to serve HTTP // and, as such, the HTTP challenge (unless // Default.AltHTTPPort is set). HTTPPort = 80 // HTTPSPort is the port on which to serve HTTPS // and, as such, the TLS-ALPN challenge // (unless Default.AltTLSALPNPort is set). HTTPSPort = 443 ) // Variables for conveniently serving HTTPS. var ( httpLn, httpsLn net.Listener lnMu sync.Mutex httpWg sync.WaitGroup ) // Maximum size for the stack trace when recovering from panics. const stackTraceBufferSize = 1024 * 128 golang-github-caddyserver-certmagic-0.25.2/certmagic_test.go000066400000000000000000000014731514710434200241260ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "os" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) // TODO var defaultTestLogger = zap.New(zapcore.NewCore( zapcore.NewConsoleEncoder(zap.NewDevelopmentEncoderConfig()), os.Stderr, zap.DebugLevel, )) golang-github-caddyserver-certmagic-0.25.2/config.go000066400000000000000000001376511514710434200224060ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bytes" "context" "crypto" "crypto/rand" "crypto/tls" "crypto/x509" "crypto/x509/pkix" "encoding/asn1" "encoding/json" "encoding/pem" "errors" "fmt" "io/fs" weakrand "math/rand/v2" "net" "net/http" "net/url" "strings" "time" "github.com/mholt/acmez/v3" "github.com/mholt/acmez/v3/acme" "go.uber.org/zap" "golang.org/x/crypto/ocsp" "golang.org/x/net/idna" ) // Config configures a certificate manager instance. // An empty Config is not valid: use New() to obtain // a valid Config. type Config struct { // How much of a certificate's lifetime becomes the // renewal window, which is the span of time at the // end of the certificate's validity period in which // it should be renewed; for most certificates, the // global default is good, but for extremely short- // lived certs, you may want to raise this to ~0.5. // Ratio is remaining:total lifetime. RenewalWindowRatio float64 // An optional event callback clients can set // to subscribe to certain things happening // internally by this config; invocations are // synchronous, so make them return quickly! // Functions should honor context cancellation. // // An error should only be returned to advise // the emitter to abort or cancel an upcoming // event. Some events, especially those that have // already happened, cannot be aborted. For example, // cert_obtaining can be canceled, but // cert_obtained cannot. Emitters may choose to // ignore returned errors. OnEvent func(ctx context.Context, event string, data map[string]any) error // DefaultServerName specifies a server name // to use when choosing a certificate if the // ClientHello's ServerName field is empty. DefaultServerName string // FallbackServerName specifies a server name // to use when choosing a certificate if the // ClientHello's ServerName field doesn't match // any available certificate. // EXPERIMENTAL: Subject to change or removal. FallbackServerName string // The state needed to operate on-demand TLS; // if non-nil, on-demand TLS is enabled and // certificate operations are deferred to // TLS handshakes (or as-needed). // TODO: Can we call this feature "Reactive/Lazy/Passive TLS" instead? OnDemand *OnDemandConfig // Adds the must staple TLS extension to the CSR. MustStaple bool // Sources for getting new, managed certificates; // the default Issuer is ACMEIssuer. If multiple // issuers are specified, they will be tried in // turn until one succeeds. Issuers []Issuer // How to select which issuer to use. // Default: UseFirstIssuer (subject to change). IssuerPolicy IssuerPolicy // If true, private keys already existing in storage // will be reused. Otherwise, a new key will be // created for every new certificate to mitigate // pinning and reduce the scope of key compromise. // Default: false (do not reuse keys). ReusePrivateKeys bool // The source of new private keys for certificates; // the default KeySource is StandardKeyGenerator. KeySource KeyGenerator // CertSelection chooses one of the certificates // with which the ClientHello will be completed; // if not set, DefaultCertificateSelector will // be used. CertSelection CertificateSelector // OCSP configures how OCSP is handled. By default, // OCSP responses are fetched for every certificate // with a responder URL, and cached on disk. Changing // these defaults is STRONGLY discouraged unless you // have a compelling reason to put clients at greater // risk and reduce their privacy. OCSP OCSPConfig // The storage to access when storing or loading // TLS assets. Default is the local file system. Storage Storage // CertMagic will verify the storage configuration // is acceptable before obtaining a certificate // to avoid information loss after an expensive // operation. If you are absolutely 100% sure your // storage is properly configured and has sufficient // space, you can disable this check to reduce I/O // if that is expensive for you. // EXPERIMENTAL: Subject to change or removal. DisableStorageCheck bool // SubjectTransformer is a hook that can transform the // subject (SAN) of a certificate being loaded or issued. // For example, a common use case is to replace the // left-most label with an asterisk (*) to become a // wildcard certificate. // EXPERIMENTAL: Subject to change or removal. SubjectTransformer func(ctx context.Context, domain string) string // Disables both ARI fetching and the use of ARI for renewal decisions. // TEMPORARY: Will likely be removed in the future. DisableARI bool // Set a logger to enable logging. If not set, // a default logger will be created. Logger *zap.Logger // required pointer to the in-memory cert cache certCache *Cache } // NewDefault makes a valid config based on the package // Default config. Most users will call this function // instead of New() since most use cases require only a // single config for any and all certificates. // // If your requirements are more advanced (for example, // multiple configs depending on the certificate), then use // New() instead. (You will need to make your own Cache // first.) If you only need a single Config to manage your // certs (even if that config changes, as long as it is the // only one), customize the Default package variable before // calling NewDefault(). // // All calls to NewDefault() will return configs that use the // same, default certificate cache. All configs returned // by NewDefault() are based on the values of the fields of // Default at the time it is called. // // This is the only way to get a config that uses the // default certificate cache. func NewDefault() *Config { defaultCacheMu.Lock() if defaultCache == nil { defaultCache = NewCache(CacheOptions{ // the cache will likely need to renew certificates, // so it will need to know how to do that, which // depends on the certificate being managed and which // can change during the lifetime of the cache; this // callback makes it possible to get the latest and // correct config with which to manage the cert, // but if the user does not provide one, we can only // assume that we are to use the default config GetConfigForCert: func(Certificate) (*Config, error) { return NewDefault(), nil }, Logger: Default.Logger, }) } certCache := defaultCache defaultCacheMu.Unlock() return newWithCache(certCache, Default) } // New makes a new, valid config based on cfg and // uses the provided certificate cache. certCache // MUST NOT be nil or this function will panic. // // Use this method when you have an advanced use case // that requires a custom certificate cache and config // that may differ from the Default. For example, if // not all certificates are managed/renewed the same // way, you need to make your own Cache value with a // GetConfigForCert callback that returns the correct // configuration for each certificate. However, for // the vast majority of cases, there will be only a // single Config, thus the default cache (which always // uses the default Config) and default config will // suffice, and you should use NewDefault() instead. func New(certCache *Cache, cfg Config) *Config { if certCache == nil { panic("a certificate cache is required") } certCache.optionsMu.RLock() getConfigForCert := certCache.options.GetConfigForCert defer certCache.optionsMu.RUnlock() if getConfigForCert == nil { panic("cache must have GetConfigForCert set in its options") } return newWithCache(certCache, cfg) } // newWithCache ensures that cfg is a valid config by populating // zero-value fields from the Default Config. If certCache is // nil, this function panics. func newWithCache(certCache *Cache, cfg Config) *Config { if certCache == nil { panic("cannot make a valid config without a pointer to a certificate cache") } if cfg.OnDemand == nil { cfg.OnDemand = Default.OnDemand } if !cfg.MustStaple { cfg.MustStaple = Default.MustStaple } if cfg.Issuers == nil { cfg.Issuers = Default.Issuers if cfg.Issuers == nil { // at least one issuer is absolutely required if not nil cfg.Issuers = []Issuer{NewACMEIssuer(&cfg, DefaultACME)} } } if cfg.RenewalWindowRatio == 0 { cfg.RenewalWindowRatio = Default.RenewalWindowRatio } if cfg.OnEvent == nil { cfg.OnEvent = Default.OnEvent } if cfg.KeySource == nil { cfg.KeySource = Default.KeySource } if cfg.DefaultServerName == "" { cfg.DefaultServerName = Default.DefaultServerName } if cfg.FallbackServerName == "" { cfg.FallbackServerName = Default.FallbackServerName } if cfg.Storage == nil { cfg.Storage = Default.Storage } if cfg.Logger == nil { cfg.Logger = Default.Logger } // absolutely don't allow a nil storage, // because that would make almost anything // a config can do pointless if cfg.Storage == nil { cfg.Storage = defaultFileStorage } // absolutely don't allow a nil logger either, // because that would result in panics if cfg.Logger == nil { cfg.Logger = defaultLogger } cfg.certCache = certCache return &cfg } // ManageSync causes the certificates for domainNames to be managed // according to cfg. If cfg.OnDemand is not nil, then this simply // allowlists the domain names and defers the certificate operations // to when they are needed. Otherwise, the certificates for each // name are loaded from storage or obtained from the CA if not already // in the cache associated with the Config. If loaded from storage, // they are renewed if they are expiring or expired. It then caches // the certificate in memory and is prepared to serve them up during // TLS handshakes. To change how an already-loaded certificate is // managed, update the cache options relating to getting a config for // a cert. // // Note that name allowlisting for on-demand management only takes // effect if cfg.OnDemand.DecisionFunc is not set (is nil); it will // not overwrite an existing DecisionFunc, nor will it overwrite // its decision; i.e. the implicit allowlist is only used if no // DecisionFunc is set. // // This method is synchronous, meaning that certificates for all // domainNames must be successfully obtained (or renewed) before // it returns. It returns immediately on the first error for any // of the given domainNames. This behavior is recommended for // interactive use (i.e. when an administrator is present) so // that errors can be reported and fixed immediately. func (cfg *Config) ManageSync(ctx context.Context, domainNames []string) error { return cfg.manageAll(ctx, domainNames, false) } // ManageAsync is the same as ManageSync, except that ACME // operations are performed asynchronously (in the background). // This method returns before certificates are ready. It is // crucial that the administrator monitors the logs and is // notified of any errors so that corrective action can be // taken as soon as possible. Any errors returned from this // method occurred before ACME transactions started. // // As long as logs are monitored, this method is typically // recommended for non-interactive environments. // // If there are failures loading, obtaining, or renewing a // certificate, it will be retried with exponential backoff // for up to about 30 days, with a maximum interval of about // 24 hours. Cancelling ctx will cancel retries and shut down // any goroutines spawned by ManageAsync. func (cfg *Config) ManageAsync(ctx context.Context, domainNames []string) error { return cfg.manageAll(ctx, domainNames, true) } // ClientCredentials returns a list of TLS client certificate chains for the given identifiers. // The return value can be used in a tls.Config to enable client authentication using managed certificates. // Any certificates that need to be obtained or renewed for these identifiers will be managed accordingly. func (cfg *Config) ClientCredentials(ctx context.Context, identifiers []string) ([]tls.Certificate, error) { err := cfg.manageAll(ctx, identifiers, false) if err != nil { return nil, err } var chains []tls.Certificate for _, id := range identifiers { certRes, err := cfg.loadCertResourceAnyIssuer(ctx, id) if err != nil { return chains, err } chain, err := tls.X509KeyPair(certRes.CertificatePEM, certRes.PrivateKeyPEM) if err != nil { return chains, err } chains = append(chains, chain) } return chains, nil } func (cfg *Config) manageAll(ctx context.Context, domainNames []string, async bool) error { if ctx == nil { ctx = context.Background() } if cfg.OnDemand != nil && cfg.OnDemand.hostAllowlist == nil { cfg.OnDemand.hostAllowlist = make(map[string]struct{}) } for _, domainName := range domainNames { domainName = normalizedName(domainName) // if on-demand is configured, defer obtain and renew operations if cfg.OnDemand != nil { cfg.OnDemand.hostAllowlist[domainName] = struct{}{} continue } // TODO: consider doing this in a goroutine if async, to utilize multiple cores while loading certs // otherwise, begin management immediately err := cfg.manageOne(ctx, domainName, async) if err != nil { return err } } return nil } func (cfg *Config) manageOne(ctx context.Context, domainName string, async bool) error { // if certificate is already being managed, nothing to do; maintenance will continue certs := cfg.certCache.getAllMatchingCerts(domainName) for _, cert := range certs { if cert.managed { return nil } } // first try loading existing certificate from storage cert, err := cfg.CacheManagedCertificate(ctx, domainName) if err != nil { if !errors.Is(err, fs.ErrNotExist) { return fmt.Errorf("%s: caching certificate: %v", domainName, err) } // if we don't have one in storage, obtain one obtain := func() error { var err error if async { err = cfg.ObtainCertAsync(ctx, domainName) } else { err = cfg.ObtainCertSync(ctx, domainName) } if err != nil { return fmt.Errorf("%s: obtaining certificate: %w", domainName, err) } cert, err = cfg.CacheManagedCertificate(ctx, domainName) if err != nil { return fmt.Errorf("%s: caching certificate after obtaining it: %v", domainName, err) } return nil } if async { // Leave the job name empty so as to allow duplicate 'obtain' // jobs; this is because Caddy calls ManageAsync() before the // previous config is stopped (and before its context is // canceled), which means that if an obtain job is still // running for the same domain, Submit() would not queue the // new one because it is still running, even though it is // (probably) about to be canceled (it might not if the new // config fails to finish loading, however). In any case, we // presume it is safe to enqueue a duplicate obtain job because // either the old one (or sometimes the new one) is about to be // canceled. This seems like reasonable logic for any consumer // of this lib. See https://github.com/caddyserver/caddy/issues/3202 jm.Submit(cfg.Logger, "", obtain) return nil } return obtain() } // for an existing certificate, make sure it is renewed; or if it is revoked, // force a renewal even if it's not expiring renew := func() error { // first, ensure status is not revoked (it was just refreshed in CacheManagedCertificate above) if !cert.Expired() && cert.ocsp != nil && cert.ocsp.Status == ocsp.Revoked { _, err = cfg.forceRenew(ctx, cfg.Logger, cert) return err } // ensure ARI is updated before we check whether the cert needs renewing // (we ignore the second return value because we already check if needs renewing anyway) if !cfg.DisableARI && cert.ari.NeedsRefresh() { cert, _, err = cfg.updateARI(ctx, cert, cfg.Logger) if err != nil { cfg.Logger.Error("updating ARI upon managing", zap.Error(err)) } } // otherwise, simply renew the certificate if needed if cert.NeedsRenewal(cfg) { var err error if async { err = cfg.RenewCertAsync(ctx, domainName, false) } else { err = cfg.RenewCertSync(ctx, domainName, false) } if err != nil { return fmt.Errorf("%s: renewing certificate: %w", domainName, err) } // successful renewal, so update in-memory cache _, err = cfg.reloadManagedCertificate(ctx, cert) if err != nil { return fmt.Errorf("%s: reloading renewed certificate into memory: %v", domainName, err) } } return nil } if async { jm.Submit(cfg.Logger, "renew_"+domainName, renew) return nil } return renew() } // renewLockLease extends the lease duration on an existing lock if the storage // backend supports it. The lease duration is calculated based on the retry attempt // number and includes the certificate obtain timeout. This prevents locks from // expiring during long-running certificate operations with retries. func (cfg *Config) renewLockLease(ctx context.Context, storage Storage, lockKey string, attempt int) error { l, ok := storage.(LockLeaseRenewer) if !ok { return nil } leaseDuration := maxRetryDuration if attempt < len(retryIntervals) && attempt >= 0 { leaseDuration = retryIntervals[attempt] } leaseDuration = leaseDuration + DefaultACME.CertObtainTimeout log := cfg.Logger.Named("renewLockLease") log.Debug("renewing lock lease", zap.String("lockKey", lockKey), zap.Int("attempt", attempt)) err := l.RenewLockLease(ctx, lockKey, leaseDuration) if err == nil { locksMu.Lock() locks[lockKey] = storage locksMu.Unlock() } return err } // ObtainCertSync generates a new private key and obtains a certificate for // name using cfg in the foreground; i.e. interactively and without retries. // It stows the renewed certificate and its assets in storage if successful. // It DOES NOT load the certificate into the in-memory cache. This method // is a no-op if storage already has a certificate for name. func (cfg *Config) ObtainCertSync(ctx context.Context, name string) error { return cfg.obtainCert(ctx, name, true) } // ObtainCertAsync is the same as ObtainCertSync(), except it runs in the // background; i.e. non-interactively, and with retries if it fails. func (cfg *Config) ObtainCertAsync(ctx context.Context, name string) error { return cfg.obtainCert(ctx, name, false) } func (cfg *Config) obtainCert(ctx context.Context, name string, interactive bool) error { if len(cfg.Issuers) == 0 { return fmt.Errorf("no issuers configured; impossible to obtain or check for existing certificate in storage") } log := cfg.Logger.Named("obtain") name = cfg.transformSubject(ctx, log, name) // if storage has all resources for this certificate, obtain is a no-op if cfg.storageHasCertResourcesAnyIssuer(ctx, name) { return nil } // ensure storage is writeable and readable // TODO: this is not necessary every time; should only perform check once every so often for each storage, which may require some global state... err := cfg.checkStorage(ctx) if err != nil { return fmt.Errorf("failed storage check: %v - storage is probably misconfigured", err) } log.Info("acquiring lock", zap.String("identifier", name)) // ensure idempotency of the obtain operation for this name lockKey := cfg.lockKey(certIssueLockOp, name) err = acquireLock(ctx, cfg.Storage, lockKey) if err != nil { return fmt.Errorf("unable to acquire lock '%s': %v", lockKey, err) } defer func() { log.Info("releasing lock", zap.String("identifier", name)) if err := releaseLock(ctx, cfg.Storage, lockKey); err != nil { log.Error("unable to unlock", zap.String("identifier", name), zap.String("lock_key", lockKey), zap.Error(err)) } }() log.Info("lock acquired", zap.String("identifier", name)) f := func(ctx context.Context) error { // renew lease on the lock if the certificate store supports it attempt, ok := ctx.Value(AttemptsCtxKey).(*int) if ok { err = cfg.renewLockLease(ctx, cfg.Storage, lockKey, *attempt) if err != nil { return fmt.Errorf("unable to renew lock lease '%s': %v", lockKey, err) } } // check if obtain is still needed -- might have been obtained during lock if cfg.storageHasCertResourcesAnyIssuer(ctx, name) { log.Info("certificate already exists in storage", zap.String("identifier", name)) return nil } log.Info("obtaining certificate", zap.String("identifier", name)) if err := cfg.emit(ctx, "cert_obtaining", map[string]any{"identifier": name}); err != nil { return fmt.Errorf("obtaining certificate aborted by event handler: %w", err) } // If storage has a private key already, use it; otherwise we'll generate our own. // Also create the slice of issuers we will try using according to any issuer // selection policy (it must be a copy of the slice so we don't mutate original). var privKey crypto.PrivateKey var privKeyPEM []byte var issuers []Issuer if cfg.ReusePrivateKeys { privKey, privKeyPEM, issuers, err = cfg.reusePrivateKey(ctx, name) if err != nil { return err } } else { issuers = make([]Issuer, len(cfg.Issuers)) copy(issuers, cfg.Issuers) } if cfg.IssuerPolicy == UseFirstRandomIssuer { weakrand.Shuffle(len(issuers), func(i, j int) { issuers[i], issuers[j] = issuers[j], issuers[i] }) } if privKey == nil { privKey, err = cfg.KeySource.GenerateKey() if err != nil { return err } privKeyPEM, err = PEMEncodePrivateKey(privKey) if err != nil { return err } } csr, err := cfg.generateCSR(privKey, []string{name}, false) if err != nil { return err } // try to obtain from each issuer until we succeed var issuedCert *IssuedCertificate var issuerUsed Issuer var issuerKeys []string for i, issuer := range issuers { issuerKeys = append(issuerKeys, issuer.IssuerKey()) log.Debug(fmt.Sprintf("trying issuer %d/%d", i+1, len(cfg.Issuers)), zap.String("issuer", issuer.IssuerKey())) if prechecker, ok := issuer.(PreChecker); ok { err = prechecker.PreCheck(ctx, []string{name}, interactive) if err != nil { continue } } // TODO: ZeroSSL's API currently requires CommonName to be set, and requires it be // distinct from SANs. If this was a cert it would violate the BRs, but their certs // are compliant, so their CSR requirements just needlessly add friction, complexity, // and inefficiency for clients. CommonName has been deprecated for 25+ years. useCSR := csr if issuer.IssuerKey() == zerosslIssuerKey { useCSR, err = cfg.generateCSR(privKey, []string{name}, true) if err != nil { return err } } issuedCert, err = issuer.Issue(ctx, useCSR) if err == nil { issuerUsed = issuer break } // err is usually wrapped, which is nice for simply printing it, but // with our structured error logs we only need the problem string errToLog := err var problem acme.Problem if errors.As(err, &problem) { errToLog = problem } log.Error("could not get certificate from issuer", zap.String("identifier", name), zap.String("issuer", issuer.IssuerKey()), zap.Error(errToLog)) } if err != nil { cfg.emit(ctx, "cert_failed", map[string]any{ "renewal": false, "identifier": name, "issuers": issuerKeys, "error": err, }) // only the error from the last issuer will be returned, but we logged the others return fmt.Errorf("[%s] Obtain: %w", name, err) } issuerKey := issuerUsed.IssuerKey() // success - immediately save the certificate resource metaJSON, err := json.Marshal(issuedCert.Metadata) if err != nil { log.Error("unable to encode certificate metadata", zap.Error(err)) } certRes := CertificateResource{ SANs: namesFromCSR(csr), CertificatePEM: issuedCert.Certificate, PrivateKeyPEM: privKeyPEM, IssuerData: metaJSON, issuerKey: issuerUsed.IssuerKey(), } err = cfg.saveCertResource(ctx, issuerUsed, certRes) if err != nil { return fmt.Errorf("[%s] Obtain: saving assets: %v", name, err) } log.Info("certificate obtained successfully", zap.String("identifier", name), zap.String("issuer", issuerUsed.IssuerKey())) certKey := certRes.NamesKey() cfg.emit(ctx, "cert_obtained", map[string]any{ "renewal": false, "identifier": name, "issuer": issuerUsed.IssuerKey(), "storage_path": StorageKeys.CertsSitePrefix(issuerKey, certKey), "private_key_path": StorageKeys.SitePrivateKey(issuerKey, certKey), "certificate_path": StorageKeys.SiteCert(issuerKey, certKey), "metadata_path": StorageKeys.SiteMeta(issuerKey, certKey), "csr_pem": pem.EncodeToMemory(&pem.Block{ Type: "CERTIFICATE REQUEST", Bytes: csr.Raw, }), }) return nil } if interactive { err = f(ctx) } else { err = doWithRetry(ctx, log, f) } return err } // reusePrivateKey looks for a private key for domain in storage in the configured issuers // paths. For the first private key it finds, it returns that key both decoded and PEM-encoded, // as well as the reordered list of issuers to use instead of cfg.Issuers (because if a key // is found, that issuer should be tried first, so it is moved to the front in a copy of // cfg.Issuers). func (cfg *Config) reusePrivateKey(ctx context.Context, domain string) (privKey crypto.PrivateKey, privKeyPEM []byte, issuers []Issuer, err error) { // make a copy of cfg.Issuers so that if we have to reorder elements, we don't // inadvertently mutate the configured issuers (see append calls below) issuers = make([]Issuer, len(cfg.Issuers)) copy(issuers, cfg.Issuers) for i, issuer := range issuers { // see if this issuer location in storage has a private key for the domain privateKeyStorageKey := StorageKeys.SitePrivateKey(issuer.IssuerKey(), domain) privKeyPEM, err = cfg.Storage.Load(ctx, privateKeyStorageKey) if errors.Is(err, fs.ErrNotExist) { err = nil // obviously, it's OK to not have a private key; so don't prevent obtaining a cert continue } if err != nil { return nil, nil, nil, fmt.Errorf("loading existing private key for reuse with issuer %s: %v", issuer.IssuerKey(), err) } // we loaded a private key; try decoding it so we can use it privKey, err = PEMDecodePrivateKey(privKeyPEM) if err != nil { return nil, nil, nil, err } // since the private key was found in storage for this issuer, move it // to the front of the list so we prefer this issuer first issuers = append([]Issuer{issuer}, append(issuers[:i], issuers[i+1:]...)...) break } return } // storageHasCertResourcesAnyIssuer returns true if storage has all the // certificate resources in storage from any configured issuer. It checks // all configured issuers in order. func (cfg *Config) storageHasCertResourcesAnyIssuer(ctx context.Context, name string) bool { for _, iss := range cfg.Issuers { if cfg.storageHasCertResources(ctx, iss, name) { return true } } return false } // RenewCertSync renews the certificate for name using cfg in the foreground; // i.e. interactively and without retries. It stows the renewed certificate // and its assets in storage if successful. It DOES NOT update the in-memory // cache with the new certificate. The certificate will not be renewed if it // is not close to expiring unless force is true. func (cfg *Config) RenewCertSync(ctx context.Context, name string, force bool) error { return cfg.renewCert(ctx, name, force, true) } // RenewCertAsync is the same as RenewCertSync(), except it runs in the // background; i.e. non-interactively, and with retries if it fails. func (cfg *Config) RenewCertAsync(ctx context.Context, name string, force bool) error { return cfg.renewCert(ctx, name, force, false) } func (cfg *Config) renewCert(ctx context.Context, name string, force, interactive bool) error { if len(cfg.Issuers) == 0 { return fmt.Errorf("no issuers configured; impossible to renew or check existing certificate in storage") } log := cfg.Logger.Named("renew") name = cfg.transformSubject(ctx, log, name) // ensure storage is writeable and readable // TODO: this is not necessary every time; should only perform check once every so often for each storage, which may require some global state... err := cfg.checkStorage(ctx) if err != nil { return fmt.Errorf("failed storage check: %v - storage is probably misconfigured", err) } log.Info("acquiring lock", zap.String("identifier", name)) // ensure idempotency of the renew operation for this name lockKey := cfg.lockKey(certIssueLockOp, name) err = acquireLock(ctx, cfg.Storage, lockKey) if err != nil { return fmt.Errorf("unable to acquire lock '%s': %v", lockKey, err) } defer func() { log.Info("releasing lock", zap.String("identifier", name)) if err := releaseLock(ctx, cfg.Storage, lockKey); err != nil { log.Error("unable to unlock", zap.String("identifier", name), zap.String("lock_key", lockKey), zap.Error(err)) } }() log.Info("lock acquired", zap.String("identifier", name)) f := func(ctx context.Context) error { // renew lease on the certificate store lock if the store implementation supports it; // prevents the lock from being acquired by another process/instance while we're renewing attempt, ok := ctx.Value(AttemptsCtxKey).(*int) if ok { err = cfg.renewLockLease(ctx, cfg.Storage, lockKey, *attempt) if err != nil { return fmt.Errorf("unable to renew lock lease '%s': %v", lockKey, err) } } // prepare for renewal (load PEM cert, key, and meta) certRes, err := cfg.loadCertResourceAnyIssuer(ctx, name) if err != nil { return err } // check if renew is still needed - might have been renewed while waiting for lock timeLeft, leaf, needsRenew := cfg.managedCertNeedsRenewal(certRes, false) if !needsRenew { if force { log.Info("certificate does not need to be renewed, but renewal is being forced", zap.String("identifier", name), zap.Duration("remaining", timeLeft)) } else { log.Info("certificate appears to have been renewed already", zap.String("identifier", name), zap.Duration("remaining", timeLeft)) return nil } } log.Info("renewing certificate", zap.String("identifier", name), zap.Duration("remaining", timeLeft)) if err := cfg.emit(ctx, "cert_obtaining", map[string]any{ "renewal": true, "identifier": name, "forced": force, "remaining": timeLeft, "issuer": certRes.issuerKey, // previous/current issuer }); err != nil { return fmt.Errorf("renewing certificate aborted by event handler: %w", err) } // reuse or generate new private key for CSR var privateKey crypto.PrivateKey if cfg.ReusePrivateKeys { privateKey, err = PEMDecodePrivateKey(certRes.PrivateKeyPEM) } else { privateKey, err = cfg.KeySource.GenerateKey() } if err != nil { return err } // if we generated a new key, make sure to replace its PEM encoding too! if !cfg.ReusePrivateKeys { certRes.PrivateKeyPEM, err = PEMEncodePrivateKey(privateKey) if err != nil { return err } } csr, err := cfg.generateCSR(privateKey, []string{name}, false) if err != nil { return err } // try to obtain from each issuer until we succeed var issuedCert *IssuedCertificate var issuerUsed Issuer var issuerKeys []string for _, issuer := range cfg.Issuers { // TODO: ZeroSSL's API currently requires CommonName to be set, and requires it be // distinct from SANs. If this was a cert it would violate the BRs, but their certs // are compliant, so their CSR requirements just needlessly add friction, complexity, // and inefficiency for clients. CommonName has been deprecated for 25+ years. useCSR := csr if issuer.IssuerKey() == "zerossl" { useCSR, err = cfg.generateCSR(privateKey, []string{name}, true) if err != nil { return err } } issuerKeys = append(issuerKeys, issuer.IssuerKey()) if prechecker, ok := issuer.(PreChecker); ok { err = prechecker.PreCheck(ctx, []string{name}, interactive) if err != nil { continue } } // if we're renewing with the same ACME CA as before, have the ACME // client tell the server we are replacing a certificate (but doing // this on the wrong CA, or when the CA doesn't recognize the certID, // can fail the order) -- TODO: change this check to whether we're using the same ACME account, not CA if !cfg.DisableARI { if acmeData, err := certRes.getACMEData(); err == nil && acmeData.CA != "" { if acmeIss, ok := issuer.(*ACMEIssuer); ok { if acmeIss.CA == acmeData.CA { ctx = context.WithValue(ctx, ctxKeyARIReplaces, leaf) } } } } issuedCert, err = issuer.Issue(ctx, useCSR) if err == nil { issuerUsed = issuer break } // err is usually wrapped, which is nice for simply printing it, but // with our structured error logs we only need the problem string errToLog := err var problem acme.Problem if errors.As(err, &problem) { errToLog = problem } log.Error("could not get certificate from issuer", zap.String("identifier", name), zap.String("issuer", issuer.IssuerKey()), zap.Error(errToLog)) } if err != nil { cfg.emit(ctx, "cert_failed", map[string]any{ "renewal": true, "identifier": name, "remaining": timeLeft, "issuers": issuerKeys, "error": err, }) // only the error from the last issuer will be returned, but we logged the others return fmt.Errorf("[%s] Renew: %w", name, err) } issuerKey := issuerUsed.IssuerKey() // success - immediately save the renewed certificate resource metaJSON, err := json.Marshal(issuedCert.Metadata) if err != nil { log.Error("unable to encode certificate metadata", zap.Error(err)) } newCertRes := CertificateResource{ SANs: namesFromCSR(csr), CertificatePEM: issuedCert.Certificate, PrivateKeyPEM: certRes.PrivateKeyPEM, IssuerData: metaJSON, issuerKey: issuerKey, } err = cfg.saveCertResource(ctx, issuerUsed, newCertRes) if err != nil { return fmt.Errorf("[%s] Renew: saving assets: %v", name, err) } log.Info("certificate renewed successfully", zap.String("identifier", name), zap.String("issuer", issuerKey)) certKey := newCertRes.NamesKey() cfg.emit(ctx, "cert_obtained", map[string]any{ "renewal": true, "remaining": timeLeft, "identifier": name, "issuer": issuerKey, "storage_path": StorageKeys.CertsSitePrefix(issuerKey, certKey), "private_key_path": StorageKeys.SitePrivateKey(issuerKey, certKey), "certificate_path": StorageKeys.SiteCert(issuerKey, certKey), "metadata_path": StorageKeys.SiteMeta(issuerKey, certKey), "csr_pem": pem.EncodeToMemory(&pem.Block{ Type: "CERTIFICATE REQUEST", Bytes: csr.Raw, }), }) return nil } if interactive { err = f(ctx) } else { err = doWithRetry(ctx, log, f) } return err } // generateCSR generates a CSR for the given SANs. If useCN is true, CommonName will get the first SAN (TODO: this is only a temporary hack for ZeroSSL API support). func (cfg *Config) generateCSR(privateKey crypto.PrivateKey, sans []string, useCN bool) (*x509.CertificateRequest, error) { csrTemplate := new(x509.CertificateRequest) for _, name := range sans { // identifiers should be converted to punycode before going into the CSR normalizedName, err := idna.ToASCII(name) if err != nil { return nil, fmt.Errorf("converting identifier '%s' to ASCII: %v", name, err) } // TODO: This is a temporary hack to support ZeroSSL API... if useCN && csrTemplate.Subject.CommonName == "" && len(normalizedName) <= 64 { csrTemplate.Subject.CommonName = normalizedName continue } if ip := net.ParseIP(normalizedName); ip != nil { csrTemplate.IPAddresses = append(csrTemplate.IPAddresses, ip) } else if strings.Contains(normalizedName, "@") { csrTemplate.EmailAddresses = append(csrTemplate.EmailAddresses, normalizedName) } else if u, err := url.Parse(normalizedName); err == nil && strings.Contains(normalizedName, "/") { csrTemplate.URIs = append(csrTemplate.URIs, u) } else { csrTemplate.DNSNames = append(csrTemplate.DNSNames, normalizedName) } } if cfg.MustStaple { csrTemplate.ExtraExtensions = append(csrTemplate.ExtraExtensions, mustStapleExtension) } // IP addresses aren't printed here because I'm too lazy to marshal them as strings, but // we at least print the incoming SANs so it should be obvious what became IPs cfg.Logger.Debug("created CSR", zap.Strings("identifiers", sans), zap.Strings("san_dns_names", csrTemplate.DNSNames), zap.Strings("san_emails", csrTemplate.EmailAddresses), zap.String("common_name", csrTemplate.Subject.CommonName), zap.Int("extra_extensions", len(csrTemplate.ExtraExtensions)), ) csrDER, err := x509.CreateCertificateRequest(rand.Reader, csrTemplate, privateKey) if err != nil { return nil, err } return x509.ParseCertificateRequest(csrDER) } // RevokeCert revokes the certificate for domain via ACME protocol. It requires // that cfg.Issuers is properly configured with the same issuer that issued the // certificate being revoked. See RFC 5280 §5.3.1 for reason codes. // // The certificate assets are deleted from storage after successful revocation // to prevent reuse. func (cfg *Config) RevokeCert(ctx context.Context, domain string, reason int, interactive bool) error { for i, issuer := range cfg.Issuers { issuerKey := issuer.IssuerKey() rev, ok := issuer.(Revoker) if !ok { return fmt.Errorf("issuer %d (%s) is not a Revoker", i, issuerKey) } certRes, err := cfg.loadCertResource(ctx, issuer, domain) if err != nil { return err } if !cfg.Storage.Exists(ctx, StorageKeys.SitePrivateKey(issuerKey, domain)) { return fmt.Errorf("private key not found for %s", certRes.SANs) } err = rev.Revoke(ctx, certRes, reason) if err != nil { return fmt.Errorf("issuer %d (%s): %v", i, issuerKey, err) } err = cfg.deleteSiteAssets(ctx, issuerKey, domain) if err != nil { return fmt.Errorf("certificate revoked, but unable to fully clean up assets from issuer %s: %v", issuerKey, err) } } return nil } // TLSConfig is an opinionated method that returns a recommended, modern // TLS configuration that can be used to configure TLS listeners. Aside // from safe, modern defaults, this method sets two critical fields on the // TLS config which are required to enable automatic certificate // management: GetCertificate and NextProtos. // // The GetCertificate field is necessary to get certificates from memory // or storage, including both manual and automated certificates. You // should only change this field if you know what you are doing. // // The NextProtos field is pre-populated with a special value to enable // solving the TLS-ALPN ACME challenge. Because this method does not // assume any particular protocols after the TLS handshake is completed, // you will likely need to customize the NextProtos field by prepending // your application's protocols to the slice. For example, to serve // HTTP, you will need to prepend "h2" and "http/1.1" values. Be sure to // leave the acmez.ACMETLS1Protocol value intact, however, or TLS-ALPN // challenges will fail (which may be acceptable if you are not using // ACME, or specifically, the TLS-ALPN challenge). // // Unlike the package TLS() function, this method does not, by itself, // enable certificate management for any domain names. func (cfg *Config) TLSConfig() *tls.Config { return &tls.Config{ // these two fields necessary for TLS-ALPN challenge GetCertificate: cfg.GetCertificate, NextProtos: []string{acmez.ACMETLS1Protocol}, // the rest recommended for modern TLS servers MinVersion: tls.VersionTLS12, CurvePreferences: []tls.CurveID{ tls.X25519, tls.CurveP256, }, CipherSuites: preferredDefaultCipherSuites(), PreferServerCipherSuites: true, } } // getACMEChallengeInfo loads the challenge info from either the internal challenge memory // or the external storage (implying distributed solving). The second return value // indicates whether challenge info was loaded from external storage. If true, the // challenge is being solved in a distributed fashion; if false, from internal memory. // If no matching challenge information can be found, an error is returned. func (cfg *Config) getACMEChallengeInfo(ctx context.Context, identifier string, allowDistributed bool) (Challenge, bool, error) { // first, check if our process initiated this challenge; if so, just return it chalData, ok := GetACMEChallenge(identifier) if ok { return chalData, false, nil } // if distributed solving is disabled, and we don't have it in memory, return an error if !allowDistributed { return Challenge{}, false, fmt.Errorf("distributed solving disabled and no challenge information found internally for identifier: %s", identifier) } // otherwise, perhaps another instance in the cluster initiated it; check // the configured storage to retrieve challenge data (requires storage) if cfg.Storage == nil { return Challenge{}, false, errors.New("challenge was not initiated internally and no storage is configured for distributed solving") } var chalInfo acme.Challenge var chalInfoBytes []byte var tokenKey string for _, issuer := range cfg.Issuers { ds := distributedSolver{ storage: cfg.Storage, storageKeyIssuerPrefix: storageKeyACMECAPrefix(issuer.IssuerKey()), } tokenKey = ds.challengeTokensKey(identifier) var err error chalInfoBytes, err = cfg.Storage.Load(ctx, tokenKey) if err == nil { break } if errors.Is(err, fs.ErrNotExist) { continue } return Challenge{}, false, fmt.Errorf("opening distributed challenge token file %s: %v", tokenKey, err) } if len(chalInfoBytes) == 0 { return Challenge{}, false, fmt.Errorf("no information found to solve challenge for identifier: %s", identifier) } err := json.Unmarshal(chalInfoBytes, &chalInfo) if err != nil { return Challenge{}, false, fmt.Errorf("decoding challenge token file %s (corrupted?): %v", tokenKey, err) } return Challenge{Challenge: chalInfo}, true, nil } func (cfg *Config) transformSubject(ctx context.Context, logger *zap.Logger, name string) string { if cfg.SubjectTransformer == nil { return name } transformedName := cfg.SubjectTransformer(ctx, name) if logger != nil && transformedName != name { logger.Debug("transformed subject name", zap.String("original", name), zap.String("transformed", transformedName)) } return transformedName } // checkStorage tests the storage by writing random bytes // to a random key, and then loading those bytes and // comparing the loaded value. If this fails, the provided // cfg.Storage mechanism should not be used. func (cfg *Config) checkStorage(ctx context.Context) error { if cfg.DisableStorageCheck { return nil } key := fmt.Sprintf("rw_test_%d", weakrand.Int()) contents := make([]byte, 1024*10) // size sufficient for one or two ACME resources // This is how ChaCha8.Read works, without handling the case where the slice length is not a multiple of 8. // This also avoids the use of a mutex and an import. for i := 0; i < len(contents); i += 8 { v := weakrand.Uint64() contents[i] = byte(v) contents[i+1] = byte(v >> 8) contents[i+2] = byte(v >> 16) contents[i+3] = byte(v >> 24) contents[i+4] = byte(v >> 32) contents[i+5] = byte(v >> 40) contents[i+6] = byte(v >> 48) contents[i+7] = byte(v >> 56) } err := cfg.Storage.Store(ctx, key, contents) if err != nil { return err } defer func() { deleteErr := cfg.Storage.Delete(ctx, key) if deleteErr != nil { cfg.Logger.Error("deleting test key from storage", zap.String("key", key), zap.Error(err)) } // if there was no other error, make sure // to return any error returned from Delete if err == nil { err = deleteErr } }() loaded, err := cfg.Storage.Load(ctx, key) if err != nil { return err } if !bytes.Equal(contents, loaded) { return fmt.Errorf("load yielded different value than was stored; expected %d bytes, got %d bytes of differing elements", len(contents), len(loaded)) } return nil } // storageHasCertResources returns true if the storage // associated with cfg's certificate cache has all the // resources related to the certificate for domain: the // certificate, the private key, and the metadata. func (cfg *Config) storageHasCertResources(ctx context.Context, issuer Issuer, domain string) bool { issuerKey := issuer.IssuerKey() certKey := StorageKeys.SiteCert(issuerKey, domain) keyKey := StorageKeys.SitePrivateKey(issuerKey, domain) metaKey := StorageKeys.SiteMeta(issuerKey, domain) return cfg.Storage.Exists(ctx, certKey) && cfg.Storage.Exists(ctx, keyKey) && cfg.Storage.Exists(ctx, metaKey) } // deleteSiteAssets deletes the folder in storage containing the // certificate, private key, and metadata file for domain from the // issuer with the given issuer key. func (cfg *Config) deleteSiteAssets(ctx context.Context, issuerKey, domain string) error { err := cfg.Storage.Delete(ctx, StorageKeys.SiteCert(issuerKey, domain)) if err != nil { return fmt.Errorf("deleting certificate file: %v", err) } err = cfg.Storage.Delete(ctx, StorageKeys.SitePrivateKey(issuerKey, domain)) if err != nil { return fmt.Errorf("deleting private key: %v", err) } err = cfg.Storage.Delete(ctx, StorageKeys.SiteMeta(issuerKey, domain)) if err != nil { return fmt.Errorf("deleting metadata file: %v", err) } err = cfg.Storage.Delete(ctx, StorageKeys.CertsSitePrefix(issuerKey, domain)) if err != nil { return fmt.Errorf("deleting site asset folder: %v", err) } return nil } // lockKey returns a key for a lock that is specific to the operation // named op being performed related to domainName and this config's CA. func (cfg *Config) lockKey(op, domainName string) string { return fmt.Sprintf("%s_%s", op, domainName) } // managedCertNeedsRenewal returns true if certRes is expiring soon or already expired, // or if the process of decoding the cert and checking its expiration returned an error. // If there wasn't an error, the leaf cert is also returned, so it can be reused if // necessary, since we are parsing the PEM bundle anyway. func (cfg *Config) managedCertNeedsRenewal(certRes CertificateResource, emitLogs bool) (time.Duration, *x509.Certificate, bool) { certChain, err := parseCertsFromPEMBundle(certRes.CertificatePEM) if err != nil || len(certChain) == 0 { return 0, nil, true } var ari acme.RenewalInfo if !cfg.DisableARI { if ariPtr, err := certRes.getARI(); err == nil && ariPtr != nil { ari = *ariPtr } } remaining := time.Until(expiresAt(certChain[0])) return remaining, certChain[0], cfg.certNeedsRenewal(certChain[0], ari, emitLogs) } func (cfg *Config) emit(ctx context.Context, eventName string, data map[string]any) error { if cfg.OnEvent == nil { return nil } return cfg.OnEvent(ctx, eventName, data) } // CertificateSelector is a type which can select a certificate to use given multiple choices. type CertificateSelector interface { SelectCertificate(*tls.ClientHelloInfo, []Certificate) (Certificate, error) } // OCSPConfig configures how OCSP is handled. type OCSPConfig struct { // Disable automatic OCSP stapling; strongly // discouraged unless you have a good reason. // Disabling this puts clients at greater risk // and reduces their privacy. DisableStapling bool // A map of OCSP responder domains to replacement // domains for querying OCSP servers. Used for // overriding the OCSP responder URL that is // embedded in certificates. Mapping to an empty // URL will disable OCSP from that responder. ResponderOverrides map[string]string // Optionally specify a function that can return the URL // for an HTTP proxy to use for OCSP-related HTTP requests. HTTPProxy func(*http.Request) (*url.URL, error) } // certIssueLockOp is the name of the operation used // when naming a lock to make it mutually exclusive // with other certificate issuance operations for a // certain name. const certIssueLockOp = "issue_cert" // Constants for PKIX MustStaple extension. var ( tlsFeatureExtensionOID = asn1.ObjectIdentifier{1, 3, 6, 1, 5, 5, 7, 1, 24} ocspMustStapleFeature = []byte{0x30, 0x03, 0x02, 0x01, 0x05} mustStapleExtension = pkix.Extension{ Id: tlsFeatureExtensionOID, Value: ocspMustStapleFeature, } ) golang-github-caddyserver-certmagic-0.25.2/config_test.go000066400000000000000000000111571514710434200234350ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bytes" "context" "encoding/json" "os" "reflect" "testing" "time" "github.com/caddyserver/certmagic/internal/testutil" "github.com/mholt/acmez/v3/acme" ) func TestSaveCertResource(t *testing.T) { ctx := context.Background() am := &ACMEIssuer{CA: "https://example.com/acme/directory"} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata_tmp"}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() domain := "example.com" certContents := "certificate" keyContents := "private key" cert := CertificateResource{ SANs: []string{domain}, PrivateKeyPEM: []byte(keyContents), CertificatePEM: []byte(certContents), IssuerData: mustJSON(acme.Certificate{ URL: "https://example.com/cert", }), issuerKey: am.IssuerKey(), } err := testConfig.saveCertResource(ctx, am, cert) if err != nil { t.Fatalf("Expected no error, got: %v", err) } siteData, err := testConfig.loadCertResource(ctx, am, domain) if err != nil { t.Fatalf("Expected no error reading site, got: %v", err) } siteData.IssuerData = bytes.ReplaceAll(siteData.IssuerData, []byte("\t"), []byte("")) siteData.IssuerData = bytes.ReplaceAll(siteData.IssuerData, []byte("\n"), []byte("")) siteData.IssuerData = bytes.ReplaceAll(siteData.IssuerData, []byte(" "), []byte("")) if !reflect.DeepEqual(cert, siteData) { t.Errorf("Expected '%+v' to match '%+v'\n%s\n%s", cert.IssuerData, siteData.IssuerData, string(cert.IssuerData), string(siteData.IssuerData)) } } type mockStorageWithLease struct { *FileStorage renewCalled bool renewError error lastLockKey string lastDuration time.Duration } func (m *mockStorageWithLease) RenewLockLease(ctx context.Context, lockKey string, leaseDuration time.Duration) error { m.renewCalled = true m.lastLockKey = lockKey m.lastDuration = leaseDuration return m.renewError } func TestRenewLockLeaseDuration(t *testing.T) { ctx := context.Background() tmpDir, err := os.MkdirTemp(os.TempDir(), "certmagic-test*") testutil.RequireNoError(t, err, "allocating tmp dir") defer os.RemoveAll(tmpDir) mockStorage := &mockStorageWithLease{ FileStorage: &FileStorage{Path: tmpDir}, } // Test attempt 0 cfg := &Config{Logger: defaultTestLogger} cfg.renewLockLease(ctx, mockStorage, "test-lock", 0) expected := retryIntervals[0] + DefaultACME.CertObtainTimeout testutil.RequireEqual(t, expected, mockStorage.lastDuration) // Test attempt beyond array bounds cfg.renewLockLease(ctx, mockStorage, "test-lock", 999) expected = maxRetryDuration + DefaultACME.CertObtainTimeout testutil.RequireEqual(t, expected, mockStorage.lastDuration) } // Test that lease renewal works when storage supports it func TestRenewLockLeaseWithInterface(t *testing.T) { ctx := context.Background() tmpDir, err := os.MkdirTemp(os.TempDir(), "certmagic-test*") testutil.RequireNoError(t, err, "allocating tmp dir") defer os.RemoveAll(tmpDir) mockStorage := &mockStorageWithLease{ FileStorage: &FileStorage{Path: tmpDir}, } cfg := &Config{Logger: defaultTestLogger} err = cfg.renewLockLease(ctx, mockStorage, "test-lock", 0) testutil.RequireNoError(t, err) testutil.RequireEqual(t, true, mockStorage.renewCalled) } // Test that no error occurs when storage doesn't support lease renewal func TestRenewLockLeaseWithoutInterface(t *testing.T) { ctx := context.Background() tmpDir, err := os.MkdirTemp(os.TempDir(), "certmagic-test*") testutil.RequireNoError(t, err, "allocating tmp dir") defer os.RemoveAll(tmpDir) storage := &FileStorage{Path: tmpDir} cfg := &Config{Logger: defaultTestLogger} err = cfg.renewLockLease(ctx, storage, "test-lock", 0) testutil.RequireNoError(t, err) } func mustJSON(val any) []byte { result, err := json.Marshal(val) if err != nil { panic("marshaling JSON: " + err.Error()) } return result } golang-github-caddyserver-certmagic-0.25.2/crypto.go000066400000000000000000000270271514710434200224540ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto" "crypto/ecdsa" "crypto/ed25519" "crypto/elliptic" "crypto/rand" "crypto/rsa" "crypto/tls" "crypto/x509" "encoding/json" "encoding/pem" "errors" "fmt" "hash/fnv" "io/fs" "sort" "strings" "github.com/klauspost/cpuid/v2" "github.com/zeebo/blake3" "go.uber.org/zap" "golang.org/x/net/idna" ) // PEMEncodePrivateKey marshals a private key into a PEM-encoded block. // The private key must be one of *ecdsa.PrivateKey, *rsa.PrivateKey, or // *ed25519.PrivateKey. func PEMEncodePrivateKey(key crypto.PrivateKey) ([]byte, error) { var pemType string var keyBytes []byte switch key := key.(type) { case *ecdsa.PrivateKey: var err error pemType = "EC" keyBytes, err = x509.MarshalECPrivateKey(key) if err != nil { return nil, err } case *rsa.PrivateKey: pemType = "RSA" keyBytes = x509.MarshalPKCS1PrivateKey(key) case ed25519.PrivateKey: var err error pemType = "ED25519" keyBytes, err = x509.MarshalPKCS8PrivateKey(key) if err != nil { return nil, err } default: return nil, fmt.Errorf("unsupported key type: %T", key) } pemKey := pem.Block{Type: pemType + " PRIVATE KEY", Bytes: keyBytes} return pem.EncodeToMemory(&pemKey), nil } // PEMDecodePrivateKey loads a PEM-encoded ECC/RSA private key from an array of bytes. // Borrowed from Go standard library, to handle various private key and PEM block types. func PEMDecodePrivateKey(keyPEMBytes []byte) (crypto.Signer, error) { // Modified from original: // https://github.com/golang/go/blob/693748e9fa385f1e2c3b91ca9acbb6c0ad2d133d/src/crypto/tls/tls.go#L291-L308 // https://github.com/golang/go/blob/693748e9fa385f1e2c3b91ca9acbb6c0ad2d133d/src/crypto/tls/tls.go#L238 keyBlockDER, _ := pem.Decode(keyPEMBytes) if keyBlockDER == nil { return nil, fmt.Errorf("failed to decode PEM block containing private key") } if keyBlockDER.Type != "PRIVATE KEY" && !strings.HasSuffix(keyBlockDER.Type, " PRIVATE KEY") { return nil, fmt.Errorf("unknown PEM header %q", keyBlockDER.Type) } if key, err := x509.ParsePKCS1PrivateKey(keyBlockDER.Bytes); err == nil { return key, nil } if key, err := x509.ParsePKCS8PrivateKey(keyBlockDER.Bytes); err == nil { switch key := key.(type) { case *rsa.PrivateKey, *ecdsa.PrivateKey, ed25519.PrivateKey: return key.(crypto.Signer), nil default: return nil, fmt.Errorf("found unknown private key type in PKCS#8 wrapping: %T", key) } } if key, err := x509.ParseECPrivateKey(keyBlockDER.Bytes); err == nil { return key, nil } return nil, fmt.Errorf("unknown private key type") } // parseCertsFromPEMBundle parses a certificate bundle from top to bottom and returns // a slice of x509 certificates. This function will error if no certificates are found. func parseCertsFromPEMBundle(bundle []byte) ([]*x509.Certificate, error) { var certificates []*x509.Certificate var certDERBlock *pem.Block for { certDERBlock, bundle = pem.Decode(bundle) if certDERBlock == nil { break } if certDERBlock.Type == "CERTIFICATE" { cert, err := x509.ParseCertificate(certDERBlock.Bytes) if err != nil { return nil, err } certificates = append(certificates, cert) } } if len(certificates) == 0 { return nil, fmt.Errorf("no certificates found in bundle") } return certificates, nil } // fastHash hashes input using a hashing algorithm that // is fast, and returns the hash as a hex-encoded string. // Do not use this for cryptographic purposes. func fastHash(input []byte) string { h := fnv.New32a() h.Write(input) return fmt.Sprintf("%x", h.Sum32()) } // saveCertResource saves the certificate resource to disk. This // includes the certificate file itself, the private key, and the // metadata file. func (cfg *Config) saveCertResource(ctx context.Context, issuer Issuer, cert CertificateResource) error { metaBytes, err := json.MarshalIndent(cert, "", "\t") if err != nil { return fmt.Errorf("encoding certificate metadata: %v", err) } issuerKey := issuer.IssuerKey() certKey := cert.NamesKey() all := []keyValue{ { key: StorageKeys.SitePrivateKey(issuerKey, certKey), value: cert.PrivateKeyPEM, }, { key: StorageKeys.SiteCert(issuerKey, certKey), value: cert.CertificatePEM, }, { key: StorageKeys.SiteMeta(issuerKey, certKey), value: metaBytes, }, } return storeTx(ctx, cfg.Storage, all) } // loadCertResourceAnyIssuer loads and returns the certificate resource from any // of the configured issuers. If multiple are found (e.g. if there are 3 issuers // configured, and all 3 have a resource matching certNamesKey), then the newest // (latest NotBefore date) resource will be chosen. func (cfg *Config) loadCertResourceAnyIssuer(ctx context.Context, certNamesKey string) (CertificateResource, error) { // we can save some extra decoding steps if there's only one issuer, since // we don't need to compare potentially multiple available resources to // select the best one, when there's only one choice anyway if len(cfg.Issuers) == 1 { return cfg.loadCertResource(ctx, cfg.Issuers[0], certNamesKey) } type decodedCertResource struct { CertificateResource issuer Issuer decoded *x509.Certificate } var certResources []decodedCertResource var lastErr error // load and decode all certificate resources found with the // configured issuers so we can sort by newest for _, issuer := range cfg.Issuers { certRes, err := cfg.loadCertResource(ctx, issuer, certNamesKey) if err != nil { if errors.Is(err, fs.ErrNotExist) { // not a problem, but we need to remember the error // in case we end up not finding any cert resources // since we'll need an error to return in that case lastErr = err continue } return CertificateResource{}, err } certs, err := parseCertsFromPEMBundle(certRes.CertificatePEM) if err != nil { return CertificateResource{}, err } certResources = append(certResources, decodedCertResource{ CertificateResource: certRes, issuer: issuer, decoded: certs[0], }) } if len(certResources) == 0 { if lastErr == nil { lastErr = fmt.Errorf("no certificate resources found") // just in case; e.g. no Issuers configured } return CertificateResource{}, lastErr } // sort by date so the most recently issued comes first sort.Slice(certResources, func(i, j int) bool { return certResources[j].decoded.NotBefore.Before(certResources[i].decoded.NotBefore) }) cfg.Logger.Debug("loading managed certificate", zap.String("domain", certNamesKey), zap.Time("expiration", expiresAt(certResources[0].decoded)), zap.String("issuer_key", certResources[0].issuer.IssuerKey()), zap.Any("storage", cfg.Storage), ) return certResources[0].CertificateResource, nil } // loadCertResource loads a certificate resource from the given issuer's storage location. func (cfg *Config) loadCertResource(ctx context.Context, issuer Issuer, certNamesKey string) (CertificateResource, error) { certRes := CertificateResource{issuerKey: issuer.IssuerKey()} // don't use the Lookup profile because we might be loading a wildcard cert which is rejected by the Lookup profile normalizedName, err := idna.ToASCII(certNamesKey) if err != nil { return CertificateResource{}, fmt.Errorf("converting '%s' to ASCII: %v", certNamesKey, err) } keyBytes, err := cfg.Storage.Load(ctx, StorageKeys.SitePrivateKey(certRes.issuerKey, normalizedName)) if err != nil { return CertificateResource{}, err } certRes.PrivateKeyPEM = keyBytes certBytes, err := cfg.Storage.Load(ctx, StorageKeys.SiteCert(certRes.issuerKey, normalizedName)) if err != nil { return CertificateResource{}, err } certRes.CertificatePEM = certBytes metaBytes, err := cfg.Storage.Load(ctx, StorageKeys.SiteMeta(certRes.issuerKey, normalizedName)) if err != nil { return CertificateResource{}, err } err = json.Unmarshal(metaBytes, &certRes) if err != nil { return CertificateResource{}, fmt.Errorf("decoding certificate metadata: %v", err) } return certRes, nil } // hashCertificateChain computes the unique hash of certChain, // which is the chain of DER-encoded bytes. It returns the // hex encoding of the hash. func hashCertificateChain(certChain [][]byte) string { h := blake3.New() for _, certInChain := range certChain { h.Write(certInChain) } return fmt.Sprintf("%x", h.Sum(nil)) } func namesFromCSR(csr *x509.CertificateRequest) []string { var nameSet []string // TODO: CommonName should not be used (it has been deprecated for 25+ years, // but ZeroSSL CA still requires it to be filled out and not overlap SANs...) if csr.Subject.CommonName != "" { nameSet = append(nameSet, csr.Subject.CommonName) } nameSet = append(nameSet, csr.DNSNames...) nameSet = append(nameSet, csr.EmailAddresses...) for _, v := range csr.IPAddresses { nameSet = append(nameSet, v.String()) } for _, v := range csr.URIs { nameSet = append(nameSet, v.String()) } return nameSet } // preferredDefaultCipherSuites returns an appropriate // cipher suite to use depending on hardware support // for AES-NI. // // See https://github.com/mholt/caddy/issues/1674 func preferredDefaultCipherSuites() []uint16 { if cpuid.CPU.Supports(cpuid.AESNI) { return defaultCiphersPreferAES } return defaultCiphersPreferChaCha } var ( defaultCiphersPreferAES = []uint16{ tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, } defaultCiphersPreferChaCha = []uint16{ tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, } ) // StandardKeyGenerator is the standard, in-memory key source // that uses crypto/rand. type StandardKeyGenerator struct { // The type of keys to generate. KeyType KeyType } // GenerateKey generates a new private key according to kg.KeyType. func (kg StandardKeyGenerator) GenerateKey() (crypto.PrivateKey, error) { switch kg.KeyType { case ED25519: _, priv, err := ed25519.GenerateKey(rand.Reader) return priv, err case "", P256: return ecdsa.GenerateKey(elliptic.P256(), rand.Reader) case P384: return ecdsa.GenerateKey(elliptic.P384(), rand.Reader) case RSA2048: return rsa.GenerateKey(rand.Reader, 2048) case RSA4096: return rsa.GenerateKey(rand.Reader, 4096) case RSA8192: return rsa.GenerateKey(rand.Reader, 8192) } return nil, fmt.Errorf("unrecognized or unsupported key type: %s", kg.KeyType) } // DefaultKeyGenerator is the default key source. var DefaultKeyGenerator = StandardKeyGenerator{KeyType: P256} // KeyType enumerates the known/supported key types. type KeyType string // Constants for all key types we support. const ( ED25519 = KeyType("ed25519") P256 = KeyType("p256") P384 = KeyType("p384") RSA2048 = KeyType("rsa2048") RSA4096 = KeyType("rsa4096") RSA8192 = KeyType("rsa8192") ) golang-github-caddyserver-certmagic-0.25.2/crypto_test.go000066400000000000000000000051331514710434200235050ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. // //go:debug rsa1024min=0 package certmagic import ( "bytes" "crypto" "crypto/ecdsa" "crypto/ed25519" "crypto/elliptic" "crypto/rand" "crypto/rsa" "crypto/x509" "testing" ) func TestEncodeDecodeRSAPrivateKey(t *testing.T) { privateKey, err := rsa.GenerateKey(rand.Reader, 128) // make tests faster; small key size OK for testing if err != nil { t.Fatal(err) } // test save savedBytes, err := PEMEncodePrivateKey(privateKey) if err != nil { t.Fatal("error saving private key:", err) } // test load loadedKey, err := PEMDecodePrivateKey(savedBytes) if err != nil { t.Error("error loading private key:", err) } // test load (should fail) _, err = PEMDecodePrivateKey(savedBytes[2:]) if err == nil { t.Error("loading private key should have failed") } // verify loaded key is correct if !privateKeysSame(privateKey, loadedKey) { t.Error("Expected key bytes to be the same, but they weren't") } } func TestSaveAndLoadECCPrivateKey(t *testing.T) { privateKey, err := ecdsa.GenerateKey(elliptic.P384(), rand.Reader) if err != nil { t.Fatal(err) } // test save savedBytes, err := PEMEncodePrivateKey(privateKey) if err != nil { t.Fatal("error saving private key:", err) } // test load loadedKey, err := PEMDecodePrivateKey(savedBytes) if err != nil { t.Error("error loading private key:", err) } // verify loaded key is correct if !privateKeysSame(privateKey, loadedKey) { t.Error("Expected key bytes to be the same, but they weren't") } } // privateKeysSame compares the bytes of a and b and returns true if they are the same. func privateKeysSame(a, b crypto.PrivateKey) bool { return bytes.Equal(privateKeyBytes(a), privateKeyBytes(b)) } // privateKeyBytes returns the bytes of DER-encoded key. func privateKeyBytes(key crypto.PrivateKey) []byte { var keyBytes []byte switch key := key.(type) { case *rsa.PrivateKey: keyBytes = x509.MarshalPKCS1PrivateKey(key) case *ecdsa.PrivateKey: keyBytes, _ = x509.MarshalECPrivateKey(key) case ed25519.PrivateKey: return key } return keyBytes } golang-github-caddyserver-certmagic-0.25.2/dnsutil.go000066400000000000000000000254051514710434200226140ustar00rootroot00000000000000package certmagic import ( "context" "errors" "fmt" "net" "strings" "sync" "time" "github.com/miekg/dns" "go.uber.org/zap" ) // Code in this file adapted from go-acme/lego, July 2020: // https://github.com/go-acme/lego // by Ludovic Fernandez and Dominik Menke // // It has been modified. // FindZoneByFQDN determines the zone apex for the given fully-qualified // domain name (FQDN) by recursing up the domain labels until the nameserver // returns a SOA record in the answer section. The logger must be non-nil. // // EXPERIMENTAL: This API was previously unexported, and may be changed or // unexported again in the future. Do not rely on it at this time. func FindZoneByFQDN(ctx context.Context, logger *zap.Logger, fqdn string, nameservers []string) (string, error) { if !strings.HasSuffix(fqdn, ".") { fqdn += "." } soa, err := lookupSoaByFqdn(ctx, logger, fqdn, nameservers) if err != nil { return "", err } return soa.zone, nil } func lookupSoaByFqdn(ctx context.Context, logger *zap.Logger, fqdn string, nameservers []string) (*soaCacheEntry, error) { logger = logger.Named("soa_lookup") if !strings.HasSuffix(fqdn, ".") { fqdn += "." } fqdnSOACacheMu.Lock() defer fqdnSOACacheMu.Unlock() if err := ctx.Err(); err != nil { return nil, err } // prefer cached version if fresh if ent := fqdnSOACache[fqdn]; ent != nil && !ent.isExpired() { logger.Debug("using cached SOA result", zap.String("entry", ent.zone)) return ent, nil } ent, err := fetchSoaByFqdn(ctx, logger, fqdn, nameservers) if err != nil { return nil, err } // save result to cache, but don't allow // the cache to grow out of control if len(fqdnSOACache) >= 1000 { for key := range fqdnSOACache { delete(fqdnSOACache, key) break } } fqdnSOACache[fqdn] = ent return ent, nil } func fetchSoaByFqdn(ctx context.Context, logger *zap.Logger, fqdn string, nameservers []string) (*soaCacheEntry, error) { var err error var in *dns.Msg labelIndexes := dns.Split(fqdn) for _, index := range labelIndexes { if err := ctx.Err(); err != nil { return nil, err } domain := fqdn[index:] in, err = dnsQuery(ctx, domain, dns.TypeSOA, nameservers, true) if err != nil { continue } if in == nil { continue } logger.Debug("fetched SOA", zap.String("msg", in.String())) switch in.Rcode { case dns.RcodeSuccess: // Check if we got a SOA RR in the answer section if len(in.Answer) == 0 { continue } // CNAME records cannot/should not exist at the root of a zone. // So we skip a domain when a CNAME is found. if dnsMsgContainsCNAME(in) { continue } for _, ans := range in.Answer { if soa, ok := ans.(*dns.SOA); ok { return newSoaCacheEntry(soa), nil } } case dns.RcodeNameError: // NXDOMAIN default: // Any response code other than NOERROR and NXDOMAIN is treated as error return nil, fmt.Errorf("unexpected response code '%s' for %s", dns.RcodeToString[in.Rcode], domain) } } return nil, fmt.Errorf("could not find the start of authority for %s%s", fqdn, formatDNSError(in, err)) } // dnsMsgContainsCNAME checks for a CNAME answer in msg func dnsMsgContainsCNAME(msg *dns.Msg) bool { for _, ans := range msg.Answer { if _, ok := ans.(*dns.CNAME); ok { return true } } return false } func dnsQuery(ctx context.Context, fqdn string, rtype uint16, nameservers []string, recursive bool) (*dns.Msg, error) { m := createDNSMsg(fqdn, rtype, recursive) var in *dns.Msg var err error for _, ns := range nameservers { in, err = sendDNSQuery(ctx, m, ns) if err == nil && len(in.Answer) > 0 { break } } return in, err } func createDNSMsg(fqdn string, rtype uint16, recursive bool) *dns.Msg { m := new(dns.Msg) m.SetQuestion(fqdn, rtype) // See: https://caddy.community/t/hard-time-getting-a-response-on-a-dns-01-challenge/15721/16 m.SetEdns0(1232, false) if !recursive { m.RecursionDesired = false } return m } func sendDNSQuery(ctx context.Context, m *dns.Msg, ns string) (*dns.Msg, error) { udp := &dns.Client{Net: "udp", Timeout: dnsTimeout} in, _, err := udp.ExchangeContext(ctx, m, ns) // two kinds of errors we can handle by retrying with TCP: // truncation and timeout; see https://github.com/caddyserver/caddy/issues/3639 truncated := in != nil && in.Truncated timeoutErr := err != nil && strings.Contains(err.Error(), "timeout") if truncated || timeoutErr { tcp := &dns.Client{Net: "tcp", Timeout: dnsTimeout} in, _, err = tcp.ExchangeContext(ctx, m, ns) } return in, err } func formatDNSError(msg *dns.Msg, err error) string { var parts []string if msg != nil { parts = append(parts, dns.RcodeToString[msg.Rcode]) } if err != nil { parts = append(parts, err.Error()) } if len(parts) > 0 { return ": " + strings.Join(parts, " ") } return "" } // soaCacheEntry holds a cached SOA record (only selected fields) type soaCacheEntry struct { zone string // zone apex (a domain name) primaryNs string // primary nameserver for the zone apex expires time.Time // time when this cache entry should be evicted } func newSoaCacheEntry(soa *dns.SOA) *soaCacheEntry { return &soaCacheEntry{ zone: soa.Hdr.Name, primaryNs: soa.Ns, expires: time.Now().Add(time.Duration(soa.Refresh) * time.Second), } } // isExpired checks whether a cache entry should be considered expired. func (cache *soaCacheEntry) isExpired() bool { return time.Now().After(cache.expires) } // systemOrDefaultNameservers attempts to get system nameservers from the // resolv.conf file given by path before falling back to hard-coded defaults. func systemOrDefaultNameservers(path string, defaults []string) []string { config, err := dns.ClientConfigFromFile(path) if err != nil || len(config.Servers) == 0 { return defaults } return config.Servers } // populateNameserverPorts ensures that all nameservers have a port number // If not, the the default DNS server port of 53 will be appended. func populateNameserverPorts(servers []string) { for i := range servers { _, port, _ := net.SplitHostPort(servers[i]) if port == "" { servers[i] = net.JoinHostPort(servers[i], "53") } } } // checkDNSPropagation checks if the expected record has been propagated to all authoritative nameservers. func checkDNSPropagation(ctx context.Context, logger *zap.Logger, fqdn string, recType uint16, expectedValue string, checkAuthoritativeServers bool, resolvers []string) (bool, error) { logger = logger.Named("propagation") if !strings.HasSuffix(fqdn, ".") { fqdn += "." } // Initial attempt to resolve at the recursive NS - but do not actually // dereference (follow) a CNAME record if we are targeting a CNAME record // itself if recType != dns.TypeCNAME { r, err := dnsQuery(ctx, fqdn, recType, resolvers, true) if err != nil { return false, fmt.Errorf("CNAME dns query: %v", err) } if r.Rcode == dns.RcodeSuccess { fqdn = updateDomainWithCName(r, fqdn) } } if checkAuthoritativeServers { authoritativeServers, err := lookupNameservers(ctx, logger, fqdn, resolvers) if err != nil { return false, fmt.Errorf("looking up authoritative nameservers: %v", err) } populateNameserverPorts(authoritativeServers) resolvers = authoritativeServers } logger.Debug("checking authoritative nameservers", zap.Strings("resolvers", resolvers)) return checkAuthoritativeNss(ctx, fqdn, recType, expectedValue, resolvers) } // checkAuthoritativeNss queries each of the given nameservers for the expected record. func checkAuthoritativeNss(ctx context.Context, fqdn string, recType uint16, expectedValue string, nameservers []string) (bool, error) { for _, ns := range nameservers { r, err := dnsQuery(ctx, fqdn, recType, []string{ns}, true) if err != nil { return false, fmt.Errorf("querying authoritative nameservers: %v", err) } if r.Rcode != dns.RcodeSuccess { if r.Rcode == dns.RcodeNameError || r.Rcode == dns.RcodeServerFailure { // if Present() succeeded, then it must show up eventually, or else // something is really broken in the DNS provider or their API; // no need for error here, simply have the caller try again return false, nil } return false, fmt.Errorf("NS %s returned %s for %s", ns, dns.RcodeToString[r.Rcode], fqdn) } for _, rr := range r.Answer { switch recType { case dns.TypeTXT: if txt, ok := rr.(*dns.TXT); ok { record := strings.Join(txt.Txt, "") if record == expectedValue { return true, nil } } case dns.TypeCNAME: if cname, ok := rr.(*dns.CNAME); ok { // TODO: whether a DNS provider assumes a trailing dot or not varies, and we may have to standardize this in libdns packages if strings.TrimSuffix(cname.Target, ".") == strings.TrimSuffix(expectedValue, ".") { return true, nil } } default: return false, fmt.Errorf("unsupported record type: %d", recType) } } } return false, nil } // lookupNameservers returns the authoritative nameservers for the given fqdn. func lookupNameservers(ctx context.Context, logger *zap.Logger, fqdn string, resolvers []string) ([]string, error) { var authoritativeNss []string zone, err := FindZoneByFQDN(ctx, logger, fqdn, resolvers) if err != nil { return nil, fmt.Errorf("could not determine the zone for '%s': %w", fqdn, err) } r, err := dnsQuery(ctx, zone, dns.TypeNS, resolvers, true) if err != nil { return nil, fmt.Errorf("querying NS resolver for zone '%s' recursively: %v", zone, err) } for _, rr := range r.Answer { if ns, ok := rr.(*dns.NS); ok { authoritativeNss = append(authoritativeNss, strings.ToLower(ns.Ns)) } } if len(authoritativeNss) > 0 { return authoritativeNss, nil } return nil, errors.New("could not determine authoritative nameservers") } // Update FQDN with CNAME if any func updateDomainWithCName(r *dns.Msg, fqdn string) string { for _, rr := range r.Answer { if cn, ok := rr.(*dns.CNAME); ok { if cn.Hdr.Name == fqdn { return cn.Target } } } return fqdn } // RecursiveNameservers are used to pre-check DNS propagation. It // picks user-configured nameservers (custom) OR the defaults // obtained from resolv.conf and defaultNameservers if none is // configured and ensures that all server addresses have a port value. // // EXPERIMENTAL: This API was previously unexported, and may be // be unexported again in the future. Do not rely on it at this time. func RecursiveNameservers(custom []string) []string { var servers []string if len(custom) == 0 { servers = systemOrDefaultNameservers(defaultResolvConf, defaultNameservers) } else { servers = make([]string, len(custom)) copy(servers, custom) } populateNameserverPorts(servers) return servers } var defaultNameservers = []string{ "8.8.8.8:53", "8.8.4.4:53", "1.1.1.1:53", "1.0.0.1:53", } var dnsTimeout = 10 * time.Second var ( fqdnSOACache = map[string]*soaCacheEntry{} fqdnSOACacheMu sync.Mutex ) const defaultResolvConf = "/etc/resolv.conf" golang-github-caddyserver-certmagic-0.25.2/dnsutil_test.go000066400000000000000000000157571514710434200236640ustar00rootroot00000000000000package certmagic // Code in this file adapted from go-acme/lego, July 2020: // https://github.com/go-acme/lego // by Ludovic Fernandez and Dominik Menke // // It has been modified. import ( "context" "net" "reflect" "runtime" "sort" "strings" "testing" "go.uber.org/zap" ) func TestLookupNameserversOK(t *testing.T) { testCases := []struct { fqdn string nss []string }{ { fqdn: "physics.georgetown.edu.", nss: []string{"ns.b1ddi.physics.georgetown.edu.", "ns4.georgetown.edu.", "ns5.georgetown.edu.", "ns6.georgetown.edu."}, }, } for i, test := range testCases { test := test i := i t.Run(test.fqdn, func(t *testing.T) { t.Parallel() nss, err := lookupNameservers(context.Background(), zap.NewNop(), test.fqdn, RecursiveNameservers(nil)) if err != nil { t.Errorf("Expected no error, got: %v", err) } sort.Strings(nss) sort.Strings(test.nss) if !reflect.DeepEqual(test.nss, nss) { t.Errorf("Test %d: expected %+v but got %+v", i, test.nss, nss) } }) } } func TestLookupNameserversErr(t *testing.T) { testCases := []struct { desc string fqdn string error string }{ { desc: "invalid tld", fqdn: "_null.n0n0.", error: "could not determine the zone", }, } for i, test := range testCases { test := test i := i t.Run(test.desc, func(t *testing.T) { t.Parallel() _, err := lookupNameservers(context.Background(), zap.NewNop(), test.fqdn, nil) if err == nil { t.Errorf("expected error, got none") } if !strings.Contains(err.Error(), test.error) { t.Errorf("Test %d: Expected error to contain '%s' but got '%s'", i, test.error, err.Error()) } }) } } var findXByFqdnTestCases = []struct { desc string fqdn string zone string primaryNs string nameservers []string expectedError string skipTest bool }{ { desc: "domain is a CNAME", fqdn: "scholar.google.com.", zone: "google.com.", primaryNs: "ns1.google.com.", nameservers: RecursiveNameservers(nil), }, { desc: "domain is a non-existent subdomain", fqdn: "foo.google.com.", zone: "google.com.", primaryNs: "ns1.google.com.", nameservers: RecursiveNameservers(nil), }, { desc: "domain is a eTLD", fqdn: "example.com.ac.", zone: "ac.", primaryNs: "a0.nic.ac.", nameservers: RecursiveNameservers(nil), }, { desc: "domain is a cross-zone CNAME", fqdn: "cross-zone-example.assets.sh.", zone: "assets.sh.", primaryNs: "gina.ns.cloudflare.com.", nameservers: RecursiveNameservers(nil), }, { desc: "NXDOMAIN", fqdn: "test.loho.jkl.", zone: "loho.jkl.", nameservers: []string{"1.1.1.1:53"}, expectedError: "could not find the start of authority for test.loho.jkl.: NXDOMAIN", }, { desc: "several non existent nameservers", fqdn: "scholar.google.com.", zone: "google.com.", primaryNs: "ns1.google.com.", nameservers: []string{":7053", ":8053", "1.1.1.1:53"}, // Windows takes a super long time to timeout and this negatively impacts CI. // Essentially, we know this works, but Windows is just slow to give up. skipTest: runtime.GOOS == "windows", }, { desc: "only non existent nameservers", fqdn: "scholar.google.com.", zone: "google.com.", nameservers: []string{":7053", ":8053", ":9053"}, expectedError: "could not find the start of authority for scholar.google.com.:", // Windows takes a super long time to timeout and this negatively impacts CI. // Essentially, we know this works, but Windows is just slow to give up. skipTest: runtime.GOOS == "windows", }, { desc: "no nameservers", fqdn: "test.ldez.com.", zone: "ldez.com.", nameservers: []string{}, expectedError: "could not find the start of authority for test.ldez.com.", }, } func TestFindZoneByFqdn(t *testing.T) { for i, test := range findXByFqdnTestCases { t.Run(test.desc, func(t *testing.T) { if test.skipTest { t.Skip("skipping test") } clearFqdnCache() zone, err := FindZoneByFQDN(context.Background(), zap.NewNop(), test.fqdn, test.nameservers) if test.expectedError != "" { if err == nil { t.Errorf("test %d: expected error, got none", i) return } if !strings.Contains(err.Error(), test.expectedError) { t.Errorf("test %d: expected error to contain '%s' but got '%s'", i, test.expectedError, err.Error()) } } else { if err != nil { t.Errorf("test %d: expected no error, but got: %v", i, err) } if zone != test.zone { t.Errorf("test %d: expected zone '%s' but got '%s'", i, zone, test.zone) } } }) } } func TestResolveConfServers(t *testing.T) { var testCases = []struct { fixture string expected []string defaults []string }{ { fixture: "testdata/resolv.conf.1", defaults: []string{"127.0.0.1:53"}, expected: []string{"10.200.3.249", "10.200.3.250:5353", "2001:4860:4860::8844", "[10.0.0.1]:5353"}, }, { fixture: "testdata/resolv.conf.nonexistent", defaults: []string{"127.0.0.1:53"}, expected: []string{"127.0.0.1:53"}, }, } for i, test := range testCases { t.Run(test.fixture, func(t *testing.T) { result := systemOrDefaultNameservers(test.fixture, test.defaults) sort.Strings(result) sort.Strings(test.expected) if !reflect.DeepEqual(test.expected, result) { t.Errorf("Test %d: Expected %v but got %v", i, test.expected, result) } }) } } func TestRecursiveNameserversAddsPort(t *testing.T) { type want struct { port string } custom := []string{"127.0.0.1", "ns1.google.com:43"} expectations := []want{{port: "53"}, {port: "43"}} results := RecursiveNameservers(custom) if !reflect.DeepEqual(custom, []string{"127.0.0.1", "ns1.google.com:43"}) { t.Errorf("Expected custom nameservers to be unmodified. got %v", custom) } if len(results) != len(expectations) { t.Errorf("%v wrong results length. got %d, want %d", results, len(results), len(expectations)) } var hasCustom bool for i, res := range results { hasCustom = hasCustom || strings.HasPrefix(res, custom[0]) if _, port, err := net.SplitHostPort(res); err != nil { t.Errorf("%v Error splitting result %d into host and port: %v", results, i, err) } else { if port != expectations[i].port { t.Errorf("%v Expected result %d to have port %s but got %s", results, i, expectations[i].port, port) } } } if !hasCustom { t.Errorf("%v Expected custom resolvers to be included, but they weren't: %v", results, custom) } } func TestRecursiveNameserversDefaults(t *testing.T) { results := RecursiveNameservers(nil) if len(results) < 1 { t.Errorf("%v Expected at least 1 records as default when nil custom", results) } results = RecursiveNameservers([]string{}) if len(results) < 1 { t.Errorf("%v Expected at least 1 records as default when empty custom", results) } } func clearFqdnCache() { fqdnSOACacheMu.Lock() fqdnSOACache = make(map[string]*soaCacheEntry) fqdnSOACacheMu.Unlock() } golang-github-caddyserver-certmagic-0.25.2/doc_test.go000066400000000000000000000020701514710434200227270ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "fmt" "log" "net/http" ) // This is the simplest way for HTTP servers to use this package. // Call HTTPS() with your domain names and your handler (or nil // for the http.DefaultMux), and CertMagic will do the rest. func ExampleHTTPS() { http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) { fmt.Fprintf(w, "Hello, HTTPS visitor!") }) err := HTTPS([]string{"example.com", "www.example.com"}, nil) if err != nil { log.Fatal(err) } } golang-github-caddyserver-certmagic-0.25.2/filestorage.go000066400000000000000000000352101514710434200234310ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "encoding/json" "errors" "fmt" "io" "io/fs" "os" "path" "path/filepath" "runtime" "time" "github.com/caddyserver/certmagic/internal/atomicfile" ) // FileStorage facilitates forming file paths derived from a root // directory. It is used to get file paths in a consistent, // cross-platform way or persisting ACME assets on the file system. // The presence of a lock file for a given key indicates a lock // is held and is thus unavailable. // // Locks are created atomically by relying on the file system to // enforce the O_EXCL flag. Acquirers that are forcefully terminated // will not have a chance to clean up their locks before they exit, // so locks may become stale. That is why, while a lock is actively // held, the contents of the lockfile are updated with the current // timestamp periodically. If another instance tries to acquire the // lock but fails, it can see if the timestamp within is still fresh. // If so, it patiently waits by polling occasionally. Otherwise, // the stale lockfile is deleted, essentially forcing an unlock. // // While locking is atomic, unlocking is not perfectly atomic. File // systems offer native atomic operations when creating files, but // not necessarily when deleting them. It is theoretically possible // for two instances to discover the same stale lock and both proceed // to delete it, but if one instance is able to delete the lockfile // and create a new one before the other one calls delete, then the // new lock file created by the first instance will get deleted by // mistake. This does mean that mutual exclusion is not guaranteed // to be perfectly enforced in the presence of stale locks. One // alternative is to lock the unlock operation by using ".unlock" // files; and we did this for some time, but those files themselves // may become stale, leading applications into infinite loops if // they always expect the unlock file to be deleted by the instance // that created it. We instead prefer the simpler solution that // implies imperfect mutual exclusion if locks become stale, but // that is probably less severe a consequence than infinite loops. // // See https://github.com/caddyserver/caddy/issues/4448 for discussion. // See commit 468bfd25e452196b140148928cdd1f1a2285ae4b for where we // switched away from using .unlock files. type FileStorage struct { Path string } // Exists returns true if key exists in s. func (s *FileStorage) Exists(_ context.Context, key string) bool { _, err := os.Stat(s.Filename(key)) return !errors.Is(err, fs.ErrNotExist) } // Store saves value at key. func (s *FileStorage) Store(_ context.Context, key string, value []byte) error { filename := s.Filename(key) err := os.MkdirAll(filepath.Dir(filename), 0700) if err != nil { return err } fp, err := atomicfile.New(filename, 0o600) if err != nil { return err } _, err = fp.Write(value) if err != nil { // cancel the write fp.Cancel() return err } // close, thereby flushing the write return fp.Close() } // Load retrieves the value at key. func (s *FileStorage) Load(_ context.Context, key string) ([]byte, error) { // i believe it's possible for the read call to error but still return bytes, in event of something like a shortread? // therefore, i think it's appropriate to not return any bytes to avoid downstream users of the package erroniously believing that // bytes read + error is a valid response (it should not be) xs, err := os.ReadFile(s.Filename(key)) if err != nil { return nil, err } return xs, nil } // Delete deletes the value at key. func (s *FileStorage) Delete(_ context.Context, key string) error { return os.RemoveAll(s.Filename(key)) } // List returns all keys that match prefix. func (s *FileStorage) List(ctx context.Context, prefix string, recursive bool) ([]string, error) { var keys []string walkPrefix := s.Filename(prefix) err := filepath.Walk(walkPrefix, func(fpath string, info os.FileInfo, err error) error { if err != nil { return err } if info == nil { return fmt.Errorf("%s: file info is nil", fpath) } if fpath == walkPrefix { return nil } if ctxErr := ctx.Err(); ctxErr != nil { return ctxErr } suffix, err := filepath.Rel(walkPrefix, fpath) if err != nil { return fmt.Errorf("%s: could not make path relative: %v", fpath, err) } keys = append(keys, path.Join(prefix, suffix)) if !recursive && info.IsDir() { return filepath.SkipDir } return nil }) return keys, err } // Stat returns information about key. func (s *FileStorage) Stat(_ context.Context, key string) (KeyInfo, error) { fi, err := os.Stat(s.Filename(key)) if err != nil { return KeyInfo{}, err } return KeyInfo{ Key: key, Modified: fi.ModTime(), Size: fi.Size(), IsTerminal: !fi.IsDir(), }, nil } // Filename returns the key as a path on the file // system prefixed by s.Path. func (s *FileStorage) Filename(key string) string { return filepath.Join(s.Path, filepath.FromSlash(key)) } // obtainLock will attempt to obtain a lock for the given name up to the // number of attempts given. // if attempts is negative then it will try forever. func (s *FileStorage) obtainLock(ctx context.Context, name string, attempts int) (bool, error) { filename := s.lockFilename(name) // sometimes the lockfiles read as empty (size 0) - this is either a stale lock or it // is currently being written; we can retry a few times in this case, as it has been // shown to help (issue #232) var emptyCount int for { // if attempts is negative then we should allow the loop // to retry until the context is done, otherwise we decrement // the remaining attempts if there are any here to ensure we // don't miss it due to continue statements throughout the loop switch { case attempts == 0: return false, nil case attempts > 0: attempts-- } err := createLockfile(filename) if err == nil { // got the lock, yay return true, nil } if !os.IsExist(err) { // unexpected error return false, fmt.Errorf("creating lock file: %v", err) } // lock file already exists var meta lockMeta f, err := os.Open(filename) if err == nil { err2 := json.NewDecoder(f).Decode(&meta) f.Close() if errors.Is(err2, io.EOF) { emptyCount++ if emptyCount < 8 { // wait for brief time and retry; could be that the file is in the process // of being written or updated (which involves truncating) - see issue #232 select { case <-time.After(250 * time.Millisecond): case <-ctx.Done(): return false, ctx.Err() } continue } else { // lockfile is empty or truncated multiple times; I *think* we can assume // the previous acquirer either crashed or had some sort of failure that // caused them to be unable to fully acquire or retain the lock, therefore // we should treat it as if the lockfile did not exist defaultLogger.Sugar().Infof("[%s] %s: Empty lockfile (%v) - likely previous process crashed or storage medium failure; treating as stale", s, filename, err2) } } else if err2 != nil { return false, fmt.Errorf("decoding lockfile contents: %w", err2) } } switch { case os.IsNotExist(err): // must have just been removed; try again to create it continue case err != nil: // unexpected error return false, fmt.Errorf("accessing lock file: %v", err) case fileLockIsStale(meta): // lock file is stale - delete it and try again to obtain lock // (NOTE: locking becomes imperfect if lock files are stale; known solutions // either have potential to cause infinite loops, as in caddyserver/caddy#4448, // or must give up on perfect mutual exclusivity; however, these cases are rare, // so we prefer the simpler solution that avoids infinite loops) defaultLogger.Sugar().Infof("[%s] Lock for '%s' is stale (created: %s, last update: %s); removing then retrying: %s", s, name, meta.Created, meta.Updated, filename) if err = os.Remove(filename); err != nil { // hopefully we can replace the lock file quickly! if !errors.Is(err, fs.ErrNotExist) { return false, fmt.Errorf("unable to delete stale lockfile; deadlocked: %w", err) } } continue default: // lockfile exists and is not stale; // just wait a moment and try again, // or return if context cancelled select { case <-time.After(fileLockPollInterval): case <-ctx.Done(): return false, ctx.Err() } } } } // Lock obtains a lock named by the given name. It blocks // until the lock can be obtained or an error is returned. func (s *FileStorage) Lock(ctx context.Context, name string) error { ok, err := s.obtainLock(ctx, name, -1) if !ok && err == nil { return errors.New("unable to obtain lock") } return err } // TryLock attempts to obtain a lock named by the given name. // If the lock was obtained it will return true, otherwise it will // return false along with any errors that may have occurred. func (s *FileStorage) TryLock(ctx context.Context, name string) (bool, error) { return s.obtainLock(ctx, name, 2) } // Unlock releases the lock for name. func (s *FileStorage) Unlock(_ context.Context, name string) error { return os.Remove(s.lockFilename(name)) } func (s *FileStorage) String() string { return "FileStorage:" + s.Path } func (s *FileStorage) lockFilename(name string) string { return filepath.Join(s.lockDir(), StorageKeys.Safe(name)+".lock") } func (s *FileStorage) lockDir() string { return filepath.Join(s.Path, "locks") } func fileLockIsStale(meta lockMeta) bool { ref := meta.Updated if ref.IsZero() { ref = meta.Created } // since updates are exactly every lockFreshnessInterval, // add a grace period for the actual file read+write to // take place return time.Since(ref) > lockFreshnessInterval*2 } // createLockfile atomically creates the lockfile // identified by filename. A successfully created // lockfile should be removed with removeLockfile. func createLockfile(filename string) error { err := atomicallyCreateFile(filename, true) if err != nil { return err } go keepLockfileFresh(filename) return nil } // keepLockfileFresh continuously updates the lock file // at filename with the current timestamp. It stops // when the file disappears (happy path = lock released), // or when there is an error at any point. Since it polls // every lockFreshnessInterval, this function might // not terminate until up to lockFreshnessInterval after // the lock is released. func keepLockfileFresh(filename string) { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] defaultLogger.Sugar().Errorf("active locking: %v\n%s", err, buf) } }() for { time.Sleep(lockFreshnessInterval) done, err := updateLockfileFreshness(filename) if err != nil { defaultLogger.Sugar().Errorf("Keeping lock file fresh: %v - terminating lock maintenance (lockfile: %s)", err, filename) return } if done { return } } } // updateLockfileFreshness updates the lock file at filename // with the current timestamp. It returns true if the parent // loop can terminate (i.e. no more need to update the lock). func updateLockfileFreshness(filename string) (bool, error) { f, err := os.OpenFile(filename, os.O_RDWR, 0644) if os.IsNotExist(err) { return true, nil // lock released } if err != nil { return true, err } defer f.Close() // read contents metaBytes, err := io.ReadAll(io.LimitReader(f, 2048)) if err != nil { return true, err } var meta lockMeta if err := json.Unmarshal(metaBytes, &meta); err != nil { // see issue #232: this can error if the file is empty, // which happens sometimes when the disk is REALLY slow return true, err } // truncate file and reset I/O offset to beginning if err := f.Truncate(0); err != nil { return true, err } if _, err := f.Seek(0, io.SeekStart); err != nil { return true, err } // write updated timestamp meta.Updated = time.Now() if err = json.NewEncoder(f).Encode(meta); err != nil { return false, err } // sync to device; we suspect that sometimes file systems // (particularly AWS EFS) don't do this on their own, // leaving the file empty when we close it; see // https://github.com/caddyserver/caddy/issues/3954 return false, f.Sync() } // atomicallyCreateFile atomically creates the file // identified by filename if it doesn't already exist. func atomicallyCreateFile(filename string, writeLockInfo bool) error { // no need to check this error, we only really care about the file creation error _ = os.MkdirAll(filepath.Dir(filename), 0700) f, err := os.OpenFile(filename, os.O_CREATE|os.O_WRONLY|os.O_EXCL, 0644) if err != nil { return err } defer f.Close() if writeLockInfo { now := time.Now() meta := lockMeta{ Created: now, Updated: now, } if err := json.NewEncoder(f).Encode(meta); err != nil { return err } // see https://github.com/caddyserver/caddy/issues/3954 if err := f.Sync(); err != nil { return err } } return nil } // homeDir returns the best guess of the current user's home // directory from environment variables. If unknown, "." (the // current directory) is returned instead. func homeDir() string { home := os.Getenv("HOME") if home == "" && runtime.GOOS == "windows" { drive := os.Getenv("HOMEDRIVE") path := os.Getenv("HOMEPATH") home = drive + path if drive == "" || path == "" { home = os.Getenv("USERPROFILE") } } if home == "" { home = "." } return home } func dataDir() string { baseDir := filepath.Join(homeDir(), ".local", "share") if xdgData := os.Getenv("XDG_DATA_HOME"); xdgData != "" { baseDir = xdgData } return filepath.Join(baseDir, "certmagic") } // lockMeta is written into a lock file. type lockMeta struct { Created time.Time `json:"created,omitempty"` Updated time.Time `json:"updated,omitempty"` } // lockFreshnessInterval is how often to update // a lock's timestamp. Locks with a timestamp // more than this duration in the past (plus a // grace period for latency) can be considered // stale. const lockFreshnessInterval = 5 * time.Second // fileLockPollInterval is how frequently // to check the existence of a lock file const fileLockPollInterval = 1 * time.Second // Interface guard var _ Storage = (*FileStorage)(nil) golang-github-caddyserver-certmagic-0.25.2/filestorage_test.go000066400000000000000000000037761514710434200245040ustar00rootroot00000000000000package certmagic_test import ( "bytes" "context" "os" "testing" "github.com/caddyserver/certmagic" "github.com/caddyserver/certmagic/internal/testutil" ) func TestFileStorageStoreLoad(t *testing.T) { ctx := context.Background() tmpDir, err := os.MkdirTemp(os.TempDir(), "certmagic*") testutil.RequireNoError(t, err, "allocating tmp dir") defer os.RemoveAll(tmpDir) s := &certmagic.FileStorage{ Path: tmpDir, } err = s.Store(ctx, "foo", []byte("bar")) testutil.RequireNoError(t, err) dat, err := s.Load(ctx, "foo") testutil.RequireNoError(t, err) testutil.RequireEqualValues(t, dat, []byte("bar")) } func TestFileStorageStoreLoadRace(t *testing.T) { ctx := context.Background() tmpDir, err := os.MkdirTemp(os.TempDir(), "certmagic*") testutil.RequireNoError(t, err, "allocating tmp dir") defer os.RemoveAll(tmpDir) s := &certmagic.FileStorage{ Path: tmpDir, } a := bytes.Repeat([]byte("a"), 4096*1024) b := bytes.Repeat([]byte("b"), 4096*1024) err = s.Store(ctx, "foo", a) testutil.RequireNoError(t, err) done := make(chan struct{}) go func() { err := s.Store(ctx, "foo", b) testutil.RequireNoError(t, err) close(done) }() dat, err := s.Load(ctx, "foo") <-done testutil.RequireNoError(t, err) testutil.RequireEqualValues(t, 4096*1024, len(dat)) } func TestFileStorageWriteLock(t *testing.T) { ctx := context.Background() tmpDir, err := os.MkdirTemp(os.TempDir(), "certmagic*") testutil.RequireNoError(t, err, "allocating tmp dir") defer os.RemoveAll(tmpDir) s := &certmagic.FileStorage{ Path: tmpDir, } // cctx is a cancelled ctx. so if we can't immediately get the lock, it will fail cctx, cn := context.WithCancel(ctx) cn() // should success err = s.Lock(cctx, "foo") testutil.RequireNoError(t, err) // should fail err = s.Lock(cctx, "foo") testutil.RequireError(t, err) err = s.Unlock(cctx, "foo") testutil.RequireNoError(t, err) // shouldn't fail err = s.Lock(cctx, "foo") testutil.RequireNoError(t, err) err = s.Unlock(cctx, "foo") testutil.RequireNoError(t, err) } golang-github-caddyserver-certmagic-0.25.2/go.mod000066400000000000000000000011631514710434200217040ustar00rootroot00000000000000module github.com/caddyserver/certmagic go 1.24.0 require ( github.com/caddyserver/zerossl v0.1.5 github.com/klauspost/cpuid/v2 v2.3.0 github.com/libdns/libdns v1.1.1 github.com/mholt/acmez/v3 v3.1.6 github.com/miekg/dns v1.1.72 github.com/zeebo/blake3 v0.2.4 go.uber.org/zap v1.27.1 go.uber.org/zap/exp v0.3.0 golang.org/x/crypto v0.48.0 golang.org/x/net v0.50.0 ) require ( go.uber.org/multierr v1.11.0 // indirect golang.org/x/mod v0.33.0 // indirect golang.org/x/sync v0.19.0 // indirect golang.org/x/sys v0.41.0 // indirect golang.org/x/text v0.34.0 // indirect golang.org/x/tools v0.42.0 // indirect ) golang-github-caddyserver-certmagic-0.25.2/go.sum000066400000000000000000000107641514710434200217400ustar00rootroot00000000000000code.pfad.fr/check v1.1.0 h1:GWvjdzhSEgHvEHe2uJujDcpmZoySKuHQNrZMfzfO0bE= code.pfad.fr/check v1.1.0/go.mod h1:NiUH13DtYsb7xp5wll0U4SXx7KhXQVCtRgdC96IPfoM= github.com/caddyserver/zerossl v0.1.5 h1:dkvOjBAEEtY6LIGAHei7sw2UgqSD6TrWweXpV7lvEvE= github.com/caddyserver/zerossl v0.1.5/go.mod h1:CxA0acn7oEGO6//4rtrRjYgEoa4MFw/XofZnrYwGqG4= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/go-jose/go-jose/v4 v4.1.3 h1:CVLmWDhDVRa6Mi/IgCgaopNosCaHz7zrMeF9MlZRkrs= github.com/go-jose/go-jose/v4 v4.1.3/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y= github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= github.com/letsencrypt/challtestsrv v1.4.2 h1:0ON3ldMhZyWlfVNYYpFuWRTmZNnyfiL9Hh5YzC3JVwU= github.com/letsencrypt/challtestsrv v1.4.2/go.mod h1:GhqMqcSoeGpYd5zX5TgwA6er/1MbWzx/o7yuuVya+Wk= github.com/letsencrypt/pebble/v2 v2.10.0 h1:Wq6gYXlsY6ubqI3hhxsTzdyotvfdjFBxuwYqCLCnj/U= github.com/letsencrypt/pebble/v2 v2.10.0/go.mod h1:Sk8cmUIPcIdv2nINo+9PB4L+ZBhzY+F9A1a/h/xmWiQ= github.com/libdns/libdns v1.1.1 h1:wPrHrXILoSHKWJKGd0EiAVmiJbFShguILTg9leS/P/U= github.com/libdns/libdns v1.1.1/go.mod h1:4Bj9+5CQiNMVGf87wjX4CY3HQJypUHRuLvlsfsZqLWQ= github.com/mholt/acmez/v3 v3.1.6 h1:eGVQNObP0pBN4sxqrXeg7MYqTOWyoiYpQqITVWlrevk= github.com/mholt/acmez/v3 v3.1.6/go.mod h1:5nTPosTGosLxF3+LU4ygbgMRFDhbAVpqMI4+a4aHLBY= github.com/miekg/dns v1.1.72 h1:vhmr+TF2A3tuoGNkLDFK9zi36F2LS+hKTRW0Uf8kbzI= github.com/miekg/dns v1.1.72/go.mod h1:+EuEPhdHOsfk6Wk5TT2CzssZdqkmFhf8r+aVyDEToIs= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/zeebo/assert v1.1.0 h1:hU1L1vLTHsnO8x8c9KAR5GmM5QscxHg5RNU5z5qbUWY= github.com/zeebo/assert v1.1.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0= github.com/zeebo/blake3 v0.2.4 h1:KYQPkhpRtcqh0ssGYcKLG1JYvddkEA8QwCM/yBqhaZI= github.com/zeebo/blake3 v0.2.4/go.mod h1:7eeQ6d2iXWRGF6npfaxl2CU+xy2Fjo2gxeyZGCRUjcE= github.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo= github.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc= go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= go.uber.org/zap/exp v0.3.0 h1:6JYzdifzYkGmTdRR59oYH+Ng7k49H9qVpWwNSsGJj3U= go.uber.org/zap/exp v0.3.0/go.mod h1:5I384qq7XGxYyByIhHm6jg5CHkGY0nsTfbDLgDDlgJQ= golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts= golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos= golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8= golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w= golang.org/x/net v0.50.0 h1:ucWh9eiCGyDR3vtzso0WMQinm2Dnt8cFMuQa9K33J60= golang.org/x/net v0.50.0/go.mod h1:UgoSli3F/pBgdJBHCTc+tp3gmrU4XswgGRgtnwWTfyM= golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k= golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk= golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA= golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k= golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= golang-github-caddyserver-certmagic-0.25.2/handshake.go000066400000000000000000001155451514710434200230650ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/tls" "errors" "fmt" "io/fs" "net" "strings" "sync" "time" "github.com/mholt/acmez/v3" "go.uber.org/zap" "golang.org/x/crypto/ocsp" "golang.org/x/net/idna" ) // GetCertificate gets a certificate to satisfy clientHello. In getting // the certificate, it abides the rules and settings defined in the Config // that matches clientHello.ServerName. It tries to get certificates in // this order: // // 1. Exact match in the in-memory cache // 2. Wildcard match in the in-memory cache // 3. Managers (if any) // 4. Storage (if on-demand is enabled) // 5. Issuers (if on-demand is enabled) // // This method is safe for use as a tls.Config.GetCertificate callback. // // GetCertificate will run in a new context, use GetCertificateWithContext to provide // a context. func (cfg *Config) GetCertificate(clientHello *tls.ClientHelloInfo) (*tls.Certificate, error) { return cfg.GetCertificateWithContext(clientHello.Context(), clientHello) } func (cfg *Config) GetCertificateWithContext(ctx context.Context, clientHello *tls.ClientHelloInfo) (*tls.Certificate, error) { if err := cfg.emit(ctx, "tls_get_certificate", map[string]any{"client_hello": clientHelloWithoutConn(clientHello)}); err != nil { cfg.Logger.Error("TLS handshake aborted by event handler", zap.String("server_name", clientHello.ServerName), zap.String("remote", clientHello.Conn.RemoteAddr().String()), zap.Error(err)) return nil, fmt.Errorf("handshake aborted by event handler: %w", err) } if ctx == nil { // tests can't set context on a tls.ClientHelloInfo because it's unexported :( ctx = context.Background() } ctx = context.WithValue(ctx, ClientHelloInfoCtxKey, clientHello) // special case: serve up the certificate for a TLS-ALPN ACME challenge // (https://www.rfc-editor.org/rfc/rfc8737.html) // "The ACME server MUST provide an ALPN extension with the single protocol // name "acme-tls/1" and an SNI extension containing only the domain name // being validated during the TLS handshake." if clientHello.ServerName != "" && len(clientHello.SupportedProtos) == 1 && clientHello.SupportedProtos[0] == acmez.ACMETLS1Protocol { challengeCert, distributed, err := cfg.getTLSALPNChallengeCert(clientHello) if err != nil { cfg.Logger.Error("tls-alpn challenge", zap.String("remote_addr", clientHello.Conn.RemoteAddr().String()), zap.String("server_name", clientHello.ServerName), zap.Error(err)) return nil, err } cfg.Logger.Info("served key authentication certificate", zap.String("server_name", clientHello.ServerName), zap.String("challenge", "tls-alpn-01"), zap.String("remote", clientHello.Conn.RemoteAddr().String()), zap.Bool("distributed", distributed)) return challengeCert, nil } // get the certificate and serve it up cert, err := cfg.getCertDuringHandshake(ctx, clientHello, true) return &cert.Certificate, err } // getCertificateFromCache gets a certificate that matches name from the in-memory // cache, according to the lookup table associated with cfg. The lookup then // points to a certificate in the Instance certificate cache. // // The name is expected to already be normalized (e.g. lowercased). // // If there is no exact match for name, it will be checked against names of // the form '*.example.com' (wildcard certificates) according to RFC 6125. // If a match is found, matched will be true. If no matches are found, matched // will be false and a "default" certificate will be returned with defaulted // set to true. If defaulted is false, then no certificates were available. // // The logic in this function is adapted from the Go standard library, // which is by the Go Authors. // // This function is safe for concurrent use. func (cfg *Config) getCertificateFromCache(hello *tls.ClientHelloInfo) (cert Certificate, matched, defaulted bool) { name := normalizedName(hello.ServerName) if name == "" { // if SNI is empty, prefer matching IP address if hello.Conn != nil { addr := localIPFromConn(hello.Conn) cert, matched = cfg.selectCert(hello, addr) if matched { return } } // use a "default" certificate by name, if specified if cfg.DefaultServerName != "" { normDefault := normalizedName(cfg.DefaultServerName) cert, defaulted = cfg.selectCert(hello, normDefault) if defaulted { return } } } else { // if SNI is specified, try an exact match first cert, matched = cfg.selectCert(hello, name) if matched { return } // try replacing labels in the name with // wildcards until we get a match labels := strings.Split(name, ".") for i := range labels { labels[i] = "*" candidate := strings.Join(labels, ".") cert, matched = cfg.selectCert(hello, candidate) if matched { return } } } // a fallback server name can be tried in the very niche // case where a client sends one SNI value but expects or // accepts a different one in return (this is sometimes // the case with CDNs like Cloudflare that send the // downstream ServerName in the handshake but accept // the backend origin's true hostname in a cert). if cfg.FallbackServerName != "" { normFallback := normalizedName(cfg.FallbackServerName) cert, defaulted = cfg.selectCert(hello, normFallback) if defaulted { return } } // otherwise, we're bingo on ammo; see issues // caddyserver/caddy#2035 and caddyserver/caddy#1303 (any // change to certificate matching behavior must // account for hosts defined where the hostname // is empty or a catch-all, like ":443" or // "0.0.0.0:443") return } // selectCert uses hello to select a certificate from the // cache for name. If cfg.CertSelection is set, it will be // used to make the decision. Otherwise, the first matching // unexpired cert is returned. As a special case, if no // certificates match name and cfg.CertSelection is set, // then all certificates in the cache will be passed in // for the cfg.CertSelection to make the final decision. func (cfg *Config) selectCert(hello *tls.ClientHelloInfo, name string) (Certificate, bool) { logger := cfg.Logger.Named("handshake") choices := cfg.certCache.getAllMatchingCerts(name) if len(choices) == 0 { if cfg.CertSelection == nil { logger.Debug("no matching certificates and no custom selection logic", zap.String("identifier", name)) return Certificate{}, false } logger.Debug("no matching certificate; will choose from all certificates", zap.String("identifier", name)) choices = cfg.certCache.getAllCerts() } logger.Debug("choosing certificate", zap.String("identifier", name), zap.Int("num_choices", len(choices))) if cfg.CertSelection == nil { cert, err := DefaultCertificateSelector(hello, choices) logger.Debug("default certificate selection results", zap.Error(err), zap.String("identifier", name), zap.Strings("subjects", cert.Names), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash)) return cert, err == nil } cert, err := cfg.CertSelection.SelectCertificate(hello, choices) logger.Debug("custom certificate selection results", zap.Error(err), zap.String("identifier", name), zap.Strings("subjects", cert.Names), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash)) return cert, err == nil } // DefaultCertificateSelector is the default certificate selection logic // given a choice of certificates. If there is at least one certificate in // choices, it always returns a certificate without error. It chooses the // first non-expired certificate that the client supports if possible, // otherwise it returns an expired certificate that the client supports, // otherwise it just returns the first certificate in the list of choices. func DefaultCertificateSelector(hello *tls.ClientHelloInfo, choices []Certificate) (Certificate, error) { if len(choices) == 1 { // Fast path: There's only one choice, so we would always return that one // regardless of whether it is expired or not compatible. return choices[0], nil } if len(choices) == 0 { return Certificate{}, fmt.Errorf("no certificates available") } // Slow path: There are choices, so we need to check each of them. now := time.Now() best := choices[0] for _, choice := range choices { if err := hello.SupportsCertificate(&choice.Certificate); err != nil { continue } best = choice // at least the client supports it... if now.After(choice.Leaf.NotBefore) && now.Before(expiresAt(choice.Leaf)) { return choice, nil // ...and unexpired, great! "Certificate, I choose you!" } } return best, nil // all matching certs are expired or incompatible, oh well } // getCertDuringHandshake will get a certificate for hello. It first tries // the in-memory cache. If no exact certificate for hello is in the cache, the // config most closely corresponding to hello (like a wildcard) will be loaded. // If none could be matched from the cache, it invokes the configured certificate // managers to get a certificate and uses the first one that returns a certificate. // If no certificate managers return a value, and if the config allows it // (OnDemand!=nil) and if loadIfNecessary == true, it goes to storage to load the // cert into the cache and serve it. If it's not on disk and if // obtainIfNecessary == true, the certificate will be obtained from the CA, cached, // and served. If obtainIfNecessary == true, then loadIfNecessary must also be == true. // An error will be returned if and only if no certificate is available. // // This function is safe for concurrent use. func (cfg *Config) getCertDuringHandshake(ctx context.Context, hello *tls.ClientHelloInfo, loadOrObtainIfNecessary bool) (Certificate, error) { logger := logWithRemote(cfg.Logger.Named("handshake"), hello) // First check our in-memory cache to see if we've already loaded it cert, matched, defaulted := cfg.getCertificateFromCache(hello) if matched { logger.Debug("matched certificate in cache", zap.Strings("subjects", cert.Names), zap.Bool("managed", cert.managed), zap.Time("expiration", expiresAt(cert.Leaf)), zap.String("hash", cert.hash)) if cert.managed && cfg.OnDemand != nil && loadOrObtainIfNecessary { // On-demand certificates are maintained in the background, but // maintenance is triggered by handshakes instead of by a timer // as in maintain.go. return cfg.optionalMaintenance(ctx, cfg.Logger.Named("on_demand"), cert, hello) } return cert, nil } name, err := cfg.getNameFromClientHello(hello) if err != nil { return Certificate{}, err } // By this point, we need to load or obtain a certificate. If a swarm of requests comes in for the same // domain, avoid pounding manager or storage thousands of times simultaneously. We use a similar sync // strategy for obtaining certificate during handshake. certLoadWaitChansMu.Lock() waiter, ok := certLoadWaitChans[name] if ok { // another goroutine is already loading the cert; just wait certLoadWaitChansMu.Unlock() timeout := time.NewTimer(2 * time.Minute) select { case <-timeout.C: return Certificate{}, fmt.Errorf("timed out waiting to load certificate for %s", name) case <-ctx.Done(): timeout.Stop() return Certificate{}, ctx.Err() case <-waiter.done: timeout.Stop() } // If the leader got a result from an external cert manager, use it // directly — these certs are not added to the cache, so a recursive // cache lookup would miss. For cached certs (on-demand, managed), // the waiter result will be empty and we fall through to the // original recursive lookup. if !waiter.cert.Empty() || waiter.err != nil { return waiter.cert, waiter.err } return cfg.getCertDuringHandshake(ctx, hello, false) } // no other goroutine is currently trying to load this cert waiter = &certLoadWaiter{done: make(chan struct{})} certLoadWaitChans[name] = waiter certLoadWaitChansMu.Unlock() // unblock others and clean up when we're done defer func() { certLoadWaitChansMu.Lock() close(waiter.done) delete(certLoadWaitChans, name) certLoadWaitChansMu.Unlock() }() // If an external Manager is configured, try to get it from them. // Only continue to use our own logic if it returns empty+nil. externalCert, err := cfg.getCertFromAnyCertManager(ctx, hello, logger) if err != nil { waiter.err = err return Certificate{}, err } if !externalCert.Empty() { waiter.cert = externalCert return externalCert, nil } // Make sure a certificate is allowed for the given name. If not, it doesn't make sense // to try loading one from storage (issue #185) or obtaining one from an issuer. if err := cfg.checkIfCertShouldBeObtained(ctx, name, false); err != nil { return Certificate{}, fmt.Errorf("certificate is not allowed for server name %s: %w", name, err) } // We might be able to load or obtain a needed certificate. Load from // storage if OnDemand is enabled, or if there is the possibility that // a statically-managed cert was evicted from a full cache. cfg.certCache.mu.RLock() cacheSize := len(cfg.certCache.cache) cfg.certCache.mu.RUnlock() // A cert might have still been evicted from the cache even if the cache // is no longer completely full; this happens if the newly-loaded cert is // itself evicted (perhaps due to being expired or unmanaged at this point). // Hence, we use an "almost full" metric to allow for the cache to not be // perfectly full while still being able to load needed certs from storage. // See https://caddy.community/t/error-tls-alert-internal-error-592-again/13272 // and caddyserver/caddy#4320. cfg.certCache.optionsMu.RLock() cacheCapacity := float64(cfg.certCache.options.Capacity) cfg.certCache.optionsMu.RUnlock() cacheAlmostFull := cacheCapacity > 0 && float64(cacheSize) >= cacheCapacity*.9 loadDynamically := cfg.OnDemand != nil || cacheAlmostFull if loadDynamically && loadOrObtainIfNecessary { // Check to see if we have one on disk loadedCert, err := cfg.loadCertFromStorage(ctx, logger, hello) if err == nil { return loadedCert, nil } logger.Debug("did not load cert from storage", zap.String("server_name", hello.ServerName), zap.Error(err)) if cfg.OnDemand != nil { // By this point, we need to ask the CA for a certificate return cfg.obtainOnDemandCertificate(ctx, hello) } return loadedCert, nil } // Fall back to another certificate if there is one (either DefaultServerName or FallbackServerName) if defaulted { logger.Debug("fell back to default certificate", zap.Strings("subjects", cert.Names), zap.Bool("managed", cert.managed), zap.Time("expiration", expiresAt(cert.Leaf)), zap.String("hash", cert.hash)) return cert, nil } logger.Debug("no certificate matching TLS ClientHello", zap.String("server_name", hello.ServerName), zap.String("remote", hello.Conn.RemoteAddr().String()), zap.String("identifier", name), zap.Uint16s("cipher_suites", hello.CipherSuites), zap.Float64("cert_cache_fill", float64(cacheSize)/cacheCapacity), // may be approximate! because we are not within the lock zap.Bool("load_or_obtain_if_necessary", loadOrObtainIfNecessary), zap.Bool("on_demand", cfg.OnDemand != nil)) return Certificate{}, fmt.Errorf("no certificate available for '%s'", name) } // loadCertFromStorage loads the certificate for name from storage and maintains it // (as this is only called with on-demand TLS enabled). func (cfg *Config) loadCertFromStorage(ctx context.Context, logger *zap.Logger, hello *tls.ClientHelloInfo) (Certificate, error) { name, err := cfg.getNameFromClientHello(hello) if err != nil { return Certificate{}, err } loadedCert, err := cfg.CacheManagedCertificate(ctx, name) if errors.Is(err, fs.ErrNotExist) { // If no exact match, try a wildcard variant, which is something we can still use labels := strings.Split(name, ".") labels[0] = "*" loadedCert, err = cfg.CacheManagedCertificate(ctx, strings.Join(labels, ".")) } if err != nil { return Certificate{}, fmt.Errorf("no matching certificate to load for %s: %w", name, err) } logger.Debug("loaded certificate from storage", zap.Strings("subjects", loadedCert.Names), zap.Bool("managed", loadedCert.managed), zap.Time("expiration", expiresAt(loadedCert.Leaf)), zap.String("hash", loadedCert.hash)) loadedCert, err = cfg.handshakeMaintenance(ctx, hello, loadedCert) if err != nil { logger.Error("maintaining newly-loaded certificate", zap.String("server_name", name), zap.Error(err)) } return loadedCert, nil } // optionalMaintenance will perform maintenance on the certificate (if necessary) and // will return the resulting certificate. This should only be done if the certificate // is managed, OnDemand is enabled, and the scope is allowed to obtain certificates. func (cfg *Config) optionalMaintenance(ctx context.Context, log *zap.Logger, cert Certificate, hello *tls.ClientHelloInfo) (Certificate, error) { newCert, err := cfg.handshakeMaintenance(ctx, hello, cert) if err == nil { return newCert, nil } log.Error("renewing certificate on-demand failed", zap.Strings("subjects", cert.Names), zap.Time("not_after", expiresAt(cert.Leaf)), zap.Error(err)) if cert.Expired() { return cert, err } // still has time remaining, so serve it anyway return cert, nil } // checkIfCertShouldBeObtained checks to see if an on-demand TLS certificate // should be obtained for a given domain based upon the config settings. If // a non-nil error is returned, do not issue a new certificate for name. func (cfg *Config) checkIfCertShouldBeObtained(ctx context.Context, name string, requireOnDemand bool) error { if requireOnDemand && cfg.OnDemand == nil { return fmt.Errorf("not configured for on-demand certificate issuance") } if !SubjectQualifiesForCert(name) { return fmt.Errorf("subject name does not qualify for certificate: %s", name) } if cfg.OnDemand != nil { if cfg.OnDemand.DecisionFunc != nil { if err := cfg.OnDemand.DecisionFunc(ctx, name); err != nil { return fmt.Errorf("decision func: %w", err) } return nil } if len(cfg.OnDemand.hostAllowlist) > 0 { if _, ok := cfg.OnDemand.hostAllowlist[name]; !ok { return fmt.Errorf("certificate for '%s' is not managed", name) } } } return nil } // obtainOnDemandCertificate obtains a certificate for hello. // If another goroutine has already started obtaining a cert for // hello, it will wait and use what the other goroutine obtained. // // This function is safe for use by multiple concurrent goroutines. func (cfg *Config) obtainOnDemandCertificate(ctx context.Context, hello *tls.ClientHelloInfo) (Certificate, error) { log := logWithRemote(cfg.Logger.Named("on_demand"), hello) name, err := cfg.getNameFromClientHello(hello) if err != nil { return Certificate{}, err } // We must protect this process from happening concurrently, so synchronize. obtainCertWaitChansMu.Lock() wait, ok := obtainCertWaitChans[name] if ok { // lucky us -- another goroutine is already obtaining the certificate. // wait for it to finish obtaining the cert and then we'll use it. obtainCertWaitChansMu.Unlock() log.Debug("new certificate is needed, but is already being obtained; waiting for that issuance to complete", zap.String("subject", name)) // TODO: see if we can get a proper context in here, for true cancellation timeout := time.NewTimer(2 * time.Minute) select { case <-timeout.C: return Certificate{}, fmt.Errorf("timed out waiting to obtain certificate for %s", name) case <-wait: timeout.Stop() } // it should now be loaded in the cache, ready to go; if not, // the goroutine in charge of that probably had an error return cfg.getCertDuringHandshake(ctx, hello, false) } // looks like it's up to us to do all the work and obtain the cert. // make a chan others can wait on if needed wait = make(chan struct{}) obtainCertWaitChans[name] = wait obtainCertWaitChansMu.Unlock() unblockWaiters := func() { obtainCertWaitChansMu.Lock() close(wait) delete(obtainCertWaitChans, name) obtainCertWaitChansMu.Unlock() } log.Info("obtaining new certificate", zap.String("server_name", name)) // set a timeout so we don't inadvertently hold a client handshake open too long // (timeout duration is based on https://caddy.community/t/zerossl-dns-challenge-failing-often-route53-plugin/13822/24?u=matt) var cancel context.CancelFunc ctx, cancel = context.WithTimeout(ctx, 180*time.Second) defer cancel() // obtain the certificate (this puts it in storage) and if successful, // load it from storage so we and any other waiting goroutine can use it var cert Certificate err = cfg.ObtainCertAsync(ctx, name) if err == nil { // load from storage while others wait to make the op as atomic as possible cert, err = cfg.loadCertFromStorage(ctx, log, hello) if err != nil { log.Error("loading newly-obtained certificate from storage", zap.String("server_name", name), zap.Error(err)) } } // immediately unblock anyone waiting for it unblockWaiters() return cert, err } // handshakeMaintenance performs a check on cert for expiration and OCSP validity. // If necessary, it will renew the certificate and/or refresh the OCSP staple. // OCSP stapling errors are not returned, only logged. // // This function is safe for use by multiple concurrent goroutines. func (cfg *Config) handshakeMaintenance(ctx context.Context, hello *tls.ClientHelloInfo, cert Certificate) (Certificate, error) { logger := cfg.Logger.Named("on_demand").With( zap.Strings("identifiers", cert.Names), zap.String("server_name", hello.ServerName)) renewIfNecessary := func(ctx context.Context, hello *tls.ClientHelloInfo, cert Certificate) (Certificate, error) { if cert.Leaf == nil { return cert, fmt.Errorf("leaf certificate is unexpectedly nil: either the Certificate got replaced by an empty value, or it was not properly initialized") } if cfg.certNeedsRenewal(cert.Leaf, cert.ari, true) { // Check if the certificate still exists on disk. If not, we need to obtain a new one. // This can happen if the certificate was cleaned up by the storage cleaner, but still // remains in the in-memory cache. if !cfg.storageHasCertResourcesAnyIssuer(ctx, cert.Names[0]) { logger.Debug("certificate not found on disk; obtaining new certificate") return cfg.obtainOnDemandCertificate(ctx, hello) } // Otherwise, renew the certificate. return cfg.renewDynamicCertificate(ctx, hello, cert) } return cert, nil } // Check OCSP staple validity if cert.ocsp != nil && !freshOCSP(cert.ocsp) { logger.Debug("OCSP response needs refreshing", zap.Int("ocsp_status", cert.ocsp.Status), zap.Time("this_update", cert.ocsp.ThisUpdate), zap.Time("next_update", cert.ocsp.NextUpdate)) err := stapleOCSP(ctx, cfg.OCSP, cfg.Storage, &cert, nil) if err != nil { // An error with OCSP stapling is not the end of the world, and in fact, is // quite common considering not all certs have issuer URLs that support it. if errors.Is(err, ErrNoOCSPServerSpecified) { logger.Debug("stapling OCSP", zap.Error(err)) } else { logger.Warn("stapling OCSP", zap.Error(err)) } } else { logger.Debug("successfully stapled new OCSP response", zap.Int("ocsp_status", cert.ocsp.Status), zap.Time("this_update", cert.ocsp.ThisUpdate), zap.Time("next_update", cert.ocsp.NextUpdate)) } // our copy of cert has the new OCSP staple, so replace it in the cache cfg.certCache.mu.Lock() cfg.certCache.cache[cert.hash] = cert cfg.certCache.mu.Unlock() } // Check ARI status, but it's only relevant if the certificate is not expired (otherwise, we already know it needs renewal!) if !cfg.DisableARI && cert.ari.NeedsRefresh() && time.Now().Before(cert.Leaf.NotAfter) { // update ARI in a goroutine to avoid blocking an active handshake, since the results of // this do not strictly affect the handshake; even though the cert may be updated with // the new ARI, it is also updated in the cache and in storage, so future handshakes // will utilize it go func(hello *tls.ClientHelloInfo, cert Certificate, logger *zap.Logger) { // TODO: a different context that isn't tied to the handshake is probably better // than a generic background context; maybe a longer-lived server config context, // or something that the importing package sets on the Config struct; for example, // a Caddy config context could be good, so that ARI updates will continue after // the handshake goes away, but will be stopped if the underlying server is stopped // (for now, use an unusual timeout to help recognize it in log patterns, if needed) ctx, cancel := context.WithTimeout(context.Background(), 8*time.Minute) defer cancel() var err error // we ignore the second return value here because we check renewal status below regardless cert, _, err = cfg.updateARI(ctx, cert, logger) if err != nil { logger.Error("updating ARI", zap.Error(err)) } _, err = renewIfNecessary(ctx, hello, cert) if err != nil { logger.Error("renewing certificate based on updated ARI", zap.Error(err)) } }(hello, cert, logger) } // We attempt to replace any certificates that were revoked. // Crucially, this happens OUTSIDE a lock on the certCache. if certShouldBeForceRenewed(cert) { logger.Warn("on-demand certificate's OCSP status is REVOKED; will try to forcefully renew", zap.Int("ocsp_status", cert.ocsp.Status), zap.Time("revoked_at", cert.ocsp.RevokedAt), zap.Time("this_update", cert.ocsp.ThisUpdate), zap.Time("next_update", cert.ocsp.NextUpdate)) return cfg.renewDynamicCertificate(ctx, hello, cert) } // Since renewal conditions may have changed, do a renewal if necessary return renewIfNecessary(ctx, hello, cert) } // renewDynamicCertificate renews the certificate for name using cfg. It returns the // certificate to use and an error, if any. name should already be lower-cased before // calling this function. name is the name obtained directly from the handshake's // ClientHello. If the certificate hasn't yet expired, currentCert will be returned // and the renewal will happen in the background; otherwise this blocks until the // certificate has been renewed, and returns the renewed certificate. // // If the certificate's OCSP status (currentCert.ocsp) is Revoked, it will be forcefully // renewed even if it is not expiring. // // This function is safe for use by multiple concurrent goroutines. func (cfg *Config) renewDynamicCertificate(ctx context.Context, hello *tls.ClientHelloInfo, currentCert Certificate) (Certificate, error) { logger := logWithRemote(cfg.Logger.Named("on_demand"), hello) name, err := cfg.getNameFromClientHello(hello) if err != nil { return Certificate{}, err } timeLeft := time.Until(expiresAt(currentCert.Leaf)) revoked := currentCert.ocsp != nil && currentCert.ocsp.Status == ocsp.Revoked // see if another goroutine is already working on this certificate obtainCertWaitChansMu.Lock() wait, ok := obtainCertWaitChans[name] if ok { // lucky us -- another goroutine is already renewing the certificate obtainCertWaitChansMu.Unlock() // the current certificate hasn't expired, and another goroutine is already // renewing it, so we might as well serve what we have without blocking, UNLESS // we're forcing renewal, in which case the current certificate is not usable if timeLeft > 0 && !revoked { logger.Debug("certificate expires soon but is already being renewed; serving current certificate", zap.Strings("subjects", currentCert.Names), zap.Duration("remaining", timeLeft)) return currentCert, nil } // otherwise, we'll have to wait for the renewal to finish so we don't serve // a revoked or expired certificate logger.Debug("certificate has expired, but is already being renewed; waiting for renewal to complete", zap.Strings("subjects", currentCert.Names), zap.Time("expired", expiresAt(currentCert.Leaf)), zap.Bool("revoked", revoked)) // TODO: see if we can get a proper context in here, for true cancellation timeout := time.NewTimer(2 * time.Minute) select { case <-timeout.C: return Certificate{}, fmt.Errorf("timed out waiting for certificate renewal of %s", name) case <-wait: timeout.Stop() } // it should now be loaded in the cache, ready to go; if not, // the goroutine in charge of that probably had an error return cfg.getCertDuringHandshake(ctx, hello, false) } // looks like it's up to us to do all the work and renew the cert wait = make(chan struct{}) obtainCertWaitChans[name] = wait obtainCertWaitChansMu.Unlock() unblockWaiters := func() { obtainCertWaitChansMu.Lock() close(wait) delete(obtainCertWaitChans, name) obtainCertWaitChansMu.Unlock() } logger = logger.With( zap.String("server_name", name), zap.Strings("subjects", currentCert.Names), zap.Time("expiration", expiresAt(currentCert.Leaf)), zap.Duration("remaining", timeLeft), zap.Bool("revoked", revoked), ) // Renew and reload the certificate renewAndReload := func(ctx context.Context, cancel context.CancelFunc) (Certificate, error) { defer cancel() // Make sure a certificate for this name should be renewed on-demand err := cfg.checkIfCertShouldBeObtained(ctx, name, true) if err != nil { // if not, remove from cache (it will be deleted from storage later) cfg.certCache.mu.Lock() cfg.certCache.removeCertificate(currentCert) cfg.certCache.mu.Unlock() unblockWaiters() if logger != nil { logger.Error("certificate should not be obtained", zap.Error(err)) } return Certificate{}, err } logger.Info("attempting certificate renewal") // otherwise, renew with issuer, etc. var newCert Certificate if revoked { newCert, err = cfg.forceRenew(ctx, logger, currentCert) } else { err = cfg.RenewCertAsync(ctx, name, false) if err == nil { // load from storage while in lock to make the replacement as atomic as possible newCert, err = cfg.reloadManagedCertificate(ctx, currentCert) } } // immediately unblock anyone waiting for it; doing this in // a defer would risk deadlock because of the recursive call // to getCertDuringHandshake below when we return! unblockWaiters() if err != nil { logger.Error("renewing and reloading certificate", zap.String("server_name", name), zap.Error(err)) } return newCert, err } // if the certificate hasn't expired, we can serve what we have and renew in the background if timeLeft > 0 { ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute) go renewAndReload(ctx, cancel) return currentCert, nil } // otherwise, we have to block while we renew an expired certificate ctx, cancel := context.WithTimeout(ctx, 90*time.Second) return renewAndReload(ctx, cancel) } // getCertFromAnyCertManager gets a certificate from cfg's Managers. If there are no Managers defined, this is // a no-op that returns empty values. Otherwise, it gets a certificate for hello from the first Manager that // returns a certificate and no error. func (cfg *Config) getCertFromAnyCertManager(ctx context.Context, hello *tls.ClientHelloInfo, logger *zap.Logger) (Certificate, error) { // fast path if nothing to do if cfg.OnDemand == nil || len(cfg.OnDemand.Managers) == 0 { return Certificate{}, nil } // try all the GetCertificate methods on external managers; use first one that returns a certificate var upstreamCert *tls.Certificate var err error for i, certManager := range cfg.OnDemand.Managers { upstreamCert, err = certManager.GetCertificate(ctx, hello) if err != nil { logger.Error("external certificate manager", zap.String("sni", hello.ServerName), zap.String("cert_manager", fmt.Sprintf("%T", certManager)), zap.Int("cert_manager_idx", i), zap.Error(err)) continue } if upstreamCert != nil { break } } if err != nil { return Certificate{}, fmt.Errorf("external certificate manager indicated that it is unable to yield certificate: %v", err) } if upstreamCert == nil { logger.Debug("all external certificate managers yielded no certificates and no errors", zap.String("sni", hello.ServerName)) return Certificate{}, nil } var cert Certificate if err = fillCertFromLeaf(&cert, *upstreamCert); err != nil { return Certificate{}, fmt.Errorf("external certificate manager: %s: filling cert from leaf: %v", hello.ServerName, err) } logger.Debug("using externally-managed certificate", zap.String("sni", hello.ServerName), zap.Strings("names", cert.Names), zap.Time("expiration", expiresAt(cert.Leaf))) return cert, nil } // getTLSALPNChallengeCert is to be called when the clientHello pertains to // a TLS-ALPN challenge and a certificate is required to solve it. This method gets // the relevant challenge info and then returns the associated certificate (if any) // or generates it anew if it's not available (as is the case when distributed // solving). True is returned if the challenge is being solved distributed (there // is no semantic difference with distributed solving; it is mainly for logging). func (cfg *Config) getTLSALPNChallengeCert(clientHello *tls.ClientHelloInfo) (*tls.Certificate, bool, error) { chalData, distributed, err := cfg.getACMEChallengeInfo(clientHello.Context(), clientHello.ServerName, true) if err != nil { return nil, distributed, err } // fast path: we already created the certificate (this avoids having to re-create // it at every handshake that tries to verify, e.g. multi-perspective validation) if chalData.data != nil { return chalData.data.(*tls.Certificate), distributed, nil } // otherwise, we can re-create the solution certificate, but it takes a few cycles cert, err := acmez.TLSALPN01ChallengeCert(chalData.Challenge) if err != nil { return nil, distributed, fmt.Errorf("making TLS-ALPN challenge certificate: %v", err) } if cert == nil { return nil, distributed, fmt.Errorf("got nil TLS-ALPN challenge certificate but no error") } return cert, distributed, nil } // getNameFromClientHello returns a normalized form of hello.ServerName. // If hello.ServerName is empty (i.e. client did not use SNI), then the // associated connection's local address is used to extract an IP address. func (cfg *Config) getNameFromClientHello(hello *tls.ClientHelloInfo) (string, error) { // IDNs must be converted to punycode for use in TLS certificates (and SNI), but not // all clients do that, so convert IDNs to ASCII according to RFC 5280 section 7 // using profile recommended by RFC 5891 section 5; this solves the "σςΣ" problem // (see https://unicode.org/faq/idn.html#22) where not all normalizations are 1:1. // The Lookup profile, for instance, rejects wildcard characters (*), but they // should never be used in the ClientHello SNI anyway. name, err := idna.Lookup.ToASCII(strings.TrimSpace(hello.ServerName)) if err != nil { return "", err } if name != "" { return name, nil } if cfg.DefaultServerName != "" { return normalizedName(cfg.DefaultServerName), nil } return localIPFromConn(hello.Conn), nil } // logWithRemote adds the remote host and port to the logger. func logWithRemote(l *zap.Logger, hello *tls.ClientHelloInfo) *zap.Logger { if hello.Conn == nil || l == nil { return l } addr := hello.Conn.RemoteAddr().String() ip, port, err := net.SplitHostPort(addr) if err != nil { ip = addr port = "" } return l.With(zap.String("remote_ip", ip), zap.String("remote_port", port)) } // localIPFromConn returns the host portion of c's local address // and strips the scope ID if one exists (see RFC 4007). func localIPFromConn(c net.Conn) string { if c == nil { return "" } localAddr := c.LocalAddr().String() ip, _, err := net.SplitHostPort(localAddr) if err != nil { // OK; assume there was no port ip = localAddr } // IPv6 addresses can have scope IDs, e.g. "fe80::4c3:3cff:fe4f:7e0b%eth0", // but for our purposes, these are useless (unless a valid use case proves // otherwise; see issue #3911) if scopeIDStart := strings.Index(ip, "%"); scopeIDStart > -1 { ip = ip[:scopeIDStart] } return ip } // normalizedName returns a cleaned form of serverName that is // used for consistency when referring to a SNI value. func normalizedName(serverName string) string { return strings.ToLower(strings.TrimSpace(serverName)) } // obtainCertWaitChans is used to coordinate obtaining certs for each hostname. var ( obtainCertWaitChans = make(map[string]chan struct{}) obtainCertWaitChansMu sync.Mutex ) // certLoadWaiter coordinates concurrent certificate loading for the same name. // The leader populates the result and closes the channel; waiters read the result // after the channel is closed. This allows externally-managed certificates (which // are not cached) to be shared directly with waiting goroutines. type certLoadWaiter struct { done chan struct{} cert Certificate err error } // TODO: this lockset should probably be per-cache var ( certLoadWaitChans = make(map[string]*certLoadWaiter) certLoadWaitChansMu sync.Mutex ) type serializableClientHello struct { CipherSuites []uint16 ServerName string SupportedCurves []tls.CurveID SupportedPoints []uint8 SignatureSchemes []tls.SignatureScheme SupportedProtos []string SupportedVersions []uint16 RemoteAddr, LocalAddr net.Addr // values copied from the Conn as they are still useful/needed conn net.Conn // unexported so it's not serialized } // clientHelloWithoutConn returns the data from the ClientHelloInfo without the // pesky exported Conn field, which often causes an error when serializing because // the underlying type may be unserializable. func clientHelloWithoutConn(hello *tls.ClientHelloInfo) serializableClientHello { if hello == nil { return serializableClientHello{} } var remote, local net.Addr if hello.Conn != nil { remote = hello.Conn.RemoteAddr() local = hello.Conn.LocalAddr() } return serializableClientHello{ CipherSuites: hello.CipherSuites, ServerName: hello.ServerName, SupportedCurves: hello.SupportedCurves, SupportedPoints: hello.SupportedPoints, SignatureSchemes: hello.SignatureSchemes, SupportedProtos: hello.SupportedProtos, SupportedVersions: hello.SupportedVersions, RemoteAddr: remote, LocalAddr: local, conn: hello.Conn, } } type helloInfoCtxKey string // ClientHelloInfoCtxKey is the key by which the ClientHelloInfo can be extracted from // a context.Context within a DecisionFunc. However, be advised that it is best practice // that the decision whether to obtain a certificate is be based solely on the name, // not other properties of the specific connection/client requesting the connection. // For example, it is not advisable to use a client's IP address to decide whether to // allow a certificate. Instead, the ClientHello can be useful for logging, etc. const ClientHelloInfoCtxKey helloInfoCtxKey = "certmagic:ClientHelloInfo" golang-github-caddyserver-certmagic-0.25.2/handshake_test.go000066400000000000000000000104061514710434200241120ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "crypto/tls" "crypto/x509" "net" "testing" ) func TestGetCertificate(t *testing.T) { c := &Cache{ cache: make(map[string]Certificate), cacheIndex: make(map[string][]string), logger: defaultTestLogger, } cfg := &Config{Logger: defaultTestLogger, certCache: c} // create a test connection for conn.LocalAddr() l, _ := net.Listen("tcp", "127.0.0.1:0") defer l.Close() conn, _ := net.Dial("tcp", l.Addr().String()) if conn == nil { t.Errorf("failed to create a test connection") } defer conn.Close() hello := &tls.ClientHelloInfo{ServerName: "example.com", Conn: conn} helloSub := &tls.ClientHelloInfo{ServerName: "sub.example.com", Conn: conn} helloNoSNI := &tls.ClientHelloInfo{Conn: conn} helloNoMatch := &tls.ClientHelloInfo{ServerName: "nomatch", Conn: conn} // When cache is empty if cert, err := cfg.GetCertificate(hello); err == nil { t.Errorf("GetCertificate should return error when cache is empty, got: %v", cert) } if cert, err := cfg.GetCertificate(helloNoSNI); err == nil { t.Errorf("GetCertificate should return error when cache is empty even if server name is blank, got: %v", cert) } // When cache has one certificate in it firstCert := Certificate{Names: []string{"example.com"}, Certificate: tls.Certificate{Leaf: &x509.Certificate{DNSNames: []string{"example.com"}}}} c.cacheCertificate(firstCert) if cert, err := cfg.GetCertificate(hello); err != nil { t.Errorf("Got an error but shouldn't have, when cert exists in cache: %v", err) } else if cert.Leaf.DNSNames[0] != "example.com" { t.Errorf("Got wrong certificate with exact match; expected 'example.com', got: %v", cert) } if _, err := cfg.GetCertificate(helloNoSNI); err == nil { t.Errorf("Did not get an error with no SNI and no DefaultServerName, but should have: %v", err) } // When retrieving wildcard certificate wildcardCert := Certificate{ Names: []string{"*.example.com"}, Certificate: tls.Certificate{Leaf: &x509.Certificate{DNSNames: []string{"*.example.com"}}}, hash: "(don't overwrite the first one)", } c.cacheCertificate(wildcardCert) if cert, err := cfg.GetCertificate(helloSub); err != nil { t.Errorf("Didn't get wildcard cert, got: cert=%v, err=%v ", cert, err) } else if cert.Leaf.DNSNames[0] != "*.example.com" { t.Errorf("Got wrong certificate, expected wildcard: %v", cert) } // When cache is NOT empty but there's no SNI if _, err := cfg.GetCertificate(helloNoSNI); err == nil { t.Errorf("Expected TLS allert when no SNI and no DefaultServerName, but got: %v", err) } // When no certificate matches, raise an alert if _, err := cfg.GetCertificate(helloNoMatch); err == nil { t.Errorf("Expected an error when no certificate matched the SNI, got: %v", err) } // When default SNI is set and SNI is missing, retrieve default cert cfg.DefaultServerName = "example.com" if cert, err := cfg.GetCertificate(helloNoSNI); err != nil { t.Errorf("Got an error with no SNI with DefaultServerName, but shouldn't have: %v", err) } else if cert == nil || cert.Leaf.DNSNames[0] != "example.com" { t.Errorf("Expected default cert, got: %v", cert) } // When default SNI is set and SNI is missing but IP address matches, retrieve IP cert ipCert := Certificate{ Names: []string{"127.0.0.1"}, Certificate: tls.Certificate{Leaf: &x509.Certificate{IPAddresses: []net.IP{net.ParseIP("127.0.0.1")}}}, hash: "(don't overwrite the first or second one)", } c.cacheCertificate(ipCert) if cert, err := cfg.GetCertificate(helloNoSNI); err != nil { t.Errorf("Got an error with no SNI but matching IP, but shouldn't have: %v", err) } else if cert == nil || len(cert.Leaf.IPAddresses) == 0 { t.Errorf("Expected IP cert, got: %v", cert) } } golang-github-caddyserver-certmagic-0.25.2/httphandlers.go000066400000000000000000000261371514710434200236350ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "fmt" "net/http" "net/url" "strings" "github.com/mholt/acmez/v3/acme" "go.uber.org/zap" ) // HTTPChallengeHandler wraps h in a handler that can solve the ACME // HTTP challenge. cfg is required, and it must have a certificate // cache backed by a functional storage facility, since that is where // the challenge state is stored between initiation and solution. // // If a request is not an ACME HTTP challenge, h will be invoked. func (am *ACMEIssuer) HTTPChallengeHandler(h http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if am.HandleHTTPChallenge(w, r) { return } h.ServeHTTP(w, r) }) } // HandleHTTPChallenge uses am to solve challenge requests from an ACME // server that were initiated by this instance or any other instance in // this cluster (being, any instances using the same storage am does). // // If the HTTP challenge is disabled, this function is a no-op. // // If am is nil or if am does not have a certificate cache backed by // usable storage, solving the HTTP challenge will fail. // // It returns true if it handled the request; if so, the response has // already been written. If false is returned, this call was a no-op and // the request has not been handled. func (am *ACMEIssuer) HandleHTTPChallenge(w http.ResponseWriter, r *http.Request) bool { if am == nil { return false } if am.DisableHTTPChallenge { return false } if !LooksLikeHTTPChallenge(r) { return false } return am.distributedHTTPChallengeSolver(w, r) } // distributedHTTPChallengeSolver checks to see if this challenge // request was initiated by this or another instance which uses the // same storage as am does, and attempts to complete the challenge for // it. It returns true if the request was handled; false otherwise. func (am *ACMEIssuer) distributedHTTPChallengeSolver(w http.ResponseWriter, r *http.Request) bool { if am == nil { return false } host := hostOnly(r.Host) chalInfo, distributed, err := am.config.getACMEChallengeInfo(r.Context(), host, !am.DisableDistributedSolvers) if err != nil { if am.DisableDistributedSolvers { // Distributed solvers are disabled, so the only way an error can be returned is if // this instance didn't initiate the challenge (or if the process exited after, but // either way, we don't have the challenge info). Assuming this is a legitimate // challenge request, we may still be able to solve it if we can present the correct // account thumbprint with the token, since the token is given to us in the URL path. // // NOTE: About doing this, RFC 8555 section 8.3 says: // // Note that because the token appears both in the request sent by the // ACME server and in the key authorization in the response, it is // possible to build clients that copy the token from request to // response. Clients should avoid this behavior because it can lead to // cross-site scripting vulnerabilities; instead, clients should be // explicitly configured on a per-challenge basis. A client that does // copy tokens from requests to responses MUST validate that the token // in the request matches the token syntax above (e.g., that it includes // only characters from the base64url alphabet). // // Also, since we're just blindly solving a challenge, we're unable to mitigate DNS // rebinding attacks, because we don't know what host to expect in the URL. So this // is not ideal, but we do at least validate the copied token is in the base64url set. if strings.HasPrefix(r.URL.Path, acmeHTTPChallengeBasePath) && strings.Count(r.URL.Path, "/") == 3 && r.Method == http.MethodGet { tokenStart := strings.LastIndex(r.URL.Path, "/") + 1 token := r.URL.Path[tokenStart:] if allBase64URL(token) { if err := am.solveHTTPChallengeBlindly(w, r); err != nil { am.Logger.Error("solving http-01 challenge blindly", zap.String("identifier", host), zap.Error(err)) } return true } } } // couldn't get challenge info even with distributed solver am.Logger.Warn("looking up info for HTTP challenge", zap.String("uri", r.RequestURI), zap.String("identifier", host), zap.String("remote_addr", r.RemoteAddr), zap.String("user_agent", r.Header.Get("User-Agent")), zap.Error(err)) return false } return solveHTTPChallenge(am.Logger, w, r, chalInfo.Challenge, distributed) } // solveHTTPChallengeBlindly will try to respond correctly with an http-01 challenge response. // The request must be an http-01 challenge request. We cannot know for sure the ACME CA that // is requesting this, so we have to guess as we load the account to use for a thumbprint as // part of the response body. It is a no-op if the last component of the URL path contains // characters outside of the base64url alphabet. func (am *ACMEIssuer) solveHTTPChallengeBlindly(w http.ResponseWriter, r *http.Request) error { tokenStart := strings.LastIndex(r.URL.Path, "/") + 1 token := r.URL.Path[tokenStart:] if allBase64URL(token) { acct, err := am.getAccountToUse(r.Context(), am.CA) // assume production CA, I guess if err != nil { return fmt.Errorf("getting an account to use: %v", err) } thumbprint, err := acct.Thumbprint() if err != nil { return fmt.Errorf("could not encode account thumbprint: %v", err) } w.Header().Add("Content-Type", "text/plain") _, _ = w.Write([]byte(token + "." + thumbprint)) r.Close = true am.Logger.Info("served key authentication", zap.String("identifier", hostOnly(r.Host)), zap.String("challenge", "http-01"), zap.String("remote", r.RemoteAddr), zap.Bool("distributed", false), zap.Bool("blind", true), zap.String("ca", am.CA)) } return nil } // allBase64URL returns true if all characters of s are in the base64url alphabet. func allBase64URL(s string) bool { for _, c := range s { if (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-' || c == '_' { continue } return false } return true } // solveHTTPChallenge solves the HTTP challenge using the given challenge information. // If the challenge is being solved in a distributed fahsion, set distributed to true for logging purposes. // It returns true the properties of the request check out in relation to the HTTP challenge. // Most of this code borrowed from xenolf's built-in HTTP-01 challenge solver in March 2018. func solveHTTPChallenge(logger *zap.Logger, w http.ResponseWriter, r *http.Request, challenge acme.Challenge, distributed bool) bool { challengeReqPath := challenge.HTTP01ResourcePath() if r.URL.Path == challengeReqPath && strings.EqualFold(hostOnly(r.Host), challenge.Identifier.Value) && // mitigate DNS rebinding attacks r.Method == http.MethodGet { w.Header().Add("Content-Type", "text/plain") w.Write([]byte(challenge.KeyAuthorization)) r.Close = true logger.Info("served key authentication", zap.String("identifier", challenge.Identifier.Value), zap.String("challenge", "http-01"), zap.String("remote", r.RemoteAddr), zap.Bool("distributed", distributed)) return true } return false } // SolveHTTPChallenge solves the HTTP challenge. It should be used only on HTTP requests that are // from ACME servers trying to validate an identifier (i.e. LooksLikeHTTPChallenge() == true). It // returns true if the request criteria check out and it answered with key authentication, in which // case no further handling of the request is necessary. func SolveHTTPChallenge(logger *zap.Logger, w http.ResponseWriter, r *http.Request, challenge acme.Challenge) bool { return solveHTTPChallenge(logger, w, r, challenge, false) } // LooksLikeHTTPChallenge returns true if r looks like an ACME // HTTP challenge request from an ACME server. func LooksLikeHTTPChallenge(r *http.Request) bool { return r.Method == http.MethodGet && strings.HasPrefix(r.URL.Path, acmeHTTPChallengeBasePath) } // LooksLikeZeroSSLHTTPValidation returns true if the request appears to be // domain validation from a ZeroSSL/Sectigo CA. NOTE: This API is // non-standard and is subject to change. func LooksLikeZeroSSLHTTPValidation(r *http.Request) bool { return r.Method == http.MethodGet && strings.HasPrefix(r.URL.Path, zerosslHTTPValidationBasePath) } // HTTPValidationHandler wraps the ZeroSSL HTTP validation handler such that // it can pass verification checks from ZeroSSL's API. // // If a request is not a ZeroSSL HTTP validation request, h will be invoked. func (iss *ZeroSSLIssuer) HTTPValidationHandler(h http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if iss.HandleZeroSSLHTTPValidation(w, r) { return } h.ServeHTTP(w, r) }) } // HandleZeroSSLHTTPValidation is to ZeroSSL API HTTP validation requests like HandleHTTPChallenge // is to ACME HTTP challenge requests. func (iss *ZeroSSLIssuer) HandleZeroSSLHTTPValidation(w http.ResponseWriter, r *http.Request) bool { if iss == nil { return false } if !LooksLikeZeroSSLHTTPValidation(r) { return false } return iss.distributedHTTPValidationAnswer(w, r) } func (iss *ZeroSSLIssuer) distributedHTTPValidationAnswer(w http.ResponseWriter, r *http.Request) bool { if iss == nil { return false } logger := iss.Logger if logger == nil { logger = zap.NewNop() } host := hostOnly(r.Host) valInfo, distributed, err := iss.getDistributedValidationInfo(r.Context(), host) if err != nil { logger.Warn("looking up info for HTTP validation", zap.String("host", host), zap.String("remote_addr", r.RemoteAddr), zap.String("user_agent", r.Header.Get("User-Agent")), zap.Error(err)) return false } return answerHTTPValidation(logger, w, r, valInfo, distributed) } func answerHTTPValidation(logger *zap.Logger, rw http.ResponseWriter, req *http.Request, valInfo acme.Challenge, distributed bool) bool { // ensure URL matches validationURL, err := url.Parse(valInfo.URL) if err != nil { logger.Error("got invalid URL from CA", zap.String("file_validation_url", valInfo.URL), zap.Error(err)) rw.WriteHeader(http.StatusInternalServerError) return true } if req.URL.Path != validationURL.Path { rw.WriteHeader(http.StatusNotFound) return true } rw.Header().Add("Content-Type", "text/plain") req.Close = true rw.Write([]byte(valInfo.Token)) logger.Info("served HTTP validation credential", zap.String("validation_path", valInfo.URL), zap.String("challenge", "http-01"), zap.String("remote", req.RemoteAddr), zap.Bool("distributed", distributed)) return true } const ( acmeHTTPChallengeBasePath = "/.well-known/acme-challenge" zerosslHTTPValidationBasePath = "/.well-known/pki-validation/" ) golang-github-caddyserver-certmagic-0.25.2/httphandlers_test.go000066400000000000000000000034431514710434200246670ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "net/http" "net/http/httptest" "os" "testing" ) func TestHTTPChallengeHandlerNoOp(t *testing.T) { am := &ACMEIssuer{CA: "https://example.com/acme/directory", Logger: defaultTestLogger} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata_tmp"}, Logger: defaultTestLogger, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() // try base paths and host names that aren't // handled by this handler for _, url := range []string{ "http://localhost/", "http://localhost/foo.html", "http://localhost/.git", "http://localhost/.well-known/", "http://localhost/.well-known/acme-challenging", "http://other/.well-known/acme-challenge/foo", } { req, err := http.NewRequest("GET", url, nil) if err != nil { t.Fatalf("Could not craft request, got error: %v", err) } rw := httptest.NewRecorder() if am.HandleHTTPChallenge(rw, req) { t.Errorf("Got true with this URL, but shouldn't have: %s", url) } } } golang-github-caddyserver-certmagic-0.25.2/internal/000077500000000000000000000000001514710434200224115ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.25.2/internal/atomicfile/000077500000000000000000000000001514710434200245255ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.25.2/internal/atomicfile/README000066400000000000000000000002561514710434200254100ustar00rootroot00000000000000# atomic file this is copied from https://github.com/containerd/containerd/blob/main/pkg%2Fatomicfile%2Ffile.go see https://github.com/caddyserver/certmagic/issues/296 golang-github-caddyserver-certmagic-0.25.2/internal/atomicfile/file.go000066400000000000000000000113601514710434200257740ustar00rootroot00000000000000/* Copyright The containerd Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ /* Package atomicfile provides a mechanism (on Unix-like platforms) to present a consistent view of a file to separate processes even while the file is being written. This is accomplished by writing a temporary file, syncing to disk, and renaming over the destination file name. Partial/inconsistent reads can occur due to: 1. A process attempting to read the file while it is being written to (both in the case of a new file with a short/incomplete write or in the case of an existing, updated file where new bytes may be written at the beginning but old bytes may still be present after). 2. Concurrent goroutines leading to multiple active writers of the same file. The above mechanism explicitly protects against (1) as all writes are to a file with a temporary name. There is no explicit protection against multiple, concurrent goroutines attempting to write the same file. However, atomically writing the file should mean only one writer will "win" and a consistent file will be visible. Note: atomicfile is partially implemented for Windows. The Windows codepath performs the same operations, however Windows does not guarantee that a rename operation is atomic; a crash in the middle may leave the destination file truncated rather than with the expected content. */ package atomicfile import ( "errors" "fmt" "io" "os" "path/filepath" "sync" ) // File is an io.ReadWriteCloser that can also be Canceled if a change needs to be abandoned. type File interface { io.ReadWriteCloser // Cancel abandons a change to a file. This can be called if a write fails or another error occurs. Cancel() error } // ErrClosed is returned if Read or Write are called on a closed File. var ErrClosed = errors.New("file is closed") // New returns a new atomic file. On Unix-like platforms, the writer (an io.ReadWriteCloser) is backed by a temporary // file placed into the same directory as the destination file (using filepath.Dir to split the directory from the // name). On a call to Close the temporary file is synced to disk and renamed to its final name, hiding any previous // file by the same name. // // Note: Take care to call Close and handle any errors that are returned. Errors returned from Close may indicate that // the file was not written with its final name. func New(name string, mode os.FileMode) (File, error) { return newFile(name, mode) } type atomicFile struct { name string f *os.File closed bool closedMu sync.RWMutex } func newFile(name string, mode os.FileMode) (File, error) { dir := filepath.Dir(name) f, err := os.CreateTemp(dir, "") if err != nil { return nil, fmt.Errorf("failed to create temp file: %w", err) } if err := f.Chmod(mode); err != nil { return nil, fmt.Errorf("failed to change temp file permissions: %w", err) } return &atomicFile{name: name, f: f}, nil } func (a *atomicFile) Close() (err error) { a.closedMu.Lock() defer a.closedMu.Unlock() if a.closed { return nil } a.closed = true defer func() { if err != nil { _ = os.Remove(a.f.Name()) // ignore errors } }() // The order of operations here is: // 1. sync // 2. close // 3. rename // While the ordering of 2 and 3 is not important on Unix-like operating systems, Windows cannot rename an open // file. By closing first, we allow the rename operation to succeed. if err = a.f.Sync(); err != nil { return fmt.Errorf("failed to sync temp file %q: %w", a.f.Name(), err) } if err = a.f.Close(); err != nil { return fmt.Errorf("failed to close temp file %q: %w", a.f.Name(), err) } if err = os.Rename(a.f.Name(), a.name); err != nil { return fmt.Errorf("failed to rename %q to %q: %w", a.f.Name(), a.name, err) } return nil } func (a *atomicFile) Cancel() error { a.closedMu.Lock() defer a.closedMu.Unlock() if a.closed { return nil } a.closed = true _ = a.f.Close() // ignore error return os.Remove(a.f.Name()) } func (a *atomicFile) Read(p []byte) (n int, err error) { a.closedMu.RLock() defer a.closedMu.RUnlock() if a.closed { return 0, ErrClosed } return a.f.Read(p) } func (a *atomicFile) Write(p []byte) (n int, err error) { a.closedMu.RLock() defer a.closedMu.RUnlock() if a.closed { return 0, ErrClosed } return a.f.Write(p) } golang-github-caddyserver-certmagic-0.25.2/internal/atomicfile/file_test.go000066400000000000000000000051511514710434200270340ustar00rootroot00000000000000/* Copyright The containerd Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package atomicfile_test import ( "fmt" "os" "path/filepath" "testing" "github.com/caddyserver/certmagic/internal/atomicfile" "github.com/caddyserver/certmagic/internal/testutil" ) func TestFile(t *testing.T) { const content = "this is some test content for a file" dir := t.TempDir() path := filepath.Join(dir, "test-file") f, err := atomicfile.New(path, 0o644) testutil.RequireNoError(t, err, "failed to create file") n, err := fmt.Fprint(f, content) testutil.AssertNoError(t, err, "failed to write content") testutil.AssertEqual(t, len(content), n, "written bytes should be equal") err = f.Close() testutil.RequireNoError(t, err, "failed to close file") actual, err := os.ReadFile(path) testutil.AssertNoError(t, err, "failed to read file") testutil.AssertEqual(t, content, string(actual)) } func TestConcurrentWrites(t *testing.T) { const content1 = "this is the first content of the file. there should be none other." const content2 = "the second content of the file should win!" dir := t.TempDir() path := filepath.Join(dir, "test-file") file1, err := atomicfile.New(path, 0o600) testutil.RequireNoError(t, err, "failed to create file1") file2, err := atomicfile.New(path, 0o644) testutil.RequireNoError(t, err, "failed to create file2") n, err := fmt.Fprint(file1, content1) testutil.AssertNoError(t, err, "failed to write content1") testutil.AssertEqual(t, len(content1), n, "written bytes should be equal") n, err = fmt.Fprint(file2, content2) testutil.AssertNoError(t, err, "failed to write content2") testutil.AssertEqual(t, len(content2), n, "written bytes should be equal") err = file1.Close() testutil.RequireNoError(t, err, "failed to close file1") actual, err := os.ReadFile(path) testutil.AssertNoError(t, err, "failed to read file") testutil.AssertEqual(t, content1, string(actual)) err = file2.Close() testutil.RequireNoError(t, err, "failed to close file2") actual, err = os.ReadFile(path) testutil.AssertNoError(t, err, "failed to read file") testutil.AssertEqual(t, content2, string(actual)) } golang-github-caddyserver-certmagic-0.25.2/internal/testutil/000077500000000000000000000000001514710434200242665ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.25.2/internal/testutil/readme.md000066400000000000000000000023171514710434200260500ustar00rootroot00000000000000# testutil some testing functions copied out of github.com/stretchr/testify, which were originally used by atomicfile ``` MIT License Copyright (c) 2012-2020 Mat Ryer, Tyler Bunnell and contributors. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ``` golang-github-caddyserver-certmagic-0.25.2/internal/testutil/testutil.go000066400000000000000000000045721514710434200265020ustar00rootroot00000000000000package testutil import ( "bytes" "fmt" "reflect" "strings" "testing" ) func AssertNoError(t *testing.T, err error, msgAndArgs ...string) { if err != nil { Fail(t, fmt.Sprintf("Received unexpected error:\n%+v", err), msgAndArgs...) } } func AssertEqual(t *testing.T, a, b any, msgAndArgs ...string) { if !ObjectsAreEqual(a, b) { Fail(t, fmt.Sprintf("Not equal: \n"+ "expected: %v\n"+ "actual : %v", a, b), msgAndArgs...) } } func RequireNoError(t *testing.T, err error, msgAndArgs ...string) { if err != nil { Failnow(t, fmt.Sprintf("Received unexpected error:\n%+v", err), msgAndArgs...) } } func RequireError(t *testing.T, err error, msgAndArgs ...string) { if err == nil { Failnow(t, fmt.Sprintf("Received no error when expecting error:\n%+v", err), msgAndArgs...) } } func RequireEqual(t *testing.T, a, b any, msgAndArgs ...string) { if !ObjectsAreEqual(a, b) { Failnow(t, fmt.Sprintf("Not equal: \n"+ "expected: %v\n"+ "actual : %v", a, b), msgAndArgs...) } } func RequireEqualValues(t *testing.T, a, b any, msgAndArgs ...string) { if !ObjectsAreEqualValues(a, b) { Failnow(t, fmt.Sprintf("Not equal: \n"+ "expected: %v\n"+ "actual : %v", a, b), msgAndArgs...) } } func Fail(t testing.TB, xs string, msgs ...string) { var testName string // Add test name if the Go version supports it if n, ok := t.(interface { Name() string }); ok { testName = n.Name() } t.Errorf("error %s:%s\n%s\n", testName, xs, strings.Join(msgs, "")) } func Failnow(t testing.TB, xs string, msgs ...string) { Fail(t, xs, msgs...) t.FailNow() } func ObjectsAreEqual(expected, actual any) bool { if expected == nil || actual == nil { return expected == actual } exp, ok := expected.([]byte) if !ok { return reflect.DeepEqual(expected, actual) } act, ok := actual.([]byte) if !ok { return false } if exp == nil || act == nil { return exp == nil && act == nil } return bytes.Equal(exp, act) } func ObjectsAreEqualValues(expected, actual any) bool { if ObjectsAreEqual(expected, actual) { return true } actualType := reflect.TypeOf(actual) if actualType == nil { return false } expectedValue := reflect.ValueOf(expected) if expectedValue.IsValid() && expectedValue.Type().ConvertibleTo(actualType) { // Attempt comparison after type conversion return reflect.DeepEqual(expectedValue.Convert(actualType).Interface(), actual) } return false } golang-github-caddyserver-certmagic-0.25.2/maintain.go000066400000000000000000001057171514710434200227370ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/x509" "encoding/json" "encoding/pem" "errors" "fmt" "io/fs" "path" "runtime" "strings" "time" "github.com/mholt/acmez/v3/acme" "go.uber.org/zap" "golang.org/x/crypto/ocsp" ) // maintainAssets is a permanently-blocking function // that loops indefinitely and, on a regular schedule, checks // certificates for expiration and initiates a renewal of certs // that are expiring soon. It also updates OCSP stapling. It // should only be called once per cache. Panics are recovered, // and if panicCount < 10, the function is called recursively, // incrementing panicCount each time. Initial invocation should // start panicCount at 0. func (certCache *Cache) maintainAssets(panicCount int) { log := certCache.logger.Named("maintenance") log = log.With(zap.String("cache", fmt.Sprintf("%p", certCache))) defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Error("panic", zap.Any("error", err), zap.ByteString("stack", buf)) if panicCount < 10 { certCache.maintainAssets(panicCount + 1) } } }() certCache.optionsMu.RLock() renewalTicker := time.NewTicker(certCache.options.RenewCheckInterval) ocspTicker := time.NewTicker(certCache.options.OCSPCheckInterval) certCache.optionsMu.RUnlock() log.Info("started background certificate maintenance") ctx, cancel := context.WithCancel(context.Background()) defer cancel() for { select { case <-renewalTicker.C: err := certCache.RenewManagedCertificates(ctx) if err != nil { log.Error("renewing managed certificates", zap.Error(err)) } case <-ocspTicker.C: certCache.updateOCSPStaples(ctx) case <-certCache.stopChan: renewalTicker.Stop() ocspTicker.Stop() log.Info("stopped background certificate maintenance") close(certCache.doneChan) return } } } // RenewManagedCertificates renews managed certificates, // including ones loaded on-demand. Note that this is done // automatically on a regular basis; normally you will not // need to call this. This method assumes non-interactive // mode (i.e. operating in the background). func (certCache *Cache) RenewManagedCertificates(ctx context.Context) error { log := certCache.logger.Named("maintenance") // configs will hold a map of certificate hash to the config // to use when managing that certificate configs := make(map[string]*Config) // we use the queues for a very important reason: to do any and all // operations that could require an exclusive write lock outside // of the read lock! otherwise we get a deadlock, yikes. in other // words, our first iteration through the certificate cache does NOT // perform any operations--only queues them--so that more fine-grained // write locks may be obtained during the actual operations. var renewQueue, reloadQueue, deleteQueue, ariQueue certList certCache.mu.RLock() for certKey, cert := range certCache.cache { if !cert.managed { continue } // the list of names on this cert should never be empty... programmer error? if len(cert.Names) == 0 { log.Warn("certificate has no names; removing from cache", zap.String("cert_key", certKey)) deleteQueue = append(deleteQueue, cert) continue } // get the config associated with this certificate cfg, err := certCache.getConfig(cert) if err != nil { log.Error("unable to get configuration to manage certificate; unable to renew", zap.Strings("identifiers", cert.Names), zap.Error(err)) continue } if cfg.OnDemand != nil { continue } // ACME-specific: see if if ACME Renewal Info (ARI) window needs refreshing if !cfg.DisableARI && cert.ari.NeedsRefresh() { configs[cert.hash] = cfg ariQueue = append(ariQueue, cert) } // if time is up or expires soon, we need to try to renew it if cert.NeedsRenewal(cfg) { configs[cert.hash] = cfg // see if the certificate in storage has already been renewed, possibly by another // instance that didn't coordinate with this one; if so, just load it (this // might happen if another instance already renewed it - kinda sloppy but checking disk // first is a simple way to possibly drastically reduce rate limit problems) storedCertNeedsRenew, err := cfg.managedCertInStorageNeedsRenewal(ctx, cert) if err != nil { // hmm, weird, but not a big deal, maybe it was deleted or something log.Warn("error while checking if stored certificate is also expiring soon", zap.Strings("identifiers", cert.Names), zap.Error(err)) } else if !storedCertNeedsRenew { // if the certificate does NOT need renewal and there was no error, then we // are good to just reload the certificate from storage instead of repeating // a likely-unnecessary renewal procedure reloadQueue = append(reloadQueue, cert) continue } // the certificate in storage has not been renewed yet, so we will do it // NOTE: It is super-important to note that the TLS-ALPN challenge requires // a write lock on the cache in order to complete its challenge, so it is extra // vital that this renew operation does not happen inside our read lock! renewQueue.insert(cert) } } certCache.mu.RUnlock() // Update ARI, and then for any certs where the ARI window changed, // be sure to queue them for renewal if necessary for _, cert := range ariQueue { cfg := configs[cert.hash] cert, changed, err := cfg.updateARI(ctx, cert, log) if err != nil { log.Error("updating ARI", zap.Error(err)) } if changed && cert.NeedsRenewal(cfg) { // it's theoretically possible that another instance already got the memo // on the changed ARI and even renewed the cert already, and thus doing it // here is wasteful, but I have never heard of this happening in reality, // so to save some cycles for now I think we'll just queue it for renewal // (notice how we use 'insert' to avoid duplicates, in case it was already // scheduled for renewal anyway) renewQueue.insert(cert) } } // Reload certificates that merely need to be updated in memory for _, oldCert := range reloadQueue { timeLeft := expiresAt(oldCert.Leaf).Sub(time.Now().UTC()) log.Info("certificate expires soon, but is already renewed in storage; reloading stored certificate", zap.Strings("identifiers", oldCert.Names), zap.Duration("remaining", timeLeft)) cfg := configs[oldCert.hash] // crucially, this happens OUTSIDE a lock on the certCache _, err := cfg.reloadManagedCertificate(ctx, oldCert) if err != nil { log.Error("loading renewed certificate", zap.Strings("identifiers", oldCert.Names), zap.Error(err)) continue } } // Renewal queue for _, oldCert := range renewQueue { cfg := configs[oldCert.hash] err := certCache.queueRenewalTask(ctx, oldCert, cfg) if err != nil { log.Error("queueing renewal task", zap.Strings("identifiers", oldCert.Names), zap.Error(err)) continue } } // Deletion queue certCache.mu.Lock() for _, cert := range deleteQueue { certCache.removeCertificate(cert) } certCache.mu.Unlock() return nil } func (certCache *Cache) queueRenewalTask(ctx context.Context, oldCert Certificate, cfg *Config) error { log := certCache.logger.Named("maintenance") timeLeft := expiresAt(oldCert.Leaf).Sub(time.Now().UTC()) log.Info("certificate expires soon; queuing for renewal", zap.Strings("identifiers", oldCert.Names), zap.Duration("remaining", timeLeft)) // Get the name which we should use to renew this certificate; // we only support managing certificates with one name per cert, // so this should be easy. renewName := oldCert.Names[0] // queue up this renewal job (is a no-op if already active or queued) jm.Submit(cfg.Logger, "renew_"+renewName, func() error { timeLeft := expiresAt(oldCert.Leaf).Sub(time.Now().UTC()) log.Info("attempting certificate renewal", zap.Strings("identifiers", oldCert.Names), zap.Duration("remaining", timeLeft)) // perform renewal - crucially, this happens OUTSIDE a lock on certCache err := cfg.RenewCertAsync(ctx, renewName, false) if err != nil { if cfg.OnDemand != nil { // loaded dynamically, remove dynamically certCache.mu.Lock() certCache.removeCertificate(oldCert) certCache.mu.Unlock() } return fmt.Errorf("%v %v", oldCert.Names, err) } // successful renewal, so update in-memory cache by loading // renewed certificate so it will be used with handshakes _, err = cfg.reloadManagedCertificate(ctx, oldCert) if err != nil { return ErrNoRetry{fmt.Errorf("%v %v", oldCert.Names, err)} } return nil }) return nil } // updateOCSPStaples updates the OCSP stapling in all // eligible, cached certificates. // // OCSP maintenance strives to abide the relevant points on // Ryan Sleevi's recommendations for good OCSP support: // https://gist.github.com/sleevi/5efe9ef98961ecfb4da8 func (certCache *Cache) updateOCSPStaples(ctx context.Context) { logger := certCache.logger.Named("maintenance") // temporary structures to store updates or tasks // so that we can keep our locks short-lived type ocspUpdate struct { rawBytes []byte parsed *ocsp.Response } type updateQueueEntry struct { cert Certificate certHash string lastNextUpdate time.Time cfg *Config } type renewQueueEntry struct { oldCert Certificate cfg *Config } updated := make(map[string]ocspUpdate) var updateQueue []updateQueueEntry // certs that need a refreshed staple var renewQueue []renewQueueEntry // certs that need to be renewed (due to revocation) // obtain brief read lock during our scan to see which staples need updating certCache.mu.RLock() for certHash, cert := range certCache.cache { // no point in updating OCSP for expired or "synthetic" certificates if cert.Leaf == nil || cert.Expired() { continue } cfg, err := certCache.getConfig(cert) if err != nil { logger.Error("unable to get automation config for certificate; maintenance for this certificate will likely fail", zap.Strings("identifiers", cert.Names), zap.Error(err)) continue } // always try to replace revoked certificates, even if OCSP response is still fresh if certShouldBeForceRenewed(cert) { renewQueue = append(renewQueue, renewQueueEntry{ oldCert: cert, cfg: cfg, }) continue } // if the status is not fresh, get a new one var lastNextUpdate time.Time if cert.ocsp != nil { lastNextUpdate = cert.ocsp.NextUpdate if cert.ocsp.Status != ocsp.Unknown && freshOCSP(cert.ocsp) { // no need to update our staple if still fresh and not Unknown continue } } updateQueue = append(updateQueue, updateQueueEntry{cert, certHash, lastNextUpdate, cfg}) } certCache.mu.RUnlock() // perform updates outside of any lock on certCache for _, qe := range updateQueue { cert := qe.cert certHash := qe.certHash lastNextUpdate := qe.lastNextUpdate if qe.cfg == nil { // this is bad if this happens, probably a programmer error (oops) logger.Error("no configuration associated with certificate; unable to manage OCSP staples", zap.Strings("identifiers", cert.Names)) continue } err := stapleOCSP(ctx, qe.cfg.OCSP, qe.cfg.Storage, &cert, nil) if err != nil { if cert.ocsp != nil { // if there was no staple before, that's fine; otherwise we should log the error logger.Error("stapling OCSP", zap.Strings("identifiers", cert.Names), zap.Error(err)) } continue } // By this point, we've obtained the latest OCSP response. // If there was no staple before, or if the response is updated, make // sure we apply the update to all names on the certificate if // the status is still Good. if cert.ocsp != nil && cert.ocsp.Status == ocsp.Good && (lastNextUpdate.IsZero() || lastNextUpdate != cert.ocsp.NextUpdate) { logger.Info("advancing OCSP staple", zap.Strings("identifiers", cert.Names), zap.Time("from", lastNextUpdate), zap.Time("to", cert.ocsp.NextUpdate)) updated[certHash] = ocspUpdate{rawBytes: cert.Certificate.OCSPStaple, parsed: cert.ocsp} } // If the updated staple shows that the certificate was revoked, we should immediately renew it if certShouldBeForceRenewed(cert) { qe.cfg.emit(ctx, "cert_ocsp_revoked", map[string]any{ "subjects": cert.Names, "certificate": cert, "reason": cert.ocsp.RevocationReason, "revoked_at": cert.ocsp.RevokedAt, }) renewQueue = append(renewQueue, renewQueueEntry{ oldCert: cert, cfg: qe.cfg, }) } } // These write locks should be brief since we have all the info we need now. for certKey, update := range updated { certCache.mu.Lock() if cert, ok := certCache.cache[certKey]; ok { cert.ocsp = update.parsed cert.Certificate.OCSPStaple = update.rawBytes certCache.cache[certKey] = cert } certCache.mu.Unlock() } // We attempt to replace any certificates that were revoked. // Crucially, this happens OUTSIDE a lock on the certCache. for _, renew := range renewQueue { _, err := renew.cfg.forceRenew(ctx, logger, renew.oldCert) if err != nil { logger.Info("forcefully renewing certificate due to REVOKED status", zap.Strings("identifiers", renew.oldCert.Names), zap.Error(err)) } } } // storageHasNewerARI returns true if the configured storage has ARI that is newer // than that of a certificate that is already loaded, along with the value from // storage. func (cfg *Config) storageHasNewerARI(ctx context.Context, cert Certificate) (bool, acme.RenewalInfo, error) { storedCert, err := cfg.loadStoredACMECertificateMetadata(ctx, cert) if err != nil || storedCert.RenewalInfo == nil || storedCert.RenewalInfo.RetryAfter == nil { return false, acme.RenewalInfo{}, err } // prefer stored info if it has a window and the loaded one doesn't, // or if the one in storage has a later RetryAfter (though I suppose // it's not guaranteed, typically those will move forward in time) if (!cert.ari.HasWindow() && storedCert.RenewalInfo.HasWindow()) || (cert.ari.RetryAfter == nil || storedCert.RenewalInfo.RetryAfter.After(*cert.ari.RetryAfter)) { return true, *storedCert.RenewalInfo, nil } return false, acme.RenewalInfo{}, nil } // loadStoredACMECertificateMetadata loads the stored ACME certificate data // from the cert's sidecar JSON file. func (cfg *Config) loadStoredACMECertificateMetadata(ctx context.Context, cert Certificate) (acme.Certificate, error) { metaBytes, err := cfg.Storage.Load(ctx, StorageKeys.SiteMeta(cert.issuerKey, cert.Names[0])) if err != nil { return acme.Certificate{}, fmt.Errorf("loading cert metadata: %w", err) } var certRes CertificateResource if err = json.Unmarshal(metaBytes, &certRes); err != nil { return acme.Certificate{}, fmt.Errorf("unmarshaling cert metadata: %w", err) } var acmeCert acme.Certificate if err = json.Unmarshal(certRes.IssuerData, &acmeCert); err != nil { return acme.Certificate{}, fmt.Errorf("unmarshaling potential ACME issuer metadata: %v", err) } return acmeCert, nil } // updateARI updates the cert's ACME renewal info, first by checking storage for a newer // one, or getting it from the CA if needed. The updated info is stored in storage and // updated in the cache. The certificate with the updated ARI is returned. If true is // returned, the ARI window or selected time has changed, and the caller should check if // the cert needs to be renewed now, even if there is an error. // // This will always try to ARI without checking if it needs to be refreshed. Call // NeedsRefresh() on the RenewalInfo first, and only call this if that returns true. func (cfg *Config) updateARI(ctx context.Context, cert Certificate, logger *zap.Logger) (updatedCert Certificate, changed bool, err error) { logger = logger.With( zap.Strings("identifiers", cert.Names), zap.String("cert_hash", cert.hash), zap.String("ari_unique_id", cert.ari.UniqueIdentifier), zap.Time("cert_expiry", cert.Leaf.NotAfter)) updatedCert = cert oldARI := cert.ari // synchronize ARI fetching; see #297 lockName := "ari_" + cert.ari.UniqueIdentifier if _, ok := cfg.Storage.(TryLocker); ok { ok, err := tryAcquireLock(ctx, cfg.Storage, lockName) if err != nil { return cert, false, fmt.Errorf("unable to obtain ARI lock: %v", err) } if !ok { logger.Debug("attempted to obtain ARI lock but it was already taken") return cert, false, nil } } else if err := acquireLock(ctx, cfg.Storage, lockName); err != nil { return cert, false, fmt.Errorf("unable to obtain ARI lock: %v", err) } defer func() { if err := releaseLock(ctx, cfg.Storage, lockName); err != nil { logger.Error("unable to release ARI lock", zap.Error(err)) } }() // see if the stored value has been refreshed already by another instance gotNewARI, newARI, err := cfg.storageHasNewerARI(ctx, cert) // when we're all done, log if something about the schedule is different // ("WARN" level because ARI window changing may be a sign of external trouble // and we want to draw their attention to a potential explanation URL) defer func() { changed = !newARI.SameWindow(oldARI) if changed { logger.Warn("ARI window or selected renewal time changed", zap.Time("prev_start", oldARI.SuggestedWindow.Start), zap.Time("next_start", newARI.SuggestedWindow.Start), zap.Time("prev_end", oldARI.SuggestedWindow.End), zap.Time("next_end", newARI.SuggestedWindow.End), zap.Time("prev_selected_time", oldARI.SelectedTime), zap.Time("next_selected_time", newARI.SelectedTime), zap.String("explanation_url", newARI.ExplanationURL)) } }() if err == nil && gotNewARI { // great, storage has a newer one we can use cfg.certCache.mu.Lock() var ok bool updatedCert, ok = cfg.certCache.cache[cert.hash] if !ok { // cert is no longer in the cache... why? what's the right thing to do here? cfg.certCache.mu.Unlock() updatedCert = cert // return input cert, not an empty one updatedCert.ari = newARI // might as well give it the new ARI for the benefit of our caller, but it won't be updated in the cache or in storage logger.Warn("loaded newer ARI from storage, but certificate is no longer in cache; newer ARI will be returned to caller, but not persisted in the cache", zap.Time("selected_time", newARI.SelectedTime), zap.Timep("next_update", newARI.RetryAfter), zap.String("explanation_url", newARI.ExplanationURL)) return } updatedCert.ari = newARI cfg.certCache.cache[cert.hash] = updatedCert cfg.certCache.mu.Unlock() logger.Info("reloaded ARI with newer one in storage", zap.Timep("next_refresh", newARI.RetryAfter), zap.Time("renewal_time", newARI.SelectedTime)) return } if err != nil { logger.Error("error while checking storage for updated ARI; updating ARI now", zap.Error(err)) } // of the issuers configured, hopefully one of them is the ACME CA we got the cert from for _, iss := range cfg.Issuers { if ariGetter, ok := iss.(RenewalInfoGetter); ok && iss.IssuerKey() == cert.issuerKey { newARI, err = ariGetter.GetRenewalInfo(ctx, cert) // be sure to use existing newARI variable so we can compare against old value in the defer if err != nil { // could be anything, but a common error might simply be the "wrong" ACME CA // (meaning, different from the one that issued the cert, thus the only one // that would have any ARI for it) if multiple ACME CAs are configured logger.Error("failed updating renewal info from ACME CA", zap.String("issuer", iss.IssuerKey()), zap.Error(err)) continue } // when we get the latest ARI, the acme package will select a time within the window // for us; of course, since it's random, it's likely different from the previously- // selected time; but if the window doesn't change, there's no need to change the // selected time (the acme package doesn't know the previous window to know better) // ... so if the window hasn't changed we'll just put back the selected time if newARI.SameWindow(oldARI) && !oldARI.SelectedTime.IsZero() { newARI.SelectedTime = oldARI.SelectedTime } // then store the updated ARI (even if the window didn't change, the Retry-After // likely did) in cache and storage // be sure we get the cert from the cache while inside a lock to avoid logical races cfg.certCache.mu.Lock() updatedCert, ok = cfg.certCache.cache[cert.hash] if !ok { // cert is no longer in the cache; this can happen for several reasons (past expiration, // rejected by on-demand permission module, random eviction due to full cache, etc), but // it probably means we don't have use of this ARI update now, so while we can return it // to the caller, we don't persist it anywhere beyond that... cfg.certCache.mu.Unlock() updatedCert = cert // return input cert, not an empty one updatedCert.ari = newARI // might as well give it the new ARI for the benefit of our caller, but it won't be updated in the cache or in storage logger.Warn("obtained ARI update, but certificate no longer in cache; ARI update will be returned to caller, but not stored", zap.Time("selected_time", newARI.SelectedTime), zap.Timep("next_update", newARI.RetryAfter), zap.String("explanation_url", newARI.ExplanationURL)) return } updatedCert.ari = newARI cfg.certCache.cache[cert.hash] = updatedCert cfg.certCache.mu.Unlock() // update the ARI value in storage var certData acme.Certificate certData, err = cfg.loadStoredACMECertificateMetadata(ctx, cert) if err != nil { err = fmt.Errorf("got new ARI from %s, but failed loading stored certificate metadata: %v", iss.IssuerKey(), err) return } certData.RenewalInfo = &newARI var certDataBytes, certResBytes []byte certDataBytes, err = json.Marshal(certData) if err != nil { err = fmt.Errorf("got new ARI from %s, but failed marshaling certificate ACME metadata: %v", iss.IssuerKey(), err) return } certResBytes, err = json.MarshalIndent(CertificateResource{ SANs: cert.Names, IssuerData: certDataBytes, }, "", "\t") if err != nil { err = fmt.Errorf("got new ARI from %s, but could not re-encode certificate metadata: %v", iss.IssuerKey(), err) return } if err = cfg.Storage.Store(ctx, StorageKeys.SiteMeta(cert.issuerKey, cert.Names[0]), certResBytes); err != nil { err = fmt.Errorf("got new ARI from %s, but could not store it with certificate metadata: %v", iss.IssuerKey(), err) return } logger.Info("updated and stored ACME renewal information", zap.Time("selected_time", newARI.SelectedTime), zap.Timep("next_update", newARI.RetryAfter), zap.String("explanation_url", newARI.ExplanationURL)) return } } err = fmt.Errorf("could not fully update ACME renewal info: either no issuer supporting ARI is configured for certificate, or all such failed (make sure the ACME CA that issued the certificate is configured)") return } // CleanStorageOptions specifies how to clean up a storage unit. type CleanStorageOptions struct { // Optional custom logger. Logger *zap.Logger // Optional ID of the instance initiating the cleaning. InstanceID string // If set, cleaning will be skipped if it was performed // more recently than this interval. Interval time.Duration // Whether to clean cached OCSP staples. OCSPStaples bool // Whether to cleanup expired certificates, and if so, // how long to let them stay after they've expired. ExpiredCerts bool ExpiredCertGracePeriod time.Duration } // CleanStorage removes assets which are no longer useful, // according to opts. func CleanStorage(ctx context.Context, storage Storage, opts CleanStorageOptions) error { const ( lockName = "storage_clean" storageKey = "last_clean.json" ) if opts.Logger == nil { opts.Logger = defaultLogger.Named("clean_storage") } opts.Logger = opts.Logger.With(zap.Any("storage", storage)) // storage cleaning should be globally exclusive if err := acquireLock(ctx, storage, lockName); err != nil { return fmt.Errorf("unable to acquire %s lock: %v", lockName, err) } defer func() { if err := releaseLock(ctx, storage, lockName); err != nil { opts.Logger.Error("unable to release lock", zap.Error(err)) return } }() // cleaning should not happen more often than the interval if opts.Interval > 0 { lastCleanBytes, err := storage.Load(ctx, storageKey) if !errors.Is(err, fs.ErrNotExist) { if err != nil { return fmt.Errorf("loading last clean timestamp: %v", err) } var lastClean lastCleanPayload err = json.Unmarshal(lastCleanBytes, &lastClean) if err != nil { return fmt.Errorf("decoding last clean data: %v", err) } lastTLSClean := lastClean["tls"] if time.Since(lastTLSClean.Timestamp) < opts.Interval { nextTime := time.Now().Add(opts.Interval) opts.Logger.Info("storage cleaning happened too recently; skipping for now", zap.String("instance", lastTLSClean.InstanceID), zap.Time("try_again", nextTime), zap.Duration("try_again_in", time.Until(nextTime)), ) return nil } } } opts.Logger.Info("cleaning storage unit") if opts.OCSPStaples { err := deleteOldOCSPStaples(ctx, storage, opts.Logger) if err != nil { opts.Logger.Error("deleting old OCSP staples", zap.Error(err)) } } if opts.ExpiredCerts { err := deleteExpiredCerts(ctx, storage, opts.Logger, opts.ExpiredCertGracePeriod) if err != nil { opts.Logger.Error("deleting expired certificates staples", zap.Error(err)) } } // TODO: delete stale locks? // update the last-clean time lastCleanBytes, err := json.Marshal(lastCleanPayload{ "tls": lastCleaned{ Timestamp: time.Now(), InstanceID: opts.InstanceID, }, }) if err != nil { return fmt.Errorf("encoding last cleaned info: %v", err) } if err := storage.Store(ctx, storageKey, lastCleanBytes); err != nil { return fmt.Errorf("storing last clean info: %v", err) } return nil } type lastCleanPayload map[string]lastCleaned type lastCleaned struct { Timestamp time.Time `json:"timestamp"` InstanceID string `json:"instance_id,omitempty"` } func deleteOldOCSPStaples(ctx context.Context, storage Storage, logger *zap.Logger) error { ocspKeys, err := storage.List(ctx, prefixOCSP, false) if err != nil { // maybe just hasn't been created yet; no big deal return nil } for _, key := range ocspKeys { // if context was cancelled, quit early; otherwise proceed select { case <-ctx.Done(): return ctx.Err() default: } ocspBytes, err := storage.Load(ctx, key) if err != nil { logger.Error("while deleting old OCSP staples, unable to load staple file", zap.Error(err)) continue } resp, err := ocsp.ParseResponse(ocspBytes, nil) if err != nil { // contents are invalid; delete it err = storage.Delete(ctx, key) if err != nil { logger.Error("purging corrupt staple file", zap.String("storage_key", key), zap.Error(err)) } continue } if time.Now().After(resp.NextUpdate) { // response has expired; delete it err = storage.Delete(ctx, key) if err != nil { logger.Error("purging expired staple file", zap.String("storage_key", key), zap.Error(err)) } } } return nil } func deleteExpiredCerts(ctx context.Context, storage Storage, logger *zap.Logger, gracePeriod time.Duration) error { issuerKeys, err := storage.List(ctx, prefixCerts, false) if err != nil { // maybe just hasn't been created yet; no big deal return nil } for _, issuerKey := range issuerKeys { siteKeys, err := storage.List(ctx, issuerKey, false) if err != nil { logger.Error("listing contents", zap.String("issuer_key", issuerKey), zap.Error(err)) continue } for _, siteKey := range siteKeys { // if context was cancelled, quit early; otherwise proceed select { case <-ctx.Done(): return ctx.Err() default: } siteAssets, err := storage.List(ctx, siteKey, false) if err != nil { logger.Error("listing site contents", zap.String("site_key", siteKey), zap.Error(err)) continue } for _, assetKey := range siteAssets { if path.Ext(assetKey) != ".crt" { continue } certFile, err := storage.Load(ctx, assetKey) if err != nil { return fmt.Errorf("loading certificate file %s: %v", assetKey, err) } block, _ := pem.Decode(certFile) if block == nil || block.Type != "CERTIFICATE" { return fmt.Errorf("certificate file %s does not contain PEM-encoded certificate", assetKey) } cert, err := x509.ParseCertificate(block.Bytes) if err != nil { return fmt.Errorf("certificate file %s is malformed; error parsing PEM: %v", assetKey, err) } if expiredTime := time.Since(expiresAt(cert)); expiredTime >= gracePeriod { logger.Info("certificate expired beyond grace period; cleaning up", zap.String("asset_key", assetKey), zap.Duration("expired_for", expiredTime), zap.Duration("grace_period", gracePeriod)) baseName := strings.TrimSuffix(assetKey, ".crt") for _, relatedAsset := range []string{ assetKey, baseName + ".key", baseName + ".json", } { logger.Info("deleting asset because resource expired", zap.String("asset_key", relatedAsset)) err := storage.Delete(ctx, relatedAsset) if err != nil { logger.Error("could not clean up asset related to expired certificate", zap.String("base_name", baseName), zap.String("related_asset", relatedAsset), zap.Error(err)) } } } } // update listing; if folder is empty, delete it siteAssets, err = storage.List(ctx, siteKey, false) if err != nil { continue } if len(siteAssets) == 0 { logger.Info("deleting site folder because key is empty", zap.String("site_key", siteKey)) err := storage.Delete(ctx, siteKey) if err != nil { return fmt.Errorf("deleting empty site folder %s: %v", siteKey, err) } } } } return nil } // forceRenew forcefully renews cert and replaces it in the cache, and returns the new certificate. It is intended // for use primarily in the case of cert revocation. This MUST NOT be called within a lock on cfg.certCacheMu. func (cfg *Config) forceRenew(ctx context.Context, logger *zap.Logger, cert Certificate) (Certificate, error) { if cert.ocsp != nil && cert.ocsp.Status == ocsp.Revoked { logger.Warn("OCSP status for managed certificate is REVOKED; attempting to replace with new certificate", zap.Strings("identifiers", cert.Names), zap.Time("expiration", expiresAt(cert.Leaf))) } else { logger.Warn("forcefully renewing certificate", zap.Strings("identifiers", cert.Names), zap.Time("expiration", expiresAt(cert.Leaf))) } renewName := cert.Names[0] // if revoked for key compromise, we can't be sure whether the storage of // the key is still safe; however, we KNOW the old key is not safe, and we // can only hope by the time of revocation that storage has been secured; // key management is not something we want to get into, but in this case // it seems prudent to replace the key - and since renewal requires reuse // of a prior key, we can't do a "renew" to replace the cert if we need a // new key, so we'll have to do an obtain instead var obtainInsteadOfRenew bool if cert.ocsp != nil && cert.ocsp.RevocationReason == acme.ReasonKeyCompromise { err := cfg.moveCompromisedPrivateKey(ctx, cert, logger) if err != nil { logger.Error("could not remove compromised private key from use", zap.Strings("identifiers", cert.Names), zap.String("issuer", cert.issuerKey), zap.Error(err)) } obtainInsteadOfRenew = true } var err error if obtainInsteadOfRenew { err = cfg.ObtainCertAsync(ctx, renewName) } else { // notice that we force renewal; otherwise, it might see that the // certificate isn't close to expiring and return, but we really // need a replacement certificate! see issue #4191 err = cfg.RenewCertAsync(ctx, renewName, true) } if err != nil { if cert.ocsp != nil && cert.ocsp.Status == ocsp.Revoked { // probably better to not serve a revoked certificate at all logger.Error("unable to obtain new to certificate after OCSP status of REVOKED; removing from cache", zap.Strings("identifiers", cert.Names), zap.Error(err)) cfg.certCache.mu.Lock() cfg.certCache.removeCertificate(cert) cfg.certCache.mu.Unlock() } return cert, fmt.Errorf("unable to forcefully get new certificate for %v: %w", cert.Names, err) } return cfg.reloadManagedCertificate(ctx, cert) } // moveCompromisedPrivateKey moves the private key for cert to a ".compromised" file // by copying the data to the new file, then deleting the old one. func (cfg *Config) moveCompromisedPrivateKey(ctx context.Context, cert Certificate, logger *zap.Logger) error { privKeyStorageKey := StorageKeys.SitePrivateKey(cert.issuerKey, cert.Names[0]) privKeyPEM, err := cfg.Storage.Load(ctx, privKeyStorageKey) if err != nil { return err } compromisedPrivKeyStorageKey := privKeyStorageKey + ".compromised" err = cfg.Storage.Store(ctx, compromisedPrivKeyStorageKey, privKeyPEM) if err != nil { // better safe than sorry: as a last resort, try deleting the key so it won't be reused cfg.Storage.Delete(ctx, privKeyStorageKey) return err } err = cfg.Storage.Delete(ctx, privKeyStorageKey) if err != nil { return err } logger.Info("removed certificate's compromised private key from use", zap.String("storage_path", compromisedPrivKeyStorageKey), zap.Strings("identifiers", cert.Names), zap.String("issuer", cert.issuerKey)) return nil } // certShouldBeForceRenewed returns true if cert should be forcefully renewed // (like if it is revoked according to its OCSP response). func certShouldBeForceRenewed(cert Certificate) bool { return cert.managed && len(cert.Names) > 0 && cert.ocsp != nil && cert.ocsp.Status == ocsp.Revoked } type certList []Certificate // insert appends cert to the list if it is not already in the list. // Efficiency: O(n) func (certs *certList) insert(cert Certificate) { for _, c := range *certs { if c.hash == cert.hash { return } } *certs = append(*certs, cert) } const ( // DefaultRenewCheckInterval is how often to check certificates for expiration. // Scans are very lightweight, so this can be semi-frequent. This default should // be smaller than *DefaultRenewalWindowRatio/3, which // gives certificates plenty of chance to be renewed on time. DefaultRenewCheckInterval = 10 * time.Minute // DefaultRenewalWindowRatio is how much of a certificate's lifetime becomes the // renewal window. The renewal window is the span of time at the end of the // certificate's validity period in which it should be renewed. A default value // of ~1/3 is pretty safe and recommended for most certificates. DefaultRenewalWindowRatio = 1.0 / 3.0 // DefaultOCSPCheckInterval is how often to check if OCSP stapling needs updating. DefaultOCSPCheckInterval = 1 * time.Hour ) golang-github-caddyserver-certmagic-0.25.2/ocsp.go000066400000000000000000000215401514710434200220720ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bytes" "context" "crypto/x509" "encoding/pem" "errors" "fmt" "io" "log" "net/http" "time" "golang.org/x/crypto/ocsp" ) // ErrNoOCSPServerSpecified indicates that OCSP information could not be // stapled because the certificate does not support OCSP. var ErrNoOCSPServerSpecified = errors.New("no OCSP server specified in certificate") // stapleOCSP staples OCSP information to cert for hostname name. // If you have it handy, you should pass in the PEM-encoded certificate // bundle; otherwise the DER-encoded cert will have to be PEM-encoded. // If you don't have the PEM blocks already, just pass in nil. // // If successful, the OCSP response will be set to cert's ocsp field, // regardless of the OCSP status. It is only stapled, however, if the // status is Good. // // Errors here are not necessarily fatal, it could just be that the // certificate doesn't have an issuer URL. func stapleOCSP(ctx context.Context, ocspConfig OCSPConfig, storage Storage, cert *Certificate, pemBundle []byte) error { if ocspConfig.DisableStapling { return nil } if pemBundle == nil { // we need a PEM encoding only for some function calls below bundle := new(bytes.Buffer) for _, derBytes := range cert.Certificate.Certificate { pem.Encode(bundle, &pem.Block{Type: "CERTIFICATE", Bytes: derBytes}) } pemBundle = bundle.Bytes() } var ocspBytes []byte var ocspResp *ocsp.Response var ocspErr error var gotNewOCSP bool // First try to load OCSP staple from storage and see if // we can still use it. ocspStapleKey := StorageKeys.OCSPStaple(cert, pemBundle) cachedOCSP, err := storage.Load(ctx, ocspStapleKey) if err == nil { resp, err := ocsp.ParseResponse(cachedOCSP, nil) if err == nil { if freshOCSP(resp) { // staple is still fresh; use it ocspBytes = cachedOCSP ocspResp = resp } } else { // invalid contents; delete the file // (we do this independently of the maintenance routine because // in this case we know for sure this should be a staple file // because we loaded it by name, whereas the maintenance routine // just iterates the list of files, even if somehow a non-staple // file gets in the folder. in this case we are sure it is corrupt.) err := storage.Delete(ctx, ocspStapleKey) if err != nil { log.Printf("[WARNING] Unable to delete invalid OCSP staple file: %v", err) } } } // If we couldn't get a fresh staple by reading the cache, // then we need to request it from the OCSP responder if ocspResp == nil || len(ocspBytes) == 0 { ocspBytes, ocspResp, ocspErr = getOCSPForCert(ocspConfig, pemBundle) // An error here is not a problem because a certificate // may simply not contain a link to an OCSP server. if ocspErr != nil { // For short-lived certificates, this is fine and we can ignore // logging because OCSP doesn't make much sense for them anyway. if cert.Lifetime() < 7*24*time.Hour { return nil } // There's nothing else we can do to get OCSP for this certificate, // so we can return here with the error to warn about it. return fmt.Errorf("no OCSP stapling for %v: %w", cert.Names, ocspErr) } gotNewOCSP = true } if ocspResp.NextUpdate.After(expiresAt(cert.Leaf)) { // uh oh, this OCSP response expires AFTER the certificate does, that's kinda bogus. // it was the reason a lot of Symantec-validated sites (not Caddy) went down // in October 2017. https://twitter.com/mattiasgeniar/status/919432824708648961 return fmt.Errorf("invalid: OCSP response for %v valid after certificate expiration (%s)", cert.Names, expiresAt(cert.Leaf).Sub(ocspResp.NextUpdate)) } // Attach the latest OCSP response to the certificate; this is NOT the same // as stapling it, which we do below only if the status is Good, but it is // useful to keep with the cert in order to act on it later (like if Revoked). cert.ocsp = ocspResp // If the response is good, staple it to the certificate. If the OCSP // response was not loaded from storage, we persist it for next time. if ocspResp.Status == ocsp.Good { cert.Certificate.OCSPStaple = ocspBytes if gotNewOCSP { err := storage.Store(ctx, ocspStapleKey, ocspBytes) if err != nil { return fmt.Errorf("unable to write OCSP staple file for %v: %v", cert.Names, err) } } } return nil } // getOCSPForCert takes a PEM encoded cert or cert bundle returning the raw OCSP response, // the parsed response, and an error, if any. The returned []byte can be passed directly // into the OCSPStaple property of a tls.Certificate. If the bundle only contains the // issued certificate, this function will try to get the issuer certificate from the // IssuingCertificateURL in the certificate. If the []byte and/or ocsp.Response return // values are nil, the OCSP status may be assumed OCSPUnknown. // // Borrowed from xenolf. func getOCSPForCert(ocspConfig OCSPConfig, bundle []byte) ([]byte, *ocsp.Response, error) { // TODO: Perhaps this should be synchronized too, with a Locker? certificates, err := parseCertsFromPEMBundle(bundle) if err != nil { return nil, nil, err } // We expect the certificate slice to be ordered downwards the chain. // SRV CRT -> CA. We need to pull the leaf and issuer certs out of it, // which should always be the first two certificates. If there's no // OCSP server listed in the leaf cert, there's nothing to do. And if // we have only one certificate so far, we need to get the issuer cert. issuedCert := certificates[0] if len(issuedCert.OCSPServer) == 0 { return nil, nil, ErrNoOCSPServerSpecified } // apply override for responder URL respURL := issuedCert.OCSPServer[0] if len(ocspConfig.ResponderOverrides) > 0 { if override, ok := ocspConfig.ResponderOverrides[respURL]; ok { respURL = override } } if respURL == "" { return nil, nil, fmt.Errorf("override disables querying OCSP responder: %v", issuedCert.OCSPServer[0]) } // configure HTTP client if necessary httpClient := http.DefaultClient if ocspConfig.HTTPProxy != nil { httpClient = &http.Client{ Transport: &http.Transport{ Proxy: ocspConfig.HTTPProxy, }, Timeout: 30 * time.Second, } } // get issuer certificate if needed if len(certificates) == 1 { if len(issuedCert.IssuingCertificateURL) == 0 { return nil, nil, fmt.Errorf("no URL to issuing certificate") } resp, err := httpClient.Get(issuedCert.IssuingCertificateURL[0]) if err != nil { return nil, nil, fmt.Errorf("getting issuer certificate: %v", err) } defer resp.Body.Close() issuerBytes, err := io.ReadAll(io.LimitReader(resp.Body, 1024*1024)) if err != nil { return nil, nil, fmt.Errorf("reading issuer certificate: %v", err) } issuerCert, err := x509.ParseCertificate(issuerBytes) if err != nil { return nil, nil, fmt.Errorf("parsing issuer certificate: %v", err) } // insert it into the slice on position 0; // we want it ordered right SRV CRT -> CA certificates = append(certificates, issuerCert) } issuerCert := certificates[1] ocspReq, err := ocsp.CreateRequest(issuedCert, issuerCert, nil) if err != nil { return nil, nil, fmt.Errorf("creating OCSP request: %v", err) } reader := bytes.NewReader(ocspReq) req, err := httpClient.Post(respURL, "application/ocsp-request", reader) if err != nil { return nil, nil, fmt.Errorf("making OCSP request: %v", err) } defer req.Body.Close() ocspResBytes, err := io.ReadAll(io.LimitReader(req.Body, 1024*1024)) if err != nil { return nil, nil, fmt.Errorf("reading OCSP response: %v", err) } ocspRes, err := ocsp.ParseResponse(ocspResBytes, issuerCert) if err != nil { return nil, nil, fmt.Errorf("parsing OCSP response: %v", err) } return ocspResBytes, ocspRes, nil } // freshOCSP returns true if resp is still fresh, // meaning that it is not expedient to get an // updated response from the OCSP server. func freshOCSP(resp *ocsp.Response) bool { nextUpdate := resp.NextUpdate // If there is an OCSP responder certificate, and it expires before the // OCSP response, use its expiration date as the end of the OCSP // response's validity period. if resp.Certificate != nil && resp.Certificate.NotAfter.Before(nextUpdate) { nextUpdate = resp.Certificate.NotAfter } // start checking OCSP staple about halfway through validity period for good measure refreshTime := resp.ThisUpdate.Add(nextUpdate.Sub(resp.ThisUpdate) / 2) return time.Now().Before(refreshTime) } golang-github-caddyserver-certmagic-0.25.2/ocsp_test.go000066400000000000000000000142531514710434200231340ustar00rootroot00000000000000package certmagic import ( "bytes" "context" "crypto" "errors" "io" "net/http" "net/http/httptest" "testing" "golang.org/x/crypto/ocsp" ) const certWithOCSPServer = `-----BEGIN CERTIFICATE----- MIIBgjCCASegAwIBAgICIAAwCgYIKoZIzj0EAwIwEjEQMA4GA1UEAxMHVGVzdCBD QTAeFw0yMzAxMDExMjAwMDBaFw0yMzAyMDExMjAwMDBaMCAxHjAcBgNVBAMTFU9D U1AgVGVzdCBDZXJ0aWZpY2F0ZTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABIoe I/bjo34qony8LdRJD+Jhuk8/S8YHXRHl6rH9t5VFCFtX8lIPN/Ll1zCrQ2KB3Wlb fxSgiQyLrCpZyrdhVPSjXzBdMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAU+Eo3 5sST4LRrwS4dueIdGBZ5d7IwLAYIKwYBBQUHAQEEIDAeMBwGCCsGAQUFBzABhhBv Y3NwLmV4YW1wbGUuY29tMAoGCCqGSM49BAMCA0kAMEYCIQDg94xY/+/VepESdvTT ykCwiWOS2aCpjyryrKpwMKkR0AIhAPc/+ZEz4W10OENxC1t+NUTvS8JbEGOwulkZ z9yfaLuD -----END CERTIFICATE-----` const certWithoutOCSPServer = `-----BEGIN CERTIFICATE----- MIIBUzCB+aADAgECAgIgADAKBggqhkjOPQQDAjASMRAwDgYDVQQDEwdUZXN0IENB MB4XDTIzMDEwMTEyMDAwMFoXDTIzMDIwMTEyMDAwMFowIDEeMBwGA1UEAxMVT0NT UCBUZXN0IENlcnRpZmljYXRlMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEih4j 9uOjfiqifLwt1EkP4mG6Tz9LxgddEeXqsf23lUUIW1fyUg838uXXMKtDYoHdaVt/ FKCJDIusKlnKt2FU9KMxMC8wDAYDVR0TAQH/BAIwADAfBgNVHSMEGDAWgBT4Sjfm xJPgtGvBLh254h0YFnl3sjAKBggqhkjOPQQDAgNJADBGAiEA3rWetLGblfSuNZKf 5CpZxhj3A0BjEocEh+2P+nAgIdUCIQDIgptabR1qTLQaF2u0hJsEX2IKuIUvYWH3 6Lb92+zIHg== -----END CERTIFICATE-----` // certKey is the private key for both certWithOCSPServer and // certWithoutOCSPServer. const certKey = `-----BEGIN EC PRIVATE KEY----- MHcCAQEEINnVcgrSNh4HlThWlZpegq14M8G/p9NVDtdVjZrseUGLoAoGCCqGSM49 AwEHoUQDQgAEih4j9uOjfiqifLwt1EkP4mG6Tz9LxgddEeXqsf23lUUIW1fyUg83 8uXXMKtDYoHdaVt/FKCJDIusKlnKt2FU9A== -----END EC PRIVATE KEY-----` // caCert is the issuing certificate for certWithOCSPServer and // certWithoutOCSPServer. const caCert = `-----BEGIN CERTIFICATE----- MIIBazCCARGgAwIBAgICEAAwCgYIKoZIzj0EAwIwEjEQMA4GA1UEAxMHVGVzdCBD QTAeFw0yMzAxMDExMjAwMDBaFw0yMzAyMDExMjAwMDBaMBIxEDAOBgNVBAMTB1Rl c3QgQ0EwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAASdKexSor/aeazDM57UHhAX rCkJxUeF2BWf0lZYCRxc3f0GdrEsVvjJW8+/E06eAzDCGSdM/08Nvun1nb6AmAlt o1cwVTAOBgNVHQ8BAf8EBAMCAQYwEwYDVR0lBAwwCgYIKwYBBQUHAwkwDwYDVR0T AQH/BAUwAwEB/zAdBgNVHQ4EFgQU+Eo35sST4LRrwS4dueIdGBZ5d7IwCgYIKoZI zj0EAwIDSAAwRQIgGbA39+kETTB/YMLBFoC2fpZe1cDWfFB7TUdfINUqdH4CIQCR ByUFC8A+hRNkK5YNH78bgjnKk/88zUQF5ONy4oPGdQ== -----END CERTIFICATE-----` const caKey = `-----BEGIN EC PRIVATE KEY----- MHcCAQEEIDJ59ptjq3MzILH4zn5IKoH1sYn+zrUeq2kD8+DD2x+OoAoGCCqGSM49 AwEHoUQDQgAEnSnsUqK/2nmswzOe1B4QF6wpCcVHhdgVn9JWWAkcXN39BnaxLFb4 yVvPvxNOngMwwhknTP9PDb7p9Z2+gJgJbQ== -----END EC PRIVATE KEY-----` func TestStapleOCSP(t *testing.T) { ctx := context.Background() storage := &FileStorage{Path: t.TempDir()} t.Run("disabled", func(t *testing.T) { cert := mustMakeCertificate(t, certWithOCSPServer, certKey) config := OCSPConfig{DisableStapling: true} err := stapleOCSP(ctx, config, storage, &cert, nil) if err != nil { t.Error("unexpected error:", err) } else if cert.Certificate.OCSPStaple != nil { t.Error("unexpected OCSP staple") } }) t.Run("no OCSP server", func(t *testing.T) { cert := mustMakeCertificate(t, certWithoutOCSPServer, certKey) err := stapleOCSP(ctx, OCSPConfig{}, storage, &cert, nil) if !errors.Is(err, ErrNoOCSPServerSpecified) { t.Error("expected ErrNoOCSPServerSpecified in error", err) } }) // Start an OCSP responder test server. responses := make(map[string][]byte) responder := startOCSPResponder(t, responses) t.Cleanup(responder.Close) ca := mustMakeCertificate(t, caCert, caKey) // The certWithOCSPServer certificate has a bogus ocsp.example.com endpoint. // Use the ResponderOverrides option to point to the test server instead. config := OCSPConfig{ ResponderOverrides: map[string]string{ "ocsp.example.com": responder.URL, }, } t.Run("ok", func(t *testing.T) { cert := mustMakeCertificate(t, certWithOCSPServer, certKey) tpl := ocsp.Response{ Status: ocsp.Good, SerialNumber: cert.Leaf.SerialNumber, } r, err := ocsp.CreateResponse( ca.Leaf, ca.Leaf, tpl, ca.PrivateKey.(crypto.Signer)) if err != nil { t.Fatal("couldn't create OCSP response", err) } responses[cert.Leaf.SerialNumber.String()] = r bundle := []byte(certWithOCSPServer + "\n" + caCert) err = stapleOCSP(ctx, config, storage, &cert, bundle) if err != nil { t.Error("unexpected error:", err) } else if !bytes.Equal(cert.Certificate.OCSPStaple, r) { t.Error("expected OCSP response to be stapled to certificate") } }) t.Run("revoked", func(t *testing.T) { cert := mustMakeCertificate(t, certWithOCSPServer, certKey) tpl := ocsp.Response{ Status: ocsp.Revoked, SerialNumber: cert.Leaf.SerialNumber, } r, err := ocsp.CreateResponse( ca.Leaf, ca.Leaf, tpl, ca.PrivateKey.(crypto.Signer)) if err != nil { t.Fatal("couldn't create OCSP response", err) } responses[cert.Leaf.SerialNumber.String()] = r bundle := []byte(certWithOCSPServer + "\n" + caCert) err = stapleOCSP(ctx, config, storage, &cert, bundle) if err != nil { t.Error("unexpected error:", err) } else if cert.Certificate.OCSPStaple != nil { t.Error("revoked OCSP response should not be stapled") } }) t.Run("no issuing cert", func(t *testing.T) { cert := mustMakeCertificate(t, certWithOCSPServer, certKey) err := stapleOCSP(ctx, config, storage, &cert, nil) expected := "no OCSP stapling for [ocsp test certificate]: " + "no URL to issuing certificate" if err == nil || err.Error() != expected { t.Errorf("expected error %q but got %q", expected, err) } }) } func mustMakeCertificate(t *testing.T, cert, key string) Certificate { t.Helper() c, err := makeCertificate([]byte(cert), []byte(key)) if err != nil { t.Fatal("couldn't make certificate:", err) } return c } func startOCSPResponder( t *testing.T, responses map[string][]byte, ) *httptest.Server { h := func(w http.ResponseWriter, r *http.Request) { ct := r.Header.Get("Content-Type") if ct != "application/ocsp-request" { t.Errorf("unexpected request Content-Type %q", ct) } b, _ := io.ReadAll(r.Body) request, err := ocsp.ParseRequest(b) if err != nil { t.Fatal(err) } w.Header().Set("Content-Type", "application/ocsp-response") w.Write(responses[request.SerialNumber.String()]) } return httptest.NewServer(http.HandlerFunc(h)) } golang-github-caddyserver-certmagic-0.25.2/ratelimiter.go000066400000000000000000000153151514710434200234520ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "log" "runtime" "sync" "time" ) // NewRateLimiter returns a rate limiter that allows up to maxEvents // in a sliding window of size window. If maxEvents and window are // both 0, or if maxEvents is non-zero and window is 0, rate limiting // is disabled. This function panics if maxEvents is less than 0 or // if maxEvents is 0 and window is non-zero, which is considered to be // an invalid configuration, as it would never allow events. func NewRateLimiter(maxEvents int, window time.Duration) *RingBufferRateLimiter { if maxEvents < 0 { panic("maxEvents cannot be less than zero") } if maxEvents == 0 && window != 0 { panic("NewRateLimiter: invalid configuration: maxEvents = 0 and window != 0 would not allow any events") } rbrl := &RingBufferRateLimiter{ window: window, ring: make([]time.Time, maxEvents), started: make(chan struct{}), stopped: make(chan struct{}), ticket: make(chan struct{}), } go rbrl.loop() <-rbrl.started // make sure loop is ready to receive before we return return rbrl } // RingBufferRateLimiter uses a ring to enforce rate limits // consisting of a maximum number of events within a single // sliding window of a given duration. An empty value is // not valid; use NewRateLimiter to get one. type RingBufferRateLimiter struct { window time.Duration ring []time.Time // maxEvents == len(ring) cursor int // always points to the oldest timestamp mu sync.Mutex // protects ring, cursor, and window started chan struct{} stopped chan struct{} ticket chan struct{} } // Stop cleans up r's scheduling goroutine. func (r *RingBufferRateLimiter) Stop() { close(r.stopped) } func (r *RingBufferRateLimiter) loop() { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: ring buffer rate limiter: %v\n%s", err, buf) } }() for { // if we've been stopped, return select { case <-r.stopped: return default: } if len(r.ring) == 0 { if r.window == 0 { // rate limiting is disabled; always allow immediately r.permit() continue } panic("invalid configuration: maxEvents = 0 and window != 0 does not allow any events") } // wait until next slot is available or until we've been stopped r.mu.Lock() then := r.ring[r.cursor].Add(r.window) r.mu.Unlock() waitDuration := time.Until(then) waitTimer := time.NewTimer(waitDuration) select { case <-waitTimer.C: r.permit() case <-r.stopped: waitTimer.Stop() return } } } // Allow returns true if the event is allowed to // happen right now. It does not wait. If the event // is allowed, a ticket is claimed. func (r *RingBufferRateLimiter) Allow() bool { select { case <-r.ticket: return true default: return false } } // Wait blocks until the event is allowed to occur. It returns an // error if the context is cancelled. func (r *RingBufferRateLimiter) Wait(ctx context.Context) error { select { case <-ctx.Done(): return context.Canceled case <-r.ticket: return nil } } // MaxEvents returns the maximum number of events that // are allowed within the sliding window. func (r *RingBufferRateLimiter) MaxEvents() int { r.mu.Lock() defer r.mu.Unlock() return len(r.ring) } // SetMaxEvents changes the maximum number of events that are // allowed in the sliding window. If the new limit is lower, // the oldest events will be forgotten. If the new limit is // higher, the window will suddenly have capacity for new // reservations. It panics if maxEvents is 0 and window size // is not zero; if setting both the events limit and the // window size to 0, call SetWindow() first. func (r *RingBufferRateLimiter) SetMaxEvents(maxEvents int) { newRing := make([]time.Time, maxEvents) r.mu.Lock() defer r.mu.Unlock() if r.window != 0 && maxEvents == 0 { panic("SetMaxEvents: invalid configuration: maxEvents = 0 and window != 0 would not allow any events") } // only make the change if the new limit is different if maxEvents == len(r.ring) { return } // the new ring may be smaller; fast-forward to the // oldest timestamp that will be kept in the new // ring so the oldest ones are forgotten and the // newest ones will be remembered sizeDiff := len(r.ring) - maxEvents for i := 0; i < sizeDiff; i++ { r.advance() } if len(r.ring) > 0 { // copy timestamps into the new ring until we // have either copied all of them or have reached // the capacity of the new ring startCursor := r.cursor for i := 0; i < len(newRing); i++ { newRing[i] = r.ring[r.cursor] r.advance() if r.cursor == startCursor { // new ring is larger than old one; // "we've come full circle" break } } } r.ring = newRing r.cursor = 0 } // Window returns the size of the sliding window. func (r *RingBufferRateLimiter) Window() time.Duration { r.mu.Lock() defer r.mu.Unlock() return r.window } // SetWindow changes r's sliding window duration to window. // Goroutines that are already blocked on a call to Wait() // will not be affected. It panics if window is non-zero // but the max event limit is 0. func (r *RingBufferRateLimiter) SetWindow(window time.Duration) { r.mu.Lock() defer r.mu.Unlock() if window != 0 && len(r.ring) == 0 { panic("SetWindow: invalid configuration: maxEvents = 0 and window != 0 would not allow any events") } r.window = window } // permit allows one event through the throttle. This method // blocks until a goroutine is waiting for a ticket or until // the rate limiter is stopped. func (r *RingBufferRateLimiter) permit() { for { select { case r.started <- struct{}{}: // notify parent goroutine that we've started; should // only happen once, before constructor returns continue case <-r.stopped: return case r.ticket <- struct{}{}: r.mu.Lock() defer r.mu.Unlock() if len(r.ring) > 0 { r.ring[r.cursor] = time.Now() r.advance() } return } } } // advance moves the cursor to the next position. // It is NOT safe for concurrent use, so it must // be called inside a lock on r.mu. func (r *RingBufferRateLimiter) advance() { r.cursor++ if r.cursor >= len(r.ring) { r.cursor = 0 } } golang-github-caddyserver-certmagic-0.25.2/solvers.go000066400000000000000000000665631514710434200226410ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/tls" "encoding/json" "fmt" "log" "net" "net/http" "path" "runtime" "strings" "sync" "sync/atomic" "time" "github.com/libdns/libdns" "github.com/mholt/acmez/v3" "github.com/mholt/acmez/v3/acme" "github.com/miekg/dns" "go.uber.org/zap" ) // httpSolver solves the HTTP challenge. It must be // associated with a config and an address to use // for solving the challenge. If multiple httpSolvers // are initialized concurrently, the first one to // begin will start the server, and the last one to // finish will stop the server. This solver must be // wrapped by a distributedSolver to work properly, // because the only way the HTTP challenge handler // can access the keyAuth material is by loading it // from storage, which is done by distributedSolver. type httpSolver struct { closed int32 // accessed atomically handler http.Handler address string } // Present starts an HTTP server if none is already listening on s.address. func (s *httpSolver) Present(ctx context.Context, _ acme.Challenge) error { solversMu.Lock() defer solversMu.Unlock() si := getSolverInfo(s.address) si.count++ if si.listener != nil { return nil // already be served by us } // notice the unusual error handling here; we // only continue to start a challenge server if // we got a listener; in all other cases return ln, err := robustTryListen(s.address) if ln == nil { return err } // successfully bound socket, so save listener and start key auth HTTP server si.listener = ln go s.serve(ctx, si) return nil } // serve is an HTTP server that serves only HTTP challenge responses. func (s *httpSolver) serve(ctx context.Context, si *solverInfo) { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: http solver server: %v\n%s", err, buf) } }() defer close(si.done) httpServer := &http.Server{ Handler: s.handler, BaseContext: func(listener net.Listener) context.Context { return ctx }, } httpServer.SetKeepAlivesEnabled(false) err := httpServer.Serve(si.listener) if err != nil && atomic.LoadInt32(&s.closed) != 1 { log.Printf("[ERROR] key auth HTTP server: %v", err) } } // CleanUp cleans up the HTTP server if it is the last one to finish. func (s *httpSolver) CleanUp(_ context.Context, _ acme.Challenge) error { solversMu.Lock() defer solversMu.Unlock() si := getSolverInfo(s.address) si.count-- if si.count == 0 { // last one out turns off the lights atomic.StoreInt32(&s.closed, 1) if si.listener != nil { si.listener.Close() <-si.done } delete(solvers, s.address) } return nil } // tlsALPNSolver is a type that can solve TLS-ALPN challenges. // It must have an associated config and address on which to // serve the challenge. type tlsALPNSolver struct { config *Config address string } // Present adds the certificate to the certificate cache and, if // needed, starts a TLS server for answering TLS-ALPN challenges. func (s *tlsALPNSolver) Present(ctx context.Context, chal acme.Challenge) error { // we pre-generate the certificate for efficiency with multi-perspective // validation, so it only has to be done once (at least, by this instance; // distributed solving does not have that luxury, oh well) - update the // challenge data in memory to be the generated certificate cert, err := acmez.TLSALPN01ChallengeCert(chal) if err != nil { return err } key := challengeKey(chal) activeChallengesMu.Lock() chalData := activeChallenges[key] chalData.data = cert activeChallenges[key] = chalData activeChallengesMu.Unlock() // the rest of this function increments the // challenge count for the solver at this // listener address, and if necessary, starts // a simple TLS server solversMu.Lock() defer solversMu.Unlock() si := getSolverInfo(s.address) si.count++ if si.listener != nil { return nil // already be served by us } // notice the unusual error handling here; we // only continue to start a challenge server if // we got a listener; in all other cases return ln, err := robustTryListen(s.address) if ln == nil { return err } // we were able to bind the socket, so make it into a TLS // listener, store it with the solverInfo, and start the // challenge server si.listener = tls.NewListener(ln, s.config.TLSConfig()) go func() { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: tls-alpn solver server: %v\n%s", err, buf) } }() defer close(si.done) for { conn, err := si.listener.Accept() if err != nil { if atomic.LoadInt32(&si.closed) == 1 { return } log.Printf("[ERROR] TLS-ALPN challenge server: accept: %v", err) continue } go s.handleConn(conn) } }() return nil } // handleConn completes the TLS handshake and then closes conn. func (*tlsALPNSolver) handleConn(conn net.Conn) { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: tls-alpn solver handler: %v\n%s", err, buf) } }() defer conn.Close() tlsConn, ok := conn.(*tls.Conn) if !ok { log.Printf("[ERROR] TLS-ALPN challenge server: expected tls.Conn but got %T: %#v", conn, conn) return } err := tlsConn.Handshake() if err != nil { log.Printf("[ERROR] TLS-ALPN challenge server: handshake: %v", err) return } } // CleanUp removes the challenge certificate from the cache, and if // it is the last one to finish, stops the TLS server. func (s *tlsALPNSolver) CleanUp(_ context.Context, chal acme.Challenge) error { solversMu.Lock() defer solversMu.Unlock() si := getSolverInfo(s.address) si.count-- if si.count == 0 { // last one out turns off the lights atomic.StoreInt32(&si.closed, 1) if si.listener != nil { si.listener.Close() <-si.done } delete(solvers, s.address) } return nil } // DNS01Solver is a type that makes libdns providers usable as ACME dns-01 // challenge solvers. See https://github.com/libdns/libdns // // Note that challenges may be solved concurrently by some clients (such as // acmez, which CertMagic uses), meaning that multiple TXT records may be // created in a DNS zone simultaneously, and in some cases distinct TXT records // may have the same name. For example, solving challenges for both example.com // and *.example.com create a TXT record named _acme_challenge.example.com, // but with different tokens as their values. This solver distinguishes // between different records with the same name by looking at their values. // DNS provider APIs and implementations of the libdns interfaces must also // support multiple same-named TXT records. type DNS01Solver struct { DNSManager } // Present creates the DNS TXT record for the given ACME challenge. func (s *DNS01Solver) Present(ctx context.Context, challenge acme.Challenge) error { dnsName := challenge.DNS01TXTRecordName() if s.OverrideDomain != "" { dnsName = s.OverrideDomain } keyAuth := challenge.DNS01KeyAuthorization() zrec, err := s.DNSManager.createRecord(ctx, dnsName, "TXT", keyAuth) if err != nil { return err } // remember the record and zone we got so we can clean up more efficiently s.saveDNSPresentMemory(dnsPresentMemory{ dnsName: dnsName, zoneRec: zrec, }) return nil } // Wait blocks until the TXT record created in Present() appears in // authoritative lookups, i.e. until it has propagated, or until // timeout, whichever is first. func (s *DNS01Solver) Wait(ctx context.Context, challenge acme.Challenge) error { // prepare for the checks by determining what to look for dnsName := challenge.DNS01TXTRecordName() if s.OverrideDomain != "" { dnsName = s.OverrideDomain } keyAuth := challenge.DNS01KeyAuthorization() // wait for the record to propagate memory, err := s.getDNSPresentMemory(dnsName, "TXT", keyAuth) if err != nil { return err } return s.DNSManager.wait(ctx, memory.zoneRec) } // CleanUp deletes the DNS TXT record created in Present(). // // We ignore the context because cleanup is often/likely performed after // a context cancellation, and properly-implemented DNS providers should // honor cancellation, which would result in cleanup being aborted. // Cleanup must always occur. func (s *DNS01Solver) CleanUp(ctx context.Context, challenge acme.Challenge) error { dnsName := challenge.DNS01TXTRecordName() if s.OverrideDomain != "" { dnsName = s.OverrideDomain } keyAuth := challenge.DNS01KeyAuthorization() // always forget about the record so we don't leak memory defer s.deleteDNSPresentMemory(dnsName, keyAuth) // recall the record we created and zone we looked up memory, err := s.getDNSPresentMemory(dnsName, "TXT", keyAuth) if err != nil { return err } if err := s.DNSManager.cleanUpRecord(ctx, memory.zoneRec); err != nil { return err } return nil } // DNSManager is a type that makes libdns providers usable for performing // DNS verification. See https://github.com/libdns/libdns // // Note that records may be manipulated concurrently by some clients (such as // acmez, which CertMagic uses), meaning that multiple records may be created // in a DNS zone simultaneously, and in some cases distinct records of the same // type may have the same name. For example, solving ACME challenges for both example.com // and *.example.com create a TXT record named _acme_challenge.example.com, // but with different tokens as their values. This solver distinguishes between // different records with the same type and name by looking at their values. type DNSManager struct { // The implementation that interacts with the DNS // provider to set or delete records. (REQUIRED) DNSProvider DNSProvider // The TTL for the temporary challenge records. TTL time.Duration // How long to wait before starting propagation checks. // Default: 0 (no wait). PropagationDelay time.Duration // Maximum time to wait for temporary DNS record to appear. // Set to -1 to disable propagation checks. // Default: 2 minutes. PropagationTimeout time.Duration // Preferred DNS resolver(s) to use when doing DNS lookups. Resolvers []string // Override the domain to set the TXT record on. This is // to delegate the challenge to a different domain. Note // that the solver doesn't follow CNAME/NS record. OverrideDomain string // An optional logger. Logger *zap.Logger // Remember DNS records while challenges are active; i.e. // records we have presented and not yet cleaned up. // This lets us clean them up quickly and efficiently. // Keyed by domain name (specifically the ACME DNS name). // The map value is a slice because there can be multiple // concurrent challenges for different domains that have // the same ACME DNS name, for example: example.com and // *.example.com. We distinguish individual memories by // the value of their TXT records, which should contain // unique challenge tokens. // See https://github.com/caddyserver/caddy/issues/3474. records map[string][]dnsPresentMemory recordsMu sync.Mutex } func (m *DNSManager) createRecord(ctx context.Context, dnsName, recordType, recordValue string) (zoneRecord, error) { logger := m.logger() zone, err := FindZoneByFQDN(ctx, logger, dnsName, RecursiveNameservers(m.Resolvers)) if err != nil { return zoneRecord{}, fmt.Errorf("could not determine zone for domain %q: %v", dnsName, err) } rr := libdns.RR{ Type: recordType, Name: libdns.RelativeName(dnsName+".", zone), Data: recordValue, TTL: m.TTL, } logger.Debug("creating DNS record", zap.String("dns_name", dnsName), zap.String("zone", zone), zap.String("record_name", rr.Name), zap.String("record_type", rr.Type), zap.String("record_data", rr.Data), zap.Duration("record_ttl", rr.TTL)) results, err := m.DNSProvider.AppendRecords(ctx, zone, []libdns.Record{rr}) if err != nil { return zoneRecord{}, fmt.Errorf("adding temporary record for zone %q: %w", zone, err) } if len(results) != 1 { return zoneRecord{}, fmt.Errorf("expected one record, got %d: %v", len(results), results) } return zoneRecord{zone, results[0].RR()}, nil } // wait blocks until the TXT record created in Present() appears in // authoritative lookups, i.e. until it has propagated, or until // timeout, whichever is first. func (m *DNSManager) wait(ctx context.Context, zrec zoneRecord) error { logger := m.logger() // if configured to, pause before doing propagation checks // (even if they are disabled, the wait might be desirable on its own) if m.PropagationDelay > 0 { select { case <-time.After(m.PropagationDelay): case <-ctx.Done(): return ctx.Err() } } // skip propagation checks if configured to do so if m.PropagationTimeout == -1 { return nil } // timings timeout := m.PropagationTimeout if timeout == 0 { timeout = defaultDNSPropagationTimeout } const interval = 2 * time.Second // how we'll do the checks checkAuthoritativeServers := len(m.Resolvers) == 0 resolvers := RecursiveNameservers(m.Resolvers) recType := dns.TypeTXT if zrec.record.RR().Type == "CNAME" { recType = dns.TypeCNAME } absName := libdns.AbsoluteName(zrec.record.Name, zrec.zone) var err error start := time.Now() for time.Since(start) < timeout { select { case <-time.After(interval): case <-ctx.Done(): return ctx.Err() } logger.Debug("checking DNS propagation", zap.String("fqdn", absName), zap.String("record_type", zrec.record.Type), zap.String("expected_data", zrec.record.Data), zap.Strings("resolvers", resolvers)) var ready bool ready, err = checkDNSPropagation(ctx, logger, absName, recType, zrec.record.Data, checkAuthoritativeServers, resolvers) if err != nil { return fmt.Errorf("checking DNS propagation of %q (relative=%s zone=%s resolvers=%v): %w", absName, zrec.record.Name, zrec.zone, resolvers, err) } if ready { return nil } } return fmt.Errorf("timed out waiting for record to fully propagate; verify DNS provider configuration is correct - last error: %v", err) } type zoneRecord struct { zone string record libdns.RR } // CleanUp deletes the DNS TXT record created in Present(). // // We ignore the context because cleanup is often/likely performed after // a context cancellation, and properly-implemented DNS providers should // honor cancellation, which would result in cleanup being aborted. // Cleanup must always occur. func (m *DNSManager) cleanUpRecord(_ context.Context, zrec zoneRecord) error { logger := m.logger() // clean up the record - use a different context though, since // one common reason cleanup is performed is because a context // was canceled, and if so, any HTTP requests by this provider // should fail if the provider is properly implemented // (see issue #200) timeout := m.PropagationTimeout if timeout <= 0 { timeout = defaultDNSPropagationTimeout } ctx, cancel := context.WithTimeout(context.Background(), timeout) defer cancel() logger.Debug("deleting DNS record", zap.String("zone", zrec.zone), zap.String("record_name", zrec.record.Name), zap.String("record_type", zrec.record.Type), zap.String("record_data", zrec.record.Data)) _, err := m.DNSProvider.DeleteRecords(ctx, zrec.zone, []libdns.Record{zrec.record}) if err != nil { return fmt.Errorf("deleting temporary record for name %q in zone %q: %w", zrec.zone, zrec.record, err) } return nil } func (m *DNSManager) logger() *zap.Logger { logger := m.Logger if logger == nil { logger = zap.NewNop() } return logger.Named("dns_manager") } const defaultDNSPropagationTimeout = 2 * time.Minute // dnsPresentMemory associates a created DNS record with its zone // (since libdns Records are zone-relative and do not include zone). type dnsPresentMemory struct { dnsName string zoneRec zoneRecord } func (s *DNSManager) saveDNSPresentMemory(mem dnsPresentMemory) { s.recordsMu.Lock() if s.records == nil { s.records = make(map[string][]dnsPresentMemory) } s.records[mem.dnsName] = append(s.records[mem.dnsName], mem) s.recordsMu.Unlock() } func (s *DNSManager) getDNSPresentMemory(dnsName, recType, value string) (dnsPresentMemory, error) { s.recordsMu.Lock() defer s.recordsMu.Unlock() var memory dnsPresentMemory var found bool for _, mem := range s.records[dnsName] { if mem.zoneRec.record.Type == recType && mem.zoneRec.record.Data == value { memory = mem found = true break } } if !found { return dnsPresentMemory{}, fmt.Errorf("no memory of presenting a DNS record for %q (usually OK if presenting also failed)", dnsName) } return memory, nil } func (s *DNSManager) deleteDNSPresentMemory(dnsName, keyAuth string) { s.recordsMu.Lock() defer s.recordsMu.Unlock() for i, mem := range s.records[dnsName] { if mem.zoneRec.record.Data == keyAuth { s.records[dnsName] = append(s.records[dnsName][:i], s.records[dnsName][i+1:]...) return } } } // DNSProvider defines the set of operations required for // ACME challenges or other sorts of domain verification. // A DNS provider must be able to append and delete records // in order to solve ACME challenges. Find one you can use // at https://github.com/libdns. If your provider isn't // implemented yet, feel free to contribute! type DNSProvider interface { libdns.RecordAppender libdns.RecordDeleter } // distributedSolver allows the ACME HTTP-01 and TLS-ALPN challenges // to be solved by an instance other than the one which initiated it. // This is useful behind load balancers or in other cluster/fleet // configurations. The only requirement is that the instance which // initiates the challenge shares the same storage and locker with // the others in the cluster. The storage backing the certificate // cache in distributedSolver.config is crucial. // // Obviously, the instance which completes the challenge must be // serving on the HTTPChallengePort for the HTTP-01 challenge or the // TLSALPNChallengePort for the TLS-ALPN-01 challenge (or have all // the packets port-forwarded) to receive and handle the request. The // server which receives the challenge must handle it by checking to // see if the challenge token exists in storage, and if so, decode it // and use it to serve up the correct response. HTTPChallengeHandler // in this package as well as the GetCertificate method implemented // by a Config support and even require this behavior. // // In short: the only two requirements for cluster operation are // sharing sync and storage, and using the facilities provided by // this package for solving the challenges. type distributedSolver struct { // The storage backing the distributed solver. It must be // the same storage configuration as what is solving the // challenge in order to be effective. storage Storage // The storage key prefix, associated with the issuer // that is solving the challenge. storageKeyIssuerPrefix string // Since the distributedSolver is only a // wrapper over an actual solver, place // the actual solver here. solver acmez.Solver } // Present invokes the underlying solver's Present method // and also stores domain, token, and keyAuth to the storage // backing the certificate cache of dhs.acmeIssuer. func (dhs distributedSolver) Present(ctx context.Context, chal acme.Challenge) error { infoBytes, err := json.Marshal(chal) if err != nil { return err } err = dhs.storage.Store(ctx, dhs.challengeTokensKey(challengeKey(chal)), infoBytes) if err != nil { return err } err = dhs.solver.Present(ctx, chal) if err != nil { return fmt.Errorf("presenting with embedded solver: %v", err) } return nil } // Wait wraps the underlying solver's Wait() method, if any. Implements acmez.Waiter. func (dhs distributedSolver) Wait(ctx context.Context, challenge acme.Challenge) error { if waiter, ok := dhs.solver.(acmez.Waiter); ok { return waiter.Wait(ctx, challenge) } return nil } // CleanUp invokes the underlying solver's CleanUp method // and also cleans up any assets saved to storage. func (dhs distributedSolver) CleanUp(ctx context.Context, chal acme.Challenge) error { err := dhs.storage.Delete(ctx, dhs.challengeTokensKey(challengeKey(chal))) if err != nil { return err } err = dhs.solver.CleanUp(ctx, chal) if err != nil { return fmt.Errorf("cleaning up embedded provider: %v", err) } return nil } // challengeTokensPrefix returns the key prefix for challenge info. func (dhs distributedSolver) challengeTokensPrefix() string { return path.Join(dhs.storageKeyIssuerPrefix, "challenge_tokens") } // challengeTokensKey returns the key to use to store and access // challenge info for domain. func (dhs distributedSolver) challengeTokensKey(domain string) string { return path.Join(dhs.challengeTokensPrefix(), StorageKeys.Safe(domain)+".json") } // solverInfo associates a listener with the // number of challenges currently using it. type solverInfo struct { closed int32 // accessed atomically count int listener net.Listener done chan struct{} // used to signal when our own solver server is done } // getSolverInfo gets a valid solverInfo struct for address. func getSolverInfo(address string) *solverInfo { si, ok := solvers[address] if !ok { si = &solverInfo{done: make(chan struct{})} solvers[address] = si } return si } // robustTryListen calls net.Listen for a TCP socket at addr. // This function may return both a nil listener and a nil error! // If it was able to bind the socket, it returns the listener // and no error. If it wasn't able to bind the socket because // the socket is already in use, then it returns a nil listener // and nil error. If it had any other error, it returns the // error. The intended error handling logic for this function // is to proceed if the returned listener is not nil; otherwise // return err (which may also be nil). In other words, this // function ignores errors if the socket is already in use, // which is useful for our challenge servers, where we assume // that whatever is already listening can solve the challenges. func robustTryListen(addr string) (net.Listener, error) { var listenErr error for i := 0; i < 2; i++ { // doesn't hurt to sleep briefly before the second // attempt in case the OS has timing issues if i > 0 { time.Sleep(100 * time.Millisecond) } // if we can bind the socket right away, great! var ln net.Listener ln, listenErr = net.Listen("tcp", addr) if listenErr == nil { return ln, nil } // if it failed just because the socket is already in use, we // have no choice but to assume that whatever is using the socket // can answer the challenge already, so we ignore the error connectErr := dialTCPSocket(addr) if connectErr == nil { return nil, nil } // Hmm, we couldn't connect to the socket, so something else must // be wrong, right? wrong!! Apparently if a port is bound by another // listener with a specific host, i.e. 'x:1234', we cannot bind to // ':1234' -- it is considered a conflict, but 'y:1234' is not. // I guess we need to assume the conflicting listener is properly // configured and continue. But we should tell the user to specify // the correct ListenHost to avoid conflict or at least so we can // know that the user is intentional about that port and hopefully // has an ACME solver on it. // // History: // https://caddy.community/t/caddy-retry-error/7317 // https://caddy.community/t/v2-upgrade-to-caddy2-failing-with-errors/7423 // https://github.com/caddyserver/certmagic/issues/250 if strings.Contains(listenErr.Error(), "address already in use") || strings.Contains(listenErr.Error(), "one usage of each socket address") { log.Printf("[WARNING] %v - be sure to set the ACMEIssuer.ListenHost field; assuming conflicting listener is correctly configured and continuing", listenErr) return nil, nil } } return nil, fmt.Errorf("could not start listener for challenge server at %s: %v", addr, listenErr) } // dialTCPSocket connects to a TCP address just for the sake of // seeing if it is open. It returns a nil error if a TCP connection // can successfully be made to addr within a short timeout. func dialTCPSocket(addr string) error { conn, err := net.DialTimeout("tcp", addr, 250*time.Millisecond) if err == nil { conn.Close() } return err } // GetACMEChallenge returns an active ACME challenge for the given identifier, // or false if no active challenge for that identifier is known. func GetACMEChallenge(identifier string) (Challenge, bool) { activeChallengesMu.Lock() chalData, ok := activeChallenges[identifier] activeChallengesMu.Unlock() return chalData, ok } // The active challenge solvers, keyed by listener address, // and protected by a mutex. Note that the creation of // solver listeners and the incrementing of their counts // are atomic operations guarded by this mutex. var ( solvers = make(map[string]*solverInfo) solversMu sync.Mutex ) // activeChallenges holds information about all known, currently-active // ACME challenges, keyed by identifier. CertMagic guarantees that // challenges for the same identifier do not overlap, by its locking // mechanisms; thus if a challenge comes in for a certain identifier, // we can be confident that if this process initiated the challenge, // the correct information to solve it is in this map. (It may have // alternatively been initiated by another instance in a cluster, in // which case the distributed solver will take care of that.) var ( activeChallenges = make(map[string]Challenge) activeChallengesMu sync.Mutex ) // Challenge is an ACME challenge, but optionally paired with // data that can make it easier or more efficient to solve. type Challenge struct { acme.Challenge data any } // challengeKey returns the map key for a given challenge; it is the identifier // unless it is an IP address using the TLS-ALPN challenge. func challengeKey(chal acme.Challenge) string { if chal.Type == acme.ChallengeTypeTLSALPN01 && chal.Identifier.Type == "ip" { reversed, err := dns.ReverseAddr(chal.Identifier.Value) if err == nil { return reversed[:len(reversed)-1] // strip off '.' } } return chal.Identifier.Value } // solverWrapper should be used to wrap all challenge solvers so that // we can add the challenge info to memory; this makes challenges globally // solvable by a single HTTP or TLS server even if multiple servers with // different configurations/scopes need to get certificates. type solverWrapper struct{ acmez.Solver } func (sw solverWrapper) Present(ctx context.Context, chal acme.Challenge) error { activeChallengesMu.Lock() activeChallenges[challengeKey(chal)] = Challenge{Challenge: chal} activeChallengesMu.Unlock() return sw.Solver.Present(ctx, chal) } func (sw solverWrapper) Wait(ctx context.Context, chal acme.Challenge) error { if waiter, ok := sw.Solver.(acmez.Waiter); ok { return waiter.Wait(ctx, chal) } return nil } func (sw solverWrapper) CleanUp(ctx context.Context, chal acme.Challenge) error { activeChallengesMu.Lock() delete(activeChallenges, challengeKey(chal)) activeChallengesMu.Unlock() return sw.Solver.CleanUp(ctx, chal) } // Interface guards var ( _ acmez.Solver = (*solverWrapper)(nil) _ acmez.Waiter = (*solverWrapper)(nil) _ acmez.Waiter = (*distributedSolver)(nil) ) golang-github-caddyserver-certmagic-0.25.2/solvers_test.go000066400000000000000000000064451514710434200236710ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "testing" "github.com/mholt/acmez/v3/acme" ) func Test_challengeKey(t *testing.T) { type args struct { chal acme.Challenge } tests := []struct { name string args args want string }{ { name: "ok/dns-dns", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeDNS01, Identifier: acme.Identifier{ Type: "dns", Value: "*.example.com", }, }, }, want: "*.example.com", }, { name: "ok/http-dns", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeHTTP01, Identifier: acme.Identifier{ Type: "dns", Value: "*.example.com", }, }, }, want: "*.example.com", }, { name: "ok/tls-dns", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "dns", Value: "*.example.com", }, }, }, want: "*.example.com", }, { name: "ok/http-ipv4", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeHTTP01, Identifier: acme.Identifier{ Type: "ip", Value: "127.0.0.1", }, }, }, want: "127.0.0.1", }, { name: "ok/http-ipv6", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeHTTP01, Identifier: acme.Identifier{ Type: "ip", Value: "2001:db8::1", }, }, }, want: "2001:db8::1", }, { name: "ok/tls-ipv4", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "ip", Value: "127.0.0.1", }, }, }, want: "1.0.0.127.in-addr.arpa", }, { name: "ok/tls-ipv6", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "ip", Value: "2001:db8::1", }, }, }, want: "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.b.d.0.1.0.0.2.ip6.arpa", }, { name: "fail/tls-ipv4", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "ip", Value: "127.0.0.1000", }, }, }, want: "127.0.0.1000", // reversing this fails; default to identifier value }, { name: "fail/tls-ipv6", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "ip", Value: "2001:db8::10000", }, }, }, want: "2001:db8::10000", // reversing this fails; default to identifier value }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { if got := challengeKey(tt.args.chal); got != tt.want { t.Errorf("challengeKey() = %v, want %v", got, tt.want) } }) } } golang-github-caddyserver-certmagic-0.25.2/storage.go000066400000000000000000000313641514710434200225770ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "fmt" "path" "regexp" "strings" "sync" "time" "go.uber.org/zap" ) // Storage is a type that implements a key-value store with // basic file system (folder path) semantics. Keys use the // forward slash '/' to separate path components and have no // leading or trailing slashes. // // A "prefix" of a key is defined on a component basis, // e.g. "a" is a prefix of "a/b" but not "ab/c". // // A "file" is a key with a value associated with it. // // A "directory" is a key with no value, but which may be // the prefix of other keys. // // Keys passed into Load and Store always have "file" semantics, // whereas "directories" are only implicit by leading up to the // file. // // The Load, Delete, List, and Stat methods should return // fs.ErrNotExist if the key does not exist. // // Processes running in a cluster should use the same Storage // value (with the same configuration) in order to share // certificates and other TLS resources with the cluster. // // Implementations of Storage MUST be safe for concurrent use // and honor context cancellations. Methods should block until // their operation is complete; that is, Load() should always // return the value from the last call to Store() for a given // key, and concurrent calls to Store() should not corrupt a // file. // // For simplicity, this is not a streaming API and is not // suitable for very large files. type Storage interface { // Locker enables the storage backend to synchronize // operational units of work. // // The use of Locker is NOT employed around every // Storage method call (Store, Load, etc), as these // should already be thread-safe. Locker is used for // high-level jobs or transactions that need // synchronization across a cluster; it's a simple // distributed lock. For example, CertMagic uses the // Locker interface to coordinate the obtaining of // certificates. Locker // Store puts value at key. It creates the key if it does // not exist and overwrites any existing value at this key. Store(ctx context.Context, key string, value []byte) error // Load retrieves the value at key. Load(ctx context.Context, key string) ([]byte, error) // Delete deletes the named key. If the name is a // directory (i.e. prefix of other keys), all keys // prefixed by this key should be deleted. An error // should be returned only if the key still exists // when the method returns. Delete(ctx context.Context, key string) error // Exists returns true if the key exists either as // a directory (prefix to other keys) or a file, // and there was no error checking. Exists(ctx context.Context, key string) bool // List returns all keys in the given path. // // If recursive is true, non-terminal keys // will be enumerated (i.e. "directories" // should be walked); otherwise, only keys // prefixed exactly by prefix will be listed. List(ctx context.Context, path string, recursive bool) ([]string, error) // Stat returns information about key. Stat(ctx context.Context, key string) (KeyInfo, error) } // Locker facilitates synchronization across machines and networks. // It essentially provides a distributed named-mutex service so // that multiple consumers can coordinate tasks and share resources. // // If possible, a Locker should implement a coordinated distributed // locking mechanism by generating fencing tokens (see // https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html). // This typically requires a central server or consensus algorithm // However, if that is not feasible, Lockers may implement an // alternative mechanism that uses timeouts to detect node or network // failures and avoid deadlocks. For example, the default FileStorage // writes a timestamp to the lock file every few seconds, and if another // node acquiring the lock sees that timestamp is too old, it may // assume the lock is stale. // // As not all Locker implementations use fencing tokens, code relying // upon Locker must be tolerant of some mis-synchronizations but can // expect them to be rare. // // This interface should only be used for coordinating expensive // operations across nodes in a cluster; not for internal, extremely // short-lived, or high-contention locks. type Locker interface { // Lock acquires the lock for name, blocking until the lock // can be obtained or an error is returned. Only one lock // for the given name can exist at a time. A call to Lock for // a name which already exists blocks until the named lock // is released or becomes stale. // // If the named lock represents an idempotent operation, callers // should always check to make sure the work still needs to be // completed after acquiring the lock. You never know if another // process already completed the task while you were waiting to // acquire it. // // Implementations should honor context cancellation. Lock(ctx context.Context, name string) error // Unlock releases named lock. This method must ONLY be called // after a successful call to Lock, and only after the critical // section is finished, even if it errored or timed out. Unlock // cleans up any resources allocated during Lock. Unlock should // only return an error if the lock was unable to be released. Unlock(ctx context.Context, name string) error } type TryLocker interface { // TryLock attempts to acquire the lock for name, and returns a // boolean that reports whether the lock was successfully aquired // or not along with any errors that may have occurred. // // Implementations should honor context cancellation. TryLock(ctx context.Context, name string) (bool, error) // Unlock releases named lock. This method must ONLY be called // after a successful call to TryLock, and only after the critical // section is finished, even if it errored or timed out. Unlock // cleans up any resources allocated during TryLock. Unlock should // only return an error if the lock was unable to be released. Unlock(ctx context.Context, name string) error } // LockLeaseRenewer is an optional interface that can be implemented by a Storage // implementation to support renewing the lease on a lock. This is useful for // long-running operations that need to be synchronized across a cluster. type LockLeaseRenewer interface { // RenewLockLease renews the lease on the lock for the given lockKey for the // given leaseDuration. This is used to prevent the lock from being acquired // by another process. RenewLockLease(ctx context.Context, lockKey string, leaseDuration time.Duration) error } // KeyInfo holds information about a key in storage. // Key and IsTerminal are required; Modified and Size // are optional if the storage implementation is not // able to get that information. Setting them will // make certain operations more consistent or // predictable, but it is not crucial to basic // functionality. type KeyInfo struct { Key string Modified time.Time Size int64 IsTerminal bool // false for directories (keys that act as prefix for other keys) } // storeTx stores all the values or none at all. func storeTx(ctx context.Context, s Storage, all []keyValue) error { for i, kv := range all { err := s.Store(ctx, kv.key, kv.value) if err != nil { for j := i - 1; j >= 0; j-- { s.Delete(ctx, all[j].key) } return err } } return nil } // keyValue pairs a key and a value. type keyValue struct { key string value []byte } // KeyBuilder provides a namespace for methods that // build keys and key prefixes, for addressing items // in a Storage implementation. type KeyBuilder struct{} // CertsPrefix returns the storage key prefix for // the given certificate issuer. func (keys KeyBuilder) CertsPrefix(issuerKey string) string { return path.Join(prefixCerts, keys.Safe(issuerKey)) } // CertsSitePrefix returns a key prefix for items associated with // the site given by domain using the given issuer key. func (keys KeyBuilder) CertsSitePrefix(issuerKey, domain string) string { return path.Join(keys.CertsPrefix(issuerKey), keys.Safe(domain)) } // SiteCert returns the path to the certificate file for domain // that is associated with the issuer with the given issuerKey. func (keys KeyBuilder) SiteCert(issuerKey, domain string) string { safeDomain := keys.Safe(domain) return path.Join(keys.CertsSitePrefix(issuerKey, domain), safeDomain+".crt") } // SitePrivateKey returns the path to the private key file for domain // that is associated with the certificate from the given issuer with // the given issuerKey. func (keys KeyBuilder) SitePrivateKey(issuerKey, domain string) string { safeDomain := keys.Safe(domain) return path.Join(keys.CertsSitePrefix(issuerKey, domain), safeDomain+".key") } // SiteMeta returns the path to the metadata file for domain that // is associated with the certificate from the given issuer with // the given issuerKey. func (keys KeyBuilder) SiteMeta(issuerKey, domain string) string { safeDomain := keys.Safe(domain) return path.Join(keys.CertsSitePrefix(issuerKey, domain), safeDomain+".json") } // OCSPStaple returns a key for the OCSP staple associated // with the given certificate. If you have the PEM bundle // handy, pass that in to save an extra encoding step. func (keys KeyBuilder) OCSPStaple(cert *Certificate, pemBundle []byte) string { var ocspFileName string if len(cert.Names) > 0 { firstName := keys.Safe(cert.Names[0]) ocspFileName = firstName + "-" } ocspFileName += fastHash(pemBundle) return path.Join(prefixOCSP, ocspFileName) } // Safe standardizes and sanitizes str for use as // a single component of a storage key. This method // is idempotent. func (keys KeyBuilder) Safe(str string) string { str = strings.ToLower(str) str = strings.TrimSpace(str) // replace a few specific characters repl := strings.NewReplacer( " ", "_", "+", "_plus_", "*", "wildcard_", ":", "-", "..", "", // prevent directory traversal (regex allows single dots) ) str = repl.Replace(str) // finally remove all non-word characters return safeKeyRE.ReplaceAllLiteralString(str, "") } // CleanUpOwnLocks immediately cleans up all // current locks obtained by this process. Since // this does not cancel the operations that // the locks are synchronizing, this should be // called only immediately before process exit. // Errors are only reported if a logger is given. func CleanUpOwnLocks(ctx context.Context, logger *zap.Logger) { locksMu.Lock() defer locksMu.Unlock() for lockKey, storage := range locks { if err := storage.Unlock(ctx, lockKey); err != nil { logger.Error("unable to clean up lock in storage backend", zap.Any("storage", storage), zap.String("lock_key", lockKey), zap.Error(err)) continue } delete(locks, lockKey) } } func acquireLock(ctx context.Context, storage Storage, lockKey string) error { err := storage.Lock(ctx, lockKey) if err == nil { locksMu.Lock() locks[lockKey] = storage locksMu.Unlock() } return err } func tryAcquireLock(ctx context.Context, storage Storage, lockKey string) (bool, error) { locker, ok := storage.(TryLocker) if !ok { return false, fmt.Errorf("%T does not implement TryLocker", storage) } ok, err := locker.TryLock(ctx, lockKey) if ok && err == nil { locksMu.Lock() locks[lockKey] = storage locksMu.Unlock() } return ok, err } func releaseLock(ctx context.Context, storage Storage, lockKey string) error { err := storage.Unlock(context.WithoutCancel(ctx), lockKey) if err == nil { locksMu.Lock() delete(locks, lockKey) locksMu.Unlock() } return err } // locks stores a reference to all the current // locks obtained by this process. var locks = make(map[string]Storage) var locksMu sync.Mutex // StorageKeys provides methods for accessing // keys and key prefixes for items in a Storage. // Typically, you will not need to use this // because accessing storage is abstracted away // for most cases. Only use this if you need to // directly access TLS assets in your application. var StorageKeys KeyBuilder const ( prefixCerts = "certificates" prefixOCSP = "ocsp" ) // safeKeyRE matches any undesirable characters in storage keys. // Note that this allows dots, so you'll have to strip ".." manually. var safeKeyRE = regexp.MustCompile(`[^\w@.-]`) // defaultFileStorage is a convenient, default storage // implementation using the local file system. var defaultFileStorage = &FileStorage{Path: dataDir()} golang-github-caddyserver-certmagic-0.25.2/storage_test.go000066400000000000000000000055431514710434200236360ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "path" "testing" ) func TestPrefixAndKeyBuilders(t *testing.T) { am := &ACMEIssuer{CA: "https://example.com/acme-ca/directory"} base := path.Join("certificates", "example.com-acme-ca-directory") for i, testcase := range []struct { in, folder, certFile, keyFile, metaFile string }{ { in: "example.com", folder: path.Join(base, "example.com"), certFile: path.Join(base, "example.com", "example.com.crt"), keyFile: path.Join(base, "example.com", "example.com.key"), metaFile: path.Join(base, "example.com", "example.com.json"), }, { in: "*.example.com", folder: path.Join(base, "wildcard_.example.com"), certFile: path.Join(base, "wildcard_.example.com", "wildcard_.example.com.crt"), keyFile: path.Join(base, "wildcard_.example.com", "wildcard_.example.com.key"), metaFile: path.Join(base, "wildcard_.example.com", "wildcard_.example.com.json"), }, { // prevent directory traversal! very important, esp. with on-demand TLS // see issue #2092 in: "a/../../../foo", folder: path.Join(base, "afoo"), certFile: path.Join(base, "afoo", "afoo.crt"), keyFile: path.Join(base, "afoo", "afoo.key"), metaFile: path.Join(base, "afoo", "afoo.json"), }, { in: "b\\..\\..\\..\\foo", folder: path.Join(base, "bfoo"), certFile: path.Join(base, "bfoo", "bfoo.crt"), keyFile: path.Join(base, "bfoo", "bfoo.key"), metaFile: path.Join(base, "bfoo", "bfoo.json"), }, { in: "c/foo", folder: path.Join(base, "cfoo"), certFile: path.Join(base, "cfoo", "cfoo.crt"), keyFile: path.Join(base, "cfoo", "cfoo.key"), metaFile: path.Join(base, "cfoo", "cfoo.json"), }, } { if actual := StorageKeys.SiteCert(am.IssuerKey(), testcase.in); actual != testcase.certFile { t.Errorf("Test %d: site cert file: Expected '%s' but got '%s'", i, testcase.certFile, actual) } if actual := StorageKeys.SitePrivateKey(am.IssuerKey(), testcase.in); actual != testcase.keyFile { t.Errorf("Test %d: site key file: Expected '%s' but got '%s'", i, testcase.keyFile, actual) } if actual := StorageKeys.SiteMeta(am.IssuerKey(), testcase.in); actual != testcase.metaFile { t.Errorf("Test %d: site meta file: Expected '%s' but got '%s'", i, testcase.metaFile, actual) } } } golang-github-caddyserver-certmagic-0.25.2/testdata/000077500000000000000000000000001514710434200224065ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.25.2/testdata/resolv.conf.1000066400000000000000000000002031514710434200247210ustar00rootroot00000000000000domain company.com nameserver 10.200.3.249 nameserver 10.200.3.250:5353 nameserver 2001:4860:4860::8844 nameserver [10.0.0.1]:5353 golang-github-caddyserver-certmagic-0.25.2/zerosslissuer.go000066400000000000000000000233611514710434200240650ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/x509" "encoding/json" "fmt" "net" "net/http" "strconv" "strings" "time" "github.com/caddyserver/zerossl" "github.com/mholt/acmez/v3" "github.com/mholt/acmez/v3/acme" "go.uber.org/zap" ) // ZeroSSLIssuer can get certificates from ZeroSSL's API. (To use ZeroSSL's ACME // endpoint, use the ACMEIssuer instead.) Note that use of the API is restricted // by payment tier. type ZeroSSLIssuer struct { // The API key (or "access key") for using the ZeroSSL API. // REQUIRED. APIKey string // Where to store verification material temporarily. // All instances in a cluster should have the same // Storage value to enable distributed verification. // REQUIRED. (TODO: Make it optional for those not // operating in a cluster. For now, it's simpler to // put info in storage whether distributed or not.) Storage Storage // How many days the certificate should be valid for. ValidityDays int // The host to bind to when opening a listener for // verifying domain names (or IPs). ListenHost string // If HTTP is forwarded from port 80, specify the // forwarded port here. AltHTTPPort int // To use CNAME validation instead of HTTP // validation, set this field. CNAMEValidation *DNSManager // Delay between poll attempts. PollInterval time.Duration // An optional (but highly recommended) logger. Logger *zap.Logger } // Issue obtains a certificate for the given csr. func (iss *ZeroSSLIssuer) Issue(ctx context.Context, csr *x509.CertificateRequest) (*IssuedCertificate, error) { client := iss.getClient() identifiers := namesFromCSR(csr) if len(identifiers) == 0 { return nil, fmt.Errorf("no identifiers on CSR") } logger := iss.Logger if logger == nil { logger = zap.NewNop() } logger = logger.With(zap.Strings("identifiers", identifiers)) logger.Info("creating certificate") cert, err := client.CreateCertificate(ctx, csr, iss.ValidityDays) if err != nil { return nil, fmt.Errorf("creating certificate: %v", err) } logger = logger.With(zap.String("cert_id", cert.ID)) logger.Info("created certificate") defer func(certID string) { if err != nil { err := client.CancelCertificate(context.WithoutCancel(ctx), certID) if err == nil { logger.Info("canceled certificate") } else { logger.Error("unable to cancel certificate", zap.Error(err)) } } }(cert.ID) var verificationMethod zerossl.VerificationMethod if iss.CNAMEValidation == nil { verificationMethod = zerossl.HTTPVerification logger = logger.With(zap.String("verification_method", string(verificationMethod))) httpVerifier := &httpSolver{ address: net.JoinHostPort(iss.ListenHost, strconv.Itoa(iss.getHTTPPort())), handler: iss.HTTPValidationHandler(http.NewServeMux()), } var solver acmez.Solver = httpVerifier if iss.Storage != nil { solver = distributedSolver{ storage: iss.Storage, storageKeyIssuerPrefix: iss.IssuerKey(), solver: httpVerifier, } } // since the distributed solver was originally designed for ACME, // the API is geared around ACME challenges. ZeroSSL's HTTP validation // is very similar to the HTTP challenge, but not quite compatible, // so we kind of shim the ZeroSSL validation data into a Challenge // object... it is not a perfect use of this type but it's pretty close valInfo := cert.Validation.OtherMethods[identifiers[0]] fakeChallenge := acme.Challenge{ Identifier: acme.Identifier{ Value: identifiers[0], // used for storage key }, URL: valInfo.FileValidationURLHTTP, Token: strings.Join(cert.Validation.OtherMethods[identifiers[0]].FileValidationContent, "\n"), } if err = solver.Present(ctx, fakeChallenge); err != nil { return nil, fmt.Errorf("presenting validation file for verification: %v", err) } defer solver.CleanUp(ctx, fakeChallenge) } else { verificationMethod = zerossl.CNAMEVerification logger = logger.With(zap.String("verification_method", string(verificationMethod))) // create the CNAME record(s) records := make(map[string]zoneRecord, len(cert.Validation.OtherMethods)) for name, verifyInfo := range cert.Validation.OtherMethods { zr, err := iss.CNAMEValidation.createRecord(ctx, verifyInfo.CnameValidationP1, "CNAME", verifyInfo.CnameValidationP2+".") // see issue #304 if err != nil { return nil, fmt.Errorf("creating CNAME record: %v", err) } defer func(name string, zr zoneRecord) { if err := iss.CNAMEValidation.cleanUpRecord(ctx, zr); err != nil { logger.Warn("cleaning up temporary validation record failed", zap.String("dns_name", name), zap.Error(err)) } }(name, zr) records[name] = zr } // wait for them to propagate for name, zr := range records { if err := iss.CNAMEValidation.wait(ctx, zr); err != nil { // allow it, since the CA will ultimately decide, but definitely log it logger.Warn("failed CNAME record propagation check", zap.String("domain", name), zap.Error(err)) } } } logger.Info("validating identifiers") cert, err = client.VerifyIdentifiers(ctx, cert.ID, verificationMethod, nil) if err != nil { return nil, fmt.Errorf("verifying identifiers: %v", err) } switch cert.Status { case "pending_validation": logger.Info("validations initiated; waiting for certificate to be issued") cert, err = iss.waitForCertToBeIssued(ctx, client, cert) if err != nil { return nil, fmt.Errorf("waiting for certificate to be issued: %v", err) } case "issued": logger.Info("validations succeeded; downloading certificate bundle") default: return nil, fmt.Errorf("unexpected certificate status: %s", cert.Status) } bundle, err := client.DownloadCertificate(ctx, cert.ID, false) if err != nil { return nil, fmt.Errorf("downloading certificate: %v", err) } logger.Info("successfully downloaded issued certificate") return &IssuedCertificate{ Certificate: []byte(bundle.CertificateCrt + bundle.CABundleCrt), Metadata: cert, }, nil } func (iss *ZeroSSLIssuer) waitForCertToBeIssued(ctx context.Context, client zerossl.Client, cert zerossl.CertificateObject) (zerossl.CertificateObject, error) { ticker := time.NewTicker(iss.pollInterval()) defer ticker.Stop() for { select { case <-ctx.Done(): return cert, ctx.Err() case <-ticker.C: var err error cert, err = client.GetCertificate(ctx, cert.ID) if err != nil { return cert, err } if cert.Status == "issued" { return cert, nil } if cert.Status != "pending_validation" { return cert, fmt.Errorf("unexpected certificate status: %s", cert.Status) } } } } func (iss *ZeroSSLIssuer) pollInterval() time.Duration { if iss.PollInterval == 0 { return defaultPollInterval } return iss.PollInterval } func (iss *ZeroSSLIssuer) getClient() zerossl.Client { return zerossl.Client{AccessKey: iss.APIKey} } func (iss *ZeroSSLIssuer) getHTTPPort() int { useHTTPPort := HTTPChallengePort if HTTPPort > 0 && HTTPPort != HTTPChallengePort { useHTTPPort = HTTPPort } if iss.AltHTTPPort > 0 { useHTTPPort = iss.AltHTTPPort } return useHTTPPort } // IssuerKey returns the unique issuer key for ZeroSSL. func (iss *ZeroSSLIssuer) IssuerKey() string { return zerosslIssuerKey } // Revoke revokes the given certificate. Only do this if there is a security or trust // concern with the certificate. func (iss *ZeroSSLIssuer) Revoke(ctx context.Context, cert CertificateResource, reason int) error { var r zerossl.RevocationReason switch reason { case acme.ReasonKeyCompromise: r = zerossl.KeyCompromise case acme.ReasonAffiliationChanged: r = zerossl.AffiliationChanged case acme.ReasonSuperseded: r = zerossl.Superseded case acme.ReasonCessationOfOperation: r = zerossl.CessationOfOperation case acme.ReasonUnspecified: r = zerossl.UnspecifiedReason default: return fmt.Errorf("unsupported reason: %d", reason) } var certObj zerossl.CertificateObject if err := json.Unmarshal(cert.IssuerData, &certObj); err != nil { return err } return iss.getClient().RevokeCertificate(ctx, certObj.ID, r) } func (iss *ZeroSSLIssuer) getDistributedValidationInfo(ctx context.Context, identifier string) (acme.Challenge, bool, error) { if iss.Storage == nil { return acme.Challenge{}, false, nil } ds := distributedSolver{ storage: iss.Storage, storageKeyIssuerPrefix: StorageKeys.Safe(iss.IssuerKey()), } tokenKey := ds.challengeTokensKey(identifier) valObjectBytes, err := iss.Storage.Load(ctx, tokenKey) if err != nil { return acme.Challenge{}, false, fmt.Errorf("opening distributed challenge token file %s: %v", tokenKey, err) } if len(valObjectBytes) == 0 { return acme.Challenge{}, false, fmt.Errorf("no information found to solve challenge for identifier: %s", identifier) } // since the distributed solver's API is geared around ACME challenges, // we crammed the validation info into a Challenge object var chal acme.Challenge if err = json.Unmarshal(valObjectBytes, &chal); err != nil { return acme.Challenge{}, false, fmt.Errorf("decoding HTTP validation token file %s (corrupted?): %v", tokenKey, err) } return chal, true, nil } const ( zerosslIssuerKey = "zerossl" defaultPollInterval = 5 * time.Second ) // Interface guards var ( _ Issuer = (*ZeroSSLIssuer)(nil) _ Revoker = (*ZeroSSLIssuer)(nil) )